This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.
Many clinical trials investigating treatment efficacy require an interim analysis. Recently we have been running a large, multisite, randomized, placebo-controlled, double-blind clinical trial investigating the effect of repetitive transcranial magnetic stimulation (rTMS) treatment for improving or stabilizing the cognition of patients diagnosed with Alzheimer disease.
The objectives of this paper are to report on recruitment, adherence, and adverse events (AEs) to date, and to describe in detail the protocol for interim analysis of the clinical trial data. The protocol will investigate whether the trial is likely to reach its objectives if continued to the planned maximum sample size.
The specific requirements of the analytic protocol are to (1) ensure the double-blind nature of the data while doing the analysis, (2) estimate the predictive probabilities of success (PPoSs), (3) estimate the numbers needed to treat, (4) re-estimate the initial required sample size. The initial estimate of sample size was 208. The interim analysis will be based on 150 patients who will be enrolled in the study and finish at least 8 weeks of the study. Our protocol for interim analysis, at the very first stage, is to determine the response rate for each participant to the treatment (either sham or active), while ensuring the double-blind nature of the data. The blinded data will be analyzed by a statistician to investigate the treatment efficacy. We will use Bayesian PPoS to predict the success rate and determine whether the study should continue.
The enrollment has been slowed significantly due to the COVID-19 pandemic and lockdown. Nevertheless, so far 133 participants have been enrolled, while 22 of these have been withdrawn or dropped out for various reasons. In general, rTMS has been found tolerable with no serious AE. Only 2 patients dropped out of the study due to their intolerability to rTMS pulses.
Overall, the study with the same protocol is going as expected with no serious AE or any major protocol deviation.
ClinicalTrials.gov NCT02908815; https://clinicaltrials.gov/ct2/show/NCT02908815
DERR1-10.2196/31183
Clinical trials investigating treatment efficacy often incorporate an interim analysis of outcomes. Interim analysis is conducted for a variety of different reasons, which may include detecting unbalanced patterns of adverse events (AEs) in treatment arms with the potential to indicate harm to participants, or determining on statistical grounds whether continuing data collection to the originally planned sample size is likely to provide a definitive answer to the question framed by the primary hypothesis. Recently we have been running a large, multisite, randomized, placebo-controlled, double-blind clinical trial for investigating the effect of repetitive transcranial magnetic stimulation (rTMS) treatment for improving or stabilizing cognition in patients in the mild to moderate stage of Alzheimer disease (AD). All 3 sites are located in urban centers of countries with a socialized health care system (Winnipeg, Montreal, and Melbourne). The details of the protocol are described in [
The goal of the interim analysis is to investigate whether continuing the trial to its planned sample size of 208 is likely to achieve the goal of determining whether active rTMS treatment benefits patients with AD beyond the placebo effect. The objectives of this interim analysis are to (1) ensure the double-blind nature of the data while doing the analysis, (2) estimate the predictive probabilities of success (PPoSs), (3) estimate the numbers needed to treat, (4) re-estimate the initial required sample size.
The initial estimate of sample size was 208. The interim analysis will be based on 150 patients who will be enrolled in the study and finish at least 8 weeks of the study. Our protocol for interim analysis is explained in detail in the following steps.
At the very first stage, the data will be prepared for analysis by a single investigator (ZM) who is aware of group assignment but who does not contribute to the data analysis. This individual will randomly sort and relabel the study participants as P1 (patient 1), P2, P3, etc. The same individual will then randomly sort and label the 3 arms of the intervention (2-week active, 4-week active, and sham) as Group 1, Group 2, and Group 3 before forwarding the data to the statistician. The data will be analyzed by a research assistant and statisticians blind to information about participants contributing to the study, who will also remain unaware of the group (sham versus active) assignment.
A patient is considered as a (positive) responder to rTMS treatment if s/he meets either one of the 3 criteria below. These criteria are derived based on similar literature monitoring improvement/decline in patients with Alzheimer [
Note that in ADAS-Cog and NPI-Q assessments, negative changes from baseline represent improvement, whereas for ADCS-ADL a positive change from baseline implies improvement. In order to avoid confusion, the criteria for responders are written using the term “improvement,” which means a change from baseline toward better cognitive or behavioral function (ie, a positive value for ADCS-ADL and a negative value for ADAS-Cog and NPI-Q).
We define the responders/nonresponders by applying the following criteria. Note that the AND is a logical AND.
Having 3+ score improvement in the ADAS-Cog score (compared with baseline) at
Having a nonsignificant improvement (<3 score) in ADAS-Cog
Having a nonsignificant worsening (<3 score) in ADAS-Cog
The above definition of responders is a slightly stricter version of the definitions of responders commonly used in studies to investigate the effect of donepezil (Aricept); for a review, see [
Among the responder groups, we will identify patients with small, moderate, and marked responses, and then estimate the “number needed to treat (NNT)” for each type of response, as NNT is also a measure of the efficacy of the treatment [
Because rTMS treatment has been suggested as an alternative nonmedication treatment for AD, it makes sense to define its success rate similar to the trials investigating the efficacy of a “standard” medication.
The most commonly used medication for AD is donepezil (Aricept). Several studies have shown significant differences in the number of responders to donepezil versus sham/placebo [
To guide decision making regarding whether to discontinue or continue the clinical trial until reaching the planned target sample, we will derive predictive probabilities for the study if it continues as opposed to relying only on the
Based on the above literature [
For the interim analysis, we will use the primary outcome measure (ADAS-Cog) and 2 secondary measures of ADCS-ADL and NPI-Q, which are the most commonly used tests to evaluate improvement or decline of a patient with Alzheimer over time in clinical trials. The changes in these measures compared with baseline will be analyzed.
As the very first step, descriptive basic statistics will be provided to compare the mean and standard deviation of values among the 3 study sites. As the ADAS-Cog, ADCS-ADL, and NPI-Q are all continuous variables, we will use an analysis of covariance (ANCOVA) model to compare the 3 treatment groups (4 and 2 weeks of active and sham).
The models of efficacy will contain covariates for baseline score, treatment effect, and center effect. The parameters for the efficacy as well as futility models are the changes from baseline of the 3 outcome measures among responders (all 3 levels) and nonresponders in each of the 2 active treatment groups versus sham. The standard assumptions on covariance will be tested before running the ANCOVA. If they fail the normality tests, we will use equivalent nonparametric tests (ie, ranked ANCOVA) [
To enroll patients into the study, we use their age and Alzheimer severity measured by the Clinical Dementia Rating Scale sum of boxes score [
We will also investigate the occurrence, if any, of serious AEs that lead to withdrawal of participants from the study in relation to the site and treatment dose.
Conditional power is basically the power of the test, that is, the probability to not reject the null hypothesis when it is false. At interim analysis, the conditional power is estimated as the probability of rejecting the null hypothesis of no effect, given a specific alternative hypothesis. PPoSs [
At an interim analysis, sample the parameter of interest θ from the current posterior given current data X(θ). The parameter θ is the responses of patients in the study.
Complete the data set by sampling future samples X(m), observations not yet observed at the interim analysis, from the predictive distribution.
Use the complete data set to calculate the success criteria (
Repeat the first 3 steps for a total of B times (B is an arbitrary but reasonable number defined by the statistician); the PPoS is the proportion of simulated trials that achieve success.
In our study, the majority of patients are on a stable dose of an acetylcholinesterase inhibitor (AChEI) medication. No participant changes or starts an AChEI medication after being enrolled into the study. However, because 35% (47/133) of participants so far are not on any AChEI medication, it should be considered as a confounding variable when analyzing the results.
As the number of participants is still small given the number of independent variables, we will use permutation statistical analysis that tests whether the observations are independent and does not make any assumption about the data’s distribution. If we find the intervention arms unbalanced (statistically) in terms of the number of nonmedicated patients, we will have to adjust our analysis for such a confounding variable.
The NNT is another measure to summarize effects of a treatment based on the relative risks. Thus, many clinical trials do calculate the NNT at the end of study or at interim analysis. The NNT for 1 patient to be a responder (either in mild, moderate, or marked response groups) will be calculated by predictive probabilities and method presented by [
At interim analysis for efficacy, a trial can be stopped early by reassessing the sample size based on existing data in case the sample size was overestimated. By contrast, if the sample size initially was underestimated, at the interim analysis, the PPoS can give a better estimation of what sample size is needed for the data to support the study’s hypotheses. We will reassess the sample size at the interim analysis by the method introduced in [
As of May 1, 2021, a total of 523 patients were screened, of whom 133 were enrolled across the 3 sites of the study (62 in Manitoba, 39 in Quebec, and 32 in Australia) and randomized to different arms of the study. Of the 133 participants, 110 have completed the 6-month study period, 1 is waiting to start the treatment (still on hold due to the pandemic), and 22 have withdrawn or discontinued due to different reasons detailed below.
The percentage of the withdrawn cases so far is therefore 16.5% (22/133), which is much higher than our initial 10% estimation. However, partial data for approximately 40% (9/22) of the withdrawn patients can be used for analysis as those were discontinued/withdrawn during the follow-up period after finishing the treatment. The withdrawn cases are categorized into 5 groups. See
Information on withdrawn patients.
Category | Number | Reasons |
Became ineligible during the pandemic | 2 | They were enrolled before lockdown, but their cognitive function declined rapidly during the lockdown and they became ineligible. |
Principal investigator withdrawal—noncompliant | 3 | One changed medication during the study and 2 developed an illness and were withdrawn by the principal investigator for safety concerns. |
Participant withdrawal—changed mind before treatment | 7 | No particular reason was given. |
Participant withdrawal—too much anxiety due to treatment | 2 | They found the pulses too painful to tolerate. |
Unrelated health and family issues | 8 | Two could not finish treatment due to the pandemic, 2 could not finish treatment due to unrelated health issues, and 4 could not attend all assessment sessions due to unrelated health conditions but their data up to a point can be used. |
In this study, at the end of each treatment session, the treatment administrator asks patients via a checklist about any related or nonrelated symptoms and asks them to identify the level of discomfort associated with receiving TMS pulses on a scale of 0-10 as shown in
The expected side effects of rTMS are scalp pain/sensitivity, leg jerking, toothache, jaw clenching, or eye twitches during the treatment. These symptoms should abate immediately after the end of treatment at each session. If the duration of any of the above symptoms is prolonged, then it is considered as an unexpected side effect.
Other expected side effects include lingering eye twitches, headache, feeling exhausted, or having slight dizziness after the treatment that may last for a few hours. These symptoms are expected to diminish without requiring medication. If they are sustained more than a few hours, they should be listed as unexpected.
Other unexpected side effects that may or may not be related to rTMS treatment include nightmares, prolonged feeling of disorientation, confusion, nausea, fatigue, dizziness, agitation, eye redness, and neck stiffness. Seizure is a rare documented side effect of rTMS among at-risk individuals; for this reason, the screening process is designed to exclude such patients from participating and a protocol for management of an unexpected seizure is in place at each site.
Information about possible side effects is written on the consent forms and explained to each participant and his/her caregiver so they are knowledgeable when signing to provide their informed consent prior to enrollment into the study. AEs, whether expected or unexpected, are managed according to the protocols developed for the Ethics Board at each site. The association of serious AEs, that is, any prolonged side effects beyond a day or any side effect that needed medical intervention, with the treatment protocol is determined by the teams’ physician(s) after consultation with the site PI, patient, and any caregivers involved in the study. Nonserious expected AEs are referred to the site PI for documentation. Nonserious unexpected AEs are referred to the PI, who will consult with the team’s physicians as necessary to determine their association with the treatment protocol. The extensive list of side effects is reported to the Data Safety Monitoring Board of the study as well as to the Ethics Board of the University of Manitoba on a regular basis.
To date, there has been no serious AE. Nevertheless, out of the 133 participants, 89 reported minor typical AE of the rTMS treatment and 12 have reported unexpected AE. The most commonly reported AE has been mild to moderate discomfort and sensitivity to the pulses, with a pain scale of 2-7, while receiving them; however, this reported AE generally reduced over the sessions; most discomfort is reported in the first few sessions of the treatment. The second most commonly reported AE has been fatigue, headache, or both immediately after the treatment, which subsided within a couple of hours without any pain medication. There were also reports of dizziness, disorientation, and nausea after treatment but with much less frequency.
In terms of tolerability, because only 2 of 133 participants withdrew due to finding the rTMS pulses too painful and causing excessive anxiety, we may say overall the participants have tolerated the treatment protocol well.
In our clinical trial, we enroll patients who are either on a stable dose of an AChEI medication or not taking any cognition-enhancing medication; most importantly they should not change their medication or no-medication status (or dosage, if applicable) during the course of the study (6 months). Thus, we investigated whether the number of nonmedicated patients could have any effect on the interim analysis. Overall, of the 133 participants, 46 were not on cognition-enhancing medication during the time in which they were participating in the study.
Statistical analysis of the data using chi-square and permutation tests of independent variables showed that the mean number of nonmedicated patients did not differ significantly across the 3 arms of the study (
Overall, the study has been going as expected. In general, participants have found the rTMS treatment tolerable and have been compliant with the study protocol; the side effects have been minor and expected in general. Most participants in the sham group have received real treatment at the end of the 6-month study period. Medication can be a confounding variable. Because of the slow enrollment rate of patients with Alzheimer with the strict inclusion/exclusion criteria as in this study, we did not stratify or otherwise control for medication status during the randomization process. To date, 35% (47/133) of patients were not taking cognitive-enhancing medication during participation in the study, and the distribution of such participants does not differ across time or across arms of the trial. Should these results change at the interim analysis, we will adjust for this variable in our statistical analysis.
Interim analysis.
Peer review reports.
acetylcholinesterase Inhibitor
Alzheimer disease
Alzheimer Disease Assessment Scale-Cognitive Subscale
Alzheimer Disease Co-operative Study-Activities of Daily Living Inventory
analysis of covariance
Neuropsychiatric Inventory–Questionnaire
predictive probability of success
repetitive transcranial magnetic stimulation
Treatment Satisfaction Questionnaire for Medication
PBF is supported by a National Health and Medical Research Council (NHMRC) Practitioner Fellowship (1078567). PBF has received equipment for research from MagVenture A/S, Nexstim, Neuronetics and Brainsway Ltd and funding for research from Neuronetics. He is a founder of TMS Clinics Australia.