The Optimizing Detection and Prediction of Cognitive Function in Multiple Sclerosis (CogDetect-MS) study applies an observational design that combines microlongitudinal (ie, frequent, repeated “burst” measures across 14 consecutive days) and longitudinal (ie, 1- and 2-year follow-up) data collection methods in a sample with MS. Participant recruitment and data collection are conducted across 3 sites: the University of Michigan (UM; lead site and data coordinating center) in Ann Arbor, Michigan; Wayne State University (WSU) in Detroit, Michigan; and the University of Washington (UW) in Seattle, Washington. Ambulatory data are managed by researchers at the Pennsylvania State University, who return scored and combined ambulatory datasets to UM.

This multisite study has received a single institutional review board (IRB) approval from the medical IRB at UM (HUM00199732; participating site approvals: UM: HUM00213744, WSU: SITE00000462, and UW: SITE00000461). Initial IRB approval was obtained on January 27, 2022. All volunteers provided written informed consent. Participants are compensated US $600 for the full completion of the study (US $200 for each visit—T1, T2, and T3). For those who do not complete the full study, the compensation schedule is as follows: US $50 per laboratory visit and US $150 per home monitoring period (for <14 days of data, compensation is graded with US $4 per day for days 1-5, US $10 per day for days 6-10, and US $20 per day for days 11-14).

The aim was to recruit 250 participants, with the expectation that 210 would also provide data at the final (2-year) follow-up. Participants were recruited through existing participant registries; electronic health record queries; institution-specific, participant-recruitment websites; clinic- and community-based recruitment; posting of flyers; and outreach to local partners, such as the local chapters of the National MS Society. Inclusion criteria (assessed by self-report) were (1) 18 years of age or older, (2) able to fluently converse and read in English, (3) MS diagnosis (confirmed via medical record review; all relapsing and progressive subtypes included), and (4) able to ambulate either independently or with the use of a cane or walker (or similar device) for at least 50% of the time at baseline; participants who lose ability to ambulate over the course of the study are retained, as this criteria only applies to initial enrollment. Exclusion criteria were (1) MS relapse within the past 30 days (may become eligible after 30 days; criteria used at T1, T2, and T3) and (2) inability to use study data collection tools (ie, ActiGraph wGT3X-BT [ActiGraph], smartphone app; volunteers “pass” this final exclusion criterion by independently completing a trial of the ambulatory assessment battery during the laboratory visit).

Volunteers underwent an initial prescreening by telephone to determine general inclusion or exclusion criteria and were fully screened at the T1 laboratory visit to establish study eligibility. MS diagnosis was either preconfirmed through medical record review or initially gathered by self-report and later confirmed through record review. Written informed consent procedures were either conducted remotely (via Zoom [Zoom Video Communications] or telephone, and a signature obtained via e-consent in REDCap [Research Electronic Data Capture; Vanderbilt University]) prior to the T1 laboratory visit or in person at the laboratory visit.

Participation in this study involves assessments at baseline (T1), 1-year follow-up (T2), and 2-year follow-up (T3). Each assessment period includes a ~2.5-hour laboratory visit immediately followed by a 14-day ambulatory monitoring period (measurement “burst”). At each laboratory visit, certified examiners administer cognitive and physical function test batteries and demonstrate the use of a study-specific smartphone (programmed with a data collection app) and an ActiGraph wGT3X-BT accelerometer for the collection of ambulatory data. A battery of web-based self-report surveys is also completed at each time point, either prior to (within 30 days of laboratory visit) or during the laboratory visit.

During each ambulatory monitoring period, participants continuously wear an ActiGraph wGT3X-BT to passively collect physical activity data. At 4 intervals throughout the day (wake, midday, and bedtime), participants complete a set of brief, valid, and reliable cognitive tests assessing processing speed and working memory [13] along with a battery of ecological momentary assessment (EMA; real-time self-report) measures of somatic symptoms, mood, functioning, behaviors, and context on the smartphone app. The wake-up and bedtime assessments are initiated by the participant when waking up (ie, waking and not necessarily getting out of bed) and going to bed (ie, “lights out” and not necessarily when getting into bed). The other 2 assessments are prompted by an audible alert on a quasi-random schedule determined by their usual waking time. At the end of each home monitoring period, participants return the ActiGraph wGT3X-BT and smartphone in a prepaid mailer to the laboratory for data download.

Survey data are collected via a secure, study-specific REDCap website. REDCap is an open-source, secure, HIPAA (Health Insurance Portability and Accountability Act)–compliant, web-based platform designed to support data capture for research studies. It has been designed specifically to protect patient privacy and confidentiality while assisting investigators in clinical research. REDCap provides an interface for data entry and validation, auditing features for tracking data manipulation, the ability to import data from external sources, calculated data fields, branching logic, and the capability to export data to many statistical packages. System-level and app-level security include Secure Sockets Layer encryption of internet traffic (https pages), hosting in a secure data center with nightly backup, fine-grained control over user rights, detailed audit trails, record locking, and deidentification features for data export. REDCap was initially developed by Vanderbilt University but now has collaborative support from a wide consortium of many domestic and international partners.

Self-report EMAs and ambulatory cognitive tests are administered via a customized app (Figure 1; Wear-IT, developed by the Real Time Science Lab, Pennsylvania State University) installed on a Motorola g⁸ Power mobile phone with a 6.4″ display (1080×2300 pixels). The phone is loaned to participants for use during the study; it is not associated with any phone number and is used with an inactive SIM card for keeping accurate time on the phone; thus, there are no signals sent or received via the phone. Phones are loaned to study participants to ensure device consistency across participants and across time periods and because ambulatory cognitive assessment apps on personal phones have not been validated at the time the study launched. Response times are recorded in milliseconds. Data are stored onboard the smartphone until it is returned to the laboratory for data download.

The ActiGraph wGT3X-BT triaxial accelerometer (Figure 2) is used to measure physical activity. It is mounted on a fabric band on the nondominant wrist. In cases of hemiparesis, the accelerometer is placed on the nonparetic side. It is lightweight (19 g) and compact (3.3×4.6×1.5 cm) and measures movement using a capacitive accelerometer that digitizes a voltage detected from movement at a sampling rate of 30 Hz. The samples are summed over a 60-second epoch period and output as activity counts. Higher activity counts relate to more physical activity. We use a wrist-worn placement, as this placement has been used extensively in physical activity studies and to validate the ActiGraph in MS [24-35].

Smartphone app data (EMA and cognitive tests) are combined, and time synced with accelerometer date by the Wear-IT team.

To ensure standard test protocol administration across study sites and time, a rigorous examiner certification process was established. To achieve initial certification to administer tests, research staff were required to read the study protocol and manual of procedures, read the NIHTB Administration Manual, watch all training videos, and pass quizzes at the end of each training video. Training videos were from the NIH Toolbox eLearning Course or were custom-made by the study investigators, who had expertise in motor testing (NEF) or neuropsychological test administration (ALK, DME, and KNA). The videos provide detailed instructions on general best practices for test administration and how to administer each non-NIHTB test. After these initial training activities, research staff practiced the full laboratory-visit protocol with at least 5 nonparticipants (eg, fellow laboratory staff), video recording the final testing session. This video along with all accompanying case report forms and test materials were evaluated by 2 investigators—one with expertise in administering motor tests and one with expertise in administering neuropsychological tests. Together, the evaluators decided whether the examiner passed or failed the certification. Failure is defined as 2 or more minor errors or 1 or more major errors. A major error is defined as any error that indicates a lack of understanding of the proper standardized administration of any test or any scoring error that is large enough to change the interpretation of the data. Errors are reviewed with the examiner and their site principal investigator (ALK, NEF, or KNA). If the assessment is failed, the examiner practices at least 1 more time and submits a new certification video for review; this process can continue until the examiner passes certification. After initial certification, the examiner can begin testing study participants and is required to video record the first testing session with a person with MS; this video is also reviewed for consistency with study protocol, and feedback shared with the examiner. To ensure continued adherence to testing protocol, examiners record the laboratory visit for every 10th session, and this recording is reviewed by investigators as with the earlier certification videos. Consent for video recording is included in the study consent form.

The principal investigators (ALK and NEF) and lead research coordinator (KP) from the data coordinating center (UM) conduct in-person data audits at each site on an annual basis. Data related to adverse events, protocol deviations, study personnel training, screening procedures, participant withdrawal or termination, and enrollment procedures and documentation were audited for all participants enrolled at the site; data related to eligibility screening and documentation, study visit tracking, participant contact information, compensation record of human participants, and data collection were audited for a random subsample of all participants enrolled at the site. Audit reports, detailing findings, and required responses to the audit were produced and delivered to the site principal investigator (ALK, NEF, or KNA) and lead site research coordinator.

To ensure multisite coordination and fidelity of procedures, the full study team, including all examiners and investigators at all study sites, met weekly to discuss study-related questions and troubleshoot any issues that had arisen during the prior week for the first 18 months of the study. As fewer questions arose, and study teams were immersed in recruitment and testing, meetings were shifted to every other week (months 18-36). After month 36, meetings were shifted to once per month. UM keeps a record of all meeting agendas and meeting minutes, and any clarifications to the study manual of procedures are recorded by the UM team and updated in a shared folder that includes all study-related documents.

UM serves as the data coordinating center for the study. Data from all sites are fully accessible to the investigators and staff at UM, who conduct monthly data checks to assess for data completeness and quality. Data double entry of case report forms from each study site and data cleaning, scoring, and merging to produce final, analyzable datasets are completed by UM staff and investigators.

We conducted analyses to determine the sample size needed to address all study aims. The goal of the first study aim is to determine whether the ambulatory tests are able to detect cognitive decline from baseline (T1) to 1-year (T2) or 2-year follow-up (T3) for individuals where clinic-based tests do not detect decline. The proportion of participants who show no decline or improvement, absolute but subtle decline (change <1/2 SD), meaningful decline (change between 1/2 SD-1 SD) [71], or clinically significant decline (≥1 SD decline) [71,72] will be calculated for both ambulatory and clinic-based neurocognitive tests. For each cognitive domain, a binary variable will be created for each participant indicating whether the ambulatory and clinic-based cognitive measures are consistent with each other (eg, agree) about the degree of change or laboratory-based or ambulatory measures indicate a larger degree of decline. We will test whether the proportion of cases where ambulatory measures indicated a larger degree of decline (relative to clinic-based tests) is statistically different from 0; sample size analyses for this test indicate that a sample of 199 will have 95% power (with critical α=.01) to detect significance, where ambulatory cognitive tests show a greater level of decline compared to clinic-based tests in as few as 1.5% (n=3) of cases. This suggests that our expected final sample size of 210 has the power to detect even modest differences in analyses comparing proportions of the sample that show a decline on ambulatory cognitive tests but not on clinic-based tests.

Effects sizes from a prior study of perceived cognitive functioning in daily life in MS [73-75] informed sample size estimation for the second and third aims, which examines factors that predict later cognitive decline or that are predicted by cognitive changes. We calculated the sample size needed to test the aims in a linear regression framework [76], which is a relatively conservative estimate, given that the repeated measures design imparts greater measurement reliability and therefore greater power [77]. We based our estimates on models that included up to 6 covariates (see list of covariates below) and 6 predictor variables of interest (eg, sleep quality, physical activity, pain, fatigue, mood, and stress) in predicting any given cognitive variable. The sample size for these models was expected to provide a conservative estimate for power required for the third aim (which had fewer predictors in each model). Critical α (P) value was set at .01. Our analyses indicated that a sample size of 214 will have 95% power (critical t=2.34, 1-sided significance test) to detect an association between cognitive functioning, and the variable expected to show the weakest association with cognition: mood (effect size f²=0.075).

The anticipated findings of this study are 3-fold. First, we anticipate that ambulatory cognitive tests will be more sensitive to subtle cognitive changes in people with MS over 1-2 years. Second, we expect to identify modifiable factors (eg, mood, sleep, and physical activity) that precede and predict later cognitive decline on a short- and long-term scale. This information could help to prevent future decline or mitigate current cognitive dysfunction. Third, we hypothesize that cognitive changes will predict changes in social and physical function on a short- and long-term scale; such information will help to delineate the full impact of cognitive change in MS.

The collection of intensive longitudinal data in a large, heterogenous sample will allow for an in-depth characterization of individuals with MS and provide multiple avenues for future research. Data from this study are expected to inform comprehensive models of cognitive change in MS as well as provide insights on potential targets for intervention development to help people with MS optimize cognitive function.

One limitation of this study is that the inclusion criteria require participants to be able to ambulate. The rationale for this criterion is that we would like to collect meaningful accelerometer data in order to explore the associations between physical activity and cognitive function. However, this criterion limits the generalizability of the findings to those with more significant mobility limitations.

This study has a number of notable strengths. Technology-enabled assessment of day-to-day cognitive function in the lived environment has the potential to greatly improve the sensitivity and ecological validity of cognitive assessment in people with MS. The advancement of measurement sensitivity is critical, as cognitive changes can be subtle and compound slowly over time; however, despite the small magnitude of these changes, individuals with MS often report distress over noticeable changes in their cognition that are not detected on standard laboratory-based cognitive tests. Another advantage is the intensive within-person design that allows for the exploration of dynamic associations between potentially modifiable predictors of cognitive dysfunction while accounting for “third variables” such as a person’s disease severity, age, and sex. AnΩΩΩhone. This allows for exploring different trajectories of change over time.

Future work that capitalizes on the findings of this study and advances in assessment methods can also be used to improve treatment decision-making, including the timing and type of treatment approach. We will use these findings to design and test trials of behavioral, medical, and combination therapies to prevent cognitive decline and improve cognitive functioning in people with MS. Additionally, our innovative assessment methods may also be used to improve the measurement of outcomes in clinical trials of both pharmacological and nonpharmacological interventions, enriching understanding of the effects of such interventions. To our knowledge, there are few studies that track cognitive and other functional domains in MS beyond 2 years; thus, long-term outcomes will yield a rich dataset for understanding longitudinal function in persons with MS.