This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.
With the rapid aging of the global population, experts anticipate a surge in the prevalence of mild cognitive impairment (MCI) and dementia worldwide. It is argued that developing more sensitive, easy to administer, and valid MCI screening tools for use in primary care settings may initiate timely clinical and personal care planning and treatment, enabling early access to programs and services. Including functional competence measures in screening tests makes them more ecologically valid and may help to identify cognitive deficits at an earlier stage.
We aim to conduct a preliminary evaluative study comparing the sensitivity, specificity, and reliability of the BrainFx Screen (referred to as SCREEN hereafter), a novel digital tool designed to assess functional competence and detect early signs of cognitive impairment, with the Quick Mild Cognitive Impairment, a validated and highly sensitive tool that detects MCI in the older adult population. We will also investigate the perceived usefulness and integration of the SCREEN into primary care practice to identify demonstrable impacts on clinical workflow and health care providers’ (HCP) perceptions of its success as a screening tool. Patients’ perceptions of completing the SCREEN and its impact on their quality of life will also be explored.
This study has a concurrent, mixed methods, prospective, and quasi-experimental design. Participants will be recruited from 5 primary care family health teams (FHTs; defined by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants will include HCPs, patients, care partners, and FHT administrative executives. Patients 55 years and older with no history of diagnoses for MCI, dementia, or Alzheimer disease rostered in one of the FHTs participating in the study will be eligible to participate. Their care partners will help triangulate the qualitative data collected from patients. Participating FHTs will identify an occupational therapist from their site to participate in the study; this HCP will both administer the research protocol and participate in semistructured in-depth interviews and questionnaires. Principal component analysis will be conducted on the SCREEN data to understand the test components better. Tests comparing sensitivity, specificity, and test-retest reliability will assess the validity of SCREEN as a screening tool for MCI.
This paper describes the study protocol and its activities to date. Data collection was halted early because of COVID-19 restrictions on research activity, and data analysis is currently in progress.
At the end of the project, we anticipate having an initial comparative evaluation of the SCREEN as a tool for early detection of MCI in primary care older adult patient populations. Resource constraints on this research study limit our ability to conduct a randomized controlled trial; however, the results will assist developers of the SCREEN in determining whether rigorous controlled testing is warranted.
DERR1-10.2196/25520
More than cancer and cardiovascular disease, declining cognition threatens an individual’s ability to age in place by living independently at home alone or with family caregivers [
MCI is a neurocognitive disorder that describes a state between normal cognition and dementia is characterized by a slight but noticeable deterioration of cognitive abilities that predominantly impairs memory and thinking skills [
Early detection and diagnosis of declining cognition may not only allow health care providers (HCPs) to intervene in cases where the condition is reversible but also provide early and optimal management, tailored treatment planning, and timely access to education and psychosocial support to those at high risk for dementia [
Not screening for cognitive changes may be a lost opportunity to identify individuals before they are biomarker-positive [
Typically, screening for MCI is triggered when someone raises concerns about their memory and thinking abilities with their general practitioner [
The
Although not commonly used in Canada, the
Designing cognitive screens with greater ecological validity (ie, designing questions that are reflective of relevant life activities [
BrainFx is a for-profit firm that creates proprietary neurological assessment software designed to identify signs of brain function impairment. The BrainFx Screen (SCREEN) is an unvalidated, digitally administered, 15-minute, 7-question screen designed to identify early signs of MCI by assessing functional deficits that may not be readily identified by existing screens (refer to
Summary of SCREEN activities.
Activity | Description | Time to complete (s) |
Abstract reasoning | Twenty everyday items are displayed, and the patient touches the item on the screen and slides each item, one at a time, into 1 of the 5 categories into which they best belong, while being timed. | 90 |
Constructive ability | Two rounds of a photo being displayed and breaking into 9 pieces. The patient touches the pieces and slides each piece into a grid to reassemble, while being timed. | 90 |
Prioritizing | Five everyday activities or tasks are presented, and the patient is told what time of day it is (eg, 7 PM), and the patient touches the screen and slides each item to prioritize the order in which the activities or tasks should be completed. | 60 |
Numerical problem solving | Ten math questions requiring 1- or 2-digit answers are presented for a patient response using a numerical pad (+, −, ×, and /) while being timed. | 90 |
Visual-spatial ability | Two rounds of patient selecting (by touch) into which shape a word fits best, while being timed. | 30 |
Divided attention | The patient watches a pot on the stove about to boil over (denoted by boiling water and red signal) and must touch the pot and move it to the sink to dump out the water, while also touching the screen to match as many objects as they can within the kitchen scene. | 90 |
Route finding | A map is presented with roads and multiple locations. In the first round, the patient traces the most efficient route between 2 locations, while being timed. In the second round, the patient traces the most efficient route between 2 locations but is instructed to make 2 stops on the way, while being timed. | 90 |
The objectives of this research study are as follows:
To evaluate the psychometric properties of the SCREEN to assess functional competence and detect early signs of cognitive impairment when administered in a primary care setting to adults 55 years and older
To investigate the integration and use of the SCREEN in primary care practice and any demonstrable impact on clinical workflow and planning
To explore HCPs’, patients’, and care partners’ perspectives on adopting and using the SCREEN.
The study has a concurrent, mixed method, prospective, and quasi-experimental design. Participants will be recruited from 5 primary care family health teams (FHTs; defined by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. FHTs that employ a registered occupational therapist as staff will be eligible to participate in the study, and participating FHTs will receive a nominal compensatory payment for their time spent collecting data for the study by administering the SCREEN, Qmci, and Geriatric Anxiety Scale (GAS) [
Patients 55 years and older with no history of diagnoses for MCI, dementia, or Alzheimer disease rostered in one of the FHTs participating in the study will be eligible to participate. The age of eligibility includes those 55 years and older to capture an at-risk population with no current diagnosis of MCI who might be healthy or experiencing early symptoms of MCI that may or may not be apparent to the participant. Prospective participants may also be excluded based on a diagnosis with any of the following conditions that are associated with MCI or dementia-like symptoms (
Aged 55 years and older
Able to read and think in English
Rostered with a participating family health team
Depression:
Symptom-free for the last 6 months
No history of hospitalization
Use of low-dose antidepressants
Brain injury
Stroke
Taking prescription neuroleptics, hypnotics, or antiepileptics medications:
Whether taken for pain and not epilepsy
Low dose
Stable (no dose or medication change for the last 6 months)
Less than grade 6 education
Diagnosis of mild cognitive impairment, dementia, or Alzheimer disease
Color blindness
Paralysis (in hands)
Physical handicap that may influence test results
Epilepsy
Severe vision or hearing impairments (hearing aids are acceptable)
Psychopathology
Major depression (that has required hospitalization)
Diagnosed psychiatric disorders
Diagnosis of condition with susceptibility to causing dementia or cognitive deficits
Alcohol or drug dependence
Sleep apnea (with no use of continuous positive airway pressure machine)
Recruitment of patients will include diverse media strategies in both clinical and community settings. At the FHT, recruitment posters and 1-page summaries of the study will be posted in waiting rooms; exam rooms; and the FHT’s website and social media platforms, including Facebook and Twitter, where available. Recruitment posters will also be posted at public establishments local to the FHT (eg, YMCAs, libraries, pharmacies, and recreational sites). Interested participants may self-identify to the FHT HCP via telephone or to the research team via a dedicated study email or phone number. The research team will host information sessions at each participating FHT to provide clinical and nonclinical health team staff with information about the study to support seamless recruitment and onboarding of new patient participants.
Once enrolled in the study, the HCP will ask patients to identify a care partner (defined as someone who might be concerned about and or interested in the patient’s well-being), if there is one, who might be interested in participating in qualitative interviews and questionnaires as part of the study. This is not a requirement of the study; qualitative data collected from care partners will help triangulate the data collected from the patient participants. There are no eligibility requirements for care partners other than having a self-identified relationship with the patient and being able to read and write in English.
Participating FHTs will identify 1 occupational therapist from their site to participate in the study. This HCP will both administer the research protocol and participate in semistructured, in-depth interviews and questionnaires. To be eligible to participate, the HCP must have the approval to participate from the appropriate corporate agent (the Executive Director of the FHT). Before starting data collection, the HCP must complete a web-based training program—consisting of 3 self-directed training modules—and learn how to administer the Qmci to become a certified BrainFx administrator. The research team will conduct in-person training to cover the research protocol and administrative processes.
Where available, the managing director or equivalent at the FHT will be interviewed to better understand contextual factors such as workload, funding, and patient population characteristics that may impact the results.
The study protocol has been reviewed and has received ethics clearance from the Wilfrid Laurier University Research Ethics Committee (ORE# 5820) and has been reviewed and approved through each FHT’s research approval process. All participants (HCPs, patients, care partners, and administrative executives) will read and sign an information and informed consent package before participating in the study. We will conform to recommendations for acquiring informed consent and conducting qualitative interviews with persons with dementia when recruiting patients who may be affected by a neurocognitive disease [
The GAS-10 is a 10-item self-report screen for anxiety in older adults [
The SCREEN (version 0.5, beta) will be administered on a tablet (ASUS ZENPAD 10.1“ WXGA IPS Display, 1920×1200), powered by a quad-core 1.5 GHz, 64-bit MediaTek MTK 8163A processor with 2 GB RAM and 16 GB storage. The tablet comes with a tablet stand for optional use and a dedicated stylus that is recommended for completion of a subset of activities. At the start of the study, HCPs will be provided with identical tablets, preloaded with the BrainFx app software for use for the duration of the study.
Using a standardized administration protocol developed by BrainFx, the HCP will instruct the patient to use either their finger or the provided stylus to complete the SCREEN. Following acclimation to the tablet, the patient will be required to complete a short survey to collect demographic information (eg, age, the highest level of education attained) and any history of pre-existing conditions and questions about the patients’ state of well-being at the time of testing (eg, self-reported concerns about their thinking, mood, hours slept, and pain). The questionnaire will be immediately followed by 7 activities that are modeled after everyday real-world actions purported to evaluate functional competence related to a variety of cognitive domains, including abstract reasoning, divided attention, or visual-spatial abilities (refer to
At the end of each SCREEN, the patient will be prompted by the app to consent to contribute their scores to a database of results maintained by BrainFx, known as the LBB. The mean and SD of the LBB database will be updated in response to the addition of every new SCREEN. The patient’s performance on the SCREEN will be evaluated by comparing their results with the global reference population (ie, all available SCREEN results in the LBB at the time of testing). Filters are available that allow the HCP to compare the patients’ results with subcohorts using factors such as gender, education, age, or primary diagnosis. Individual SCREEN results reports display the individual’s activity score, the LBB mean (for the global reference population unless the operator selects subcohorts based on selected filters), and whether the activity score falls within 1 SD of the LBB mean. If the patient’s activity score falls 1 or more SD below the global mean, it is classified as an
The Qmci is a sensitive and specific screen that differentiates normal cognition from MCI [
In the Task Technology Fit (TTF) model, technologies refer to any tool(s) used to complete a task, and the task itself is any number of actions performed to complete the task. Operationally,
In this study, the questions on the TTF questionnaire are designed to measure how the SCREEN system (ie, assessment, handouts, hardware, and technical support) influences the HCP’s ability to screen for MCI. Tasks include, but are not limited to, collecting data from the patient that are relevant to MCI and using those data to make necessary decisions (eg, the decision to refer patients for further neuropsychiatric evaluation or provide particular intervention recommendations for MCI [
Care partners will complete the 12-item Zarit Burden Interview (ZBI) [
The summary of the study protocol for the data collection process is included for reference in
Study protocol data collection process. AOC: area of concern; ED: executive director; EMR: electronic medical record; ET: evaluation team; FHT: family health team; GAS: Geriatric Anxiety Scale; HCP: health care provider; IC: informed consent; MCI: mild cognitive impairment; PEF: patient encounter form; Qmci: Quick Mild Cognitive Impairment Screen.
Data collection for the psychometric evaluation study will take place in person at the patients’ respective FHT. Standard operating procedures have been developed for the research study, which will be followed by the research team and HCP staff trained in the study protocol and administration of all data collection tools. A member of the research team will observe at least one in-clinic HCP data collection visit to confirm adherence to the protocol and training procedures.
To assess the reliability and perceived usefulness of the SCREEN, depending upon when a patient enters the study, they may repeat the screening protocol up to 4 times. Each visit is structured around the administration of 2 MCI screens; the
The patients’ first appointment will take approximately 45-60 minutes (to account for onboarding), and all subsequent appointments (up to 4 screening test pairs, depending on the patient’s study entry date) will take approximately 45 minutes to complete. Rolling recruitment of patients will occur over an 18-month period and will end when a minimum of 2 and a maximum of 4 SCREEN-Qmci test pairs are completed every 12 weeks. In older adults, with no diagnosis of MCI or dementia, measures of cognitive abilities, including verbal fluency, attention, and intelligence [
The GAS-10 will be administered just before and immediately after the administration of the first MCI screen (eg, the SCREEN) and immediately followed by the administration of the second MCI screen (eg, the Qmci) at each appointment. After completing the 2 MCI screens, the HCP will manually calculate the results for the Qmci, log in to the BrainFx portal to retrieve the SCREEN report, and review both sets of results with the patient. The Qmci cut-off score for distinguishing MCI from normal cognition is ≤67/100 [
To determine the sensitivity and specificity of the SCREEN in comparison with the Qmci, the results of both screens will be classified in binary format as healthy or not healthy, where
Following the review of the results at each appointment, the HCP will provide patients with a handout that summarizes recommendations for supporting cognitive health. If the SCREEN identifies any
Semistructured, in-depth interviews will be conducted, in person or by telephone, with HCPs, executive directors, and a subset of patients and their care partners throughout the study. Interviews will be audio-recorded and transcribed verbatim. Data collection will be guided by the
HCPs will complete a TTF questionnaire (
Task Technology Fit model.
The quality of the information I receive from the BrainFx SCREEN and Report is enough to meet my clinical needs.
The BrainFx SCREEN and Report provides me with the right data I need to better support patients and caregivers.
I am able to quickly locate the results of the BrainFx SCREEN on a patient chart.
The data elements on the BrainFx Report are easy to understand or it is easy to find out.
Technical support to access the BrainFx app was always available when I needed it.
I can count on the BrainFx system to be “up and running” and available when I need it.
The tablets were subject to unexpected or inconvenient downtimes, which makes it harder for me to do my work.
The BrainFx app was subject to frequent problems.
It was easy to learn the BrainFx system.
I frequently deal with nonroutine cases of older adults with cognitive issues.
I frequently deal with routine cases of older adults with cognitive issues.
Identifying cases of mild cognitive impairment usually takes more than one clinician.
Sharing relevant and timely information with other care providers is important when diagnosing cognitive impairment.
The BrainFx system was convenient and easy to use.
There was not enough training for me on how to administer the BrainFx SCREEN using the tablets.
There was not enough training for me on how to use the BrainFx Report.
BrainFx support took an interest in helping me to solve problems to avoid disruptions to my workflow.
My overall effectiveness in detecting mild cognitive impairment increased when I used the BrainFx SCREEN and Report.
My ability to target interventions for individual patients and their needs was improved with the BrainFx SCREEN and Report.
My ability to target interventions for individual caregivers and their needs was improved with the BrainFx SCREEN and Report.
I waste less time interpreting test results and preparing interventions with BrainFx SCREEN and Report.
I spend less time writing up charting cognitive test results with the BrainFx SCREEN and Report.
The BrainFx SCREEN and Report provide better information to patients and their caregivers.
The quality of my follow-up recommendations to patients and caregivers has improved with my use of the BrainFx SCREEN and Report.
Descriptive and inferential analyses will be conducted using SPSS Statistics for Windows, Version 26 (IBM Corp). Qualitative analysis will be conducted using NVivo version 12 (QSR International Pty Ltd).
Descriptive data will be described using frequencies and percentiles and compared using the chi-square test or Fisher exact test as necessary. Continuous data will be analyzed for central tendency and variability; categorical data will be presented as proportions. Normality will be tested using the Shapiro-Wilk test, and nonparametric tests will be performed using the Mann-Whitney U test
MCI test outcome data will be coded into a binary format of
To assess the perceived usefulness of adopting the tablet-based SCREEN in a real-world clinical setting, HCPs, FHT executive administrators, patients, and their care partners will be interviewed upon entry and exit from the study. All HCPs and patients will be interviewed twice, and care partners will be sampled until saturation [
This funded research was launched in January 2019, and enrollment was conducted in February 2020. Quantitative data collection was interrupted in March 2020 because of the COVID-19 pandemic and the shutting down of all nonessential in-person clinical visits; qualitative data collection was concluded in July 2020. The results are forthcoming.
This research study will assess the psychometric properties and perceived usefulness of a novel tablet-based tool to screen for MCI in adults 55 years and older in the primary care setting. A fundamental objective of a screening test is to reduce morbidity or mortality in an at-risk population through early detection and treatment [
The assessment of this novel screen for MCI will be executed within the constraints of limited financial resources. As such, it will methodologically constrain our use of a presumed
Although patient participation in the study is voluntary (there is no randomization or selection for particular traits), we anticipate that those with concerns about their cognition and those more comfortable with the use of tablet technology may be more likely to self-select into the study. Our data collection and statistical analysis will account for both the potentialities.
activities of daily living
electronic medical record
family health team
Geriatric Anxiety Scale
health care provider
instrumental activities of daily living
Living Brain Bank
mild cognitive impairment
Mini-Mental State Exam
Montreal Cognitive Assessment 8.1
Quick Mild Cognitive Impairment
Task Technology Fit
Zarit Burden Interview
Participating FHTs will receive a nominal compensatory payment for their OTs’ time spent administering the SCREEN, collecting data for the research study, and communicating with the research team. The research team would like to thank the Ontario Centers of Excellence Health Technologies Fund on behalf of the Ontario Ministry of Health and Long-Term Care for their support of this research study and research assistants Kelly Xu, Sharmin Sharker, and Muhammad Umair.
None declared.