A Pragmatic Cluster Randomized Trial of an Electronic Clinical Decision Support System to Improve Chronic Kidney Disease Management in Primary Care: Design, Rationale, and Implementation Experience

Background The diagnosis of chronic kidney disease (CKD) is based on laboratory results easily extracted from electronic health records; therefore, CKD identification and management is an ideal area for targeted electronic decision support efforts. Early CKD management frequently occurs in primary care settings where primary care providers (PCPs) may not implement all the best practices to prevent CKD-related complications. Few previous studies have employed randomized trials to assess a CKD electronic clinical decision support system (eCDSS) that provided recommendations to PCPs tailored to each patient based on laboratory results. Objective The aim of this study was to report the trial design and implementation experience of a CKD eCDSS in primary care. Methods This was a 3-arm pragmatic cluster-randomized trial at an academic general internal medicine practice. Eligible patients had 2 previous estimated-glomerular-filtration-rates by serum creatinine (eGFRCr) <60 mL/min/1.73m2 at least 90 days apart. Randomization occurred at the PCP level. For patients of PCPs in either of the 2 intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine albumin-creatinine-ratio) at the beginning of the study period, to be completed when acquiring labs for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. The eCDSS delivered individualized guidance on cardiovascular risk-reduction, potassium and proteinuria management, and patient education. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education. The primary clinical outcome is blood pressure change from baseline at 6 months after the end of the trial, and the main secondary outcome is provider awareness of CKD diagnosis. We also collected process, patient-centered, and implementation outcomes. Results A multidisciplinary team (primary care internist, nephrologists, pharmacist, and informaticist) designed the eCDSS to integrate into the current clinical workflow. All 81 PCPs contacted agreed to participate and were randomized. Of 995 patients initially eligible by eGFRCr, 413 were excluded per protocol and 58 opted out or withdrew, resulting in 524 patient participants (188 usual care; 165 eCDSS; and 171 eCDSS+). During the 12-month intervention period, 53.0% (178/336) of intervention patient participants completed triple-marker labs. Among these, 138/178 (77.5%) had a PCP appointment after the triple-marker labs resulted; the eCDSS was opened for 73.9% (102/138), with orders or education signed for 81.4% (83/102). Conclusions Successful integration of an eCDSS into primary care workflows and high eCDSS utilization rates at eligible visits suggest this tailored electronic approach is feasible and has the potential to improve guideline-concordant CKD care. Trial Registration ClinicalTrials.gov NCT02925962; https://clinicaltrials.gov/ct2/show/NCT02925962 (Archived by WebCite at http://www.webcitation.org/78qpx1mjR) International Registered Report Identifier (IRRID) DERR1-10.2196/14022

1a-i) Identify the mode of delivery in the title Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if o ine products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.
Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 1a-ii) Non-web-based components or important co-interventions in title Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").
Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not in title, but described within the protocol (the pharmacist outreach in the eCDSS+ intervention arm) 1a-iii) Primary condition or target group in the title Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Keep in mind the needs of systematic reviewers and indexers by including important synonyms. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "This was a 3-arm pragmatic cluster-randomized trial at an academic general internal medicine practice." "The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triplemarker testing. The eCDSS delivered individualized guidance on cardiovascular riskreduction, potassium and proteinuria management, and patient education. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education." https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lNS… 11/69 subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "This was a 3-arm pragmatic cluster-randomized trial at an academic general internal medicine practice." "The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triplemarker testing. The eCDSS delivered individualized guidance on cardiovascular riskreduction, potassium and proteinuria management, and patient education. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education." 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-toface assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. o ine), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional o ine trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lNS… 12/69 subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "This was a 3-arm pragmatic cluster-randomized trial at an academic general internal medicine practice. Eligible patients had 2 previous estimated-glomerularltration-rates by serum creatinine (eGFRCr) <60 mL/min/1.73m2 at least 90 days apart. Randomization occurred at the PCP level. For patients of PCPs in either of the 2 intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine albumin-creatinine-ratio) at the beginning of the study period, to be completed when acquiring labs for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. The eCDSS delivered individualized guidance on cardiovascular risk-reduction, potassium and proteinuria management, and patient education. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education." 1b-iv) RESULTS section in abstract must contain use data Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) INTRODUCTION 2a) In INTRODUCTION: Scienti c background and explanation of rationale Does your paper address subitem 1b-iv? Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "All 81 PCPs contacted agreed to participate and were randomized. Of 995 patients initially eligible by eGFRCr, 413 were excluded per protocol and 58 opted out or withdrew, resulting in 524 patient participants (188 usual care; 165 eCDSS; and 171 eCDSS+). During the 12-month intervention period, 53.0% (178/336) of intervention patient participants completed triple-marker labs. Among these, 138/178 (77.5%) had a PCP appointment after the triple-marker labs resulted; the eCDSS was opened for 73.9% (102/138), with orders or education signed for 81.4% (83/102)." 1b-v) CONCLUSIONS/DISCUSSION in abstract for negative trials Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-v?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No; this is a protocol paper. Primary outcome of trial not reported. Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "This was a 3-arm cluster randomized controlled trial with randomization at the provider level. There was 1 usual care control arm and 2 intervention arms. " Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No major trial changes 3b-i) Bug xes, Downtimes, Content Changes Bug xes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug xes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have in uenced study design such as staff changes, system failures/downtimes, etc. [2].

4a) Eligibility criteria for participants
Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "The study team devoted substantial time to troubleshoot unanticipated problems with the eCDSS programming and capture of outcomes. Every update or change to the EHR processes had the potential to disrupt the trial implementation. The team regularly validated the eCDSS and data capture through manual chart reviews. The inaccuracies of EHR data extraction are well-documented [57]. Despite the presence of an explicit trial infrastructure in our EHR, implementation of the intervention and data collection was far from seamless." subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "All providers practicing at the general internal medicine practice with a primary care panel were eligible for inclusion and randomization.
Eligible patients were identi ed using a previously described and validated algorithm [40]. (Patients were considered eligible for inclusion if they were aged 18 to 80 years; had a preferred language of English, Spanish, or Chinese; had at least 2 outpatient eGFRCr=30 to 59 milliliters/minute (mL/min) at least 90 days apart, one of which was within the 12 months before August 14, 2017 (when providers were rst contacted for study recruitment); and had a primary care visit in the past 18 months.
(see Figure 1 for the Consolidated Standards of Reporting Trial diagram) Participants were automatically excluded (based on EHR data) if they were deceased; diagnosed with end-stage renal disease; engaged with a nephrology clinic with 2 or more visits in the past 12 months; kidney transplant recipients; or diagnosed with dementia. PCPs were then sent a list of potential participants for inclusion and asked to exclude participants for additional criteria that were unreliably captured when extracted from the EHR: current pregnancy, life expectancy <6 months, limited communication ability owing to impaired cognition or severe mental illness, New York Heart Association Class III/IV heart failure, or known ejection fraction <25%. PCPs could also exclude any other participants they felt would be inappropriate for study staff to contact.

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clari ed. Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No; not relevant as intervention directed at clinicians 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. o ine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email con rmation, phone calls) were used to detect/prevent these. "We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP's assignment. The study statistician was blinded to the identi cation of the PCPs and patients and will be similarly blinded for the analyses." 4a-iii) Information giving during recruitment Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results. Primary outcome data on BP will be extracted from the EHR. Baseline BP was de ned as the most recent ambulatory BP measurement before nal enrollment on October 4, 2017. All patient participant ambulatory BP measurements were collected during the intervention period and will continue to be collected for the 6 months after the end of the intervention.
Secondary outcomes will be collected primarily using automated EHR data extraction algorithms. The outcomes acquired through data extraction include all secondary outcomes except for provider and patient-centered outcomes, which are collected via surveys. Up to 25 participants in each of the 3 arms were surveyed to assess patient knowledge about CKD and NSAID 5) The interventions for each group with su cient details to allow replication, including how and when they were actually administered 4b-i) Report if outcomes were (self-)assessed through online questionnaires Clearly report if outcomes were (self-)assessed through online questionnaires (as common in webbased trials) or otherwise.
Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "All participating intervention PCPs will be surveyed about their perception of the feasibility and utility of the eCDSS and pharmacist calls (for CDSS+ providers)."

4b-ii) Report how institutional a liations are displayed
Report how institutional a liations are displayed to potential participants [on ehealth media], as a liations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Not a required item -describe only if this may bias results) Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Patients receive care at UCSF so aware of institutional a liation of study Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "The EHR used within this practice is Epic (EpicCare, Epic Systems)." "To optimize the chance of a successful intervention, we worked with a multidisciplinary team (general internist, informatics specialist, and nephrologists) to develop the eCDSS logic. The team focused on areas with the strongest evidence base for CKD management in primary care and developed the logic to apply guideline recommendations in real-time for individual patients."

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results. subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "To optimize the chance of a successful intervention, we worked with a multidisciplinary team (general internist, informatics specialist, and nephrologists) to develop the eCDSS logic. The team focused on areas with the strongest evidence base for CKD management in primary care and developed the logic to apply guideline recommendations in real-time for individual patients." "We elicited feedback from PCPs in focus group settings during development of the eCDSS and during implementation planning, which is a recommended practice to increase adoption and uptake of eCDSS [23]. Focus group discussions centered on eCDSS messaging and orders, as well as ways to facilitate integration into the usual clinical work ow."

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial. Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).
Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not relevant; eCDSS created only once Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "The study team devoted substantial time to troubleshoot unanticipated problems with the eCDSS programming and capture of outcomes. Every update or change to the EHR processes had the potential to disrupt the trial implementation. The team regularly validated the eCDSS and data capture through manual chart reviews. The inaccuracies of EHR data extraction are well-documented" 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing owcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing owcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scienti c reporting.
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lNS… 29/69 "After the triple-marker testing was ordered, the next time the patient participant visited the lab for their regular clinical care, the triple-marker labs were also collected. We programmed the eCDSS to trigger the rst time that the patient participant visited their PCP after all 3 lab results were available. (see Figure 2 for the study work ow) The eCDSS best practice advisory (BPA) appeared above the current medications in the electronic chart ( Figures 3 and 4)." "For patient participants categorized as high risk, when the PCP accepted the BPA, a SmartSet (EpicCare, Epic Systems) of tailored orders and recommendations appeared ( Figure 5)." "In addition to a reminder of BP targets in CKD, the SmartSet for patient participants classi ed as high risk for progression and/or complications of CKD used real-time EHR lab and medication data to recommend any of the following that applied:

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speci c group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not entirely relevant, as it is an CDSS system embedded in local EHR "After the triple-marker testing was ordered, the next time the patient participant visited the lab for their regular clinical care, the triple-marker labs were also collected. We programmed the eCDSS to trigger the rst time that the patient participant visited their PCP after all 3 lab results were available." Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lNS… 33/69 "Therefore, we utilized several theory-based strategies to encourage provider uptake of the eCDSS and participant uptake of provider and/or pharmacist recommendations. We classify these strategies using the capability, opportunity, motivation, behavior framework (or COM-B), which asserts that capability, opportunity, and motivation are essential conditions that impact behavior [44,45].
To encourage provider uptake and use of CKD guidelines, we addressed capability barriers (such as knowledge about the guidelines or the eCDSS or forgetting to discuss CKD owing to limited bandwidth

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study See Figure 3 5-x) Clarify the level of human involvement Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability). Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "To optimize the chance of a successful intervention, we worked with a multidisciplinary team (general internist, informatics specialist, and nephrologists) to develop the eCDSS logic." "The second intervention arm ( subitem not at all important 1 2 3 4 5 essential 5-xi) Report any prompts/reminders used Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study See gure 3. "We programmed the eCDSS to trigger the rst time that the patient participant visited their PCP after all 3 lab results were available. (see Figure 2 for the study work ow) The eCDSS best practice advisory (BPA) appeared above the current medications in the electronic chart" Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study See gure 6 (outcomes) "Primary Outcome A summary of pre-planned study outcomes is shown in Figure  6. In addition to the effectiveness outcomes described above, the implementation outcomes are described in detail in Figure 6 using the reach, effectiveness, adoption, implementation, and Primary outcome data on BP will be extracted from the EHR.
Baseline BP was de ned as the most recent ambulatory BP measurement before nal enrollment on October 4, 2017. All patient participant ambulatory BP measurements were collected during the intervention period and will continue to be collected for the 6 months after the end of the intervention.
Secondary outcomes will be collected primarily using automated EHR data extraction algorithms. The outcomes acquired through data extraction include all secondary outcomes except for provider and patient-centered outcomes, which are collected via surveys. Up to 25 participants in each of the 3 arms were surveyed to assess patient knowledge about CKD and NSAID avoidance. Patient participants in the eCDSS arm were surveyed 2 weeks after a visit in which the SmartSet was utilized; those in the eCDSS+ arm were surveyed 2 weeks after their pharmacist follow-up call, and those in the usual care arm were surveyed 2 weeks after a visit with their PCP during the study period. All participating intervention PCPs will be surveyed about their perception of the feasibility and utility of the eCDSS and pharmacist calls (for CDSS+ providers)." Does your paper address subitem 6a-i?
Copy and paste relevant sections from manuscript text N/A: no online questionnaires currently 6a-ii) Describe whether and how "use" (including intensity of use/dosage) was de ned/measured/monitored Describe whether and how "use" (including intensity of use/dosage) was de ned/measured/monitored (logins, log le analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.
Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text See gure 6: implementation outcomes "In addition to the effectiveness outcomes described above, the implementation outcomes are described in detail in Figure 6 using the reach, effectiveness, adoption, implementation, and subitem not at all important 1 2 3 4 5 essential 6b) Any changes to trial outcomes after the trial commenced, with reasons 7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).
Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text N/A: protocol paper. Planned qualitative data collection reported.
"Patient participants in the eCDSS arm were surveyed 2 weeks after a visit in which the SmartSet was utilized; those in the eCDSS+ arm were surveyed 2 weeks after their pharmacist follow-up call, and those in the usual care arm were surveyed 2 weeks after a visit with their PCP during the study period. All participating intervention PCPs will be surveyed about their perception of the feasibility and utility of the eCDSS and pharmacist calls (for CDSS+ providers)." Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group 8b) Type of randomisation; details of any restriction (such as blocking and block size) 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP's assignment." Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP's assignment." 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP's assignment." Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP's assignment." 11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).
Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "The study statistician was blinded to the identi cation of the PCPs and patients and will be similarly blinded for the analyses." Care providers and patients were not blinded.
11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator". subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is a protocol paper that reports planned analyses: "Baseline demographic and clinical characteristics will be summarized and study balance by study arm will be assessed using descriptive statistics as well as methods that account for the lack of independence among patients in the same cluster [48]. Balance will be assessed with logit link generalized estimating equation (GEE) for categorical variables and linear GEE models for continuous variables. Clustering will be accounted for using an exchangeable correlation matrix with robust standard errors. We anticipate controlling for characteristics that are associated with outcomes but differ at baseline between arms and using multivariable regression analyses with GEE to account for cluster randomization in the reporting of the nal outcomes of our study. Sensitivity analyses will include as treated analyses with inverse probability weighting to determine impact of participants changing providers/study arms as well as to determine impact of any differences in the number of assessments or clinic visits by study arm. Primary analyses will follow intention-to-treat principles." 12a-i) Imputation techniques to deal with attrition / missing values Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4] Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A: protocol paper Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Baseline demographic and clinical characteristics will be summarized and study balance by study arm will be assessed using descriptive statistics as well as methods that account for the lack of independence among patients in the same cluster [48]. Balance will be assessed with logit link generalized estimating equation (GEE) for categorical variables and linear GEE models for continuous variables. Clustering will be accounted for using an exchangeable correlation matrix with robust standard errors. We anticipate controlling for characteristics that are associated with outcomes but differ at baseline between arms and using multivariable regression analyses with GEE to account for cluster randomization in the reporting of the nal outcomes of our study. Sensitivity analyses will include as treated analyses with inverse probability weighting to determine impact of participants changing providers/study arms as well as to determine impact of any differences in the number of assessments or clinic visits by study arm. Primary analyses will follow intention-to-treat principles." subitem not at all important 1 2 3 4 5 essential Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "We recruited PCPs with eligible patients by eGFRCr by email between August 14, 2017 and September 8, 2017. PCPs were given 2 weeks to either opt-out entirely or exclude individual patients. We then randomized participating PCPs as described below. Participation letters were mailed to eligible intervention arm patients between September 1, 2017 and September 27, 2017, and patients were given 2 weeks to either return an opt-out card or call the study coordinator to opt out. Patients opting out after the study start date of October 4, 2017, were considered withdrawn from the study.
The usual care group PCPs received no additional contact beyond recruitment from the study team, and the usual care patients were never contacted directly by the study team"

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lNS… 51/69 RESULTS 13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Given that the study physician ordered labs at the start of the intervention period, safety checks were put into place to ensure adequate review of the laboratory results. A nephrologist (LL) from the study team reviewed weekly laboratory results speci cally to identify the following: eGFRCr decline >30% from baseline, ACR ≥1000 mcg/mg, adherence to nephrology referrals, and any discordance >30% between eGFRCr and eGFRCys. In all of these situations, the nephrologist contacted the PCP to ensure appropriate follow-up. Speci cally, in cases of discordance between the 2 types of eGFR levels, the nephrologist advised PCPs to dose medications based on eGFRCys when the clinical scenario suggested that eGFRcr may not be accurate." https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lNS… 52/69 Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not fully applicable: protocol paper.
"Recruitment Rate At the provider level, all 81 eligible PCPs (47 faculty attending physicians, 31 resident physicians, and 3 NPs) agreed to participate (100%). In total, 79 of the 81 providers (98%) were included because 2 providers had no remaining eligible participants after exclusion and opt-outs. At the patient level, 995 patients who met the initial eligibility criteria were identi ed. A total of 316 / 995 (31.8%) potential patient participants were excluded using automated algorithms, and clinicians excluded another 90 patients (Figure 1). A total of 582 patient participants were distributed to the 3 arms based on provider randomization. After an additional 3 patients were excluded (owing to patient deaths not recorded in EHR before randomization) and 55 / 582 patients (9.5%) opted out or withdrew, 524 patient participants ( 13b) For each group, losses and exclusions after randomisation, together with reasons intervention participants.
Implementation Outcomes: Adoption SmartSet Use: Of the 102 participants in which the BPA was opened, orders were signed or patient education" Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT ow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Also see Figure 1: consort diagram ""Recruitment Rate At the provider level, all 81 eligible PCPs (47 faculty attending physicians, 31 resident physicians, and 3 NPs) agreed to participate (100%). In total, 79 of the 81 providers (98%) were included because 2 providers had no remaining eligible participants after exclusion and opt-outs. At the patient level, 995 patients who met the initial eligibility criteria were identi ed. A total of 316 / 995 (31.8%) potential patient participants were excluded using automated algorithms, and clinicians excluded another 90 patients (Figure 1). A total of 582 patient participants were distributed to the 3 arms based on provider randomization. After an additional 3 patients were excluded (owing to patient deaths not recorded in EHR before randomization) and 55 / 582 patients (9.5%) opted out or withdrew, 524 patient participants ( included in each analysis and whether the analysis was by original assigned groups Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A: trial did not end or stop early Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study See Table 1 15-i) Report demographics associated with digital divide issues In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study See 16-i) Report multiple "denominators" and provide de nitions Report multiple "denominators" and provide de nitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speci c pre-de ned time points of interest (in absolute and relative numbers per group). Always clearly de ne "use" of the intervention.
Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study See gure 1 (consort diagram). Data collection not nalized. This is a protocol paper

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Baseline demographic and clinical characteristics will be summarized and study balance by study arm will be assessed using descriptive statistics as well as methods that account for the lack of independence among patients in the same cluster [48]. Balance will be assessed with logit link generalized estimating equation (GEE) for categorical variables and linear GEE models for continuous variables. Clustering will be accounted for using an exchangeable correlation matrix with robust standard errors. We anticipate controlling for characteristics that are associated with outcomes but differ at baseline between arms and using multivariable regression analyses with GEE to account for cluster randomization in the reporting of the nal outcomes of our study. Sensitivity analyses will include as treated analyses with inverse probability weighting to determine impact of participants changing providers/study arms as well as to determine impact of any differences in the number of assessments or clinic visits by study arm. Primary analyses will follow intention-to-treat principles." Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study subitem not at all important 1 2 3 4 5 essential 17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended 17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational de nitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is de ned (e.g., timeout after idle time) [1] (report under item 6a).
Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).
Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).
Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not fully applicable -protocol paper.
"In this report, we describe the rationale, design, and initial implementation outcomes of a 3-arm pragmatic trial that assessed the feasibility and preliminary effectiveness of an eCDSS to improve CKD management in primary care compared with usual care. This study builds on previous pragmatic trials focused on EHR-based interventions to improve CKD management [28,29,51,52] and supports the feasibility of using EHRs to identify study participants, intervene in early CKD management, and measure study outcomes. Given its design as a pragmatic trial (per Pragmatic Explanatory Continuum Indicator Summary, or PRECIS, criteria [53]), we found high rates of participation by providers and low opt-out rates by patients. We also found high rates of eCDSS use by providers." Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events. Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "As a pilot study, this study was limited by its inclusion of patients and providers from a single institution. Therefore, both the intervention and strategies to encourage intervention uptake were adapted to meet the needs of this single institution that uses Epic Systems' EHR. However, we have found that the patient population in this clinic is similar to others [60], and Epic is a widely used EHR in multiple health systems. As a pragmatic trial, our sample size was smaller than initially anticipated and will thus ultimately impact our power to detect changes in our primary outcome. The short follow-up period for this pilot study will also limit our ability to assess some clinical outcomes. However, we were still able to use this pilot study to determine the feasibility of this intervention."

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "As a pilot study, this study was limited by its inclusion of patients and providers from a single institution. Therefore, both the intervention and strategies to encourage intervention uptake were adapted to meet the needs of this single institution that uses Epic Systems' EHR. However, we have found that the patient population in this clinic is similar to others [60], and Epic is a widely used EHR in multiple health systems. As a pragmatic trial, our sample size was smaller than initially anticipated and will thus ultimately impact our power to detect changes in our primary outcome. The short follow-up period for this pilot study will also limit our ability to assess some clinical outcomes. However, we were still able to use this pilot study to determine the feasibility of this intervention." 21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.
Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Trial Registration: ClinicalTrials.gov NCT02925962; https://clinicaltrials.gov/ct2/show/NCT02925962 (Archived by WebCite at http://www.webcitation.org/78qpx1mjR)" Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is a protocol paper. Full protocol can be accessed by contacting the author team.
Does your paper address CONSORT subitem 25? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "LK and CP received funding from the National Institute of Diabetes and Digestive In addition to the usual declaration of interests ( nancial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.
Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is an internal eCDSS was developed by the authorship team.
As a result of using this checklist, did you make changes in your manuscript? * What were the most important changes you made as a result of using this checklist? STOP -Save this form as PDF before you click submit To generate a record that you lled in this form, we recommend to generate a PDF of this page (on a Mac, simply select "print" and then select "print as PDF") before you submit it.
When you submit your (revised) paper to JMIR, please upload the PDF as supplementary le.
Don't worry if some text in the textboxes is cut off, as we still have the complete information in our database. Thank you!