This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
The diagnosis of chronic kidney disease (CKD) is based on laboratory results easily extracted from electronic health records; therefore, CKD identification and management is an ideal area for targeted electronic decision support efforts. Early CKD management frequently occurs in primary care settings where primary care providers (PCPs) may not implement all the best practices to prevent CKD-related complications. Few previous studies have employed randomized trials to assess a CKD electronic clinical decision support system (eCDSS) that provided recommendations to PCPs tailored to each patient based on laboratory results.
The aim of this study was to report the trial design and implementation experience of a CKD eCDSS in primary care.
This was a 3-arm pragmatic cluster-randomized trial at an academic general internal medicine practice. Eligible patients had 2 previous estimated-glomerular-filtration-rates by serum creatinine (eGFRCr) <60 mL/min/1.73m2 at least 90 days apart. Randomization occurred at the PCP level. For patients of PCPs in either of the 2 intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine albumin-creatinine-ratio) at the beginning of the study period, to be completed when acquiring labs for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. The eCDSS delivered individualized guidance on cardiovascular risk-reduction, potassium and proteinuria management, and patient education. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education. The primary clinical outcome is blood pressure change from baseline at 6 months after the end of the trial, and the main secondary outcome is provider awareness of CKD diagnosis. We also collected process, patient-centered, and implementation outcomes.
A multidisciplinary team (primary care internist, nephrologists, pharmacist, and informaticist) designed the eCDSS to integrate into the current clinical workflow. All 81 PCPs contacted agreed to participate and were randomized. Of 995 patients initially eligible by eGFRCr, 413 were excluded per protocol and 58 opted out or withdrew, resulting in 524 patient participants (188 usual care; 165 eCDSS; and 171 eCDSS+). During the 12-month intervention period, 53.0% (178/336) of intervention patient participants completed triple-marker labs. Among these, 138/178 (77.5%) had a PCP appointment after the triple-marker labs resulted; the eCDSS was opened for 73.9% (102/138), with orders or education signed for 81.4% (83/102).
Successful integration of an eCDSS into primary care workflows and high eCDSS utilization rates at eligible visits suggest this tailored electronic approach is feasible and has the potential to improve guideline-concordant CKD care.
ClinicalTrials.gov NCT02925962; https://clinicaltrials.gov/ct2/show/NCT02925962 (Archived by WebCite at http://www.webcitation.org/78qpx1mjR)
DERR1-10.2196/14022
Chronic kidney disease (CKD) is common in adults; in the United States, 14% of all adults and nearly half of individuals aged ≥70 years have CKD [
With the adoption of electronic health records (EHRs), there have been opportunities to implement low-cost interventions to improve patient care. Given that CKD diagnosis is based on laboratory results easily extracted from EHRs, CKD identification and management is an ideal area for targeted electronic efforts [
Team-based approaches for providing chronic disease management in primary care practice have also been shown to improve patient outcomes [
Given the promise of both eCDSS and team-based care to improve guideline-concordant CKD care in primary care practice, we designed a pragmatic cluster randomized 3-arm trial (usual care; eCDSS; and eCDSS plus pharmacist follow-up). This trial aimed to assess the feasibility and impact of a CKD eCDSS with individualized recommendations and any additional benefit of pharmacist outreach to follow-up compared with usual care. In this paper, we have reported the study design and initial implementation outcomes of this trial.
This was a 3-arm cluster randomized controlled trial with randomization at the provider level. There was 1 usual care control arm and 2 intervention arms. For patients of PCPs in the intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine ACR) at the beginning of the study period, to be completed when the patient visited the lab for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education after a PCP visit in which the eCDSS was utilized. We planned for an 18-month trial beginning October 4, 2017, that included a 12-month intervention period and subsequent 6-month follow-up period. This trial was registered on ClinicalTrials.gov (NCT02925962). The University of California San Francisco Human Research Protection Program approved the protocol for this study.
This study was conducted at a general internal medicine practice (with 2 locations) at the University of California San Francisco that cares for a diverse population of more than 24,000 adult patients. The PCPs in this practice include faculty attending physicians, resident physicians, and nurse practitioners (NPs). This practice is a certified primary care medical home (PCMH) that follows a team-based approach to care; each of the 10 teams includes 10 to 14 part-time PCPs, 2 medical assistants, a licensed vocational nurse, and administrative staff. Additional personnel of the PCMH include 5 registered nurses, 2 pharmacy technicians, and a clinical pharmacist available 1 day a week. The EHR used within this practice is Epic (EpicCare, Epic Systems).
To optimize the chance of a successful intervention, we worked with a multidisciplinary team (general internist, informatics specialist, and nephrologists) to develop the eCDSS logic. The team focused on areas with the strongest evidence base for CKD management in primary care and developed the logic to apply guideline recommendations in real-time for individual patients. In the first step, patients were classified as either at high risk (eGFRCys <60 mL/min and/or ACR >30 mcg/mg) or low risk (eGFRCys >60 mL/min and ACR <30 mcg/mg) for experiencing CKD-related complications [
We elicited feedback from PCPs in focus group settings during development of the eCDSS and during implementation planning, which is a recommended practice to increase adoption and uptake of eCDSS [
All providers practicing at the general internal medicine practice with a primary care panel were eligible for inclusion and randomization.
Eligible patients were identified using a previously described and validated algorithm [
Flow diagram of participant selection. Note that 1 clinician who was randomized did not ultimately participate because their only eligible patient opted out. eGFRCr: estimated glomerular filtration rate by serum creatinine; eCDSS: electronic clinical decision support system trial arm; eCDSS+: electronic clinical decision support system and pharmacist follow-up trial arm; N/A: not applicable.
We recruited PCPs with eligible patients by eGFRCr by email between August 14, 2017 and September 8, 2017. PCPs were given 2 weeks to either opt-out entirely or exclude individual patients. We then randomized participating PCPs as described below. Participation letters were mailed to eligible intervention arm patients between September 1, 2017 and September 27, 2017, and patients were given 2 weeks to either return an opt-out card or call the study coordinator to opt out. Patients opting out after the study start date of October 4, 2017, were considered withdrawn from the study.
The usual care group PCPs received no additional contact beyond recruitment from the study team, and the usual care patients were never contacted directly by the study team. Usual care PCPs did have access to the same labs and medications to use at their clinical discretion without any recommendations or direction from the study team.
We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP’s assignment. The study statistician was blinded to the identification of the PCPs and patients and will be similarly blinded for the analyses.
Before the eCDSS rollout but after randomization, all participating PCPs randomized to an intervention arm were asked to watch an educational video that provided background on guideline-based CKD care, as well as information on what to expect when an enrolled patient participant came in for a visit. PCPs were given an incentive of US $10 and a chocolate bar upon watching the video. Overall, 69.8% (72.0% eCDSS and 67.9% eCDSS+) viewed the video.
Before the eCDSS rollout, between October 5, 2017 and October 12, 2017, one of the study investigators (LK) ordered triple-marker testing for each patient participant from the eCDSS arm (n=169) and each patient participant from the eCDSS+ arm (n=177). We excluded 48 patients who opted out. In total, 10 patients withdrew later.
After the triple-marker testing was ordered, the next time the patient participant visited the lab for their regular clinical care, the triple-marker labs were also collected. We programmed the eCDSS to trigger the first time that the patient participant visited their PCP after all 3 lab results were available. (see
Flow diagram of study workflow for intervention arms. BPA: best practice advisory; PCP: primary care provider.
Electronic clinical decision support system trial arm low-risk best practice advisory. CKD: chronic kidney disease; Cr: serum creatinine; eGFR: estimated glomerular filtration rate.
Electronic clinical decision support system trial arm high-risk best practice advisory. BP: blood pressure; CKD: chronic kidney disease; mg/g: milligrams per grams.
Electronic clinical decision support system trial arm example SmartSet. ACEi/ARB: angiotensin converting enzyme inhibitor / angiotensin II receptor blocker; AVS: after visit summary; CKD: chronic kidney disease; eCDSS: electronic clinical decision support system; CDSS+: electronic clinical decision support system and pharmacist follow-up trial arm; K+: potassium; mg/g: milligrams per grams; NSAID: nonsteroidal anti-inflammatory drug; OTC: over-the-counter;.
In addition to a reminder of BP targets in CKD, the SmartSet for patient participants classified as high risk for progression and/or complications of CKD used real-time EHR lab and medication data to recommend any of the following that applied:
Cardiovascular risk reduction: recommendation for use of statin in all individuals aged ≥50 years not already on a statin, as recommended by Kidney Disease: Improving Global Outcomes (KDIGO) guidelines [
Potassium management: diet and diuretic recommendations.
Proteinuria management: initiation or titration of renin-angiotensin blockade with angiotensin converting enzyme inhibitors (ACEi) or angiotensin II receptor blockers (ARB).
Nephrology referral for highest risk participants: highest risk was defined as one of: confirmed eGFRCys <30 mL/min; potassium >5.5 mEq/L; ACR >300 mcg/mg, systolic blood pressure (SBP) >150 millimeters of mercury (mm Hg) despite 3 agents including a diuretic; >3% risk of five-year progression kidney failure based on Tangri equation [
Patient education: patient education materials to populate in the after-visit summary (AVS) focused on CKD general information, avoidance of nonsteroidal anti-inflammatory drugs (NSAIDs), and dietary recommendations.
The second intervention arm (eCDSS+) included a pharmacist follow-up visit by telephone. If the SmartSet was signed during the PCP visit, a pharmacist would call the participant within 2 weeks of the visit to reinforce CKD-related teaching and medication changes from the visit as well as complete a comprehensive medication review. Information on the telephone encounter was documented in the EHR and sent to the PCP; any urgent clinical issues identified during the call were highlighted for the PCP as needing their follow-up. Providers in the eCDSS+ group also were encouraged to make a warm handoff to the pharmacist by including anticipatory guidance about the pharmacist phone call within the after-visit summary and distributing a business card with the pharmacist’s photo and phone number for their patient participants at the time of using the SmartSet.
Given that the study physician ordered labs at the start of the intervention period, safety checks were put into place to ensure adequate review of the laboratory results. A nephrologist (LL) from the study team reviewed weekly laboratory results specifically to identify the following: eGFRCr decline >30% from baseline, ACR ≥1000 mcg/mg, adherence to nephrology referrals, and any discordance >30% between eGFRCr and eGFRCys. In all of these situations, the nephrologist contacted the PCP to ensure appropriate follow-up. Specifically, in cases of discordance between the 2 types of eGFR levels, the nephrologist advised PCPs to dose medications based on eGFRCys when the clinical scenario suggested that eGFRcr may not be accurate.
Implementation studies have shown that interventions designed with a theoretical basis are more likely to be successful [
To encourage provider uptake and use of CKD guidelines, we addressed
To address PCP
To address PCP
Although the eCDSS itself was a provider-facing intervention, we used strategies to encourage patient participant behavior change. To address
For patient participants, the EHR was used to collect baseline data about demographic characteristics (age, sex, race-ethnicity, preferred language, and insurance status); medical co-morbidities (cerebrovascular disease, congestive heart failure, coronary artery disease, diabetes mellitus, hyperlipidemia, and hypertension); current medication prescriptions (statin, ACEi/ARB, and diuretics); previous documentation of CKD on problem list and/or as a visit diagnosis, most recent blood pressure, eGFRCr, eGFRCys (if available), and urine ACR (if available) before enrollment.
Primary outcome data on BP will be extracted from the EHR. Baseline BP was defined as the most recent ambulatory BP measurement before final enrollment on October 4, 2017. All patient participant ambulatory BP measurements were collected during the intervention period and will continue to be collected for the 6 months after the end of the intervention.
Secondary outcomes will be collected primarily using automated EHR data extraction algorithms. The outcomes acquired through data extraction include all secondary outcomes except for provider and patient-centered outcomes, which are collected via surveys. Up to 25 participants in each of the 3 arms were surveyed to assess patient knowledge about CKD and NSAID avoidance. Patient participants in the eCDSS arm were surveyed 2 weeks after a visit in which the SmartSet was utilized; those in the eCDSS+ arm were surveyed 2 weeks after their pharmacist follow-up call, and those in the usual care arm were surveyed 2 weeks after a visit with their PCP during the study period. All participating intervention PCPs will be surveyed about their perception of the feasibility and utility of the eCDSS and pharmacist calls (for CDSS+ providers).
A summary of pre-planned study outcomes is shown in
Systolic blood pressure change
Diastolic blood pressure change
Controlled blood pressure (defined as <140/90)
Process
PCP awareness of CKD diagnosis (CKD inclusion on problem list or visit diagnosis)
ACEi/ARB utilization for albuminuria
Statin therapy for eligible high-risk CKD patients
Initiation of statin therapy
Total use of statin therapy
Patient-centered
Knowledge about CKD
Awareness of NSAID avoidance in CKD
Reach
Recruitment rate of providers and patients
Proportion that complete triple marker screen
Proportion with PCP visit after complete triple marker screening
Proportion of participants where BPA is available for PCP
Proportion of participants identified as high- vs low-risk CKD
Adoption
Proportion where PCP signed Smart Set when BPA was available
Proportion of CDSS+ participants that received pharmacist phone call after PCP signed Smart Set
Implementation
Proportion of orders signed or patient education provided by PCP after opening Smart Set
Proportion referred to nephrology and followed up with nephrology after signing Smart Set
Proportion of low-risk CKD patients that receive repeat triple screen
Maintenance
PCP satisfaction with eCDSS
PCP intent to continue using eCDSS
Our main secondary outcome is PCP awareness of the patient participant’s CKD diagnosis, as measured by inclusion of CKD-related International Statistical Classification of Diseases and Related Health Problems-10th Revision codes on the problem list or as a visit diagnosis. Additional secondary outcomes include process of care, patient-centered, and implementation outcomes.
There are 2 main process of care outcomes: (1) the proportion of patient participants with albuminuria prescribed ACEi/ARB and (2) appropriate use of statins. Appropriate statin use is defined both as the proportion of all patient participants aged ≥50 years prescribed statin therapy and the proportion of participants initiated on statin therapy after study start date.
Patient-centered outcomes include patient knowledge about CKD and NSAID avoidance as measured by patient surveys.
In addition to the effectiveness outcomes described above, the implementation outcomes are described in detail in
Baseline demographic and clinical characteristics will be summarized and study balance by study arm will be assessed using descriptive statistics as well as methods that account for the lack of independence among patients in the same cluster [
Based on preliminary data from our institution’s EHR, it was determined that there would be a maximum of 1400 participants in the practice who could meet inclusion criteria. The study was powered for the clinical outcome of BP change, with calculations performed using the clustersampsi command for Stata version 11.2. We assumed a 2-tailed alpha level of 0.05, an intraclass correlation coefficient of 0.025 based on studies in primary care settings [
Implementation outcomes at the end of the 12-month intervention period on October 4, 2018, are summarized below:
At the provider level, all 81 eligible PCPs (47 faculty attending physicians, 31 resident physicians, and 3 NPs) agreed to participate (100%). In total, 79 of the 81 providers (98%) were included because 2 providers had no remaining eligible participants after exclusion and opt-outs. At the patient level, 995 patients who met the initial eligibility criteria were identified. A total of 316 / 995 (31.8%) potential patient participants were excluded using automated algorithms, and clinicians excluded another 90 patients (
Baseline patient participant characteristics are shown in
Baseline characteristics of all included patients (N=524).
Patient characteristic | Values | |
Age (years), mean (SD) | 70.3 (8.9) | |
Female, n (%) | 236 (45) | |
White | 277 (53) | |
Asian | 118 (23) | |
Black/African American | 70 (13) | |
Hispanic | 38 (7) | |
Other | 20 (4) | |
English | 469 (90) | |
Chinese | 39 (7) | |
Spanish | 16 (3) | |
Private insurance | 146 (28) | |
Medicare | 215 (41) | |
Medicaid | 163 (31) | |
Cerebrovascular disease | 36 (7) | |
Congestive heart failure | 40 (8) | |
Coronary artery disease | 95 (18) | |
Diabetes mellitus | 199 (38) | |
Hyperlipidemia | 302 (58) | |
Hypertension | 377 (72) | |
Angiotensin converting enzyme inhibitor/angiotensin II receptor blocker | 319 (61) | |
Diuretic | 199 (38) | |
Statin | 353 (67) | |
Chronic kidney disease (CKD) on problem list | 61 (12) | |
CKD on problem list or primary care visit diagnosis by ICD-10c codes | 247 (47) | |
Systolic BPa (mm Hg), median (Q1-Q3) | 128 (117-140) | |
Diastolic BP (mm Hg), median (Q1-Q3) | 68 (62.5-75) | |
BP controlledb (<140/90), n (%) | 373 (71) | |
Estimated glomerular filtration rate based by serum creatinine (mL/min), mean (SD) | 56.0 (11.8) |
aBP: blood pressure.
bBP control reported based on only the most recent measurement before the start of the study.
cICD-10: International Classification of Diseases, Tenth Revision, Clinical Modification.
Of the 336 patient participants in the intervention arms, 178 (53.0%) completed the triple-marker screening.
Of the 178 patient participants who completed a triple-marker screen, 138 (77.5%) had a PCP visit during the 12-month intervention period. This was 41.1% (138/336) of all intervention participants.
Of the 138 participants with a PCP visit, the BPA was opened for 102 participants (73.9%). This was 30.3% (102/336) of all intervention participants.
Of the 102 participants in which the BPA was opened, orders were signed or patient education was given from the SmartSet for 83 participants (81.4%).
In this report, we describe the rationale, design, and initial implementation outcomes of a 3-arm pragmatic trial that assessed the feasibility and preliminary effectiveness of an eCDSS to improve CKD management in primary care compared with usual care. This study builds on previous pragmatic trials focused on EHR-based interventions to improve CKD management [
This study has many innovative components. We utilized a 3-armed trial design, which included a usual care group as well as a group with pharmacist-patient engagement after the provider-facing electronic intervention. In addition, our clinical eCDSS was mostly automated and individualized to patients based on laboratory results, rather than a generic CDSS for all participants. We also implemented the recommended triple-marker risk stratification approach to better characterize participants’ risk for CKD progression. Our participants included non-English speakers, who have been excluded from many other trials. This trial also included multiple types of outcomes allowing for a deeper understanding of the intervention’s impact on care processes. Reviews of eCDSS have found that few studies assess unintended consequences or adverse effects of a CDSS, which can be widespread as CDSS interventions often automate multiple actions [
One of the greatest challenges of this trial was its dependence on data analysts experienced at working with the EHR to program and implement the intervention as well as extract outcomes data. The study team devoted substantial time to troubleshoot unanticipated problems with the eCDSS programming and capture of outcomes. Every update or change to the EHR processes had the potential to disrupt the trial implementation. The team regularly validated the eCDSS and data capture through manual chart reviews. The inaccuracies of EHR data extraction are well-documented [
In this trial, we found much higher reach and adoption of eCDSS by PCPs than previous studies where adoption rates were frequently <50% [
We experienced many of the same challenges as other pragmatic trials during implementation of this EHR-dependent intervention. There was a period of time between provider recruitment, randomization, participants being contacted to opt out, and the study start date; as a result, 18 patients passed away before randomization and 2 additional patients passed away before the start of the study. Importantly, nearly half of the participants in the intervention arms never completed the triple-marker screen and therefore never had the opportunity to receive the eCDSS interventions. Another one-fifth of participants who completed the triple-marker screen did not have a PCP visit after the triple-marker screening and also did not receive the eCDSS. These barriers, common in many eCDSS trials, prevented the intervention group from receiving the desired intervention.
As a pilot study, this study was limited by its inclusion of patients and providers from a single institution. Therefore, both the intervention and strategies to encourage intervention uptake were adapted to meet the needs of this single institution that uses Epic Systems’ EHR. However, we have found that the patient population in this clinic is similar to others [
As the prevalence of CKD grows, primary care teams will increasingly be responsible for management of CKD. Clinical decision support tools may be a low-cost effective solution to enhance guideline-concordant care for this underdiagnosed disease. We describe the design and implementation of a pragmatic clustered randomized controlled trial to evaluate the feasibility and effectiveness an EHR-embedded electronic clinical decision support tool to improve management of CKD in primary care. Results from this study can guide design of future pragmatic eCDSS trials to improve CKD care.
angiotensin converting enzyme inhibitor
urine albumin-creatinine ratio
angiotensin II receptor blocker
blood pressure
best practice advisory
chronic kidney disease
electronic clinical decision support system trial arm
electronic clinical decision support system and pharmacist follow-up trial arm
estimated glomerular filtration rate
estimated glomerular filtration rate by serum creatinine
estimated glomerular filtration rate by serum cystatin-c
electronic health record
generalized estimating equation
International Classification of Diseases, Tenth Revision, Clinical Modification
Kidney Disease: Improving Global Outcomes
nurse practitioner
nonsteroidal anti-inflammatory drug
primary care medical home
primary care provider
LK and CP received funding from the National Institute of Diabetes and Digestive and Kidney Diseases (1R18DK110959). EK received funding through the National Research Service Award (T32HP19025). The funders played no role in review or approval of this manuscript for publication. The authors wish to thank the PCPs and patients who participated in this study.
EK analyzed and interpreted data and drafted the manuscript. LK, LL, and MS conceived and designed the study; interpreted data; and substantively revised the manuscript. AR, SP, JS acquired and analyzed data. RS designed the study; analyzed data; and substantively revised the manuscript. CP conceived and designed the study; interpreted data; and substantively revised the manuscript. All authors read and approved the final manuscript.
CP is currently the chief medical officer of Cricket Health. The other authors declare they have no conflicts of interest.
CONSORT‐EHEALTH checklist (V 1.6.1).