Chronic kidney disease (CKD) is common in adults; in the United States, 14% of all adults and nearly half of individuals aged ≥70 years have CKD [1]. CKD is an important predictor of morbidity and cardiovascular mortality [2]. Although most patients with early CKD are seen by primary care providers (PCPs), studies have consistently shown that patients and PCPs remain largely unaware of the patient’s CKD diagnosis until the disease is more advanced [3,4]. Even when early CKD is recognized, PCPs frequently are unaware of the best practices for risk stratification and prevention of CKD-related complications [3]. In particular, both CKD staging and complication risk stratification are greatly improved using a triple marker strategy: urine albumin-creatinine ratio (ACR), estimated glomerular filtration rate (eGFR) based on both serum creatinine levels (eGFR_Cr) and cystatin-c levels (eGFR_Cys) [5-9]. Early detection of CKD and risk stratification enable clinicians and patients to take individualized actions that improve outcomes and have the potential to attenuate progression, such as blood pressure management and renin-angiotensin blockade [10-13], statin therapy [13-16], avoidance of nephrotoxic medications [17,18], and glucose management in individuals with diabetes [19-21].

With the adoption of electronic health records (EHRs), there have been opportunities to implement low-cost interventions to improve patient care. Given that CKD diagnosis is based on laboratory results easily extracted from EHRs, CKD identification and management is an ideal area for targeted electronic efforts [22,23]. Previous studies have shown the feasibility of electronic clinical decision support systems (eCDSS) to improve guideline-concordant care for patients with CKD [24-29]. However, most studies have not been randomized nor have they provided individualized recommendations for patients based on laboratory results. Instead, they have utilized standard reminders or checklists. In addition, concerns about the eCDSS being burdensome or disruptive to workflow have hindered the development and testing of more complex, individualized eCDSS [23,30,31].

Team-based approaches for providing chronic disease management in primary care practice have also been shown to improve patient outcomes [32,33]. For example, blood pressure (BP) control is improved when managed by nonphysician members of the team, including nurses and pharmacists [34-37]. Initial studies have suggested that CKD, like other chronic diseases, may also benefit from a team-based approach [38,39].

Given the promise of both eCDSS and team-based care to improve guideline-concordant CKD care in primary care practice, we designed a pragmatic cluster randomized 3-arm trial (usual care; eCDSS; and eCDSS plus pharmacist follow-up). This trial aimed to assess the feasibility and impact of a CKD eCDSS with individualized recommendations and any additional benefit of pharmacist outreach to follow-up compared with usual care. In this paper, we have reported the study design and initial implementation outcomes of this trial.

This was a 3-arm cluster randomized controlled trial with randomization at the provider level. There was 1 usual care control arm and 2 intervention arms. For patients of PCPs in the intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine ACR) at the beginning of the study period, to be completed when the patient visited the lab for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education after a PCP visit in which the eCDSS was utilized. We planned for an 18-month trial beginning October 4, 2017, that included a 12-month intervention period and subsequent 6-month follow-up period. This trial was registered on ClinicalTrials.gov (NCT02925962). The University of California San Francisco Human Research Protection Program approved the protocol for this study.

This study was conducted at a general internal medicine practice (with 2 locations) at the University of California San Francisco that cares for a diverse population of more than 24,000 adult patients. The PCPs in this practice include faculty attending physicians, resident physicians, and nurse practitioners (NPs). This practice is a certified primary care medical home (PCMH) that follows a team-based approach to care; each of the 10 teams includes 10 to 14 part-time PCPs, 2 medical assistants, a licensed vocational nurse, and administrative staff. Additional personnel of the PCMH include 5 registered nurses, 2 pharmacy technicians, and a clinical pharmacist available 1 day a week. The EHR used within this practice is Epic (EpicCare, Epic Systems).

To optimize the chance of a successful intervention, we worked with a multidisciplinary team (general internist, informatics specialist, and nephrologists) to develop the eCDSS logic. The team focused on areas with the strongest evidence base for CKD management in primary care and developed the logic to apply guideline recommendations in real-time for individual patients. In the first step, patients were classified as either at high risk (eGFR_Cys <60 mL/min and/or ACR >30 mcg/mg) or low risk (eGFR_Cys >60 mL/min and ACR <30 mcg/mg) for experiencing CKD-related complications [5,17]. After risk stratification, the eCDSS logic focused on 5 domains of CKD care for high-risk patients: (1) cardiovascular risk reduction; (2) potassium management; (3) proteinuria management; (4) appropriateness for nephrology referral; and (5) patient education. For low-risk patients, the logic defaulted to a recommendation for repeat triple-marker testing in 6 months.

We elicited feedback from PCPs in focus group settings during development of the eCDSS and during implementation planning, which is a recommended practice to increase adoption and uptake of eCDSS [23]. Focus group discussions centered on eCDSS messaging and orders, as well as ways to facilitate integration into the usual clinical workflow.

All providers practicing at the general internal medicine practice with a primary care panel were eligible for inclusion and randomization.

Eligible patients were identified using a previously described and validated algorithm [40]. (Patients were considered eligible for inclusion if they were aged 18 to 80 years; had a preferred language of English, Spanish, or Chinese; had at least 2 outpatient eGFR_Cr=30 to 59 milliliters/minute (mL/min) at least 90 days apart, one of which was within the 12 months before August 14, 2017 (when providers were first contacted for study recruitment); and had a primary care visit in the past 18 months. (see Figure 1 for the Consolidated Standards of Reporting Trial diagram) Participants were automatically excluded (based on EHR data) if they were deceased; diagnosed with end-stage renal disease; engaged with a nephrology clinic with 2 or more visits in the past 12 months; kidney transplant recipients; or diagnosed with dementia. PCPs were then sent a list of potential participants for inclusion and asked to exclude participants for additional criteria that were unreliably captured when extracted from the EHR: current pregnancy, life expectancy <6 months, limited communication ability owing to impaired cognition or severe mental illness, New York Heart Association Class III/IV heart failure, or known ejection fraction <25%. PCPs could also exclude any other participants they felt would be inappropriate for study staff to contact.

We recruited PCPs with eligible patients by eGFR_Cr by email between August 14, 2017 and September 8, 2017. PCPs were given 2 weeks to either opt-out entirely or exclude individual patients. We then randomized participating PCPs as described below. Participation letters were mailed to eligible intervention arm patients between September 1, 2017 and September 27, 2017, and patients were given 2 weeks to either return an opt-out card or call the study coordinator to opt out. Patients opting out after the study start date of October 4, 2017, were considered withdrawn from the study.

The usual care group PCPs received no additional contact beyond recruitment from the study team, and the usual care patients were never contacted directly by the study team. Usual care PCPs did have access to the same labs and medications to use at their clinical discretion without any recommendations or direction from the study team.

We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP’s assignment. The study statistician was blinded to the identification of the PCPs and patients and will be similarly blinded for the analyses.

Before the eCDSS rollout but after randomization, all participating PCPs randomized to an intervention arm were asked to watch an educational video that provided background on guideline-based CKD care, as well as information on what to expect when an enrolled patient participant came in for a visit. PCPs were given an incentive of US $10 and a chocolate bar upon watching the video. Overall, 69.8% (72.0% eCDSS and 67.9% eCDSS+) viewed the video.

Before the eCDSS rollout, between October 5, 2017 and October 12, 2017, one of the study investigators (LK) ordered triple-marker testing for each patient participant from the eCDSS arm (n=169) and each patient participant from the eCDSS+ arm (n=177). We excluded 48 patients who opted out. In total, 10 patients withdrew later.

After the triple-marker testing was ordered, the next time the patient participant visited the lab for their regular clinical care, the triple-marker labs were also collected. We programmed the eCDSS to trigger the first time that the patient participant visited their PCP after all 3 lab results were available. (see Figure 2 for the study workflow) The eCDSS best practice advisory (BPA) appeared above the current medications in the electronic chart (Figures 3 and 4). PCPs could choose to open the BPA by clicking an “accept” button. We also allowed PCPs to view the recommendations by hovering over the BPA before accepting it. For patient participants categorized as low risk, when the PCP accepted the BPA, triple-marker labs with an expected completion date 6 months hence prepopulated were automatically pending for the PCP to sign. For patient participants categorized as high risk, when the PCP accepted the BPA, a SmartSet (EpicCare, Epic Systems) of tailored orders and recommendations appeared (Figure 5). If the PCP did not open the BPA and sign the orders in the SmartSet during the first PCP visit after the triple-marker labs were available, the BPA would trigger at up to 2 additional PCP visits during the study period. The eCDSS did not trigger if the patient participant saw a provider who was not their PCP.

In addition to a reminder of BP targets in CKD, the SmartSet for patient participants classified as high risk for progression and/or complications of CKD used real-time EHR lab and medication data to recommend any of the following that applied:

The second intervention arm (eCDSS+) included a pharmacist follow-up visit by telephone. If the SmartSet was signed during the PCP visit, a pharmacist would call the participant within 2 weeks of the visit to reinforce CKD-related teaching and medication changes from the visit as well as complete a comprehensive medication review. Information on the telephone encounter was documented in the EHR and sent to the PCP; any urgent clinical issues identified during the call were highlighted for the PCP as needing their follow-up. Providers in the eCDSS+ group also were encouraged to make a warm handoff to the pharmacist by including anticipatory guidance about the pharmacist phone call within the after-visit summary and distributing a business card with the pharmacist’s photo and phone number for their patient participants at the time of using the SmartSet.

Given that the study physician ordered labs at the start of the intervention period, safety checks were put into place to ensure adequate review of the laboratory results. A nephrologist (LL) from the study team reviewed weekly laboratory results specifically to identify the following: eGFR_Cr decline >30% from baseline, ACR ≥1000 mcg/mg, adherence to nephrology referrals, and any discordance >30% between eGFR_Cr and eGFR_Cys. In all of these situations, the nephrologist contacted the PCP to ensure appropriate follow-up. Specifically, in cases of discordance between the 2 types of eGFR levels, the nephrologist advised PCPs to dose medications based on eGFR­_Cys when the clinical scenario suggested that eGFR_cr may not be accurate.

Implementation studies have shown that interventions designed with a theoretical basis are more likely to be successful [42,43]. Therefore, we utilized several theory-based strategies to encourage provider uptake of the eCDSS and participant uptake of provider and/or pharmacist recommendations. We classify these strategies using the capability, opportunity, motivation, behavior framework (or COM-B), which asserts that capability, opportunity, and motivation are essential conditions that impact behavior [44,45].

To encourage provider uptake and use of CKD guidelines, we addressed capability barriers (such as knowledge about the guidelines or the eCDSS or forgetting to discuss CKD owing to limited bandwidth). During the recruitment phase, we educated eligible PCPs about the prevalence of CKD within their panel by sending a list of their potentially eligible participants along with information about the importance of recognizing and managing CKD early. Shortly before intervention implementation, we sent a training video about the CKD guidelines and the eCDSS to the participating intervention PCPs. The eCDSS itself also served as a prompt to discuss CKD during the visits.

To address PCP opportunity barriers, we designed the eCDSS to fit into the physician workflow and to contain all the necessary CKD-related orders and patient education in one easily accessible place during a clinical visit.

To address PCP motivation barriers, we provided a small incentive to the participating intervention PCPs who watched the training video. Finally, to enhance use of the BPA, study personnel sent an individual email reminder to the intervention PCP the day before each eligible patient participant visit.

Although the eCDSS itself was a provider-facing intervention, we used strategies to encourage patient participant behavior change. To address capability barriers (such as knowledge of CKD or forgetting the provider recommendations), the CDSS SmartSet had CKD-based patient educational handouts in the patient’s preferred language. The pharmacist phone call was also designed to reinforce the knowledge provided at the office visit and motivate the patient participant to adhere to medication and avoid NSAIDS.

For patient participants, the EHR was used to collect baseline data about demographic characteristics (age, sex, race-ethnicity, preferred language, and insurance status); medical co-morbidities (cerebrovascular disease, congestive heart failure, coronary artery disease, diabetes mellitus, hyperlipidemia, and hypertension); current medication prescriptions (statin, ACEi/ARB, and diuretics); previous documentation of CKD on problem list and/or as a visit diagnosis, most recent blood pressure, eGFR­­_Cr, eGFR­­_Cys (if available), and urine ACR (if available) before enrollment.

Primary outcome data on BP will be extracted from the EHR. Baseline BP was defined as the most recent ambulatory BP measurement before final enrollment on October 4, 2017. All patient participant ambulatory BP measurements were collected during the intervention period and will continue to be collected for the 6 months after the end of the intervention.

Secondary outcomes will be collected primarily using automated EHR data extraction algorithms. The outcomes acquired through data extraction include all secondary outcomes except for provider and patient-centered outcomes, which are collected via surveys. Up to 25 participants in each of the 3 arms were surveyed to assess patient knowledge about CKD and NSAID avoidance. Patient participants in the eCDSS arm were surveyed 2 weeks after a visit in which the SmartSet was utilized; those in the eCDSS+ arm were surveyed 2 weeks after their pharmacist follow-up call, and those in the usual care arm were surveyed 2 weeks after a visit with their PCP during the study period. All participating intervention PCPs will be surveyed about their perception of the feasibility and utility of the eCDSS and pharmacist calls (for CDSS+ providers).

Baseline demographic and clinical characteristics will be summarized and study balance by study arm will be assessed using descriptive statistics as well as methods that account for the lack of independence among patients in the same cluster [48]. Balance will be assessed with logit link generalized estimating equation (GEE) for categorical variables and linear GEE models for continuous variables. Clustering will be accounted for using an exchangeable correlation matrix with robust standard errors. We anticipate controlling for characteristics that are associated with outcomes but differ at baseline between arms and using multivariable regression analyses with GEE to account for cluster randomization in the reporting of the final outcomes of our study. Sensitivity analyses will include as treated analyses with inverse probability weighting to determine impact of participants changing providers/study arms as well as to determine impact of any differences in the number of assessments or clinic visits by study arm. Primary analyses will follow intention-to-treat principles.

Based on preliminary data from our institution’s EHR, it was determined that there would be a maximum of 1400 participants in the practice who could meet inclusion criteria. The study was powered for the clinical outcome of BP change, with calculations performed using the clustersampsi command for Stata version 11.2. We assumed a 2-tailed alpha level of 0.05, an intraclass correlation coefficient of 0.025 based on studies in primary care settings [49,50], and a standard deviation of 5 mmHg. Assuming 23 clinicians per arm with at least 15 patients per clinician (ie, 345 patients per arm), we estimated that we would have 80% power to detect a difference as small as 1.27 mmHg between arms.

Implementation outcomes at the end of the 12-month intervention period on October 4, 2018, are summarized below:

In this report, we describe the rationale, design, and initial implementation outcomes of a 3-arm pragmatic trial that assessed the feasibility and preliminary effectiveness of an eCDSS to improve CKD management in primary care compared with usual care. This study builds on previous pragmatic trials focused on EHR-based interventions to improve CKD management [28,29,51,52] and supports the feasibility of using EHRs to identify study participants, intervene in early CKD management, and measure study outcomes. Given its design as a pragmatic trial (per Pragmatic Explanatory Continuum Indicator Summary, or PRECIS, criteria [53]), we found high rates of participation by providers and low opt-out rates by patients. We also found high rates of eCDSS use by providers.

This study has many innovative components. We utilized a 3-armed trial design, which included a usual care group as well as a group with pharmacist-patient engagement after the provider-facing electronic intervention. In addition, our clinical eCDSS was mostly automated and individualized to patients based on laboratory results, rather than a generic CDSS for all participants. We also implemented the recommended triple-marker risk stratification approach to better characterize participants’ risk for CKD progression. Our participants included non-English speakers, who have been excluded from many other trials. This trial also included multiple types of outcomes allowing for a deeper understanding of the intervention’s impact on care processes. Reviews of eCDSS have found that few studies assess unintended consequences or adverse effects of a CDSS, which can be widespread as CDSS interventions often automate multiple actions [54,55]. This trial integrated safety measures and their tracking by a nephrologist allowing not only for measurement of safety but also the ability to ensure that any new concerning clinical findings found owing to the trial would be addressed. As previously recommended, it is important to first pilot interventions before widespread implementation to ensure there are no adverse effects in a small population where safety can be more easily assessed [56].

One of the greatest challenges of this trial was its dependence on data analysts experienced at working with the EHR to program and implement the intervention as well as extract outcomes data. The study team devoted substantial time to troubleshoot unanticipated problems with the eCDSS programming and capture of outcomes. Every update or change to the EHR processes had the potential to disrupt the trial implementation. The team regularly validated the eCDSS and data capture through manual chart reviews. The inaccuracies of EHR data extraction are well-documented [57]. Despite the presence of an explicit trial infrastructure in our EHR, implementation of the intervention and data collection was far from seamless. Studies that rely on the EHR to deliver interventions and collect outcomes must ensure adequate time and resources for a multidisciplinary team to validate the intervention and review data inaccuracies in an ongoing manner for all EHR-based trials.

In this trial, we found much higher reach and adoption of eCDSS by PCPs than previous studies where adoption rates were frequently <50% [55,58]. We believe there are multiple explanations. Few electronic alerts are currently used in this practice, so providers have not yet experienced fatigue associated with the growing presence of alerts [59]. Moreover, the eCDSS was designed by a multidisciplinary team, facilitating buy-in from all stakeholders and ensuring all perspectives were considered in determining how the eCDSS would impact workflow. Finally, study staff (including a lead investigator who works within the primary care practice) served as on-the-ground champions and made significant efforts to educate, remind, and incentivize PCPs.

We experienced many of the same challenges as other pragmatic trials during implementation of this EHR-dependent intervention. There was a period of time between provider recruitment, randomization, participants being contacted to opt out, and the study start date; as a result, 18 patients passed away before randomization and 2 additional patients passed away before the start of the study. Importantly, nearly half of the participants in the intervention arms never completed the triple-marker screen and therefore never had the opportunity to receive the eCDSS interventions. Another one-fifth of participants who completed the triple-marker screen did not have a PCP visit after the triple-marker screening and also did not receive the eCDSS. These barriers, common in many eCDSS trials, prevented the intervention group from receiving the desired intervention.

As a pilot study, this study was limited by its inclusion of patients and providers from a single institution. Therefore, both the intervention and strategies to encourage intervention uptake were adapted to meet the needs of this single institution that uses Epic Systems’ EHR. However, we have found that the patient population in this clinic is similar to others [60], and Epic is a widely used EHR in multiple health systems. As a pragmatic trial, our sample size was smaller than initially anticipated and will thus ultimately impact our power to detect changes in our primary outcome. The short follow-up period for this pilot study will also limit our ability to assess some clinical outcomes. However, we were still able to use this pilot study to determine the feasibility of this intervention.

As the prevalence of CKD grows, primary care teams will increasingly be responsible for management of CKD. Clinical decision support tools may be a low-cost effective solution to enhance guideline-concordant care for this underdiagnosed disease. We describe the design and implementation of a pragmatic clustered randomized controlled trial to evaluate the feasibility and effectiveness an EHR-embedded electronic clinical decision support tool to improve management of CKD in primary care. Results from this study can guide design of future pragmatic eCDSS trials to improve CKD care.