Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 13.01.20 in Vol 9, No 1 (2020): January

Preprints (earlier versions) of this paper are available at, first published Feb 26, 2019.

This paper is in the following e-collection/theme issue:


    Evaluation of a Health Information Technology–Enabled Panel Management Platform to Improve Anticoagulation Control in a Low-Income Patient Population: Protocol for a Quasi-Experimental Design

    1Center for Vulnerable Populations, University of California, San Francisco, CA, United States

    2Smith Center for Outcomes Research in Cardiology, Beth Israel Deaconess Medical Center, Harvard University, Boston, MA, United States

    3Outpatient Pharmacy, Zuckerberg San Francisco General Hospital, San Francisco, CA, United States

    Corresponding Author:

    Valy Fontil, MD, MS

    Center for Vulnerable Populations

    University of California

    2789 25th Street

    Suite 350

    San Francisco, CA, 94110

    United States

    Phone: 1 (415)206 4087



    Background: Warfarin is one of the most commonly prescribed medications in the United States, and it causes a significant proportion of adverse drug events. Patients taking warfarin fall outside of the recommended therapeutic range 30% of the time, largely because of inadequate laboratory monitoring and dose adjustment. This leads to an increased risk of blood clots or bleeding events. We propose a comparative effectiveness study to examine whether a technology-enabled anticoagulation management program can improve long-term clinical outcomes compared with usual care.

    Objective: Our proposed intervention is the implementation of an electronic dashboard (integrated into a preexisting electronic health record) and standardized workflow to track patients’ laboratory results, identify patients requiring follow-up, and facilitate the use of a validated nomogram for dose adjustment. The primary outcome of this study is the time in therapeutic range (TTR) at 6 months post intervention (a validated metric of anticoagulation quality among patients receiving warfarin).

    Methods: We will employ a pre-post quasi-experimental design with a nonequivalent usual-care comparison site and a difference-in-differences approach to compare the effectiveness of a technology-enabled anticoagulation management program compared with usual care at a large university-affiliated safety-net clinic.

    Results: We used a commercially available health information technology (HIT) platform to host a registry of patients on warfarin therapy and create the electronic dashboard for panel management. We developed the intervention with, and for, frontline clinician users, using principles of human-centered design. This study is funded until September 2020 and is approved by the University of California, San Francisco Institutional Review Board until June 22, 2020. We completed data collection in September 2019 and expect to complete our proposed analyses by February 2020.

    Conclusions: We anticipate that the intervention will increase TTR among patients taking warfarin and that the use of this HIT platform will facilitate tracking and monitoring of patients on warfarin, which could enable outreach to those overdue for visits or laboratory monitoring. We will use these findings to iteratively improve the platform in preparation for a larger, multiple-site, pragmatic clinical trial. If successful, our study will demonstrate the integration of HIT platforms into existing electronic health records to improve patient care in real-world clinical settings.

    International Registered Report Identifier (IRRID): DERR1-10.2196/13835

    JMIR Res Protoc 2020;9(1):e13835



    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100



    Warfarin is one of the most commonly prescribed medications in the United States, with more than 20 million Americans taking warfarin anticoagulation therapy to prevent the formation of blood clots. As one of the most common culprits of adverse drug events (ADEs) in outpatient settings [1], warfarin dosing must be individualized to be effective; overdosing of warfarin can cause serious bleeding complications, whereas underdosing does not provide adequate protection against thromboembolism. As such, the National Action Plan for ADE Prevention targets blood thinners as a high priority for intervention [2].

    Despite more than 50 years of clinical experience, studies suggest that patients taking warfarin fall outside of the recommended therapeutic range 30% of the time [3]. Therapeutic range is based on a laboratory test called the international normalized ratio (INR), and a patient’s time in therapeutic range (TTR) is associated with lower risk of developing blood clots from underdosing of warfarin or bleeding events from overdosing [4]. Therefore, warfarin treatment requires periodic monitoring with blood tests to inform dose adjustment [5].

    Unfortunately, solutions for efficient monitoring of warfarin treatment are still lacking, especially in settings such as safety-net clinics with relatively limited resources and health technology infrastructure. In safety-net clinics caring for low-income, uninsured, or underinsured patients, maintaining warfarin in the therapeutic range poses an additional challenge because of limited health literacy and educational attainment for a large proportion of the patient population [6,7], prevalent impairment in cognitive function [8], and various socioeconomic challenges [9]. Hence, proactive and efficient strategies that support outpatient warfarin monitoring are urgently needed.

    As anticoagulation therapy relies on well-established, standardized care protocols with clear safety and efficacy targets (eg, INR monitoring), it is well suited to health information technology (HIT) approaches that can identify patients who need a specific treatment, monitoring, or intervention and help health care providers execute the appropriate management plan. These software platforms have been widely used for outreach and management of chronic illnesses, including diabetes, asthma, cancer, depression, and congestive heart failure [10-14], where their use improves health outcomes and is cost-effective [15-17]. However, many HIT interventions are not adequately integrated with current electronic health records (EHRs), forcing health systems to add layers of complexity to clinician workflow that are seldom feasible in safety-net clinics with limited resources [18]. Moreover, current platforms are not customized for anticoagulation therapy to facilitate the attainment and tracking of safety and efficacy targets. This study aims to resolve this limitation by implementing a locally customized, user-designed HIT tool that interfaces with the EHR and focuses on improving the quality of anticoagulation therapy.


    In this study, we propose to design and test an electronic dashboard for panel management of patients taking warfarin anticoagulation therapy at a large university-affiliated safety-net clinic. We have previously developed an electronic registry of patients taking warfarin and a team-based workflow for scheduling or rescheduling clinic appointments for patients outside of the therapeutic range [19]. This prior pilot intervention led to an improvement in patient attendance to visits for anticoagulation management [19]. We will use a commercially available HIT platform to host our patient registry and create an electronic dashboard for panel management that will enable providers to track patients’ INR, implement a workflow for scheduling patients to visits, and use a validated electronic anticoagulation treatment nomogram for dose adjustment during the visit.


    Study Design

    This is a pre-post quasi-experimental design with a nonequivalent usual care comparison site. We will use a difference-in-differences approach to compare the effectiveness of a technology-enabled panel management anticoagulation program with usual care in improving TTR.

    Study Population and Setting

    Clinical Sites

    The site that will undergo the intervention is a large university-affiliated anticoagulation clinic within a network of 12 safety-net clinics in San Francisco that serve a racially and ethnically diverse population of low-income patients. A team of 3 part-time clinical pharmacists, 1 part-time nurse practitioner, a clinical coordinator, and a physician medical director provides care to approximately 250 patients who receive primary care in any of the 12 clinics in the city’s public health network. This network is operated by the San Francisco Department of Public Health (SFDPH), and its member clinics use the same electronic health records. Our comparison site is another anticoagulation clinic site within this network, which is chosen for its similarity with the intervention site with regard to its university affiliation and the demographic composition of its patient population. The comparison site delivers anticoagulation to approximately 50 patients receiving primary care at the family health center; 2 part-time clinical pharmacists and a physician medical director staff this clinic.

    As both clinics are within the SFDPH, they follow the same protocols for standard management (monitoring intervals of 4 weeks before establishing therapeutic control and 12 weeks once therapeutic control has been achieved for 3 consecutive visits). The standard management includes a protocol for patients scheduled to undergo high-risk surgical operations. Over the past 3 years, these clinics have sought to use an internal guideline for switching patients from warfarin to direct-acting oral anticoagulants (DOACs) based on the duration of treatment, most recent INR, warfarin medication adherence, drug-drug interactions, indication for anticoagulation, and kidney and liver function.

    Time Horizon

    The period of analysis will include a 12-month preintervention period and a 12-month postintervention period that will each include an enrollment period in the first 6 months of pre and postintervention.

    Enrollment Inclusion Criteria

    The study will include all adult patients aged 18 years or above on anticoagulation therapy (such as warfarin, apixaban, rivaroxaban, fondaparinux, dabigatran, and edoxaban) identified by ICD-10 (International Classification of Diseases, tenth revision) Z79.01, who present to anticoagulation clinic for initiation or management of anticoagulation therapy over 2 enrollment periods of 6 months at the 2 large academic safety-net clinics (intervention and comparison site) before and after the start of the intervention. We chose the enrollment period of 6 months to enable at least 6 months of follow-up for every patient monitored in the study in both pre and postintervention periods.


    The intervention uses a new electronic dashboard (Figure 1) for panel management of patients on warfarin anticoagulation therapy to improve therapeutic efficacy. The intervention will feature the following elements:

    • An electronic dashboard that displays medication information, TTR, and other laboratory results. The dashboard will present relevant data on patients prescribed anticoagulants to facilitate more effective and efficient panel management. The dashboard is customized to show information important for the safe management of these medications (such as current medications, laboratory results, attendance rate to scheduled visits, and language spoken by patients). The dashboard can be used to sort patients according to desired criteria such as diagnoses, specific TTR parameters, or patients with missed scheduled appointments or laboratory monitoring. Finally, the dashboard can be used to create tasks, assign workflow, and send reminders to clinic providers.
    • A panel management workflow that will include patient outreach activities, such as appointment reminders and rescheduling of missed appointments; communication from the pharmacist to the patient’s primary care provider or medical specialists, such as alerts to consider novel alternative anticoagulation therapy (eg, DOAC); and suspected or pending adverse effects based on early signs, such as bruising or drug interactions.
    • Integration of evidence-based treatment nomogram that will be accessible on the platform for use by the pharmacist to guide therapeutic treatment decisions. Although multiple guidelines addressing anticoagulant management are available to clinical providers, they are not integrated at the point of care. The dashboard will display relevant clinical guidelines while providers are managing their patients. As an example, the warfarin dosing nomogram will be shown on the screen at the same time the providers are managing a patient on warfarin.
    Figure 1. Screenshot of electronic dashboard and panel management interface.
    View this figure
    Electronic Dashboard Development

    We developed the dashboard in partnership with our frontline anticoagulation providers using the principles of human-centered design [20], which emphasizes the use of rapid prototyping, early and continuous stakeholder engagement, and iterative development and evaluation of interventions. We partnered with a commercially available health care technology platform (CipherHealth LLC) to host the electronic dashboard [21]. Our study team includes a frontline anticoagulation provider (SL) who conducted informal interviews with other frontline anticoagulation providers at the intervention clinic site to assess their needs and potential usability of the dashboard and an expert cardiologist (DK) who collaborated to create the initial dashboard prototype. We developed subsequent alpha versions of the prototype collaboratively with 2 additional coinvestigators, including an implementation scientist (VF) and a patient safety and digital health technology researcher (US), who are both practicing primary care physicians at the intervention site. We provided the final alpha version to the anticoagulation providers for use over a period of 4 weeks, and we conducted direct observations and informal interviews that led to additional modifications of the platform based on user feedback.

    Usual Care Comparison

    Under current practice (ie, usual care), patients who receive care at either site are referred to an anticoagulation clinic where a pharmacist or nurse practitioner assesses the patient and manages their anticoagulation therapy on an ongoing basis. At each visit, either in person or virtual (by phone), the provider reviews the indication, treatment duration, and relevant history and proactively screens whether a patient qualifies to be transitioned to a DOAC medication. The frequency of follow-up visits is determined by INR results, INR trends, and other clinically relevant factors, including age and bleeding or clotting risks. Current practice does not include panel management or the use of a HIT tool that facilitates panel management and integration of an evidence-based treatment nomogram at the point of care.

    Outcomes and Measurements

    Primary and Secondary Outcomes

    The primary outcome will be TTR, a well-established intermediate clinical outcome that is the most widely used metric of anticoagulation quality among patients receiving warfarin [22-25]. It is defined as the proportion of time during which the INR is within the range deemed therapeutic for the indication for which the patient is receiving warfarin therapy. For instance, an INR between 2 and 3 inclusive is therapeutic for patients receiving warfarin for secondary prevention of venous thromboembolism, but an INR between 2.5 and 3.5 inclusive is considered therapeutic for patients with mechanical mitral valves. We will calculate TTR using the Rosendaal method [26], which uses linear interpolation to assign INR values for each day between consecutive INR values. We will calculate the TTR for patients with at least 5 INR laboratory values over the course of at least 6 months. Patients who stop coming to the clinic before or are transitioned to DOACs before this threshold will be censored. The secondary outcome will be proportion in therapeutic range defined as the proportion of laboratory INR values that are within the therapeutic range (ie, number of INR values in range divided by the total number of laboratory INR tests performed, regardless of the interval between these values).

    Exploratory Endpoints

    We will examine a number of process measures that are on the pathway to achieving therapeutic goals for anticoagulation, including time from out-of-range INR value to patient contact and time from warfarin initiation to first therapeutic INR (Table 1). We will measure other process-of-care metrics, including (1) patient’s attendance rate to scheduled visits for anticoagulation management, (2) proportion of patients not meeting monitoring guidelines (defined as those who do not have INR testing at least every 56 days based on National Quality Forum recommendations [16]), (3) appropriate duration of therapy (defined as the days of observed warfarin therapy divided by the recommended days of therapy), and (4) provision of telephonic or other remote visits for anticoagulation management. We will also report clinical endpoints such as bleeding complications (hospital visits for bleeding and minor vs major hemorrhagic strokes), incident deep venous thrombosis (DVT), incident pulmonary embolism (PE), and ischemic stroke, although we are likely not adequately powered to detect statistically significant differences between the control and intervention arms. In addition to these process measures for warfarin patients, we will also measure adherence to protocols for screening and transitioning patients to DOACs, which requires assessing patients for medication adherence, drug-drug interactions, and liver and renal function.

    Table 1. Variables, outcomes, and process measures with definitions.
    View this table
    Independent Variables

    We will measure baseline patient demographics such as age, gender, race and ethnicity, and percent poverty in zip code. Other independent variables will include baseline history of chronic kidney disease, ischemic heart disease, stroke, atrial fibrillation, mitral valve replacement (MVR), atrial valve replacement (AVR), DVT, pulmonary arterial hypertension (PAH), and PE.

    Analysis Plan


    We will use the difference-in-differences approach to compare the effectiveness of the technology-enabled anticoagulation management program with usual care in improving TTR. Implementation using a linear mixed model (LMM) will allow us to adjust for patient-level confounders; in addition, many patients will provide TTR outcomes both before and after implementation at the intervention clinic and in the matching periods at the control clinic, providing further natural control of confounding. We will also conduct a series of exploratory analyses of secondary outcomes, subgroup analyses to examine heterogeneity of effect, and exploratory mediation analyses to understand the processes by which the intervention might provide greater effectiveness.

    Difference-in-Differences Analytic Approach

    The primary analysis will use an LMM for potentially repeated values of TTR, which are calculated for each patient using the Rosendaal method described above and transformed as needed to meet normality assumptions. The LMM will include a random effect for patient to account for the within-patient correlation of the repeated TTRs and will also include fixed effects for clinical site, intervention period (pre vs post), and their interaction. The LMM coefficient for the interaction between clinical site and intervention period will be interpretable as the difference of differences, the focus of interest. In addition, adjusted mean TTRs by clinic and period will be obtained as linear combinations of the coefficients in the model. To further control confounding, the LMM will include patient-level factors, including baseline gender, race, age, desired value of INR, tobacco use, concurrent use of medications that interfere with warfarin, and presence of a high-risk comorbid condition at baseline (hypertension; diabetes; ischemic heart disease; congestive heart failure; stroke; and pulmonary, hepatic, or renal disease). We have chosen these covariates because they are the known predictive factors for the efficacy of warfarin therapy used to calculate the SAMe-TT2R2 score (sex, age, medical history, treatment, tobacco use, and race), a widely used clinical prediction rule for achieving and maintaining a desired TTR of greater than 65%, based on common clinical factors that influence INR and anticoagulation control [27]. We define baseline as the time of warfarin initiation for each patient. The model will also adjust for reasons for anticoagulation therapy such as having a baseline diagnosis of atrial fibrillation, MVR, AVR, DVT, PAH, and PE.

    Analysis of Process Measures

    We will employ the difference-in-differences analytic approach described above, using LMMs or generalized LMMs as appropriate to each outcome, to compare pre-post differences between the intervention and usual care clinics in the process outcomes listed in Table 1.

    Subgroup Analysis to Test for Heterogeneity of Treatment Effects

    We will produce strata-specific analyses stratified by the demographic characteristics and test for interactions by intervention arm and demographic subgroups such as age groups, race/ethnicity, and gender.

    Multiple Hypothesis Testing

    We will report the results of the hypothesis test for the primary outcome without adjustment for multiple hypothesis testing. No formal penalization for multiple hypothesis testing is planned for the analysis of process measures or subgroup analyses as these will be treated as exploratory and hypothesis generating. We will report 95% confidence intervals for all our estimates.

    Multiple Imputation for Missing Data

    We do not anticipate any missing data for our primary outcome. In the event of missing data, we will use a multiple imputation strategy to address missing data. In addition to multiple imputation under the standard assumption that data are missing at random (conditional on observed covariates and outcomes), we would also implement sensitivity analyses using imputation under plausible missing not at random scenarios.

    Data Source, Collection, Management, and Safety

    Consistent with the principles of pragmatic clinical trials, we will use EHR data for patient identification and assessment of intervention implementation and outcomes. We have established agreements between CipherHealth and the SFDPH to ensure data security. Data will flow between SFDPH EHRs and CipherHealth through secure file transfer protocols. SFDPH and CipherHealth programmers will regularly review data security procedures. All individuals who have access to patient data are either (1) clinicians with prior access or (2) research staff approved by the University of California, San Francisco internal review board, who have completed training in research ethics and compliance. Identifiable data will only be used for standard patient care and management; all data extracted from the platform for analyses will be deidentified.

    Minimum Detectable Effects

    Preliminary analysis showed that 350 patients would provide 80% power in a test with 2-sided significance level of 5% to detect a difference of differences of 2 to 5 TTR percentage points, based on the proposed analysis using an adjusted LMM. The minimum detectable effect will depend on the intraclass correlation of the repeated TTR measures (assumed range: 0.80-0.95) and the percentage of patients providing measurements in both periods (assumed range: 50%-100%). These calculations assume that the standard deviation of the TTR outcome is 20 percentage points, based on published literature on TTR [28], and these calculations are penalized for covariate adjustment using an adaptation of the variance inflation factor [29].


    We used a commercially available HIT platform to host a registry of patients on warfarin therapy and create an electronic dashboard for panel management. We developed the HIT dashboard interface and co-designed the intervention with, and for, frontline clinician users. This study is funded until September 2020 and is approved by the University of California San Francisco Institutional Review Board until June 22, 2020. We completed data collection as of July 2019 and expect to complete our proposed analyses by February 2020.


    We propose a quasi-experimental study to evaluate the implementation of an integrated HIT intervention for management of anticoagulation and examine pre-post outcomes on clinical and implementation outcomes. The HIT platform will facilitate tracking and monitoring of patients on warfarin and enable outreach to those overdue for visits or laboratory monitoring.

    We developed the dashboard in partnership with our frontline anticoagulation providers using the principles of human-centered design [20], which emphasizes the use of rapid prototyping, early and continuous stakeholder engagement, and iterative development and evaluation of interventions. An important constraint was its integration into the existing electronic health record with minimal disruption of the existing workflow of anticoagulation service providers. We believe this emphasis on usability is critical to the successful adoption of a HIT intervention.

    We anticipate that the intervention will facilitate tracking and monitoring of patients on warfarin, allowing providers to reach out to those with abnormal results (who may need dose adjustments) as well as those overdue for visits or laboratory monitoring. Coupled with increased use of validated nomograms for dose adjustment (which has been previously shown to be a key predictor of improved TTR), our technology-enabled panel management program will increase the proportion of time in TTR among patients taking warfarin. Insights from this study will help us improve and adapt our HIT platform in preparation for a larger, pragmatic clinical trial powered for clinical outcomes. Such a trial would also examine the cost-effectiveness of the intervention.

    This project is innovative with significant implications for use of HIT to improve clinical care. First, use of an HIT-enabled intervention to improve anticoagulation management in resource-constrained settings, such as safety-net clinics, could improve outcomes and reduce adverse events at the national level, as these clinics often provide care for a disproportionate share of high-risk and high-cost patients. A common complaint of many widely used electronic health record systems is the relative lack of customizability for individual health systems and clinicians. We developed a HIT interface with, and for, frontline clinician users in safety-net clinic. Safety-net clinics often lack the resources for HIT-based interventions, yet these sites might gain the most from such interventions given the clinician’s workload and disease burden. Patients experiencing socioeconomic vulnerabilities tend to have chaotic lives; modest interventions involving reminders and other notifications have the potential to make significant improvements to appointment adherence and thereby therapeutic control. Moreover, technology-enabled panel management, as in our intervention, will help clinicians triage patients according to risk for harm or loss to follow-up and thereby more efficiently allocate limited resources in this setting.

    If successful, our study will demonstrate an example of integration of HIT platforms into existing electronic health records to improve patient care in real-world clinical settings. As we designed the intervention to work within the existing clinic personnel and electronic health record, it also promises to be sustainable and adaptable to other clinics within and outside of the network that share a similar data infrastructure, allowing for the retention of customizations made at pilot sites. However, modified panel management workflows will vary from site to site as the ability to tailor the intervention is, in part, what makes it novel. Finally, the intervention facilitates the use of evidence-based dosing algorithms, thereby promoting the adoption of evidence-based practice in anticoagulation clinics—an important strategy to optimize health outcomes at the population level.


    The authors would like to thank all the frontline clinicians in the anticoagulation clinic whose input has critically informed the design, implementation, and analysis plan outlined above. This research was supported by the Agency for Healthcare Research and Quality (AHRQ) grants P30HS023558 and R21HS21322 and National Institutes of Health (NIH) K23HL136899. The contents herein are solely the responsibility of the authors and do not necessarily represent the official views of the AHRQ or the NIH.

    Authors' Contributions

    VF contributed to conceptualization of the study and played the leading role in constructing the study design and writing the manuscript. DK played a leading role in conceptualization of the study, designing the intervention, and development of the dashboard. He also contributed to writing, reviewing, and editing the manuscript. RC played a leading role in coordinating the dashboard development and implementation of the intervention. He also played a significant role in writing the manuscript. SYL contributed to designing the intervention and led its implementation. She also contributed to writing, reviewing, and editing of the manuscript. US played the leading role in conceptualization of the study, designing the intervention, developing the dashboard, and reviewing and editing of the manuscript.

    Conflicts of Interest

    None declared.


    1. Budnitz DS, Pollock DA, Weidenbach KN, Mendelsohn AB, Schroeder TJ, Annest JL. National surveillance of emergency department visits for outpatient adverse drug events. J Am Med Assoc 2006 Oct 18;296(15):1858-1866. [CrossRef] [Medline]
    2. Harris Y, Hu DJ, Lee C, Mistry M, York A, Johnson TK. Advancing medication safety: establishing a national action plan for adverse drug event prevention. Jt Comm J Qual Patient Saf 2015 Aug;41(8):351-360. [CrossRef] [Medline]
    3. Lader E, Martin N, Cohen G, Meyer M, Reiter P, Dimova A, et al. Warfarin therapeutic monitoring: is 70% time in the therapeutic range the best we can do? J Clin Pharm Ther 2012 Aug;37(4):375-377. [CrossRef] [Medline]
    4. Budnitz DS, Shehab N, Kegler SR, Richards CL. Medication use leading to emergency department visits for adverse drug events in older adults. Ann Intern Med 2007 Dec 4;147(11):755-765. [CrossRef] [Medline]
    5. Carlson B. Declaring war on warfarin misdosing. Biotechnol Healthc 2008 Jul;5(2):54-55 [FREE Full text] [Medline]
    6. Kripalani S, Robertson R, Love-Ghaffari MH, Henderson LE, Praska J, Strawder A, et al. Development of an illustrated medication schedule as a low-literacy patient education tool. Patient Educ Couns 2007 Jun;66(3):368-377. [CrossRef] [Medline]
    7. Machtinger EL, Wang F, Chen L, Rodriguez M, Wu S, Schillinger D. A visual medication schedule to improve anticoagulation control: a randomized, controlled trial. Jt Comm J Qual Patient Saf 2007 Oct;33(10):625-635. [CrossRef] [Medline]
    8. Paradise HT, Berlowitz DR, Ozonoff A, Miller DR, Hylek EM, Ash AS, et al. Outcomes of anticoagulation therapy in patients with mental health conditions. J Gen Intern Med 2014 Jun;29(6):855-861 [FREE Full text] [CrossRef] [Medline]
    9. Rao SR, Reisman JI, Kressin NR, Berlowitz DR, Ash AS, Ozonoff A, et al. Explaining racial disparities in anticoagulation control: results from a study of patients at the Veterans Administration. Am J Med Qual 2015 May;30(3):214-222. [CrossRef] [Medline]
    10. Hoffman RM, Steel S, Yee E, Massie L, Schrader R, Moffett M, et al. A system-based intervention to improve colorectal cancer screening uptake. Am J Manag Care 2011 Jan;17(1):49-55 [FREE Full text] [Medline]
    11. Casalino L, Gillies RR, Shortell SM, Schmittdiel JA, Bodenheimer T, Robinson JC, et al. External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases. J Am Med Assoc 2003;289(4):434-441. [CrossRef] [Medline]
    12. Schmittdiel J, Bodenheimer T, Solomon NA, Gillies RR, Shortell SM. Brief report: The prevalence and use of chronic disease registries in physician organizations. A national survey. J Gen Intern Med 2005 Sep;20(9):855-858 [FREE Full text] [CrossRef] [Medline]
    13. van der Wees PJ, Friedberg MW, Guzman EA, Ayanian JZ, Rodriguez HP. Comparing the implementation of team approaches for improving diabetes care in community health centers. BMC Health Serv Res 2014 Dec 3;14:608 [FREE Full text] [CrossRef] [Medline]
    14. Young AS, Chaney E, Shoai R, Bonner L, Cohen AN, Doebbeling B, et al. Information technology to support improved care for chronic illness. J Gen Intern Med 2007 Dec;22(Suppl 3):425-430 [FREE Full text] [CrossRef] [Medline]
    15. Use of patient registries in US primary care practices. Am Fam Physician 2007 Jun 1;75(11):1629 [FREE Full text] [Medline]
    16. Bray P, Thompson D, Wynn JD, Cummings DM, Whetstone L. Confronting disparities in diabetes care: the clinical effectiveness of redesigning care management for minority patients in rural primary care practices. J Rural Health 2005;21(4):317-321. [CrossRef] [Medline]
    17. East J, Krishnamurthy P, Freed B, Nosovitski G. Impact of a diabetes electronic management system on patient care in a community clinic. Am J Med Qual 2003;18(4):150-154. [CrossRef] [Medline]
    18. Howard J, Clark EC, Friedman A, Crosson JC, Pellerano M, Crabtree BF, et al. Electronic health record impact on work burden in small, unaffiliated, community-based primary care practices. J Gen Intern Med 2013 Jan;28(1):107-113 [FREE Full text] [CrossRef] [Medline]
    19. Lee S, Cherian R, Ly I, Horton C, Salley AL, Sarkar U. Designing and implementing an electronic patient registry to improve warfarin monitoring in the ambulatory setting. Jt Comm J Qual Patient Saf 2017 Jul;43(7):353-360 [FREE Full text] [CrossRef] [Medline]
    20. Harte R, Glynn L, Rodríguez-Molinero A, Baker PM, Scharf T, Quinlan LR, et al. A human-centered design methodology to enhance the usability, human factors, and user experience of connected health systems: a three-phase methodology. JMIR Hum Factors 2017 Mar 16;4(1):e8 [FREE Full text] [CrossRef] [Medline]
    21. CipherHealth: Patient Engagement Company, Patient Outreach.   URL: [accessed 2019-02-11]
    22. Ansell J, Hirsh J, Hylek E, Jacobson A, Crowther M, Palareti G. Pharmacology and management of the vitamin K antagonists: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest 2008 Jun;133(6 Suppl):160S-198S. [CrossRef] [Medline]
    23. Reynolds MW, Fahrbach K, Hauch O, Wygant G, Estok R, Cella C, et al. Warfarin anticoagulation and outcomes in patients with atrial fibrillation: a systematic review and metaanalysis. Chest 2004 Dec;126(6):1938-1945. [CrossRef] [Medline]
    24. Vestergaard AS, Skjøth F, Larsen TB, Ehlers LH. The importance of mean time in therapeutic range for complication rates in warfarin therapy of patients with atrial fibrillation: A systematic review and meta-regression analysis. PLoS One 2017;12(11):e0188482 [FREE Full text] [CrossRef] [Medline]
    25. Duxbury BM. Assessment of therapeutic control of anticoagulation. Acta Haematol 1985;73(2):118-119 [FREE Full text] [CrossRef] [Medline]
    26. Rosendaal F, Cannegieter S, van der Meer F, Briët E. A method to determine the optimal intensity of oral anticoagulant therapy. Thromb Haemost 1993 Mar 1;69(3):236-239. [CrossRef] [Medline]
    27. Zulkifly H, Lip GY, Lane DA. Use of the SAMe-TTR score to predict anticoagulation control in atrial fibrillation and venous thromboembolism patients receiving vitamin K antagonists: A review. Heart Rhythm 2018 Apr;15(4):615-623. [CrossRef] [Medline]
    28. Bereznicki LR, Jackson SL, Peterson GM. Supervised patient self-testing of warfarin therapy using an online system. J Med Internet Res 2013 Jul 12;15(7):e138 [FREE Full text] [CrossRef] [Medline]
    29. Hsieh FY, Bloch DA, Larsen MD. A simple method of sample size calculation for linear and logistic regression. Stat Med 1998;17(14):1623-1634. [CrossRef]


    ADE: adverse drug event
    AHRQ: Agency for Healthcare Research and Quality
    AVR: atrial valve replacement
    DOAC: direct-acting oral anticoagulant
    DVT: deep venous thrombosis
    EHR: electronic health record
    HIT: health information technology
    INR: international normalized ratio
    LMM: linear mixed model
    NIH: National Institutes of Health
    MVR: mitral valve replacement
    PAH: pulmonary arterial hypertension
    PE: pulmonary embolism
    SFDPH: San Francisco Department of Public Health
    TTR: time in therapeutic range

    Edited by G Eysenbach; submitted 26.02.19; peer-reviewed by J Hay, A Meyer; comments to author 25.04.19; revised version received 29.07.19; accepted 04.09.19; published 13.01.20

    ©Valy Fontil, Dhruv Kazi, Roy Cherian, Shin-Yu Lee, Urmimala Sarkar. Originally published in JMIR Research Protocols (, 13.01.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.