Background: In the United States, poor adherence accounts for up to 70% of all medication-related hospital admissions, resulting in $100 billion in health care costs annually. In pediatrics, adherence is largely dependent on caregivers. In a high-risk hematopoietic stem cell transplant (HSCT) population, caregivers are isolated with their child due to infection risk and must manage challenging treatment regimens at home, often with limited time and support. Complex behavioral interventions, typically employed to address adherence, are difficult to deliver and manage in the context of these daily tasks. The most successful adherence interventions, and thus improved clinical outcomes, have included mobile health (mHealth) reminder approaches and a direct measure of adherence.
Objective: This is a 3-phase project, with this protocol describing phase 2, to determine the usability and feasibility of an mHealth app (BMT4me) designed to promote adherence to immunosuppressant medication and to track symptoms among children who received HSCT.
Methods: This study uses an iterative convergent mixed methods design to develop and assess the usability and feasibility of an adherence digital health intervention. We will recruit 15 caregivers of pediatric patients receiving HSCT to complete user testing. Qualitative and quantitative data will be integrated to enhance and expand upon study findings.
Results: Enrollment began in September 2021 and is ongoing. A total of 7 caregivers have enrolled. We anticipate completion by fall 2022. We anticipate high usability scores and a better understanding of unique features within the app that are needed for HSCT families post transplant. To date, usability scores among enrolled participants are greater than 70%. Feedback from qualitative interviews is being used to further adapt the app by adding specific weekly logs, call provider options, and voice to text.
Conclusions: This protocol describes a mixed methods usability and feasibility study to develop and implement a smartphone app for caregivers of children receiving HSCT. The app was designed to improve immunosuppressant adherence and to track symptoms in the acute phase post discharge. Study findings will inform further refinement of the app and the feasibility of a pilot randomized controlled trial examining efficacy on clinical outcomes.
Trial Registration: ClinicalTrials.gov NCT04976933; https://clinicaltrials.gov/ct2/show/NCT04976933
International Registered Report Identifier (IRRID): DERR1-10.2196/39098
In the United States, poor adherence accounts for up to 70% of all medication-related hospital admissions , resulting in $100 billion in health care costs annually [ ]. An estimated 50% to 80% of pediatric patients are nonadherent to medications [ - ]. Reasons for nonadherence are multifactorial. The most important determinants of nonadherence are consistently documented as complexity and duration of treatment regimens, as well as forgetfulness [ - ]. Thus, children undergoing difficult hematopoietic stem cell transplants (HSCTs) who require medication indefinitely are at high risk for medication nonadherence.
HSCT is the transplantation of hematopoietic stem cells derived from bone marrow (ie, “bone marrow transplant” or peripheral blood stem cells). HSCT became the common treatment for life-threatening malignant (eg, leukemia, lymphoma) and nonmalignant (eg, aplastic anemia, sickle cell disease, immune deficiencies) disorders in the mid-1900s. Recipients of bone marrow transplants must adhere to multifaceted outpatient regimens that require strict personal hygiene, environmental restrictions, and complex medication regimens. These include wearing masks, avoiding large crowds, frequent mouth care, and taking multiple time-sensitive medications throughout the day.
Lack of adherence to any of these regimens can be life-threatening . Adherence to immunosuppressant medications during the acute phase (ie, the first 100 days) post transplant is critical to prevent graft versus host disease and avoid graft failure. Studies in children with HSCT report suboptimal adherence rates (52%-78%) that worsen over time [ - ]. Nonadherence to oral medications has been associated with a greater incidence of infections in children during the outpatient HSCT treatment phase [ ]. Providers also report adherence as a major concern for outpatient pediatric HSCT recipients [ ]. However, no studies have examined adherence specifically to immunosuppressant medications, which are key to engraftment and ultimately survival.
Multiple factors influence adherence, but, ultimately, the final common pathway to adherence is human behavior . For children, caregivers are ultimately responsible for adherence, including refilling prescriptions, retrieving medications, and administering them correctly. In a high-risk HSCT population, caregivers are susceptible to fatigue and stress from being isolated with their child due to infection risk and independently managing complex treatment regimens at home, often with limited time and support. Unfortunately, behavioral interventions employed to address adherence are often difficult to deliver and manage in the context of these daily tasks. Specifically, these interventions can involve 8 to 12 face-to-face sessions, tracking behaviors, practicing skills, and other homework, which can be overwhelming to caregivers.
Alternatively, behavioral economics (BE) theory suggests that small “nudges” can produce and sustain behavior change . A BE approach is a significant paradigm shift from the complex cognitive behavioral theories that inform most adherence interventions. Instead, BE assumes decision-making can be influenced through low-intensity interventions to lead patients to optimal choices [ ]. Adherence to medication and exercise programs using BE-designed interventions in adults have some initial promising data [ - ]. Within pediatrics, for example, BE has been successful in increasing fruit consumption and improving vaccination rates [ - ].
With mobile health (mHealth) access nearly ubiquitous and an estimated 250 billion mHealth app downloads in 2022 , technology has great potential to improve adherence [ ]. mHealth interventions based on BE principles may be ideal, as they provide sophisticated yet simple “nudges” to individuals [ ]. Such reminder apps target forgetfulness, a common reason for poor adherence [ , ]. These mHealth interventions are ideally suited for complex HSCT regimens requiring multiple-dose medications that are time sensitive.
A recent systematic review in solid organ transplant reported the most successful interventions, resulting in increased adherence and improved clinical outcomes (eg, complications, hospital admissions, survival) have used mHealth reminders and a direct measure of adherence . Although direct measures of adherence (eg, serum assays) can avoid bias inherent in self- or parent reports, no studies have used direct measures of adherence or mHealth interventions to promote immunosuppression adherence in the acute phase after an HSCT (ie, discharge to day 100). Furthermore, the feasibility, acceptability, and efficacy of BE-designed mHealth interventions have not been examined in the HSCT population. Thus, this study utilizes a BE approach to mHealth app development, as well as an iterative approach of cyclical evaluation and revision, to maximize the usability and adoption of the digital intervention, BMT4me.
BMT4me () is a digital health intervention designed to aid in medication management, optimize adherence, and track symptoms or side effects in real time. The app reminds users of medication doses, records the time medication is taken, and asks reasons for missed doses through pop-up notifications. Medications can be typed in the app by the user or entered using an image-to-text option, which converts the medication label into text. As symptom severity can affect adherence and inform the need for medical intervention, it is illustrated by both corresponding emojis and numbers on a 1-to-10 scale to aid understanding and for convenient entry. Additional features include a notes page for recording any details of care to communicate with the provider and the ability to upload pictures. The app can be downloaded to both iOS and Android devices. Currently, availability is limited to the English language, and no clinical data is shared with the medical team during the research study.
Research Aim and Rationale for Mixed Methods Methodology
The overall aim of this mixed methods study is to design and test a theory-driven mHealth app that can virtually assist caregivers of children post HSCT in improving medication adherence, tracking symptoms, and recording pertinent medical information via the journal feature. This app is different from other medication adherence apps available to patients with cancer in several ways: (1) features (eg, reminders, engaging emojis) in the app are designed to support medication adherence by applying BE principles to provide reminders and tailored notifications for caregivers; (2) it is specific to children after an HSCT; and (3) it provides more than medication reminders, with the design encouraging caregivers to engage with the app via symptom tracking and a journal feature that allows uploading of pictures.
Overall Study Design
The larger mixed methods mHealth app development study consists of 3 phases: (1) prototype development, (2) feasibility and usability, and (3) usability and clinical efficacy (). This protocol corresponds to phase 2, feasibility and usability among 15 caregivers immediately upon discharge from the inpatient unit post HSCT. Inclusion criteria include (1) caregiver of child aged 2 to 18 years undergoing allogeneic transplant, (2) access to a smartphone (Android or iOS), and (3) English speaking. Findings from phase 1 informed phase 2, and thus findings from phase 2 will lead to further refinement prior to phase 3. In phase 3, the smartphone app will be compared to usual care in a pilot randomized controlled trial.
All phases of the study have been approved by the Nationwide Children’s Hospital Institutional Review Board (approval number STUDY00000910).
Phase 2 Study Design
This is a convergent mixed methods design focused on analysis of quantitative and qualitative data simultaneously collected to obtain feedback on the adherence mHealth app interface, usability, and flow and content of the software. Usability testing is the principal means of determining if the system meets its intended purpose . Purposive sampling will be used to identify caregivers of children across the spectrum of age, sex, and diagnoses of pediatric patients treated with HSCT. Caregivers will be evaluated for engagement and understanding of the app in the following steps ( ):
- Step 1. Unobtrusive observation to measure the intuitiveness of the interface (10-15 minutes). Initially, caregivers will be briefly introduced to the program and asked to start interacting with the adherence mHealth app. The design team will not intervene or provide instructions and will observe and record participants’ progress.
- Step 2. Interactive observation (10-15 minutes). The design team will begin to interrupt individual participants during the interaction by posing questions regarding observed cues that imply success or delay in program use. Caregivers will be encouraged to verbalize their criticisms of the mHealth app. Since discussion will be based on evolving patterns of interaction, pre-prescribed questions are not possible.
- Step 3. Debriefing (15-20 minutes). Caregivers will be asked to reflect on their experience with the app and its interface and contents, and to indicate their opinions regarding the strengths and weaknesses of the app. These evaluations will be used to establish criteria for refining the content and interactive app features for future development.
- Step 4. Passive use observation. Caregivers will be asked to continue using the mHealth app on their own device until day 100 post HSCT or weaning of the immunosuppressant, whichever occurs first. Passive data modules will be added to the app to capture phone activity (screen on and off times) and will log the participants’ use of the app (eg, time, date, duration of use). Changes in app use and phone activity will be descriptively analyzed and plotted. Subjective measures and standard-of-care immunosuppression serum assays will be recorded weekly. Such passive observational effort of app use and adherence, with a mixed method approach, will be one of the first to be reported in an HSCT population.
At the conclusion, caregivers will complete a 15-minute semistructured interview related to participation in the intervention, as well as the System Usability Scale . Open-ended feedback related to suggestions for further app development will undergo qualitative analysis.
Trained research assistants will recruit and onboard eligible participants. The participants will sign written consent forms. The research assistant will then conduct one-on-one audio-recorded sessions, which will include a facilitator and a note taker. A project scientist may also participate. The measures listed below will be collected either electronically or in paper-and-pencil form from each caregiver. All participants will receive US $25 at enrollment and US $25 at the conclusion of the study after the exit interview and measures are complete.
The BMT4me App
The BMT4me app will collect daily data on medication-taking (taken or not taken), the time medication was taken, barriers or reasons for missed doses, symptoms, and any notes the caregiver completes. Passive use data on phone activity and app use will be collected.
Demographic Data Form
The caregiver will report on basic background characteristics, including parent and child age, sex, race, ethnicity, education level, and family income.
Medication Adherence Measure
The Medication Adherence Measure (MAM) is a semistructured interview specific to pediatrics, conducted with the parent, to obtain an individual score in each module. The score is represented as percentages of the number of required doses. A total summary score can be calculated across all medications as well separately. This allows for quantification of the degree of adherence on a continuum. MAM has demonstrated adequate convergent validity with MEMS (medication event monitoring system) Caps (r=−0.40, P<.05) . MEMS Caps are electronic monitors that objectively measure adherence. They contain microelectronic circuits that date and time-stamp each instance the container is opened for a dose of medication. Data from the MEMS Caps will be downloaded using cloud- or computer-based software at each study visit [ , ].
System Usability Scale
The System Usability Scale is a 10-item questionnaire routinely used to evaluate the functionality and acceptability of mHealth apps [, ]. Items are rated on a 5-point scale and scores range from 0 to 100 [ ]. Reliability (0.91) and validity (0.81 correlation with a 7-point scale of “user-friendliness”) have been well established [ ]. A score of >68% is considered above average [ ].
Satisfaction will be assessed via semistructured interviews and electronic surveys with caregivers. Caregivers will be asked for feedback regarding participation in the intervention, benefits, burden, barriers, suggested modifications, and overall satisfaction. Suggested modifications to the app and advice to the health care team will also be solicited.
Medication Level Variability Index
The medication level variability index is the calculation of the standard deviation of serum assays of immunosuppressants that has been shown to correlate with adherence and clinical outcomes in the solid organ transplant population . Immunosuppressant serum assays are collected weekly during the acute phase. The degree of variation among levels will also be calculated.
All interviews will be transcribed verbatim. During qualitative analysis, investigators will use an iterative process of reading, summarizing, and rereading data. The qualitative team is composed of 3 individuals with formal research training in psychology, behavioral health, and nursing. The interview transcripts will be organized and coded using NVIVO software (QSR International). In-vivo coding will occur as new themes emerge from the data. The data will be triangulated by comparing and integrating quantitative survey data with qualitative findings.
Quantitative data will be summarized with descriptive statistics (frequencies, means, and SDs). During the passive use observation period, passive data modules will capture phone activity and caregivers’ application use (eg, time, date, duration of use). Descriptive statistics will be used to analyze phone activity. Correlation analysis will be used to investigate use behavior over time. Acceptability will be assessed by averaging total scores from the System Usability Scale. Consistent with the literature , scores >68% on the System Usability Scale will be considered acceptable. The proportion of participants that enroll and complete the study will be examined to assess study feasibility.
The nature of the study is exploratory rather than confirmatory. Thus, the objective of the analysis is effect size estimation, rather than formal hypothesis testing. Threats to power (eg, participant attrition due to patient death, early taper of immunosuppressant medication) are not a primary concern. We will compute pre-post effect sizes (eg, a standardized mean difference) to inform the future randomized controlled trial (phase 3) assessing the efficacy of the intervention.
Enrollment of caregivers of children post HSCT from the free-standing Midwestern Children’s Hospital began in September 2021 and is ongoing. Findings from phase 2 will be used to inform further development and refinement of the app for testing in the pilot randomized controlled trial (phase 3) anticipated to begin in fall 2022. We anticipate high usability scores and a better understanding of unique features within the app that are needed for HSCT families post transplant. To date, usability scores among enrolled participants are greater than 70%. Feedback from qualitative interviews is being used to further adapt the app by adding specific weekly logs, call provider options, and voice to text. The results of this study will be disseminated through presentations at scientific conferences and publication in peer-reviewed journals.
To our knowledge, this is one of the first studies to use a systematic, phased approach to the development of a digital health intervention and evaluation of usability and feasibility to improve outcomes in pediatric HSCT. From this study, we anticipate high usability scores and a better understanding of unique features within the app that are needed for HSCT families post transplant. The long-term goal of this research is to develop novel digital interventions to increase adherence and ultimately improve clinical outcomes for these high-risk children. In addition, there is potential to adapt our work for children with other chronic conditions.
The strengths of the mixed methods approach and proposed digital health intervention include constructs from behavioral health and BE theory that drive data collection and analysis, as well as the use of multiple methods of data collection and multiple data sources to gain a comprehensive understanding of patient needs. The development of the intervention has undergone multiple iterations based on feedback from multiple stakeholders, including pediatric patients, caregivers, physicians, and nurses, each offering unique and important perspectives.
Potential limitations of the study include the use of 1 pilot recruitment site; however, there is a diverse group of patients requiring HSCT at this institution. A small number of patient caregivers will be recruited, and a larger user population could yield additional data. At this time, only English-speaking caregivers are eligible; opportunities to increase diversity by adding additional languages are planned in future iterations.
This protocol describes the second phase of a multiphase, theory-driven digital health app intervention to improve adherence and symptom tracking in children post HSCT. The study findings will provide important knowledge regarding the feasibility of testing in a larger randomized controlled trial. The goal of the entire project is to improve adherence and clinical outcomes in children post HCST, which may have important implications for improving adherence in other clinical populations.
The research reported in this publication was supported by the National Institute of Nursing Research of the National Institutes of Health under award number K99NR019115. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
We are grateful for the support of the Information Technology Research Informatics group at the Nationwide Children’s Hospital, especially Brannon Oiler, Brandon Abbott, and John Luna for app design and development.
MS conceived and designed the study. MS wrote the initial draft of the manuscript. MS, ES, JS, WL, AP, and CG made substantial contributions to the planning and design of the study and contributed to the revision of the manuscript. All authors read and approved the final version of the manuscript.
Conflicts of Interest
Peer-review report by the National Institute of Nursing Research Initial Review Group (NRRC) (National Institutes of Health, USA).PDF File (Adobe PDF File), 151 KB
- Bissonnette JM. Adherence: a concept analysis. J Adv Nurs 2008 Sep;63(6):634-643. [CrossRef] [Medline]
- Drotar DD, Crawford P, Bonner M. Collaborative decision-making and promoting treatment adherence in pediatric chronic illness. PI 2010 Mar;2:1-7. [CrossRef]
- Drotar D. Strategies of adherence promotion in the management of pediatric chronic conditions. J Dev Behav Pediatr 2013;34(9):716-729. [CrossRef] [Medline]
- McGrady M, Hommel KA. Medication adherence and health care utilization in pediatric chronic illness: a systematic review. Pediatrics 2013 Oct;132(4):730-740 [FREE Full text] [CrossRef] [Medline]
- Fiese B, Everhart RS. Medical adherence and childhood chronic illness: family daily management skills and emotional climate as emerging contributors. Curr Opin Pediatr 2006 Oct;18(5):551-557. [CrossRef] [Medline]
- Landier W. Age span challenges: adherence in pediatric oncology. Semin Oncol Nurs 2011 May;27(2):142-153. [CrossRef] [Medline]
- Shemesh E, Shneider BL, Savitzky JK, Arnott L, Gondolesi GE, Krieger NR, et al. Medication adherence in pediatric and adolescent liver transplant recipients. Pediatrics 2004 Apr;113(4):825-832. [CrossRef] [Medline]
- Quine L, Steadman L, Thompson S, Rutter DR. Adherence to anti-hypertensive medication: proposing and testing a conceptual model. Br J Health Psychol 2012 Feb;17(1):202-219. [CrossRef] [Medline]
- Varni JW, Wallander JL. Adherence to health-related regimens in pediatric chronic disorders. Clin Psych Rev 1984 Jan;4(5):585-596. [CrossRef]
- Hoodin F. Psychological and behavioral correlates of medical adherence among adult bone marrow transplant recipients (Dissertation). Detroit, MI: Wayne St. University; 1993.
- Phipps S, DeCuir-Whalley S. Adherence issues in pediatric bone marrow transplantation. J Pediatr Psychol 1990 Aug;15(4):459-475. [CrossRef] [Medline]
- Pai AL, Rausch J, Drake S, Morrison CF, Lee JL, Nelson A, et al. Poor adherence is associated with more infections after pediatric hematopoietic stem cell transplant. Biol Blood Marrow Transplant 2018 Feb;24(2):381-385 [FREE Full text] [CrossRef] [Medline]
- McGrady ME, Williams SN, Davies SM, Pai AL. Adherence to outpatient oral medication regimens in adolescent hematopoietic stem cell transplant recipients. Eur J Oncol Nurs 2014 Apr;18(2):140-144 [FREE Full text] [CrossRef] [Medline]
- Skeens M, Akard TF, Gilmer MJ, Garee A, Miller A. Toward a better understanding of pediatric BMT adherence: an exploration of provider perceptions. Bio Blood Marrow Transplant 2014 Feb;20(2):S315. [CrossRef]
- Patel MS, Volpp KG, Asch DA. Nudge units to improve the delivery of health care. N Engl J Med 2018 Jan 18;378(3):214-216. [CrossRef]
- Linnemayr S, MacCarthy S, Kim A, Giguere R, Carballo-Dieguez A, Barreras JL. Behavioral economics-based incentives supported by mobile technology on HIV knowledge and testing frequency among Latino/a men who have sex with men and transgender women: Protocol for a randomized pilot study to test intervention feasibility and acceptability. Trials 2018 Oct 05;19(1):540 [FREE Full text] [CrossRef] [Medline]
- Stevens J. Behavioral economics as a promising framework for promoting treatment adherence to pediatric regimens. J Pediatr Psychol 2014 Sep;39(10):1097-1103. [CrossRef] [Medline]
- van Mierlo T, Hyatt D, Ching AT, Fournier R, Dembo RS. Behavioral economics, wearable devices, and cooperative games: results from a population-based intervention to increase physical activity. JMIR Serious Games 2016 Jan 28;4(1):e1 [FREE Full text] [CrossRef] [Medline]
- Mitchell MS, Goodman JM, Alter DA, John LK, Oh PI, Pakosh MT, et al. Financial incentives for exercise adherence in adults: systematic review and meta-analysis. Am J Prev Med 2013 Nov;45(5):658-667. [CrossRef] [Medline]
- Angellotti E, Wong JB, Pierce A, Hescott B, Pittas AG. Combining Wireless Technology and Behavioral Economics to Engage Patients (WiBEEP) with cardiometabolic disease: a pilot study. Pilot Feasibility Stud 2019 Jan 15;5(1):7 [FREE Full text] [CrossRef] [Medline]
- Chang LL, DeVore AD, Granger BB, Eapen ZJ, Ariely D, Hernandez AF. Leveraging behavioral economics to improve heart failure care and outcomes. Circulation 2017 Aug 22;136(8):765-772. [CrossRef]
- Malo TL, Hall ME, Brewer NT, Lathren CR, Gilkey MB. Why is announcement training more effective than conversation training for introducing HPV vaccination? A theory-based investigation. Implement Sci 2018 Apr 19;13(1):57 [FREE Full text] [CrossRef] [Medline]
- Opel DJ, Taylor JA, Zhou C, Catz S, Myaing M, Mangione-Smith R. The relationship between parent attitudes about childhood vaccines survey scores and future child immunization status: a validation study. JAMA Pediatr 2013 Nov 01;167(11):1065-1071 [FREE Full text] [CrossRef] [Medline]
- Taylor NK, Buttenheim AM. Improving utilization of and retention in PMTCT services: can behavioral economics help? BMC Health Serv Res 2013 Oct 10;13(1):406 [FREE Full text] [CrossRef] [Medline]
- Barth S, de Jong MD, Junger M. Lost in privacy? Online privacy from a cybersecurity expert perspective. Telemat Inform 2022 Mar;68:101782. [CrossRef]
- Camacho E, LoPresti M, Appelboom G, Dumont ELP, Taylor B, Connolly ES. The ubiquitous role of smartphones in mobile health. BBIJ 2014 Sep 27;1(1):14-19. [CrossRef]
- Asch DA, Muller RW, Volpp KG. Automated hovering in health care — watching over the 5000 hours. N Engl J Med 2012 Jul 05;367(1):1-3. [CrossRef]
- Skrabal Ross X, Gunn KM, Suppiah V, Patterson P, Olver I. A review of factors influencing non-adherence to oral antineoplastic drugs. Support Care Cancer 2020 Sep 25;28(9):4043-4050. [CrossRef] [Medline]
- Unni E, Sternbach N, Goren A. Using the Medication Adherence Reasons Scale (MAR-Scale) to identify the reasons for non-adherence across multiple disease conditions. PPA 2019 Jun;Volume 13:993-1004. [CrossRef]
- Duncan S, Annunziato RA, Dunphy C, LaPointe Rudow D, Shneider BL, Shemesh E. A systematic review of immunosuppressant adherence interventions in transplant recipients: Decoding the streetlight effect. Pediatr Transplant 2018 Feb 07;22(1):e13086 [FREE Full text] [CrossRef] [Medline]
- Alwashmi MF, Hawboldt J, Davis E, Fetters MD. The iterative convergent design for mobile health usability testing: mixed methods approach. JMIR Mhealth Uhealth 2019 Apr 26;7(4):e11656. [CrossRef]
- Fedele DA, Cushing CC, Fritz A, Amaro CM, Ortega A. Mobile health interventions for improving health outcomes in youth: a meta-analysis. JAMA Pediatr 2017 May 01;171(5):461-469 [FREE Full text] [CrossRef] [Medline]
- Hommel KA, Davis CM, Baldassano RN. Objective versus subjective assessment of oral medication adherence in pediatric inflammatory bowel disease. Inflamm Bowel Dis 2009 Apr;15(4):589-593 [FREE Full text] [CrossRef] [Medline]
- Quittner AL, Modi AC, Lemanek KL, Ievers-Landis CE, Rapoff MA. Evidence-based assessment of adherence to medical treatments in pediatric psychology. J Pediatr Psychol 2008 Oct 18;33(9):916-36; discussion 937 [FREE Full text] [CrossRef] [Medline]
- Ingerski LM, Hente EA, Modi AC, Hommel KA. Electronic measurement of medication adherence in pediatric chronic illness: a review of measures. J Pediatr 2011 Oct;159(4):528-534 [FREE Full text] [CrossRef] [Medline]
- Lewis JR, Sauro J, editors. The Factor Structure of the System Usability Scale. Berlin, Heidelberg: Springer; 2009.
- Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction 2008 Jul 30;24(6):574-594. [CrossRef]
- Shemesh E, Bucuvalas JC, Anand R, Mazariegos GV, Alonso EM, Venick RS, et al. The Medication Level Variability Index (MLVI) predicts poor liver transplant outcomes: a prospective multi-site study. Am J Transplant 2017 Oct 22;17(10):2668-2678 [FREE Full text] [CrossRef] [Medline]
|BE: behavioral economics|
|HSCT: hematopoietic stem cell transplant|
|MAM: Medication Adherence Measure|
|MEMS: medication event monitoring system|
|mHealth: mobile health|
Edited by T Leung; This paper was peer reviewed by the National Institute of Nursing Research Initial Review Group (NRRC) (National Institutes of Health, USA). See the Multimedia Appendix for the peer-review report; submitted 28.04.22; accepted 13.05.22; published 21.07.22Copyright
©Micah Skeens, Emre Sezgin, Jack Stevens, Wendy Landier, Ahna Pai, Cynthia Gerhardt. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 21.07.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.