Published on in Vol 11, No 3 (2022): March

Preprints (earlier versions) of this paper are available at, first published .
mHealth Intervention to Improve Treatment Outcomes Among People With HIV Who Use Cocaine: Protocol for a Pilot Randomized Controlled Trial

mHealth Intervention to Improve Treatment Outcomes Among People With HIV Who Use Cocaine: Protocol for a Pilot Randomized Controlled Trial

mHealth Intervention to Improve Treatment Outcomes Among People With HIV Who Use Cocaine: Protocol for a Pilot Randomized Controlled Trial


1Department of Communication, University of Missouri, Columbia, MO, United States

2Department of Communication, University at Albany, State University of New York, New York, NY, United States

3Department of Geography, University of Connecticut, Storrs, CT, United States

4Harvard T H Chan School of Public Health, Harvard University, Boston, MA, United States

5Department of Internal Medicine, Section of Infectious Diseases, Yale University, New Haven, CT, United States

Corresponding Author:

Yerina S Ranjit, PhD

Department of Communication

University of Missouri

108 Switzler Hall

Columbia, MO, 65211

United States

Phone: 1 573 822 0881


Related ArticleThis is a corrected version. See correction statement in:

Background: Antiretroviral therapy is effective in reducing HIV-related morbidity, mortality, and transmission among people with HIV. However, adherence and persistence to antiretroviral therapy are crucial for successful HIV treatment outcomes. People with HIV who use cocaine have poor access to HIV services and lower retention in care.

Objective: The primary goal of this paper is to provide a detailed description of a mobile health intervention. This study is designed to improve medication adherence among people with HIV who use cocaine. A secondary goal is to list the important challenges and adaptations incorporated in the study design.

Methods: This study, titled Project SMART, used a wireless technology–based intervention, including cellular-enabled electronic pillboxes called TowerView Health and smartphones, to provide reminders and feedback on adherence behavior. The intervention design was based on the theoretical frameworks provided by the self-determination theory and the Motivation Technology Model. The 12-week pilot randomized controlled trial with four arms provided three types of feedback: automated feedback, automated+clinician feedback, and automated feedback+social network feedback.

Results: The study was funded by the National Institute of Drug Abuse (R21DA039842) on August 1, 2016. The institutional review board for the study was approved by Yale University on March 21, 2017. Data collection lasted from June 2017 to January 2020. The final enrollment was 71 participants, of whom 57 (80%) completed the study. The data are currently undergoing analysis, and the manuscript is being developed for publication in early 2022.

Conclusions: Implementing complex mobile health interventions for high-risk and marginalized populations with multicomponent interventions poses certain challenges, such as finding companies with adequate technology for clients and financial stability and minimizing the research-related burden for the study population. Conducting feasibility studies is important to recognize these challenges and the opportunity to address these challenges with solutions while keeping the design of a randomized controlled trial as true as possible.

Trial Registration: NCT04418076;

International Registered Report Identifier (IRRID): DERR1-10.2196/28332

JMIR Res Protoc 2022;11(3):e28332




Antiretroviral therapy (ART), a daily medication regimen, is extremely effective in reducing HIV-related morbidity and mortality and reducing transmission among people with HIV [1-3]. ART suppresses HIV viral load (VL) and slows the progression of the virus in the body by keeping a high CD4 cell count [4]. The Centers for Disease Control and Prevention defines viral suppression as having <200 copies of HIV per milliliter of blood [4]. However, successful HIV treatment outcomes depend on patients’ adherence to and persistence in ART [5-9]. When adherence is optimal (≥90%) [10], improvements are seen not only for HIV outcomes but also for several comorbidities, including substance use disorders, tuberculosis, viral hepatitis, and depression [11-15]. Among the approximately 1.2 million people with HIV in the United States, nonadherence to ART results in as few as 19% achieving viral suppression [16]. For people with HIV with co-occurring substance use disorders, there are additional negative health outcomes. Active drug use is associated with clinicians not prescribing or delaying prescribing ART [17], which can overshadow a patient’s health care priorities, including medication adherence and retention in HIV care [18]. As a result, people with HIV who use drugs experience worse clinical outcomes and greater mortality than people who do not use drugs [19]. People who use cocaine especially have poor measures of success in accessing and retaining HIV care [20,21]. They are less likely to receive ART compared with people who use other drugs [21,22], who have poor adherence to ART when they do receive it [23-25], and who have an extraordinarily low likelihood of achieving viral suppression [26]. Overall, AIDS-related death rates among people who use cocaine are 3 times higher than among those who do not use cocaine, controlling for age, race, duration of illness, and self-reported high ART adherence [26]. Effective strategies for improving ART adherence are urgently needed to address this health crisis among people with HIV who use cocaine.

For people with HIV who use cocaine, currently, the only effective adherence intervention is directly administered ART, which is expensive and labor intensive [27]. International guidelines suggest that adherence interventions need to be scaled back, both in terms of cost and personnel [28]. Mobile health (mHealth) technologies satisfy these recommendations and can provide innovative, efficacious, and cost-effective strategies for improving ART adherence and optimizing HIV treatment outcomes [29,30]. In fact, the World Health Organization guidelines now strongly recommend sending SMS text messages to promote adherence and retention in general [31]. mHealth has already shown promise in resource-limited settings [32] and improvement in treatment efficacy among patients with diabetes [33,34], tuberculosis [35], malaria [36], asthma [37], and HIV [38-40]. mHealth interventions have also been efficacious in reducing substance use [41]. In addition, bidirectional SMS text messaging interventions have significantly improved adherence to ART among people with HIV who use substances [42]. However, there has never been any systematic development or testing of mHealth interventions for people with HIV who use cocaine. Moreover, existing mHealth interventions have not optimized the integration of feedback to determine what type of feedback (eg, predetermined by researchers or clinicians, tailored to patients’ preferences, dynamically tailored based on patient’s adherence level, or feedback from friends and family members) improves outcomes. Prior studies have shown the effects of dynamically tailored feedback via SMS text messages on viral suppression compared with standard HIV care [43-45]. However, in these studies, the feedback was designed for people with HIV in a regulated clinical setting, which may not be effective for people with HIV using drugs in an unregulated community setting.

Study Objective

The primary objective of this paper is to provide a detailed description of a pilot mHealth-based randomized controlled trial (RCT) for improving ART adherence among people with HIV who use cocaine. A secondary goal is to list the critical challenges faced during the duration of the study and adaptations incorporated into the study design.

Pilot RCTs are small-scale studies designed to evaluate the feasibility and acceptability of a proposed intervention. Findings related to feasibility prepare investigators to conduct large-scale interventions, whereas acceptability shows how the population for whom the intervention is designed and the researchers who implement the study react to the intervention [46]. As part of a National Institute on Drug Abuse–funded study, we developed and conducted a feasibility study titled Project SMART, which developed a wireless technology–based intervention with multiple forms of feedback to improve ART adherence. In this paper, we aim to describe the research design of this pilot study, including the timeline, assessments, technologies, and challenges faced by researchers in implementing a multicomponent intervention. We also outline some practical adjustments made to the protocol during the study. By sharing our experiences, we aim to inform and educate other researchers interested in designing mHealth interventions for populations with similar characteristics. The following sections describe the methodologies and components of the pilot study as these are preanalysis protocol descriptions. The primary and secondary outcomes will be analyzed in forthcoming papers.

Study Setting

This study was conducted between May 2017 and January 2020. Participants were recruited from community centers, HIV care clinics, and a mobile clinic that has strategic liaisons with patients and providers throughout the Yale University community. The study took place within an HIV and opioid use disorder treatment facility, which is in an urban location in the northeastern part of the United States. The study was funded by the National Institute of Drug Abuse (R21DA039842).

Recruitment, Screening for Eligibility, and Consent

Recruitment started in May 2017 and continued until January 2020. Institutional review board (IRB)-approved flyers containing a toll-free number were posted in numerous venues such as community centers, HIV care clinics, and a mobile clinic that had strategic liaisons with patients and providers throughout the community. The initial screening eligibility criteria included patients (1) with a self-reported HIV diagnosis, (2) with self-reported use of cocaine ever, (3) having ART prescription and taking ART, and (4) willing and able to use a smartphone and electronic pillbox for a 16-week study period. After screening, the research staff obtained informed consent from the participants at their first visit and explained the purpose, protocol, expectations, and possible risks and benefits of the study.

Study Design

The design of this study is based on the fundamental components of transactional communication, which are the sender, receiver, and message [47]. In the transactional model of face-to-face communication, the sender and receiver engage in communication facilitated by feedback (verbal or nonverbal). With technologies such as the electronic pillbox, this exchange can take place only when the technology is responsive in some form to its user [48]. Interactive technologies enhance the engagement of its users [48]. Self-determination theory posits that the motivation to perform an action is influenced if an individual’s need for competence, autonomy, and relatedness is fulfilled [49]. Using this assumption, the Motivation Technology Model (MTM) predicts that individuals can be motivated to change their behaviors if and when they achieve a sense of relatedness while using the technology [50]. MTM argues that this interactive nature of technology, which fulfills the need to be socially connected, can motivate individuals to perform or maintain certain health behaviors [50]. This perception of interactivity can be established when the technology provides timely responses and feedback to its users [50]. On the basis of these theoretical concepts of the self-determination theory and MTM, this study uses feedback as the central motivational factor to study adherence behavior.

The study design of Project SMART is shown in Figure 1. This was a 12-week pilot RCT that examined the use of two mHealth tools (cellular-enabled electronic pillboxes and smartphones) and four types of feedback—(1) no feedback, (2) automated feedback, (3) automated+clinician feedback, and (4) automated feedback+social network feedback (Figure 1)—on primary (ART adherence and persistence) and secondary outcomes (HIV viral suppression, cocaine use, retention in HIV care, and retention in RCT). The study period was 16 weeks, with a 12-week intervention period and a 4-week postintervention period. The following sections describe each of the components of the study design in detail.

Figure 1. Study design of Project SMART. RN: registered nurse.
View this figure

mHealth Tools


Two mHealth tools—an electronic pillbox and smartphones—were used for intervention delivery. All study participants received these two mHealth tools but differed in the type of communication feedback they received: no feedback, automated feedback, automated and clinician feedback, and automated and social network feedback. The types of feedback determined the four arms of the RCT: 1 control arm and 3 intervention arms.

Electronic Pillbox

As shown in Figure 2, the TowerView Health pillbox is 10×7.5×1.5 inches, which is comparable with a handheld device such as an iPad, and can be linked to a smartphone. It has the capacity to hold pills or tablets for a weekly medication regimen of up to 4 dosages a day. The pills are nestled in 28 individual wells, each with its own sensor, to indicate when a dosage has been removed. The box has a built-in alarm that emits both a low sound and a flash of light. The pillboxes have 4G cellular capability via an integrated chip and hence have the ability to communicate with other devices such as smartphones. This capability is useful for transmitting real-time adherence data to the secure TowerView Health server and communicating with linked smartphones via SMS text messages.

Figure 2. Electronic pillbox TowerView Health used for Project SMART.
View this figure
Medication and Refills

At the start of study enrollment, the clinician associated with Project SMART requested permission from participants’ physicians to dispense ART medications from the selected pharmacy for the study period. The pharmacy collaborated with TowerView Health and delivered 4 sets (one for each week) of blister packs, with 1 months’ worth of ART medication specifically designed to fit the dimensions of the pillbox. Although the pillboxes are designed to hold medication pills independently, blister packs were used by TowerView Health for ease of delivery and organization as they did not require manual insertion of individual pills. Our clinical staff placed these blister packs in the box in the participants’ presence for the first time and trained the participants to replace the blister packs at the end of each week. Participants were instructed to bring their pillbox to the research site each month, at which time the research staff resupplied participants with a month’s supply of medications. A 1-month supply of medication is the community standard of care for the dispensation of medications for patients with HIV [51].

Automated Feedback Messages

All participants’ phones were connected to their assigned pillbox. The pillbox was programmed to send SMS text messages in the event of a missed dosage or messages of encouragement in the event of a successful daily dosage regimen. For example, when the pillbox was opened, and a segment of the blister pack was broken for medication intake, the sensor within the individual wells detected pressure, prompting a signal to be sent to the TowerView Health server. This then activated the automatic delivery of an SMS text message to the participant’s smartphone. The content of the SMS text messages or feedback delivered to a participant’s smartphone was determined by the research team. Refer to section Randomization Trial and Intervention for examples of messages sent to participants.


The study used Samsung phones with an Android operating system, which were purchased through Yale University’s agreement with Verizon. The participants received either a Samsung 7 or Samsung 8 model for the duration of the study, and the phone was configured for our study. The messaging app was activated so that participants could start receiving SMS text messages from the electronic pillbox. A total of 2 apps, CommCare and SMS Back-Up and Restore, were installed before handing out a phone to a participant (the role of these apps is described in the Study Instruments section). The research staff trained the participants on how to charge and use the electronic pillbox and smartphone (including using the voicemail and SMS text messaging features). In addition, the research staff programmed pertinent contact information (eg, clinic care and research staff contact) on each participant’s phone.

Randomization Trial and Intervention

After completing the baseline interview, participants were assigned to 1 of the 4 arms systematically at a 1:1:1:1 ratio. Arm A was the control group, and participants in this group did not receive feedback as an intervention via the mHealth tools. The in-built and automatic feedback features of the TowerView Health smart pillboxes were also deactivated.

Arm B participants received automated feedback messages as an intervention. The box sent text dosage reminders to the participants’ phones at the dosage time, and the rim around the box flashed blue, purple, and magenta light-emitting diode lights with a gentle ringing alarm. An example of the SMS text message is as follows: “Hi [participant name], don’t forget to take your [dose time, eg, 3 PM] medications.” If the pillbox was not opened at a dosage time, the box sent a reminder at 30 minutes and another at 90 minutes past the missed dosage time. The reminder said: “Hello [participant name], here’s a quick reminder to take your missed pill of [dose time].” If at the end of the day, the participant took all the dosages, they received a note of encouragement on their phones; for example, “Congratulations, [participant name]! You took all your pills yesterday.” These kinds of positive feedback were accompanied by emoticons such as a thumbs-up or clap. The positive messages varied daily and were selected from a set of messages designed by the team before the start of the study.

Arm C participants, in addition to the automated feedback from the pillbox, received feedback about their ART adherence from a clinician via phone calls on a weekly basis. The clinician’s phone conversation was guided by the real-time adherence data provided by the TowerView Health dashboard.

For arm D, in addition to the automated feedback from the pillbox, participants received SMS text message reminders to take their medication from a person in their social network or a preselected social designee. The designee was a family member, friend, or acquaintance (excluding clinicians) who was willing to be part of the intervention and consented to their participation. The research team sent a weekly notification to the designee’s mobile phone, prompting them to deliver the intervention via SMS text messages to the participant. The predefined automated notification was:

This is from Project SMART. In the next hour, send a text to [participant’ name] @ [participant’ number] to remind them to take their medication on time. Thank you for your help.!

The designee then sent SMS text messages to remind the participant’s phone about their ART dosages. No predefined SMS text messages for dyads (participant–social designee) were developed for this intervention. Instead, the actual contents of the SMS text message reminders between dyads were recorded as intervention data.

Participants in arms B, C, and D received their interventions for a duration of 12 weeks. After 12 weeks, the built-in feedback features of the electronic pillbox were deactivated such that during the 4-week postintervention period, the pillboxes only measured participants’ pill-taking behavior, similar to arm A.

Study Instruments


Project SMART collected data using a range of instruments and standardized assessments administered at various time points of a participant’s visits during preintervention (screening and baseline), intervention (4, 8, and 12 weeks), and postintervention (16 weeks) phases, spanning a period of 16 weeks. Table 1 contains a list of survey instruments.

Table 1. Study activity and instruments.
Study activity and measuresPreinterventionInterventionPostintervention, week 16

Before 4 weeks to
before 1 week
Preparation timeWeek 0Week 4Week 8Week 12
Study activity

Screening for eligibility

Release of information


Device feature setting and preparation

Acquiring permission to change pharmacy and ordering medication

Device and medication distribution

Assigning social network designee


Weekly smartphone survey (conducted weekly)

Demographic and social information


Social circumstances

Health care status

Health literacy [52]

Social support [52]

Trust in physician [52]

Mental health and risk factors


Depression (CES-Db) [53]

AUDIT-Cc [54]

Cocaine use disorder [55]

Substance use disorder

ASI-Lited to assess cocaine use, polysubstance use, and severity [56,57]

Timeline follow back [58]

LifeWindows (LW-IMB-AAQe) questionnaire to measure the information, motivation, and behavioral constructs of adherence [59,60]


Visual analog scale [61,62]

Wilson adherence scale [63]

Barriers to taking medication

Technology perceptions

Technology acceptance [64]

Credibility of SMS text message source [65]

Attitudes toward text and study

Geolocation and social network

Activity space

Social network

Social network designee

Smartphone survey

Phone call history

Payments (US $)

Research interviewsN/Af30N/AN/A30203020

Return of the devices (smartphone and pillbox)N/AN/AN/AN/AN/AN/AN/A90

Weekly smartphone surveysN/AN/AN/AN/AN/AN/AN/A80

Incentive to social network designeeN/AN/AN/AN/AN/AN/AN/A20

aBINI: Brief Inventory of Neurocognitive Impairment.

bCES-D: Center for Epidemiologic Studies–Depression.

cAUDIT-C: Alcohol Use Disorders Identification Test–Concise.

dASI-Lite: Addiction Severity Index–Lite.

eLW-IMB-AAQ: Life Windows Information Motivation Behavioral skills ART Adherence Questionnaire.

fN/A: not applicable.


The screening instrument included questions regarding HIV status, health insurance, current prescription of ART, social designees, and ability and willingness to use a smart pillbox and smartphone for the duration of the study. This instrument determined the eligibility of a participant.


The research staff completed the baseline survey after participants provided consent and a signed medical release of information for self-reported HIV status and HIV treatment regimen. Neurocognitive impairment (NCI) was assessed at baseline before assigning participants to the study arms. In addition, questions regarding their social networks (requested for arm D) were also administered at baseline. Phlebotomy measures using Quest Diagnostics were also conducted at baseline to conduct tests that included HIV-1 RNA testing (Amplicor 1.5, range 50-750,000 copies/mL), CD4 lymphocyte count using flow cytometry, and HIV genotypic mutations (if VL>500 copies/mL).

Weekly Survey

During the intervention period (12 weeks), participants completed a short survey per week using the CommCare app on their smartphones. CommCare is a survey platform designed specifically for mobile devices. The survey had a multiple-choice format with the questions regarding cocaine use in the past 7 days, triggers of cocaine use, protected and unprotected sexual intercourse following cocaine use, ART adherence in the past 7 days, and reasons for nonadherence. The research team sent an automated SMS text message reminder to complete the survey.

Monthly Research Interviews

A total of 4 monthly interviews took place at weeks 4, 8, 12, and 16, the latter of which was the postintervention interview. At the week 8 follow-up, participants also completed a set of questions on the acceptability and feasibility of the study components using the Technology Acceptance Model. The questions were about the ease of use and usefulness of smart pillboxes and smartphones, attitude toward adherence-related SMS text messages, and perceived credibility of those messages. For participants in arm D of the RCT, there was an additional instrument measuring the acceptability and receptivity of the social network intervention.

SMS Text Messages

SMS text message data were collected from arm D participants at weeks 4, 8, and 12. A total of 4 SMS text messages (1 SMS text message per week) sent by the social designee to the participant were downloaded from participants’ phones by a research staff member. These messages were then saved in a secure and password-protected drive at X University’s Health Insurance Portability and Accountability Act–compliant server.

Study Completion and Device Recycling

At the week 16 interview, study participants were asked to return their pillbox and their smartphone. Incentives were built in for device returns (Table 1). The study team disinfected the phones and boxes and applied a factory reset to remove all the saved information from the prior user.


All participants were given payments in the form of gift cards for their time and participation, following the guidelines of our IRB. Participants were paid for monthly interviews, answering the weekly smartphone survey, and returning the smart pillboxes and smartphones at the end of the study. Refer to Table 1 for a description of the payments.

Analytical Plan

Primary Outcomes

The primary outcomes of this study were ART adherence and persistence, measured as a composite score ranging from 0% to 100%, and as the number of continuous days of taking ART without a 7-day (or 3-day) gap [66,67], respectively.

Secondary Outcomes

Outcome analyses will be conducted for (1) drug outcomes (cocaine use), (2) HIV treatment outcomes (VL<20 copies/mL; change in CD4+ T lymphocytes), and (3) ART-related information, motivation, and behavioral skills constructs. The general framework for discerning differences between study conditions on the main outcome variables will be to perform a 4 (intervention condition)×3 (repeated follow-up assessments) mixed design analysis by entering preintervention scores as covariates and additional theoretically and empirically relevant variables (such as baseline mental status). For variables that are approximately normally distributed, we will use multivariate analyses of covariance, with significant multivariate tests with Bonferroni correction followed by subsequent analyses of covariance. Variables that violate distributional assumptions of normality will be analyzed using generalized linear model procedures, which enable the use of nonparametric functions such as non–Gaussian error models. On the basis of a review of the literature, the generalized linear model is the most appropriate approach for variables with strong positive skew, such as adherence [63,68].

Ethical Oversight

The IRB at Yale University reviewed and approved all study procedures (IRB:1508016342). Additional protections were provided by the National Institute of Health, which issued a certificate of confidentiality.

Sample Size

On the basis of a review of pilot studies that have ranged in sample size from 12 per group [69-71] to 9% to 50% [72,73] of a large RCT’s sample size (typically 20-30 per arm), we aimed to enroll 20 participants for each of the arms for a total of 80 participants. The final enrollment was 71 participants, of whom 57 (80%) completed the study. Figure 3 shows a complete CONSORT (Consolidated Standards of Reporting Trials) diagram. This was deemed appropriate for obtaining preliminary findings and optimizing a large-scale RCT for future studies [74].

Figure 3. Study flow of participants. ART: antiretroviral therapy; RA: research assistant.
View this figure


Participants were assigned to their arms after completing the baseline interview and before receiving ART medication, a pillbox, and a smartphone. Owing to the time required in acquiring permission to change pharmacy and receive medication for 4 weeks from the pharmacy, there was a time-lapse between baseline interview and actual start time of the study, termed preparation time (Table 1). During the preparation time, participants were randomized to their respective arms, the pillbox communicative features were set to the requirements of the study arms, and the smartphones were set up with the survey (CommCare) and the SMS text message (SMS Back-Up and Restore) apps.

We followed a systematic sequence of assigning participants to the study arms—the first participant to arm A, second to arm B, third to arm C, and fourth to arm D; this sequence was repeated until the desired sample size was reached. In addition, during the baseline interview, questions related to the NCI were administered using the Brief Inventory of Neurocognitive Impairment questionnaire [75,76]. As it is well-documented that individuals with underlying mental illness experience worse outcomes, we stratified participants based on the presence or absence of NCI using 155 out of 228 as the cutoff point [75,76]. Consecutive participants with high NCI were placed in different arms. The data are currently undergoing analysis, and the manuscript is being developed for publication in early 2022.

Study Challenges and Adaptations

Smart Pillbox

The research team chose the Health Insurance Portability and Accountability Act–compliant TowerView Health electronic pillbox because of its ability to record adherence data in real time and send automated reminders to a participant’s smartphone via SMS text messages. The smart features made this particular device more useful than frequently used pillboxes or bottles such as Glow Caps, MEMS Caps, or Wisepill in adherence research. As a start-up company based in Philadelphia, TowerView Health seemed to be the right choice for the study of the aforementioned reasons and for their active adherence research [77]. However, because of the unstable nature of start-up companies, this study experienced challenges worth noting.

First, the pillbox required a customized medication blister pack that would fit easily in the box. The local pharmacies at our study sites were not equipped with a printer to deliver these blister packs, creating the first barrier to adopting this pillbox. However, a pharmacy that had the capability to produce the customized blister packs required for the pillboxes was identified, located approximately 30 miles from the research site. The research team, along with TowerView Health, established a memorandum of understanding with the pharmacy that it would deliver participants’ HIV medication after receiving the prescription from the health care provider. This required participants to change their pharmacy for HIV medication for the duration of the study, which sometimes contributed to complications for the timely delivery of medication.

Second, some devices had technical issues similar to any other cellular device with wireless capabilities, such as loss of connection or cellular towers not responding. Our team worked diligently with the company’s engineering department and our participants to resolve such technical issues. One such incident, where pillboxes were not connected to the server because of negligence from the company’s side, led to data loss. Our research team was able to address this data loss by replacing faulty pillboxes with new devices and extending the study period for the affected participants to compensate for the data loss. We reached this solution by fully disclosing the errors and discussing the serious consequences on the study outcomes with our participants and the IRB. With IRB approval and full consent of the affected participants, we were able to resume regular intervention delivery and data collection.

Third, toward the end of Project SMART, when we were 8 participants away from reaching our target enrollment, TowerView Health informed us that their company had ceased to exist because of their inability to find investors. This was an unexpected challenge for our team; nevertheless, we addressed this issue by introducing a new method for the collection of adherence data for the last 8 participants. We extended the traditional and well-accepted pill count method of adherence to a picture format, whereby participants uploaded pictures of their ART pills in a blister pack on a weekly basis [78,79]. The collected pictures will allow us to count the pills taken in a given week and calculate the necessary adherence metrics based on prescribed dosages and time elapsed from the last refill [78-80]. Although we lost the real-time adherence data collected by the electronic pillboxes, the picture pill count provides an appropriate alternative to other more invasive and labor-intensive methods of measuring adherence.


Most participants were able to return their smartphones at the end of the study, which was then factory reset and reused for other study participants. However, some (9/57, 16%) participants reported losing or damaging their phones, such that phones had to be replaced. mHealth studies with incentives have reported a similar proportion of returned devices with return incentives. This is encouraging as the return rate among people with multiple challenges because of cocaine use is comparable with that of other populations.

The use of prepared Samsung phones for the participants provided numerous lessons for future digital health interventions. Each participant phone needed 2 separate apps (CommCare and SMS Back-Up and Restore) downloaded from the Google Play Store installed before participant enrollment. To ensure the consistency of the mobile phone hardware and software provided to each participant, the research team needed to register a new Google account on each Samsung device for each new participant. To prepare batches of phones with separate Google accounts, the research team needed to validate each new account by receiving an SMS text message on their personal phones. At the beginning of the study, Google account permissions allowed a single phone number to validate numerous new accounts. However, midway through the study, Google enacted more stringent account validation protocols, and the research team could not set up numerous accounts using 1 phone number. Thus, the creation of Google accounts to set up participant devices became a significant bottleneck for the project. The research team overcame this problem by using multiple phones to set up the accounts.

Time Length for Cocaine Use

One of the initial inclusion criteria was cocaine use in the past 30 days. This criterion was limiting, as most of the eligible participants reported using it infrequently or trying to quit. Hence, to address low recruitment, the study was adapted to first open up to people with HIV who had used cocaine in the past 60 days and then to people who had ever used to reach the target sample size, which was approved by the IRB.

Demands of the Study

Project SMART was conducted among people with HIV who were active cocaine users. Although most participants were able to adhere to the study protocol of using the electronic pillbox, using their smartphones to complete weekly surveys, and completing in-person interviews at the study site, almost one-fifth were unable to complete the study (14/71, 20%). The reasons for dropping out of the study were (1) comorbidities such as cancer and heart conditions, which required an extended period of hospitalization, affecting the ability to use the pillbox and participate in the study; (2) incarceration because of violent behavior while using illicit substances; (3) inability to answer phone calls and reminders for in-person interviews, leading to regularly missed in-person interviews; and (4) not being able to receive permission from nurses at the community homes to use medication adherence devices.

The RCT was a complex study that placed multiple demands on a population that already experienced constant instabilities, including insecurities related to health, social relations, or living conditions, because of drug use and other marginalized identities. The components of the study added to the burdens of living, including traveling to the research site, participating 4 times in an hour-long interview, learning how to use new technology, and completing multiple tasks throughout the period of the study. These demands created additional challenges for our participants, which resulted in a high dropout rate. Hence, in the future, similar mHealth interventions should aim to minimize these demands to reduce the burden on marginalized participants and maintain retention in the study.

This study lends itself to the emerging theoretical work on complex health intervention frameworks [81]. Complex health interventions are characterized by the complexity of the intervention at the following levels: the design of the intervention, the people involved (those who implement as well as those to receive it), at the organization and workforce, and at the outcome level. The 2 important constructs of this framework are the core function and forms of the intervention. Core function refers to the main purpose or goal that the intervention seeks to achieve, and form includes the strategies or activities of the intervention to achieve those goals [81]. The complex health intervention framework, which was based on research on patient-centered medical home interventions, argues that the core function of health interventions can be achieved through forms, which may vary greatly based on the cultural and contextual needs of the individuals that they serve. Hence, complex health interventions will change based on the context-specific needs of the patients or populations they serve as well as those who implement the intervention [81]. In this study, we acknowledge that the electronic pillbox was designed for individuals who have greater stability and fewer comorbidities than individuals with HIV who use cocaine. Therefore, future interventions should account for the complex and changing needs of this population, their health care providers (eg, nurses at community homes), and other characteristics in the early stages of the implementation and adapt the services to the changing demands of the intervention. Although this study was able to adjust the intervention in the latter part of the study by replacing the electronic pillbox with a manual blister package, earlier assessment and adaptation would have better facilitated the core function. In addition, the framework suggests that researchers should develop a function and form matrix in anticipation of challenges they may face in implementing complex interventions [81]. Hence, in the future, researchers should create a framework with the core functions of the interventions and possible adaptations of the forms to be prepared for the complexities that could arise at the various levels of the intervention.


This paper provides detailed information about an RCT involving an mHealth intervention to improve adherence to ART for people with HIV who use cocaine. The 4-armed RCT with a control group sought to assess the feasibility and acceptability of technology interventions such as smart pillboxes and smartphones among at-risk, vulnerable populations, for whom currently the only effective adherence intervention is directly administered ART, which is expensive and labor intensive. The pilot study described here presents the different components of a technology-based intervention that can provide options for scaled-back strategies, both in terms of cost and personnel, as well as highlighting some challenges and adaptations required to consider when planning for a large-scale study.


The authors would like to express their gratitude to the research and clinical staff of Yale Center for Community and Clinical Research Office, who navigated through the complexity of this study with positivity and sincerity. The authors would also like to acknowledge the administrative prowess of Dr Ruthanne Marcus, who ensured the functioning of Project SMART. Finally, the entire team would like to thank the research participants without whose cooperation this study would not have been possible.

Funding support for this work was provided by R21 (AK) and K01 (DG).

Conflicts of Interest

FLA declares the following conflicts of interest, which are unrelated to this research: research grant support to Yale University where he is listed as principal investigator or co-principal investigator (National Institute of Health, National Institute on Drug Abuse, National Institute on Alcohol Abuse and Alcoholism, Health Resources and Services Administration, Substance Abuse and Mental Health Services Administration, Gilead Foundation, Merck Pharmaceuticals); Speakers Bureau (Gilead Sciences, Practice Point Communications Simply Speaking HIV, Clinical Care Options); and Advisory Board Membership (Abbvie, Gilead Sciences, Alliance for Patient Access).

Multimedia Appendix 1

Peer review report from the Behavioral and Social Consequences of HIV/AIDS Study Section - Center for Scientific Review (National Institutes of Health).

PDF File (Adobe PDF File), 125 KB

  1. Hammond R, Harry TC. Efficacy of antiretroviral therapy in Africa: effect on immunological and virological outcome measures - a meta-analysis. Int J STD AIDS 2008 May 01;19(5):291-296. [CrossRef] [Medline]
  2. Bartlett J, Fath MJ, Demasi R, Hermes A, Quinn J, Mondou E, et al. An updated systematic overview of triple combination therapy in antiretroviral-naive HIV-infected adults. AIDS 2006 Oct 24;20(16):2051-2064. [CrossRef] [Medline]
  3. HIV treatment overview. HIV - U.S. Department of Health & Human Services.   URL: [accessed 2022-01-28]
  4. HIV treatment. Centers for Disease Control and Prevention. 2021.   URL: [accessed 2022-01-28]
  5. Velasco-Hernandez J, Gershengorn H, Blower S. Could widespread use of combination antiretroviral therapy eradicate HIV epidemics? Lancet Infect Dis 2002 Aug;2(8):487-493. [CrossRef]
  6. Montaner JS, Hogg R, Wood E, Kerr T, Tyndall M, Levy AR, et al. The case for expanding access to highly active antiretroviral therapy to curb the growth of the HIV epidemic. Lancet 2006 Aug;368(9534):531-536. [CrossRef]
  7. Lima V, Johnston K, Hogg R, Levy A, Harrigan P, Anema A, et al. Expanded access to highly active antiretroviral therapy: a potentially powerful strategy to curb the growth of the HIV epidemic. J Infect Dis 2008 Jul 01;198(1):59-67. [CrossRef] [Medline]
  8. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, HPTN 052 Study Team. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med 2011 Aug 11;365(6):493-505 [FREE Full text] [CrossRef] [Medline]
  9. Altice F, Evuarherhe O, Shina S, Carter G, Beaubrun AC. Adherence to HIV treatment regimens: systematic literature review and meta-analysis. Patient Prefer Adher 2019 Apr;Volume 13:475-490. [CrossRef]
  10. Ortego C, Huedo-Medina TB, Llorca J, Sevilla L, Santos P, Rodríguez E, et al. Adherence to highly active antiretroviral therapy (HAART): a meta-analysis. AIDS Behav 2011 Oct 6;15(7):1381-1396. [CrossRef] [Medline]
  11. Altice FL, Kamarulzaman A, Soriano VV, Schechter M, Friedland GH. Treatment of medical, psychiatric, and substance-use comorbidities in people infected with HIV who use drugs. Lancet 2010 Jul;376(9738):367-387. [CrossRef]
  12. Uthman OA, Okwundu C, Gbenga K, Volmink J, Dowdy D, Zumla A, et al. Optimal timing of antiretroviral therapy initiation for HIV-infected adults with newly diagnosed pulmonary tuberculosis. Ann Intern Med 2015 Jul 07;163(1):32-39. [CrossRef]
  13. Zhang S, Rust G, Cardarelli K, Felizzola J, Fransua M, Stringer HG. Adherence to highly active antiretroviral therapy impact on clinical and economic outcomes for Medicaid enrollees with human immunodeficiency virus and hepatitis C coinfection. AIDS Care 2015 Mar 27;27(7):829-835 [FREE Full text] [CrossRef] [Medline]
  14. Paschen-Wolff MM, Campbell AN, Tross S, Choo T, Pavlicova M, Jarlais DD. DSM-5 substance use disorder symptom clusters and HIV antiretroviral therapy (ART) adherence. AIDS Care 2020 May 04;32(5):645-650 [FREE Full text] [CrossRef] [Medline]
  15. Surie D, Borgdorff MW, Cain KP, Click ES, DeCock KM, Yuen CM. Assessing the impact of antiretroviral therapy on tuberculosis notification rates among people with HIV: a descriptive analysis of 23 countries in sub-Saharan Africa, 2010-2015. BMC Infect Dis 2018 Sep 26;18(1):481 [FREE Full text] [CrossRef] [Medline]
  16. Gardner EM, McLees MP, Steiner JF, Del Rio C, Burman WJ. The spectrum of engagement in HIV care and its relevance to test-and-treat strategies for prevention of HIV infection. Clin Infect Dis 2011 Mar 15;52(6):793-800 [FREE Full text] [CrossRef] [Medline]
  17. Westergaard RP, Ambrose BK, Mehta SH, Kirk GD. Provider and clinic-level correlates of deferring antiretroviral therapy for people who inject drugs: a survey of North American HIV providers. J Int AIDS Soc 2012 Feb 23;15(1):10 [FREE Full text] [CrossRef] [Medline]
  18. Lucas GM, Cheever LW, Chaisson RE, Moore RD. Detrimental effects of continued illicit drug use on the treatment of HIV-1 infection. J Acquir Immune Defic Syndr 2001;27(3):251-259. [CrossRef]
  19. Lewden C, Chene G, Morlat P, Raffi F, Dupon M, Dellamonica P, Agence Nationale de Recherches sur le Sida et les Hepatites Virales (ANRS) CO8 APROCO-COPILOTE Study Group, Agence Nationale de Recherches sur le Sida et les Hepatites Virales (ANRS) CO3 AQUITAINE Study Group. HIV-infected adults with a CD4 cell count greater than 500 cells/mm3 on long-term combination antiretroviral therapy reach same mortality rates as the general population. J Acquir Immune Defic Syndr 2007 Sep 01;46(1):72-77. [CrossRef] [Medline]
  20. Cunningham CO, Sohler NL, Berg KM, Shapiro S, Heller D. Type of substance use and access to HIV-related health care. AIDS Patient Care STDS 2006 Jun;20(6):399-407. [CrossRef] [Medline]
  21. Metsch LR, Pereyra M, Brewer TH. Use of HIV health care in HIV-seropositive crack cocaine smokers and other active drug users. J Subst Abuse 2001 Sep;13(1-2):155-167. [CrossRef]
  22. Sohler NL, Wong MD, Cunningham WE, Cabral H, Drainoni M, Cunningham CO. Type and pattern of illicit drug use and access to health care services for HIV-infected people. AIDS Patient Care STDS 2007 Jun;21 Suppl 1(s1):68-76. [CrossRef] [Medline]
  23. Arnsten JH, Demas PA, Grant RW, Gourevitch MN, Farzadegan H, Howard AA, et al. Impact of active drug use on antiretroviral therapy adherence and viral suppression in HIV-infected drug users. J Gen Intern Med 2002 May;17(5):377-381. [CrossRef]
  24. Ingersoll K. The impact of psychiatric symptoms, drug use, and medication regimen on non-adherence to HIV treatment. AIDS Care 2004 Feb 12;16(2):199-211. [CrossRef] [Medline]
  25. Sharpe TT, Lee LM, Nakashima AK, Elam-Evans LD, Fleming PL. Crack cocaine use and adherence to antiretroviral treatment among HIV-infected black women. J Community Health 2004 Apr;29(2):117-127. [CrossRef]
  26. Cook J, Burke-Miller JK, Cohen MH, Cook RL, Vlahov D, Wilson TE, et al. Crack cocaine, disease progression, and mortality in a multicenter cohort of HIV-1 positive women. AIDS 2008 Jul 11;22(11):1355-1363 [FREE Full text] [CrossRef] [Medline]
  27. Maru DS, Bruce RD, Walton M, Mezger JA, Springer SA, Shield D, et al. Initiation, adherence, and retention in a randomized controlled trial of directly administered antiretroviral therapy. AIDS Behav 2008 Mar 18;12(2):284-293 [FREE Full text] [CrossRef] [Medline]
  28. Thompson MA, Mugavero MJ, Amico KR, Cargill VA, Chang LW, Gross R, et al. Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: evidence-based recommendations from an International Association of Physicians in AIDS Care panel. Ann Intern Med 2012 Jun 05;156(11):817-833 [FREE Full text] [CrossRef] [Medline]
  29. Boyer EW, Smelson D, Fletcher R, Ziedonis D, Picard RW. Wireless technologies, ubiquitous computing and mobile health: application to drug abuse treatment and compliance with HIV therapies. J Med Toxicol 2010 Jun 10;6(2):212-216 [FREE Full text] [CrossRef] [Medline]
  30. Lester R, Gelmon L, Plummer FA. Cell phones: tightening the communication gap in resource-limited antiretroviral programmes? AIDS 2006 Nov 14;20(17):2242-2244. [CrossRef] [Medline]
  31. World Health Organization. Wireless Technologies, Ubiquitous Computing and Mobile Health: Application to Drug Abuse Treatment and Compliance With HIV Therapies. Geneva: World Health Organization; 2013. [CrossRef]
  32. Leach-Lemens C. Using mobile phones in HIV care and prevention. HIV & AIDS Treat Pract 2009;137:7 [FREE Full text]
  33. Hanauer DA, Wentzell K, Laffel N, Laffel LM. Computerized Automated Reminder Diabetes System (CARDS): e-mail and SMS cell phone text messaging reminders to support diabetes management. Diabetes Technol Ther 2009 Feb;11(2):99-106 [FREE Full text] [CrossRef] [Medline]
  34. Fjeldsoe BS, Marshall AL, Miller YD. Behavior change interventions delivered by mobile telephone short-message service. Am J Prev Med 2009 Feb;36(2):165-173. [CrossRef] [Medline]
  35. Hoffman JA, Cunningham JR, Suleh AJ, Sundsmo A, Dekker D, Vago F, et al. Mobile direct observation treatment for tuberculosis patients: a technical feasibility pilot using mobile phones in Nairobi, Kenya. Am J Prev Med 2010 Jul;39(1):78-80. [CrossRef] [Medline]
  36. Asiimwe C, Gelvin D, Lee E, Ben Amor Y, Quinto E, Katureebe C, et al. Use of an innovative, affordable, and open-source short message service-based tool to monitor malaria in remote areas of Uganda. Am J Trop Med Hyg 2011 Jul;85(1):26-33 [FREE Full text] [CrossRef] [Medline]
  37. Ryan D, Pinnock H, Lee AJ, Tarassenko L, Pagliari C, Sheikh A, et al. The CYMPLA trial. Mobile phone-based structured intervention to achieve asthma control in patients with uncontrolled persistent asthma: a pragmatic randomised controlled trial. Prim Care Respir J 2009 Dec 25;18(4):343-345 [FREE Full text] [CrossRef] [Medline]
  38. Lester R, Kariri A. Mobilizing cell phones to improve antiretroviral adherence and follow-up in Kenya: a randomized controlled trial in progress. WHO Essential Medicines Monitor. 2009.   URL: [accessed 2022-01-28]
  39. Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 2010 Nov;376(9755):1838-1845. [CrossRef]
  40. Harris LT, Lehavot K, Huh D, Yard S, Andrasik MP, Dunbar PJ, et al. Two-way text messaging for health behavior change among human immunodeficiency virus-positive individuals. Telemed J E Health 2010 Dec;16(10):1024-1029 [FREE Full text] [CrossRef] [Medline]
  41. Kazemi DM, Borsari B, Levine MJ, Li S, Lamberson KA, Matta LA. A systematic review of the mHealth interventions to prevent alcohol and substance abuse. J Health Commun 2017 May 10;22(5):413-432 [FREE Full text] [CrossRef] [Medline]
  42. Ingersoll KS, Dillingham RA, Hettema JE, Conaway M, Freeman J, Reynolds G, et al. Pilot RCT of bidirectional text messaging for ART adherence among nonurban substance users with HIV. Health Psychol 2015 Dec;34S(Suppl):1305-1315 [FREE Full text] [CrossRef] [Medline]
  43. Lewis MA, Uhrig JD, Bann CM, Harris JL, Furberg RD, Coomes C, et al. Tailored text messaging intervention for HIV adherence: a proof-of-concept study. Health Psychol 2013 Mar;32(3):248-253. [CrossRef] [Medline]
  44. Uhrig JD, Lewis MA, Bann CM, Harris JL, Furberg RD, Coomes CM, et al. Addressing HIV knowledge, risk reduction, social support, and patient involvement using SMS: results of a proof-of-concept study. J Health Commun 2012 May 02;17 Suppl 1(sup1):128-145. [CrossRef] [Medline]
  45. Horvath T, Azman H, Kennedy GE, Rutherford GW. Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection. Cochrane Database Syst Rev 2012 Mar 14(3):CD009756 [FREE Full text] [CrossRef] [Medline]
  46. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457 [FREE Full text] [CrossRef] [Medline]
  47. Shannon CE, Weaver W. The Mathematical Theory of Communication. Champaign, Illinois: University of Illinois Press; 1949.
  48. Sundar SS, Bellur S, Oh J, Jia H, Kim H. Theoretical importance of contingency in human-computer interaction. Commun Res 2014 May 22;43(5):595-625. [CrossRef]
  49. Ryan RM, Deci EL. Self-Determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness. New York: Guilford Press; 2017.
  50. Sundar S, Bellur S, Jia H. Motivational technologies: a theoretical framework for designing preventive health applications. In: Proceedings of the International Conference on Persuasive Technology. 2012 Presented at: International Conference on Persuasive Technology; June 6-8, 2012; Linköping, Sweden p. 112-122. [CrossRef]
  51. Dybul M, Fauci AS, Bartlett JG, Kaplan JE, Pau AK, Panel on Clinical Practices for the Treatment of HIV. Guidelines for using antiretroviral agents among HIV-infected adults and adolescents. Recommendations of the Panel on Clinical Practices for Treatment of HIV. MMWR Recomm Rep 2002 May 17;51(RR-7):1-55 [FREE Full text] [Medline]
  52. Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004 Sep;36(8):588-594 [FREE Full text] [Medline]
  53. Radloff LS. The CES-D Scale: a self report depression scale for research in the general population. Appl Psychol Measur 2016 Jul 26;1(3):385-401. [CrossRef]
  54. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med 1998 Sep 14;158(16):1789-1795. [CrossRef] [Medline]
  55. American Psychiatric Association. Diagnostic And Statistical Manual Of Mental Disorders. 5 Ed. Washington, DC: American Psychiatric Association Publishing; 2013.
  56. McLellan A, Kushner H, Metzger D, Peters R, Smith I, Grissom G, et al. The fifth edition of the addiction severity index. J Subst Abuse Treat 1992 Jun;9(3):199-213. [CrossRef]
  57. Rosen C, Henson BR, Finney JW, Moos RH. Consistency of self-administered and interview-based Addiction Severity Index composite scores. Addiction 2000 Mar;95(3):419-425. [CrossRef] [Medline]
  58. Robinson SM, Sobell LC, Sobell MB, Leo GI. Reliability of the Timeline Followback for cocaine, cannabis, and cigarette use. Psychol Addict Behav 2014 Mar;28(1):154-162. [CrossRef] [Medline]
  59. Amico KR, Toro-Alfonso J, Fisher JD. An empirical test of the information, motivation and behavioral skills model of antiretroviral therapy adherence. AIDS Care 2005 Aug 20;17(6):661-673. [CrossRef] [Medline]
  60. Fisher JD, Amico KR, Fisher WA, Cornman DH, Shuper PA, Trayling C, LifeWindows Team. Computer-based intervention in HIV clinical care setting improves antiretroviral adherence: the LifeWindows Project. AIDS Behav 2011 Nov 31;15(8):1635-1646. [CrossRef] [Medline]
  61. Amico K, Fisher WA, Cornman DH, Shuper PA, Redding CG, Konkle-Parker DJ, et al. Visual analog scale of ART adherence: association with 3-day self-report and adherence barriers. J Acquir Immune Defic Syndr 2006 Aug 01;42(4):455-459. [CrossRef] [Medline]
  62. Wilson IB, Lee Y, Michaud J, Fowler FJ, Rogers WH. Validation of a new three-item self-report measure for medication adherence. AIDS Behav 2016 Nov 20;20(11):2700-2708 [FREE Full text] [CrossRef] [Medline]
  63. Neal DJ, Simons JS. Inference in regression models of heavily skewed alcohol use data: a comparison of ordinary least squares, generalized linear models, and bootstrap resampling. Psychol Addict Behav 2007 Dec;21(4):441-452. [CrossRef]
  64. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989 Sep;13(3):319. [CrossRef]
  65. Hass RG. Effects of source characteristics on cognitive responses on persuasion. In: Cognitive Responses in Persuasion. Hillsdale, N.J: Lawrence Erlbaum Associates; 1981.
  66. Ing EC, Bae JW, Maru DS, Altice FL. Medication persistence of HIV-infected drug users on directly administered antiretroviral therapy. AIDS Behav 2013 Jan 22;17(1):113-121. [CrossRef] [Medline]
  67. Bae J, Guyer W, Grimm K, Altice FL. Medication persistence in the treatment of HIV infection: a review of the literature and implications for future clinical care and research. AIDS 2011 Jan 28;25(3):279-290. [CrossRef] [Medline]
  68. Ng VK, Cribbie RA. Using the Gamma Generalized Linear Model for modeling continuous, skewed and heteroscedastic outcomes in psychology. Curr Psychol 2016 Jan 9;36(2):225-235. [CrossRef]
  69. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharmaceut Statist 2005 Oct;4(4):287-291. [CrossRef]
  70. Birkett MA, Day SJ. Internal pilot studies for estimating sample size. Stat Med 1994 Dec 15;13(23-24):2455-2463. [CrossRef] [Medline]
  71. Browne RH. On the use of a pilot sample for sample size determination. Stat Med 1995 Sep 15;14(17):1933-1940. [CrossRef] [Medline]
  72. Cocks K, Torgerson DJ. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol 2013 Feb;66(2):197-201. [CrossRef] [Medline]
  73. Wittes J, Brittain E. The role of internal pilot studies in increasing the efficiency of clinical trials. Stat Med 1990 Jan;9(1-2):65-71. [CrossRef] [Medline]
  74. Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res 2016 Jun 19;25(3):1057-1073 [FREE Full text] [CrossRef] [Medline]
  75. Stern Y, McDermott MP, Albert S, Palumbo D, Selnes OA, McArthur J, Dana Consortium on the Therapy of HIV-DementiaRelated Cognitive Disorders. Factors associated with incident human immunodeficiency virus-dementia. Arch Neurol 2001 Mar 01;58(3):473-479. [CrossRef] [Medline]
  76. Anand P, Springer SA, Copenhaver MM, Altice FL. Neurocognitive impairment and HIV risk factors: a reciprocal relationship. AIDS Behav 2010 Dec 16;14(6):1213-1226 [FREE Full text] [CrossRef] [Medline]
  77. George J. DreamIt Health company makes pill taking easier with help from big name partners. Philadelphia Business Journal. 2015.   URL: https:/​/www.​​philadelphia/​blog/​health-care/​2015/​05/​dreamit-health-company-makes-pill-taking-easier.​html [accessed 2022-01-28]
  78. Wu P, Johnson B, Nachega J, Wu B, Ordonez C, Hare A, et al. The combination of pill count and self-reported adherence is a strong predictor of first-line ART failure for adults in South Africa. Curr HIV Res 2014 Nov 21;12(5):366-375. [CrossRef] [Medline]
  79. Basu S, Garg S. Methods for estimating medication adherence to antiretroviral therapy: response to Mehta et al. (2016). J Family Med Prim Care 2017;6(1):171-172 [FREE Full text] [CrossRef] [Medline]
  80. Holstad MM, Higgins M, Bauman M, Farber EW, Waldrop-Valverde D, Okwandu O, et al. Picture pill count: an innovative, reliable, valid and feasible method to measure adherence to ART. AIDS Behav 2019 Aug 30;23(8):2210-2217. [CrossRef] [Medline]
  81. Jolles MP, Lengnick-Hall R, Mittman BS. Core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med 2019 Jun 8;34(6):1032-1038 [FREE Full text] [CrossRef] [Medline]

ART: antiretroviral therapy
CONSORT: Consolidated Standards of Reporting Trials
IRB: institutional review boardmHealth: mobile healthMTM: Motivation Technology Model
NCI: neurocognitive impairment
RCT: randomized controlled trial
VL: viral load

Edited by G Eysenbach; Peer reviewed by M Yablowitz, J Montoya; This paper was peer reviewed by the Behavioral and Social Consequences of HIV/AIDS Study Section - Center for Scientific Review (National Institutes of Health). See the Multimedia Appendix for the peer-review report; submitted 01.03.21; accepted 22.11.21; published 07.03.22


©Yerina S Ranjit, Archana Krishnan, Debarchana Ghosh, Claire Cravero, Xin Zhou, Frederick L Altice. Originally published in JMIR Research Protocols (, 07.03.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.