This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
HIV prevalence is high among young men who have sex with men (YMSM) and young transgender women who have sex with men (YTWSM), particularly among minorities. Despite its proven efficacy and safety, the uptake of and adherence to pre-exposure prophylaxis (PrEP) among YMSM and YTWSM is currently limited. To date, evidence-based interventions to promote and sustain PrEP adherence have been limited and not shown to be highly efficacious. Given the widespread adoption of smartphones, mobile apps can be utilized to increase PrEP adherence for many YMSM and YTWSM.
The study consists of a formative research phase to develop an app-based intervention, P3 (Prepared, Protected, emPowered), to increase PrEP adherence, and a randomized controlled trial (RCT) to test its efficacy. P3 is a mobile app built on an established health platform, which includes social networking and game-based components to encourage PrEP adherence among YMSM and YTWSM. P3+ includes all P3 features plus adherence counseling delivered via two-way text messaging (short message service, SMS) through the app.
The formative research phase includes usability testing to assess users’ comprehension of P3’s educational content, understanding and use of intervention features, and overall impressions of app functionality, followed by app refinements. A subsequent field trial will identify and resolve any remaining technical challenges. A three-arm RCT (P3, P3+, and standard of care) will then be conducted at 6 iTech subject recruitment venues to assess intervention efficacy and to conduct a comparison of costs to deliver the 2 intervention arms.
This is an ongoing research project with initial results from the formative work expected in 2020 and those from the RCT in 2021.
P3 aims to provide an engaging, interactive experience that is highly appealing for the target population, leveraging technology already heavily integrated into the lives of young people, and thus meeting users’ needs in a familiar, stimulating way. If efficacious, P3 could be a sustainable, easily disseminated, lower-cost PrEP intervention for YMSM and YTWSM. Further, the research aims to determine the processes that are essential to developing and implementing future health-related gamification interventions.
ClinicalTrials.gov NCT03320512; https://clinicaltrials.gov/ct2/show/NCT03320512 (Archived by WebCite at http://www.webcitation.org/74OVZkICl)
DERR1-10.2196/10448
Between 2002 and 2011, rates of new HIV infections declined by >30% overall in the United States but increased by 132% among young men who have sex with men (YMSM) aged 13-24 years [
The use of smartphones to deliver HIV prevention and care interventions has grown substantially in recent years due to: (1) wide-scale adoption of smartphone technology among high-risk groups, (2) the ability to deliver interventions in real time within risk contexts, and (3) low implementation costs [
A smartphone-delivered PrEP adherence intervention is well suited for YMSM and YTWSM, given that they have a high uptake and utilization of smartphone technology [
Social engagement and provision of support are powerful tools for behavior change, both of which can be achieved with smartphone technologies. Social networking sites often provide venues for health-related activities including searching for information and connecting with peers by sharing information, posting questions, and joining special-interest groups [
Although smartphone apps can deliver individualized, tailored content through complex algorithms, added individual adherence counseling delivered by trained counselors might provide a degree of personalization that could improve outcomes. The available literature suggests that some tools, including technology-based tools, may be more beneficial to patient adherence when combined with education or counseling [
P3 development is guided by evidence-based interventions and health behavior change theories including social cognitive theory (SCT), narrative communication (eg, storytelling), and the principles of persuasive technology [
The aims of this project are to conduct formative research that includes usability testing to assess users’ comprehension of P3’s educational content, understanding and use of intervention features, and overall impressions of app functionality followed by app refinements. A subsequent field trial will identify and resolve any remaining technical challenges with the app and allow for the finalization of study procedures, including biologic specimen collection. A three-arm randomized control trial (RCT) including P3, P3+, and standard of care (SOC) will then be conducted at 6 iTech subject recruitment venues (SRVs) to assess intervention efficacy and to conduct a comparison of costs to deliver the 2 intervention arms.
The research and ethics presented in this study have been reviewed and approved by the Institutional Review Board of the University of North Carolina at Chapel Hill, USA (17-9551). A Certificate of Confidentiality has been obtained from the National Institute of Child Health and Human Development, and a waiver of parental consent will be obtained for participants who are 15-17 years old. The study is also registered on ClinicalTrials.gov (NCT03320512).
Building off of the Empower platform developed by our technology partner, Ayogo, we created a prototype for P3 (
P3+ will include all features of the P3 app but also includes adherence counseling, via two-way text messaging (short message service, SMS) sessions with trained counselors based on the Next Step Counseling (NSC) adherence counseling curriculum [
NSC is an interactive, client-centered motivational intervention that was developed and implemented during the Pre-exposure Prophylaxis Initiative (iPrEx) trial and was found to be highly acceptable among men who have sex with men [
To ensure diverse representation of youth, we will test P3 in 6 US cities with iTech SRVs (Boston, MA; Philadelphia, PA; Chicago, IL; New York City, NY; Houston, TX; Atlanta, GA) [
Usability testing will be conducted to assess user comprehension of the educational content, understanding and use of intervention features, and overall impressions of the app functionality. Subsequently, field testing will be conducted to identify and resolve any remaining app technical challenges and allow for the finalization of study procedures, including biologic specimen collection.
Usability testing will be conducted at 2 iTech SRVs (Chicago, Illinois; Boston, Massachusetts) with 8-12 YMSM and YTWSM who are on PrEP, have used PrEP in the past, or are considering initiating PrEP. Participants will be English speaking, HIV-uninfected YMSM and YTWSM between the ages of 16 and 24 years, who are familiar with Android or iOS smartphones. In-person and Web-based methods will be used to recruit participants.
Usability testing will be conducted by members of the research team in accordance with the National Institutes of Health usability guidelines [
Field testing of P3 and P3+ will be done at 3 iTech SRVs (Philadelphia, Pennsylvania; Houston, Texas; Bronx, New York), with 15-24 YMSM and YTWSM who are starting PrEP or are nonadherent to PrEP. Eligible participants will be those who (1) are aged 16-24 years; (2) were assigned male sex at birth; (3) report sex with or intentions to have sex with men or TW; (4) have reliable daily access to an Android or iOS smartphone with a data plan; (5) are able to speak and read English; (6) are HIV-uninfected (confirmed by self-report at enrollment visit); and (7) initiated PrEP within the last 60 days and have an active PrEP prescription (prescription confirmed by study staff) OR are on PrEP >60 days but self-report adherence on average <6 pills per week over the past month and have an active PrEP prescription (prescription confirmed by study staff). Individuals who cannot be consented due to active substance use or psychological condition will be considered ineligible.
Privacy features: These include avatars, pseudonyms, confidential personal identification number to open the app, and app time-out after 5 minutes of inactivity.
App progression meter: Visual display of current app “level,” virtual bank account funds, and in-game currency, which is visible to other participants. Participants level up and earn in-game currency based on app use. Participants redeem currency to unlock narratives and other app features.
Social prompts: Daily discussions foster community and peer sharing, model successful behaviors, and provide reinforcement (eg, how do you remember your meds?). Notifications are sent when someone has commented or “liked” a particular post.
Medication reminder system: Personalized, discreet reminders and habit-building solutions promote pre-exposure prophylaxis (PrEP) adherence.
Tailored adherence strategies: The app uses information provided during the initial set up (eg, time of day PrEP is taken) to suggest adherence strategies (eg, take when I brush my teeth). Tailored feedback on new strategies is provided when adherence falters.
Refill reminders: Participants can create personalized reminders for PrEP refills.
Adherence dashboard (portal): Provides the study team with an easily interpretable, real-time overview of each user’s PrEP tracking and app use. Automated “canned” and tailored messages can be delivered to provide support and encouragement. A message library will be developed with input from Youth Advisory Boards at the participating iTech subject recruitment venues. All 6 of the recruitment sites have active Youth Advisory Boards that meet monthly to discuss and provide input on planned and ongoing studies [
Adherence Counseling: Next Step Counseling conducted in-app through text-messaging. Key features include reviewing participant adherence experiences, exploring adherence facilitators and barriers, identifying adherence needs and strategies to meet needs, and developing an adherence action plan [
Daily Quests: Actionable routine tasks help users set goals and build knowledge and skills.
Brain games: Quizzes and interactive exercises help users check knowledge and skills.
Multimedia library: Includes PrEP-related information and information about safer sex, relationships, and general health and wellness. Users are prompted with a reflection question after each article to apply the material to their lives. A visual shows progress toward completing each section.
“Choose-your-own adventure” narratives feature young men who have sex with men or young transwomen who have sex with men navigating common situations that impact PrEP care and adherence (eg, substance use, stigma). Playing through story paths allows users to face hard choices that impact health and practice problem solving.
Screenshot of the home screen.
Screenshot of the home screen: daily tasks.
Screenshot of unlocking the narrative collections.
Recruitment methods will be the same as those used for usability testing. Youth who meet initial eligibility criteria (1-5 above) and indicate that they meet the HIV-uninfected and PrEP prescription criteria (6-7 above) will be scheduled for an in-person enrollment visit. To confirm they meet the study eligibility criteria, they will be asked to report on their HIV status and provide evidence of an active PrEP prescription. Participants who report that they are HIV-positive will be linked to care at one of the clinical care sites and will not be enrolled. Youth confirmed as eligible will complete the informed consent process, provide baseline dried blood spots (DBS) to test for TFV-DP or emtricitabine-triphosphate (FTC-TP) levels, provide hair samples to test for FTC levels [
Study staff will help participants download the app on to their phones and provide a tour of the app to highlight features. The adherence counselor will also make initial contact with them during the enrollment visit to ensure that the features of P3+ are also understood. Participants will be compensated for their time at the end of the visit.
During the 1-month study, participants will be asked to use the P3 app daily. They will be asked to schedule and have at least 2 counseling “chat” check-ins with the adherence counselor and will be asked to engage in two-way texting with the adherence counselor to ensure the full technological functioning of the texting portal. Individuals will be instructed to contact study staff immediately to report difficulties with any app components or to report any problems with their phone or phone service.
At the end of the 1-month field trial, participants will return to the site to complete posttrial procedures. These procedures include DBS and hair collection and a posttrial CASI survey that assesses app, DBS, and hair collection feasibility and acceptability. A semistructured qualitative exit interview conducted using Health Insurance Portability and Accountability Act of 1996 (HIPAA)–compliant videoconferencing software will focus on how participants used the app over the 1-month field trial, any technical challenges they may have encountered, and whether and how they thought that use of the intervention could translate into behavior change. Participants will be compensated for participation at visit completion. A field-testing report will be compiled and reviewed by the investigator team and the technology partner. The report will include: (1) list of app functionality issues, including any issues with the texting portal; (2) recommended design improvements that will be used in the final iterative stages of app adaptation; and (3) recommended changes to DBS and hair collection procedures.
The second phase of the trial will consist of a three-arm RCT to test intervention efficacy among YMSM and YTWSM who are starting PrEP or are nonadherent to PrEP (
We will enroll up to 240 participants. Eligible participants will be those who: (1) are aged 16-24 years; (2) were assigned male sex at birth; (3) report sex with or intentions to have sex with men or TW; (4) have reliable daily access to an Android or iOS smartphone with a data plan; (5) are able to speak and read English; (6) are HIV-uninfected (confirmed by self-report at enrollment visit); and (7) are not currently on PrEP but plan to initiate in the next 7 days and have an active PrEP prescription (prescription confirmed by study staff) OR initiated PrEP within the last 30 days and have an active PrEP prescription (prescription confirmed by study staff) OR are on PrEP >30 days but self-report adherence on average <6 pills per week over the past month and have an active PrEP prescription (prescription confirmed by study staff). Individuals who participated in the field trial will not be eligible for participation in the RCT.
We will utilize 6 iTech SRVs to enroll YMSM and YTWSM. Multiple methods will be used for recruitment including in-person, venue-based (including recruitment from local PrEP clinics or providers), and Web-based recruitment mechanisms. For those initially eligible, contact information will be acquired, and an in-person enrollment visit will be scheduled. Youth who are confirmed as eligible for the study will be guided through an informed consent or assent process by research staff.
Randomized controlled trial study design. CASI: computer-assisted self-interviewing; DBS: dried blood spots; 3M: 3 month; 6M: 6 month; SOC: standard of care.
Retention procedures include sending emails, text reminders, and calendar invites to participants for follow-up survey and visit completion.
Those providing consent will also provide a baseline DBS sample and complete a baseline CASI survey. Participants will then be randomized in a 1:1:1 ratio into the SOC, P3, or P3+ condition based on a randomization sequence developed by the Analytic Core lead statistician and loaded into a HIPAA-compliant Web-based CASI platform form. The randomization sequence will be stratified by SRV.
Participants randomized to the P3 or P3+ arms will be guided by study staff through the app download process and given an app site tour to highlight features. Those in the P3+ arm will be provided with additional information about the adherence counseling portal.
Participants (n=80) will receive the P3 app on their Android or iOS smartphone device. P3 includes gamification features built into a technology platform that helps users to build self-efficacy and habituate positive health habits. The app includes social features, a PrEP-specific knowledge center, game elements, loss aversion mechanics, and a daily habit-building interface. They will have access to the app over a 6-month period but will only receive financial incentives for the first 3 months.
Participants (n=80) will receive the P3+ app on their Android or iOS smartphone device. This version of the app will include all features of the P3 app as well as two-way text messaging sessions with trained counselors based on the NSC adherence counseling curriculum [
SRV staff will inform participants (n=80) of SOC services that should be available to them at their prescribing PrEP provider’s practice and will provide them with written materials regarding PrEP adherence. According to Centers for Disease Control and Prevention guidelines [
At app intervention completion (month 3), participants will complete an in-person visit and will provide a DBS sample for TFV-DP and FTC-TP testing, hair samples for FTC testing, and complete a follow-up CASI survey. The assessment should occur as soon after the 3-month enrollment period as possible. At month 3, those participants in either the P3 or P3+ arm will be allowed to continue using the app but will not be provided incentives for usage nor have access to the adherence counselor (P3+ arm only). The 6-month assessments will include DBS and hair collection, and a follow-up CASI survey. These assessments should occur as soon after the 6-month enrollment period as possible.
A sample of 20 users (10 in the P3 and 10 in the P3+ arm) including both high and low users will complete a qualitative exit interview, conducted using HIPAA-compliant videoconferencing software, which will allow for an in-depth and nuanced understanding of the usage of P3 or P3+ over the 3-month intervention trial. High users of the P3 arm are defined as those who use the app on average >4 days during the intervention period; moderate to low users are those who use the app on average <3 days. High users of the P3+ arm include those who use the app >4 days on average and participate in >3 adherence counseling sessions; moderate to low users are those who use the app <3 days and participate in <2 counseling sessions. The Web-based interview will last 45-60 minutes, and audio only will be recorded using the software platform and transcribed by secure transcription service. Participants can opt to include video chat, but no video will be recorded. Interviews will be semistructured and focus on how participants used or did not use P3 or P3+ over the 3-month trial and what additional components (either technology-based on in-person) might be useful to further impact behavior change.
Participants will be compensated with US $50 upon completion of their enrollment visit, and US $50 each at the 3- and 6-month visits (US $150 total). Participants randomized to the P3 or P3+ arms have the possibility of earning an additional US $135 for app usage over the 3-month app intervention period. Compensation can be provided in person or mailed to subjects, depending on what is allowed at the specific SRV.
Baseline assessments will be conducted at the enrollment visit, with follow-up assessments conducted at 3 and 6 months.
The primary outcome is PrEP adherence, measured by the levels of TFV-DP and FTC-TP in DBS plasma, consistent with >4 doses per week at the 3- and 6-month follow-ups.
Secondary outcomes include self-reported retention in PrEP clinical care, PrEP persistence, sexual risk behaviors, sexually transmitted infection incidence, and SCT constructs including self-efficacy, outcome expectations, and self-regulation.
To compare costs between the intervention arms, we will collect information on (1) time spent by study staff for training and supervision of adherence counselor(s); (2) time participants spend in the adherence counseling sessions; and (3) costs associated with the delivery of both P3 and P3+, including the costs of providing financial incentives. The procedures we will use to quantify the resources required to deliver the interventions will be organized in standard expenditure categories: personnel, supplies, equipment, services, space, and overhead.
The primary analyses will assess the efficacy of the P3 or P3+ intervention at 3- and 6-month follow-ups. Efficacy is defined as the difference in the proportion of patients with protective levels of TFV-DP or FTC-TP when comparing the intervention arms to the SOC. The primary test of the efficacy of the intervention (P3 or P3+) will be of the null hypothesis that there is no difference in the proportion of patients with protective levels of TFV-DP and FTC-TP at either the 3- or 6-month visit against the 2-sided alternative of some difference in proportion. These tests will be conducted at the 0.05 level of significance, and we will present corresponding 95% CIs for the treatment effects.
For this analysis, we will use longitudinal targeted minimum loss-based estimation (TMLE) [
A detailed description and figure describing the power calculation are included in
We will conduct an analysis parallel to the primary analysis exactly as above but with the 2 P3 intervention arms separated (ie, P3 vs SOC, P3+ vs SOC). For each of the above analyses, we will additionally study the efficacy of the P3 or P3+ arms as measured by continuous levels of TFV-DP or FTC-TP in order to assess subclinical differences in adherence. Specifically, we will estimate the ratio of geometric mean levels in the intervention and SOC groups using longitudinal TMLE. We will also assess intervention efficacy on a panel of self-reported outcomes including retention in PrEP care, PrEP persistence, sexual risk, incident sexually transmitted infection, and SCT model constructs. For binary outcomes, we define intervention efficacy as a difference in proportions, while for continuous outcomes, we define efficacy as a difference in means.
Incremental cost effectiveness ratio between the 2 intervention arms will be defined as ΔC/ΔE, where ΔC denotes the estimated difference in mean costs of the intervention and ΔE reflects the estimated difference in mean effectiveness between the interventions. ΔC and ΔE will be estimated using longitudinal TMLE as with the primary analysis, and closed-form influence function-based Wald-style 95% CIs will be constructed for these estimates.
Usability testing will begin in winter 2017. Following app revisions based on usability test results, field testing will begin (anticipated start: spring 2018). Field testing and final app refinements will be completed over the course of 6 months. We anticipate completion of Phases 1 and 2 and app refinement by fall 2018. The RCT is anticipated to begin in winter 2018 and will be completed over the course of 2 years (completed in winter 2020), and it is anticipated that analyses and results will be ready for dissemination by May 2021.
P3 is a novel smartphone app intervention designed to improve adherence among YMSM and YTWSM initiating or nonadherent to PrEP. Phase 1 of the study (usability testing and field testing) will guide app development and refinement. In Phase 2, in addition to testing the efficacy of P3 alone, this study will test the efficacy of P3+ (medication adherence counseling delivered through the app). If successful, our interventions will improve PrEP adherence, retention in PrEP clinical care, and PrEP persistence in our subject population. Given the significant health sequelae associated with HIV infection and the paucity of scalable intervention programs for this population of young adults, the knowledge to be gained from this research is significant.
Sustainable, integrated HIV prevention interventions are critically needed to improve health outcomes and reduce HIV incidence, particularly among communities of YMSM and YTWSM. Interventions that take advantage of technology-based platforms have great potential to encourage health promotion behaviors [
P3 is the first intervention to include gamification to increase PrEP adherence among YMSM and YTWSM. Gamification uses game design components and principles of psychology outside of gaming contexts, thus providing opportunities for sophisticated engagement of participants in Web-based behavioral interventions [
The measurement of plasma for intracellular drug concentration will be the primary outcome of this study. This approach is considered the gold standard for determining if and how often an antiretroviral has been taken. However, there are limitations to this method. Namely, the collection of plasma is invasive, requires substantial processing to enable intracellular measurements, and has long turn-around times. Furthermore, the results represent only a short-term measure of drug-taking behavior. Given the rapid processing time of the proposed hair sampling procedure (matrix-assisted laser desorption electrospray ionization) which requires minimal sample processing, and the sample can be analyzed within 1 hour [
Anticipated limitations of this study include the use of self-reported secondary outcomes, which may be subject to social desirability bias. However, this is metered by using DBS to determine protective levels of TFV-DP and FTC-TP as a primary outcome. Contamination may also be an issue, as participants at each of the 6 SRVs can be randomized to any of the 3 arms, and participants from P3 or P3+ could show the program to those in the SOC arm. The possibility that providing financial incentives for app use may not be sustainable or impact behavior is a consideration. While economic approaches have shown mixed effects for changing behaviors, it is clear that some populations respond to financial incentives [
We anticipate that due to the developmentally and age-appropriate content featured in the P3 app, these interventions will improve PrEP adherence. Based on the results of this study, particularly the differences between the P3 and P3+ arms, modifications to the intervention can be made, and the most appropriate intervention components can be deployed on a larger scale.
Power calculation.
computer-assisted self-interviewing
dried blood spots
emtricitabine-triphosphate
Health Insurance Portability and Accountability Act of 1996
Next Step Counseling
Prepared, Protected, emPowered
pre-exposure prophylaxis
randomized controlled trial
social cognitive theory
standard of care
subject recruitment venue
short message service
tenofovir-diphosphate
targeted minimum loss-based estimation
transgender women
young men who have sex with men
young transgender women who have sex with men
This work was supported by the National Institutes of Health Adolescent Medicine Trials Network for HIV/AIDS Interventions (ATN 142; multiple principal investigators: SL & LH-W) as part of and the University of North Carolina/Emory Center for Innovative Technology (iTech; principal investigators: LH-W and PSS, 1U19HD089881). The content is solely the responsibility of the authors and does not represent the official views of the funding agencies. The authors would like to thank Adi Ferrara, MS, ELS for help in the preparation of this manuscript.
None declared.