Background: Daily and on-demand pre-exposure prophylaxis (PrEP) has been well demonstrated to effectively prevent HIV acquisition for men who have sex with men (MSM). More than half of the MSM PrEP users in Taiwan prefer on-demand PrEP; however, on-demand PrEP involves a complicated dosing regimen because it requires precoital and postcoital dosing and sex events are hard to anticipate. Although there are a growing number of mobile apps designed to improve access to HIV prevention services and HIV medication adherence, few mobile apps focus on adherence to PrEP or are designed to accommodate a complicated, on-demand PrEP dosing schedule.
Objective: The aim of this project is to evaluate the usability of a newly developed mobile app (UPrEPU) to assist MSM PrEP users to self-monitor their adherence to either daily or on-demand PrEP using a user-centered scheme.
Methods: This research will be conducted in 2 phases: app development and usability study. In the app development phase, we will first conduct formative research with end users and stakeholders through in-depth interviews; the results will provide PrEP users’ and PrEP navigators’ personas as material used in the app conceptualization stage. PrEP navigators are individuals in the health care system that help HIV-negative individuals who need assistance in accessing PrEP care. A low-fidelity prototype of the app feature will be formatted by applying a participatory design approach to engage PrEP users, designers, and app developers in the design process of the app. Then, a high-fidelity prototype of the app will be developed for the usability study and refined iteratively by the multidisciplinary team and new internal testers. Internal testers include the research team consisting of experts in public health, infectious disease, and industrial design and a close network of the research team that is taking PrEP. In the usability study phase, we will enroll 70 MSM PrEP users and follow them up for 4 months. Usability, feasibility, and effectiveness of adherence monitoring will be evaluated.
Results: Refinement of the UPrEPU app is currently ongoing. The usability study commenced in May 2020.
Conclusions: The UPrEPU app is one of the first apps designed to help MSM PrEP users to self-manage their PrEP schedule better regardless of dosing modes. With a design-thinking approach and adapting to the cultural context in Taiwan’s MSM population, this novel app will have substantial potential to be acceptable and feasible and contribute to the reduction of new HIV infections.
Trial Registration: ClinicalTrials.gov NCT04248790; https://clinicaltrials.gov/ct2/show/NCT04248790
International Registered Report Identifier (IRRID): PRR1-10.2196/20360
Men who have sex with men (MSM) in Taiwan have been disproportionally affected by the HIV epidemic since 2006 . Since 2015, approximately 2000 MSM per year have been newly diagnosed with HIV infections in Taiwan [ ]. HIV biomedical prevention interventions, such as treatment as prevention and pre-exposure prophylaxis (PrEP), have been increasingly used in the past decade. PrEP has been well demonstrated in several clinical trials, and open-label studies have been conducted to provide high-level protection for both heterosexual and MSM against acquiring HIV [ - ]. PrEP has thus been included globally as one of the major components of the HIV prevention toolbox, including in World Health Organization guidelines [ , ], in the United States [ ], and in the United Kingdom [ ].
Daily PrEP — taking tenofovir disoproxil fumarate/emtricitabine (TDF/FTC) at the same time every day — has been recommended across genders and at-risk populations. For MSM, there is an alternative PrEP-dosing regimen: on-demand PrEP, referring to a double dose of TDF/FTC taken 2-24 hours before each sexual intercourse, followed by 2 single doses of TDF/FTC taken 24 and 48 hours after the first drug intake . The IPERGAY study showed that on-demand PrEP in MSM was more efficacious than placebo administered to controls [ ].
In France and the Netherlands, 23%-55% of PrEP users reported using on-demand PrEP [, ]. Taiwan has a higher acceptance and willingness to use on-demand PrEP than daily PrEP [ ]. Since PrEP was introduced in Taiwan in 2016, MSM have enjoyed the flexibility of choosing between daily and an on-demand PrEP schedule based on the recommendations from Taiwan’s National PrEP Guidelines [ ]. A high proportion of Taiwanese MSM choose on-demand PrEP [ ]. Based on real-world data, 56.7% of MSM report the use of on-demand PrEP at baseline and in 49.7% of follow-up visits [ ]. PrEP users in Taiwan have various ways to access PrEP, including partially or fully subsidized by government-led demonstration projects, pay out of pocket for Truvada, or pay out of pocket for a generic brand of PrEP. Users are recommended to visit the clinics at least every 3 months for clinical measurements, laboratory measurements, and prescription refills based on Taiwan’s PrEP guidelines [ ]. MSM in Taiwan have also been uncertain about communicating with sexual partners on social apps regarding their use of PrEP, thereby indicating their struggles with the social stigma related to PrEP [ ].
The efficacy of PrEP relies heavily on adherence. The on-demand dosing regimen, however, can be complicated for most users because it requires that individuals anticipate sex in the next 2-24 hours and remember to take the pills before and after sex; therefore, adherence to on-demand PrEP can be hard to achieve.
Using mobile phone apps to seek sex partners has become common among MSM [- ]. Adopting mobile phone apps to improve access to HIV prevention services and adherence to antiretroviral therapy among people living with HIV has grown increasingly popular [ - ]; however, only a few studies have focused on self-monitoring of PrEP adherence among MSM or were designed for daily dosing PrEP only [ , ], without meeting the growing need of on-demand PrEP and the possibility of switching between the 2 dosing regimens.
In response to the complexity of different dosing regimens and potential switching between the 2 dosing regimens, we developed a self-monitoring tool (UPrEPU app) to improve PrEP adherence regardless of the MSM user’s choice of dosing regimen and lifestyle. The aim of this study is to use a user-centered approach to evaluate the usability of this app. Users were involved at the beginning stage and throughout the development of this app to facilitate identifying users’ needs, enhancing participants’ engagement, and improving the app’s usability .
This research protocol will be conducted in 2 phases: (1) app development phase, during which the app design and development process will be led by a multidisciplinary team with a user-centered design approach to ensure that the app will be acceptable and designed in line with PrEP users’ needs and context of use, and (2) usability study phase, during which we will evaluate the feasibility and usability of this app with a pilot sample.
Phase 1: App Development
The understanding stage will involve in-depth interviews with PrEP users and navigators as well as development of PrEP users’ personas and a journey map.
To identify PrEP users’ needs and the requirements of the mobile app–based PrEP adherence self-monitoring intervention, MSM who have experience using PrEP will be invited to participate in individual interviews. We will recruit a maximum of 30 participants from various routes, including the online LGBTQ community, PrEP clinics, and PrEP users’ online groups. Participants will undergo an in-depth online audio interview using a camera such as that available through Skype, Google Hangouts, or Line. The interview will explore the following: (1) PrEP users’ experience such as motivation, choices of PrEP-dosing regimens, and potential barriers to PrEP; (2) technologies used to self-manage adherence to PrEP, find sexual partners, and access PrEP-related information; (3) interactions between PrEP users and PrEP navigators.
PrEP navigators will also be invited to participate in individual interviews to understand their needs, usability requirements, and ideas related to this app. We will ask infectious disease physicians to refer PrEP navigators who are interested in participating in our study to contact us. The interviews will be online audio interviews conducted via Skype, Google Hangouts, or Line. During the interview, the following information will be obtained: (1) information related to the PrEP navigators’ routine work, (2) how they follow up with PrEP users and their dosing regimens, (3) tools they use to communicate with PrEP users at follow-ups, (4) challenges they encounter at follow-ups. All interviews will be audio-recorded for transcription and analysis.
Personas and users’ journey maps, a visualization of the process of an individual going through to reach a goal, are well-established user-centered approaches in user interface design to contribute to the creation of usable products, are able to provide an understanding of users for designers, and focus on users’ needs and pain points [- ]. Several PrEP users’ personas and journey maps will be developed by the research team based on results from the previous stage. It can help developers and users visualize typical days in the PrEP users’ lives and PrEP managers’ daily routines. Personas will include demographics; goals, needs, and motivations; frustrations; and technologies they have applied to the PrEP-related issue. The journey maps will depict the PrEP service from the perspectives of PrEP users and PrEP navigators. In journey maps, the goals, actions, touch points, thoughts, emotions, and opportunities for stages — before, during, and after providing or receiving PrEP services — will be described. Personas and journey maps will be used in the upcoming participatory design workshop as an illustrative scenario for participants to generate shared understandings and design ideas [ ].
During the app conceptualization stage, we will conduct a participatory design workshop (idea generation and prototyping) and develop a high-fidelity prototype.
The workshop aims to generate ideas for features of the UPrEPU app. We will apply a participatory design approach in the workshop to engage PrEP users, designers, and app developers in the app design process . Designers include researchers with expertise in either human-computer interactions or public health. The participatory design approach between users and developers can help facilitate engagement and communication that focus on the users’ needs and search for technological solutions by brainstorming, exploring, and iteratively and cooperatively evaluating design ideas [ , ]. The workshop will be held in 1 day for 6 hours and will be held only once. The workshop will be led by a facilitator. In order to better generate ideas for app features, personas and journey maps developed from Phase 1 will be used in the workshop as an illustrative scenario to facilitate discussion and understanding for what PrEP users and navigators may encounter in daily life. A low-fidelity paper prototype of this app will be developed through the evolution and agreement of participants in the workshop.
A high-fidelity prototype of the UPrEPU app will be developed for both iOS and Android operation systems. During the development, the multidisciplinary team will work cooperatively to refine the prototype. After a full prototype is completed, new internal testers will access all features of the UPrEPU app to identify any remaining glitches, bugs, or usability concerns. Afterwards, a polished version of a final app prototype will be developed based on the feedback from the internal testers for an external usability study. We expect that the functions of the app may include sex and medication diary, medication adherence reminders based on users’ expected sexual behaviors and dosing regimens, education, and resources, although the final prototype will be developed based on the participatory design workshop.
Phase 2: Usability Study
After the internal test for the UPrEPU app is completed and optimized through feedback from the internal testers, we will evaluate the feasibility and usability of the app through a pilot study. This pilot study will also examine the preliminary effectiveness of the app in monitoring adherence to PrEP. We will enroll 70 MSM in 2 urban cities in Taiwan. Participants will be followed for 4 months and will complete an assessment at the beginning and every month. If they decide to join the study, participants will be asked to wear a wearable device that collects physiological sensor data.
Eligible participants will be HIV-negative men who meet the following criteria: (1) 20 years of age or older; (2) reside in Taiwan and able to understand, read, and speak Mandarin Chinese; (3) tested negative for HIV 3 months prior to enrollment for current PrEP users; (4) have laboratory results eligible to initiate PrEP based on Taiwan’s PrEP guidelines ; (5) currently taking PrEP or willing to initiate PrEP after enrollment; (6) report having ≥4 episodes of anal intercourse with men in the previous 1 month; (7) own an Android or Apple operating system (iOS) smartphone and are willing to download the study app; and (8) willing to wear the device the research team provides during the study period. Participants with the following characteristics will be excluded from the study: (1) have abnormal kidney function (creatinine clearance rate ≤60 mL/minute) or (2) currently on medications that might interact with PrEP, such as drugs containing lamivudine, at the baseline assessment. Participants who experience HIV seroconversion during the study period will be terminated from participation and referred to HIV care services. Their data will be still included in the analysis.
We will enroll participants from 2 medical centers in 2 major cities of Taiwan. Both medical centers have provided PrEP since 2016. Potential study participants will be referred by physicians and case navigators from the clinics. Their eligibility will be assessed by a screening tool available online. Afterwards, trained research assistants will inform those who are eligible about the purpose of the study and the information that will be collected in the study. Individuals who express interest in this study will be required to provide signed informed consent. For those who do not consent to participate, the reasons for declining participation will be documented. Eligible participants will download the UPrEPU app on their phone and answer a baseline questionnaire in the app regarding sociodemographics, mental health scales, sexual behaviors, and PrEP use at the baseline visit at the study site. Participants will also receive a wearable device that collects physiological sensor data. Participants will be encouraged to use all app components over the next 4 months and wear the wearable devices at all times.
Participants will be followed up monthly for the 4 months at the 2 medical centers where they were recruited. Each monthly visit will include rapid testing for HIV antigen and antibodies, measuring TDF/FTC concentration, and completing a follow-up questionnaire on their mental health scales, sexual behaviors, and PrEP use. Depression and anxiety will be assessed using the Patient Health Questionnaire-9 and General Anxiety Disorder-7, respectively. We collect these 2 mental health indicators since studies have shown that mental health status may be associated with adherence to medications .
Kidney function and sexually transmitted infections will be assessed at the end of the follow-up. The system usability scale for the UPrEPU app will be assessed during the first follow-up visit and at the end of the study. A face-to-face, semistructured qualitative interview will be conducted at each visit to assess the feasibility of the app. It will focus on any technical challenges participants may have encountered, recommendations for app improvement, and their satisfaction and comfort in using this app. All interviews will take around 30 minutes and will be audio-recorded for transcription and analysis.
Users can report bugs in the system and provide feedback in the app and to the customer service portal at any time. The bugs will be reported directly to our technology partners, whereas the issues reported and feedback will be documented, reviewed by the investigator team, and used in the next iterative stages of the app adaptation.
The primary outcomes include usability, feasibility, and effectiveness of adherence monitoring. Usability of the app will be measured using the system usability scale, a 10-item, 5-point Likert scale, which gives a global view of subjective assessment of usability . A score above 50 out of 100 indicates acceptable [ ].
The primary feasibility outcomes include frequency of app logins, use of app components such as PrEP-taking and sexual behavior reports, and the length of time the app is used based on the app analytics. Descriptive statistics will be used, and confidence intervals will be calculated to evaluate the primary feasibility outcomes. Secondary feasibility outcomes of this app will be analyzing data using thematic analysis methods from qualitative interviews focusing on technical challenges and satisfaction of the app during the follow-up .
We will examine the app users’ effectiveness of adherence monitoring with this app by comparing the pill-taking reports in the PrEP-taking diary component of the app with the tenofovir and emtricitabine drug concentrations in dried blood spot samples in the previous 7 days before the follow-up day. We will use correlation analysis to examine the consistency between the drug concentration in the dried blood spot samples and self-reported PrEP diary in the app. Higher correlation reflects higher effectiveness of adherence monitoring.
Participants will receive incentives for completing each follow-up visit and in-depth interview. This includes US $20 cash for each follow-up visit and US $33 cash for twice completing the in-depth interviews. Participants whose cumulative frequency of logins and use of features reach 90% will receive an additional US $33 at the end of the study.
Trial Registration, Ethics, Consent, and Institutional Board Approval
The research and ethics presented in this study have been reviewed and approved by the Institutional Reviewer Board of National Cheng-Kung University in Tainan City, Taiwan (A-ER-107-337).
The usability study began enrollment in May 2020, and participants were followed up for 4 months. Study results will be available in 2021.
This paper provides an outline of a protocol for a research study that aims to evaluate the usability of a newly developed mobile app intervention using a user-centered scheme for self-monitoring adherence to either daily or on-demand PrEP.
The UPrEPU app is a novel smartphone app that accommodates the flexibility of switching dosing regimens and assists PrEP users adapt their dynamic lifestyles to better self-manage their PrEP taking, with the overarching goal of improving their adherence to PrEP. The UPrEPU app aims to be an acceptable, feasible, and capable tool for self-monitoring PrEP adherence in MSM populations.
Self-management of PrEP use and adherence to PrEP are particularly important during and after the COVID-19 pandemic. The COVID-19 pandemic may have had an impact on individuals’ ability to access HIV prevention medicine or treatment or may have limited access to health care services due to reduced hours ; in turn, MSM may be more likely to switch to an on-demand dosing regimen to cope with the possible shortage of pills.
A few mobile health interventions, with some of the most common features such as gamification, notifications, medication log, and education, target HIV prevention or PrEP adherence [, - ]. Our study is different from the rest of the mobile apps for PrEP adherence in its emphasis on incorporating sex behavior and medication logs to facilitate a more precise and personal reminder for whether the user has enough protection based on the correct dosage of PrEP. UPrEPU is the first app designed for both on-demand and daily dosing regimens for PrEP users, adopting a user-centered design. As the first app developed and operating in traditional Chinese focusing on PrEP adherence, we emphasized a participatory design throughout the development process to adapt to the cultural context in Taiwan’s MSM populations and that is likely in other Chinese-speaking communities around the world.
We anticipate a few limitations in the study protocol. First, we will only recruit individuals ≥20 years of age who are able to be consented legally. Therefore, our study results may not apply to younger MSM or adolescents since the usability or feasibility in such populations might be different. Also, the study might be more likely to recruit participants who have higher incomes and socioeconomic status based on the characteristics of the smartphone ownership eligibility criteria, which also affect the generalizability of the results. Second, the study requests that participants in phase 2 use the app and wear a portable device collecting physiological sensor data. Specifically, higher incentives will be received if participants use the app more frequently. We are likely to recruit highly motivated participants. The feasibility might be biased toward the higher end compared to when the app is released for download to the public in the future. Last, it is possible that providing a financial incentive for app use may impact participants’ behavior ; however, other factors such as social norms and intrinsic motivations may also influence participants’ behavior [ , ]. Our results from this pilot study will not completely reflect how people may use the app in real-world settings. Given that the effectiveness of PrEP depends on adherence and that switching between 2 PrEP dosing regimens is common and complex in real-world settings [ ], the importance of obtaining related information and knowledge from this research is significant. The user-centered design approach can help us address the concerns regarding sensitive, private issues raised by marginalized populations. Findings from this study can be further used to design adaptive interventions to address different needs in various contexts and behavioral patterns. This study protocol describes the details of conceptualization of mobile apps and usability evaluation. Future studies could benefit from this study protocol by using a user-centered design approach in complex behaviors such as different drug-dosing regimens contingent on lifestyle behaviors to other mobile health interventions.
The authors acknowledge and thank PrEP navigators, PrEP users, participants of participatory design workshops, and internal and external app testers for their contribution to the UPrEPU app development and refinement. We thank all the user interface design and user experience design team, technology partners, and staff involved in the development phase of the UPrEPU app. The study is funded by the Taiwan Ministry of Science and Technology (MOST 108-2636-B-006-004, 109-2636-B-006-004, 108-2221-E-003-026, 109-2221-E-003-010-MY3, 109-2223-E-007-001-MY3), the Taiwan Ministry of Health and Welfare, Centers for Disease Control (MOHW 109-CDC-C-114-000104), and the National Cheng-Kung University Hospital, Tainan, Taiwan (NCKUH-10902066). The MOST, Taiwan CDC, and the NCKUH had no role in the design of the study and collection, analysis, and interpretation of data or in writing the manuscript.
Conflicts of Interest
- Lo Y, Chuang J, Huang Y, Liu D, Chou J. GBD 2017 and HIV estimates for Taiwan. The Lancet HIV 2020 Apr;7(4):e224. [CrossRef] [Medline]
- Grant RM, Lama JR, Anderson PL, McMahan V, Liu AY, Vargas L, iPrEx Study Team. Preexposure chemoprophylaxis for HIV prevention in men who have sex with men. N Engl J Med 2010 Dec 30;363(27):2587-2599 [FREE Full text] [CrossRef] [Medline]
- McCormack S, Dunn DT, Desai M, Dolling DI, Gafos M, Gilson R, et al. Pre-exposure prophylaxis to prevent the acquisition of HIV-1 infection (PROUD): effectiveness results from the pilot phase of a pragmatic open-label randomised trial. The Lancet 2016 Jan 2;387(10013):53-60 [FREE Full text] [CrossRef] [Medline]
- Baeten JM, Donnell D, Ndase P, Mugo NR, Campbell JD, Wangisi J, Partners PrEP Study Team. Antiretroviral prophylaxis for HIV prevention in heterosexual men and women. N Engl J Med 2012 Aug 02;367(5):399-410 [FREE Full text] [CrossRef] [Medline]
- Thigpen MC, Kebaabetswe PM, Paxton LA, Smith DK, Rose CE, Segolodi TM, TDF2 Study Group. Antiretroviral preexposure prophylaxis for heterosexual HIV transmission in Botswana. N Engl J Med 2012 Aug 02;367(5):423-434. [CrossRef] [Medline]
- Choopanya K, Martin M, Suntharasamai P, Sangkum U, Mock PA, Leethochawalit M, et al. Antiretroviral prophylaxis for HIV infection in injecting drug users in Bangkok, Thailand (the Bangkok Tenofovir Study): a randomised, double-blind, placebo-controlled phase 3 trial. The Lancet 2013 Jun 15;381(9883):2083-2090. [CrossRef] [Medline]
- Molina J, Capitant C, Spire B, Pialoux G, Cotte L, Charreau I, ANRS IPERGAY Study Group. On-Demand Preexposure Prophylaxis in Men at High Risk for HIV-1 Infection. N Engl J Med 2015 Dec 03;373(23):2237-2246. [CrossRef] [Medline]
- Guidance on Pre-Exposure Oral Prophylaxis (PrEP) for Serodiscordant Couples, Men and Transgender Women Who Have Sex with Men at High Risk of HIV: Recommendations for Use in the Context of Demonstration Projects. World Health Organization. 2012. URL: http://www.who.int/hiv/pub/guidance_prep/en/ [accessed 2018-10-01]
- Policy brief: pre-exposure prophylaxis (PrEP): WHO expands recommendation on oral pre-exposure prophylaxis of HIV infection (PrEP). World Health Organization. 2015. URL: https://apps.who.int/iris/bitstream/handle/10665/197906/WHO_HIV_2015.48_eng.pdf [accessed 2020-08-03]
- Saag MS, Benson CA, Gandhi RT, Hoy JF, Landovitz RJ, Mugavero MJ, et al. Antiretroviral Drugs for Treatment and Prevention of HIV Infection in Adults: 2018 Recommendations of the International Antiviral Society-USA Panel. JAMA 2018 Jul 24;320(4):379-396 [FREE Full text] [CrossRef] [Medline]
- McCormack S, Fidler S, Fisher M, British HIV Association, British Association for Sexual Health and HIV. The British HIV Association/British Association for Sexual Health and HIV Position Statement on pre-exposure prophylaxis in the UK. Int J STD AIDS 2012 Jan;23(1):1-4. [CrossRef] [Medline]
- European Guidelines for treatment of HIV-positive adults in Europe. European AIDS Clinincal Society (EACS). 2020. URL: http://www.eacsociety.org/guidelines/eacs-guidelines/eacs-guidelines.html [accessed 2018-10-01]
- Hoornenborg E, Achterbergh RC, van der Loeff MFS, Davidovich U, van der Helm JJ, Hogewoning A, et al. Men who have sex with men more often chose daily than event-driven use of pre-exposure prophylaxis: baseline analysis of a demonstration study in Amsterdam. J Intern AIDS Soc 2018 Mar 30;21(3):e25105 [FREE Full text] [CrossRef] [Medline]
- Molina JM, Ghosn J, Béniguel L, Rojas-Castro D, Algarte-Genin M, Pialoux G, Prevenir ANRS study group. Incidence of HIV-infection in the ANRS Prévenir study in Paris region with daily or on-demand PrEP with TDF/FTC. 2018 Presented at: 22nd International AIDS Conference; July 23-27, 2018; Amsterdam, Netherlands.
- Lee Y, Chang S, Lin K, Chang L, Liu W, Wu C, et al. Awareness and willingness towards pre-exposure prophylaxis against HIV infection among individuals seeking voluntary counselling and testing for HIV in Taiwan: a cross-sectional questionnaire survey. BMJ Open 2017 Oct 16;7(10):e015142 [FREE Full text] [CrossRef] [Medline]
- Yen-Hao Chu I, Wen-Wei Ku S, Li C, Toh HS, Yang C, Wu K, et al. Taiwan guideline on oral pre-exposure prophylaxis for HIV prevention - 2018 update. J Microbiol Immunol Infect 2020 Feb;53(1):1-10 [FREE Full text] [CrossRef] [Medline]
- Wu H, Strong C, Ku S. Syphilis acquisition and dosing schedule for pre-exposure prophylaxis (PrEP) users in Taiwan PrEP demonstration project. 2018 Presented at: 22nd International AIDS Conference; July 23-27, 2018; Amsterdam, Netherlands.
- Wu H, Ku SW, Li C, Ko N, Yu T, Chung A, et al. Factors Associated with Preferred Pre-Exposure Prophylaxis Dosing Regimen Among Men Who Have Sex with Men in Real-World Settings: A Mixed-Effect Model Analysis. AIDS Behav 2020 Jul 08. [CrossRef] [Medline]
- Huang P, Wu H, Strong C, Jan F, Mao L, Ko N, et al. Unspeakable PrEP: a qualitative study of sexual communication, problematic integration, and uncertainty management among men who have sex with men in Taiwan. Journal of Applied Communication Research 2019 Nov 24;47(6):611-627. [CrossRef]
- Grosskopf NA, LeVasseur MT, Glaser DB. Use of the Internet and mobile-based "apps" for sex-seeking among men who have sex with men in New York City. Am J Mens Health 2014 Nov;8(6):510-520 [FREE Full text] [CrossRef] [Medline]
- Phillips G, Magnus M, Kuo I, Rawls A, Peterson J, Jia Y, et al. Use of geosocial networking (GSN) mobile phone applications to find men for sex by men who have sex with men (MSM) in Washington, DC. AIDS Behav 2014 Sep;18(9):1630-1637. [CrossRef] [Medline]
- Goedel WC, Duncan DT. Geosocial-Networking App Usage Patterns of Gay, Bisexual, and Other Men Who Have Sex With Men: Survey Among Users of Grindr, A Mobile Dating App. JMIR Public Health Surveill 2015;1(1):e4 [FREE Full text] [CrossRef] [Medline]
- Ko N, Tseng P, Huang Y, Chen Y, Hsu S. Seeking sex partners through the internet and mobile phone applications among men who have sex with men in Taiwan. AIDS Care 2016 Jul;28(7):927-931. [CrossRef] [Medline]
- Jenkins Hall W, Sun CJ, Tanner AE, Mann L, Stowers J, Rhodes SD. HIV-Prevention Opportunities With GPS-Based Social and Sexual Networking Applications for Men Who Have Sex With Men. AIDS Educ Prev 2017 Feb;29(1):38-48. [CrossRef] [Medline]
- Muessig KE, LeGrand S, Horvath KJ, Bauermeister JA, Hightow-Weidman LB. Recent mobile health interventions to support medication adherence among HIV-positive MSM. Curr Opin HIV AIDS 2017 Sep;12(5):432-441 [FREE Full text] [CrossRef] [Medline]
- Dillingham R, Ingersoll K, Flickinger TE, Waldman AL, Grabowski M, Laurence C, et al. PositiveLinks: A Mobile Health Intervention for Retention in HIV Care and Clinical Outcomes with 12-Month Follow-Up. AIDS Patient Care STDS 2018 Jun;32(6):241-250 [FREE Full text] [CrossRef] [Medline]
- Trang K, Sullivan PS, Hinton DE, Worthman CM, Le MG, Jovanovic T. Feasibility, acceptability, and design of a mobile health application for high-risk men who have sex with men in Hanoi, Vietnam. The Lancet Global Health 2020 Apr;8:S14. [CrossRef]
- LeGrand S, Knudtson K, Benkeser D, Muessig K, Mcgee A, Sullivan PS, et al. Testing the Efficacy of a Social Networking Gamification App to Improve Pre-Exposure Prophylaxis Adherence (P3: Prepared, Protected, emPowered): Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2018 Dec 18;7(12):e10448 [FREE Full text] [CrossRef] [Medline]
- Sharpe JD, Kamara MT. A systematic evaluation of mobile apps to improve the uptake of and adherence to HIV pre-exposure prophylaxis. Sex Health 2018 Nov;15(6):587-594. [CrossRef] [Medline]
- Goldenberg T, McDougal SJ, Sullivan PS, Stekler JD, Stephenson R. Preferences for a Mobile HIV Prevention App for Men Who Have Sex With Men. JMIR Mhealth Uhealth 2014 Oct 29;2(4):e47 [FREE Full text] [CrossRef] [Medline]
- Aquino Jr PT, Filgueiras LVL. User modeling with personas. 2005 Presented at: Latin American conference on Human-computer interaction; October 23-26, 2005; Cuernavaca, Mexico. [CrossRef]
- Floyd IR, Cameron Jones M, Twidale MB. Resolving Incommensurable Debates: A Preliminary Identification of Persona Kinds, Attributes, and Characteristics. Artifact 2008 Apr;2(1):12-26. [CrossRef]
- Matthews T, Judge T, Whittaker S. How do designers and user experience professionals actually perceive and use personas? 2012 Presented at: SIGCHI conference on human factors in computing systems; May 2012; Austin Texas. [CrossRef]
- Carroll JM. Encountering Others: Reciprocal Openings in Participatory Design and User-Centered Design. Human–Computer Interaction 2009 Nov 11;11(3):285-290. [CrossRef]
- Bjerkan J, Hedlund M, Hellesø R. Patients' contribution to the development of a web-based plan for integrated care - a participatory design study. Inform Health Soc Care 2015 Mar;40(2):167-184. [CrossRef] [Medline]
- Clemensen J, Rothmann MJ, Smith AC, Caffery LJ, Danbjorg DB. Participatory design methods in telemedicine research. J Telemed Telecare 2017 Oct 01;23(9):780-785. [CrossRef] [Medline]
- Kuhns LM, Hotton AL, Garofalo R, Muldoon AL, Jaffe K, Bouris A, et al. An Index of Multiple Psychosocial, Syndemic Conditions Is Associated with Antiretroviral Medication Adherence Among HIV-Positive Youth. AIDS Patient Care STDS 2016 Apr;30(4):185-192 [FREE Full text] [CrossRef] [Medline]
- Brooke J. SUS-A quick and dirty usability scale. 1996. URL: https://hell.meiert.org/core/pdf/sus.pdf [accessed 2020-11-26]
- Bangor A, Kortum PT, Miller JT. An Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction 2008 Jul 30;24(6):574-594. [CrossRef]
- Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006 Jan;3(2):77-101. [CrossRef]
- Emanuel EJ, Persad G, Upshur R, Thome B, Parker M, Glickman A, et al. Fair Allocation of Scarce Medical Resources in the Time of Covid-19. N Engl J Med 2020 May 21;382(21):2049-2055. [CrossRef] [Medline]
- Sullivan PS, Driggers R, Stekler JD, Siegler A, Goldenberg T, McDougal SJ, et al. Usability and Acceptability of a Mobile Comprehensive HIV Prevention App for Men Who Have Sex With Men: A Pilot Study. JMIR Mhealth Uhealth 2017 Mar 09;5(3):e26 [FREE Full text] [CrossRef] [Medline]
- Mitchell JT, LeGrand S, Hightow-Weidman LB, McKellar MS, Kashuba AD, Cottrell M, et al. Smartphone-Based Contingency Management Intervention to Improve Pre-Exposure Prophylaxis Adherence: Pilot Trial. JMIR Mhealth Uhealth 2018 Sep 10;6(9):e10456 [FREE Full text] [CrossRef] [Medline]
- Biello KB, Marrow E, Mimiaga MJ, Sullivan P, Hightow-Weidman L, Mayer KH. A Mobile-Based App (MyChoices) to Increase Uptake of HIV Testing and Pre-Exposure Prophylaxis by Young Men Who Have Sex With Men: Protocol for a Pilot Randomized Controlled Trial. JMIR Res Protoc 2019 Jan 07;8(1):e10694 [FREE Full text] [CrossRef] [Medline]
- Whiteley L, Mena L, Craker LK, Healy MG, Brown LK. Creating a Theoretically Grounded Gaming App to Increase Adherence to Pre-Exposure Prophylaxis: Lessons From the Development of the Viral Combat Mobile Phone Game. JMIR Serious Games 2019 Mar 27;7(1):e11861 [FREE Full text] [CrossRef] [Medline]
- Kane RL, Johnson PE, Town RJ, Butler M. A structured review of the effect of economic incentives on consumers' preventive behavior. Am J Prev Med 2004 Nov;27(4):327-352. [CrossRef] [Medline]
- Fehr E, Falk A. Psychological foundations of incentives. European Economic Review 2002 May;46(4-5):687-724. [CrossRef]
- Gneezy U, Meier S, Rey-Biel P. When and Why Incentives (Don't) Work to Modify Behavior. Journal of Economic Perspectives 2011 Nov 01;25(4):191-210. [CrossRef]
|MSM: men who have sex with men|
|PrEP: pre-exposure prophylaxis|
|SUS: system usability scale|
|TDF/FTC: tenofovir disoproxil fumarate/emtricitabine|
Edited by G Eysenbach; submitted 18.05.20; peer-reviewed by M Hurmuz, N Lorente, N Tarantino; comments to author 08.07.20; revised version received 07.08.20; accepted 10.11.20; published 01.12.20Copyright
©Carol Strong, Huei-Jiuan Wu, Yuan-Chi Tseng, Chien-Wen Yuan, Yi-Fang Yu, Jay Chiehen Liao, Yi-Wen Chen, Yi-Chen Hung, Chia-Wen Li, Po-Hsien Huang, Nai-Ying Ko, Stephane Wen-Wei Ku. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 01.12.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.