During the intervention period, pilot study participants received the enhanced version of the Blued app: Blued+. The installation was integrated with a regular Blued app update to avoid disrupting users. A notification about the update was sent to study participants. Once study participants agreed to update the Blued app on their smartphone, the update to the enhanced Blued+ version was automatic.

The theoretical foundation that structured the integration of the HIV prevention services into this app environment was the Extended, Unified Theory of Acceptance and Use of Technology (UTAUT2). UTAUT2 is a frequently used model for the development and assessment of technology-based health interventions, with over 20,000 citations and recent validation for mobile technology assessment in China [30,31]. Domains relevant to Blued+ were performance expectancy (PE), effort expectancy (EE), social influence (SI), and habit (H) (Table 1). Blued+ sought to provide users services in a manner they deem valuable (PE), to decrease expected effort expended by using functions and design layout consistent with the existing Blued app (EE), to include health-positive messages encouraging self-care in direct messaging for social influence (SI), and to use reminder systems to help users build HIV prevention habits (H).

“Video walk-throughs” of telehealth intervention apps have been previously suggested [25]. In Multimedia Appendix 2, we provided a video walk-through of the Blued+ app to facilitate a detailed understanding of how intervention features were offered to users. Screenshots of some pages from the Blued+ app, translated into English, were presented in Figure 2. Services in the HIV prevention intervention package included PrEP provision, in-home HIV tests and linkage to care, and prevention supplies such as condoms and lubricants. Additionally, participants received weekly health messages focused on PrEP, sexual health, and self-care. All these services were provided at no cost.

Health messages were adapted from a set of messages used in an app-based messaging and services intervention (M-cubed study) that was found to increase HIV testing and PrEP use among men in the United States [32]. The messages were initially translated into Chinese by staff in the research team, then reviewed and revised by staff at Blued to ensure their cultural and contextual appropriateness for Chinese men who have sex with men. Health messages in the study were sent at intervals of 1-3 weeks with a total of 29 messages sent out to each participant.

Topics included promotion of PrEP use (n=11), condom and lubricant use (n=6), HIV testing (n=6), engagement in health care (n=2), and health-positive messages encouraging self-care (n=3). PrEP use support messages included information on PrEP and PrEP medication, PrEP effectiveness, PrEP costs, side effects, dealing with PrEP stigma, and how to access PrEP. Condom use messages included traditional messages encouraging consistent condom use and encouraging condom use with lubricants for pleasure. HIV testing messages included information about rapid testing, where to get tested, and encouraging regular testing every 6 months. Messages about engagement in health care encouraged users to talk to their health care providers and offered the in-app clinician counseling option. Positive self-support messages were adapted from a set of self-care and self-love messages provided by a community PrEP educator who focused on mental well-being and harm reduction behaviors. For example, one of the self-love messages was “To love oneself is the beginning of a lifelong romance” [Oscar Wilde]. A team-based translation approach was used to translate these messages into Chinese. Two bilingual staff from the research team and two bilingual local staff members worked together to ensure the messages were accurate and culturally appropriate. For most messages, a hyperlink with an appropriate destination was included within the message. For instance, a hyperlink within a PrEP use message “Could PrEP be right for you? Chat with our Blued+ provider” would direct users to the page where they could consult with a PrEP provider. A hyperlink within the message “Got a hot date? Don’t go unprepared. Get some free condoms and lube today!” would direct users to the page to order condoms and lubricants. A full version of all health messages with English translations can be found in Multimedia Appendix 3.

The clinical eligibility for PrEP use in China are (1) laboratory-confirmed HIV-negative, (2) no contraindications to emtricitabine or tenofovir disoproxil fumarate, (3) creatinine clearance of at least 60 mL/min (reported by a laboratory), (4) willingness to adhere to a PrEP regimen, and (5) willingness to attend PrEP maintenance visits every 3 months.

Participants interested in accessing PrEP could click on the link in PrEP use messages or access services directly in the Blued+ app that schedules a PrEP counseling session with an in-app clinician. After counseling, participants who decided to initiate PrEP could get lab tests for HIV, syphilis, hepatitis B surface antigen (HBsAg), and serum creatinine from any local tier-2 or tier-3 hospital. The age and weight of the participant were collected by self-report to calculate the creatinine clearance rate. Prior to the study launch, 8 staff from Blued (4 in each city) went to a hospital of their choice and got lab tests to trial the process and estimate its cost. The cost associated with completing these lab tests was estimated to range from US $27 (190 CNY) to US $50 (350 CNY). Reimbursement for tests was sent once participants uploaded the test results to Blued+ in the app. The test results were reviewed by a study clinician, and participants were prescribed PrEP in the platform if they were eligible for PrEP. After the prescription, research staff received a notification and shipped the medication within 1 week of an order, in batches of 90 pills to cover daily PrEP use for 3 months. Participants received tracking numbers once the medicine was shipped. Participants could refill every 3 months if they received appropriate laboratory testing specified in the Chinese guidelines [29]. PrEP medicines used in this study were pills containing 200 mg emtricitabine and 300 mg tenofovir disoproxil fumarate.

In Blued+, participants could order in-home HIV tests by clicking a link in HIV testing health messages or clicking a tab in the Blued+ app. The ordering page was designed to be easy to navigate to reduce users’ effort expectancy. Similar to the procedures for shipping PrEP medication, once the order was placed, the Blued+ pharmacy received a notification and prepared the shipment. Three types of in-home HIV test kits, using capillary blood, urine, or oral swabs, were offered to maximize participant choice. During the intervention period, each participant could order a package of 10 blood test kits, 1 urine test kit, or 1 oral test kit. The aim of offering 10 blood test kits (anticipated to be the most frequently selected option) was to provide enough test kits for the 12-month intervention period, allowing participants to test as frequently as they would like. Participants could upload photos of their test results to the app and receive free counseling (regarding the test kit and result interpretation) and linkage to care when appropriate from a study coordinator at Blued. A free message app account number of the in-app clinician was also provided to participants so that they could reach out for HIV test counseling and other related questions. Participants were able to use the Blued+ platform to voluntarily upload their HIV self-test results for further counseling. Experienced counselors would provide further explanation of the test result and referral to clinics for confirmatory tests, when necessary, through phone calls.

Three types of condoms with different thicknesses and brands (ultrathin vs standard and premium vs budget brand) and water-based and silicone-based condom-compatible lubricants were available for participants to order from the platform. The types of condoms and lubricants were selected based on our target users’ preferences as assessed during FGDs (see above). The ordering process for condoms and lubricants was similar to ordering HIV testing kits. When participants added condoms to their “cart” in Blued+, a note would pop up reminding them to add lubricants to use with condoms.

For condoms, each participant was offered 48 condoms per quarter. This number was based on 2-3 sexual activities per week, a common sex frequency for men who have sex with men in China [33]. The Blued+ platform allowed participants to choose luxury condoms of either premium brand A (ultrathin 20 condoms) or premium brand B (air-feel 20 condoms). This gave participants the opportunity to try different brands and thicknesses. Additionally, each participant could order 28 standard condoms from a budget brand.

It is recommended that 5-10 mL of condom-compatible lubricant be used with each anal sexual activity [34,35]. Participants were offered a total of 340 mL of lubricants per quarter, which included 1 container of water-based lubricant (100 mL) and 2 containers of silicone-based lubricant (120 mL each).

Although the standard of care has some similarities to the intervention, there were notable differences informed in part by the study’s qualitative assessments. The health page in the intervention was designed to be simpler and easier to navigate. Health messages in the intervention covered a wide range of HIV prevention–related topics and health-positive self-care messages, whereas the health messages in the standard of care only covered a narrower range of HIV prevention topics such as condom use and HIV testing, and there was no systematic delivery of these messages. For HIV prevention products, the health portal in Blued+ offered options of different condoms, lubricants, and in-home HIV test kits based on user preferences. For PrEP care access, the intervention offered in-app PrEP counseling and prescribing, and medication mailing when appropriate. All these services were provided at no cost to participants in the pilot.

Data were collected through Sojump, a widely used and secure online survey platform in China that offers survey design features including multiple question types and branching logic. The survey link was sent to participants through the direct message function in the Blued app. Blued user IDs, tied to unique phone numbers, were used to identify and exclude duplicate responses within the cohort survey as well as between the cohort survey and the repeated cross-sectional survey.

The enrollment survey included questions in the domains of (1) sociodemographic characteristics, (2) relationship status, (3) sexual history, (4) HIV and STI testing history, (5) PrEP knowledge or attitudes, (6) PrEP intention and use (including adherence), (7) HIV knowledge, (8) PrEP stigma, (9) mental health, and (10) prevention supplies order history. Sociodemographic characteristics questions were adapted from the annual American Men's Internet Survey (AMIS) [36]. HIV knowledge was assessed using an 8-item scale from the Sustainable Health Center Implementation PrEP Pilot (SHIPP) study [37]. PrEP stigma was measured using a previously validated scale [38]. Mental health was measured using the Patient Health Questionnaire for Depression and Anxiety (PHQ-4) [39]. The full enrollment survey can be found in Multimedia Appendix 4. Follow-up surveys include the domains of (1) relationship status, (2) sexual history, (3) HIV and STI testing history, (4) PrEP knowledge or attitudes, (5) PrEP intention and use, (6) prevention supplies order history, and (7) acceptability of Blued+. HIV knowledge, PrEP stigma, and mental health were only included in the month 6 and month 12 follow-up surveys because they were believed to be less labile. Acceptability of Blued+ was measured by the widely used System Usability Scale that assesses the subjective usability of products and systems [40,41]. The acceptability questions were only included in the month 12 follow-up survey. Questions in the pilot cohort survey and cross-sectional survey were identical. The average survey completion time ranged from 10 to 20 minutes across participants at each time point.

In addition to self-report data regarding PrEP adherence collected from the cohort survey, we conducted an exploratory assessment of the suitability and acceptability of biomarker data collection to measure drug levels in blood samples. Participants who initiated PrEP were asked at an interim visit between months 9 and 12 to visit a local laboratory for the collection of a blood specimen to create a dried blood spot to measure tenofovir diphosphate (TFV-DP) levels in plasma. For this procedure, participants were first asked to complete a supplemental survey and to receive an HIV screening test. The survey assessed PrEP pill usage in greater detail than in the cohort survey to evaluate adherence to their regimen of either daily use or on-demand use in the past 2 months. A trained phlebotomist at the designated local laboratory then collected finger-prick blood onto a paper card to create a dried blood spot card for future testing for TFV-DP. For daily pill use, TFV-DP levels provide an indicator of the mean number of days per week that PrEP is ingested over a period of approximately 1-2 months [42]. For daily users, a cut point of >700 fmol/punch TFV-DP indicates >4 doses/week and thus serves as a surrogate for substantial protection from HIV infection. For event-based PrEP users, adherence was determined by evaluating if the TFV-DP level matches with the self-report behavior data on sexual frequency.

The primary analysis will be focused on the feasibility and acceptability of the pilot intervention, assessed based on a series of benchmarks (Table 2); these benchmarks were established in 2018 when the research was first proposed. The intervention will be determined as feasible if a minimum proportion of participants receive services for home HIV testing (30%) and PrEP prescription (20%) at month 12. Minimum targets were determined based on modeling that indicate levels of service uptake required to substantially influence an HIV epidemic [43]. The intervention will be considered acceptable if it reaches a System Usability Scale rating benchmark score of “good” (≥71 out of 100). Analyses will be conducted using SAS 9.4 (SAS Institute) or similar statistical software packages.

This is a pilot study assessing feasibility and acceptability; therefore, our sample was not powered to detect significant differences in the intervention period compared to the standard-of-care period (before baseline) or to the set of three cross-sectional comparisons. A sample of 200 pilot study participants per intervention city was selected based on budget and time constraints. This sample size was also anticipated to provide estimates of key parameters with sufficient precision. For instance, a 95% CI for the prevalence of our binary benchmarks (home HIV test orders, condom package orders, PrEP medication orders) would have a half-width of 5.1 percentage points. This is calculated assuming an estimated population SD of 0.5 and Pr(w)=0.9 [45], where Pr(w) represents the probability that the width of a future CI will not exceed 95% and accounts for the fact that the SD is not known and will therefore lead to CIs that vary from sample to sample [46].

Additional analysis will include assessing preliminary efficacy with descriptive endpoints and measuring changes from baseline to intervention periods in HIV testing, condom use, and PrEP use. Descriptive analyses will include comparisons of the proportions of users reporting use of condoms, PrEP, and HIV testing in the pilot relative to the set of three comparison groups (Table 2). Longitudinal analyses will assess trends within the pilot cohort while accounting for autocorrelation, using models such as generalized estimating equations to measure predictors of the binary outcome of HIV testing, condom use, and PrEP uptake. Models will contain the 7 time periods with primary comparisons of 0-month (first baseline) measurement to 6-month (end wash-out baseline period) measurement and to 12-month (end intervention period) measurement. Time will be considered as a fixed effect to compare proportions tested across intervals, with a random intercept for individuals. TFV-DP data will be reported descriptively to provide context for the self-report PrEP use data.