Published on in Vol 7, No 7 (2018): July

Preprints (earlier versions) of this paper are available at, first published .
Reducing HIV Vulnerability Through a Multilevel Life Skills Intervention for Adolescent Men (The iREACH Project): Protocol for a Randomized Controlled Trial

Reducing HIV Vulnerability Through a Multilevel Life Skills Intervention for Adolescent Men (The iREACH Project): Protocol for a Randomized Controlled Trial

Reducing HIV Vulnerability Through a Multilevel Life Skills Intervention for Adolescent Men (The iREACH Project): Protocol for a Randomized Controlled Trial


1School of Nursing, University of Pennsylvania, Philadelphia, PA, United States

2Rollins School of Public Health, Emory University, Atlanta, GA, United States

3School of Nursing, University of Michigan, Ann Arbor, MI, United States

4School of Public Health, Division of Epidemiology and Community Health, University of Minnesota, Minneapolis, MN, United States

5School of Medicine, University of Michigan, Ann Arbor, MI, United States

6Milken Institute School of Public Health, George Washington University, Washington, DC, United States

7School of Medicine, University of North Carolina, Chapel Hill, NC, United States

Corresponding Author:

Jose Bauermeister, MPH, PhD

School of Nursing

University of Pennsylvania

418 Curie Boulevard

Suite 222L

Philadelphia, PA, 19104

United States

Phone: 1 2158989993


Background: Few HIV interventions have demonstrated efficacy in reducing HIV risk among adolescent men who have sex with men (AMSM), and fewer still have recognized the unique needs of AMSM based on race/ethnicity or geographical setting. Recognizing that youths’ HIV vulnerability is intricately tied to their development and social context, delivering life skills training during adolescence might delay the onset or reduce the consequences of risk factors for HIV acquisition and equip AMSM with the skills to navigate HIV prevention. This protocol describes the development and testing of iREACH, an online multilevel life skills intervention for AMSM.

Objective: This randomized controlled trial (RCT) aims to test the efficacy of an online-delivered life skills intervention, iREACH, on cognitive and behavioral HIV-related outcomes for AMSM.

Methods: iREACH is a prospective RCT of approximately 600 cisgender adolescent males aged 13 to 18 years who report same-sex attractions. The intervention will be tested with a racial/ethnically diverse sample (≥50% racial/ethnic minority) of AMSM living in four regions in the United States: (1) Chicago to Detroit, (2) Washington, DC to Atlanta, (3) San Francisco to San Diego, and (4) Memphis to New Orleans.

Results: This project is currently recruiting participants. Recruitment began in March 2018.

Conclusions: iREACH represents a significant innovation in the development and testing of a tailored life skills-focused intervention for AMSM, and has the potential to fill a significant gap in HIV prevention intervention programming and research for AMSM.

Registered Report Identifier: RR1-10.2196/10174

JMIR Res Protoc 2018;7(7):e10174




Between 2000 and 2010, the annual number of new HIV diagnoses among young men who have sex with men (YMSM; age 13-24 years) in the United States more than doubled [1]. In 2016, youth aged 13 to 24 years made up 21% of all new HIV diagnoses in the United States. Most (81%) of those new diagnoses occurred among young gay and bisexual men. Young black/African American and Hispanic/Latino gay and bisexual men were especially affected [1]. The HIV epidemic among YMSM is characterized by strong racial and ethnic disparities in HIV incidence. Data from the National HIV Behavioral Survey showed 26% of African American youth surveyed (aged 18-24 years) tested HIV positive compared to only 3% of white youth aged 18 to 24 years [2]. Although initially attributed to greater engagement in risk behaviors among racial/ethnic minorities, recent analyses [3-7] illustrate that racial/ethnic disparities in HIV are likely driven by social determinants of health, including access to health insurance and social network properties, and by more limited coverage of effective HIV treatment for those living with HIV. These findings underscore the importance of understanding and addressing the structural factors driving the HIV disparities faced by marginalized YMSM in the United States.

Few HIV interventions have demonstrated efficacy for reducing HIV risk among adolescent men who have sex with men (AMSM). We employ the term AMSM to refer to cisgender males aged 13 to 18 years who may express same-sex attractions and/or engage in same-sex behaviors, yet may or may not identify as gay, bisexual, queer, and/or questioning. AMSM also represent a younger age range than YMSM, which typically includes ages up to 24 years. Mustanski [8] notes that relative to HIV prevention research with adult populations, AMSM have received less research attention, with a paucity of longitudinal studies with follow-up periods of greater than 12 months or the testing of interventions that recognize the unique developmental context of AMSM. Recognizing that AMSM’s HIV vulnerability is intricately tied to their developmental stage and social context [9,10], research has posited that delivering life skills training during adolescence may delay the onset of or reduce the consequences of risk. The World Health Organization describes life skills as “the ability for adaptive and positive behavior that enables individuals to deal effectively with the demands and challenges of everyday life.” For AMSM, life skills training may include a set of resources tailored to their individual and social contexts, allowing them to learn about and manage their HIV risk. A review [11] concluded that behavioral interventions that teach life skills are highly effective for HIV risk reduction among adult men who have sex with men (MSM); however, there is limited evidence examining whether life skills training for AMSM is an efficacious HIV prevention strategy.

Life skills training programs may be suited to electronic delivery given the proven appeal of e-interventions among youth, the suitability for delivering tailored content specific to each user’s HIV risk behaviors and context, and the opportunity to reach AMSM residing in diverse geographic locations [3,4]. Furthermore, given that MSM often rely on online technologies to build their social and sexual networks, receive social support, and obtain relevant health information [12-14], an e-delivered platform may reach AMSM who might otherwise not be able to access LGBTQ+–friendly (LGBTQ+: lesbian, gay, bisexual, transgender, queer, and additional identities) resources and services (eg, rural AMSM). In this protocol paper, we describe our plans to test the efficacy of iREACH, a life skills training Web-based app designed for racially and ethnically diverse AMSM living in four diverse regions of the United States that include rural and urban settings.


The primary objective of this randomized controlled trial (RCT) is to test the efficacy of an e-delivered life skills intervention, iREACH, on cognitive and behavioral HIV-related outcomes for AMSM. We will recruit a large and diverse sample of AMSM (N=600; ≤50% non-Hispanic white) living in four regions disproportionately burdened by HIV prevalence across the United States. We have two secondary aims for this project: (1) to examine the differential efficacy of iREACH in shaping the psychosocial mediators (eg, personal competency) associated with our outcomes based on engagement with the intervention, and (2) to explore how socioecological determinants at the individual (eg, race/ethnicity, urbanity) and regional (eg, socioeconomic disadvantage, HIV prevalence) level are associated with intervention efficacy.

Trial Design

We will conduct a prospective RCT of 600 online-recruited cisgender AMSM (age 13-18 years) followed for 12 months with study assessments at each 3-month interval. A racially/ethnically diverse sample (at least 50% racial/ethnic minority) of AMSM living in four regions in the United States: (1) Chicago, IL to Detroit, MI; (2) Washington, DC to Atlanta, GA; (3) San Francisco, CA to San Diego, CA; and (4) Memphis, TN to New Orleans, LA. Regions were identified by inspection of HIV prevalence rate maps on Eligible counties are those that include the major interstate highway that connects the two anchor cities (ie, I-94 for Chicago to Detroit; I-95 for Washington, DC to Atlanta; I-5 for San Francisco to San Diego; and I-55 from Memphis to New Orleans). Each region includes urban, suburban, and rural counties, as classified by the 2006 National Center for Health Statistics urban-rural classification scheme for counties [15].

Eligibility Criteria

Eligible individuals must (1) have been assigned a male sex at birth and identify as male at the time of enrollment into the study (cisgender male), (2) be between the ages of 13 and 18 years (inclusive), (3) speak and read English, (4) report same-sex attractions and/or behaviors, (5) have access to the Internet, (6) live in one of the zip codes at least partially contained in the 109 counties included in the four regions selected for this trial, and (7) self-report as HIV-negative at time of enrollment.

Recruitment, Screening, Consent, and Enrollment

Potential participants will click on targeted banner advertisements (Figure 1) based on our eligibility criteria (eg, sex, age, region) placed on commonly used social media sites (ie, Facebook), which will direct them to a home page containing basic study information. We will also recruit through community events (eg, LGBTQ+ pride events). Interested individuals will consent to complete an online screener. As part of the screener, we will verify that the participant lives in one of the counties selected for this trial based on their reported zip code. Individuals who do not meet the eligibility criteria will see a screen that thanks them for their interest and provides HIV testing and counseling information and resources. We will not indicate why they were ineligible to avoid unintentional disclosure of their study involvement and to protect against fraud. Eligible individuals will be taken to the study consent form (a waiver of parental consent has been obtained for minor participants). AMSM who do not consent will be taken to a screen thanking them for their interest. AMSM who consent will submit a cell phone number and their cell phone carrier as part of the registration process, an email and/or short message service (SMS) text message containing a code will be immediately sent to verify the user. This process has been found to be acceptable among adolescents [16] and MSM [17,18] in previous studies. After verification, participants will be asked to provide their contact information, including an email address, a cell phone number, social media handles, and a mailing address, and will be asked to provide a nickname or name of choice to be referred to throughout the study. Participants will then be directed to the baseline survey, which is estimated to take approximately 30 minutes to complete.

Completed baseline survey data and participant information will be manually reviewed and checked for duplications or possible fraud by study staff members. Responses from the screening survey will be checked against responses in the baseline survey (eg, age) to ensure consistency, and IP addresses, email addresses, and phone numbers will be reviewed to check for multiple registrations. Proxy IP addresses will be flagged for further scrutiny, and reviews of the baseline survey data will be conducted to check for suspicious response patterns and realistic completion times. Within 48 hours, participants will receive their log-in credentials, their incentive payment for completion of the baseline survey, and will be assigned in a 1:1 ratio for the intervention or attention control condition using stratified randomization [19] by race and region. Screener and baseline data will be used by iREACH to inform personalized, tailored content for AMSM assigned to the intervention condition.

Intervention Content: iREACH

iREACH (Figure 2) is a tailored Web app intervention for AMSM aged 13 to 18 years. The intervention component of the app aims to facilitate participants lowering their vulnerability to HIV infection by (1) providing life skills educational modules tailored to their unique needs and characteristics, (2) setting goals and encouraging participants to use relevant services available locally to help achieve them, and (3) accessing LGBTQ+–welcoming resources across the life skills areas. Individuals in the experimental arm will have access to iREACH over the 12 months of the study. Within the Web app, they will learn life skills content through activity-based learning across 14 key life areas, set goals in those areas and monitor progress toward these goals, work on these goals using the peer mentor video chat feature, and locate nearby LGBTQ+–welcoming community resources to achieve these goals. We describe iREACH’s main components subsequently.

Figure 1. Examples of advertisements used to recruit racial/ethnically diverse adolescent men who have sex with men.
View this figure
Figure 2. Screen image of iREACH intervention app.
View this figure

The life skills educational content in iREACH covers 14 topics. As shown in Figure 3, content is written at the eighth grade literacy level and presented in an interactive format, which includes infographics, GIF (Graphics Interchange Format) images, interactive activities, and accordion (drop-down) headers to improve the ease of navigation and level of cognitive effort. Tailored (ie, personalized) content is derived from AMSM’s baseline sociodemographic characteristics (eg, age, region) and answers to follow-up surveys inform new personalized content. For example, although we will only enroll participants who identify as male at baseline, iREACH is responsive to the gender development of AMSM during the study: if a participant identifies as transgender or gender nonconforming during the intervention, they can edit their profile name and preferred pronouns. Similarly, if a participant reports being newly diagnosed with HIV during the study, iREACH is designed to provide HIV-specific content (eg, how to keep up with their treatment plan, how Ryan White funds can assist with medication, transportation, and social support).

Goal Setting

Participants will be able to set, monitor, and track their goals through the “My Goals” component of the Web app. The variety of possible goals is based on AMSM’s wide range of potential needs and experiences across the four regions. As shown in Figure 4, participants select the primary goal (eg, autonomy, competence, relatedness, and self-actualization) they wish to work on, followed by the relevant life skills domain (eg, work, relationships, and sex). Participants can then select from a list of prepopulated common goals or can create their own goal, and determine whether it is a short-term (now), medium-term (soon), or long-term (later) goal. Using a progress navigation bar, participants can track their progress on their goals (Figure 5), delete goals that are no longer relevant, and review goals marked as completed. Participants may also review and receive feedback on their goals through peer mentor sessions (described subsequently).

Local Resources

Participants who click on the “Locator” button will access a prepopulated list of national (eg, crisis hotlines) and local resources (eg, gay-straight alliances, HIV testing locations). We identified 1833 eligible resources across our four study regions; each resource was verified through mailers and phone calls using protocols adapted from HIV testing [20] and pre-exposure prophylaxis (PrEP) [21] locators. A random subsample (20%) of resources will be selected for review every 6 months and updated as necessary.

The Locator section filters resources for the participants' regions automatically and clusters the resources linked to each life skill area. Participants can further refine the resources based on their county, zip code, and/or current distance to the resource (as computed by their browser’s geolocation if activated). If a local resource does not exist in their area or if a participants moves out of the designed regions during the study, the Locator will provide national hotlines and websites. We detail the proportion of services identified, verified and included in our Locator in Table 1.

Peer Mentors

Building on the Web app’s life skills content and goal-setting structure, iREACH acknowledges that AMSM’s achievement of these goals can be enhanced by role playing and coaching. LGBTQ+ supportive peers may be limited in some communities; therefore, iREACH provides participants with access to trained peer mentors who can help AMSM personalize the life skills lessons, set new goals and/or support those already identified, and provide peer-to-peer social support. Participants can schedule and attend these peer mentor appointments (approximately 30 minutes, on average) through the “Sessions” button.

These sessions are housed within the VSee video chat telemedicine platform, accessible from their Web app. AMSM concerned about privacy may use VSee in four ways: (1) video in which both they and the peer mentor can be seen, (2) video in which only the peer mentor can be seen, (3) audio only, or (4) a text-only chat interface. Peer mentors (YMSM aged 18-29 years) will be trained to use motivational interviewing principles in their exchanges with AMSM, and supervised by study team members experienced in motivational interviewing and counseling (eg, psychologist, professional counselor). Peer mentors will use a social problem-solving approach to help participants set goals that are achievable and moderately challenging and to identify potential solutions to problems. Participants will be able to schedule a maximum of three sessions per week. This service will be available throughout the full year of their assignment to the intervention.

Figure 3. Screen images of iREACH intervention app illustrating age-appropriate literacy and interactive format. SO: significant other.
View this figure
Figure 4. Screen images of iREACH intervention app illustrating goal-setting activities.
View this figure
Figure 5. Screen image of iREACH intervention app illustrating goal progress tracking.
View this figure
Table 1. Number of local resources identified and included in the iREACH locator by regiona.
n (%)
Mental health
and substance
use, n (%)
n (%)
n (%)
Shelters and
n (%)
Intimate partner
violence resource,
n (%)
Total resources in
region (N=1933),
n (%)
Chicago, IL to Detroit, MI376 (47.1)472 (59.1)152 (19.0)31 (3.9)91 (11.4)103 (12.9)798 (43.5)
Washington DC to Atlanta, GA233 (43.7)317 (59.5)123 (23.1)16 (3.0)31 (5.8)37 (6.9)533 (29.1)
San Francisco, CA to San Diego, CA154 (47.0)239 (72.9)81 (24.7)11 (3.4)28 (8.5)32 (9.8)328 (17.9)
Memphis, TN to New Orleans, LA86 (50.6)82 (48.2)7 (4.1)6 (3.5)8 (4.7)9 (5.3)170 (9.3)
National hotlines0 (0.0)3 (75.1)3 (75.4)2 (50.4)1 (25.2)2 (50.3)4 (.22)

aResources often offer multiple types of services; percentages reflect the proportion of local resources offering a specific type of resource. Regions comprise all counties touching the major interstate corridor connecting the two anchor cities.

bLGBTQ+: lesbian, gay, bisexual, transgender, queer, and additional identities.

Additional Engagement

To promote on-going user engagement, intervention participants will also gain badges (Figure 6) to reinforce continued participation on the site [22,23]. Badges are unlocked as participants complete tasks on the Web app (eg, setting goals, reading content, scheduling peer mentor sessions), and continuously engage in the site (eg, returning to the Web app, using it on weekdays and/or weekends, logging on the site on different hours of the day). Participants also can access a message board where they can start and discuss topics with each other. Peer mentors will monitor these boards and facilitate discussions as needed.

Information-Only Attention Control Arm

Those AMSM assigned to the attention control arm (N=300) will only receive access to the “Locator” component of the intervention (Figure 7). Although the provision of a service locator is a form of an intervention, albeit weak, and it may decrease our ability to detect intervention effect, we felt that withholding referrals to services would be unethical given AMSM’s vulnerability to HIV and sexually transmitted infections (STIs). At the end of the RCT, participants in the attention control condition will be given full access to the iREACH intervention for 3 months.

Figure 6. Screen image of iREACH intervention app illustrating badge earning.
View this figure
Figure 7. Screen images of information-only attention control condition illustrating service locator information.
View this figure


Because our participants will be ages 13 to 18 years and many might not have begun sexual activity or used HIV prevention services, we set our primary outcomes as cognitive factors linked to the ability to use HIV prevention and behavioral intentions to use HIV prevention. Our secondary outcomes are behavioral factors. Cognitive factors can be measured for all AMSM, regardless of whether they have initiated sexual activity with partners, and are broadly grouped into (1) knowledge, (2) attitudes, (3) norms, (4) self-efficacy, and (5) perceived behavioral capacity. Behavioral intention outcomes include self-reported intentions to adopt HIV risk reduction strategies and HIV prevention services. We also include secondary outcomes that apply to the subset of AMSM who are sexually active, and include self-reported HIV risk-taking behavior (both sexual and drug/alcohol use) and the use of HIV prevention services (eg, testing) and prevention activities (eg, abstinence, condoms, PrEP). We detail study measures in Table 2.

Table 2. Measures planned for a randomized controlled trial of a life skills intervention for adolescent men who have sex with men in the United States.
DomainAssessment time
BaselineMonth 3aMonth 6Month 9Month 12Month 15b
Primary outcomes

HIV knowledge [24]XXXX

Condom use/communication efficacy [16]XXXX

Outcomes required for analysis


Patient provider communication around sexual orientation [25]XX

Stigma [26]XX

Resilience [27]XX

Psychological needs [28]XXXX

Future life goals [29]XX
Secondary outcomes

Internalized homonegativity [30]XXXX

PrEP use and willingness [31]XXXX

Sex behaviors [32,33]XXXX

Substance abuse [34,35]XXXX

Depression [36]XXXX

Anxiety [37]XXXX

Self-Esteem [38]XXXX

Peer influence [33]XX

Family support [39]XX

Discrimination [40]XX

Online behaviors [41,42]XX

Societal reaction to sexual orientation [43]XX

Ethnicity beliefs [44]XX

Relationship history [45]XXXX

Intervention acceptability [46,47]XX

aThe iREACH intervention question set will only be available to the intervention group for month 3.

bOnly participants initially assigned to the control condition will complete survey at month 15 after receiving access to the iREACH intervention.


Participants receive US $30 for the baseline and 12-month follow-up and US $25 for the 3-, 6-, and 9-month follow-ups. Control participants will receive US $30 for completing the 15-month follow-up. These incentives are small enough to avoid coercion, yet sufficiently substantial to promote retention.

Recruitment and Randomization

Sample Composition

We used Census information on population structure by race/ethnicity for each region to inform our recruitment goals. Briefly, using 2010 Census data, we calculated the number of men in the eligible age group and by race/ethnicity in each region (Table 3). We then used data from the Youth Behavioral Risk Factor Surveillance System [48] to estimate the proportion of AMSM in each age group who could meet eligibility criteria based on same-sex behavior, identity, or attraction for our study, and applied that age-specific proportion to the estimated total AMSM population by race in each region. This resulted in an estimated number of eligible AMSM per region and racial/ethnic group. Within each region, we calculated a proportional recruitment target under the assumption that the 150 AMSM in that region would be recruited proportional to their population prevalence.

Table 3. Estimated eligible adolescent men who have sex with men by race and ethnicity and proportional and planned study enrollment for each region.
Race/ethnicityRegiona, n

San Francisco/San DiegoAtlanta/Washington, DCDetroit/ChicagoMemphis/New OrleansTotalb

Black/African American643484213,48837389053343758056238155
American Indian or Alaska Native6641111681131121132801350
Native Hawaiian/Other Pacific Islander362011240130013101350

aRegions comprise all counties touching the major interstate corridor connecting the two anchor cities.

bTotals within race/ethnicity groups may not sum to total planned enrollment due to rounding.

cEst: estimate. Estimated total adolescent MSM (aged 13-18 years) in the region who have same-sex sexual experience, same-sex attraction, or are gay/bisexually identified.

dProp: proportion. Reflects the target recruitment for each racial/ethnic subgroup, assuming that enrollment is evenly distributed by region, and proportionally distributed by population prevalence of eligible AMSM within region.

eGoal: trial recruitment goal.

To recruit a sample that is diverse in terms of race/ethnicity, we will need to substantially oversample some racial/ethnic subgroups based on these calculations. To achieve a sample that is 50% or more nonwhite adolescent men, we will oversample Hispanic AMSM by a factor of 1.2, Asian MSM by a factor of 1.5, multiracial AMSM by a factor of 3.8, Native American AMSM by a factor of 25.6, and Native Hawaiian/Pacific Islander MSM by a factor of 100 (Table 4). Therefore, we have budgeted more resources for recruitment in these subgroups, and will direct substantial recruitment focus toward these groups through selection of race/ethnicity concordant models in advertisements and through identifying community events or support groups that are especially relevant to these subgroups [49].

Table 4. Overall recruitment goals of adolescent men who have sex with men (AMSM) based on allocation based on population prevalence, planned recruitment, and ratio of planned to proportional allocations for a randomized controlled trial of a life skills intervention.
Race/ethnicityProportionala, nPlannedb, nRelative recruitmentc, n
African American/black1411551.1
American Indian or Alaska Native25025.6d
Native Hawaiian/Other Pacific Islander150>100.0d

aProportional recruitment is determined by multiplying the estimated proportion of all AMSM in the four regions by the total study enrollment (N=600).

bPlanned enrollment is an empirically determined set of requirement targets.

cRelative recruitment is the ratio of the planned/proportional recruitment numbers, representing the relative under- or overrepresentation of the population in the planned sample, relative to their representation in the overall population of the study areas.

dThese relative recruitment values are calculated from proportional recruitment numbers with more precision than those displayed in the proportional column (eg, for Native Hawaiian/Other Pacific Islander, the proportional sample is calculated as 0.0016 person, but it is depicted as 1 since fractional recruitment of participants is not possible) . Therefore, the relative recruitment differs from the simple ratio of the planned to proportional for these groups.

These data also make clear that the distribution of participants will not be equal across the four geographic regions within racial/ethnic groups; therefore, we will stratify randomization within race and region. Based on estimated AMSM population sizes and historical data on sampling fractions, we anticipate more than 35% of participants will come from rural areas [50]. We recognize the challenge of recruiting early adolescents given that same-sex attraction and same-sex sexual experience become more common with age. Therefore, we realistically expect to include a greater number of participants in the older age range.


Recruitment will utilize both virtual and physical venues. Targeted advertisements, representing age and ethnic diversity, will be promoted on Facebook, Instagram, Snapchat, and organizational websites. In addition, supplemental advertising on other social media platforms (YouTube, Tumblr, Twitter, and Reddit) may be used to expand recruitment efforts. The social media campaigns will be monitored and adjusted throughout the recruitment phase. Across all regions, physical recruitment materials will be developed and distributed among organizations that serve and support LGBTQ+ youth (eg, homeless youth organizations, community LGBTQ+ centers, HIV resources). Physical recruitment materials will also be distributed during large community events, including pride festivals in the study regions.

Strategies to Ensure Sample Diversity

We will develop ads that promote AMSM’s interest by including diverse images of youth (Figure 1) and targeting specific sociodemographic characteristics and interests.

During the formative phase of the study, facilitated focus group discussions were conducted online to identify the most effective recruitment strategies to optimize diversity and to ensure that culturally appropriate strategies are employed to engage and retain enough Asian/Pacific Islander, Native American, and Alaskan Native participants. These include promoting the study to agencies, social media groups, and in social media forums highly utilized by youth in these specific ethnic groups, and incorporating recruitment messages that encourage community engagement. Materials will avoid identifying candidates as AMSM in the recruitment text to avoid unintended disclosure.


To be evaluated as potential “best-evidence” interventions through the Centers for Disease Controls and Prevention’s Prevention Synthesis Research activity [51], data must be available for at least a single follow-up time point for more than 70% of participants. As indicated subsequently, a detailed retention plan for the study will draw on previously successful retention protocols to achieve 80% or more retention at the first follow-up visit. We will use successful best practices from previous studies [17,49] to retain participants (eg, comprehensive locator information that includes participants’ cell phone number, email, Facebook and/or other social media usernames), while being sensitive to undue disclosure of AMSM participating in the study. In addition, we allow participants to specify the day of the week and time of day when they would like to receive electronic follow-up surveys [17].

We have a preplanned schedule of follow-up assessments utilizing a variety of methods. Initially, a respondent who does not respond to an electronic notification that a survey is due will automatically receive additional notifications 48 hours after the initial notification the survey is available. If the participant has still not completed the assessment 7 days after the third electronic notification, the retention activities are escalated to a research staff member who will use the participant’s contact preferences provided on registration (eg, by SMS text message). If still unresponsive, other available contact information (eg, phone call) will be used. Each contact is logged in an electronic retention system (Study Management and Retention Toolkit [SMART]) developed by Emory University [17,18,52]. The SMART system also maintains electronic lists of participants’ retention status, and automatically creates notification lists for retention staff to ensure that a systematic process is followed and carefully documented for retention.

Statistical Methods

The primary outcome analyses seek to test the efficacy of iREACH compared to the information-only control condition to improve cognitive factors and behavioral intentions (eg, comfort discussing sexuality, HIV prevention attitudes, norms, self-efficacy) and behavioral factors (eg, condom use, HIV testing, PrEP use). Psychosocial and demographic characteristics will be described for all participants and by intervention group. These will be compared between treatment groups using t tests or Wilcoxon rank sum tests for continuous variables and chi-square tests for categorical variables. We will determine from the analyses stratified by treatment arm whether or not a failure of randomization occurred. We will use the general framework of generalized linear mixed models to test for intervention effects over time. For some binary outcomes, such as HIV testing, we will perform an aggregate analysis after collapsing across the repeated measures using simple logistic regression comparing whether the probability of having tested at least once over the entire follow-up period is different across treatment groups. To ensure robustness, we will also apply an exchangeable working correlation structure to its corresponding generalized estimating equation model.

Sample Size and Power Calculations for Primary Analyses

Our expected sample size for analyses across both conditions is N=600 (intervention: n=300; control: n=300), assuming a 15% to 20% loss to follow-up. We estimated the minimum detectable effect sizes at 80% power, for comparisons of the two groups for the primary cognitive and behavioral intentions outcomes. For mean differences, our sample size calculations are based on a two-sample t test assuming equal variance using a two-sided significance of .05. At 80% power, we can detect a between-arm difference of d=0.22 at the final follow-up. For repeated measure analyses, assuming a within-person correlation of .25, we would be able to detect a difference of 0.08. For proportions, our sample size calculations are based on a two-sample test of proportions using a two-sided significance of .05. To have 80% power to compare the intervention to the control group, we require at least 500 participants to find a 12.5% difference between arms in cross-sectional analyses. Assuming within-person correlation of .25, we can detect an 8.8% difference.

Secondary Analysis

To examine the effects of our intervention on the psychosocial correlates (eg, personal competence), we will run a regression with only group assignment in the model. Among participants assigned to the intervention arm, we will test whether the intervention effects vary as a function of AMSM’s varying engagement with the intervention as measured by paradata metrics (eg, frequency of site log-ins, time spent on intervention components). Among participants assigned to the control arm, we will describe AMSM’s varying engagement with the resource guide as measured by paradata metrics (eg, frequency of site log-ins) and test whether engagement is associated with changes over time within the control arm. Finally, we will use multilevel models to examine how regional characteristics influence AMSM’s outcomes [53]. We will link individual and regional level data using participants’ residential address at enrollment. Our regional unit of analysis will be county-level to ensure enough participants per region and to avoid inadvertent identification of participants. Exploratory analyses will examine whether county-level characteristics (eg, economic disadvantage, racial composition, HIV prevalence) are associated with individual-level outcomes. Analyses will be adapted for binary, count, or continuous outcomes accordingly.

Intervention Exposure, Fidelity, and Dosage

We will measure intervention exposure using paradata from the intervention, including counts of user sessions, session lengths, pages visited, and functions utilized [54]. For the peer mentor component, we will record the number and duration of peer mentor sessions, as well as the domains covered in these sessions. This information will assist in examining whether intervention dosage influences the efficacy of the intervention, and inform wider implementation and scalability.

The protocol has been reviewed and approved by the University of Pennsylvania Institutional Review Board (825686) and is registered on (NCT03155841).

This project is currently recruiting participants. Recruitment began in March 2018.

Limitations and Anticipated Challenges

There are several potential challenges to the success of our trial. First, we propose to recruit a diverse (in terms of race/ethnicity, rurality, and socioeconomic status) sample of adolescents aged 13 to 18 years. This poses two potential challenges. First, we may experience more success in recruiting AMSM at the older ages of this range (eg, 16 years and older). To counteract this challenge, we have elicited feedback from AMSM during our formative planning and have planned for a broad range of social media outlets utilized commonly by these age groups in our recruitment. We will also leverage youth-focused social media outlets in each of the four sampled regions when available. Second, we recognize that AMSM living in the urban centers of each of our sampled regions and/or who are non-Hispanic white might be easier to recruit into the study. There is now strong evidence of high levels of Internet, personal computer, and mobile phone use across all race and ethnic groups [41], indicating that our modes of recruitment should not bias toward any particular demographic. We will ensure racial and ethnic representation in all advertising. Many social media outlets allow advertising targeted by zip codes, allowing us to target recruitment ads to all zip codes in each of our sampled regions. We will monitor the characteristics of our enrolled sample closely; if we are enrolling urban youth at a faster pace than rural youth then we will increase our advertising efforts in rural zip codes. Third, given time and resource constraints, our intervention focuses on English-speaking AMSM. Although this decision may exclude Spanish-only speakers, recent Census data suggests that 88% of Hispanic youth younger than 17 years in the United States are proficient in English [55]. If the intervention is found to be efficacious, we will explore opportunities to translate the intervention. Fourth, there is a possibility that peer mentor sessions may be popular among AMSM and might require us to increase the number of available slots per day. If so, we will adjust our schedule and hire/train new peer mentors to meet the demand. Finally, we are vigilant of events that may lead to unintended disclosure of sexuality, behavior, and research participation to the participant’s parents/guardians. To minimize the possibility of unintended disclosures, we have integrated widely used Web privacy features, such as password authentication, automatically logging users out after a period of inactivity, and not including sensitive details in communications sent to participants. We have also structured the intervention to give participants some control over their own privacy, such as allowing them to indicate preferred methods of receiving communication (ie SMS text message, email, or social media private message), and remind participants to be active in protecting their information. If a parent or guardian becomes aware of AMSM’s participation in the trial and chooses to reach out to the study team, a member of the research team will be available to speak with the parent/guardian; however, to protect AMSM’s privacy, we will not confirm their child’s participation in the trial or share data with parents.


As the numbers of HIV-infected AMSM continue to grow, innovative methods to scale up HIV prevention to AMSM are required. A life skills intervention delivered to a diverse sample of AMSM may reduce the existing HIV disparities across social categories (eg, race/ethnicity, rurality, socioeconomic status) by promoting equitable access to psychosocial and sexual health resources, and by reducing HIV-related risk factors that may be learned in adolescence and sustained across adulthood. The tools and framework used in this project may be directly applicable to US-based studies of this population. Along with promoting HIV risk reduction, our most significant contribution will be the development of an online intervention that builds on AMSM’s life skills and addresses the psychosocial needs during this developmental period. In fact, given the dearth of studies focused on AMSM, our trial may represent one of the first systematic evaluations of a psychosocial and behavioral intervention for this age group.


This study is supported by a U01 grant from the National Institute of Minority Health and Health Disparities (1U01MD011274-01), and the Centers for AIDS Research at the University of Pennsylvania (P30AI045008) and at Emory University (P30AI050409). Views expressed in this manuscript do not necessarily represent the views of the funding agency.

Conflicts of Interest

None declared.

  1. Centers for Disease Control and Prevention. Diagnoses of HIV Infection in the United States and Dependent Areas. Atlanta, GA: Centers for Disease Control and Prevention; 2016.
  2. Wejnert C, Le B, Rose CE, Oster AM, Smith AJ, Zhu J, Gabriela Paz-Bailey for the NHBS Study Group. HIV infection and awareness among men who have sex with men-20 cities, United States, 2008 and 2011. PLoS One 2013;8(10):e76878 [FREE Full text] [CrossRef] [Medline]
  3. Millett G, Peterson J, Flores S, Hart T, Jeffries W, Wilson P, et al. Comparisons of disparities and risks of HIV infection in black and other men who have sex with men in Canada, UK, and USA: a meta-analysis. Lancet 2012 Jul 28;380(9839):341-348. [CrossRef] [Medline]
  4. Ayala G, Bingham T, Kim J, Wheeler DP, Millett GA. Modeling the impact of social discrimination and financial hardship on the sexual risk of HIV among Latino and Black men who have sex with men. Am J Public Health 2012 May;102 Suppl 2:S242-S249. [CrossRef] [Medline]
  5. Millett GA, Ding H, Marks G, Jeffries WL, Bingham T, Lauby J, et al. Mistaken assumptions and missed opportunities: correlates of undiagnosed HIV infection among black and Latino men who have sex with men. J Acquir Immune Defic Syndr 2011 Sep 01;58(1):64-71. [CrossRef] [Medline]
  6. Oster AM, Wiegand RE, Sionean C, Miles IJ, Thomas PE, Melendez-Morales L, et al. Understanding disparities in HIV infection between black and white MSM in the United States. AIDS 2011 May 15;25(8):1103-1112. [CrossRef] [Medline]
  7. Millett G, Flores S, Peterson J, Bakeman R. Explaining disparities in HIV infection among black and white men who have sex with men: a meta-analysis of HIV risk behaviors. AIDS 2007 Oct 01;21(15):2083-2091. [CrossRef] [Medline]
  8. Mustanski B. Future directions in research on sexual minority adolescent mental, behavioral, and sexual health. J Clin Child Adolesc Psychol 2015;44(1):204-219 [FREE Full text] [CrossRef] [Medline]
  9. Harper G, Brodsky A, Bruce D. What's good about being gay?: Perspectives from youth. J LGBT Youth 2012;9(1):22-41 [FREE Full text] [CrossRef] [Medline]
  10. Harper GW, Riplinger AJ. HIV prevention interventions for adolescents and young adults: what about the needs of gay and bisexual males? AIDS Behav 2013 Mar;17(3):1082-1095. [CrossRef] [Medline]
  11. Johnson W, Diaz R, Flanders W, Goodman M, Hill A, Holtgrave D, et al. Behavioral interventions to reduce risk for sexual transmission of HIV among men who have sex with men. Cochrane Database Syst Rev 2008 Jul 16(3):CD001230. [CrossRef] [Medline]
  12. Allison S, Bauermeister JA, Bull S, Lightfoot M, Mustanski B, Shegog R, et al. The intersection of youth, technology, and new media with sexual health: moving the research agenda forward. J Adolesc Health 2012 Sep;51(3):207-212 [FREE Full text] [CrossRef] [Medline]
  13. Hightow-Weidman LB, Muessig KE, Bauermeister J, Zhang C, LeGrand S. Youth, technology, and HIV: recent advances and future directions. Curr HIV/AIDS Rep 2015 Dec;12(4):500-515 [FREE Full text] [CrossRef] [Medline]
  14. Pingel ES, Bauermeister JA, Johns MM, Eisenberg A, Leslie-Santana M. “A safe way to explore”: reframing risk on the Internet amidst young gay men's search for identity. J Adolesc Res 2013 Jul;28(4):453-478 [FREE Full text] [CrossRef] [Medline]
  15. Ingram D, Franco S. 2013 NCHS urban-rural classification scheme for counties. Vital Health Stat 2 2014 Apr(166):1-73 [FREE Full text] [Medline]
  16. Bauermeister JA, Pingel ES, Jadwin-Cakmak L, Harper GW, Horvath K, Weiss G, et al. Acceptability and preliminary efficacy of a tailored online HIV/STI testing intervention for young men who have sex with men: the Get Connected! program. AIDS Behav 2015 Oct;19(10):1860-1874 [FREE Full text] [CrossRef] [Medline]
  17. Khosropour CM, Johnson BA, Ricca AV, Sullivan PS. Enhancing retention of an Internet-based cohort study of men who have sex with men (MSM) via text messaging: randomized controlled trial. J Med Internet Res 2013 Aug 27;15(8):e194 [FREE Full text] [CrossRef] [Medline]
  18. Khosropour CM, Lake JG, Sullivan PS. Are MSM willing to SMS for HIV prevention? J Health Commun 2014;19(1):57-66. [CrossRef] [Medline]
  19. Kernan WN, Viscoli CM, Makuch RW, Brass LM, Horwitz RI. Stratified randomization for clinical trials. J Clin Epidemiol 1999 Jan;52(1):19-26. [Medline]
  20. Bauermeister JA, Pingel ES, Jadwin-Cakmak L, Meanley S, Alapati D, Moore M, et al. The use of mystery shopping for quality assurance evaluations of HIV/STI testing sites offering services to young gay and bisexual men. AIDS Behav 2015 Oct;19(10):1919-1927 [FREE Full text] [CrossRef] [Medline]
  21. Siegler AJ, Wirtz S, Weber S, Sullivan PS. Developing a Web-based geolocated directory of HIV pre-exposure prophylaxis-providing clinics: the PrEP Locator protocol and operating procedures. JMIR Public Health Surveill 2017 Sep 06;3(3):e58 [FREE Full text] [CrossRef] [Medline]
  22. Hightow-Weidman L, Muessig K, Bauermeister J, LeGrand S, Fiellin L. The future of digital games for HIV prevention and care. Curr Opin HIV AIDS 2017 Sep;12(5):501-507. [CrossRef] [Medline]
  23. Simoni JM, Kutner BA, Horvath KJ. Opportunities and challenges of digital technology for HIV treatment and prevention. Curr HIV/AIDS Rep 2015 Dec;12(4):437-440 [FREE Full text] [CrossRef] [Medline]
  24. Garofalo R, Gayles T, Bottone PD, Ryan D, Kuhns LM, Mustanski B. Racial/Ethnic difference in HIV-related knowledge among young men who have sex with men and their association with condom errors. Health Educ J 2015 Sep;74(5):518-530 [FREE Full text] [CrossRef] [Medline]
  25. Meanley S, Gale A, Harmell C, Jadwin-Cakmak L, Pingel E, Bauermeister JA. The role of provider interactions on comprehensive sexual healthcare among young men who have sex with men. AIDS Educ Prev 2015 Feb;27(1):15-26. [CrossRef] [Medline]
  26. Smolenski DJ, Diamond PM, Ross MW, Rosser BR. Revision, criterion validity, and multigroup assessment of the reactions to homosexuality scale. J Pers Assess 2010 Nov;92(6):568-576 [FREE Full text] [CrossRef] [Medline]
  27. Connor KM, Davidson JR. Development of a new resilience scale: the Connor-Davidson Resilience Scale (CD-RISC). Depress Anxiety 2003;18(2):76-82. [CrossRef] [Medline]
  28. Sheldon K, Hilpert J. The balanced measure of psychological needs (BMPN) scale: An alternative domain general measure of need satisfaction. Motiv Emot 2012 Mar 3;36(4):439-451. [CrossRef]
  29. Zimbardo P, Boyd J. Putting time in perspective: a valid, reliable individual-differences metric. J Pers Soc Psychol 1999;77(6):1271-1288. [CrossRef]
  30. Herek G, Gillis J, Cogan J. Internalized stigma among sexual minority adults: insights from a social psychological perspective. J Couns Psych 2009;56(1):32-43. [CrossRef]
  31. Sullivan P, Sineath C, Kahle E, Sanchez T. Awareness, willingness and use of oral PrEP among a national sample of US men who have sex with men. 2015 Jul Presented at: AIDSImpact; Jul 28-31, 2015; Amsterdam.
  32. Mellins CA, Elkington KS, Bauermeister JA, Brackis-Cott E, Dolezal C, McKay M, et al. Sexual and drug use behavior in perinatally HIV-infected youth: mental health and family influences. J Am Acad Child Adolesc Psychiatry 2009 Aug;48(8):810-819 [FREE Full text] [CrossRef] [Medline]
  33. Bauermeister JA, Elkington K, Brackis-Cott E, Dolezal C, Mellins CA. Sexual behavior and perceived peer norms: comparing perinatally HIV-infected and HIV-affected youth. J Youth Adolesc 2009 Sep;38(8):1110-1122 [FREE Full text] [CrossRef] [Medline]
  34. Bauermeister JA, Zimmerman MA, Johns MM, Glowacki P, Stoddard S, Volz E. Innovative recruitment using online networks: lessons learned from an online study of alcohol and other drug use utilizing a web-based, respondent-driven sampling (webRDS) strategy. J Stud Alcohol Drugs 2012 Sep;73(5):834-838 [FREE Full text] [Medline]
  35. Saunders JB, Aasland OG, Babor TF, de LFJR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption--II. Addiction 1993 Jun;88(6):791-804. [Medline]
  36. Radloff L. The CES-D Scale. Appl Psych Meas 2016 Jul 26;1(3):385-401. [CrossRef]
  37. Derogatis L, Melisaratos N. The Brief Symptom Inventory: an introductory report. Psychol Med 2009 Jul 9;13(03):595-605. [CrossRef]
  38. Rosenberg M. Society and the Adolescent Self-Image. Princeton, NJ: Princeton University Press; 2015.
  39. Elkington KS, Bauermeister JA, Zimmerman MA. Do parents and peers matter? A prospective socio-ecological examination of substance use and sexual risk among African American youth. J Adolesc 2011 Oct;34(5):1035-1047 [FREE Full text] [CrossRef] [Medline]
  40. Williams D, Mohammed S. Discrimination and racial disparities in health: evidence and needed research. J Behav Med 2009 Feb;32(1):20-47 [FREE Full text] [CrossRef] [Medline]
  41. Lenhart A. Pew Research Center. 2015 Apr 09. Teens, social media & technology overview 2015   URL: [accessed 2018-06-04] [WebCite Cache]
  42. Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov 14;8(4):e27 [FREE Full text] [CrossRef] [Medline]
  43. Herek G, Glunt E. Identity and community among gay and bisexual men in the AIDS era: preliminary findings from the Sacramento men's health study. In: Herek G, Greene B, editors. AIDS, Identity, and Community: The HIV Epidemic and Lesbians and Gay Men. Thousand Oaks, CA: SAGE Publications; 1995:55-84.
  44. Phinney J, Kohatsu E. Ethnic and racial identity development and mental health. In: Schulenberg J, Maggs JL, Hurrelmann K, editors. Health Risks and Developmental Transitions During Adolescence. New York: Cambridge University Press; 1997:420-443.
  45. Bauermeister JA, Johns MM, Sandfort TG, Eisenberg A, Grossman AH, D'Augelli AR. Relationship trajectories and psychological well-being among sexual minority youth. J Youth Adolesc 2010 Oct;39(10):1148-1163 [FREE Full text] [CrossRef] [Medline]
  46. DeLone W, McLean E. The DeLone and McLean model of information systems success: a ten-year update. J Manage Inform Syst 2003;19(4):9-30.
  47. DeLone W, McLean E. Information systems success: the quest for the dependent variable. Inform Syst Res 1992 Mar;3(1):60-95. [CrossRef]
  48. Centers for Disease ControlPrevention (CDC), Brener N, Kann L, Shanklin S, Kinchen S, Eaton DK, Centers for Disease Control and Prevention (CDC). Methodology of the Youth Risk Behavior Surveillance System--2013. MMWR Recomm Rep 2013 Mar 01;62(RR-1):1-20 [FREE Full text] [Medline]
  49. Sullivan PS, Khosropour CM, Luisi N, Amsden M, Coggia T, Wingood GM, et al. Bias in online recruitment and retention of racial and ethnic minority men who have sex with men. J Med Internet Res 2011 May 13;13(2):e38 [FREE Full text] [CrossRef] [Medline]
  50. Sanchez TH, Sineath RC, Kahle EM, Tregear SJ, Sullivan PS. The annual American Men's Internet Survey of Behaviors of Men Who Have Sex With Men in the United States: protocol and key indicators report 2013. JMIR Public Health Surveill 2015;1(1):e3 [FREE Full text] [CrossRef] [Medline]
  51. Compendium of Evidence-Based Interventions and Best Practices for HIV Prevention. Atlanta, GA: Centers for Disease Control and Prevention; 2017. PRS Efficacy Criteria for Best-Evidence Risk Reduction (RR) Individual-Level, Group-Level, and Couple-Level Interventions   URL: [accessed 2018-06-04] [WebCite Cache]
  52. Sullivan P, Rosenberg E, Sanchez T, Kelley CF, Luisi N, Cooper HL, et al. Explaining racial disparities in HIV incidence in black and white men who have sex with men in Atlanta, GA: a prospective observational cohort study. Ann Epidemiol 2015 Jun;25(6):445-454 [FREE Full text] [CrossRef] [Medline]
  53. Bauermeister J, Connochie D, Eaton L, Demers M, Stephenson R. Geospatial indicators of space and place: a review of multilevel studies of HIV prevention and care outcomes among young men who have sex with men in the United States. J Sex Res 2017;54(4-5):446-464 [FREE Full text] [CrossRef] [Medline]
  54. Bauermeister JA, Golinkoff JM, Muessig KE, Horvath KJ, Hightow-Weidman LB. Addressing engagement in technology-based behavioural HIV interventions through paradata metrics. Curr Opin HIV AIDS 2017 Sep;12(5):442-446. [CrossRef] [Medline]
  55. Krogstad JM. Pew Research Center. 2016 Apr 20. Rise in English proficiency among US Hispanics is driven by the young   URL: http:/​/www.​​fact-tank/​2016/​04/​20/​rise-in-english-proficiency-among-u-s-hispanics-is-driven-by-the-young/​ [accessed 2018-06-04] [WebCite Cache]

AMSM: adolescent men who have sex with men
LGBTQ+: lesbian, gay, bisexual, transgender, queer, and additional identities
MSM: men who have sex with men
RCT: randomized controlled trial
SMART: Study Management and Retention Toolkit
SMS: short message service
STI: sexually transmitted infections
YMSM: young men who have sex with men

Edited by G Eysenbach; submitted 19.02.18; peer-reviewed by M Kipke, A Algarin, K Nelson; comments to author 25.04.18; revised version received 15.05.18; accepted 15.05.18; published 10.07.18


©Jose Bauermeister, Patrick S Sullivan, Laura Gravens, James Wolfe, Kristina Countryman, Neena Smith-Bankhead, Ryan A Drab, Gregory Sallabank, Jordan D Helms, Kristie Khatibi, Rebecca Filipowicz, Keith Joseph Horvath, Erin Bonar, Amanda Castel, Lisa Hightow-Weidman, Jodie Guest, Rob Stephenson. Originally published in JMIR Research Protocols (, 10.07.2018.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.