Published on in Vol 5, No 2 (2016): Apr-Jun

Developing a Video-Based eHealth Intervention for HIV-Positive Gay, Bisexual, and Other Men Who Have Sex with Men: Study Protocol for a Randomized Controlled Trial

Developing a Video-Based eHealth Intervention for HIV-Positive Gay, Bisexual, and Other Men Who Have Sex with Men: Study Protocol for a Randomized Controlled Trial

Developing a Video-Based eHealth Intervention for HIV-Positive Gay, Bisexual, and Other Men Who Have Sex with Men: Study Protocol for a Randomized Controlled Trial


1Public Health Solutions, Research and Evaluation, New York, NY, United States

2Center for HIV/AIDS Educational Studies and Training (CHEST), Department of Psychology, and the Graduate Center of the City University of New York (CUNY), Hunter College, New York, NY, United States

3Center for HIV/AIDS Educational Studies and Training (CHEST), Graduate School of Public Health and Health Policy, City University of New York, New York, NY, United States

4Division of Infectious Diseases, Departments of Medicine and Epidemiology, Columbia University, New York, NY, United States

5Rollins School of Public Health, Department of Epidemiology, Emory University, Atlanta, GA, United States

6Smart + Strong, New York, NY, United States

Corresponding Author:

Sabina Hirshfield, PhD

Public Health Solutions

Research and Evaluation

40 Worth Street, 5th Floor

New York, NY, 10013

United States

Phone: 1 6466196676

Fax:1 6466196777


Background: Gay, bisexual, and other men who have sex with men (GBMSM) accounted for 67% of new US human immunodeficiency virus (HIV) infections in 2012; however, less than 40% of HIV-positive GBMSM are virally suppressed. Preventing transmission from virally unsuppressed men who have condomless anal sex (CAS) with serodiscordant partners is a public health imperative. New HIV infections in GBMSM are attributed in part to online access to sex partners; therefore, low-cost eHealth interventions are a unique opportunity to reach men where they meet partners.

Objective: To describe the protocol of a randomized controlled trial evaluating whether video-based messaging delivered online may lead to reductions in serodiscordant CAS and increased HIV disclosure.

Methods: Sex Positive![+] is a two-arm, phase III, video-based randomized controlled trial delivered online to GBMSM living with HIV. Participants in the intervention arm receive 10 video vignettes grounded in social learning and social cognitive theories that are designed to elicit critical thinking around issues of HIV transmission and disclosure. Participants in the attention control arm receive 10 video vignettes that focus on healthy living. All videos are optimized for mobile viewing. The study protocol includes five online assessments conducted over a 1-year period among 1500 US white, black, or Hispanic/Latino GBMSM living with HIV who report suboptimal antiretroviral therapy (ART) adherence or a detectable viral load in the past 12 months and recent CAS (past 6 months) with HIV-negative or unknown status male partners. Compared to the control arm, we hypothesize that men who watch the intervention videos will report at 12-month follow-up significantly fewer serodiscordant CAS partners, increased HIV disclosure, and improved social cognition (eg, condom use self-efficacy, perceived responsibility).

Results: Participant recruitment began in June 2015 and ended in December 2015.

Conclusions: This protocol describes the underlying theoretical framework and measures, study design, recruitment challenges, and antifraud measures for an online, video-based randomized controlled trial that has the potential to decrease HIV transmission risk behaviors among HIV-positive GBMSM who struggle with ART adherence. The Sex Positive![+] intervention allows for participation through multiple Internet-based mediums and has the potential to reach and engage a broader population of HIV-positive GBMSM who are virally unsuppressed.

ClinicalTrial: NCT02023580; (Archived by WebCite at

JMIR Res Protoc 2016;5(2):e125



Gay, bisexual, and other men who have sex with men (GBMSM) accounted for 67% of new human immunodeficiency virus (HIV) infections in the United States in 2012 [1]. Although antiretroviral therapy (ART) has improved survival with HIV, the low level of ART adherence presents a significant public health challenge in terms of the potential to transmit HIV [2]. Further, only three of the 96 evidence-based interventions defined by the Centers for Disease Control and Prevention have been designed for GBMSM living with HIV [3]. Among GBMSM with diagnosed HIV in the United States and Puerto Rico in 2010, it was estimated that 74% of men aged 18 to 24 years were virally unsuppressed compared to less than 40% in men aged 55 years and older. By race/ethnicity, the highest proportion of virally unsuppressed cases were among black GBMSM (63%), followed by Hispanic/Latino (58.5%), and white men (56%) [4]. This is particularly concerning because a recent study found that, compared to heterosexual males with HIV, GBMSM living with HIV who had unsuppressed viral loads had eight times the odds of engaging in serodiscordant condomless sex [5].

Preventing transmission in virally unsuppressed GBMSM who have condomless anal sex (CAS) with serodiscordant partners can have a great public health impact. Nationally, white, black, and Hispanic GBMSM continue to account for approximately 95% of newly diagnosed HIV infections among men [6]. Further, black and Hispanic men are also overrepresented as being virally unsuppressed [5,7,8]. Interventions designed to reduce CAS among GBMSM should be cost-effective and scalable, which are goals of the National HIV/AIDS Strategy [9]. Because new infections in GBMSM have been attributed in part to increased access to sex partners online [10,11], it is critical to deliver online behavioral interventions to GBMSM living with HIV to reach many high-risk men at a relatively low cost [12], engage men where they meet sex partners [13], and enable men to participate privately on a computer, tablet, or mobile phone with smartphone capabilities on their own schedule versus in a structured clinical setting [14].

Users of Technology

Primarily because of the anonymity and accessibility of online spaces (websites, mobile apps), GBMSM have been early adopters of technologies designed for sexual partnering [15]. However, as promising as technology may be for HIV prevention and because Hispanic/Latino and black populations are more likely to own mobile phones and use mobile apps compared to white populations [16,17], HIV prevention technology also introduces often-overlooked methodological pitfalls such as low engagement of racial/ethnic minorities and recruitment bias. Historically, online HIV prevention work has had low representation of minority GBMSM [18-20]. There are two likely explanations for this disparity. First, black men constitute 13% of the US male population [21], which may account for the smaller proportion who complete online surveys. Second, online recruitment bias may occur, such that researchers may not be using targeted language or graphics, or recruiting from sites that cater to minority GBMSM [22].

eHealth Interventions

Electronic health (eHealth) interventions have the potential to reach and engage GBMSM living with HIV from diverse racial/ethnic groups and socioeconomic statuses. These interventions are critical for risk-reduction efforts [23,24], can reach geographically dispersed men [12,25], and can adapt offline interventions [26]. However, few interventions have demonstrated efficacy in reducing HIV transmission from GBMSM who are HIV-positive. Recent eHealth and mobile health (mHealth) interventions for HIV-positive populations have focused on ART adherence [27,28], rather than sexual risk [29], and have been delivered through text messaging and on computers [30-33]. eHealth assessments of GBMSM living with HIV have been used to tailor provider-delivered interventions to decrease CAS with serodiscordant partners. Nevertheless, a recent provider-delivered eHealth intervention had low participation and retention, due mainly to factors such as time commitment and clinical setting [34]. Other eHealth interventions address HIV disclosure and condom use among GBMSM who are HIV-positive but are still in early stages of implementation (eg, Miranda et al [35]).

Cost-Effectiveness of eHealth Interventions

There is a great need for cost-effective HIV prevention strategies for HIV-positive populations, particularly GBMSM who constitute the majority of those living with HIV [36,37]. Even modest intervention effects can have a significant public health impact because the two most important factors that determine cost-effectiveness are (1) the HIV prevalence of the target population (preventing transmission from men who are HIV-positive rather than preventing acquisition by men who are HIV-negative) and (2) the cost per person reached [38]. Effective HIV prevention interventions that use digital media are also likely to be highly cost-effective because they can be easily replicated after development, require minimal staffing, and have unlimited geographic reach [31,38-40].

Aims and Objectives

The aim of this paper is to describe the study protocol of a randomized controlled trial (RCT) evaluating the effectiveness of a video-based messaging intervention delivered online, by comparing intervention and attention control groups on reductions in serodiscordant CAS and increases in HIV disclosure to sex partners. Through our prior work [18,19,41], we have identified a risk-reduction intervention approach for GBMSM living with HIV. The Sex Positive![+] study is conducted over a 1-year period among 1500 US white, black, and Hispanic/Latino GBMSM who were virally unsuppressed at some point during the past year, or who report suboptimal adherence to ART, and report recent CAS with serodiscordant male partners. Sex Positive![+] encompasses many characteristics found to reduce risk among HIV-positive populations in that it is theory-driven, has intervention content focused on HIV transmission behaviors, uses videos that demonstrate risk reduction and health behaviors through modeling, is delivered in an intensive manner, and is delivered over a 1-year period [42].

Trial Design

Sex Positive![+] is a two-arm randomized controlled phase III clinical trial with a 1:1 allocation ratio.

Ethics Statement

The Institutional Review Board at Public Health Solutions in New York, NY, approved all study procedures. A waiver of documentation of written consent was obtained given the Internet-based research approach. Although Federal regulation requires that researchers obtain written informed consent for research on human subjects, under 45 CFR § 46.117(c) [43], written consent can be waived for research that involves minimal risk to participants and involves no procedures for which written consent is normally required outside of the research context. This research meets that criterion and we use an alternative approach where participants click a button signifying that they have read the informed consent page and agree to participate in the study. An advantage of online studies is that the consent form is available for the participant to review and/or print at any time. This strategy complies with the requirement of 45 CFR § 46.117(c) that participants are given a written statement describing the research and risks.

A Data and Safety Monitoring Board has been established to conduct semiannual reviews of study activities and to ensure participant safety, validity, and integrity of the data. The Board is comprised of experts, independent of the trial or funding agency, in RCTs, Internet research, Web design, and HIV-positive populations. Furthermore, a Certificate of Confidentiality has been obtained from the National Institute of Mental Health to help protect the privacy of HIV-positive participants enrolled in this health-related study.


The target sample is 1500 high-risk, virally unsuppressed or less than 90% ART-adherent, US white, black, and Hispanic/Latino GBMSM living with HIV. Individuals participating in any aspect of the study must (1) be biologically male, (2) be age 18 or older, (3) be able to read and respond in English, (4) reside within the United States or a US territory, (5) report CAS with any HIV-negative or unknown status (serodiscordant) male partners in the past 6 months, (6) identify as HIV-positive, (7) report a detectable viral load, not being on ART and not knowing their viral load in the past year, or an undetectable viral load but less than 90% ART-adherent in the past 30 days, (8) identify as white, black, or Hispanic/Latino, (9) be willing to participate in an online intervention study for 12 months, and (10) have a working email address and mobile phone for intervention follow-up. We use quota sampling and targeted recruitment to ensure balanced representation of white (n=500), black (n=500), and Hispanic/Latino (n=500) men. Further, we include the following black racial/ethnic categories: black, African American, Caribbean, African, or multiethnic black [44]. In addition, we use targeted recruitment to ensure that 20% of the sample are men between the ages of 18 and 29 years. This group is overrepresented in the current HIV epidemic, particularly young men of color, and is less likely to be adherent to HIV medications or in care [4]. Men who meet study criteria, consent, and register to participate are automatically randomized into one of the two study arms.


Men are identified for the study through social networking websites and gay-oriented sexual networking websites, dating websites, global positioning system (GPS)-based mobile phone apps that utilize targeted recruitment by city, race, and ethnicity, and online bulletin boards. By recruiting from different types of websites and mobile phone apps, we increase our chances of reaching a broader, more diverse pool of men with HIV. The goals of recruitment are to identify eligible participants online who are willing to participate in an online, longitudinal HIV risk-reduction intervention. Based on previous research findings [22], study banner advertisements mirror the racial/ethnic composition of each study subgroup. One of our sources of recruitment is POZ Personals, the dating site for POZ Magazine, which distributes internal system messages to a defined subset of US male members who are HIV-positive, at least 18 years of age, and self-identify as gay or bisexual.

Primary Outcome Measures

Both HIV status disclosure to sex partners and serodiscordant CAS are assessed at each of five survey time points (baseline, 3-, 6-, 9-, and 12-month follow-up). The recall period for the primary outcome measures is the past 3 months for each of the five online assessments (Table 1).

Table 1. Survey instrument summary of primary outcomes assessed at baseline and at 3-, 6-, 9-, and 12-month follow-ups.
Primary outcomes measuresDescription or lead questionItems
Serodiscordant condomless anal sexPast 3 months, three most recent male sexual partners in one-on-one encountersSexual behavior by partner type (eg, one-time, repeat, exchange): insertive and/or receptive oral sex with or without condoms and ejaculation (y/n), insertive and/or receptive anal sex with or without condoms and ejaculation (y/n), drug and/or alcohol use prior to or during sex (y/n), total number of anal sex acts (with or without condoms) for each sex partner
HIV disclosurePast 3 months, HIV disclosure with three most recent sex partners in one-on-one encountersDemographic questions related to most recent partner(s) include: race/ethnicity, age, partner relationship type, partner serostatus, one-time vs repeat partner, and exchange vs nonexchange partner. HIV disclosure questions include: knowing partner(s) serostatus before or after having sex, asking partner(s) status, telling partner(s) one’s serostatus, who disclosed their serostatus first (participant or partner), how they learned about the partner(s) serostatus (eg, asking, telling, online profile)

Secondary Outcome Measures

Secondary outcomes assessed at each time point include self-reported adherence to HIV medications, viral load, and CD4 count. Syndemic factors, or co-occurring epidemics, thought to be related to HIV transmission risk will also be assessed at various survey time points, including drug and alcohol use [45,46], depressive and anxiety symptoms [47-49], condom use and HIV disclosure self-efficacy [50], sexual compulsivity [51], HIV stigma, and interpersonal violence [52-54]. Process measures are used to track participants’ interest in, acceptability of, and verification of video viewing by having men complete brief postvideo online surveys. These surveys are designed to assess participants’ likes and dislikes as well as to elicit critical thinking about the video as it pertains to study outcomes [19,55].

Sample Size

Based on the prevalence of behaviors in our previous studies, we calculated true proportions and sample sizes using chi-square tests for this two-arm design. We estimated that by enrolling approximately 750 men per group and retaining 75% at 12-month follow-up, we would have 80% power at a 5% alpha level to detect a minimum reduction of 8% in the number of serodiscordant CAS partners between the intervention and attention control arms.

Intervention Content

Video messages are an effective way to deliver HIV prevention to GBMSM [19,56-58]. The first of our three theoretically grounded HIV prevention videos (from the HIV Big Deal project) tackling issues of CAS, HIV disclosure, and testing was rigorously evaluated among HIV-negative, HIV-positive, and untested GBMSM recruited online [19]. In our single-session video pilot for 971 GBMSM, we found significant reductions in CAS in the most recent encounter (9% decrease) and significant increases in HIV disclosure at 3-month follow-up (13% increase) compared to baseline [19]. In our subsequent online, single-session RCT for 3092 GBMSM that used videos from the HIV Big Deal project, we found significant reductions in CAS among men in the video study arm at 60-day follow-up (8% decrease) compared to baseline; HIV-positive men in the video study arm reduced their CAS (14% decrease), including with HIV-negative or status unknown partners, at 60-day follow-up (13% decrease) compared to baseline. Men living with HIV were also significantly more likely than men who were HIV-negative or untested to complete follow-up (57% vs 51%, P=.002) [18].

Theoretical Framework for the Intervention Videos

Employing a dramatic video series grounded in social learning and social cognitive theories [59,60], the Sex Positive![+] study engages learners through storytelling and promotes critical thinking on issues of HIV disclosure to sex partners, medication adherence and viral suppression, and serodiscordant CAS. In collaboration with a local production team, including a scriptwriter, producer, and director, we produced Just a Guy, a 6-episode video series that follows the story of “Guy,” an openly gay man living with HIV in Brooklyn, NY. The video series is based, in part, on the HIV Big Deal project described previously, which was launched online in 2008 [61]. According to social learning theory, individuals learn through the observation of others’ attitudes, behaviors, and the outcomes of those behaviors [59]. Videos developed for the intervention described in this paper include elements of social learning and attitude change theories, both of which informed the instructional design and delivery of our pilot online video intervention [19] and online feasibility trial of GBMSM [12]. More specifically, the intervention relies on three critical design dimensions including the medium, the degree of realism, and modeling (Table 2).

Table 2. Survey instrument summary of theoretical constructs assessed at baseline and at 3-, 6-, 9-, and 12-month follow-ups.
Construct, topic, and lead questionsResponse options
Construct: self-efficacy

Topic: disclosure to sex partners

Lead question: How confident are you that you could tell a potential sex partner your HIV status...Not at all confident, not very confident, somewhat confident, very confident, extremely confident your online or mobile phone app dating profile? an email? a text message?

...over the phone? person?

Topic: safer sex [91]

Lead question: Now think about future sexual encounters with HIV-negative or unknown HIV status male partners. How confident are you that you could have anal sex with a condom...Not at all confident, not very confident, somewhat confident, very confident, extremely confident, prefer not to answer

When you feel depressed?

When you think that your partner does not want to use condoms?

When you are drunk or high on drugs?

When you are really sexually aroused?
Construct: self-regulatory skills

Topic: sexual compulsivity [92]

Lead question: Below are statements about sex that you may agree or disagree with.Not at all like me, slightly like me, mainly like me, very much like me, prefer not to answer

My sexual appetite has gotten in the way of my relationships.

My sexual thoughts and behaviors are causing problems in my life.

My desires to have sex have disrupted my daily life.

I sometimes fail to meet my commitments and responsibilities because of my sexual behaviors.

I sometimes get so horny I could lose control.

I find myself thinking about sex while at work.

I feel that sexual thoughts and feelings are stronger than I am.

I have to struggle to control my sexual thoughts and behavior.

I think about sex more than I would like to.

It has been difficult for me to find sex partners who desire having sex as much as I want to.
Construct: outcome expectancies

Topic: condoms and anal sex (adapted from Bimbi et al [93])

Lead question: Below is a list of statements that you may agree or disagree with.Strongly agree, somewhat agree, neutral, somewhat disagree, strongly disagree, prefer not to answer

I am more likely to use a condom with men who are HIV-negative or of unknown status.

I am more likely to have anal insertive sex (top) without a condom while drinking or high.

I am more likely to have anal receptive sex (bottom) without a condom while drinking or high.

I am less likely to have anal sex with men who are HIV-negative or of unknown status.
Construct: perceived responsibility

Topic: personal and partner responsibility for preventing HIV transmission [94]

Lead question: Below is a list of statements that you may agree or disagree with.Strongly agree, somewhat agree, neutral, somewhat disagree, strongly disagree, prefer not to answer

It is very important for me to use condoms to protect my sex partners from HIV.

HIV-positive gay men have a responsibility to keep other gay men from becoming positive.

When HIV-positive and HIV-negative men have sex with each other, they have an equal responsibility for being safe.

HIV-positive gay men have a special obligation to have safe sex with men who are negative or do not know their HIV status.

I feel responsible for protecting my partners from HIV.

If my partner is HIV-negative, he should not put the responsibility on me for safer sex.

If men who are HIV-negative want to have risky sex, it is their choice to do so.

It should be the responsibility of someone who is HIV-negative—not someone who is positive—to make sure their sex is safe.

I feel it is my partner’s responsibility to protect himself from HIV if he is negative.

It is my responsibility to protect others from getting HIV.

Increasingly, eHealth HIV behavioral interventions are incorporating digital media, ranging from brief, untailored video interventions to complex computer-tailored multimedia interventions that target individual behaviors [62,63]. Online video-based interventions are an appealing and effective medium to deliver HIV prevention content to GBMSM [56,57]. Furthermore, video has greater potential to engage learners than conventional text or graphics in Web-based or print materials [64-66].

Storytelling, often more effective than exposition, is characterized by realism. From the perspectives of social learning and social cognitive theories [67], and attitude change theories [68], plausible “stories like mine” and “characters like me” are critical factors for engagement [69,70]. A Community Advisory Group, assembled for the Sex Positive![+] study, recommended that the videos feature an HIV-positive main character who overcomes a “victim” status and develops a sense of empowerment that positively impacts his personal relationships and physical health.

Social learning and cognitive theories emphasize the role of outcome expectancies regarding HIV disclosure and condom use self-efficacy and modeling of self-regulatory skills [59,60,71-73]. The tenets of social learning and social cognitive theories are embedded in the content, dialog, and nonverbal cues of the intervention videos, with the goal of preventing risk behavior before it happens. This intervention aims to reduce sexual risk behaviors by modeling HIV disclosure and discussions about safer sex. For example, in Just a Guy, the viewer learns through a nonverbal cue that Guy and Matt—a potential love interest—used a condom for anal sex the night before as Matt places the condom wrapper on his forehead on waking up. In this scene, Matt learns that Guy is HIV-positive and the two have a heated discussion about HIV disclosure responsibility. The video does not attempt to answer the question of personal responsibility or assert any one behavioral prescription, but rather encourages the viewer to think critically and discuss the issue with their sex partners.

Attention Control Videos

The two study arms are designed to be equal in the number of sessions, video length, study duration, and interest level. All videos are available for free viewing on the Sex Positive![+] study website, although men can view only videos that are assigned to their study arm. The control arm receives 10 healthy living videos that cover a range of topics including nutrition, physical exercise, smoking, and sleep quality. Attention control videos were selected from video-sharing websites and voted on by members of the research team.

Video Boosters

Because the effects of most preventive interventions tend to gradually wane over time, the inclusion of follow-up booster sessions can support prior skills learned to sustain an intervention impact [74]. Based on our team’s experience with intervention effects attenuating at 6 months [75], participants in the intervention arm will receive four video boosters after they complete the 6-month assessment survey. We edited Ask Me, Tell Me [76], the fourth video installment of our HIV Big Deal series, into three booster episodes (each episode is approximately 3 minutes in length). This particular prevention video from the HIV Big Deal series was selected for its emphasis on the importance of discussing one’s history of sexually transmitted infections (STIs) with sex partners and reducing episodes of CAS with serodiscordant partners. The final booster video focuses on the issue of social support for persons living with HIV and was selected from a video-sharing website. Participants in the attention control arm receive four additional healthy living videos after they complete the 6-month assessment survey.

Intervention Activities


Participants in the intervention and attention control arms complete assessments at baseline and at 3-, 6-, 9-, and 12-month time points. To reduce the chance of instrument reactivity, we provide detailed online survey assessments at baseline and 12-month follow-up and brief assessments at 3-, 6-, and 9-month follow-ups. Participants will receive a text message and/or email with a survey link when it is time for them to complete a follow-up survey or watch a video. The dissemination of intervention and attention control videos occurs between the baseline and 3-month assessments, spaced 1 week apart. Following the 6-month assessment, participants in both arms receive four video boosters, spaced 1 week apart. The intervention and attention control videos are only available to study participants via a secure URL and men cannot forward video links to anyone, thus preventing potential cross-contamination between arms. All intervention activities occur online and are optimized for mobile performance.

Administrative Platform

For complex online interventions, developing a user-friendly administrative platform for the deployment and monitoring of data collection and intervention activities is critical. The online administrative platform enables study staff to screen potential participants, obtain consent, register, randomize participants into one of two study arms (intervention or attention control), monitor recruitment and retention, and flag suspicious cases. The administrative platform can be programmed to produce reports on participants, such as who completed certain study activities, who needs to be sent incentives, who needs to receive retention calls, and so forth. In addition, the study dashboard that participants see when they log in provides information on what study activities they have completed or need to complete, their personal information (eg, name, address, phone number) that can be updated, their communication preferences (eg, receiving texts or calls), and how much in incentives they have earned. The dashboard can also host hyperlinks to provide health information (eg, nutrition and HIV, ART adherence) and track which links participants click on.

Eligibility, Screening, and Consent

Men who click on a study banner, email, or online classified advertisement are directed to a brief, secure screener survey housed on the online administrative platform. Those who are determined to be eligible for study inclusion are directed to the study landing page and registration platform, which describes the study and provides a consent form for intervention activities. Men who are determined to be ineligible are provided with a message indicating that they are not eligible, thanking them for their time, and directing them to HIV prevention and other health resources, including an invitation to join a participant registry for future study opportunities.

Registration, Verification, and Randomization

After consent, participants are guided through online study registration, including the creation of a log-in username and password and collection and automated verification of their email address and mobile phone number. Then, they are automatically randomized into the intervention or attention control arm [77] through a stratified block randomization (by race/ethnicity and age), which will balance groups within a 5% range [77,78]. On accessing their study account, men are instructed to complete the baseline survey, which is hosted on a secure server on Survey Gizmo that is compliant with the Health Insurance Portability and Accountability Act. Men remain in their original assignment group (intention to treat) and are sent text message and email notifications for each of their intervention activities even if they discontinue participation.


Participants can receive up to a total of US $115 in gift cards (distributed electronically and via direct mail). Figure 1 provides the incentive schedule for Sex Positive![+].

Figure 1. Study incentive structure.
View this figure
Study Retention

Historically, online research has had lower retention rates than offline research because there are fewer social constraints compared to in-person interviewing [79]. However, recent advances in retention protocols and technology have greatly improved researchers’ ability to retain participants in HIV prevention trials, with 90% retention at 6 months and 82% retention at 12 months [80,81]. To ensure minimal attrition, Sex Positive![+] conducts multiple sessions, offers video boosters to maintain study interest, provides incentives for each study activity, sends reminder emails and text messages for videos and follow-up assessments, and uses the online administrative platform to create daily lists of nonresponsive participants for the retention coordinator to contact.

Protection Against Fraud

An advantage of online research is data validity for sensitive information. Indeed, a growing number of studies indicate higher reporting of sexual risk and substance-using behaviors with computer-based surveys compared to mail, phone, and in-person surveys [82-84]. However, compared to the gold standard of in-person interviewing, a limitation of online research—as with mail and phone surveys—is the challenge of verifying a participant’s identity [85]. Based on recommendations made during an open session of our Data and Safety Monitoring Board, as well as a meeting of Internet experts about the issue of online fraud in research studies [86], the Sex Positive![+] study implements several protections aimed at reducing the likelihood of fraud. Specifically, (1) contact information is verified during registration using multiple methods; (2) duplicate detection (of Internet Protocol [IP] address and mailing address) software is used to detect instances of participants attempting to create multiple study profiles; (3) mobile phone numbers are investigated to determine whether they are voice over IP (VoIP) numbers—an individual can obtain multiple VoIP numbers on the Internet, typically for no charge, that can be routed to the individual’s mobile device (eg, this helps a potential participant to sign up numerous times with unique phone numbers); (4) proxy IP addresses and invalid mailing addresses are flagged for further scrutiny by the research team; (5) trap questions are used in the baseline survey to flag cases with inconsistent data—“I am HIV-negative”—or careless responses (eg, straightlining); (6) study staff conduct weekly analyses of new screener data to identify individuals (by IP address) who make multiple attempts to join the study; (7) compensation is kept sufficiently low to reduce the chances of participating solely to gain incentive payments; and (8) to ensure participant authenticity, the final study incentive is mailed to a verified physical address.


Primary analyses for intervention efficacy will examine whether participants in the intervention arm report fewer CAS acts with serodiscordant partners, a higher percentage of anal sex acts with condoms, fewer sexual partners, and more disclosure of HIV status with partners compared to participants in the attention control arm. Dose-response analyses will allow us to examine whether a certain number of intervention videos—“doses”—are necessary to effect study goals.

Even with a robust retention plan, incentives, and survey programming that requires responses, missing data are inevitable because participants can “refuse” survey items or drop out of the study. For this study, data from the online screener will verify that blocked randomization produced equivalent groups and will also be used to assess possible sample attrition bias. Although there are many ways to handle missing data, our experience suggests that maximum likelihood estimation is the best approach, using the appropriate algorithm for estimation purposes.

We will also conduct analyses to assess the savings in averted HIV-related lifetime treatment costs, the total number of quality-adjusted life years (QALYs) saved by preventing a single HIV infection, and the cost of developing and implementing the Sex Positive![+]study. In general terms, after the 12-month follow-up we will analyze up to three partner-by-partner sexual encounters, for biological male partners only, as well as global number of condomless anal or vaginal sex partners (ie, biological male and female partners, transidentified females, and transidentified males) at each of the four follow-up assessment time points. This will provide an estimate of the number of secondary infections expected among the participant’s serodiscordant sex partners. The main analysis will assume that study participants with an undetectable viral load at a particular time point are noninfectious. Although a separate estimate will be calculated for each participant at each of the four follow-up time points, the sum of these time point findings will estimate the total number of secondary infections expected among the participant’s sex partners during the entire study period. We will then calculate the total number of estimated infections prevented by the intervention arm versus the control by measuring the difference in the mean number of expected secondary infections for all men in the intervention and control arms over the 12-month period. Based on the total number of prevented infections, we will calculate the corresponding savings in averted HIV-related lifetime treatment costs as well as the total number of QALYs saved by preventing a single HIV infection. Finally, we will calculate the net cost of the intervention, the cost per infection prevented, and the net cost per QALY saved (ie, the cost-utility ratio) [87]. The intervention can be considered “cost saving” if the net cost is negative and “cost-effective” if the cost-utility ratio is less than US $100,000 per QALY saved [88].

Participant recruitment began in June 2015 and ended in December 2015.

Those eHealth interventions, such as Sex Positive![+], that allow participation through multiple Internet-based mediums (ie, computer and mobile access) have the potential to reach and engage a broader population of GBMSM with HIV. More specifically, this type of online intervention can reach men living with HIV who are outside of HIV epicenters, who may be beyond the reach of traditional prevention services, and are poorly represented in research. Furthermore, the online administrative platform and videos will be accessible to a much larger population at a relatively low cost following completion and evaluation of the study. For populations with limited Internet access, the intervention can be adapted for use in HIV clinics and community-based organizations via private kiosks, laptops or tablets, or in small group settings. Thus, this self-administered, online video-based intervention can be implemented in various settings at minimal cost.


This study protocol has several limitations that deserve mention. All men were recruited online, through social networking and gay-oriented sexual networking websites and mobile phone apps. As such, the findings may not be generalizable to HIV-positive GBMSM who do not own a mobile phone or have Internet access, access these types of websites or mobile phone apps, to men who do not identify as gay, to individuals exposed to a study banner or email but choose not to click on it, or to men who do not identify as black, white, or Hispanic/Latino. Study content is only available in English, which limits its reach to participants. There is a need to translate content into Spanish because it is the second most-spoken language in the United States and represents a subpopulation of GBMSM with high rates of HIV [4,89]. Lastly, a potential limitation is that participants self-report their health outcomes, specifically viral load. However, a recent validation study of 639 individuals with HIV from an ongoing prospective study in New York found that participant recall of viral load agreed with the Department of Health’s registry data 85% of the time [90].


In conclusion, the Sex Positive![+] study addresses the lack of interventions designed for GBMSM living with HIV. This protocol describes the underlying theoretical framework and measures, study design, recruitment challenges, and antifraud measures. If efficacious, it will have a significant impact on reducing HIV transmission risk in a disproportionately affected population. Although this eHealth intervention is being implemented with virally unsuppressed men or men who struggle with ART adherence, it can be adapted for delivery in other settings and with other populations.


This research was supported by a grant from the National Institute of Mental Health (R01 MH100973, PI: Hirshfield).

Authors' Contributions

SH, MJD, JTP, CG, RG, RS, PSS, and MAC planned aspects of the trial and wrote the manuscript. STH, IA, MJD, ISY worked on trial implementation and collaborated on the writing of the manuscript. All authors have read and approved the final version of the manuscript.

Conflicts of Interest

None declared.

  1. Centers for Disease Control and Prevention. 2014. HIV Surveillance Report, 2012   URL: [accessed 2016-05-26] [WebCite Cache]
  2. Kilmarx PH, Mermin J. Prevention with people with HIV in the United States: the nexus of HIV prevention and treatment. J Acquir Immune Defic Syndr 2012 Jul 1;60(3):219-220. [CrossRef] [Medline]
  3. Centers for Disease Control and Prevention. 2015. Compendium of evidence-based interventions and best practices for HIV prevention   URL: [accessed 2016-05-26] [WebCite Cache]
  4. Singh S, Bradley H, Hu X, Skarbinski J, Hall HI, Lansky A, Centers for Disease Control and Prevention (CDC). Men living with diagnosed HIV who have sex with men: progress along the continuum of HIV care--United States, 2010. MMWR Morb Mortal Wkly Rep 2014 Sep 26;63(38):829-833 [FREE Full text] [Medline]
  5. Hall HI, Holtgrave DR, Tang T, Rhodes P. HIV transmission in the United States: considerations of viral load, risk behavior, and health disparities. AIDS Behav 2013 Jun;17(5):1632-1636. [CrossRef] [Medline]
  6. Centers for Disease Control and Prevention. HIV Surveillance Report: Diagnoses of HIV Infection in the United States and Dependent Areas, 2014. 2015 Nov.   URL: [accessed 2016-05-26] [WebCite Cache]
  7. Gant Z, Bradley H, Hu X, Skarbinski J, Hall HI, Lansky A, Centers for Disease Control and Prevention (CDC). Hispanics or Latinos living with diagnosed HIV: progress along the continuum of HIV care - United States, 2010. MMWR Morb Mortal Wkly Rep 2014 Oct 10;63(40):886-890 [FREE Full text] [Medline]
  8. Whiteside YO, Cohen SM, Bradley H, Skarbinski J, Hall HI, Lansky A, Centers for Disease Control and Prevention. Progress along the continuum of HIV care among blacks with diagnosed HIV- United States, 2010. MMWR Morb Mortal Wkly Rep 2014 Feb 7;63(5):85-89 [FREE Full text] [Medline]
  9. The White House Office of National AIDS Policy. National HIV/AIDS Strategy for the United States: Updated to 2020. 2015 Jul.   URL: [accessed 2016-05-26] [WebCite Cache]
  10. Beymer MR, Weiss RE, Bolan RK, Rudy ET, Bourque LB, Rodriguez JP, et al. Sex on demand: geosocial networking phone apps and risk of sexually transmitted infections among a cross-sectional sample of men who have sex with men in Los Angeles County. Sex Transm Infect 2014 Nov;90(7):567-572 [FREE Full text] [CrossRef] [Medline]
  11. Rendina HJ, Jimenez RH, Grov C, Ventuneac A, Parsons JT. Patterns of lifetime and recent HIV testing among men who have sex with men in New York City who use Grindr. AIDS Behav 2014 Jan;18(1):41-49 [FREE Full text] [CrossRef] [Medline]
  12. Chiasson MA, Parsons JT, Tesoriero JM, Carballo-Dieguez A, Hirshfield S, Remien RH. HIV behavioral research online. J Urban Health 2006 Jan;83(1):73-85 [FREE Full text] [CrossRef] [Medline]
  13. Rosser BR, Oakes JM, Horvath KJ, Konstan JA, Danilenko GP, Peterson JL. HIV sexual risk behavior by men who use the Internet to seek sex with men: results of the Men's INTernet Sex Study-II (MINTS-II). AIDS Behav 2009 Jun;13(3):488-498 [FREE Full text] [CrossRef] [Medline]
  14. Wolitski RJ, Gómez CA, Parsons JT. Effects of a peer-led behavioral intervention to reduce HIV transmission and promote serostatus disclosure among HIV-seropositive gay and bisexual men. AIDS 2005 Apr;19 Suppl 1:S99-S109. [Medline]
  15. Grov C, Breslow AS, Newcomb ME, Rosenberger JG, Bauermeister JA. Gay and bisexual men's use of the Internet: research from the 1990s through 2013. J Sex Res 2014;51(4):390-409 [FREE Full text] [CrossRef] [Medline]
  16. Duggan M. Pew Research Center. 2013 Sep 19. Cell phone activities 2013   URL: [accessed 2016-05-26] [WebCite Cache]
  17. Grov C, Ventuneac A, Rendina HJ, Jimenez RH, Parsons JT. Perceived importance of five different health issues for gay and bisexual men: implications for new directions in health education and prevention. Am J Mens Health 2013 Jul;7(4):274-284 [FREE Full text] [CrossRef] [Medline]
  18. Hirshfield S, Chiasson MA, Joseph H, Scheinmann R, Johnson WD, Remien RH, et al. An online randomized controlled trial evaluating HIV prevention digital media interventions for men who have sex with men. PLoS One 2012;7(10):e46252 [FREE Full text] [CrossRef] [Medline]
  19. Chiasson MA, Shaw FS, Humberstone M, Hirshfield S, Hartel D. Increased HIV disclosure three months after an online video intervention for men who have sex with men (MSM). AIDS Care 2009 Sep;21(9):1081-1089. [CrossRef] [Medline]
  20. Rosser BR, Horvath KJ, Hatfield LA, Peterson JL, Jacoby S, Stately A. Predictors of HIV disclosure to secondary partners and sexual risk behavior among a high-risk sample of HIV-positive MSM: results from six epicenters in the US. AIDS Care 2008 Sep;20(8):925-930 [FREE Full text] [CrossRef] [Medline]
  21. US Census Bureau. The Black Alone or in Combination Population in the United States: 2012. Washington, DC: Government Printing Office; 2012.   URL: [accessed 2016-05-26] [WebCite Cache]
  22. Sullivan PS, Khosropour CM, Luisi N, Amsden M, Coggia T, Wingood GM, et al. Bias in online recruitment and retention of racial and ethnic minority men who have sex with men. J Med Internet Res 2011;13(2):e38 [FREE Full text] [CrossRef] [Medline]
  23. Hooper S, Rosser BR, Horvath KJ, Oakes JM, Danilenko G, Men's INTernet Sex II (MINTS-II) Team. An online needs assessment of a virtual community: what men who use the internet to seek sex with men want in Internet-based HIV prevention. AIDS Behav 2008 Nov;12(6):867-875 [FREE Full text] [CrossRef] [Medline]
  24. Bauermeister JA, Pingel ES, Jadwin-Cakmak L, Harper GW, Horvath K, Weiss G, et al. Acceptability and preliminary efficacy of a tailored online HIV/STI testing intervention for young men who have sex with men: the Get Connected! Program. AIDS Behav 2015 Feb 1. [CrossRef] [Medline]
  25. Sun CJ, Stowers J, Miller C, Bachmann LH, Rhodes SD. Acceptability and feasibility of using established geosocial and sexual networking mobile applications to promote HIV and STD testing among men who have sex with men. AIDS Behav 2015 Mar;19(3):543-552. [CrossRef] [Medline]
  26. Card JJ, Kuhn T, Solomon J, Benner TA, Wingood GM, DiClemente RJ. Translating an effective group-based HIV prevention program to a program delivered primarily by a computer: methods and outcomes. AIDS Educ Prev 2011 Apr;23(2):159-174. [CrossRef] [Medline]
  27. Lewis MA, Uhrig JD, Bann CM, Harris JL, Furberg RD, Coomes C, et al. Tailored text messaging intervention for HIV adherence: a proof-of-concept study. Health Psychol 2013 Mar;32(3):248-253. [CrossRef] [Medline]
  28. Simoni JM, Kutner BA, Horvath KJ. Opportunities and challenges of digital technology for HIV treatment and prevention. Curr HIV/AIDS Rep 2015 Sep 28. [CrossRef] [Medline]
  29. Pellowski JA, Kalichman SC. Recent advances (2011-2012) in technology-delivered interventions for people living with HIV. Curr HIV/AIDS Rep 2012 Dec;9(4):326-334 [FREE Full text] [CrossRef] [Medline]
  30. Horvath KJ, Oakes JM, Rosser BR, Danilenko G, Vezina H, Amico KR, et al. Feasibility, acceptability and preliminary efficacy of an online peer-to-peer social support ART adherence intervention. AIDS Behav 2013 Jul;17(6):2031-2044 [FREE Full text] [CrossRef] [Medline]
  31. Hightow-Weidman LB, Muessig KE, Bauermeister J, Zhang C, LeGrand S. Youth, technology, and HIV: recent advances and future directions. Curr HIV/AIDS Rep 2015 Dec;12(4):500-515. [CrossRef] [Medline]
  32. Gamage DG, Sidat M, Read T, Cummings R, Bradshaw CS, Howley K, et al. Evaluation of Health Map: a patient-centred web-based service for supporting HIV-infected patients. Sex Health 2011 Jun;8(2):194-198. [CrossRef] [Medline]
  33. Harris LT, Lehavot K, Huh D, Yard S, Andrasik MP, Dunbar PJ, et al. Two-way text messaging for health behavior change among human immunodeficiency virus-positive individuals. Telemed J E Health 2010 Dec;16(10):1024-1029 [FREE Full text] [CrossRef] [Medline]
  34. Bachmann LH, Grimley DM, Gao H, Aban I, Chen H, Raper JL, et al. Impact of a computer-assisted, provider-delivered intervention on sexual risk behaviors in HIV-positive men who have sex with men (MSM) in a primary care setting. AIDS Educ Prev 2013 Apr;25(2):87-101. [CrossRef] [Medline]
  35. Miranda J, Côté J, Godin G, Blais M, Otis J, Guéhéneuc YG, et al. An internet-based intervention (Condom-Him) to increase condom use among HIV-positive men who have sex with men: protocol for a randomized controlled trial. JMIR Res Protoc 2013;2(2):e39 [FREE Full text] [CrossRef] [Medline]
  36. Marseille E, Shade SB, Myers J, Morin S. The cost-effectiveness of HIV prevention interventions for HIV-infected patients seen in clinical settings. J Acquir Immune Defic Syndr 2011 Mar 1;56(3):e87-e94. [CrossRef] [Medline]
  37. Lee MB, Leibowitz A, Rotheram-Borus MJ. Cost-effectiveness of a behavioral intervention for seropositive youth. AIDS Educ Prev 2005 Apr;17(2):105-118. [CrossRef] [Medline]
  38. Cohen DA, Wu S, Farley TA. Comparing the cost-effectiveness of HIV prevention interventions. J Acquir Immune Defic Syndr 2004 Nov 1;37(3):1404-1414. [Medline]
  39. Claborn KR, Fernandez A, Wray T, Ramsey S. Computer-based HIV adherence promotion interventions: a systematic review: Translation Behavioral Medicine. Transl Behav Med 2015 Sep;5(3):294-306 [FREE Full text] [CrossRef] [Medline]
  40. Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, internet and Web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep 2015 Jan 28. [CrossRef] [Medline]
  41. Hirshfield S, Grov C, Parsons JT, Anderson I, Chiasson MA. Social media use and HIV transmission risk behavior among ethnically diverse HIV-positive gay men: results of an online study in three U.S. states. Arch Sex Behav 2015 Oct;44(7):1969-1978. [CrossRef] [Medline]
  42. Crepaz N, Lyles CM, Wolitski RJ, Passin WF, Rama SM, Herbst JH, HIV/AIDS Prevention Research Synthesis (PRS) Team. Do prevention interventions reduce HIV risk behaviours among people living with HIV? A meta-analytic review of controlled trials. AIDS 2006 Jan 9;20(2):143-157. [CrossRef] [Medline]
  43. US Department of Health and Human Services, Office for Human Research Protections. 2009 Jan 15. 45 CFR 46: Protection of human subjects: documentation of informed consent   URL: [accessed 2016-05-26] [WebCite Cache]
  44. Mayer KH, Wang L, Koblin B, Mao C, Mannheimer S, Magnus M. An evolving concentrated epidemic: comparison of newly diagnosed, previously diagnosed and HIV-uninfected black men who have sex with men in 6 US cities. 2012 Presented at: International AIDS Conference; July 22-27, 2012; Washington, DC.
  45. Velasquez MM, von Sternberg K, Johnson DH, Green C, Carbonari JP, Parsons JT. Reducing sexual risk behaviors and alcohol use among HIV-positive men who have sex with men: a randomized clinical trial. J Consult Clin Psychol 2009 Aug;77(4):657-667 [FREE Full text] [CrossRef] [Medline]
  46. Hirshfield S, Remien RH, Chiasson MA. Crystal methamphetamine use among men who have sex with men: results from two national online studies. J Gay Lesbian Psychotherapy 2006 Aug 14;10(3-4):85-93. [CrossRef]
  47. Salomon EA, Mimiaga MJ, Husnik MJ, Welles SL, Manseau MW, Montenegro AB, et al. Depressive symptoms, utilization of mental health care, substance use and sexual risk among young men who have sex with men in EXPLORE: implications for age-specific interventions. AIDS Behav 2009 Aug;13(4):811-821 [FREE Full text] [CrossRef] [Medline]
  48. Rabkin JG. HIV and depression: 2008 review and update. Curr HIV/AIDS Rep 2008 Nov;5(4):163-171. [Medline]
  49. Leserman J. Role of depression, stress, and trauma in HIV disease progression. Psychosom Med 2008 Jun;70(5):539-545. [CrossRef] [Medline]
  50. Parsons JT, Huszti HC, Crudder SO, Rich L, Mendoza J. Maintenance of safer sexual behaviours: evaluation of a theory-based intervention for HIV seropositive men with haemophilia and their female partners. Haemophilia 2000 May;6(3):181-190. [Medline]
  51. Grov C, Parsons JT, Bimbi DS. Sexual compulsivity and sexual risk in gay and bisexual men. Arch Sex Behav 2010 Aug;39(4):940-949 [FREE Full text] [CrossRef] [Medline]
  52. Benoit E, Downing MJJ. Childhood sexual experiences among substance-using non-gay identified Black men who have sex with men and women. Child Abuse Negl 2013 Sep;37(9):679-690 [FREE Full text] [CrossRef] [Medline]
  53. Mimiaga MJ, Noonan E, Donnell D, Safren SA, Koenen KC, Gortmaker S, et al. Childhood sexual abuse is highly associated with HIV risk-taking behavior and infection among MSM in the EXPLORE Study. J Acquir Immune Defic Syndr 2009 Jul 1;51(3):340-348 [FREE Full text] [CrossRef] [Medline]
  54. Welles SL, Baker AC, Miner MH, Brennan DJ, Jacoby S, Rosser BR. History of childhood sexual abuse and unsafe anal intercourse in a 6-city study of HIV-positive men who have sex with men. Am J Public Health 2009 Jun;99(6):1079-1086 [FREE Full text] [CrossRef] [Medline]
  55. Tarnowski KJ, Simonian SJ. Assessing treatment acceptance: the Abbreviated Acceptability Rating Profile. J Behav Ther Exp Psychiatry 1992 Jun;23(2):101-106. [Medline]
  56. Chiasson MA, Hirshfield S, Rietmeijer CA. HIV prevention and care in the digital age. J Acquir Immune Defic Syndr 2010 Dec;55 Suppl 2:S94-S97. [CrossRef] [Medline]
  57. Blas MM, Alva IE, Carcamo CP, Cabello R, Goodreau SM, Kimball AM, et al. Effect of an online video-based intervention to increase HIV testing in men who have sex with men in Peru. PLoS One 2010;5(5):e10448 [FREE Full text] [CrossRef] [Medline]
  58. Rosser BR, Hatfield LA, Miner MH, Ghiselli ME, Lee BR, Welles SL, Positive Connections Team. Effects of a behavioral intervention to reduce serodiscordant unsafe sex among HIV positive men who have sex with men: the Positive Connections randomized controlled trial study. J Behav Med 2010 Apr;33(2):147-158 [FREE Full text] [CrossRef] [Medline]
  59. Bandura A. Social Learning Theory. Englewood Cliffs, NJ: Prentice Hall; 1977.
  60. Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall; 1986.
  61. HIV Big Deal. 2011.   URL: [accessed 2016-05-26] [WebCite Cache]
  62. Guse K, Levine D, Martins S, Lira A, Gaarde J, Westmorland W, et al. Interventions using new digital media to improve adolescent sexual health: a systematic review. J Adolesc Health 2012 Dec;51(6):535-543. [CrossRef] [Medline]
  63. Noar SM, Black HG, Pierce LB. Efficacy of computer technology-based HIV prevention interventions: a meta-analysis. AIDS 2009 Jan 2;23(1):107-115. [CrossRef] [Medline]
  64. Collins A. Design issues for learning environments. In: Vosniadou S, De Corte E, Glaser R, Mandl H, editors. International Perspectives on the Design of Technology-Supported Learning Environments. Mahwah, NJ: L Erlbaum Associates; 1996:347-362.
  65. Bransford J, Sherwood R, Hasselbring T, Kinzer C, Williams S. Anchored instruction: why we need it and how technology can help. In: Nix D, Spiro R, editors. Cognition, Education, and Multimedia: Exploring Ideas in High Technology. Hillsdale, NJ: L Erlbaum Associates; 1990:115-141.
  66. Jonassen D, Howland J, Moore J, Marra R. Learning to Solve Problems with Technology: A Constructivist Perspective. 2nd Ed. Upper Saddle River, NJ: Merrill Prentice-Hall; 2003.
  67. Resnick LB. The 1987 presidential address: learning in school and out. Educ Researcher 1987 Dec 01;16(9):13-54. [CrossRef]
  68. Allport G. Attitudes. In: Murchison C, editor. Handbook of Social Psychology. Worchester: Clark University Press; 1935:798-844.
  69. Jonassen DH, Hernandez-Serrano J. Case-based reasoning and instructional design: using stories to support problem solving. Educ Technol Res Dev 2002 Jun;50(2):65-77. [CrossRef]
  70. Chiasson MA, Shuchat-Shaw F, Humberstone M. The Morning After. New York: The Steinhardt School of Education, New York University & Medical and Health Research Association; 2005.   URL: [accessed 2016-05-26] [WebCite Cache]
  71. Bandura A. Psychological Modeling: Conflicting Theories. New York: Chicago Liber-Atherton; 1974:1-64.
  72. Piaget J. Mechanisms of Perception. London: Rutledge & Kegan Paul; 1969.
  73. Schank R, Berman T. The pervasive role of stories in knowledge and action. In: Green M, Strange JJ, Brock TC, editors. Narrative Impact: Social and Cognitive Foundations. Mahway, NJ: Lawrence Erlbaum Associates; 2002:287-313.
  74. Wu Y, Stanton BF, Galbraith J, Kaljee L, Cottrell L, Li X, et al. Sustaining and broadening intervention impact: a longitudinal randomized trial of 3 adolescent risk reduction approaches. Pediatrics 2003 Jan;111(1):e32-e38. [Medline]
  75. Parsons JT, Golub SA, Rosof E, Holder C. Motivational interviewing and cognitive-behavioral intervention to improve HIV medication adherence among hazardous drinkers: a randomized controlled trial. J Acquir Immune Defic Syndr 2007 Dec 1;46(4):443-450 [FREE Full text] [Medline]
  76. Ahlberg T. HIV Big Deal. 2011. Ask me tell me   URL: [accessed 2016-05-26] [WebCite Cache]
  77. Kernan WN, Viscoli CM, Makuch RW, Brass LM, Horwitz RI. Stratified randomization for clinical trials. J Clin Epidemiol 1999 Jan;52(1):19-26. [Medline]
  78. Friedman L, Furberg C, DeMets D. Fundamentals of Clinical Trials. 3rd Ed. St Louis: Mosby-Year Book, Inc; 1996.
  79. Birnbaum MH. Human research and data collection via the internet. Annu Rev Psychol 2004;55:803-832. [CrossRef] [Medline]
  80. Khosropour CM. Retention and HIV testing of MSM enrolled in a 12-month online study. 2011 Presented at: Sex Tech Conference; April 1,2011; San Francisco, CA.
  81. Rosser BR, Oakes JM, Konstan J, Hooper S, Horvath KJ, Danilenko GP, et al. Reducing HIV risk behavior of men who have sex with men through persuasive computing: results of the Men's INTernet Study-II. AIDS 2010 Aug 24;24(13):2099-2107 [FREE Full text] [CrossRef] [Medline]
  82. Link MW, Mokdad AH. Alternative modes for health surveillance surveys: an experiment with web, mail, and telephone. Epidemiology 2005 Sep;16(5):701-704. [Medline]
  83. Perlis TE, Des Jarlais DC, Friedman SR, Arasteh K, Turner CF. Audio-computerized self-interviewing versus face-to-face interviewing for research data collection at drug abuse treatment programs. Addiction 2004 Jul;99(7):885-896. [CrossRef] [Medline]
  84. Elford J, Bolding G, Davis M, Sherr L, Hart G. Web-based behavioral surveillance among men who have sex with men: a comparison of online and offline samples in London, UK. J Acquir Immune Defic Syndr 2004 Apr 1;35(4):421-426. [Medline]
  85. Grey JA, Konstan J, Iantaffi A, Wilkerson JM, Galos D, Rosser BR. An updated protocol to detect invalid entries in an online survey of men who have sex with men (MSM): how do valid and invalid submissions compare? AIDS Behav 2015 Oct;19(10):1928-1937. [CrossRef] [Medline]
  86. Teitcher JE, Bockting WO, Bauermeister JA, Hoefer CJ, Miner MH, Klitzman RL. Detecting, preventing, and responding to “fraudsters” in internet research: ethics and tradeoffs. J Law Med Ethics 2015;43(1):116-133 [FREE Full text] [CrossRef] [Medline]
  87. Pinkerton SD, Holtgrave DR. A method for evaluating the economic efficiency of HIV behavioral risk reduction interventions. AIDS Behav 1998;2(3):189-201.
  88. Braithwaite RS, Meltzer DO, King JTJ, Leslie D, Roberts MS. What does the value of modern medicine say about the $50,000 per quality-adjusted life-year decision rule? Med Care 2008 Apr;46(4):349-356. [CrossRef] [Medline]
  89. Ryan C. Language Use in the United States: American Community Survey. Washington, DC: US Census Bureau; 2013.   URL: [accessed 2016-05-26] [WebCite Cache]
  90. Messeri P, McAllister-Hollod L, Irvine M. Validating self-reported HIV test results using surveillance registry data. CHAIN Report 2012-8 2013.
  91. Parsons JT, Halkitis PN, Bimbi D, Borkowski T. Perceptions of the benefits and costs associated with condom use and unprotected sex among late adolescent college students. J Adolesc 2000 Aug;23(4):377-391. [CrossRef] [Medline]
  92. Kalichman SC, Rompa D. Sexual Sensation Seeking and Sexual Compulsivity Scales: reliability, validity, and predicting HIV risk behavior. J Pers Assess 1995 Dec;65(3):586-601. [CrossRef] [Medline]
  93. Bimbi DS, Nanin JE, Parsons JT, Vicioso KJ, Missildine W, Frost DM. Assessing gay and bisexual men's outcome expectancies for sexual risk under the influence of alcohol and drugs. Subst Use Misuse 2006;41(5):643-652. [CrossRef] [Medline]
  94. Wolitski RJ, Flores SA, O'Leary A, Bimbi DS, Gómez CA. Beliefs about personal and partner responsibility among HIV-seropositive men who have sex with men: measurement and association with transmission risk behavior. AIDS Behav 2007 Sep;11(5):676-686. [CrossRef] [Medline]

ART: antiretroviral therapy
CAS: condomless anal sex
GBMSM: gay, bisexual, and other men who have sex with men
GPS: Global Positioning System
HIV: human immunodeficiency virus
IP: Internet Protocol
QALY: quality-adjusted life years
RCT: randomized controlled trial
STI: sexually transmitted infection
VoIP: voice over Internet Protocol

Edited by G Eysenbach; submitted 22.01.16; peer-reviewed by R W, L Kuhns; comments to author 18.02.16; revised version received 26.03.16; accepted 28.03.16; published 17.06.16


©Sabina Hirshfield, Martin J Downing Jr, Jeffrey T Parsons, Christian Grov, Rachel J Gordon, Steven T Houang, Roberta Scheinmann, Patrick S Sullivan, Irene S Yoon, Ian Anderson, Mary Ann Chiasson. Originally published in JMIR Research Protocols (, 17.06.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.