This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Human immunodeficiency virus (HIV) infections are increasing among young men who have sex with men (YMSM), yet few HIV prevention programs have studied this population. Keep It Up! (KIU!), an online HIV prevention program tailored to diverse YMSM, was developed to fill this gap. The KIU! 2.0 randomized controlled trial (RCT) was launched to establish intervention efficacy.
The objective of the KIU! study is to advance scientific knowledge of technology-based behavioral HIV prevention, as well as improve public health by establishing the efficacy of an innovative electronic health (eHealth) prevention program for ethnically and racially diverse YMSM. The intervention is initiated upon receipt of a negative HIV test result, based on the theory that testing negative is a teachable moment for future prevention behaviors.
This is a two-group, active-control RCT of the online KIU! intervention. The intervention condition includes modules that use videos, animation, games, and interactive exercises to address HIV knowledge, motivation for safer behaviors, self-efficacy, and behavioral skills. The control condition reflects HIV information that is readily available on many websites, with the aim to understand how the KIU! intervention improves upon information that is currently available online. Follow-up assessments are administered at 3, 6, and 12 months for each arm. Testing for urethral and rectal sexually transmitted infections (STIs) is completed at baseline and at 12-month follow-up for all participants, and at 3- and 6-month follow-ups for participants who test positive at baseline. The primary behavioral outcome is unprotected anal sex at all follow-up points, and the primary biomedical outcome is incident STIs at 12-month follow-up.
Consistent with study aims, the KIU! technology has been successfully integrated into a widely-used health technology platform. Baseline enrollment for the RCT was completed on December 30, 2015 (N=901), and assessment of intervention outcomes is ongoing at 3-, 6-, and 12-month time points. Upon collection of all data, and after the efficacy of the intervention has been evaluated, we will explore whether the KIU! intervention has differential efficacy across subgroups of YMSM based on ethnicity/race and relationship status.
Our approach is innovative in linking an eHealth solution to HIV and STI home testing, as well as serving as a model for integrating scalable behavioral prevention into other biomedical prevention strategies.
Clinicaltrials.gov NCT01836445; https://clinicaltrials.gov/ct2/show/NCT01836445 (Archived by WebCite at http://www.webcitation.org/6myMFlxnC)
In the United States, young men who have sex with men (YMSM) are the group most affected by the human immunodeficiency virus (HIV) and acquired immune deficiency syndrome (AIDS) [
Despite the burden of HIV among YMSM, few proven individual-level HIV prevention programs have been created specifically for this population [
To address the limited availability of EBIs for diverse YMSM, Keep It Up! (KIU!), an online HIV prevention intervention, was developed [
The overarching goal of the KIU! 2.0 project is to advance knowledge of technology-based behavioral HIV prevention, as well as improve public health by establishing the efficacy of an innovative electronic health (eHealth) prevention program for YMSM. We will accomplish these goals with three specific aims. First, we will integrate the KIU! intervention into a widely-used health technology platform to increase its scalability, adaptability, and potential for broad implementation. Second, we will test the efficacy of the KIU! intervention in a multisite RCT by (1) enrolling ethnically diverse HIV-negative YMSM (N=900; >65% ethnic/racial minorities) primarily in Atlanta, Chicago, and New York; (2) randomizing participants to either the KIU! intervention or an HIV knowledge control condition; and (3) measuring intervention outcomes at baseline and follow-up assessments at 3, 6, and 12 months.
The primary behavioral outcome will be the count of CAS acts, and the primary biomedical outcome will be incidences of sexually transmitted infections (STIs). Secondary behavioral outcomes include alcohol and drug use prior to sex, risky sex after substance use, condom errors, factors from the Information-Motivation-Behavioral Skills (IMB) theoretical model of HIV risk reduction [
This is a two-group, active-control, double-blinded RCT of the online KIU! 2.0 intervention. Participants are randomized into two groups in equal proportions, and are blinded to which group is the intervention of interest. Consent materials indicate that we are evaluating two versions of an online HIV prevention program. Study investigators and staff who have contact with participants for enrollment and retention activities are also blinded to the arm in which participants are enrolled. The KIU! intervention includes seven modules that are completed across three sessions, at least 24 hours apart, totaling approximately 2 hours of content. Across these modules, the KIU! intervention uses diverse delivery methods (eg, videos, animation, and games) to address HIV knowledge, motivate safer behaviors, teach behavioral skills, and instill self-efficacy for preventive behaviors. The intervention is available on desktop, laptop, and tablet computers. Due to the Adobe Flash components of the intervention, KIU! 2.0 is not available on mobile devices. An earlier version of the intervention (KIU! 1.0) that did not contain the enhanced booster content at 3- and 6-month follow-ups, has been reported [
All procedures performed in this study are approved by the Emory University, Hunter College, and Northwestern University Institutional Review Boards. Informed consent is obtained from all individual participants included in this study.
All interested participants are assessed for eligibility by completing a brief screener. Study inclusion criteria include (1) being between the ages of 18 and 29, (2) assigned male at birth and having current male gender identity, (3) receiving an HIV-negative test result from a study site or remote HIV testing, (4) reporting at least one act of CAS with a male partner in the prior 6 months, (5) not being in a behaviorally monogamous relationship lasting longer than 6 months, (6) having the ability to read English at an 8thgrade level, and (7) having an email address that can be used for research contact for retention purposes.
Participants are recruited from a variety of sources including (1) HIV testing clinics and mobile testing units of our partner CBOs in Atlanta, Chicago, and New York; (2) university-based HIV testing at research sites in Atlanta and New York; (3) local health department clinics in Chicago; (4) street outreach by university staff in Atlanta, Chicago, and New York; (5) local and national print, online, and telephone-recorded ads; (6) referrals from completed observational studies and research participant registries at the university sites; (7) referrals from CBOs not affiliated with the study who provide HIV testing; and (8) nationwide online ads on social media apps linked with remote, at-home HIV testing (see
Participants complete self-report assessments at baseline, immediately postintervention, and 3, 6, and 12-months postintervention. Assessments are completed via the Internet using a Computer-Assisted Self Interview. Participants are also mailed kits to collect urine and rectal swabs for STI testing at baseline and 12-month follow-up. Participants who test positive for an STI at baseline also provide samples for STI testing at the 3- and 6-month follow-ups, in addition to the 12-month follow-up.
In the context of a National Institutes of Health R34 grant, we collaborated with local CBOs to develop and pilot test KIU!, an interactive online HIV prevention project tailored to ethnically and racially diverse YMSM [
Recruitment strategies workflow, Keep It Up! 2.0.
The KIU! intervention includes seven modules completed across three sessions, completed at least 24 hours apart (ie, across at least 3 days), which total approximately 2 hours to complete. An innovative aspect of KIU! is that each module is based on a particular setting or situation that is relevant to the lives of YMSM, with developmentally appropriate health behavior change content embedded within each of these settings (see
Intervention modules, Keep It Up! 2.0.
Module | Style | Content | |
Session 1 | Healthy and Whole Person | Diverse peer videos | The first module welcomes and engages participants in the KIU! intervention. Diverse YMSM are interviewed on the streets of Atlanta, Chicago, and New York and discuss connections to family, community, and romantic partners for setting positive norms for condom use and obtaining support from family of origin and choice [ |
Hooking up |
Stylized animation with three scenarios | This animated module follows three diverse YMSM chatting online with a focus on identifying triggers for CAS. Embedded content focuses on the effects of mood on risk [ |
|
Session 2 | The Club Game | Virtual reality game | In this interactive game, participants address pros/cons of condom use, steps to correct condom use, consequences of excessive alcohol consumption or drug use, issues with presuming HIV status in others, and effects of sexual arousal on decision making [ |
Dating |
Illustrated story in Flash animation | The power dynamics between an older and younger man in a dating relationship are explored as well as how YMSM can assert healthy behaviors [ |
|
Session 3 | A Serious Relationship | Illustrated story in Flash animation and scripted scenarios on video | An illustrated story about dating considers ways to get sexual, emotional, and health needs met in relationships and how ongoing condom use can be an important aspect of that. The module also includes a video of a YMSM who receives an HIV-positive diagnosis while in a relationship. It wraps up with a video with actors portraying examples of good and bad communication about condom use. |
Setting Risk |
Health educator video and HIV prevention goals worksheet | Participants develop three realistic and practical goals based on topics covered in the intervention such as consistent condom use, regular HIV testing, and improving communication with partners. The purpose is to plan to engage in behaviors that preserve emotional, sexual, and physical health, and to troubleshoot obstacles to successful implementation of the goals. | |
Sex in the City | Scripted soap opera -style video | A diverse cast of YMSM highlights the risks in making assumptions about a partner’s HIV status or monogamy, the limits of serosorting in HIV negative YMSM, the importance of regular testing, and skills for negotiating condom use within relationships. The soap opera is divided into four short videos that are shown across multiple sessions of the intervention. Part 1 is shown in the first session, part 2 in the second session, and parts 3 and 4 in the third session. | |
3 month |
3 month |
Scripted |
A series of videos follow a young man named Antoine as he learns the importance of regular HIV testing and condom use after a condom failure due to incorrect use by his partner. Also included is video follow-up of a character from the “Sex in the City” soap opera who received an HIV negative test result and is working to maintain his risk reduction strategies. Participants are also given information about pre-exposure prophylaxis and other biomedical prevention strategies in various formats (video, fact sheet, and embedded Twitter feed). At the end of the booster, participants have a chance to revisit intervention modules and goals, troubleshoot obstacles to meeting goals, and set new goals or re-affirm existing ones. |
6 month |
6 month |
Scripted |
A series of videos follow Antoine as he navigates the dating scene before entering a serious relationship in which stopping condom use is discussed. In addition, participants have a chance to revisit 3 month booster content and goals, troubleshoot obstacles to meeting goals, and set new goals or re-affirm existing ones. |
The control condition contains the same number of modules as the KIU! intervention condition, with the same requirement to participate across three sessions. Participants in the control condition also complete follow-up assessments and STI testing at the same time points as those in the KIU! condition. The control content reflects HIV information that is currently available on many websites, with the aim to understand how the KIU! intervention improves upon what is currently available online. Information on transmission, treatment, and prevention is provided through static slides with text and images. The control condition is didactic, not tailored to YMSM, noninteractive, and focuses on HIV/STI knowledge. Modifications were made to the control arm prior to the launch of KIU! 2.0 to include facts about biomedical prevention strategies. The use of this approach as a control condition ensures that both groups have equivalent access to the Internet for HIV-related content.
In the current study, there are two booster sessions paired with follow-up assessments for both the intervention and control arms. These sessions occur at 3- and 6-month time points. At all follow-up time points, data collection occurs prior to booster session content, to prevent any effect on participant responses. The content provided at each follow-up varies by study arm.
The 3- and 6-month booster sessions reinforce learning from the intervention and provide additional HIV prevention information. The 3-month booster for the KIU! intervention focuses on the importance of repeat HIV testing, following the CDC’s recommendation of twice annual HIV testing among high-risk MSM [
At the 3- and 6-month follow-up sessions, participants review content from the control modules. The slides on HIV information displayed in the initial modules are rearranged and administered at the 3-month follow-up. Slides with information on biomedical prevention strategies such as preexposure prophylaxis (PrEP), microbicides, and male circumcision are also included at the 3-month follow-up. The slides with STI information are rearranged and administered at the 6-month follow-up. Similar to the intervention condition, only the study measures are administered at the 12-month follow-up.
To assess eligibility for participation in KIU! 2.0, individuals who are recruited online are mailed the FDA-approved, at-home, oral fluid OraQuick HIV test kit (see
At-home HIV testing workflow, Keep It Up! 2.0.
To enroll in the study, potential participants are mailed at-home urine and rectal swab sample collection kits in a nondescript box to test for urethral and rectal gonorrhea (NG) and chlamydia (CT) at baseline. Easy-to-understand instructions for collecting and returning the samples are provided with the kits. In addition to the written instructions provided with the rectal STI kit, a video with instructions for properly collecting the rectal samples is shared with participants. The protocol for diagnostic testing of STI samples has changed as the study has progressed. Initially, the biotechnology company Identigene tested urine samples, while Emory University tested rectal swabs, for NG and CT. Both laboratories used the Nucleic Acid Amplification Test (NAAT) method, which is the gold standard method of diagnostic testing. As of March 2012, the CDC Division of Sexually Transmitted Diseases (STDs) Prevention laboratory provides diagnostic testing of the test kit samples using the NAAT method. Participants mail the kit to the CDC lab using prepaid boxes provided by the study. After STI test results are received by the KIU! 2.0 study team from the CDC, they are delivered to potential participants using a secure, encrypted email. To open the email and access their results, individuals must enter the unique study identification number that is provided to them with their test kit. Participants can print a hard copy of their results, and may speak to research staff if they wish. If positive STI test results arise, study staff provide local referrals for free or low-cost treatment and make a legally required confidential report to the appropriate health department. Across all stages of the study, the RAs prompt participants to access their results if they have not been viewed within 14 days of being made available. If a participant does not access his results after this reminder, the RA calls or sends additional reminders every 7 days. A minimum of three attempts at contact are made for both the reminders to return kits, and to access test results. If a participant does not respond to these attempts, the RA makes additional attempts for the duration of the study (as feasible) unless the participant explicitly asks to no longer be contacted. STI testing at follow-up follows the same protocol.
The online KIU! intervention was integrated into the online Web-based patient reported outcome (PRO) platform, Assessment Center (AC) [
All participant tracking and retention activities are centrally managed at the lead site, Northwestern University, where the tracking technology is based. A supplemental database housed on REDCap, an online application, is used to log staff contact with participants and participant progress in the study. In consideration of difficult-to-reach participants across study sites, the Atlanta- and New York-based research staff members also assist with tracking and retention activities. The belief is that participants will be more responsive to contacts made from local sites, especially if they were recruited from these sites.
Upon enrollment, participants are randomly assigned by the online program (AC) to receive the KIU! intervention or HIV knowledge control arm. Participants do not know which group is the intervention under evaluation. Study investigators are blinded to the arm in which participants are enrolled. Randomization was performed using 6 permuted blocks of size 4, and stratified by race and HIV testing site at baseline [
Participant workflow, Keep It Up! 2.0.
A total of 2984 potential participants have been screened across all recruitment sources. Of those screened, approximately half were eligible, and 901 participants were enrolled to make up the final study sample (see
Demographic characteristics of enrolled Keep It Up! 2.0 participants.
Characteristics | n (%) | |
Total | 901 | |
White | 330 (36.6) | |
Latino | 260 (28.9) | |
Black | 219 (24.3) | |
Other | 92 (10.2) | |
Gay | 777 (86.2) | |
Bisexual | 104 (11.5) | |
Other | 20 (2.2) | |
Serious relationship | 175 (19.5) | |
Casual dating | 223 (24.8) | |
Not in a relationship | 501 (55.7) | |
Catholic | 154 (17.1) | |
Protestant | 102 (11.3) | |
No religious affiliation | 408 (45.3) | |
Other (eg, Jewish, Muslim) | 237 (26.3) | |
High school or less | 113 (12.5) | |
Some college | 252 (28.0) | |
College degree | 418 (46.4) | |
Graduate degree | 118 (13.1) | |
Yes | 328 (36.4) | |
No | 573 (63.6) | |
Full time | 451 (50.1) | |
Part time | 250 (27.8) | |
Unemployed | 199 (22.1) | |
Yes | 560 (62.2) | |
No | 341 (37.8) | |
Yes | 252 (30.0) | |
No | 589 (70.0) | |
Age, mean (SD) | 24.3 (2.9) | |
Length (months) of serious relationship, mean (SD) | 25.40 (28.6) |
Knowledge, motivation, skills (ie, partner sexual communication, correct condom use), and behavioral outcomes (ie, number of insertive and receptive CAS acts, condom errors) are measured at baseline and the 3-, 6-, and 12-month follow-up assessment time points. We measure intervention acceptability and tolerability immediately postintervention. Whenever possible, we selected measures designed for YMSM that were previously tested with diverse populations, to minimize cultural bias and maximize sensitivity and comparability to other studies. We follow participants for 12 months to assess behavioral outcomes far enough postintervention to allow for the potential occurrence of risk behaviors and HIV testing. This assessment plan also allows us to model possible degradation of treatment effects over time, and to assess outcomes 6 months after the final booster session, which meets CDC criteria for being classified as a tier I best-evidence HIV prevention program [
The
The H-RASP measure includes a subset of questions specific to alcohol and drug use prior to sex, and is used to assess substance use as a risk factor for CAS. Substance use is being assessed as a risk factor because YMSM, in comparison to their heterosexual counterparts, are more likely to use a variety of different substances (including alcohol and illicit drugs), to initiate drug use at an earlier age, and to experience more rapid increases in substance use over time [
To assess biomedical outcomes, urine and rectal samples are tested for NG and CT with the FDA-cleared Gen-Probe APTIMA Combo 2 Assay. All participants are tested at baseline and at the 12-month follow-up. Participants who test positive for an STI at baseline are also tested at the 3- and 6-month follow-ups. We test for both urethral and rectal NG and CT, as recent research shows rectal infections to be just as common, if not more so, than urethral infections, particularly among MSM of color [
The
The
The
The
We use standard measures of age, ethnicity, education, and socioeconomic status. For YMSM, we use tailored items for gender identity, sexual orientation identity, and anatomic sex at birth.
The
The study team modified this measure at follow-up to better reflect PrEP use after it became FDA approved. For example, the baseline PrEP measure that was programmed before FDA approval of PrEP asks, “How many times have you taken anti-HIV medications?” under the assumption that participants might have been receiving PrEP inconsistently, as it was not readily available to most of the population. This question was removed in the follow-up assessments. New questions such as, “On a typical week, how many days did you miss taking your medication?” were added to the follow-up measure to reflect that participants who now take PrEP likely have a prescription for the medication, and to reflect the importance of assessing adherence.
The
Univariate summary statistics will be computed for all potential covariates. These summary statistics will be stratified by treatment arm, and then compared statistically through tests of two independent binomial proportions for binary variables, and two-sample t-tests for continuous variables to assess a failure of randomization. A Cochran-Mantel-Hanzel test of two independent binomial proportions will be used for the primary outcome measure of incident STIs at the 12-month endpoint, stratified by race and site, and an analogous stratified test for the count of CAS acts. These tests will set Cronbach alpha at .05, two-sided, and unadjusted for risk factors, except for the strata variables (race, site) used in the experimental design of the study. Ordinary generalized linear models and quasi-likelihood will be used to model the primary 12-month efficacy endpoints while adjusting for potential risk factors. Generalized linear mixed models and generalized estimating equations for multiple correlated, longitudinal CAS measures will be used to estimate the time-averaged treatment effect and time trends using all follow-up outcome measures, while adjusting for other potential time-dependent risk factors. The same regression modeling procedures will be used for secondary outcomes, such as condom errors, IMB factors, and receipt of an HIV test. All statistical analyses will be performed under an intent-to-treat principle [
To address potential adverse effects of participants’ use of PrEP during the study, we will use methods of causal inference under Rubin’s causal model [
This evaluation of KIU!, a promising eHealth HIV prevention intervention for YMSM, is an important contribution to the field of HIV prevention for several reasons. To begin, while numerous funded studies regarding the Internet and HIV risk have been undertaken, there have been relatively few funded efficacy RCTs of HIV prevention eHealth projects, particularly among YMSM. Rates of HIV are on the rise among MSM in the period of
The KIU! intervention content and recruitment approaches also represent innovations in the field. Intervention content is based on the IMB theory of HIV risk behavior change [
Regarding participant recruitment, KIU! is unique in linking a behavioral HIV prevention project to a clinical encounter (ie, HIV testing) as one of its recruitment strategies. Currently, most testing clinics have limited time and resources to provide prevention resources. This approach produces innovative research on how to catalyze prevention by capitalizing on a key clinical encounter that could then be generalized to other biomedical strategies that require embedded behavioral prevention (eg, PrEP). Such an intervention could play an important role in providing accessible prevention for YMSM, particularly YMSM seeking HIV testing. This approach represents an opportunity to develop a cost-effective and easy-to-use intervention that will engage and motivate participants, while teaching risk reduction behaviors. Additionally, recruitment may be extended beyond the clinic setting to include more traditional recruitment efforts, such as community and street outreach and organization referrals, as well as increasingly common online advertising. As demonstrated in this study, recruitment of diverse YMSM from a variety of sources is feasible for online HIV prevention research. Documenting these efforts will produce research on differences in retention and risk profiles of YMSM recruited from a variety of sources.
Another important contribution to the field of HIV prevention is our approach to incorporate STI testing into an online HIV prevention project, primarily through remote self-testing. This approach is in response to calls to incorporate STI testing and treatment into HIV prevention efforts [
There are important limitations in considering the promise of KIU! 2.0 in its current form. The first limitation concerns access to the Internet for the delivery of online interventions. The Internet has become an important delivery approach for eHealth tools. Online interventions can be convenient for users as they are accessible from anywhere that there is a connection to the Internet. Additionally, online interventions can be used in private settings, which also improves accessibility and engagement without the fear of stigma, particularly for YMSM and other high-risk populations. Although the digital divide is narrowing, the promise of eHealth interventions may be limited for those without consistent and reliable Internet access. Second, issues related to the technology required to deliver and maintain eHealth interventions may serve as a study limitation. Currently, KIU! can only be accessed on laptops or computer tablets because content is not formatted for access on mobile phones. This factor may limit intervention access, particularly among subpopulations who primarily access the Internet via smartphones. Additionally, as with all Web-based applications, regular maintenance is required to ensure that the intervention is compatible with new and updated Web browsers, and to fix emerging bugs that impede participants’ ability to complete intervention sessions. Technical support and ongoing maintenance will present a financial challenge to future implementation after this trial is completed. Third, there is the challenge of deciding when and if to update eHealth intervention content during an ongoing RCT as new advances emerge (eg, PrEP) [
The overarching goal of KIU! 2.0 is to advance scientific knowledge of Internet-based behavioral HIV prevention, and improve public health by establishing the efficacy of an innovative eHealth prevention program for YMSM. This research is making significant progress towards achieving the specified aims. First, the KIU! technology has been successfully integrated into a widely-used health technology platform to increase its scalability, adaptability, and potential for broad implementation. Second, baseline enrollment for the RCT is complete (N=901) and we are currently assessing intervention outcomes (ie, count of CAS acts and STI incidence) via follow-up assessments at 3, 6, and 12 months. Finally, upon collection of all data, and after the efficacy of the intervention has been evaluated, we will explore whether the KIU! intervention has differential efficacy across subgroups of YMSM based on ethnicity/race, relationship status, and other variables. Our approach is innovative in linking an eHealth solution to HIV and STI testing, and serves as a model for integrating scalable behavioral prevention into other biomedical prevention strategies.
Participant recruitment for KIU! 2.0 is complete. Follow-up data is currently being collected and will be completed in early 2017.
Assessment Center
acquired immune deficiency syndrome
condomless anal sex
community-based organization
Centers for Disease Control and Prevention
chlamydia
evidence-based intervention
electronic health
Food and Drug Administration
human immunodeficiency virus
HIV-Risk Assessment for Sexual Partnerships
Information-Motivation-Behavioral Skills
Keep It Up!
men who have sex with men
Nucleic Acid Amplification Test
gonorrhea
preexposure prophylaxis
patient reported outcome
research assistant
randomized controlled trial
sexually transmitted disease
sexually transmitted infection
young men who have sex with men
This study was supported by a grant from the National Institute on Drug Abuse and National Institute of Mental Health (R01DA035145). We thank the CDC Division of Sexually Transmitted Diseases Prevention laboratory for their analyses of the urine and rectal samples collected for this research. We also thank Identigene and Lars Mouritsen for their input in the development of the STI-tracking and reporting protocols, as well as providing at-home urethral STI testing services. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Drug Abuse, the National Institute of Mental Health, the National Institutes of Health, or the CDC.
None declared.