This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
In the United States, young men who have sex with men (YMSM) remain disproportionately affected by human immunodeficiency virus (HIV) and other sexually transmitted infections (STIs). Although routine HIV/STI screening is pivotal to the timely diagnosis of HIV and STIs, initiation of appropriate treatment, and reduced onward disease transmission, repeat screening is underused. Novel interventions that incorporate elements of games, an approach known as gamification, have the potential to increase routinization of HIV/STI screening among YMSM.
The study aims to test the hypothesis that an incentive-based intervention that incorporates elements of gamification can increase routine HIV/STI screening among YMSM in California.
The study consists of a formative research phase to develop the intervention and an implementation phase where the intervention is piloted in a controlled research setting. In the formative research phase, we use an iterative development process to design the intervention, including gathering information about the feasibility, acceptability, and expected effectiveness of potential game elements (eg, points, leaderboards, rewards). These activities include staff interviews, focus group discussions with members of the target population, and team meetings to strategize and develop the intervention. The final intervention is called Stick To It and consists of 3 components: (1) online enrollment, (2) Web-based activities consisting primarily of quizzes and a countdown “timer” to facilitate screening reminders, and (3) in-person activities that occur at 2 sexual health clinics. Participants earn points through the Web-based activities that are then redeemed for chances to win various prizes during clinic visits. The pilot study is a quasi-experimental study with a minimum of 60 intervention group participants recruited at the clinics, at community-based events, and online. We will compare outcomes in the intervention group with a historical control group consisting of individuals meeting the inclusion criteria who attended study clinics in the 12 months prior to intervention implementation. Eligible participants in the pilot study (1) are 18 to 26 years old, (2) were born or identify as male, 3) report male sexual partners, and 4) have a zip code of residence within defined areas in the vicinity of 1 of the 2 implementation sites. The primary outcome is repeat HIV/STI screening within 6 months.
This is an ongoing research study with initial results expected in the fourth quarter of 2017.
We will develop and pilot test a gamification intervention to encourage YMSM to be regularly screened for HIV/STIs. The results from this research will provide preliminary evidence about the potential effectiveness of using gamification to amplify health-related behavioral change interventions. Further, the research aims to determine the processes that are essential to developing and implementing future health-related gamification interventions.
Clinicaltrials.gov NCT02946164; https://clinicaltrials.gov/ct2/show/NCT02946164 (Archived by WebCite at http://www.webcitation.org/6ri3G4HwD)
New and innovative strategies for human immunodeficiency virus (HIV) infection prevention are urgently needed to increase the uptake of health services among men who have sex with men (MSM). A growing body of evidence suggests that, under the right circumstances, financial incentives can increase the demand for HIV screening, change short-term sexual behavior, enhance linkage to care after HIV diagnosis, and promote adherence to antiretroviral therapy [
We hypothesize that gamification for HIV and sexually transmitted infection (STI) prevention may work well to reengage and motivate a new generation of MSM in a cost-effective and sustainable way. A focus on young men who have sex with men (YMSM) is particularly warranted because this population remains disproportionally affected by HIV infection. Although HIV diagnoses among gay and bisexual men have stabilized in recent years, YMSM continue to experience the greatest burden of HIV compared with any other group in the United States, with young men 13 to 24 years of age accounting for 27% of new diagnoses among all gay and bisexual men [
To test the potential of gamification for HIV/STI prevention, our 2-year study developed and will pilot Stick To It, an HIV/STI prevention intervention for YMSM (18-26 years of age) that incorporates elements of gamification. The objective of the intervention is to increase repeat HIV/STI screening, defined as screening at least every 3 months. HIV/STI screening is critical both as the gateway to HIV/STI treatment, which can lead to reduced onward transmission, and as a critical first step to access prevention strategies such as preexposure prophylaxis [
The study consists of 2 distinct phases. The first phase is the formative research phase, in which we used focus groups, structured interviews, and rapid prototyping of intervention elements to determine the most effective and appropriate design for the intervention. The second phase consists of the implementation of the intervention in a controlled research design. First, we present the methodology of the formative phase and the insights gained from the different elements therein, leading to the final design of the intervention. Second, we present the protocol for the implementation phase.
Our intervention was implemented at 2 sexual health clinics, 1 in northern California (Oakland), and 1 in southern California (Hollywood). Phase 1 formative research took place at the 2 clinics and at meeting locations on a university campus from November 2015 to July 2016. Our goal in phase 1 was to design Stick To It, an intervention incorporating gamification intended to motivate increased repeat HIV/STI screening among YMSM. Gamification, “the use of game design elements in non-game contexts,” [
Because the formative phase of gamification design involves an iterative process of gathering information about the effectiveness of game elements and refining the intervention, we simultaneously present both the design of the formative phase of our study and the results generated by each stage of the formative research.
Theory of change for how a potential gamification intervention may facilitate improved health outcomes among young men who have sex with men (YMSM). HIV: human immunodeficiency virus; STI: sexually transmitted infection.
The phase 1 research activities consisted of 4 components: (1) 10 strategy and development meetings of the entire project team, consisting of the research team, clinic staff, and a gamification consultant from the private sector; (2) 4 in-depth interviews with clinic staff about the clinic and its population; (3) 11 focus groups with 29 members of the target population (YMSM, 18-26 years of age); and (4) a series of weekly meetings of a smaller design team, consisting of 3 members of the research team and 1 of the site coordinators.
We held 10 meetings of the entire project team. Participants were the research team, including the project manager, the 2 site-specific project coordinators, the regional coordinators and research director of the implementation partner, and the gamification consultant. At the initial meetings, the emphasis was on broad game design-related issues such as the theme and aesthetics of the intervention design. Subsequent meetings consisted of extensive discussion of the game elements to be included, such as how to use points; whether to use a leaderboard; the frequency of participant interaction with the intervention; how much interaction between participants to include, if any; when or if to incorporate prizes; and how to use chance elements to determine prizes or other outcomes. Over time, these discussions were increasingly informed by the focus group findings. An important goal of these discussions was to integrate (1) extensive testing of game elements among the target population to determine the most suitable, engaging, and effective intervention design, as this is the cornerstone of gamification [
We conducted structured in-depth interviews with 4 staff members from both implementation sites. Interviewees were purposely selected to include testing counselors and clinic managers. The primary purpose of the interviews was to assess the kinds of game elements that would be feasible, appropriate, and best suited to increase engagement between clinic visits based on staff knowledge of the institutional setting and target population. The primary game elements discussed were theme and core game mechanics. The potential core game mechanics that we initially considered were (1) games of chance (eg, dice or spin wheels, gumball machines) and a system of points awarded for achievement of tasks and goals within the intervention, (2) prizes awarded based on accumulated points and outcomes of the chance elements, and (3) leaderboards that display the relative ranking of participants’ points or accomplishments and are used to create an element of competition. We used the insights gained from the staff interviews to develop the set of potential themes, core mechanics, and game elements presented at the initial focus groups. We also asked staff members to briefly pilot the games of chance in the clinic and report their perspectives about the feasibility and acceptability of each approach.
After staff interviews, we conducted an initial set of 4 focus groups with the target population in both implementation areas. These discussions followed a standard structured format designed to elicit detailed information about preferences and characteristics, with particular focus on use of smartphones and Internet technology, as well as experiences with different types of games. Special emphasis was placed on issues of concern to race and ethnic minorities and the lesbian, gay, bisexual, transgender, questioning community. In addition, we tested the themes and game mechanics that emerged from the staff interviews with focus group participants to determine those that would be most appropriate and effective.
We conducted 7 additional focus groups, which focused more narrowly on the specific game elements of the intervention and followed a progression of increasing detail and specificity of game elements as the design and content of the intervention evolved. This stage of the formative research was intended to implement the iterative game design testing process, an important component of game testing [
Among the 11 focus groups held, a total of 29 participants attended, with between 2 and 9 participants per group session. The mean age of participants was 24 years old. The self-identified race/ethnicity of focus group participants was as follows: American Indian/Alaskan native: 1 (3%); Asian/Pacific Islander: 3 (10%); black/African American: 4 (14%); Hispanic/Latino: 10 (34%); white: 9 (31%); mixed or other race/ethnicity: 2 (7%).
Focus group results: key insights and decisions about game elements to be used in the Stick To It intervention for young men who have sex with men (YMSM) in California, 2015-2016.
Game element | Key insights from focus group discussions | Design decisions for Stick To It |
Theme | Prefer contemporary to retro themes. |
Base theme on experiences and interests of YMSM. |
Reminders to screen at self-scheduled time | Reminders need to be fun and rewarding. |
Send communications via SMS. |
Web-based activities to accompany reminders and earn points | Prefer polls and quizzes to games. |
Use multiple-choice quizzes, with points for answering questions, and additional points for correct answers. |
Chance element for awarding prizes for screening | Guarantee participants always win at least a small prize as a means to deflect from the anxiety and stress associated with screening encounters. |
Use gumball machine at the clinic as a game of chance. |
aSMS: short message service.
The smaller design team held a series of weekly meetings. The design team iteratively redesigned components of the intervention on the basis of the results of the other formative research components. Each meeting of the design team generated a revised set of game elements, which were then taken into the subsequent focus groups and strategy and development meetings for consideration.
The final intervention consists of 3 components: online enrollment, Web-based activities, and in-person activities that occur at the clinic. Participants earn points through the Web-based activities, which they then redeem for a chance to win prizes during clinic visits. All content was developed prior to the implementation phase of the intervention.
In the first component, participants go through an online enrollment process and introduction to the intervention. The enrollment process consists of answering study eligibility questions, providing consent to participate in the study, and answering basic demographic questions. The introduction to the intervention consists of reading a short tutorial, inputting the date of last HIV/STI screening—used to set the countdown timer for the next recommended screening—and being invited to answer a 5-question multiple-choice quiz on a topic related to sexual health. Quizzes include questions that test knowledge of sexual health information that members of the target population may value (eg, STI symptoms, treatment, transmission risk, and prevention strategies) and questions primarily intended to be humorous or whimsical. Participants earn points for each step of this process (except for providing consent).
The second component, Web-based activities, occur on the intervention website (stick2it.org) and are intended to accomplish two primary goals. The first is to provide multiple reminders of the approaching quarterly screening date, which is accomplished through the prominent display of the countdown timer on the personalized webpage dashboard using a graphic presentation that was designed as part of the aesthetic theme of the intervention. The second is to provide participants the opportunity to accumulate points, which increases their chance of winning more valuable prizes at the clinic, thus increasing their motivation to receive quarterly HIV/STI screening. The online activities are primarily triggered by a series of short message service (SMS) text messages, inviting participants to visit the intervention website and complete various activities, such as taking the latest quiz, viewing their screening countdown timer, and inviting friends to join the intervention. Messages are sent with increasing frequency as the participant approaches their quarterly screening date, beginning with a 3-week interval, increasing to a 1-week interval. To complete each quiz, participants click a link in the SMS message directing them to their home page on the intervention website, where participants can also view their countdown timer. Points are awarded for answering the quiz questions, with additional points awarded for answering correctly. Participants are provided with an opportunity to take a new quiz every 3 weeks throughout the course of the intervention. In addition, participants earn points when they invite eligible friends who subsequently enroll in the intervention. Invited friends must meet all participant inclusion criteria.
The third component of the intervention involves the game of chance and takes place at the clinic. Participants also receive additional points for visiting the clinic for screening. Participants’ accumulated points are redeemable for prizes only at the clinic (for any reason, including screening or treatment). The redemption process works as follows. Gumball machines are located at each clinic. This was both the game of chance overwhelmingly preferred by the target population through our focus group testing and the most feasible to implement at the clinics. During their clinic visit, participants can draw between 1 and 5 gumballs, depending on their accumulated points. The number of points earned corresponds with the number of gumball draws available to the participant. The color combinations of the gumballs that are drawn determine the prize received. Points redeemed for gumball draws are deducted from the participant’s accumulated points at the time of redemption. The system of points and prizes was designed to result in an average prize cost of US $5 per screening visit. After screening, the countdown timer is reset for the next quarterly screening date.
Schematic of final intervention design. HIV: human immunodeficiency virus; STI: sexually transmitted infection.
Phase 2 will take place from October 2016 to June 2017 and will address the following aims: (1) evaluate the preliminary effectiveness of the Stick To It intervention on repeat HIV/STI screening in a pilot study, (2) evaluate the acceptability of and participant engagement with the Stick To It intervention, (3) evaluate the feasibility and cost of implementing an effectiveness evaluation on a larger scale, and (4) evaluate the cultural competence of the intervention as designed and implemented.
Our study targets individuals who (1) are 18 to 26 years old, (2) were born or identify as male, (3) report male sexual partners at the time of enrollment, and (4) have a zip code of residence within defined areas in the vicinity of 1 of the 2 implementation sites. We selected YMSM aged 18 to 26 years as inclusion criteria for this pilot study given the substantial variations in the intervention design that would be required to include adolescent-aged YMSM. Additionally, a minimum inclusion age of 18 years will ensure that participants can provide independent consent to participate in the intervention.
Participants will be recruited through three main channels.
Clinic staff will identify eligible individuals who visit the clinics for screenings, treatment, or other purposes and will inform them of the study and invite them to participate. In addition, placing flyers in the clinics and displaying the gumball machines in examination rooms will facilitate conversations about the intervention with potential participants. Those who are interested will be given a handout directing them to the intervention website, where they will be invited to complete an eligibility questionnaire and, if eligible, provide consent to participate in the study. Those with a smartphone will be invited to sign up while still at the clinic. Those who are eligible and provide consent will enter the study.
We will place flyers in bars, other clinics, on mobile testing vans, and at selected community events. The advertisements and flyers include links to the intervention website, where individuals are invited to take the eligibility questionnaire and provide consent.
We will post advertisements on social networking platforms such as Facebook, Twitter, Instagram, Craigslist, and Grindr (a geolocation-based social networking app used by YMSM).
Owing to potential spillover effects between the treatment and control groups within local peer groups, we deemed a randomized design to be impracticable. Instead, our study will use a quasi-experimental study design with a historical control group consisting of eligible participants identified within the medical records of the same clinics over the 12 months prior to the intervention phase of the study. We preregistered the study (clinicaltrials.gov NCT02946164). All eligible individuals recruited at the clinics, at community-based events, and online will be assigned to the treatment group. We will recruit a minimum of 60 participants and a maximum of 200 participants into the treatment group and identify an identical number of individuals to serve as a historical control group. To our knowledge, this will be the first intervention using gamification to target YMSM for repeat HIV/STI screening. Accordingly, we chose a large sample size range as the upper bound, as it is not possible for us to predict what the demand will be for such an intervention among the target population.
Given the pilot nature of the study, the proposed analyses will likely be underpowered for formal testing purposes. Nevertheless, if we assume that half of the men in the comparison group will attend 1 or more repeat HIV/STI screening visits over 6 months of follow-up, with 60 men in each group, we will have 80% power to detect a minimum change of at least 27 percentage points (from 50% to 77%), which is comparable to the increase observed in an incentive-based intervention previously implemented by the community-based organization that operates the 2 clinics in our study [
For the primary outcome of repeat HIV/STI screening, we will use routinely collected client data from the clinics’ electronic patient tracking and medical record system. A set of quantitative and qualitative secondary outcomes will be used to determine the acceptability and feasibility of Stick To It, and whether the intervention is suitable for a future effectiveness study on a larger scale. These implementation outcomes are as follows. First is the ability to recruit the study population: the percentage of eligible men approached who are willing to enroll and whether we can fully enroll the cohort within 6 weeks of study initiation. Second is participant engagement with the intervention: among Stick To It participants, we will determine (1) the proportion who successfully invited 1 or more other eligible individuals to the intervention, (2) the number of completed quizzes, and (3) the average number of quizzes participants completed. Third is cost: we will track the incremental cost of the program, relative to standard of care, which we will identify using the ingredients approach, in accordance with guidance from the World Health Organization [
For each participant, we will collect data for a period of 6 months following their entry into the study. The primary outcome is repeat HIV/STI screening, defined as the number of screenings participants receive over the 6-month observation period, an indicator of retention in the program and its effectiveness.
In addition, we will collect qualitative information about the experiences of participants and clinic staff, described below. We will use the following methods to collect our secondary outcomes.
A survey offered to all participants at the end of the study period will assess HIV/STI screening at other clinics (self-reported), barriers and facilitators to screening, and perceptions of the Stick To It intervention.
We will conduct up to 10 in-depth interviews with members of the clinic staff at the end the study period. We will gather detailed information on their perceptions of the intervention, including implementation challenges and successes, adaptability and integration with clinic operations, perceptions of effectiveness, and suggestions for future improvements.
We will also conduct in-depth exit interviews with up to 40 Stick To It participants from both locations. Our goal is to gather detailed information about whether the intervention was relevant, motivating, and culturally competent. Further, we hope to assess the level of satisfaction with the process of earning and redeeming points, the rewards used, and the program’s potential applicability to others. We will purposefully select a diverse group of men from different racial and ethnic groups and men who had different levels of engagement with the intervention. Topical areas will include the participant’s perception of HIV/STI risk, perceptions about and experience with HIV/STI screening, willingness to discuss sexual health with others, particularly motivating elements of the game, and any unexpected challenges or adverse events.
We will examine whether Stick To It was culturally appropriate, relevant, and nonstigmatizing for YMSM, with an emphasis on African American and Latino YMSM, who are disproportionately affected by HIV [
In-depth interviews will be conducted in English by trained staff and will follow standard procedures [
Summary of study design, screening and recruitment process, and data collection.
For quantitative outcomes, we will first assess missing data, describe participant characteristics using frequency tables, and provide descriptive statistics (eg, means, standard deviations, medians, ranges, proportions) for the primary and secondary outcomes stratified by group. For the binary outcomes, we will compare the intervention and comparison groups by computing unadjusted and adjusted prevalence ratios and 95% confidence intervals using generalized linear models, such as log-binomial regression [
Qualitative data analysis will be conducted with ATLAS.ti (Scientific Software Development GmbH) following standard procedures [
This is an ongoing research study with initial results expected in the fourth quarter of 2017.
The ongoing study is intended to explore the effectiveness of incorporating gamification into an incentive-based intervention to increase repeat HIV/STI screening among YMSM. Given that this population remains disproportionally affected by HIV and STIs, novel approaches that supplement proven engagement strategies may serve an important public health function. Namely, early HIV diagnosis, as achieved through repeat HIV screening, has been shown to facilitate timely linkage to medical care, initiation of antiretroviral therapy, and attainment of viral suppression, resulting in reduced transmission risk [
Given the success of nongamification interventions involving incentives, and considering the potential of gamification to harness and amplify intrinsic motivation for both short-term and long-term behavior change, we hypothesize that this approach may be particularly effective in increasing routine HIV/STI screening among YMSM. We emphasize that a fundamental principle of gamification is to enhance the motivational effectiveness of known behavior change mechanisms, such as reminders and incentives, and to amplify the existing motivation of individuals to accomplish desired behaviors. However, we do not expect stand-alone gamification interventions to replace or compete with existing types of interventions, but rather to enhance established strategies. Accordingly, we coupled gamification with an incentive-based structure for this very reason.
A secondary but important goal of this study is to explore the process of designing gamification interventions for health-related behavior change and to learn best practices for future intervention design. This is crucial, due to the fact that gamification interventions potentially require an extensive formative design process to determine the optimal set of game elements and intervention design required to incorporate such elements. One conjecture is that the success of any given health-related gamification intervention, and thus of health-related gamification more generally, may depend on highly context-specific and population-specific formative research. Thus, determining best practices for the formative phase is necessary to achieve the goal of being able to consistently and predictably design cost-effective interventions in a range of different circumstances. This paper, in addition to presenting the protocol of the study, is an attempt at documenting what formative research to develop a gamification-based intervention entails.
It is also important to recognize the limitations of this study. As previously discussed, we will use a historical control group for comparison. Although a randomized design has advantages, it was not practical because participants randomly assigned to the control group would have been exposed to some components of the intervention, such as recruitment materials posted in the clinic waiting and examination rooms (eg, comparison group contamination). Additionally, given the pilot state of the intervention, a cluster randomized design was neither warranted nor feasible. Although a historical comparison group will help us understand whether there is a preliminary signal of the intervention’s effectiveness and whether further study is warranted, it has important limitations. Most importantly, we will be unable to make causal statements about the effect of the intervention because we will be unable to adequately control for time trends between the intervention and comparison groups and the selection bias that may result from differences between the 2 groups. Nevertheless, a historical comparison group will still provide us with baseline data on repeat testing among YMSM in the same geographic region and therefore a benchmark to which we can make preliminary comparisons about the intervention’s potential effectiveness.
Stick To It points system/prize decoder.
Intervention website screenshots.
Recruitment tools (flyers).
Recruitment tools (video).
human immunodeficiency virus
men who have sex with men
short message service
sexually transmitted infection
young men who have sex with men
The authors thank the following individuals for their contributions to this work: Sagar Amin; Branden Barger; Dale Gluth, MA; Jamie Murkey, MPH; Sean Young, MS, PhD; Gabe Zichermann; and the Stick To It Community Advisory Board. Publication was made possible in part by support from the Berkeley Research Impact Initiative sponsored by the University of California, Berkeley Library. Funding for the study was provided by the National Institute of Mental Health (McCoy R34MH106359). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.
None declared.