Published on in Vol 8 , No 2 (2019) :February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/12164, first published .
Pilot Testing the Feasibility of a Game Intervention Aimed at Improving Help Seeking and Coping Among Sexual and Gender Minority Youth: Protocol for a Randomized Controlled Trial

Pilot Testing the Feasibility of a Game Intervention Aimed at Improving Help Seeking and Coping Among Sexual and Gender Minority Youth: Protocol for a Randomized Controlled Trial

Pilot Testing the Feasibility of a Game Intervention Aimed at Improving Help Seeking and Coping Among Sexual and Gender Minority Youth: Protocol for a Randomized Controlled Trial

Protocol

1University of Pittsburgh, Pittsburgh, PA, United States

2College of Health and Human Performance, University of Florida, Gainesville, FL, United States

3School of Psychological Sciences and Health, University of Strathclyde, Glasgow, United Kingdom

4Schell Games, Pittsburgh, PA, United States

5Yale University, New Haven, CT, United States

Corresponding Author:

Robert WS Coulter, MPH, PhD

University of Pittsburgh

3414 Fifth Avenue, Room 101

Pittsburgh, PA, 15213

United States

Phone: 1 716 523 8564

Email: robert.ws.coulter@pitt.edu


Background: Sexual and gender minority youth (SGMY; eg, lesbian, gay, bisexual, and transgender youth) experience myriad substance use and mental health disparities compared with their cisgender (nontransgender) heterosexual peers. Despite much research showing these disparities are driven by experiences of bullying and cyberbullying victimization, few interventions have aimed to improve the health of bullied SGMY. One possible way to improve the health of bullied SGMY is via a Web-accessible game intervention. Nevertheless, little research has examined the feasibility of using a Web-accessible game intervention with SGMY.

Objective: This study aimed to describe the protocol for a randomized controlled trial (RCT) pilot, testing the feasibility and limited efficacy of a game-based intervention for increasing help-seeking–related knowledge, intentions, self-efficacy, behaviors, productive coping skills use, and coping flexibility and reducing health risk factors and behaviors among SGMY.

Methods: We enrolled 240 SGMY aged 14 to 18 years residing in the United States into a 2-arm prospective RCT. The intervention is a theory-based, community-informed, computer-based, role playing game with 3 primary components: encouraging help-seeking behaviors, encouraging use of productive coping, and raising awareness of Web-based resources. SGMY randomized to both the intervention and control conditions will receive a list of SGMY-inclusive resources, covering a variety of health-related topics. Control condition participants received only the list of resources. Notably, all study procedures are conducted via the internet. We conveniently sampled SGMY using Web-based advertisements. Study assessments occur at enrollment, 1 month after enrollment, and 2 months after enrollment. The primary outcomes of this feasibility study include implementation procedures, game demand, and game acceptability. Secondary outcomes include help-seeking intentions, self-efficacy, and behaviors; productive coping strategies and coping flexibility; and knowledge and use of Web-based resources. Tertiary outcomes include bullying and cyberbullying victimization, loneliness, mental health issues, substance use, and internalized sexual and gender minority stigma.

Results: From April to July 2018, 240 participants were enrolled and randomized. Half of the enrolled participants (n=120) were randomized into the intervention condition and half (n=120) into the control condition. At baseline, 52.1% (125/240) of the participants identified as gay or lesbian, 26.7% (64/240) as bisexual, 24.2% (58/240) as queer, and 11.7% (28/240) as another nonheterosexual identity. Nearly half (113/240) of participants were a gender minority: 36.7% (88/240) were cisgender boys, and 16.3% (39/240) were cisgender girls. There were no differences in demographic characteristics between intervention and control condition participants.

Conclusions: Web-accessible game interventions overcome common impediments of face-to-face interventions and present a unique opportunity to reach SGMY and improve their health. This trial will provide data on feasibility and limited efficacy that can inform future Web-based studies and a larger RCT aimed at improving health equity for SGMY.

Trial Registration: ClinicalTrials.gov NCT03501264; https://clinicaltrials.gov/ct2/show/NCT03501264 (Archived by WebCite at http://www.webcitation.org/72HpafarW)

International Registered Report Identifier (IRRID): DERR1-10.2196/12164

JMIR Res Protoc 2019;8(2):e12164

doi:10.2196/12164

Keywords



Background

Sexual minority youth (eg, lesbian, gay, bisexual, or queer youth) and gender minority youth (ie, youth who identify as a gender different from their sex assigned at birth) experience myriad substance use and mental health disparities [1-27]. In particular, sexual orientation–related disparities have been known for over 20 years [1]: sexual minority youth, compared with heterosexual youth, have approximately 176% higher odds of cigarette use, 155% higher alcohol use, 34% higher heavy alcohol use (eg, binge drinking), and 56% higher marijuana use [2]. Currently, many of these sexual orientation–related disparities appear to be growing larger [3,4]. Similarly, compared with cisgender (ie, nontransgender) youth, gender minority youth have significantly higher use of cigarettes, alcohol, and marijuana [5,27]. These sexual and gender minority youth (SGMY) disparities are also present for novel substances, such as electronic cigarette use [5]. Regarding mental health disparities, SGMY have significantly higher anxiety, depression, and suicidality [20-27]. Meta-analyses show that sexual minority youth, compared with heterosexual youth, have 96% higher odds of having suicidal thoughts, 120% higher odds of making suicide plans, and 218% higher odds of making suicide attempts [20]. Altogether, these substantial and persistent health disparities make SGMY a priority population for interventions that attempt to reduce health inequities.

SGMY also experience disparities in bullying and cyberbullying victimization compared with their cisgender heterosexual peers [5,27-33]. For example, according to the 2015 Youth Risk Behavior Survey (YRBS), gay, lesbian, and bisexual adolescents (compared with heterosexuals) had nearly doubled the prevalence of bullying and cyberbullying victimization [33]. Importantly, research shows that these bullying disparities also contribute to SGMY disparities in substance use and mental health issues [5,13,27,34,35]. Therefore, interventions that help bullied SGMY and reduce bullying victimization may, in turn, reduce substance use and mental health disparities.

In addition to this greater prevalence of bullying, SGMY have unique factors that also contribute to health disparities. When SGMY are bullied, in addition to the typical fears of disclosing their bullying victimization experiences to others, they often fear having to disclose their sexual and gender minority (SGM) status to adults, thereby putting them at risk for further discrimination and harassment [36,37], likely preventing them from reaching out for help. Even when SGMY consider suicide, large proportions of them do not seek help [38], and sexual minority youth have more trouble than heterosexuals with identifying people to talk to about their emotional worries [39]. Moreover, compared with heterosexuals, sexual minority youth are more likely to use nonproductive coping strategies (eg, self-blaming, giving up, ignoring problems, and worrying) to manage the stressors in their lives [40,41], which likely exacerbates their health. Thus, interventions that aim to improve help-seeking behaviors and productive coping strategies (eg, solving problems, seeking relaxing diversions, and being physically active) among bullied SGMY may substantially reduce substance use and mental health disparities.

Notably, few interventions have been rigorously tested to examine whether they are efficacious in reducing substance use and mental health disparities among SGMY [42]. Moreover, 1 possible way to improve the health of bullied SGMY is via a Web-accessible (ie, downloadable via the internet) game intervention. Game interventions can be easily accessible through the internet, thereby providing an effective way to reach large numbers of SGMY, including SGMY living in rural areas and high structural stigma locations (ie, areas with less SGM-inclusive policies, institutions, and attitudes). Furthermore, Web-based game interventions are advantageous because some SGMY may be out to only a few people offline, making recruitment and attendance in face-to-face interventions for SGMY quite challenging [43,44]. Web-accessible interventions may be accessed by those who are insufficiently supported in offline programs [45-47]. In addition, Web-accessible gaming programs about sex [48,49], mental health [50], alcohol use [51-55], smoking [56-58], and asthma [59-62] are effective for youth in general. The Web-based environment is also relatively safe for LGB youth to gain coping skills [63,64]. Other advantages of Web-accessible interventions are increased fidelity and cost-effectiveness [65-71]. Overall, Web-accessible game interventions overcome common impediments of face-to-face interventions and present a unique opportunity to reach SGMY to improve their health. Little research, with few exceptions [72,73], has examined the feasibility of using a Web-accessible game intervention with SGMY populations. Therefore, pilot testing the feasibility of such a study is critical for successfully conducting a large-scale intervention. Feasibility comprises a wide range of topics [74], such as the testing of implementation procedures—how well the study was implemented as planned; intervention demand—the actual use of intervention among participants; intervention acceptability—how the participants react to the intervention; intervention integration—how well the game fits into the participants’ lives; intervention adaptation and expansion—the changes necessary for future iterations of the intervention and translation into new environments; and finally, limited efficacy testing—can be used to get estimates for variability and precision to power a larger randomized controlled trial (RCT). To adequately address the multiple dimensions of feasibility related to a game-based intervention tailored to SGMY, a carefully planned pilot trial is needed.

Study Aims

This paper describes the protocol for a pilot RCT assessing the feasibility and limited efficacy of the game-based intervention to increase help-seeking–related knowledge, intentions, self-efficacy and behaviors, productive coping skills use, and coping flexibility and reduce health risk factors and behaviors among SGMY. The primary, secondary, tertiary, and exploratory aims of the study are described below.

Primary Aim

Our primary aim is to evaluate implementation procedures for the RCT of the game-based intervention as well as to determine the level of game demand and game acceptability. We hypothesize having high implementation fidelity, game demand, and game acceptability (see Table 1 in the Methods section for specific targets).

Secondary Aim

Our secondary aim is to test the limited efficacy for the game in increasing the following short-term outcomes: help-seeking intentions, self-efficacy, and behaviors; productive coping strategies and coping flexibility; and knowledge and use of Web-based resources. We hypothesize that compared with the control participants, the intervention participants would have greater improvements in all short-term outcomes.

Tertiary Aim

Our tertiary aim is to test the limited efficacy for the game in reducing long-term outcomes: bullying and cyberbullying victimization; loneliness; mental health issues (ie, stress, anxiety, depression, and suicidality); substance use; and internalized SGM stigma. We hypothesize that intervention participants, compared with control participants, will have greater decreases in all long-term outcomes.

Exploratory Aim

We have built in an exploratory aim meant to better our understanding of participants’ responses to the game and research procedures and how to improve both. This exploratory aim concerns implementation procedures, integration, and the adaptation and expansion of the game. Given the exploratory nature of this aim, we have no a priori hypotheses.


Study Design

The purpose of this research study is to pilot a 2-arm RCT of a game-based intervention to improve help-seeking behaviors and productive coping strategies to reduce substance use, victimization, and mental health issues among SGMY. The study is led by a team with expertise in SGMY research at the Center for LGBT Health Research in the University of Pittsburgh’s Graduate School for Public Health, University of Pittsburgh (Principal Investigators: Friedman and Egan). We engage in team science by partnering with an expert in bullying research, help-seeking research, an interventionist, a biostatistician, a health and game researcher, and a professional game development company. The study is funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development at the National Institutes of Health (R21HD083561) and is registered as a clinical trial (NCT03501264) [75].

Study Population and Study Flow

All study procedures related to screening, consenting, surveying, and reminding are completed using REDCap, a free and secure Health Insurance Portability and Accountability Act-compliant website for managing Web-based surveys and databases. Study participant flow is depicted in Figure 1.

First, to determine eligibility, respondents completed a brief Web-based self-reported screening questionnaire before entering in this study. Respondents were eligible if they were English literate, lived in the United States of America, were aged between 14 to 18 years, had experienced bullying or cyberbullying victimization in the past year, had a sexual minority identity (ie, gay, lesbian, bisexual, or queer) or a gender minority identity (ie, considered themselves to be transgender or nonbinary), had a personal computer or Mac laptop or desktop computer where they could download games, and had an email address. Eligible respondents were then directed to a Web-based informed consent form. Participants voluntarily consented using a click-to-consent procedure. To protect participants from having to reveal their sexual or gender minority identities to their caregivers, thereby potentially putting them in harm’s way, we received a waiver of parental consent [76]. This allowed participants to self-consent. Ineligible respondents were thanked for their time and no additional contact was made.

Once eligible respondents agreed to voluntarily participate in the study, they were emailed a link to the baseline (T1) survey to complete. The T1 survey contained 24 pages with a median of 8.5 items per page (mean 10.3; range 3-26). After completion of the T1 survey, the study team randomized participants to intervention or control conditions (full descriptions of conditions are provided in later sections). Immediately after randomization, participants in the intervention condition automatically received an email with a REDCap survey link containing information that guided participants through the procedures for downloading and installing the game intervention onto their desktop or laptop computers with Microsoft Windows or Apple Mac operating systems. Every 3 days thereafter, participants were automatically sent up to 5 email reminders to download the game. Moreover, 1 day after randomization, participants in both the intervention and control conditions were automatically sent a list of resources related to study outcomes.

The first follow-up (T2) survey was activated 4 weeks after T1 survey completion and remained open for 4 weeks. The T2 survey was similar to the T1 survey; however, participants in the intervention condition who self-reported having played the game also completed questions about their gaming experience. The T2 survey contained 26 pages with a median of 8.5 items per page (mean 11.4; range 3-47).

The final follow-up survey (T3) is activated 8 weeks after T1 survey completion and remains open for 8 weeks. The T3 survey is similar to the T1 survey; however, intervention condition participants who do not complete the gaming experience questions at T2 but self-report having played the game in the T3 survey will be asked questions about their gaming experience at T3. The T3 survey contained 26 pages with a median of 8.5 items per page (mean 11.4; range 3-47).

For each survey, up to 5 email reminders to complete surveys are automatically emailed to participants every 4 days. While completing each survey, participants were able to change their answers by clicking a back button. The following incentives are given after the completion of each survey: US $10 for T1; US $25 for T2; and US $50 for T3. At the end of each survey, participants select if they wanted a gift card to Apple iTunes or Google Play.

Recruitment

Participants were conveniently sampled and recruited throughout the United States using website advertisements posted on social media platforms. This passive approach allowed SGMY from multiple geographic locations (eg, rural and urban, and East and West) to enroll in the study without overextending our limited resources. Facebook was our primary recruitment site. Facebook is an appropriate recruitment platform because it is highly utilized by adolescents; approximately 71% of teens use Facebook [77]. We created a formal side-bar Facebook ad (in-line ads were not used). We also recruited participants from Instagram using the same advertisements. We also recruited participants from SGM-related Web-based gaming groups, such as Geeks OUT, Gay Geeks (Facebook group), GaymerX (Facebook group), Transmission Gaming, and Reddit Gaymer forums. We advertised via Pitt+Me, a community hosted by the University of Pittsburgh comprising patients, volunteers, and researchers working together as partners in research and clinical trials to advance health care. Finally, we advertised on a dedicated Facebook page for the study. Only study team members were able to post comments on this page. An individual who was interested in participating clicked on the advertisement and was directed to the Web-based screening questionnaire. To ensure representation of both sexual and gender minorities, enrollment was monitored weekly. Specific ads were created and used to target underrepresented groups. Depending on the prior week’s enrollment numbers, we tailored which ads were used for the upcoming week.

Randomization

We used permuted block allocation (using blocks of several sizes) to randomize individual participants to the intervention or control conditions. The permuted blocks were created using the ralloc package for Stata. Randomization was performed in REDCap using the Randomization Module.

Control Materials

All participants randomized into the control condition received a list of national SGM-inclusive resources. This list included general lesbian, gay, bisexual, transgender, and queer (LGBTQ) resources (eg, GLAAD resource list), LGBTQ bullying resources (eg, The Trevor Project), general bullying victimization resources (eg, StopBullying.gov), child abuse resources (eg, Child Abuse Resource Center), dating violence resources (eg, National Domestic Violence Hotline), suicide and mental health resources (National Suicide Prevention Lifeline), substance use resources (eg, National Institute of Drug Abuse for Teens), LGBTQ homelessness (eg, True Colors Fund), and LGBTQ emergency hotlines (eg, Gay, Lesbian, Bisexual, and Transgender National Help Center Hotline). These materials were delivered via email the day after participants were randomized to the control condition. In addition, after completion of the final T3 survey, control participants were offered a free download of the intervention game. No additional follow-up was initiated to assess their game use, game satisfaction, or changes in outcomes.

Intervention Materials

Participants assigned to the intervention condition received a list of SGM-inclusive resources (the same as the control condition materials) and the game intervention. After randomization, intervention condition participants received an email with a link and instructions on how to download the game. Participants were asked to download the game to their computer to play it. The game intervention was based on empirical interviews with SGMY about their gaming preferences, undergirded by etiologic and behavioral change theories and built in collaboration with expert educational game developers.

Gaming Preferences of Sexual and Gender Minority Youth

Before developing the game, we conducted one-on-one, in-depth interviews with 20 SGMY about their gaming preferences (publication of these results is currently underway). Most SGMY enjoyed playing action-oriented games, and when asked what they liked about their favorite games, the most common response concerned the ability to personalize characters. Therefore, in the game, we incorporated selection of pronouns (Figure 2) and character customization, including skin color, hair, body type, and clothing (Figure 3). SGMY liked engaging, unique storylines, as well as challenging (but not too challenging) mini objectives and missions. Having a multiplayer aspect of the game that requires teamwork and interaction with others was also mentioned. We also incorporated these elements into the game.

Figure 1. 2-Arm randomized controlled trial design and data collection schedule.
View this figure
Figure 2. Game intervention name and pronoun selection.
View this figure
Figure 3. Game intervention character customization.
View this figure
Theoretical Underpinnings

The conceptual model of the game intervention (Figure 4) is based on 3 theories that inform the etiology and behavioral change strategies of our short-term and long-term outcomes. First, social cognitive theory [78,79] suggests that behavior change (eg, help-seeking behaviors) is facilitated by developing self-efficacy and skills. Self-efficacy and social skills can be achieved through behavioral rehearsal, witnessing outcomes of their choices, and feedback [80-82]. Second, stress and coping theory [83] suggests that specific types of appraisals predict different coping strategies. With bullying, youth who blame themselves, perceive little control, and view the situation primarily as a threat as opposed to a challenge are likely to engage in nonproductive coping [84-87]. On the other hand, identification of the problem, direct problem solving, and help seeking constitutes productive coping [88]. Third, the social and emotional learning framework [89] identifies 4 main competencies to promote positive health outcomes: awareness of self and others, responsible decision making, positive attitudes and values, and social interaction skills. We considered each of these theories during the process of developing the core game mechanics of the player experience.

Elements of these theories are embedded in the game in unique ways. First, the game encourages help-seeking behaviors by having players create a team with other nonplayable characters and to pair a lonely nonplayable character with an appropriate mentor (Figure 5). Second, the game encourages use of productive coping strategies through active listening and helping a nonplayable character overcome anger in a healthy way (Figure 6). Third, the game raises awareness of Web-based resources through collecting pages from a virtual notebook that contain information about external resources and bullying information (Figures 7 and 8).

Figure 4. Game intervention conceptual model.
View this figure
Figure 5. Interaction with a nonplayable character who is lonely.
View this figure
Figure 6. Interaction with a nonplayable character who is angry.
View this figure
Figure 7. Virtual notebook page about bullying.
View this figure
Figure 8. Virtual notebook page about the importance of making contact after being bullied.
View this figure
Game Development

The game was developed in collaboration with Schell Games, an education and entertainment game development company located in Pittsburgh, Pennsylvania, United States. The intervention game is a role-playing game inspired by Japanese role-playing games, which typically involve exploring a space, talking to nonplayable characters, and fighting enemies through turn-based battles. The game runs standalone on desktops and laptops with Windows or Mac operating systems. Before implementation, we user-tested the intervention using think-aloud interviews with 3 SGMY, who found technical bugs and recommended password protection. Prior research shows that user testing with 3 participants results in finding at least 60% of major usability problems [90]. Schell Games revised the game accordingly before the implementation of the RCT.

Game Play

In the game, the player takes on the role of a Singular, a superhuman individual with special gifts, who is located in a school. The player is told that because of their uniqueness, Singulars face prejudice, often driven by fear and misunderstanding. After customizing their character, the player is tasked with finding a team to help them complete their final mission, which is to become a world-class superhero by defeating the robots in the Holochamber Challenge. The player then explores the school, talking to potential peers (ie, nonplayable characters) who can join their team. Some peers deal with problems related to bullying, confidence issues, and anger, which prevent them from performing properly or from wanting to join the player’s team. The player’s mini objectives are to do the following for each of their peers: (1) best identify the nonplayable characters’ problems, (2) find the best individuals or resources to help the nonplayable characters, and (3) help the nonplayable characters properly communicate or utilize their newfound resources. If the player is successful and finds the best way to help the nonplayable characters, then the nonplayable characters will join their team or help them by giving them an item or ability that will help them achieve success in the final Holochamber Challenge (Figure 9). After the player finishes the Holochamber Challenge, the events that took place are evaluated. For every nonplayable character that is successfully helped, the player is given a positive ending; for example, “Thanks to Invisibella’s mentoring, Violet Phantom eventually felt comfortable to reveal herself as a Singular. Despite past events, a majority of her other friends and family were supportive. Knowing she was not alone and had support helped her no longer feel afraid.” For every nonplayable character that was not successfully helped, the player is given a negative ending; for example, “Gigaton didn't learn to control his anger in this round. One day he snapped and turned on those bullying him. Though no one was seriously injured, the outburst resulted in Gigaton getting in trouble with the Principal.” The players are then encouraged to replay the game and given hints to help them receive positive endings. Each player can unlimitedly replay the game.

Data Collection

We are collecting data in 2 distinct ways. First, we use self-administered surveys via REDCap, where participants completed surveys on their computer, tablet, or phone. Participants complete surveys at T1, T2, and T3 (see the section Study Population and Study Flow for more details). Surveys were created based on the Checklist for Reporting Results of Internet E-Surveys [91]. Second, we collect game play data from participants in the intervention condition. These game play data are transferred via a secure file-transfer-protocol system. Text files containing milestones achieved, time played, and player choices are automatically sent to a secure server housed at the University of Pittsburgh. Participants’ game play data are tracked using a unique identification number that does not rely on identifiable information (eg, IP addresses).

Outcomes

Primary Outcomes

To answer our primary aim, we will assess the following primary feasibility outcomes: success of the implementation procedures used in our RCT, game demand, and game acceptability. Table 1 details our primary outcomes, assessments, and investigator-generated hypotheses.

Success of our implementation procedures are assessed using a variety of measures. We hypothesize the following: 240 participants will enroll in our trial and complete the first survey, greater than or equal to 80% of eligible participants will consent to participate in the study, 120 participants will be randomized to the intervention condition and 120 to the control condition, greater than or equal to 80% of enrolled participants will complete the T2 survey, greater than or equal to 80% of enrolled participants will complete the T3 survey, and greater than or equal to 75% of participants will complete all surveys. Testing the feasibility of our implementation procedures is essential to inform the design of our future larger-scale RCT.

Figure 9. The player’s final mission: defeat the Holochamber Challenge with their teammates.
View this figure
Table 1. Primary outcomes, assessments, and hypotheses.
Primary outcomeAssessmentHypothesis
Implementation procedures

Study populationMeasured as the total number of participants who were consented and who completed T1a240 participants

Participation rateMeasured as the total number of people who agreed to participate in the study divided by the total number of people who were eligible to participate in the study≥80%

Number of randomized participantsMeasured as the total number of participants randomized240 participants

Randomization successAssessed by comparing intervention and control conditions across all demographic and potential confounding variables at baselineNo differences between intervention and control condition at baseline

Retention rate for T2bMeasured as the total number of participants who completed T2 divided by the total number of participants enrolled in the study≥80%

Retention rate for T3cMeasured as the total number of participants who completed T3 divided by total number of participants enrolled in the study≥80%

Retention rate in T2 and T3Measured as the total number of participants that completed both T2 and T3 surveys divided by the total number of participants enrolled in the study≥75%
Game demand

Game downloadAssessed on T2 and T3 surveys: did you download the game titled “Singularities”?—Yes; No; Unsure≥80% selected “Yes”

Any game playAssessed on T2 and T3 surveys: did you play the game titled “Singularities”?—Yes; No; Unsure≥80% selected “Yes”

Any game playTotal number people who played the game based on game play data from the secure file-transfer-protocol system divided by total number of participants randomized to intervention condition≥80% played

Total time of game playAssessed on T2 and T3 surveys: in the past month, about how long did you play the game “Singularities”?—I did not play the game; Less than 1 hour; 1 hour; 2 hours; 3 hours; 4 hours; 5 hours; 6 hours; 7 hours; 8 hours or more≥75% selected 1 hour or greater

Total time of game playThe number of hours the game was played based on game play data from the secure file-transfer-protocol system≥75% played 1 hour or greater
Game acceptability

Gaming experience [92]

Assessed on T2 and T3 surveys: response options used a 5-point Likert scale (range: 0-4)—Not at all, Slightly, Moderately, Fairly, Extremely; Question stem was “Please indicate how you felt while playing the game for each of the questions below”; Competence subscale —I felt skillful, I felt competent, I was good at it, I felt successful, I was fast at reaching the game’s targets; Sensory and imaginative immersion subscale —I was interested in the game’s story, it was aesthetically pleasing, I felt imaginative, I felt that I could explore things, I found it impressive, It felt like a rich experience; Flow subscale —I was fully occupied with the game, I forgot everything around me, I lost track of time, I was deeply concentrated in the game, I lost connection with the outside world; Tension and annoyance subscale (reverse coded)—I felt annoyed, I felt irritable, I felt frustrated; Negative affect subscale (reverse coded)—It gave me a bad mood, I thought about other things, I found it tiresome, I felt bored; Positive affect subscale —I felt content, I thought it was fun, I felt happy, I felt good, I enjoyed itMean scores ≥2 for each subscale separately

Desire to play game againAssessed on T2 and T3 surveys: I would like to play this game again—Strongly Disagree; Disagree; Neutral; Agree; Strongly Agree≥75% of intervention condition participants selected “Agree” or “Strongly Agree”

Likelihood to recommend game to friendsAssessed on T2 and T3 surveys: how likely would you be to recommend that your friends play this game?—Definitely; Very Probably; Probably; Possibly; Probably Not; Definitely Not≥75% of intervention condition participants selected “Definitely,” “Very Probably,” or “Probably”

aT1: baseline.

bT2: first follow-up.

cT3: final follow-up.

Game demand is assessed among the intervention condition participants in 2 ways, via downloading and playing the game. Download of the game is assessed via self-reported surveys (T2 and T3). We hypothesize ≥80% of intervention condition participants will download the game. Game play is assessed via self-reported surveys (T2 and T3) and game play data. We hypothesize ≥80% of intervention condition participants will play the game and ≥75% of intervention participants will play the game for 1 hour or greater.

Game acceptability among the intervention condition participants is assessed via self-reported surveys at T2 and T3. If participants reported playing the game, we ask about their overall impressions of the game using the Gaming Experience Questionnaire. The Gaming Experience Questionnaire is a multidimensional scale that assesses the following domains about the participants’ feelings and thoughts during the game: competence, sensory and imaginative immersion, game flow, game tension and annoyance (reverse coded), negative affect (reverse coded), and positive affect. We hypothesize that the mean score for each subscale will be greater than 2, representing a moderately good gaming experience. In addition, we ask participants if they would be interested in playing the game again, and whether they would recommend the game to their friends. We hypothesize ≥75% of participants will “agree” or “strongly agree” that they would like to play the game again, and ≥75% would “definitely,” “very probably,” or “probably” recommend the game to their friends.

Secondary Outcomes

To answer our secondary aim, we will measure the following short-term outcomes: help-seeking intentions, self-efficacy, and behaviors; productive coping strategy usage and coping flexibility; and knowledge and use of Web-based resources. All secondary outcomes are assessed via self-reported surveys at all time points. Table 2 details our secondary outcomes, items, response options, and coding procedures.

Help-seeking intentions are assessed using adapted version of the General Help Seeking Questionnaire [93]. In addition, 2 different sets of questions assess how likely participants are to seek help from a variety of sources about (1) emotional problems and (2) suicidal ideation. Help sources include a wide variety of people (eg, doctor and counselor) and places (eg, websites).

We adapted the original items by adding “Phone or text/chat help line (such as the National Suicide Prevention Lifeline or the Trevor Project)” and “teacher.” We will create 2 separate overall mean scores for how likely participants are to reach out concerning (1) emotional problems and (2) suicidal ideation. In addition, we will create a score for each help source individually. Higher mean scores indicate greater likelihood of seeking help from these sources.

Help-seeking self-efficacy is assessed using 2 subscales from Bandura’s Multidimensional Scales of Perceived Self Efficacy [94,95]: the enlisting social resources subscale and the enlisting parental and community support subscale. Each subscale has 4 items, which are used to calculate an overall mean score measuring how well participants think they can get support from different sources (eg, “How well can you get a friend to help you when you have social problems” and “How well can you get your parents to take part in school activities”).

Help-seeking behaviors are assessed using items adapted from the Help-Seeking Behaviors Scale [96]. We adapted the scale to include the wide variety of help sources used in the aforementioned help-seeking intention scales. Therefore, we created 8 different subscales (1 for each help source) assessed with 3 items each (eg, “In the past month, how often have you asked a teacher for help?”). Each subscale will be analyzed separately.

Coping skill usage is assessed using the Adolescent Coping Scale Second Edition Short Form [88]. This scale uses 18 items measuring 2 different dimensions: productive (problem solving) coping and nonproductive (passive avoidant) coping. We hypothesize that the game-based intervention (vs the control) will increase productive coping and decrease nonproductive coping.

Coping flexibility is assessed using the Coping Flexibility Scale [97], including 2 separate subscales: evaluation coping and adaptive coping. The evaluation coping subscale uses 5 items to assess how well a person monitors and evaluates coping outcomes (eg, “I am aware of how successful or unsuccessful my attempts to cope with stress have been.”). The adaptive coping subscale uses 5 items to assess how well a person uses an alternative coping strategy to produce a desirable outcome (eg, “When a stressful situation has not improved, I try to think of other ways to cope with it”). Each subscale will be examined separately.

Knowledge and use of Web-based resources are measured using 2 different scales developed by the investigative team. We created a list of SGM-inclusive resources that match what was provided in the game-based intervention and control materials, such as the Trevor Project. Use of Web-based resources is measured for the past month. We will create 1 summary score for knowledge and 1 for use; each summary score adds together all “yes” response options.

Table 2. Secondary outcomes, items, response options, and coding procedures.
Secondary outcomeItemsResponse optionsCoding procedure
Help-seeking intentions, self-efficacy, and behaviors

Help-seeking intentions for personal or emotional problems, adapted from [93]If you were having a personal or emotional problem, how likely is that you would seek help from the following people?—parent or guardian; other relative or family member; teacher; intimate partner (such as girlfriend or boyfriend); friend (someone not related to you); mental health professional (such as a psychologist, social worker, counselor); phone or text/chat help line (such as the National Suicide Prevention Lifeline or the Trevor Project); website help resources (such as StopBullying.gov); doctor or primary care provider; minister or religious leader (eg, priest, rabbi, chaplain); I would not seek help from anyone; I would seek help from another person/place not listed above7-point Likert scale, ranging from “extremely unlikely” (1) to “extremely likely” (7)Mean score averaged across all items and for each item individually

Help-seeking intentions for suicidality, adapted from [93]If you were experiencing suicidal thoughts, how likely is that you would seek help from the following people?—parent or guardian; other relative or family member; teacher; intimate partner (such as girlfriend or boyfriend); friend (someone not related to you); mental health professional (such as a psychologist, social worker, counselor); phone or text/chat help line (such as the National Suicide Prevention Lifeline or the Trevor Project); website help resources (such as StopBullying.gov); doctor or primary care provider; minister or religious leader (eg, priest, rabbi, and chaplain); I would not seek help from anyone; I would seek help from another person/place not listed above7-point Likert scale, ranging from “extremely unlikely” (1) to “extremely likely” (7)Mean score averaged across all items and for each item individually

Help-seeking self-efficacy [94,95]Please rate how certain you are that you can do each of the things described; Enlisting social resources subscale —how well can you get teachers to help you when you get stuck on schoolwork? how well can you get another student to help you when you get stuck on schoolwork? how well can you get adults to help you when you have social problems? how well can you get a friend to help you when you have social problems?; Enlisting parental and community support subscale —how much can you get your parent(s) to help you with a problem? how well can you get your brother(s) and sister(s) to help you with a problem? how well can you get your parents to take part in school activities? how well can you get people outside the school to take an interest in your school (community groups, churches)?7-point Likert scale, ranging from “not at all well” (1) to “very well” (7)Mean score for each subscale separately

Help-seeking behaviors, adapted from [96]In the past month, how often have you: Parent and guardian help-seeking subscale —asked your parents/guardians for help? talked to your parents/guardians about personal problems? talked to your parents/guardians about problems at school?; Relative/family help-seeking subscale —asked a relative/family member for help? talked to a relative/family member about personal problems? talked to a relative/family member about problems at school?; Teacher help-seeking subscale —asked a teacher for help? talked to a teacher about personal problems? talked to a teacher about problems at school?; Friend help-seeking subscale —asked a friend for help? talked to a friend about personal problems? talked to a friend about problems at school?; Mental health provider help-seeking subscale —asked a mental health professional (such as a psychologist, social worker, counselor) for help? talked to a mental health professional about personal problems? talked to a mental health professional about problems at school?; Help line help-seeking subscale —asked a person from a phone or text/chat help line (such as the National Suicide Prevention Lifeline or the Trevor Project) for help? talked to a person from a phone or text/chat help line about personal problems? talked to a person from a phone or text/chat help line about problems at school?; Doctor help-seeking subscale —asked a doctor or nurse for help? talked to a doctor or nurse about personal problems? talked to a doctor or nurse about problems at school?5-point Likert scale: Never; Rarely; Occasionally; A moderate amount; A great dealMean score for each subscale separately
Coping strategies and flexibility

Coping skill usage [88]Assessed using the Short form of the Adolescent Coping Scale Second Edition Short Form (ACS-2) [88]. For proprietary reasons, we do not list specific items. Productive (problem-solving) coping subscale contains 10 items; Nonproductive (passive avoidant) coping subscale contains 8 items (reverse coded)5-point Likert Scale, ranging from “never” (1) to “very often” (5)Mean score for each subscale separately

Coping flexibility [97]Please indicate how these situations apply to you by choosing one of the following for each situation: Evaluation coping subscale —I only use certain ways to cope with stress (reverse-coded); I am aware of how successful or unsuccessful my attempts to cope with stress have been; I fail to notice when I have been unable to cope with stress (reverse-coded); if I feel that I have failed to cope with stress, I change the way in which I deal with stress; after coping with stress, I think about how well my ways of coping with stress worked or did not work; Adaptive coping subscale —when a stressful situation has not improved, I try to think of other ways to cope with it; when stressed, I use several ways to cope and make the situation better; when I haven’t coped with a stressful situation well, I use other ways to cope with that situation; if a stressful situation has not improved, I use other ways to cope with that situation; if I have failed to cope with stress, I think of other ways to cope4-point Likert Scale: Not applicable; Somewhat applicable; Applicable; Very applicableMean score for each subscale separately
Knowledge and use of Web-based resources

Knowledge of Web-based resourcesHave you heard of any of these websites?—The Trevor Project; It Gets Better; GLAAD; Accredited Schools online; Teen Line; GSAa Network. (Links to each website were provided for reference.)Yes; No; UnsureSummary score that adds together all “yes” response options

Use of Web-based resourcesHave you visited any of these websites in the past month?—The Trevor Project; It Gets Better; GLAAD; Accredited Schools online; Teen Line; GSAa Network. (Links to each website were provided for reference.)Yes; No; UnsureSummary score that adds together all “yes” response options

aGSA: gay-straight alliance or gender and sexuality alliance.

Tertiary Outcomes

To answer our tertiary aim, we will measure the following long-term outcomes: bullying and cyberbullying victimization, loneliness, mental health issues, substance use, and internalized SGM stigma. All tertiary outcomes are assessed via self-reported surveys at T1, T2, and T3. Table 3 details our tertiary outcomes, items, response options, and coding procedures.

Bullying victimization is measured using 2 different scales, 1 for traditional face-to-face bullying and 1 for cyberbullying. We assess past-month bullying victimization using an adapted version of the University of Illinois Victimization Scale [98,99], which consists of 6 items assessing the number of times respondents experienced harassment in the past month. In addition, 4 response options range from “never” to “7 or more.” These response options are assigned a value from 1 to 4, and a mean score is calculated, with a higher mean score indicating greater bullying victimization. We assess past-month cyberbullying victimization using 4 items adapted from an internally consistent cyberbullying perpetration scale [100]. Response options use a 5-point Likert scale including “not sure,” “never,” “rarely,” “occasionally,” and “often,” which we will recode to a 4-point scale, where “never” equals 1 and “often” equals 4. We will average scores across all items, with a higher mean score indicating greater cyberbullying victimization.

Loneliness is measured via Robert’s Version of the University of California, Los Angeles Loneliness Scale [101], a validated measure of loneliness. Overall, 8 items (eg, “I feel isolated from others”) are used to calculate a mean score for how lonely participants felt.

Mental health issues include stress, anxiety, depression, and suicidality. Past-month stress is assessed using the Perceived Stress Scale [102,103], which contains 10 items used to calculate a mean score. Past-week anxiety symptoms are assessed using the Severity Measure for Generalized Anxiety Disorder—Child Age 11-17 [104,105], which contains 10 items used to calculate a mean score. Past-week depressive symptoms are assessed using the Patient Health Questionnaire-9 for children aged from 11 to 17 years [106], which contains 9 items used to calculate a mean score. Suicidality is assessed with 3 questions adapted from the 2017 YRBS with items measuring suicidal thoughts, plans, and attempts. Although the YRBS questions measured past year suicidality, we adapted them to assess for suicidality in the past month. Each of these 3 suicidality items will be modeled separately as dichotomous (presence vs absence) variables.

Substance use includes alcohol, cigarette, electronic cigarette, and marijuana use. All items are assessed based on the YRBS and adapted to measure past-month use. Alcohol use is assessed via 2 questions: 1 measuring the frequency of past-month alcohol use and 1 item assessing frequency of binge drinking (having 5 or more drinks in a row). Similar to prior research [7], each item will be modeled as continuous variable representing the number of days used. Cigarette smoking is assessed via 2 items measuring past-month number of days smoked and average quantity smoked per day. Cigarette smoking will be modeled continuously by multiplying smoking frequency by quantity, similar to prior research [10]. Electronic cigarette smoking and marijuana use are both assessed using single items that measure past-month frequency of use. Each will be modeled as a continuous variable representing the number of days used. For all substance use items, we will use midpoints for categories with a range (eg, “1 or 2 days” will be coded as 1.5).

Internalized SGM stigma is measured using 2 different scales. To assess internalized gender minority stigma, we use the Transgender Identity Survey [107,108], which consists of 26 items that are divided into 4 different subscales: pride (reverse coded), passing, alienation, and shame. We adapted this scale by adding the term “nonbinary” to each item. To assess internalized sexual minority stigma, we adapted the Transgender Identity Survey [107,108] items by replacing “transgender or nonbinary” with “lesbian, gay, bisexual, or queer”; otherwise, the subscales, response options, and scoring are identical.

Table 3. Tertiary outcomes, items, response options, and coding procedures.
Tertiary outcomeItemsResponse optionsCoding procedure
Bullying and cyberbullying victimization

Bullying victimization [98,99]How many times did these things happen to you in the last 30 days?—other students picked on me; other students called me names; I got hit and pushed by other students; I was threatened by other students; students spread rumors or told lies about me; I was excluded or kept out of a group of friends on purpose.Never; 1 or 2 times; 3 or 4 times; 5 or 6 times; 7 or more timesMean score

Cyberbullying victimization, adapted from [100]How often did the following things happen to you in the last month?—someone made rude comments about me online; someone spread rumors about me online, whether they were true or not; someone made aggressive or threatening comments to me online; someone sent a text message that said rude or mean things about meNot sure; Never; Rarely; Occasionally; OftenMean score
Loneliness

Loneliness [101]Indicate how often each of the statements below is descriptive of you—I feel in tune with the people around me (reverse coded); I lack companionship; I do not feel alone (reverse coded); I feel part of a group of friends (reverse coded); I am no longer close to anyone; I feel left out; I feel isolated from others; I can find companionship when I want it (reverse coded)4-point Likert scale: Never; Rarely; Sometimes; OftenMean score
Mental health issues

Stress [103,109]For each item, mark the description that best represents how often you have felt or thought that way during the past month—Been upset because of something that happened unexpectedly; Felt that you were unable to control the important things in your life; Felt nervous and ― “stressed”; Felt confident about your ability to handle your personal problems; Felt that things were going your way; Found that you could not cope with all the things that you had to do; Been able to control irritations in your life; Felt that you were on top of things; Been angered because of things that were outside of your control; Felt difficulties were piling up so high that you could not overcome them5-point Likert scale: Never; Almost never; Sometimes; Fairly often; Very oftenMean score

Anxiety [104,105]During the past 7 days, I have: Felt moments of sudden terror, fear, or fright; Felt anxious, worried, or nervous; Had thoughts of bad things happening such as family tragedy, ill health, loss of a job, or accidents; Felt a racing heart, sweaty, trouble breathing, faint, or shaky; Felt tense muscles, felt on edge or restless, or had trouble relaxing or trouble sleeping; Avoided, or did not approach or enter situations about which I worry; Left situations early or participated only minimally because of worries; Spent lots of time making decisions, putting off making decisions, or preparing situations, because of worries; Sought reassurance from others because of worries; Needed help to cope with anxiety (eg, alcohol or medication, superstitious objects, or other people)5-point Likert scale: Never; Occasionally; Half the time; Most of the time; All of the timeMean score

Depression [106]How often have you been bothered by each of the following symptoms during the past 7 days?—Feeling down, depressed, irritable, or hopeless; Little interest or pleasure in doing things; Trouble falling asleep, staying asleep, or sleeping too much; Poor appetite, weight loss, or overeating; Feeling tired, or having little energy; Feeling bad about yourself—or feeling that you are a failure, or that you have let yourself or your family down; Trouble concentrating on things like school work, reading, or watching TV; Moving or speaking so slowly that other people have noticed; Or the opposite—being so fidgety or restless that you were moving around a lot more than usual; Thoughts that you would be better off dead, or of hurting yourself in some way4-point Likert scale: Not at all; Several days; More than half the days; Nearly every dayMean score

Suicidal ideation, adapted from [110]During the past month, did you ever seriously consider attempting suicide?Yes; NoDichotomously as yes or no

Suicide plan, adapted from [110]During the past month, did you make a plan about how you would attempt suicide?Yes; NoDichotomously as yes or no

Suicide attempt, adapted from [110]During the past month, how many times did you actually attempt suicide?

0 times; 1 time; 2 or 3 times; 4 or 5 times; 6 or more timesDichotomously as yes or no, similar to prior research [6]
Substance use

Alcohol use [110]During the past 30 days, on how many days did you have at least one drink of alcohol?0 days; 1 or 2 days; 3 to 5 days; 6 to 9 days; 10 to 19 days; 20 to 29 days; All 30 daysCount variable using midpoints (eg, “1 or 2 days” equals 1.5), similar to prior research [7]

Binge alcohol use [111]During the past 30 days, on how many days did you have 5 or more drinks of alcohol in a row (within a couple of hours)?0 days; 1 day; 2 days; 3 to 5 days; 6 to 9 days; 10 to 19 days; 20 or more daysCount variable using midpoints (eg, “1 or 2 days” equals 1.5), similar to prior research [7]

Cigarette smoking [110]During the past 30 days, on how many days did you smoke cigarettes? During the past 30 days, on the days you smoked, how many cigarettes did you smoke per day?For the question pertaining to number of days: 0 days; 1 or 2 days; 3 to 5 days; 6 to 9 days; 10 to 19 days; 20 to 29 days; All 30 days. For the question pertaining to number of cigarettes: I did not smoke cigarettes during the past 30 days; Less than 1 cigarette per day; 1 cigarette per day; 2 to 5 cigarettes per day; 6 to 10 cigarettes per day; 11 to 20 cigarettes per day; More than 20 cigarettes per dayContinuously by multiplying frequency by quantity, similar to prior research [10]

Electronic cigarette use [110]During the past 30 days, on how many days did you use an electronic vapor product?0 days; 1 or 2 days; 3 to 5 days; 6 to 9 days; 10 to 19 days; 20 to 29 days; All 30 daysCount variable using midpoints (eg, “1 or 2 days” equals 1.5)

Marijuana use [110]During the past 30 days, how many times did you use marijuana? (Marijuana is also called grass, pot, or weed.)0 times; 1 or 2 times; 3 to 9 times; 10 to 19 times; 20 to 39 times; 40 or more timesCount variable using midpoints (eg, “1 or 2 days” equals 1.5)
Internalized sexual and gender minority stigma

Internalized gender minority stigma, adapted from [107,108]Pride subscale contains 8 items (reverse coded); Passing subscale contains 7 items; Alienation subscale contains 3 items; Shame subscale contains 8 items; To obtain the specific scale items, please contact Robert WS Coulter and Walter O Bockting7-point Likert Scale: Strongly disagree; Disagree; Somewhat disagree; Neither agree nor disagree; Somewhat agree; Agree; Strongly agreeMean score for each subscale separately

Internalized Sexual Minority Stigma, adapted from [107,108]Pride subscale contains 8 items (reverse coded); Passing subscale contains 7 items; Alienation subscale contains 3 items; Shame subscale contains 8 items; To obtain the specific scale items, please contact Robert WS Coulter and Walter O Bockting7-point Likert Scale: Strongly Disagree; Disagree; Somewhat Disagree; Neither Agree nor Disagree; Somewhat Agree; Agree; Strongly AgreeMean score for each subscale separately
Exploratory Outcomes

To answer our exploratory aim, we assess several outcomes meant to better our understanding of participants’ responses to the game and how to improve the game in the future. Our exploratory aim concerns implementation procedures, intervention integration, and intervention adaptation and expansion outcomes. Table 4 details our exploratory outcomes, research questions, and assessments.

Multiple implementation procedures will be explored to inform our future research. These include the following: how long it takes to enroll participants on the internet, which venues participants were recruited from, how long it takes participants to complete the surveys, how many participants complete the game, and which game milestones are achieved by participants. These data are obtained from multiple sources, including the self-reported surveys in REDCap and the game play data.

Implementation integration—or how well the game fits into the participants’ lives—will be assessed in a variety of ways. We will examine how many participants were excluded from our study based on not having access to a computer or not having an email address. We will examine if the participants had any problems downloading or playing game and how easy it was for them to participate in the RCT. We will also explore if participants had issues with game security, privacy, or interference. These outcomes are tracked using participants’ responses to the screening questionnaire, T2 survey, and T3 surveys, as well as from participants’ correspondences with the project email address.

Adaptation and expansion of the game intervention will be assessed via the T2 and T3 surveys. We will ask participants if they prefer to play the game on a different format (eg, on their phones), their favorite and least favorite parts of the game, how they would change and improve the game, and whether they think other SGMY would be interested in the game. Finally, we will explore whether the responses to the Gaming Experience Questionnaire [92] subscale scores differed by participants’ gender, sexual orientation, race, ethnicity, and age.

Table 4. Exploratory outcomes, research questions, and assessments.
Exploratory outcomeExploratory questionAssessment
Implementation procedures

Enrollment periodHow long does it take to get 240 people to enroll in the study?Measured as number of months between enrollment of first and last participant.

Recruitment venuesHow many people were recruited from which venue?Assessed using unique links that track the number of clicks on each advertisement.

Survey completion timeHow many days before participants completed each survey?Measured as the number of days between the first survey invitation was sent to the participant and the day they completed the survey.

Game completionHow many intervention condition participants completed the game?Assessed via game play data from the secure file-transfer-protocol (FTP) system as well as T2a and T3b surveys: Did you complete the game? Yes; No; Unsure

Game milestonesWhat milestones did the participants achieve in the game?Assessed via game play data from the secure file-transfer-protocol (FTP) system.
Integration

Computer accessHow many youths were excluded because they were without a personal computer or Mac?Assessed via the screening questionnaire: Do you have a laptop or desktop computer (either personal computer or Mac) where you can download games? Yes; No; Unsure

Email address accessHow many youths were excluded because they were without an email address?Assessed via the screening questionnaire: do you have an email address? Yes; No

Ease of downloadHow easily was the game downloaded without contacting our research coordinator?Assessed via the number of emails to our project email address.

Personal computer versus Mac useWhat participants used personal computer and Mac computers to download the game?Assessed as percentages via the game download materials in REDCap.

Ease of participationHow many times was the research staff contacted by participants with questions about surveys or intervention materials?Assessed via the number and types of emails to our project email address.

Game problemsHow many problems and what types of problems did participants encounter with the game?Assessed via T2 and T3 surveys: Did you have any problems in the game?—Yes; No; If yes, please describe; Open-ended text box

Game securityHow important was the use of password protection in the game?Assessed via T2 and T3 surveys: How important was it to have the game be protected by a password?—Very Important; Important; Moderately Important; Slightly Important; Not Important

Game privacyHow safe did our participants feel playing the game?Assessed via T2 and T3 surveys: How often were you concerned about other people seeing you play the game?—Always; Very Often; Sometimes; Rarely; Never

Game interferenceDid the gaming intervention interfere with participants’ regular activities?Assessed via T2 and T3 surveys: During the past 30 days, how many times did playing the game interfere with school, work, or other responsibilities (like being late, missing school, or making it hard to concentrate)—Never; 1 time; 2 times; 3 times; 4 times; 5 or more times
Adaptation and expansion

Future gaming platformsWould our participants like to play the game on other platforms, such as phone?Assessed via T2 and T3 surveys: In the future where would you like to play this game? My computer only; My phone only; Both my computer and phone

Gaming experience [92] by demographicsWere there certain groups that had better or worse experiences with the game?We will examine whether the Gaming Experience Questionnaire [92] subscale scores (assessed via T2 and T3 surveys) differed by participants’ gender, sexual orientation, race, ethnicity, and age.

Future appeal to sexual and gender minority youthWill other sexual and gender minority youth enjoy the game?Assessed via T2 and T3 surveys: Do you think other LGBTQ people your age would like to play this game? Definitely; Very Probably; Probably; Possibly; Probably Not; Definitely Not

Recommended game changesWhat would participants change about the game?Assessed via T2 and T3 surveys: What would you change about the game? Open-ended text box

Favorite parts of gameWhat did participants like best about the game?Assessed via T2 and T3 surveys: What were your favorite parts of the game? Open-ended text box

Least favorite arts of gameWhat did participants like least about the game?Assessed via T2 and T3 surveys: What were your least favorite parts of the game? Open-ended text box

Recommended game improvementsHow would participants improve the game?Assessed via T2 and T3 surveys: How would you improve the game? Open-ended text box

aT2: first follow-up.

bT3: final follow-up.

Demographics and Potential Confounders

Demographic variables include age, grade in school, race, ethnicity, parent’s highest education level, and eligibility for free or reduced-price lunch at school. We also measure gender using the 2-step process assessing current gender identity and sex assigned at birth [112]. Moreover, 2 items assess gender expression. We measure sexual orientation using 3 questions assessing sexual attraction, behavior, and identity.

In addition to demographic variables, we assess variables that may confound the relationship between the intervention and our outcomes. Structural stigma is assessed by collecting participant’s residential zip code. Each zip code will then be given a state-level structural stigma score using Hatzenbuehler scale [113], based on density of same-sex couples, inclusive policies (eg, employee nondiscrimination policies), public opinion, and percentage of high schools with gay-straight alliance (also known as a gender and sexuality alliance; GSA). We have participants self-report whether or not their school had a GSA. We assess the participants’ overall school environment with 5 questions from the California School Climate Survey. We assess participants’ level of perceived social support using the Multidimensional Scale of Perceived Social Support, measuring support from family and friends [114]. Finally, we measure level of “outness” regarding participant’s sexual orientation and gender identity.

Statistical Analysis

For the measures of implementation procedures, demand, acceptability, integration, and adaptation and expansion (ie, primary and exploratory outcomes), we will report results using descriptive statistics (ie, percentages and frequencies for categorical variables or means and SDs for continuous variables) along with 95% CIs. We will compare our results against our a priori benchmarks (see Table 1). For the limited-efficacy outcomes (ie, secondary and tertiary outcomes), we will use repeated measures (ie, multilevel) statistical models using linear or logistic regression, depending on the distribution of the outcomes. To examine whether there are greater improvements over time in the intervention versus control conditions, we will test the interaction term of time by condition, wherein we will focus on the point estimates and 95% CIs. We will conduct our primary analyses as an intent-to-treat analysis, wherein everyone in the intervention condition is treated equally. As an additional exploratory exercise, we will conduct an analysis wherein we assess whether intervention effects were stronger with more intensive uptake of the game intervention. For these analyses, instead of a binary intervention indicator variable (1.0 for intervention vs 0.0 for control), we will use an intensity-adjusted intervention variable. The intensity-adjusted intervention variable will be coded as 1.0 for those who complete the game, 0.5 for those who play the game, and 0.0 for those who never played the game or were in the control condition. All analyses will be completed in Stata 15.0 (StataCorp LLC, College Station, Texas), and significance will be set as at P<.05.

Qualitative Analysis

For the open-ended questions, we will conduct qualitative data analysis. First, a group of investigators will read all of the participants’ responses and inductively identify the main themes for each item. Second, using these themes, we will develop a codebook with code names and definitions. Finally, 2 independent coders will code the qualitative data using Dedoose. If the 2 coders cannot reach consensus, a third coder will resolve any disagreements.

Sample Size and Power Calculation

This is a feasibility study, which is primarily being conducted to inform a larger RCT. As such, we powered this study based on our primary outcomes [74,115]. For our primary outcomes regarding the success of implementation procedures (eg, retention rate at T2), assuming 240 participants and 5% type 1 error rate, we will be able to estimate 95% CI width of no more than 0.13. For game demand among 120 intervention condition participants, we will be able to estimate 95% CI width of no more than 0.18. For our secondary and tertiary outcomes, we are primarily interested in effect size and CI estimation; we are not necessarily interested in finding statistically significant effects [74,115]. Estimations of the effect size and CI width will help us power a future, larger RCT.

Ethics Statement

All study procedures were approved by the Human Research Protection Office at the University of Pittsburgh. To protect participants from having to reveal their sexual or gender minority identities to their caregivers, thereby potentially putting them in harm’s way, we received a waiver of parental consent [76]. This allowed participants to self-consent. We did not collect personally identifying information other than email, phone, and zip code. Incentives, in the form of either Apple or Google electronic gift cards, are sent via email. This study was covered by a Certificate of Confidentiality from the National Institutes of Health.


Enrollment and Randomization

Overall, 2153 individuals clicked the link to the screening questionnaire (Figure 10). In total, 988 individuals completed the screening questionnaire, of which 407 individuals met all eligibility criteria. Overall, 304 individuals consented to participate, and 240 participants completed the T1 survey and were randomized. Of those who were eligible, 59.0% (240/407) were enrolled into the study. Half of the enrolled participants (n=120) were randomized into the intervention condition and half (n=120) into the control condition. All 240 participants were enrolled into the RCT between April 2018 and July 2018. The final surveys (T3) will be completed in November 2018.

Sample Demographics

At baseline, there were no significant differences in demographic characteristics between intervention and control condition participants (Table 5). Overall, 52.1% (125/240) of participants identified as gay or lesbian, 26.7% (64/240) as bisexual, 24.2% (58/240) as queer, 11.7% (28/240) as another nonheterosexual identity, 4.6% (11/240) as unsure, and 2.1% (5/240) as heterosexual (all of whom were gender minorities). Nearly half (47.1%; 113/240) of participants were a gender minority, 36.7% (88/240) were cisgender boys, and 16.3% (39/240) were cisgender girls. Among the 113 gender minority participants, 54.9% (62/113) identified as a transgender boy, 33.6% (38/113) as nonbinary, 18.6% (21/113) as a boy, 17.7% (20/113) as genderqueer, 6.2% (7/113) as a transgender girl, 5.3% (6/113) as a girl, and 5.3% (6/113) as another identity (data not shown; participants could select multiple options). Overall, 81.3% (195/240) of participants identified as white, 20.4% (49/240) as Hispanic/Latinx, 10.8% (26/240) as black or African American, 7.5% (18/240) as American Indian or Alaska Native, and 5.8% (14/240) as Asian. Moreover, 36.7% (88/240) of participants were eligible for free or reduced-price lunch, and 52.9% (127/240) had a parent/guardian with a college degree. Figure 11 shows the geographic distribution of participants across the United States of America. In total, participants lived in 30 states.

Figure 10. Consolidated Standards of Reporting Trials (CONSORT) flow diagram.
View this figure
Table 5. Baseline demographic characteristics for the total sample and by intervention condition (N=240).
Demographic characteristicsTotal (N=240), n (%)Condition


Intervention (n=120), n (%)Control (n=120), n (%)P valuea
Sexual identityb

Gay or lesbian125 (52)62 (52)63 (53).89

Bisexual64 (27)32 (27)32 (27)>.99

Queer58 (24)33 (28)25 (21).23

Unsure11 (5)8 (7)3 (3).12

Another nonheterosexual identity28 (12)11 (9)17 (14).23

Heterosexual5 (2)3 (3)2 (2)>.99
Gender.24

Cisgender girls39 (16)18 (15)21 (18)

Cisgender boys88 (37)39 (33)49 (41)

Gender minorityc113 (47)63 (53)50 (42)
Age in years.98

1435 (15)16 (13)19 (16)

1566 (28)33 (28)33 (28)

1671 (30)37 (31)34 (28)

1756 (23)28 (23)28 (23)

1812 (5)6 (5)6 (5)
Race/ethnicityb

White195 (81)96 (80)99 (83).62

Asian14 (6)5 (4)9 (8).27

American Indian or Alaska Native18 (8)6 (5)12 (10).14

Black or African American26 (11)15 (13)11 (9).41

Native Hawaiian or other Pacific Islander5 (2)3 (3)2 (2)>.99

Hispanic or Latinx49 (20)28 (23)21 (18).26

Another race21 (9)9 (8)12 (10).49
Free or reduced-price lunch eligibility.98

Yes88 (37)44 (37)44 (37)

No109 (45)54 (45)55 (46)

Unsure43 (18)22 (18)21 (18)
Highest education level of parent(s) or guardian(s).75

Less than high school degree22 (9)10 (8)12 (10)

High school degree38 (16)22 (18)16 (13)

Some college43 (18)23 (19)20 (17)

College degree127 (53)61 (51)66 (55)

Unsure10 (4)4 (3)6 (5)

aGenerally, P values were derived using chi-square test statistics; however, when expected cell sizes were less than 5, P values were derived using Fisher exact tests.

bParticipants could select more than one option; therefore, the percentages may not add to 100% (N=240). P values were derived for each response option separately.

cGender minority includes all participants whose gender identity did not match their assigned sex at birth.

Figure 11. Geographic distribution of participants in the United States of America.
View this figure

Principal Findings

In this pilot RCT, a game-based intervention, we successfully enrolled and randomized 240 SGMY participants over a 4-month period. Recruitment was most successful through Facebook and Instagram. Over half of the enrolled sample identified as gay or lesbian and nearly half were gender minority youth. There were no demographic differences between intervention and control conditions.

This study protocol offers several strengths and novel methods that can inform future intervention studies with SGMY. Our game intervention is one of the first Web-accessible programs targeting SGMY who are victims of bullying or cyberbullying. Furthermore, this study engages a younger cohort of individuals focusing on youth aged 14 to 18 years, as opposed to the typical focus of SGMY-relevant intervention research, which concentrates on those aged 18 years and older. This is also one of the first bullying prevention programs to address social and emotional learning (eg, coping strategies) with respect to bullying among SGMY utilizing a gaming format to attract participation and build retention. Finally, the internet-based distribution of the game intervention has the potential to reach far greater numbers of SGMY than traditional face-to-face interventions.

Limitations

Despite the many strengths of this study protocol, it is not without limitations. Of particular concern is selection bias. Participants in our study were conveniently sampled via the internet. Participants were also required to have a computer with internet access and an email address. Therefore, our study results may not be generalizable to all SGMY (eg, those who are homelessness or have a low socioeconomic status). In addition, participants can only access the game intervention on desktop or laptop computers with Windows or Mac operating systems. In future iterations, we plan to make the game intervention compatible with smartphones, thereby increasing the accessibility and potential usability of the game. Although we used and adapted measures with strong psychometric properties, most of the scales were not validated specifically among SGMY. Finally, an 8-week follow-up period may be too brief to affect our identified long-term outcomes; however, we chose to measure these outcomes as they are important health outcomes to inform a longer-term intervention study.

Conclusions

SGMY experience great disparities in bullying victimization, substance use, and mental health; however, there are few scientifically tested interventions currently available to reduce these disparities. This paper describes our protocol for the study of a Web-accessible game intervention aimed at improving the health of SGMY. Specifically, the outcomes of this pilot study are to assess the feasibility of implementing a game intervention to SGMY and limited efficacy of this intervention: increasing help-seeking–related knowledge, intentions, self-efficacy, and behaviors; increasing productive coping skills use and coping flexibility; and reducing health risk factors and behaviors among SGMY. Our study protocol directly informs the scientific development of a future, larger RCT testing the limited efficacy of our game intervention.

Acknowledgments

This research was supported by the National Institutes of Health, primarily by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R21HD083561 to JEE and MSF). In addition, this study was partially supported by the National Institute on Drug Abuse (F31DA037647 to RWSC), the National Center for Advancing Translational Sciences (TL1TR001858 for RWSC), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (K24HD075862 to EM), and the National Institute of Mental Health (T32MH094174 to ERH). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The National Institutes of Health was not involved in the study design, the writing of the protocol, or the decision to submit for publication.

Conflicts of Interest

None declared.

  1. Garofalo R, Wolf RC, Kessel S, Palfrey SJ, DuRant RH. The association between health risk behaviors and sexual orientation among a school-based sample of adolescents. Pediatrics 1998 May;101(5):895-902. [Medline]
  2. Marshal MP, Friedman MS, Stall R, King KM, Miles J, Gold MA, et al. Sexual orientation and adolescent substance use: a meta-analysis and methodological review. Addiction 2008 Apr;103(4):546-556 [FREE Full text] [CrossRef] [Medline]
  3. Watson RJ, Lewis NM, Fish JN, Goodenow C. Sexual minority youth continue to smoke cigarettes earlier and more often than heterosexuals: findings from population-based data. Drug Alcohol Depend 2018 Mar 1;184:64-70. [CrossRef] [Medline]
  4. Watson RJ, Goodenow C, Porta C, Adjei J, Saewyc E. Substance use among sexual minorities: has it actually gotten better? Subst Use Misuse 2018 Jun 7;53(7):1221-1228. [CrossRef] [Medline]
  5. Coulter RW, Bersamin M, Russell ST, Mair C. The effects of gender- and sexuality-based harassment on lesbian, gay, bisexual, and transgender substance use disparities. J Adolesc Health 2018 Jun;62(6):688-700. [CrossRef] [Medline]
  6. Coulter RW, Kessel Schneider S, Beadnell B, O'Donnell L. Associations of outside- and within-school adult support on suicidality: moderating effects of sexual orientation. Am J Orthopsychiatry 2017;87(6):671-679 [FREE Full text] [CrossRef] [Medline]
  7. Coulter RW, Birkett M, Corliss HL, Hatzenbuehler ML, Mustanski B, Stall RD. Associations between LGBTQ-affirmative school climate and adolescent drinking behaviors. Drug Alcohol Depend 2016 Apr 1;161:340-347 [FREE Full text] [CrossRef] [Medline]
  8. Russell ST, Driscoll AK, Truong N. Adolescent same-sex romantic attractions and relationships: implications for substance use and abuse. Am J Public Health 2002 Feb;92(2):198-202. [Medline]
  9. Corliss HL, Rosario M, Wypij D, Wylie SA, Frazier AL, Austin SB. Sexual orientation and drug use in a longitudinal cohort study of U.S. adolescents. Addict Behav 2010 May;35(5):517-521 [FREE Full text] [CrossRef] [Medline]
  10. Corliss HL, Rosario M, Birkett MA, Newcomb ME, Buchting FO, Matthews AK. Sexual orientation disparities in adolescent cigarette smoking: intersections with race/ethnicity, gender, and age. Am J Public Health 2014 Jun;104(6):1137-1147. [CrossRef] [Medline]
  11. Newcomb ME, Birkett M, Corliss HL, Mustanski B. Sexual orientation, gender, and racial differences in illicit drug use in a sample of US high school students. Am J Public Health 2014 Feb;104(2):304-310. [CrossRef] [Medline]
  12. Rosario M, Corliss HL, Everett BG, Reisner SL, Austin SB, Buchting FO, et al. Sexual orientation disparities in cancer-related risk behaviors of tobacco, alcohol, sexual behaviors, and diet and physical activity: pooled Youth Risk Behavior Surveys. Am J Public Health 2014 Feb;104(2):245-254. [CrossRef] [Medline]
  13. Rosario M, Corliss HL, Everett BG, Russell ST, Buchting FO, Birkett MA. Mediation by peer violence victimization of sexual orientation disparities in cancer-related tobacco, alcohol, and sexual risk behaviors: pooled youth risk behavior surveys. Am J Public Health 2014 Jun;104(6):1113-1123. [CrossRef] [Medline]
  14. Corliss HL, Rosario M, Wypij D, Fisher LB, Austin SB. Sexual orientation disparities in longitudinal alcohol use patterns among adolescents: findings from the Growing Up Today Study. Arch Pediatr Adolesc Med 2008 Nov;162(11):1071-1078 [FREE Full text] [CrossRef] [Medline]
  15. Corliss HL, Wadler BM, Jun HJ, Rosario M, Wypij D, Frazier AL, et al. Sexual-orientation disparities in cigarette smoking in a longitudinal cohort study of adolescents. Nicotine Tob Res 2013 Jan;15(1):213-222 [FREE Full text] [CrossRef] [Medline]
  16. Talley AE, Hughes TL, Aranda F, Birkett M, Marshal MP. Exploring alcohol-use behaviors among heterosexual and sexual minority adolescents: intersections with sex, age, and race/ethnicity. Am J Public Health 2014 Feb;104(2):295-303 [FREE Full text] [CrossRef] [Medline]
  17. Fish JN, Watson RJ, Porta CM, Russell ST, Saewyc EM. Are alcohol-related disparities between sexual minority and heterosexual youth decreasing? Addiction 2017 Nov;112(11):1931-1941. [CrossRef] [Medline]
  18. White Hughto JM, Biello KB, Reisner SL, Perez-Brumer A, Heflin KJ, Mimiaga MJ. Health risk behaviors in a representative sample of bisexual and heterosexual female high school students in Massachusetts. J Sch Health 2016 Jan;86(1):61-71 [FREE Full text] [CrossRef] [Medline]
  19. Matthews DD, Blosnich JR, Farmer GW, Adams BJ. Operational definitions of sexual orientation and estimates of adolescent health risk behaviors. LGBT Health 2014 Mar;1(1):42-49 [FREE Full text] [Medline]
  20. Marshal MP, Dietz LJ, Friedman MS, Stall R, Smith HA, McGinley J, et al. Suicidality and depression disparities between sexual minority and heterosexual youth: a meta-analytic review. J Adolesc Health 2011 Aug;49(2):115-123 [FREE Full text] [CrossRef] [Medline]
  21. Seil KS, Desai MM, Smith MV. Sexual orientation, adult connectedness, substance use, and mental health outcomes among adolescents: findings from the 2009 New York City Youth Risk Behavior Survey. Am J Public Health 2014 Oct;104(10):1950-1956 [FREE Full text] [CrossRef] [Medline]
  22. Hatzenbuehler ML, Birkett M, Van Wagenen A, Meyer IH. Protective school climates and reduced risk for suicide ideation in sexual minority youths. Am J Public Health 2014 Feb;104(2):279-286. [CrossRef] [Medline]
  23. Russell ST, Joyner K. Adolescent sexual orientation and suicide risk: evidence from a national study. Am J Public Health 2001 Aug;91(8):1276-1281. [Medline]
  24. Stone DM, Luo F, Ouyang L, Lippy C, Hertz MF, Crosby AE. Sexual orientation and suicide ideation, plans, attempts, and medically serious attempts: evidence from local Youth Risk Behavior Surveys, 2001-2009. Am J Public Health 2014 Feb;104(2):262-271. [CrossRef] [Medline]
  25. Stone DM, Luo F, Lippy C, McIntosh WL. The role of social connectedness and sexual orientation in the prevention of youth suicide ideation and attempts among sexually active adolescents. Suicide Life Threat Behav 2015 Aug;45(4):415-430. [CrossRef] [Medline]
  26. Eisenberg ME, Resnick MD. Suicidality among gay, lesbian and bisexual youth: the role of protective factors. J Adolesc Health 2006 Nov;39(5):662-668 [FREE Full text] [CrossRef] [Medline]
  27. Reisner SL, Greytak EA, Parsons JT, Ybarra ML. Gender minority social stress in adolescence: disparities in adolescent bullying and substance use by gender identity. J Sex Res 2015;52(3):243-256 [FREE Full text] [CrossRef] [Medline]
  28. Friedman MS, Marshal MP, Guadamuz TE, Wei C, Wong CF, Saewyc E, et al. A meta-analysis of disparities in childhood sexual abuse, parental physical abuse, and peer victimization among sexual minority and sexual nonminority individuals. Am J Public Health 2011 Aug;101(8):1481-1494 [FREE Full text] [CrossRef] [Medline]
  29. Russell ST, Everett BG, Rosario M, Birkett M. Indicators of victimization and sexual orientation among adolescents: analyses from Youth Risk Behavior Surveys. Am J Public Health 2014 Feb;104(2):255-261. [CrossRef] [Medline]
  30. Coulter RW, Herrick AL, Friedman MR, Stall RD. Sexual-orientation differences in positive youth development: the mediational role of bullying victimization. Am J Public Health 2016 Apr;106(4):691-697. [CrossRef] [Medline]
  31. Kessel Schneider K, O'Donnell L, Stueve A, Coulter RW. Cyberbullying, school bullying, and psychological distress: a regional census of high school students. Am J Public Health 2012 Jan;102(1):171-177. [CrossRef] [Medline]
  32. Goodenow C, Watson RJ, Adjei J, Homma Y, Saewyc E. Sexual orientation trends and disparities in school bullying and violence-related experiences, 1999-2013. Psychol Sex Orientat Gend Divers 2016 Dec;3(4):386-396 [FREE Full text] [CrossRef] [Medline]
  33. Kann L, Olsen EO, McManus T, Harris WA, Shanklin SL, Flint KH, et al. Sexual identity, sex of sexual contacts, and health-related behaviors among students in grades 9-12 - United States and selected sites, 2015. MMWR Surveill Summ 2016 Aug 12;65(9):1-202. [CrossRef] [Medline]
  34. Meyer IH. Minority stress and mental health in gay men. J Health Soc Behav 1995 Mar;36(1):38-56. [Medline]
  35. Meyer IH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychol Bull 2003 Sep;129(5):674-697 [FREE Full text] [CrossRef] [Medline]
  36. Lasser J, Tharinger D. Visibility management in school and beyond: a qualitative study of gay, lesbian, bisexual youth. J Adolesc 2003 Apr;26(2):233-244. [Medline]
  37. Plöderl M, Faistauer G, Fartacek R. The contribution of school to the feeling of acceptance and the risk of suicide attempts among Austrian gay and bisexual males. J Homosex 2010;57(7):819-841. [CrossRef] [Medline]
  38. Lytle MC, Silenzio VM, Homan CM, Schneider P, Caine ED. Suicidal and help-seeking behaviors among youth in an online lesbian, gay, bisexual, transgender, queer, and questioning social network. J Homosex 2017 Oct 11:1916-1933. [CrossRef] [Medline]
  39. Lucassen MF, Merry SN, Robinson EM, Denny S, Clark T, Ameratunga S, et al. Sexual attraction, depression, self-harm, suicidality and help-seeking behaviour in New Zealand secondary school students. Aust N Z J Psychiatry 2011 May;45(5):376-383. [CrossRef] [Medline]
  40. Lock J, Steiner H. Relationships between sexual orientation and coping styles of gay, lesbian, and bisexual adolescents from a community high school. J Gay Lesbian Med Assoc 1999 Sep;3(3):77-82. [CrossRef]
  41. Sornberger MJ, Smith NG, Toste JR, Heath NL. Nonsuicidal self-injury, coping strategies, and sexual orientation. J Clin Psychol 2013 Jun;69(6):571-583. [CrossRef] [Medline]
  42. Institute of Medicine (US) Committee on Lesbian, Gay, Bisexual, and Transgender Health Issues and Research Gaps and Opportunities. The Health of Lesbian, Gay, Bisexual, and Transgender People Building a Foundation for Better Understanding. Washington, DC: National Academies Press; 2011.
  43. Mustanski B, Lyons T, Garcia SC. Internet use and sexual health of young men who have sex with men: a mixed-methods study. Arch Sex Behav 2011 Apr;40(2):289-300 [FREE Full text] [CrossRef] [Medline]
  44. Egan J. The New York Times. 2000 Dec 31. Lonely Gay Teen Seeking Same   URL: https://www.nytimes.com/2000/12/31/magazine/l-lonely-gay-teen-seeking-same-172898.html [accessed 2019-01-14] [WebCite Cache]
  45. Stueve A, O'Donnell LN, Duran R, San Doval A, Blome J. Time-space sampling in minority communities: results with young Latino men who have sex with men. Am J Public Health 2001 Jun;91(6):922-926. [Medline]
  46. Valleroy LA, MacKellar DA, Karon JM, Rosen DH, McFarland W, Shehan DA, et al. HIV prevalence and associated risks in young men who have sex with men. Young Men's Survey Study Group. JAMA 2000 Jul 12;284(2):198-204. [Medline]
  47. Muhib FB, Lin LS, Stueve A, Miller RL, Ford WL, Johnson WD, Community Intervention Trial for Youth Study Team. A venue-based method for sampling hard-to-reach populations. Public Health Rep 2001;116(Suppl 1):216-222 [FREE Full text] [CrossRef] [Medline]
  48. Gilbert LK, Temby JR, Rogers SE. Evaluating a teen STD prevention Web site. J Adolesc Health 2005 Sep;37(3):236-242. [CrossRef] [Medline]
  49. Lou CH, Zhao Q, Gao ES, Shah IH. Can the internet be used effectively to provide sex education to young people in China? J Adolesc Health 2006 Nov;39(5):720-728. [CrossRef] [Medline]
  50. Woolderink M, Smit F, van der Zanden R, Beecham J, Knapp M, Paulus A, et al. Design of an internet-based health economic evaluation of a preventive group-intervention for children of parents with mental illness or substance use disorders. BMC Public Health 2010 Aug 10;10:470 [FREE Full text] [CrossRef] [Medline]
  51. Elliott JC, Carey KB, Bolles JR. Computer-based interventions for college drinking: a qualitative review. Addict Behav 2008 Aug;33(8):994-1005 [FREE Full text] [CrossRef] [Medline]
  52. Walters ST, Miller E, Chiauzzi E. Wired for wellness: e-interventions for addressing college drinking. J Subst Abuse Treat 2005 Sep;29(2):139-145. [CrossRef] [Medline]
  53. Schinke SP, Schwinn TM, Di Noia J, Cole KC. Reducing the risks of alcohol use among urban youth: three-year effects of a computer-based intervention with and without parent involvement. J Stud Alcohol 2004 Jul;65(4):443-449 [FREE Full text] [Medline]
  54. Hallett J, Maycock B, Kypri K, Howat P, McManus A. Development of a web-based alcohol intervention for university students: processes and challenges. Drug Alcohol Rev 2009 Jan;28(1):31-39. [CrossRef] [Medline]
  55. Carey KB, Scott-Sheldon LA, Elliott JC, Bolles JR, Carey MP. Computer-delivered interventions to reduce college student drinking: a meta-analysis. Addiction 2009 Nov;104(11):1807-1819 [FREE Full text] [CrossRef] [Medline]
  56. Walters ST, Wright JA, Shegog R. A review of computer and internet-based interventions for smoking behavior. Addict Behav 2006 Feb;31(2):264-277. [CrossRef] [Medline]
  57. Buller DB, Borland R, Woodall WG, Hall JR, Hines JM, Burris-Woodall P, et al. Randomized trials on consider this, a tailored, internet-delivered smoking prevention program for adolescents. Health Educ Behav 2008 Apr;35(2):260-281 [FREE Full text] [CrossRef] [Medline]
  58. Patten CA, Croghan IT, Meis TM, Decker PA, Pingree S, Colligan RC, et al. Randomized clinical trial of an internet-based versus brief office intervention for adolescent smoking cessation. Patient Educ Couns 2006 Dec;64(1-3):249-258. [CrossRef] [Medline]
  59. Jan RL, Wang JY, Huang MC, Tseng SM, Su HJ, Liu LF. An internet-based interactive telemonitoring system for improving childhood asthma outcomes in Taiwan. Telemed J E Health 2007 Jun;13(3):257-268. [CrossRef] [Medline]
  60. Runge C, Lecheler J, Horn M, Tews JT, Schaefer M. Outcomes of a web-based patient education program for asthmatic children and adolescents. Chest 2006 Mar;129(3):581-593. [CrossRef] [Medline]
  61. Joseph CL, Peterson E, Havstad S, Johnson CC, Hoerauf S, Stringer S, Asthma in Adolescents Research Team. A web-based, tailored asthma management program for urban African-American high school students. Am J Respir Crit Care Med 2007 May 1;175(9):888-895 [FREE Full text] [CrossRef] [Medline]
  62. Chan DS, Callahan CW, Hatch-Pigott VB, Lawless A, Proffitt HL, Manning NE, et al. Internet-based home monitoring and education of children with asthma is comparable to ideal office-based care: results of a 1-year asthma in-home monitoring trial. Pediatrics 2007 Mar;119(3):569-578. [CrossRef] [Medline]
  63. Maczewski M. Exploring identities through the internet: youth experiences online. Child Youth Care Forum 2002 Apr;31(2):111-129. [CrossRef]
  64. Hillier L, Harrison L. Building realities less limited than their own: young people practising same-sex attraction on the internet. Sexualities 2007 Feb 1;10(1):82-100. [CrossRef]
  65. Schneider F, Schulz DN, Pouwels LH, de Vries H, van Osch LA. Using a proactive dissemination strategy to optimize reach of an internet-delivered lifestyle intervention. Eur Health Psychol 2014 Dec 1;16:603 [FREE Full text]
  66. Martinez O, Wu E, Shultz AZ, Capote J, López Rios J, Sandfort T, et al. Still a hard-to-reach population? Using social media to recruit Latino gay couples for an HIV intervention adaptation study. J Med Internet Res 2014 Apr 24;16(4):e113 [FREE Full text] [CrossRef] [Medline]
  67. Kurti AN, Logan H, Manini T, Dallery J. Physical activity behavior, barriers to activity, and opinions about a smartphone-based physical activity intervention among rural residents. Telemed J E Health 2015 Jan;21(1):16-23 [FREE Full text] [CrossRef] [Medline]
  68. Moessner M, Bauer S, Ozer F, Wolf M, Zimmer B, Kordy H. Cost-effectiveness of an internet-based aftercare intervention after inpatient treatment in a psychosomatic hospital. Psychother Res 2014;24(4):496-503. [CrossRef] [Medline]
  69. Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PLoS One 2014 May 20;9(5):e98118 [FREE Full text] [CrossRef] [Medline]
  70. Elbert NJ, van Os-Medendorp H, van Renselaar W, Ekeland AG, Hakkaart-van Roijen L, Raat H, et al. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses. J Med Internet Res 2014 Apr 16;16(4):e110 [FREE Full text] [CrossRef] [Medline]
  71. Christensen H, Batterham P, Calear A. Online interventions for anxiety disorders. Curr Opin Psychiatry 2014 Jan;27(1):7-13. [CrossRef] [Medline]
  72. Lucassen MF, Merry SN, Hatcher S, Frampton CM. Rainbow SPARX: a novel approach to addressing depression in sexual minority youth. Cogn Behav Pract 2015 May;22(2):203-216. [CrossRef]
  73. Schwinn TM, Thom B, Schinke SP, Hopkins J. Preventing drug use among sexual-minority youths: findings from a tailored, web-based intervention. J Adolesc Health 2015 May;56(5):571-573. [CrossRef] [Medline]
  74. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457 [FREE Full text] [CrossRef] [Medline]
  75. U.S. National Library of Medicine, Clinical Trials. 2018. Evaluation of an Online Intervention to Help LGBTQ Youth Cope With Bullying   URL: https://clinicaltrials.gov/ct2/show/NCT03501264 [accessed 2018-09-08] [WebCite Cache]
  76. Mustanski B. Ethical and regulatory issues with conducting sexuality research with LGBT adolescents: a call to action for a scientifically informed approach. Arch Sex Behav 2011 Aug;40(4):673-686. [CrossRef] [Medline]
  77. Lenhart A, Duggan M, Perrin A, Stepler R, Rainie L, Parker K. Pew Research Center. 2015 Apr 9. Teen, Social Media and Technology Overview 2015   URL: http://www.pewinternet.org/2015/04/09/teens-social-media-technology-2015/ [WebCite Cache]
  78. Bandura A. Human agency in social cognitive theory. Am Psychol 1989 Sep;44(9):1175-1184. [Medline]
  79. Bandura A. Social cognitive theory: an agentic perspective. Annu Rev Psychol 2001;52:1-26. [CrossRef] [Medline]
  80. Bandura A. Social Foundations Of Thought And Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice Hall; Oct 11, 1985:640.
  81. Bandura A. Social cognitive theory of self-regulation. Organ Behav Hum Decis Process 1991 Dec;50(2):248-287. [CrossRef]
  82. Bandura A. Health promotion by social cognitive means. Health Educ Behav 2004 Apr;31(2):143-164. [CrossRef] [Medline]
  83. Lazarus RS. Psychological Stress and the Coping Process. New York, New York: McGraw-Hill; 1966:466.
  84. Folkman S. Stress, coping, and hope. Psychooncology 2010 Sep;19(9):901-908. [CrossRef] [Medline]
  85. Folkman S. Personal control and stress and coping processes: a theoretical analysis. J Pers Soc Psychol 1984 Apr;46(4):839-852. [Medline]
  86. Folkman S, Lazarus RS, Dunkel-Schetter C, DeLongis A, Gruen RJ. Dynamics of a stressful encounter: cognitive appraisal, coping, and encounter outcomes. J Pers Soc Psychol 1986 May;50(5):992-1003. [Medline]
  87. Lazarus RS, Folkman S. Transactional theory and research on emotions and coping. Eur J Pers 1987 Sep;1(3):141-169. [CrossRef]
  88. Frydenberg E, Lewis R. Adolescent Coping Scale-Second Edition (ACS-2): User Manual. Melbourne, VIC: Australian Council for Educational Research (ACER Press); 2011.
  89. Payton JW, Wardlaw DM, Graczyk PA, Bloodworth MR, Tompsett CJ, Weissberg RP. Social and emotional learning: a framework for promoting mental health and reducing risk behavior in children and youth. J Sch Health 2000 May;70(5):179-185. [Medline]
  90. Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. 1993 Presented at: Proceedings of the INTERACT'93 and CHI'93 conference on Human Factors in Computing Systems; April 24-29, 1993; Amsterdam, The Netherlands p. 206-213. [CrossRef]
  91. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [CrossRef] [Medline]
  92. Johnson D, Gardner MJ, Perry R. Validation of two game experience scales: The Player Experience of Need Satisfaction (PENS) and Game Experience Questionnaire (GEQ). Int J Hum Comput Stud 2018 Oct;118:38-46. [CrossRef]
  93. Wilson CJ, Deane FP, Ciarrochi J, Rickwood D. Measuring help-seeking intentions: properties of the general help seeking questionnaire. Can J Couns Psychother 2005;39(1):15-28 [FREE Full text]
  94. Choi N, Fuqua DR, Griffin BW. Exploratory analysis of the structure of scores from the multidimensional scales of perceived self-efficacy. Educ Psychol Meas 2001 Jun 1;61(3):475-489. [CrossRef]
  95. Bandura A. Multidimensional Scales of Perceived Self-Efficacy. Stanford, CA: Stanford University; 1989.
  96. Pham YK, McWhirter EH, Murray C. Measuring help-seeking behaviors: factor structure, reliability, and validity among youth with disabilities. J Adolesc 2014 Apr;37(3):237-246. [CrossRef] [Medline]
  97. Kato T. Development of the Coping Flexibility Scale: evidence for the coping flexibility hypothesis. J Couns Psychol 2012 Apr;59(2):262-273. [CrossRef] [Medline]
  98. Espelage DL, Holt MK. Bullying and victimization during early adolescence: Peer influences and psychosocial correlates. J Emot Abus 2001 Mar;2(2-3):123-142. [CrossRef]
  99. Second Step: Skills for Social and Academic Success. 2013. Illinois Bully Scale: Student Version   URL: http://www.secondstep.org/Portals/0/G3/BPU/Evaluation_Tools/IllinoisBullyScaleStudent.pdf [WebCite Cache]
  100. Low S, Espelage D. Differentiating cyber bullying perpetration from non-physical bullying: commonalities across race, individual, and family predictors. Psychol Violence 2013;3(1):39-52. [CrossRef]
  101. Roberts RE, Lewinsohn PM, Seeley JR. A brief measure of loneliness suitable for use with adolescents. Psychol Rep 1993 Jun;72(3 Pt 2):1379-1391. [CrossRef] [Medline]
  102. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983 Dec;24(4):385-396. [Medline]
  103. Cohen S, Williamson G. Perceived stress in a probability sample of the United States. In: Spacapan S, Oskamp S, editors. The Social Psychology of Health: The Claremont Symposium on Applied Social Psychology. Newbury Park, CA: Sage Publications; 1988:31-67.
  104. Lebeau RT, Glenn DE, Hanover LN, Beesdo-Baum K, Wittchen HU, Craske MG. A dimensional approach to measuring anxiety for DSM-5. Int J Methods Psychiatr Res 2012 Dec;21(4):258-272. [CrossRef] [Medline]
  105. American Psychiatric Association. 2013. Severity Measure for Generalized Anxiety Disorder-Child Age 11-17   URL: https:/​/www.​psychiatry.org/​File%20Library/​Psychiatrists/​Practice/​DSM/​APA_DSM5_Severity-Measure-For-Generalized-Anxiety-Disorder-Child-Age-11-to-17.​pdf [accessed 2018-08-28] [WebCite Cache]
  106. Johnson JG, Harris ES, Spitzer RL, Williams JB. The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health 2002 Mar;30(3):196-204. [Medline]
  107. Gonzalez C, Miner MH, Bockting WO. An examination of demographic characteristics, components of sexuality, and minority stress as predictors of cannabis, excessive alcohol, and illicit/nonprescription drug use among transgender people in the United States. Journal of Primary Prevention 2017;38(4):419-445 (forthcoming). [Medline]
  108. Bockting WO. Transgender identity, sexuality, and coming out: Implications for HIV risk and prevention. In: Proceedings of the NIDA-sponsored satellite sessions in association with the XIV International AIDS Conference, Barcelona, Spain, July 7-11, 2002. U.S. Bethesda, MD: National Institute on Drug Abuse, U.S. Department of Health and Human Services, National Institutes of Health; 2003:163-172.
  109. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983 Dec;24(4):385-396. [Medline]
  110. Centers for Disease Control and Prevention. 2017. Youth risk behavior surveillance system: High school survey   URL: https://www.cdc.gov/healthyyouth/data/yrbs/index.htm [WebCite Cache]
  111. Centers for Disease Control and Prevention. 2015. Youth risk behavior surveillance system: High school survey   URL: https://www.cdc.gov/healthyyouth/data/yrbs/index.htm [WebCite Cache]
  112. The GenIUSS Group. 2014. Best practices for asking questions to identify transgender and other gender minority respondents on population-based surveys Los Angeles, CA: The Williams Institute   URL: https://williamsinstitute.law.ucla.edu/wp-content/uploads/geniuss-report-sep-2014.pdf [accessed 2018-08-28] [WebCite Cache]
  113. Hatzenbuehler ML, Jun HJ, Corliss HL, Austin SB. Structural stigma and cigarette smoking in a prospective cohort study of sexual minority and heterosexual youth. Ann Behav Med 2014 Feb;47(1):48-56 [FREE Full text] [CrossRef] [Medline]
  114. Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of perceived social support. J Pers Assess 1988 Mar;52(1):30-41. [CrossRef]
  115. Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci 2011 Oct;4(5):332-337 [FREE Full text] [CrossRef] [Medline]


GSA: gay-straight alliance or gender and sexuality alliance
LGBTQ: lesbian, gay, bisexual, transgender, and queer
RCT: randomized controlled trial
SGM: sexual and gender minority
SGMY: sexual and gender minority youth
T1: baseline
T2: first follow-up
T3: final follow-up
YRBS: Youth Risk Behavior Survey


Edited by G Eysenbach; submitted 10.09.18; peer-reviewed by P Janulis, Y Perry, EM Lachmar, R Rosen; comments to author 21.11.18; accepted 10.12.18; published 15.02.19

Copyright

©Robert WS Coulter, Jordan M Sang, William Louth-Marquez, Emmett R Henderson, Dorothy Espelage, Simon C Hunter, Matthew DeLucas, Kaleab Z Abebe, Elizabeth Miller, Brooke A Morrill, Kimberly Hieftje, Mark S Friedman, James E Egan. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 15.02.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.