Published on in Vol 10, No 4 (2021): April

Preprints (earlier versions) of this paper are available at, first published .
A Gamified, Social Media–Inspired, Web-Based Personalized Normative Feedback Alcohol Intervention for Lesbian, Bisexual, and Queer-Identified Women: Protocol for a Hybrid Trial

A Gamified, Social Media–Inspired, Web-Based Personalized Normative Feedback Alcohol Intervention for Lesbian, Bisexual, and Queer-Identified Women: Protocol for a Hybrid Trial

A Gamified, Social Media–Inspired, Web-Based Personalized Normative Feedback Alcohol Intervention for Lesbian, Bisexual, and Queer-Identified Women: Protocol for a Hybrid Trial

Authors of this article:

Sarah C Boyle 1 Author Orcid Image ;   Joseph W LaBrie 1 Author Orcid Image


HeadsUp Labs, Department of Psychology, Loyola Marymount University, Los Angeles, CA, United States

Corresponding Author:

Sarah C Boyle, PhD

HeadsUp Labs

Department of Psychology

Loyola Marymount University

1 LMU Drive

Suite 3787

Los Angeles, CA, 90045

United States

Phone: 1 8182909940


Background: Sexual minority women are more likely to drink alcohol, engage in heavy drinking, and experience alcohol-related problems than heterosexual women. However, culturally tailored interventions for this population have been slow to emerge.

Objective: This type 1 effectiveness-implementation trial examines the feasibility and efficacy of a gamified, culturally tailored, personalized normative feedback (PNF) alcohol intervention for sexual minority women who psychologically identify as lesbian, bisexual, or queer (LBQ).

Methods: The core components of a PNF intervention were delivered within LezParlay, a fun, social media–inspired, digital competition designed to challenge negative stereotypes about LBQ women and increase visibility. The competition was advertised on the web through social media platforms and collaboration with LBQ community organizations. After 2 rounds of play by a large cohort of LBQ women, a subsample of 500 drinkers already taking part in the competition were invited to participate in the evaluation study. Study participants were randomized to receive 1 of 3 unique sequences of PNF (ie, alcohol and stigma coping, alcohol and control, or control topics only) over 2 intervention rounds. Randomization was fully automated by the web app, and both researchers and participants were blinded.

Results: Analyses will evaluate whether PNF on alcohol use reduces participants’ drinking and negative consequences at 2 and 4 months postintervention; examine whether providing PNF on stigma-coping behaviors, in addition to alcohol use, further reduces alcohol use and consequences beyond PNF on alcohol alone; identify mediators and moderators of intervention efficacy; and examine broader LezParlay app engagement, acceptability, and perceived benefits.

Conclusions: This incognito intervention approach is uniquely oriented toward engaging and preventing alcohol-related risks among community populations of LBQ women who may view their heavy drinking as normative and not in need of change because of the visibility of alcohol use in sexual minority community spaces. Thus, this intervention strategy diverges from, and is intended to complement, more intensive programs being developed to meet the needs of LBQ women already motivated to reduce their consumption.

Trial Registration: NCT03884478;

International Registered Report Identifier (IRRID): DERR1-10.2196/24647

JMIR Res Protoc 2021;10(4):e24647




The Institute of Medicine first identified sexual minority women (SMW) as a medically underserved population that disproportionately engage in hazardous drinking and carry the burden of alcohol dependence two decades ago [1]. At present, sexual orientation–based alcohol-use disparities remain with national survey data, revealing lesbian and bisexual women to be among the heaviest drinking female populations [2-4]. For instance, compared with heterosexual women of the same age, lesbian and bisexual women are more than twice as likely to engage in heavy drinking [4,5], 4-7 times more likely to meet the Diagnostic and Statistical Manual of Mental Disorders criteria for alcohol dependence [5,6], and 9-11 times more likely to report that their drinking has led to serious interpersonal consequences [5,6]. In addition, relative to their sexual majority female peers, SMW are more likely to continue patterns of heavy drinking as they age [7,8], increasing their risk for several cancers [8,9] and cardiovascular diseases [10,11]. Despite these striking disparities and consequences, targeted alcohol intervention and prevention efforts for SMW have been slow to emerge [12,13].

Recent research suggests that personalized normative feedback (PNF) may represent a welcomed low-risk, high-reward strategy for reducing alcohol use among SMW, particularly those who psychologically identify as lesbian, bisexual, or queer (LBQ) [14-16]. Founded in the social norms approach [17,18], modern PNF interventions first prompt nontreatment-seeking members of a target group or community to answer a series of web-based survey questions about their perceptions of peers’ alcohol use and their own drinking. Group members then receive individualized digital reports that use bar charts to highlight discrepancies among their perceptions of peers’ drinking, peers’ actual drinking, and their own drinking [19]. To increase the appeal and cost-effectiveness of PNF for this population, we introduced a culturally tailored, social media–inspired, gamified intervention format designed to simultaneously engage heavy drinking LBQ women just as well as their lighter-drinking and alcohol-abstaining peers and remedy the oft-cited attentional and motivational limitations associated with traditional web-based PNF intervention formats.

Why a Brief, Social Norms–Based Alcohol Intervention for SMW?

Although social norms are among the most predictive and commonly targeted antecedents to alcohol use in other heavy drinking populations [20-23] and a growing body of the literature identifies sexual minority–specific peer substance use norms as appropriate targets for intervention and prevention efforts [14,15,24-27], the dominant perspective for understanding heavy drinking among SMW is not that of social norms but rather of sexual minority stress [28-30]. This model emphasizes the role of stigma in SMW’s increased alcohol use, explaining that the increased drinking and dependence among SMW may derive from separate and combined effects of distal stressors, including experiences of prejudice, rejection, harassment, discrimination, and violence originating from a heterosexual society [28,30,31], as well as proximal stressors rooted within the individual, including internalization of stigma, concealment, and inadequate or problematic coping. Growing support for the link between sexual minority stress exposure and coping-motivated drinking [32-38] has encouraged the development of intensive, sexual identity–affirming programs that seek to reduce alcohol use and improve mental health by increasing individuals’ understanding of stigma-related processes and bolstering their adaptive coping skills and resources [39,40]. Despite the promise of these interventions for SMW seeking treatment for their problematic drinking [41,42], these strategies are unlikely to engage the larger population of SMW who do not view their drinking as problematic or lack motivation to reduce their consumption.

In contrast, web-based PNF interventions have the potential to cost-effectively reach and motivate reductions in drinking among SMW not seeking treatment and those who do not yet view their drinking as excessive or a risk to their overall health. Similar to the heavy drinking populations of college students [20,43], military personnel [21,22], and working adults [44] commonly targeted by web-based PNF alcohol interventions, bisexual- and lesbian-identified SMW have been found to overestimate descriptive peer drinking norms [13,14]. Over time, these perceptions of norms have been found to relate to drinking behavior among lesbian and bisexual SMW in the standard reciprocal feed-forward fashion observed in other groups for whom PNF has been effective [15]. Building on these findings, the primary aim of this trial is to examine the extent to which PNF designed to correct sexual identity and age-specific drinking norms is efficacious in augmenting these normative perceptions, thereby reducing LBQ women’s alcohol use and negative consequences. Recent research also suggests that in addition to general peer drinking norms (ie, frequency or quantity of consumption), this population also overestimates peer norms specific to coping-motivated drinking following collectively experienced sexual minority stressors such as the Pulse Nightclub shooting [45] and the 2016 US presidential election [46], with these misperceptions also contributing to their current and future drinking beyond the self-reported stress impact of these events. As such, the secondary aim of this trial is to evaluate whether delivering PNF on stigma-coping behaviors, in addition to alcohol use, further reduces alcohol use and consequences beyond PNF on alcohol use alone.

Testing the Boundaries of PNF Efficacy Among SMW

Few longitudinal studies have simultaneously examined the perceptions of sexual identity–specific substance use norms and concurrently experienced sexual minority stressors as predictors of substance use [47], and no published studies investigating alcohol use among SMW have considered how these predictors may influence or interact with one another over time. Rather, sexual minority stress and stigma have long been positioned in the literature as primary targets for substance use interventions, and the implicit perspective among many lesbian, gay, and bisexual health researchers seems to be that social disadvantage and stigma-related processes may render evidence-based intervention strategies that are effective in other populations ineffective among sexual minorities. Indeed, as the sexual minority stress model [30] positions sexual minority communities as stress-buffering resources and does not consider how perceptions of substance use norms may be artificially inflated by these social environments, it is certainly possible that sexual minority stress and stigma-related processes may render a PNF alcohol intervention ineffective.

However, 2 longstanding social psychological theories support the prediction that correcting misperceived sexual identity–specific drinking norms may be maximally effective among SMW who experience severe interpersonal sexual minority stressors such as violence and harassment because of the sexual minority status. Specifically, self-categorization theory [48-50] contends that when an intergroup threat is experienced in a self-relevant domain, as would be the case when an LBQ woman experiences prejudice, harassment, or violence because of her sexual minority status, she will be particularly likely to turn to perceived in-group norms to guide her behavior. Similarly, terror management theory [51-53] posits that embracing the cultural standards and norms of a self-relevant group can protect against the deeply rooted fears of mortality likely to arise from such threatening intergroup experiences. Thus, following from both theories, to the extent that an LBQ woman experiences prejudice or victimization because of her sexual identity and has inflated perceptions of how much same-identity peers drink, she may be especially likely to increase her drinking as a means of conforming to this inflated normative standard. However, if PNF were to correct her misperceptions of these norms, she should also be especially motivated to align her drinking to the risk-reducing true norm. This study directly examines whether interpersonal stigma exposure moderates the efficacy of PNF designed to correct misperceived norms for LBQ peers’ drinking and coping behaviors.

Additional potential moderators of PNF intervention efficacy examined in this trial include preintervention alcohol consumption and negative consequences, sexual identity, age, race or ethnicity, and relationship status. Relative to lighter drinkers, heavier drinkers naturally have greater room for behavior change following PNF’s correction of drinking norms [19,22]. Thus, heavier drinking SMW are expected to exhibit larger reductions in their drinking post-PNF relative to their lighter-drinking peers. Although sexual identity, race or ethnicity, age, and relationship status are factors associated with variability in alcohol consumption among SMW [54], to date, no published studies have compared the relative sizes of discrepancies between the perceived and actual norms or the strengths of relationships between the perceived norms and alcohol consumption among subgroups of SMW who differ in these demographic characteristics. As such, this trial also evaluates whether these demographic characteristics moderate PNF intervention efficacy among LBQ women.

Addressing PNF Intervention Limitations and Engaging a Hard-to-Reach Population

An extensive study of web-based PNF interventions among college students suggests that these interventions lead to reliable but relatively short-term and modest reductions in drinking [20]. Researchers have identified several issues that, if remedied, could considerably increase the impact of this strategy. In particular, doubts about the credibility of actual drinking norms derived from previously collected data sources [55,56], defensive reactions among heavy drinkers [57,58], general inattention to feedback [59,60], and low motivation among participants [61] have been proposed as barriers to greater public health impact. The real-world suitability of this approach has also drawn criticism [62], as researchers have struggled to implement web-based PNF interventions as well as engage and retain heavy drinkers outside of study settings where participation is mandatory or participants are offered compensation at the point of recruitment [61,63]. Beyond these issues, sexual minority communities present unique challenges for intervention dissemination. Unlike universities and military bases where mandatory PNF interventions can easily target new cohorts of students and recruits, there is no single institution from which SMW can be easily recruited. In contrast, SMW must hear about community programs and judge whether the programs are credible and worthwhile.

One tool commonly used to increase the credibility and appeal of health promotion programs for minority populations is cultural tailoring, which refers to the development of interventions, messages, and materials to conform with specific cultural characteristics of the target group [64]. Recommended cultural tailoring practices for SMW include the development of programs and materials that reflect the social identities, values, and lived experiences of LBQ women as well as the involvement of LBQ community members and trusted community organizations in program promotion and delivery [12,65,66]. Following these recommendations and seeking to bolster intervention relevance, engagement, and motivation, PNF designed to correct drinking and stigma-coping norms were delivered within a larger digital competition called LezParlay. This competition was strategically crafted to reflect deep-structure cultural themes, including community members’ awareness of negative LBQ stereotypes [67-69], desire for increased identity visibility [70-72], and enjoyment of intracommunity competition and sport [73-75]. Consistent with the recommendations for surface-structure intervention tailoring [12,65,76], the LezParlay competition was also developed by an LBQ woman in the target age range and jointly promoted on the web by 4 collaborating community organizations (ie, HER social app, Autostraddle, Lez Do Brunch, and the Los Angeles Lesbian, Gay, Bisexual, and Transgender [LGBT] Center) trusted as sources of health and social information by LBQ women. In addition to cultural tailoring, LezParlay draws upon the self-determination theory (SDT [77,78]) and the nascent gamification literature [79-82] to leverage 4 evidence-based game mechanics (ie, copresence, a system of points, user-generated content, and chance-based uncertainty) to both remedy the limitations associated with traditional PNF intervention formats and foster basic psychological needs for relatedness, competence, and autonomy in this population (see Multimedia Appendix 1 [83-102] for an overview of LezParlay game mechanics and supporting literature).

One round of LezParlay was played monthly over an 8-month period, with a variable cash prize awarded monthly to the top scoring player exhibiting the greatest accuracy in their perceptions of LBQ peers. During the first 3 weeks of each round, players were invited to size-up fellow LBQ players by browsing their social media–like profiles, submit guesses about negative stereotype-related behaviors and experiences of age- and sexual identity–matched LBQ peers (eg, What percentage of [lesbian/bisexual/queer] players in their [20s/30s/40s/50s+] own a pair of Birkenstocks? How many days per week does the typical [lesbian/bisexual/queer] player in her [20s/30s/40s/50s+] drink?), select an amount of points to wager on these guesses being true of other age- and sexual identity–matched players, and earn points for reporting on their own corresponding behaviors and experiences. In the last week of each month, all players received individualized, detailed results (ie, PNF) for a subset of the round’s questions. Animated charts and text detailed the accuracy of the player’s perceptions, how their behaviors and experiences compared with LBQ peers, summarized the stereotypes challenged, and provided their perceptual accuracy–based rank and score. Importantly, all actual norms featured in the detailed results were organically derived from the round-specific reports of players’ behaviors and experiences. Multimedia Appendix 1 provides additional literature supporting this innovative approach to the PNF intervention as well as detailed descriptions of the LezParlay round play and detailed results (ie, PNF screens).

This Study

This registered clinical trial sought to evaluate whether LezParlay-delivered PNF on alcohol use reduces alcohol consumption and negative consequences relative to PNF on control topics (aim 1); examine whether providing PNF on coping behaviors, in addition to alcohol use, further reduces alcohol use and consequences beyond alcohol PNF alone (aim 2); identify mediators (ie, perceived norms) and moderators (ie, interpersonal stigma exposure, baseline drinking, sexual identity, age, relationship status, and race or ethnicity) of intervention efficacy (aim 3); and examine broader LezParlay competition engagement, acceptability, and perceived benefits (aim 4). The following sections provide an overview of the trial design, LezParlay competition promotion efforts, randomized controlled trial (RCT) subsample recruitment, measures, and analysis plan.

Trial Design

Following recent recommendations for testing the real-world feasibility and impact of normative feedback interventions [62,103], LezParlay was examined through a type 1 hybrid-effectiveness-implementation trial [104,105]. That is, in contrast to recruiting LBQ women into a transparent, incentivized, alcohol intervention study, LezParlay was advertised as it would be in the real world—as a free, web-based competition designed to test LBQ stereotypes and increase visibility. Only after several rounds of play were a subsample of 500 drinkers, already participating in the competition, invited to participate in an incentivized evaluation study. These players were covertly randomized to receive 1 of 3 unique sequences of feedback (ie, alcohol+coping, alcohol only, or control only) over 2 consecutive rounds of play. Short-term reductions in norms and drinking were assessed 2 months later organically within the competition through a replay bonus, which invited players to boost their scores by guessing, betting, and reporting on alcohol use and control topics a second time. Following the competition, 4 months postintervention, evaluation study participants then completed a feedback survey assessing competition acceptability, perceived benefits, and feature requests for the next version of the competition. At the end of this survey, participants reported their alcohol use one final time.

Application Technology

LezParlay is a low-cost, device-responsive, HTML5 progressive web application integrated with Facebook Connect and Construct 3 game engine as well as a text message application programming interface, and email server. Although it provided a native app–like feel on Android and Apple smartphones, the app could be accessed on any internet-connected electronic device and did not require players to visit an app store or download any software. Instead, SMW simply accessed the LezParlay web app by URL [106] and were provided instructions for saving the web app to their computer’s desktop or smartphone’s home screen for easy access.

Competition Promotion

The competition was open to all LBQ women aged 21 years or above, regardless of birth sex. Players learned about LezParlay through 1 of 4 promotion strategies taking place over a 3-month period. First, before the launch of the first round, local SMW were invited to sign up through flyers and promotional items distributed at LBQ community events in Los Angeles (ie, a weekend brunch for queer women organized by a community group Lez Do Brunch and a queer casino night jointly organized by the Los Angeles LGBT Center and the Los Angeles Women’s Network). Next, as the first round began, marketing campaigns on the HER social app, the leading dating or social app for LBQ women, invited users in their 3 largest markets (ie, Los Angeles, New York City, and Chicago) to LezParlay via push notifications and in-app advertisements. An advertisement was also placed in the electronic newsletter of Autostraddle, the leading independently owned news website for queer women. During the first 3 rounds of the competition, targeted campaigns on Facebook and Instagram (Facebook Inc) also advertised LezParlay to LBQ women residing in the United States. All recruitment materials were linked to LezParlay’s informational landing page [107]. This page presented an overview of the competition and provided a sign-up button that redirected interested women to view and accept the terms of service and privacy policy (basic consent for competition participation) before creating an account. The institutional review board of Loyola Marymount University approved all recruitment materials, procedures, and intervention materials (protocol number LMUIRB2018SU14).


After consenting to take part in the competition, users were prompted to link a valid mobile phone number to their account, and they could elect to log in with a unique email address and password combination or use their existing Facebook credentials. Next, users created their LezParlay public profile, which included a username of their choice and their sexual identity, age group, relationship status, and pronouns. Users also had the option of uploading a profile photo or Bitmoji to represent them; entering a brief textual self-description; and connecting their Facebook, Twitter, and/or Instagram accounts so that other players could learn about them. Following account creation, players were directed to a home screen that displayed a timer counting down to the close of the current round as well as buttons to play the current round, browse player profiles, submit and vote on the questions to be parlayed in future rounds, view round winners and leaderboards, edit public profile, and change account settings. The specifics of round play and the format of the detailed results (ie, PNF) delivered at the end of each round are detailed in Multimedia Appendix 1.

RCT Subsample Recruitment

There was no upper limit on the number of SMW who could take part in LezParlay, and new players were accepted on a rolling basis throughout the competition. We aimed to recruit a minimum of 1200 LBQ women to sign up during the first 2 monthly rounds to ensure that meaningful and stable sexual identity– and age group–specific actual norms for drinking and coping behaviors could be delivered in intervention rounds 3 and 4. From this larger pool of players, 500 drinkers were recruited to participate in a LezParlay evaluation study (RCT) during the third month of play. Acting as baseline (T1) for the RCT, round 3 featured questions about alcohol use, stigma experiences, and a group of nonhealth-related control questions submitted by players. Upon submitting answers to alcohol-related questions in round 3, players were covertly screened for evaluation study eligibility based on their answers (ie, number of drinking days per week and peak drinks on a single day during the past 2 months) as well as their geolocation and the number of previous rounds played. Those who played at least one previous round, were in the United States, and reported drinking alcohol on 3 or more days per week or having 3 or more drinks on their peak drinking occasion were invited to take part in the evaluation study at the end of the round. Interested potential participants advanced to an informed consent screen that explained that the goal of the study was to evaluate the impact and format of detailed results received in LezParlay and gather player feedback to inform the next version of the competition. The information further detailed that participation in the evaluation study simply involved playing and viewing detailed results in subsequent rounds and completing a brief feedback survey at the end of the competition. Participants could earn up to US $40 in electronic gift cards of their choice to play subsequent rounds and complete the feedback survey. Those who checked a box indicating that they understood what the study participation entailed and desired to participate were welcomed into the study as LezParlay official testers.

RCT Design, Randomization, and Debriefing

The web app’s Qualtrics integration ensured that Qualtrics Research Suite’s automated randomizer, commonly used in RCTs evaluating psychosocial interventions [40,41,55], could be used to randomize evaluation study participants to a PNF condition at the point of study enrollment in round 3. Randomization determined the sequence of topics on which participants received detailed results across intervention rounds 3 and 4: alcohol+coping, alcohol+control, or control only. Members of the research team were blinded to participant condition assignment, and the study participants were not aware that any sort of randomization was taking place. Rather, when detailed results were sent at the end of each round, players were prompted to choose among 3 animated, graphical doors to determine the 1 to 2 round topics on which they would view detailed results (see Figure S2 in Multimedia Appendix 1). Although the results topics were truly determined via chance in most rounds of the competition, the doors of evaluation study participants were fixed to open to their randomly assigned feedback topics regardless of the door they selected in rounds 3 and 4. Upon completing the feedback survey at the end of the competition, participants were debriefed regarding the study’s research questions and the fixed sequences of health or control feedback they were randomized to receive in rounds 3 and 4 of the competition.

Intervention Rounds

All players taking part in the third round of LezParlay estimated the drinking behaviors of the typical same-sexual identity player in their age group during the previous 2 months, reporting on their perceptions of the typical player’s (1) maximum number of drinks consumed on a single occasion, (2) average number of drinks consumed per occasion, and (3) average number of drinking days per week [103]. Players also estimated the number of negative alcohol-related consequences experienced over the previous 2 months by a typical player in their sexual identity and age group from a list of 8 negative consequences (ie, had a hangover or illness, got in a physical or verbal fight, had problems with significant others, missed a social engagement or event, had problems with friends or family, performed poorly at work or school, had problems with money, and had an unwanted or regrettable sexual experience). Players then answered parallel items assessing their own drinking and consequences over the corresponding 2-month period.

Players taking part in round 4 of the competition were prompted to think about how other players deal with stress and sexual minority stigma and asked to estimate the percentage of time (ie, 0%-100%) a typical player in their sexual identity and age group tried to feel better during the past month by (1) drinking alcohol; (2) taking a drug; (3) meditating, using relaxation techniques, or exercising; and (4) talking to a close other or mental health professional. Players were then prompted to think about how they themselves dealt with stress and stigma and responded to parallel items. The actual norms variably delivered to evaluation study participants in PNF at the end of rounds 3 and 4 were derived by computing the actual average response of all players submitting responses in each sexual identity and age group.

Participants randomized to receive PNF on control topics received detailed results for nonhealth-related topics in round 3 (eg, household repair ability, frequency of home improvement store visits, or tool box ownership) and round 4 (eg, time in between relationships, texting exes, and partners being confused for sisters). Multimedia Appendix 1 includes a link to view an example of detailed results for a control topic and a treatment topic delivered in the intervention rounds.

RCT Measures

Demographic and Psychosocial Covariates

At sign up, all players reported their sexual identity, relationship status, and age group. Upon enrolling in the evaluation study, participants also reported their race, ethnicity, and actual age in years. The feedback survey at the end of the competition prompted the study participants to rereport their relationship status and sexual identity.

Perceived Alcohol-Related Norms and Behaviors

As described previously, perceived drinking norms and alcohol-use behaviors were assessed organically in competition rounds 3 (T1; baseline) and 7 (T2; 2-month follow-up) by items modeled after Baer’s quantity, frequency, and max measure [108] in combination with additional norm and behavior items, respectively, examining negative alcohol-related consequences. These items were assessed a final time at the end of the postcompetition survey (T3; 4-month follow-up). Measures at each timepoint were referenced from the previous 2-month period. As done in previous gamified PNF pilot studies with college students [43,109,110], composite measures of perceived alcohol-use norms and alcohol-use behavior at baseline and follow-up will be computed by z scoring and then averaged across respective sets of individual items at each timepoint. In addition to these composites, 3 key outcomes of interest in alcohol intervention research are to be examined individually pre- and postintervention: (1) estimated drinks per week over the previous 2 months (computed by multiplying the reported number of drinking days per week and the average number of drinks per occasion at each timepoint), (2) peak drinks on 1 occasion over the previous 2 months, and (3) the number of negative alcohol-related consequences over the previous 2 months.

Interpersonal Stigma Exposure

Interpersonal stigma exposure was also assessed at baseline (round 3) and follow-up (as a replay bonus topic in round 7). Players guessed about the stigma experiences of other players and reported on their own stigma experiences over the previous 2 months. Stigma-related norms were not corrected in the competition, and these perceptions were only assessed to make sense for players to report on their own recent interpersonal stigma exposure (a theorized moderator of conditional effects on drinking) at the same time that they were reporting on their alcohol use and negative consequences in the game. Players’ recent exposure to severe interpersonal stigma was assessed by their responses to 2 items: (1) “During the past 2 months, how many times have you been physically harmed due to your sexual identity?” and (2) “During the past 2 months, how many times have you been verbally harassed or threatened (online or in person) due to your sexual identity?” Associations between pairs of stigma items at each timepoint are expected, and we anticipate combining responses to derive severe interpersonal stigma scores.

Feasibility Measures

Reach and Engagement

Data from Google Analytics and the app’s back end will allow us to examine the total number of players who signed up to participate in the LezParlay competition in the absence of traditional study participation incentives; identify the promotional channels that brought them to the app; and detail the players’ demographic characteristics, states of residence, average number of log-ins, and number of rounds completed.


Feedback surveys prompted study participants to rate numerous aspects of the competition (the stereotype challenge concept, topics and questions, detailed results, leaderboards, the ability to browse player profiles, the ability to submit questions, the ability to bet points on the accuracy of guesses, text messages, and email communications from LezParlay) on Likert-type scales ranging from did not like at all (0) to liked very much (5).

Perceived Benefits

A single yes or no item asked participants whether they felt that participating in the LezParlay competition was psychologically beneficial. Those selecting yes in response were asked to enter text describing the perceived benefits.

Improvements and Requested Features

A final free-response item asked participants to share recommendations they had for improving the competition and describing the features they would like to see in the next version.


This project was funded by the National Institute on Alcohol Abuse and Alcoholism in April 2018. The institutional review board approval was granted in August 2018, and the LezParlay app was developed per approved protocol between August 2018 and November 2018. The competition launched in January 2019, and the collection of all efficacy and feasibility data was completed in September 2019. A total of 2677 LBQ women participated in the LezParlay competition, with 500 LBQ drinkers recruited into the efficacy trial. Data cleaning and analysis were delayed by several months because of COVID-19–related delays and is underway as of January 2021. The results are expected to be published in summer 2021.

Data Analytic Plan for Evaluating Intervention Efficacy

An intent-to-treat approach will be used to examine LezParlay treatment effects at 2 and 4 months postintervention on 4 outcomes: composite alcohol use, estimated number of drinks per week, peak number of drinks on 1 occasion, and number of negative alcohol-related consequences. Preliminary analyses will examine potential biases related to attrition and missing data [111,112], inspect outcome distributions, and evaluate potential baseline differences among conditions. As the latter 3 outcomes are count variables (ie, estimated drinks per week, peak drinks, and negative consequences), they are likely to be substantially skewed and best approximated by either Poisson or negative binomial distributions.

Main Effects

At 2 and 4 months after the delivery of treatment PNF, participants in both conditions receiving treatment PNF on alcohol use (ie, alcohol+coping and alcohol only) are expected to report reduced drinks per week, peak drinks, and negative consequences relative to those in the control PNF condition. Furthermore, participants in the alcohol+coping condition are expected to exhibit larger reductions in their alcohol use and negative consequences at postintervention follow-ups than participants in the alcohol-only PNF condition. Multilevel models (MLMs [113,114]) with full maximum likelihood specification will be used to test these predictions. Time will be specified as a level 1 varying predictor nested within individuals (level 2). Intercept treatment differences will represent treatment differences at baseline (eg, conditional differences in drinking at baseline), and slope differences will represent changes over time (eg, did participants in treatment conditions reduce their drinking between baseline and follow-up assessments more than control participants). The intercept includes a random effect, which will model the subject-specific heterogeneity in alcohol-related outcomes, thereby controlling for correlated data due to individuals. Main effect models will also control for covariates: age, sexual identity, race, ethnicity, relationship status, and severe interpersonal stigma exposure.

Tests of Mediation and Moderation

Tests of mediation will examine whether perceived drinking norms at the 2-month follow-up mediate relationships between condition and alcohol-use outcomes at the 4-month follow-up. PROCESS bootstrap tests [115,116] will be used to test the mediation. These models will control for baseline measures of potential mediating variables (ie, norms) and outcomes (ie, alcohol use and consequences). Moderation analyses will be examined within an MLM framework and will examine whether the efficacy of treatment PNF varied as a function of participants’ baseline drinking, sexual identity, exposure to severe interpersonal stigma, or other demographic characteristics. In the presence of significant interactions, exploratory moderated mediation models [115,117,118] may simultaneously estimate the conditional direct and indirect effects associated with the different levels of moderating variables.

Power Analysis

Informed by previous research examining the effects of web-based alcohol PNF on changes in normative perceptions and drinking in other populations (Cohen d=0.22 [20,119]), the comparably larger effect size revealed in a similar gamified PNF intervention for college students (Cohen d=0.46 [109]), power analyses using the standard 0.80 power of detecting a significant effect, P<.05, and an effect size of Cohen d=0.30 indicate a sample size of 375 (125 participants in each condition) to be sufficient to detect small-to-medium effects using repeated measures MLMs (ie, 2 levels, 3 arms, and randomization at the individual level) as well as tests of mediation and moderation. Thus, our sample size of 500 will allow us to detect modest effects with even 30% attrition.

Data Analytic Plan for Evaluating Feasibility

Descriptive statistics will allow us to assess SMW’s level of interest in the LezParlay competition and engagement with the app (ie, total number of sign-ups and average number of log-ins), recruitment origins (eg, HER app ad, Facebook, Instagram, and player referral), acceptability (mean rating overall and by competition component), and perceived psychological benefits (ie, proportion of evaluation study participants who reported benefits). Qualitative text entry responses to items assessing the perceived benefits of the LezParlay competition and improvements or features requested for the next version will also be coded by theme or category using a generic inductive qualitative coding approach [120].

Trial Overview

Joining the growing number of gamified health interventions being developed for sexual minority men and youth [121-126] and extending gamified PNF pilot work with college students [43,83,109,110], this project leverages evidence-based digital game mechanics informed by SDT to deliver PNF within a novel digital competition designed to challenge LBQ stereotypes and increase visibility. Notably, several deep-structure themes well documented among LBQ women are incorporated into the incognito intervention to bolster both relevance to and resilience within this population. For instance, awareness of stigmatizing sexual identity–based stereotypes is well documented among LBQ women [63-66], and social norms theory predicts that players are likely to overestimate undesirable stereotypical behaviors among peers of same-sexual identity (eg, promiscuity and infidelity among bisexual women, unhealthy relationship behaviors, and transphobic attitudes among lesbians). As such, we anticipate that revealing and reinforcing true norms for such experiences and attitudes may carry psychological benefits (ie, reducing identity-related stigma and increasing collective self-esteem) for participants beyond reduced alcohol consumption.

This hybrid trial also follows recent recommendations for improved design and evaluation of social norms–based health interventions [62], as both mediators and moderators of intervention effectiveness are examined, and, importantly, the LezParlay competition is framed and advertised as it would be outside of the study setting. Only later is an incentivized clinical trial subsample of drinkers recruited from the larger population of players already engaging with the competition (notably in the absence of traditional study participation incentives). This hybrid design allows the research team to cost-effectively assess the feasibility of drawing large numbers of SMW to the broader LezParlay competition via targeted promotional channels, player engagement with different areas of the web app, and competition acceptability and ways in which the competition might be improved. Simultaneously, recruiting a subsample of alcohol-consuming SMW already taking part in the competition into an incognito RCT allows for the evaluation of whether PNF on alcohol use and stigma-coping behaviors meaningfully reduces alcohol consumption and negative consequences relative to control PNF. In sum, this design allows critical questions about feasibility and efficacy to be jointly addressed, with minimal costs to internal or external validity.

The examination of demographics in terms of both LezParlay engagement and moderators of intervention efficacy will identify the groups of SMW most engaged and impacted by this strategy. Furthermore, as previous work with SMW has not examined potential interactions between perceptions of sexual identity–specific drinking norms and experiences with violence or harassment because of sexual minority status, examining the potential interplay between these established predictors of drinking allows this project to make a significant contribution to the larger sexual minority health literature. Similarly, as normative feedback interventions have not been widely considered as potential strategies for reducing problematic substance use or other health-risk behaviors among members of stigmatized health disparity populations, determining whether minority identity–based violence and harassment makes PNF more or less efficacious is a critical research question that may also carry intervention development implications for these populations (eg, racial and ethnic minorities and gender minorities).

Limitations and Future Directions

Funded by the National Institutes of Health’s exploratory/developmental R21 grant mechanism, the goal of this initial trial is to evaluate the feasibility and efficacy associated with the LezParlay competition as a minimally viable product taking the form of an extremely low-cost, progressive web app designed and coded by the first author, who is a member of the target population. Thus, representing the preliminary step in a larger program of LezParlay-gamified PNF intervention research, findings from this trial will not speak to the feasibility or efficacy of delivering the intervention through a more sophisticated and polished native smartphone app that would likely be more desirable and user-friendly and thus better equipped to attract and retain LBQ women. In addition, PNF is the only intervention component featured in the initial version of LezParlay and, furthermore, only static descriptive norms for drinking and coping are corrected. Findings from this study may suggest high feasibility for the gamified approach and stereotype challenge framing (ie, large numbers of SMW are engaged by the competition and participants and report psychological benefits), but treatment PNF fails to meaningfully reduce drinkers’ consumption and negative alcohol-related consequences relative to control. In this event, the stereotype challenge concept and game mechanics may be retained; however, future versions of the LezParlay app might expand PNF to correct additional types of alcohol and coping norms (ie, injunctive, affective, and dynamic norms) and/or deliver additional intervention components such as skills training around healthy coping strategies and/or local alcohol treatment information (ie, referral to treatment).

The intervention’s real-time, organic generation of actual norms from LezParlay players’ round-specific reports of their behaviors represents both a point of innovation and a potential limitation. Foremost, this approach diverges significantly from traditional PNF interventions that derive actual drinking norms from an existing data source or a separate norms documentation study conducted before the recruitment of the intervention sample [19-22]. As PNF research suggests that the practice of using previously collected actual norms data may undermine intervention efficacy by diminishing PNF credibility and interest [55-57,110], LezParlay sought to increase the psychological proximity and relatability of the peer group by making the individuals on whom actual norms are based digitally present and visible. As a point of innovation, this format may help extend PNF interventions to sexual minorities and other hard-to-reach populations for whom existing norms data do not exist and would be difficult and/or costly to collect through a separate survey study. Furthermore, trial results from a similar gamified PNF intervention for college students revealed that the actual drinking norms similarly derived in-game among visible peers differed very little from those derived from separate survey samples of students at the same university [109]. However, as this approach has not previously been tested with adult LBQ women, it is critical to establish that the actual norms derived in real time from LezParlay rounds are sufficiently risk-reducing, stable, and approximately equivalent to those derived from comparable survey samples containing the same age and sexual identity groups [14-16,91].

Another limitation pertains to this study’s organic assessment of alcohol outcomes and some moderators within the rounds of the competition. Although this is a major strength, in that it eliminates the demand characteristics that often plague transparent alcohol intervention studies and substantially decreases trial costs, this also meant that the key constructs could only be assessed by a few items, and the language of the items could not be too formal or clinical in tone. Although Baer’s frequency, quantity, and max measure [103] fit well in this regard as a short, validated measure of alcohol use, it would have also been valuable to include longer validated survey measures to more formally assess alcohol-related outcomes and screen for alcohol use disorder. In the event that LezParlay feasibility and efficacy are demonstrated in the initial trial, we anticipate seeking additional funding for a larger trial that will include traditional survey-based baseline and follow-up assessments and test an expanded set of potential mediators and moderators.

A final limitation pertains to this study’s narrow focus on the direct effect of PNF treatment on alcohol-related outcomes. Norms for other health behaviors (ie, stigma coping, smoking, exercise, and health care utilization) were also corrected within the broader competition; however, the RCT was not designed to examine potential PNF-related changes in these behaviors. Similarly, players were expected to overestimate several undesirable stereotypical behaviors among same-sexual identity peers in nonintervention rounds of the competition (eg, promiscuity and infidelity among bisexual women, unhealthy relationship behaviors, and transphobic attitudes among lesbians). Although outside the scope of this initial trial, revealing and reinforcing true norms for these experiences and attitudes in LezParlay may carry psychological benefits for SMW partaking in the competition (ie, reducing identity-related stigma, increasing feelings of belonging, and/or collective self-esteem). Thus, assessing pre-post competition changes in these constructs and evaluating the extent to which challenging negative stereotypes through the larger LezParlay competition might buffer stigma-related processes and thereby reducing drinking and improving other health outcomes remain to be critical next steps in the larger program of research.


This hybrid trial will examine the efficacy and feasibility of an innovative, culturally tailored, evidence-based alcohol intervention for LBQ moderate-to-heavy drinkers, thereby narrowing the disparity in alcohol intervention research and practice. This incognito, gamified intervention approach is uniquely oriented toward engaging and preventing alcohol-related risks among LBQ community members who may view their heavy drinking as normative and not in need of change because of the visibility of alcohol consumption in LBQ community spaces. Thus, this intervention strategy diverges from and is intended to complement more intensive intervention programs being developed to meet the needs of SMW already motivated to reduce their consumption and those seeking culturally tailored treatment for alcohol-use disorder and comorbid mental health problems [40,41].


Manuscript preparation was supported by developmental/exploratory grant R21AA025767-01A from the National Institute on Alcohol Abuse and Alcoholism. The content is solely the responsibility of the authors and does not necessarily reflect the official views of the National Institute on Alcohol Abuse and Alcoholism or its affiliates.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Overview of LezParlay game mechanics, supporting literature, round play, and detailed results.

DOCX File , 490 KB

Multimedia Appendix 2

Peer-review report by the National Institute on Alcohol Abuse and Alcoholism.

PDF File (Adobe PDF File), 186 KB

  1. Institute of Medicine. Lesbian health status and health risks. In: Solarz A, editor. Current Assessment and Directions for the Future. Washington, DC: National Academies Press; 1999.
  2. Green KE, Feinstein BA. Substance use in lesbian, gay, and bisexual populations: an update on empirical research and implications for treatment. Psychol Addict Behav 2012 Jun;26(2):265-278 [FREE Full text] [CrossRef] [Medline]
  3. Hughes TL, Wilsnack SC, Kantor LW. The Influence of Gender and Sexual Orientation on Alcohol Use and Alcohol-Related Problems: Toward a Global Perspective. Alcohol Res 2016;38(1):121-132 [FREE Full text] [Medline]
  4. Fish JN. Sexual Orientation-Related Disparities in High-Intensity Binge Drinking: Findings from a Nationally Representative Sample. LGBT Health 2019 Jul;6(5):242-249 [FREE Full text] [CrossRef] [Medline]
  5. McCabe S, Hughes T, Bostwick W, West B, Boyd C. Sexual orientation, substance use behaviors and substance dependence in the United States. Addiction 2009 Aug 1;104(8):1333-1345. [CrossRef]
  6. Drabble L, Trocki K. Alcohol Consumption, Alcohol-Related Problems, and Other Substance Use Among Lesbian and Bisexual Women. Journal of Lesbian Studies 2008 Sep 22;9(3):19-30. [CrossRef]
  7. Burgard SA, Cochran SD, Mays VM. Alcohol and tobacco use patterns among heterosexually and homosexually experienced California women. Drug and Alcohol Dependence 2005 Jan;77(1):61-70. [CrossRef]
  8. Fredriksen-Goldsen KI, Kim H, Barkan SE, Muraco A, Hoy-Ellis CP. Health disparities among lesbian, gay, and bisexual older adults: results from a population-based study. Am J Public Health 2013 Oct;103(10):1802-1809 [FREE Full text] [CrossRef] [Medline]
  9. Fredriksen-Goldsen K, Hoy-Ellis C, Brown M. Addressing behavioral cancer risks from a LGBT health equity perspective. In: Boehmer U, Elk R. editors. Cancer LGBT Community Unique Perspect Risk Surviv Cham: Springer International Publishing; 2015.
  10. Caceres BA, Brody A, Luscombe RE, Primiano JE, Marusca P, Sitts EM, et al. A Systematic Review of Cardiovascular Disease in Sexual Minorities. Am J Public Health 2017 Apr;107(4):e13-e21. [CrossRef] [Medline]
  11. Farmer GW, Jabson JM, Bucholz KK, Bowen DJ. A Population-Based Study of Cardiovascular Disease Risk in Sexual-Minority Women. Am J Public Health 2013 Oct;103(10):1845-1850. [CrossRef]
  12. Bloom AW. Advances in Substance Abuse Prevention and Treatment Interventions Among Racial, Ethnic, and Sexual Minority Populations. Alcohol Res 2016;38(1):47-54 [FREE Full text] [Medline]
  13. Hughes TL, Veldhuis CB, Drabble LA, Wilsnack SC. Research on alcohol and other drug (AOD) use among sexual minority women: A global scoping review. PLoS ONE 2020 Mar 18;15(3):e0229869. [CrossRef]
  14. Boyle SC, LaBrie JW, Witkovic YD. Do lesbians overestimate alcohol use norms? Exploring the potential utility of personalized normative feedback interventions to reduce high-risk drinking in Southern California lesbian communities. Journal of Gay & Lesbian Social Services 2016 Jul 15;28(3):179-194. [CrossRef]
  15. Ehlke SJ, Stamates AL, Kelley ML, Braitman AL. Bisexual women’s reports of descriptive drinking norms for heterosexual, bisexual, and lesbian women. Psychology of Sexual Orientation and Gender Diversity 2019 Jun;6(2):256-263. [CrossRef]
  16. Litt DM, Lewis MA, Rhew IC, Hodge KA, Kaysen DL. Reciprocal relationships over time between descriptive norms and alcohol use in young adult sexual minority women. Psychology of Addictive Behaviors 2015 Dec;29(4):885-893. [CrossRef]
  17. Perkins HW, editor. The social norms approach to preventing school and college age substance abuse: A handbook for educators, counselors, clinicians. San Francisco, CA: Jossey-Bass; 2003.
  18. Perkins HW, Berkowitz AD. Perceiving the Community Norms of Alcohol Use among Students: Some Research Implications for Campus Alcohol Education Programming. International Journal of the Addictions 1986 Oct;21(9-10):961-976. [CrossRef] [Medline]
  19. Miller DT, Prentice DA. Changing Norms to Change Behavior. Annu Rev Psychol 2016;67:339-361. [CrossRef] [Medline]
  20. Dotson KB, Dunn ME, Bowers CA. Stand-Alone Personalized Normative Feedback for College Student Drinkers: A Meta-Analytic Review, 2004 to 2014. PLoS ONE 2015 Oct 8;10(10):e0139518. [CrossRef]
  21. Neighbors C, Walker D, Rodriguez L, Walton T, Mbilinyi L, Kaysen D, et al. Normative Misperceptions of Alcohol Use Among Substance Abusing Army Personnel. Military Behavioral Health 2014 Jun 03;2(2):203-209. [CrossRef]
  22. Pedersen ER, Parast L, Marshall GN, Schell TL, Neighbors C. A randomized controlled trial of a web-based, personalized normative feedback alcohol intervention for young-adult veterans. J Consult Clin Psychol 2017 May;85(5):459-470 [FREE Full text] [CrossRef] [Medline]
  23. Davis JP, Houck JM, Rowell LN, Benson JG, Smith DC. Brief Motivational Interviewing and Normative Feedback for Adolescents: Change Language and Alcohol Use Outcomes. Journal of Substance Abuse Treatment 2016 Jun;65:66-73. [CrossRef]
  24. Mereish EH, Goldbach JT, Burgess C, DiBello AM. Sexual orientation, minority stress, social norms, and substance use among racially diverse adolescents. Drug and Alcohol Dependence 2017 Sep;178:49-56. [CrossRef]
  25. Ramchand R, Fisher MP, Griffin BA, Becker K, Iguchi MY. Drug Use Among Gay and Bisexual Men at Weekend Dance Parties: The Role of Intentions and Perceptions of Peers’ Behaviors. AIDS Behav 2012 Dec 28;17(4):1540-1549. [CrossRef]
  26. Talley AE, Gilbert PA, Mitchell J, Goldbach J, Marshall BDL, Kaysen D. Addressing gaps on risk and resilience factors for alcohol use outcomes in sexual and gender minority populations. Drug and Alcohol Review 2016 Apr 13;35(4):484-493. [CrossRef]
  27. Hamilton CJ, Mahalik JR. Minority stress, masculinity, and social norms predicting gay men's health risk behaviors. Journal of Counseling Psychology 2009 Jan;56(1):132-141. [CrossRef]
  28. Hatzenbuehler ML. How does sexual minority stigma “get under the skin”? A psychological mediation framework. Psychological Bulletin 2009 Sep;135(5):707-730. [CrossRef]
  29. Lick DJ, Durso LE, Johnson KL. Minority Stress and Physical Health Among Sexual Minorities. Perspect Psychol Sci 2013 Sep;8(5):521-548. [CrossRef] [Medline]
  30. Meyer IH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychol Bull 2003 Sep;129(5):674-697 [FREE Full text] [CrossRef] [Medline]
  31. Lehavot K, Simoni JM. The impact of minority stress on mental health and substance use among sexual minority women. Journal of Consulting and Clinical Psychology 2011 Apr;79(2):159-170. [CrossRef]
  32. Lewis RJ, Winstead BA, Lau-Barraco C, Mason TB. Social Factors Linking Stigma-Related Stress with Alcohol Use among Lesbians. Journal of Social Issues 2017 Sep 21;73(3):545-562. [CrossRef]
  33. McCabe SE, Bostwick WB, Hughes TL, West BT, Boyd CJ. The Relationship Between Discrimination and Substance Use Disorders Among Lesbian, Gay, and Bisexual Adults in the United States. Am J Public Health 2010 Oct;100(10):1946-1952. [CrossRef]
  34. Feinstein BA, Newcomb ME. The role of substance use motives in the associations between minority stressors and substance use problems among young men who have sex with men. Psychology of Sexual Orientation and Gender Diversity 2016 Sep;3(3):357-366. [CrossRef]
  35. Livingston NA, Christianson N, Cochran BN. Minority stress, psychological distress, and alcohol misuse among sexual minority young adults: A resiliency-based conditional process analysis. Addictive Behaviors 2016 Dec;63:125-131. [CrossRef]
  36. Dorn-Medeiros CM, Doyle C. Alcohol as Coping: Internalized Homophobia and Heterosexism’s Role in Alcohol Use among Lesbians. Journal of LGBT Issues in Counseling 2019 Jan 07;12(3):142-157. [CrossRef]
  37. Nawyn SJ, Richman JA, Rospenda KM, Hughes TL. Sexual Identity and Alcohol-Related Outcomes: Contributions of Workplace Harassment. Journal of Substance Abuse 2000 May;11(3):289-304. [CrossRef]
  38. McNair R, Pennay A, Hughes T, Brown R, Leonard W, Lubman DI. A model for lesbian, bisexual and queer-related influences on alcohol consumption and implications for policy and practice. Cult Health Sex 2016;18(4):405-421. [CrossRef] [Medline]
  39. Chaudoir SR, Fisher JD, Simoni JM. Understanding HIV disclosure: a review and application of the Disclosure Processes Model. Soc Sci Med 2011 May;72(10):1618-1629 [FREE Full text] [CrossRef] [Medline]
  40. Millar BM, Wang K, Pachankis JE. The moderating role of internalized homonegativity on the efficacy of LGB-affirmative psychotherapy: Results from a randomized controlled trial with young adult gay and bisexual men. Journal of Consulting and Clinical Psychology 2016 Jul;84(7):565-570. [CrossRef]
  41. Pachankis JE, McConocha EM, Clark KA, Wang K, Behari K, Fetzner BK, et al. A transdiagnostic minority stress intervention for gender diverse sexual minority women’s depression, anxiety, and unhealthy alcohol use: A randomized controlled trial. Journal of Consulting and Clinical Psychology 2020 Jul;88(7):613-630. [CrossRef]
  42. Benz MB, Palm Reed L, Bishop LS. Stigma and help-seeking: the interplay of substance use and gender and sexual minority identity. Addict Behav 2019 Oct;97:63-69. [CrossRef] [Medline]
  43. Boyle SC, Earle AM, LaBrie JW, Smith DJ. PNF 2.0? Initial evidence that gamification can increase the efficacy of brief, web-based personalized normative feedback alcohol interventions. Addictive Behaviors 2017 Apr;67:8-17. [CrossRef]
  44. Walters S, Woodall W. Mailed feedback reduces consumption among moderate drinkers who are employed. Prevention Science 2003 Dec 1;4(4):294. [CrossRef]
  45. Boyle SC, LaBrie JW, Costine LD, Witkovic YD. “It's how we deal”: Perceptions of LGB peers' use of alcohol and other drugs to cope and sexual minority adults' own coping motivated substance use following the Pulse nightclub shooting. Addictive Behaviors 2017 Feb;65:51-55. [CrossRef]
  46. Boyle S, Costine L, LaBrie J. Cross-sectional and feed-forward relationships between peer coping norms and LGB adults own coping-motivated alcohol use following stressors. Alcoholism Clinical Experimental Research 2017;41:259A.
  47. Boyle SC, LaBrie JW, Omoto AM. Normative substance use antecedents among sexual minorities: A scoping review and synthesis. Psychology of Sexual Orientation and Gender Diversity 2020 Jun;7(2):117-131. [CrossRef]
  48. Turner J, Hogg M, Oakes P, Reicher S, Wetherell M. Rediscovering the social group: A self-categorization theory. Cambridge, MA: Basil Blackwell; 1987.
  49. Turner J, Reynolds K. Self-categorization theory. In: Van Lange PA, Kruglanski AW, Higgins ET, editors. Handbook of Theories in Social Psychology. UK: Sage Publications; 2011:399-417.
  50. Hogg MA, Reid SA. Social Identity, Self-Categorization, and the Communication of Group Norms. Commun Theory 2006 Feb;16(1):7-30. [CrossRef]
  51. Greenberg J, Pyszczynski T, Solomon S. The causes and consequences of a need for self-esteem: A terror management theory. In: Baumeister RF, editor. Public Self Private Self. New York: Springer; 1986:189-212.
  52. Greenberg J, Solomon S, Pyszczynski T. Terror management theory of self-esteem and cultural worldviews: Empirical assessments and conceptual refinements. Advances in Experimental Social Psychology 1997;29:61-139. [CrossRef]
  53. Jonas E, Martens A, Niesta Kayser D, Fritsche I, Sullivan D, Greenberg J. Focus theory of normative conduct and terror-management theory: The interactive impact of mortality salience and norm salience on social judgment. Journal of Personality and Social Psychology 2008;95(6):1239-1251. [CrossRef]
  54. Hughes TL, Wilsnack SC, Kantor LW. The Influence of Gender and Sexual Orientation on Alcohol Use and Alcohol-Related Problems: Toward a Global Perspective. Alcohol Res 2016;38(1):121-132 [FREE Full text] [Medline]
  55. Hummer JF, Davison GC. Examining the Role of Source Credibility and Reference Group Proximity on Personalized Normative Feedback Interventions for College Student Alcohol Use: A Randomized Laboratory Experiment. Subst Use Misuse 2016 Nov 09;51(13):1701-1715. [CrossRef] [Medline]
  56. Miller MB, Leffingwell TR. What do college student drinkers want to know? Student perceptions of alcohol-related feedback. Psychology of Addictive Behaviors 2013;27(1):214-222. [CrossRef]
  57. Granfield R. Alcohol use in college: Limitations on the transformation of social norms. Addiction Research & Theory 2009 Jul 11;13(3):281-292. [CrossRef]
  58. Steers MN, Coffman AD, Wickham RE, Bryan JL, Caraway L, Neighbors C. Evaluation of Alcohol-Related Personalized Normative Feedback With and Without an Injunctive Message. J. Stud. Alcohol Drugs 2016 Mar;77(2):337-342. [CrossRef]
  59. Lewis MA, Neighbors C. An examination of college student activities and attentiveness during a web-delivered personalized normative feedback intervention. Psychology of Addictive Behaviors 2015;29(1):162-167. [CrossRef]
  60. Rodriguez LM, Neighbors C, Rinker DV, Lewis MA, Lazorwitz B, Gonzales RG, et al. Remote versus in-lab computer-delivered personalized normative feedback interventions for college student drinking. Journal of Consulting and Clinical Psychology 2015 Jun;83(3):455-463. [CrossRef]
  61. Bedendo A, Ferri CP, de Souza AAL, Andrade ALM, Noto AR. Pragmatic randomized controlled trial of a web-based intervention for alcohol use among Brazilian college students: Motivation as a moderating effect. Drug and Alcohol Dependence 2019 Jun;199:92-100. [CrossRef]
  62. Dempsey RC, McAlaney J, Bewick BM. A critical appraisal of the social norms approach as an interventional strategy for health-related behavior and attitude change. Front Psychol 2018 Nov 6;9:2180 [FREE Full text] [CrossRef] [Medline]
  63. Palfai TP, Winter M, Lu J, Rosenbloom D, Saitz R. Personalized Feedback as a Universal Prevention Approach for College Drinking: A Randomized Trial of an e-Mail Linked Universal Web-Based Alcohol Intervention. J Primary Prevent 2014 Jan 14;35(2):75-84. [CrossRef]
  64. Pasick RJ, D'onofrio CN, Otero-Sabogal R. Similarities and Differences Across Cultures: Questions to Inform a Third Generation for Health Promotion Research. Health Education Quarterly 2016 Aug 26;23(1_suppl):142-161. [CrossRef]
  65. Germanos R, Deacon R, Mooney-Somers J. The Social and Cultural Significance of Women's Sexual Identities Should Guide Health Promotion. LGBT Health 2015 Jun;2(2):162-168. [CrossRef]
  66. Mason TB, Lewis RJ. Reducing obesity among lesbian women: Recommendations for culturally tailored interventions. Psychology of Sexual Orientation and Gender Diversity 2014 Dec;1(4):361-376. [CrossRef]
  67. Huxley C, Clarke V, Halliwell E. Resisting and Conforming to the ‘Lesbian Look’: The Importance of Appearance Norms for Lesbian and Bisexual Women. J. Community Appl. Soc. Psychol 2013 Jul 23;24(3):205-219. [CrossRef]
  68. Fingerhut AW, Abdou CM. The Role of Healthcare Stereotype Threat and Social Identity Threat in LGB Health Disparities. Journal of Social Issues 2017 Sep 21;73(3):493-507. [CrossRef]
  69. Gordon LE. Bringing the U-Haul: Embracing and Resisting Sexual Stereotypes in a Lesbian Community. Sexualities 2016 Nov 07;9(2):171-192. [CrossRef]
  70. Fox J, Warber KM. Queer Identity Management and Political Self-Expression on Social Networking Sites: A Co-Cultural Approach to the Spiral of Silence. J Commun 2014 Dec 22;65(1):79-100. [CrossRef]
  71. Gabbay SG, Wahler JJ. Lesbian Aging. Journal of Gay & Lesbian Social Services 2002 Oct 29;14(3):1-21. [CrossRef]
  72. Herrera AP. Theorizing the lesbian hashtag: Identity, community, and the technological imperative to name the sexual self. Journal of Lesbian Studies 2017 Nov 27;22(3):313-328. [CrossRef]
  73. Davis-Delano LR. Sport as Context for the Development of Women’s Same-Sex Relationships. Journal of Sport and Social Issues 2014 Feb 04;38(3):263-285. [CrossRef]
  74. Dolance S. “A whole stadium full”: Lesbian community at women's national basketball association games. Journal of Sex Research 2005 Feb;42(1):74-83. [CrossRef]
  75. Mock SE, Misener K, Havitz ME. A League of Their Own? A Longitudinal Study of Ego Involvement and Participation Behaviors in LGBT-Focused Community Sport. Leisure Sciences 2019 Sep 17:1-18. [CrossRef]
  76. Resnicow K, Soler R, Braithwaite R, Ahluwalia J, Butler J. Cultural sensitivity in substance use prevention. J Community Psychol 2000 Apr 26;28(3):271-290 [FREE Full text] [CrossRef]
  77. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist 2000;55(1):68-78. [CrossRef]
  78. Deci EL, Ryan RM, editors. Handbook of Self-determination Research. Rochester: University Rochester Press; 2004.
  79. Kapp K. The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. New York, USA: John Wiley & Sons; 2012.
  80. King D, Greaves F, Exeter C, Darzi A. 'Gamification': influencing health behaviours with games. J R Soc Med 2013 Mar;106(3):76-78. [CrossRef] [Medline]
  81. Kawachi I. It’s All in the Game—The Uses of Gamification to Motivate Behavior Change. JAMA Intern Med 2017 Nov 01;177(11):1593. [CrossRef]
  82. Johnson D, Deterding S, Kuhn K, Staneva A, Stoyanov S, Hides L. Gamification for health and wellbeing: A systematic review of the literature. Internet Interventions 2016 Nov;6:89-106. [CrossRef]
  83. Boyle SC, Earle AM, McCabe N, LaBrie JW. Increasing Chance-Based Uncertainty Reduces Heavy Drinkers’ Cognitive Reactance to Web-Based Personalized Normative Feedback. J. Stud. Alcohol Drugs 2018 Jul;79(4):601-610. [CrossRef]
  84. Deci E, Ryan R. Self-determination. In: Weiner IB, Craighead WE, editors. The Corsini Encyclopedia of Psychology. Hoboken: John Wiley & Sons; 2010:1-2.
  85. Deci EL, Ryan RM. Handbook of self-determination research. Rochester: University Rochester Press; 2004.
  86. Mandryk RL, Inkpen KM, Calvert TW. Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour & Information Technology 2006 Mar;25(2):141-158. [CrossRef]
  87. Weibel D, Wissmath B, Habegger S, Steiner Y, Groner R. Playing online games against computer- vs. human-controlled opponents: Effects on presence, flow, and enjoyment. Computers in Human Behavior 2008 Sep;24(5):2274-2291. [CrossRef]
  88. Lim S, Reeves B. Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player. International Journal of Human-Computer Studies 2010 Jan;68(1-2):57-68. [CrossRef]
  89. Colwell J. Needs met through computer game play among adolescents. Personality and Individual Differences 2007 Dec;43(8):2072-2082. [CrossRef]
  90. Kern R. Imagining community: Visibility, bonding, and audiences. Sexualities 2014 May 13;17(4):434-450. [CrossRef]
  91. Boyle SC, Kettering V, Young SH, LaBrie JW. Alcoholism Treatment Quarterly 2020 Feb 11;38(4):415-429. [CrossRef]
  92. Uysal A, Yildirim I. Self-determination theory in digital games. In: Bostan B, editor. Gamer Psychology and Behavior. Switzerland: Springer International Publishing; 2016:123-135.
  93. Ryan RM, Rigby CS, Przybylski A. The Motivational Pull of Video Games: A Self-Determination Theory Approach. Motiv Emot 2006 Nov 29;30(4):344-360. [CrossRef]
  94. Przybylski AK, Rigby CS, Ryan RM. A motivational model of video game engagement. Review of General Psychology 2010;14(2):154-166. [CrossRef]
  95. Boyle E, Connolly TM, Hainey T. The role of psychology in understanding the impact of computer games. Entertainment Computing 2011 Jan;2(2):69-74. [CrossRef]
  96. Sheldon K, Filak V. Manipulating autonomy, competence, and relatedness support in a game-learning context: New evidence that all three needs matter. Br J Soc Psychol 2008 Jun 1;47(2):267-283. [CrossRef]
  97. Rigby S, Ryan R. The Player Experience of Need Satisfaction (PENS): An applied model and methodology for understanding key components of the player experience. 2007 Sep.   URL: [accessed 2020-12-20]
  98. Ozcelik E, Cagiltay NE, Ozcelik NS. The effect of uncertainty on learning in game-like environments. Computers & Education 2013 Sep;67:12-20. [CrossRef]
  99. Liang R. Designing for unexpected encounters with digital products: Case studies of serendipity as felt experience. Int Journal of Design Apr 1 2012;6(1):41-58. [CrossRef]
  100. Hou J. Uses and gratifications of social games: blending social networking and game play. First Monday 2011 Jul 2;16(7):e1. [CrossRef]
  101. van der Spek E. Towards designing for competence and engagement in serious games. In: Ma M, Oliveira MF, Hauge JB, Duin H, Thoben KD, editors. Serious Games Development and Applications. Berlin: Springer; 2012:98-109.
  102. Willson M, Leaver T. Social, casual and mobile games: The changing gaming landscape. New York: Bloomsbury Academic; 2017.
  103. Radtke T, Ostergaard M, Cooke R, Scholz U. Web-Based Alcohol Intervention: Study of Systematic Attrition of Heavy Drinkers. J Med Internet Res 2017 Jun 28;19(6):e217. [CrossRef]
  104. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012 Mar;50(3):217-226 [FREE Full text] [CrossRef] [Medline]
  105. Landes SJ, McBain SA, Curran GM. An introduction to effectiveness-implementation hybrid designs. Psychiatry Res 2019 Oct;280:112513. [CrossRef] [Medline]
  106. Boyle S.   URL:; [accessed 2020-12-20]
  107. Boyle S. LezParlay Competition Informational Landing Page.   URL: [accessed 2020-12-20]
  108. Baer JS. Etiology and secondary prevention of alcohol problems with young adults. In: Baer JS, Marlatt GA, McMahon RJ, editors. Addictive behaviors across the life span: Prevention, treatment, and policy issues. Thousand Oaks: Sage Publications; 1993:111-137.
  109. Earle AM, LaBrie JW, Boyle SC, Smith D. In pursuit of a self-sustaining college alcohol intervention: Deploying gamified PNF in the real world. Addict Behav 2018 May;80:71-81. [CrossRef] [Medline]
  110. LaBrie JW, de Rutte JL, Boyle SC, Tan CN, Earle AM. Leveraging copresence to increase the effectiveness of gamified personalized normative feedback. Addictive Behaviors 2019 Dec;99:106085. [CrossRef]
  111. Bhaskaran K, Smeeth L. What is the difference between missing completely at random and missing at random? Int J Epidemiol 2014 Aug;43(4):1336-1339 [FREE Full text] [CrossRef] [Medline]
  112. Little RJ, Rubin DB. Statistical Analysis With Missing Data. Hoboken: John Wiley & Sons; 2019.
  113. Burton P, Gurrin L, Sly P. Extending the simple linear regression model to account for correlated responses: An introduction to generalized estimating equations and multi-level mixed modelling. Statist. Med 1998 Dec 04;17(11):1261-1291. [CrossRef]
  114. Goldstein H. Multilevel Statistical Models: 4th Edition. Hoboken: John Wiley & Sons; 2011.
  115. Hayes AF. Introduction to Mediation, Moderation, and Conditional Process Analysis: a Regression-based Approach. 2nd Edition. New York: Guilford Publications; 2017.
  116. Hayes AF, Preacher KJ. Statistical mediation analysis with a multicategorical independent variable. Br J Math Stat Psychol 2014 Nov;67(3):451-470. [CrossRef] [Medline]
  117. Edwards JR, Lambert LS. Methods for integrating moderation and mediation: a general analytical framework using moderated path analysis. Psychol Methods 2007 Mar;12(1):1-22. [CrossRef] [Medline]
  118. Morgan-Lopez AA, Mackinnon DP. Demonstration and evaluation of a method for assessing mediated moderation. Behavior Research Methods 2006 Feb;38(1):77-87. [CrossRef]
  119. Walters ST, Neighbors C. Feedback interventions for college alcohol misuse: What, why and for whom? Addictive Behaviors 2005 Jul;30(6):1168-1182. [CrossRef]
  120. Thomas DR. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation 2006 Jun 01;27(2):237-246. [CrossRef]
  121. Hightow-Weidman LB, Muessig KE, Bauermeister JA, LeGrand S, Fiellin LE. The future of digital games for HIV prevention and care. Curr Opin HIV AIDS 2017 Sep;12(5):501-507. [CrossRef] [Medline]
  122. LeGrand S, Muessig KE, McNulty T, Soni K, Knudtson K, Lemann A, et al. Epic Allies: Development of a Gaming App to Improve Antiretroviral Therapy Adherence Among Young HIV-Positive Men Who Have Sex With Men. JMIR Serious Games 2016 May 13;4(1):e6. [CrossRef]
  123. Fiellin LE, Kyriakides TC, Hieftje KD, Pendergrass TM, Duncan LR, Dziura JD, et al. The design and implementation of a randomized controlled trial of a risk reduction and human immunodeficiency virus prevention videogame intervention in minority adolescents: PlayForward: Elm City Stories. Clin Trials 2016 Aug;13(4):400-408. [CrossRef] [Medline]
  124. Flores-Aranda J, Goyette M, Larose-Osterrath C. Online Intervention as Strategy to Reach Men Who Have Sex With Other Men and Who Use Substances in a Sexual Context. Development of the Project. Front Psychiatry 2019;10:183 [FREE Full text] [CrossRef] [Medline]
  125. Mejia CM, Acland D, Buzdugan R, Grimball R, Natoli L, McGrath MR, et al. An Intervention Using Gamification to Increase Human Immunodeficiency Virus and Sexually Transmitted Infection Screening Among Young Men Who Have Sex With Men in California: Rationale and Design of Stick To It. JMIR Res Protoc 2017 Jul 17;6(7):e140. [CrossRef]
  126. Coulter RW, Sang JM, Louth-Marquez W, Henderson ER, Espelage D, Hunter SC, et al. Pilot Testing the Feasibility of a Game Intervention Aimed at Improving Help Seeking and Coping Among Sexual and Gender Minority Youth: Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2019 Feb 15;8(2):e12164. [CrossRef]

LBQ: lesbian, bisexual, or queer
LGBT: lesbian, gay, bisexual, and transgender
MLM: multilevel model
PNF: personalized normative feedback
RCT: randomized controlled trial
SDT: self-determination theory
SMW: sexual minority women

Edited by F Drozd; This paper was peer reviewed by J Mooney-Somers, R Bush; externally peer reviewed by the National Institute on Alcohol Abuse and Alcoholism. See the Multimedia Appendices for the peer-review report; submitted 28.09.20; accepted 20.03.21; published 16.04.21


©Sarah C Boyle, Joseph W LaBrie. Originally published in JMIR Research Protocols (, 16.04.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.