This study uses an iterative series of MRTs to refine and test engagement strategies embedded within the MHA Online Screening Program. This approach is consistent with optimization frameworks that facilitate continual improvement of intervention components through successive trials [36]. In this study, eligible MHA visitors (henceforth, participants) may be randomized at 2 prespecified decision points embedded within the usual visitor flow of the MHA Online Screening Program, which could include progression through the PHQ-9 depression screening test, an optional postscreening survey that collects participant information (eg, demographics), screening results, and targeted Next Steps Content Pages. The first decision point will be the Screening Results Page, where MHA visitors are typically shown information about their PHQ-9 score and provided with a set of recommended resources as “Next Steps”. The second decision point will occur when participants navigate to a targeted Next Steps Content Page from the Screening Results Page. These 2 decision points represent critical moments where MHA visitors are at risk of disengagement.

Distinct engagement strategies were designed for placement at the 2 decision points on the MHA website, including the Screening Results Page and the targeted Next Steps Content Pages. At each decision point, participants may be randomized to receive no engagement strategy (Z_t) or an engagement strategy targeting 1 of 4 HAPA behavioral determinants, including outcome expectancy (A_t), risk perception (B_t), self-efficacy (C_t), or SGM-specific barriers to engagement (D_t) (Figure 1). Primarily, this series of MRTs will test the effect of delivering HAPA-based engagement strategies on proximal engagement compared to standard website content (ie, MHA pages without embedded engagement strategies).

Initial comparisons will be conducted across all available strategies. Over the study period, the set of strategies included in randomization may be refined based on accumulating evidence related to the performance of strategy content, delivery format, or timing. This iterative approach has previously been used to optimize engagement with DMH interventions, including on MHA [37], where strategies are refined and reevaluated in subsequent MRTs [40]. In this study, any engagement strategy refinements will be determined between MRT iterations and informed by findings from preceding MRTs. Refinements between MRT iterations will follow prespecified decision rules developed in consultation with MHA that balance optimization opportunities (eg, removing underperforming strategy variations within HAPA behavioral determinants or restricting to the highest-performing strategy variation within HAPA behavioral determinants) with feasibility and maintenance of a consistent MRT structure, including eligibility criteria, decision points, randomization to receive no engagement strategy (comparator) or an engagement strategy targeting HAPA behavioral determinants (intervention), and outcome measures. These refinements will be documented using the Framework for Reporting Adaptations and Modifications-Enhanced, which logs when the modification was made, what was modified, and the rationale for the modification, among other factors, to ensure transparency and analytic consistency [41]. No modifications to intervention content or randomization procedures are anticipated within an ongoing MRT unless they are necessary to address implementation errors identified through quality assurance (QA) monitoring. This approach supports optimization through efficient, progressive refinement of displayed strategies over the study period and may enable more direct comparisons among top-performing strategies while maintaining a consistent experimental framework across the series of MRTs. In alignment with similar studies, 3 to 4 MRTs are anticipated within the series, with the exact number of iterations depending on opportunities for continued strategy refinement and study time constraints [40]. Identifying the most effective strategies and delivery points will ultimately facilitate scalable implementation within the broader MHA website.

MHA visitors will be eligible for the trial if they complete the PHQ-9 depression screener in English and indicate on the optional postscreening survey that they (1) live in the United States, (2) are aged ≥14 years, and (3) identify as LGBTQ+ (lesbian, gay, bisexual, transgender, queer, and others).

Prior findings indicated that HAPA’s immediate behavioral determinants were associated with engagement behaviors (eg, time on resources, number of resources visited, and types of resources visited) among MHA’s SGM visitors. In this trial, engagement strategies targeting these behavioral determinants, delivered at key decision points, are hypothesized to increase proximal engagement by addressing participants’ outcome expectancies, perceived risks, self-efficacy, and SGM-specific barriers to engaging with DMH resources. Engagement strategies are designed to enhance participants’ beliefs that the Next Steps resources are relevant, helpful, and worth trying (target: outcome expectancy); strengthen feelings of safety and trust (target: perceived risk); increase confidence in their ability to use the resources (target: self-efficacy); and reinforce feelings of pride or support (target: SGM-specific barriers to engagement), thereby increasing proximal engagement relative to standard website content. To activate these mechanisms, engagement strategies incorporate evidence-based behavioral design elements, including authority cues, social proof, normative feedback, goal-gradient effects, and visually salient interface features (eg, highlighted calls to action) [42-44].

On the Screening Results Page, strategies tested in the initial MRT will be delivered as inline engagement messages designed to reinforce an immediate behavioral determinant to nudge participants to select their Next Step after viewing their PHQ-9 screening results. On the Next Steps Content Pages, strategies tested in the initial MRT will be delivered as engagement cards embedded within the existing page layout. Based on the preliminary work, an initial bank of HAPA-informed engagement strategy variations was developed. The bank includes 12 engagement message variations and 8 engagement card variations. Each variation was designed to target 1 of 4 HAPA behavioral determinants. During the intervention design process, HAPA behavioral determinants were operationalized based on how each manifested for SGM visitors, specifically (eg, risk perception: safety and trust concerns and SGM-specific barriers to engagement: stigma). For each determinant, 3 message variations were developed for the first decision point, and 2 card variations were developed for the second decision point. All variations within each determinant differ in wording but are unified by their focus on influencing the same behavioral determinant while maintaining consistent visual placement and interface design.

On the Screening Results Page, participants will be randomized to see a brief inline engagement message targeting 1 of the 4 HAPA behavioral determinants or MHA’s standard Screening Results Page. At this initial decision point, engagement messages are designed to quickly reinforce engagement behaviors while minimizing cognitive burden at a high-friction transition point where most visitors disengage immediately after viewing their screening results. HAPA-based engagement messages that will be delivered at the first decision point in the initial MRT are displayed in Figure 2.

Once a participant initiates engagement by selecting one of the targeted Next Steps Content Pages, they may be randomized to see an engagement card embedded directly within the content page layout that targets 1 of the 4 HAPA behavioral determinants or MHA’s standard page design. These cards function as small user interface components that visually stand out but do not interrupt navigation. Cards are designed to provide a more visible but nondisruptive nudge to reinforce continued engagement. Each card will be embedded within the website interface and can be read without redirecting participants or requiring them to leave the page. HAPA-based engagement cards that will be delivered at the second decision point in the initial MRT are displayed in Figure 3.

MRTs are often more statistically efficient than traditional randomized trial designs because they leverage both between-subject and within-subject contrasts of proximal outcomes. However, the efficiency gains of an MRT depend on the number of decision points contributed per participant. The current MRTs will include 2 decision points within a single visitor session on the MHA website. This design provides fewer within-person contrasts than high-frequency MRTs (eg, daily notification studies), as each participant can contribute at most 2 randomizations across the session. Consequently, statistical power in this study is primarily driven by the between-subject sample size rather than the repeated within-person contrasts. This structure reflects constraints of the intervention context. The current trial is a web-based, naturalistic engagement study that embeds randomizations within the existing ecosystem of a real-world DMH platform, where pull intervention delivery must align with the typical website navigation flow rather than repeated daily push notifications.

The minimum sample size was calculated for MRTs with binary proximal outcomes [36,45-47]. Calculations assumed 80% power, a 2-sided α of .05, and a treatment randomization probability of 0.80 for the primary pooled analysis (ie, probability of being assigned any HAPA-based engagement strategy vs control at a given decision point). An expected availability of 0.70 was used in the calculation. This reflects the expected availability averaged over time [46], including full availability at decision point 1 (as all eligible participants view the Screening Results Page) and a conservatively lower availability at decision point 2 (as preliminary analyses indicated that under standard conditions, most MHA visitors bounce after receiving their screening results, with fewer navigating to content pages). Participants who do not navigate to a Next Steps Content Page are not available for randomization at decision point 2. Under the assumption of a constant success probability under the null of 0.30 (a conservative estimate given that 63% of visitors leave the MHA website after viewing screening results) and a proximal treatment effect of 1.20 on the relative risk scale, an estimated 2300 participants will be required to detect a marginal causal excursion effect with 80% power. These assumptions reflect engagement patterns observed in prior analyses of MHA’s site use and account for the time-bounded, single-session nature of the intervention. Given the volume of visitors to MHA’s Online Screening Program, it is possible that the final sample size will exceed this minimum requirement. However, final enrollment numbers will ultimately reflect the naturalistic rate at which eligible visitors access MHA’s Online Screening Program throughout the duration of the study. Importantly, the study uses an iterative series of MRTs in which successive MRTs are conducted and engagement strategies are refined between iterations. Each MRT iteration requires approximately 2300 participants to detect an effect within the primary pooled analysis. To power secondary analyses comparing specific HAPA behavioral determinants against control or specific message or card against control, each MRT iteration will need a larger sample. These analyses will be interpreted as exploratory.

Given the real-world setting of this study and focus on naturalistic engagement as the primary outcome, no active recruitment or retention procedures will be implemented.

Analyses will estimate the marginal causal excursion effects of delivering engagement strategies on proximal engagement with MHA content. Data will be analyzed using the weighted centered least squares estimator, which provides consistent estimates of causal excursion effects when including time-varying factors that may be influenced by prior intervention exposure [36,45,48]. Analyses will examine whether, on average, across decision points, engagement strategies (A_t-D_t) increase proximal engagement with targeted MHA content relative to the control (Z_t). A log-linear model will be used, as the primary proximal outcome is binary (engaged vs not engaged with targeted content). For the primary analysis, all HAPA-based engagement strategies (A_t-D_t) will be pooled so that the estimated marginal causal excursion effect represents the average effect of delivering any engagement strategy compared with the control (Z_t). Effects will be expressed as the log of a risk ratio (ie, the probability of engagement when an engagement strategy is delivered relative to the probability of engagement when the control is delivered at a decision point). To examine whether intervention effects differ across the 2 decision points, models will include an interaction between intervention assignment and a decision point indicator. Baseline characteristics (eg, sexual orientation, gender identity, age, baseline PHQ-9 score, and intentions for using MHA) and time-varying contextual factors (eg, number and type of pages visited prior to a decision point) will also be examined as potential moderators of proximal intervention effects by including interaction terms in the log-linear model. Models will control for race or ethnicity and income.

Secondary analyses will examine whether engagement strategies targeting distinct HAPA behavioral determinants differ in their relative effectiveness at increasing proximal engagement with targeted MHA content. Using the same weighted centered least squares estimation approach, models will compare the proximal effects of strategies targeting outcome expectancy (A_t), risk perception (B_t), self-efficacy (C_t), and SGM-specific barriers to engagement (D_t). These analyses will estimate marginal causal excursion effects for each strategy type. Additional analyses will examine whether variations of engagement strategies within the bank that target the same HAPA behavioral determinant differentially impact proximal engagement. These exploratory comparisons will be conducted by extending the primary regression models to include indicators for specific engagement strategy variations and estimating their relative causal excursion effects on proximal engagement outcomes. Exploratory analyses will also examine additional engagement outcomes (as described in the Measures section), including time spent on pages and the number of additional pages visited following exposure to intervention content.

Proximal outcomes at both decision points are defined using website event data and are expected to be largely complete. Missing proximal outcomes may occur due to both technical factors (eg, logging errors) and behavioral processes (eg, participants who disengage before a decision point are not available for randomization). Indicators required for eligibility assessment are expected to be complete, but some moderators and covariates may be missing as postscreening survey items are optional.

45,48

The primary analysis will use all available decision points with observed proximal outcomes (ie, complete-case analysis at each decision point), an approach consistent with recommendations for addressing missing data in MRTs [49]. If missing data are ≤10%, results will be interpreted with minimal concern for bias. If missingness exceeds this threshold, patterns of missing data will be summarized at each decision point and compared across experimental conditions and observed participant characteristics. Exploratory analyses will examine whether intervention effects differ depending on patterns of missingness.

This trial protocol was reviewed by the single institutional review board (sIRB) at the University of Washington and approved on December 3, 2025 (STUDY00017726). The QT-Digital Mental Health Engagement study is considered a multisite clinical trial, as MHA is involved in the following research activities: (1) performing research procedures; (2) administering study interventions; and (3) obtaining, using, or analyzing identifiable data. The external Data Safety and Monitoring Board was provided with protocol documents prior to the initiation of data collection and did not identify any foreseeable risks. Any major protocol modifications will be reviewed by the sIRB and the Data Safety and Monitoring Board and updated on the ClinicalTrials.gov registration.

A waiver of consent was approved by the sIRB to facilitate the examination of naturalistic engagement. This waiver was supported by (1) MHA’s privacy policy, which informs visitors who use the site’s screening tools that deidentified data may be shared with universities for research and quality improvement purposes (Multimedia Appendix 1); and (2) the study’s minimal risk classification, as participants can continue to access MHA’s content across all conditions. In this study, formal consent procedures may also prevent naturalistic engagement. Prior research has found that participants recruited explicitly for DMH research engage differently than users who access DMH and are not identified as research participants [50]. Although participants will be randomized to different engagement strategies, these strategies introduce very minimal risk and are intended to optimize engagement with MHA’s available DMH resources.

Along with the overall waiver of consent, a waiver of parental consent was approved for participants aged 14 to 17 years, as the protocol was designed for a subject population (here, SGM adolescents seeking readily available DMH resources via MHA’s platform) for which parental permission is not a reasonable requirement to protect this study subsample (45 CFR 46.408(c)). Generally, adolescents represent approximately 59% of MHA’s SGM visitors. In alignment, as of April 13, 2026, 52% (2004/3878) of study participants were SGM visitors aged 14 to 17 years. Including this user population is critical to understanding and optimizing the engagement behaviors of MHA’s SGM visitors, generally, and for this priority subpopulation. Adolescent-parental relational and communication issues are often cited as the main barriers to mental health care for SGM adolescents [51]. Besides disrupting natural engagement, requiring parental consent may cause safety concerns for those not ready to disclose their sexual or gender identity [52-54] or for those seeking DMH services due to relational concerns with their parents. The lower cutoff age of 14 years was chosen, as these adolescents represent high school–aged youth whom past research shows have similar health literacy as adults [55] and sufficient maturity and rational decision-making to self-consent to health and mental health care [56].

MRT participants will be subject to minimal levels of risk, no greater than if they were to use the MHA platform without enrollment in this study. Participants reached and will engage with the MHA platform and Online Screening Program of their own volition. Although the study outcome is engagement, it is expected that most participants will have mental health symptoms or diagnoses. All MHA visitors have access to contact information for “warmlines” for each of the 50 states and phone and text crisis lines (eg, suicide and interpersonal violence), including those tailored to SGM populations. MHA visitors displaying any heightened level of risk for suicide (eg, anyone who responds “several days,” “more than half the days,” or “nearly every day” on the PHQ-9) are immediately prompted with the message “If you need immediate help, you can reach the Suicide & Crisis Lifeline by calling or texting 988 or using the chat box at 988lifeline.org. You can also text “MHA to 741-741 to reach the CrisisText Line. Warmlines (embedded link to warmline resources) are an excellent place for non-crisis support.” These mechanisms remain in place for study participants.

Along with the overall waiver of consent, a waiver of parental consent was approved for participants aged 14 to 17 years, as the protocol was designed for a subject population (here, SGM adolescents seeking readily available DMH resources via MHA’s platform) for which parental permission is not a reasonable requirement to protect this study subsample (45 CFR 46.408(c)). Generally, adolescents represent approximately 59% of MHA’s SGM visitors. In alignment, as of April 13, 2026, 52% (2004/3878) of study participants were SGM visitors aged 14 to 17 years. Including this user population is critical to understanding and optimizing the engagement behaviors of MHA’s SGM visitors, generally, and for this priority subpopulation. Adolescent-parental relational and communication issues are often cited as the main barriers to mental health care for SGM adolescents [51]. Besides disrupting natural engagement, requiring parental consent may cause safety concerns for those not ready to disclose their sexual or gender identity [52-54] or for those seeking DMH services due to relational concerns with their parents. The lower cutoff age of 14 years was chosen, as these adolescents represent high school–aged youth whom past research shows have similar health literacy as adults [55] and sufficient maturity and rational decision-making to self-consent to health and mental health care [56].

MHA collects engagement data and optional demographic survey responses as part of standard operations. MHA’s privacy policy provides visitors with information about how data are collected, stored, and shared (Multimedia Appendix 1). IP addresses are recorded and stored indefinitely in MHA’s server access logs but are not processed to identify visitors. MHA will have access to participant identifiers, including IP addresses and ZIP codes or email addresses, if provided by visitors as part of the optional demographic survey. UW researchers will not have access to any of these direct identifiers as they are removed from datasets before transmission from MHA to UW via a secure Box account. IP addresses, specifically, are hashed using an MD5 algorithm. This process creates an indirect identifier that cannot be reversed to recover the original IP address. MHA’s procedures for data cleaning, including deidentification procedures, have been published elsewhere [37]. The deidentified data will be stored on UW servers in accordance with the University’s data security requirements.