SARA’s data collection module deals with procuring self-reported data. SARA currently supports 2 types of self-reported data collection. The first is provided by active tasks. Active tasks, which were first introduced in the Apple Research Kit [52], constitute intuitive user interactions that allow researchers to objectively measure reaction time, spatial memory, gait, problem-solving skills, etc. SARA’s first active task serves as a measure of spatial memory, a random sequence of 5 seashells lights up in a 2-dimensional grid of 9 seashells. Participants are then asked to repeat the sequence. The second active task is a tapping task in which participants tap 2 buttons alternately for 10 seconds. The number of completed taps gives a measure of reaction time. The reason for choosing the tapping and spatial tasks is that spatial memory and reaction time may vary based on substance use-related intoxication [53,54].

The second type of data collection is a daily survey in which participants report their daily feelings and activities (ie, stress level, mood, loneliness, hopefulness, amount of free time, and excitement) [55-58]. On Sundays, an extra set of questions assesses the past week’s frequency of substance use (alcohol, cannabis, and tobacco), motives and riskiness of use (for alcohol and marijuana), impulsivity, and intentions to avoid alcohol and marijuana use in the upcoming week [8,59-63]. Multimedia Appendix 1 contains the survey questions and the screenshots of the active tasks.

In the current version, participants self-report 1 survey and 2 active tasks (1 tapping task and 1 spatial task) every day between 6 pm and midnight. We selected this time window for the following reasons: (i) 6 pm to midnight provides a large enough time period to self-report, (ii) to capture a summary of most of the day using the survey in the evening, and (iii) to capture intoxication via active tasks since substances are typically used by youth in the evening [64].

The engagement module of SARA contains several engagement strategies to increase self-report completion. These engagement strategies can be grouped as follows: (1) postdata collection rewards, (2) predata collection incentives, and (3) reminders to self-report. In addition, SARA contains other enhancements to support engagement, for example, occasional human support by SMS text messages or phone calls when participants temporarily disengage; a small amount of money, and an overall coherent user experience so that the app is easy to use, and the engagement strategies appear to be part of one app. Since SARA focuses on reducing financial incentives, we took several measures so that the engagement strategies are indeed effective for the target population.

First, we grounded the strategies in behavioral theories on influencing the intended action. Second, we followed a user-centered design process, where we conducted 3 focus groups (N=21, mean age 19.9 years) and a pilot study (N=17, mean age 21.2 years); a majority of the participants in these studies (67% in focus groups and 100% in the pilot study) reported binge drinking (5+ drinks at 1 occasion) or marijuana use in the past 3 months. We describe below the engagement strategies that are supported by theories and the user-centered design process.

We used the cross-platform JavaScript framework Ionic to build the Android and iPhone versions of SARA. The aquarium part uses the Phaser 2D game library for the animations. All the self-reported data in SARA are encrypted to comply with the Health Insurance Portability and Accountability Act (HIPAA). All data are stored in Amazon S3, a HIPAA-compliant data storage service, and the communication between Amazon S3 and the mobile phone app is encrypted with RSA 2048 and AES-256 [80].

The 30-day MRT is conducted at the University of Michigan. Study participants are recruited from the University of Michigan Hospital Pediatric and Adult Emergency Departments. The study is approved by the Institutional Review Board of the University of Michigan Health System (HUM00121553) and is registered at ClinicalTrials.gov (NCT03255317). Patients are eligible for screening if they are aged between 14 and 24 years, understand English, are medically stable, are able to provide informed consent or assent (eg, not cognitively impaired or intoxicated), and are accompanied by a parent or guardian (for patients aged between 14 and 17 years). Individuals are eligible if they 1) have an Android or an iPhone mobile phone on which the app can be downloaded and 2) screen positive for past-month binge drinking (≥4 drinks for females and ≥5 drinks for males, on at least 1 occasion) or past-month cannabis use without a medical marijuana card.

At the University of Michigan Emergency Department, recruiters monitor incoming admissions and identify patients in the target age range who do not meet the screening exclusion criteria (eg, presentation for sexual assault, droplet precautions, active vomiting, roomed in critical care). After completing an online consent or assent, participants self-administer a screening survey on a tablet, which contains questions regarding their demographics [84,85], health behaviors such as alcohol and marijuana use [8] and sleep habits [86], cell phone capabilities, and social media use. The screening survey takes approximately 8 min to complete, and participants receive a small gift valued at US $1.00 (eg, headphones, water bottle, mobile phone armband) for completing it. Those who report past-month binge drinking or marijuana use and have access to their mobile phone while in the Emergency Department provide written consent or assent and self-administer an approximately 20-min baseline survey with items about (1) substance use frequency, consequences, overdose, and driving under the influence [86-88]; (2) violence involvement, injury, and risky sex behaviors [6,89-92]; (3) coping and mindfulness [93,94]; (4) social influences [95-97]; and (5) motivation or self-efficacy to reduce their alcohol and marijuana use [98,99].

Then, the recruiters collect contact information from the participants. Recruiters ensure that participants install SARA on their phone during the intake since in the pilot study prior to this trial, we have found that many potential participants did not install the app after they left the intake session. Basic instructions on how to use the app are provided during intake, followed by motivational statements regarding participant’s stated barriers to using SARA regularly. Participants receive US $20 cash for completing their intake visit.

Following the intake session, participants start the 30-day MRT with SARA. Each day, participants are requested to complete 1 survey and 2 active tasks between 6 pm and midnight. Participants are randomized at the following 4 decision points each day:

Note that not all the engagement strategies in SARA are randomized. The randomizations in 1-4 mentioned above are motivated by scientific questions concerning the abovementioned engagement strategies in SARA. Specifically, we randomize the push notifications because they can interrupt the participant and are intrusive, thus potentially reducing engagement. The other engagement strategies are not push notifications, so there is low risk of interruption; for example, the participant decides whether to open the app, complete self-reports, and receive the rewards. We randomize the memes and life insights because of the scientific question of how funny and informative contents respectively can influence self-report completion over time [44].

The proximal outcomes of the randomizations are whether participants completed the survey and active tasks. For the inspirational message notifications at 4 pm and the reminder notifications at 6 pm, the proximal outcome is whether the survey or the active tasks are completed in the evening on the same day. For the 2 reinforcement interventions, the proximal outcome is whether the survey or the active tasks, respectively, are completed on the following day. Note that our engagement strategies are primarily designed to encourage self-reporting on the same or next day. Although the strategies may have longer term effects as well [81,82], we are interested in their effects on the same day or the next day; if these effects are consistently higher, then longer term adherence will be higher too. In addition, poststudy open-ended feedback from the participants will be analyzed to refine future versions of SARA.

Our primary analyses will concern the following 2 hypotheses:

We selected these hypotheses as primary since our team found these hypotheses to be the most interesting scientifically. The 4 pm randomization is designed to address the question of whether a notification intended to facilitate reciprocation (by providing an inspirational message before self-report time) is useful, and the randomization of participants upon survey completion is designed to investigate whether providing a postdata collection reward increases data collection. Furthermore, we conjecture that these hypotheses have the greatest potential to be supported. In particular, as the 4 pm notification is proximal in time to the data collection (2 hours prior), there are fewer extraneous distracting circumstances that can occur during this short time window that would reduce the intervention effect. Both hypotheses also consider a contrast between an active agent (ie, the 4 pm notification or the meme) versus nothing. Since these 2 hypotheses are primary, we divide the standard P value of .05 by 2.

Our secondary analyses will concern the following 2 hypotheses:

We consider the first hypothesis as secondary because the randomization for the 6 pm reminder is between 2 active agents (reminder with vs without a persuasive message). This implies that the additional effect of the persuasive messages may be small; thus, this hypothesis has a more exploratory nature. We consider the second hypothesis as secondary because data must accumulate before the visualizations in life insights become interesting. In addition, this hypothesis has a more exploratory nature because our life insights are only visualizations of past data; in the future, we aim for potentially more potent life insights using prediction tools [103,104].

For both primary and secondary analyses, we will control for the following 3 covariates to reduce variance in the outcome of self-input completion: (1) whether the survey or the active tasks were fully completed on the previous day, (2) whether SMS text messages or phone calls were made in the last 24 hours, and (3) whether the app was opened in the prior 72 hours outside when a survey or active task was completed. We will not include other baseline variables such as gender and age as covariates in the primary analyses because inclusion of a covariate uses up degrees of freedom. We also anticipate that the within-person covariate “whether the survey or active tasks were fully completed on the previous day” will capture some of the variance due to baseline gender or age. We will also not include time as a covariate in the primary analyses for the abovementioned reasons as well. Note that the statistical methods [83] will adjust the standard errors of the estimated effects to account for within-person correlation across time in the outcome. For more details about our primary analysis plan, please refer to the Center for Open Science document [102].

We plan to run exploratory analyses to examine how the effectiveness of engagement strategies changes over time (we conjecture that the effectiveness will decrease). We will also run additional exploratory analyses to assess effect moderation. We will examine how the effect of engagement strategies is moderated by gender, weekdays versus weekends, and whether the day is Sunday versus other days of the week (we expect that the completion rate for the longer Sunday’s surveys may be lower than that for other days). We initially planned to assess age and phone type (Android or iPhone) as moderators; however, we did not include these variables because recruitment thus far indicates that very few participants in the study have Android phones or are below the age of 18 years.

We started recruiting for the study on August 21, 2017. Recruitment concluded on February 28, 2018. We recruited 73 participants for the study.

In mHealth, it is common to collect time-varying measures of the participants’ context (such as stress, mood, and loneliness from the self-report assessments). The provision of engagement strategies is time-varying as well; that is, at each decision point, participants can receive different options of the engagement strategy. A key statistical issue is that covariates (measures of the participant’s context) at a time point can be affected by past engagement strategies. For such a setting, Boruvka et al [83] proposed a method to estimate the causal effects of interventions on continuous outcomes. However, in SARA, we are dealing with a binary outcome (whether participants self-reported or not). Multimedia Appendix 8 contains details of a method that we have developed, which extends the work of Boruvka et al to binary outcomes. We will use RStudio 1.1.453 to run the statistical analysis.

The open-ended qualitative data from exit interviews will be coded using thematic analysis [105]. The qualitative analysis will be performed using NVivo 11.

Since our outcome is adherence to self-report completion, not completing self-reports is not missing data for our study. However, missingness can happen in the study if participants uninstall the app in the middle of the study. To account for such missingness, we will conduct the following 3 versions of the primary analysis: (1) only include participants who had the app installed for 30 days, (2) include all participants and include only days when the app was installed, and (3) include all participants and data for both installed and uninstalled days; for the uninstalled days, the 4 pm and 6 pm notifications will be imputed, and the outcome will be considered as “self-report noncompletion.” More details on how missingness will be accommodated can be found in the Center of Open Science document [102].