We recruited adolescents from social media advertisements, including paid advertisements on Instagram (Meta Platforms, Inc) and Facebook (Meta Platforms, Inc) and posts by research team members on TikTok (ByteDance). Advertisements displayed different combinations of study aspects, including the study name (TEAM300) and logo; engaging phrases (eg, Join the squad!); language about compensation and online format; and relevant images of teenagers, alcohol content in media, and general media use (Figure 4 depicts illustrative advertisements). Advertisements promoted diversity and inclusion by featuring young people from a variety of racial and ethnic backgrounds, with specific visuals and text refined based on feedback from our YAB (refer to the subsequent sections). Study staff ultimately developed advertisements as both still and animated graphics to fit social media feed and story layouts.

Clicking a digital study advertisement directed the individual to a study landing page in Qualtrics that provided basic information about the study, both in brief written and video format, as well as a link to our study website. Those interested in participating were directed to a 14-item anonymous screening survey that assessed basic eligibility criteria. We enabled various security features on Qualtrics (eg, ReCaptcha 2/3 and bot detection) to reduce instances of fraud.

After completing the screening survey, eligible adolescents who were aged 18 years continued directly to the online consent form. Those who consented then provided contact information. Eligible adolescents younger than 18 years provided contact information and subsequently received an email containing study information to be forwarded via email to their parent or legal guardian (hereafter referred to as parents) so that they could provide parental permission. The email contained a link to a Qualtrics landing page for parents, which provided a study overview, an informational video, links to the study website, and a link to the parental consent form. The project website contained a brief description of study procedures and compensation as well as 2 informational videos, 1 geared toward the participants and 1 geared toward parents. The research team, including our staff and YAB members from groups considered historically underrepresented, was featured on our study website. Two frequently asked questions sections were also provided on the website, 1 for adolescents and 1 for parents. The frequently asked questions responded to questions such as “Are there any risks to participating?” “Are you asking me to share anything on my phone or in my social media accounts?” and “What are you asking my child about?” All website materials were written at an 8th-grade reading level or lower. Parents were invited to contact research staff to answer any questions using the study contact information provided, or they could provide online consent for youth participation without additional information needed from project staff. There were no instances of parents reaching out to the team for additional study information before providing parental consent. Once parental permission was obtained within Qualtrics, the participant then received an email with a link to complete their own digital assent form.

Eligible participants who provided consent or assent (with documented parental consent) were directed to an online scheduling platform to select a 45-minute group orientation session. Once scheduled, an RA confirmed the orientation session appointment over the phone. During these calls, staff also confirmed the identities of the participants by asking the participants to confirm their contact information and state the name of their high school, verifying the high school location against the zip code previously given by the participant. We encountered approximately 40 nonlegitimate cases, including issues of contact information not matching previously submitted details, inaccurate high school names or locations, or inability to provide requested information.

Orientation sessions were conducted via Zoom in groups ranging from 2 to 10 (median 4) participants. Orientations, which were delivered by RAs, incorporated a slide deck as a visual aid to reinforce study information. Our RAs were diverse across both gender and racial and ethnic groups. The slide layouts, content, and flow of topics were adjusted and refined across time via staff, peers, and YAB feedback, ultimately resulting in standardized orientation session content across cohorts. The presentation included general study elements such as study staff names and position titles, overall study aims, participation timelines, and compensation structure, as well as specific procedural elements such as screenshots of the daily phone app reports and weekly surveys that the participants would complete. Figure 5 shows sample orientation content presented using Canva slides. The goals of the orientation sessions were to promote participant study engagement and teach the participants (1) examples of references to alcohol that might be encountered in day-to-day use of media, (2) the different types of reports, (3) how to submit a self-initiated report about the different types of media content they noticed, (4) how to download and navigate the study app, and (5) how to identify each type of survey and corresponding submission details. The orientation sessions included built-in knowledge checks throughout to ensure that the participants fully comprehended the study protocol.

At the end of orientation, the RA assisted the participants in real time in downloading and logging into the EMA app, MetricWire (MetricWire, Inc), with their study-assigned (nonpersonal) emails and assisted in adjusting phone settings and permissions. Youth were instructed to keep their device with them at all times, except while at school and sleeping, and to ensure they were receiving notifications. They were also provided instructions to adjust notifications in MetricWire (eg, turn off or mute or silence notifications) if desired, such as while sleeping or during sports practice.

We asked the participants to keep their phone on and charged and not to change phone settings for the duration of the study. The participants were explicitly told that researchers were not taking any information from their phone or device, putting anything on their phone or device, or trying to change their behavior. We emphasized that the participants should not deviate from their normal activities to find alcohol content (Figure 6).

Smartphone use is highly prevalent in this age group and more evenly distributed across race, ethnicity, and household income levels than PCs (resulting in limited bias associated with access) [45]. Thus, we did not anticipate that participants would have any difficulty accessing the survey software. In focus groups conducted before this study [46,47] and in our YAB, we probed device ownership, text plans, and access to Wi-Fi as barriers to participation, and none of the participants indicated this as a concern. It is possible that potentially interested participants were dissuaded from enrolling upon learning that the study procedures involved frequent surveys administered on a smartphone; this might produce a small bias in our sample. We will assess this potential bias using a baseline survey item: “Do you have a data plan for your smartphone?” (response options: yes, unlimited data; yes, but limited data; and no, no data plan), along with indicators of disadvantage such as socioeconomic status, which is measured by free or reduced-cost lunch eligibility and parental education level. It is also worth noting that the digital divide may be less apparent in our study because surveys only require a few minutes of data per day. Following attendance of the orientation session, the participants were emailed a link to the Qualtrics baseline survey. This assessment was requested to be completed before the first day of EMA data collection and included a section where the participants were asked to rate their reasons for joining our study to use the participant’s rationale to personalize our reminder and outreach messages. (For any cohort that did not receive this section in the baseline survey, it was administered instead at their follow-up survey.)

During orientation, the participants were provided with detailed and illustrated guidance on how to use the study app to report alcohol content that they observed. They were instructed to report each instance of alcohol content in media by initiating a survey continuously available on the study app’s dashboard.

To facilitate coding of alcohol content within the context of a movie, show, video, or song, we instructed the participants to enter a time stamp indicating exactly when the reference to alcohol occurred. When movies, shows, or videos had multiple references to alcohol, the participants were instructed to report on the most noticeable or important scene or example using a free-text entry item that accompanied the event-contingent report. For YouTube, we provided a field for the participants to paste a hyperlink, which could be used to obtain a static image. Finally, the participants were encouraged to submit a comment at the end of the report to describe additional scenes or any other notable information.

For participant reports of alcohol references in images, we similarly asked them to submit these images in event-contingent reports. We taught the participants how to deidentify an image by marking over or cropping out usernames or faces before submitting the image. We recommended that they delete any screenshots or photos with identifying information from their phone or gallery after submitting the report.

Before the start of EMA data collection, staff emailed the participants a participant instruction guide consisting of key reminders about participation rules and expectations (Figure 7). The participants were told that they could refer to this condensed guide at any point for clarification about study terms or survey details. Shortly before each subsequent data burst, staff again emailed the participants the link to the guide, encouraging them to review the information as a refresher of study protocols before starting the EMA portion again.

The first 3-week EMA phase began no more than 12 days after the orientation session. During this monitoring period, the participants completed 2 types of EMA reports on their own personal smartphone devices: scheduled reports (scheduled media reports) and self-initiated reports (self-media reports; Figure 8). The participants were prompted to complete scheduled reports during 4 intervals each day: 5 AM to 2 PM (morning), 2 PM to 5 PM (afternoon), 5 PM to 8 PM (evening), and 8 PM to 11 PM (night). These intervals were chosen to best align with adolescents’ schedules, with the selection of a very early morning survey prompt informed by discussion with members of the YAB. The prompt for the morning report was delivered at 5 AM, and the survey stayed available on the app dashboard until 2 PM each day, with reminders issued at 8 AM and 12 PM if necessary. The participants were prompted to complete afternoon, evening, and night reports at random times within their respective time blocks. Access to each of those reports expired 1 hour after the initial notification.

The participants could also complete a bedtime report, available from 8 PM to midnight, if they needed to go to sleep before receiving their prompt for the night report. These scheduled prompts assessed whether the participants had come across alcohol content in the media since their last submitted report. If the participants endorsed having any unreported alcohol content, they were asked whether it was one or multiple instances; for multiple exposures, they were instructed to think back on only the most recent instance of alcohol content when completing the subsequent set of questions. The participants who did not report encountering alcohol content were administered a set of general media questions that included self-reported media platform use and digital interactions unrelated to alcohol. This ensured that all the participants had the same opportunities to earn compensation based on survey compliance regardless of their daily alcohol content exposure.

Self-initiated media reports allowed youth to record exposure to media alcohol content in vivo. The participants initiated an assessment whenever they encountered an alcohol reference in the media. If the participants encountered >1 exposure in a time block, their first self-initiated media report fully assessed all alcohol media exposure content, including attitudes, cognitions, and context. Subsequent self-initiated media reports were abbreviated to reduce participant burden when submitting multiple exposures.

In all surveys, the participants were asked to share a screenshot, take a photo, type a text description, or include a link to a video. Images of alcohol content could be uploaded when submitting reports or taken within the app itself (videos uploads were also allowed). Youth were asked to describe the nature of the exposure, including the format or medium and source. They also indicated whether they reacted to the image (eg, like and comment). They also completed survey items assessing attitudes, cognitions (identification, liking, expectancies, and prototypes), and context related to the content. Abbreviated self-initiated media reports assessed only format or medium, source, and reaction. Measures and items are detailed in Table 1.

At the end of each week during the EMA protocol, the participants received a link to complete an online survey (on days 8, 15, and 22 of each burst). Items assessed past-week exposure to alcohol content, perceived peer and friend norms about alcohol use, and affect. This was followed by a 7-day TLFB assessment, which captured daily alcohol and other substance use over the past week.

Follow-up surveys were administered 1 week before each burst. These surveys were identical to the baseline, except they excluded stable sociodemographic information (eg, sex assigned at birth and race or ethnicity).

Study personnel contacted the participants daily via text or email and as needed to check progress, encourage compliance, and address technical issues. In addition, we created study social media accounts and encouraged the participants to follow them as a means to promote engagement and a sense of study community. Posts included the results of weekly polls as well as other fun content. We also communicated with the participants in between data collection bursts, including text check-ins, emailing certificates of burst completion, and mailing study swag, and we implemented a weekly survey raffle to encourage compliance with the weekly survey. A companion paper describing techniques for and data on retention and compliance is currently being prepared for submission.

All procedures were approved by the Brown University institutional review board (protocol number 2009002813). Informed consent was obtained from the participants who were aged 18 years, and both parental consent and verbal assent was obtained for the participants who were younger than 18 years. Consenting materials were written at an 8th-grade reading level.

We informed the participants during the orientation session that to protect data and prevent unauthorized access, study data were deidentified and stored in password-protected internal servers. We also encouraged the participants to ensure they enabled passwords to protect their phones and to exit out of the app and survey when complete. To protect against the risk that concerned parents might seek information from the study about their child, the consent and assent forms explained that we could not provide the parent with information that their child had given us.

Given that the participants could submit images that contained identifying information (eg, social media posts of friends), the participants were instructed to crop out usernames or faces and to cover or mark over faces and personal information. They were also told to delete screenshots or photos with identifying information from their phone and gallery after submitting the image.

One of the keys to achieving high EMA response rates is designing compensation schedules that are linked to target response rates. The participants were coached to achieve a response rate of at least 3 reports (either prompted or self-initiated) per day. A complete day was defined as at least 1 report in 3 out of 4 time blocks (morning, afternoon, evening, and night or bedtime), and the participants were compensated US $5 per complete day. To encourage consistent responding, the participants could also earn a US $10 weekly bonus by having at least 6 complete days in a given week. In addition to EMA compensation, the participants could earn US $30 for completing both the baseline survey and the orientation session. They could also earn US $5 for each weekly survey completion and US $35 and US $40, respectively, for completing the first follow-up and second follow-up surveys. These incentives together meant that the participants could earn up to US $555 for participating. Level of compensation was commensurate to level of participant involvement and consistent with regional compensation norms for similar studies. All compensation was given in the form of Amazon e-gift cards. The compensation schedule was clearly delineated in the orientation materials and in the consent forms.

Tables 1 and 2 present an overview of study measures. Across all survey types, skip patterns, error checks, and validation features were built in to ensure high-quality data and minimize data entry errors. Items were programmed in MetricWire, which is a commercially available app with features that are appropriate for smartphone-based experience sampling and EMA research. MetricWire also allows the user to take photographs or videos from within the app and permits uploads of screenshots or saved photos from a user’s phone.

For all reports of alcohol content in the media, the report first assessed the format or medium (all that apply): Snapchat, Instagram, TikTok, Facebook, YouTube, movie, TV show, music (eg, radio and Spotify), or other. For social media (Snapchat, Instagram, TikTok, and Facebook), we assessed who posted or shared the content, with the following response options: “sponsored ad/commercial that appeared,” “company/brand I follow,” “influencer/celebrity I follow,” “friend or similar-aged family member I know well,” “older friend or family member I know well,” and “someone I know but not well.” We also asked whether the content was posted publicly or privately and about who created the content, with the following response options: “same person who shared it,” “company/brand I follow,” “influencer/celebrity I follow,” “friend or similar-aged family member I know well,” “older friend or family member I know well,” “someone I know but not well,” and “someone I do not know/not sure.” Participant engagement with the content was assessed, including “liked,” “other reaction (eg, emoji),” “commented,” “shared/posted publicly,” “shared/posted privately,” “saved it or added it to a list,” “followed user/account or related group,” “blocked or deleted the user/account,” “clicked on an affiliated link,” “discussed with someone else, in person,” and “nothing (I kept scrolling).” If “shared/posted privately” was endorsed, we asked who they shared it with; response options were “close friend/best friend,” “significant other (eg, boyfriend or girlfriend),” “friend/acquaintance,” “peer (eg, classmate or coworker),” “adult family member,” “sibling/cousin similar in age,” and “other.”

For music, we asked the participants the song name and artist in a text field. For YouTube videos, we asked the participants to enter the link (URL). We also asked whether the alcohol content was part of the YouTube video or in a sponsored ad/commercial that appeared before/during the video. Likewise, for movies and TV shows, we asked whether the alcohol content was part of the movie or show or in a sponsored ad/commercial that appeared. We also asked the participants to enter the name of the movie or show and time stamp (if known) in a text field.

For all types of content, we assessed whether there were people shown or described in the alcohol content. If there were multiple people, we asked the participant to think of the person who was most engaged with alcohol. If there were people shown, we asked who the person was with the following response options: “friend/peer,” “celebrity/influencer,” “somebody I did not know,” and “other.” We asked the participants to evaluate the person along 4 domains: I want to be like this person, I wish I could do what they do, I wish I could have as much fun as they have, and I felt like the person was similar to me; each item was rated on a scale from 1 (strongly disagree) to 7 (strongly agree). The participants rated their perceptions of the person’s or character’s (1) popularity, (2) attractiveness, (3) coolness, (4) immaturity, (5) confusion, and (6) self-centeredness (prototypes) [74] using a scale from 0 (not at all) to 3 (very). The participants reported on their own attitudes as well as beliefs about teenagers their age (perceived norms) with respect to the following domains: (1) content promotes or discourages alcohol drinking, (2) liking or disliking content, (3) disapproval or approval of the content, (4) content makes drinking seem better or worse than it really is, and (5) more or less likely to drink upon viewing content (response options: better, neutral, and worse). We also asked whether the creators of the content were intending to promote or discourage drinking.

We asked respondents whether they were with someone or others (in person) when they encountered the content and, if so, who it was: “alone,” “with a close friend or best friend,” “with a significant other (eg, boyfriend or girlfriend),” “with a friend or acquaintance,” “with a peer (classmate or coworker),” “with an adult family member,” and “with a sibling or cousin similar in age.” We also asked how many other people the respondent was digitally interacting with (such as texting, online messaging, video chat, etc) when they encountered the content and, if so, how many people (none, 1, 2 or 3, and ≥4). Finally, we asked whether the participant came across any examples of marijuana (weed and cannabis), e-cigarette use, or cigarette smoking that day (bedtime report only).

In total, the self-media report contained 32 items. The scheduled media reports contained 32 items if the participant endorsed encountering alcohol content. However, participants were shown only a subset of these items due to planned skip patterns (eg, if the participant endorsed YouTube, they were not shown the 6 items about other media types, and if the participant noted there were no people shown or described in the content, they were not shown the 3 items specific to people). The shortened versions of self-media reports contained 13 items, although again participants were only shown a subset of these items. If the participant did not endorse encountering alcohol content in the scheduled media report, they completed 6 items. Consequently, most reports took between 1 and 3 minutes.

Coding manuals were developed for both image and text submissions. We strove to make the coding systems similar to one another, as the participants were given the choice of whether to upload an image or to enter text for each digital media type (eg, taking a screenshot of a song or lyrics on Spotify and entering the name of the song or relevant lyrics into the text box). The participants were also provided an open-ended text box at the end of the survey where additional information could be entered (eg, describing a scene in a film for which a screenshot was provided). For the image coding, student coders underwent a rigorous training process to ensure interrater reliability and consensus. This involved 2 rounds of training with 10 coders (80 images) each. Interrater reliability of the training images was >90%. Codes with lower reliability (eg, social and sophistication) were redefined using advisory board input, and trainers were retrained. A companion paper describing the content coding process is currently being prepared for submission.

A 7-day TLFB assessment queried alcohol use, cigarette use, e-cigarette use, cannabis use, and sleep for each day, using an online TLFB [77]. We also assessed whether the participants experienced any of the following alcohol-related problems, which were a subset of items from the Rutgers Alcohol Problem Index [56]: hangover, nausea or vomiting, self-injury, driving after drinking, blackout, rude or aggressive behavior, and unwanted sex. Alcohol use and related consequences were assessed on a weekly rather than daily basis to separate the assessment of media alcohol content from the assessment of drinking behavior.

We assessed whether the participant had planned or intended to drink but did not, whether they were at a place where alcohol was available, and whether they were offered alcohol in the past week. We assessed willingness to drink alcohol, use marijuana or weed, and use an e-cigarette if offered by a friend or classmate using the following prompt: “If a friend or classmate offered you [substance], would you use it?” Response options ranged from 0 (definitely not) to 3 (definitely yes) [78]. We assessed descriptive norms for close friends (“How often do you think that your close friends drank alcohol in the past week?”) and peers (“How often do you think that the typical student in your grade drank alcohol in the past week?”); response options ranged from 0 (did not drink alcohol) to 4 (drank alcohol daily or nearly daily), adapted from a measure used by Wood et al [60] to reflect past-week use. Injunctive norms were assessed by asking how most of their close friends feel about kids drinking alcohol, getting drunk, using marijuana or weed, or using e-cigarettes; responses ranged from 1 (strongly disapprove) to 5 (strongly approve) [60]. We also assessed positive and negative affect (5 items each) using the Positive and Negative Affect Schedule [64]; responses ranged from 1 (very slightly or not at all) to 5 (extremely).

In addition, the participants reported whether they had seen any alcohol-related content in the past week (and if so, the media platform and whether they engaged with it), as well whether they came across any content that specifically referred to positive and negative consequences of drinking. Finally, we asked whether they had publicly or privately posted any text, images, or videos of alcohol in the past week.

At baseline and preceding each burst, a web-based survey assessed sociodemographics, media, alcohol consumption and related problems, other substance use, drinking context, and source of alcohol (Table 2 provides a list of all measures).