Smartphone ownership is nearly universal in the life of American adolescents: 95% of adolescents now report either owning or having access to a smartphone, over 90% of adolescents use the internet daily, and 45% say they are online on a near-constant basis [1]. Recent statistics suggest comparable smartphone ownership among teens across gender, race and ethnicities, and socioeconomic backgrounds, with 93% of low-income adolescents reporting access to a smartphone [1]. However, research suggests key demographic differences in how youth use the internet for health. In a nationally representative survey in the United States, over 80% of adolescents reported that they had ever sought health advice on the internet, with African American and Hispanic adolescents reporting the use of web-based platforms for health information more frequently than white adolescents [2]. Lesbian, gay, bisexual, transgender, and queer (LGBTQ) youth report searching for health information online more often than heterosexual youth due to privacy-related reasons, lack of health education inclusive of their sexual orientation or gender identity, and not having anyone to ask for accurate information [3]. Integrating digital technologies into health program delivery presents an enormous opportunity to connect with youth who rely on digital health information.

Recent research assessing technology-based interventions for health has shown promise in a variety of topics and settings, including increased adherence and knowledge [4]. One systematic review of mobile apps for health and fitness found high acceptability but limited rigorous research to determine efficacy and establish evidence for best practices [5]. There is growing evidence that technology-based sexual and reproductive health (SRH) interventions can be effective [6,7]. A recent meta-analysis of 15 years of research found that technology-based interventions for youth had a significant effect on improving condom use, delaying sex, and increasing sexual health knowledge, although the effects on other sexual health outcomes were more limited [8]. Online HIV prevention interventions have shown similar efficacy as their in-person equivalents; online programs have been found to increase condom use, reduce the number of sexual partners, and reduce sexually transmitted infection (STI) incidence [9]. Similarly, Bedsider.org, a web-based educational network that provides peer feedback on birth control methods and lists youth-friendly locations to access contraceptives, has been shown to reduce unprotected sex and increase contraceptive use [10]. The enormous potential of technology-based interventions has led to the proliferation of youth-focused SRH interventions that incorporate social media, texting, and other digital components.

However, gaps remain in the research into the effectiveness of technology-based SRH interventions, particularly with certain youth populations. The analysis of long-term efficacy and the use of randomized controlled trial designs thus far are limited [8,11,12]. One systematic review of text and mobile phone app interventions for adolescents found no significant improvements in preventative sexual health behavior [13]. Furthermore, a review of apps designed for sexual health education found that the majority narrowly focused on STIs and pregnancy prevention and did not integrate evidence-based components of effective sexual health education [14]. Blended learning, which combines online and in-person instruction, is also increasing in popularity in sexual health education with mixed results [15]. Further research is needed to determine if technologies can reinforce the messaging and skill development provided in person. In addition, it is important to evaluate the viability of technology as a mechanism to reach marginalized youth populations who may not receive adequate SRH information through traditional approaches.

Sexual health education can provide critical information, but the content and quality of the curricula vary substantially [16]. Furthermore, traditional programming that focuses on pregnancy prevention often ignores the broader health and developmental issues that youth face. Incorporating educational and career success, healthy life skills, and healthy relationships into sexual health education may build youth self-efficacy in making positive life choices that impact sexual health and overall well-being [17]. However, few sexual education programs cover these more comprehensive topics [18].

Although the adolescent pregnancy rate is declining nationwide, substantial disparities persist [19]. In addition, the rates of STIs are increasing among adolescents and disproportionately affect youth from certain racial, ethnic, geographic, and socioeconomic backgrounds [20]. Too often, sexual health education and services do not reach these youth or do not reflect their experiences and backgrounds [21]. In many cases, youth who are at the greatest need for comprehensive programming are less likely to receive it. For example, youth who frequently move, are unstably housed, or in foster care may miss school-based programming [21,22]. Similarly, few sex education curricula are designed to be inclusive of same-sex partners, sexual orientation, or gender identity [23]. In addition, many of the most common sexual health curricula were developed decades ago with limited adolescent input during the design phase. These curricula often seem dated and culturally irrelevant to youth [21].

Youth who are unstably housed, homeless, or in foster care are at a higher risk of unplanned pregnancy and STIs [24,25]. These young people often have limited access to consistent health care, are more at risk for coerced sexual activity, are more likely to have experienced trauma, and are more likely to exchange sex for money or other basic resources [24]. As a result, the rate of STI and HIV infection in homeless youth is 2 to 10 times higher than that of other adolescents [26]. Youth of color and sexual minority youth also have an elevated risk for adolescent pregnancy and STIs [19,25].

Many vulnerable youth who may need SRH services worry about provider attitudes, privacy and confidentiality, and stigma [27]. Providers are rarely competent in LGBTQ-specific health concerns, so LGBTQ youth often face discrimination when seeking health services [27]. Providing relevant information about accessible, confidential, and respectful health services for these youth can increase their use of contraceptives and improve the overall health service utilization [28].

Adolescents representing the diverse target populations helped to develop ITK by engaging in a user-centered design process, which recognizes that youth are the experts in their own lives [29,30]. Through a series of workshops, youth brainstormed ideas for intervention content and design and developed rapid prototypes alongside the design team. Youth then reviewed ideas and provided feedback on multiple design and content iterations.

ITK was designed using a trauma-informed approach and a positive youth development framework. This behavioral approach to adolescent development views youth as having assets that can be cultivated to reach their full developmental potential [31,32]. It focuses on creating an environment that supports protective factors, which promote personal strengths and resilience.

The in-person, classroom component of the intervention is divided into 3 modules that can be implemented in 1 day or over the course of a few days. The total implementation time is approximately 6.5 hours, with each module lasting approximately 2 hours. Module 1: Sexual health and contraceptive use teaches youth the functions of the sexual and reproductive system; sexual orientation and inclusivity of all gender and sexual identities; how pregnancy occurs; birth control methods and correct condom usage; and STI prevention, symptoms, testing, and treatment options. Module 2: Healthy relationships helps youth understand healthy relationships, including communication, consent, and sexual violence prevention, and teaches life skills, such as stress management, identifying strengths, and goal setting. Module 3: Educational and career success informs youth of education and career options and teaches them about financial aid, resume and cover letter writing, interview skills, and budgeting.

The web-based component incorporates digital technologies to complement and reinforce the key content covered in the classroom intervention. It uses technology-based strategies to digitalize the components of the existing curricula, including text message reminders, gamification, geo-location, and web-based resources such as videos. Youth can access this information through a downloadable app or a website (Figure 1).

This enables health educators to engage with youth on platforms easily accessed by them (such as mobile phones and tablets) to provide high-quality education and training while offering supportive tools that allow youth to practice new knowledge and skills. In addition to introducing the app and explaining how the app works at the beginning of the cohort, health educators demonstrate and support the participants in using one or more key features of the app during each session. Health educators remind participants that the app supplements the in-class education, offers reinforcement of messaging, and has additional resources that youth can access after the session. Taking the knowledge gained in the classroom, participants can generate health or career goals and set reminders to keep them on track. Participants can also complete short quizzes for points. Under the Resources tab, users can search for and find local services and resources using a Yelp-like feature that allows them to locate clinics on a map and rate the services after they have used them. In addition, in the Knowledge section of the mobile app, users can explore curated articles and videos. Youth are able to take this information and resources with them after completing the class, enabling them to refer to materials and locate services as needed. Participants also sign up to receive text messages for 1 month after the in-person sessions end, with information and reinforcement of key messages. They can also schedule reminders to complete activities to achieve their personal sexual health, relationship, and career development goals that they selected.

The first primary outcome is condom/contraceptive use or no sex in the last 3 months. This is assessed at 3 months by asking participants how often they used birth control, including condoms, when they had vaginal sex in the past 3 months; and how often they used a condom when they had anal sex in the past 3 months. The second primary outcome is the use of any clinical health services in the last 3 months. This is assessed at 9 months by asking participants whether they have received mental health services or counseling, substance abuse treatment, or sexual health services from a doctor, counselor, therapist, social worker, or clinic in the past 3 months.

The secondary outcome measures are as follows:

The outcomes and covariates are shown in Table 1. All measures are collected at baseline, 3 months, and 9 months, except where noted in the table.

A target sample of 1360 youth (680 per study arm) will be enrolled in the study. The number of cohorts to be enrolled in the study is 68 per study arm, with an estimated 10 youth per cohort. We estimate 85% (1156/1360) retention at 3 months (578 per arm) and 80% (1088/1360) retention at 9 months (544 per arm).

Sample size calculations were based on the first primary (binary) outcome, which is condom/contraceptive use or no sex in the past 3 months (assessed at 3 months). We selected an intracluster correlation coefficient (ICC) of 0.02, which is within the range of previous group-randomized trials of school-based HIV, STI, and pregnancy prevention interventions in the United States [34]. With a two-sided significance level of 5% and a power of 80%, this sample size is sufficient to detect an increase in condom/contraceptive use or no sex from 60% to 67% at 3 months, with an estimated 15% loss to follow-up.

We have also planned for adequate study power for the second primary (binary) outcome, any use of clinical services in the past 3 months (at 9 months). With a two-sided significance level of 5%, a power of 80%, and an ICC of 0.02, this sample size is sufficient to detect an increase in use of clinical services from 30% to 38% at 9 months, with an estimated 20% loss to follow-up.

Participants complete the baseline survey on a tablet, although paper surveys are available as needed. The baseline survey includes questions from all domains, as shown in Table 1.

All participants are asked to complete web-based follow-up surveys 3 and 9 months after baseline. A link to complete the follow-up surveys is sent via email and/or a text message to their mobile phone; participants are asked to state their preferred method of survey delivery at baseline. When the follow-up surveys are due, participants receive the survey via the method of their choice.

The 3- and 9-month follow-up surveys collect repeat measures of the outcomes measured at baseline. They also ask youth to report what websites and apps they have used to search for health information and services in the past 3 months.

Protocols have been established to ensure the confidentiality of personal information about participants. Data are encrypted and transmitted securely through Qualtrics, a tool for collecting web-based surveys that meets the security requirements for UCSF research. Surveys are void of participant identifiers such as names and addresses. Participants are only identified through their study identification numbers. Electronic data are stored on encrypted, password-protected computers within a secure network, inside a building with limited access. Audio-recorded data, such as focus groups and interviews, and any paper-based surveys are stored in a secure and locked file cabinet. Only authorized personnel have access to the data, and all data are de-identified before analyses.

We expect that retention may be challenging due to the high mobility of the study’s target population. We developed a protocol to maximize retention via incentives and reminders. Study participants can receive up to US $60 for completing all surveys. The following is the incentive structure:

At baseline, detailed contact information, including telephone numbers, email addresses, and mailing addresses, is collected. Participants are asked to provide this information for themselves as well as at least two alternative contacts, such as a parent, case manager, relative, friend, or other trusted adult. Alternate contacts are only used if the contact information for the participant is no longer accurate. EOC staff give participants small cards with the dates they will receive their follow-up surveys. Participants receive ongoing reminders about the study through text messages approximately every 2 months.

When youth are eligible to receive their follow-up survey, they are sent the survey link through their choice of text message or email. They receive two additional requests to complete the survey, at 3 and 7 days. If participants do not complete the follow-up survey during the initial outreach period, researchers call youth or their alternate contacts and offer to resend the survey or have the youth complete the survey verbally over the phone.

If the researchers are unable to contact the youth via email, text messages, or phone, EOC staff return to the implementation site, administer paper-based follow-up surveys, and collect new contact information for the youth. Surveys are mailed to the UCSF for processing.

We will compare participants who refuse to participate and those lost to follow-up with the baseline sample to assess whether they vary by study arm, sociodemographic factors, or site. We will report any nonrandom loss to follow-up and consider the impact on the interpretation of our results.

We will report rates of and reasons for missing data, and we will assess whether participants with missing data differ systematically from others on sociodemographic characteristics, site, or trial arm. We plan to use multiple imputation to impute missing values for the predictor variables.

The intervention arm (ITK) will be compared with the control arm (standard of care) for all primary analyses. The analysis population will include all enrolled participants. We will compare changes in condom/contraceptive use or no sex in the last 3 months from baseline to 3-month follow-up. We also will compare changes in the use of any clinical health services in the last 3 months from baseline to 9-month follow-up. We will use mixed-effects logistic regression analysis with random effects at the individual, cohort, and site levels to account for clustering. Unadjusted models will include a variable for the study group, time, and interaction between study group and time. Additional models will be estimated adjusting for sociodemographic characteristics known to be associated with the primary outcomes (age, race/ethnicity, gender, sexual orientation, rural or urban location, and housing status). We will conduct a correlation analysis among the control variables and consider the directionality of the relationship between control variables and outcome variables. For subgroup analyses, we will also test the interactions of the study group with age, race/ethnicity, gender, sexual orientation, rural or urban location, and housing status.

The analysis of the secondary outcomes will be similar to that of the primary outcomes. To compare outcomes between the treatment and control arms longitudinally, we will use mixed-effects logistic regression analyses for binary outcomes and mixed-effects linear regression for continuous outcomes.

In addition to the outcome data, we collect a variety of measures to verify fidelity to the curriculum, identify potential challenges, and receive feedback from youth participants. Data sources and methods include the following: immediate post survey, attendance logs, implementation logs, interviews, focus groups, site observations, and web-based analytics.

Participants in the intervention group are asked to complete a survey on the last day after all intervention activities have been completed. The immediate post survey is generally administered on tablets but is available in a paper-based format if needed. It assesses reproductive health knowledge and perceptions of the in-person and technology-based components.

Health educators collect attendance data for each participant. The UCSF uses this to measure the number of youth served and the amount of the intervention (dosage) they receive. The UCSF reviews attendance logs in conjunction with the number of surveys collected to ensure accuracy.

Conducted for each intervention cohort, this information tracks the fidelity of ITK delivery and any adaptations made. The UCSF reviews these fidelity checklists and provides technical assistance as needed.

The UCSF interviews health educators annually to assess their perception of the program and identify implementation challenges and successes.

The UCSF conducts 6-8 youth focus groups annually. A subset of participants (purposively sampled to ensure different backgrounds are represented) are invited to share their perceptions of the program and provide feedback on how it can be improved.

The UCSF conducts quarterly site observations, with sites purposively selected to represent different youth populations. They focus on factors that may affect the quality of implementation, the extent to which the intervention is delivered with fidelity, implementation challenges, and needs for additional technical assistance or training.

Web-based analytics capture the extent and type of technology used by participants, including the use of tools, resources, and referrals on the app and website at the aggregate level. This provides an important measure of dosage and utility.