This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
American Indian and Alaska Native (AI/AN) youth face multiple health challenges compared to other racial/ethnic groups, which could potentially be ameliorated by the dissemination of evidence-based adolescent health promotion programs. Previous studies have indicated that limited trained personnel, cultural barriers, and geographic isolation may hinder the reach and implementation of evidence-based health promotion programs among AI/AN youth. Although Internet access is variable in AI/AN communities across the United States, it is swiftly and steadily improving, and it may provide a viable strategy to disseminate evidence-based health promotion programs to this underserved population.
We explored the potential of using the Internet to disseminate evidence-based health promotion programs on multiple health topics to AI/AN youth living in diverse communities across 3 geographically dispersed regions of the United States. Specifically, we assessed the Internet’s potential to increase the reach and implementation of evidence-based health promotion programs for AI/AN youth, and to engage AI/AN youth.
This randomized controlled trial was conducted in 25 participating sites in Alaska, Arizona, and the Pacific Northwest. Predominantly AI/AN youth, aged 12-14 years, accessed 6 evidence-based health promotion programs delivered via the Internet, which focused on sexual health, hearing loss, alcohol use, tobacco use, drug use, and nutrition and physical activity. Adult site coordinators completed computer-based education inventory surveys, connectivity and bandwidth testing to assess parameters related to program reach (computer access, connectivity, and bandwidth), and implementation logs to assess barriers to implementation (program errors and delivery issues). We assessed youths’ perceptions of program engagement via ratings on ease of use, understandability, credibility, likeability, perceived impact, and motivational appeal, using previously established measures.
Sites had sufficient computer access and Internet connectivity to implement the 6 programs with adequate fidelity; however, variable bandwidth (ranging from 0.24 to 93.5 megabits per second; mean 25.6) and technical issues led some sites to access programs via back-up modalities (eg, uploading the programs from a Universal Serial Bus drive). The number of youth providing engagement ratings varied by program (n=40-191; 48-60% female, 85-90% self-identified AI/AN). Across programs, youth rated the programs as easy to use (68-91%), trustworthy (61-89%), likeable (59-87%), and impactful (63-91%). Most youth understood the words in the programs (60-83%), although some needed hints to complete the programs (16-49%). Overall, 37-66% of the participants would recommend the programs to a classmate, and 62-87% found the programs enjoyable when compared to other school lessons.
Findings demonstrate the potential of the Internet to enhance the reach and implementation of evidence-based health promotion programs, and to engage AI/AN youth. Provision of back-up modalities is recommended to address possible connectivity or technical issues. The dissemination of Internet-based health promotion programs may be a promising strategy to address health disparities for this underserved population.
Clinicaltrials.gov NCT01303575; https://clinicaltrials.gov/ct2/show/NCT01303575 (Archived by WebCite at http://www.webcitation.org/6m7DO4g7c)
As of 2012, an estimated 5.2 million individuals in the United States identified as American Indian and Alaska Native (AI/AN) alone or combined with other races, comprising 2% of the nation’s population [
AI/AN youth also face disparities related to obesity and diabetes. Previous studies have indicated that 20-30% of AI/AN children are obese [
These health disparities could potentially be ameliorated by the dissemination and implementation of evidence-based adolescent health promotion programs. The United States Department of Health and Human Services’ Office of Adolescent Health, Substance Abuse and Mental Health Services Administration, and Centers for Disease Control and Prevention (CDC) have identified adolescent health promotion programs proven to impact behavior change across various health domains, including sexual health, violence and substance use prevention, physical activity, and nutrition [
Limited data exist regarding the reach and implementation of evidence-based health promotion programs among AI/AN youth. The Bureau of Indian Education (BIE) supports 183 primary and secondary schools in 23 states, serving approximately 50,000 students. A school health assessment of BIE schools conducted in New Mexico in 2006 reported that 38 of 39 schools (97%) provided health education and/or health promotion activities, and 67% used a comprehensive health education curriculum [
Technology-based programs may offer a viable strategy to increase the reach and implementation of evidence-based health promotion programs in this underserved population [
Recently, Internet-based sexual health promotion programs have been specifically developed or adapted for AI/AN youth, with high satisfaction ratings reported [
This study presents a secondary analysis of data collected during the implementation phase of a randomized controlled trial (Clinicaltrials.gov NCT01303575) that assessed the effectiveness of Native It’s Your Game (Native IYG; a Web-based sexual health education program adapted for AI/AN youth) relative to a comparison suite of 5 evidence-based Internet-based health promotion programs, across 3 geographically dispersed regions in the United States. A detailed description of the adaptation process for Native IYG is provided in a supplemental file [
Participants were primarily self-identified AI/AN youth aged 12-14 years and adult site coordinators (teachers, counselors, nurses, wellness coordinators, and college students) that were recruited from 25 study sites. The sites were located in 13 urban and 12 rural/tribal settings in Alaska, Arizona, and the Pacific Northwest (Oregon, Idaho, and Washington), and comprised schools, tribal community centers, after school programs, and summer youth programs. Given the importance of confidentiality when partnering with AI/AN communities, specific tribal names have been withheld. The study was approved by the Alaska Area Institutional Review Board (IRB), the Portland Area Indian Health Service IRB, the University of Texas Health Science Center at Houston (UTHealth) IRB, and 16 tribal organizations (ie, tribal councils, tribal health boards, villages, and community agencies).
Study activities were coordinated regionally by 3 organizations that collectively serve 295 federally recognized AI/AN tribes. Regional staff used convenience sampling to recruit tribal communities that were interested in participating in an early adolescent sexual health study. Regional staff sent flyers to schools, tribal community centers, after school programs, and summer camp programs, and advertised on organizational websites, social media outlets, and/or via newsletters. Participating sites were randomized to a treatment condition (Native IYG, n=14) or a comparison condition (n=11) featuring a suite of 5 evidence-based Internet-based health promotion programs that were not focused on sexual health.
Site coordinators at each study site completed the Collaborative Institutional Training Initiative Program’s online certification in human subject research and a live webinar (tailored for treatment or control conditions) coordinated by UTHealth research staff, which explained intervention content and protocols for logging-in youth, documentation, and maintaining confidentiality.
Native IYG is a 13-lesson, multimedia sexual health education curriculum for AI/AN youth (aged 12-14 years). The curriculum was adapted from an Internet-based curriculum for urban middle schools titled
The 5 evidence-based, Internet-based programs that comprised the comparison suite addressed hearing loss (Dangerous Decibels), alcohol use (N-Squad), tobacco use (A Smoking Prevention Interactive Experience; ASPIRE), drug use (Reconstructors), and physical activity and nutrition (The Quest to Lava Mountain). Each program has undergone usability testing with non-AI/AN youth as part of its own formative development, and has demonstrated efficacy to positively impact health behaviors and/or related psychosocial factors in other adolescent populations. The Dangerous Decibels Virtual Exhibit was developed by the Oregon Museum of Science and Industry as an online component of a public health campaign to reduce the incidence and prevalence of noise-induced hearing loss and tinnitus (ringing in the ear) by improving knowledge, attitudes, and protective behaviors of school-aged children [
Site coordinators logged participants onto the programs on desktop or laptop computers located in quiet locations (eg, a computer lab, empty classroom, or library). Sites with insufficient bandwidth to accommodate simultaneous Internet access by multiple users were provided with uploadable versions of their respective programs on a Universal Serial Bus (USB) drive or digital video disc (DVD).
Prior to implementation, site coordinators completed a computer-based education inventory survey [
We used Likert-type scales adapted from previous studies to assess youths’ perceptions of engagement [
Ratings were collected via an Internet-based Qualtrics usability survey administered at the completion of each program. Demographic characteristics (gender, age, and self-identified race/ethnicity) were collected during the study’s baseline survey, using an Internet-based Qualtrics self-report survey [
We used descriptive statistics (frequencies, median/mean, and/or range) to summarize data on reach parameters (computer access, connectivity, and bandwidth) and implementation (program errors and technical issues). Regarding youth engagement, for each parameter we calculated the percent of youth who rated each program favorably, and the range of ratings across all 6 programs, from lowest to highest.
Twenty-four site coordinators (24/25, 96%) provided complete or partial information related to Internet reach parameters. Eighteen computer labs, one classroom, and one after school classroom across the 3 regions were primarily composed of personal computers (13/20, 65%) and Mac computers (6/20, 30%), were mostly wired (16/18, 89%), and most had access to the Internet (22/24, 92%). Primary Web browsers included Chrome (7/20, 35%), Safari (5/20, 25%), Internet Explorer (5/20, 25%), Firefox (2/20, 10%), and Mozilla (1/20, 5%). Download speeds ranged from 0.24 megabits per second (Mbps) to 93.5 Mbps (mean 25.6 Mbps, standard deviation 31.14; median 6.37 Mbps).
At some sites, the method of program delivery changed during implementation. Treatment sites commenced the study accessing Native IYG as an online streaming program (n=12) or as an uploadable program from a USB drive (n=2). During the study, several sites that initially accessed Native IYG as an online streaming program converted to uploading Native IYG from a USB drive (n=4) due to the inability of local bandwidth to accommodate larger Native IYG video files while providing simultaneous streaming for multiple youth.
Comparison condition sites commenced the study by accessing the suite of health promotion programs via online streaming (n=8) or a combination of online streaming and an uploadable program from DVDs (n=2). Information regarding program access was missing from one site. During the study, most sites continued to access these programs via Internet connections (n=8).
The most commonly reported problems that were documented by site coordinators during implementation included frozen screens (4/6 programs), activities taking a long time to load (3/6 programs)—both of which were related to multiple simultaneous users—and trouble navigating the programs (3/6 programs).
During implementation, 387 youth received Native IYG, of whom 191 (49%) provided feedback; 136 youth received the comparison suite of programs, of whom 108 (79%) completed at least one feedback survey. Across programs, participants were 48-60% female, with a mean age of 13.1-13.3 years, and 85-90% self-identified as AI/AN (
Demographic characteristics for AI/AN youth who provided ratings for each program (n=40-191): Alaska, Arizona, and Pacific Northwest, 2012-2014.
Native IYG |
Dangerous Decibels |
N-Squad |
ASPIRE |
Reconstructors |
Lava Mountain |
||
Female | 114 (59.7) | 30 (48.4) | 34 (54.8) | 25 (48.1) | 24 (53.3) | 21 (52.5) | |
Male | 77 (40.3) | 32 (51.6) | 28 (45.2) | 27 (51.9) | 21 (46.7) | 19 (47.5) | |
Self-identify as AI/AN, n (%) | 164 (85.9) | 54 (87.1) | 56 (90.3) | 46 (88.5) | 40 (88.9) | 34 (85.0) | |
Mean age (standard deviation) | 13.1 (0.98) | 13.2 (0.83) | 13.3 (0.84) | 13.3 (0.77) | 13.2 (0.78) | 13.3 (0.88) |
Youth generally rated the programs as easy to use (68-91%) and the majority understood the words in the programs (60-83%). However, some participants needed adult assistance or hints to complete the programs (16-49%). Over half of the youth reported that the program content was credible, rating it
AI/AN youth face multiple health challenges compared to youth of other racial/ethnic groups. Viable program delivery strategies that overcome limited personnel with training in health education, cultural barriers, and geographic isolation (ie, the Internet) are needed to increase the reach and implementation fidelity of evidence-based adolescent health promotion programs in tribal communities. We examined the potential of using the Internet to deliver 6 evidence-based health promotion programs to AI/AN youth living in urban and rural settings in 3 geographically diverse regions of the United States. The health topics addressed included sexual health, hearing loss, substance use, physical activity, and nutrition. Despite variability in connectivity and bandwidth, most sites were able to access the programs via the Internet. However, technical and connectivity issues led some sites to access the programs via back-up modalities (ie, uploading the programs from a USB drive or DVD). Practitioners interested in implementing Internet-based programs in tribal communities are advised to provide contingency plans as back-ups to technical failures. We also recommend conducting bandwidth assessments, especially when dealing with multiple simultaneous users, prior to implementing an Internet-based program.
Adult site coordinators from each tribal community (including teachers, counselors, nurses, wellness coordinators, and college students) facilitated youths’ access to the Internet-based programs. Regarding implementation fidelity and program errors, site coordinators reported issues with frozen screens (4/6 programs), activities taking a long time to load (3/6 programs), and trouble navigating the programs (3/6 programs). Given that 16-49% of youth needed assistance to complete the programs, some adult oversight is recommended when implementing Internet-based programs for youth. Although the site coordinators received webinar training prior to program implementation, the training focused primarily on research study-specific protocols (eg, logging students onto computers using study identification numbers and reporting technical issues). The training related to actual program implementation, such as reviewing specific program content, was limited to approximately one hour, indicating that a broad range of personnel (including those not certified in health education) may be able to implement Internet-based health promotion programs.
Overall, AI/AN youth rated the programs favorably. Although fewer youth rated the programs as being
It is worth noting that youth rated Native IYG most favorably across all programs on 10 of the 13 usability parameters, possibly because this program was specifically adapted for AI/AN youth. This finding aligns with previous studies that point to the value of cultural tailoring [
Across all programs, we experienced attrition in the number of youth who received the programs compared to those who completed a feedback survey. This attrition may have been due to respondent burden or fatigue in completing surveys after each program. However, the relatively high motivational appeal ratings across programs (eg, at least 60% of youth stated that the programs were
This study has several strengths. First, most feasibility studies for technology-based health promotion programs feature small numbers of participants (typically <30 users), as they do not require statistical significance to determine major usability problems [
Despite these strengths, several limitations should be noted. First, the sample was restricted to early adolescent AI/AN youth, aged 12-14 years; thus, findings may not apply to older AI/AN youth. Second, social desirability may have biased youths’ ratings of the programs; however, the fact that some items scored lower than others (eg, comparing programs to their favorite video game) suggests that youth answered honestly, based on their experience with each program. Third, the intervention program, Native IYG, was specifically adapted for AI/AN youth. In contrast, no attempt was made to adapt the comparison group programs for AI/AN youth; this factor may have negatively impacted program ratings relative to Native IYG. Finally, although the use of percent ranges to evaluate engagement is sufficient for this kind of exploratory trial, more thorough quantitative analyses, such as appropriately powered randomized pilot tests of the programs’ impact on behavioral determinants or randomized controlled efficacy field trials, are needed to determine long-term behavioral impact.
Despite these limitations, this study demonstrates the potential of using the Internet to disseminate evidence-based health promotion programs to AI/AN youth across 3 separate geographic regions. These findings may have broader implications for understanding the degree to which Internet-based programs may enhance the reach and implementation of evidence-based health promotion programs in tribal communities, and provide an educational format that is engaging for AI/AN youth.
American Indian and Alaska Native (AI/AN)
Bureau of Indian Education
Centers for Disease Control and Prevention
digital video disc
Institutional Review Board
It’s Your Game
megabits per second
Universal Serial Bus
University of Texas Health Science Center at Houston
All authors contributed to the conception and design of the study, and the process of drafting and revising the manuscript. This study was funded by the Centers for Disease Control and Prevention (CDC #5U48DP001949-02). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC. We would like to thank the youth and site coordinators for their participation in the study.
CMM served as joint Principal Investigator with RS on the conceptualization and design of the study, drafted the initial manuscript, and approved the final manuscript as submitted. SCR, CJ, and GG served as Regional Principal Investigators for the study, contributed to the intervention design, coordinated site recruitment in their respective regions, codesigned the data collection instrument, critically reviewed and revised the manuscript, and approved the final manuscript as submitted. WEL, AVP, and LM were program developers for Dangerous Decibels, ASPIRE, N-Squad, and Reconstructors, respectively. These authors critically reviewed and revised the manuscript, and approved the final manuscript as submitted. KA-F served as a representative for The Cooper Institute (the program developer for The Quest to Lava Mountain), critically reviewed and revised the manuscript, and approved the final manuscript as submitted. JT coordinated intervention development and data collection across all 3 regions, assisted in data cleaning and initial analyses, drafted sections of the initial manuscript, critically reviewed and revised the manuscript, and approved the final manuscript as submitted. RCA conducted statistical analyses, critically reviewed and revised the manuscript, and approved the final manuscript as submitted. MFP served as Co-Investigator for the study and codesigned the data collection instrument, study design, and analyses. She critically reviewed and revised the manuscript and approved the final manuscript as submitted. RS served as joint Principal Investigator with CMM on the conceptualization and design of the study, critically reviewed and revised the manuscript, and approved the final manuscript as submitted.
None declared.