Although many young people make the transition to adolescence with only minor behavioral problems and school-related difficulties, a significant number of at-risk youth develop problem behaviors that are serious, that may last a lifetime, and that could impair later functioning. During the past decade, substance use has remained a serious public health concern, with 35% of eighth graders reporting having tried alcohol and associated increases in substance use during the adolescent years [1]. Early adolescence (ages 11-14 years) is a time of rapid biological and social transition. Interactions between parents and their child’s middle school are significantly more formalized and less frequent than in elementary school [2]. As a result, parents tend to become less involved in their child’s overall adjustment, which may subsequently lead to a variety of behavioral and social problems in high school. As a result, middle school is an ideal developmental period for family-centered prevention that targets reduction of problem behavior and substance use through teaching and supporting effective parenting skills.

Research during the past two decades has established that skillful family management, including applying positive parenting skills, setting limits, monitoring, and effectively solving problems, during adolescence protects youth from a variety of mental health and behavioral problems. Most family-centered intervention studies have focused on parenting skills as a direct target of intervention, guided by a theoretical model whereby parenting skills are the mediator that links early risk with a profile of later behavioral risk, including problem behavior, substance use, and school failure [3-6]. Fortunately, positive changes in parental family management skills have been linked to meaningful improvements in adolescent behavior across family cultures and ethnic groups [7,8]. Even among adolescents who exhibit risk, such as affiliation with deviant peers, improved family management skills by parents has been shown to decrease the growth of externalizing behavior during adolescence [9].

In this paper we describe the background, research design, and intervention components of an ongoing project funded by the US National Institute on Drug Abuse (R01DA037628) that is intended to develop and test the usability, feasibility, and efficacy of an Internet-based version of the Family Check-Up (FCU) program as a universal prevention intervention that targets middle school children. In Multimedia Appendix 1 we present the summary statement generated by peer reviewers in the US National Institute on Drug Abuse Study Section prior to our research being funded.

The FCU is a strengths-based, family-centered intervention that promotes family management and parent skill enhancement and addresses child and adolescent adjustment problems [10]. It has two components: (1) an ecological strengths-based self-report assessment of child behavior, parenting skills, family dynamics, and life stressors, followed by focused feedback; and (2) parent management training, which focuses on supporting positive behavior, setting healthy limits, supervision, and building relationships [11]. Depending on the particular program configuration used, the FCU family feedback session can be held at the family home, a clinic, a school, or a community center, and its delivery is typically facilitated by a counselor or coach (in school settings) or a therapist (in community mental health settings). The FCU can be delivered as both a preventive checkup and as an intensive intervention for high-risk families.

Multiple federally funded grants have examined the FCU in randomized controlled trials based in public schools that involved ethnically and socioeconomically diverse young children and middle-school–aged youths [10,12-15]. Strong effects have been found on both proximal and distal outcomes, including substance use, health behavior, and depression. The FCU delivered in middle school has been linked to long-term improvement in academic outcomes (self-regulation, grade point average, school attendance and engagement, and teacher-rated child problem behavior over time) [14,16,17] and various nonacademic outcomes (eg, depression, substance use, and high-risk sexual behavior) [17-20]. It was also related to decreased arrest rates, problem behavior, and substance use. These positive effects have been found to persist through high school and the early-adult years [5,21,22].

The FCU was found to have direct effects on putative mediators, such as youth self-regulation, and on outcomes such as deviant peer affiliation, substance use, and family conflict [17,23]. When the FCU was delivered in schools, teachers reported reduced problem behavior across the 3 years of middle school [14] and in school-related outcomes [24]. The putative mediators associated with changes in behavior across these intervention trials included parenting skill enhancement and youth self-regulation [17,23], with a particular focus on positive parenting across the life span. Improved self-regulation during the middle school years predicted reduced risk behavior during the transition to adulthood more than 10 years later [20].

Electronic health (eHealth) interventions delivered via the internet are rapidly being developed for a wide variety of target behaviors, and they have shown encouraging efficacy in controlled trials, for example, for smoking cessation [25-27], depression treatment [28-30], and obesity management [31,32].These programs can be stand-alone (fully automated), which reduces their cost of delivery while greatly increasing their reach (their public health impact), or they can include live contact with coaches or counselors in face-to-face sessions or through telephone calls [33], which increases participant adherence through accountability to a coach who is seen as trustworthy, benevolent, and having expertise. Mohr et al [33] posited a model of supportive accountability that describes how participant engagement and follow-through in eHealth interventions can be encouraged by the human support provided by a coach, for example, when participants receive brief calls from the coach. Using the term coach implies that the coach’s interaction with families need not require the skills of a highly trained clinician [34]. This level of coach support has been found to enhance the efficacy of eHealth interventions for tobacco cessation [35-37] and depression [38-40].

A number of studies have examined Web-based parent-training programs. Some programs have adopted a video teleconference approach to enable coaches or therapists to observe family interactions and guide treatment activities at parents’ homes (eg, the work of Comer and colleagues [41-43] on internet-facilitated Parent Child Interaction Therapy). Other parent-training programs include multimedia and program content designed for parents to use on their own or under the guidance of trained coaches. Examples of these studies are Incredible Years [44], a mobile phone–based version of Helping the Noncompliant Child [45], a Web-based implementation of the Strongest Years program delivered in Sweden [46], the Parenting Wisely program [47], some of the tests of the Triple-P Online program [48,49], and ezParent, a tablet-based intervention designed for a low-income, ethnic minority population of parents [50].

Although a variety of parent interventions in public schools have motivated positive change in parenting and reduced problem behavior [12,51,52], few children and adolescents ever receive treatment for these problems when interventions are fully disseminated, and only a very small percentage of parents participate in parenting or family interventions to address behavioral problems [53,54]. Several likely reasons could explain this problem, including inadequate funding for implementation, schools’ competing priorities, complicated logistical requirements for treatments, inadequate time for teachers and staff to be trained and to deliver the program, and parents who are difficult to recruit [55,56]. In a randomized effectiveness trial of the FCU model, we found that schools were unable to administer the FCU to families in a systematic way, although schools were generally supportive of delivering family-centered interventions from the school. The lack of trained staff and time for implementation were key factors that limited the uptake of the intervention, which was associated with improved outcomes for high-risk students, such as increased parental monitoring and decreased negative school contacts, despite poor implementation [20,57]. This research inspired our efforts to develop an eHealth intervention version of the FCU that could be administered to families with little or no staffing from schools. We used an iterative approach to development that was guided by family and school focus groups, testing of the eHealth version, and adaptations based on feedback. In the next section, we describe our development process, intervention modules, and study design.

We used an integrated technology architecture for the FCU Online website, its administration website, and coach portal, which involved sharing a common database. This resulted in a seamless development process that enhanced quality control and user data tracking. Program components were fully tested on a preproduction server before being moved to the live production environment.

The FCU program comprises 3 separate but complementary entities: assessment and feedback, skills sessions in the parent website, and an administration website (Figure 1).

Families in all 3 conditions are mailed a follow-up questionnaire at 3 months, 6 months, and 12 months that is identical to the preassessment in order to assess changes in constructs over time (see Table 2). Each follow-up questionnaire is estimated to require 30 minutes to complete. The 3-month follow-up questionnaire for parents in the 2 intervention conditions also includes a 2-page website feedback survey that requires approximately 10 minutes to complete. All families receive remuneration for their time: US $100 for completing the baseline assessment and US $100 for completing each of the 3 follow-up assessments, for a total of $400. Families can also earn a US $50 bonus if they complete all 4 assessments.

At multiple points in the iterative development process, we have gathered information from potential users about what was working in the program and what needed to be adapted or reformulated. The first of these focus groups included 6 parents and the dean of students from a participating middle school. Three other focus groups included various school staff from 4 schools (2 suburban and 2 rural). The first included 2 principals, 2 vice principals, and 1 school counselor; the second included 1 principal, 2 deans of students (1 of whom was also called a family liaison staff), and 1 counselor; and the third included 1 principal, 1 counselor, and 1 behavioral specialist. We integrated the feedback from these groups into the development and design of the program (eg, enable users to go back and retake the FCU assessment, provide more tips and the shorter the better, have children looking at mobile phones in the pictures so the images appear more accurate and up-to-date, allow different credential levels on the administration site).

Next, we conducted usability testing with 5 participants to examine the acceptability and feasibility of the program. Usability testers met individually with a research staff member in 90-minute sessions during which they interacted with portions of the FCU Online program while testers used think-aloud techniques to describe their ideas and thoughts. Usability test participants were also asked to complete the 10-item System Usability Scale [85] to examine the acceptability and feasibility of the program. Items in this scale include “I think that I would like to use this website frequently” and “I thought the website was easy to use.” We then used these qualitative and quantitative data to further improve the design of the program.

Finally, we conducted a pilot study with 7 participants in either the FCU Online-only or the FCU Online + Coach condition. Parents were then given 2 weeks to use the website and meet with a coach (if applicable). Next, parents provided verbal feedback about their experience with the assessment process in general, and with the website specifically, during a debriefing interview with project staff members. This feedback was used to further improve the surveys and program logic.

Families are randomly assigned to a condition and will be the unit of analysis for all models. Mixed-effects analyses will be based on a hierarchical linear modeling approach in which students are nested within schools; primary outcomes are nested within individual students at level 1 of the model; and between-participant predictors (fixed effects), such as treatment condition and child and parent demographics, will be examined at level 2. This approach will (1) account for the correlated within-participant errors created by nesting of repeated measurements within study participants, (2) allow us to examine longitudinal trajectories within a unified and flexible framework that also facilitates examination of potential moderating and mediating variables, and (3) enable us to test for potential dependencies (school-level effects) in the data. For each of the 3 pairwise contrasts between conditions, we will examine intervention effects by modeling longitudinal trajectories across time with mixed-effects models using SAS PROC MIXED (SAS Institute) or Mplus software (Muthén & Muthén).

Using an intent-to-treat approach with 100 participants per condition, with alpha set to .017 (to adjust for multiple contrasts), there is sufficient power (>.80) to detect a condition effect of Cohen d=.42 or larger (moderately small effect size) between either intervention condition and the control group on primary outcomes, which include effective parenting skills and reductions in child problem behavior. Previous FCU efficacy trials have demonstrated medium to large effects for tobacco use, alcohol use, cannabis use, antisocial behavior, and arrest rates [15,21].

This paper describes the innovative FCU Online eHealth intervention randomized controlled trial for parents of middle school children. Our report focuses on the background, research design, and intervention components of a trial that will develop and test the usability, feasibility, and efficacy of an eHealth version of the FCU program that targets middleschool children. The rigorous study design will allow for comparisons of two versions of the FCU Online program (FCU Online + Coach, FCU Online-only) and the waitlist control condition.

A fundamental strength of the FCU is that it is scalable at multiple levels, depending on the barriers and resources available (family resources and school resources). For schools, barriers in staffing and coach support may prohibit use of the FCU Online + Coach, and these schools can use the FCU Online-only version, which requires limited staffing support to provide access to families. Barriers for families include time, transportation, and internet access. The FCU Online program can be delivered to families in their home and on their own schedule. The website is accessible by phone and can also be used “on the go.” Schools may also provide computer access to enable families to complete the program. This provides a high-reach, scalable approach to help families that is accessible to urban, suburban, and rural communities.

Another strength of this study is the use of multiple urban, suburban, and rural schools throughout the state of Oregon. This diverse population will allow us to examine rural versus urban participation and consumer satisfaction. We plan to implement the FCU Online program in 1 model school at the end of the study, which will provide additional information about dissemination.

Our development approach is also a strength of this study. Specifically, our use of an integrated technology architecture and a shared database facilitates data sharing and consistent programming procedures for the FCU Online website, its administration website, and its coach portal. Similarly, our use of an iterative formative research development process helped confirm program functionality and refine the program’s user experience design and user interface.

A potential limitation is the financial remuneration of the maximum amount of US $450 that participants are scheduled to receive in this efficacy trial for time and effort spent completing the assessments. We believe that this level of remuneration is equivalent across conditions and therefore it should not differentially affect groupwise outcomes. Moreover, it is contingent upon assessment completion rather than participation in the intervention. However, given the likelihood that the FCU Online program would not be implemented in the real world with such significant financial consequences, it will be important to assess the FCU Online program within a more practical context. In addition, previous research on the FCU has demonstrated prevention effects emerging over long-term, multiyear follow-up. As such, potential long-term prevention effects of the FCU may not be detected within the 1-year follow-up period of this study. Another possible limitation is that participants using the current version of FCU Online program must be proficient in English.

Programs such as the FCU Online represent an important next-generation direction in delivering behavioral health programs to parents and caretakers of school-aged children. Previous researchers and clinicians have been working to integrate evidence-based behavioral health prevention and intervention programs (also termed mental health programs) into schools for more than three decades (eg, [86,87]). Nearly all these attempts identified a consistent set of barriers when moving from “hothouse” efficacy demonstrations to the real-life frontiers of community educational settings and service providers. These barriers primarily include lack of resources (particularly in rural settings); stigma and parental resistance associated with behavioral health screening and diagnostic methods [88]; competing responsibilities of intervention staff; lack of support from school administrators and teachers, who often have no exposure to or training in behavioral health practices that are evidence based; difficulty in engaging families; and administrative and staff turnover, which creates a tremendous and ongoing staff training problem. The fact that many evidence-based practices are not flexible in terms of allowing shorter sessions and briefer interventions, and that most are developed for single issues (eg, anxiety, depression, or oppositional defiant disorder), makes it difficult to integrate various behavioral health programs [57,87,89], which creates additional barriers to uptake in real-life community settings.

In the context of all these barriers, schools are faced with increasing challenges, such as climbing rates of mental health issues, high rates of behavioral problems, children exposed to trauma, and school violence [90,91]. The FCU Online program has the potential to help address many of the aforementioned barriers that more traditional school-based behavioral health implementation strategies have yet to solve. For example, strengths of the FCU Online include its ability to be used within any behavioral health service delivery model or strategy (eg, school-only, school plus community behavioral health clinicians, and school-based health clinics), the ability of parents and clinicians to titrate (ie, use indicated modules or the entire program), and the ability to select families for intervention in a nonstigmatizing manner (ie, nonuse of Diagnostic and Statistical Manual of Mental Disorders diagnostic labels). The FCU includes a contextualized assessment of known family, child, and parenting constructs associated with behavioral health and educational outcomes (common elements approach); engagement of parents with multiple and parent-preferred levels as opposed to the more usual face-to-face–only strategies; presentation through multiple methods (eg, human video; video animations; text, graphs, and tools; exercises and forms; and additional literature) of evidence-based parenting methods and exemplars that are not confounded by clinician talent and training; and low response cost for schools to integrate at whatever level their desire or resources allow.

In addition, it remains for additional research to demonstrate the extent to which adding complementary program content (eg, stress management skills training, healthy eating) aimed at parents might enhance impact and whether benefits might accrue from adding online content for children. For example, embedding content from an evidenced-based curriculum, such as Coping Power, that includes both child and parent components into FCU Online might enhance outcomes over time [92]. Additional research on implementation and sustainability of effective interventions might explore whether targeted eHealth interventions for families, such as the FCU Online program, could be but one element of a multicomponent, school-based mental health program that provides access to engaging internet-based resources and tools [93,94]. Future analyses (following the examples set by the recent review by Finan et al [95] and the study by Heinrichs [96]) should also examine the possible impacts of different amounts of behavioral health prevention dollars used (1) to incentivize recruitment and assessment completion and (2) to encourage the practice of program strategies that might sustain treatment effects long after research payments are no longer an option.

Plans for future development of the FCU Online program include more-varied approaches and reporting to accommodate multiple parents and caretakers per child, multiple children within a participating family, expansion of program content for use by Spanish-speaking users, and expansion to health-related topics (eg, obesity management and prevention, and health promotion).