We are using 9 measures and 1 student questionnaire (Health Risk and Protective Factors) that were developed and employed in our prior implementation study (Table 1), with some modifications (pilot-tested in the Bahamas) based on validated scales from school implementation studies [39-41]. The new scales include assessments of teacher autonomy (5 items), perceived principal supportiveness (4 items), teacher self-efficacy (3 items), teacher attitudes toward sex education in schools (8 items), and teacher confidence (5 items). These scales are included in teachers’ measures of impression before and after teaching and will be further tested in the proposed study. These measures will be administered to the teachers before and after implementing FOYC+CImPACT to assess factors influencing fidelity of intervention implementation.

Every teacher will be assigned to a school coordinator whose responsibility will be to collect the forms and return them biweekly by email, mail, in person, or carrier to the central data system at the research office. The school coordinator will also be responsible for completing the school coordinator biweekly assessment, which contains information about all scheduled activities and measures for each teacher assigned to her and their completion or otherwise. It is important to have this information independently observed and recorded so that the MOE and research team are able to determine the status of implementation at all times and the data forms are continually tracked. Project manager will review this data biweekly.

In the first quarter of year 1, all members of the research team including the 45 school coordinators participated in a 1.5-day training workshop, beginning with an overview of the Fast Track initiative and the FOYC+CImPACT program, followed by roles, responsibilities, data flow, ethical issues, and practice with the forms. Procedure manuals were distributed. A 1-day refresher training will be conducted in subsequent years for new and existing personnel.

Based on the principles of communities of practice, the MOE created SAM, a 2-tiered group mentorship program, to deploy the strength of high-performing teachers to help teachers who are struggling.

An additional program, enhanced SAM, will be offered to teachers who still have difficulties in teaching sensitive topics. As part of enhanced SAM, these teachers will be observed in the classroom by the team leader who will provide onsite assistance. These strategies, consistent with the principles of communities of practice [29,30], have evolved as part of the Bahamian school system’s culture to support new/challenged teachers.

This trial examines the effect of each training/implementation component for at-risk and moderate-performing teachers and whether the presence or absence of a component has an impact on the performance of other components. An advantage of the multiphase optimization strategy (MOST) design is its ability to identify active components making significant contributions to the overall effect from those that are not doing so, hence not worth retaining. For our MOST trial, there are two intervention components, one with 2 levels (ie, BMF) and the other with 3 levels (ie, SAM), which corresponds to a full factorial with 6 experimental conditions (Table 2). We chose the teacher implementation checklist to identify high-performing teachers as a measure of success (optimization criteria). Using our 7-item preimplementation school screening tool [38], we identified teachers who are at risk for not implementing the FOYC intervention curriculum and moderate- and high-performing teachers. All at-risk teachers and moderate-performing teachers were invited to participate in a factorial experimentation, using a MOST trial design. Workshop/video was provided to all teachers and treated as a constant component in the experiment. At the beginning of the trial, at-risk and moderate-performing teachers were asked to attend a 2-day curriculum workshop and each received an educational video. Following the teacher training, teachers were randomly assigned to 1 of the 6 experimental conditions and asked to teach the intervention curriculum for 1 semester. The primary outcome is implementation fidelity as assessed through the teacher implementation checklist and observer form. This effectiveness information is used to decide which set of components to select for at-risk or moderate-performing teachers. By conducting the proposed optimization phase of the MOST trial in year 1, we are determining the role of BMF and SAM (including enhanced SAM) and/or their combinations with the training workshop in improving implementation fidelity. The optimized teacher training and implementation packages will be used to train all at-risk and moderate-performing teachers for the subsequent national implementation of FOYC+CImPACT (12 islands in years 2 to 5).

Each stage of implementation will be assessed and tracked via the 9 measures. National implementation by grade begins in year 2 with grade 6 (full program). This will be followed in subsequent years by the grade 7 (boosters) as well as continued grade 6 full program, etc, such that by year 5, all classes of grades 6 through 9 in the 115 government schools nationwide will be participating. National implementation will begin with the teacher workshops. After the workshops, teachers will begin the actual implementation in the classrooms, beginning with the FOYC sessions during regular HFLE class time and scheduling the CImPACT sessions at a time when parents can attend. Data regarding the teaching process will be measured, including teacher perceptions before and after teaching FOYC+CImPACT, teacher checklist of what was taught, and observation by the school coordinator. Teacher compliance and success in scheduling, conducting at least 85% of all activities, and completing all forms will be tracked in biweekly and programmic assessments by the school coordinator. The project manager will meet at least weekly with the national school coordinators to review and document on the programmatic assessments any changes made by MOE relevant to the implementation of FOYC+CImPACT. Three times per year, the implementation committee will review program progress. The research team will assemble, organize, and enter all of the data using the Autodata System’s Scannable Office.

To test the main effects and interactions among intervention components, we will use a standard analysis of variance (ANOVA) using effect coding (for two level: –1 = no and +1 = yes; for three level: –1 = no SAM, 0 = SAM, and +1 = enhanced SAM) rather than dummy coding (0, 1 or 0, 1, 2, etc). Effect coding has several advantages over dummy coding. First, effect coding produces estimates of main effects and interactions that are consistent with the classic definitions of ANOVA effects [42]. Second, effect coding preserves power for interactions [43].

We will make a preliminary selection of components that have achieved main effects (exceeding statistical significance or demonstrating medium-to-large effect size). This preliminary selection will then be reevaluated in light of any substantial interaction effects that have been detected to gain an understanding of how the components work in combination. Depending on the optimization criteria identified (implementing at least 85% of core activities), this would then be combined with other information (eg, cost, feasibility, scalability) to make a final selection of components [44]. This information will guide assembly of an optimized implementation package that achieves target outcomes with the least resource consumption and participant burden.

Percentages of core activities (30 core activities in FOYC and 5 core activities in CImPACT) completed by all grade 6 teachers will be computed to assess whether the teachers can implement the prevention program with fidelity (delivery of more than 85% of core activities). For grade 7 to 9 teachers, percentages of core activities in booster sessions completed with fidelity will be computed to assess implementation fidelity

To assess whether delivery is sustained over time, percentages of core activities completed with fidelity by grade 6 teachers among the next cohorts of grade 6 students (and among grades 7 to 9 teachers, percentage of core activities in the boosters among the next cohorts of students) will be computed and compared across the cohorts of student classes. Sustainability refers to teachers’ continued implementation of the intervention with fidelity (at least 50% of core activities) and adherence to program principles in 3 years (at 3 yearly follow-ups).

Bivariate analysis and mixed-effects modeling will be conducted to assess the relationship between implementation fidelity and student outcomes. In bivariate analysis, we will categorize the fidelity score into 3 levels: high fidelity (teachers complete more than 85% of core activities with fidelity), average fidelity (teachers complete 70% to 85% of core activities with fidelity), and low fidelity (teachers complete less than 70% of core activities with fidelity). The differences in grade 6 student outcomes at baseline and year-end follow-up across 3 levels of implementation fidelity and differences in the change scores between groups will be assessed using ANOVA (for HIV/AIDS knowledge, condom-use skills, perceptions) and Pearson chi-square tests (for self-reported behaviors). The test statistics (F score, chi-square) will be adjusted for the clustering effects of classroom and/or school using variance inflation factors.

The association of teacher implementation fidelity with student outcomes will be further examined using mixed-effects modeling (for knowledge, skills, and perceptions) and generalized linear mixed modeling (for self-reported behaviors), controlling for clustering effects of classroom/school, student age, sex, and baseline differences. Mixed-effects modeling analyses will be run using combined grade 6 teacher and grades 7 to 9 teacher fidelity score to assess overall effects of implementation fidelity of FOYC+CImPACT on student outcomes. Bivariate analysis and mixed-effects modeling will also be performed to assess the impact of different levels and types of curricular changes on student outcomes. To assess whether student improvements gained from grade 6 national implementation are sustained over time, a generalized estimating equation model will be used to examine the difference in student knowledge, skills, perceptions, and self-reported behaviors across the time points. Analyses will be performed using the SAS 9.4 (SAS Institute Inc) statistical software package.

Structural equation modeling analysis will be conducted to examine the relationships among factors influencing teacher fidelity of implementation and student outcomes using the Mplus 8 (Muthen & Muthen). We developed a hypothetical conceptual model (Figure 1) based on a synthesis of the empirical literature and our implementation work [21,45,46]. The model posits that teacher attitudes toward the intervention and self-efficacy in teaching the curriculum have a direct effect on fidelity of implementation, which will impact student outcomes. Ongoing interest and support from the school administration will reinforce teacher perception that this program is a high priority for their school administrators and indeed the nation. The identification of high-performing teachers as mentors will empower these teachers by increasing their sense of autonomy and role in the community. Given the importance of self-efficacy in implementation fidelity, components have been added to the implementation effort to increase real and perceived self-efficacy (workshop/video and site-based assistance and mentorship) [22,45]. Biweekly monitoring and feedback are hypothesized to increase teacher fidelity of implementation. The curriculum implementation committee is expected to improve teacher and school administrator attitudes toward the intervention. A starting model will be estimated to investigate the interrelationships among factors influencing fidelity of implementation. A full model will be constructed by including student outcome latent variables in the revised fidelity model, using the cluster option in Mplus. Standardized regression coefficients for all paths will be estimated using robust maximum likelihood estimation. A good model fit is indicated when the standardized root mean square residual and root mean square error of approximation are less than 0.05 and comparative fit index is greater than 0.95 [47,48].