This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
In Australia, secondary school educators are well positioned to recognize mental illness among students and provide support. However, many report that they lack the knowledge and confidence to do so, and few mental health training programs available for educators are evidence based. To address this gap, the Black Dog Institute (BDI) developed a web-based training program (Building Educators’ Skills in Adolescent Mental Health [BEAM]) that aims to improve mental health knowledge, confidence, and helping behaviors among secondary school educators in leadership positions. A pilot study of the training program found it to be positively associated with increased confidence and helping behaviors among educators and reduced personal psychological distress. An adequately powered randomized controlled trial (RCT) is needed.
The primary objective of this cluster RCT is to evaluate the effectiveness of the BEAM program for improving educators’ confidence in managing student mental health. The trial will also evaluate the effect of the BEAM program in increasing educators’ frequency of providing help to students and improving their mental health knowledge and reducing educators’ psychological distress and stigma toward students with mental health issues.
The target sample size is 234 educators from 47 secondary schools across New South Wales, Australia. Four waves of recruitment and enrollment into the trial are planned. Schools will participate in one wave only and will be randomized to the intervention or waitlist control conditions. Participants from the same school will be assigned to the same condition. Assessments will be conducted at baseline, posttest (10 weeks after baseline), and follow-up (22 weeks after baseline) using the BDI eHealth research platform. Intervention participants will receive access to the BEAM program for 10 weeks upon completion of baseline, and the control condition will receive access for 10 weeks upon completion of the follow-up assessment.
Recruitment for this trial began on July 21, 2020, with the first baseline assessments occurring on August 17, 2020. To date, 295 participants from 71 schools have completed baseline. Due to the unexpected success of recruitment in the first 3 waves, the final fourth wave has been abandoned. Intervention participants are currently receiving the program, with follow-up due for completion in March 2021.
This is one of the first RCTs to examine the effectiveness of a web-based adolescent mental health training program for Australian secondary school educators in leadership positions. If found to be effective, this training program will offer a sustainable and scalable delivery method for upskilling educators in caring for students’ mental health.
Australian New Zealand Clinical Trials Registry ACTRN12620000876998; https://covid-19.cochrane.org/studies/crs-14669208
DERR1-10.2196/25870
Over half of all mental illnesses experienced by adults begin before the age of 18 years [
Despite the recommendation by government bodies for greater training in adolescent mental health [
Although all 6 programs were shown to be effective for improving mental health knowledge, none have yet been shown to be effective in increasing helping behaviors, for example, recommending or referring a student to seek professional help, or in reducing the distress experienced by educators [
To fill this gap, the Black Dog Institute (BDI) developed the Building Educators’ Skills in Adolescent Mental Health (BEAM) program. BEAM is a web-based training program on adolescent mental health for secondary school educators in leadership positions, such as year advisors, heads of well-being, and principals
A recent pilot study examined the acceptability of the BEAM program among secondary school year advisors (N=71) from NSW, Australia. After using the program for 6 weeks, the year advisors reported significantly higher levels of self-reported confidence in their ability to care for students’ mental health and lower levels of personal psychological distress. Year advisors also reported an increased frequency of helping behaviors at the 19-week follow-up. However, many of the year advisors did not complete the entire program (59/70, 84%) reporting that the 6-week duration was insufficient, and barriers such as forgetfulness hampered their completion. In preparation for this trial, several modifications were made to the program to increase engagement and completion: program access was extended to 10 weeks, SMS reminders in addition to email reminders were embedded, sequential module completion was removed so that participants can complete the modules in any order, a
The primary objective of this trial is to evaluate the effectiveness of the BEAM program in improving secondary school educators’ confidence in recognizing and responding to their students’ mental health needs. The secondary objective is to assess BEAM’s effectiveness in increasing the frequency of help provided to students, improving educators’ mental health knowledge, reducing educators’ stigma toward others with mental illnesses, and reducing their own psychological distress.
The primary hypothesis is that educators who are allocated to receive the BEAM program will report significantly higher levels of confidence at posttest (primary endpoint) when compared with the waitlist control condition and that these effects will be sustained at follow-up. It is also hypothesized that educators who receive the BEAM program will report greater improvements in mental health knowledge, stigma, and psychological distress at posttest and a greater frequency of helping behaviors at follow-up, when compared with those in the control condition.
This study is a cluster randomized controlled effectiveness trial with 2 parallel conditions (the BEAM program and waitlist control), with measurements taken at baseline, posttest (10 weeks from baseline completion), and follow-up (22 weeks from baseline completion). This study protocol was approved by the primary ethics body of the University of New South Wales (UNSW) Human Research Ethics Committee (HREC; HC200257). Approval was also sought and obtained from the NSW Department of Education State Education Research Applications Process (SERAP2020222) to conduct research within government schools and from the Catholic Schools Office Dioceses of Maitland-Newcastle, Canberra-Goulburn (schools located in the Goulburn area only), and Wollongong to conduct research with schools located within their dioceses.
This research project is also guided by a trial management committee consisting of experts in research and trial design and service and program implementation to oversee and provide guidance on the study procedures. This committee meets bimonthly or more frequently on an as-needed basis.
To be eligible to participate, secondary school educators must be (1) employed in a school leadership position that includes responsibility for student well-being (eg, year advisors, directors or heads of student well-being, principals, directors of pastoral care, student coordinators, and heads of year); (2) currently working in this role at a government, Catholic, or independent secondary school in NSW, Australia, for the duration of the study; and (3) obtain their principal’s consent for their involvement.
Secondary school educators who participated in the pilot study are not eligible to participate in this trial.
The target sample size for this trial is 234 participants from at least 47 schools. This calculation is conservatively based on the participation of an average of 5 educators per school and an intraclass correlation of 0.07, which yields a design effect of 1.28. To detect a standardized effect size of 0.50, (a minimum of a moderate effect size is required to warrant future value and benefit of the program) with 80% power and α=.05 (2-tailed), an individually randomized trial would require 64 participants per arm. This number is inflated by the design effect to 82 to allow for clustering, with the recruitment increased to conservatively allow for attrition of up to 30%. This yields a minimum of 117 participants per arm.
Cluster randomization at the school level is used to avoid potential contamination and bias effects from other participants, reduce administrative tasks for schools, and enable implementation of the peer-to-peer component of the program. As such, all participants from the same school will be allocated to the same condition and complete the trial at the same time using randomly permuted block randomization with block sizes of 2 and 4. Schools will be assigned to either the intervention or control arm using a 1:1 allocation, stratified by school size (<400 or >400 students) and index of community socioeducational advantage (ICSEA) level (<1000 or >1000) as per a computer-generated randomization schedule. Randomization is conducted by a statistician not involved in the day-to-day running of the trial to avoid influence or bias. The research team will be aware of the allocation once registration is scheduled to begin because they are responsible for providing the link to participants for the intervention or control program study website. Participants will be unaware of which group they are allocated to during registration and baseline. Upon completion of baseline, they will be informed, via email, of their allocation. This is because the intervention participants will receive immediate instructions and access to the program, and the waitlist control participants will be asked to wait for their next survey.
A passive approach to recruitment is being undertaken using study advertisements. The advertisements are being placed on the BDI’s website and social media channels (Facebook, LinkedIn, and Twitter), within BDI newsletters, and circulated to BDI mailing lists and contacts. The study is also being advertised in the NSW Health School-Link e-newsletters (an NSW Health service that connects schools with local mental health services), Teacher Magazine, and Catholic Diocese bulletins. After viewing the study advertisements, interested educators are directed to a web-based expression of interest form on the BDI website, which collects their name, school, suburb, email, and role at their school. Once registered, prospective participants are encouraged to share the study information with their colleagues to increase cluster sizes and encourage peer-to-peer interaction. To promote a representative sample from metropolitan, regional, and rural locations, participation will be open to all government, Catholic, and independent schools within the Australian state of NSW for which we have ethical approval.
After expressing interest, the educators are emailed the study information. Interested participants consult their principal to obtain a signed letter of support. The school and its educators are randomized after the letter of support is received by the research team. Only 1 signed letter of support is required per school.
Educator consent is obtained online. Prospective participants provide consent by confirming the declaration statements in the web-based participant information sheet and consent form (PISCF). Participants can download the PISCF before providing consent, and they are emailed a copy for their records.
Participants can withdraw consent at any time without providing a reason by contacting the research team, replying to any email communication with the word
This trial will include 4 waves of registration and enrollment. A predetermined cut-off date indicates which wave a school participates in, as determined by the date on which the letter of support is received. Using waves ensures that data collection does not occur during summer school holidays, allows flexibility for when schools enroll in the trial based on their schedule, and ensures that participants from the same school commence the trial concurrently. Once all registered participants from a single school commence the baseline survey, no other prospective colleagues from that school can register. This prevents the influence that knowledge of group allocation might have on future participant enrollment. To avoid disappointment, educators are asked to tell their colleagues about the study at the time of recruitment.
Participants are sent an email directing them to the study registration website 1 week before the scheduled baseline start date. Here, they are asked to confirm their eligibility, register their personal details (including their name, school, and email), provide consent, and create a study account. They have the option to enter their mobile phone number to receive SMS notifications and reminders. Once completed, they await further instructions and access to the baseline survey.
On the day each wave is due to begin, participants receive an email (and optional SMS) inviting them to complete the baseline survey. The survey is accessible for 7 days, and participants who do not complete it are automatically withdrawn. This process is repeated for the 10-week and 22-week assessments. All participants receive 2 email reminders (and 2 optional SMS reminders) for the survey completions.
The BEAM program is a web-based training program accessible on any internet-enabled device. Each of the 5 modules consists of information, web-based interactive activities, and downloadable resources related to adolescent mental health. In this trial, participants can complete the 5 program modules in any order; however, an initial module is suggested to participants based on their response to the
This study uses a waitlist control condition. Participants in the control condition will receive access to the intervention at no cost immediately after they complete the follow-up survey (22 weeks post baseline). If they do not complete the follow-up survey, they will receive access immediately after the survey has closed. They will receive access to the full program whether they have not completed the survey or not.
Participants in both conditions will receive an Aus $15 (US $11.59) e-gift voucher to thank them for their time and completion of the posttest survey. They will also receive an Aus $15 (US $11.59) e-gift voucher after completing the follow-up survey. All e-gift cards will be emailed within 5 working days of a participant completing the survey and will be issued through GiftPay.
Schedule of outcome measures.
Measure | Data collection timepoint | ||
|
Baseline | Posttest (10 weeks) | Follow-up (22 weeks) |
Demographics and background | ✓a | —b | — |
Experience of mental health | ✓ | — | — |
Self-care | ✓ | ✓ | ✓ |
Experience in mental health training | ✓ | — | — |
School factors | ✓ | ✓ | ✓ |
Impact of COVID-19 on helping behaviors | ✓ | ✓ | ✓ |
Perceived mental health knowledge and awareness | ✓ | ✓ | ✓ |
Mental health knowledge | ✓ | ✓ | ✓ |
Stigma | ✓ | ✓ | ✓ |
Confidence | ✓ | ✓ | ✓ |
Helping behaviors | ✓ | ✓ | ✓ |
Psychological distress | ✓ | ✓ | ✓ |
Module recommendation question | ✓ | — | — |
Barriers to use | — | ✓ | — |
Program satisfaction | — | ✓ | — |
Process evaluation | — | ✓ | — |
Program impact on future behaviors | — | — | ✓ |
aIndicates the timepoint measure is administered.
bIndicates the measure is not administered at that timepoint.
The primary outcome for this trial is educators’ confidence in recognizing and responding to students’ mental health needs. This is measured using an adapted version of the confidence to recognize, refer, and support subscale from the study by Sebbens et al [
A modified version of the Help Provided to Students Questionnaire by Jorm et al [
This is assessed using the Perceived Knowledge and the Perceived Awareness subscales from the Mental Health Literacy and Capacity Survey for Educators [
For the awareness subscale, participants are asked to rate their level of perceived awareness on a set of 5 statements (eg,
This consists of 2 constructs: mental health literacy and the recognition of common mental illnesses. These 2 constructs are measured using an adapted version of the 12-item Mental Health Knowledge Schedule (MAKS; [
A modified version of the Personal Stigma subscale from the Depression Stigma Scale from Griffiths et al [
The Distress Questionnaire-5 (DQ-5) [
Participants are asked to provide their name, age, gender identity, whether they identify as Aboriginal or Torres Strait Islander, current role at their school, duration of employment at their current school (years), overall experience as an educator (years), and experience in their current role (years) at baseline.
Participants are asked to indicate whether they have had a personal, family, or close friend experience a mental illness (answered
Participants are also asked to rate how frequently they engage in self-care. This is to be answered on a 6-point scale (
Participants are asked to rate how important they believe mental health training is for educators (answered 0,
Participants are asked to indicate their school location (
In response to COVID-19, participants in this trial are asked questions regarding helping behaviors that may have changed due to the pandemic. The first 3 questions, “Have you reached out to students in a way that is different than you have done so in the past because of COVID-19? (eg, connecting via technology)?”, “Have you implemented a service, program, or educational information session about mental health?”, and “Are there any other ways you have responded to your students’ mental health that isn’t covered here?” will require a
The following measures will be obtained only from participants assigned to BEAM and will be used in subsequent research into factors that may moderate or mediate outcomes.
Program use will be measured by the number of completed lessons (maximum of 27), collected automatically by the eHealth research platform.
Program barriers will be identified using a 13-item list at posttest.
Participants will be asked to report if they experienced any of the listed barriers throughout the trial (eg,
Participants are asked to rate the extent to which they agreed with a set of 14 statements about the BEAM program (such as
Participants are asked whether they completed the peer coaching activities with a colleague (answered
Participants are asked to indicate whether the program content or resources were shared with other school staff and if any well-being programs had been implemented during the trial period (answered
The primary analysis will use a mixed model repeated measures analysis of variance (MMRM), accounting for repeated assessments within individuals and a random effect to account for clustering within schools. Models will include the factors time, condition (intervention vs control), and their interaction, with the critical test of effectiveness being planned contrasts of this interaction from baseline to postintervention (the trial primary endpoint) and follow-up (secondary endpoint). An unconstrained variance-covariance matrix will be used to accommodate within-participant effects. The method of Kenward and Roger [
Secondary and additional outcome analyses will involve contrasts comparing changes from baseline to follow-up analyses of secondary outcomes (mental health knowledge, stigma, helping behaviors, and psychological distress) from baseline to other occasions of measurement, using an MMRM approach, as described above. If the intervention is found to be effective, exploratory analyses will examine evidence for moderation effects, that is, whether the intervention was more effective for certain subgroups of the sample. This may include teacher attributes such as gender or age and school characteristics such as ICSEA status.
Approval was obtained from the primary ethics body (UNSW HREC) on April 21, 2020, SERAP on July 21, 2020, Maitland-Newcastle Catholic Diocese on May 27, 2020, Canberra-Goulburn Catholic Diocese on June 12, 2020, and Wollongong Catholic Diocese on August 4, 2020. Recruitment of educators started on July 21, 2020, and the baseline for waves 1, 2, and 3 are complete. To date, 465 educators have expressed interest in participating in the trial. In total, 308 educators have registered and 295 have completed baseline, representing 71 schools. Due to the unexpected success of recruitment in the first 3 waves, the decision has been made to not go ahead with the final fourth wave. Intervention participants are currently receiving the program with follow-up due for completion in March 2021. It is planned that the results will be presented at both national and international conferences and submitted to peer-reviewed journals. The results will also be disseminated to stakeholders through reports and presentations and on the BDI website.
This protocol describes the RCT of the BEAM program, a study that aims to evaluate the effectiveness of a new web-based mental health training program for secondary school educators in leadership positions. Through the provision of mental health information and interactive activities, the BEAM program aims to improve educators’ knowledge of adolescent mental health. It is anticipated that their confidence in managing their students’ needs and the frequency of help provided will thereby increase, whereas their stigma toward mental ill-health and their own levels of psychological distress will reduce.
There remains a significant lack of available evidence-based mental health training programs for educators [
If the effectiveness of the BEAM program is demonstrated through this trial, there are significant implications for how mental health training can be delivered to educators. For example, traditional educator training is typically delivered through face-to-face didactic-style workshops that require participants to take leave and have their classroom duties covered by another staff member [
There are limitations to this trial that are acknowledged, including the reliance on self-report questionnaires, which may be susceptible to biases. Self-report outcomes, such as confidence, may not match actual educator behavior; however, we will use validated measures where possible and assess teacher reports of actual helping behavior.
Another limitation is that the trial is being conducted among educators who self-selected to participate from within NSW only. The results may not be generalizable to interstate educators or educators who chose not to participate. Although Australia has a national education curriculum, schools are governed by each of the States’ and Territories’ Department of Education, and Catholic schools are governed by the Diocese to which they belong. Each has their own set of frameworks, policies, and rules that guide staffing. For example, there are structural differences between the states of South Australia (SA) and NSW, such that secondary school begins in year 8 in SA and year 7 in NSW. It may also be the case that participating educators are employed at better-resourced schools and have the time to take part. Furthermore, participation is limited to educators from schools for which we have approval from their governing ethical body and could obtain support from their school principal. Not all Catholic Dioceses in NSW granted approval to conduct this research, and gaining principal support might not have been possible for all interested educators.
A further limitation is the use of waitlist control and trial length. The total amount of time a control participant is required to wait without any access to the intervention is 22 weeks, which may affect attrition. We have conservatively estimated a 30% attrition in our target sample size, and a monetary reimbursement will be used to motivate completion of the posttest and follow-up surveys. The waitlist control also only enables the assessment of whether the intervention is more effective than the passage of time. Future follow-up studies to compare the intervention with active controls are needed once effectiveness relative to waitlist control is established.
Despite these limitations, this trial has several strengths, including the RCT design to examine the effectiveness of BEAM and the clustering and randomization at the school level to reduce the risk of bias and contamination. Once a school begins in the trial, no other educators from that school can enroll, ensuring that participants are not influenced by the participation of other staff members from their school. The standardized delivery of the intervention helps maximize the fidelity of the training being delivered to educators, ensuring that all participants receive the same standardized intervention. Other strengths include the suite of measures included to examine a range of outcomes; the long-term follow-up period; targeted recruiting approach to include educators from metropolitan, regional, and rural areas; exploration of possible moderators; and stratification of school variables to account for factors hypothesized to influence the results. If shown to be effective, the assessment of BEAM through an RCT will provide a novel method for delivering mental health training to secondary school educators.
Overview of the Building Educators’ Skills in Adolescent Mental Health Training program.
CONSORT (Consolidated Standards of Reporting Trials) flow diagram that will be used to outline participation through the Building Educators’ Skills in Adolescent Mental Health Training program for secondary school educators. NSW: New South Wales.
Black Dog Institute
Building Educators’ Skills in Adolescent Mental Health
Distress Questionnaire-5
Human Research Ethics Committee
index of community socioeducational advantage
Mental Health Knowledge Schedule
Mental Health First Aid
mixed model repeated measures analysis of variance
New South Wales
participant information sheet and consent form
randomized controlled trial
South Australia
State Education Research Applications Process
Teachers As Accompagnateurs
University of New South Wales
BP and CC conceived the study and the trial design. BP prepared the protocol, and both BP and CC initiated and coordinated the trial. AM contributed to the randomization, and PB and AM contributed to the statistical analyses. All authors contributed to the refinement of the protocol, and all authors read and approved the final manuscript. This project is funded by the Balnaves Foundation. PB is supported by National Health and Medical Research Council Fellowship 1159707.
BO is a section editor for JMIR Mental Health. No other authors have any conflicts of interest to declare.