This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.
Serious disruptive behavior among adolescents is a prevalent and often persistent problem. This highlights the importance of adequate and effective treatment to help adolescents with disruptive behavior problems react less hostile and aggressive. In order to create a treatment environment in which behavioral change can be enhanced, treatment motivation plays an essential role. Regarding treatment itself, a focus on challenging self-serving cognitive distortions in order to achieve behavioral change is important. Street Temptations (ST) is a new training program that was developed to address both treatment motivation and cognitive distortions in adolescents with disruptive behavior problems. One of the innovative aspects of ST is the use of virtual reality (VR) techniques to provide adolescents during treatment with visually presented daily social scenarios to activate emotional engagement and dysfunctional cognitions. By using the VR scenarios as an integral starting point of ST’s sessions and transferring the power of the VR experience into playful and dynamic exercises to practice social perspective–taking, adolescents are encouraged to reflect on both their own behavior and that of others. This focus on reflection is grounded in ST’s main treatment mechanism to influence treatment motivation and cognitive distortions, namely, mentalizing (ie, reflective functioning).
The aim of this study is to describe the research protocol to evaluate the effects of ST on treatment motivation and cognitive distortions. We take a closer look at the use of ST and the methodology used, namely, the repeated single-case experimental design (SCED).
The effects of ST are studied through a multiple baseline SCED, using both quantitative and qualitative data. In total, 18 adolescents from secure residential youth care facilities and secondary special education schools are randomly assigned to 1 of the 3 different baseline conditions. Throughout the baseline phase (1, 2, or 3 weeks), intervention phase (4 weeks), and follow-up phase (1, 2, or 3 weeks), daily measurements on treatment motivation and cognitive distortions are conducted. Secondary study parameters are assessed before baseline, after intervention, and after follow-up. Qualitative data are collected after intervention, as well as at 3 months and 6 months after the intervention.
Data collection for this study started in November 2021 and is planned to be completed by August 2023. The results will be published in peer-reviewed journals and presented at national and international conferences.
ST aims to improve the disruptive behavior problems of adolescents. This study will be the first to gain insights into the effectiveness of ST. The strengths of this study include its thorough and individually focused design (SCED), the focus on a residential as well as a secondary special education setting, and the ecological validity. The implications for practice are discussed.
Central Committee on Research Involving Human Subjects NL75545.029.20. Netherlands Trial Register NL9639; https://www.trialregister.nl/trial/9639
PRR1-10.2196/33555
Disruptive behavior of young children is among the most frequent reasons for referral to child and adolescent mental health care services worldwide [
Well-established, evidence-based treatments for adolescents with disruptive behavior problems, such as the Multisystemic Therapy and Treatment Foster Care Oregon, primarily focus on a small portion of adolescents with disruptive behavior problems, that is, adolescents with judicial involvement within the context of forensic youth care [
Treatment motivation is considered to be one of the preconditions for treatment to be effective [
The self-serving cognitive distortions mentioned above are associated with disruptive behavior problems [
According to Gibbs et al [
Taken together, adequate treatment options for adolescents with disruptive behavior problems are needed to prevent escalation of their problems. Treatment motivation is an important requisite to increase the chances of successful behavior change. In terms of content, emotionally engaging social perspective–taking opportunities can challenge self-serving cognitive distortions and in that way induce behavioral change.
Street Temptations (ST) is a new and innovative training program that was developed by Garage2020 in cocreation with Levvel, a secure residential facility and youth care provider in Amsterdam, The Netherlands, to influence treatment motivation as well as cognitive distortions of adolescents with disruptive behavior problems. In order to achieve this effect, ST’s exercises focus entirely on social perspective–taking opportunities. ST specifically aims to work with scenarios that are emotionally engaging, as this is the type of situation that should be focused on in treatment [
The term VR indicates a replacement of the physical environment by a 3D computer or an artificially generated interactive environment [
The focus on reflection is grounded in the assumably main therapeutic mechanism of ST, that is, mentalizing. The concept of mentalizing, operationalized as reflective functioning, refers to “the mental process by which an individual implicitly and explicitly interprets the actions of himself or herself and others as meaningful on the basis of intentional mental states such as personal desires, needs, feelings, beliefs, and reasons” [
Since ST is a newly developed program, so far, only test runs regarding the feasibility and potential of the program have been conducted [
This study applies a randomized, nonconcurrent, multiple baseline SCED across single participants [
Participants are randomized to a 1-, 2-, or 3-week baseline phase. Randomization to varying baseline periods enables us to determine whether change in measurements is exclusively related to the application of the intervention. The random assignment is similar to the way in which a random assignment is used in between-participants designs [
Overview of the study design with 3 different conditions. The daily repeated measure starts directly after T0, on the same day. Moreover, 3 and 6 months after T2, adolescents are approached again to participate in follow-up interviews. D: daily repeated measure; E: end of daily measure study period; I: first interview with adolescents and trainers; S: start of the study, application, informed consent, and eligibility check; T0: pretreatment assessment and randomization; T1: posttreatment assessment; T2: short-term follow-up assessment; wk: week.
Participants are recruited among adolescents from secure residential youth care facilities and secondary special education schools in the Netherlands. Both populations are characterized by serious externalizing problems. These problems are often accompanied by internalizing problems, sometimes in combination with psychiatric and addiction problems. Adolescents who meet the following criteria are eligible for inclusion: (1) aged between 12 and 18 years, (2) antisocial or externalizing behavioral problems, (3) deficits regarding cognitive distortions or treatment motivation, (4) presence or risk of delinquent behavior, (5) assigned to ST after multidisciplinary consultation, (6) expected stay of at least 2 months, and (7) basic understanding of mobile apps. A potential participant who meets any of the following criteria is excluded from participation: (1) severe physical impairment such as deafness and blindness, (2) severe psychiatric problems such as psychosis or high risk of suicide requiring immediate intervention, (3) trauma from serious violence, (4) epilepsy or serious problems regarding motion sickness, and (5) insufficient understanding of the spoken and written Dutch language.
According to SCED research standards, SCEDs need at least 3 attempts to demonstrate an intervention effect. Each of these attempts needs to be at a different time point, requiring a multiple baseline SCED to have at least 3 baseline conditions. Additionally, each phase must include a minimum of 3, preferably 5, data points to qualify as an attempt to demonstrate an effect [
Overview of the sample distribution.
Within the residential facilities, all referred adolescents are screened by the clinicians and ST trainers. Regarding the participating schools, adolescents from selected classes are screened during the first few weeks of the new academic year. This screening process is chosen to minimize the risk of nonresponse. Screening is only done by the professionals from the facilities and schools. All professionals involved are extensively briefed on the population the study focuses on. When an adolescent is thought to be eligible to participate in the study, the professional informs the adolescent about the study. The adolescent is shown a short video, in which the researcher briefly introduces herself and the project. Written information is provided as well. When the adolescent is interested to participate and gives oral permission to be approached, the researcher plans an informed consent appointment. Written informed consent is signed when the adolescent agrees to participate in the study. In the case of a minor, parents or a legal guardian signs a written informed consent as well. Adolescents that do not agree to participate in the study do not start with ST and receive treatment as usual.
After the informed consent procedure, the researchers decide whether a participant is eligible to take part based on the inclusion and exclusion criteria. When necessary, a clinician will be consulted to make an informed decision. After official enrollment, premeasurements are conducted, and randomization takes place. Because trainers, clinicians, and adolescents will notice in which condition adolescents are participating, allocation is not blinded. Appointments for the ST-intake session and following sessions are made according to the randomization. Directly after the premeasurements have been conducted, the daily measurements are set up by installing the data collection app (m-Path). Everything about the app and data collection is explained as well as tested with the adolescent. From then on, the app automatically sends out notifications for the daily assessments within set time frames. This continues until the last day of the follow-up phase, according to the randomization. During the baseline and intervention phase, the researcher, ST trainers, and participants are in touch on a regular basis. Together, they check how things are going and whether there are any particularities regarding ST, the daily measurements, or in general. The intensity (eg, frequency, duration) of these contact moments will be kept the same across participants as much as possible. If necessary, for example, to encourage participants to fill in their daily measurements, the researcher will be in touch more often. Contact moments will be registered per participant in order to take possible variations regarding contact moments into account when analyzing treatment effects.
ST trainers inform the researcher when the last ST session takes place. After the last session has taken place, 2 appointments are scheduled with the participating adolescent: 1 for the posttreatment assessment and 1 for the interview. This is done separately in order to reduce the burden on the adolescent. Additionally, 1 appointment is scheduled with the ST trainer for the interview with the trainer. After the last ST session, participants enter the follow-up phase. The researcher schedules the last appointment at the end of this phase to conduct the follow-up measurements and to close the study period together with the ST trainer and participant. At 3 months and 6 months after the end of the study period, the adolescents are approached again to participate in the 2 additional follow-up interviews. When a participant decides to leave the study prematurely, it is possible to finish the remaining ST sessions. The decision to do so or not will always be made in consultation with the trainer and clinician. Owing to the inevitable heterogeneity with regard to both the problems of the adolescents as well as the moment at which they participate in ST within their treatment process, no restrictions are imposed with regard to cointervention.
ST consists of seven 45-60-minute sessions. During the sessions, input and direction from the adolescents form the main lead regarding the exercises to offer adolescents a creative and alternative way to develop certain skills and to freely share their personal story. In this way, ST aims to add to more traditional modes that are not always tailored to the needs of adolescents [
By incorporating mentalizing as the main therapeutic mechanism, ST aims to influence both treatment motivation and cognitive distortions. In order to develop the needed motivation to engage in treatment for behavioral change, it is in the first place necessary to acknowledge problematic behavior and to seek help for this behavior [
The sessions are divided over 2 modules, A and B, which are executed in a fixed order. Before starting with the first module (ie, A), there is an intake session during which the adolescent chooses a personal learning goal. This goal focuses on mentalizing abilities, for example, “I want to learn that how I see a situation doesn’t have to be the same as how somebody else sees the same situation” or “I want to learn to listen to what somebody else thinks, feels, or would want to do in a situation so that I can better understand that person.” Each consecutive session ends with discussing the personal learning objective.
Module A revolves around 3 main characters. Each session starts with watching a 360° VR video. This video (see
Screen capture of the virtual reality video.
Examples of the cards used in the sessions.
In module B, the exercises revolve mainly around a personal experience chosen by the adolescent. Adolescents visualize the scenario of this situation for the trainer by using Street View VR, which means that the adolescent will use the VR glasses to virtually go to the place of their personal experience. While virtually being present in that place, the trainer watches along with the stream of the VR glasses and the adolescent explains what exactly happened in that place. Thereafter, the same perspective-switching exercises as described above are executed but with different perspectives. One session is about the perspective from an unknown passerby, and the other session is about the perspective from someone in the social network of the adolescent. In an additional exercise, a fictional character is created. This character is put in different situations based on the situation cards, and the adolescent has to make and substantiate a choice in that situation based on the character. As in module A, in module B, all exercises take place outside of the VR environment as well. The use of VR serves to present a scenario on which the exercises will be based. During the last session, the trainer and adolescent reflect on the progress that has been made regarding the personal learning goal. They also evaluate the program all together and what the adolescent has learned in addition to the set learning objective. It is possible to involve, for example, the mentor in this final session and have the adolescent explain what has been done and learned.
ST trainers receive a 2-day training course, provided by the first author and a highly experienced psychotherapist who is also a registered teacher and supervisor. The training focuses on the theoretical background of ST, working with VR, the ST protocol, and practicing the learned skills by participating in and reflecting on role plays with experience experts. In addition to the training, ST trainers are guided throughout the research period by participating in monthly supervision sessions. These sessions are also facilitated by the first author and the clinician from the ST training. Besides the supervision, trainers are encouraged to engage in peer consultation. Lastly, they are able to receive telephonic consultation by the first author or clinician on request. To gain insight into the extent to which trainers commit to the protocol, trainers are required to fill out session forms.
The main study parameters are assessed once a day in the format of an idiographic digital self-report questionnaire for the adolescents. The items are based on the questionnaires that are assessed at T0 and will be presented in a random order each day.
Treatment motivation is measured using the Dutch Adolescent Treatment Motivation Questionnaire (ATMQ) [
Cognitive distortions are assessed using the self-report How I Think questionnaire (HIT) [
The secondary study parameters include change in reflective functioning and social perspective–taking as well as a qualitative exploration of the overall experiences with regard to ST and VR.
Reflective functioning is measured using the Reflective Functioning Questionnaire for Youths (RFQY) [
Social perspective–taking is assessed using the Perspective Taking subscale of the Interpersonal Reactivity Index [
The ST evaluation is done by conducting semistructured interviews with adolescents as well as trainers based on the Change Interview [
The VR evaluation is done by adding questions regarding this topic to the above-described interviews. All respondents are asked to reflect on their experience with VR in general and working with the VR material, what they believe VR did or did not add to ST, and what they think of the video used in ST. Additionally, they are asked how they think the VR component could be improved.
Sociodemographic information such as age, sex, education level, ethnicity, living situation, and possible experience with minor criminal activity is collected using a demographic questionnaire developed by the researchers. Information regarding diagnostic background and treatment history is collected using file information. When recent IQ data are missing, the Screener for Intelligence and Learning Disabilities [
An overview of all the measurement tools and data collection moments is given in
Overview of the measurement tools and informants.
Variable | Measure | Informant | |
|
|||
|
Treatment motivation | ATMQa: Daily questionnaire | Adolescent |
|
Cognitive distortions | HITb: Daily questionnaire | Adolescent |
|
|||
|
Reflective functioning | RFQYc, SRIS-Yd | Adolescent |
|
Social perspective-taking | PTe-subscale | Adolescent |
|
STf evaluation | Semistructured interview | Adolescent, trainer |
|
VRg evaluation | Semistructured interview | Adolescent, trainer |
|
|||
|
Demographics | Questions | Adolescent, clinician |
|
Diagnostic and treatment history | File information | Clinician |
|
Intelligence | File information, SCILh | Clinician, adolescent |
aATMQ: Adolescent Treatment Motivation Questionnaire.
bHIT: How I Think questionnaire.
cRFQY: Reflective Functioning Questionnaire for Youths.
dSRIS-Y: Self-Reflection and Insight Scale for Youth.
ePT: Perspective Taking.
fST: Street Temptations.
gVR: virtual reality.
hSCIL: Screener for Intelligence and Learning Disabilities.
Overview of the data collection moments. ATMQ: Adolescent Treatment Motivation Questionnaire; HIT: How I Think questionnaire; IRI: Interpersonal Reactivity Index; PT: Perspective Taking; RFQY: Reflective Functioning Questionnaire for Youths; SRIS-Y: Self-Reflection and Insight Scale for Youth.
The primary outcome measures are the daily self-reported questions regarding cognitive distortions and treatment motivation. The resulting data will be presented as quantitative data. Within the context of SCED, including the multiple baseline design, the primary method for data evaluation regarding these repeated measurements is visual analyses [
Ethics approval for this study was obtained in June 2021, which was granted by the independent Medical Ethical Committee of Vrije Universiteit medical center (reference number: 2021.0114). This study will be conducted according to the principles of the World Medical Association Declaration of Helsinki [
Participant recruitment was started in November 2021. Data collection for this study is expected to be completed by August 2023. Analysis will be conducted after data collection has been completed. The results will be published in peer-reviewed journals and presented at national and international conferences.
In order to help adolescents with disruptive behavior problems develop and increase the necessary skills to react less hostile and aggressive, challenging self-serving cognitive distortions and enhancing motivation for behavioral change seem particularly important to focus on in treatment [
Our study has multiple strengths. First, by using a SCED instead of a more traditional group comparison design, a lot of individual information is collected throughout the entire treatment process while respecting each participants’ personal variability [
In addition to the strengths, our study also poses several potential challenges. First, ST is a newly developed intervention that has not been implemented yet. We are therefore dependent on the willingness of organizations to participate and the capacity available to carry out ST in addition to the standard care that is provided. Owing to the hectic work environment of both residential care and special education settings, it may be difficult for organizations to find the time and energy to participate. To increase our chances of success, we focus on the participation of multiple organizations and locations so that our dependency is not too vulnerable. Second, although we deliberately chose a design that requires a relatively small sample size, nonresponse and dropouts are still realistic challenges. We focus on a hard-to-reach sample, and data collection demands a lot from the participating adolescents. We have tried to reduce the required effort from participants by making the daily measurement as short and easy as possible. Additionally, personal reminders will be used when assessments are not completed, and we will be in touch with participants regularly in order to keep them motivated. Third, although they are validated measures, we only use self-report questionnaires regarding the quantitative measurements. This may cause social desirability bias as well as compromise validity. However, we do use a mixed methods approach as we combine our quantitative measures with qualitative data collection. This triangulation helps us to improve the interpretation of the results and decreases the deficiency of only using self-report [
ST is a new, innovative training program that specifically aims to meet the needs of adolescents by, among other things, integrating the potential of VR in the exercises. When the results are positive, ST can be further developed, implemented, and researched. In addition, when our described SCED proves to be viable for research in clinical practice, this will enhance the possibilities of clinical research. Adolescents with disruptive behavior problems usually form a hard-to-reach population, which is not easily captured in larger group designs such as randomized controlled trials. This often results in studies that are difficult to conduct, with high risks of, for example, not meeting the required sample size. This study may show alternatives for conducting good scientific research in hectic clinical environments. In this way, our study can provide both a contribution to science as well as to clinical practice.
To date, no research has been conducted into the effectiveness of ST. Our study will be the first to gain insights into the value of ST in helping adolescents with disruptive behavior problems react less hostile and aggressive. Based on the results, ST can be further developed. In addition, the foundation that will be laid with this study allows us to design follow-up studies, for example, to compare the effectiveness of ST with other treatments.
Adolescent Treatment Motivation Questionnaire
hostile attribution bias
How I Think questionnaire
Reflective Functioning Questionnaire for Youths
single-case experimental design
Self-Reflection and Insight Scale for Youth
Street Temptations
virtual reality
This study was funded by the Dutch Ministry of Health, Welfare, and Sport. The funding agency has no role in the design of the study.
REKS conceptualized and designed the final study protocol. AP was a major reviewer, and RJLL and LVD reviewed the final study protocol. REKS took the lead in writing the manuscript. All authors read, edited, and approved the final manuscript.
Garage2020, where REKS and LVD are affiliated with, developed Street Temptations and an educational program for professionals to learn about the scientific and practical background of Street Temptations.