Background: Parent training is recommended as the first-line treatment for attention-deficit/hyperactivity disorder (ADHD) in preschool children. The New Forest Parenting Programme (NFPP) is an evidence-based parenting program developed specifically to target preschool ADHD.
Objective: The objective of this trial is to investigate whether the NFPP can be effectively delivered for children referred through official community pathways in everyday clinical practice.
Methods: A multicenter randomized controlled parallel arm trial design is employed. There are two treatment arms, NFPP and treatment as usual. NFPP consists of eight individually delivered parenting sessions, where the child attends during three of the sessions. Outcomes are examined at three time points (T1, T2, T3): T1 (baseline), T2 (week 12, post intervention), and T3 (6 month follow/up). 140 children between the ages of 3-7, with a clinical diagnosis of ADHD, informed by the Development and Well Being Assessment, and recruited from three child and adolescent psychiatry departments in Denmark will take part. Randomization is on a 1:1 basis, stratified for age and gender.
Results: The primary endpoint is change in ADHD symptoms as measured by the Preschool ADHD-Rating Scale (ADHD-RS) by T2. Secondary outcome measures include: effects on this measure at T3 and T2 and T3 measures of teacher reported Preschool ADHD-RS scores, parent and teacher rated scores on the Strength & Difficulties Questionnaire, direct observation of ADHD behaviors during Child’s Solo Play, observation of parent-child interaction, parent sense of competence, and family stress. Results will be reported using the standards set out in the Consolidated Standards of Reporting Trials Statement for Randomized Controlled Trials of nonpharmacological treatments.
Conclusions: The trial will provide evidence as to whether NFPP is a more effective treatment for preschool ADHD than the treatment usually offered in everyday clinical practice.
Trial Registration: ClinicalTrials.gov NCT01684644; https://clinicaltrials.gov/ct2/show/NCT01684644?term= NCT01684644&rank=1 (Archived by WebCite at http://www.webcitation/6eOOAe8Qe)
Attention Deficit/Hyperactivity-Disorder Costs for Children
Attention deficit/hyperactivity-disorder (ADHD) is a neuro-developmental disorder with symptoms frequently occurring in early childhood, however, there is a parenting program that was developed specifically for the management of ADHD in preschool children, noted below [, ]. ADHD is one of the most common psychiatric disorders in childhood [ ] and the most common reason for referral to child mental health services (CAMHS) [ ]. Studies indicate that ADHD is as common in preschoolers as it is in school-age children, with population based prevalence rates ranging between 2 and 5% [ , ]. Impairment is equally common [ ], including marked impairments in relationships with parents, siblings, and peers [ - ]; social and preacademic skills [ ]; and neuropsychological functioning [ ]. Preschool children with ADHD are at greater risk of placement in special educational classes and use more special needs services [ ]. Young children with ADHD are more likely to suffer physical injury and accidental poisoning related to impulsive behaviors [ ]. ADHD symptoms in preschoolers show persistence over time [ , ]. Lahey et al [ ] found that 4-6 year old children who met full criteria for ADHD were highly likely to continue to meet criteria 3 years later. Preschool hyperactivity is associated with long-term economic burden. Evidence from a longitudinal health economic analysis of costs incurred across childhood, adolescence, and young adulthood found significantly higher costs for this group of individuals in areas of mental health, education, social service, and the criminal justice system [ ]. In a large register-based study, it was found that childhood ADHD, by the time the child is 10, reduces parental socioeconomic status by lowering labor supply and earnings, and reduces parental relationship stability [ ]. Left untreated, preschool ADHD is associated with long-term poor outcomes for the patient [ ]. Hence, it is of central public priority generally, and for CAMHS specifically, to offer evidence-based and cost-effective treatments at an early stage of development in an attempt to reduce the long-term risks and change the negative trajectories that ADHD presents to the individual and to society [ ].
Although medication is recommended as the front-line treatment for older children and adolescents with ADHD, international treatment guidelines do not recommend pharmacological therapies for preschool ADHD [, ]. Indeed, available data suggest that stimulant medication such as methylphenidate is less efficacious in the preschool years and side effects are more common [ ]. Long-term effects on growth and brain development are currently not definitively established. Because of these factors, psychosocial behavioral treatments for ADHD, including behavioral parent training and behaviorally based day-care interventions, are recommended as first-line treatments for ADHD in the preschool years [ ]. A number of psychosocial interventions have been applied and evaluated for the treatment of preschool ADHD. A recent systematic review found that behavioral parent training is valuable as a treatment for preschool ADHD [ ]. Furthermore, Sonuga-Barke et al [ ] found that trials evaluating behaviorally based parent interventions had the largest effects for preschool children with ADHD.
Whereas many behaviorally based interventions are generic in nature, in that they were originally developed to help parents manage preschoolers’ oppositional and challenging behavior, the New Forest Parenting Programme (NFPP)  is a parenting program that was developed specifically for the management of ADHD in preschool children. In addition to behavioral strategies, it includes ideas for games and activities aimed at targeting some of the self-regulatory and attention deficits that cause impairment in the condition. Supporting the child’s development through parental scaffolding is a key component of the NFPP, and everyday play scenarios in the home constitute opportunities for parents to train and improve the child’s ADHD symptoms. A single practitioner delivers NFPP to families in their home during eight weekly sessions.
Each session lasts approximately 90 minutes and is either for the parent only (five sessions) or the parent and child together. In an evaluation of the NFPP, Sonuga-Barke et al  found that the intervention reduced ADHD symptoms as reported by parents and improved behavior during a play observation task, as rated by blinded observers. Similarly, Thompson et al [ ] reported that NFPP improved parent-reported ADHD symptoms in a sample of preschool children with ADHD.
At present, the evidence of the effect of NFPP is based on efficacy research carried out by the expert team of researchers and clinicians who developed the program originally, with therapists receiving support from lead investigators, and with children who have not been referred for treatment through regular community pathways. Yet, it is unexplored if the findings from the original research can be replicated in different clinical and cultural contexts, including in a different language. This leaves open the question of whether effects shown in specialist research led services can be translated in the everyday practice of established clinical CAMHS . In a review of implementation studies [ ], it was shown that there is a risk that interventions will lose impact and potency when implemented in everyday clinical settings. Hence, it is emphasised that the transition of efficacious treatments has to be carefully planned and adapted to the clinical setting in which the treatment is being implemented [ , ].
Evidence-based parent training interventions specifically designed for preschool ADHD, such as the NFPP, are not currently provided by Danish CAMHS. The development and implementation of effective psychosocial treatments for young children with ADHD represent an important health policy objective. The trial described in this protocol is designed to evaluate the effectiveness of the NFPP in the treatment of young children diagnosed with ADHD referred through established clinical pathways to child psychiatry departments in Denmark. It is the first time that the NFPP has been tested in a European country outside the United Kingdom, and the first time the NFPP has been tested on children referred through established clinical pathways to tertiary child psychiatry services, to our knowledge. For these purposes, the NFPP program has been adjusted to fit the clinical and cultural context of Danish child and adolescent psychiatry services.
Aim of the Study
The objective of the trial is to examine whether the NFPP can be implemented effectively as a treatment for preschool ADHD in everyday clinical settings of existing services. To do this, we will examine ADHD and other outcomes at the end of treatment and also at a 6-month follow-up.
A randomized controlled trial will be carried out to investigate the effectiveness of the NFPP against a treatment as usual condition (TAU) in the treatment of ADHD in a clinical sample of young children referred to, assessed, and diagnosed at tertiary, preschool child psychiatry services in Denmark. The effectiveness of the NFPP intervention will be examined at three time points: T1 (baseline), T2 (week 12, post intervention), and T3 (6 month follow/up).
The study is approved by the Ethics Committee for Central Danish Region (No: 1-10-72-140-12), and is approved by the Danish Data Protection Agency. The trial is registered at ClinicalTrial.gov identity no: NCT01684644, September 4, 2012.
Participants will be enlisted from the regional, hospital-based child psychiatry departments for preschool children at the three participating centers in the trial: Centre for Child and Adolescent Psychiatry-Risskov; Centre for Child and Adolescent Psychiatry-Herning; and Centre for Child and Adolescent Psychiatry-Glostrup. The departments are highly specialized, tertiary CAMH services, and lie at the end of the referral pathway for young children with mental health problems in Denmark. Prior to their referral to specialist child psychiatry services, children will usually have received an initial assessment and intervention by primary care professionals, such as an early years educational psychologist, a community pediatrician, and social services.
Inclusion criteria are: (1) child age between 3-7 years old; (2) child must have received a clinical diagnosis of ADHD, as informed by the Development and Well-Being Assessment (DAWBA) ; and (3) Danish must be first language spoken in the home.
Child exclusion criteria are: (1) intellectual disabilities (IQ < 70); (2) diagnosis of autism spectrum disorders; and (3) child currently receiving pharmacological or other psychosocial treatment for ADHD.
Parent exclusion criteria are: (1) severe psychiatric disorder (eg, untreated psychosis or untreated bipolar or severe depressive disorder) and (2) severe social adversity in the home, as defined by active child protection issues.
A total of 140 children (age 3-7) referred through official referral routes to a general child psychiatry service for preschool children and their parents will be recruited into the trial. A clinical psychologist or a specialist in child and adolescent psychiatry initially screens all referrals for eligibility. Referrals indicating noneligibility, due to direct descriptions of exclusion criteria in the referral letter, receive standard clinical assessment from the preschool child psychiatry team. At this point, all other referrals are regarded as eligible and parents are sent a letter including a personal access code asking them to complete the Internet DAWBA  or the Preschool DAWBA (DAWBA-preschool for 2-4 year olds) [ ]. The DAWBA generates computerized diagnostic profiles, which are scored by child and adolescent psychiatrists and clinical psychologists trained in DAWBA rating. The DAWBA is employed at all three participating sites in an attempt to homogenize diagnostic practices and is used to compliment and inform the clinical, diagnostic assessment process.
In conjunction with the DAWBA, eligible children will undergo standard clinical assessment at the referral site. For referrals to preschool child psychiatry services in Denmark, this standard clinical assessment usually consists of: interview with parents regarding the child’s developmental history and current difficulties; child cognitive assessment and neuropsychological tests or neuropsychological tests; child medical examination; and, at Risskov and Herning, a semistructured observation of the child at the child’s day-care facility. Results from clinical assessments and results from the Internet DAWBA are presented and discussed at weekly multidisciplinary clinical team meetings where a specialist doctor in child and adolescent psychiatry issues final clinical diagnoses.
Parents of children meeting the research criteria are invited to join the study, and are included in the trial, once written consent is obtained.shows the recruitment flow according to Consolidated Standards of Reporting Trials (CONSORT) guidelines [ ].
Since outcome may be dependent on treatment center and regional variations in referral pathways, as well as gender and age of the child, the study uses stratified allocation to treatment condition to ensure a balanced number of participants over conditions. Randomization is based on the defined strata as well as on the assigned treatment condition of earlier randomized participants within the strata. Parents will be randomized to one of the two arms using an allocation ratio of 1:1 for NFPP and TAU. Randomization to NFPP and TAU (1:1) will be generated by a Web-based randomization computer program within the Internet data management service Trialpartner , which allows for on-the-spot randomization of participants into an arm of the study. Randomization is done in blocks of size four or six and in 12 strata defined by center, gender, and age of the child (3-5 and 6-7 years). Once the parent has provided written informed consent to participate in the study, the administrator completes participant registration on the Internet in Trialpartner and enters a randomization request, at which point the randomization program randomly assigns the treatment condition, for example, NFPP or TAU, to the respective participant ID number. Trialpartner simultaneously generates an automated email message for the administrator and the main investigator of the study, in which participant ID, site, and treatment allocation is outlined. Parents are informed of their respective treatment condition following completion of the T1 visit, so that the first assessment is not influenced by participant’s knowledge of treatment condition. Information about treatment allocation is delivered to parents in a telephone call by the researcher who initially informed the parent of the study.
All T1, T2, and T3 assessment schedules administered to parents and to day-care staff are set up electronically in Trialpartner  by the The Data Management Unit in The Central Denmark Region and are administered for completion on the Internet to parents and day-care staff by a research assistant. All data are stored and managed by The Data Management Unit in The Central Denmark Region. Data will remain locked until all participants have completed T3.
Research assistants carrying out research assessments and observations at the three time-points will be blinded. Parents are instructed not to reveal treatment allocation to research assistants. All raters of videotaped observations will be blinded. Research assistants and raters are postgraduate psychology students and newly qualified psychologists. Parents, NFPP therapists, NFPP supervisor, and TAU mental health professionals will not be blinded. Efforts will be made to keep teachers blind to treatment allocation. Yet full blinding cannot be guaranteed, as participants may spontaneously talk about treatment to teachers.
Sample Size Estimation
The primary endpoint is parent ADHD ratings at T2. Sample size has been estimated at 126 children to obtain power at 0.80, ES=.50 (Cohen’s d). The usual convention that is generally proposed and accepted for randomized trials is that power in a study ought to be 0.80 when ES is estimated at 0.50 . An estimated effect size of 0.5 is deemed reasonable for a trial evaluating the effect of parenting interventions in the treatment of ADHD. By comparison, a recent meta-analysis of parent training interventions for ADHD obtained an ES of 0.4 [ ].
For a design comparing two groups, for example, NFPP versus TAU, and allowing for a participant drop out of 10%, a total of 140 children, for example, a sample of 70 children per group is needed to attain a power of 0.80 . A drop out rate of 7% was reported in a Dutch parent training trial implemented in routine clinical CAMHS with a similar age group [ ]. In this light, the present trial has chosen to operate with a potential drop out rate of 10%. The preschool child psychiatry sites at Risskov and Herning are expected to enroll 2/3 of participants and the Glostrup site is expected to enroll 1/3 of participants for the trial.
Adapted New Forest Parenting Programme
The original NFPP  was developed to be delivered in the child’s home over the course of 8 sessions. However, the catchment areas of the three Danish hospital-based sites participating in this study cover large geographical areas. The distance between the furthermost points in the catchment area of Risskov, for example, is in excess of 150 km, which makes home-visits very time consuming and difficult to carry out to the degree outlined in the original NFPP protocol. Second, child psychiatric services in Denmark do not have a tradition of offering home-based assessment or treatment. As a rule, these are offered on an outpatient basis. To optimize the acceptability and deliverability of the NFPP in Danish CAMHS, the current study has made changes to the delivery of the original program and offers the majority of the 8 treatment sessions in the clinic. The third and the fifth treatment sessions, where the child is present, will be offered in the home. The other six sessions will be offered in the clinic in designated child and family friendly treatment rooms. The content of the NFPP manual remains otherwise unchanged.
The NFPP is centered around five broad themes .
- The importance of psychoeducation about the nature and manifestation of preschool ADHD in order for parents to understand reasons for the child’s behavior.
- Emphasis on scaffolding to help parents work from the child’s level of development and skills.
- The importance of parent-child interaction, and how it might be enhanced to support child development and reduce parental stress.
- Guiding parents in the use of behavioral strategies to improve behavior and ADHD symptoms.
- Training to improve child’s ADHD related, neuropsychological deficits, where parents are instructed to play with the child using attention training games, and helping the child manage delay, waiting, and self-regulation.
The NFPP manual, course materials, hand-outs, and a treatment DVD have been translated from English into Danish by a professional translation agency and A-ML with permission from the original developers . The original developers have approved all corrections or changes to original wording.
In this trial, families are also given basic play materials that are recommended in the manual for use in specific training tasks with the child, for example, two card games and a special relaxation mat for the child.
Treatment as Usual
TAU is likely to be different at the three sites, reflecting the different clinical practice and traditions in everyday outpatient child psychiatry in Denmark. In Risskov, TAU traditionally consists of three sessions of psychoeducation delivered in groups of parents to referred children diagnosed with ADHD. In Herning, TAU typically consists of one individual session with parents, following diagnosis, and one psychoeducation session in a group of parents. In Glostrup, it consists of four psychoeducation sessions in groups for parents. Children do not participate in the psychoeducation groups at any of these services. Clinical decisions may be taken in individual cases that give rise to intrasite variations in TAU. Based on the individual clinical need of the child and the parents, treatment may occasionally be offered by a mental health professional through 2-3 individual sessions to parents. TAU at all sites is carried out by existing clinic staff with many years of clinical experience in relation to preschool ADHD. The nature of treatment provided to parents in the TAU groups will be carefully recorded.
Fidelity to Treatment Manual
The NFPP intervention is manual based. All treatment sessions will be videotaped. To verify treatment fidelity based on the NFPP intervention manual, independent raters will code a number of treatment sessions per therapist.
New Forest Parenting Programme Training of Danish Therapists and Supervisor
During 2010 and 2011, the lead investigator A-ML participated during two NFPP training events held by the developers of the NFPP, for example, Dr Margaret Thompson (MT) and Ms Cathy Laver-Bradbury’s (CLB), University of Southampton in the United Kingdom. A-ML subsequently saw two young children diagnosed with ADHD and their parents from the preschool child psychiatry clinic in Risskov for 8 sessions NFPP under close supervision from MT and CLB. All treatment sessions were videotaped and translated in order for MT and CLB to supervise A-ML to achieve NFPP therapist certification. Following therapist certification, A-ML embarked upon NFPP supervisor certification and received biweekly supervision from MT and CLB in preparation to supervise Danish therapists employed in this trial.
A total of four therapists: a clinical psychologist and a pedagog from the preschool team at the Glostrup site and a clinical psychologist and a nurse specialist from the Risskov site are employed to train as NFPP therapists for the trial. They have all undergone a 4-day NFPP training course conducted by MT and CLB. Following training, all four therapists provided NFPP treatment to two families each, and received weekly supervision in relation to these families by A-ML. MT and CLB conducted a one-day top up training session after six months, in order to clarify specific questions in relation to the NFPP intervention. Therapists will receive 2-hour clinical supervision sessions in groups of two by A-ML throughout the trial. In turn, A-ML will receive biweekly supervisor supervision via Skype calls by MT and CLB to ensure adherence to the NFPP.
Assessment and Outcome Measures
Measures administered in this trial are listed in.
|Primary outcome measure|
|Preschool ADHD-RS (parent rated) ||T1, T2, T3|
|Secondary outcome measures|
|Preschool ADHD-RS (teacher rated) ||T1, T2, T3|
|Child Solo Play - observation measure||T1, T2, T3|
|Behavioral symptoms and impact|
|SDQaP2-4 & P4-16 - SDQaand impact supplement (parent rated) [, ]||T1, T2, T3|
|SDQaT2-4 & T4-16 - SDQaand impact|
supplement (teacher rated) [ , ]
|T1, T2, T3|
|The Adult ADHD Self-Report Scale (ASRS-V1.1) ||T1|
|PSOCb||T1, T2, T3|
|FSIc||T1, T2, T3|
|Positive and constructive parenting|
|GIPCIe(jigsaw/tidy up/freeplay) observation measure ||T1, T2, T3|
|Parent mental health|
|GHQd:12 ||T1, T2, T3|
a SDQ: Strength and Difficulties Questionnaire
bPSOC: Parenting Sense of Competence Scale
cFSI: Family Strain Index
d GHQ: General Health Questionnaire
e GIPCI: Global Impressions of Parent-Child Interactions
Primary Outcome Measure
Child Attention Deficit / Hyperactivity-Disorder: Attention Deficit / Hyperactivity-Disorder Rating Scale-IV–Preschool Version
The ADHD Rating Scale (ADHD RS)-IV-Preschool Version  is a questionnaire adapted for the use in preschoolers from the 18-item ADHD-RS-IV [ ]. In this preschool version, symptom statements have been modified to be more appropriate to the developmental level of preschoolers. The scale includes 18 behavioral descriptors of ADHD as determined by the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) [ ]. Raters can consist of parents and teachers who respond to each question on a 4-point scale from 0 (not at all) to 3 (very often) to reflect the child’s behavior. Scores can be obtained of Inattention, Hyperactive/Impulsive, and Total Scales. Internal consistency for the three scales has been established in the range from .92 to .95 on the Teacher Version and from .85 to .92 on the Parent Version. Test-retest reliability correlations for the three scales ranged from .93 to .96 for the Teacher Version and between .8 and .87 for the Parent Version [ ].
Attention Deficit / Hyperactivity-Disorder Rating Scale-IV-Preschool Version Translated
The ADHD RS-IV-Preschool Version has been translated into Danish for the purpose of the present study with permission from and formal approval by Kara McGoey (see) according to international guidelines for the translation of questionnaires [ ].
Secondary Outcome Measures
Child Solo Play
The Child Solo Play  instrument is an independent, direct observation measure administered by a research assistant during 5 minutes of solo play with a standard activity, multipurpose toy. Patterns of attending to and switching from one activity to another during play with the toy are measured and indexed in terms of attention and engagement. High index scores represent more attention and less switching. Good test-retest reliability (Pearson r=.81), interrater reliability (Pearson r=.76), and validity—discriminating children with ADHD from non-ADHD children—have been established. Research assistants, for example, postgraduate psychology students and newly qualified psychologists, employed in the trial are trained in the administration and scoring of Child’s Solo Play by the original developer of the measure and coauthor on this paper, DD.
Parenting Sense of Competence Scale The Parenting Sense of Competence Scale (PSOC)  is the most frequently used questionnaire applied to measure parental self-efficacy and satisfaction [ ]. It measures parental competence on two dimensions: Satisfaction and Efficacy. Separate scales are developed for mothers and fathers, who respond to 16 questions on a 6-point Likert-scale (ranging from strongly agree to strongly disagree) [ , ]. There are 9 questions that are related to Satisfaction and seven are related to Efficacy. The Satisfaction section examines parents’ anxiety, motivation, and frustration, while the Efficacy section looks at parents’ competence, capability levels, and problem-solving abilities in their parental role. A number of studies have demonstrated the scale’s good psychometric properties reporting internal consistency of .75 [ ], and internal consistencies of .72 (mothers) and .76 (fathers) [ ]. Charlotte Johnston gave permission and approval for the PSOC to be translated for the purpose of the present study (see ) according to international guidelines for the translation of questionnaires [ ].
Strength and Difficulties Questionnaire Parent and Teacher Version
The Strength and Difficulties Questionnaire (SDQ)  is a brief behavioral five factor instrument developed to assess emotional and behavioral problems in children. The SDQ consists of 25 questions scored on a 3 point Likert scale (“not true”, “somewhat true”, “certainly true”). The questions cover five domains of child psychopathology by five subscales: hyperactivity/inattention, conduct problems, emotional symptoms, peer relationship problems, and prosocial behavior. Each area is covered by five questions. In addition, the SDQ contains an impact supplement inquiring about distress, burden, and impairment of the child. The questionnaire is widely used for clinical as well as research [ ]. The standard SDQ is developed for use in the age-range of 4-17 years. The early years SDQ is developed for use in the age range of 2-4 years. A teacher version is developed for each scale, and a scale for completion by the young person is developed for use in the age range 11-17.
The SDQ has been used extensively in European as well as non-European contexts, and has been translated into more than 60 languages . A recent review of the psychometric properties of the parent and teacher versions of the SDQ included 48 studies from 17 different cultural settings and a total of 131,223 raters [ ]. The internal reliability and factor structure of the Danish versions of the SDQ were established in a sample of 71,840 parent and teacher raters of 5-, 7-, and 10- to 12-year-old children included in four large-scale Danish cohorts [ ]. The predictive validity of the SDQ has been demonstrated in another large Danish study where the SDQ was shown to identify a group of children with highly increased risk of later being diagnosed and treated for ADHD or treated for ADHD in school age [ ].
The early years SDQ—parent and teacher version Goodman —has been translated into Danish for the purpose of the present study with personal permission and approval by Robert Goodman (see ) following standard guidelines for the translation of questionnaires [ ]. The early years SDQ has shown good psychometric properties in a recent study [ ].
Family Strain Index The Family Strain Index (FSI)  is a 6-item parent-report questionnaire developed to assess the effects of ADHD on families to better understand and address the level of stress, strain, and burden that families experience. The FSI is designed to measure two major aspects of stress and demand, for example, the “emotional” and the “restrictiveness” experiences in the context of living with a child with ADHD. The FSI has demonstrated good internal consistency across the six items (Cronbach alpha= .83 to .87) and for the full scale (Cronbach alpha=.87) [ ].
The FSI has been translated into Danish for the purpose of the present study (see) following standard guidelines for the translation of questionnaires [ ] with permission from and approval by the original developer Anne Riley.
Global Impressions of Parent-Child Interactions
The Global Impressions of Parent-Child Interactions (GIPCI-R)  is a direct, semistructured observation schedule developed to evaluate parent and child interaction. Parent-Child dyads are videotaped in 15 minutes sessions during three tasks, each of 5 minutes duration: “jigsaw”, “free play”, and “tidy-up”. Videotapes are rated using the GIPCI coding manual at a later stage by a rater who is blinded to the randomization of the child. Child behavior items include: respect, disruptive, social skills, and destruction. Parents items rated include: responsiveness, warmth, praise, enjoyment, scaffolding, criticism, and punishment. Ratings produce global summary scores for child and parent with a higher score reflecting a more positive outcome. Thompson et al [ ] found adequate interrater reliability for child scores (.62: range .48-.77) and for parent scores (.64: range .48 to .79). Good internal consistency was established for parent and child scales (.84; .87). Training in the administration and scoring is carried out by one of the authors of this paper, DD.
The Adult Attention Deficit/Hyperactivity-Disorder Self-Report Scale
The Adult ADHD Self-Report Scale (ADRS-v1.1) is an 18-item Symptom Checklist covering the 18 DSM-IV-TR criteria for ADHD. The scale has been developed in conjunction with the World Health Organization (WHO), and the Workgroup on Adult ADHD . The first 6 questions have been found most sensitive in the screening for ADHD in adults [ ]. The 18-item scale has been translated into Danish and has been approved by the original investigators of the WHO Work Group. The scale provides a method of identifying ADHD symptoms in adults and is a powerful tool to discriminate DSM-IV cases from noncases [ ]. The scale has good psychometric properties and is widely used in scientific research [ ]. The validity and clinical feasibility of the Danish version of the ADHD-RS has been demonstrated in a multicenter study [ ]. The measure will be used to evaluate the extent to which parental ADHD symptoms are associated to outcome [ ].
The General Health Questionnaire
The General Health Questionnaire (GHQ)  is the most common assessment measure for assessing of mental well-being in adults. Developed as a screening tool to detect those likely to have or be at risk of developing psychiatric disorders, it is a measure of the common mental health problems/domains of depression, anxiety, somatic symptoms, and social withdrawal. It is available in a variety of versions using 12, 28, 30, or 60 items, which have all been translated into Danish. The present study will use the 12-item version GHQ:12. The measure will be used to evaluate the extent to which parental mental health is associated to outcome [ ].
The data will be analyzed on an intention to treat basis. All outcomes will be analyzed with a repeated measure model with a randomization arm, gender, age (above or below 5 year), center, year, and a random level for each child as covariates. We will estimate the effect of the intervention at T2 (intervention, TAU) in terms of the difference between the change from T1-T2 and its effect at T3 in terms of change between the T3-T1. Missing data will be imputed assuming Missingness At Random in relevant analysis models including additional control variables if relevant. Outcomes with more than fifty percent missing will not be analyzed. To examine the effect of outliers, a sensitivity analysis will be performed excluding observation with residuals exceeding 2.5*SD. In addition, four other sensitivity analyses will be performed for each analysis exploring how sensitive the results are for reasonable deviations from the Missing At Random assumption. The missing data will be predicted based on the analysis of the observed data. In the first sensitivity analysis, we will reanalyze the data including the predicted value for the missing values, but add .2*SD to the missing data in the intervention group only. In the second sensitivity analysis, we will subtract .2*SD from the missing data in the intervention group only. In the third, we add .2*SD to the missing data in the TAU group only, and, in the last, subtract .2*SD from the missing data in the TAU group only.
The results for the primary outcome as a supplement will be split according to gender, single parenting (yes/no), oppositional defiant disorder/conduct disorder-comorbidity, and maternal ADHD symptoms. A number of scientific articles will be generated on the basis of collected data from this trial, including publications outlining moderator and mediator analyses. Results will be reported according to the CONSORT statement for nonpharmacological interventions .
Time-Line Funding for this trial was granted in 2011 and in 2014. Enrollment of participants started in August, 2012, and the data collection is expected to be finalized at the end of December, 2015. The primary outcome paper will be submitted for publication mid-year 2016.
Attention Deficit/Hyperactivity-Disorder in Children
This trial addresses the need to investigate the effect of nonpharmacological treatments for young children with ADHD in everyday clinical practice. It is the first trial of the NFPP to be based on the recruitment of children referred directly to a child psychiatry department through official community pathways, to our knowledge. In this sense, this randomized trial tests the effectiveness of the NFPP with the children, families, and clinicians in the clinical setting for which the intervention is ultimately intended, and compares directly to the usual clinical care on offer for young children diagnosed with ADHD. The trial is also the first evaluation of the NFPP outside an English speaking context, to our knowledge.
Given the controversies surrounding the ADHD diagnoses in young children, and the challenge that confronts assessment of preschool children with ADHD symptoms, diagnostic procedures require particular attention . This trial is the first to include young children with a formal clinical diagnosis of ADHD following standard clinical and DAWBA assessment, to our knowledge. The clinical assessment ensures thorough medical, psychological, and psychosocial evaluation of the child and the child’s environment, and the adjunct systematic DAWBA evaluation ensures standardization in the assessment procedure for children entering the trial.
Reducing Chance of Bias
To reduce the chance of bias, the trial includes measures involving direct observation of children and of parenting by blinded raters as well as ratings by naïve informants (ie, day-care staff) along with parent ratings. This will allow for the investigation of rater effect on child outcome, and facilitate the exploration of whether child outcome is stable across settings.
Identifying efficacious treatments for young children with ADHD is important, but finding evidence from trials in settings created for research (eg, university labs, specialist clinics) or with participants recruited through advertisements does not guarantee that a treatment will work well in everyday clinical practice, across different populations, clinical contexts, cultures, and countries . The current trial has carefully adapted the NFPP to fit clinical and cultural needs and commenced an evaluation of its effectiveness compared to the usual treatment interventions offered to young children with ADHD and their families referred through official community pathways. To these ends, the trial will provide important information about the effectiveness and replicability of the NFPP in the treatment of young children with ADHD in an everyday CAMHS setting. This level of investigation is needed to shrink the gap between intervention research and clinical practice [ ].
ADHD is a disorder that presents with major personal and social costs. The immediate and long-term impact and adverse outcomes of ADHD for individuals and their families are considerable . The benefits of early intervention for ADHD is an area of current scientific and public health focus, but is not fully documented [ ]. It is therefore important that carefully implemented effectiveness trials of evidence-based treatments for young children are translated and implemented into different cultures with a view to enhance outcome for a broader population of children with ADHD and their families.
This trial is based upon work supported by the Danish foundation TrygFonden. Acknowledgments are extended to the Danish foundation Helsefonden and the Central Denmark Region Foundation for Psychiatric Research for financial support in the preparation and implementation of this trial. We thank the Central Management at the Centre of Child & Adolescent Psychiatry in the Central Denmark Region and the Capital Denmark Region, respectively, for persistently supporting the implementation of the trial in everyday clinical practice. We gratefully acknowledge the clinical advice and commitment of MT and CLB, Southampton University, and their assistance in early discussions regarding the trial. We also acknowledge the hard work and dedication by Rikke Lambek, Aarhus University, and Kirsten Krabek Frantzen, Aarhus University Hospital, in the translation of measures for the trial. We thank original authors Robert Goodman, Kings College London; Kara McGoey, Duquesne University; George DuPaul, Lehigh University; Charlotte Johnston, University of British Columbia, and Anne Riley, Johns Hopkins University, for their permission to translate outcome measures and for their efforts in approving translations and back translations of their respective measures.
Conflicts of Interest
ES-B and DD are the original codevelopers of the NFPP and benefit from royalties from the self-help version of the program.
Multimedia Appendix 1
Adfærd.PDF File (Adobe PDF File), 93KB
Multimedia Appendix 2
Spørgeskema om barnets styrker og vanskeligheder (SDQ skema SMÅBØRN, til forælder).PDF File (Adobe PDF File), 116KB
Multimedia Appendix 3
Family strain.PDF File (Adobe PDF File), 51KB
Multimedia Appendix 4
Parenting Sense of Competence Scale, PSOC, Charlotte Johnston, University of British Columbia.PDF File (Adobe PDF File), 100KB
- Sonuga-Barke EJ, Daley D, Thompson M, Laver-Bradbury C, Weeks A. Parent-based therapies for preschool attention-deficit/hyperactivity disorder: A randomized, controlled trial with a community sample. J Am Acad Child Adolesc Psychiatry 2001 Apr;40(4):402-408. [CrossRef] [Medline]
- Sonuga-Barke Edmund J S, Halperin JM. Developmental phenotypes and causal pathways in attention deficit/hyperactivity disorder: Potential targets for early intervention? J Child Psychol Psychiatry 2010 Apr;51(4):368-389. [CrossRef] [Medline]
- Wichstrøm L, Berg-Nielsen TS, Angold A, Egger HL, Solheim E, Sveen TH. Prevalence of psychiatric disorders in preschoolers. J Child Psychol Psychiatry 2012 Jun;53(6):695-705. [CrossRef] [Medline]
- Goodman R, Ford T, Richards H, Gatward R, Meltzer H. The development and well-being assessment: Description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry 2000 Jul;41(5):645-655. [Medline]
- Lavigne JV, Gibbons RD, Christoffel KK, Arend R, Rosenbaum D, Binns H, et al. Prevalence rates and correlates of psychiatric disorders among preschool children. J Am Acad Child Adolesc Psychiatry 1996 Feb;35(2):204-214. [CrossRef] [Medline]
- Lavigne JV, Lebailly SA, Hopkins J, Gouze KR, Binns HJ. The prevalence of ADHD, ODD, depression, and anxiety in a community sample of 4-year-olds. J Clin Child Adolesc Psychol 2009 May;38(3):315-328. [CrossRef] [Medline]
- Greenhill LL, Posner K, Vaughan BS, Kratochvil CJ. Attention deficit hyperactivity disorder in preschool children. Child Adolesc Psychiatr Clin N Am 2008 Apr;17(2):347-366, ix. [CrossRef] [Medline]
- Pelham WE, Milich R. Peer relations in children with hyperactivity/attention deficit disorder. J Learn Disabil 1984 Nov;17(9):560-567. [Medline]
- Wehmeier PM, Schacht A, Barkley RA. Social and emotional impairment in children and adolescents with ADHD and the impact on quality of life. J Adolesc Health 2010 Mar;46(3):209-217. [CrossRef] [Medline]
- DuPaul GJ, McGoey KE, Eckert TL, VanBrakle J. Preschool children with attention-deficit/hyperactivity disorder: Impairments in behavioral, social, and school functioning. J Am Acad Child Adolesc Psychiatry 2001 May;40(5):508-515. [CrossRef] [Medline]
- Gardner DM, Gerdes AC. A review of peer relationships and friendships in youth with ADHD. J Atten Disord 2015 Oct;19(10):844-855. [CrossRef] [Medline]
- Daley D, Birchwood J. ADHD and academic performance: Why does ADHD impact on academic performance and what can be done to support ADHD children in the classroom? Child Care Health Dev 2010 Jul;36(4):455-464. [CrossRef] [Medline]
- Sonuga-Barke Edmund J S, Dalen L, Daley D, Remington B. Are planning, working memory, and inhibition associated with individual differences in preschool ADHD symptoms? Dev Neuropsychol 2002;21(3):255-272. [CrossRef] [Medline]
- DuPaul GJ, Gormley MJ, Laracy SD. School-based interventions for elementary school students with ADHD. Child Adolesc Psychiatr Clin N Am 2014 Oct;23(4):687-697. [CrossRef] [Medline]
- Lee SS, Lahey BB, Owens EB, Hinshaw SP. Few preschool boys and girls with ADHD are well-adjusted during adolescence. J Abnorm Child Psychol 2008 Apr;36(3):373-383. [CrossRef] [Medline]
- Biederman J, Petty CR, Evans M, Small J, Faraone SV. How persistent is ADHD? A controlled 10-year follow-up study of boys with ADHD. Psychiatry Res 2010 May 30;177(3):299-304 [FREE Full text] [CrossRef] [Medline]
- Lahey BB, Pelham WE, Loney J, Kipp H, Ehrhardt A, Lee SS, et al. Three-year predictive validity of DSM-IV attention deficit hyperactivity disorder in children diagnosed at 4-6 years of age. Am J Psychiatry 2004 Nov;161(11):2014-2020. [CrossRef] [Medline]
- Chorozoglou M, Smith E, Koerting J, Thompson MJ, Sayal K, Sonuga-Barke Edmund J S. Preschool hyperactivity is associated with long-term economic burden: Evidence from a longitudinal health economic analysis of costs incurred across childhood, adolescence and young adulthood. J Child Psychol Psychiatry 2015 Sep;56(9):966-975 [FREE Full text] [CrossRef] [Medline]
- Kvist AP, Nielsen HS, Simonsen M. The importance of children's ADHD for parents' relationship stability and labor supply. Soc Sci Med 2013 Jul;88:30-38. [CrossRef] [Medline]
- Uchida M, Spencer TJ, Faraone SV, Biederman J. Adult outcome of ADHD: An overview of results from the MGH longitudinal family studies of pediatrically and psychiatrically referred youth with and without ADHD of both sexes. J Atten Disord 2015 Sep 22. [CrossRef] [Medline]
- Subcommittee on Attention-Deficit/Hyperactivity Disorder, Steering Committee on Quality Improvement and Management, Wolraich M, Brown L, Brown RT, DuPaul G, et al. ADHD: Clinical practice guideline for the diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Pediatrics 2011 Nov;128(5):1007-1022 [FREE Full text] [CrossRef] [Medline]
- Kendall T, Taylor E, Perez A, Taylor C, Guideline DG. Diagnosis and management of attention-deficit/hyperactivity disorder in children, young people, and adults: Summary of NICE guidance. BMJ 2008;337:a1239. [Medline]
- Greenhill L, Kollins S, Abikoff H, McCracken J, Riddle M, Swanson J, et al. Efficacy and safety of immediate-release methylphenidate treatment for preschoolers with ADHD. J Am Acad Child Adolesc Psychiatry 2006 Nov;45(11):1284-1293. [CrossRef] [Medline]
- Charach A, Carson P, Fox S, Ali MU, Beckett J, Lim CG. Interventions for preschool children at high risk for ADHD: A comparative effectiveness review. Pediatrics 2013 May;131(5):e1584-e1604 [FREE Full text] [CrossRef] [Medline]
- Sonuga-Barke Edmund J S, Brandeis D, Cortese S, Daley D, Ferrin M, Holtmann M, van der Oord Saskia, Wong Ian C K, European ADHD Guidelines Group. Nonpharmacological interventions for ADHD: Systematic review and meta-analyses of randomized controlled trials of dietary and psychological treatments. Am J Psychiatry 2013 Mar;170(3):275-289. [CrossRef] [Medline]
- Thompson Margaret J J, Laver-Bradbury C, Ayres M, Le PE, Mead S, Dodds C, Sonuga-Barke Edmund J S. A small-scale randomized controlled trial of the revised new forest parenting programme for preschoolers with attention deficit hyperactivity disorder. Eur Child Adolesc Psychiatry 2009 Oct;18(10):605-616. [CrossRef] [Medline]
- Santucci LC, Thomassin K, Petrovic L, Weisz JR. Building evidence-based interventions for the youth, providers, and contexts of real-world mental-health care. Child Dev Perspect 2015 Apr 11;9(2):67-73 [FREE Full text] [CrossRef]
- Barwick M, Boydell K, Stasiulis E, Ferguson H, Blase K, Fixsen D. Ontario Canada: Children's Mental Health Ontario. Knowledge transfer and implementation of evidence-based practices in childrens mental health URL: http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/CMHO-KTAndImplementationOfEBPInChildrensMentalHealth-2005.pdf [accessed 2016-01-09] [WebCite Cache]
- Weisz JR, Gray JS. Evidence-based psychotherapy for children and adolescents: Data from the present and a model for the future. Child Adolesc Ment Health 2008 May;13(2):54-65. [CrossRef]
- Goodman R. Report. London Assessing preschool childrens emotional and behavioural adjustment URL: http://www.dawba.info/EarlyYears/EarlyYearsDAWBAReport9jan11.pdf [accessed 2016-01-09] [WebCite Cache]
- Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration. Ann Intern Med 2008 Feb 19;148(4):295-309. [Medline]
- Datamanagement U. https://trialpartner.dk. Central Denmark Region, Oluf Palmes Allé 15, 8200 Aarhus N, Denmark: Datamanagement Unit; 2012. Trialpartner URL: http://www.webcitation.org/6eUITZ1GZ [accessed 2016-01-12] [WebCite Cache]
- Kazdin A. Research design in clinical psychology: Pearson New International Edition. Harlow, Essex: Pearson Education, Limited; 2013:9781292042787.
- Coates J, Taylor JA, Sayal K. Parenting interventions for ADHD: A systematic literature review and meta-analysis. J Atten Disord 2015 Oct;19(10):831-843. [CrossRef] [Medline]
- Cohen J. A power primer. Psychol Bull 1992 Jul;112(1):155-159. [Medline]
- van den Hoofdakker Barbara J, van der Veen-Mulders Lianne, Sytema S, Emmelkamp Paul M G, Minderaa RB, Nauta MH. Effectiveness of behavioral parent training for children with ADHD in routine clinical practice: A randomized controlled study. J Am Acad Child Adolesc Psychiatry 2007 Oct;46(10):1263-1271. [CrossRef] [Medline]
- McCann DC, Thompson M, Daley D, Barton J, Laver-Bradbury C, Hutchings J, et al. Study protocol for a randomized controlled trial comparing the efficacy of a specialist and a generic parenting programme for the treatment of preschool ADHD. Trials 2014;15:142 [FREE Full text] [CrossRef] [Medline]
- McGoey KE, DuPaul GJ, Haley E, Shelton TL. Parent and teacher ratings of attention-deficit/hyperactivity disorder in preschool: The ADHD rating scale-IV preschool version. J Psychopathol Behav Assess 2007 Jun 28;29(4):269-276. [CrossRef]
- Goodman R. The extended version of the strengths and difficulties questionnaire as a guide to child psychiatric caseness and consequent burden. J Child Psychol Psychiatry 1999 Jul;40(5):791-799. [Medline]
- Kessler RC, Adler L, Ames M, Demler O, Faraone S, Hiripi E, et al. The World Health Organization adult ADHD self-report scale (ASRS): A short screening scale for use in the general population. Psychol Med 2005 Feb;35(2):245-256. [Medline]
- Johnston C, Mash E. A measure of parenting satisfaction and efficacy. Journal of Clinical Child Psychology 1989 Jun;18(2):167-175. [CrossRef]
- Riley AW, Lyman LM, Spiel G, Döpfner M, Lorenzo MJ, Ralston SJ. The family strain index (FSI). Reliability, validity, and factor structure of a brief questionnaire for families of children with ADHD. Eur Child Adolesc Psychiatry 2006 Dec;15 Suppl 1:I72-I78. [CrossRef] [Medline]
- Rush A. Symptom-specific measures for disorders usually first diagnosed in infancy, childhood, or adolescence. In: Handbook of psychiatric measures. Washington, DC: American Psychiatric Association; 2000.
- Goldberg D. Manual of the general health questionnaire. UK: NFER; 1978.
- DuPaul G, Reid R, Anastopoulos AD, Lambert MC, Watkins MW, Power TJ. Parent and teacher ratings of attention-deficit/hyperactivity disorder symptoms: Factor structure and normative data. Psychol Assess 2016 Feb;28(2):214-225. [CrossRef] [Medline]
- Diagnostic and statistical manual of mental disorders: DSM-IV. Washington, DC: American Psychiatric Association; 2000.
- Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000 Dec 15;25(24):3186-3191. [Medline]
- Gilmore L, Cuskelly M. Factor structure of the parenting sense of competence scale using a normative sample. Child Care Health Dev 2009 Jan;35(1):48-55. [CrossRef] [Medline]
- Vostanis P. Strengths and difficulties questionnaire: Research and clinical applications. Curr Opin Psychiatry 2006 Jul;19(4):367-372. [CrossRef] [Medline]
- Stone LL, Otten R, Engels Rutger C M E, Vermulst AA, Janssens Jan M A M. Psychometric properties of the parent and teacher versions of the strengths and difficulties questionnaire for 4- to 12-year-olds: A review. Clin Child Fam Psychol Rev 2010 Sep;13(3):254-274 [FREE Full text] [CrossRef] [Medline]
- Niclasen J, Teasdale TW, Andersen AN, Skovgaard AM, Elberling H, Obel C. Psychometric properties of the Danish strength and difficulties questionnaire: The SDQ assessed for more than 70,000 raters in four different cohorts. PLoS One 2012;7(2):e32025 [FREE Full text] [CrossRef] [Medline]
- Rimvall MK, Elberling H, Rask CU, Helenius D, Skovgaard AM, Jeppesen P. Predicting ADHD in school age when using the strengths and difficulties questionnaire in preschool age: A longitudinal general population study, CCC2000. Eur Child Adolesc Psychiatry 2014 Nov;23(11):1051-1060. [CrossRef] [Medline]
- Croft S, Stride C, Maughan B, Rowe R. Validity of the strengths and difficulties questionnaire in preschool-aged children. Pediatrics 2015 May;135(5):e1210-e1219. [CrossRef] [Medline]
- Kessler RC, Adler LA, Gruber MJ, Sarawate CA, Spencer T, Van Brunt David L. Validity of the World Health Organization adult ADHD self-report scale (ASRS) screener in a representative sample of health plan members. Int J Methods Psychiatr Res 2007;16(2):52-65 [FREE Full text] [CrossRef] [Medline]
- Kessler RC, Adler L, Barkley R, Biederman J, Conners CK, Demler O, et al. The prevalence and correlates of adult ADHD in the United States: Results from the National Comorbidity Survey Replication. Am J Psychiatry 2006 Apr;163(4):716-723 [FREE Full text] [CrossRef] [Medline]
- Szomlaiski N, Dyrborg J, Rasmussen H, Schumann T, Koch SV, Bilenberg N. Validity and clinical feasibility of the ADHD rating scale (ADHD-RS) A Danish nationwide multicenter study. Acta Paediatr 2009 Feb;98(2):397-402. [CrossRef] [Medline]
- Sonuga-Barke Edmund J S, Daley D, Thompson M. Does maternal ADHD reduce the effectiveness of parent training for preschool children's ADHD? J Am Acad Child Adolesc Psychiatry 2002 Jun;41(6):696-702. [CrossRef] [Medline]
- Reyno SM, McGrath PJ. Predictors of parent training efficacy for child externalizing behavior problems--a meta-analytic review. J Child Psychol Psychiatry 2006 Jan;47(1):99-111. [CrossRef] [Medline]
- Ghuman J, Ghuman H. ADHD in preschool children. New York: Oxford University Press; 2014:9780199948925.
- Weisz JR, Krumholz LS, Santucci L, Thomassin K, Ng MY. Shrinking the gap between research and practice: Tailoring and testing youth psychotherapies in clinical care contexts. Annu Rev Clin Psychol 2015;11:139-163. [CrossRef] [Medline]
- Daley D, Jacobsen R, Lange A-M, Sorensen A, Walldorf J. Costing adult attention deficit hyperactivity disorder: Impact on the individual and society. New York: Oxford University Press; 2015:0198745559.
- Halperin JM, Bédard ACV, Curchack-Lichtin JT. Preventive interventions for ADHD: A neurodevelopmental perspective. Neurotherapeutics 2012 Jul;9(3):531-541 [FREE Full text] [CrossRef] [Medline]
|ADHD: attention deficit/hyperactivity-disorder|
|ADHD RS: ADHD Rating Scale-IV|
|ADRS-v1.1: Adult ADHD Self-Report Scale|
|CAMHS: child mental health services|
|CLB: Ms Cathy Laver-Bradbury|
|CONSORT: Consolidated Standards of Reporting Trials|
|DAWBA: Development and Well-Being Assessment|
|DSM-IV: Diagnostic and Statistical Manual of Mental Disorders|
|FSI: Family Strain Index|
|GHQ: General Health Questionnaire|
|GIPCI-R: Global Impressions of Parent-Child Interactions|
|MT: Dr Margaret Thompson|
|NFPP: New Forest Parenting Programme|
|PSOC: Parenting Sense of Competence Scale|
|SDQ: Strength and Difficulties Questionnaire|
|TAU: treatment as usual condition|
|WHO: World Health Organization|
Edited by G Eysenbach; submitted 04.11.15; peer-reviewed by S Van der Oord; comments to author 20.11.15; revised version received 23.11.15; accepted 25.11.15; published 13.04.16Copyright
©Anne-Mette Lange, David Daley, Morten Frydenberg, Charlotte U Rask, Edmund Sonuga-Barke, Per H Thomsen. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 13.04.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.