Mobile health apps for smartphones and tablet devices have become a lucrative business, with worldwide expenditure estimated to be over US $92 billion [1]. Apps are increasingly being used to monitor, assess, and improve mental health. There are now more than 10,000 publicly available mental health–specific apps [2]. Most of these apps lack published evidence for their effectiveness, making it difficult for clinicians and consumers to know which app is the most appropriate [3]. Currently, choices are made using reviews and ratings available in app stores [4], but these can produce unreliable results [5].

Although effective treatments for anxiety and depression exist, many people do not access these for various reasons [6]. However, with ownership of smartphones being at 70% of the global population and rising [7], mental health apps potentially offer a partial solution to limitations in service availability and acceptability.

Published reviews have found that mental health apps can be effective for reducing anxiety [8] and depression [9] with an overall effect size of small to moderate [10]. Within this research, there are some notable shortcomings, including substantial heterogeneity across studies. For example, there have been differences in dosage [11,12] duration of interventions [13,14] and the absence of long-term follow-up data [15].

Another limitation of previous research is that most of it has been carried out by individuals who have developed the app, who have stood to gain financially from its sales, and/or who were otherwise associated with it [3]. For instance, a recent review of app stores found that only 1.02% of mental health apps offering therapeutic treatment for anxiety and/or depression had been evaluated using independent research [3]. Furthermore, in a meta-analysis of 9 studies on apps targeting anxiety [8] and in another meta-analysis of 18 studies on apps targeting depression [9], none involved independent research or replication (note that some studies were included in both meta-analyses).

A possible reason for the lack of research on apps is the time factor for large-scale experimental designs. Specifically, randomized controlled trials (RCTs) that demonstrate the efficacy of an intervention by measuring and comparing the outcomes of matched treatment and control groups are often lengthy to conduct. This is a barrier to achieving results in a time frame that is acceptable to the profit-driven app market. In the time it takes to complete an RCT, the app being studied may have been updated or disappeared from the market altogether as newer apps with enhanced features emerge in its place. Furthermore, RCTs are not necessarily the most appropriate study design for every situation, with another limitation of RCTs being lower ecological validity [16].

Single-case research designs address the issue of ecological validity by testing the effectiveness of an intervention for individuals (ie, performance under real-world conditions). However, single-case designs can also control for threats to internal validity and thus test for the efficacy of a treatment. Such designs go beyond a study with a sample of one participant and involve continuous and repeated measurements, random assignment, sequential introduction of the treatment, and specific data analysis and statistics [17]. Robust results in clinical psychology and behavioral science can be demonstrated when benefits are shown in 3 to 5 cases [18-20]. A single-case design is also safer than an RCT for vulnerable participants because their well-being is monitored by gathering and analyzing data more frequently during the study, and treatment can be altered if there is a clinically significant decline in status [21,22]. For mental health apps, single-case designs are a viable alternative for accelerating the evidence base [23,24].

The main objective of this study is to use a single-case design to examine the effectiveness of 5 mental health apps that purport to have efficacy for reducing symptoms of anxiety and/or depression.

This study seeks to answer the following research questions: (1) Do the apps in this study provide clinically significant improvements in symptoms of anxiety and/or depression? (2) What individual characteristics of participants influence the results in this regard? and (3) What individual characteristics of the apps influence the results?

The only hypothesis to be tested in this study is based on question 1:

The Howard et al [25] model proposed that the outcome of any psychotherapeutic intervention will involve progressive reductions in subjective distress, then symptomatology, and, finally, an increase in overall life functioning.

This study is registered with the Australian and New Zealand Clinical Trials Registry (ANZCTR), which is a primary registry in the World Health Organization Registry Network (registration number: ACTRN12619001302145p).

A multiple baseline across-individuals design will be employed. Multiple baseline designs in mental health intervention studies are those where a baseline period of stability of symptoms is established before the intervention is introduced. In this way, each participant acts as their own control, and internal validity is demonstrated when there is no change in symptoms until after the treatment is introduced [18]. In the design to be used in this study, all participants commence the baseline period at the same time but start the treatment at different times after a minimum number of daily data readings (at least 20) have been received. This sequential commencement approach further strengthens the internal validity by reducing the likelihood of history, maturation, or other extraneous factors explaining any observed emotional or behavioral change that occurs simultaneously with the introduction of the treatment. The multiple data recordings allow for the use of analytical techniques such as a time series analysis [20,26] and will involve participants reporting ratings of subjective units of distress (SUDS) via SMS text messaging using a 10-point scale. In this design, 4 or more baselines are recommended [18,20], and these will follow the pattern shown in Figure 1.

This research will use a prescribed dosage approach, as if the app was a digital antidepressant: one 10-min dose of app use per day for 5 days per week. The 10-week intervention period creates equivalence with one 50-min session per week for 10 weeks, which is the annual maximum number of psychology sessions rebated under Australia’s Medicare system [27]. The rationale for a minimum 3-week postintervention period (to demonstrate the stability of the treatment effect) is similar to that described earlier, namely, that 20 daily SUDS ratings are needed for a valid data analysis. The justification for using the 5 chosen apps are as follows: (1) each has some evidence of efficacy published in a peer-reviewed journal, (2) all have publicly available free versions, and (3) all can fit the prescribed dosage approach. The chosen apps use 3 popular evidence-based frameworks employed across mental health settings for treating anxiety and/or depression: cognitive behavioral therapy (CBT), mindfulness, and positive psychology. Finally, the rationale for using 10 participants per app is that, even accounting for a 50% to 60% attrition rate [28], approximately 3 to 5 participants per app will provide enough data for a valid statistical analysis using time series conventions. As previously noted, this number of replications in a single-case design with similarly presenting individuals can produce robust and generalizable findings if the results are comparable in each case [18-20].

Commencing in January 2020, participants will be recruited throughout Australia by advertising the proposed study to nongovernment organizations that run programs for clients with mental illness (eg, the Benevolent Society), contacting associations of mental health professionals that may alert their members to the proposed study (eg, Australian Psychological Society), and contacting support groups and other organizations in the mental health sector (eg, Mental Health Victoria), requesting they advertise the proposed study on their various social media platforms. The advertisement is shown in Multimedia Appendix 1. Recruitment will cease once 50 participants are recruited. Owing to the nature of the proposed study design, new participants cannot commence after the study has started because the multiple baseline design requires participants to begin at the same time and then have specifically staggered phase commencements after that. Figure 1 demonstrates this process.

All 50 participants (10 for each app) will be randomized to an app and their position in the single-case design (ie, P1 to P10) using the web-based random number generator, Research Randomizer [29]. The full inclusion and exclusion criteria are presented in Textboxes 1 and 2, respectively. A financial reimbursement will be offered to participants of Aus $0.50 (US $0.33) per daily SUDS rating sent via a text message. The researchers acknowledge that financial payments have the potential to interfere with ecological validity, because a person in the community would not normally be paid for using a mental health app, and intrinsic motivation, because people could potentially use the app for the benefit of financial remuneration rather than for the value of improving their mental health. However, the low amount of remuneration being offered of approximately Aus $45.00 (US $29.41) on average is not considered payment for participation in the proposed study but rather reimbursement of personal expenses incurred while taking part in the proposed study. Given that the effectiveness of mental health apps has the potential to benefit those from low socioeconomic groups, being reimbursed for providing in excess of 80 text messages will alleviate reasons that a potential participant of lower socioeconomic background could provide for being out of pocket for the cost of sending text messages from their mobile phone. Therefore, it is envisaged that prospective participants will more likely have motivation to improve their mental health beyond receiving financial remuneration. In addition, this financial incentive will not be advertised, and participants will only learn about this when they provide consent when completing the demographics questionnaire.

Participants will supply their own smartphones and/or tablet devices. In total, 5 different apps will be used: (1) Destressify [30,31], (2) MoodMission [32-35], (3) Smiling Mind [36,37], (4) MindShift [15,38], and (5) SuperBetter [39,40].

All the apps are supported by published research demonstrating statistically significant efficacy for the treatment of anxiety and/or depression. Each app has an accompanying website with further information and an accessible privacy policy. Detailed information about each app and its accompanying research is provided in Multimedia Appendix 2.

A number of measures of participants’ experiences and outcomes will be used, as described in the following sections.

Figures 1 and 2 illustrate the phases of this research. Recruitment commenced in January 2020.

Ethics approval was granted by the University of New England Human Research Ethics Committee on November 1, 2019 (approval number: HE19-186). This research will be conducted under the guidelines of the National Statement on Ethical Conduct in Human Research by the Australian National Health and Research Council [66]. Any changes to procedures outlined in this protocol will be forwarded to the University of New England Human Research Ethics Committee for approval before implementation. Such changes will also be acknowledged on the trial registration at the ANZCTR.

Information about and descriptions of the previous research on each app are provided in Multimedia Appendix 2. Previous research on mental health apps is lacking, and there are other issues impacting this research. The methodologies employed in the studies based on the apps used in the proposed study are heterogeneous, and this is in keeping with other previous research on other mental health apps [3]. The studies here had varying attrition rates (the Destressify research [30] reported 19.9%; MoodMission [34], 54.8%; Smiling Mind [36], 17.7%; MindShift [15], 46.7%; and SuperBetter [39], 73.9%), but the varying intervention times and methodologies may have contributed to these attrition rates. Some studies were conducted by researchers who either developed the app or had an otherwise pre-existing association with the app being tested (in the case of MoodMission and SuperBetter). Other studies had participants with varying degrees of severity of anxiety and depression at commencement that were measured with varying outcome instruments (Destressify, MoodMission, and Smiling Mind did not specify the participants’ mental health status in their inclusion criteria; MindShift had participants with moderate-to-high levels of anxiety, as indicated by their ratings on at least one scale of the Patient Health Questionnaire; and SuperBetter had participants who scored higher than 16 on the Center for Epidemiological Studies Depression Scale). Nevertheless, all studies were published in peer-reviewed journals and authored by individuals who have associations with legitimate academic institutions and mental health organizations. Therefore, we hypothesize that our results will reveal the apps to be effective for reducing symptoms of anxiety and depression, as reflected in the previous research.

This study is unique with respect to any published study of mental health apps that could be located. This independent research, using a single-case methodology, allows for an in-depth examination of personal factors that may impact the effectiveness of these apps. It will therefore add value to existing studies on these specific apps. However, it is also anticipated that this research will be automated to the point where the design can be used to examine larger samples with the rigor of an RCT experimental design, thereby having a positive impact on increasing future research at a faster rate. It also provides an opportunity for evidence-based mental health treatment to reach those who are not already receiving it.

If mental health apps have demonstrated effectiveness, they could be incorporated by clinicians into face-to-face therapy to enhance the experience of consumers. For example, some apps allow users to complete homework tasks set by their therapist or to make thought diary entries that can be shared digitally with their therapist. Other information such as physiological readings and user-entered information such as SUDS ratings can also be sent digitally to clinicians to gain a more accurate reading of their client’s emotional health between sessions [68].

There are potential benefits for health systems and mental health consumers if apps can gain increased legitimacy for their ability to effectively manage anxiety and depression. These include the following: more economical for low socioeconomic groups to obtain mental health treatment compared with face-to-face services [69], improved access for those in rural areas where there may be limited treatment options [70], reduced stigma [71] because of anonymous assistance, access for children and adolescents who are already large consumers of smartphones and the internet [72], and it is simply a preferred way to receive mental health information for some [73]. Therefore, it is important to increase research on the efficacy and effectiveness of mental health apps using appropriate and scientifically validated methodologies in addition to RCTs, as the widely considered gold standard of RCTs may not be the most appropriate for analyzing mental health apps [23,74].

This study will examine a number of mental health apps that differ in several ways: (1) having different theoretical frameworks (MoodMission and MindShift use CBT, Destressify and Smiling Mind use mindfulness, and SuperBetter uses positive psychology), (2) being developed by different teams in different countries (Destressify was developed in the United States by individuals with an interest in mindfulness meditation, MoodMission was developed in Australia by a team of psychologists and researchers at Monash University, Smiling Mind was set up as a not-for-profit organization in Australia by mental health and meditation experts, MindShift was developed in Canada by a not-for-profit mental health organization, and SuperBetter was developed in the United States by a game designer and mental health researchers from Stanford University and the University of Pennsylvania), and (3) containing different aesthetic qualities and types of activities with different aims (Destressify focuses on reducing stress, MoodMission focuses on providing short activities designed to help an individual in response to how they are feeling at that time, Smiling Mind focuses on teaching mindfulness skills in a structured format using guided meditation, MindShift focuses on reducing anxiety by using a number of different interventions such as graded exposure and using a thought journal, and SuperBetter is very colorful with a playful tone that may appeal to individuals who like video games). Multimedia Appendix 2 provides further information about each app. Having a diversity of apps is important because there may be differences in the way consumers react to different aspects of an app. It is known that face-to-face therapy outcomes can be influenced by client-therapist rapport [75], client motivation [42], and chronicity/history of mental illness [43]. Therefore, there may be different aspects of a mental health app that contribute to its effectiveness, such as gamification, aesthetics, usability/interface [76], and evidence-based framework.

In sum, the reasons mentioned earlier support the need for a vigorous research agenda on the effectiveness of mental health apps, and this study methodology can assist in realizing this.

This study has some limitations. First, it may not be possible to generalize the findings if the outcomes for participants with the same condition differ in significant ways. Second, there is no certainty that participants will provide daily SUDS ratings for 16 weeks, despite the minimal effort involved. Third, with many brands of smartphones using different versions of software, there is a risk that the technology between phone and app may not be compatible for some participants.

This study also has several strengths. By using questionnaires that consider subject distress, symptoms, and life functioning at different time points, the design allows for a comprehensive approach toward the impact of the apps. The use of the DASS-21 and OQ-45.2 questionnaires at multiple time points will allow an examination of issues such as suicidality, dysfunctional coping activities (such as excessive alcohol consumption), physical health, and sleep disturbance. The single-case design will also provide in-depth information about individual responses and offers a way that clinicians may be able to contribute to the evaluation process (see the Conclusions section). Finally, there is the ambitious goal of offering a future methodology that could be applied to larger RCTs via a highly digitized procedure.

The evidence base for mental health apps that offer treatments for anxiety and depression is currently low. This study may assist in improving this situation in several ways. First, it may allow more clinicians to participate in the research process. Marshall et al [74] have outlined a proposal to establish a centralized database where clinicians and researchers contribute information and data on the effectiveness of mental health apps by using a standardized protocol that forms the basis of this research. Such a repository of information on mental health apps would mean an ever-increasing knowledge base that clinicians, researchers, government authorities, and academic institutions could refer to. Although there are existing websites that offer professional reviews with useful insights into mental health apps (eg, PsyberGuide [77], Head To Health [78], Reachout Australia [79], Health Navigator [80], and the NHS App Library [81]), these are based on professionals’ perspectives and not systematic and scientific observations. The results of increased research, such as that outlined in the proposed study, have the potential to add valuable empirical data to such websites to reinforce the reviews posted there.

If a collaborative scientific methodology was used by clinicians and researchers to rate the effectiveness of mental health apps, this would also potentially allow more transparent categorization of mental health apps in the various app stores [74]. Currently, reliance on app stores leads to potential confusion for consumers as ratings and reviews may be unreliable or even fake [5]. Using the methodology in this study is one way that, if willing, the app stores could certify mental health apps as having reached an acceptable level of independently verified effectiveness [3]. This would allow consumers to more clearly identify apps validated by scientific research.

Finally, given the large number of consumers who own a smartphone globally [82], if more people are able to use efficacious mental health apps on their phones, it could potentially free up scarce face-to-face services in communities struggling to meet the demand for interventions to address mild-to-moderate mental health problems.