Paranoia refers to concerns that others may intend to harm you [1,2]. Freeman’s [1] theoretical model of paranoia suggests a hierarchy, whereby more severe forms of paranoid worries in clinical populations (eg, believing others are trying to kill you) build on milder, everyday forms of social evaluative concerns (eg, believing others are talking about you behind your back), which are commonly experienced in the general population. Paranoia is often a distressing experience that can negatively interfere with many aspects of life, including people’s everyday activities [3] and interpersonal relationships [4]. Many people can experience paranoia, including those with psychosis spectrum disorders, but it is also a transdiagnostic symptom and can be present in other mental health conditions as well, including depression and anxiety disorders [5]. Paranoia can also be experienced by nonclinical populations across the lifespan [3,6].

To improve the lives of those experiencing paranoia, developing support (eg, to reduce paranoia or sustain recovery) for this population has been identified as a research priority by patients, carers, researchers, and health care professionals in an international priority setting partnership [7]. One promising treatment mode for those experiencing paranoia is digital mental health interventions (DMHIs), or treatment that incorporates technology to improve users’ mental health [8]. An early value assessment conducted by the National Institute for Health and Care Excellence identified specific DMHIs as suitable for full recommendation, subject to further data collection [9]. Early recommendation of this sort can improve the provision of care by increasing treatment options and access to support using fewer resources than in-person treatments [9]. Preliminary evidence suggests that DMHIs may be an acceptable and feasible treatment for paranoia [10].

While other DMHIs for paranoia have been developed, most are delivered via virtual reality, require clinical support, or focus on other psychotic experiences (eg, voices) and pay limited attention to more common, social evaluative concerns within paranoia (eg, that others are talking about you negatively) [10]. Successful Treatment of Paranoia (STOP) was developed to address these limitations by providing a self-delivered, smartphone-based transdiagnostic intervention specifically targeting interpretation biases underpinning paranoia. The intervention and its development have been described in detail elsewhere [11,12]. A brief description is included to contextualize our work.

STOP is a self-administered psychological intervention using the principles of cognitive bias modification for paranoia (CBM-pa). CBM-pa aims to address the cognitive interpretation bias that is thought to cause and maintain the symptoms of paranoia [13]. Individuals displaying this interpretation bias often interpret neutral or ambiguous social scenarios as personally threatening [13]. There is preliminary evidence from a feasibility randomized controlled trial (RCT) that interpretation biases can be modified by CBM-pa that was delivered via a desktop computer in a research laboratory setting (N=63) [14].

STOP in the RCT included a weekly session administered through a smartphone app either on users’ personal smartphones or, if needed to avoid digital exclusion, on a study phone with a data plan provided free of charge to participants. Two versions of the intervention of varying lengths were evaluated to investigate “dose” effects (6 vs 12 weeks). Users scheduled weekly sessions at convenient times, during which they read 40 everyday social scenarios. In the sessions, users were encouraged to interpret ambiguous social scenarios in a nonthreatening way by completing a word puzzle and then answering a question about the meaning of the scenario (Figure 1 includes an intervention example). Session content was designed by clinicians, academics, and the McPin Foundation research charity’s STOP lived experience advisory panel (LEAP) of 7 people who give voice to intended end users of the app to increase its clinical relevance and acceptability to users [11]. Automated phone and email reminders were sent at midnight 1 day before the scheduled session and again if a session was not completed on the scheduled day.

Program theories underpin interventions and explain how, why, for whom, and in which circumstances interventions are assumed to work and can be represented visually in a logic model [15]. A logic model for the STOP intervention was created after conducting a document analysis of the trial protocol and publications associated with the intervention (Figure 2) [11,12]. An initial program theory was developed from the logic model (Textbox 1).

We will conduct a process evaluation to better understand the factors influencing DMHI implementation both during and after the trial. Exploring factors that may influence implementation is an important research focus as the implementation of DMHIs is a persistent challenge [16,17], and it is also acknowledged that interventions may be used differently across settings and populations [18,19]. There is some empirical evidence for this as reviews have demonstrated DMHI use can vary due to several factors, including patient characteristics (eg, age) and intervention features (eg, including human support) [20]. However, implementation remains a key area of uncertainty in the STOP trial as it is currently unknown what, or if any, factors influence the implementation of STOP (Figure 2, context box). A limited understanding of implementing STOP contrasts with a comparatively better understanding of how STOP is assumed to work, for which there is a growing evidence base [13,14] and plans to evaluate mechanisms of action in other trial work [12]. Intervention implementation is an important uncertainty to address given that STOP has the potential to be implemented in a variety of settings (as users choose where and when sessions are accessed) and by a variety of potential end users (as users may not need to be under the care of a health service or have a formal mental health diagnosis to use STOP) [12].

Our work aims to address these knowledge gaps by exploring the factors that influence the implementation of a self-administered digital intervention for paranoia in and after an RCT. The overarching objective of this evaluation is to develop an evidence-based understanding of implementing STOP by investigating how much, by whom, and under what circumstances STOP is and may be implemented in and after a trial. A secondary objective is to use this knowledge to identify the barriers and facilitators to implementing STOP in and after a trial.

We will conduct a mixed methods process evaluation of STOP. The protocol is reported using an amended version of the process evaluation recommendations from the Medical Research Council (MRC) process evaluation framework (Table S1 in Multimedia Appendix 1) [18]. We amended the reporting standards to fit the reporting of a protocol paper by removing sections that are not applicable (ie, those relating to reporting evaluation findings). Details of the core research team (CH, CD, JY, and PJ) are provided in Multimedia Appendix 2.

The STOP trial was preregistered at ISRCTN (ISRCTN17754650) and approved by the London-Stanmore Research Ethics Committee (21/LO/0896, January 14, 2022) and the Medicines and Healthcare products Regulatory Agency (CI/2022/0033/GB, July 19, 2022), including approval for collecting additional qualitative data with STOP users and health care professionals. An additional qualitative study involving STOP trial participants was conducted and approved separately by the University of Bath Social Sciences Research Ethics Committee (0179-353). Written informed consent was obtained from all participants prior to data collection, with the option to withdraw at any time without specifying a reason. Participants were compensated for their time: trial participants received up to £115 (US $153; £20 [US $26.6] per follow-up assessment, an additional £20 [US $26.6] at the trial end, and £15 [US $19.95] after completing an additional interview), health care professionals received a £20 (US $26.6) voucher, and nonrandomized STOP users received £40 (US $53.2). All participants were assigned unique study identifiers, and only fully anonymized and deidentified data will be accessed for analyses. The process evaluation was approved by the University of Bath Social Sciences Research Ethics Committee as a requirement of the doctorate (5841-6196). Further ethics approval was not required because the process evaluation did not introduce additional trial procedures or data collection and will only use existing data from participants who consented to data archiving.

We will use the 2015 MRC process evaluation guidance as a methodological framework, which assumes that an efficacy evaluation in isolation provides a limited understanding of an intervention [18]. Instead of evaluating intervention efficacy, process evaluation in the MRC framework explores 3 proximal outcomes which can complement efficacy evaluations (Textbox 2) [18].

This mixed methods study will conduct analyses on STOP trial routine data and secondary analyses on sociodemographic data from another paranoia population in England.

To summarize who agreed to take part and try STOP in a trial (STOP uptake), to describe factors that influenced uptake, and to characterize and assess the representativeness of those agreeing to try STOP compared with another paranoia population in England.

STOP RCT participants and recruitment procedures are described earlier (in the Efficacy RCT section) and in full in the protocol paper [12].

Measures will include intervention uptake, factors influencing uptake, and the characteristics of individuals who agreed to try STOP compared with a paranoia population in England. Intervention uptake will be summarized using trial records to report the number and percentage of those agreeing to try STOP out of those who were approached to take part in the trial at different time points in the RCT (eg, consent, screening, and randomization) to summarize the uptake of STOP in a trial.

Factors influencing uptake will be identified from qualitative reasons reported by those agreeing to and declining to try STOP at different time points in the RCT (eg, consent, screening, and randomization), using STOP trial documents (eg, CONSORT diagrams, STOP case report forms, and recruitment trackers) to describe factors that facilitated or hindered the uptake of STOP in a trial.

Characteristics of those agreeing to try STOP compared with a paranoia population in England will be assessed using sociodemographic data obtained from STOP trial baseline data and from the Adult Psychiatric Morbidity Survey (AMPS) conducted in 2023 to 2024 by Public Health England, which provides data about treated and untreated psychiatric disorders among adults in England [23], to characterize and assess the representativeness of those agreeing to try STOP. We will compare those who agreed to try STOP with the subsample of responders endorsing items indicating paranoid thinking, which is a method that has been adopted in other analyses of AMPS [24,25]. We are unable to prespecify the variables or which items and measures will be used to indicate paranoia because data from the AMPS are not available at the time of publication [23]. However, based on the most recently available AMPS dataset published in 2016 [26], probable variables include age, sex, ethnicity, living arrangement, region, employment status, and current treatment.

Rates of uptake will be summarized descriptively. Qualitative data from STOP trial documents will be analyzed in the latest version of NVivo (Lumivero) to provide an audit trail using a framework analysis approach [27,28], and findings from all trial documents will be triangulated. We will use the Consolidated Framework for Implementation Research (CFIR) [29] as a deductive framework to identify factors influencing uptake. Data that cannot be coded using a priori codes from the framework will be inductively coded in an “other” category. Qualitative analysis will be led by CH, but codes and themes will be developed and reviewed by the core research team (CD, JY, and PJ). STOP trial participants will be characterized using descriptive statistics from the trial’s main outcome paper, supplemented as necessary by further information. Exploratory quantitative analyses will be conducted using relevant packages in the most recent version of R software (R Foundation for Statistical Computing) to test for between-group differences using chi-square tests and 2-tailed t tests where appropriate. If necessary, a P value correction will be used to account for multiple testing.

Data collection is complete, but analysis has not yet started.

This mixed methods study will use STOP trial routine data and will conduct secondary analyses of qualitative data generated from interviews with STOP users who consented to data archiving. The interviews were originally conducted as part of a qualitative evaluation of the host STOP trial or other research work, and not as part of the planned process evaluation, and have been previously disseminated as peer-reviewed publications [30,31].

To summarize how much STOP was used in an RCT, to characterize users of STOP, and to describe factors that predicted or influenced its use.

Participants include 182 trial participants who were randomized to the intervention arms, and 10 STOP intervention users consenting to participate in interviews for data archiving. The eligibility criteria for trial participants are described earlier (in the Efficacy RCT section). No exclusion criteria were added to participate in the interview studies, although STOP users needed an email account and access to a phone or computer to take part [30,31]. Interviewees were predominantly female (15/25, 60%) and White (16/25, 64%) [30].

The recruitment procedure for the host STOP trial is described earlier (in the Efficacy RCT section). In the qualitative studies, 2 samples participated in a one-off interview. The first sample comprises 8 participants who had completed the host STOP trial, were randomized to an intervention arm, provided informed consent for an optional qualitative interview, and consented to data archiving [30]. The second sample included 12 participants who were ineligible for the host trial but had tested a 6-week version of the STOP intervention on a dedicated study phone and consented to a one-off interview; the 2 participants who consented to data archiving will be included in our analyses [31].

Measures will include intervention use, predictors of use, and factors that hindered or facilitated use. Intervention use in the trial is the primary outcome of study 2 and will be quantified and treated as a count variable based on the number of sessions completed (0-6 or 0-12) for 6 or 12 weeks, which was electronically monitored by a digital health platform (HealthMachine; Avegen Health). Acceptable and target session completion was defined in the study protocol as the user entering a response on 20 (50%) or 30 (75%) training items per session, and both completion rates will be considered.

Predictors of use will be selected from 10 quantitative variables in STOP trial routine baseline data. Our selection was informed by available trial data and an existing digital engagement model [32]. The digital complexity model proposes a person-centered approach to treatment and is informed by established models of engagement [32], such as the cumulative complexity model [33] and has been adopted in other evaluations [34]. Predictors include user variables (life demands, resources, and clinical characteristics) and STOP intervention variables (delivery device and therapy burden). User life demands include self-reported employment and marital status. User clinical characteristics include self-reported number of years living with paranoia, self-reported paranoia severity measured using a validated clinical measure (the Paranoia Worries Questionnaire) [35], and interpretation bias severity measured using the Similarity Rating Task. We selected the most sensitive measures of paranoia and interpretation bias severity as predictors after observing data in the host trial. User resources include self-reported dyslexia and premorbid IQ measured using the Wechsler Test of Adult Reading [36]. Intervention components include the STOP delivery device (users’ own smartphone or a study-provided phone) and intervention length (6 or 12 weeks, based on the allocated trial arm), both recorded in electronic databases used for the trial (MACRO; Ennov and HealthMachine).

Barriers to intervention use will be identified from qualitative self-reported reasons for lack of use from 3 sources. First, users reported reasons for missed sessions during check-in calls with a trained researcher. Second, reasons for withdrawal from the host trial are recorded in an electronic database (MACRO). Third, pooled interviews with STOP users will be used to identify whether any factors that hindered use were reported outside of routine trial data.

Facilitators to intervention use will also be identified from pooled interviews with STOP to determine whether any factors that facilitated intervention use were reported outside of routine trial data.

A semistructured topic guide was created for each qualitative study (Supplementary Materials S1 and S2 in Multimedia Appendix 3). The topic guides were developed to explore users’ experiences of a digital therapeutic relationship [30] and the perceived acceptability of STOP [37]. Trained researchers (TT and LE) conducted all one-off interviews online, which were recorded and transcribed. Anonymized transcripts will be accessed from secure institutional servers.

Quantitative analyses will be conducted using relevant packages in the most recent version of R. To contextualize the substudy findings, STOP users will be characterized using descriptive statistics, including clinical and sociodemographic variables, taken from the trial’s main outcome paper and supplemented as necessary by further information, and reported per intervention arm. Chi-square tests or pairwise comparisons will be used to test for differences between groups. To report intervention use, descriptive statistics will summarize the total number and proportion of intervention users in the host trial completing 0 to 6 or 0 to 12 sessions using acceptable and target rates. To test predictors of engagement, we will first assess the nature of the distribution and then run a linear regression analysis for normally distributed data or Poisson or negative binomial regression if data are not normally distributed (depending on overdispersion in the data). We will qualitatively analyze self-reported barriers and facilitators to intervention use using the same approach in study 1 (in the Analysis section) as analysis will be led by CH, supported by NVivo, and will deductively use the CFIR framework [29] in framework analysis [27,28], while inductively coding other data not captured in the initial framework.

Data collection is complete, but analyses have not yet started.

This qualitative study conducted interviews and focus groups with health care professionals and will conduct secondary analyses on interviews with STOP users (described under study 2).

To identify the hypothetical barriers and facilitators to implementing STOP after the trial using the perspectives of STOP users and health care professionals.

Participants include STOP users and health care professionals. The STOP user participants are described previously (in the Participants section for study 2). Twenty health care professionals from 6 NHS trusts in England participated in an interview (n=11) or focus group (n=2). These participants were aged 26 to 61 (mean 38, SD 11.736) years and were predominantly clinical psychologists (15/20, 75%), female (15/20, 75%) and White (16/20, 80%).

The recruitment processes for STOP users are described earlier (in the Recruitment section for study 2). Health care professionals were recruited from NHS trusts in England. Participants were those who provided informed consent to take part in a one-off interview or focus group after trialing a tester version of the STOP intervention or had viewed a video demonstrating the app.

The procedure for the STOP user interviews has been described previously (in the Procedure section for study 2). The health care professional topic guide was developed to explore the perceived acceptability of STOP (Supplementary Material S3 and S4 in Multimedia Appendix 3). Trained researchers (CH, AK, and HS) conducted the interviews and focus groups using Microsoft Teams, which were recorded and transcribed.

Descriptive statistics for anonymized basic demographic data will be tabulated. To date, interviews and focus groups have been transcribed verbatim. Next, we will qualitatively analyze data using the same approach in study 1 (in the Analysis section) with analysis led by CH, supported by NVivo, and deductively using the CFIR framework [29] in framework analysis [27,28], while inductively coding other data not captured in the initial framework.

Data collection and transcription are complete, and preliminary analyses have started.

Our process evaluation is expected to provide a detailed understanding of factors that influence implementation within a trial context. An evidence-based understanding of implementing STOP in an RCT will address a key uncertainty in the STOP logic model (Figure 2) and provide a more accurate understanding of actual implementation processes, which may differ from hypothesized ones [38]. Our process evaluation therefore offers unique insights beyond other STOP trial projects and can complement and contextualize efficacy findings. Insights into intervention-related implementation barriers and facilitators will inform future iterations of STOP. We will also develop an implementation framework for posttrial delivery of STOP, which will inform the upscaling of the intervention outside of an RCT setting; a persistent challenge [16,17] that must be addressed to avoid underuse of research efforts and to ensure the provision of optimal care to those experiencing mental health difficulties. Our framework may also be useful for other researchers or clinicians seeking to integrate self-administered DMHIs into routine care.

Our findings can address important gaps in research and knowledge in this field. The process evaluation is conducted in a context where there is interest in developing treatments for paranoia [7], including those that are digitally delivered [2]. While an oft-cited advantage of digital treatments is the potential to implement this mode in a variety of settings [39], the literature has focused on service settings [38,40,41], which often have limited access to the resources needed to implement DMHIs [42]. Therefore, we will address this limited opportunity to learn about DMHI implementation in other settings, including settings without clinical support or guidelines, which may be of interest to other evaluators in this field. Furthermore, the process evaluation methodology is often underutilized [43,44]. Our work therefore addresses a methodological gap and aligns with wider priorities to strengthen implementation research in digital mental health [16,45] and may add to the growing evidence base that a concurrent investigation of efficacy and implementation using process evaluation can improve intervention evaluation and development in trials [18,46]. Finally, an evidence-based understanding about implementing STOP has implications for the provision of care to those experiencing paranoia. Our work could be applied to a care context where health care professionals can assume that those with paranoia are unsuitable for this treatment mode due to patients’ illness factors, such as preexisting concerns about technology [47,48]. Evidence suggests that such staff views can be overgeneralizing, as paranoia does not preclude engagement with DMHIs [10] or can coexist with their use [49].

This process evaluation has several strengths. First, to our knowledge, we are the first to explicitly apply process evaluation methodology to evaluate a DMHI used by a transdiagnostic sample of individuals with paranoia. Furthermore, we undertake a rigorous approach to process evaluation by adopting an established framework for this evaluation method [18]. An additional strength is that most process evaluation data were collected as part of a large-scale clinical trial and thus were subject to quality control checks and may be less subject to responder bias, meaning that more reliable data will be used. Furthermore, we will use mixed methods data and a range of data sources, which may identify different factors affecting implementation. Triangulation of mixed methods data in analyses will enable an integrated interpretation of process evaluation findings, which will support the development of an evidence-based understanding of implementation that may be used in future work. Finally, we transparently report on the planned process evaluation approach, scope, and methods to contribute to good practice and share our work with other stakeholders.

However, the process evaluation should be considered in line with several limitations. Uptake rates may be limited by incomplete recording of individuals who declined participation at the initial approach, as data were only available for contacts made directly by STOP research assistants or shared with the trial team. Our use of secondary interview data with STOP users has several limitations. These studies did not explicitly address our evaluation aims, which may have limited the depth or relevance of feedback from STOP users to our process evaluation. Furthermore, the use of secondary data meant that we were unable to analyze data during data collection, limiting our ability to explore emergent themes. Qualitative data were also collected using opportunity sampling, which may introduce bias toward responders who may be more interested in intervention evaluation or had more positive experiences of STOP. Future evaluation work in the field could mitigate this limitation by prospectively embedding a process evaluation within trials. Additionally, we could not prespecify which variables we will use to assess sample representativeness in study 1 as the AMPS protocol or data are not available at the time of publication. Finally, the protocol will be published after data collection was completed for the process evaluation studies. To mitigate these limitations and enhance transparency, we will prespecify our analysis plans on Open Science Framework before starting analyses [22].

Overall, our work will develop an evidence-based understanding of the implementation of STOP in and after a trial, including identifying barriers to and facilitators of its implementation. Our process evaluation is also expected to develop a hypothetical implementation framework for posttrial delivery inside and outside of mental health services, which will inform upscaling and future iterations of the STOP intervention. Expected findings are anticipated to also have implications beyond the STOP trial and intervention. Our findings may also highlight the importance of process evaluation when trialing interventions by generating evidence that can inform the development or upscaling of other self-administered DMHIs.