In total, 300 nationally recruited individuals were diagnosed with a primary diagnosis of AD (n=142, 47.3%) or ED (n=158, 52.7%) and were included in the RCT. Participant recruitment was carried out through social media, newspaper advertisements, and information provided to health care clinics. Participants self-referred to the study web page, where they signed digital informed consent and completed a screening battery consisting of sociodemographic and clinical background questions as well as self-report symptom questionnaires. Participants were subsequently clinically assessed by a psychologist using a structured diagnostic interview, including Mini International Neuropsychiatric Interview [31], self-rated ED [32], and the Adjustment Disorder New Module-8 (ADNM-8) [33]. For inclusion, participants needed to (1) fulfill the criteria for a primary diagnosis of AD or ED, (2) be aged between 18 and 65 years, (3) have regular access to a computer with internet access, and (4) be able to read and write in the Swedish language. Exclusion criteria included (1) drug use or addiction during the past 6 months, (2) current or past psychosis or bipolar disorder, (3) current risk of suicide, (4) changed psychopharmacological treatment in the past month, (5) other ongoing psychological treatment, and (6) previous experience of CBT for AD or ED in the past year.

Patients were randomized to one out of two 12-week internet-delivered treatments (CBT and general health-promoting advice). They both consisted of web-based text-based modules with related exercises and assignments. Patients were guided sequentially through the modules by a therapist via a secure web-based platform. The therapists’ primary role was to provide feedback on exercises, support in problem-solving, and to give emotional and technical support via weekly asynchronous text messages. Therapists were licensed clinical psychologists or clinical psychology students in their final year of training. Because this study will not evaluate the individual treatments, they will not be further described here. A full description of the treatments is described in the study by Sennerstam et al [34].

Predictors were gathered through self-report measures that were administered in the web-based study platform, clinical assessment conducted before inclusion to the study, and remote cognitive testing. Table 1 presents all predictors included in the study.

Information on age (interval), sex (male, female, other, or prefer not to disclose), relationship status (in relationship, single, or widowed), number of children, educational attainment (in 9 categories between <9 years of school to PhD), employment status (eg, student, unemployed, or full-time work), and employment type (in 11 categories, eg, employed in the private sector, by the municipality, or other) was gathered before the start of treatment using the web-based study platform. Self-rated reading and computer skills were rated separately on a 5-step ordinal scale from poor to very good. Patients also reported if they were Swedish native speakers.

During the clinical interview, patients reported their medication regimen, specifying both the number (0-4) and type of medication (antidepressants, anxiolytics, sleep medication, and pain medication and yes or no). Primary diagnosis (AD or ED), and possible secondary psychiatric diagnosis (eg, anxiety or depressive disorder) was assessed by the clinician using Mini International Neuropsychiatric Interview, self-rated ED (ordinal categories ranging from no to yes—to a high degree) [32], and the ADNM-8 [33]. Using ADNM-8, the patient was asked about which specific stressors had been present in the past 2 years (in 16 options, eg, too much or too little work or financial difficulties). The clinician assessed the length of the current episode (in months), and the age of the patients first episode (in years). Sick-leave status upon inclusion in the study (0%-100% in 5 steps), length of current sick-leave episode (0-1 months to >12 months in 5 categories) was self-reported.

Alcohol use was assessed using the Alcohol Use Disorder Identification Test [40,41]. This 10-item screening instrument evaluates alcohol consumption, drinking behavior, and alcohol-related problems over the past year. It contains items, such as How often do you have six or more drinks on one occasion? rated on various ordinal scales, typically ranging from 0 to 4.

Symptoms of anxiety were measured using the Generalized Anxiety Disorder-7 scale [42]. This screening tool assesses generalized anxiety symptoms over the past 2 weeks. It comprises 7 items, such as not being able to stop or control worrying rated on a 4-point ordinal scale ranging from 0 not at all to 3 nearly every day.

Symptoms of burnout were measured using the Shirom-Melamed Burnout Questionnaire [43,44]. It aims to measure 3 components of burnout; emotional and physical fatigue, cognitive weariness, and listlessness and contains statements such as I have difficulty concentrating rated on a 7-point scale ranging from 1 never or almost never to 7 always or almost always with some items using reversed scoring.

Symptoms of depression were measured using Montgomery-Åsberg Depression Rating Scale [45]. It is a 9-item questionnaire used to measure different aspects of depression such as concentration difficulties, suicidal thoughts, sadness, and affected appetite with answers rated on a 7-point ordinal scale from 0 to 6.

Symptoms of exhaustion disorder were measured using the 9-item Karolinska Exhaustion Disorder Scale [46]. Measuring different aspects of exhaustion such as fatigue, endurance, and sleep impairment, answers are rated on an ordinal scale from 0 to 6 (eg, ability to concentrate; ranging from 0 “I do not have any difficulty concentrating, and can read, watch TV and converse normally” to 6 “I cannot concentrate on anything at all.”)

Functional disability was measured using The World Health Organization Disability Assessment Schedule (2.0) [47], developed to assess functioning in the last 30 days in 6 different life domains, including cognition, mobility, self-care, relationships, life activities, and societal participation. It contains statements, such as “I have difficulty standing for longer periods such as 30 minutes.” Answers are rated on a 5-point ordinal scale ranging from 0 never to 4 extreme or unable. A 12-item version was used.

Quality of life was assessed using the EQ-5D-5L [48,49] and the Brunnsviken Brief Quality of Life Scale [50]. The EQ-5D-5L contains 5 dimensions: mobility, self-care, usual activities, pain or discomfort, and anxiety or depression each rated on 5 levels of severity from no problems to extreme problems. The Brunnsviken Brief Quality of Life Scale is a 12-item questionnaire that assesses 6 life areas (leisure time, view on life, learning, creativity, view of self, and friends and friendship). Ratings range from 0 strongly disagree, to 4 strongly agree, on statements of the importance and satisfaction of each area.

Insomnia severity was measured using the Insomnia Severity Index [51]. The Insomnia Severity Index is a 7-item questionnaire designed to assess aspects of insomnia, including difficulty falling asleep, difficulty staying asleep, and satisfaction with sleep. Ratings are given using an ordinal scale ranging from 0 to 4.

Self-rated health was assessed using Self-Rated Health 5 asking patients to rate their general health on a scale of 1, very bad to 5 very good [52].

Somatoform symptoms were assessed using the Patient Health Questionnaire [53]. It consists of 15 questions covering somatic symptoms commonly seen in primary care, such as back pain, headache, and nausea. Answers are rated on a 3-point ordinal scale ranging from not at all bothered to bothered a lot.

Subjective memory impairment was measured using the 6-item Questionnaire of Everyday Memory Problems (6-QEMP) [54]. A 5-item version has previously been used to assess subjective memory problems in this patient population [55,56]. The present version was adapted by Stigsdotter Neely for use in patients with stress-related disorders with statements, such as “How do you think your memory functions now compared to before your stress-related mental health problems?.” The answers are rated on a 5-point ordinal scale.

Clinician Treatment Expectancy was judged after patient assessment, upon inclusion in the study, by clinicians rating the probability of the patient improving after treatment on a scale of 0 no improvement to 10 full remission.

The Treatment Credibility Scale was administered 3 weeks after the start of treatment [57]. Patients were asked questions about their impression of the treatment and if they thought they would improve. It included statements such as “How logical do you think this treatment is?” and “How confidently would you recommend this treatment to a friend with the same problems as you?” on a scale of 0 not at all to 10 very logical, very confidently.

Attention and processing speed were measured using the Symbol Digit Modality Test. A test originally developed by Smith [58,59] that measures visual detection, attention, and processing speed. A key with 9 different symbols and matching numbers is shown on the upper part of the display. At the center one of these 9 symbols are shown and the task of the participant is to choose the corresponding number using the key as guidance. The test score is the number of correct entries in 90 seconds. Comparable substitution tasks are considered sensitive to treatment effects for patients with multiple sclerosis [60] and depression [61], and it has been used in patients with stress-related disorders [62].

Executive functioning was measured using the Verbal Fluency Test (FAS) Word Fluency Test and the Stroop test. FAS was first described by Spreen and Benton [63], and it measures spontaneous verbal fluency and selective attention and shifting. The participant is tasked with producing words beginning with a certain alphabet letter (F, A, and S). Names, numbers, or repeated words are not allowed. The test score is the number of correct words beginning with the letter. FAS and similar word fluency tasks have been shown to be impaired in patients with stress-related exhaustion [62].

The Stroop test, originally developed by Stroop [64] and described by Jensen and Rohwer [65], measures executive functioning, inhibition, as well as updating and processing speed [66]. The test has 2 parts, (1) 20 color words are presented (green, yellow, blue, or red) and they are colored congruent to their meaning (eg, the word red colored in red). In the bottom part of the display, the color words are displayed on 4 buttons. The task is to, as quickly and thoroughly as possible, click the correct button. (2) Twenty color words are presented but displayed in an incongruent color (eg, the word red colored in green). The task of the participant is to click the button containing the color of the word as quickly and thoroughly as possible. Test score is calculated as an index (number of correct answers in part 2 divided by average time in seconds from part 2) and for interference (average time in part one–average time in part one). Performance of Stroop in patients with stress-related disorders has been shown to be impaired in 2 studies [62,67], but not in others [68,69].

Memory and learning were assessed using the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) Word List Learning Test and Corsi block-tapping test forward. CERAD was originally developed for use with Alzheimer disease [70] but is similar to other word-list tasks used in this patient population. It measures verbal learning and episodic memory. In the learning part of the test, a word list containing 10 words is presented over 3 trials and the task after every trial is to recall the words from the list. For every presentation the order is mixed. In the delayed recall part of the test (trial 4) that occurs after 5 to 10 minutes, the participant is asked to recall the words. Test score for the learning time is number of correct words in trial 1 to 3, and in the delayed recall part, number of correct words in trial 4. Similar word-list tasks have been used previously to assess memory functioning in patients with stress-related disorders [62,69].

Corsi block-tapping test forward gives information about visual ability of attention, short-term memory and working memory [71]. It contains 2 parts, but in this test battery, only the first part of the test is used. Nine blocks are displayed, and the testing platform starts by lighting up a sequence of blocks. The task is to repeat the sequence of blocks that the platform has displayed. The task starts out easy with only 2 blocks, but the difficulty increases by adding a longer sequence of blocks until the participant enters the incorrect sequence twice at the same number of blocks. The test score is the maximum number of correct repeated blocks. A cross-sectional study comparing patients with stress-related disorders to a healthy normative group found impaired performance on this test [62].

We will transform categorical variables into a format suitable for numerical analysis. For binary categorical variables, we will use label encoding. For multinomial variables, we will apply one-hot encoding. In addition, for ordinal data, which have a natural order, we will transform the categories into integers.

Predictor variables with over 20% missing data will be excluded from the analysis. Categorical variables exhibiting low variance, as determined by predictors with <5% of a certain response will be removed. For instance, by removing the variable “Sleep medication” if it only occurs in 3 out of 300 patients. This approach aims to reduce unnecessary complexity in the predictions and to minimize the risk of overfitting. To control for multicollinearity, variables with a correlation coefficient ≥0.8 will be removed from the traditional prediction analysis but will be retained for the ML model development. Data that are highly skewed will be transformed if deemed appropriate.

Cognitive test results will be manually reviewed before model fitting to validate a proper result. Comments pertaining to technical difficulties and disturbances that may have affected the test result will be assessed by two of the authors and lead to exclusion if so judged. Participants who have noted during screening that Swedish is not their native language will be excluded from the analysis for CERAD and FAS. We will standardize the raw scores from the cognitive tests using normative regression models with age, education and sex as covariates. This standardization process will convert raw scores into Z scores, as previously described by Franke Foyen et al [62] and for a full overview of the multiple linear regression models used and how they were calculated, see the studies by Mindmore [77] and van den Hurk et al [78].

Patients who have missing data for the posttreatment PSS-10, ie, the missing outcome variable for the primary aim, will be replaced by a PSS-10 process measurement at week 10 if available; If not, the patient will be excluded from the analysis. The number of participants excluded from the final models will be described.

To prepare our primary outcome, RCI for the PSS-10 before to after treatment will be computed using the following formula [39]:

Cronbach α=0.83 from normative data will be used [38]. Patients exhibiting an RCI of −1.96 will be classified as responders.

Descriptive statistics will be used to summarize the sample characteristics and pretreatment variables, including mean or median, SDs and IQR for continuous variables, and proportions for categorical variables.

For the traditional regression analysis, data will be imputed using the KNN imputer. The imputer, a nonparametric imputation method, works by imputing missing values based on the k-nearest neighbors; in this study k will be determined by cross-validation. It uses the Euclidean distance metric to find the nearest neighbors and can be used for both numerical and categorical data. Each missing value is imputed using values from its k-nearest neighbors. After imputation, we will run univariate logistic regressions for each predictor listed in Table 1 using RCI as a target variable. Predictors that are statistically significant in the univariate analyses will then be included in an ablation study, a systematic approach to evaluate feature importance. This method involves iteratively removing each significant predictor from a full model, measuring the change in explained variance, and then reinserting it, thereby quantifying each predictor’s unique contribution to the model’s explanatory power in the context of all other features.

For an introduction on the technical terms introduced in this section, see the review article by Bzdok and Meyer-Lindenberg [13].

As the ultimate goal of any model is to predict an outcome in unseen data, the ML models will be developed using a training set, and then evaluated on a test set stratified on main diagnosis (AD or ED) and responder status. In total, 70% of the data will be used for selecting predictor variables and training the models, and 30% for testing the prediction accuracy of the models. The choice of 70 to 30 was due to the limited size of our sample, as fewer observations in the testing data makes it difficult to use uncommon predictors. No external validation set is currently available at the time of writing.

Standardization and imputation will be applied on the training and test data separately to avoid data leakage. Numerical data will be standardized and all missing data will be imputed using the KNN imputer.

We will train and evaluate 4 different ML classifiers, a multiple logistic regression (LogReg) classifier using elastic net, a random forest (RF) classifier, a support vector machine (SVM) classifier, and an AdaBoost classifier. For a review of the models used, see the textbook by Geron [79]. In short, the LogReg classifier works by modeling the probability of a binary outcome based on one or more predictor variables, using the logistic function to ensure the output is between 0 and 1. We will use elastic net regularization to facilitate feature selection and prevent overfitting. Elastic net combines L1 (lasso) and L2 (ridge) penalties, encouraging sparsity and maintaining stability in the model. The RF classifier works by building multiple decision trees on random subsets of data and predictors. Each tree’s prediction is based on splits that minimize variance in the target variable, with the final model ensembling these predictions. The SVM classifier works by finding the hyperplane that maximizes the margin between different classes in the feature space. SVM is particularly effective in high-dimensional spaces and when the number of dimensions exceeds the number of samples. AdaBoost, the final classifier, works by combining multiple weak classifiers, typically decision trees, into a single strong classifier. It sequentially fits these weak learners on repeatedly modified versions of the data, focusing more on misclassified instances to improve overall accuracy.

We will conduct 5-fold cross-validation using randomized search for hyperparameter tuning and training evaluation to enhance the external generalizability and robustness of the results. This process involves defining a hyperparameter space, then randomly selecting a predetermined number of samples—in this case, 10—from this space, and conducting 5-fold cross-validation for each selected set of hyperparameters. Fivefold cross-validation is done by partitioning the data into 5 subsets, training the model on 4 subsets, and validating it on the remaining subset. This process is repeated 5 times, with each subset used exactly once as the validation data. The best performing hyperparameters will be chosen for the final models that are trained and then evaluated on the test set.

The hyperparameter ranges for the LogReg will include C values from 0.01 to 100 and l1_ratio values from 0 to 1. For RF, the parameter ranges will include the number of estimators from 5 to 1200, minimum samples required to split a node from 10 to 200, maximum depths from 5 to 750, and a binary indicator for bootstrapping. For SVM, the parameter range for the randomized search will include regularization parameter C values from 0.01 to 1 and for AdaBoost, the parameter ranges for the randomized search will include the number of estimators, ranging from 1 to 1500, and learning rates from 0.001 to 2.5.

The models developed to identify the responder status will be evaluated using BACC, precision and recall, both in the training set obtained through k-fold cross-validation and in the test set. Predictor importance in the RF model will be determined using Scikit-learn’s Feature importance function, which quantifies each predictor’s contribution to the model’s balanced classification accuracy. Area under the curve will be used to assess the models’ capability to distinguish between classes accurately. The approach will aim to provide a clear understanding of the models’ effectiveness and the role of various predictors. Our primary outcome of interest for comparison will be BACC in each model in the test set with the aim that (1) the model should perform better than a dummy model that simply predicts the most common responder status, and (2) that the model should perform 67% BACC or above to be deemed clinically useful [28]. Furthermore, the models will be statistically compared using bootstrap sampling. Specifically, we will generate 5000 bootstrap samples from the test set, calculating the BACC for each model on each sample. The distributions of these bootstrap BACCs will be compared and we will conclude that there is a statistically significant difference between models if the CIs do not overlap.