Background: Suicide is a severe public health problem, resulting in a high number of attempts and deaths each year. Early detection of suicidal thoughts and behaviors (STBs) is key to preventing attempts. We discuss passive sensing of digital and behavioral markers to enhance the detection and prediction of STBs.
Objective: The paper presents the protocol for a systematic review that aims to summarize existing research on passive sensing of STBs and evaluate whether the STB prediction can be improved using passive sensing compared to prior prediction models.
Methods: A systematic search will be conducted in the scientific databases MEDLINE, PubMed, Embase, PsycINFO, and Web of Science. Eligible studies need to investigate any passive sensor data from smartphones or wearables to predict STBs. The predictive value of passive sensing will be the primary outcome. The practical implications and feasibility of the studies will be considered as secondary outcomes. Study quality will be assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). If studies are sufficiently homogenous, we will conduct a meta-analysis of the predictive value of passive sensing on STBs.
Results: The review process started in July 2022 with data extraction in September 2022. Results are expected in December 2022.
Conclusions: Despite intensive research efforts, the ability to predict STBs is little better than chance. This systematic review will contribute to our understanding of the potential of passive sensing to improve STB prediction. Future research will be stimulated since gaps in the current literature will be identified and promising next steps toward clinical implementation will be outlined.
Trial Registration: OSF Registries osf-registrations-hzxua-v1; https://osf.io/hzxua
International Registered Report Identifier (IRRID): DERR1-10.2196/42146
Suicide is a common cause of death, resulting in over 700,000 deaths worldwide each year, while the total number of suicide attempts is even higher . Especially among adolescents and young adults, suicide is a leading cause of death [ ]. Identification of suicidal thoughts and behaviors (STBs) can support help seeking to prevent suicide attempts [ ]. Years of research have been conducted to identify risk factors to meet this demand of health care [ - ]. Yet, a recent systematic review of prediction models for suicide attempts and deaths reported that the predictive validity associated with a positive result for suicide mortality was extremely low (≤0.01 in most models) [ ]. Additionally, Franklin and colleagues [ ] found that research on suicide risk factors in the past 50 years carries almost no predictive power due to several reasons. First, the prediction of a suicidal event is difficult in general because of the low probability of the occurrence of suicide [ , ]. Second, a major shortcoming of prior investigations was the repeated focus on an identical set of risk factors that have been shown to produce little predictive value for when an attempt might occur. For example, while the first suicidal thoughts or ideas about specific suicide methods often occur years before a suicide attempt, the transfer from ideation to action often occurs within days or hours before the attempt [ ]. Third, similar measurement methods were repeatedly used, predominantly questionnaires [ ]. However, questionnaires fall short in describing the strong fluctuations in risk factors for suicidal behavior like emotional distress, hopelessness, or suicidal thoughts [ , ], making short-term prediction nearly impossible. In addition, it is difficult to model the complex interactions of different risk factors using questionnaire data [ ].
In recent years, promising new ways to identify STBs have emerged. The widespread use of smartphones in everyday life provides a source of data that allows real-time monitoring . Passive sensing is one of many terms that are used to describe the passive collection of behavioral data via smartphones or other wearable devices [ ]. Depending on the type of device, different sensors can be used to collect a variety of data points. In general, behavioral (eg, movement via GPS), physiological (eg, heart rate), and social (eg, social media engagement) signals can be measured through the sensors [ ]. The objectivity and seamless provision of information over a period of time brings advantages compared to subjective questionnaires at a certain point in time [ , ]. Therefore, research has focused on the potential and feasibility of passive sensing in mental health [ - ]. Machine learning methods can then build predictive models from these huge data sets [ , ].
Despite a number of unanswered questions and methodological challenges , some researchers have explored the potential of passive sensing for mental health. Studies investigating the relationship between passive sensing and psychological symptoms have been conducted [ , , ]. For example, Zulueta and colleagues [ ] found that increased accelerometer activity was correlated with a change in mood disturbances. Furthermore, increased social media use was found to be positively correlated to depressive symptoms [ ]. Quantitative variables such as the number of outgoing calls, unique numbers texted, or absolute distance traveled showed a predictive value for depressive symptoms in an investigation by Place and colleagues [ ]. Across several studies, there is a promising trend that passive sensing can be reliably used to predict some symptoms or behaviors. However, other studies report less-promising results, which tempers expectations and highlights the need for replication trials to confirm preliminary findings [ , ].
The Proposed Review
Prior reviews have been conducted to summarize the possibilities and limitations of passive sensing in suicide prediction [, , ]. Given the dynamic development of the field, an updated review is needed to observe the latest advances in the field. The proposed systematic review aims to summarize the research on passive sensing in suicide prediction and identify both major advances and obstacles. In particular, we intend to create a better understanding of how current findings may translate into practice. For this purpose, the review will address (1) whether passive sensing data show the ability to improve the prediction of STBs compared to traditional methods of data collection, (2) what the comparative predictive power of different sensor types is, and (3) what analysis methods have been reported in the literature.
This protocol is based on the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) checklist .
We will include studies that addressed passive data generation via smartphones or wearables in the context of STBs. All individuals regardless of age or gender with any STB (suicidal ideation, suicide attempt, death by suicide) will be included. Studies reporting on nonsuicidal self-injury will be excluded. If studies investigate both STBs and nonsuicidal self-injury, they will be included. Studies will be included regardless of whether or not the participants were receiving treatment. Studies will be eligible if they report results on the association between passive sensing and STBs. In addition, we will include study protocols and conduct a search of international study registries to preview upcoming research. Articles will be translated into English if necessary.
A web-based systematic database search will be performed using the following search terms: (mobile sens* OR smart sens* OR smartphone sens* OR passive sens* OR passive monitor* OR sensor OR sensors OR digital phenotyp* OR wearable* OR passive data OR real-time data OR real-world data) AND suicid*. To ensure the sensitivity of the search, the search string was validated by a test set of 7 hand-searched relevant articles ([ - ]). The search string was optimized using an iterative process until a coverage rate of 100% was reached. In July 2022, the search will be conducted via the databases MEDLINE, PubMed, Embase, PsycINFO, and Web of Science ( ). We will perform forward searches via Google Scholar and backward searches in the reference lists. Gray literature will not be searched.
The selection of relevant articles will be conducted by 2 independent researchers using the online tool Covidence. First, all titles and abstracts resulting from the search will be screened against the eligibility criteria. Second, the full texts of the articles selected in the first step will be obtained and screened in more detail. Disagreements will be resolved in discussion with a third reviewer. Duplicates will be identified and excluded. In the case of multiple reports of the same study, all available data will be reported. All steps of the selection process will be described in detail and will be visualized in a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart .
All data will be managed via Covidence, which will be used for the whole selection process. At the end of the process, the 2 independent reviewers will extract information according to a data extraction sheet. The following variables will be extracted from all included articles: authors, publication year, journal, population variables (age, gender, STBs), data collection device (smartphone, smartwatch, smart home, etc), type and frequency of passive data (social media, text messages, screen time, etc), analysis methods (machine learning, predictive models, qualitative techniques, etc), and assessment length. If any important data are missing, the authors will be contacted. Any qualitative or quantitative data describing the predictive ability of passive sensor data will be extracted. Results on the practical relevance and feasibility of passive sensing will be considered as secondary outcomes.
Risk of Bias Assessment
To assess the risk of bias at the study level, 2 independent reviewers will use the signaling questions of the Prediction Model Risk of Bias Assessment Tool (PROBAST) . According to the tool, 4 domains will be analyzed: participants, predictors, outcome, and analysis. In total, 19 signaling questions will be answered with “yes,” “probably yes,” “no,” “probably no,” or “no information.” Afterward, the level of risk of bias will be estimated. Disagreement between the reviewers will be solved via discussion. If discussion does not lead to agreement, a third reviewer will be consulted.
All extracted characteristics of the identified studies will be described narratively. The relevant results will be presented in text form and visualized in tables. If an appropriate number of studies report associations between identical sensor data and a quantitative measure for STBs, we will perform meta-analytic pooling. The meta-analytic pooled correlation will be estimated using a random-effects model with a maximum likelihood estimator. We will treat the heterogeneity (ie, the variability between the studies in terms of methodology and sample characteristics) as random . In this way, we will estimate both the average true effect and the amount of heterogeneity among the true effects [ ]. If heterogeneity is zero, the average true effect displays the true effect.
The selection process started in July 2022. Data extraction started at the beginning of September 2022. Results are expected in December 2022.
The aim of this systematic review is to summarize the potential of passively generated data to predict STBs. Research in this area is new but has developed rapidly in recent years. Consequently, we expect a rather heterogenous set of reports and trial designs. Therefore, one aspect of this review will be to identify key variables that future trials should report in order to increase comparability in future systematic reviews. For example, an extended form of the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) Checklist  for sensing data could be developed. In addition, this review offers a chance to identify valuable predictors to improve the prevention of suicide. It will present an updated summary of existing knowledge in this fast-growing field and exceed prior reviews’ quality through preregistration and by using a systematic approach. Next to predictive values of quantitative sensor data, feasibility aspects of sensing studies will be pointed out as well. Hence, new inspirations for further research regarding methodological possibilities of data collection and study design will be stimulated. At the same time, recent evidence will be critically evaluated in order to create further demands for research that will advance the path to clinical applicability.
Funding for the article processing fee was provided by the University of Freiburg’s Open Access Publishing program. No further external funding was received for this study.
All study data will be made publicly available on the Open Science Framework website.
Conflicts of Interest
The test set of 7 hand-searched relevant articles.DOCX File , 21 KB
Database search strings.DOCX File , 16 KB
- Suicide. World Health Organization. 2021 Jun 17. URL: https://www.who.int/news-room/fact-sheets/detail/suicide [accessed 2022-06-24]
- Glenn CR, Nock MK. Improving the short-term prediction of suicidal behavior. Am J Prev Med 2014 Sep;47(3):176-180 [FREE Full text] [CrossRef] [Medline]
- Chang BP, Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, et al. Biological risk factors for suicidal behaviors: a meta-analysis. Transl Psychiatry 2016 Sep 13;6(9):e887-e887 [FREE Full text] [CrossRef] [Medline]
- Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol Bull 2017 Feb;143(2):187-232. [CrossRef] [Medline]
- Gonda X, Pompili M, Serafini G, Montebovi F, Campi S, Dome P, et al. Suicidal behavior in bipolar disorder: epidemiology, characteristics and major risk factors. J Affect Disord 2012 Dec 20;143(1-3):16-26. [CrossRef] [Medline]
- Nock MK, Borges G, Bromet EJ, Alonso J, Angermeyer M, Beautrais A, et al. Cross-national prevalence and risk factors for suicidal ideation, plans and attempts. Br J Psychiatry 2008 Feb;192(2):98-105 [FREE Full text] [CrossRef] [Medline]
- Qin P, Agerbo E, Westergård-Nielsen N, Eriksson T, Mortensen PB. Gender differences in risk factors for suicide in Denmark. Br J Psychiatry 2000 Dec;177:546-550. [CrossRef] [Medline]
- Belsher BE, Smolenski DJ, Pruitt LD, Bush NE, Beech EH, Workman DE, et al. Prediction models for suicide attempts and deaths: a systematic review and simulation. JAMA Psychiatry 2019 Jun 01;76(6):642-651. [CrossRef] [Medline]
- Millner AJ, Lee MD, Nock MK. Describing and measuring the pathway to suicide attempts: a preliminary study. Suicide Life Threat Behav 2017 Jun;47(3):353-369. [CrossRef] [Medline]
- Ballard ED, Gilbert JR, Wusinich C, Zarate CA. New methods for assessing rapid changes in suicide risk. Front Psychiatry 2021 Jan 26;12:598434 [FREE Full text] [CrossRef] [Medline]
- Kleiman EM, Turner BJ, Fedor S, Beale EE, Huffman JC, Nock MK. Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies. J Abnorm Psychol 2017 Aug;126(6):726-738. [CrossRef] [Medline]
- Turecki G, Brent DA. Suicide and suicidal behaviour. Lancet 2016 Mar 19;387(10024):1227-1239 [FREE Full text] [CrossRef] [Medline]
- Abdullah S, Choudhury T. Sensing technologies for monitoring serious mental illnesses. IEEE MultiMedia 2018 Jan;25(1):61-75. [CrossRef]
- Baumeister H, Montag C. Digital phenotyping and mobile sensing in psychoinformatics—a rapidly evolving interdisciplinary research endeavor. In: Baumeister H, Montag C, editors. Digital Phenotyping and Mobile Sensing: New Developments in Psychoinformatics. Studies in Neuroscience, Psychology and Behavioral Economics (SNPBE) book series. Cham, Switzerland: Springer; 2019:xiii-xixx.
- Allen NB, Nelson BW, Brent D, Auerbach RP. Short-term prediction of suicidal thoughts and behaviors in adolescents: can recent developments in technology and computational science provide a breakthrough? J Affect Disord 2019 May 01;250:163-169 [FREE Full text] [CrossRef] [Medline]
- Areán PA, Pratap A, Hsin H, Huppert T, Hendricks K, Heagerty P, et al. Perceived utility and characterization of personal google search histories to detect data patterns proximal to a suicide attempt in individuals who previously attempted suicide: pilot cohort study. J Med Internet Res 2021 May 06;23(5):e27918 [FREE Full text] [CrossRef] [Medline]
- Betthauser LM, Stearns-Yoder KA, McGarity S, Smith V, Place S, Brenner LA. Mobile app for mental health monitoring and clinical outreach in veterans: mixed methods feasibility and acceptability study. J Med Internet Res 2020 Aug 11;22(8):e15506 [FREE Full text] [CrossRef] [Medline]
- Brietzke E, Hawken ER, Idzikowski M, Pong J, Kennedy SH, Soares CN. Integrating digital phenotyping in clinical characterization of individuals with mood disorders. Neurosci Biobehav Rev 2019 Sep;104:223-230. [CrossRef] [Medline]
- Melcher J, Hays R, Torous J. Digital phenotyping for mental health of college students: a clinical review. Evid Based Ment Health 2020 Nov;23(4):161-166. [CrossRef] [Medline]
- Bernert RA, Hilberg AM, Melia R, Kim JP, Shah NH, Abnousi F. Artificial intelligence and suicide prevention: a systematic review of machine learning investigations. Int J Environ Res Public Health 2020 Aug 15;17(16) [FREE Full text] [CrossRef] [Medline]
- Kirtley OJ, van Mens K, Hoogendoorn M, Kapur N, de Beurs D. Translating promise into practice: a review of machine learning in suicide research and prevention. Lancet Psychiat 2022 Mar;9(3):243-252. [CrossRef] [Medline]
- Onnela J. Opportunities and challenges in the collection and analysis of digital phenotyping data. Neuropsychopharmacology 2021 Jan;46(1):45-54 [FREE Full text] [CrossRef] [Medline]
- MacLeod L, Suruliraj B, Gall D, Bessenyei K, Hamm S, Romkey I, et al. A mobile sensing app to monitor youth mental health: observational pilot study. JMIR Mhealth Uhealth 2021 Oct 26;9(10):e20638 [FREE Full text] [CrossRef] [Medline]
- Vlisides-Henry RD, Gao M, Thomas L, Kaliush PR, Conradt E, Crowell SE. Digital phenotyping of emotion dysregulation across lifespan transitions to better understand psychopathology risk. Front Psychiatry 2021;12:618442 [FREE Full text] [CrossRef] [Medline]
- Zulueta J, Piscitello A, Rasic M, Easter R, Babu P, Langenecker SA, et al. Predicting mood disturbance severity with mobile phone keystroke metadata: a biaffect digital phenotyping study. J Med Internet Res 2018 Jul 20;20(7):e241 [FREE Full text] [CrossRef] [Medline]
- Place S, Blanch-Hartigan D, Rubin C, Gorrostieta C, Mead C, Kane J, et al. Behavioral indicators on a mobile sensing platform predict clinically validated psychiatric symptoms of mood and anxiety disorders. J Med Internet Res 2017 Mar 16;19(3):e75 [FREE Full text] [CrossRef] [Medline]
- Cornet VP, Holden RJ. Systematic review of smartphone-based passive sensing for health and wellbeing. J Biomed Inform 2018 Jan;77:120-132 [FREE Full text] [CrossRef] [Medline]
- Zarate D, Stavropoulos V, Ball M, de Sena Collier G, Jacobson NC. Exploring the digital footprint of depression: a PRISMA systematic literature review of the empirical evidence. BMC Psychiatry 2022 Jun 22;22(1):421 [FREE Full text] [CrossRef] [Medline]
- Kleiman EM, Nock MK. Real-time assessment of suicidal thoughts and behaviors. Curr Opin Psychol 2018 Aug;22:33-37 [FREE Full text] [CrossRef] [Medline]
- Torous J, Larsen ME, Depp C, Cosco TD, Barnett I, Nock MK, et al. Smartphones, sensors, and machine learning to advance real-time prediction and interventions for suicide prevention: a review of current progress and next steps. Curr Psychiatry Rep 2018 Jun 28;20(7):51. [CrossRef] [Medline]
- Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015 Jan 01;4:1 [FREE Full text] [CrossRef] [Medline]
- Sheridan DC, Baker S, Dehart R, Lin A, Hansen M, Tereshchenko LG, et al. Heart rate variability and its ability to detect worsening suicidality in adolescents: a pilot trial of wearable technology. Psychiatry Investig 2021 Oct;18(10):928-935 [FREE Full text] [CrossRef] [Medline]
- Kleiman EM, Bentley KH, Maimone JS, Lee HS, Kilbury EN, Fortgang RG, et al. Can passive measurement of physiological distress help better predict suicidal thinking? Transl Psychiatry 2021 Dec 02;11(1):611 [FREE Full text] [CrossRef] [Medline]
- Haines-Delmont A, Chahal G, Bruen AJ, Wall A, Khan CT, Sadashiv R, et al. Testing suicide risk prediction algorithms using phone measurements with patients in acute mental health settings: feasibility study. JMIR Mhealth Uhealth 2020 Jun 26;8(6):e15901 [FREE Full text] [CrossRef] [Medline]
- Coyne SM, Hurst JL, Dyer WJ, Hunt Q, Schvanaveldt E, Brown S, et al. Suicide risk in emerging adulthood: associations with screen time over 10 years. J Youth Adolescence 2021 Feb 02;50(12):2324-2338. [CrossRef]
- Sels L, Homan S, Ries A, Santhanam P, Scheerer H, Colla M, et al. SIMON: a digital protocol to monitor and predict suicidal ideation. Front Psychiatry 2021 Jul 1;12:554811 [FREE Full text] [CrossRef] [Medline]
- Berrouiguet S, Barrigón ML, Castroman JL, Courtet P, Artés-Rodríguez A, Baca-García E. Combining mobile-health (mHealth) and artificial intelligence (AI) methods to avoid suicide attempts: the Smartcrises study protocol. BMC Psychiatry 2019 Sep 07;19(1):277 [FREE Full text] [CrossRef] [Medline]
- Bruen AJ, Wall A, Haines-Delmont A, Perkins E. Exploring suicidal ideation using an innovative mobile app-strength within me: the usability and acceptability of setting up a trial involving mobile technology and mental health service users. JMIR Ment Health 2020 Sep 28;7(9):e18407 [FREE Full text] [CrossRef] [Medline]
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021 Mar 29;372:n71 [FREE Full text] [CrossRef] [Medline]
- Andaur Navarro CL, Damen JAA, Takada T, Nijman SWJ, Dhiman P, Ma J, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ 2021 Oct 20;375:n2281 [FREE Full text] [CrossRef] [Medline]
- Viechtbauer W. Conducting meta-analyses in with the package. J Stat Soft 2010;36(3). [CrossRef]
- Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015 Jan 06;162(1):55-63. [CrossRef] [Medline]
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|PRISMA-P: Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols|
|PROBAST: Prediction Model Risk of Bias Assessment Tool|
|STB: suicidal thoughts and behavior|
|TRIPOD: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis|
Edited by T Leung; submitted 26.08.22; peer-reviewed by P Santangelo, M Kapsetaki; comments to author 08.10.22; revised version received 19.10.22; accepted 25.10.22; published 29.11.22Copyright
©Tanita Winkler, Rebekka Büscher, Mark Erik Larsen, Sam Kwon, John Torous, Joseph Firth, Lasse B Sander. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 29.11.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.