Postoperative delirium of cardiovascular surgery (PODOCVS) is an acute brain dysfunction characterized by inattention, impaired consciousness, and cognitive impairment that fluctuates in presence and severity across time [1].

Studies show that PODOCVS affects approximately 26% to 52% of patients with cardiovascular surgery [2]. As a common complication of cardiovascular surgery, PODOCVS worsens patient outcomes [3] and consumes enormous financial resources in excess of US $164 billion per year [4]. PODOCVS is independently associated with major surgical trauma, advanced age, cardiopulmonary bypass, compromised brain health, and transfer to the cardiovascular intensive care unit post surgery [2,5,6]. PODOCVS occurs during the early postoperative period and is associated with adverse outcomes including prolonged mechanical ventilation times and hospital stays, long-term nursing needs, worsening of pre-existing cognitive impairment, new dementia, increased anxiety and depression, and premature mortality [7]. In terms of pharmacological therapies, ramelteon and suvorexant can reportedly reduce delirium, but premedication to prevent delirium is not recommended for all patients [8]. However, due to the lack of specific treatments, strategies for managing delirium currently focus only on its prevention and early detection [9,10]. Therefore, identifying populations at high risk of developing delirium and providing targeted and effective interventions are crucial [9,11]. Risk stratification and identification of vulnerable patients offer an extremely efficient method of protecting them from the initial adverse outcomes of PODOCVS. The pathogenesis of PODOCVS is multifactorial and, as such, multivariable prediction models for stratifying PODOCVS risk may enable early personalized treatment interventions [12-14]. This would significantly reduce the incidence of delirium, shorten hospital stays, cut medical costs, and reduce the incidence of postoperative complications and mortality risk [4].

At present, there are many PODOCVS prediction models used in clinical practice. These can be divided into traditional statistical models and machine learning (ML) models. Although statistical models (eg, logistic regression) are favorable in terms of model interpretability, Choi et al [15] indicate that ML is preferred for predictive accuracy. The predictive power of traditional statistics-based models (eg, the E-PRE-DELIRIC [Early Prediction Model for Delirium in the intensive care unit] model) varies greatly across regional cohorts with an area under the curve (AUC) of 0.54, which is below the receiver operating characteristic curve of 0.75 [16,17]. Mufti et al [18] and Xue et al [19] have shown that the predictive accuracy of ML algorithms was superior to that of conventional statistical models in the area of PODOCVS [20]. ML models can handle complex, multidimensional data without the limitations of traditional statistical methods. Although there is an expanding body of published literature on the use of ML models in cardiovascular health care, whether their performance in predicting PODOCVS is superior to that of traditional statistical models remains to be verified [21].

The accuracy of predictive models for patients at risk of developing delirium has been reported to be insufficient when applied to patients with cardiac surgery [17]. Hence, predictive models for delirium risk should be constructed for patients with cardiac surgery specifically [22]. In recent years, an increasing number of studies have focused on developing or validating predictive models to estimate PODOCVS risk. However, health care professionals are uncertain as to which model to use for patients undergoing cardiac surgery in specific patterns, settings, and populations. Hence, the quality of existing models and their applicability remain unclear. Therefore, it is critical to thoroughly evaluate the predictive performance, applicability, and quality of existing PODOCVS risk prediction models [23]. To this end, this study presents a protocol for the systematic review and critical assessment of the quality, performance, and applicability of current predictive models for PODOCVS risk.

The systematic review aims to thoroughly evaluate the predictive performance, applicability, and quality of existing PODOCVS risk prediction models. To provide a robust scientific foundation for health care professionals to select appropriate delirium prediction models tailored to patients with cardiac surgery in specific clinical environments. The objectives of this project are listed in Textbox 1.

This protocol was developed according to the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols) guidelines (Multimedia Appendix 1) [24].

A systematic review will be conducted according to the recommended methods for systematic reviews and meta-analyses of prediction models [24] and will adhere to the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) checklist [25]. The results of this review will be reported as per the updated PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 checklist [26]. The review will be registered in the International PROSPERO (International Prospective Register of Systematic Reviews). Any modifications to the protocol will be amended accordingly.

This review follows the PICOTS (Population, Intervention model, Comparator, Outcomes, Timing, and Setting) approach to frame the review question and determine the inclusion criteria (Table 1) [25,26].

This systematic review will include studies with a subject population of patients with cardiac surgery aged 18 years and older, and studies using developed and internally or externally validated predictive models for PODOCVS risk based on multivariate analysis. The outcome metric is postoperative delirium. Studies will not be excluded based on their publication status and language. The exclusion criteria are as follows: (1) studies focused solely on the external validation of existing models; (2) studies that construct predictive models without conducting internal/external verification; (3) studies for which data are unavailable through the accessible channels; (4) studies lacking evidence of mode performance in derivation or validation cohorts; (4) studies that only analyze predictive risk factors without constructing predictive models; and (5) various types of publications such as conference abstracts, reviews, comments, research protocols, animal model studies, and textbook materials.

Once this protocol is published, the search will be performed in the following databases: Embase, PubMed, the Web of Science Core Collection, the China National Knowledge Infrastructure (CNKI), Wan Fang, and Wei Pu (VIP). The following search terms will be used: “cardiac surgical procedure,” “cardiac surgery,” “cardiovascular surgery,” “heart surgery,” “coronary artery bypass grafting,” “Coronary Artery Bypass Grafting (CABG),” “extracorporeal circulation,” “type A aortic dissection,” “type B aortic dissection,” “subacute delirium,” “delirium,” “postoperative delirium,” “delirium of mixed origin,” “deliri*,” “psychosis,” “intensive care delirium,” “neurological complications,” “risk prediction,” “model,” “risk score,” and “risk assessment”. The retrieval time limit will range from each corresponding database’s inception until May 20, 2024 (updated until January 31, 2025). The search will be performed by 2 authors (WYK and ZXL). We will perform both forward and backward citation searches for the included studies and relevant previous systematic reviews. The complete search strings are presented in Multimedia Appendix 2.

EndNote’s “Delete duplicates” function will be used, followed by the manual deletion of any remaining duplicates. Subsequently, the authors (ZXL and WYK) will evaluate the acceptability of the titles and abstracts. In case of an eventual disagreement between the authors who analyze the eligibility of the documents, the opinion of a third author (LLJ) will be considered. Any inconsistencies will be resolved through discussion to arrive at a consensus. Thereafter, the selected full texts will finally be included in the review.

The following information will be identified from each selected study and summarized in tables for qualitative analysis: author name, country, the aim of the study, model development sample size, external validation sample size, internal validation sample size, participants, follow-up, study design (prospective study or retrospective study), events per variable, the main outcome, predictors in the final model, modeling methodology, number of models, model characteristics (ie, modeling methods, model validation methods, and candidate predictors) and model performances (ie, discrimination, calibration, and classification measures; Table 2). Data will be extracted independently by 2 authors (ZXL and WYK), who will enter the data into different Microsoft Excel spreadsheets. A third author will verify their agreement and resolve disagreements by reanalyzing the data (LLJ). A further limitation of the planned review is that we will not contact study authors to request unreported information, as we will explicitly document instances where information is missing.

Two authors (ZXL and WYK) will independently use the CHARMS checklist [25] to extract the relevant key information. Candidate studies with uncertain model characteristics (ie, modeling methods, model validation methods, and candidate predictors) and model performances (ie, discrimination, calibration, and classification measures) will be discussed with a third author (LLJ) before inclusion.

We will use the Prediction Model Risk of Bias Assessment Tool (PROBAST) to evaluate the quality of the candidate studies, as evaluated independently by 2 authors (ZXL and WYK). Similarly, uncertain cases will be discussed with a third author (LLJ) before inclusion. PROBAST [27] comprises 4 domains, namely, participants, predictors, outcome, and analysis. Each domain includes 20 signaling questions used to evaluate the risk of bias and applicability. These signaling questions rely on factual information and are categorized as either “Yes” or “Probably yes” (Y), “No” or “Probably no” (N), or “No information” (NI). After evaluating each domain using signaling questions specific to that domain, they will be categorized as either “High,” “Low,” or “Unclear.” By synthesizing the results obtained from each dimension, an overall assessment will be made regarding the predictive models’ risk of bias and applicability.

The evaluation of the predictive models’ performance will be based on an assessment of their discrimination and calibration [28]. “Discrimination” refers to a model’s ability to distinguish between individuals with different outcomes or conditions. It assesses how well the model correctly ranks individuals based on their risk of experiencing a certain event or condition. Common methods for evaluating discrimination include the c-statistic (also referred to as the area under the receiver operating characteristic curve) and the concordance index [28]. “Calibration” refers to the conformity between a model’s predicted probabilities or risk evaluations and the observed actual outcomes. Calibration is often assessed by comparing the predicted and observed event rates among different risk groups, or by using calibration plots [20,28].

Ethics approval is not required for this systematic review, as it does not require primary data collection. The protocol will be registered with the PROSPERO. The results of this systematic review will be disseminated through publication in an academic journal and scientific conferences.

This protocol aims to provide a detailed description of the process for conducting a systematic review of existing global predictive models for PODOCVS, focusing on their predictive performance, applicability, and quality. Due to differences in the basic characteristics of study populations, protocols, surgical methods, and evaluation methods and frequency, the reported incidence of PODOCVS varies greatly [29-31]. PODOCVS can cause serious adverse outcomes [32], and prevention is the most effective strategy for minimizing its occurrence and poor prognosis [2]. Accurate risk estimation for surgical patients using predictive models can aid clinical decision-making and inform policy, thereby guiding the optimal allocation of often-limited resources. By identifying, summarizing, and evaluating published PODOCVS prediction models, this systematic review will serve as a comprehensive resource for PODOCVS stakeholders, including health care workers, policy makers, and researchers. Although numerous PODOCVS prediction models exist, most of them have not been developed, validated, and assessed according to established guidelines for predictive research. This has given rise to significant biases in PODOCVS risk estimation, deficiencies in the statistical methods used, and a lack of internal and external validation.

While the clinical outcomes of PODOCVS are heterogeneous, many predictors of poor clinical outcomes have been identified. These include age (≥60 years old), sex (male), education level, history of living alone, frailty, BMI (>30 kg/m²), high EuroSCORE (≥14), disease severity, changes in lifestyle patterns before and after admission, personality traits (high irritability), and history of smoking and alcohol consumption [33-35]. These interrelated factors’ relative influence on patient outcomes can be described through multivariable modeling and subsequently used to construct a predictive model to estimate patients with cardiac surgery risk of developing postoperative delirium [36]. Therefore, to ensure the validity of the included predictive models, we will only select models that are internally or externally validated during development. Studies focused solely on the external validation of existing models or otherwise unvalidated predictive models will be excluded since several existing externally validated models have exhibited significant variability in their predictive accuracy across different populations. The reason for this may be that postoperative delirium in patients with cardiac surgery is closely related to the type of surgery that patients receive [2,37]. The planned systematic review will concisely summarize key information presented across all identified predictive model studies. Patients with cardiac surgery health care providers and policy makers can then use this information to assess a model’s applicability to the patients with cardiac surgery population in different scenarios.

In recent years, ML-based predictive models have shown great success [38]. ML models can process vast quantities of multidimensional and unstructured data to reveal the roles and relationships between multiple variables, providing an effective means for accurately predicting delirium risk in patients with cardiac surgery [9,39,40]. Although ML-based predictive models exhibit good predictive performance, research has shown that they lack explanatory power. The reason for this may be that patients’ clinical data are not uniform and standardized, and key data are often omitted [41]. Which of the 2 model types (ie, ML-based or statistical models) exhibits superior discrimination and calibration in the prediction of PODOCVS risk has not been demonstrated. To some extent, this affects the selection and use of predictive models by clinical medical staff. Therefore, this review will also analyze whether ML models can provide more accurate predictions as compared with traditional statistical models.

Strengths of this study include its strict adherence to the updated PRISMA 2020 checklist and its planned adherence to the CHARMS checklist. The use of a consensus-based analytical approach fosters collective accountability in interpretative decisions, thereby strengthening the robustness and credibility of the findings.

Some limitations of the study warrant consideration. We will only conduct qualitative descriptions and not perform a meta-analysis. We anticipate that inconsistencies in the predictors included in the models, follow-up duration, clinical settings in which the models were developed, types of cardiovascular surgery, and geographic distribution of the study populations may pose challenges to conducting a meta-analysis. Even if each predictive model we included has undergone internal or external validation, its limited clinical application necessitates further evaluation of its generalizability to ensure both applicability and scientific rigor. Although the study uses a comprehensive methodology, one limitation is the omission of a search for grey literature, which could potentially result in the exclusion of studies reported in nontraditional publications. Another limitation of the planned review is that we will not contact study authors to request unreported data, although missing information will be explicitly documented.

We present a protocol for the systematic review of prediction models of postoperative delirium for patients with cardiac surgery. With the aim of identifying, summarizing, and appraising the available risk models, we hope to provide a current reference to stakeholders engaged in patients with cardiac surgery care, policy, and research. In addition, we also hope to explore whether PODOCVS ML models can provide more accurate predictions as compared with PODOCVS traditional statistical models.