Clinical decision support systems (CDSS) play an important role in health care by providing evidence-based guidance and recommendations to clinical staff. Typical use cases include medication management, disease diagnosis and management, treatment planning, risk assessment, and workflow optimization. The use of these systems aims to increase the accuracy and effectiveness of clinical staff [1,2].

The increasing digital transformation of health care means that more and more health care data is available in digital form, which means that artificial intelligence (AI) applications are also becoming more relevant in this area. More and more AI applications are being developed in health care. In clinical practice, AI has a wide range of potential applications, including disease diagnosis, treatment selection, and patient monitoring [3]. AI-based CDSS is essential in this context.

Although studies have shown that CDSS can reduce medical errors and improve outcomes, they have also shown that CDSS are not being used to their full potential [4-10]. It can be assumed that the challenges of non–AI-based CDSS also apply to AI-based CDSS. Further challenges arise with respect to AI-based CDSS [11]: pointed out in the viewpoint is that the use of deep learning and other analytic methods brings additional challenges. These methods generate insights in ways that are not directly traceable, meaning that clinical staff cannot apply the same validation as with traditional clinical decision support tools. As a result of this lack of transparency, trust in the AI system may decrease [11,12].

Another challenge is integration into workflows [13,14]. When developing AI-based technologies for clinical use, it is crucial to consider existing workflows in both the design and development phases. This will ensure that the technology can be used effectively and make a positive contribution to patient care. Consideration of workflows and the needs of clinical staff will ensure the successful integration of AI technologies into everyday clinical practice. Poor integration processes can have a negative impact on uptake and adoption, as illustrated by a case study on the implementation of a CDSS [14]. For successful integration, it is essential that clinical staff develop the skills to interpret the results appropriately. This “black box” nature of AI described by [11] and the lack of transparency of the basis for decision making can make practical implementation difficult and may also be a factor in low user adoption [11,15].

The notion of the user’s acceptance of new technologies is derived from the Theory of Reasoned Action [16]. According to various technology acceptance models (TAMs; eg, TAM and UTAUT [Unified Theory of Acceptance and Use of Technology]) based on this theory, acceptance encompasses both the intention to use technology and its influence on actual use behavior [4,17,18]. For researching clinicians’ acceptance of CDSS these models serve as a foundation [19-22].

Acceptance of CDSSs is crucial for their successful use. Involving users at an early stage can improve the acceptance and use of information technology by taking into account their needs, preferences, and experiences. This can help to optimize the user experience and increase the effectiveness of the technologies [15,17].

Involving future users is possible through participatory research approaches. Such approaches aim to plan and carry out research processes with people who investigate their social world and meaningful actions as lifeworld-situated living and working practices that are to be improved by developing appropriate innovations [23]. In addition, a participatory approach may also help to avoid problems that arise when transferring the AI-based CDSS to new patient populations for instance due to overfitting of training data or lack of generalizability [24]. This problem can be counteracted by involving expert knowledge from practice, for instance by reviewing the operationalization of health care–related concepts, feature selection, and data quality.

The stage of participation can be determined using the Wright et al [25] stage model. This model consists of 9 stages and is divided into four areas: (1) nonparticipation (stages 12), (2) preparticipation (stages 3-5), (3) participation (stages 6-8), and (4) beyond participation (stage 9). It is also used to determine the stage of participation in the studies reviewed. In order to be able to categorize the studies, particular attention should be paid to the methodological description of the studies. The second version of the GRIPP (Guidance for Reporting Involvement of Patients and Public), a tool to improve the reporting of patient and public involvement in research, shows in its fourth section of the long-form reporting checklist on the methodology of the work (design, people involved, stages of involvement, and level or type of involvement) the relevant aspects for describing the involvement of groups of people [26]. The content can also be transferred to other stakeholder groups (not only patient and public involvement). An alternative to the Wright et al [25] stage model and GRIPP 2 could have been the participatory ergonomics framework, which focuses on the active involvement of participants, particularly workers, in ergonomic interventions to improve work conditions and processes [27]. However, the Wright model and GRIPP 2 were selected because of their established use in health care, their broad applicability, their transferability to different stakeholder groups, and their alignment with the objectives of this review.

According to a recent review, clinical professionals (future users) are already involved in the development of AI-based CDSS, but only in about 30% of cases. The focus is on the creation of predictive CDSS specifications or the evaluation of system implementations. However, clinical experts are less likely to be involved in the development phases to check clinical validity, select model features, process data, or act as a gold standard [28].

Nevertheless, the development of AI applications poses challenges to participatory methods: A precondition for carrying out participatory methods on the topic of AI is that the participants have a basic understanding of AI. Furthermore, implementing design ideas is difficult, as a realistic prototype is often hard to realize. In addition, evaluating the results is only possible to a limited extent. Much has to be done via simulations or imagination because it often requires a long testing period. Another difficulty is often the lack of comprehensibility of the AI decisions for the user [29].

Although clinical staff are already involved in the research and technology development process of AI-based CDSS, no overarching overview has been identified that summarizes and presents the main approaches, methods, and practices for participatory research and technology development of AI-based CDSS for clinical staff. Hence, this gap of knowledge should be addressed by this scoping review.

A preliminary search of MEDLINE was conducted, and no current or ongoing systematic reviews or scoping reviews on the topic were identified.

The objective of the review is to provide an overview and systematization of participatory approaches of various disciplines for developing, piloting, implementing, and evaluating AI-based information systems in a clinical setting.

The following research questions will be addressed:

We are going to conduct one scoping review. The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) is used as a basic tool [30] in combination with the Joanna Briggs Institute (JBI) approach to scoping reviews. This approach ensures that the scoping review is transparent, reproducible, and methodologically sound [31].

The inclusion and exclusion criteria that were applied to the studies are shown in Textbox 2.

The retrieved references will be checked for duplicates and transmitted to Covidence (software for managing and streamlining reviews, operated by Veritas Health Innovation Ltd) [40] for the screening steps. We will use Zotero (a free and open-source literature management program) [41] as a bibliographic tool. Two independent reviewers (TR and PG) will screen all titles and abstracts separately for inclusion or exclusion. Disagreements will be solved by including a third reviewer. Afterward, the same procedure will be applied to the full-text screening, which is carried out by 3 independent reviewers (TR, PG, and CH). Reasons for excluding a study will be assessed in each of these steps. The results of the search and the study inclusion process will be reported in full in the final scoping review and presented in a PRISMA-ScR flow diagram [30].

A data charting form has been developed jointly by the authors to identify the variables to be extracted. The 2 reviewers will chart the data independently, discuss the results, and continually update the data charting form in an iterative process, with changes detailed in the scoping review. Any reviewer disagreements will be resolved by discussion or with additional reviewers. Where appropriate, authors of papers will be contacted to request missing or additional data as required. A draft extraction form is provided (Multimedia Appendix 2). Data on the participation process is also extracted to assess the stage of participation.

Data Analysis and presentation will follow the recommendations of the Joanna Briggs Institute scoping review methodology group [42]. First, the extracted data will be presented in a logical and descriptive way (diagrams and tables), guided by the objectives and questions of the scoping review. Additional relevant data items may be identified during the data extraction process. If additional items are extracted that were not prespecified in the review protocol, this will be made clear in the final report together with a rationale as to why it occurred. The extracted data will be summarized in a narrative synthesis to bring together findings relating to challenges and facilitators for participatory design processes. Given the breadth of scoping review questions, the analysis will also use qualitative content analysis. In order to identify and structure relevant aspects of the research questions, the analysis will follow an inductive approach. Following an open coding process, a coding framework will be developed and reviewed by all authors. This approach aims to provide insights into participatory design and research practice for AI-based CDSS and highlight areas for future research.

The main objective of this study is to identify, clarify, and map key approaches, methods, and procedures for participatory research and technology development of AI-based CDSS in the context of clinical staff. It will also analyze, synthesize, and develop existing approaches, concepts, and conceptualizations. The insights gained from this process will serve as a basis for designing, developing, and testing participatory processes specifically designed for clinical staff. Various stakeholders can use the results to design, develop, and review participatory processes that address the development of an AI-based CDSS for clinical staff.

Scoping reviews have limitations, particularly in that they focus on the collection and synthesis of data and do not assess the strength of evidence or the risk of bias in the research. Therefore, further research is needed to assess and analyze the quality of existing studies on the participatory development of AI-based decision support systems. It should be noted that only papers written in English or German were considered, which meant that potentially relevant studies in other languages could not be included. Furthermore, despite the comprehensive inclusion criteria, some relevant sources of information may not be included.

Up to now, no review of this scope and objective has been identified. Hence, this review will be the first to address this specific knowledge gap targeting clinical staff. Additionally, one aim of this review is to identify further research and knowledge gaps and to give hints where further reviews would be helpful.