There is growing interest in artificial intelligence (AI) in medical research and clinical practice. Judicious use of algorithms could potentially improve the quality of care and accessibility to health services [1,2]. AI in health care is mostly used for training supervised models on previously collected data [1,3]. Some health specialties benefit from these tools that allow the application of AI models safely with a controlled risk of bias. For instance, the use of AI in medical imaging allows an improvement in the detection of some anomalies [4,5], and initiatives in radiology seem to improve efficiency and health outcomes [6]. Similarly, AI models could potentially have a positive impact on populations in vulnerable situations who access the health care system through primary care [1,7]. Unfortunately, AI remains poorly integrated into primary and community health services [8].

A potential use and implementation of AI models in the primary health care context could be in citizen portals by promoting health-oriented behaviors and preventing the onset of chronic diseases [8]. As elsewhere in Canada and worldwide, such portals are currently implemented in some electronic medical records in the province of Québec and serve as an interface between clinicians and patients [9]. These portals can provide access to laboratory results, reports of health questionnaires, or communication with health care professionals.

Citizen portals with integrated AI could lead to virtual assistance throughout the health care system or in situations where vulnerable populations do not have direct and rapid access to physicians’ advice (eg, remote, isolated, vulnerable, and stigmatized populations) [1,2]. However, since these AI models are deployed in communities and vulnerable populations are concerned, there are risks of biases affecting patient care quality and safety, and their sources must be explored and identified prior to implementation [10]. We need to take a specific interest in medical and ethical parameters to ensure user quality and safety, as these models may be biased especially by ethnicity, sex, or gender [11-14]. Moreover, inequitable access to resources and recommendations could potentially put some users at risk [10].

As defined by the Canadian Institutes of Health Research, community-based primary health care (CBPHC) “covers the broad range of primary prevention (including public health) and primary care services within the community, including health promotion and disease prevention; the diagnosis, treatment, and management of chronic and episodic illness; rehabilitation support; and end of life care” [15]. The current literature identifies several potential benefits of AI to develop and integrate clinical tools for population health, care team well-being, and efficiency of the CBPHC system [1,16-18]. Integration of AI technologies into CBPHC could help in many ways especially “to help people to get the care (including prevention) they need” [15]. However, there is a lack of understanding on how the risk of bias is considered in the development of these AI models and to what extent they perpetuate or introduce potential biases toward groups that could be considered vulnerable in terms of their characteristics (eg, sex, gender identity, sexual orientation, race, ethnicity, socioeconomic status) [16,19]. In response to this concern, the US Food and Drug Administration recently “calls for efforts to develop a methodology to develop devices to be well suited for a racially and ethnically diverse patient population” [1,16].

In a recent scoping review with the objective to “assess the nature and extent of the body of research on AI to primary care,” the authors concluded that AI “is at an early stage of maturity (...) and more evaluation studies are needed” [7]. To the best of our knowledge, no reviews are available on methods to assess risk of bias and health inequity in primary care algorithms. We recently found a scoping review protocol aiming “to summarize the extent to which AI systems in primary care examine the inherent bias toward or against vulnerable populations and appraise how these systems have mitigated the impact of such biases during their development” [20]. Unless we are mistaken, the results of this protocol have not yet been published, and we still lack an overview of bias mitigation interventions and the characterization of groups and subgroups to which they are applied in CBPHC.

The aim of this rapid qualitative systematic review is to identify relevant methods (eg, frameworks, tools, checklists) for a risk of bias assessment for vulnerable or diverse groups in the development or deployment of algorithms in CBPHC and identify mitigation strategies deployed to promote and increase equity, diversity, and inclusion in CBPHC algorithms.

The research questions are:

This protocol follows the Cochrane Collaboration guidance for rapid reviews [21] and the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) guidelines [22,23]. PRISMA-P guidelines recommend the use of PICO (Participant, Intervention, Comparator, and Outcome) to frame the review question and develop a search strategy [22,23]. However, other options are available depending on the review objective. For a scoping review, the Joanna Briggs Institute recommends the PCC (Population or Participant, Concept, and Context) search framework [24]. Although we are not in a scoping review process, the PCC search framework [25] seems more appropriate and consistent with our open-ended qualitative research question (Table 1).

In November 2022, an information specialist developed a search strategy based on the main concepts of our primary review question and adapted it to the most relevant databases (PubMed, CINAHL, Web of Science, and PsychINFO). We used five relevant articles to test the sensitivity, and the search covered the last 5 years (2017-2022) aligned with the concept’s emergence time frame. The database search strategies can be found in Multimedia Appendix 1.

We include all studies on methods developed or tested to assess risk of bias in algorithms that are relevant in CBPHC. In December 2022, all citations were exported to the web-based collaborative tool Covidence (Veritas Health Innovation) [26] where 581 duplicates were removed by the automated function. Since February 2023, two reviewers have independently screened the titles and abstracts of the identified records (N=1022) and assessed the inclusion and exclusion criteria (Table 2). Conflicts on study selection are resolved by consensus, and the opinion of a senior researcher is sought if required. At the full-text assessment step, two reviewers (and the information specialist if required) will search and obtain all the full texts of the selected references, and they will import the PDF files into Covidence.

Team members will complete the extraction on a structured prespecified grid in Covidence, and the data will be reviewed and pretested by the research team. We will extract descriptive data (title, year of publication, authors, country), study type (published, study design), and population data (populations targeted by the intervention, number of participants). Characteristics of populations will be mainly extracted to our prespecified grid according to the PROGRESS (place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, and social capital) framework [27]. This framework addresses “health inequalities and unfair differences in disease burden,” and these characteristics “that stratify health opportunities and outcomes” are “place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, and social capital” [27]. We will add to that list the “Plus” in the acronym PROGRESS-Plus, referring to the “1) personal characteristics associated with discrimination (e.g. age, disability) 2) features of relationships (e.g. smoking parents, excluded from school) 3) time-dependent relationships (e.g. leaving the hospital, respite care, other instances where a person may be temporarily at a disadvantage)” [28].

We will also extract concept data: identification and characteristics of the assessment method used/developed; identification and characteristics of the algorithm, including purpose (eg, diagnosis- or prognosis-related, used for disease detection or surveillance), type (eg, machine learning, natural language processing, expert systems), and any other available detail (eg, input information/parameters, outcomes); and context data, including details about the specific settings (ie, identification of potential biases for vulnerable or diverse groups) and the results of the intervention (eg, bias mitigation data).

We will appraise the quality of empirical studies included based on experiment, observation, or simulation (randomized controlled trial, quasi–randomized controlled trial, prospective cohort study, pre-post study, observational study, mixed methods study, qualitative study) by applying the Mixed Methods Appraisal Tool (MMAT) [29].

We will synthesize data using structured narrative summaries around our main concept/context and provide a list of available methods. The development process and main features of the methods (eg, potential sources of bias/items covered, signaling questions, differences and similarities, strengths and weaknesses) will be described. Furthermore, we will present mitigation strategies deployed with these methods (eg, algorithm parameter adjustments, mitigation or elimination of bias).

We obtained approval by the Ethics board of the “Comité d’éthique de la recherche sectoriel en Santé des Populations et Première Ligne du CIUSSS de la Capitale-Nationale” for the Protecting and Engaging Vulnerable Populations in the Development of Predictive Models in Primary Health Care for Inclusive, Diverse and Equitable AI (PREMIA) project (#2023-2726).

We anticipate structured narrative summaries of identified bias mitigation methods would be useful to researchers, data scientists, ethicists, and other CBPHC providers. We also anticipate that identification and documentation of these methods would be particularly relevant for multidisciplinary teams: “AI initiatives should have two main goals: they should be designed and utilized in a manner that does not create or maintain health disparities currently experienced by vulnerable groups, and they should address and remove existing health disparities” [1]. We hypothesize that giving methods and tools to multidisciplinary teams, with the participation of vulnerable populations in the entire process, would be useful to achieve inclusive AI when applied to health. We also expect that few studies will present quantitative bias mitigation data and results. Nevertheless, we will present these data in qualitative structured summaries.

Other reviews generally focused on a few specific population characteristics. In a recent study on hospital readmission rates, the authors focused on racial and socioeconomic status biases in algorithms due to an increased risk in these groups [19]. In another example, the authors explored the source of age-related bias in AI systems [30]. There are many other “subgroups or unprotected groups” that can be considered to address “health inequalities and unfair differences in disease burden” [31]. We expect that our focus on primary health care will offer a different insight and that the PROGRESS-Plus framework will be helpful to identify potential interventions to mitigate health inequalities [27,28]. We anticipate listing and describing which vulnerable or diverse groups are considered (and which could be considered).

The main strength of our review is that we are applying an equity lens [27,28] to describe mitigation strategies in primary health care AI models. However, this review has three main limitations. First, its rapid nature implies that we limit our search to the last 5 years and to 4 databases only. Some relevant studies might not be included. However, given the recent interest in AI in primary care, this is unlikely. Second, we will consider all types of health conditions and various types of populations, which could bring important heterogeneity in the results and limit the possibility of synthesizing this information. Finally, applying the findings of this review to existing AI-based algorithms could present challenges because of the different contexts and populations in which they were developed.

This rapid review is the first step of the larger PREMIA project. This work will be presented to key stakeholders, and we will conduct qualitative data collection to identify and evaluate biases in one AI model. Giving a voice to diverse groups [32,33] at the development or deployment stages of CBPHC AI-based algorithms could have a substantial impact on this review. Recommendations will be implemented into tools able to guide developers, researchers, and decision makers on strategies to reduce the risk of bias in AI primary health care programs and services, with the objective to protect vulnerable groups and not further exacerbate inequalities.

Identifying methods developed and deployed to mitigate biases toward vulnerable populations is needed to develop equitable primary health care. AI models can have many benefits for patients, but factors that stratify health opportunities and outcomes need to be addressed to mitigate inequalities toward members of vulnerable or diverse groups. To our knowledge, no reviews are currently available to identify risk of bias mitigation strategies in CBPHC AI algorithms. This rapid review will contribute to providing comprehensive structured narrative summaries of risk of bias assessment methods. This knowledge could be useful to CBPHC researchers and other stakeholders to identify potential sources of bias in AI algorithm development and eventually try to reduce or eliminate them.