Published on in Vol 13 (2024)

This is a member publication of University of Birmingham (Jisc)

Preprints (earlier versions) of this paper are available at, first published .
AI as a Medical Device for Ophthalmic Imaging in Europe, Australia, and the United States: Protocol for a Systematic Scoping Review of Regulated Devices

AI as a Medical Device for Ophthalmic Imaging in Europe, Australia, and the United States: Protocol for a Systematic Scoping Review of Regulated Devices

AI as a Medical Device for Ophthalmic Imaging in Europe, Australia, and the United States: Protocol for a Systematic Scoping Review of Regulated Devices


1Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom

2Oxford Eye Hospital, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom

3Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, United Kingdom

4Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHS Foundation Trust, Birmingham, United Kingdom

5Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom

6Sydney Eye Hospital, Sydney, Australia

7Retinal Consultants Medical Group, Sacramento, CA, United States

8NIHR Birmingham Biomedical Research Centre, Birmingham, United Kingdom

9Centre for Regulatory Science and Innovation, Birmingham Health Partners, Birmingham, United Kingdom

10Institute of Ophthalmology, University College London, London, United Kingdom

11NIHR Moorfields Biomedical Research Centre, London, United Kingdom

Corresponding Author:

Alastair K Denniston, PhD

Institute of Inflammation and Ageing

University of Birmingham

College of Medical and Dental Sciences


Birmingham, B15 2TT

United Kingdom

Phone: 44 01213716905


Background: Artificial intelligence as a medical device (AIaMD) has the potential to transform many aspects of ophthalmic care, such as improving accuracy and speed of diagnosis, addressing capacity issues in high-volume areas such as screening, and detecting novel biomarkers of systemic disease in the eye (oculomics). In order to ensure that such tools are safe for the target population and achieve their intended purpose, it is important that these AIaMD have adequate clinical evaluation to support any regulatory decision. Currently, the evidential requirements for regulatory approval are less clear for AIaMD compared to more established interventions such as drugs or medical devices. There is therefore value in understanding the level of evidence that underpins AIaMD currently on the market, as a step toward identifying what the best practices might be in this area. In this systematic scoping review, we will focus on AIaMD that contributes to clinical decision-making (relating to screening, diagnosis, prognosis, and treatment) in the context of ophthalmic imaging.

Objective: This study aims to identify regulator-approved AIaMD for ophthalmic imaging in Europe, Australia, and the United States; report the characteristics of these devices and their regulatory approvals; and report the available evidence underpinning these AIaMD.

Methods: The Food and Drug Administration (United States), the Australian Register of Therapeutic Goods (Australia), the Medicines and Healthcare products Regulatory Agency (United Kingdom), and the European Database on Medical Devices (European Union) regulatory databases will be searched for ophthalmic imaging AIaMD through a snowballing approach. PubMed and clinical trial registries will be systematically searched, and manufacturers will be directly contacted for studies investigating the effectiveness of eligible AIaMD. Preliminary regulatory database searches, evidence searches, screening, data extraction, and methodological quality assessment will be undertaken by 2 independent review authors and arbitrated by a third at each stage of the process.

Results: Preliminary searches were conducted in February 2023. Data extraction, data synthesis, and assessment of methodological quality commenced in October 2023. The review is on track to be completed and submitted for peer review by April 2024.

Conclusions: This systematic review will provide greater clarity on ophthalmic imaging AIaMD that have achieved regulatory approval as well as the evidence that underpins them. This should help adopters understand the range of tools available and whether they can be safely incorporated into their clinical workflow, and it should also support developers in navigating regulatory approval more efficiently.

International Registered Report Identifier (IRRID): DERR1-10.2196/52602

JMIR Res Protoc 2024;13:e52602




There is a growing capacity-demand mismatch within ophthalmology, increasing the risk of sight loss from treatment delays [1,2]. Artificial intelligence (AI) has the potential to help address these challenges. AI’s strength lies in its ability to produce high-throughput analyses and glean meaningful insights from complex multimodal and multidimensional data sets through pattern recognition. This is well aligned with ophthalmology services, where disease diagnosis and management depend heavily on multimodal imaging [3]. AI therefore has the potential to help improve speed and access to care at reduced costs.

However, despite the exponential increase in the number of AI as a medical device (AIaMD) being developed and receiving regulatory approval, relatively few have been seamlessly integrated into routine clinical practice [4-6]. This so-called “AI chasm” limits the deployment of AIaMD to achieve patient benefit at scale [7]. This AI chasm results from a wide range of interdependent factors at the policy, organizational, and individual levels [8-13]. Key elements include ensuring adequate clinical evaluation to support regulatory decisions, such that the evidence base underpinning such tools is aligned with their intended use and is safe for the target population, and also clarifying how these regulatory requirements align with commissioners’ needs.

The evidential requirements for software (including AIaMD) may be more ambiguous compared to more established interventions, making it more difficult for AI developers to explicitly understand the nature and extent of evidence they need to generate to gain regulatory approval. Attempts to study regulator approved AIaMD are impeded by the usability of public databases as well as the private nature of much of the information submitted by applicants, making it difficult for researchers, clinicians, or commissioners attempting to understand the evidence underpinning approved AIaMD. A review of Food and Drug Administration (FDA) approvals in the United States found that few submissions included comparisons between AI and human performance and that only a small proportion reported prospective data [14]. The reporting of sample size and number of sites in the validation studies was generally poor. The review did not assess whether participant characteristics such as gender and ethnicity were reported. Although guidelines for performing and presenting AI studies have been developed [15-18], there is no clear “best practice” for providers to ensure the safety and effectiveness of the AIaMD they adopt [14].

Review Objectives

This scoping review will focus on AIaMD that contributes to clinical decision-making (relating to screening, diagnosis, prognosis, and treatment) using ophthalmic imaging as an input. The aim is to identify and characterize AIaMD for ophthalmic imaging, which have received regulatory approval in 4 countries with established regulatory pathways for clinical use, in order to support providers in procurement decisions and developers in generating evidence to support applications to regulators.

The objectives are as follows:

  1. To identify regulator-approved AIaMD for ophthalmic imaging in Europe, Australia, and the United States
  2. To report the characteristics of those AIaMD and the regulatory approvals granted to them
  3. To report the available evidence for the effectiveness and efficacy of approved AIaMD

Protocol Registration and Reporting

The protocol and subsequent review will adhere to the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) [19] and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) [20] reporting guidelines, respectively (note that PRISMA-AI [21] is still in development). The protocol was registered with the Open Science Framework’s website.

Eligibility Criteria

This review will focus on AIaMD for ophthalmic imaging that helps inform clinical management. All AIaMD for ophthalmic imaging where the AIaMD or its manufacturer are recorded in the US FDA, Medicines and Healthcare products Regulatory Agency (MHRA), European Database on Medical Devices (EUDAMED), or Australian Register of Therapeutic Goods (ARTG) databases will be included. All 4 countries are members of the International Medical Regulators Device Forum and have a track record of admitting AIaMD to their markets. No restrictions will be placed on the type of ophthalmic imaging modality involved or the intended use of the AIaMD.

The AIaMD will have a partial or fully data-led mechanism (eg, regression modeling, random forest, or convolutional neural networks). Any AIaMD that exclusively uses rule-based mechanisms (eg, a priori decision trees, best practice alerts, and normal or abnormal threshold alerts) will be excluded.

With regard to the evidence underpinning each AIaMD, only original research that comprises a clinical evaluation of the AIaMD in human participants will be included. This may include randomized controlled trials or prospective or retrospective observational studies. Systematic reviews and meta-analyses, case series, case reports, commentaries, and expert opinions will not be eligible. No date or language restrictions will be applied to the electronic search.

Search Strategy and Sources of Information

To identify potentially eligible AIaMD, the FDA (United States), ARTG (Australia), MHRA (United Kingdom), and EUDAMED (European Union) regulatory databases will be searched through a snowballing approach [22]. This will involve an exhaustive review of the product class codes and predicate devices (if applicable) with which each known eligible device is associated. This strategy has been adopted due to limitations in the search functionality of these databases. No AI tools will be used to assist the search.

The snowball search will commence with a list of 15 AIaMD for ophthalmic imaging (Textbox 1). This represents the sum of the authors’ awareness of regulated products and a pragmatic search of relevant academic literature [23].

Textbox 1. Initial list of ophthalmic imaging artificial intelligence as a medical device (AIaMD) used in snowball search.

Ophthalmic imaging AIaMD

  • LumineticsCore (previously known as IDx-DR), Digital Diagnostics
  • Eyeart, Eyenuk Inc
  • RetmarkerDR, Retmarker SA
  • SELENA+, eyRIS Pte Ltd
  • Automated Retinal Disease Assessment, Verily Life Sciences
  • Medios AI, Remidio
  • OphtAI, Evolucare, ACDIS
  • RetCAD, Thirona Retina B.V.
  • DeepDee AI, DeepDee
  • Eyetelligence, Eyetelligence Pty Ltd
  • CARA, Diagnos
  • RetinaLyze, RetinaLyze System A/S (Ltd)

Next, PubMed,, and the International Clinical Trials Registry Platform will be systematically searched for each eligible AIaMD and its manufacturer by combining both search terms with an “OR” Boolean operator. These searches will be limited to relevant ophthalmology-specific studies using relevant key terms such as “retin*” for AIaMD relating to diabetic retinopathy screening.

In addition, manufacturers’ websites will also be reviewed for any peer-reviewed publications. The manufacturers of all eligible AIaMD will be contacted directly for clarification and as an additional source of peer-reviewed publications and ongoing studies. A preliminary scoping search highlighted that not all studies mention the AI device name or manufacturer, and some devices undergo a name change from one version to the next. Hence, this additional search will ensure that the data captured are as comprehensive as possible.

Study Selection

Two authors will search the regulatory databases independently. They will come to a consensus decision about the eligibility of any AIaMD identified. Any unresolved disagreements will be arbitrated by a third author. There may be instances where an AIaMD’s eligibility or its regulatory approval status cannot be determined with publicly available evidence, as eligible devices may use proprietary AI that is kept confidential. If this is the case, correspondence with the manufacturer will be undertaken to seek clarification. If further clarification is not possible, the ambiguity about the AIaMD’s eligibility and the rationale for including or excluding it will be recorded.

The search for evidence will be undertaken by an independent author. After deduplication, the titles and abstracts will be independently screened by 2 authors to assess their relevance to the eligible AIaMD. Discrepancies will be resolved by discussion and by arbitration with an additional author if necessary. The full texts will be screened, and a further round of arbitration will take place as needed.

Data Extraction

Data extraction will be undertaken in 2 phases, using standardized data extraction forms designed and piloted for the purposes of this review.

Phase 1

The characteristics of each eligible AIaMD for ophthalmic imaging with regulatory approval in Europe, Australia, and the United States at the time of the search will be obtained. Where available, this will include:

  1. Type of regulator approval and date of approval
  2. Class assigned under FDA, TGA, UK MDR (Medical Devices Regulations 2002) and/or EU MDR (Regulation (EU) 2017/745)
  3. Intended use statement
  4. Ophthalmic imaging modality
  5. Model type and architecture (eg, deep learning with machine learning)
  6. Recall indications on the regulatory databases, as available

Some AIaMD may be approved in 2 or more jurisdictions, and the data will be extracted accordingly. Google searches will be used to supplement data extraction if the required data are unavailable from the FDA, EUDAMED, MHRA, or ARTG databases.

Phase 2

Published evidence underpinning the effectiveness and efficacy of each eligible AIaMD will be obtained. The following data will be extracted from each included study:

  1. Study characteristics: title, author name, publication status, funding source, conflicts of interest, and author affiliations with manufacturers
  2. Study methodology: study duration, study design (randomized, prospective observational, and retrospective observational), etc
  3. Validation: external validation, reference standards, and comparison between AI and humans
  4. Data set or cohort details: source of data set, size of data set or number of participants, setting, number of countries, number of centers, and participant demographics (age, gender, and ethnicity)
  5. Model performance: metrics including sensitivity, specificity, area under the curve, etc, with 95% CIs; clinical outcomes as described

Assessment of Methodological Quality

Two review authors will independently assess the methodological quality of each included clinical validation study. Appropriate quality assessment tools will be used for each study type, for example, the QUADAS-2 tool for evaluating the risk of bias and applicability of primary diagnostic accuracy studies [24], the Cochrane risk of bias-2 tool for randomized controlled trials [25], and the ROBINS-I (Risk of Bias in Nonrandomized Studies of Interventions) tool for nonrandomized studies [26]. QUADAS-AI (quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies) [27] and PROBAST-AI (Prediction Model Risk of Bias Assessment Tool for Artificial Intelligence) [28] are under development at the time of writing but will be used where appropriate if available. Any disagreements will be resolved through discussion or by involving a third reviewer where consensus cannot be reached.

Data Synthesis

Study-Level Data

The extracted data will be synthesized using narrative and tabular approaches. A summary of the findings will be presented, using descriptive statistics to describe the characteristics of the included studies. For example, mean and SD will be used to describe continuous variables, while percentages will be used to describe proportions. If appropriate, a meta-analysis of AIaMD diagnostic accuracy will be considered, but this may not be feasible if there is significant heterogeneity in study methods and AI methodology.

AIaMD-Level Data

The data for each AIaMD will be synthesized to give an overview of the characteristics of its regulatory approval or approvals and its clinical validation studies, again through narrative and tabular approaches.

Ethical Considerations

Ethical approval is not required, as this is a protocol for a systematic scoping review. All relevant data have been published, and no primary or proprietary data will be collected. This decision has been verified with the Newcastle University Ethics Committee.

Preliminary searches were conducted in February 2023, and screening is underway. Data extraction, data synthesis, and assessment of methodological quality commenced in October 2023. We anticipate that the scoping review will be completed and submitted for peer review by April 2024.

This will be the first review to examine and synthesize evidence on AIaMD for ophthalmic imaging. The key aim is to better understand the landscape of AIaMD for ophthalmic imaging on the market and the level of evidence that supports their regulatory approval.

Strengths and Limitations

Due to the limited and variable functionality of regulatory databases, it was not possible to conduct searches with standard systematic review methodologies based on research databases. This has necessitated several design considerations. We have sought to minimize publication bias and improve the completeness of the search process by undertaking a snowballing approach informed by expert knowledge, in combination with a database search. This was necessitated by limitations in the search functionality of the regulatory databases, but we acknowledge that this approach facilitates the mitigation rather than the removal of these limitations. In addition, our PubMed search strategy may be hampered by incomplete reporting or no mention of the AI device’s name or manufacturer. We have attempted to mitigate this by supplementing our search with reviewing manufacturers’ websites, corresponding with manufacturers, and performing adjunct searches of major clinical trial registries in order to ensure that our search is as comprehensive as possible.

Strengths include the international scope of the review. We recognize that there may be differences in regulatory requirements across territories (and in the transparency of reporting of data supporting those applications) and have therefore included 4 major jurisdictions. We have also assembled an international team of authors with representation from the United States, Europe, and Australia to support the interpretation of the data. The inclusion of critical appraisal to identify any methodological variations or shortcomings of existing clinical validation studies is a further strength not common to previous regulatory reviews of AIaMDs, and will help guide any future improvements.


We describe the protocol for a systematic scoping review that seeks to map and examine AIaMD for ophthalmic imaging that has received regulatory approvals for commercial use. We anticipate that our findings may be of interest to ophthalmic professionals, AI model developers, health care commissioners, and policy makers, with the overall aim of improving transparency to help inform safe AIaMD implementation, thereby optimizing patient care.

Data Availability

Data sharing is not applicable to this article, which is a protocol for a scoping review, as no data sets were generated or analyzed.

Conflicts of Interest

HDJH is funded by the National Institute for Health Research through a doctoral fellowship award (NIHR301467) and is on the advisory board for Siloton, an optical coherence tomography imaging equipment manufacturer. PAK has received personal fees from AbbVie, Google Health, Roche, Apellis, Novartis, RetinAI, Bitfount, and nonfinancial support from Bayer outside the submitted work. The remaining authors do not have any conflicts of interest or financial disclosures to declare.

  1. Foot B, MacEwen C. Surveillance of sight loss due to delay in ophthalmic treatment or review: frequency, cause and outcome. Eye (Lond). May 2017;31(5):771-775. [FREE Full text] [CrossRef] [Medline]
  2. Facing workforce shortages and backlogs in the aftermath of COVID-19: the 2022 census of the ophthalmology consultant, trainee and SAS workforce. The Royal College of Ophthalmologists. 2023. URL: https:/​/www.​​wp-content/​uploads/​2023/​03/​2022-Ophthalmology-census-Facing-workforce-shortages-and-backlogs-in-the-aftermath-of-COVID-19.​pdf [accessed 2024-02-22]
  3. Ringel MJ, Tang EM, Tao YK. Advances in multimodal imaging in ophthalmology. Ther Adv Ophthalmol. 2021;13:25158414211002400. [FREE Full text] [CrossRef] [Medline]
  4. Zhang J, Whebell S, Gallifant J, Lertvittayakumjorn P, Mattie H, Budhdeo S, et al. AI for Human Health—Dashboard. URL: [accessed 2023-03-11]
  5. Guo Y, Hao Z, Zhao S, Gong J, Yang F. Artificial intelligence in health care: bibliometric analysis. J Med Internet Res. 2020;22(7):e18228. [FREE Full text] [CrossRef] [Medline]
  6. FDA. Artificial Intelligence and Machine Learning (AI/ML)-enabled medical devices. U.S. Food and Drug Administration. 2022. URL: https:/​/www.​​medical-devices/​software-medical-device-samd/​artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices [accessed 2023-02-19]
  7. Keane PA, Topol EJ. With an eye to AI and autonomous diagnosis. NPJ Digit Med. 2018;1:40. [FREE Full text] [CrossRef] [Medline]
  8. Tseng RMWW, Gunasekeran DV, Tan SSH, Rim TH, Lum E, Tan GSW, et al. Considerations for artificial intelligence real-world implementation in ophthalmology: providers' and patients' perspectives. Asia Pac J Ophthalmol (Phila). 2021;10(3):299-306. [FREE Full text] [CrossRef] [Medline]
  9. Ibrahim H, Liu X, Zariffa N, Morris AD, Denniston AK. Health data poverty: an assailable barrier to equitable digital health care. Lancet Digit Health. 2021;3(4):e260-e265. [FREE Full text] [CrossRef] [Medline]
  10. Aristidou A, Jena R, Topol EJ. Bridging the chasm between AI and clinical implementation. Lancet. 2022;399(10325):620. [CrossRef] [Medline]
  11. Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health. Mar 2021;3(3):e195-e203. [FREE Full text] [CrossRef] [Medline]
  12. Kramer DB, Xu S, Kesselheim AS. How does medical device regulation perform in the United States and the European union? A systematic review. PLoS Med. 2012;9(7):e1001276. [FREE Full text] [CrossRef] [Medline]
  13. Taribagil P, Hogg HJ, Balaskas K, Keane PA. Integrating artificial intelligence into an ophthalmologist's workflow: obstacles and opportunities. Expert Rev Ophthalmol. 2023;18(1):45-56. [FREE Full text] [CrossRef]
  14. Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med. 2021;27(4):582-584. [CrossRef] [Medline]
  15. Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. Sep 2020;26(9):1364-1374. [FREE Full text] [CrossRef] [Medline]
  16. Rivera SC, Liu X, Chan AW, Denniston AK, Calvert MJ, SPIRIT-AI and CONSORT-AI Working Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension. BMJ. Sep 09, 2020;370:m3210. [FREE Full text] [CrossRef] [Medline]
  17. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924-933. [FREE Full text] [CrossRef] [Medline]
  18. Sounderajah V, Ashrafian H, Golub RM, Shetty S, De Fauw J, Hooft L, et al. Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol. BMJ Open. 2021;11(6):e047709. [FREE Full text] [CrossRef] [Medline]
  19. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. PRISMA-P Group. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. Jan 02, 2015;350:g7647. [FREE Full text] [CrossRef] [Medline]
  20. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [FREE Full text] [CrossRef] [Medline]
  21. Cacciamani GE, Chu TN, Sanford DI, Abreu A, Duddalwar V, Oberai A, et al. PRISMA AI reporting guidelines for systematic reviews and meta-analyses on AI in healthcare. Nat Med. 2023;29(1):14-15. [CrossRef] [Medline]
  22. Wohlin C, Kalinowski M, Romero Felizardo KR, Mendes E. Successful combination of database search and snowballing for identification of primary studies in systematic literature studies. Inf Softw Technol. 2022;147:106908. [FREE Full text] [CrossRef]
  23. Grzybowski A, Brona P. Approval and certification of ophthalmic AI devices in the European Union. Ophthalmol Ther. Apr 2023;12(2):633-638. [FREE Full text] [CrossRef] [Medline]
  24. Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. Oct 18, 2011;155(8):529-536. [FREE Full text] [CrossRef] [Medline]
  25. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. Aug 28, 2019;366:l4898. [FREE Full text] [CrossRef] [Medline]
  26. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. Oct 12, 2016;355:i4919. [FREE Full text] [CrossRef] [Medline]
  27. Sounderajah V, Ashrafian H, Rose S, Shah NH, Ghassemi M, Golub R, et al. A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI. Nat Med. 2021;27(10):1663-1665. [CrossRef] [Medline]
  28. Collins GS, Dhiman P, Navarro CLA, Ma J, Hooft L, Reitsma JB, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021;11(7):e048008. [FREE Full text] [CrossRef] [Medline]

AI: artificial intelligence
AIaMD: artificial intelligence as a medical device
ARTG: Australian Register of Therapeutic Goods
EUDAMED: European Database on Medical Devices
FDA: Food and Drug Administration
MHRA: Medicines and Healthcare products Regulatory Agency
PRISMA-P: Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols
PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews
PROBAST-AI: Prediction Model Risk of Bias Assessment Tool for Artificial Intelligence
QUADAS-AI: quality assessment tool for artificial intelligence–centered diagnostic test accuracy studies
ROBINS-I: Risk of Bias in Nonrandomized Studies of Interventions

Edited by A Mavragani; submitted 09.09.23; peer-reviewed by H Demir, B Jacob, JJ Beunza, D Chrimes; comments to author 17.12.23; revised version received 10.02.24; accepted 20.02.24; published 14.03.24.


©Ariel Yuhan Ong, Henry David Jeffry Hogg, Aditya U Kale, Priyal Taribagil, Ashley Kras, Eliot Dow, Trystan Macdonald, Xiaoxuan Liu, Pearse A Keane, Alastair K Denniston. Originally published in JMIR Research Protocols (, 14.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.