Dental caries is the most prevalent chronic disease in childhood [1]. It causes mineral loss within the dental hard tissue, which results in visual changes in the enamel in the initial stages [2]. With increasing severity, demineralization advances into the dentine-pulp complex, and a cavity can develop [2]. More severe caries can result in pain, difficulty eating, and infection, and its management is more invasive and costly from an individual and public health perspective. In contrast, managing early caries is noninvasive, and damage to the tooth surface is reversible [3]. Epidemiological research should use methods that enable early disease detection to better target caries prevention and timely intervention programs during the early years.

In epidemiological research, visual examination (VE) is the standard method for detecting dental caries on visible tooth surfaces. VE can be subjective, particularly for early disease detection and monitoring. Validated indices, such as the International Caries Detection and Assessment System (ICDAS), improve the accuracy of VE for early caries detection [4-6]. ICDAS classifies caries based on severity, and a strong correlation exists between ICDAS and histological caries depth [7,8]. Studies using ICDAS report good inter- and intraexaminer reliability. Training and calibration improve examiner accuracy and reliability and are essential for standardizing examiners [9-11].

Epidemiological research, particularly multisite studies, requires the presence of multiple qualified dental examiners to undertake VE at the time of data collection, which is time and resource intensive. Digital photography has been suggested as an alternative to VE in these settings, where images are taken and assessed by trained and calibrated examiners. Boye et al [12] demonstrated high examiner agreement (weighted kappa, κ_w>0.9) between VE and remote assessment of 2D images. High accuracy for caries detection has been reported using 2D images compared to a gold standard examiner; however, the threshold used for disease detection in these studies is mostly tooth cavitation. Such thresholds cannot inform and drive early disease management strategies. Novel methods that facilitate the remote assessment of dentitions without compromising accuracy relative to VE for early disease detection would have broad applications in the research setting and beyond. Intraoral scanning technology provides such a method, yet its performance for caries detection in children’s primary teeth has yet to be established.

Intraoral scanners (IOSs) are handheld devices that create 3D digital models of the teeth and surrounding structures [13]. Digital representation of the teeth is achieved using specialized hardware (to scan the teeth) and software (to combine the data received and create 3D models). Scanners shine light onto a tooth region, and the distance between the area of interest and the scanner sensor is calculated via optical triangulation, confocal imaging, or active wave front sampling [14,15]. Numerous data points regarding tissue geometry are generated and used to construct a 3D model [14]. Red, green, and blue signals received from the tissues are applied to the model as color texture. Recently, light-induced fluorescence technology has been incorporated into the hardware of a commercially available IOS to enhance early caries detection (TRIOS 4 and 5) [16].

Light fluorescence technology is suitable for detecting and quantifying early carious lesions [17]. Exposure to blue-violet wavelength light causes sound dental tissue to auto-fluoresce green and demineralized carious dental tissue to appear darker [17]. The diagnostic capability of light fluorescence is enhanced because porphyrins, a metabolite of bacteria associated with dental caries, emit red fluorescence when exposed to light at this wavelength [18].

The interpretation of light fluorescence can be subjective, as it depends on the skill and experience of the dental practitioner. A study investigating occlusal caries detection of permanent teeth with and without fluorescence found no difference in accuracy between on-screen assessment and VE, suggesting that on-screen assessment of 3D models could be used for early caries detection and as a potential alternative to VE [19]. To our knowledge, there are no studies that have investigated the validity of on-screen assessment of 3D models for caries detection in primary teeth.

Recent advances have seen the development of computational methods as a more objective measure to quantify dental caries and support clinical diagnosis. TRIOS has developed an automated caries scoring system for their software that can automatically classify caries on occlusal surfaces based on color and fluorescence changes (TRIOS Patient Monitoring [TPM]; version 2.3; 3Shape TRIOS A/S) [16]. When comparing the diagnostic performance as quantified by the area under the curve (AUC), early in vitro and in vivo validation studies suggest that the automated score is comparable to VE at the early (AUC VE 0.71 vs 0.76) and moderate disease thresholds (AUC VE 0.90 vs 0.90), based on a histological reference standard for permanent teeth [20]. A separate investigation into the validity of this algorithm in primary teeth concluded it had comparable performance (AUC 0.88) to VE (AUC 0.96) for caries detection of primary tooth occlusal surfaces [21]. The latter was a small in vitro feasibility study; consequently, results should be interpreted cautiously and not generalized to the clinical setting. The in vivo performance of automated assessment of 3D models for caries detection in primary teeth has yet to be established.

One of the problems with investigating early caries detection methods in realistic settings is the choice of a reference standard for method comparison, as histological or operative references are rarely feasible or ethical. It is argued that when VE alone is used as a gold standard, measures of diagnostic accuracy (sensitivity and specificity) may overestimate the true accuracy of the caries detection method under investigation [22,23]. Instead, reporting on the diagnostic agreement has been suggested to be more appropriate. Diagnostic agreement quantifies the similarity between methods in detecting the outcome of interest to be used interchangeably [24].

This study describes the protocol and statistical analysis plan for determining the diagnostic agreement between VE, on-screen, and automated caries detection methods using 3D models in children.

The specific study objectives are to, first, determine and compare the diagnostic agreement between VE and on-screen visual assessment of 3D models in approximate natural colors with and without fluorescence to detect and classify carious lesions on visible tooth surfaces in primary teeth. The second objective is to determine the diagnostic agreement between VE and the automated caries scoring system for detecting and classifying occlusal carious lesions in primary teeth. Finally, the third objective is to explore the impact of caries threshold, tooth surface examined (smooth vs occlusal), and enamel defects’ presence on the reported agreement estimates between caries detection methods.

This is a diagnostic agreement study, as VE is not considered a gold standard in this setting [23]. Dental caries in this study is defined as primary coronal caries. Data sources (age at dental visit, sex, VE data, and 3D models) will be retrospectively sourced from a sample of children who underwent VE using the ICDAS and intraoral scanning with the TRIOS 4 to produce 3D digital models.

The 3D models for each eligible participant will be subject to an on-screen assessment by 4 qualified dental practitioners. Each practitioner will examine all 3D models at 2 separate time points. An on-screen assessment of 3D models based on tissue geometry and approximate natural colors (excitation to visible white light) will take place first (Figure 1A), followed by an on-screen assessment with the addition of fluorescence texture (excitation to light at 415 nm; Figure 1B). An interval of at least 4 weeks will follow to minimize recall bias before the on-screen assessment process is replicated. Occlusal surfaces of primary molars will be examined using the automated caries scoring system at a single time point only (Figure 1C). Examiners will be blinded to all clinical examination data and automated caries scores when scoring the 3D scans.

The study sample will be drawn from eligible participants in an existing randomized controlled trial, the Melbourne Infant Study Bacille Calmette Guérin (BCG) for Allergy & Infection Reduction (MISBAIR; ClinicalTrials.gov NCT01906853; for a complete description of the trial, please refer to the MISBAIR study protocol) [25]. English-speaking immunocompetent pregnant women scheduled for birth at hospitals servicing Werribee, Ballarat, Geelong, and Melbourne in Victoria, Australia, from mid-2013 to mid- to late 2016 provided consent for their infants to participate in a longitudinal and randomized controlled trial to determine if BCG immunization within the first 10 days of life prevents the development of allergy and infection in early childhood. Infants were excluded if they had any conditions at birth that would contraindicate the use of live vaccines. The MISBAIR trial included a 5-year-old study visit where a clinical assessment (including dental assessment) was undertaken (219/536, 40.9% remaining MISBAIR participants). All study visits occurred between January 31, 2021, and March 31, 2022, at the Murdoch Children’s Research Institute in Melbourne.

MISBAIR participant data will be included for analysis in this study if participants meet the following criteria: (1) attended the 5-year-old study visit for MISBAIR (target age 4-6 years), (2) completed of a VE during the 5-year-old study visit, (3) acquired an intraoral scan at the same time as the VE, and (4) provided informed consent for the analysis of dental data and on-screen assessment of 3D models.

For each participant, a tooth surface will be eligible for on-screen assessment if it (1) has been examined as part of the dental exam, (2) is from a primary tooth, and (3) is visible on the 3D model.

Tooth surfaces will be excluded from on-screen assessment if (1) they have a sealant, direct or indirect restoration; (2) the tooth surface visibility on the 3D model is impeded by calculus, debris, or plaque; and (3) less than one-third of the surface is visible due to insufficient or missing scan data.

Tooth surfaces will be excluded from the application of the automated caries scoring system if they (1) have a sealant, direct or indirect restoration; (2) have an enamel defect; (3) are not a primary molar occlusal surface; and (4) less than one-third of the surface is visible due to insufficient or missing scan data.

This is a retrospective study drawing all eligible participants from the MISBAIR trial who underwent the dental assessment at their 5-year-old study visit.

For agreement analyses, ICDAS and modified ICDAS codes will be collapsed for each surface into four categories labeled sound surface (0), initial caries (1), moderate (2), and extensive caries (3). A binary dental caries variable will be created for each surface, and the detection threshold will be initial caries (0 vs 1, 2, and 3).

Data checking and cleaning will take place before analysis. Descriptive statistics for patient characteristics such as age, sex, caries prevalence, and the proportion of children with enamel defects will be reported. The total number of tooth sites included in the analysis will be reported.

Descriptive characteristics for the rater population will be reported, including qualification, clinical background, years of experience, and training results for each rater. Interrater and intrarater reliability for on-screen assessment of 3D models based on color and fluorescence using the intraclass correlation coefficient will be reported for all examiners. These estimates will be obtained from a multilevel model fitted to the surface-level data, accounting clustering (see below), and zero inflation.

To assess agreement among methods, we will undertake both surface-level and individual-level analyses. At the surface level, caries data naturally have 3 levels with surfaces clustered within a tooth, which are clustered within an individual. Therefore, for this analysis, we will use multilevel logistic regression to estimate method agreement while accounting for clustering for the tooth. Specifically, separate multilevel models will be fitted to compare VE with each of the alternative methods by including the variable method (VE vs on-screen assessment of 3D models in approximate natural colors, on-screen assessment of 3D models in approximate natural colors featuring fluorescence, and automated system) as the fixed effect. The coefficient estimates for the variable method will be used to indicate the level of agreement. The multilevel model will be fitted with a zero-inflated binomial distribution.

At the individual level, a numerical variable representing the number of lesions per individual will be generated and analyzed with 2 approaches. First, Bland-Altman method agreement analysis will be performed using a regression approach. Specifically, a model will be constructed to regress the difference between each pair of methods on their average [36]. The analysis will be repeated for each possible pair of methods. Second, for each possible pair of methods, an intraclass correlation coefficient will be estimated from a multilevel model fitted to the paired data consisting of the 2 measures for each individual obtained from the 2 methods, fitted with a zero-inflated Poisson distribution.

A sensitivity analysis will explore the impact of the diagnostic threshold used on agreement estimates. Analyses will be repeated with the caries data dichotomized at alternative thresholds: the moderate and extensive thresholds. In the analysis of the moderate threshold, sound and initial surfaces will be labeled caries absent (0) and all remaining categories (moderate-extensive) will be labeled caries present (1). In the analysis of the extensive threshold, sound, initial, and moderate caries will be combined and labeled as caries absent (0) and only teeth with extensive caries will be marked as caries present (1). To evaluate if the surface location and presence of enamel defects impact the diagnostic agreement estimates, these variables will be included as additional variables in the regression models for the surface-level analyses, including interaction terms with the method. The surface location variable will have 3 levels for each visible tooth surface (occlusal, smooth, and proximal). Enamel defects will be a binary variable, defined as present or absent per person. The sensitivity analysis to compare the agreement between on-screen assessment of 3D models in approximate natural colors and on-screen assessment of 3D models in approximate natural colors featuring fluorescence will include additional variables, rater and timing, as fixed effects.

The MISBAIR trial has ethical and governance approval from Mercy Health Human Research Ethics Committee (HREC, No. R12-28) and Royal Children’s Hospital (RCH) HREC (HREC No. 33025). Informed consent to participate in the study and to share dental data to validate and develop intraoral scanning technology for dental health assessment was obtained from participants' parents or guardians. Participant records sourced have a unique identifier and are not reidentifiable to the dental team. This diagnostic agreement study has ethical approval from the RCH HREC (RCH HREC No. 88321). It will be conducted according to the principles outlined in the Declaration of Helsinki. This protocol has registration with the Australian New Zealand Clinical Trials Registry (ANZCTR; registration ACTRN12622001237774p).