Autism spectrum disorder (ASD), a neurodevelopmental disorder impacting both communication and behavior, affects approximately 1 in every 44 children in the United States [1]. Targeted early intervention during a critical neurodevelopmental window is recommended to maximize long-term cognitive gains [2,3], adaptive behaviors [4-6], social and emotional functioning [7,8], and verbal fluency [8,9]. ASD evaluations have traditionally been managed in specialist settings; however, a rapid rise in prevalence rates over the past half century [10] has outpaced specialist capacity and led to prolonged wait times for diagnostic evaluations [10-13]. Currently, families in the United States may wait as long as 18 months between initial screening by their primary care clinician and diagnosis by a specialist [11].

Rising ASD prevalence rates, combined with specialist shortages, have contributed to significant diagnostic delays in the United States. Although ASD can be reliably diagnosed at as early as 18 months, the mean age of ASD diagnosis has remained >4 years since the Centers for Disease Control and Prevention began tracking prevalence rates [1,14-16]. The average 3 years delay from first parental concern to eventual diagnosis recorded in the literature is even longer for demographic groups that include girls, children of color, and children who are rural residing or of lower socioeconomic status [17-20]. An estimated 27% of children with ASD in the United States remain undiagnosed at the age of 8 years [20].

Given the importance of early diagnosis and intervention, the American Academy of Pediatrics has recommended that all children be screened in primary care at 18 and 24 months, in addition to whenever caregivers express concern [12]. When they feel confident, upon a failed screen, primary care clinicians are encouraged to make the ASD diagnosis themselves using the Diagnostic and Statistical Manual of Mental Disorders–fifth edition (DSM-5) criteria [12]. Alternatively, they can refer to specialists for further assessment. However, studies suggest that clinicians do not consistently screen or refer to these recommendations [21]. Currently, <1% of all ASD diagnoses are made by primary care clinicians in the United States [21,22].

The low rates of ASD diagnosis in primary care settings may be explained by several factors. Clinician–identified barriers include a lack of ASD-specific training and low self-efficacy in making the diagnosis, as well as time constraints that limit their ability to comprehensively evaluate, review results, and discuss treatment plans with families in primary care [23-26]. Complex presentations in children with ASD, including multiple overlapping comorbidities and a diverse spectrum of symptoms, can also confound the diagnosis [27]. Although diagnostic tools have been developed to support accurate evaluations [28], gold standard ASD diagnostic instruments with the highest validity [28-30] are typically poorly suited to use in primary care. This is because they are time intensive and typically require in-person delivery in a clinical facility and specialist training for reliable administration. Even when clinicians receive specialist training, primary care reimbursement rates may be insufficient to justify the effort [31].

Diagnosis aids tailored specifically for use in primary care are urgently needed to streamline ASD evaluations and enhance primary care diagnostic capacity. In response to this need, an artificial intelligence (AI)–based Software as a Medical Device [32] was developed to support health care providers in diagnosing or ruling out ASD in children aged 18 to 72 months with concerns for developmental delay. The device is indicated for use by health care providers who have prescriptive authority in their state of practice. For the purposes of this study the device will be used exclusively by primary care Extension for Community Health Outcomes (ECHO) Autism clinicians. The device was developed using patient record data from thousands of children with diverse conditions, presentations, and comorbidities who were either diagnosed with ASD or confirmed not to have ASD based on standardized diagnostic tools [29,30] and representing both sexes across the supported age range. The underlying device algorithm was iteratively improved, supplemented with ASD expert input, and prospectively validated for over 7 years [33-38]. Key steps involved in device use are illustrated in Figure 1.

The device integrates 3 independent inputs within its gradient-boosted decision tree algorithm: a brief caregiver questionnaire (completed via a mobile app), a video analyst questionnaire based on the analysis of 2 short home videos uploaded by the caregiver (completed in the video analyst portal), and a health care provider questionnaire (completed in the health care provider portal). The device evaluates inputs based on the predictive features that are most indicative of ASD. If the information provided is sufficiently granular, the device will provide an ASD Positive or ASD Negative output. When a highly predictive determination cannot be made, the device abstains from providing a result (No Result output). Following strong pivotal trial results [39], the device received Food and Drug Administration (FDA) marketing authorization on June 2, 2021 [40]. In a prospective, double-blinded multisite pivotal trial, the device was found to facilitate timely and accurate ASD diagnostic evaluation in nearly a third of the study cohort while minimizing false negatives to maintain clinical safety [39]. However, real-world evidence is needed to learn more about the utility and impact of the device in everyday US primary care settings.

ECHO Autism primary care [41,42] is a virtual learning program in which primary care clinicians receive guidance from ASD experts regarding best practices for ASD screening, identification, and management of medical and psychiatric comorbidities. The diagnostic component of the ECHO Autism primary care model involves training clinicians to conduct diagnostic assessments for ASD using the Screening Tool for Autism in Toddlers and Young Children (STAT), a level 2 ASD screener used as a diagnostic observational measure. The ECHO Autism STAT model provides training and mentorship to equip clinicians to diagnose ASD in cases that are clear and unambiguous and appropriately identify and refer complex cases for further assessment [43]. A 12-month pilot study with 18 participating clinicians was conducted using the ECHO Autism STAT model [43]. In this study, participating clinicians presented the results of their assessment to the ECHO Autism expert hub team for specialist input before guiding the diagnostic pathway to include either a diagnosis of ASD or a referral for further specialist evaluation. This pilot study showed that 73% of the participants reported acceptance of referrals for ASD diagnostic evaluations following program completion, and 80% reported an increase in the number of children with ASD in their caseloads. In addition, the pilot demonstrated a 2- to 6-month decrease in time to access services when compared with standard-of-care diagnostic pathways [41].

The objective of this study is to assess the feasibility and impact of integrating the device into the existing primary care diagnostic pathway, the ECHO Autism STAT model.

The study was approved by the institutional review board of the University of Missouri-Columbia (institutional review board–assigned project number 2075722) and registered with ClinicalTrials.gov (NCT05223374).

This prospective observational study was designed to obtain real-world evidence pertaining to the feasibility and impact of integrating the device into a primary care ASD diagnostic pathway. Demographic, clinical, and qualitative user experience data will be obtained from eligible participants. All decisions regarding patient care will be determined by the ECHO Autism clinicians, as the study is noninterventional. All clinical outcomes will be assessed by the ECHO Autism clinician, as in routine clinical practice.

Up to 15 University of Missouri ECHO Autism clinicians in both rural and suburban areas across the United States will be recruited.

Study enrollment is expected to take approximately 12 months. Patient participation will occur from the time the ECHO Autism clinician identifies a concern for ASD through +21 or −21 days of the 3-month follow-up visit.

There are no physical risks associated with participation in the study. Caregivers may feel some psychological discomfort when answering questions about their children’s behaviors. Children might also feel some psychological discomfort when being filmed for the video analysis. Videos will be used to evaluate participants’ behaviors as part of the diagnostic process. The videos will be captured within the mobile app, stored on Amazon’s secure cloud service (Amazon S3), and analyzed by the device manufacturer through a secure web portal. By agreeing to participate in the study, participants will grant and release all rights, titles, and interests in the videos to the device manufacturer. Another potential risk is the possible release of sensitive medical, behavioral, or educational information. To mitigate the risk of releasing sensitive personal information, the device manufacturer will adhere to confidentiality protocols aligned with the US Health Insurance Portability and Accountability Act and FDA 21 Code of Federal Regulations Part 11, as well as other federal, national, and local laws and regulations, to protect the privacy of personal information. All content within the mobile app will be stored on Amazon S3 and analyzed by the device manufacturer through a secure web portal. When data are published, they will be anonymized in a manner that prevents the identification of specific caregivers or participants. To the best of the manufacturer’s knowledge, there were no data or security breaches in any prior studies involving the device.

A potential benefit of the study is that device use could support ECHO Autism clinicians in identifying participants with ASD earlier than they would otherwise have been identified, thus enabling earlier therapy, which is known to positively affect long-term outcomes.

Participation is completely voluntary. Caregivers may withdraw consent for study participation at any time by informing the principal investigator or the study staff. The principal investigator may also withdraw a participant from the study if determined that further participation would not be in the participant’s best interest. Participants may be discontinued by the study staff if determined unable to fully meet the requirements of participation, including the completion of study visits and necessary assessments. The reasons for discontinuation will be recorded for all participants. If lost to follow-up, the research team will attempt to contact the caregiver by telephone, followed by registered mail, to establish the reason for withdrawal as completely as possible.

To our knowledge, this collaborative observational study will be the first to evaluate the use of a novel AI-based ASD diagnosis aid as part of a real-world primary care diagnostic pathway. It is anticipated that this study will provide valuable feasibility data regarding device integration into a primary care workflow. The study results may help clarify the length of time between initial clinician concern and diagnosis when an AI-based device is used as part of the evaluation process. The results may also provide insights into device performance outside of a research setting, including the percentage of primary care participants for whom it is able to provide a determinate ASD positive or negative result, and concordance between device output and clinical diagnostic determination. Qualitative user experience measures captured as part of the study are anticipated to shed light on the acceptability of the device to both families and clinicians and may help drive future product improvements.

Although to date, no previous AI-based ASD diagnosis aid primary care–focused studies have been conducted, there is literature supporting the feasibility of diagnosing and managing ASD in primary care settings. For example, a recent systematic review focused on training clinicians to diagnose and manage ASD in primary care [44] found that with appropriate training and support, primary care clinicians can render diagnostic decisions that closely align with specialty team assessments. The reviewed studies also reported reduced wait times for diagnosis when a proportion of evaluations were moved to the primary care setting.

Despite well-documented bottlenecks in ASD diagnosis and treatment initiation in the United States [11,45], to date, very few diagnostic tools have been designed to enhance primary care diagnostic capacity. The integration of the device into the ECHO Autism primary care diagnostic pathway in this study offers several potential benefits over existing tools [28-30]. As it was designed with primary care use in mind, the device is more time-efficient than existing instruments and does not require specialist training to administer. For example, in the pivotal trial, the total time burden associated with device use was <30 minutes (median time for caregiver questionnaire completion was 4 minutes and 56 seconds; median time for video analyst scoring from the time video review began to submission of scores was 10 minutes and 54 seconds; self-reported time for health care providers to complete their questionnaire was approximately 10 minutes) [39].

The device is also amenable to remote administration. Caregivers will complete their questionnaires and upload videos remotely using an app on their mobile phones. Video analysts will receive inputs and upload questionnaire outputs remotely through an analyst portal. Pivotal trial data showed no evidence of performance degradation in cases where health care providers completed their questionnaires remotely versus in person. Remote administration may support diagnostic access for families residing in rural and remote locations, as well as families without access to transportation. Remote administration is also useful during public health emergencies such as the COVID-19 pandemic.

Compared with other ASD assessment instruments where the child is observed in a clinical setting, the device leverages home videos that provide naturalistic data about the child’s behavior outside of the clinical setting. These data may help to overcome some of the challenges clinicians and caregivers have described capturing typical child behavior within time-limited clinical encounters [46]. For example, in the clinical setting, the child may become overly reactive to changes in the environment or may not display the behaviors of concern they exhibit at home. Without a picture of the child’s typical behavior, it can be challenging for the clinician to make a confident diagnostic assessment.

The device was also trained on a large and diverse data set in terms of race and ethnicity, socioeconomic status, and gender. Although the pivotal trial was not powered for statistical inference on these covariates, our initial analysis indicated no evidence of inconsistent device performance across these variables [39]. This finding is encouraging and points to the potential of the device to address some of the well-documented disparities in current diagnostic approaches whereby girls and African American and Hispanic children are misdiagnosed more often, as well as diagnosed at a later age, on average, than other groups [47,48].

The device itself and the overall study design have several constraints and limitations. Device use requires access to a smartphone and wireless internet, which not all low-income American families have [49]. In addition, the device is only available in English. Thus, many culturally and linguistically diverse potential participants will be excluded from the study. Selection bias may also be a concern in this study. For example, parents with a higher number of concerns about their child’s development may be more motivated to enroll in the study and complete all device inputs than parents with milder concerns. Attrition is another concern in this study as participation involves multiple steps and engagement over a 3-month period. Given the multiple steps involved, it is possible that study completers may end up representing a more motivated cohort than study leavers. The clinicians who will be recruited into the trial may also represent a biased sample, as by enrolling in ECHO training, they have already demonstrated an interest in enhancing their knowledge of developmental disabilities. In addition, owing to the ECHO Autism curriculum training they receive, clinicians may have more experience and background in ASD diagnosis than other primary care clinicians. In other settings, clinicians with less interest in ASD may be less motivated or confident in using the device and may be less willing to diagnose in the primary care setting.

The study follows participants for a total of 3 months. Additional follow-up studies are recommended to assess the stability of device performance over a longer period. For the purposes of this study, the sponsors will cover the device cost and associated clinician fees. However, if the device were to become a standard tool in primary care settings, it is unknown to what extent insurance companies would accept and reimburse these costs. Questions also remain regarding whether and how families with no health insurance can access the device. We recommend that future device studies be conducted in populations who have not completed the ECHO Autism diagnostic training program. Such research could help answer important questions about the feasibility of device use in primary care clinician populations with less ASD-specific knowledge and training.

If device integration into primary care practice proves feasible and efficacious, it may support clinicians in diagnosing more children in the primary care setting and reducing current delays between the first ASD concern and eventual diagnosis. Streamlining primary care ASD diagnosis could potentially reduce strain on specialist services by allowing for the diagnosis of less ambiguous cases in primary care and quicker referral of complex cases to relevant specialist teams. Ultimately, we hope that decreasing the time to diagnosis and age of diagnosis through enhanced primary care capacity will allow a greater proportion of children to commence early intervention during a critical neurodevelopmental window.