A prospective study in Ontario in 2005 [28] concluded that for similar cases, ED costs were 3 to 4 times higher than the costs incurred in a family physician’s office or a walk-in clinic. However, compared costs were not adjusted for comorbidities or severity of disease and did not include out-of-pocket expenses (eg, parking) and indirect costs to patients (eg, loss of income). Other studies, mainly from the United States, have reached similar conclusions [17,37-39] but using charges as proxies of health care costs, which has been shown to be an inaccurate costing method [40,41]. Some reports even suggest that walk-in clinics may in fact increase overall health care costs by duplicating care with frequent return visits after an initial visit [42-45].

Very few studies have considered quality of care and patient health outcomes in determining the best alternative setting for treating ambulatory emergency patients [8]. A 2017 review (Cochrane) of prospective studies comparing mortality, morbidity, and adherence to practice guidelines in walk-in clinics, primary care practices, and EDs found that none met this criterion [46]. However, three retrospective studies [47-49] and one study evaluating costs and return visits [28] suggested that (1) inappropriate use of antibiotics for self-resolving acute respiratory conditions occurs more frequently following visits to urgent care centers and family medicine offices than to EDs [47-50]; (2) the choice of antibiotics is more concordant with practice guidelines in walk-in clinics than in EDs and family medicine practices [48]; and (3) return visit likelihood within 72 hours is higher after an ED care episode than after any other outpatient clinic visit [28]. However, these fragmented and incomplete data come mostly from the United States. A comprehensive research program comparing acute care received in EDs, walk-in clinics, and primary care practices in Canada is long overdue.

Deciders often prioritize certain care settings based on potential cost savings, auctioning off care paths to the lowest bidder from the government’s perspective [51]. However, studies have shown that from a patient’s perspective, the choice to seek care in either a primary care practice, a walk-in clinic, or an ED is determined not only by ease of primary care access but also by factors such as convenience and perceived severity of illness and previous health care experiences [35,52-56]. What patients value the most differs considerably from what other stakeholders tend to value [57]. The patient’s perspective must be considered to determine the best ambulatory emergency care option. To our knowledge, no studies have compared these alternative settings from a patient’s perspective.

A time-driven activity-based costing study will first be performed for fiscal year April 1, 2021, to March 31, 2022. We shall estimate the cost of care processes administered by care providers (Objective 1) in 3 different models of ambulatory emergency care in Québec and Ontario: (1) discontinuous care in the ED (by physicians unfamiliar with the patients); (2) discontinuous care in a walk-in clinic (by physicians unfamiliar with the patients); and (3) continuous care in primary care clinic (patients attached to a primary care practice, seen by their family physician or a colleague on a same-day appointment for urgent needs).

We have confirmed the participation of 14 of the 15 planned patient recruitment sites (ED 5/5, walk-in 5/5, primary care 4/5; Multimedia Appendix 1). They have been selected in different types of urban areas, including small (Joliette), medium (Kingston), large (Québec City, Ottawa), and metropolitan (Montreal) cities. In each participating region, an ED will be paired with a nearby walk-in clinic and primary care practice. We are currently securing our final additional clinic in Ottawa with the help of BeACCoN (Better Access and Care for Complex Needs), a provincial primary care research network.

The time-driven activity-based costing method [72,85] will enable us to derive for each setting the cost of care processes (eg, triage) and traceable supplies (eg, medication) potentially provided to patients with acute respiratory conditions, which includes telemedicine. This costing method requires only 2 parameters, namely, the unit cost of supplying capacity and the duration of processes, and comprises the following steps:

The cost of a care process is proportional to the mean duration measured on field. For example, the cost of triage is estimated by adding up the expenses associated with the triage nurse, consumables, and overhead. These elements will be estimated by multiplying the mean triage duration by their unit costs as follows:

Cost of triage = mean triage duration × (unit cost of nurse + unit cost of consumables+ unit cost of overhead) = 7.1 min × (US $0.78/min + US $0.07/min + US $0.17/min) = US $7.24 (Can $1 [US $0.76])

The cost of telemedicine will be estimated following the same steps, from resource mapping and time measurement through allocation of overhead and consumables, all the way to average cost calculation.

Where applicable, the following adjustments will be made so that the estimated costs reflect the public payer’s perspective: (1) expenses paid by physicians or owners of a participating clinic will be subtracted from the yearly expenses related to the appropriate cost element (eg, salaries, overhead); (2) similarly, government funding received by a clinic apart from physician remuneration will be added.

The accounting department at each participating site will provide all financial data, except for physician charges. To calculate the unit cost for physicians, the total amount charged by all physicians per site per year will be obtained from local private billing agencies. This sum will be divided by the number of minutes spent delivering patient care, which will be obtained from physician schedules.

In addition to institution-specific costs, upon completing phase 1, we will create a list of standardized costs of care for each process and associated traceable supplies based on the average costs estimated in the 15 institutions (ED, walk-in, primary care, both provinces). Use of standardized costs will eliminate price effects because of differential costings between sites and provinces, thereby facilitating comparisons between the 3 clinical settings. In phase 2, the cost of a care episode will be calculated per individual by summing the standardized costs of care processes, supplies, and drugs received by each patient during their visit. Fixed and variable costs will be broken down to estimate and compare the care settings in terms of the marginal cost of each new patient assessed [86].

A multicenter prospective cohort study will be conducted in the institutions included in phase 1 to compare the value of care in EDs, walk-in clinics, and primary care practices (Objectives 2 and 3).

We shall include patients (1) aged 18 years and older; (2) seen in person or via telemedicine in an ED, a walk-in clinic, or the primary care practice where they are registered; (3) ambulatory during the entire visit or consultation; and (4) discharged home with a diagnosis of URTI, sinusitis, otitis media, pharyngitis, tonsillitis, bronchitis, influenza-like illness, pneumonia, acute asthma, or acute exacerbation of COPD. We shall exclude patients (1) transported by ambulance, (2) not covered by the provincial health insurance plan, (3) having consulted for a similar problem in the previous 30 days as patients with refractory diseases representing a population with different care needs, (4) living in a long-term health care facility or incarcerated, or (5) receiving palliative care.

A research nurse in collaboration with local clerks at each site will screen eligible patients after on-site registration or web-based scheduling, but before assessment by a physician, based on presenting complaints suggestive of acute respiratory conditions. After assessment and once a targeted diagnosis is confirmed, the same research nurse will prospectively (1) obtain consent from patients; (2) ensure that the discharge diagnosis, comorbidities, and disposition plans are fully documented; and (3) administer a questionnaire to assess patient experience of care and motivation for choosing one care setting over the other. Motivation will be classified into the 6 domains of the Conceptual Model of Emergency Department Use [35]. Participants will be asked to specify whether their choice of care setting was based on accessibility, convenience, their perception of the severity of illness, their beliefs and knowledge regarding these care settings, referral and advice from a care professional or an acquaintance, or costs. They will also be requested to rate their perception of illness severity. For on-site participants only, the research nurse will also (1) assess the severity using the Pandemic Medical Early Warning Score (PMEWS), a validated severity score allowing points for age, vital signs, comorbidity, social situation, and functional status [87], and (2) perform spirometry (measured parameter: forced expiratory volume in the first second [FEV₁]) on patients with acute asthma. A random sampling recruitment schedule will be planned to ensure a proportional representation of the hours of operation for each recruiting site. Participating EDs will recruit on a schedule similar to their paired participating clinics to include participants who could have consulted in an alternative setting and exclude night patients who differ significantly from patients seeking care during the day [88]. Recruitment will occur over a full year to encompass seasonal variability in the incidence of respiratory diseases.

Research assistants at each site will complete data collection from local medical records. For on-site participants and, where appropriate, for those assessed by telemedicine, they will compile the following information: age, sex, gender, ethnicity, postal code, distance from facility to home, referral by the provincial telephone consultation service (811, Telehealth), enrollment with a family physician, presenting complaints, comorbidities, regular medications, date and time of arrival and discharge, vital signs upon arrival, investigations and interventions during care episode, discharge diagnosis, and prescriptions upon discharge. A follow-up phone call will be made to all participants 10 days after the initial visit to collect data initially unavailable in medical records and to evaluate primary and secondary outcome metrics. Patient-reported outcome and cost measures will be completed by the participants at this moment, either on the phone with the research assistant or independently using a secured online survey link, depending on the participant’s preference. Text messaging and email reminders will be sent to improve participant retention [89]. We shall obtain information on health outcomes (eg, mortality) and physician charges via provincial databases. The charges billed by any physician 7 days after the initial visit will be used to estimate the costs of care for subsequent return visits and hospital admissions.

A value-based comparative assessment requires the simultaneous evaluation of health outcomes and costs. Our outcome measures were chosen from a guideline on the assessment of ED performance [90] and recent literature on patient experience assessment (Table 2) [91-95]. The initial visit, from arrival at a participating site to discharge, represents the unit of analysis for all outcome measures; however, the health system or patient costs incurred during the following week will be estimated and added to the cost of the initial visit. For participants assessed in person, the outcome will be scored per care setting and further stratified per discharge diagnosis and by province, using institution-specific costs for interprovincial cost comparisons. We will analyze the same outcome measures separately in the case of patients evaluated by telemedicine, as missing data (eg, vital signs) will prevent us from adjusting for the severity of their illness.

To evaluate the quality of care in each group under study, compliance with practice guidelines (eg, corticosteroid prescription for asthma) for the treatment of respiratory diseases [104-109] will be compared (the full list of outcome measures is given in Multimedia Appendix 5 [83,91,94,96-103]). Return visits will be reported by the participants during the 10-day follow-up phone call. The Canadian Institutes of Health Research bridge grant obtained in April 2020 allowed our team to adapt questionnaires assessing patient perspective (Patient-Reported Experience Measure [PREM], patient-reported outcome measure for ED patients [PROM-ED], and cost-for-patient questionnaire [CoPaQ]) for use in any setting under evaluation. Their use for patients seen in person or by telemedicine in ambulatory patients with acute respiratory conditions will be validated further in the fall of 2020.

As our main analyses focus on patients assessed in person, our sample size calculation is based solely on their numbers. We estimate that the rate of return visit for ambulatory emergency conditions varies from 1% to 13% depending on the care setting [17,110,111]. To account for the potential similarity in outcomes among individuals in each of the 15 clusters, we assumed a realistic intracluster correlation of =0.02 based on previous studies and applied a correction to inflate our sample size calculation [112,113]. Using data from Campbell et al [28], at least 1485 patients (approximately 99 per cluster) will be needed to reveal a 5% difference in the proportion of return visits within 72 hours (eg, 5% vs 10%), assuming a 20% loss to follow-up and at least 30 participants per condition, based on multivariate logistic regression power analysis, type I error (α) at .05, and power at 80% (1−β). Assuming 240 recruitment days over a year at each site and the recruitment of at least 1 to 2 patients per day, our final cohort should include over 4000 patients and reach the minimal sample size in both participating provinces, which will allow for more robust comparisons and analyses.

All main analyses will be conducted primarily on participants assessed in person in any of the care settings. Participants assessed by telemedicine will be analyzed and compared separately between sites where it is implemented. The value delivered at each participating site and on average in each care setting type will be illustrated with an operational effectiveness graphic [114]. Adjusted costs of care for acute ambulatory respiratory conditions will be plotted on the x-axis and adjusted return visits within 72 hours on the y-axis. Points closest to 0 on both axes represent the highest value of care (Multimedia Appendix 6). Indeed, the lower the return visit proportion and cost of care, the higher the value of care. Similar graphics will be used for patient-centered outcome measures. To compare EDs, walk-in clinics, and primary care physician practices, multilevel generalized linear models will be used with probability distributions adapted to the outcome under evaluation. To adjust for confounding (differences in case mix between care settings), an overlap weights approach [115,116] will be used, wherein each individual receives a weight factor that is proportional to the inverse of the probability of choosing a particular setting. Subjects that differ fundamentally between settings are attributed a weight of 0 and are thus excluded. This approach is of the greatest interest when groups are initially very different [116]. Intuitively, overlap weights create a pseudopopulation for which treatment is independent of measured confounders, thus mimicking a randomized trial of those confounders. Overlap weights have been shown to be more robust than conventional inverse probability weighting and matching based on propensity scores [115]. The weights will be estimated using a multinomial logistic regression model in which the dependent variable is the chosen care setting and independent variables are potential confounders or risk factors [117] for the outcomes identified in the literature: age [118,119], sex and gender [118,120], ethnicity [118,121-123], registration with a family physician [20,124], comorbidities (the Charlson index; number of regular medications) [118,125,126], asthma, FEV₁ among patients with asthma, the Canadian deprivation index [127-129], patient perception of illness severity [130], and vital signs [131-135] as proxies for severity. The same independent variables will be used to adjust for differences in case mix between settings in the telemedicine cohort, excluding vital signs and FEV₁. Multiple imputation will be considered as a possible means to adjust for these variables in this cohort. Overlap weights will be calculated using the values predicted by this model. We shall verify that the care setting groups are comparable according to the measured confounders after weighting by computing standardized mean differences. Differences below 10% will be considered to indicate good balance [117]. If residual imbalances are present, the weighting model will be revised. Once an appropriate balance is achieved, separate models for each outcome will be fitted to the weighted data, for which the care setting will be the only independent variable. The robustness of results with respect to unmeasured confounding will be assessed using the E-value [136,137]. Clustering by setting (eg, province, practice unit) will be taken into account using multilevel modeling (random intercept on province and practice unit). Reported cost estimates will be calculated with item-specific standardized costs (eg, Québec and Ontario average nurse unit cost). Patients referred to the ED from a participating outpatient facility but discharged home after ED assessment will be analyzed in the care setting group where they first presented, and the ED referral will be considered as a return visit. The costs of any return visits and admissions up to 7 days after the initial visit will be estimated separately and attributed to the care setting where the initial visit took place. Results of the 3 questionnaires from the patient perspective will be reported as proportions (PREM), mean costs (CoPaQ), and median scores (PROM-ED) and adjusted using the overlap weights approach. As patients seek care for symptoms, subanalysis based on presenting complaints instead of discharge diagnosis will be conducted to provide meaningful patient-oriented results. Other subanalyses will evaluate which patient profile (eg, gender [120], motivation for choosing a facility), and institutional characteristics (eg, access to x-ray) predict high quality and low costs, keeping in mind that our value assessment might not yield similar results for all subgroups or even within a group of patients with the same diagnosis. Statistical differences will be assessed with a significance threshold set at .05.

To assess potential uncontrolled confounding of the results, sensitivity analyses will be conducted by excluding separately and concurrently the participants most likely to influence the effects of the 3 types of care settings: (1) ≥65 years; (2) with ≥1 comorbidity; (3) with either asthma or COPD; (4) with ≥1 regular medication; (5) with any abnormal vital signs; (6) in the lowest and highest quartile of the deprivation index; and (7) smokers. The analyses will be repeated using PMEWS instead of vital signs as a marker of illness severity. To control for a potential Hawthorne effect, the analyses will be repeated, with the first 3 months of recruitment excluded to focus on the data collected after the providers have become used to being observed.