This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
In Canada, 30%-60% of patients presenting to emergency departments are ambulatory. This category has been labeled as a source of emergency department overuse. Acting on the presumption that primary care practices and walk-in clinics offer equivalent care at a lower cost, governments have invested massively in improving access to these alternative settings in the hope that patients would present there instead when possible, thereby reducing the load on emergency departments. Data in support of this approach remain scarce and equivocal.
The aim of this study is to compare the value of care received in emergency departments, walk-in clinics, and primary care practices by ambulatory patients with upper respiratory tract infection, sinusitis, otitis media, tonsillitis, pharyngitis, bronchitis, influenza-like illness, pneumonia, acute asthma, or acute exacerbation of chronic obstructive pulmonary disease.
A multicenter prospective cohort study will be performed in Ontario and Québec. In phase 1, a time-driven activity-based costing method will be applied at each of the 15 study sites. This method uses time as a cost driver to allocate direct costs (eg, medication), consumable expenditures (eg, needles), overhead costs (eg, building maintenance), and physician charges to patient care. Thus, the cost of a care episode will be proportional to the time spent receiving the care. At the end of this phase, a list of care process costs will be generated and used to calculate the cost of each consultation during phase 2, in which a prospective cohort of patients will be monitored to compare the care received in each setting. Patients aged 18 years and older, ambulatory throughout the care episode, and discharged to home with one of the aforementioned targeted diagnoses will be considered. The estimated sample size is 1485 patients. The 3 types of care settings will be compared on the basis of primary outcomes in terms of the proportion of return visits to any site 3 and 7 days after the initial visit and the mean cost of care. The secondary outcomes measured will include scores on patient-reported outcome and experience measures and mean costs borne wholly by patients. We will use multilevel generalized linear models to compare the care settings and an overlap weights approach to adjust for confounding factors related to age, sex, gender, ethnicity, comorbidities, registration with a family physician, socioeconomic status, and severity of illness.
Phase 1 will begin in 2021 and phase 2, in 2023. The results will be available in 2025.
The end point of our program will be for deciders, patients, and care providers to be able to determine the most appropriate care setting for the management of ambulatory emergency respiratory conditions, based on the quality and cost of care associated with each alternative.
PRR1-10.2196/25619
Emergency departments (EDs) are specialized and costly resources designed to provide care for patients with urgent or life-threatening conditions [
Many Canadian regional health authorities have developed policies so that low-acuity ambulatory emergency patients preferably present to walk-in clinics or primary care practices [
Few studies have tested the hypothesis that walk-in or primary care clinics offer better care than EDs to ambulatory patients with acute health concerns.
A prospective study in Ontario in 2005 [
Very few studies have considered quality of care and patient health outcomes in determining the best alternative setting for treating ambulatory emergency patients [
Deciders often prioritize certain care settings based on potential cost savings,
To compare the different care setting possibilities for ambulatory emergency patients, we propose value-based assessment, an approach first described by Michael Porter in 2006 [
Our team has conducted a pilot study in which an ED and a primary care clinic offering walk-in services for frequent ambulatory acute conditions were compared in terms of costs of care and compliance with practice guidelines [
Mean cost of care and percentage of use of nonrecommended care applied to upper respiratory tract infection in a primary care practice and an emergency department.
Variable | Primary care practice (n=102) | Emergency department (n=52) | ||
Cost of carea (US $), (mean 95% CI) | 45.4 (38.4-53.4) | 59.8 (49.4-72.3) | <.001 | |
|
||||
|
Chest x-ray | 13.7 (7.7-22.0) | 26.9 (15.6-41.0) | .05 |
|
Antibiotics | 44.1 (34.3-54.3) | 5.8 (1.2-16.0) | <.001 |
aMean value adjusted for age, sex, vital signs, comorbidities, and number of regular medications for upper respiratory tract infection.
On the basis of this preliminary study, we conclude that (1) time-driven activity-based costing is feasible in ED and primary care settings without requiring advanced information technologies or rigorously coded electronic medical records, 2 major barriers to conducting research in outpatient clinics, and (2) significant variations in costs and quality of care may exist between EDs and clinics, suggesting that a multicenter cohort study is warranted. However, this retrospective study highlighted major issues that only a prospective design can resolve: comorbidities (crucial to risk adjustment), disposition plans (crucial to assessing quality of care), and discharge diagnosis are not readily extractable from databases in the outpatient setting and are often missing or incomplete in medical notes. By manually reviewing thousands of visits logged in electronic records, our research assistants identified eligible cases one chart at a time. These major hurdles apply to outpatient clinics in all Canadian provinces. A retrospective design for a multicenter cross-jurisdictional study would have major methodological flaws because of the unlikeliness of obtaining comparable information across settings. More importantly, a retrospective study on administrative databases would not allow us to assess patients’ perspectives. Finally, a randomized controlled trial is not feasible for the population and settings under study because randomization would have to occur before any contact with the health system to assign patients to their treatment group. For these reasons, we believe that a prospective cohort study is the most appropriate design for identifying the best care setting for ambulatory emergency patients.
Our goal is to compare the health outcomes and costs of care received in EDs, walk-in clinics, and primary care practices by ambulatory patients presenting with acute respiratory conditions, namely, URTI, sinusitis, otitis media, pharyngitis, tonsillitis, bronchitis, influenza-like illness, pneumonia, acute asthma, or acute exacerbation of chronic obstructive pulmonary disease (COPD). We selected these conditions because many performance metrics have been validated previously for assessing the quality of care provided [
A time-driven activity-based costing study will first be performed for fiscal year April 1, 2021, to March 31, 2022. We shall estimate the cost of care processes administered by care providers (Objective 1) in 3 different models of ambulatory emergency care in Québec and Ontario: (1) discontinuous care in the ED (by physicians unfamiliar with the patients); (2) discontinuous care in a walk-in clinic (by physicians unfamiliar with the patients); and (3) continuous care in primary care clinic (patients attached to a primary care practice, seen by their family physician or a colleague on a same-day appointment for urgent needs).
We have confirmed the participation of 14 of the 15 planned patient recruitment sites (ED 5/5, walk-in 5/5, primary care 4/5;
The time-driven activity-based costing method [
Process (eg, salbutamol in acute asthma) and resource (eg, respiratory therapist) mapping through discussion with local teams for each respiratory condition (
Validation of process maps and durations by on-field research assistants prospectively observing patients and measuring process duration using time-motion software (UMT Plus [Laubrass])
Calculation, with local administrative teams, of total annual overhead costs (eg, building maintenance) related to the care of ambulatory patients with acute conditions (
Estimation of cost per time unit ($/minute) for the following cost elements obtained by dividing yearly expenses for a cost element by the total yearly number of minutes worked by professionals to care for patients in this facility (
Estimation of the cost of traceable supplies (eg, laboratory testing)
Calculation of the average cost of each health care process (
Process mapping for upper respiratory tract infections in the emergency department (truncated). Each box represents a process with its duration. Colors identify human resources (red=nurses; yellow=clerks; green=physicians). ED: emergency department.
The cost of a care process is proportional to the mean duration measured on field. For example, the cost of triage is estimated by adding up the expenses associated with the triage nurse, consumables, and overhead. These elements will be estimated by multiplying the mean triage duration by their unit costs as follows:
Cost of triage = mean triage duration × (unit cost of nurse + unit cost of consumables+ unit cost of overhead) = 7.1 min × (US $0.78/min + US $0.07/min + US $0.17/min) = US $7.24 (Can $1 [US $0.76])
The cost of telemedicine will be estimated following the same steps, from resource mapping and time measurement through allocation of overhead and consumables, all the way to average cost calculation.
Where applicable, the following adjustments will be made so that the estimated costs reflect the public payer’s perspective: (1) expenses paid by physicians or owners of a participating clinic will be subtracted from the yearly expenses related to the appropriate cost element (eg, salaries, overhead); (2) similarly, government funding received by a clinic apart from physician remuneration will be added.
The accounting department at each participating site will provide all financial data, except for physician charges. To calculate the unit cost for physicians, the total amount charged by all physicians per site per year will be obtained from local private billing agencies. This sum will be divided by the number of minutes spent delivering patient care, which will be obtained from physician schedules.
In addition to institution-specific costs, upon completing phase 1, we will create a list of standardized costs of care for each process and associated traceable supplies based on the average costs estimated in the 15 institutions (ED, walk-in, primary care, both provinces). Use of standardized costs will eliminate price effects because of differential costings between sites and provinces, thereby facilitating comparisons between the 3 clinical settings. In phase 2, the cost of a care episode will be calculated per individual by summing the standardized costs of care processes, supplies, and drugs received by each patient during their visit. Fixed and variable costs will be broken down to estimate and compare the care settings in terms of the marginal cost of each new patient assessed [
A multicenter prospective cohort study will be conducted in the institutions included in phase 1 to compare the value of care in EDs, walk-in clinics, and primary care practices (Objectives 2 and 3).
We shall include patients (1) aged 18 years and older; (2) seen in person or via telemedicine in an ED, a walk-in clinic, or the primary care practice where they are registered; (3) ambulatory during the entire visit or consultation; and (4) discharged home with a diagnosis of URTI, sinusitis, otitis media, pharyngitis, tonsillitis, bronchitis, influenza-like illness, pneumonia, acute asthma, or acute exacerbation of COPD. We shall exclude patients (1) transported by ambulance, (2) not covered by the provincial health insurance plan, (3) having consulted for a similar problem in the previous 30 days as patients with refractory diseases representing a population with different care needs, (4) living in a long-term health care facility or incarcerated, or (5) receiving palliative care.
A research nurse in collaboration with local clerks at each site will screen eligible patients after on-site registration or web-based scheduling, but before assessment by a physician, based on presenting complaints suggestive of acute respiratory conditions. After assessment and once a targeted diagnosis is confirmed, the same research nurse will prospectively (1) obtain consent from patients; (2) ensure that the discharge diagnosis, comorbidities, and disposition plans are fully documented; and (3) administer a questionnaire to assess patient experience of care and motivation for choosing one care setting over the other. Motivation will be classified into the 6 domains of the Conceptual Model of Emergency Department Use [
Research assistants at each site will complete data collection from local medical records. For on-site participants and, where appropriate, for those assessed by telemedicine, they will compile the following information: age, sex, gender, ethnicity, postal code, distance from facility to home, referral by the provincial telephone consultation service (811, Telehealth), enrollment with a family physician, presenting complaints, comorbidities, regular medications, date and time of arrival and discharge, vital signs upon arrival, investigations and interventions during care episode, discharge diagnosis, and prescriptions upon discharge. A follow-up phone call will be made to all participants 10 days after the initial visit to collect data initially unavailable in medical records and to evaluate primary and secondary outcome metrics. Patient-reported outcome and cost measures will be completed by the participants at this moment, either on the phone with the research assistant or independently using a secured online survey link, depending on the participant’s preference. Text messaging and email reminders will be sent to improve participant retention [
A value-based comparative assessment requires the simultaneous evaluation of health outcomes and costs. Our outcome measures were chosen from a guideline on the assessment of ED performance [
Main study outcomes.
Outcome | Definition | Source | |
|
|||
|
Incidence of return visit (Oa) | Proportion of patients returning to any EDb or outpatient clinic at 72 hours and 7 days after the initial visit [ |
Follow-up call at 10 days |
|
Mean cost of care−the Ministry of Health perspective (Cc) | The cost per care episode calculated by summing the costs of all care processes delivered to a patient during the initial visit plus the costs of return visits and admissions at 72 hours and 7 days. | Electronic medical records and provincial billing databases |
|
|||
|
Median PROM-EDd patients scores (O) | Developed and validated by team member SV, the PROM-ED questionnaire provides a measurement of patient-reported outcome expressed as scores for symptom relief, understanding of health concern, reassurance, and having a plan for care [ |
Follow-up call at 10 days |
|
Median scores on a PREMe (O) | We adapted and are validating a tool from patient experience surveys used in EDs and primary care clinics in Ontario [ |
At the end of the initial visit |
|
Mean CoPaQf (C) | A questionnaire measuring patients’ and caregivers’ out-of-pocket expenses (eg, travel) and indirect costs (eg, loss of income) will be proposed to participants. This questionnaire was developed and validated by members of our team (ML, JG, SB) and further adapted for use in this study. | Follow-up call at 10 days |
|
Incidence of admission, intensive care unit, or mortality (O) | Proportion of patients who were admitted to the hospital or the intensive care unit or died because of one of the targeted respiratory conditions within 30 days [ |
Provincial databases: Med-Echo, ICES, death registries |
|
Wait times | Median and mean length of stay and time spent waiting to see a physician. | Electronic medical records |
aO: health outcome.
bED: emergency department.
cC: health cost.
dPROM-ED: patient-reported outcome measure for ED.
ePREM: patient-reported experience measure.
fCoPaQ: cost-for-patient questionnaire.
To evaluate the quality of care in each group under study, compliance with practice guidelines (eg, corticosteroid prescription for asthma) for the treatment of respiratory diseases [
As our main analyses focus on patients assessed in person, our sample size calculation is based solely on their numbers. We estimate that the rate of return visit for ambulatory emergency conditions varies from 1% to 13% depending on the care setting [
All main analyses will be conducted primarily on participants assessed in person in any of the care settings. Participants assessed by telemedicine will be analyzed and compared separately between sites where it is implemented. The value delivered at each participating site and on average in each care setting type will be illustrated with an operational effectiveness graphic [
To assess potential uncontrolled confounding of the results, sensitivity analyses will be conducted by excluding separately and concurrently the participants most likely to influence the effects of the 3 types of care settings: (1) ≥65 years; (2) with ≥1 comorbidity; (3) with either asthma or COPD; (4) with ≥1 regular medication; (5) with any abnormal vital signs; (6) in the lowest and highest quartile of the deprivation index; and (7) smokers. The analyses will be repeated using PMEWS instead of vital signs as a marker of illness severity. To control for a potential Hawthorne effect, the analyses will be repeated, with the first 3 months of recruitment excluded to focus on the data collected after the providers have become used to being observed.
From our pilot studies reported earlier until now, our team has made significant progress to reach its goal of identifying the care pathways providing the highest value to ambulatory emergency patients. We have assembled a very strong research team composed of patients, clinicians, administrators, and researchers. Together, we have created this paper. Two patient partners met with us regularly and provided helpful comments to make our research plan more patient centered. We have secured 14 of 15 planned participating sites. We have adapted the 3 patient-centered tools (PREM, PROM-ED, and CoPaQ) and are currently validating their use on ambulatory emergency patients whether they receive care in an ED, a walk-in clinic, or a primary care practice.
Our protocol has been endorsed by the Network of Canadian Emergency Researchers (NCER). The broad support for our research initiative from leading Canadian organizations in emergency (NCER, Canadian Association of Emergency Physicians) and primary care (Réseau-1, Réseau de recherche axée sur les pratiques de première ligne, BeACCoN Ontario, Réseau sur les Innovations en soins de santé de première ligne et intégrés, Strategy for Patient-Oriented Research Unit), from the Ministries of Health of Ontario and Québec, and from organizations dedicated to improving health care throughout Canada (PULSAR, Canadian Institute for Health Information, Institut national d’excellence en santé et services sociaux, ICES) demonstrates the importance of the issue being addressed.
Phase 1 will begin in 2021 and will allow us to compare the cost of care from the public payer perspective in 3 different settings and 2 Canadian provinces. We expect that the results from this phase will be available in 2023. Phase 2 will begin in 2023 and will evaluate the value of the care in each setting under study. The final results will be published in 2025 and 2026. Our 4-year program covering the period of April 1, 2021, to March 31, 2025, is presented in a Gantt diagram available in
Our unique multidimensional approach to examining the quality and cost of care using both patient and system perspectives will provide knowledge that will be helpful in determining whether EDs, walk-in clinics, or primary care practices offer the best value to patients with acute ambulatory respiratory conditions. We expect our study to yield tangible benefits for all stakeholders.
For guiding policy and decisions: Despite weak evidence, Canadian provinces have invested massively in alternative care pathways to get ambulatory patients with urgent care needs to rely less on hospital EDs. Data generated by the proposed study will have an immediate impact by providing hard evidence in support of health care planning decisions intended to improve the service quality/cost ratio and hence outcomes in the largest patient category.
For patients: Current policies are designed for statistically average ambulatory emergency patients without considering patient perspectives and the widely variable severity of each diagnosed illness. As the needs and preferences of patients with pharyngitis likely differ from those with exacerbated COPD, our stratified results per condition will enable policy makers to structure urgent care systems to provide better-adapted higher value services to each specific category of patients. Our comprehensive research initiative will bring patient preferences and perspectives into policy making.
For clinicians: Our study will be a powerful driver for quality improvement in all care settings involved. Care quality can vary considerably, and we hope to generate unique opportunities for valid and meaningful comparisons and for quality improvement initiatives throughout the country.
First, as patients choose their facility, those presenting at the 3 types of setting will likely represent different populations. However, we believe that the potential confounding bias due to self-selection of the care setting can be overcome using the overlap weights approach. Extensive testing of the robustness of our findings by sensitivity analyses should allow us to avoid reaching false conclusions under the influence of uncontrolled confounding. Second, the Québec and Ontario health systems might differ enough to yield results that will not be easy to generalize. When applicable, the sources of heterogeneity will be investigated. However, Canadian provincial health care systems have fundamental similarities that reduce the risk of poor generalizability. All are based on universal coverage; all suffer from a lack of integration between primary and urgent care resources [
Ambulatory emergency patients account for 30% to 60% of all ED visits in Canada. This burden on emergency care is now exacerbated by the COVID-19 pandemic. This category of patients is thought to be amenable to using walk-in clinics or primary care practices and is the focus of redirection strategies meant to decrease ED overuse. However, current knowledge is inadequate for reaching any firm conclusions about which care settings are best suited for this purpose. The aim of this study is to compare the value of the care that these patients receive in EDs, walk-in clinics, and primary care practices, thereby providing arm administrators and care providers with new and robust knowledge that will enable them to determine the best care setting for the management of respiratory ambulatory emergency conditions. We all agree that the system can only benefit from patients receiving timely care in the proper setting from the most suitable provider.
Participating sites per province and care setting.
Summary of overhead expenses.
Cost per time unit (Can $/min) of cost elements and estimated cost (Can $) of important care processes.
Steps of the time-driven activity-based costing method.
Complete list of study outcomes.
Example of an operational effectiveness graphic with hypothetical numbers.
Gantt diagram—value project.
Better Access and Care for Complex Needs
cost-for-patient questionnaire
chronic obstructive pulmonary disease
emergency department
forced expiratory volume in the first second
Network of Canadian Emergency Researchers
Pandemic Medical Early Warning Score
patient-reported experience measures
patient-reported outcome measure for emergency department patients
upper respiratory tract infection
This work was funded by a Canadian Institutes of Health Research Project under Grant RN #169210. The authors would like to thank Yves Bolduc, Jean-Sébastien Audette, Michel Lafrenière, Benoît Maheu, Susan Phillips, Bruno J L’Heureux, Danièle Roberge, and Myriam Nadeau for their contributions.
SB is the grant holder and nominated principal investigator of the project. MB and JG are the coprincipal investigators. All authors contributed ideas and read and approved the final manuscript.
Dr. Patrick M. Archambault has completed research contracts with Thales Digital Solutions to develop medical decision support systems. Dr. Alexandre Messier is the inventor of a redirection solution via a web application and works as a medical consultant for Logibec, the company responsible for its marketing and distribution.