This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Health care providers resort to informal temporary practices known as workarounds for handling exceptions to normal workflow that are unintentionally imposed by electronic health record (EHR) systems. Although workarounds may seem favorable at first sight, they are generally suboptimal and may jeopardize patient safety, effectiveness, and efficiency of care. Identifying workarounds and understanding their motivations, scope, and impact is pivotal to support the design of user-friendly EHRs and achieve closer alignment between EHRs and work contexts.
We propose a study protocol to identify EHR workarounds and subsequently determine their scope and impact on health care providers’ workflows, patient safety, effectiveness, and efficiency of care. First, knowing whether a workaround solely affects the health care provider who devised it, or whether its effects extends beyond the EHR user to the work context of other health care providers, is key to accurately assessing its degree of influence on the overall patient care workflow. Second, knowing whether the consequence of an EHR workaround is favorable or unfavorable provides insights into how to address EHR-related safety, effectiveness, and efficiency concerns. Knowledge of both perspectives can provide input on optimizing EHR designs.
In the study, a combination of direct observations, semistructured interviews, and qualitative coding techniques will be used to identify, analyze, and classify EHR workarounds. The research project will be conducted within three distinct pediatric care processes and settings at a large university hospital.
Data was collected using the described approach from January 2016 to March 2017. Data analysis is underway and is expected to be completed in May 2017. We aim to report the results of this study in a follow-up publication.
This study protocol provides a grounded framework to explore EHR workarounds from a holistic and integral perspective. Insights from this study can inform the design and redesign of EHRs to further align with work contexts of healthcare professionals, and subsequently lead to better organization and safer provision of care.
In recent years, an increasing number of hospitals around the world have implemented electronic health record (EHR) systems [
Multiple studies have reported on desirable outcomes of EHRs. Examples include improvements related to patient safety [
These undesirable and unanticipated consequences of EHR adoption can have negative and unintended effects on the overall health care organization and its work processes (and the outcomes thereof), and have frequently been subject to further examination. When the practices of health care providers are unintentionally but negatively influenced by mismatches between EHR designs and actual workflows, providers devise so-called
Although workarounds may solve the exceptions that EHRs impose upon the ordinary workflows of their users, they are generally suboptimal, as the EHR fails to live up to the goals of its implementation (ie, improving the practices of health care providers) and may negatively influence the safety, effectiveness, and efficiency of care. Understanding why and how workarounds occur is pivotal to develop user-friendly EHRs, and to achieve greater alignment between work context and the EHR [
To date, research into the scope and impact of EHR usage-related workarounds on overall patient care processes has been limited. First, concerning the scope of EHR workarounds, it is crucial to know whether a workaround affects a single EHR user who devised it, or whether its effects extend beyond the EHR user to the work context of other health care providers, to accurately assess its impact on the overall patient care workflow. Second, knowing whether the consequence of an EHR workaround is favorable or unfavorable provides insights into how to address EHR-related safety, effectiveness, and efficiency concerns. Knowledge of both perspectives can provide input on optimizing EHR design.
This study protocol proposes a way of identifying, analyzing, and classifying EHR workarounds to determine their scope and impact on the patient care process. Within a large university hospital, we intend to conduct direct observations of (and semistructured interviews with) health care providers while they use EHRs in three different processes, each taking place in a distinct physical environment: the preparation of outpatient consultations in private offices of health care professionals, actual outpatient consultations in examination rooms, and actual inpatient consultations with admitted patients in wards. The research design, clinical setting, and methods to be used in the research project are described in the following section.
To address the aim of determining the scope and impact of EHR-related workarounds, we adapted one of the most widely used health care human factors systems frameworks, the Systems Engineering Initiative for Patient Safety (SEIPS) framework [
In our study
In the study, we will investigate workarounds by means of direct observations and semistructured interviews in three processes. Each process will take place in a distinct physical environment: the preparation of outpatient consultations in private offices of health care professionals, actual outpatient consultation in examination rooms, and actual inpatient consultation with patients admitted into wards. Workarounds occurring within these three processes can have consequences that affect the outcomes of each process. We will determine the scope of each workaround to the patient, the health care professional, and the overall organization level, or a combination thereof. Furthermore, to determine the impact of each workaround, we will classify whether its consequence is favorable or unfavorable, and assess its impact on patient safety, patient care effectiveness, and efficiency.
Due to the unique nature of each health care setting to be studied, the direct observations and semistructured interview procedures will vary per setting. The research project involves six major chronological phases, as illustrated in
Summary of research design by process to be studied.
Process | Preparing outpatient consultation | Providing outpatient consultation | Providing inpatient consultation |
Approximately 12 physicians, 6 nurses, and 3 clerks (same staff as in |
Approximately 12 physicians, 6 nurses, and 3 clerks (same staff as in |
Approximately 12 physicians, 6 nurses, and 3 clerks | |
(1) Must have completed the required training to use EHR, and (2) must have used EHR from the moment of its implementation | (1) Must have completed the required training to use EHR, and (2) must have used EHR from the moment of its implementation | (1) Must have completed the required training to use EHR, and (2) must have used EHR from the moment of its implementation | |
Private office | Examination room | Inpatient ward | |
User-system | User-patient, user-system | User-patient, user-system | |
Direct observation while preparing outpatient consultation (1-2 hours), asking opportunistic questions while observing, semistructured follow-up interviews (1 hour) | Direct observation while providing outpatient consultation (4-6 hours), semistructured follow-up interviews (1 hour) | Direct observation during ward rounds and post-ward round EHR usage (4 hours), semistructured follow-up interviews (1 hour) | |
Transcribing and subsequent bottom-up coding of audiovisual recordings in ATLAS.ti | Transcribing and subsequent bottom-up coding of audiovisual recordings in ATLAS.ti | Transcribing and subsequent bottom-up coding of audiovisual recordings in ATLAS.ti |
Conceptual framework to study electronic health record workarounds, adapted from Holden et al [
Illustration of the six research phases to be conducted. EHR: electronic health record.
Workarounds have been identified, analyzed, and described in various health care contexts, and in different ways; these include observations [
Direct observations will be complemented with a follow-up semistructured interview with each observed health care provider. Although we will make use of an interview protocol with predefined questions that are of particular importance to maintain coherency across the cases being studied [
All direct observations and interviews will be captured by means of a small audiovisual camera positioned at a designated, static location (see
Example of a data collection setup in an outpatient consultation room.
The audiovisual recordings will be imported into a software application named ATLAS.ti and will be subject to further processing after this first research phase. All physicians, nurses, clerks, and patients will be asked for informed consent before any audiovisual recording takes place. The study has been proposed to, and discussed with, the chief of medical staff and the director of operations. We gained approval and support from these parties to proceed with the study, and no institutional review board approval was required. To protect patients’ health information, all audiovisual recordings will be stored on an encrypted hard drive set to erase itself after a series of incorrect password entries, and subject to editing, during which any patient names or contact details will be blurred or blanked out.
To study the EHR from a broader perspective within the university hospital, data will be gathered within three pediatric specialties: hematology, immunology, and infectious diseases. These specialties use the same EHR, of which the
The first process concerns health care professionals preparing outpatient visits while using the EHR in their private offices. When preparing outpatient consultations an EHR user (eg, physician or nurse) primarily relies on the availability, retrievability, and quality of the data stored in the EHR by colleagues and the user him/herself, as important clinical information is often hidden in a
The second process of an outpatient consultation concerns obtaining the medical history from patients, conducting a physical examination, and ordering laboratory tests or medication while a health care professional uses the EHR in a designated examination room. Since these activities are regularly carried out in a limited time frame per patient, the ratio between provider-system and provider-patient interaction demands careful balancing [
The third and final setting concerns health care professionals using the EHR after having made inpatient ward rounds. These rounds concern regular daily reviews and consultation of hospitalized patients with regard to their medical condition, medication, and progress. Subsequently, the EHR is used to change drug prescriptions, order blood or other laboratory tests, and document the patient visit. The interface of the EHR differs from the interface shown to users in the first and second processes, thereby providing unique insights into usability-related workarounds. Similar to the first and second processes to be studied, a semistructured interview will be conducted after the observed session of the user interacting with the EHR.
After the data collection phase, we will transcribe the audiovisual recordings of the direct observations and interviews. We will purposefully transcribe the recordings ourselves, as this will aid in data interpretation by developing affinity with the transcriptions. Each audiovisual recording of the observations and interviews will be transcribed in a separate Microsoft Word document. These files, including the original audiovisual recordings, will be imported into ATLAS.ti as
Two approaches to coding have been considered while designing this study:
In line with a bottom-up approach, a provisional coding taxonomy will be developed by the lead researcher based on impressions and notes taken during each observation and interview, before coding of the transcriptions commences. This provisional coding taxonomy provides the coding team with a birds-eye overview of what has been witnessed during the EHR sessions with users in each of the three processes. This process will generate a temporary list of codes to be assigned to the data, to prevent each coder from developing a unique list of codes, and to ensure that coders use the same names for sections of data when their interpretations of the transcriptions are identical.
To establish common ground among members of the coding team before coding the transcriptions, one or multiple plenary educational sessions will be organized in which the team will be instructed on the EHR, the coding scheme, the contents of the provisional coding taxonomy, the meaning of each code, and the basics of coding in ATLAS.ti. To achieve a sufficient level of consistency and quality among coders, they will be asked to code the same copy of a random interview transcription using the provisional coding taxonomy before the actual process of coding the transcriptions starts. The copies will then be merged in ATLAS.ti to create a single analyzable file that contains all actions performed by all coders. Results will then be compared and any discrepancies or ambiguities will be discussed. If the coding scheme turns out to be ambiguous, the lead researcher will adjust the taxonomy and coding responses will be recalibrated.
After the provisional coding taxonomy is finalized, the coding team will begin open coding. Initially, two coders will independently code five similar randomly chosen transcriptions using the provisional coding taxonomy. One or multiple codes may be assigned to each quotation. When data do not fit into codes of the provisional taxonomy, new codes may be proposed by the coders. Coders may likewise propose alternative ways of labeling the codes. The research team will then come together and compare the results of the coders. Any discrepancies related to the codes assigned for the same unit of text or video stills will be resolved through discussion. The provisional coding taxonomy will be adjusted accordingly by the lead researcher and in collaboration with the coders, if deemed necessary. Whenever the coding taxonomy is altered throughout the research project, the transcriptions that were already processed will be reviewed again to determine whether all quotations assigned to a code still match the revised coding taxonomy. The same holds true if a code has been broken into multiple codes, or multiple codes have been merged into a single one.
We expect the tentativeness of the coding taxonomy to develop itself into a set of codes that fit the data well, after this initial round of coding. Most of the remaining transcriptions will be independently coded by the coding team. An independent reviewer will review the coded transcriptions on a regular basis and signal the research team if inconsistencies are noticed (eg, continuously using an inappropriate code for quotations with similar semantics). The research team will then resolve the inconsistencies to ensure that the predefined codes are used by all coders in the same way.
When all transcriptions have been coded and validated by the research team, a random sample of identical transcriptions that have been independently coded by at least two coders will be merged in ATLAS.ti. This process will create a single analyzable file containing all actions performed by all coders. Within this file, interrater reliability and interrater agreement of codes assigned to transcriptions will be calculated. We aim to do this for 30% of the transcriptions (usually 10-20% [
Finally, the number of quotations associated with each code will be tabulated to provide insights into which codes are more prevalent, both overall and within each of the three different health care settings. A high number of quotations associated with a given code may prompt further investigation during follow-up interviews and provide clues as to why the given code occurs more often than others.
The fourth phase aims to analyze the identified workarounds regarding their scope. As previously mentioned, each workaround will be related to its impact on the patient, the health care professional, the overall organization, or a combination thereof; this is in accordance with the
The fifth phase involves determining the consequences of each identified workaround. We have been inspired by the approach of Ash et al [
An unanticipated consequence is a consequence of an EHR workaround that has not been foreseen in advance [
Undesirable unanticipated consequences are unforeseen consequences that turn out to have an unfavorable impact on an individual or the social system in which the EHR workaround occurred; these consequences can be termed
The final phase involves determining the possible impact of EHR workaround consequences. One or multiple sessions will be organized to convene all members of the research team, health care professionals, and clerks participating in the study. The impact of each workaround consequence will then be collectively analyzed from three perspectives: patient safety, effectiveness of care, and efficiency of care.
A comprehensive list of indicators to determine the impact of EHR workaround consequences regarding the three perspectives is, to our knowledge, nonexistent. We will therefore develop a list of indicators following a bottom-up approach. Based on this list of indicators and garnered insights, our final aim is to develop a model of EHR workaround consequences and their possible impact on patient safety, effectiveness, and efficiency of care that builds upon the CPOE consequences model developed by Ash et al [
Patient safety is a broad discipline that has garnered increasing attention since the 1990s and has become a cornerstone of delivering high-quality health care [
EHRs are regarded as essential to improving patient safety [
According to ISO 9241-11 (1998), effectiveness can be defined as the accuracy and completeness with which users achieve specified goals [
According to ISO 9241-11 (1998), efficiency can be defined as resources expended in relation to the accuracy and completeness with which EHR users achieve goals [
Data was collected using the described approach from January 2016 to March 2017. Data analysis is underway and is expected to be completed in May 2017. We aim to report the results of this study in a follow-up publication.
Health care providers resort to informal work practices known as
Research on the scope and impact of EHR usage-related workarounds on the overall patient care processes is currently limited. Insights into the consequences of EHR workarounds on patients, health care providers, and health care organizations provide guidance on how to address EHR-related safety, effectiveness, and efficiency concerns, and to optimize EHR designs.
Our study protocol, based on the SEIPS conceptual framework [
In addition to reporting on identified workarounds to EHR usage in an academic hospital in multiple distinct processes and settings, our final aim is to develop a model of EHR workaround consequences and their impacts on patient care that builds upon the CPOE consequences model developed by Ash et al [
Computerized Physician Order Entry
Electronic Health Record
Systems Engineering Initiative for Patient Safety
VB, KK, MJ conceived and designed the study. VB wrote the manuscript. MJ edited the manuscript. All authors read and approved the final manuscript.
None declared.