Workarounds have been identified, analyzed, and described in various health care contexts, and in different ways; these include observations [32], interviews [20,31], focus groups [30], questionnaire surveys [33], document analyses, time and motion measures, information systems data analyses, and predominantly a combination of the foregoing methods [34]. In our study, we will apply nonparticipant direct observation in combination with semistructured post-observation interviews for two main reasons. First, workarounds must be observed in vivo, while work practices and EHR use by health care professionals unfold in situ [38]. Second, and related to the former, workarounds tend to be invisible: EHR users may not have any interest in making their workarounds explicit in interviews or surveys. In fact, if users reveal their workarounds, they could be held accountable for not complying with guidelines of system use [38].

Direct observations will be complemented with a follow-up semistructured interview with each observed health care provider. Although we will make use of an interview protocol with predefined questions that are of particular importance to maintain coherency across the cases being studied [39], not all questions will necessarily be asked in a fixed order (or asked at all). New questions may be formulated during the interviews to gather more in-depth data related to the subject being discussed, or an issue being raised by the interviewee. The questions will primarily be related to the direct observations in which the interviewee is the main actor. We therefore maintain a more flexible approach by actively probing and listening to the interviewee, known as an open-ended interview [40]. This approach facilitates the use of a standardized list of questions, thereby enhancing internal validity and reliability, while retaining a degree of flexibility to adapt to situational interests ad hoc [39]. Each physician, nurse, and clerk will be observed and interviewed independently. If a potentially preventable medical error occurs while observing or interviewing a participant (and only if the workaround poses a serious and direct threat, to maintain our nonparticipant approach to observation), the participant will first be asked about the reason for their workaround, and later be informed of any potential preventable medical error(s) that may result from the workaround. In addition, although an estimation of the number of physicians, nurses, and clerks to be observed and interviewed is provided in Table 1, observations and interviews will continue until the research team agrees that data saturation is achieved.

All direct observations and interviews will be captured by means of a small audiovisual camera positioned at a designated, static location (see Figure 3 for an example setup). This procedure allows us to gather raw data from health care professionals and clerks using the system from which workarounds can be identified, to pinpoint moments in the processes when workarounds occur, and conduct follow-up analyses to gain deeper insights into how and why the workaround occurred. To mitigate the Hawthorne effect during the observations and audiovisual recordings, we will clearly communicate to the participants what is in it for them. It will be made clear that participants have no reason to use their EHR in any different way than they normally would. First, we will explain that participating in the research project is an opportunity to improve the EHR and thereby reduce potentially negative impacts on patient safety, effectiveness, and efficiency of care. Moreover, we will stress that we will be observing the EHR rather than the participant him/herself. Second, we will clearly communicate to participants that all data gathered will be anonymous, cannot be traced back to them, and will not be shared with anyone but the research team. This approach will reassure participants that they can use their EHR as they normally would, without fear of potentially being reprimanded or rebuked afterwards. Third, the audiovisual camera will be permanently and unobtrusively installed for the duration of the observation, and does not require frequent maintenance or recalibration. Finally, observers will be positioned at a safe distance from the clinician using the EHR [41].

The audiovisual recordings will be imported into a software application named ATLAS.ti and will be subject to further processing after this first research phase. All physicians, nurses, clerks, and patients will be asked for informed consent before any audiovisual recording takes place. The study has been proposed to, and discussed with, the chief of medical staff and the director of operations. We gained approval and support from these parties to proceed with the study, and no institutional review board approval was required. To protect patients’ health information, all audiovisual recordings will be stored on an encrypted hard drive set to erase itself after a series of incorrect password entries, and subject to editing, during which any patient names or contact details will be blurred or blanked out.

To study the EHR from a broader perspective within the university hospital, data will be gathered within three pediatric specialties: hematology, immunology, and infectious diseases. These specialties use the same EHR, of which the look and feel is identical, but may use additional functionalities tailored to each specialty. Within each specialty, providers will be observed while using the system in three distinct processes: the preparation of outpatient consultation, providing outpatient consultation, and providing inpatient consultation. We will deliberately analyze multiple distinct processes to create variation with regard to physical environment in which the EHR is used, tasks performed, and outcomes produced (see Figure 1). We expect to find different types of context-dependent workarounds in each of these processes. The preparation and data collection procedures per process are explained in the following subsections.

After the data collection phase, we will transcribe the audiovisual recordings of the direct observations and interviews. We will purposefully transcribe the recordings ourselves, as this will aid in data interpretation by developing affinity with the transcriptions. Each audiovisual recording of the observations and interviews will be transcribed in a separate Microsoft Word document. These files, including the original audiovisual recordings, will be imported into ATLAS.ti as primary documents within a hermeneutic unit. Within these primary documents, quotations will be created for selected text sections or video frames possibly related to a workaround resulting from EHR usage. Quotations are independent objects without any codes assigned to them and can be regarded as bookmarks within the data set. After all transcriptions have been processed, the research team will jointly review each transcription and quotation to determine whether (1) the quotation indeed relates to a workaround of the EHR, (2) there is consistency among the quotations in terms of the range (eg, selected string length or number of video frames) of the selected data, (3) minimal discrepancies exist between the audiovisual fragments and transcribed text, and (4) to ensure no relevant sections of data were overlooked.

Two approaches to coding have been considered while designing this study: top-down and bottom-up. Top-down coding would require the configuration of a set of predetermined codes from existing literature on EHR workarounds [22,28,34]. We would then simply match our data against the predetermined codes and develop new codes if a quotation would not fit in the existing classification of codes. In contrast, using bottom-up coding, we would develop the codes ourselves with the naming of codes constantly being tentative and subject to change. As more data is analyzed over time, the tentativeness of the coding taxonomy would eventually develop itself into a set of codes that fit the data well [46]. Despite being more time-demanding, we will use a bottom-up approach, as this will allow us to generate potentially new types of categories that may not emerge from a top-down approach, due to potential analytical bias (ie, forcing data into predetermined categories).

In line with a bottom-up approach, a provisional coding taxonomy will be developed by the lead researcher based on impressions and notes taken during each observation and interview, before coding of the transcriptions commences. This provisional coding taxonomy provides the coding team with a birds-eye overview of what has been witnessed during the EHR sessions with users in each of the three processes. This process will generate a temporary list of codes to be assigned to the data, to prevent each coder from developing a unique list of codes, and to ensure that coders use the same names for sections of data when their interpretations of the transcriptions are identical.

To establish common ground among members of the coding team before coding the transcriptions, one or multiple plenary educational sessions will be organized in which the team will be instructed on the EHR, the coding scheme, the contents of the provisional coding taxonomy, the meaning of each code, and the basics of coding in ATLAS.ti. To achieve a sufficient level of consistency and quality among coders, they will be asked to code the same copy of a random interview transcription using the provisional coding taxonomy before the actual process of coding the transcriptions starts. The copies will then be merged in ATLAS.ti to create a single analyzable file that contains all actions performed by all coders. Results will then be compared and any discrepancies or ambiguities will be discussed. If the coding scheme turns out to be ambiguous, the lead researcher will adjust the taxonomy and coding responses will be recalibrated.

After the provisional coding taxonomy is finalized, the coding team will begin open coding. Initially, two coders will independently code five similar randomly chosen transcriptions using the provisional coding taxonomy. One or multiple codes may be assigned to each quotation. When data do not fit into codes of the provisional taxonomy, new codes may be proposed by the coders. Coders may likewise propose alternative ways of labeling the codes. The research team will then come together and compare the results of the coders. Any discrepancies related to the codes assigned for the same unit of text or video stills will be resolved through discussion. The provisional coding taxonomy will be adjusted accordingly by the lead researcher and in collaboration with the coders, if deemed necessary. Whenever the coding taxonomy is altered throughout the research project, the transcriptions that were already processed will be reviewed again to determine whether all quotations assigned to a code still match the revised coding taxonomy. The same holds true if a code has been broken into multiple codes, or multiple codes have been merged into a single one.

We expect the tentativeness of the coding taxonomy to develop itself into a set of codes that fit the data well, after this initial round of coding. Most of the remaining transcriptions will be independently coded by the coding team. An independent reviewer will review the coded transcriptions on a regular basis and signal the research team if inconsistencies are noticed (eg, continuously using an inappropriate code for quotations with similar semantics). The research team will then resolve the inconsistencies to ensure that the predefined codes are used by all coders in the same way.

When all transcriptions have been coded and validated by the research team, a random sample of identical transcriptions that have been independently coded by at least two coders will be merged in ATLAS.ti. This process will create a single analyzable file containing all actions performed by all coders. Within this file, interrater reliability and interrater agreement of codes assigned to transcriptions will be calculated. We aim to do this for 30% of the transcriptions (usually 10-20% [47]). Interrater reliability will tell us whether there is consistency among the coding team with regard to selecting the same codes for the same unit of analysis (ie, quotation) while coding in isolation. Interrater agreement will tell us the extent to which coders are able to reconcile through discussion (and mediation by the independent reviewer in case the coders fail to reach consensus) if coding discrepancies arise for the same unit of analysis [48].

Finally, the number of quotations associated with each code will be tabulated to provide insights into which codes are more prevalent, both overall and within each of the three different health care settings. A high number of quotations associated with a given code may prompt further investigation during follow-up interviews and provide clues as to why the given code occurs more often than others.

An unanticipated consequence is a consequence of an EHR workaround that has not been foreseen in advance [49]. An unanticipated consequence can be either desirable or undesirable. Desirable unanticipated consequences are unforeseen consequences that turn out to have a favorable impact on an individual or the social system in which the EHR workaround occurred; these consequences can be described as serendipity. In the words of Ash et al [49], these consequences can be regarded as, “happy surprises”. Examples include increased collaboration and learning from alert messages, or ordering a wrong drug purposefully to trigger the alert system to suggest the right one.

Undesirable unanticipated consequences are unforeseen consequences that turn out to have an unfavorable impact on an individual or the social system in which the EHR workaround occurred; these consequences can be termed unintended consequences. Examples include health care professionals suffering from alert fatigue due to an overload of alerts generated by the EHR with low specificity [19,25], paper persistence [25], deteriorated communication and cooperation among health care professionals [51-53], workflow issues [25], difficulties in finding information in the system [24], never-ending system demands [25], or a human-computer interface that is unsuitable for a highly interruptive health care context [26].

Patient safety is a broad discipline that has garnered increasing attention since the 1990s and has become a cornerstone of delivering high-quality health care [54]. The Institute of Medicine defines patient safety as, “the prevention of harm to patients” [55]. To tailor this definition to our context, we define patient safety as any EHR-related incident that could possibly harm one or multiple patients receiving care.

EHRs are regarded as essential to improving patient safety [7]. However, recent evidence highlights substantial and often unanticipated patient risks resulting from the use of EHRs or workarounds [34,56-58]. The safe delivery of patient care can be jeopardized by bypassing security blocks (eg, working in emergency mode in nonemergency situations and thereby omitting security checks) [28], cloaking deficiencies (ie, devising workarounds rather than bringing problems to the attention of systems designers, causing problems to remain hidden) [59], and undermining standardization (eg, using an alternative way to accomplish a task, thereby not conforming to a system-enforced way of working that is designed to safeguard patient safety) [60]. Understanding how consequences of EHR workarounds could impact patient safety is therefore key when formulating design and redesign interventions for EHRs.

According to ISO 9241-11 (1998), effectiveness can be defined as the accuracy and completeness with which users achieve specified goals [61]. Workarounds often have a negative impact on the effectiveness of user-EHR interaction; specifically, they have been found to result in information on patient care (processes) or work protocols that are unstable, unavailable, or unreliable [27]. To achieve closer alignment between work context and EHR design, it will be important to understand the impacts that workaround consequences have on the accuracy and completeness of the goals that EHR users hope to achieve.

According to ISO 9241-11 (1998), efficiency can be defined as resources expended in relation to the accuracy and completeness with which EHR users achieve goals [61]. As previously mentioned, the ratio between provider-EHR and provider-patient interaction demands careful balancing [43-45]. Recent evidence confirms that EHRs claim a significant portion of physicians' time and draw attention away from their direct interactions with patients, and from their personal lives [62,63]. Physicians may spend 3-4 hours on EHR tasks and desk work for every hour of direct clinical time spent with patients [63]. Similar to the effectiveness of care potentially being jeopardized by workarounds, the same holds true for efficiency of care, as work protocols enforced by EHRs that are unstable, unavailable, or unreliable are sources of inefficiency [27].