This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Clinicians, such as respiratory therapists and physicians, are often required to set up pieces of medical equipment that use inconsistent terminology. Current lung ventilator terminology that is used by different manufacturers contributes to the risk of usage errors, and in turn the risk of ventilator-associated lung injuries and other conditions. Human factors and communication issues are often associated with ventilator-related sentinel events, and inconsistent ventilator terminology compounds these issues. This paper describes our proposed protocol, which will be implemented at the University of Waterloo, Canada when this project is externally funded.
We propose to determine whether a standardized vocabulary improves the ease of use, safety, and utility as it relates to the usability of medical devices, compared to legacy medical devices from multiple manufacturers, which use different terms.
We hypothesize that usage errors by clinicians will be lower when standardization is consistently applied by all manufacturers. The proposed study will experimentally examine the impact of standardized nomenclature on performance declines in the use of an unfamiliar ventilator product in clinically relevant scenarios. Participants will be respiratory therapy practitioners and trainees, and we propose studying approximately 60 participants.
The work reported here is in the proposal phase. Once the protocol is implemented, we will report the results in a follow-up paper.
The proposed study will help us better understand the effects of standardization on medical device usability. The study will also help identify any terms in the International Organization for Standardization (ISO) Draft International Standard (DIS) 19223 that may be associated with recurrent errors. Amendments to the standard will be proposed if recurrent errors are identified. This report contributes a protocol that can be used to assess the effect of standardization in any given domain that involves equipment, multiple manufacturers, inconsistent vocabulary, symbology, audio tones, or patterns in interface navigation. Second, the protocol can be used to experimentally evaluate the ISO DIS 19223 for its effectiveness, as researchers around the world may wish to conduct such tests and compare results.
Lung ventilators are frequently used in health care, and over 300,000 patients are ventilated in the United States every year [
The main objective of our study is to determine the ease of use, safety, and utility of standardized vocabulary as it relates to the usability of medical devices, compared to legacy medical devices from different manufacturers. We will focus specifically on lung ventilators and the “ISO Draft International Standard 19223 – Lung Ventilators and related equipment – Vocabulary and Semantics” [
Little work has been done to evaluate the impact of a standardized terminology on the usability of medical devices, especially on transitions across heterogeneous devices from different manufacturers. Bakhshi-Raiez et al [
Usability is associated with task performance, which is in turn associated with risk of complications and ventilator-associated lung injuries. Our proposed study will experimentally examine differences in task performance (ie, human error rate, task completion times) and error type (emerging from the use of a standardized versus nonstandardized nomenclature) on lung ventilator user interfaces in clinically relevant scenarios. Our research question and hypothesis are stated as follows:
Translation of terms between vocabularies.
Description | Type | Term in ISO DIS 19223 | Term in PB-840 | GE Engstrom |
Mode in which pressure is adjusted from inflation to inflation, and a set target volume is delivered | Mode | Volume-targeted pressure control (vtPC) | Volume Control Plus (VC+) | Pressure Control Ventilation - Volume Guaranteed (PCV-VG) |
Sleep apnea breathing therapy mode | Mode | Continuous positive airway pressure (CPAP) | Spontaneous (SPONT) | Continuous Positive Airway Pressure/Pressure Support Ventilation (CPAP/PSV) |
Mode in which two pressure levels are set for spontaneous breathing | Mode | Bi-level positive airway pressure (also bi-level PAP or BPAP) | BiLevel | BiLevel |
Ventilation-pattern in which a selected inflation-type (which is the primary inflation) is initiated at a set rate. Patient-trigger events may lead to additional primary inflations beyond the set rate | Mode class | Assist/Control Ventilation | Assist/control mode | “Assist control” mode is available through Volume Controlled Ventilation (VCV), Pressure Controlled Ventilation (PCV), Pressure Controlled Ventilation - Volume Guaranteed (PCV-VG) modes only |
Ventilation-pattern in which a selected inflation-type (which is the primary inflation) is initiated at a set rate; patient trigger events cause support inflations in which spontaneous breathing may occur; primary inflations are synchronized with any spontaneous breathing through “synchronization windows” | Mode class | Synchronized intermittent mandatory ventilation (SIMV) | SIMV, “mandatory breaths” can be volume or pressure-based | Several modes on - Synchronized Intermittent Mandatory Ventilation (SIMV) are present in the Engrstrom; SIMV Pressure Controlled (SIMV-PC), SIMV Volume Controlled (SIMV-VC), and SIMV Pressure Controlled, Volume Guaranteed (SIMV-PCVG). SIMV-VC, SIMV-PC, and SIMV-PCVG use a “trigger window” which is different from “synchronization windows” mentioned in the ISO DIS 19223 |
Baseline airway-pressure (BAP) or pressure level set above ambient pressure at which unassisted breathing may occur, and/or inflations may be superimposed | Setting | Baseline airway-pressure (BAP) | Positive end-expiratory pressure (PEEP) | PEEP |
Higher pressure level in the Bi-Level Mode | Setting | BAPH | PEEPH | Phigh |
Lower pressure level in the Bi-Level Mode | Setting | BAP | PEEPL | Plow |
High PEEP time or Inspiratory Time | Setting | BAPH Time or tH | TH | Thigh |
Low PEEP time or Expiratory Time | Setting | BAP Time or tL | TL | Tlow |
Setting for duration of inspiratory phase | Setting | Inspiratory Time or tI | TI | Tinsp |
Setting or measured quantity for airway pressure in an inspiratory or inflation phase | Setting, measured quantity | Inspiratory Pressure | PI | Pinsp |
Tidal Volume | Setting | VT | VT | Tidal Volume or TV |
To test this hypothesis, the proposed experimental study will compare performance declines resulting from the use of an unfamiliar ventilator model. These declines will be compared between two groups: a group of clinical participants provided with legacy ventilator models, and a group of clinical participants provided with ventilator models that are modified to include standardized nomenclature. The experiment will consider user interfaces on two types of ventilator models: one familiar to clinical participants and one unfamiliar to clinical participants. Multiple manufacturers provide lung ventilators, so clinicians (eg, respiratory therapists) within a given geographic region become trained and familiarized only with a subset of the lung ventilators that the global market is capable of providing. However, clinicians may encounter unfamiliar lung ventilators on an occasional basis. For each ventilator model’s user interface, a variant will be developed in which the original layout and navigational structure will be retained, while replacing its nomenclature with the ISO DIS 19223.
The familiar ventilator model will be the Puritan Bennett 840 (PB-840), and the unfamiliar model will be the General Electric (GE) Engstrom Carestation (referred to in this paper as
Usability of medical devices is of direct relevance to patient care. It is common for ventilators from several different manufacturers (with different terminology) to be used in hospital departments or different hospital units within a hospital system; this leads to increased risk in patient care [
Human factors and communication issues are inseparable from the issue of nomenclature inconsistency. The development of an ISO standard for lung ventilator nomenclature is a positive step towards mitigating patient safety risk with the use of lung ventilators. Our research protocol (and eventual study to evaluate the ISO DIS 19223) will help us better understand the potential benefits and barriers, if any, in the incorporation of the ISO DIS 19223 in lung ventilators.
The results of our proposed study will be valuable in the adoption of the ISO DIS 19223 in lung ventilators and in training programs and manuals. When the ISO DIS 19223 becomes widely used, insights from our proposed study will be useful to interface designers of lung ventilators. The proposed study plans to recruit experienced respiratory therapists, so any frequently recurring error(s) associated with specific ISO DIS 19223 term(s) may be of interest in clinician education, in manuals, and in the design of training materials incorporating the ISO DIS 19223. This proposed study will also provide insights about relationships between operator mental models, device nomenclature, and operator error types, which would be a useful human factor contribution applicable to other types of medical devices and instruments.
Lung ventilator models differ between manufacturers in terms of nomenclature. Differences are seen in definitions of various ventilation modes and other terminology, which affects training costs and human error in health care settings. The complexity of lung ventilators in use is a factor underlying patient complications and ventilator-associated lung injury. As an example of an adverse event, a 28-year-old in a neurological ICU was having difficulties with his/her ventilator; a respiratory therapist from a cardiothoracic ICU attending to this patient decided to change settings on the ventilator to improve the patient’s oxygen saturation [
As an example of conflict resulting from current inconsistencies in terminology, the term “breath” sometimes refers to an inflation performed by a lung ventilator, leading to ambiguity with the use of manuals and descriptions [
Chatburn [
In a 2014 Association for the Advancement of Medical Instrumentation/Food and Drug Administration summit on ventilator technology, it was noted that gaps exist in current clinical training for ventilator use and in the current state of ventilator terminology [
Advances in the standardization of lung ventilator nomenclature will have the most impact if user interfaces on lung ventilators and manuals adopt the standardized nomenclature. There has been no research done to examine or compare the effectiveness of two or more nomenclature systems in lung ventilators. The effectiveness of user interfaces is captured in the construct of usability. Therefore, usability is being incorporated in the proposed study to understand the potential impact that a standardized terminology could have. Usability is a multi-faceted construct that takes into consideration efficiency, errors, memorability, learnability, and satisfaction [
In summary, our paper reports an experimental protocol to evaluate a standardized nomenclature system for a medical device, and the evaluation will determine the extent to which the standardized nomenclature facilitates the work of clinicians in situations in which the clinician would need to operate an unfamiliar medical device. The evaluation will take clinician error, performance, and usability into consideration. The medical device we will focus on is the lung ventilator, and the standardized nomenclature is the ISO DIS 19223.
To test our hypothesis pertaining to nomenclature in medical equipment, the proposed research protocol focuses on lung ventilators while planning to involve respiratory therapy practitioners and trainees. The study will involve at least two lung ventilator models, one of which would be less popular in practice in the region of study, while the other would have a high level of familiarity due to common practice among clinical practitioners and trainees in the region of study. Mockups of interfaces on these ventilator models will be developed. Additionally, for each ventilator model being considered in the study, a variant incorporating the terminology in the ISO DIS 19223 will be developed. This protocol will involve the following tasks: (1)
In Task 1, we will develop mockups of the
There will be a total of four mockups (PB-840, PB-840-ISO, Engstrom, and Engstrom-ISO), as a variant for each model will be developed using of the ISO DIS 19223 nomenclature. In Phase 1, we will also prepare 10 clinically relevant scenarios, of which seven will be routine scenarios and three will be nonroutine critical scenarios. Each scenario will require the participant to specify or change modes and/or settings on a ventilator mockup. The training materials will be designed to familiarize participants with the ventilator mockups. Training materials for the PB-840, the GE Engstrom Carestation, and their ISO-standard variants will be prepared. Definitions of terms used on the ventilators will not be provided in training.
The objective of this task is to get feedback on experimental materials (ie, clinical scenarios, mockups) and the experimental protocol. This feedback will be elicited from experienced respiratory therapists and we will use the feedback to refine the experimental materials and protocol. The task will mainly involve a pilot run of the main experiment with think-alouds and unstructured interviews. We will recruit approximately 12 experienced respiratory therapists [
We plan to recruit 60 participants with training and/or clinical experience with ventilators. A power analysis was conducted with G*Power. Considering a medium effect size (effect size f of .25), alpha of .05, power of .80, and the assumption of a moderate correlation (r=.5) between performance measures across ventilator types, the required sample size is estimated to be 48 participants. However, participant data may need to be excluded due to performance considerations or technical issues. Therefore, a target participant pool of 60 would be appropriate for this study. This sample will consist of 30 students undergoing training in respiratory therapy and 30 registered respiratory therapists who have been in practice for over two years. This timeframe is equal to (or higher than) the experience level of
Planned timeline for recruitment of clinical participants. PB-840: Puritan Bennett 840 ventilator; GE: General Electric; DIS: Draft International Standard; RT: respiratory therapist.
Dependent Variables
Response accuracy: number of inaccurate mode or setting selections (in a scenario set of 10 scenarios)
Average time for completion (taken across ten scenarios in one set)
Responses to subjective ratings on interface evaluation questionnaire [
Independent Variables
The experiment is designed to simulate situations in which a clinician undergoes a transition from familiar to unfamiliar equipment, as such situations could give rise to clinician errors. Participants (both trainees and therapists) will be randomly placed in one of two groups: the control group or the ISO group. A summary of the experimental variables is presented in
Experimental sessions may be videotaped, with the camera focused on the mockup screens only. All participants will be required to perform 10 scenarios on each ventilator model, which will include seven common scenarios that are routine in clinical settings and three emergency/nonroutine scenarios. Nonroutine scenarios or nonroutine events are events that would appear to be atypical to health care providers, may cause disruptions in the process of care delivery, and may result in cognitive deliberation in addition to what a routine event may demand; they also represent a class of events broader than
Set up the ventilator to the mode in which two pressure levels are set for spontaneous breathing, wherein the upper pressure should be set to 20 cmH2O, lower pressure set to 7 cmH2O, time at upper pressure should be 1.5 seconds, breathing rate set to 10, and percentage of oxygen by volume set to 28%. On the ventilator provided to you, change the settings to what would be appropriate for the patient.
Change the ventilator mode to the one in which pressure control inflations should be initiated at a rate of 10 while additional patient-trigger events would increase inflations of the selected type (ie, pressure control). Set inspiratory pressure to 15 cmH2O and baseline pressure at 5cmH2O above ambient pressure. The percentage of oxygen by volume should be 30%.
Participants will be instructed to respond to each scenario using their ventilator mockups, one at a time, and a time limit will be provided for each scenario. We will log start and end times for each scenario, which will be used to calculate response times for each scenario. When a participant does not respond accurately in a given scenario, the experimenter will probe the participant with questions. The inaccurate responses will also be recorded for analysis. Responses to these probes will be audio recorded, and they will be used to classify the error(s) based on Rasmussen’s error classification [
Timeline of an experimental session for the experiment. Each experimental session is expected to last approximately 80 minutes, including time for demographic surveys. PB-840: Puritan Bennett 840 ventilator; ISO: International Organization for Standardization; DIS: Draft International Standard; S: scenario set.
Task 5 consists of statistical analyses and a qualitative analysis. The statistical analyses will mainly involve response times, accuracy, and subjective ratings, and will focus on performance differences between groups and ventilator types. We will run a mixed factorial analysis of variance to detect differences of statistical significance across groups and ventilator types. We will also apply statistical tests to detect any potential relationships between task performance and experience. Verbal responses to probe questions will be qualitatively analyzed [
The work reported here is in the proposal phase. Once the protocol is implemented, we will report the results in a follow-up paper.
In a number of domains in which time-critical tasks are performed with complex equipment, health care providers may be required to occasionally work with various manufacturers' models of equipment with which providers are unfamiliar. During such transitions, providers must cope with unfamiliar symbology, terminology, proprietary manufacturers' terms, audio tones, or patterns in interface navigation. Such transitions can be a cause for error, potentially leading to hazardous situations that result in environmental damage, patient morbidity, or mortality. To mitigate the risk of human error, efforts can be made to standardize terminology for instruction manuals, displays, and controls (eg, alarm signals of differing equipment used in critical care). Previous attempts to assess the usability of medical systems incorporating a specific terminology in the context of operation in realistic settings have been limited [
We would like to indicate a few limitations in our protocol. Replication of this protocol for a planned experimental study in an industry or university laboratory would not be bound by the same limitations. First, there is a limit on the number of ventilator models that we can integrate into our study. The protocol reported in our paper includes only two ventilator models (PB-840 and the GE Engstrom). However, it is recommended to include more ventilator models, and we may include more ventilator(s) depending on availability of other ventilators and industry participation. Second, we will be using mockups, although it would be ideal to use the actual medical devices that have the ability to record data. Finally, nurses and clinicians other than respiratory therapists are not included in our protocol. Nurses may be regular operators of lung ventilators in some developing countries, and the inclusion of such nurses in an experimental protocol may be beneficial. A follow-up study can also be conducted to compare nurses with respiratory therapists.
Architecture, Interface, Terminology, Information, Nursing, and Knowledge
baseline airway-pressure
continuous positive airway pressure
Draft International Standard
General Electric
intensive care unit
International Organization for Standardization
Puritan Bennett 840 ventilator
Pressure Controlled Ventilation
Pressure Controlled Ventilation - Volume Guaranteed
positive end-expiratory pressure
Synchronized Intermittent Mandatory Ventilation (in the ISO DIS 19223) or Synchronous Intermittent Mandatory Ventilation (in the PB-840)
Synchronized Intermittent Mandatory Ventilation - Pressure Controlled
Synchronized Intermittent Mandatory Ventilation - Pressure Controlled, Volume Guaranteed
Synchronized Intermittent Mandatory Ventilation - Volume Controlled
Systematized Nomenclature of Medicine - Clinical Terms
We would like to thank GE Healthcare and Covidien for generously donating lung ventilators to the University of Waterloo. We appreciate the support and input from Justin St-Maurice of Conestoga College and Patrick Nellis of Draeger Medical Canada Inc. Dev would like to thank Prof. Karen Feigh of Georgia Tech for her support. We are thankful for funding support from the Natural Sciences and Engineering Research Council discovery grant 132995 and from a Telus Health contract.
None declared.