The study is designed to assess the effect of an e-learning module on all three of the proposed levels of medical education translational research: knowledge and skills, patient care practice, and patient outcomes [24]. A “dilution” of the effect should be expected in the process of changing clinicians’ knowledge, in their behavior as well as actual patient outcomes [23]. Therefore, data from a large number of anesthesia procedures are likely required to obtain adequate strength. To make this feasible, we have chosen to conduct the study at all major hospitals in the Zealand Region of Denmark, where data on use of neuromuscular monitoring are automatically recorded in the anesthesia information management system MetaVision (iMDsoft^®, Düsseldorf, Germany). Furthermore, we have chosen the interrupted time series design, in which data are collected repeatedly, at fixed intervals, before and after the intervention (Figure 1).

The effect of the intervention is analyzed statistically using segmented regression analysis, testing for changes in both the level and the trend of the outcome. The design is proposed to be the strongest quasi-experimental approach for assessing the longitudinal effect of interventions and also allows description of the timing of the effect of the intervention [25]. This design is well suited for our setting, where the number of participating departments is limited to the 6 departments of anesthesia at the teaching hospitals in the Zealand Region of Denmark, making experimental designs such as cluster randomized trials unfeasible [26]. Baseline data are being obtained from the descriptive study “Use of neuromuscular blocking agents and neuromuscular monitoring in 7 Danish teaching hospitals—a cross-sectional study” (NCT02914119), in which data are collected from the same hospitals in the year leading up to the intervention. The e-learning module was implemented over a period of 2 weeks, and we area again collecting data from all departments at fixed 3-week intervals.

The investigator group has developed the e-learning module in collaboration with the Regional Unit for Development and Evaluation of Learning Technologies in the Capital Region of Denmark. Learning objectives for the module are based on a needs assessment, including focus group interviews with nurse anesthetists and anesthesiologists from 5 different Danish hospitals and clinical observations of neuromuscular monitoring use in practice (NCT02239965) as well as a previous Danish survey [18] and expert opinions. The main topics in the course are as follows: (1) background and clinical consequences of residual blockade, (2) neuromuscular monitoring and stimulation patterns, (3) practical tips on monitoring and troubleshooting equipment malfunction, and (4) reversal of neuromuscular blockade. The course participants are of different educational backgrounds and have anesthesia experience ranging from short periods to decades of employment. We have aimed to target this diversity in participant background by making the e-learning module adaptable to learners’ needs, specifically by making part of the material optional (Figure 2) [27]. Furthermore, clickable animations are included to stimulate interaction and increase learning [28], while examples of monitor output make the module relevant to participants’ daily practice. The course duration is approximately 30 minutes.

We recruited a local investigator at each study site. This person is responsible for motivating the anesthesia personnel to complete the e-learning module and introduce all new employees to it. On the first day of implementation, the local investigators introduced the e-learning module. The primary investigator also visited the departments in the intervention period and introduced the study, but without revealing the primary outcome to the participants. The participating departments ensured that the personnel could complete the module during their normal work hours, such as by having colleagues cover for them while they complete the module. Course progress was logged for each anesthetist, and email reminders were sent to ensure the highest possible completion rate. The local investigator and the head of department were also informed about the completion rate throughout the implementation period. To increase awareness about the study and the e-learning module, the local investigators distributed candies and pens with the INVERT study logo at intervals. New employees joining the departments after the intervention period were also included in the study. The module is available for ongoing access, meaning that learners can still access the module, such as for use as a clinical troubleshooting tool. All anesthetic practices and medication use are at the discretion of the individual anesthetist according to local guidelines.

Given the nature of the intervention, it was not possible to blind the anesthesia personnel. However, they are not informed of the specific outcomes, only that the study “aims to assess the effect of the e-learning module on use of neuromuscular monitoring”. The investigators in charge of data analysis will not have access to the data until after the final data collection period.

We will assess the effect of the e-learning module on knowledge, behavior, and patient outcomes. A meta-analysis of studies on the effect of Web-based learning indicates that an e-learning module will have greatest effect on knowledge, a smaller effect on skills, and a yet smaller effect on behavior and patient-related outcomes [23]. We have chosen the primary outcome based on what we believe to be of highest clinical relevance, balanced with the feasibility of collecting data on this outcome from thousands of patients.

The primary outcome is divided because the two types of relaxants are used in different clinical situations: succinylcholine for “rapid sequence induction and intubation” and nondepolarizing relaxants for nonemergent tracheal intubation.

Secondary analyses that may be performed if the data allow it include the incidence of respiratory complications, arterial oxygen desaturation, and the effect of the e-learning module at the individual level.

Data are recorded in the MetaVision anesthesia information management system either automatically or manually, by the anesthetist. TOF data are recorded automatically when the neuromuscular monitoring module is activated. The anesthesia information management system saves TOF data every minute, even if the neuromuscular monitoring module measures a TOF value every 12 seconds. To ensure that the reported TOF values at the time of tracheal extubation are not erroneously measured due to lack of fixation of the patient’s hand, movements caused by the surgical personnel, or other factors, we will collect the last five TOF values before tracheal extubation and use the highest value, given that the values increase incrementally. We will develop an algorithm for choosing the last correctly measured TOF value and will perform manual validation on part of the dataset. Data manually entered into the anesthesia information management system include the type and dose of intravenous drugs and the time of tracheal extubation. The anesthetist enters these data by pressing a button in the software and can alter the time and the details, such as the dose of medicine administered, afterward if needed. Where possible, we will seek to validate these manually entered data, such as by comparing them to changes in ventilatory data from the anesthesia machine. In case of doubt about the correctness of data for a particular case, we will access the full dataset for that case in order to investigate further.

A simulation-based sample size calculation revealed that the sample size necessary to detect an increase in neuromuscular monitoring from 40%-60% (with an 80% power) would require 3 consecutive data collection periods, each comprising approximately 100 patients per site, both before and after the implementation of the intervention. We expected to achieve this sample size by letting each period be 3 weeks long. The baseline value of 40% was based on a 1-month data sample from quality assurance at the participating hospitals, while the increase to 60% reflects the minimal effect size that we believe would make it worthwhile to disseminate the course to other departments. It is possible that the effect of the module could “wear off” after the 9 weeks of postintervention data collection if either a loss of knowledge occurs or the motivation to use neuromuscular monitoring decreases after the study period. To assess this, we extended the postintervention data collection period beyond what the sample size calculation indicated. This extension is ethically justifiable because prolonging the data collection does not affect patient treatment and because the data are already automatically collected in the anesthesia information management system.

When analyzing the data, we will use segmented regression analysis [25]. All results will be reported with 95% confidence intervals when relevant. We will test continuous variables for normality by visual inspection before summarization with the mean (standard deviation). Non-normally distributed continuous variables and ordinal variables will be described as the median (95th percentile range) and categorical variables as the number (percentage), if not otherwise specified. Analyses will be performed using SPSS version 22 (SPSS Inc., Chicago, IL, USA). The investigators JLT and LTS will conduct the analyses. We will consider a two-tailed P value of less than 0.05 to be statistically significant.

In the analysis of the secondary outcomes, we will exclude cases with missing data on the outcome studied, that is, we will perform only per-protocol analyses. However, in reporting of the median TOF ratio at tracheal extubation, we will include both an analysis where a TOF count between 0 and 3 (and hence no TOF ratio reported) is included as a TOF ratio of 0 and a per-protocol analysis where only cases with a TOF ratio are included. Furthermore, we will report the proportion of patients transferred from the operating room while still tracheally intubated and, if possible, report the cause for this, such as severe residual neuromuscular blockade.

The study solely uses data routinely registered in the anesthesia information management system. The Committees on Health Research Ethics for the Capital Region of Denmark have confirmed that the study does not need approval from the committee system and that there is no need for individual patient consent (protocol no. 16028220). The study is registered at clinicaltrials.gov (NCT02925143) and with The Danish Data Protection Agency (HGH-2015-063/04364).

One of the advantages of Web-based learning, such as an e-learning module, is that participant numbers can be increased with only small expenses [29]. If the e-learning module proves effective in increasing the use of neuromuscular monitoring, it may be implemented in all Danish anesthesia departments or in the training of anesthesiologists and nurse anesthetists both nationally and internationally.

In conclusion, with a dataset consisting of thousands of general anesthesia procedures, the INVERT study will thoroughly assess whether an e-learning module can increase anesthetists’ use of neuromuscular monitoring.