ResProt JMIR Res Protoc JMIR Research Protocols 1929-0748 JMIR Publications Toronto, Canada v8i8e13653 31392962 10.2196/13653 Protocol Protocol eRegTime, Efficiency of Health Information Management Using an Electronic Registry for Maternal and Child Health: Protocol for a Time-Motion Study in a Cluster Randomized Trial Eysenbach Gunther Tani Kassimu Bettencourt-Silva Joao Strudwick Gillian Lindberg Marie Hella MPhil 1 http://orcid.org/0000-0002-4777-9627 Venkateswaran Mahima MBBS, MSc 2 3 http://orcid.org/0000-0002-1226-5685 Abu Khader Khadija MPH 4 http://orcid.org/0000-0001-5000-7276 Awwad Tamara MPH 4 http://orcid.org/0000-0002-4750-173X Ghanem Buthaina PharmD, MPH 4 http://orcid.org/0000-0003-1125-6274 Hijaz Taghreed MSc 5 http://orcid.org/0000-0002-9672-4489 Mørkrid Kjersti MPhil, PhD 2 http://orcid.org/0000-0003-4613-1324 Frøen J Frederik MD, PhD 2
Global Health Cluster Division for Health Services Norwegian Institute of Public Health PO Box 222 Skøyen Oslo, 0213 Norway 47 92 49 34 35 frederik.froen@fhi.no
3 http://orcid.org/0000-0001-9390-8509 Faculty of Health Sciences UiT – the Arctic University of Norway Tromsø Norway Global Health Cluster Division for Health Services Norwegian Institute of Public Health Oslo Norway Centre for Intervention Science in Maternal and Child Health University of Bergen Bergen Norway Palestinian National Institute of Public Health World Health Organization Al-Bireh Occupied Palestinian Territory Ministry of Health Ramallah Occupied Palestinian Territory Corresponding Author: J Frederik Frøen frederik.froen@fhi.no 082019 07 08 2019 8 8 e13653 7 2 2019 28 3 2019 14 6 2019 16 6 2019 ©Marie Hella Lindberg, Mahima Venkateswaran, Khadija Abu Khader, Tamara Awwad, Buthaina Ghanem, Taghreed Hijaz, Kjersti Mørkrid, J Frederik Frøen. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 07.08.2019. 2019

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.

 Background

Paper-based routine health information systems often require repetitive data entry. In the West Bank, the primary health care system for maternal and child health was entirely paper-based, with care providers spending considerable amounts of time maintaining multiple files and client registers. As part of the phased national implementation of an electronic health information system, some of the primary health care clinics are now using an electronic registry (eRegistry) for maternal and child health. The eRegistry consists of client-level data entered by care providers at the point-of-care and supports several digital health interventions that are triggered by the documented clinical data, including guideline-based clinical decision support and automated public health reports.

 Objective

The aim of the eRegTime study is to investigate whether the use of the eRegistry leads to changes in time-efficiency in health information management by the care providers, compared with the paper-based systems.

 Methods

This is a substudy in a cluster randomized controlled trial (the eRegQual study) and uses the time-motion observational study design. The primary outcome is the time spent on health information management for antenatal care, informed and defined by workflow mapping in the clinics. We performed sample size estimations to enable the detection of a 25% change in time-efficiency with a 90% power using an intracluster correlation coefficient of 0.1 and an alpha of .05. We observed care providers for full workdays in 24 randomly selected primary health care clinics—12 using the eRegistry and 12 still using paper. Linear mixed effects models will be used to compare the time spent on health information management per client per care provider.

 Results

Although the objective of the eRegQual study is to assess the effectiveness of the eRegistry in improving quality of antenatal care, the results of the eRegTime study will contribute to process evaluation, supplementing the findings of the larger trial.

 Conclusions

Electronic health tools are expected to reduce workload for the care providers and thus improve efficiency of clinical work. To achieve these benefits, the implementation of such systems requires both integration with existing workflows and the creation of new workflows. Studies assessing the time-efficiency of electronic health information systems can inform policy decisions for implementations in resource-limited low- and middle-income settings.

 International Registered Report Identifier (IRRID)

DERR1-10.2196/13653

 health information systems, eHealth Time and Motion Studies workflow antenatal care developing countries Introduction Background

Robust health information systems play a central role in the strengthening of health systems and achieving universal health coverage [1-4]. There are, however, substantial gaps in the reliability, timeliness, and efficiency of health data collection, analysis, and use in many countries, hampering evidence-based decision making at all levels of the health system [5]. Common traits of many health systems include inefficient and uncoordinated data processing and management [6]. Health care providers are often obliged to repeatedly collect, compile, and report redundant health information. Therefore, time spent on direct patient care might be shortened [5,7]. The introduction of health information technologies could substantially influence care providers’ workflow and clinical work processes [8,9]. Existing evidence, primarily from high-income contexts, suggests that access to relevant health information tends to improve with the use of electronic health information systems but is often associated with time-consuming and counterintuitive user-system interactions [10-14]. There is limited evidence from low- and middle-income countries (LMICs) on how the use of electronic health information systems affect clinical workflow or efficiency [15,16]. LMIC can least afford wasting the time of a limited health workforce and may gain the most from improved efficiency of health information management [17,18]. It is therefore important to detect and understand the specific challenges faced in settings with fewer resources to successfully and sustainably implement electronic health information systems in such contexts.

In the West Bank, Palestine, an electronic health information system—the eRegistry for maternal and child health—is currently being rolled out on a national scale throughout primary health care. The eRegistry consists of electronic health (eHealth) records for antenatal, postpartum, and newborn care for use at the point of care by the care providers. The eRegistry supports automated clinical decision support, workflow management support, and referral functionalities. [1,19-21]. Care providers in primary health care clinics can access the eRegistry through desktop computers where they enter all client-related information [22]. The Palestinian eRegistry is installed in the District Health Information System 2 (DHIS2) tracker software; DHIS2 is a Web-based platform that is free and open-source and currently in use in more than 50 low- and middle-income settings largely for collection of aggregate data in a health information system or, to a lesser extent, for individual-level data in the health system.

An ongoing cluster randomized controlled trial (CRCT), the eRegQual study, is embedded in the national implementation of the eRegistry, where clinics using the eRegistry were included in the intervention arm and compared with the control arm that used paper-based records [22]. The primary objective of the eRegQual CRCT is to assess the effectiveness of the eRegistry in improving health outcomes for pregnant women and process outcomes of quality of antenatal care. Further details of the eRegQual study can be found in the published trial protocol [22].

The time-motion method is one of the more robust study designs for the collection and quantification of time data [8] and has been used to study costs and inefficiencies in the delivery of health care as well as patient safety and quality [23]. The time-motion study design in health care involves continuous observations of clinicians’ work in health facilities by recording the time taken to perform a set of predefined tasks. This study design is frequently applied in assessing whether the introduction of an eHealth tool is associated with changes in time-efficiency [24].

 Objectives

The aim of the eRegTime study is to evaluate whether the use of an eRegistry changes the time-efficiency of care providers in primary health care clinics for antenatal care. Time-efficiency will be assessed by measuring the time spent by the care providers on health information management.

 Methods

In this protocol, we have followed the Suggested Time and Motion Procedures checklist for standardized reporting of studies using the time-motion design (see Multimedia Appendix 1) [8] as well as the Standard Protocol Items: Recommendations for Intervention Trials checklist (see Multimedia Appendix 2).

Setting

In the West Bank, Palestine, primary health care clinics provide antenatal, postpartum, and newborn care.

The different cadres of health care providers that work in maternal and child health in primary health care clinics include midwives, nurses, general practitioners trained in maternal and child health care, and specialist obstetricians. Smaller clinics (less than 50 new enrollments of pregnancies a year) typically have a nurse or a midwife working throughout the week, whereas the doctor visits the clinic once every 2 weeks. Larger clinics (more than 50 new enrollments of pregnancies a year) and referral clinics have specialist obstetricians, in addition. The nurse or midwife in the clinics does the majority of the antenatal care consultation and after-consultation work that involves health information management and were the only groups of care providers observed in this study.

The phased national implementation of the eRegistry was undertaken in tandem with the eRegQual study; the intervention clusters of the eRegQual study were the clinics that received the eRegistry as part of phase 1 of implementation, whereas control clusters continued to use paper-based clinical records and were scheduled to receive the eRegistry after the end of the eRegQual study. Although 68 primary health care clinics started using the eRegistry in phase 1 of implementation, and 59 clinics continued to use paper-based clinical records, some of the smaller clinics were clubbed to form clusters before randomization for the eRegQual study. Larger clinics were considered as clusters of their own. In total, there were 60 clusters in each arm of the eRegQual study. Details of enrollment and randomization for the eRegQual study can be found in the published trial protocol [22].

The intervention evaluated in the eRegQual study—the eRegistry—is used as a point-of-care electronic data entry tool in primary health care clinics in the West Bank [22]. Guideline-based clinical decision support and automated electronic monthly reports are 2 digital health interventions currently supported by the Palestinian eRegistry. The eRegistry is intended to fully replace paper-based systems for maternal and child health in primary health care in the West Bank.

 Workflow in Primary Health Care Clinics Workflow in Clinics Using Paper-Based Systems

Pregnant women visit primary health care clinics for their first antenatal (booking) visit on specific workdays (clinics may work 1-4 days a week). The nurse or midwife in the clinics receives the pregnant women for the booking visit and documents a set of demographic data (eg, name, national identification number, address, phone number, and date of birth), and medical, surgical, and obstetric history. Afterward, the nurse or midwife measures and documents the woman’s height and weight, blood pressure, and fundal height and orders and fills out routine laboratory results appropriate for each antenatal visit. As part of the booking visit, the doctor examines women on the same workday or a few workdays later in some clinics. The nurse or midwife assists the doctor in medical and ultrasound examinations. For pregnant women identified with risk factors that warrant a referral, the nurse or midwife makes necessary arrangements for transfer to the referral health facility. There is a flexible appointment system for all subsequent antenatal visits. For uncomplicated pregnancies, the nurse or midwife documents blood pressure and fundal height, checks for fetal presentation, and orders laboratory investigations during the subsequent antenatal visits. Nurses and midwives typically do client care for pregnant and postpartum women as well as newborns in the first part of the workday. Following this, the nurse or midwife usually completes registers for antenatal care, referrals, ultrasounds, vaccines, and laboratory investigations. The nurse or midwife also compiles the data in the registers for public health reporting to the Palestinian Ministry of Health, typically concentrating this task in 1 or 2 workdays monthly. Event counts of number of pregnancies registered in the clinic, number of ultrasound examinations and laboratory tests that are performed, and number of pregnancies with risk conditions that are referred are some examples of the data that are part of standardized monthly reports submitted by care providers [25].

 Workflow in Clinics Using the Electronic Registry

All clinical tasks, as described for the control clusters, are identical in case of the intervention clusters. Only the health information management differs. The eRegistry is used by care providers to document real-time clinical data during client consultation. On the basis of the data entered at the point of care, the eRegistry generates automated decision support and workflow assistance [19,22]. Laboratory systems are not integrated in the eRegistry, and care providers need to enter the laboratory results they receive on paper into the eRegistry retrospectively. The eRegistry aggregates and submits all data that are part of the public health reports automatically every month to the Palestinian Ministry of Health.

 Study Design

The time-motion study design was employed to collect data in the eRegTime study [8,24]. Observations were conducted in a randomly selected subsample of intervention and control clusters (primary health care clinics) of the eRegQual CRCT.

Outcome Measures

The primary outcome measure is the time spent on health information management per consultation. We defined health information management as the preparations and executions of collection, aggregation, analysis, and dissemination of clinical data, both at the individual and aggregate levels [26]. To tailor the general definition of the primary outcome to fit our context, we first used workflow mapping exercises ahead of data collections for the eRegTime study (as described previously) to list all the tasks usually done by the nurse or midwife in the primary health care clinics during antenatal care on a typical workday [27]. We then defined 6 activity types corresponding to the tasks: accessing information, reporting, documentation, client care, client-related care, and miscellaneous. The primary outcome measure—health information management time—was defined as time spent on all tasks involving the activity types “information access,” “information documentation,” and “information reporting” (see Table 1) [27]. “Information access” includes all activities that involve seeking and finding relevant existing health or demographic information on the client [27]. “Information documentation” consists of all tasks that involve writing down client information in the antenatal records (electronic or paper), laboratory, and ultrasound forms [27]. “Information reporting” is defined as transferring information from the antenatal records and registers for public health reporting [27].

A total of 2 additional analysis categories were defined: (1) “client care” that includes all activities in which the care provider is fully focused on the client without any writing and (2) “client-related care” that refers to all tasks that are imperative for care of individual pregnant women undertaken between 2 antenatal care consultations (see Table 1).

Activities unrelated to care of clients, including personal activities of the care providers, and tidying and preparing the consultation room for new clients, were categorized as “miscellaneous” (see Table 1) [27].

Analysis categories including the primary outcome measure, corresponding task, and task category as defined for data collection (adapted from the study by Pizziferri et al [30] and tailored to the local context).

Analysis category Task category in data collection tool Name of task in data collection tool
Client care Outside Assisting doctor
Client care Outside Examination in other room
Client care Procedures Clinical and medical examination
Client care Procedures Injections and bloodtake
Client care Procedures Giving tablets
Client care Procedures Other
Client care Talking Education and counseling
Client care Talking Talking to family
Client care Talking History: demographic and medical
Client care or client-related carea Talking Clinical support
Client care or client-related carea Talking Call client or family
Client care or client-related carea Talking Referrals
Client care or client-related carea Talking Other
Health information management Between or after consultations Writing in statistics book
Health information management Computer-Find Client file
Health information management Computer-Find Lab or ultrasound results
Health information management Computer-Writing Client file (including history)
Health information management Computer-Writing Lab or ultrasound form
Health information management Computer-Writing Schedule appointment
Health information management Computer-Writing Text message in eRegistry
Health information management Paper-Find Client file
Health information management Paper-Find Lab or ultrasound results
Health information management Paper-Writing MCH (Maternal and Child Health) Handbook (including history)
Health information management Paper-Writing Client file (including history)
Health information management Paper-Writing Register book
Health information management Paper-Writing MCH Handbook or register book
Health information management Paper-Writing Register book or client file
Health information management Paper-Writing Client file or MCH handbook
Health information management Paper-Writing Lab, ultrasound, prescriptions, and referrals
Health information management Paper-Writing Schedule appointment
Health information management Paper-Writing Writing on other paper
Health information management Talking Explaining test results
Health information management Talking Technical support
Health information management or client-related careb Computer-Read Appointment list
Health information management or client-related careb Computer-Read Client file
Health information management or client-related careb Computer-Read Lab or ultrasound results
Health information management or client-related careb Computer-Read Guidelines, treatment
Health information management or client-related careb Computer-Read Other info
Health information management or client-related careb Paper-Read Appointment list
Health information management or client-related careb Paper-Read MCH handbook
Health information management or client-related careb Paper-Read Client file
Health information management or client-related careb Paper-Read Lab or ultrasound results
Health information management or client-related careb Paper-Read Treatment guidelines
Health information management or client-related careb Paper-Read Other info
Miscellaneous Between or after consultations Cleaning, arranging files
Miscellaneous Between or after consultations Phone and computer (personal)
Miscellaneous Between or after consultations Other: praying, eating, toilet
Miscellaneous Between or after consultations Eating, praying, toilet
Miscellaneous Between or after consultations Group education
Miscellaneous Postpartum care Postpartum care

aTask classified as client-related care if done outside of a consultation. If done within an antenatal care consultation, it is classified as client care.

bTask classified as client-related care if done outside of a consultation. If done within an antenatal care consultation, it is classified as health information management.

 Eligibility Criteria

Clusters (primary health care clinics) that are part of the eRegQual CRCT that fulfil the following criteria were eligible for inclusion in the eRegTime study: (1) Have 1 nurse or 1 midwife providing antenatal care services on a given workday (to maintain a 1:1 subject-to-observer ratio) and (2) Have, on average, at least 1 booking visit per workday (to ensure capturing a sufficient number of antenatal booking visits). After applying these inclusion criteria to the 120 clusters that are part of the eRegQual CRCT, 41 clusters were eligible for the time-motion study observations (20 intervention clusters and 21 control clusters; Figure 1).

Selection of clusters in primary health care clinics for observations.

 Sampling

For sample size estimations, we assumed that clinics using paper-based systems spend an average of 10 min on health information management per client. We also assumed unequal and higher SD around the mean health information management time (in minutes) for clinics that use the eRegistry (SD=5) compared with clinics that use paper-based systems (SD=2) because of an expected variance in computer literacy and confidence of use. Sample size calculations were made using the Stata command “clustersampsi” to detect a 25% difference at a 90% power and 5% significance using an a priori intracluster correlation coefficient of .1 [28,29]. A total of 24 primary health care clinics were selected to be observed, 12 from each arm of the CRCT, with at least 8 observed antenatal consultations per clinic (Figure 1). Statisticians that are independent of the eRegTime study team performed a random sampling of the primary health care clinics for the observations, stratified on laboratory availability.

 Data Collection Methods

We designed the data collection tool based on a Microsoft Access database template made available online by the US Agency for Healthcare Research and Quality [24], customized to the clinical workflow in the West Bank. The data collection tool was installed on handheld tablets. The data collection tool contains a list of tasks categorized under 10 task categories, and every task can be time-stamped (Figure 2) [30,31]. The task categories covered the care providers’ entire workday consisting of every clinical and nonclinical task, including after-consultation and between-consultation work.

The observers were trained to first determine the nature of the observed task and then click on the corresponding task on the data collection tool (Figure 2) [32]. The observer could end a task by clicking on the “confirm entry” button (Figure 2). In case of multitasking by the care providers, the observers were instructed to select the principal activity. After-consultation work were recorded as separate observations, as were postpartum care consultations.

The database stored the observation times for each task with an activity code linked to the tasks. In accordance with ethical approvals for the study, no personal or other demographic data related to the client or the care provider were collected, and clinic names were only being stored as computer-generated codes in the database.

A total of 4 trained observers completed the data collection. Observers were trained with simulation videos on the time-motion methodology and the task categories and in using the data collection tool (see Multimedia Appendix 3). Following training, the observers conducted practice observations in nonstudy clinics with and without the eRegistry. After this, observations and data collections were undertaken in the study clinics. The observers recorded a full workday and included all the antenatal consultations during that day. If the required number of antenatal consultations per clinic (n=8) was not achieved in 1 day, additional days of observation were carried out until the required cluster size was reached.

The field coordinators of the study received the data after each day of data collection and checked that the sample size for each clinic is reached with a sufficient number of documented observations.

 Blinding

Although neither the observers nor the care providers in the primary health care clinics can be blinded to the intervention, they both will be blinded to the outcomes of the eRegTime study and have only been informed of the overarching objective of the eRegQual CRCT (including the eRegTime study) of assessing effects of the eRegistry on the quality of care. To ensure blinding of the observers to the outcome, the data collection tool included an exhaustive list of tasks, beyond the primary outcome of the eRegTime study (see Table 1; Figure 2).

Data collection tool (data entry form) used for recording time-motion data.

 Data Analyses

The unit of measurement of the primary outcome is the time spent on health information management per client per care provider, where time will be analyzed in minutes. Statistical analyses will be performed using Stata version 15 or later (StataCorp LLC, 2017,  Stata Statistical Software: Release 15) or RStudio version 1.2.1335 or later. Descriptive statistics for the time variables will be summarized as means and SDs. We will report on the average time spent on antenatal consultation overall, and for booking visits and other antenatal visits separately, and the average time spent on each of the activity types including those that are not part of the primary outcome (including “client care,” “client-related care,” and “miscellaneous”; see Table 1).

We assume that the nurse or midwife spends, on average, an equal amount of time on after-consultation documentation work per client. The after-consultation time spent on client-related documentation and public health reporting will be averaged over the number of observed antenatal consultations and added to the time spent per consultation. Differences in the health information management time between the clinics with and without the eRegistry will be tested for significance using the linear mixed effects model to account for clustering [33,34]. In addition, as secondary analyses, we will test for differences in the health information management time separately for booking visits and other antenatal visits, and differences in time spent on other activity types in the 2 arms. Postpartum care consultations will be excluded from the analysis, as the focus of both eRegQual and eRegTime studies is on antenatal care quality and clinical processes.

Accompanying the results of the outcomes of the study, interobserver reliability assessments will be reported using kappa coefficients for the total number of clinical tasks and activity types recorded and intraclass correlation coefficients for the recorded mean times for the tasks and analysis categories (see Table 1) [35].

 Ethics Approval and Consent to Participate

The eRegTime study was approved by the Palestinian Health Research Council (PHRC/HC/208/17) and the Regional Committee for Medical and Health Research Ethics in Norway (2017/400). Permissions to conduct observations in the clinics have been obtained from the Palestinian Ministry of Health. Care providers and supervisors of the primary health care clinics will be informed of the data collection for this study. Considering the local sensitivity and hesitance related to signing documents in our study context, pregnant women will be asked for oral consent to allow the observers to be present in the rooms during consultations, and the ethics committees were notified of this. No data will be recorded on personal or individual characteristics of pregnant women, care providers, or primary health care clinics. Only completely anonymous data will be available to the researchers for analysis.

 Results

Ethical approvals for conduct of the study were obtained in April 2017. The data collection tool was designed, tested, and adjusted over 2017 and 2018, followed by which the sample was selected for the main study. Clinics included in the sample were informed about the study before start of observations, and the data collection for the eRegTime study was completed between August and December 2018. Data will be analyzed for outcomes in July and August 2019; the results are expected to be published in the second half of 2019.

 Discussion

The eRegTime study is one of the few studies that assesses the impact of an eHealth intervention on clinical workflow and time-efficiency in a middle-income context, where the impact of using digital tools routinely during clinical care is probably much bigger given the manpower and resource constraints than high-income settings.

Most studies that have assessed the time-efficiency using the time-motion design find no statistical differences in the workload of care providers following the introduction of eHealth tools [15,30,31,36]. Factors that may potentially affect the time-efficiency of care providers while using eHealth tools are duration of use of eHealth tools, computer literacy, multitasking, and interruptions [37-41]. In some settings, a period of 18 to 24 months between the implementation of the eHealth tools and the observations was considered sufficient for the stabilization of clinical work routines [15,42]. Primary health care clinics observed in the eRegTime study will be using the eRegistry for a median time of 20 months at the time of the observations. According to a questionnaire survey conducted in the intervention clusters of the eRegQual CRCT, a quarter of the nurses and the midwives had never used a computer before starting to use the eRegistry. Formative research and workflow mapping exercises showed that health care service delivery in the setting of this study was characterized by fragmented workflow and the time taken to perform the different tasks was relatively short [27]. The time-motion design is particularly suitable for data collection in such settings [30,31]. Although interruptions to the workflow and multitasking might be overlooked because of the fact that the data collection tool requires the observer to select only 1 activity at the time, the primary objective of the eRegQual study is to assess quantitative differences in time spent between clinics with and without the eRegistry; data collection methods were designed to be identical in the 2 arms.

Other methods such as work sampling and self-reported questionnaires were considered during the planning phase of this study. However, with work sampling, in which activities are recorded only at certain time intervals, there is a risk of missing certain activities. In addition, this method requires an enormously large sample size, which was not feasible in our setting [43]. The use of self-reported questionnaires poses risks of inaccurate reporting and recall bias as well as being a considerable interference to the care provider’s workflow [44]. The time-motion design was therefore considered the most suitable for this study.

We acknowledge that it may not be possible to completely eliminate the risk of care providers behaving differently because they know they are being observed [45]. We will attempt to minimize this effect by training the observers to avoid interfering with clinical work [45]. Another potential source of bias is diverging subjective interpretations of the tasks during data collection by the observers, and this will be minimized by hands-on training sessions, practice sessions with simulation videos, and “test” observations before the start of the study observations.

 Multimedia Appendix 1

Suggested Time and Motion Procedures (STAMP) checklist.

 Multimedia Appendix 2

SPIRIT protocol checklist.

 Multimedia Appendix 3

Training manual.

 Abbreviations CRCT

cluster randomized controlled trial

 DHIS2

District Health Information System 2

 eHealth

electronic health

 eRegistry

electronic registry

 LMIC

low- and middle-income country

The first author MHL, with supervision from Knut Reidar Wangen, conducted formative research for this study as part of a master thesis at the University of Oslo, sections of which also informed this protocol. The eRegTime study is conducted in cooperation with the Ministry of Health, Palestine and the Palestinian National Institute of Public Health, who will facilitate the training of the observers and the implementation of the study.

The eRegQual CRCT and the eRegTime study are funded by the European Research Council (grant agreement number: 617639; project title: A New Paradigm for Public Health Surveillance: Unlocking the Potential of Data to Empower Woman and Health Systems; project acronym: HEALTMPOWR) and the Research Council of Norway (grant agreement number: 234376; project title: Harmonized Reproductive Health Registry Communication Strategies: Using Health Data to Empower Women and Health Systems). This study is also supported by the Centre for Intervention Science in Maternal and Child Health (CISMAC), a Centre of Excellence funded by the Research Council of Norway and the University of Bergen (grant agreement number: 223269). The funding agencies have no role in the study design, data collection, and analyses of data or publication of results.

MHL designed the data collection tool, conducted formative research including mapping clinical workflows, and defining task categories. MHL and MV contributed to the study design, sample size estimations, and formulation of outcomes and the writing of this study. KM contributed to the study design and formulation of the outcomes and the formative research. BG, TA, KAK, and TH participated in the design of the data collection tool. TA, KAK, and TH conducted pilot observations in the clinics. JFF is the principal investigator of the eRegQual CRCT and contributed to the study design, objectives, outcomes, and the writing of this study. All authors have read and approved of this version of the manuscript.

None declared.

 Flenady V Wojcieszek AM Fjeldheim I Friberg IK Nankabirwa V Jani JV Myhre S Middleton P Crowther C Ellwood D Tudehope D Pattinson R Ho J Matthews J Ortega AB Venkateswaran M Chou D Say L Mehl G Frøen JF eRegistries: indicators for the WHO Essential Interventions for reproductive, maternal, newborn and child health BMC Pregnancy Childbirth 2016 12 30 16 1 293 10.1186/s12884-016-1049-y 27716088 10.1186/s12884-016-1049-y PMC5045645 World Health Organization 2015 Health in 2015: from MDGs to SDGs https://www.who.int/gho/publications/mdgs-sdgs/en/ United Nations 2015 The Millennium Development Goals Report https://www.un.org/millenniumgoals/2015_MDG_Report/pdf/MDG%202015%20rev%20(July%201).pdf Boerma T Requejo J Victora CG Amouzou A George A Agyepong I Barroso C Barros AJ Bhutta ZA Black RE Borghi J Buse K Aguirre LC Chopra M Chou D Chu Y Claeson M Daelmans B Davis A DeJong J Diaz T El Arifeen S Ewerling F Fox M Gillespie S Grove J Guenther T Haakenstad A Hosseinpoor AR Hounton S Huicho L Jacobs T Jiwani S Keita Y Khosla R Kruk ME Kuo T Kyobutungi C Langer A Lawn JE Leslie H Liang M Maliqi B Manu A Masanja H Marchant T Menon P Moran AC Mujica OJ Nambiar D Ohiri K Park LA Patton GC Peterson S Piwoz E Rasanathan K Raj A Ronsmans C Saad-Haddad G Sabin ML Sanders D Sawyer SM da Silva IC Singh NS Somers K Spiegel P Tappis H Temmerman M Vaz LM Ved RR Vidaletti LP Waiswa P Wehrmeister FC Weiss W You D Zaidi S Countdown to 2030: tracking progress towards universal coverage for reproductive, maternal, newborn, and child health Lancet 2018 04 391 10129 1538 48 10.1016/S0140-6736(18)30104-1 Azubuike MC Ehiri JE Health information systems in developing countries: benefits, problems, and prospects J R Soc Promot Health 1999 09 119 3 180 4 10.1177/146642409911900309 10518358 Krickeberg K Principles of health information systems in developing countries Health Inf Manag 2007 36 3 8 20 10.1177/183335830703600303 18195412 Lippeveld T Sauerborn R Bodart C Routine data collection methods Design and Implementation of Health Information Systems 2000 Geneva, Switzerland World Health Organization Zheng K Guo MH Hanauer DA Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research J Am Med Inform Assoc 2011 18 5 704 10 10.1136/amiajnl-2011-000083 21527407 amiajnl-2011-000083 PMC3168304 Unertl KM Novak LL Johnson KB Lorenzi NM Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review J Am Med Inform Assoc 2010 17 3 265 73 10.1136/jamia.2010.004333 20442143 17/3/265 PMC2995718 Niazkhani Z Pirnejad H Berg M Aarts J The impact of computerized provider order entry systems on inpatient clinical workflow: a literature review J Am Med Inform Assoc 2009 16 4 539 49 10.1197/jamia.M2419 19390113 M2419 PMC2705258 Moody LE Slocumb E Berg B Jackson D Electronic health records documentation in nursing: nurses' perceptions, attitudes, and preferences Comput Inform Nurs 2004 22 6 337 44 10.1097/00024665-200411000-00009 15602303 Rahimi B Vimarlund V Timpka T Health information system implementation: a qualitative meta-analysis J Med Syst 2009 10 33 5 359 68 10.1007/s10916-008-9198-9 19827262 van der Meijden MJ Tange HJ Troost J Hasman A Determinants of success of inpatient clinical information systems: a literature review J Am Med Inform Assoc 2003 10 3 235 43 10.1197/jamia.M1094 12626373 M1094 PMC342046 Harrison MI Koppel R Bar-Lev S Unintended consequences of information technologies in health care--an interactive sociotechnical analysis J Am Med Inform Assoc 2007 14 5 542 9 10.1197/jamia.M2384 17600093 M2384 PMC1975796 Mensah N Sukums F Awine T Meid A Williams J Akweongo P Kaltschmidt J Haefeli WE Blank A Impact of an electronic clinical decision support system on workflow in antenatal care: the QUALMAT eCDSS in rural health care facilities in Ghana and Tanzania Glob Health Action 2015 8 25756 10.3402/gha.v8.25756 25630707 25756 PMC4309829 Adepoju IO Albersen BJ de Brouwere V van Roosmalen J Zweekhorst M mHealth for clinical decision-making in sub-Saharan Africa: a scoping review JMIR Mhealth Uhealth 2017 03 23 5 3 e38 10.2196/mhealth.7185 28336504 v5i3e38 PMC5383806 Ngana FR Myers B Belton S Health reporting system in two subdistricts in eastern Indonesia: highlighting the role of village midwives Midwifery 2012 12 28 6 809 15 10.1016/j.midw.2011.09.005 22015220 S0266-6138(11)00138-0 Thompson A Castle E Lubeck P Makarfi PS Experience implementing OpenMRS to support maternal and reproductive health in northern Nigeria Stud Health Technol Inform 2010 160 Pt 1 332 6 20841703 Frøen JF Myhre SL Frost MJ Chou D Mehl G Say L Cheng S Fjeldheim I Friberg IK French S Jani JV Kaye J Lewis J Lunde A Mørkrid K Nankabirwa V Nyanchoka L Stone H Venkateswaran M Wojcieszek AM Temmerman M Flenady VJ eRegistries: electronic registries for maternal and child health BMC Pregnancy Childbirth 2016 01 19 16 11 10.1186/s12884-016-0801-7 26791790 10.1186/s12884-016-0801-7 PMC4721069 Myhre SL Kaye J Bygrave LA Aanestad M Ghanem B Mechael P Frøen JF eRegistries: governance for electronic maternal and child health registries BMC Pregnancy Childbirth 2016 09 23 16 1 279 10.1186/s12884-016-1063-0 27663979 10.1186/s12884-016-1063-0 PMC5035445 eRegistries – Interactive Information for Health http://eregistries.org/ Venkateswaran M Mørkrid K Ghanem B Abbas E Abuward I Baniode M Norheim OF Frøen JF eRegQual-an electronic health registry with interactive checklists and clinical decision support for improving quality of antenatal care: study protocol for a cluster randomized trial Trials 2018 01 22 19 1 54 10.1186/s13063-017-2386-5 29357912 10.1186/s13063-017-2386-5 PMC5778657 Lopetegui M Yen PY Lai A Jeffries J Embi P Payne P Time motion studies in healthcare: what are we talking about? J Biomed Inform 2014 06 49 292 9 10.1016/j.jbi.2014.02.017 24607863 S1532-0464(14)00056-2 PMC4058370 Agency for Healthcare Research and Quality: Health IT Time and Motion Studies Database https://healthit.ahrq.gov/health-it-tools-and-resources/time-and-motion-studies-database Venkateswaran M Mørkrid K Khader KA Awwad T Friberg IK Ghanem B Hijaz T Frøen JF Comparing individual-level clinical data from antenatal records with routine health information systems indicators for antenatal care in the West Bank: a cross-sectional study PLoS One 2018 13 11 e0207813 10.1371/journal.pone.0207813 30481201 PONE-D-18-16684 PMC6258527 Zeng X Reynolds R Sharp M Redefining the roles of health information management professionals in health information technology Perspect Health Inf Manag 2009 09 16 6 1f 20052321 PMC2781729 Lindberg MH UiO - DUO 2017 Efficiency of an Electronic Health Information System for Antenatal Care: A Pilot Time-Motion Study https://www.duo.uio.no/handle/10852/58673 Hemming K Marsh J A menu-driven facility for sample-size calculations in cluster randomized controlled trials Stata J 2018 11 19 13 1 114 35 10.1177/1536867X1301300109 Campbell MK Fayers PM Grimshaw JM Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research Clin Trials 2005 2 2 99 107 10.1191/1740774505cn071oa 16279131 Pizziferri L Kittler AF Volk LA Honour MM Gupta S Wang S Wang T Lippincott M Li Q Bates DW Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study J Biomed Inform 2005 06 38 3 176 88 10.1016/j.jbi.2004.11.009 15896691 S1532-0464(04)00161-3 Overhage JM Perkins S Tierney WM McDonald CJ Controlled trial of direct physician order entry: effects on physicians' time utilization in ambulatory primary care internal medicine practices J Am Med Inform Assoc 2001 8 4 361 71 10.1136/jamia.2001.0080361 11418543 PMC130081 Tang Z Weavind L Mazabob J Thomas EJ Chu-Weininger MY Johnson TR Workflow in intensive care unit remote monitoring: a time-and-motion study Crit Care Med 2007 09 35 9 2057 63 10.1097/01.ccm.0000281516.84767.96 17855819 Zyzanski SJ Flocke SA Dickinson LM On the nature and analysis of clustered data Ann Fam Med 2004 2 3 199 200 10.1370/afm.197 15209193 PMC1466665 Moen EL Fricano-Kugler CJ Luikart BW O'Malley AJ Analyzing clustered data: why and how to account for multiple observations nested within a study participant? PLoS One 2016 11 1 e0146721 10.1371/journal.pone.0146721 26766425 PONE-D-15-28428 PMC4713068 Lopetegui MA Bai S Yen PY Lai A Embi P Payne PR Inter-observer reliability assessments in time motion studies: the foundation for meaningful clinical workflow analysis AMIA Annu Symp Proc 2013 2013 889 96 24551381 PMC3900222 Lo HG Newmark LP Yoon C Volk LA Carlson VL Kittler AF Lippincott M Wang T Bates DW Electronic health records in specialty care: a time-motion study J Am Med Inform Assoc 2007 14 5 609 15 10.1197/jamia.M2318 17600102 M2318 PMC1975804 Alwan K Awoke T Tilahun B Knowledge and utilization of computers among health professionals in a developing country: a cross-sectional study JMIR Hum Factors 2015 03 26 2 1 e4 10.2196/humanfactors.4184 27025996 v2i1e4 PMC4797659 Ruxwana NL Herselman ME Conradie DP ICT applications as e-health solutions in rural healthcare in the Eastern Cape province of South Africa Health Inf Manag 2010 39 1 17 29 10.1177/183335831003900104 20335646 Palestinian Central Bureau of Statistics 2011 Palestine - Household Survey on Information and Communications Technology http://www.pcbs.gov.ps/PCBS-Metadata-en-v4.3/index.php/catalog/81 OECD Data 2017 Access to Computers From Home https://data.oecd.org/ict/access-to-computers-from-home.htm Walter SR Dunsmuir WT Westbrook JI Studying interruptions and multitasking in situ: the untapped potential of quantitative observational studies Int J Hum Comput Stud 2015 07 79 118 25 10.1016/j.ijhcs.2015.01.008 Poissant L Pereira J Tamblyn R Kawasumi Y The impact of electronic health records on time efficiency of physicians and nurses: a systematic review J Am Med Inform Assoc 2005 12 5 505 16 10.1197/jamia.M1700 15905487 M1700 PMC1205599 Finkler SA Knickman JR Hendrickson G Lipkin Jr M Thompson WG A comparison of work-sampling and time-and-motion techniques for studies in health services research Health Serv Res 1993 12 28 5 577 97 8270422 PMC1069965 Burke TA McKee JR Wilson HC Donahue RM Batenhorst AS Pathak DS A comparison of time-and-motion and self-reporting methods of work measurement J Nurs Adm 2000 03 30 3 118 25 10.1097/00005110-200003000-00003 10725940 Franke RH Kaul JD The Hawthorne experiments: first statistical interpretation Am Sociol Rev 1978 10 43 5 623 43 10.2307/2094540