Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 02.04.20 in Vol 9, No 4 (2020): April

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/16531, first published Oct 16, 2019.

This paper is in the following e-collection/theme issue:

    Protocol

    Determining the Agreement Between an Automated Respiratory Rate Counter and a Reference Standard for Detecting Symptoms of Pneumonia in Children: Protocol for a Cross-Sectional Study in Ethiopia

    1Malaria Consortium, London, United Kingdom

    2Department of Public Health Sciences, Karolinska Institutet, Solnavägen, Sweden

    3Malaria Consortium, Addis Ababa, Ethiopia

    4United Nations Children’s Fund Supply Division, Copenhagen, Denmark

    5United Nations Children’s Fund Programme Division, New York, NY, United States

    6St Paul’s Hospital Millennium Medical College, Addis Ababa, Ethiopia

    7School of Public Health and Community Medicine, Institute of Medicine, University of Gothenburg, Gothenburg, Sweden

    Corresponding Author:

    Charlotte Ward, MA, MSc

    Malaria Consortium

    The Green House 244-254 Cambridge Heath Rd

    London

    United Kingdom

    Phone: 44 0 20 3559 ext 6438

    Email: c.ward@malariaconsortium.org


    ABSTRACT

    Background: Acute respiratory infections (ARIs), primarily pneumonia, are the leading infectious cause of under-5 mortality worldwide. Manually counting respiratory rate (RR) for 60 seconds using an ARI timer is commonly practiced by community health workers to detect fast breathing, an important sign of pneumonia. However, correctly counting breaths manually and classifying the RR is challenging, often leading to inappropriate treatment. A potential solution is to introduce RR counters, which count and classify RR automatically.

    Objective: This study aims to determine how the RR count of an Automated Respiratory Infection Diagnostic Aid (ARIDA) agrees with the count of an expert panel of pediatricians counting RR by reviewing a video of the child’s chest for 60 seconds (reference standard), for children aged younger than 5 years with cough and/or difficult breathing.

    Methods: A cross-sectional study aiming to enroll 290 children aged 0 to 59 months presenting to pediatric in- and outpatient departments at a teaching hospital in Addis Ababa, Ethiopia, was conducted. Enrollment occurred between April and May 2017. Once enrolled, children participated in at least one of three types of RR evaluations: (1) agreement—measure the RR count of an ARIDA in comparison with the reference standard, (2) consistency—measure the agreement between two ARIDA devices strapped to one child, and (3) RR fluctuation—measure RR count variability over time after ARIDA attachment as measured by a manual count. The agreement and consistency of expert clinicians (ECs) counting RR for the same child with the Mark 2 ARI timer for 60 seconds was also measured in comparison with the reference standard.

    Results: Primary outcomes were (1) mean difference between the ARIDA and reference standard RR count (agreement) and (2) mean difference between RR counts obtained by two ARIDA devices started simultaneously (consistency).

    Conclusions: Study strengths included the design allowing for comparison between both ARIDA and the EC with the reference standard RR count. A limitation is that exactly the same set of breaths were not compared between ARIDA and the reference standard since ARIDA can take longer than 60 seconds to count RR. Also, manual RR counting, even when aided by a video of the child’s chest movements, is subject to human error and can result in low interrater reliability. Further work is needed to reach global consensus on the most appropriate reference standard and an acceptable level of agreement to provide ministries of health with evidence to make an informed decision on whether to scale up new automated RR counters.

    Trial Registration: ClinicalTrials.gov NCT03067558; https://clinicaltrials.gov/ct2/show/NCT03067558

    International Registered Report Identifier (IRRID): RR1-10.2196/16531

    JMIR Res Protoc 2020;9(4):e16531

    doi:10.2196/16531

    KEYWORDS

    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100



    Introduction

    Acute respiratory infections (ARIs), primarily pneumonia, are the leading infectious causes of death among children aged younger than 5 years globally, accounting for an estimated 0.9 million deaths in 2015 [1], with over 75% of these deaths clustering in sub-Saharan Africa and Southeast Asia. Deaths from pneumonia in children result mostly from delayed presentation to appropriate health care providers and inappropriate treatment [2].

    Diagnosis of pneumonia by community health workers (CHWs) and first-level health facility workers (FLHFWs), collectively known as frontline health workers, is based on counting the number of breaths in 60 seconds in children aged younger than 5 years with cough and/or difficulty breathing to assess whether the respiratory rate (RR) is high enough for a particular age to prescribe antibiotics and treat suspected pneumonia. This is defined by the World Health Organization (WHO) Integrated Management of Childhood Illness (IMCI) guidelines [3] for FLHFWs and the Integrated Community Case Management (iCCM) guidelines for CHWs [4]. Current standard practice for frontline health workers is to count RR manually by observing chest movements for 60 seconds. In practice defining a breath and counting RR can be difficult, as children breathe irregularly and faster than adults and the child may not be calm and still for a full minute. Misclassification of the observed rate remains high [5,6], often leading to inappropriate treatment [7].

    The Acute Respiratory Infection Diagnostic Aids (ARIDA) project [8] was initiated as a response to the call for better pneumonia diagnostic aids [9,10]. A target product profile (TPP) was shared with industry, academia, and partners to encourage and guide development of new automated RR counting devices [11]. The ARIDA technical specification listed in the United Nations Children’s Fund (UNICEF) request for proposals outlines that any ARIDA must automatically detect and display the RR to aid in the classification of suspected pneumonia in children from the age of 0 to 59 months and include a visual indicator for notification of above or below the age-specific fast breathing thresholds as defined by the WHO IMCI guidelines [3].

    In response to the TPP, Philips developed the Children’s Respiration Monitor (ChARM) device, which uses an accelerometer-based system to measure the RR in children 0 to 59 months and automatically classifies the breathing rate as fast or normal, based on the age of the child. ChARM is intended to be used by CHWs in low-resource settings. It is strapped around the belly of the child using an elastic belt (Figure 1). ChARM is the first product to be tested as part of the ARIDA field trials, implemented by the Malaria Consortium in Ethiopia and Nepal and sponsored by UNICEF in partnership and with funding from “la Caixa” Banking Foundation.

    Figure 1. Illustration of the Philips ChARM device positioned on a child.
    View this figure

    In Ethiopia, pneumonia is the number one cause of under-5 mortality, responsible for 16% deaths in children aged younger than 5 years in 2016 [12]. Ethiopia has scaled up iCCM of childhood illness in all regions following a national policy change supporting community-based treatment of childhood pneumonia by CHWs (locally known as health extension workers [HEWs]) in early 2010 [13]. As part of Ethiopia’s HEW program, over 42,000 HEWs have been trained for 1 year in iCCM and equipped to assess, classify, and manage pneumonia, malaria, diarrhea, and severe acute malnutrition and provide preventive and curative health services [13]. This paper presents the study design for the evaluation of agreement between an ARIDA and a reference standard RR count for children in Ethiopia.


    Methods

    Study Aims and Objectives

    This study aims to understand whether an ARIDA RR count agrees with an expert panel of pediatricians counting RR by reviewing a video of the child’s chest for 60 seconds (reference standard) for children aged younger than 5 years with cough and/or difficult breathing.

    The primary objective of this study is to determine the performance of an ARIDA, as defined by agreement and consistency, in children aged younger than 5 years with cough and/or difficulty breathing. The secondary objective is to determine the performance of expert clinicians (ECs) counting RR, as defined by agreement and consistency, in children aged younger than 5 years with cough and/or difficulty breathing. The third objective is to measure RR fluctuation over time after ARIDA device attachment in normal breathing children aged 2 to 59 months.

    Study Design

    The study is a cross-sectional study comprising three types of RR evaluations: agreement, consistency, and RR fluctuation over time.

    Study Site

    The study was conducted in the pediatric in- and outpatient departments at Saint Paul’s Hospital and Millennium Medical College in Addis Ababa, Ethiopia. This hospital was selected based on the high incidence of pneumonia in outpatient and inpatient departments, interest and willingness of hospital managers to host the study, availability of Integrated Management of Neonatal and Child Illness (IMNCI)-trained [14] expert clinicians (ECs), and availability of a suitable study room, reliable electricity supply, and access to treatment including amoxicillin and oxygen.

    Ethics Approval and Consent to Participate

    Ethical approval was obtained from the Armauer Hansen Research Institute/ALERT Ethics Review Committee (a biomedical research institute in Ethiopia) on March 7, 2017 (ref. PO02/17) and favorable ethical opinion received by the Liverpool School of Tropical Medicine Ethics Committee. All participants consented to the study by reading and signing the information and consent form.

    Study Participants

    All children attending in- and outpatient departments at Saint Paul’s Hospital and Millennium Medical College in Addis Ababa between April 5 and May 22, 2017, were potential participants in the study and were systematically screened for eligibility. Children aged 0 to <2 months were excluded from the consistency evaluation due to the anticipated difficulty in attaching two devices at once to a small child. Children aged 0 to <2 months and those with fast breathing were excluded from the fluctuation evaluation due to anticipated difficulty in measuring RR in this group for an extended period of time and also to isolate the effect of the ChARM attachment on RR from other causes of RR fluctuation. All other children aged younger than 5 years who were accompanied by a caregiver aged 18 years and older, not too agitated to be assessed by a research nurse, who did not present with general danger signs or IMNCI referral signs or device manufacturer safety exclusion criteria (wearing supportive device at area of chest/belly, skin not intact in chest/belly, born before 37 weeks of gestation [<2 months only]), were not an inpatient being managed by barrier nursing (such as severe burns, child with neutropenia, severe infectious diseases), and were not advised against research procedures by the supervising clinician were eligible to participate in the study.

    General danger signs for newborns (<28 days) include active convulsions or fits, not feeding well, fever (37.5℃ [99.5℉] or above), low body temperature (35.5℃ [95.9℉] or below), movement only when stimulated, or no movement even when stimulated and for all other age groups include lethargy or unconsciousness, not able to drink/breastfeed, vomiting everything, and active convulsions or fits. IMNCI referral signs for all children include stridor in a calm child, chest indrawing, severe dehydration, severe persistent diarrhea, very severe febrile disease, severe complicated measles, mastoiditis, complicated severe malnutrition, and severe anemia. Written informed consent was obtained from the caregiver before enrollment. Two ECs with extensive experience in assessing and treating children with suspected pneumonia using IMNCI guidelines were selected. They were required to have BSc nursing qualification and an IMNCI certificate.

    Outcome Measures

    The primary outcome for the agreement evaluation was the mean difference in RR between ARIDA and the reference standard, which summarized the lack of agreement by calculating the average deviation between measures. Similarly for the consistency evaluation, the mean difference in RR between two ARIDA devices was calculate. Table 1 shows all the outcome measures for the study by objective.

    • Primary objective: determine the performance of the ARIDA device as defined by agreement and consistency in children aged younger than 5 years with cough or difficulty breathing in a controlled setting
    • Secondary objective: determine the performance of ECs counting RR as defined by agreement and consistency in children aged younger than 5 years with cough or difficulty breathing in a controlled setting
    • Third objective: measure RR fluctuation over time after ARIDA device attachment in normal breathing children aged 2 to 59 months in a controlled setting
    Table 1. Primary objective and outcome measures.
    View this table

    Data Collection and Management

    Data were collected using an electronic data collection platform (CommCare, Dimagi) installed onto password-protected 7C Pro tablets (Tecno Mobile) and backed up to a protected cloud server. Four-digit unique identification codes were used to anonymize patient data. All videos were transferred using password-protected external hard drives, and paper consent forms were stored in opaque carriers in locked cabinets. All RR evaluation data were entered by two independent research assistants. The data manager downloaded data daily and entered it into a data checker with in-built validation checks. Source videos showing the ARIDA device with the RR count displayed were used to verify ARIDA counts should the two research assistants disagree. Other inconsistencies were rectified by tracing back to paper data entry forms or querying the counts directly with the research team.

    Training and Pretest

    The video expert panel (VEP) members and ECs were trained for two days on the WHO IMCI method to count RR including practice for half a day using videos of known RR counts [3]. They were individually evaluated using different RR videos to ensure that they were able to count RR ±2 breaths per minute (bpm) from the known RR in 4 out of 5 training videos [15]. All VEP members and ECs passed the competency assessment before starting video review.

    Following training, an 8-day pretesting of procedural activities including patient screening, patient flow, and data collection was conducted to ensure the research team was conversant with the data collection procedures, devices, and videography equipment to be used in the study. There was also a pretest of the video panel reference standard evaluations and refresher training on general danger signs, breath-counting, signs of stridor, and chest indrawing with the IMNCI training video.

    Evaluations

    Patients were screened by a research nurse in the in- and outpatient departments of the hospital using a screening checklist to ascertain the child’s eligibility. An RR classification was made during prescreening by the research nurse using the Mark 2 Acute Respiratory Infection (MK2 ARI) timer to determine whether the child had fast or normal breathing. The prescreening assessment was conducted in a separate part of the hospital and not communicated to the ECs to blind them to the RR classification. Children were enrolled prospectively based on eligibility determined by the screening procedure. The research team then decided, based on the state of the child, age, and breathing status, how many elements of the study to conduct on each child—agreement, consistency, and/or RR fluctuation. Table 2 shows the number of participants aimed to be enrolled to each element of the study by age group and breathing status.

    Table 2. Enrollment targets for each type of evaluation by age group and breathing status.
    View this table

    For the agreement evaluation, the research assistant attached an ARIDA to the child and ensured the child was positioned correctly according to device instructions: with his/her back fully supported, either on the lap of the caregiver or lying down on a bed, and the device in line with the child’s belly button and one of the nipples [16]. Once the child was calmed, usually by the research assistant clicking their fingers, the videographer started recording and the ARIDA and EC RR count started simultaneously. The EC was blinded to the ARIDA RR count by placing ARIDA on the far side of the child’s belly and using a paper cover to shield the screen. The time taken to get an ARIDA count (from when the OK button was pressed to when the device beeped to signal completion of the RR count) was also obtained by a research assistant using a stopwatch. After 60 seconds, if the EC had not obtained an RR count, the EC attempt was recorded as unsuccessful and repeated for both the EC and ARIDA. After 5 minutes or if the ARIDA displayed an error message, the ARIDA attempt was recorded as unsuccessful (with a reason) and repeated for both EC and ARIDA. If the third attempt was still unsuccessful for either device or EC, the evaluation was recorded as a failure. Fifteen different ARIDA devices were used and rotated systematically for all evaluations.

    The consistency evaluation followed the same procedure as the agreement evaluation with two ARIDA devices attached to a child using a single belt, positioned in line with each nipple and measured RR from the same starting point. Time taken to obtain each ARIDA RR count was recorded by two research assistants using stopwatches. To measure the consistency between ECs, two ECs conducted separate manual RR counts with MK2 ARI timers over an identical 60-second period. For an EC or ARIDA attempt to be successful, both ECs or both ARIDA devices had to get an RR count. For the RR fluctuation evaluation, an EC counted RR with the MK2 ARI timer for 60 seconds. Following this, the ARIDA was attached to the child and the EC did three more RR counts for 60 seconds in the following time periods: 0 to 1 minute, 2 to 3 minutes, 4 to 5 minutes. On completion of the evaluation, the research team debriefed the caregiver and ensured medical management for the child was completed by the relevant hospital staff.

    Reference Standard

    The reference standard for the agreement evaluation was a video review by two to four independent VEP members. They were all practicing pediatricians with over 5 years’ experience managing pneumonia in children aged younger than 5 years and who had received refresher training in counting RR as per WHO IMCI guidelines (3).

    First, two VEP members independently watched a video of the child’s chest movements, edited with the layover of the ChARM start and stop sound and a 60 second timer, and counted the number of breaths observed in a full minute. Beep sounds were added by the videographer in sync with the original sounds made by the ChARM device at the start (when the start button on ChARM is pushed) and at the end (when the ChARM displays the result). The sound recorded at the time of recording was also muted to allow the VEP to focus on the sound of the start and stop beeps.

    If the first two VEP members agreed (≤±2 bpm), a mean RR count was used as the reference standard. If they disagreed (>±2 bpm), a third VEP member reviewed the video and if two out of three counts agreed (≤±2 bpm), the mean of the two closest RR counts was used. If all three VEP members disagreed (>±2 bpm), the video was sent for review to a fourth VEP member. If the fourth VEP member’s count agreed (≤±2 bpm) with any of the first three VEP members’ counts, the mean of the two closest counts was used. If all four panel members disagreed (>±2 bpm), the data from this evaluation were excluded from the agreement analysis.

    Sample Size

    The primary outcome on which sample size was based was the agreement between the ARIDA and VEP RR counts. As per Bland-Altman [17], we conducted a precision-based sample size calculation based on the confidence interval for the 95% limits of agreement. The formula estimates the required number of children per age group (n) based on the desired width of the confidence interval. Using normal approximation and allowing a confidence interval of ±0.5 standard deviations of the difference between the two devices, a sample size of 46 children per age group was required for the agreement and consistency evaluations, adjusted to 52 per group for failure to get a reference standard count. For the RR fluctuation evaluation, a sample size of 30 children was used.

    Data Analysis

    Data analysis for all three RR evaluations was conducted in Stata 13 (StataCorp LLC) and Excel (Microsoft Corp). First, the number of children screened, eligible, consented, and enrolled in each type of evaluation was described. Baseline characteristics (age and sex) by screening breathing status (normal/fast) for those enrolled were described. All full-length source videos were reviewed for quality assurance purposes, and descriptive information on the video quality was recorded, including those where all four VEP members disagreed on the RR count. For the ARIDA and EC agreement and consistency evaluations, mean difference, root mean square difference, absolute mean difference, proportion of RR counts ±2 bpm from the reference standard, and positive and negative percentage agreement with 95% confidence intervals were calculated in Stata 13 by age group, and Bland-Altman plots with limits of agreement and 95% confidence intervals by age group and breathing status were created. Percentage of unsuccessful attempts and failures (defined by three unsuccessful attempts) and mean time to get an ARIDA RR count were calculated. A per-protocol analysis was used whereby children were excluded from the analysis if an RR could not be obtained simultaneously by the ARIDA and by the EC, with a VEP RR reading where at least two of the panel members were within ±2 bpm of each other. For the RR fluctuation evaluation, mean difference in the RR count between baseline and 1 minute, 1 and 3 minutes, and 3 and 5 minutes were calculated. The proportion of children with fast or normal RR classification at baseline and the change between RR classifications over time were analyzed.

    Quality Assurance, Supervision, and Monitoring

    Malaria Consortium and UNICEF (Supply Division and Ethiopia Country Office) conducted quality assurance visits every 2 weeks to the research site during data collection. All data collected from the screening and RR evaluations were checked and verified by the data manager daily. A sample of three videos was sent weekly to an independent study advisor for RR evaluation using WHO IMCI guidelines [3] and to Malaria Consortium HQ for quality assurance. The project had an 11-person Advisory Committee made up of experts on maternal and child health who provided technical oversight and reviewed the study protocol.


    Results

    The project was funded in 2016. Data were collected between 5 April until 22 May 2017. Authors are drafting the results for publication.


    Discussion

    Accurately diagnosing pneumonia in children aged younger than 5 years remains a significant problem in resource-poor settings. Manually counting RR is inherently challenging for CHWs, resulting in both over and under diagnosis and treatment. This diagnostic performance study in Ethiopia aims to provide evidence for the performance of an ARIDA device when used in a controlled setting.

    Evaluating performance of new RR counters is difficult due to the absence of an appropriate gold standard. Selecting a robust reference standard when designing this study was a challenge. The aim was to have one reference standard for evaluating any new ARIDA regardless of the technology for calculating RR. Retrospective review of video recordings by a panel of experts has been used as a reference standard for a number of pneumonia studies [18-20]. It allows many experts to assess the same patient, thus limiting bias that could arise from having one expert per child and reducing the number of experts present in the room, whose presence could agitate the child and affect their RR. It also allows the expert to review the evaluation numerous times and adjust the speed and zoom of the video to aid the counter. An interrater agreement study in northeast Tanzania measured the agreement between two pediatricians reviewing RR videos of children aged 2 to 59 months. They found that in two-thirds of cases, both pediatricians agreed on the RR within ±2bpm, which represents fair agreement (kappa=.34) and in ninety-six percent of cases, both pediatricians agreed on RR classification, representing perfect agreement (kappa=.85) [21]. Recognizing the limitations of humans counting RR using a video, in the absence of a gold standard and with recommendations from the Advisory Committee, the video reference standard was selected.

    A strength of this study is that the design allows for contemporaneous comparison between the RR count from the ARIDA, EC, and video reference standard. While the comparison is imperfect, as the ARIDA can take longer than 60 seconds to obtain a count compared with the VEPs and ECs who assessed RR over 60 seconds, it remains useful for identifying increased RR and therefore whether the RR classification of breathing status is comparable.

    To minimize RR counting errors, this study was implemented with two days of training and an assessment for the VEP members and ECs to ensure a consistent methodology for RR counting. Interrater agreement between humans could be improved with guidance about how to define a breath versus a movement and additional standardization between humans through training and practice using this guidance to count RR for a selection of videos. Furthermore, a video annotation aid that allows the panel member to mark breaths and non–breath movements directly on the video could reduce human RR counting inconsistencies and allow for discussion and consensus building between panel members about the RR of videos.

    Mean difference with 95% confidence intervals was selected as the agreement measure for the primary outcome. A disadvantage of this measure is that positive and negative bias cancel out to give a lower mean difference. An alternative is to use the limits of agreement with 95% confidence intervals on the Bland-Altman plot to visually show the agreement between the two measures and estimate the precision of the estimates. We recommend that global consensus and guidance on an acceptable level of agreement between a new automated RR counter and a reference standard as measured by the range of the limits of agreement on a Bland-Altman plot is sought in addition to global consensus on the reference standard methodology. This will provide ministries of health with evidence to make an informed decision on the performance of new RR devices to inform introduction and scale up of these devices.

    Acknowledgments

    We would like to thank St Paul’s Hospital Millennium Medical College for allowing us to conduct the research at their hospital and allowing their staff to be available for the VEP. We also extend thanks to the videographer, Tewodros Emiru, Dr Hollie Spence who helped in the additional review of video references, and the research team: Yenealem Reta, Fiseha Mihretu, Ruth Haiu, Okulu Sultan, Hanna Birkneh, Lidiya Zerihun, Atresaw Zegye, Kalkidan Belay, Sileshi Tsegaye, and Aynalem Hailem. We are also grateful for the continued support from the Ethiopian Federal Ministry of Health and members of the Advisory Committee: Quique Bassat, David Milestone, Kristoffer Gandrup-Marino, Gian Gandhi, Mark Young, Leith Greenslade, Shamin Qazi, Jim Black, Raza Izadnegahdar, Salim Sadruddin, and HRH the Infanta Cristina of Spain. The study was sponsored by UNICEF in partnership and with funding from “la Caixa” Banking Foundation. The manufacturer of the device (Philips) did not contribute any funding for this work and did not participate in any data analysis or interpretation. The Malaria Consortium research team has subsequently received funding from the Philips Foundation to conduct future work on developing an improved respiratory rate reference standard.

    Authors' Contributions

    KK, KB, CW, SM, DG, JHB, PL, and NM designed the study protocol. CW, KB, SM, and KK created the data collection tools and standard operating procedures with review from TH, CM, and JHB. TH, CW, KB, SM, and KK implemented the data collection. MP, KK, CW, KB, and SM designed and implemented the data analysis. CW drafted the paper. All authors read and approved the final manuscript.

    Conflicts of Interest

    None declared.

    References

    1. McAllister D, Liu L, Shi T, Chu Y, Reed C, Burrows J, et al. Global, regional, and national estimates of pneumonia morbidity and mortality in children younger than 5 years between 2000 and 2015: a systematic analysis. Lancet Glob Health 2019 Jan;7(1):e47-e57 [FREE Full text] [CrossRef] [Medline]
    2. Requejo J, Bryce J, Barros A, Berman P, Bhutta Z, Chopra M, et al. Countdown to 2015 and beyond: fulfilling the health agenda for women and children. Lancet 2015 Jan 31;385(9966):466-476. [CrossRef] [Medline]
    3. World Health Organization. Integrated management of childhood illness: chart booklet 2014   URL: http://apps.who.int/iris/bitstream/10665/104772/16/9789241506823_Chartbook_eng.pdf [accessed 2020-02-10]
    4. Young M, Wolfheim C, Marsh D, Hammamy D. World Health Organization/United Nations Children's Fund joint statement on integrated community case management: an equity-focused strategy to improve access to essential treatment services for children. Am J Trop Med Hyg 2012 Nov;87(5 Suppl):6-10 [FREE Full text] [CrossRef] [Medline]
    5. Mukanga D, Babirye R, Peterson S, Pariyo G, Ojiambo G, Tibenderana J, et al. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health 2011 Oct;16(10):1234-1242 [FREE Full text] [CrossRef] [Medline]
    6. Källander K, Tomson G, Nsabagasani X, Sabiiti JN, Pariyo G, Peterson S. Can community health workers and caretakers recognise pneumonia in children? Experiences from western Uganda. Trans R Soc Trop Med Hyg 2006 Oct;100(10):956-963. [CrossRef] [Medline]
    7. Muro F, Mtove G, Mosha N, Wangai H, Harrison N, Hildenwall H, et al. Effect of context on respiratory rate measurement in identifying non-severe pneumonia in African children. Trop Med Int Health 2015 Jun;20(6):757-765 [FREE Full text] [CrossRef] [Medline]
    8. UNICEF. ARIDA (Acute Respiratory Infection Diagnostic Aid) 2017   URL: https://www.unicef.org/innovation/innovation_81722.html [accessed 2020-02-10]
    9. Ginsburg AS, Sadruddin S, Klugman KP. Innovations in pneumonia diagnosis and treatment: a call to action on World Pneumonia Day, 2013. Lancet Glob Health 2013 Dec;1(6):e326-e327 [FREE Full text] [CrossRef] [Medline]
    10. Källander K, Young M, Qazi S. Universal access to pneumonia prevention and care: a call for action. Lancet Respir Med 2014 Dec;2(12):950-952. [CrossRef] [Medline]
    11. UNICEF. Target Product Profile: acute respiratory infection diagnostic aid (ARIDA). Copenhagen: UNICEF Supply Division; 2014 Nov 14.   URL: https://www.unicef.org/videoaudio/PDFs/ARIDA_-_Target_Product_Profile_(2).pdf [accessed 2020-02-11]
    12. UNICEF. Estimates of child cause of death, acute respiratory infection 2018   URL: https://data.unicef.org/wp-content/uploads/2018/02/CoD_ARI_Feb-2018_WHO_MCEE_236.xlsx [accessed 2020-02-11]
    13. Legesse H. National scale-up of integrated community case management in rural Ethiopia: implementation and early lessons learned. Ethiop Med J 2014 Oct;52 Suppl 3:15-26. [Medline]
    14. World Health Organization. 2014 Mar. Integrated management of newborn and childhood illness chart booklet   URL: https://tinyurl.com/wnhee4m [accessed 2020-03-02]
    15. USAID. 2015. Project CLEAR: pneumonia pretesting outputs   URL: https://tinyurl.com/uf8xv55 [accessed 2020-03-02]
    16. Philips. ChARM Instructions for use 2016   URL: https://tinyurl.com/t27txn4 [accessed 2020-02-11]
    17. Bland JM, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986 Feb 08;1(8476):307-310. [Medline]
    18. Sinyangwe C, Graham K, Nicholas S, King R, Mukupa S, Källander K, et al. Assessing the quality of care for pneumonia in integrated community case management: a cross-sectional mixed methods study. PLoS One 2016;11(3):e0152204 [FREE Full text] [CrossRef] [Medline]
    19. Black J, Gerdtz M, Nicholson P, Crellin D, Browning L, Simpson J, et al. Can simple mobile phone applications provide reliable counts of respiratory rates in sick infants and children? An initial evaluation of three new applications. Int J Nurs Stud 2015 May;52(5):963-969. [CrossRef] [Medline]
    20. Karlen W, Gan H, Chiu M, Dunsmuir D, Zhou G, Dumont GA, et al. Improving the accuracy and efficiency of respiratory rate measurements in children using mobile devices. PLoS One 2014;9(6):e99266 [FREE Full text] [CrossRef] [Medline]
    21. Muro F, Mosha N, Hildenwall H, Mtei F, Harrison N, Schellenberg D, et al. Variability of respiratory rate measurements in children suspected with non-severe pneumonia in north-east Tanzania. Trop Med Int Health 2017 Feb;22(2):139-147 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    ARI: acute respiratory infection
    ARIDA: Acute Respiratory Infection Diagnostic Aid
    bpm: breaths per minute
    ChARM: Children’s Respiration Monitor
    CHW: community health worker
    EC: expert clinician
    FLHFW: first-level health facility worker
    HEW: health extension worker
    iCCM: Integrated Community Case Management
    IMCI: Integrated Management of Childhood Illness
    IMNCI: Integrated Management of Neonatal and Child Illness
    RR: respiratory rate
    TPP: target product profile
    UNICEF: United Nation Children’s Fund
    VEP: video expert panel
    WHO: World Health Organization


    Edited by G Eysenbach; submitted 16.10.19; peer-reviewed by ECY Su, S Rhedin, S Sanchez; comments to author 30.11.19; revised version received 03.01.20; accepted 14.01.20; published 02.04.20

    ©Charlotte Ward, Kevin Baker, Sarah Marks, Dawit Getachew, Tedila Habte, Cindy McWhorter, Paul Labarre, Jonathan Howard-Brand, Nathan P Miller, Hayalnesh Tarekegn, Solomie Jebessa Deribessa, Max Petzold, Karin Kallander. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 02.04.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.