Background: Pneumonia is one of the leading causes of death in children aged under 5 years in both sub-Saharan Africa and Southeast Asia. The current diagnostic criterion for pneumonia is based on the increased respiratory rate (RR) in children with cough or difficulty breathing. Low oxygen saturation, measured using pulse oximeters, is indicative of severe pneumonia. Health workers often find it difficult to accurately count the number of breaths, and the current RR counting devices are often difficult to use or unavailable. Nonetheless, improved counting devices and low-cost pulse oximeters are now available on the market.
Objective: The objective of our study was to identify the most accurate, usable, and acceptable devices for the diagnosis of pneumonia symptoms by community health workers and first-level health facility workers or frontline health workers in resource-poor settings.
Methods: This was a multicenter, prospective, two-stage, observational study to assess the performance and usability or acceptability of 9 potential diagnostic devices when used to detect symptoms of pneumonia in the hands of frontline health workers. Notably, 188 possible devices were ranked and scored, tested for suitability in a laboratory, and 5 pulse oximeters and 4 RR timers were evaluated for usability and performance by frontline health workers in hospital, health facility, and community settings. The performance was evaluated against 2 references over 3 months in Cambodia, Ethiopia, South Sudan, and Uganda. Furthermore, acceptability and usability was subsequently evaluated using both qualitative and quantitative methodologies in routine practice, over 3 months, in the 4 countries.
Results: This project was funded in 2014, and data collection has been completed. Data analysis is currently under way, and the first results are expected to be submitted for publication in 2018.
Conclusions: This is the first large-scale evaluation of tools to detect symptoms of pneumonia at the community level. In addition, selecting an appropriate reference standard against which the devices were measured was challenging given the lack of existing standards and differences of opinions among experts. The findings from this study will help create a standardized and validated protocol for future studies and support further comparative testing of diagnostic devices in these settings.
Trial Registration: Australian New Zealand Clinical Trials Registry ACTRN12615000348550; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=367306&isReview=true (Archived by Website at http://www.webcitation.org/72OcvgBcf)
International Registered Report Identifier (IRRID): RR1-10.2196/10191
Pneumonia is one of the leading causes of death in children aged under 5 years, accounting for an annual 944,000 deaths globally, and 60% of these deaths occur in just 10 countries in South Asia and sub-Saharan Africa . Deaths from pneumonia in children result mostly from delayed presentation to appropriate care, inappropriate treatment, or presuming the symptoms are of malaria [ ]. Children with severe pneumonia have additional symptoms and danger signs such as chest in-drawing, stridor, and wheezing, which some health care workers are not able to adequately recognize, and subsequently, such children are treated with or referred for antibiotic treatment and oxygen therapy [ ].
To diagnose pneumonia, frontline health workers are taught to observe a child’s chest for a full minute to visually identify and count the child’s breaths or respiratory rate (RR) and assess whether the RR is higher than the normal parameters for a child of that age, as defined by the World Health Organization (WHO) . Counting the RR can be challenging in itself for health workers, and the misclassification of observed rate is common.
However, even with the deployment of the 1-minute acute respiratory infection timer in the early 1990s, counting the RR continues to prove challenging, as children breathe irregularly and faster than adults and may not sit still for a full minute. Misclassification of the observed rate is, therefore, still common, leading to incorrect diagnosis and, consequently, inappropriate treatment [- ].
In addition, the acute respiratory infection timer has several shortcomings, such as short battery life and a distracting ticking sound every second, which can lead a health worker to count the sound instead of the chest movements. In a recent observational study, only 3 of 10 Mozambican community health workers (CHWs) counted the RR in children with a cough; of them, 1 counted the ticking of the timer, resulting in an RR of 60 breaths per minute, whereas other CHWs carried the timer but never used it, and the timer did not work in one case .
In addition, delays in seeking treatment put children at risk of developing severe pneumonia, and the inability of health care workers to adequately recognize danger signs and urgently refer children to a higher level of care leads to the death of many children. Hypoxemia, a symptom of severe pneumonia, has been identified as a predictor for morbidity and mortality in children with respiratory illness . However, hypoxemia is poorly identified on the basis of clinical findings alone [ ]. While pulse oximetry can be used to measure oxygen saturation (SpO2) and is a reliable and noninvasive method for identifying children with hypoxemia, pulse oximeters are rarely available outside of higher-level facilities in resource-constrained countries.
Integrated community case management (iCCM) is an approach developed by the WHO, United Nations Children’s Fund (UNICEF), and partners  where CHWs are trained to identify and treat pneumonia, malaria, diarrhea, and malnutrition in children aged under 5 years, as well as refer severely ill children to the nearest health facility. Evidence in African countries shows that health workers, if properly trained and equipped, can potentially reduce child deaths from malaria, pneumonia, and diarrhea by up to 60% through the delivery of iCCM [ ]. The integrated management of childhood illness (IMCI) was developed by the WHO to support health workers in health centers to better manage childhood illnesses [ ].
More recently, and partly as a response to the scale-up of large iCCM projects in sub-Saharan Africa and Southeast Asia, new pneumonia diagnostic aids have been developed by industry, academia, and other partners to improve the accuracy and effectiveness of diagnosing pneumonia in resource-poor contexts .
Study Aim and Objectives
This paper presents the protocol of a study aimed to identify the most accurate, acceptable, scalable, and user-friendly RR counters and pulse oximeters for the diagnosis of pneumonia symptoms in children by CHWs and first-level health facility workers (FLHFWs) with different levels of training in 4 countries: Cambodia, Ethiopia, South Sudan, and Uganda. The two main objectives were (1) to evaluate the accuracy of 9 RR counters and pulse oximeters in the hands of CHWs and FLHFWs in 4 low-resource countries and (2) to assess the usability and acceptability of the 9 devices among CHWs and FLHFWs in 4 low-resource countries when used in the routine practice over a 3-month period.
Ethical Approval and Consent to Participate
The study was approved by the ethical review boards in each study country at the national or regional level: in Ethiopia, from the Southern Nations Nationalities Peoples’ Region Health Bureau Health Research Review Committee (Ref: 6-19/10342); in Uganda, from the Uganda National Council for Science and Technology (Ref: HS 1585); in South Sudan, from the Research and Ethics Committee at the Government of South Sudan, Ministry of Health (dated May 23, 2014); in Cambodia, from the National Ethics Committee for Health Research (Ref: 0146 NECHR); and by the Regional Ethics Committee in Stockholm, Sweden (Ref. 2017/4:10). Participants were recruited only after obtaining written informed consent. The clinical evaluation is registered with the Australia New Zealand Clinical Trials Registry (ACTRN12615000348550).
A two-phase, mixed-methods design was developed to examine the performance, acceptability, and usability of the 9 devices in the 4 countries (). The conceptual framework for the design was adapted from the WHO documents “Health technology assessment of medical devices” [ ] and “Introducing new technology safely” [ ].
The first phase, the performance evaluation element of the study, was a multicenter, single-blind comparison of the performance of devices to detect symptoms of pneumonia in the hands of frontline health workers using 2 reference standards. The second phase, the acceptability and usability evaluation element of the study, was a mixed-methods, multicenter, observational study using both qualitative and quantitative data to compare the acceptability and usability of devices to detect symptoms of pneumonia in the hands of CHWs and FLHFWs in routine practice.
This was a multicountry study implemented in Cambodia, Ethiopia, South Sudan, and Uganda as all 4 countries have a high proportion of deaths of children under 5 years caused by pneumonia (16%-21%), as well as a high incidence of pneumonia, and all have implemented the Ministry of Health-defined iCCM and IMCI programs. However, the characteristics of the health worker programs, such as the length of training, literacy level, and RR timing devices used, differed by country ().
The study sites selected for conducting phase 1 were all district hospital-level facilities selected after the analysis was conducted on patient flow to understand whether the individual research sites could support the sample size required by the study for enrollment, as it would not have been possible to achieve the sample if phase 1 had been conducted at the community level ().
Phase 1 research sites were as follows: Mpigi General Hospital, approximately 45 miles from Kampala in Uganda; Yrgalem District Hospital in Southern Nations and Nationalities and People’s Region in Ethiopia; Borkeo District Hospital in Ratanakiri province in Cambodia; and Aweil General Hospital in Northern Bahr el Ghazal state in South Sudan. For phase 2 of the study, frontline health workers were selected depending on having participated in phase 1 and being within 20 km of the health facility used in phase 1 in order to have access to the functioning oxygen equipment and severe illness case management capabilities.
|Pneumonia deaths (% of total under-5 deaths) ||17||17||21||16|
|Pneumonia incidence in children under 5 years (number of episodes per child per year) ||0.25||0.28||0.32a||0.27|
|Proportion of children aged under 5 years with suspected pneumonia and receiving antibiotics (%) ||39||7||33||47|
|Name for community health worker (CHW)||Extended village malaria worker||Health extension worker||Community drug distributor||Village health team member|
|Length of initial training||5 days (2 days malaria training + 3 days in sick child case management)||1 year||6 days||11 days (5 days basic training + 6 days in sick child case management)|
|Literacy level||Low||High||Extremely low||Low-median|
|Pneumonia diagnosis tool||Acute respiratory infection (ARI) timer||Wrist watch or ARI timer||ARI timer + beads||ARI timer|
|Catchment population per CHW||130-150 households||400-500 households||250-300 households||250-500 households|
|Average caseload per month||8||12||9||12|
aData are for Sudan.
|Description||Cambodia Ratanakiri province||Ethiopia (Dale & Shebedino Districts), SNNPRa||South Sudan (Aweil West & Center counties) Northern Bahr el Ghazal State||Uganda, Mpigi District|
|Children under 5 years population, N (%)||37,720 (20.50)||82,582 (15.61)||25,000 (19.49)||51,363 (20.50)|
|Number of community health workers||270||161||1683||650|
|Number of health centers||23||19||20||39|
|Number of hospitals||2||2||1||1|
aSNNPR: Southern Nations and Nationalities and People’s Region.
A number of activities were conducted before the field trials to select the test devices. First, the formative research was conducted to inform the attributes used in the subsequent device scoring . Second, an initial landscape review was conducted where 188 possible devices were identified [ ]; these devices were further evaluated in a review of technical specifications [ ]. Third, all potential devices were scored and ranked using 20 device attributes, including measures of usability, utility, scalability, and user acceptance. provides the table with the device scoring based on the attributes. In contrast to the respiratory timing devices identified, the 8 selected pulse oximeters had not previously been field-tested, and before taking them to the field trials, they were first tested in a laboratory for accuracy and environmental robustness. Based on the laboratory test results, the final 9 devices were taken forward to the performance evaluation phase ( ), that is, 4 RR devices (manual and assisted counters) and 5 pulse oximeters (fingertip and handheld devices) [ ].
The devices allocated to each country were based on the suitability for individual country context; overall, 9 devices were tested for performance (phase one) and the usability and acceptability in the routine practice (phase 2).
The sample size calculation for phase 1 was based on the precision of the mean difference between the device and the reference respiratory count, assuming the normal distribution. An SD of 7 for the difference was obtained in a previous study evaluating the performance of RR counters  and in requiring a maximal total length of the 95% CI of 4 units, which is the same range as the WHO-accepted maximal absolute breathing rate deviance (eg, ±2 breaths per min); the minimum sample size was 47 children per strata for independent observations. The two age strata in the study were (1) 0-60 days and (2) 2-59 months, and 3 device pairs per country gave a total sample size of 282 children. The sample size was then increased by 50% to 423 and rounded off to 430 children per country to accommodate for potential clustering at the CHW level [ ]. For the usability and acceptability of the routine practice assessment, a sample of 20 CHWs and 5 FLHFWs was recruited to do the assessments in each country over the 3 months.
The primary outcome for phase 1 was the agreement between each health worker measurement, using the test device, and that of the reference standard, calculated as the proportion of 1-minute observations that were within ±2 breaths or ±2% SpO2 for each of the 9 devices. The secondary outcomes included the agreement in classification of the breath rate into (1) normal or fast breathing and oxygen saturation into (2) normal oxygen saturation (SpO2≥90%) or hypoxemia (SpO2<90%) and included agreement statistics appropriate for situations when no gold standard exists, such as positive percent agreement, negative percent agreement, and Cohen kappa statistic.
The primary outcome for phase 2 (the usability and acceptability evaluation) was acceptability, based on users’ perceptions of the different devices used. Secondary outcomes for phase 2 included the proportion of users who accurately followed the correct procedures and the caregivers’ perceptions of, interaction with, and reaction to the devices when used on children.
In phase 1, all children aged 0-5 years presenting at the health facilities where the study was conducted were screened for eligibility and invited to participate. All young infants aged 0-60 days were eligible, as were children aged 2-59 months with a cough and/or difficulty in breathing. Children with an illness of >2-week duration or exhibiting one or more of the IMCI danger signs (severe dehydration, agitation, inconsolable, neck stiffness, active convulsions or fits, unconscious or lethargic, not breastfeeding, and vomiting everything) as well as those with severe burns, with neutropenia, with a severe infectious disease, or ineligible as advised by the supervising clinician were excluded.
In the absence of a gold standard, 2 reference standards were used for phase 1 of this study: (1) an IMCI-trained expert clinician (EC) and whose RR counting skills were standardized to ±2 breaths per minute on 5 video recorded children and (2) an automated monitoring device (Masimo Root patient monitoring and connectivity platform with Phasein ISA CO2 capnography using nasal cannulas and Radical 7 pulse oximeter)  that provided a measurement for the same period. All health workers received 2 days of training prior to data collection, including a refresher module on iCCM as well as practice sessions on RR counting and using a new device. All health workers needed to receive a pass mark of ≥90% on a competency-based assessment before participating in data collection. On consenting to the studying and entering the research room, a child was positioned comfortably on a caregiver’s lap and calmed and attached to the Masimo reference device. For the RR counters, 2 assessments were performed by the health workers and recorded, along with the corresponding Masimo reference measurements, and within 5 minutes of the health worker measurements, the ECs took 2 RR measurements using a stopwatch. For pulse oximeter devices, the health workers took 2 SpO2 measurements, and the EC also took 2 SpO2 measurements using the same pulse oximeter the health worker had used, along with simultaneous Masimo SpO2 reference measurements. The health workers were asked to classify a child into fast or normal breathing using the WHO age-specific cut-offs for RR for RR devices or for severe or nonsevere pneumonia based on the SpO2 reading being <90% for pulse oximeters.
In phase 2, to assess the usability and acceptability of the devices in routine practice by health workers and explore their acceptability to caregivers, field testing was conducted over 3 months. This was a mixed-methods study incorporating structured observations, video recordings of procedures, and qualitative exit interviews with health workers and caregivers. During the activity, 100 health workers were trained for 2 days to use an RR counter and a pulse oximeter as part of their routine iCCM or IMCI activity. Each health worker had to pass a competency-based assessment before participating in the data collection. The research team scheduled visits with each health worker 3 times (once a month) during the evaluation, each time gathering a minimum of 5 assessments of each CHW or FLHFW. The health workers took the medical history of children as per the iCCM or IMCI guidelines, and if cough and/or difficulty breathing were recorded, the health workers used the RR device to count the number of breaths in 1 minute; a procedure that was repeated twice, and the highest reading was used for classification. The observed RR was used by the health workers to decide whether or not to provide treatment for pneumonia using the national treatment guidelines. If fast breathing was detected, the health workers assessed for hypoxemia using the pulse oximeter by taking two SpO2 readings and used the lowest reading for classification. All children with signs and symptoms of severe pneumonia and with SpO2 <90% were referred.
Paper-based data collection tools were developed in local languages for both phases and collected data on screening, usability and performance, and adverse events. All data collection tools were developed in collaboration across the 4 research sites and were translated and pilot-tested in all locations before the start of the study. The tools for performance and usability included demographic information, child status, device measurement results, health worker classification of results, time taken, failures, and usability checklists. In addition, semistructured interview guides were developed for the exit interviews with caregivers and health workers (). Video recordings and photographs of the health worker log books were captured as back-up to the paper forms submitted. All completed forms and log book photos were returned to the Malaria Consortium office for double data entry using EpiData version 3.1 (EpiData Association, Odense, Denmark) and filed. The qualitative data collection consisted of in-depth interviews with health workers at the end of data collection to capture their views on the usability and acceptability of the devices. Each data form had a unique identification code to link data from different forms in the database.
The analysis of the primary outcome for phase 1, the agreement between each health worker measurement using the test device and that of the reference standards, was done following a per-protocol approach for each RR and SpO2 measurement. Agreement was calculated as the proportion of observations that were within ±2 breaths or SpO2% for each of the 9 devices, respectively, in the per-protocol population (only including children who had a cough and difficulty breathing and no danger signs). The secondary outcomes for the performance evaluation, the agreement in classification of the RR or SpO2 obtained by health workers and the reference standards was measured using Cohen kappa statistic, positive percent agreement, and negative percent agreement of each device. For all of these secondary outcomes, the unit of analysis was the child rather than the device measurements. To illustrate the agreement between different devices, Bland-Altman plots  were produced to visualize the agreement in ratings of respiratory counts between reference standards and CHWs. The primary outcome for the usability and acceptability evaluation in phase 2 used qualitative analysis to establish the acceptability, based on users’ perceptions of the different devices used. Secondary outcomes for the acceptability and usability evaluation included the proportion of users who accurately followed the correct procedures and the caregivers’ perceptions of, interaction with, and reaction to the devices when used on children.
This project was funded in 2014. Data collection started on February 4, 2015 and was completed on December 31, 2015. Data analysis is currently under way and the first results are expected to be submitted for publication in 2018.
To reduce the number of child deaths from pneumonia in low-resource settings and to minimize unnecessary use of antibiotics, it is crucial to improve the diagnosis of pneumonia symptoms by frontline health workers . This study aimed to meet this need by identifying possible improved pneumonia diagnostic devices and evaluated 9 of these in “real-life” health system contexts in 4 different countries in 2 continents. One of the key factors in designing a study like this is agreeing on the gold or reference standard to compare the devices to. The gold standard is the best single test (or a combination of tests) that is considered the current preferred method of diagnosing a particular disease [ ]. Many possible gold standards have been suggested for RR timing devices in the literature, such as simultaneous counting by a clinical expert [ ], real-time electronic monitoring, or retrospective review of video recordings by a panel of experts [ , ]. However, all these methods have limitations such as the inaccuracy of human counters [ ] and the inconsistency of counting between humans and electronic monitoring devices [ ]. On review, and through discussions with experts from the WHO, UNICEF, and academia in the scientific advisory committee for the study, it was decided that no suitable gold standard exists in this setting; therefore, it was agreed to use 2 imperfect reference standards—the automated Masimo Root patient monitoring and connectivity platform with Radical-7 pulse oximeter and capnograph using Phasein ISA CO2 module [ ] and a simultaneous assessment by a clinical expert. Furthermore, it was hoped that using 2 different types of references, in the absence of a suitable gold standard, would best account for the limitations outlined above.
We would like to thank all the country research teams who assisted with the development of the research protocols, Sarah Marks for reviewing the manuscript draft, and Jill Nicholson for coordinating the protocol design process and workshops. Funding for this study was provided by the Bill & Melinda Gates Foundation. However, the authors have not received any funding or benefits from the industry to conduct this study.
Conflicts of Interest
Multimedia Appendix 1
Complete scoring sheet for 188 possible pneumonia diagnostic aids, including respiratory rate timers, pulse oximeters and joint devices.XLS File (Microsoft Excel File), 264KB
Multimedia Appendix 2
Summary of final devices tested in the field for performance, acceptability, and usability.PDF File (Adobe PDF File), 93KB
Multimedia Appendix 3
Discussion guide for caregiver interviews.PDF File (Adobe PDF File), 48KB
- Requejo J, Bryce J, Barros A, Berman P, Bhutta Z, Chopra M, et al. Countdown to 2015 and beyond: fulfilling the health agenda for women and children. The Lancet 2015 Jan;385(9966):466-476. [CrossRef]
- Bojalil R, Kirkwood BR, Bobak M, Guiscafre H. The relative contribution of case management and inadequate care-seeking behaviour to childhood deaths from diarrhoea and acute respiratory infections in Hidalgo, Mexico. Trop Med Int Health 2007 Dec;12(12):1545-1552 [FREE Full text] [CrossRef] [Medline]
- WHO. Revised WHO classification and treatment of childhood pneumonia at health facilities. In: WHO. Geneva: WHO; 2014.
- Muro F, Mtove G, Mosha N, Wangai H, Harrison N, Hildenwall H, et al. Effect of context on respiratory rate measurement in identifying non-severe pneumonia in African children. Trop Med Int Health 2015 Jun;20(6):757-765 [FREE Full text] [CrossRef] [Medline]
- Källander K, Tomson G, Nsabagasani X, Sabiiti J, Pariyo G, Peterson S. Can community health workers and caretakers recognise pneumonia in children? Experiences from western Uganda. Trans R Soc Trop Med Hyg 2006 Oct;100(10):956-963. [CrossRef] [Medline]
- Mukanga D, Babirye R, Peterson S, Pariyo G, Ojiambo G, Tibenderana J, et al. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health 2011 Oct;16(10):1234-1242 [FREE Full text] [CrossRef] [Medline]
- Nanyonjo A. Counting breaths in children: A comparative study of the accuracy of the ARI timer alone, the ARI timer with beads and the Malaria Consortium mobile respiratory timer. Kampala: Malaria Consortium; 2012.
- Svanberg M. Can community health workers in Mozambique dignose and treat diarrhea, malaria and pneumonia in children? Evaluation of performance one year after training in Massinga district, Mozambique. Stockholm: Karolinska Institute; 2012.
- Subhi R, Adamson M, Campbell H, Weber M, Smith K, Duke T. The prevalence of hypoxaemia among ill children in developing countries: a systematic review. The Lancet Infectious Diseases 2009 Apr;9(4):219-227. [CrossRef]
- Emdin C, Mir F, Sultana S, Kazi A, Zaidi A, Dimitris M, et al. Utility and feasibility of integrating pulse oximetry into the routine assessment of young infants at primary care clinics in Karachi, Pakistan: a cross-sectional study. BMC Pediatr 2015 Sep 30;15(1):141 [FREE Full text] [CrossRef] [Medline]
- UNICEF. UNICEF. 2012. An equity-focused strategy to improve access to essential treatment services for children URL: http://www.unicef.org/health/files/iCCM_Joint_Statement_2012.pdf [accessed 2018-09-16] [WebCite Cache]
- UNICEF. Countdown to 2015. Tracking progress in maternal, neonatal and child survival: the 2008 report. New York: UNICEF; 2008.
- World Health Organization. Integrated management of childhood illnesses - caring for newborms and children in the community. Geneva: WHO; 2011.
- UNICEF. Pneumonia Diagnostics: Current Outlook and Perspectives. Copenhagen: UNICEF; 2013.
- World Health Organization. Health technology assessment of medical devices. Geneva: WHO; 2011.
- Mytton O, Velazquez A, Banken R, Mathew J, Ikonen T, Taylor K, et al. Introducing new technology safely. Qual Saf Health Care 2010 Aug;19 Suppl 2(Suppl 2):i9-14 [FREE Full text] [CrossRef] [Medline]
- A Decade of Tracking Progress for Maternal, Newborn and Child Survival - The 2015 Report. In: UNICEF. New York: UNICEF; 2015.
- Rudan I, O'Brien K, Nair H, Liu L, Theodoratou E, Qazi S, Child Health Epidemiology Reference Group (CHERG). Epidemiology and etiology of childhood pneumonia in 2010: estimates of incidence, severe morbidity, mortality, underlying risk factors and causative pathogens for 192 countries. J Glob Health 2013 Jun;3(1):010401 [FREE Full text] [CrossRef] [Medline]
- World Health Organization. World Health Statistics 2015. Luxembourg: World Health Organization; 2016.
- Spence H, Baker K, Wharton-Smith A, Mucunguzi A, Matata L, Habte T, et al. Childhood pneumonia diagnostics: community health workers' and national stakeholders' differing perspectives of new and existing aids. Glob Health Action 2017;10(1):1290340 [FREE Full text] [CrossRef] [Medline]
- Hernandez P, Baker K, Kallander K. Initial penumonia device short listing exercise report. London: Malaria Consortium; 2014.
- Peel D, Baker K, Kallander K. Technical pneumonia device landscape review report. London: Malaria Consortium; 2014.
- Wharton-Smith A, Baker K, Kallander K. Pneumonia Diagnostics - Device Selection Report. London: Malaria Consortium; 2015.
- Noordam A, Barberá; LY, Sadruddin S, van Heck PM, Chono A, Acaye G, et al. The use of counting beads to improve the classification of fast breathing in low-resource settings: a multi-country review. Health Policy Plan 2015 Jul;30(6):696-704 [FREE Full text] [CrossRef] [Medline]
- Rowe A, Lama M, Onikpo F, Deming M. Design effects and intraclass correlation coefficients from a health facility cluster survey in Benin. Int J Qual Health Care 2002 Dec;14(6):521-523. [Medline]
- Masimo. Root Patient Monitor and Connectivity Platform with ISA CO2 module capnography 2014 URL: http://www.masimo.com/siteassets/uk2/images/capnography/isa/lab8341a_sell_sheet_capnography_isa_co2.pdf [accessed 2018-09-16] [WebCite Cache]
- Bland JM, Altman DG. Agreed statistics: measurement method comparison. Anesthesiology 2012 Jan;116(1):182-185. [CrossRef] [Medline]
- Rambaud-Althaus C, Althaus F, Genton B, D'Acremont V. Clinical features for diagnosis of pneumonia in children younger than 5 years: a systematic review and meta-analysis. The Lancet Infectious Diseases 2015 Apr;15(4):439-450. [CrossRef]
- Parikh R, Mathai A, Parikh S, Chandra SG, Thomas R. Understanding and using sensitivity, specificity and predictive values. Indian J Ophthalmol 2008;56(1):45-50 [FREE Full text] [Medline]
- Black J, Gerdtz M, Nicholson P, Crellin D, Browning L, Simpson J, et al. Can simple mobile phone applications provide reliable counts of respiratory rates in sick infants and children? An initial evaluation of three new applications. Int J Nurs Stud 2015 May;52(5):963-969. [CrossRef] [Medline]
|CHW: community health worker|
|EC: expert clinician|
|FLHFW: first-level health facility workers|
|iCCM: integrated community case management|
|IMCI: integrated management of childhood illness|
|RR: respiratory rate|
|UNICEF: United Nations Children’s Fund|
|WHO: World Health Organization|
Edited by G Eysenbach; submitted 08.03.18; peer-reviewed by J Todd, M Fajardo; comments to author 26.04.18; revised version received 27.06.18; accepted 04.07.18; published 25.10.18Copyright
©Kevin Baker, Mucunguzi Akasiima, Alexandra Wharton-Smith, Tedila Habte, Lena Matata, Diana Nanyumba, Morris Okwir, Anteneh Sebsibe, Madeleine Marasciulo, Max Petzold, Karin Källander. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 25.10.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.