This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Biomedical research in the application of noncontact methods to measure heart rate (HR) and respiratory rate (RR) in the neonatal population has produced mixed results. This paper describes and discusses a protocol for conducting a method comparison study, which aims to determine the accuracy of a proposed noncontact computer vision system to detect HR and RR relative to the HR and RR obtained by 3-lead electrocardiogram (ECG) in preterm infants in the neonatal unit.
The aim of this preliminary study is to determine the accuracy of a proposed noncontact computer vision system to detect HR and RR relative to the HR and RR obtained by 3-lead ECG in preterm infants in the neonatal unit.
A single-center cross-sectional study was planned to be conducted in the neonatal unit at Flinders Medical Centre, South Australia, in May 2018. A total of 10 neonates and their ECG monitors will be filmed concurrently for 10 min using digital cameras. Advanced image processing techniques are to be applied later to determine their physiological data at 3 intervals. These data will then be compared with the ECG readings at the same points in time.
Study enrolment began in May 2018. Results of this study were published in July 2019.
The study will analyze the data obtained by the noncontact system in comparison to data obtained by ECG, identify factors that may influence data extraction and accuracy when filming infants, and provide recommendations for how this noncontact system may be implemented into clinical applications.
RR1-10.2196/13400
The conventional method for monitoring vital signs of infants in the neonatal intensive care unit (NICU) is by the use of the electrocardiogram (ECG). This form of monitoring has disadvantages. One of the main concerns is that it is reliant on the use of electrodes that contain an adhesive layer [
Preservation of the integrity of the premature infant’s skin is essential and certainly a challenge in the NICU [
An alternative technique to the use of ECG is noncontact photoplethysmography (PPGi). The PPGi principle is based on the concept that light is absorbed by blood more than by the surrounding tissue. A pulse will therefore produce small variations in the color of the skin, indicating that blood is being circulated owing to the redness of hemoglobin. Researchers have been able to extract the heart rate (HR) by filming and magnifying this variation in skin color changes [
To capture the respiratory rate (RR), movement of the chest wall can be observed. The chest rises from movement of the diaphragm during inspiration, typically from 4 mm to 12 mm [
In our previous study [
The aim of this preliminary study is to determine the accuracy of a proposed noncontact computer vision system to detect HR and RR relative to the HR and RR obtained by 3-lead ECG in preterm infants in the neonatal unit. The specific objectives are as follows:
To determine if the noncontact system is accurate for measuring neonatal HR and RR when compared with ECG.
To identify factors that may influence data extraction and accuracy when filming preterm infants and provide recommendations for method improvement.
To provide an insight on how this noncontact system may be implemented for real-time and prototype development.
This technology may have the potential to replace conventional ECG and therefore avoid the adverse effects of its use with the preterm population.
This is a single-center cross-sectional study of the proposed noncontact computer vision system based on quantitative methods. The concepts of PPGi and motion magnification will be applied to extract physiological data, with a statistical analysis of the results to quantify the relationship between the 2 methods of HR and RR monitoring using the Bland-Altman method [
This section describes the framework used to extract HR and RR based on the physiological variations caused by cardiopulmonary activity from 2 different regions.
To extract the cardiac signal, it is necessary to amplify the subtle color variations on the skin surface using the standard Eulerian Video Magnification (EVM) method [
These modifications include using a wavelet pyramid decomposition and an elliptic band-pass filter instead of the Laplacian pyramid decomposition. After amplifying the video, the region of interest (ROI) will be manually localized using the MATLAB built-in function
where
The variations in thoracoabdominal wall movement during respiration reflect directly in the spatial changes of intensity values in the video recording. The motion magnification approach proposed in previous work [
where
The measured value (
The period (p) between 2 peaks (average) is:
where p is the number of peaks of the acquired signal and t is the length of the video signal in seconds.
The end-product is a graphic panel of the 2 sources of data extraction using skin color and motion magnification. As shown in
Participants will be selected using a convenience method. To assess the accuracy of the computer vision system in different situations, recruiting participants with confounding variables is necessary, such as respiratory support covering the face, variations in melanin concentration of infant skin, being cuddled by their parent, being positioned in an incubator, or receiving phototherapy producing a blue-light source. The inclusion criteria are as follows: infants below 37 weeks of corrected gestation who are monitored using the unit’s regular ECG monitor during the time of data collection to be used as a reference to validate accuracy of the proposed system. The exclusion criteria are as follows: term infants (≥37 weeks of gestation) or preterm infants that are not on ECG monitoring, who are likely to be discharged at the time of data collection, or who have major congenital abnormalities that could possibly make them identifiable in publications. Informed written consent will be obtained. Ethical approval for this study was obtained from the Southern Adelaide Local Network Research Committee (HREC/17/SAC/340; SSA/17/SAC/341) and the University of South Australia Human Research Ethics Committee (protocol number 0000034901).
A sample size of 10 participants will provide 2 data points per participant (from PPGi and motion magnification), totaling 20 data points. This targeted sample size was determined given that this is a preliminary study investigating this novel technology still in its developmental phase. Results from this study will inform the rationale for conducting future large-scale studies.
MATLAB graphic panel of the proposed system. HR: heart rate; RR: respiratory rate.
Calculation 1.
Calculation 2.
Equation 3.
Equation 4.
Data collection will only commence after parents or guardians have acknowledged and signed the consent form provided. Participants will be filmed for approximately 10 min, which will provide adequate opportunity for data collection and variation in HR and RR. A Nikon D5300 and Nikon D610 camera will be used to film the infants and the cardiorespiratory monitor. Each camera will be mounted to a tripod and directed from approximately one meter away (
Filming will take place by each camera concurrently, with the data and time of recording and participant number being noted. Clinicians will be able to have full access to the infant if required. If the infant was to have an apneic or bradycardic episode during filming, the clinician will be alerted by the unit’s ECG monitor. Filming will continue during this event unless directed otherwise by either the clinician or parent or guardian, because capturing significant variations in HR and RR will strengthen the efficacy of the noncontact system.
The first component of data analysis is data extraction using the noncontact system. The MATLAB environment—R2016a (MathWorks) with a Microsoft Windows 10 operating system will be utilized to conduct data analysis, producing the graphic panel for each participant (
Data points from both sources will be obtained at 3, 6, and 9 min for both RR and HR. A total of 3 data points will be determined as Altman and Bland [
Owing to the potential for rapid fluctuations in infant HR and RR, the average HR and RR measurements over a 5-second period will be analyzed using the calculation previously described. The Phillips IntelliVue monitor that will be used to validate our results determines the HR by averaging the 12 most recent HR intervals from the ECG, and the RR is calculated by averaging the last 8 detected breaths [
The second component of data analysis is the statistical comparison of the 2 different methods. MedCalc Statistical Software version 18.2.1 [
The experimental set-up for data collection.
Data analysis flowchart.
First, owing to collecting more than one measurement per participant, we cannot use a formal paired
Researcher KG will confirm the ECG readings at the selected intervals and compare the data extracted using the computer vision system which will be performed by researcher AA, limiting potential bias. A Bland-Altman analysis with multiple measurements per subject will be utilized from each method of data extraction (PPGi and motion magnification).
In total, 2 scatter plots (HR and RR) will be automatically formulated encompassing data for all participants using MedCalc software. The difference between the methods (input video vs ECG monitor;
Study enrolment and data collection began in May 2018. Results of this study have recently been published [
We will be studying a real-life population in an environment that is characterized by variables demonstrated by other researchers to cause monitoring inaccuracy. Reduced environmental levels common in the NICU have been one of the main methodological challenges owing to impacting the PPGi signal to noise ratio with unwanted interference in related work [
A possible threat to our method’s internal validity is that the preexisting impedance monitoring will be utilized as the reference standard to validate data. Impedance monitoring can be influenced by patient movement or cardiac activity, thus impacting the RR signal [
The results will address the aim and objectives of this study and inform the design of future large-scale primary studies to refine methods, identify and address potential limitations of the technology, and evaluate the overall feasibility of the proposed noncontact system.
clinically acceptable difference
electrocardiogram
Eulerian Video Magnification
Fast Fourier Transform
heart rate
neonatal intensive care unit
photoplethysmography
Red, Green, Blue
region of interest
respiratory rate
All authors provided input for the development, design, and planning of the study. KG has led the conceptualization of this study and developed this protocol under the supervision of JC, JF, and MS. JC is the Principal Supervisor of this study. JH was responsible for developing the ethical clearance documents. SM provided site support and input into planning of the study. All authors contributed, edited, and reviewed the final version.
The primary researcher has no conflicts of interest to declare. Other members of the research team (AA and JC) may have input into the potential commercialization of the prototype. As such, the team will adhere to the principles of the Australian Code for the Responsible Conduct of Research [