Published on 26.09.17 in Vol 6, No 9 (2017): September
Educational Module Intervention for Radiographers to Reduce Repetition Rate of Routine Digital Chest Radiography in Makkah Region of Saudi Arabia Tertiary Hospitals: Protocol of a Quasi-Experimental Study
Background: Repetition of an image is a critical event in any radiology department. When the repetition rate of routine digital chest radiographs is high, radiation exposure of staff and patients is increased. In addition, repetition consumes the equipment’s life span, thus affecting the annual budget of the department.
Objective: The aim of this study is to determine the impact of a printed educational module on reducing the repetition rate of routine digital chest radiography among radiographers in Makkah Region tertiary hospitals.
Methods: A quasi-experimental time series with a control group will be conducted in Makkah Region tertiary hospitals for 8 months starting in the second quarter of 2017. Four hospitals out of 5 in the region will be selected; 2 of them will be selected as the control group and the other 2 as the intervention group. Stratification and a simple random sampling technique will be used to sample 56 radiographers in each group. Pre- and postintervention assessments will be conducted to determine the radiographer knowledge, motivation, and skills and repetition rate of chest radiographs. Radiographs of the chest performed by sampled radiographers in the selected hospitals will be collected for 2 weeks before and after the intervention. A piloted questionnaire will be distributed and collected by a researcher in both groups. One-way multivariate analysis of variance and 2-way repeated multivariate analysis of variance will be used to analyze the data.
Results: It is expected that the repetition rate in the intervention group will decline after implementing the intervention and the change will be statistically significant (P<.05). Furthermore, it is expected that the knowledge, motivation, and skill levels in the intervention group will increase significantly among radiographers after implementation of the intervention (P<.05). Meanwhile, knowledge, motivation, and skills in the control group will not change.
Conclusions: A quasi-experimental time series with a control will be conducted to investigate the effect of printed educational material in reducing the repetition rate of routine digital chest radiographs among radiographers in tertiary hospitals in the Makkah Region of Saudi Arabia.
JMIR Res Protoc 2017;6(9):e185
- repetition rate;
- information motivation behavioral skills model
Crowdfunding campaign to support this specific research
We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.
Suggested contribution levels: $20/$50/$100
Good quality images in routine radiography should provide an adequate picture of the body’s anatomy. Failure to obtain a good quality image requires the radiograph to be repeated. According to Foos et al , the term “repetition” refers to redoing a radiograph of a patient that was deemed clinically unacceptable. Repetition of an image is a critical event in radiology. It is recommended that the repetition rate should not exceed 5% [ - ]. The Diagnostic Imaging Quality Assurance Committee recommends that the repetition of radiographs should not exceed 5% to 7% [ ]. The American Association of Physicists in Medicine recommends keeping the repetition rate below 6%, and when it increases to 10%, corrective action should be conducted [ ]. The Australian College of Radiologists recommends an acceptable repetition rate of 2% and not more than 5% [ ].
A study by Khafaji and Hagi  reported high repetition rates of radiography in Saudi hospitals, averaging 14.9%, which is higher than the international standard. Another study reported the repetition rate in 3 Ministry of Health hospitals ranged from 7.4% to 9.7%. The same study revealed that chest radiographs have higher repetition rates compared to other radiological procedures [ ]. Related to that, it was revealed that radiographer error is one of the factors that strongly contribute to the issue of the repetition [ , ].
The production of high-quality images is based on radiographer practices. According to the World Health Organization, practice is influenced by the level of knowledge, motivation, and skills . A study conducted in Saudi Arabia by Alsharif et al [ ] showed that there is poor knowledge among radiographers in identifying image error. Another study conducted in Saudi Arabia by Ahmed et al [ ] revealed that there is variation in radiographer knowledge of radiation protection, with 58% of radiographers indicating poor knowledge. Additionally, it was revealed that the motivation level of radiographers is low and this affects production of high-quality images [ ]. Lack of knowledge and motivation dramatically influence skill level. It has been exhibited that the increase in repetition rate is due to deficiencies in radiographer skills [ ]. Radiographers with high skills tend to avoid errors in the imaging process. Skills include the ability to communicate properly with the patient and handling the equipment accurately.
Repeated radiographs have financial and health implications, especially because of increased exposure to radiation for both staff and patients . Khafaji and Hagi [ ] and Khoshinani and Heidari [ ] added that a high repetition rate in radiography consumes the digital equipment’s lifetime by 2 months each year. This increases both staff workload and waiting time for the patient in addition to affecting the achievement of the organization’s vision.
According to Almalki et al , most interventions conducted in previous studies are technical in nature. Despite that, those intervention studies show a positive impact on the repetition rate of digital radiography. However, radiographers were not included in the studies despite them being significant factors in the repetition rate.
In general, the aim of this study is to develop, implement, and evaluate the impact of printed educational material in reducing the repetition rate of routine digital chest radiography among radiographers in Makkah Region tertiary hospitals.
The design of this study is basically a quasi-experimental time series with a control group. This design was chosen because the intervention was recommended by other researchers [, ]. A quasi-experimental study is the only design that could be applied in this study. The difficulty of randomizing by location and subject and the small number of the population make the quasi-experimental design suitable in this study [ ]. In addition, there is difficulty in randomizing by subject to avoid potential contamination. Location is also a factor, since no 2 hospitals are similar.
The Makkah Region of Saudi Arabia was selected for the study because the problem has been ignored in the area and the repetition rate has not been periodically measured there [, ]. Out of 5 hospitals, 4 will be selected to be in the study since randomization is not required in this type of study [ ]. The hospitals involved are under the direction of Makkah Region health affairs. Two will be chosen as an intervention group and 2 as a control group. Preintervention assessment will be conducted in both groups at the same time during the second quarter of 2017. The intervention will be distributed to radiographers in the intervention group, and after 1 month, an assessment will be conducted. Six months after the implementation of intervention, a second postintervention assessment will be conducted. demonstrates the study flow.
The inclusion and exclusion criteria are based on the job description of a radiographer, since there are several tasks in routine radiography in the radiology department. Only radiographers who perform routine digital chest radiography will be included. Clerk radiographers, administrative radiographers of routine digital radiography, radiographers who are on long annual leave, and radiographers who are pregnant will be excluded. Non-Saudi radiographers will be excluded as well.
The method employed to sample radiographers is stratification followed by simple random sampling. The list of radiographers will be obtained from the radiographer in charge. After that, the inclusion and exclusion criteria will be applied to radiographers derived from the list. Then, radiographers will be stratified by gender, with male respondents representing 70% of the sample and female respondents forming 30%. After that, a software number generator will be used to select the targeted sample.
Matching has been used in research since the beginning of the 19th century . Exact matching is the method that will be employed in this study. Stuart and Rubin [ ] recommended the selection of the most common covariate that has an effect on the outcome in order to make matching possible. According to Loman [ ], exact matching can be employed for up to 5 variables. Based on these principles, variables that underwent matching include gender, experience, education level, training, and the type of university from which a radiographer graduated.
Respondent characteristics will be obtained from the radiographers in charge in the control group. First, a sample from the intervention group will be randomly selected using a software number generator. Since the list of radiographers in the control group and the characteristics have already been obtained, exact matching can be performed. This method will help in making the groups comparable and similar in terms of confounder distribution.
We will use the formula by Lemeshow et al  to estimate the minimum sample size required in intervention studies and to test a hypothesis of proportion of 2 population problems in terms of the radiographers sample size (see ), where p 1 and p 2 were obtained from a study by Moreira [ ] to estimate the sample size of radiographers in each group and Z1–α=1.96, Z1–β=0.842, p 1 is 0.63, p 2 is 0.88, and p–=0.755. displays the sample size estimation of the secondary outcome variables.
According to Sullivan , the attrition rate (dropout) can be calculated by the formula (desired sample size)/(percent retained). Hence, for this study, a sample size of 56 for each group was targeted.
In terms of the repetition rate of routine digital chest radiographs sample size, the same formula was used, where p 1 and p 1 were obtained from the study of Zhang and Chu  to estimate the sample size of routine digital chest radiography in each group and Z1–α=1.96, Z1–β=0.842, p 1 is 0.0584, p 2 is 0.087, and p–=0.0728. displays the sample size estimation of the primary outcome variable.
According to Sullivan , the attrition rate that may occur due to any loss of chest images can be calculated by the formula (desired sample size)/(percent retained). The sample size of routine digital chest radiographs is 1618 for each group, and this number is expected to be reached within 2 weeks. Two weeks’ time is similar to that used in the study conducted by Ahmed and Suliman [ ].
A questionnaire developed by the researcher based on the information motivation behavioral skills model is one of the instruments that will be used to evaluate the level of radiographer knowledge about imaging, as well as the motivation and skills. It consists of close-ended questions and is divided into 2 sections: demographic data of the radiographer and domain of the radiographer’s knowledge, motivation, and skills. Radiographers are expected to spend 5 minutes completing the questionnaire.
A check list was recommended and used in several studies around the world to measure the repetition rate of routine digital chest radiographs . It is an international instrument. It contains radiographer demographic data, number of radiographs performed by radiographer, number of repeated radiographs, and causes of repetition. It is completed by the researcher in order to obtain accurate results and overcome biases, using actual numbers. Therefore, its reliability does not need to be checked. Furthermore, studies conducted by Al-Malki et al [ ] and Khafaji and Hagi [ ] in Saudi Arabia used the same instrument. This means that the check list used in this study is valid.
In order to achieve accurate and precise results, the validity and readability of the questionnaire will be evaluated. Face validity will be ensured by an expert currently practicing to ensure the veracity of the meaning, wording, and sequences. Content validity will be ensured by lecturers working in the university to ensure clarity, representation, and comprehensiveness. Furthermore, factor analysis will be conducted to ensure a structural correlation between variables and factors on the instruments. Finally, reliability through the Cronbach coefficient alpha will be conducted to ensure internal consistency.
Piloted intervention will be used in this study. The intervention is in the form of printed educational material distributed to routine digital radiographers in the departments of intervention hospitals based on a specific module developed for the purpose of the study. The intervention module was developed from previous studies [- ]. The education material was developed based on the information motivation behavioral skills model. This model has 4 constructs: information, motivation, skills, and behavioral change. This model was selected because it was recommended by another researcher to study the effect of self-efficacy, attitude, and knowledge on repetition. The intervention component comprises 3 sections. The first section touches on the background of the repetition issue and the importance of producing high-quality chest images. The second section encompasses the motivation issue of repetition and dose of radiation. The third section includes important skills that should be performed by a radiographer to reduce the repetition rate of chest images. Furthermore, the educational material discusses the issue of repeated radiography and the definition, repetition rate, international standard, causes of repetition, and the burden of repeated radiography to radiographers, patients, clinicians, and the organization. Anatomical parts which should be included in chest radiography will also be included in the education material.
The primary outcome of this research is the repetition rate of routine digital chest radiographs. It is the change of the behavior based on the information motivation behavioral skills model.
The secondary outcome in this study is knowledge, motivation, and skills of radiographers. Based on the information motivation behavioral skills model, there are direct and indirect correlations between knowledge and behavioral change. There are also direct and indirect correlations between motivation and behavior change. Meanwhile, behavioral skills have a direct correlation with behavioral change.
Data analysis in this study is divided into 2 parts: descriptive and inferential. Descriptive data will be calculated in order to compute the central tendency and dispersion to add valuable statistical information to the study. Inferential data analysis will be used to meet a specific objective. Chi-square, 1-way multivariate analysis of variance, 2-way repeated measure multivariate analysis of variance, and multivariate analysis of covariance are the statistical methods that will be used to test the hypothesis. Cochran Q test will be also employed to assess the difference in proportion. The level of significance will be set at α=0.05, and all testing of hypotheses will be conducted using 2-sided tailed hypotheses. The statistical program used is SPSS version 22 (IBM Corp).
Ethics Approval and Registration
Approval from the ethics committee of the faculty of Medicine and Health Sciences of the University Putra Malaysia was obtained (reference number EXP16 P160). Approval to conduct the study was also obtained from the Ministry of Health (reference number H-02-J002). Approvals from Makkah health affairs and the hospitals that are under study were also obtained. In addition, radiographers who will be involved in the study will sign a consent form.
The researchers expect that the repetition rate and the radiographer knowledge, motivation, and skills in both the control and intervention groups before intervention are statistically not significant (P>.05). It is expected that a high repetition rate with a low level of knowledge, motivation, and skills in both groups will be found in the baseline data. We predict that after implementation of the educational material in intervention hospitals, the knowledge, motivation, and skills of radiographers will increase and the repetition rate will reduce (P<.05), but we do not expect the repetition rate, knowledge, motivation, and skills to change in the control group (P>.05). It is expected that the intervention will be effective to change the behavior and reduce the repetition rate of routine digital chest radiography (P<.05). The results are expected to be published in 2018.
This quasi-experimental time series with control group aims to investigate the effect of printed educational material on radiographer knowledge, motivation, and skills and the radiography repetition rate.
Educating radiographers helps reduce the dose of radiation exposure on patients, decreases waiting time, and increases patient satisfaction. A reduction in the repetition rate decreases the dose of radiation and reduces the workload. This intervention is significant to the organization as it reduces the burden of equipment consumption and cost as well as assists the organization in achieving its vision and goals. In addition, implementing an educational program that focuses on reducing the repetition rate of radiographs has been highly recommended [, , ].
To our knowledge, this is the first study that combines 4 outcome variables—knowledge, motivation, skills, and the repetition rate of routine digital chest radiography—and aims to investigate the effect of using printed educational material on the repetition rate of routine digital chest radiography. Furthermore and based on our knowledge, this is the first study that analyzes the repetition rate among radiographers.
The intervention module will be made available in both English and Arabic languages, and participants can choose their preferred language to complete the sessions. The intervention program was designed to be as brief as possible to increase readability. The printed educational material was selected because of the difficulty of assembling radiographers from different cities in one place at one time. There is a need to overcome the issue of bias to increase the credibility of the study. The quasi-experimental design is one of the strongest designs for this particular research.
There are some limitations to this study beginning with the research design. The threat of internal validity mostly reduces the inference of causality due to the lack of randomization. However, the researcher will make the groups comparable and similar by using the exact matching technique. Another limitation is that the result cannot be generalized to all of the hospitals in the Makkah Region due to differences in hospital equipment, which may be conventional, computed, or direct forms of radiography. These modalities are totally different than the others, but the result can be generalized on tertiary hospitals in the region.
The printed education intervention could serve as a new modality to manage the critical event of repetition among radiographers. The study aims to provide better recognition and management of the repetition rate of routine digital radiography through increasing knowledge, motivation, and skills. It also aims to educate and create awareness of the problem of repetition in radiography. There is a need to develop simple, brief, and effective interventions tailored to the needs of the radiology department to reduce the burden of repetition among radiographers.
To our knowledge, this study will be the first quasi-experimental time series study with a control group using a printed educational material intervention program for radiographers to investigate the repetition rate in chest radiography and radiographer knowledge, motivation, and skills. The results from this study will determine the effectiveness of the intervention in managing and decreasing the repetition rate of routine digital radiography among radiographers. If proven to be effective, the intervention can better serve the organization by assisting decision making in the radiology department to manage and reduce the burden caused by repetition.
The researchers would like to thank the research assistants, radiographers in charge, and Picture Archiving Communication System technicians in the hospitals under the study for their cooperation and interest in the study.
Conflicts of Interest
- Foos D, Sehnert W, Reiner B, Siegel E, Segal A, Waldman D. Digital radiography reject analysis: data collection methodology, results, and recommendations from an in-depth investigation at two hospitals. J Digit Imaging 2009;22(1):89-98 [FREE Full text]
- International Atomic Energy Agency. Radiation Protection in Diagnostic Radiology. 2009. URL: http://www-pub.iaea.org/MTCD/publications/PDF/Pub1381_web.pdf [accessed 2017-09-01] [WebCite Cache]
- Zewdeneh D, Teferi S, Admassie D. X-ray reject analysis in Tikur Anbessa and Bethzatha hospitals. Ethiop J Health Dev 2008;22(1):63-67 [FREE Full text]
- Bushong S. Radiologic Science for Technologists: Physics, Biology, and Protection. 10th Edition. Atlanta: Elsevier Health Sciences; 2013.
- New Jersey Department of Environmental Protection. Compliance Guidance for Radiographic Quality Control, 4th Edition. 2003. URL: http://www.nj.gov/dep/rpp/qa/qa_down/radcg.pdf [accessed 2017-09-01] [WebCite Cache]
- World Health Organization. Quality assurance in diagnostic radiology. 1982. URL: http://apps.who.int/iris/bitstream/10665/39095/1/9241541644.pdf [accessed 2017-09-01] [WebCite Cache]
- Lloyd P. Quality qssurance workbook for radiographers and radiological technologists. 2001. URL: http://apps.who.int/iris/bitstream/10665/42412/1/9241546425.pdf [accessed 2017-09-01] [WebCite Cache]
- Winston J, Jackson D, Wozniak D, Zeisler J, Farish S, Thoma P. Quality control recommendations for diagnostic radiography. Radiography 2001;3:72 [FREE Full text]
- Jones A, Heintz P, Geiser W, Goldman L, Jerjian K, Martin M, et al. Ongoing quality control in digital radiography: report of AAPM Imaging Physics Committee Task Group 151. Med Phys 2015 Nov;42(11):6658-6670. [CrossRef] [Medline]
- Rajani A, Sajjad Z, Masroor I, Parveen A, Naqvi M. Should we stop improving when benchmarks are met. Pak J Radiol 2016;18(1):18-19 [FREE Full text]
- Khafaji M, Hagi S. Direct digital radiograph: technicians role in obtaining good images. Saudi Med J 2014;35(8):879-881 [FREE Full text]
- Al-Malki M, Abulfaraj W, Bhuiyan S, Kinsara A. A study on radiographic repeat rate data of several hospitals in Jeddah. Radiat Prot Dosimetry 2003;103(4):323-330 [FREE Full text]
- Akintomide A, Egbe N, Bassey D, Eduwem D, Oyama E. An analysis of repeated examinations in conventional film-screen radiography. J Assoc Radiogr Niger 2011;25:20.
- Bassey C, Ojo O, Akpabio I. Repeat profile analysis in an x-ray department. J Radiol Prot 1991;11(3):179-183 [FREE Full text]
- Dieleman M, Harnmeijer J. Improving health worker performance: in search of promising practices. Geneva: World Health Organization; 2006. URL: http://www.who.int/hrh/resources/improving_hw_performance.pdf [accessed 2017-09-06] [WebCite Cache]
- Alsharif W, Davis M, McGee A, Rainford L. An investigation of Saudi Arabian MR radiographers' knowledge and confidence in relation to MR image-quality-related errors. Radiography 2017;23(2):e27-e33 [FREE Full text]
- Mohammed AR, Mohamed TEA, Hassan W. Knowledge and performance of radiographers towards radiation protection, Taif, Saudi Arabia. IOSR J Dent Med Sci Ver II 2015;14(3).
- Okaro A, Eze C, Ohagwu C. Impact of motivation on productivity of radiographers in two tertiary healthcare institutions in Enugu Metropolis Southeastern Nigeria. Am J Sci Res 2010 Mar;2010(8):26-32.
- Waaler D, Hofmann B. Image rejects/retakes—radiographic challenges. Radiat Prot Dosimetry 2010;139(1-3):375-379. [CrossRef] [Medline]
- Khoshinani H, Khoshinani H, Heidari S. Evaluation of related factors to repeated radiographs in radiology centers of Hamadan hospitals. J Paramed Sci 2014;5(4):45-49 [FREE Full text]
- Almalki A, Manaf R, Noor N. A systematic review on repetition rate of routine digital radiogrphy. Int J Curr Res 2017;9(2):46326-46330 [FREE Full text]
- Silva WD, Marques M, Nascimento AD, Marques M. Comparative study to determine technical failures affecting conventional chest radiography. Radiol Bras 2013;46(1):39-42 [FREE Full text]
- Hofmann B, Rosanowsky TB, Jensen C, Wah KHC. Image rejects in general direct digital radiography. Acta Radiol Open 2015 Oct;4(10) [FREE Full text] [CrossRef] [Medline]
- Moore K. Quasi-experimental evaluations: part 6 in a series on practical evaluation methods. 2008. URL: https://www.childtrends.org/wp-content/uploads/2008/01/Child_Trends-2008_01_16_Evaluation6.pdf [accessed 2017-09-06] [WebCite Cache]
- Scope of services of radiation protection department. Saudi Arabia: Ministry of Health; 2016. URL: http://www.moh.gov.sa/endepts/EnvironmentalHealth/Departments/Pages/RadiationProtection.aspx [accessed 2017-09-13] [WebCite Cache]
- Central Board of Accreditation for Healthcare Institutions. Radiology standards. 2015. URL: http://www.cbahi.org/cbahifiles/standard/radiology.pdf [accessed 2017-09-06] [WebCite Cache]
- Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, et al. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc 2006;13(1):16-23 [FREE Full text] [CrossRef] [Medline]
- Greenwood E. Experimental Sociology: A Study in Method. New York: King's Crown Press; 1945.
- Stuart EA, Rubin DB. Best practices in quasi-experimental designs: matching methods for causal inference. In: Best Practices in Quantitative Social Science. Thousand Oaks: Sage Publications; 2008:155-176.
- Loman T. Matching procedures in field experiments. 2003. URL: http://capacitybuilding.net/Matching [accessed 2017-09-06] [WebCite Cache]
- Lemeshow S, Hosmer J, Klar JL. Adequacy of Sample Size in Health Studies. West Sussex: John Wiley & Sons Ltd; 1990.
- Moreira IC, Ventura S, Ramos I, Rodrigues P. Development and assessment of an e-Learning course on breast imaging for radiographers: a stratified randomized controlled trial. J Med Internet Res 2015;17(1).
- Sullivan L. Power and sample size determination.: Boston University School of Public Health URL: http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Power/BS704_Power_print.html [accessed 2017-09-06] [WebCite Cache]
- Zhang M, Chu C. Optimization of the radiological protection of patients undergoing digital radiography. J Digit Imaging 2012 Feb;25(1):196-200 [FREE Full text] [CrossRef] [Medline]
- Ahmed NA, Suliman II. Quality assurance and optimization of radiation protection of patients in x-ray radiographic examinations using the rejects film analysis. J Radiol Diagnostic Imaging 2013;1(2):4.
- Pongnapang N. Practical guidelines for radiographers to improve computed radiography image quality. Biomed Imaging Interv J 2015;1(2):12 [FREE Full text]
- European Commission. European guidelines on quality criteria for diagnostic radiographic images. URL: http://cordis.europa.eu/pub/fp5-euratom/docs/eur16260.pdf [accessed 2017-09-06] [WebCite Cache]
- Ahmed N. Good positioning for PA chest x-ray exams. Columbia: AuntMinnie.com; 2001 Oct 19. URL: http://www.auntminnie.com/index.aspx?sec=ser&sub=def&pag=dis&ItemID=51950 [accessed 2017-04-02]
- Checklist guide for quality PA chest radiographs.: Conference of Radiation Control Program Directors; 2008. URL: http://www.crcpd.org/resource/resmgr/docs/QA_Collectibles/ChecklistPA_Chest.pdf [accessed 2017-09-06] [WebCite Cache]
- Elahifar M, Dehghan MG, Ranjbar SM. Assessment of radiographic film repeats rate and its causes in radiology centers of Zahedan University of Medical Sciences. ISMJ 2014;17(3):50.
- Ozsunar Y, Cetin M, Taşkin F, Yücel A, Can S, Argin M, et al. The level of quality of radiology services in Turkey: a sampling analysis. Diagn Interv Radiol 2006 Dec;12(4):166-170 [FREE Full text] [Medline]
Edited by G Eysenbach; submitted 09.05.17; peer-reviewed by P Khalighinejad, E Krupinski; comments to author 12.07.17; revised version received 01.08.17; accepted 01.08.17; published 26.09.17
©Abdullah A. Almalki, Rosliza Abdul Manaf, Muhamad Hanafiah Juni, Hayati Kadir Shahar, Noramaliza Mohd Noor, Abdelsafi Gabbad. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 26.09.2017.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.