This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Translating Emergency Knowledge for Kids was established to bridge the research-practice gap in pediatric emergency care by bringing the best evidence to Canadian general emergency departments (EDs). The first step in this process was to conduct a national needs assessment to determine the information needs and preferences of health professionals and parents in this clinical setting.
To describe the development and implementation of two electronic surveys, and determine the feasibility of collecting electronic survey data on iPads with in-person data collectors in a busy clinical environment.
Two descriptive surveys were conducted in 32 general EDs. Specific factors were addressed in four survey development and implementation stages: survey design, survey delivery, survey completion, and survey return. Feasibility of the data collection approach was determined by evaluating participation rates, completion rates, average survey time to completion, and usability of the platform. Usability was assessed with the in-person data collectors on five key variables: interactivity, portability, innovativeness, security, and proficiency.
Health professional participation rates (1561/2575, 60.62%) and completion rates (1471/1561, 94.23%) were strong. Parental participation rates (974/1099, 88.63%) and completion rates (897/974, 92.09%) were excellent. Mean time to survey completion was 28.08 minutes for health professionals and 43.23 minutes for parents. Data collectors rated the platform “positively” to “very positively” on all five usability variables.
A number of design and implementation considerations were explored and integrated into this mixed-mode survey data collection approach. Feasibility was demonstrated by the robust survey participation and completion rates, reasonable survey completion times, and very positive usability evaluation results.
The Translating Emergency Knowledge for Kids (TREKK) project aims to bridge the research-to-practice gap in pediatric emergency medicine, and reduce variable emergency care, by ensuring that practitioners in general emergency departments (EDs) have access to and apply the latest research evidence in their pediatric practice [
Despite a wealth of research, much debate remains regarding the superiority of electronic and/or mailed paper surveys [
Given the complexity of the ED environment (ie, fast-paced, high volume, high acuity), traditional paper-based surveys were too cumbersome and resource-intensive for this study. An electronic survey was determined to be the most appropriate method to meet study timelines, due to ease of implementation across a large geographic area, and decreased administrative costs [
Survey questions were developed using relevant research literature [
In addition to survey content, six features affecting response rate of web-based surveys were considered: (1) general format, (2) length, (3) disclosure of survey progress, (4) visual presentation, (5) interactivity, and (6) question/response format [
Consistent hardware was used at each site to streamline training and mitigate technological issues in survey delivery [
External validity has been identified as a methodological issue of concern in survey research [
The electronic survey platform incorporated synchronous and asynchronous data collection capability, meaning that data could be collected online and automatically uploaded to a secure server when a wireless connection was present, and data could also be collected offline, safely stored on the device, and later uploaded to a secure server once a wireless connection was available. This feature was particularly important in the ED setting, as many hospitals do not provide wireless internet and 3G/4G connectivity is limited or non-existent in rural and remote regions. This approach also addressed previously identified security concerns with cloud-based data storage [
A usability evaluation was conducted with data collectors using an anonymous 20-question electronic survey. The content of the survey was theoretically informed on the basis of a small-scale review of previous studies employing electronic platforms for data collection and the National Institutes of Health’s Usability Guidelines [
The recruitment rate for health care professionals was 68.66% (1768/2575) and participation rate was 60.62% (1561/2575), and among participants the survey completion rate was 94.23% (1471/1561). Among parents the recruitment rate could not be determined because the eligible population was dependent on people coming into the general EDs; however, the participation rate was 88.63% (974/1099) and among participants the survey completion rate was 92.09% (897/974). See
Survey recruitment, participation & completion.
Of the 1471 health care professional surveys included in the final analysis, 79 surveys did not have a time stamp (ie, participants did not press
Of the 897 parental surveys included in the final analysis, 25 did not have a time stamp and were not included in the calculation for average survey completion time. Of the 872 surveys with a time stamp, the mean time to survey completion was 43.23 minutes (standard deviation = 691.25 minutes). This estimate includes participants that left the survey open and returned to complete it at a later time (eg, 4 surveys were open for more than 1000 minutes).
Eight data collectors were approached to participate in the usability evaluation, six of whom agreed to participate and complete the survey (75% participation and completion rate). On a five-point scale, responses were largely
Data collectors' (n=6) ratings on 5 variables to evaluate the data collection platform.
Our findings make an important contribution to the web-based survey literature by addressing calls for research to examine web-based survey response and completion issues [
Utilizing data collectors to approach potential survey participants, and explain the study, eliminated the need for email or web-based invitations and completion reminders, and significantly enhanced survey participation and completion rates. We suggest that the addition of in-person data collectors offered the benefits of personal connection, and caused potential survey participants to make an active decision to participate in the study. In-person data collectors were also able to engage with potential participants and answer any questions the potential participants had about the study or the technology. Deploying a web-based survey without a mixed-mode approach allows potential participants to easily ignore electronic invitations to participate in survey research. With a mixed-mode approach, we were able to capitalize on the many benefits of web-based surveys, including improved data quality and the ability to immediately begin data analysis, while simultaneously mitigating previously reported downfalls of web-based surveys, including lower response rates and lower completion rates.
Vicente and Reis itemized six areas to consider when designing and implementing web-based surveys [
This study provides strong evidence for the feasibility of a mixed-mode approach to survey data collection using iPads and in-person data collectors, based on strong response rates, reasonable survey completion times, and very positive usability evaluation results. This study also details survey development and implementation considerations that will be useful to survey researchers working with a variety of populations. Great potential exists for utilizing a mixed-mode approach for future survey research in clinical settings.
TREKK Needs Assessment Healthcare Professional Survey.
TREKK Needs Assessment Parent Survey.
Electronic survey question design.
TREKK Usability Survey with In-person Data Collectors.
emergency department
Translating Emergency Knowledge for Kids
TREKK is funded by the Networks of Centers of Excellence – Knowledge Mobilization initiative, and supported (either financially or through in-kind contributions) by the following agencies and institutions: Pediatric Emergency Research Canada, Children’s Hospital Research Institute of Manitoba, University of Manitoba, Research Manitoba, ManitobaCPD.com, Cochrane Child Health Field, Alberta Children’s Hospital Research Institute, Children’s Hospital of Eastern Ontario, Ontario ED Local Health Integration Network Leads, Alberta Health Services, and Women and Children’s Health Research Institute. SDS is a Canada Research Chair (tier II) for knowledge translation in child health and a Population Health Investigator with Alberta Innovates – Health Solutions (formerly Alberta Heritage Foundation for Medical Research). LA is supported by Alberta Innovates – Health Solutions Graduate Studentship and Women & Children’s Health Research Institute Graduate Studentship.
We would like to acknowledge and thank the TREKK Team, including the Board of Directors, Prioritization Committee, Nodal Leaders, and Pediatric Emergency Research Canada Representatives, Coordinators, and Administrative staff for their dedication to this project.
TPK and SDS obtained research funding. SDS conceptualized and led the research study. SDS, LA and LMG designed the electronic surveys. LA conducted data collection and data analysis for the
None declared.