This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Long-distance runners are prone to injuries including Achilles tendinopathy and medial tibial stress syndrome. We have developed an Internet comprehensive self-report questionnaire examining the medical history, injury history, and running habits of adult recreational runners.
The objective of the study was to evaluate two alternative forms of test-retest reliability of a comprehensive self-report Internet questionnaire retrospectively examining the medical history, injury history, and running habits among a sample of adult recreational runners. This will contribute to the broad aims of a wider study investigating genetics and running injury.
Invitations to complete an Internet questionnaire were sent by email to a convenience pilot population (test group 1). Inclusion criteria required participants to be a recreational runner age 18 or over, who ran over 15 km per week on a consistent basis. The survey questions addressed regular running habits and any injuries (including signs, symptoms, and diagnosis) of the lower limbs that resulted in discontinuation of running for a period of 2 consecutive weeks or more, within the last 2 years. Questions also addressed general health, age, sex, height, weight, and ethnic background. Participants were then asked to repeat the survey using the Internet platform again after 10-14 days. Following analysis of test group 1, we soft-launched the survey to a larger population (test group 2), through a local running club of 900 members via email platform. The same inclusion criteria applied, however, participants were asked to complete a repeat of the survey by telephone interview after 7-10 days. Selected key questions, important to clarify inclusion or exclusion from the wider genetics study, were selected to evaluate test-retest reliability. Reliability was quantified using the kappa coefficient for categorical data.
In response to the invitation, 28 participants accessed the survey from test group 1, 23 completed the Internet survey on the first occasion, and 20 completed the Internet retest within 10-21 days. Test-retest reliability scored moderate to almost perfect (kappa=.41 to .99) for 19/19 of the key questions analyzed. Following the invitation, 122 participants accessed the survey from test group 2, 101 completed the Internet survey on the first occasion, and 50 were randomly selected and contacted by email inviting them to repeat the survey by telephone interview. There were 33 participants that consented to the telephone interview and 30 completed the questionnaire within 7-10 days. Test-retest reliability scored moderate to almost perfect for 18/19 (kappa=.41 to .99) and slight for 1/19 of the key questions analyzed.
We successfully developed a self-reported, retrospective questionnaire, delivered using Internet software, providing stable and reliable answers. We demonstrate that our survey provides a relatively quick, easy to complete, and cost effective method to collect epidemiological data from recreational runners and evaluate these participants for inclusion into a genetic study.
Several systematic reviews have examined the incidence rates and risk factors of running related injuries [
There are two types of injury that are commonly reported in association with running and exercise: bone stress injuries (including medial tibial stress syndrome and tibial stress fracture) and tendon injuries (including tendinopathy and tendon rupture) [
Published genetic studies on exercise-induced injuries primarily focus on soft tissue injuries [
Surveys should be evaluated for reliability, a critical measurement property for health-related and physical activity variables. Reliability refers to the consistency of answers obtained by the same participant when a measurement is repeated on different occasions [
Internet survey instruments are increasingly being used as a preferred method of conveniently collecting data from a wide range of participants from various locations. The reliability of survey instruments delivered on the Internet has been previously demonstrated, showing that in the instance of retrospective tobacco exposure and risk, this method of data collection is reliable [
The purpose of the study reported here is to test the test-retest reliability of an Internet survey platform from a convenience population of first responders, and pilot test participants who accepted an invitation to describe their recreational running epidemiology. This will contribute to the broad aims of the wider study, which are to recruit thousands or even tens of thousands of individuals to investigate injury epidemiology across a wide group of recreational runners in order to gain insight into how a variety of demographic factors may influence injury epidemiology; and to perform genetic assessments for the injury risk factors in a small subsample of the overall injury epidemiology study.
The questionnaire administered was designed to enable recruitment as part of the wider study entitled, “The Genetics of Exercise-Induced Injuries in Tendon and Bone.” This study has very broad aims in the range of wanting to recruit very large numbers of participants to provide injury epidemiology among recreational runners, as well as performing genetic assessments for the injury risk factors in a smaller subsample of the overall injury epidemiology study.
The platform used to deliver the survey was the Internet and commercial survey software (SurveyGizmo). The Internet survey was advertised through appropriate forums relevant to the intended population to analyze sporting injuries and training practices. Participants provided us with data on their demographics and running injuries, but the survey provides a platform for determining whether a participant met the wider genetics study’s inclusion and exclusion criteria. Participation was entirely voluntary and required consent to participate in the research processes described by checking a required “I ACCEPT” box necessary to access the questionnaire. Further information about the project was made available via a check box diverting the participant to another screen with all relevant information and contact details for further clarification if required.
On accepting conditions of participation in this project, participants were taken to the questionnaire and instructed that it would take no more than 30 minutes to complete. The software used allowed participants to exit the survey at any time and complete at a later date, allowing participants to provide their data at the time most suitable to them. A questionnaire was deemed complete when the participant had answered all required questions (customized within the software) and participants had submitted their questionnaire by checking the “SUBMIT” tab inserted at the end of the questionnaire.
The questionnaire developed here combined new items of assessment with adapted versions of the SCOFF questionnaire for examining eating disorders [
In addition, participants were asked to report on their general health, provide any knowledge of existing conditions or diseases, and any known antibiotic or prescribed drug administration. They were asked to describe their regular dietary practices including sport supplement use, as well as any body weight fluctuations. Participants were also asked to report their age, sex, height, weight, and ethnic background to collect demographic data for further epidemiology analysis.
Some of the key variables, important to clarify inclusion or exclusion from the wider genetics study previously mentioned, were then selected for test-retest reliability here (
Questionnaire items tested for reliability.
Questionnaire item | Possible responses | |
|
How many years have you been running on a regular basis? Regular is defined as at least weekly. | <1/1/2/3/4/5/6/7/8/9/10+ |
|
On average, how many kilometer per week would you run? | <15/15-20/20-30/30-40/40-50/50-60/60+ |
|
What type of terrain is the majority of your running performed on? | Bitumen/cement/hard dirt or gravel/soft dirt/grass/synthetic/treadmill |
|
In the last 2 years, have you participated in any other sports or intentional exercise on a regular basis (eg, weekly during at least one season)? | Yes/no |
|
In the last 2 years, have you had any injuries of the lower limbs, which have forced you to discontinue running for a period of 2 weeks or more? | Yes/no |
|
If yes, how many lower limb injuries have you been diagnosed with in the last 2 years? | 1/2/3/4+ |
|
Was the injury diagnosed by a professional doctor? | Yes/no |
|
Was the injury diagnosed by a professional physiotherapist? | Yes/no |
|
Was the injury an Achilles tendon injury? | Yes/no |
|
Was the injury a bone stress injury below the knee? | Yes/no |
|
Have you ever had hip, knee, or ankle surgery? | Yes/no |
|
While running do you wear orthotics? | Yes/no |
|
Have you ever smoked cigarettes? | Yes/no |
|
To your knowledge, have you ever been treated using quinolone antibiotics (eg, ciprofloxacin, norfloxacin)? | Yes/no/unsure |
|
To your knowledge, have you ever been treated using corticosteroid medication (eg, cortisone injection, prednisone tablets, prednisolone tablets, flixotide inhaler, pulmicort inhaler, QVAR inhaler, seretide accuhaler, symbicort turbuhaler, steroid cream)? | Yes/no/unsure |
|
To your knowledge, have you ever been treated using calcium tablets as prescribed by a medical doctor or taken it without a prescription? | Yes/no/unsure |
|
To your knowledge, have you ever been treated using vitamin D supplementation as prescribed by a medical doctor or taken it without prescription? | Yes/no/unsure |
|
Do you follow a gluten-free diet? | Yes/no |
|
Do you have any food allergies or avoidances? | Yes/no |
Initial recruitment involved inviting a convenience sample of participants into a pilot group. Invitation to complete “The Genetics of Exercise-Induced Injuries in Tendon and Bone Study” survey was via email platform including an outline of the purpose of the study and providing potential participants with a link to the Internet survey. Participants were recruited from within our association or among associated colleagues. Inclusion criteria for participants included being a recreational runner aged 18 or over, who ran over 15 km/week on a consistent basis.
In response to the invitation, 28 participants accessed and consented to the conditions of the study as logged by SurveyGizmo. Of these participants, 23 completed the first survey on the first occasion, and after 10-14 days were invited by email to repeat the survey for a second time, 20 completed within 10-21 days after the date of the first completion. This group was deemed test group 1 and were subjected to the Internet-Internet test-retest method.
After data from test group 1 suggested that reliability was more than moderate for 19/19 of the variables analyzed, we then sought to maximize the variability in response and strength in reliability of our questionnaire by soft launching the survey to a wider population using a different test-retest method. We invited a larger convenience sample of participants by promoting the wider genetics study through a local running club of 900 members via email platform as described earlier for group 1. The same inclusion criteria were applied and participant information was provided at the commencement of the survey, and checking the box “I ACCEPT” resulted in access to the Internet survey and provided informed consent.
In response to the invitation, 122 participants accessed and consented to the conditions of the study as logged by SurveyGizmo. Of these participants, 101 completed the first survey on the Internet on the first occasion. Of these participants, 50 were randomly selected and contacted by email platform inviting them to repeat the survey by telephone interview. In response to this invitation, 33 participants agreed to the repeat by telephone interview within 7-10 days of Internet completion and 30 answered the call within 14 days and completed the questionnaire on the second occasion. This group was deemed test group 2 and participants were subjected to the Internet-phone test-retest method.
Descriptive statistics were used to describe the cohort characteristics of participants for each group, and reported as a percentage (%) of total participant number within the group illustrated. Some of the key variables important to clarify inclusion or exclusion from the wider genetics study previously mentioned were then selected for test-retest reliability and scored using the kappa statistic, with asymptotic standard error [
This self-report questionnaire worked toward the development of a standardized instrument via which participants not only provide us with data on their demographics and running injuries, but also provide us with a platform for determining whether interested participants meet the wider genetics study’s inclusion criteria. This study analyzed the test-retest reliability of key inclusion/exclusion criteria for the wider genetics study, collected with the developed survey involving the 2 groups. The choice of different methods for test-retest analysis of each group is discussed in the “Methods” section. The results of both approaches scored using the kappa statistic, with asymptotic standard error, the valid number of responses, and number of response options, are highlighted here.
There were 20 participants that fulfilled group 1 criteria by completing the survey on the Internet then again on the Internet within 21 days after the first completion. Participants were between 18-68 years of age, with an equal distribution of males (50%, n=10) and females (50%, n=10). As much as 45% (9/20) of participants recorded that they had been running for 6 years or less and the remaining 55% (11/20), 8 years or more. Their running habits were variable, with 50% (10/20) predominantly running 15-30 km/week, 25% (5/20) running 30-40 km/week, and 25% (5/20) running 40 km or more per week (
Cohort characteristics.
Characteristics |
|
Total participants | Test group 1 |
Test group 2 |
|||
|
|||||||
|
|
N=50 | N=20 | N=30 | |||
|
|
n | % | n | % | n | % |
|
|
|
|||||
|
Male | 26 | 52 | 10 | 50 | 16 | 53 |
|
Female | 24 | 48 | 10 | 50 | 14 | 47 |
|
|
|
|||||
|
18-25 | 8 | 16 | 4 | 20 | 4 | 13 |
|
26-35 | 19 | 38 | 8 | 40 | 11 | 37 |
|
36-40 | 6 | 12 | 4 | 20 | 2 | 7 |
|
41+ | 17 | 34 | 4 | 20 | 13 | 43 |
|
|
|
|||||
|
50-60 | 14 | 28 | 6 | 30 | 8 | 27 |
|
61-75 | 24 | 48 | 9 | 45 | 15 | 50 |
|
≥76 | 12 | 24 | 5 | 25 | 7 | 23 |
|
|
|
|||||
|
150-160 | 2 | 4 | 0 | 0 | 2 | 7 |
|
161-170 | 18 | 36 | 9 | 45 | 9 | 30 |
|
171-180 | 16 | 32 | 5 | 25 | 11 | 37 |
|
≥181 | 14 | 28 | 6 | 30 | 8 | 27 |
|
|
|
|||||
|
≤6 | 21 | 42 | 9 | 45 | 12 | 40 |
|
≥8 | 29 | 58 | 11 | 55 | 18 | 60 |
|
|
|
|||||
|
15-30 | 23 | 46 | 10 | 50 | 13 | 43 |
|
31-40 | 14 | 28 | 5 | 25 | 9 | 30 |
|
≥ 41 | 13 | 26 | 5 | 25 | 8 | 27 |
aAs the participants were required to describe the average distance run per week and years of running experience within certain categories, for example, 15-20 km/week, 10+ years, no means or SD could be calculated for these variables.
There were 30 participants that fulfilled group 2 criteria by completing the survey on the Internet then again by phone interview within 14 days after Internet completion. Participants were between 18 and 67 years of age, with an almost equal distribution of males (53%, n=16) and females (47%, n=14). There were 40% (12/30) of participants that had been running for 6 years or less and the remaining 60% (18/30) 10 years or more. Their running habits were similar to test group 1 with 43% (13/30) predominantly running 15-30 km/week, 30% (9/30) 30-40 km/week, and 27% (8/30) 40 km or more per week (
Reliability was almost perfect for the type of terrain run on (kappa=.863), injuries of the lower limbs reported in the last 2 years (kappa>.99), and the number of these lower limb injuries diagnosed in the same time frame (kappa>.99). Reliability was also almost perfect for the injury being diagnosed by a doctor (kappa=.820), the injury being a bone stress injury (kappa>.99), reporting any hip/knee/ankle surgery (kappa=.857), and the use of orthotics while running (kappa>.99). Reliability was substantial for the number of years running on a regular basis (kappa=.791), average kilometer run per week (kappa=.684), other sports or intentional exercise participated in regularly in the last 2 years (kappa=.773), and reporting the injury as an Achilles tendon injury (kappa=.625). Reliability was moderate for being diagnosed by a physiotherapist (kappa=.467;
Test group 1 (Internet-Internet), test-retest reliability for running habits, injuries, and general health.
Test-retest reliability | n | Number of response options | Kappa (A-symp SE) | |
|
|
|
||
|
How many years have you been running on a regular basis? Regular is defined as at least weekly. | 20 | 11 | .791 (.100) |
|
On average, how many kilometer per week would you run? | 20 | 7 | .684 (.119) |
|
What type of terrain is the majority of your running performed on? | 20 | 7 | .863 (.088) |
|
In the last 2 years, have you participated in any other sports or intentional exercise on a regular basis (eg, weekly during at least one season)? | 20 | 2 | .773 (.216) |
|
In the last 2 years, have you had any injuries of the lower limbs, which have forced you to discontinue running for a period of 2 weeks or more? | 20 | 2 | >.99 |
|
If yes, how many lower limb injuries have you been diagnosed with in the last 2 years? | 12a | 4 | >.99 |
|
Was the injury diagnosed by a professional doctor? | 11b | 2 | .820 (.169) |
|
Was the injury diagnosed by a professional physiotherapist? | 8c | 2 | .467 (.323) |
|
Was the injury an Achilles tendon injury? | 12a | 2 | .625 (.333) |
|
Was the injury a bone stress injury below the knee? | 12a | 2 | >.99 |
|
Have you ever had hip, knee, or ankle surgery? | 20 | 2 | .857 (.138) |
|
While running do you wear orthotics? | 20 | 2 | >.99 |
|
|
|
||
|
Have you ever smoked cigarettes? | 19d | 2 | >.99 |
|
To your knowledge, have you ever been treated using quinolone antibiotics (eg, ciprofloxacin, norfloxacin)? |
|
3 | .468 (.174) |
|
To your knowledge, have you ever been treated using corticosteroid medication? |
|
3 | .492 (.165) |
|
To your knowledge, have you ever been treated using calcium tablets as prescribed by a medical doctor or taken it without a prescription? | 20 | 3 | .886 (.110) |
|
To your knowledge, have you ever been treated using vitamin D supplementation as prescribed by a medical doctor or taken it without prescription? | 20 | 3 | >.99 |
|
Do you follow a gluten-free diet? | 20 | 2 | >.99 |
|
Do you have any food allergies or avoidances? | 20 | 2 | >.99 |
an=20; valid=12; excluded=8
bn=20; valid=11; excluded=9
cn=20; valid=8; excluded=12
dn=20; valid=19; excluded=1
Reliability was almost perfect for the majority of all general health questions reported including having ever smoked cigarettes (kappa>.99), any knowledge of being treated with calcium tablets (kappa=.886), any knowledge of being treated with vitamin D supplementation (kappa>.99), reporting on whether participants followed a gluten-free diet (kappa>.99), or stating any food allergies or avoidances (kappa>.99). Reliability for any knowledge of being treated with corticosteroid medication (kappa=.492) or quinolone (kappa=.468) was moderate (
The reliability of running habit and running injury key variables in test group 1 was more variable than that of test group 1 (
Test group 2 (Internet-phone), test-retest reliability for running habits, injuries, and general health.
Test-retest reliability | n | Number of response options | Kappa (A-symp SE) | |
|
|
|
||
|
How many years have you been running on a regular basis? Regular is defined as at least weekly. | 30 | 11 | .639 (.102) |
|
On average, how many kilometer per week would you run? | 30 | 7 | .540 (.108) |
|
What type of terrain is the majority of your running performed on? | 30 | 7 | .469 (.129) |
|
In the last 2 years, have you participated in any other sports or intentional exercise on a regular basis (eg, weekly during at least one season)? | 29a | 2 | .473 (.306) |
|
In the last 2 years, have you had any injuries of the lower limbs, which have forced you to discontinue running for a period of 2 weeks or more? | 30 | 2 | .930 (.069) |
|
If yes, how many lower limb injuries have you been diagnosed with in the last 2 years? | 10b | 4 | .697 (.187) |
|
Was the injury diagnosed by a professional doctor? | 9c | 2 | .727 (.247) |
|
Was the injury diagnosed by a professional physiotherapist? | 10b | 2 | .615 (.337) |
|
Was the injury an Achilles tendon injury? | 11d | 2 | .814 (.175) |
|
Was the injury a bone stress injury below the knee? | 6e | 2 | .571 (.353) |
|
Have you ever had hip, knee, or ankle surgery? | 30 | 2 | >.99 |
|
While running do you wear orthotics? | 30 | 2 | >.99 |
|
|
|
||
|
Have you ever smoked cigarettes? | 30 | 2 | .839 (.157) |
|
To your knowledge, have you ever been treated using quinolone antibiotics (eg, ciprofloxacin, norfloxacin)? |
|
3 | .150 (.132) |
|
To your knowledge, have you ever been treated using corticosteroid medication? |
|
3 | .583 (.124) |
|
To your knowledge, have you ever been treated using calcium tablets as prescribed by a medical doctor or taken it without a prescription? | 30 | 3 | .720 (.184) |
|
To your knowledge, have you ever been treated using vitamin D supplementation as prescribed by a medical doctor or taken it without prescription? | 30 | 3 | .672 (.170) |
|
Do you follow a gluten-free diet? | 30 | 2 | .783 (.209) |
|
Do you have any food allergies or avoidances? | 29a | 2 | .922 (.077) |
an=30; valid=29; excluded=1
bn=30; valid=10; excluded=20
cn=30; valid=9; excluded=21
dn=30; valid=11; excluded=19
en=30; valid=6; excluded=24
fn=30; valid=27; excluded=3
Overall, reliability was lower for general health questions reported in test group 2 in comparison to test group 1 (
The wider study, for which this survey will provide epidemiology information and a participant cohort to recruit from, will be the first, to our knowledge, to examine the genetic predisposition of tendon and bone injury in adult recreational runners. Recruitment for the larger study will involve the use of this survey, tested here initially for test-retest reliability. The results indicate that the self-reported questionnaire, delivered using the Internet commercial software, provided stable and reliable answers for many of the most important measures required for recruiting to the larger study on tendon and bone injury.
The Internet delivery of a retrospective injury study is a convenient method for remote collection of data from participants and has been used successfully in the analysis of sports-induced injuries in surfing [
Poor reliability of the question addressing treatment with quinolone may suggest poor understanding of or unfamiliarity with “quinolone” upon first exposure to the question. It is known that test-retest reliability can be influenced by many factors. Interpretation of a question, such as familiarity of content or ambiguity, as well as memory can cause random answers [
Respondent fatigue is a well-known phenomenon affecting survey participants [
Results when addressing the 2 different test groups should be considered in the context of several limitations. Our study contained a different number of participants in each group with a variable distribution of age. This limitation resulted from the sequential manner of recruitment of the two groups, inviting participation from within our networks for test group 1, then soft launching to recruit a larger group from the running club for test group 2.
There was a slightly higher test-retest reliability for test group 1, in which the survey was repeated using the same Internet platform. It is possible that this was due to learning effects, or that the retest was administered in exactly the same format.
The slightly lower test-retest reliability for group 2, in which the survey was administered by the Internet platform and then followed up by a telephone call, may be related to the wording of the question over the telephone by the interviewer compared to the participants reading the computer screen. During the telephone assessment, it is also possible that participants felt more uncomfortable answering sensitive questions or got confused in regards to what constitutes a response for each question asked. Our findings indicate that using 2 test-retest protocols resulted in participants providing the same information with high reliability for the survey administered in this study. Maintaining consistency of the protocol, however, can improve test-retest reliability.
We successfully developed an Internet survey to meet the recruitment needs of the larger genetic injury study. We tested its reliability using 2 different test-retest methods and used a participant sample targeted to our needs (adult recreational runners) in aid of focusing our findings to support our larger study. We had equal gender participation and a broad range of running habits and participants with reportable Achilles and bone stress injuries. Despite the aforementioned limitations, the results of this reliability study demonstrate that the Internet survey developed does provide a relatively quick, easy to complete, and cost-effective method to collect epidemiological data from recreational runners and evaluate these participants for inclusion into a genetic study.
Survey: The genetics of exercise-induced injuries in tendon and bone.
These studies were supported by funding from the Collaborative Research Network for Advancing Exercise and Sports Science (CRN-AESS; RD, NV, JK, and DCH). The authors thank the CRN-AESS Scientific Committee members who contributed to this work: Professor Nuala Byrne (Bond University), Professor Matthew A Brown (University of Queensland, Diamantina Institute) and Professor Maria A. Fiatarone Singh (University of Sydney).
None declared.