Background: A current limitation in the care of patients with severe lower extremity traumatic injuries is the lack of a rigorously developed patient-reported outcome (PRO) instrument specific to lower extremity trauma patients.
Objective: This mixed methods protocol aims to describe phases I and II of the development of a PRO instrument for lower extremity trauma patients, following international PRO development guidelines.
Methods: The phase I study follows an interpretive description approach. Development of the PRO instrument begins with identifying the concepts that are important to patients, after which a preliminary conceptual framework is devised from a systematic literature review and used to generate an interview guide. Patients aged 18 years or above with limb-threatening lower extremity traumatic injuries resulting in reconstruction, amputation, or amputation after failed reconstruction will be recruited. The subjects will participate in semistructured, in-depth qualitative interviews to identify all important concepts of interest. The qualitative interview data will be coded with top-level domains, themes, and subthemes. The codes will then be utilized to refine the conceptual framework and generate preliminary items and a set of scales. The preliminary scales will be further refined via a process of conducting cognitive debriefing interviews with lower extremity trauma patients and soliciting expert opinions. Phase III will include a large-scale field test, using Rasch measurement theory to analyze the psychometric properties of the instrument; shortening and finalizing the scales; and determining the reliability, validity, and responsiveness of the instrument.
Results: Phases I and II of this study have been funded. Phase I of this study has been completed, and phase II began in January 2019 and is expected to be completed in November 2019. Phase III will begin following the completion of phase II.
Conclusions: This protocol describes the initial phases of development of a novel PRO instrument for use in lower extremity trauma patients.
International Registered Report Identifier (IRRID): DERR1-10.2196/14397
Each year there are thousands of civilian and military limb-threatening lower extremity traumatic injuries [, ]. Treatment for this is lengthy, with a time and resource burden both for patients and the health system. The outcomes include limb salvage, amputation, or delayed amputation after a failed attempt at reconstruction. It has not yet been established if attempted limb salvage or amputation results in better function, satisfaction, or quality of life for the patient [ , ]. To adequately answer these questions and other important research questions in the management of these patients, a well-defined, valid, reliable, and responsive patient-reported outcome (PRO) instrument is needed. PRO instruments are rating scales that measure concepts of interest (COI) relevant to patients such as the symptoms, appearance, function, and quality of life by asking the patients directly, without interpretation by a clinician or researcher [ ].
Numerous PRO instruments have been used in the past to study this patient population, such as the Sickness Impact Profile [, ] and the Short Musculoskeletal Function Assessment Questionnaire [ ]; however, all were developed either for the general population or specific to alternate diseases [ , ]. Despite the previous use of these instruments in lower extremity trauma patients, none were developed specifically for this cohort and, as a result, they lack content specific to this patient population. Therefore, although previous research has been rigorous and well-designed to identify differences in outcomes between the treatment cohorts, the interpretation of these results is limited by inadequate outcome measures used, as they were not designed to address the particular concerns of lower extremity trauma patients.
To ensure that all important COI of these patients are measured and to answer important clinical questions regarding limb reconstruction and amputation, a well-defined PRO instrument designed specifically for lower extremity trauma patients is needed. This instrument should cover common and unique concerns of amputees and limb-reconstruction patients.
The following protocol is the result of an international collaboration whose primary focus is the development and validation of a new PRO instrument, developed specifically for patients after lower extremity trauma, to measure all important COI from the patient’s perspective, applicable to both amputation and reconstruction patients; see.
The aim of this study is to develop a PRO instrument, with a set of independently functioning scales, that captures the multidimensional impact of lower extremity traumatic injuries on patients, useful for clinical practice, quality improvement, and research. Our team follows rigorous, state-of-the-art psychometric methodology for PRO instrument development [, - ], utilizing a multiphase mixed-methods approach [ ]. Our approach involves 3 main phases of instrument development, which are completed in an iterative and interactive manner. Briefly, the first phase identifies what the instrument should measure through the development of a preliminary conceptual framework from a literature review, followed by qualitative interviews, which are utilized to finalize the conceptual framework and develop the preliminary set of scales. The second phase refines the PRO instrument via cognitive debriefing interviews and solicitation of expert opinion. Finally, the third phase involves a large-scale field test and psychometric evaluation and refinement of items and scales. Adherence to the international guidelines for PRO instrument development ensures that the PRO instrument is scientifically sound (reliable, valid, and responsive). The first 2 phases of instrument development are presented here.
The conceptual framework describes the COI within the patient group, helping to organize and frame the domains that are of importance to patients, and therefore should be under consideration for inclusion in the PRO instrument . The preliminary conceptual framework was developed from the COI identified in our systematic review of the literature, in addition to expert opinion; see [ ]. The preliminary conceptual framework is then further refined through the qualitative portion of the study. The final conceptual framework maps all COI identified through the qualitative interviews and is used to define the domains within the preliminary set of scales.
Preliminary conceptual framework domains.
- Physical function
- Prostheses and orthotics
- Psychosocial well-being
- Satisfaction with experience
- Satisfaction with outcome
- Sexual well-being
This is a qualitative study designed to identify the concepts that are most important to lower extremity trauma patients via semistructured qualitative interviews. An interpretive description approach is utilized, with theoretical knowledge from the systematic review and clinical knowledge from the clinical team members, forming a basis for the identification of the COI for this patient group .
Inclusion criteria include patients aged 18 years or above, who suffered a lower extremity traumatic injury, distal to the midfemur, resulting in the need for reconstruction or amputation. Patients who do not speak English are excluded. A member of the clinical team recruits patients either face-to-face in the hospital or clinic or over the telephone. Interested participants who are unable to travel to the interview are offered phone interviews.
Purposeful sampling maximizes participant variability to ensure a diverse group of participants, reflected in demographic variables (age, gender, and race), injury etiology, time since injury, and injury outcome (reconstruction, amputation, or amputation after failed reconstruction). Patients having undergone successful limb salvage, early amputation, and late amputation after failed attempts at reconstruction with varying amounts of time from initial injury will be recruited. Recruitment continues until content saturation is reached, that is, no new concepts are being identified in the interviews .
Consent is provided before participation. Demographic characteristics include age, gender, and race and clinical characteristics include medical history, drug/tobacco use, injury etiology, injury characteristics, date of injury, and surgical history, in addition to postinjury findings such as return-to-work status and chronic pain medication use. Trained qualitative researchers perform interviews using an interview guide developed to reflect the preliminary conceptual framework to ensure that all previously identified COI are addressed. The interview guide is a working document that is updated throughout the study as new COI are identified; see.
Interviews are recorded and transcribed verbatim. Each interview is coded line by line to identify COI, categorized top-down into domains, followed by themes and subthemes. Interviews are coded as a team by a clinical expert (LRM) and qualitative methods expert (AK) to ensure consistency, reliability, and validity in coding. Coding questions requiring additional clinical expertise were discussed with additional clinical experts (MJG and STH) as needed. Constant comparison is used to ensure consistency in coding, allowing for continuous refinement of previously coded data . It must be noted that more than 1 item can be generated from a code if the participant uses different words/phrases to describe their thoughts or experiences. However, if someone uses the same word/phrase repeatedly, we develop 1 item for the concept as we are interested in capturing a unique concept rather than capturing the number of times a subject repeats himself or herself. The domains are used to generate the final conceptual framework, organizing all identified COI.
The codes generated in the data analysis process represent an exhaustive list of all relevant COI to lower extremity trauma patients, organized through the lens of the conceptual framework. Each domain within the conceptual framework that is relevant to clinical care or research is utilized to develop a preliminary scale. Coded text with major/minor themes is moved into an Excel spreadsheet (Microsoft Excel 2016) along with the participant’s age, gender, and surgical outcome (reconstruction, amputation, or reconstruction to amputation). Clinical data are utilized to identify the codes that are relevant across patient groups versus the codes that are relevant to only a specific patient group, such as concerns specific to patients with prostheses. Each code is examined in turn to generate a preliminary item pool for scale development. Within each scale, the items are developed from the themes and subthemes, using as many of the participants’ own words as possible. For each scale, instructions and response options are added. Our goal is to keep the scales simple and in line with the published guidelines . Khadka et al demonstrated that rating scales or questionnaires with a complicated question format, with a large number of response categories or with unlabeled categories tended to be dysfunctional and recommended that PRO instruments should have simple question formats and only a few (4-5 at most) response options and that all response options should be labeled [ ].
Refining the Preliminary Scales
The preliminary PRO instrument scales are refined via patient interviews and solicitation of expert opinion in phase II of instrument development, ensuring maximum content clarity and comprehensiveness .
Cognitive Debriefing Interviews
Semistructured interviews are conducted with lower extremity trauma patients to assess for content and clarity of wording, ensuring the instrument is comprehensive, relevant, and comprehensible. Participants meeting the same inclusion/exclusion criteria stated previously are recruited in a similar manner. Recruitment starts with the initial qualitative interview of participants and then, if needed, is expanded to additional patients meeting the above inclusion criteria, who did not participate in the first round of interviews. Participants are interviewed one-on-one by a trained qualitative interviewer, with the participant going through the scale line by line using the think-aloud technique, explaining out loud how the participant interprets each item . Participants are also asked to comment on the instructions, probed for missing content, and asked to identify problematic items and scales. Consequently, the instrument is prepared for the large-scale field test, combining revisions from the opinions of expert clinicians.
Expert Clinical Input
In concurrence with soliciting opinions from patients, the opinions of expert clinicians are solicited. This is to ensure that the PRO instrument captures all key concepts relevant to clinicians or researchers. Expert advice is sought from clinicians, including plastic and orthopedic surgeons, in addition to the physical and occupational therapists, nursing, prosthetists, and limb-loss experts. Experts provide written feedback on the instrument via a written survey.
Phases I and II of this study have been funded. Phase I of this study has been completed, and phase II began in January 2019 and is expected to be completed in November 2019. Phase III will begin following the completion of phase II.
Pilot Field Test
A pilot field test is carried out before the large-scale international field test. The goal of the pilot field test is 2-fold: first, to identify and address any administrative or logistical difficulties, and second, to perform a preliminary Rasch analysis to identify and address any psychometric issues with LIMB-Q scale performance. Inclusion criteria are similar to those described above. Participants complete the LIMB-Q in addition to providing demographic, injury, and treatment information. Rasch measurement theory (RMT) is used to drive the data analysis of the scales and is described in detail in the following section.
International Field Test
A large-scale international field test is the final stage in LIMB-Q development, with the goal to finalize the items and scales of the LIMB-Q, determining scale validity, reliability, and responsiveness. Inclusion criteria are similar to what is described above, with participants recruited from multiple centers in different countries. Centers are selected based on the interest and feasibility of recruitment. In addition to completing the LIMB-Q, participants provide demographic, injury, and treatment information. A small subset of participants complete the LIMB-Q at 2 points, 1 to 2 weeks apart, to allow for an assessment of test-retest reliability. Each scale is analyzed independently so that it may stand alone.
Rasch Measurement Theory
In the RMT, the focus is on the relationship between a person’s measurement and his or her probability of responding to an item in a scale . The RMT places individuals onto a ruler based on the likelihood of giving a certain answer [ ]. The qualitative phase of instrument development allows for the creation sample to fit on the ruler, with room to move up or down with treatment, making for scientifically sound and clinically meaningful scales. The RMT analysis identifies the subset of items that are the best indicators of a scale’s concept based on the performance and set of psychometric tests and criteria. Briefly, the psychometric testing of the instrument following the field test will involve several steps. The first will be to assess the psychometric functioning of the items and scales. The thresholds for item response options will be ordered into a hierarchy of items on the scale from the easiest to the most difficult questions. Next, 3 item-fit statistical tests will be employed to determine if the items work well together within a set. These include log residual for item-person interaction, chi-square values for item-trait interaction, and item characteristic curves. Outside of the clinically important items, items not meeting all 3 characteristics will be dropped from the scale. Finally, construct validity is confirmed by comparing the range of the construct measured by the scale with the range experienced by the population. Following these initial steps, the internal consistency, that is, the interrelatedness of items on a scale, will be confirmed with testing for unidimensionality and evaluating the Person Separation Index. In addition, differential item functioning will be evaluated, and items will be reduced accordingly. Items will be reduced that either represent redundancy or overlap on the ruler or are poorly functioning. The steps outlined above will result in a final version of the LIMB-Q and associated scoring table for each scale. Following the development of the finalized scales, construct validity, reliability with internal consistency and test-retest reliability, validity with content, construct and criterion validity, and responsiveness to change will be established.
This study is the first to our knowledge to develop a PRO instrument specific to lower extremity trauma patients, applicable to both amputees and limb-salvage patients. A major limitation in previous lower extremity trauma research has been the lack of an appropriate outcome measure that is capable of assessing all COI relevant to both limb-reconstruction patients and amputees [, ].
The outcome measures currently used to evaluate these patients include functional scales and generic or disease-specific PRO instruments not developed specifically for the lower extremity trauma population. The Lower Extremity Functional Scale  or the Locomotor Capabilities Index [ ] are functional instruments that provide data deemed important to clinicians and researchers. To evaluate the patient’s perspective, generic PRO instruments, such as the Sickness Impact Profile [ , ] and 36-Item Short Form Health Survey questionnaire [ ], or generic musculoskeletal instruments, such as the American Orthopedic Foot and Ankle Score Ankle-Hindfoot and Midfoot Rating Scales [ ] and Short Musculoskeletal Function Assessment Questionnaire [ ], along with disease-specific instruments not developed for but applied to lower extremity trauma patients, such as the Musculoskeletal Tumor Society [ ] and Toronto Extremity Salvage Score [ ], are frequently used. Although these instruments are reliable and valid for the general population, or for their respective musculoskeletal disease cohort, they lack the detail needed to measure clinical change in a reliable or meaningful way for these trauma patients [ ]. PRO instruments focusing on patients who have undergone only amputation are in use [ ]; however, there is no PRO instrument that is designed for all patients with limb-threatening lower extremity traumatic injuries that measures COI relevant to both reconstruction and amputation patient groups.
Strengths and Limitations
The goal of this protocol is to compete phases I and II of the development of a PRO instrument for lower extremity trauma patients. In future stages, our multidisciplinary team will utilize RMT to finalize the development and validation of LIMB-Q, comprising a comprehensive set of clinically meaningful scales, measuring COI important to patients with limb-threatening lower extremity injuries. The scales will be designed for use in research and direct clinical care, applicable to patients who undergo either limb salvage or amputation. The inclusion of patients at varying time points from injury in the development of this instrument will provide for an agile instrument capable of providing meaningful information to clinicians and patients over the duration of a patient’s treatment course. This instrument will have the capability to measure incremental differences in outcomes important to patients and allow for direct comparison between treatment modalities for severe lower extremity trauma. Clinically, this tool will provide patients with a structured and reliable method for communicating outcomes to providers and will provide accurate and objective feedback to improve clinical care.
The LIMB-Q will be a novel PRO instrument for use in lower extremity trauma patients, available for use in research and clinical care. Rigorously developed and validated, the LIMB-Q will have the capability to measure COI relevant to lower extremity trauma patients, to help us better care for and understand the needs of these patients with challenging and often devastating injuries.
This project has been aided by a grant from the Foundation of Orthopaedic Trauma and a grant from the Duke Surgical Center for Outcomes Research via the Duke Musculoskeletal Program.
Conflicts of Interest
Peer-review report.PDF File (Adobe PDF File), 579 KB
- Finkelstein EA, Corso PS, Miller TR. Incidence and Economic Burden of Injuries in the United States. New York: Oxford University Press; 2006.
- Owens BD, Kragh Jr JF, Macaitis J, Svoboda SJ, Wenke JC. Characterization of extremity wounds in Operation Iraqi Freedom and Operation Enduring Freedom. J Orthop Trauma 2007 Apr;21(4):254-257. [CrossRef] [Medline]
- Bosse MJ, MacKenzie EJ, Kellam JF, Burgess AR, Webb LX, Swiontkowski MF, et al. An analysis of outcomes of reconstruction or amputation after leg-threatening injuries. N Engl J Med 2002 Dec 12;347(24):1924-1931. [CrossRef] [Medline]
- MacKenzie EJ, Bosse MJ, Pollak AN, Webb LX, Swiontkowski MF, Kellam JF, et al. Long-term persistence of disability following severe lower-limb trauma. Results of a seven-year follow-up. J Bone Joint Surg Am 2005 Aug;87(8):1801-1809. [CrossRef] [Medline]
- Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002 May;11(3):193-205. [CrossRef] [Medline]
- Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health status measure. Med Care 1981 Aug;19(8):787-805. [CrossRef] [Medline]
- Bergner M, Bobbitt RA, Kressel S, Pollard WE, Gilson BS, Morris JR. The Sickness Impact Profile: conceptual formulation and methodology for the development of a health status measure. Int J Health Serv 1976;6(3):393-415. [CrossRef] [Medline]
- Swiontkowski MF, Engelberg R, Martin DP, Agel J. Short musculoskeletal function assessment questionnaire: validity, reliability, and responsiveness. J Bone Joint Surg Am 1999 Sep;81(9):1245-1260. [CrossRef] [Medline]
- Mundy LR, Grier AJ, Weissler EH, Carty MJ, Pusic AL, Hollenbeck ST, et al. Patient-reported outcome instruments in lower extremity trauma: a systematic review of the literature. Plast Reconstr Surg Glob Open 2019 May 3;7(5):e2218 [FREE Full text] [CrossRef] [Medline]
- Momoh AO, Chung KC. Measuring outcomes in lower limb surgery. Clin Plast Surg 2013 Apr;40(2):323-329 [FREE Full text] [CrossRef] [Medline]
- Lasch KE, Marquis P, Vigneux M, Abetz L, Arnould B, Bayliss M, et al. PRO development: rigorous qualitative research as the crucial foundation. Qual Life Res 2010 Oct;19(8):1087-1096 [FREE Full text] [CrossRef] [Medline]
- Brod M, Tesler LE, Christensen TL. Qualitative research and content validity: developing best practices based on science and experience. Qual Life Res 2009 Nov;18(9):1263-1278. [CrossRef] [Medline]
- Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technol Assess 2009 Feb;13(12):1-177 [FREE Full text] [CrossRef] [Medline]
- Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010 Jul;63(7):737-745. [CrossRef] [Medline]
- Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010 May;19(4):539-549 [FREE Full text] [CrossRef] [Medline]
- Klassen AF, Cano SJ, Schwitzer JA, Scott AM, Pusic AL. FACE-Q scales for health-related quality of life, early life impact, satisfaction with outcomes, and decision to have treatment: development and validation. Plast Reconstr Surg 2015 Feb;135(2):375-386. [CrossRef] [Medline]
- US Department of Health and Human Services FDA Center for Drug Evaluation and Research, US Department of Health and Human Services FDA Center for Biologics Evaluation and Research, US Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes 2006 Oct 11;4:79 [FREE Full text] [CrossRef] [Medline]
- Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1--eliciting concepts for a new PRO instrument. Value Health 2011 Dec;14(8):967-977 [FREE Full text] [CrossRef] [Medline]
- Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding. Value Health 2011 Dec;14(8):978-988 [FREE Full text] [CrossRef] [Medline]
- Riff KW, Tsangaris E, Goodacre T, Forrest CR, Pusic AL, Cano SJ, et al. International multiphase mixed methods study protocol to develop a cross-cultural patient-reported outcome instrument for children and young adults with cleft lip and/or palate (CLEFT-Q). BMJ Open 2017 Jan 11;7(1):e015467 [FREE Full text] [CrossRef] [Medline]
- Thorne S. Interpretive Description. Developing Qualitative Inquiry. Walnut Creek, CA: Left Coast Press; 2009.
- Sandelowski M. Theoretical saturation. In: Given LM, editor. The Sage Encyclopedia Of Qualitative Research Methods. Thousand Oaks, CA: Sage Publications; 2008.
- Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. Br Med J 2000 Jan 8;320(7227):114-116 [FREE Full text] [CrossRef] [Medline]
- Khadka J, Gothwal VK, McAlinden C, Lamoureux EL, Pesudovs KJ. The importance of rating scales in measuring patient-reported outcomes. Health Qual Life Outcomes 2012 Jul 13;10:80 [FREE Full text] [CrossRef] [Medline]
- van Someren MW, Barnard Y, Sandberg J. The Think Aloud Method: A Practical Guide To Modelling Cognitive Processes (Knowledge-Based Systems). London, England: Academic Press; 1994.
- Rasch G. Studies in Mathematical Psychology: I. Probabilistic Models for Some Intelligence and Attainment Tests. Oxford, England: Nielsen & Lydiche; 1960.
- Binkley JM, Stratford PW, Lott SA, Riddle DL. The Lower Extremity Functional Scale (LEFS): scale development, measurement properties, and clinical application. North American Orthopaedic Rehabilitation Research Network. Phys Ther 1999 Apr;79(4):371-383. [Medline]
- Gauthier-Gagnon C, Grise M, Lepage Y. The locomotor capabilities index: content validity. J Rehabil Outcomes Meas 1998;2(4):40-46.
- McHorney CA, Ware Jr JE, Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care 1993 Mar;31(3):247-263. [CrossRef] [Medline]
- Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS, Sanders M. Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int 1994 Jul;15(7):349-353. [CrossRef] [Medline]
- Simon MA, Aschliman MA, Thomas N, Mankin HJ. Limb-salvage treatment versus amputation for osteosarcoma of the distal end of the femur. J Bone Joint Surg Am 1986 Dec;68(9):1331-1337. [CrossRef] [Medline]
- Davis AM, Wright JG, Williams JI, Bombardier C, Griffin A, Bell RS. Development of a measure of physical function for patients with bone and soft tissue sarcoma. Qual Life Res 1996 Oct;5(5):508-516. [CrossRef] [Medline]
- Fayers PM, Machin D. Quality of Life: The Assessment, Analysis and Interpretation of Patient-Reported Outcomes. Hoboken, NJ: Wiley, John & Sons; 2007.
- Gallagher P, MacLachlan M. Development and psychometric evaluation of the Trinity Amputation and Prosthesis Experience Scales (TAPES). Rehabil Psychol 2000 May;45(2):130. [CrossRef]
|COI: concepts of interest|
|PRO: patient-reported outcome|
|RMT: Rasch measurement theory|
Edited by G Eysenbach; submitted 22.04.19; peer-reviewed by JS Howard, B Celler; comments to author 04.07.19; revised version received 09.08.19; accepted 18.08.19; published 17.10.19Copyright
©Lily R Mundy, Anne Klassen, Jordan Grier, Matthew J Carty, Andrea L Pusic, Scott T Hollenbeck, Mark J Gage. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 17.10.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.