Inclusion criteria include patients aged 18 years or above, who suffered a lower extremity traumatic injury, distal to the midfemur, resulting in the need for reconstruction or amputation. Patients who do not speak English are excluded. A member of the clinical team recruits patients either face-to-face in the hospital or clinic or over the telephone. Interested participants who are unable to travel to the interview are offered phone interviews.

Purposeful sampling maximizes participant variability to ensure a diverse group of participants, reflected in demographic variables (age, gender, and race), injury etiology, time since injury, and injury outcome (reconstruction, amputation, or amputation after failed reconstruction). Patients having undergone successful limb salvage, early amputation, and late amputation after failed attempts at reconstruction with varying amounts of time from initial injury will be recruited. Recruitment continues until content saturation is reached, that is, no new concepts are being identified in the interviews [22].

Consent is provided before participation. Demographic characteristics include age, gender, and race and clinical characteristics include medical history, drug/tobacco use, injury etiology, injury characteristics, date of injury, and surgical history, in addition to postinjury findings such as return-to-work status and chronic pain medication use. Trained qualitative researchers perform interviews using an interview guide developed to reflect the preliminary conceptual framework to ensure that all previously identified COI are addressed. The interview guide is a working document that is updated throughout the study as new COI are identified; see Figure 2.

Interviews are recorded and transcribed verbatim. Each interview is coded line by line to identify COI, categorized top-down into domains, followed by themes and subthemes. Interviews are coded as a team by a clinical expert (LRM) and qualitative methods expert (AK) to ensure consistency, reliability, and validity in coding. Coding questions requiring additional clinical expertise were discussed with additional clinical experts (MJG and STH) as needed. Constant comparison is used to ensure consistency in coding, allowing for continuous refinement of previously coded data [23]. It must be noted that more than 1 item can be generated from a code if the participant uses different words/phrases to describe their thoughts or experiences. However, if someone uses the same word/phrase repeatedly, we develop 1 item for the concept as we are interested in capturing a unique concept rather than capturing the number of times a subject repeats himself or herself. The domains are used to generate the final conceptual framework, organizing all identified COI.

The codes generated in the data analysis process represent an exhaustive list of all relevant COI to lower extremity trauma patients, organized through the lens of the conceptual framework. Each domain within the conceptual framework that is relevant to clinical care or research is utilized to develop a preliminary scale. Coded text with major/minor themes is moved into an Excel spreadsheet (Microsoft Excel 2016) along with the participant’s age, gender, and surgical outcome (reconstruction, amputation, or reconstruction to amputation). Clinical data are utilized to identify the codes that are relevant across patient groups versus the codes that are relevant to only a specific patient group, such as concerns specific to patients with prostheses. Each code is examined in turn to generate a preliminary item pool for scale development. Within each scale, the items are developed from the themes and subthemes, using as many of the participants’ own words as possible. For each scale, instructions and response options are added. Our goal is to keep the scales simple and in line with the published guidelines [24]. Khadka et al demonstrated that rating scales or questionnaires with a complicated question format, with a large number of response categories or with unlabeled categories tended to be dysfunctional and recommended that PRO instruments should have simple question formats and only a few (4-5 at most) response options and that all response options should be labeled [24].

Semistructured interviews are conducted with lower extremity trauma patients to assess for content and clarity of wording, ensuring the instrument is comprehensive, relevant, and comprehensible. Participants meeting the same inclusion/exclusion criteria stated previously are recruited in a similar manner. Recruitment starts with the initial qualitative interview of participants and then, if needed, is expanded to additional patients meeting the above inclusion criteria, who did not participate in the first round of interviews. Participants are interviewed one-on-one by a trained qualitative interviewer, with the participant going through the scale line by line using the think-aloud technique, explaining out loud how the participant interprets each item [25]. Participants are also asked to comment on the instructions, probed for missing content, and asked to identify problematic items and scales. Consequently, the instrument is prepared for the large-scale field test, combining revisions from the opinions of expert clinicians.

In concurrence with soliciting opinions from patients, the opinions of expert clinicians are solicited. This is to ensure that the PRO instrument captures all key concepts relevant to clinicians or researchers. Expert advice is sought from clinicians, including plastic and orthopedic surgeons, in addition to the physical and occupational therapists, nursing, prosthetists, and limb-loss experts. Experts provide written feedback on the instrument via a written survey.

A pilot field test is carried out before the large-scale international field test. The goal of the pilot field test is 2-fold: first, to identify and address any administrative or logistical difficulties, and second, to perform a preliminary Rasch analysis to identify and address any psychometric issues with LIMB-Q scale performance. Inclusion criteria are similar to those described above. Participants complete the LIMB-Q in addition to providing demographic, injury, and treatment information. Rasch measurement theory (RMT) is used to drive the data analysis of the scales and is described in detail in the following section.

A large-scale international field test is the final stage in LIMB-Q development, with the goal to finalize the items and scales of the LIMB-Q, determining scale validity, reliability, and responsiveness. Inclusion criteria are similar to what is described above, with participants recruited from multiple centers in different countries. Centers are selected based on the interest and feasibility of recruitment. In addition to completing the LIMB-Q, participants provide demographic, injury, and treatment information. A small subset of participants complete the LIMB-Q at 2 points, 1 to 2 weeks apart, to allow for an assessment of test-retest reliability. Each scale is analyzed independently so that it may stand alone.

In the RMT, the focus is on the relationship between a person’s measurement and his or her probability of responding to an item in a scale [13]. The RMT places individuals onto a ruler based on the likelihood of giving a certain answer [26]. The qualitative phase of instrument development allows for the creation sample to fit on the ruler, with room to move up or down with treatment, making for scientifically sound and clinically meaningful scales. The RMT analysis identifies the subset of items that are the best indicators of a scale’s concept based on the performance and set of psychometric tests and criteria. Briefly, the psychometric testing of the instrument following the field test will involve several steps. The first will be to assess the psychometric functioning of the items and scales. The thresholds for item response options will be ordered into a hierarchy of items on the scale from the easiest to the most difficult questions. Next, 3 item-fit statistical tests will be employed to determine if the items work well together within a set. These include log residual for item-person interaction, chi-square values for item-trait interaction, and item characteristic curves. Outside of the clinically important items, items not meeting all 3 characteristics will be dropped from the scale. Finally, construct validity is confirmed by comparing the range of the construct measured by the scale with the range experienced by the population. Following these initial steps, the internal consistency, that is, the interrelatedness of items on a scale, will be confirmed with testing for unidimensionality and evaluating the Person Separation Index. In addition, differential item functioning will be evaluated, and items will be reduced accordingly. Items will be reduced that either represent redundancy or overlap on the ruler or are poorly functioning. The steps outlined above will result in a final version of the LIMB-Q and associated scoring table for each scale. Following the development of the finalized scales, construct validity, reliability with internal consistency and test-retest reliability, validity with content, construct and criterion validity, and responsiveness to change will be established.