This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Nonmuscle invasive bladder cancer (NMIBC) is a chronic condition requiring intensive follow-up, repeated endoscopic examinations, tumor resections, and intravesical treatments that can occur every 3 months for life. In this clinical context, patient-reported outcomes (PROs) are a critical concern for patients and their managing clinicians. PROs have enormous potential to be integral to treatment assessment and recommendations for NMIBC; however, current PRO measures are inadequate for NMIBC because they lack key NMIBC-specific symptoms and side effects associated with contemporary treatments.
The overarching aim of this study was to develop and evaluate a patient-reported symptom index (SI) for individuals with NMIBC (the NMIBC-SI) that is acceptable to patients; reliable, valid, and responsive to differences between contemporary treatments for NMIBC; and fit for purpose as an endpoint in clinical trials.
The NMIBC-SI will be evaluated in 2 field tests across a total of 3 years. Field test 1 is a cross-sectional study design involving 225 adult NMIBC patients recruited while undergoing active treatment or those who completed final treatment within the past week. Data collected include patient demographics, clinical features of the tumor, risk category, treatment type, comorbidity, and PROs. Field test 2 is a prospective longitudinal study involving 225 newly diagnosed NMIBC-SI patients. Clinical data and patient-completed questionnaires will be collected at 4 time points during treatment: before tumor resection, 1 week after resection, end-of-induction intravesical therapy, and 1-year follow-up. Standard psychometric tests will be performed to assess the reliability, validity, responsiveness, and clinical utility of the NMIBC-SI.
Participant recruitment to field test 1 commenced in February 2017. Recruitment for field test 2 is planned to commence in January 2018. Final results are expected to be published in 2019. The NMIBC-SI will be freely available for use via registration.
This study protocol contains detailed methods that will be used across multiple international sites. Phase 2 in the development of the NMIBC-SI will enable a comprehensive evaluation of its reliability, validity, and responsiveness to ensure that the NMIBC-SI is fit for purpose in clinical research and provides an evidence base for the ongoing improvement of future therapies for NMIBC.
ClinicalTrials.gov NCT03091764; http://clinicaltrials.gov/ct2/showNCT03091764 (Archived by WebCite at http://www.webcitation.org/6umBhQeNX)
Bladder cancer is the 9th most common cancer diagnosed worldwide and the 13th most common cause of cancer death [
Nonmuscle invasive bladder cancer is a chronic disease. Its management depends on the risk of the bladder cancer recurring and progressing [
Guidelines for the management of NMIBC are based on evidence for the effectiveness of treatments in reducing risk of bladder cancer progression and recurrence. Evidence for reductions in health-related quality of life (HRQoL) and associated side effects with each treatment are not incorporated into the decision-making process because they have been poorly documented (see section below on Critical Gaps in Knowledge About HRQoL and Patient-Reported Outcomes [PROs] in NMIBC). This is despite the fact that these treatments can cause substantial side effects and local and systemic toxicity. For example, the commonly used BCG has proved effective in reducing recurrences in patients with high-grade tumors and carcinoma in situ, but it can cause moderate to severe local and systemic side effects, and only 16% are able to complete their full treatment schedule [
A PRO is any report that comes directly from patients about how they feel in relation to their health condition and its therapy, without interpretation by others [
PROs are beneficial for improving patient-clinician communication, prioritizing patient-centered care, and improving service provision [
The primary goal in managing NMIBC in patients is to completely remove the tumor and control the unpredictable risk of recurrence and progression to muscle invasion [
The most recent indexed comprehensive review of the literature investigating the impact of NMIBC on HRQoL is over 14 years old and is based on research published between 1966 and 2000 [
Our review did identify a NMIBC-specific PROM, the European Organisation for Research and Treatment of Cancer (EORTC)—EORTC QLQ-BLS24 questionnaire—developed in 1996. There is no publicly available information detailing its development, and it only recently underwent validation, 18 years after its development (now renamed the EORTC QLQ-NMIBC24) [
Therefore, a patient-reported NMIBC-specific symptom index (NMIBC-SI), including possible symptoms and side effects of contemporary treatments for NMIBC, is needed to enable accurate, robust, and clinically relevant assessment of differences in PROs among contemporary treatments and to provide an evidence base for the ongoing improvement of future therapies for NMIBC. Such a measure has been developed based on existing literature and interviews with patients and clinicians [
The overarching aim of this study was to develop and evaluate the NMIBC-SI for reliability, validity, and responsiveness to ensure that it is fit for purpose in clinical research.
To evaluate the feasibility and acceptability of the NMIBC-SI; produce a shorter version, if appropriate, by selecting items that perform best against robust psychometric criteria; examine the legitimacy of summing items into scales; identify legitimate scales and subscales; and test scaling assumptions and scale performance in a large-scale Australian sample (field test 1).
To psychometrically evaluate the measurement properties of the final version SI, testing for reliability, clinical validity (sensitivity to differences between patient groups and responsiveness to clinically important change), and interpretability of the final version NMIBC-SI in a new large-scale international sample (field test 2).
To conduct a head-to-head comparison of the new NMIBC-SI with the EORTC bladder cancer module, QLQ-NMIBC24.
To assess and compare key PROs across the full range of contemporary treatments for NMIBC and over-the-disease trajectory, including acute treatment and 1-year survivorship.
To compare PROs between patients with low-, intermediate-, and high-risk NMIBC.
This multicenter study was designed to evaluate the psychometric properties of an NMIBC-SI for patients treated for NMIBC. Guidance for developing and validating health outcome measures has been followed to ensure high quality and standardization for the development of the NMIBC-SI [
In preliminary research and phase 1 development, a conceptual framework was developed by tapping into 3 sources: (1) a systemic review and narrative analysis of the HRQoL and PRO literature, identifying several local and systematic side effects associated with contemporary treatments for NMIBC (eg, urinary problems, discomfort, and malaise); (2) in-depth qualitative interviews with a sample of NMIBC patients that explored patients’ experience of receiving treatment; and (3) in-depth qualitative interviews with treating clinicians (specialist nurses and urologists) that explored important issues from their perspective. This phase was important to ensure that high content validity was achieved and demonstrated [
In phase 1 pretesting, further qualitative interviews explored the generated list of issues with patients and clinicians for clarity and overlap and the appropriateness of the NMIBC-SI’s time frame, question stem, and response options. On the basis of information obtained from the interviews, the provisional NMIBC-SI was revised to produce a preliminary version ready for field testing.
The evaluation of the NMIBC-SI is phase 2 of this project. It will be undertaken in 2 parts: preliminary field test 1 (item reduction) and final field test 2 (psychometric properties). The preliminary field test will identify any items with poor psychometric performance for possible elimination. The final field test will be undertaken to evaluate the item-reduced version of the NMIBC-SI for reliability, validity, and responsiveness. Gold standard psychometric methods will be used [
The psychometric properties of the NMIBC-SI will be evaluated in 2 field tests, including a preliminary field test (item reduction) to identify items with poor psychometric properties for possible elimination and to identify subscales, and a final field test (psychometric evaluation) to evaluate the reliability and validity of the item-reduced version of the NMIBC-SI. The overall strategy and methods for the psychometric evaluation are based on the methods used to develop and validate PROMs in several other areas of medicine and surgery [
The purpose of the preliminary field test 1 is to produce a short (item-reduced) version of the NMIBC-SI and to undertake an initial psychometric evaluation of the item-reduced questionnaire. This will be done using a cross-sectional study design.
An item reduction strategy developed for evaluation of PROMs in other medical areas [
Development and evaluation of the symptom index for individuals with nonmuscle invasive bladder cancer.
Results of the item reduction analyses will be used to develop a shorter version of NMIBC-SI for final psychometric field testing.
Adult patients (aged ≥18 years) from participating centers diagnosed with NMIBC, who are able to read and understand English and give their written informed consent will be included in the study. Patients will be recruited while undergoing active treatment (ie, 1 week after tumor resection or intravesical therapy) or when they have completed final treatment for NMIBC within the past week.
Patients will be excluded from the study if any of the following criteria apply:
They are unconscious or confused.
They have cognitive impairment.
They are unable to speak, read, and/or write in English.
They are diagnosed with muscle invasive disease.
They are unable to provide informed consent.
Approximately 225 NMIBC patients will be required, purposively sampled to ensure representation of patients across the 3 NMIBC risk categories (
Patients from participating centers who meet the eligibility criteria will be informed about and invited to take part in the study either in person or via an invitation letter. Consecutive patients will be identified and approached to participate by either the named site investigator or the named site nurse. Accrual will be reviewed to ensure that there is balanced representation of patients in all NMIBC categories. A record of those identified as eligible, approached to participate, refusals, consenting patients, and questionnaire returns will be kept for progress monitoring and final reporting purposes (see Data Collection/Assessment section below).
A verbal explanation of the study and a patient information sheet will be provided for patients to consider. These will include detailed information about the rationale, design, and personal implications of the study. Following information provision, patients will have as much time as they need to consider participation. The right of the patient to refuse consent without giving reasons will be respected.
Assenting patients will then be invited to provide informed, written consent on the consent form at the end of the patient information sheet to collect baseline assessment data and to complete the questionnaire. The patient will remain free to withdraw from the study at any time without giving reasons and without prejudicing any further treatment. The original consent form will be filed within the investigator site file or at a designated secure location.
Following informed consent and confirmation of eligibility, participants will be provided with a study ID number and registered to the study.
The participating site staff will complete a log of all patients screened for eligibility. Screening logs will be returned to the University of Sydney.
Using the study ID number as an identifier, information will be collected for all eligible patients, including the age, gender, risk category, and treatment type.
For those who decline to participate in the study, reasons will be recorded.
High-risk tumors
Any of the following:
T1 tumor
G3 (high grade, which is a mixture of some G2 and G3) tumor
Carcinoma in situ (CIS)
Multiple and recurrent and large (>3 cm) Ta G1G2 tumors (all conditions must be presented at this point); low grade is a mixture of G1 and G2
Intermediate-risk tumors
All tumors not defined in the adjacent categories (between the category of low- and high risk)
Low-risk tumors
Primary, solitary, Ta, G1 (PUNLMP, low grade is a mixture of G1 and G2), <3 cm, no CIS
Study data will be recorded by participating site staff on case report forms and by participants on questionnaire booklets either on paper or electronically. The study data are collected and managed using REDCap electronic data capture tools hosted at the University of Sydney [
Assessments will be undertaken as follows:
Registration and baseline data
Patient questionnaire booklet
Registration and clinical data will be completed by the clinician or specialist nurse at participating centers. Hard-copy questionnaires will be collected by the clinician or specialist nurse or posted by the patient back to the site. Study ID number coded data will be sent to the University of Sydney for central data management. Individually identifiable data and master lists linking study ID numbers to individual identity will be retained by the participating sites.
Patients who meet the inclusion criteria and provide informed consent will be registered to this study. Registration and clinical information will be recorded by participating site staff including the following:
Patient study ID number
Age
Gender
Country of birth
Marital status
Living arrangements
Education
Risk category (clinical definitive category from medical records)
Tumor grade
Tumor stage
Treatment type (information about which treatment interventions the patient is currently receiving)
Comorbidity (Charlson Comorbidity Index [
Name of site staff member completing clinical data
Confirmation of eligibility
Participants will self-complete the questionnaire booklet containing the NMIBC-SI, which will be provided to them by the clinician or specialist nurse at participating centers in hard copy or by link to the Web-based version. It is anticipated that completion of the questionnaire may take up to 20 min. Completed hardcopy questionnaires will be entered into REDCap by participating site staff directly or returned to the University of Sydney for data entry.
Participants will be given a hard copy of the questionnaire or provided with a link to the Web-based questionnaire, depending on their preference.
To determine whether the NMIBC-SI fulfills fundamental prerequisites for rigorous measurement as defined by established psychometric criteria [
The purpose of the item reduction analysis is to produce a psychometrically robust version of the NMIBC-SI. Standard psychometric tests and criteria, as described in
Missing data will not be imputed. The frequency of missing data will be determined, and items with a response rate of <80.0% (180/225) will be investigated.
The purpose of the final field test 2 is to assess the reliability and clinical validity (sensitivity to differences between patient groups and responsiveness to clinically important change) and interpretability of the final version NMIBC-SI in a new large-scale international sample. This phase will be conducted using a prospective longitudinal study design. The outcome will be a patient-reported NMIBC-SI that is reliable and clinically valid, with NMIBC-specific content that complements the more generic HRQoL content of the EORTC QLQ-C30.
Adult patients (aged ≥18 years) from participating centers diagnosed with NMIBC, after imaging or flexible cystoscopy but before endoscopic resection, who are able to read and understand English and give their written informed consent will be included in the study.
Approximately 225 patients newly diagnosed with NMIBC will be required to provide sufficient subjects for statistical assessment of test-retest reliability [
Psychometric tests and criteria.
Property | Definition/test | Criteria for acceptability | ||||
1. Item analysis | Identify items for possible elimination due to weak psychometric performance; assessed on the basis of (1) exploratory factor analysis with principal axis factoring and (2) item- and scale-level analyses | Exploratory factor analysis: items with a factor-loading coefficient ≥0.4 will be retained in each subscale. |
||||
2. Acceptability | The quality of data; assessed by completeness of data and score distributions | Missing data for summary scores <20%; normal distribution of endorsement frequencies across response categories (ie, absence of skew, endorsement rates between 0.20 and 0.80); and floor or ceiling effects for summary scores <10% | ||||
3.1 Internal consistency | The extent to which items comprising a scale measure the same construct (eg, homogeneity of the scale); assessed by Cronbach alpha and item-total correlations | Cronbach alpha for summary scores ≥.70 and item-total correlations ≥.30 | ||||
3.2 Test-retest reliability (field test 2 only) | The stability of a measuring instrument; assessed by administering the instrument to respondents on two different occasions and examining the correlation between test and retest scores | Test-retest reliability and intraclass correlations for summary scores ≥.70 | ||||
4.1 Content validity | The extent to which the content of a scale is representative of the conceptual domain it is intended to cover; assessed qualitatively during the questionnaire development stage through pretesting with patients, expert opinion, and literature review | Qualitative evidence from patients, expert opinion, and literature review that items in the scale are representative of the construct being measured | ||||
Within-scale analyses | Evidence that a single entity (construct) is being measured and that items can be combined to form a summary score | Confirmatory factor analysis: (1) items with a factor-loading coefficient ≥0.4 and (2) moderate to high correlations between scale scores | ||||
Convergent validity | Evidence that the scale is correlated with other measures of the same or similar constructs; assessed on the basis of correlations between the measure and other similar measures | Correlations are expected to vary according to the degree of similarity between the constructs that are being measured by each instrument. Specific hypotheses are formulated and predictions tested on the basis of correlations | ||||
Discriminant validity | Evidence that the scale is not correlated with measures of different constructs; assessed on the basis of correlations with measures of different constructs | Low correlations between the instrument and measures of different constructs | ||||
Known groups differences | The ability of a scale to differentiate known groups; assessed by comparing scores for subgroups that are expected to differ on the construct being measured | Significant differences between known groups or difference of expected magnitude | ||||
5. Responsiveness | The ability of a scale to detect clinically significant change following treatment of known efficacy; assessed by examining within-person change scores before and after treatment and calculating an effect size statistic (mean change score divided by standard deviation of pretreatment scores) | Moderate to large effect sizes (small 0.2, moderate 0.5, and large 0.8 or higher) |
Patient-reported outcome assessment schedule.
Risk group | Assessed within 3 months |
Assessed within 4 to 10 days |
Assessed within 1 month |
Assessed within 1 month |
High | T1 | T2 | T3 | T4 |
Intermediate | T1 | T2 | T3 | T4 |
Low | T1 | T2 | T3c | T4 |
~n | 225 | 225 | 225 | 225 |
aA minimum of 25 participants from each risk group will be asked to complete an additional questionnaire pack 3 to 7 days after T3.
bPreferably before cystoscopy.
cFor the low-risk group, T3 will be 8 weeks after resection.
Patients will be recruited at diagnosis, after imaging or flexible cystoscopy but before endoscopic resection. Eligible patients will be identified by the clinician investigators and their teams at participating centers, ensuring adequate representation of patients in the 3 risk groups (see
Registration data will be collected as done for field test 1. Baseline questionnaires will be completed before endoscopic resection (
Participants will complete follow-up questionnaire packs at scheduled follow-up time points (see
Prospective assessment of newly diagnosed patients before and after treatment is required for the responsiveness analysis. It is expected that the NMIBC-SI will detect changes in symptoms due to specific treatments. As risk profile determines treatment schedule, the corresponding schedules of prospective assessments are indicated in
In addition to the planned assessments (
Participants will be sent a hard copy or link to the questionnaires 2 weeks before their scheduled follow-up time point. Up to 2 email or telephone reminders to complete follow-up questionnaires will be made if questionnaires are not completed by the follow-up date. The clinician or specialist nurse at participating centers will be responsible for sending follow-up questionnaires, emails, and reminders to participants unless participants who complete the Web-based questionnaires have provided their email address to receive automatic reminders directly from REDCap. The assessment time windows are indicated in
Participants will be given a hard copy of the questionnaire or provided with a link to the Web-based questionnaire depending on their preference.
A questionnaire pack containing the NMIBC-SI, EORTC QLQ-C30, and the EORTC QLQ-NMIBC24 will be self-completed by all participants via hard copy or Web.
Participants will complete the item-reduced version of the NMIBC-SI, the QLQ-C30, and the QLQ-NMIBC24 measures to assess construct validity (convergent, discriminant, and known groups; see
The QLQ-C30 [
The QLQ-NMIBC24 [
All measures will be administered in the same order. It is anticipated that completion of questionnaire packs may take up to 20 min.
Analyses will evaluate the measurement properties of the final version of the NMIBC-SI, using psychometric tests described for field test 1 (
Analyses of the clinical aims will include: (1) evaluation of PRO changes over time, from diagnosis to peak treatment and at 1 year; (2) comparison of PROs between the 3 risk groups at each time point; and (3) head-to-head comparison of the relative discriminatory ability of the NMIBC-SI and the EORTC QLQ-NMIBC24. The pretumor resection assessment (T1) will be at diagnosis and is considered our baseline assessment. Patients are not expected to have any treatment-related problems at this point. The subsequent 3 time points (
A patient-reported NMIBC-SI, including all symptoms and side effects associated with contemporary treatments for NMIBC, has been developed based on a conceptual framework of PROs important to patients with NMIBC and their managing clinicians [
The conceptual framework was developed by utilizing 3 sources (see
The draft version of the NMIBC-SI was pretested using structured interviews with key stakeholders (patients and clinicians), testing for face validity, relevance and comprehensiveness of content, comprehensibility, and clinical utility. Specifically, clarity and overlap of items and the appropriateness of the NMIBC-SI’s time frame, question stem, and response options were explored. On the basis of information obtained from the interviews, the provisional NMIBC-SI was revised to produce a preliminary version ready for field testing.
The NMIBC-SI was designed to complement and be administered alongside the EORTC QLQ-C30 generic cancer questionnaire [
The field testing phase is planned over 3 years. Recruitment for field test 1 commenced in February 2017 in 9 Australian centers. Recruitment for field test 2 is planned to commence in January 2018 in the same 9 Australian centers that participated in field test 1 plus an additional 10 international centers (2 in New Zealand, 4 in the United States, 2 in Canada, and 2 in Europe) following the universal approach to using the same language in different countries [
This study protocol contains detailed methods to be used across 19 international sites, including both public and private hospitals and treatment clinics that treat patients diagnosed with NMIBC. Field test 1 is a cross-sectional study and includes a sample of patients with NMIBC on active treatment. Field test 2 is a longitudinal study and includes a sample of newly diagnosed patients to enable assessment of possible treatment effects as well as responsiveness of the NMIBC-SI. No patients recruited for field test 1 will be included in field test 2. Phase 2 in the development of the NMIBC-SI will enable a comprehensive evaluation of its reliability, validity, and responsiveness to ensure that it is fit for purpose in clinical research and provides an evidence base for the ongoing improvement of future therapies for NMIBC.
Following evaluation, our NMIBC-SI will be suitable for use with English-speaking patients, diagnosed and treated for NMIBC in Australia, New Zealand, the United States, Canada, and the United Kingdom. Cross-cultural and language translations are planned following development and evaluation of the English version.
Peer Review Report.
Bacillus Calmette–Guérin
European Organisation for Research and Treatment of Cancer
health-related quality of life
nonmuscle invasive bladder cancer
symptom index for individuals with NMIBC or NMIBC-specific symptom index
patient-reported outcome
patient-reported outcome measures
symptom index
This study is sponsored by the University of Sydney with grant funding from Cancer Australia and Cancer Council NSW, Australia (ID: 1103036). The contents of this protocol are solely the responsibility of the authors and do not reflect the views of Cancer Australia or the Cancer Council NSW.
The authors thank the NMIBC-SI Working Group and participating site staff for their contribution to patient recruitment and the Australian and New Zealand Urogenital and Prostate (ANZUP) Cancer Trials Group for their collaboration. NMIBC-SI working group members are Dr Paul Anderson, Prof Maurizio Brausi, Dr Venu Chalasani, Prof Mark Frydenberg, Prof Dickon Hayne, A/Prof Shomik Sengupta, Mr Peter Stanford, and Dr Roger Watson.
The authors thank all the study participants.
None declared.