Over the last 20 years, advances in the understanding of the molecular basis of lung cancer have led to new diagnostic and treatment practices being introduced into routine care [1], but their everyday impact in general patient populations is undefined. Such improvements in lung cancer care have been an important priority because lung cancer is a major cause of death globally [2] and a significant contributor to ethnicity-related health inequity in Aotearoa New Zealand (NZ) [3] and elsewhere. Starting in 2010, the epidermal growth factor receptor (EGFR) kinase inhibitor drugs erlotinib and gefitinib were introduced into routine clinical use in NZ for the treatment of advanced lung cancer [4]. Before their introduction, clinical trials of erlotinib and gefitinib had demonstrated substantial therapeutic benefits in patients with EGFR mutation–positive lung cancer, with >50% reduction in the risk of disease progression in pooled analyses of randomized trials [5]. These clinical trials also highlighted major safety concerns with erlotinib and gefitinib due to serious adverse drug reactions [6], including interstitial lung disease, renal failure, hepatotoxicity, severe diarrhea, gastrointestinal perforation, and skin and ocular disorders [7,8]. Regulatory studies [9,10] also alerted to the potential interaction of erlotinib and gefitinib with other medicines, expected to lead to adverse outcomes from treatment. Some medicine interactions were expected to reduce lung cancer treatment effectiveness by reducing systemic exposure to erlotinib or gefitinib from their concurrent use with gastric acid suppressants [11,12] or cytochrome P450 enzyme inducers [13,14]. Other medicine interactions were expected to increase serious adverse events by increasing erlotinib or gefitinib systemic exposure from their concurrent use with cytochrome P450 enzyme inhibitors [14,15]. Other medicine interactions were expected to increase the risk of gastrointestinal perforation from concurrent use with nonsteroidal anti-inflammatory drugs or systemic corticosteroids or bleeding from prolongation of the international normalized ratio from concurrent use with coumarin anticoagulants [16,17]. Despite these expectations, the effectiveness and safety of these new personalized lung cancer treatments in the general population of NZ patients has not yet been described, and the contributions made by concomitant medicines and other factors to the occurrence of adverse outcomes remain to be defined in this setting of routine care. As clinical trials underreport harm from anticancer drugs [18] and assess their efficacy under ideal rather than everyday circumstances [19], large-scale observational studies such as the one described in this protocol are required to fully understand the safety and effectiveness of cancer treatment in real-world settings. Recent reports illustrate how real-world studies of new personalized lung cancer treatments can be informative about their effectiveness and safety in general patient populations [20] and unrepresented patient groups [21] as well as the impacts of concomitant medicines [22].

Studies using routinely collected health administrative and clinical data can provide insights into the everyday impacts of new treatment pathways, but their data sources and study methodology first need to be validated [23]. In NZ, cancer registration, hospital discharge, mortality registration, and pharmaceutical dispensing data are collected in national electronic health databases [24], thereby providing potentially useful sources of information for studies of the impacts of routine cancer care. However, bias may be introduced from errors in data recording or in the methods used for identifying the cases and health outcomes of interest, which may limit the validity of study findings. This potential for bias may be controlled in part by quantifying errors inherent in study methodology and data sources by comparison against a reference standard, such as clinical records [25].

The purpose of this paper is to present the study protocol and results of a validation substudy for a planned retrospective cohort study of the safety and effectiveness of erlotinib and gefitinib during the first decade of their use in NZ for treating advanced EGFR mutation–positive lung cancer. The overall objective of the study described in this protocol is to understand the effectiveness and safety of new personalized treatments in the general population of NZ patients with lung cancer and the contributions made by concomitant medicines and other factors to adverse outcomes in this setting of routine care. The study protocol and results of the validation substudy are detailed in the following sections.

The specific aims of the study are to (1) determine the effectiveness and safety of erlotinib and gefitinib for the treatment of EGFR mutation–positive lung cancer in the entire cohort of all patients treated in NZ over a 10-year period, (2) identify serious adverse events and assess their expected association with erlotinib and gefitinib in these patients, (3) determine the prevalence and patterns of the concomitant use of other medicines with erlotinib or gefitinib in these patients, and (4) assess whether the combined use of other medicines was associated with serious adverse events or reduced lung cancer treatment effectiveness in these patients treated with erlotinib or gefitinib.

This study will be a national, population-based, observational, data-linkage, retrospective cohort study that will analyze routinely collected administrative and clinical electronic health data. The study was registered on the Australian New Zealand Clinical Trials Registry (ACTRN12615000998549).

This study will have a whole-of-population sample comprising a single group of patients treated with erlotinib or gefitinib for EGFR mutation–positive lung cancer who were first dispensed erlotinib or gefitinib before October 1, 2020, and followed thereafter until death or for at least 1 year.

NZ had a resident population of approximately 4.4 to 5.1 million people during the period of the planned retrospective study. NZ residents were eligible for state-funded health care, including prescription medicines whose state subsidies were managed by the NZ government agency Te Pātaka Whaioranga–Pharmac [26]. From October 1, 2010, to December 31, 2013, Te Pātaka Whaioranga–Pharmac fully funded erlotinib in NZ as a second-line treatment for advanced non–small cell lung cancer without any requirement for EGFR mutation testing. From August 1, 2012, Te Pātaka Whaioranga–Pharmac fully funded gefitinib in NZ as a first-line treatment for advanced EGFR mutation–positive non–small cell lung cancer. On May 1, 2013, the National Health Committee of the NZ Ministry of Health issued recommendations for EGFR mutation testing in NZ, including that all patients with non–small cell lung cancer should be tested at diagnosis as part of standard pathology processes, except those confidently diagnosed with squamous carcinoma. From January 1, 2014, Te Pātaka Whaioranga–Pharmac restricted funding for erlotinib in NZ to the treatment of advanced EGFR mutation–positive non–small cell lung cancer.

Eligible patients will be identified first in national electronic dispensing records of erlotinib and gefitinib (Figure 1). Subsequently, the data required for the study will be compiled from these dispensing records and other sources (Figure 1). Use will be made of routinely collected health administrative and clinical electronic data sourced from various national electronic health databases, including (1) the New Zealand Cancer Registry, which is a population-based register of all legally mandated notifications of cancer diagnoses and deaths in NZ and includes all cancer types except nonmelanoma skin cancer; (2) the Pharmaceutical Information Database, which contains payment and claims information from pharmacists for state-subsidized community pharmaceutical dispensings in NZ; (3) the National Mortality Collection, which contains information on all deaths registered in NZ and includes the dates and underlying cause of death; and (4) National Minimum Dataset (Hospital Events), which is a national collection of public and private hospital discharge information for day patients and inpatients in NZ. Individual-level data on eligible cohort participants will be compiled from these various national electronic health databases by deterministic data linkage using each participant’s unique National Health Index number. Additional data on eligible cohort participants may be sourced from regional data repositories or clinical records, such as EGFR mutation subtype and testing methodology.

The process of assembling the retrospective cohort will be illustrated using a flow diagram to show the number of patients at each stage of the process, including those who were examined for eligibility, were included in the study, completed follow up, were included in the analysis or not included, and the reasons for noninclusion. Descriptive statistics will be used to analyze the demographic profiles, clinical characteristics, and treatment received by the assembled retrospective cohort. To determine the effectiveness and safety of erlotinib and gefitinib in the cohort (aim 1), time to event effectiveness (time to discontinuation of treatment with erlotinib or gefitinib, overall survival, and lung cancer–specific survival) and safety outcome measures (non–cancer-related mortality) will be analyzed using Kaplan-Meier curves and Cox proportional hazards modeling to generate summary survival statistics (median, 1-year, and 2-year survival, as well as 95% CIs) and examine predictive factors. To identify and assess serious adverse events (aim 2), summary tables and descriptive statistics will be used to display the number and percentage of patients with serious adverse events by ICD-10 code and their known association with erlotinib or gefitinib. To determine the prevalence and patterns of concomitant medicines use (aim 3), summary tables and descriptive statistics will be used to display the number and percentage of patients administered medicines concomitantly by individual specific medicine, therapeutic class, and expected adverse outcome of potential interaction with erlotinib or gefitinib. To analyze associations between high-risk concomitant medicines use and the effectiveness and safety of erlotinib and gefitinib (aim 4), study participants will be stratified according to whether they were exposed or not exposed to high-risk concomitant medicines. Differences between participants exposed to high-risk concomitant medicines and those not exposed in proportions of patients experiencing serious adverse events will be analyzed by chi-square tests and multivariable regression models. Differences between participants exposed to high-risk concomitant medicines and those not exposed in time to event end points will be analyzed using Kaplan Meier curves and Cox proportional hazards modeling. Exploratory analyses may examine associations between the use of concomitant medicines and serious adverse events or poor survival outcomes using different definitions of concurrent exposure, individual medicines, the therapeutic classes of medicines, and patient subgroups such as those defined by ethnicity. Sensitivity analyses may be undertaken, for example, to assess effects on the outcomes of modifying patient eligibility criteria.

With an expected cohort of between 800 and 900 eligible participants, with ≥750 events, the study will have 80% power (=.8) for detecting with statistical significance (=.05) differences in time to treatment discontinuation (primary effectiveness outcome measure) with hazard ratios of ≤0.76 between the groups, each including between 20% and 80% of the whole cohort. With an expected rate of serious adverse events (primary safety outcome measure) of ≥10% and complete serious adverse event data on at least 75% of the eligible cohort participants, the study will have 80% power (=.8) for detecting with statistical significance (=.05) differences of at least 2-fold in serious adverse event rates between the groups, each including between 20% and 80% of the whole cohort.

To validate national electronic health databases as the source of data for this study protocol and the aforementioned methods for determining patient eligibility and identifying the key outcomes and variables of interest, the validation substudy was undertaken in a subgroup of potentially eligible patients. From a source population of 532 patients identified in national electronic dispensing records as having been dispensed erlotinib or gefitinib in the northern region of NZ between October 1, 2010, and September 30, 2020, a total of 100 (18.8%) patients were randomly selected for this validation substudy. The northern region includes the Auckland, Counties Manukau, Waitemata, and Northland district health areas and one-third of the national population. The validation substudy was restricted to northern region patients because their clinical records were available to the investigators for manual review. A gold standard set of data was compiled for each validation substudy patient by manual review of their clinical records, including their EGFR mutation test status. These results were compared to those generated by study methods applied to data compilations from national electronic health databases. As the primary cause of death was not systematically recorded in clinical records, fatal and nonfatal serious adverse events were validated separately. In addition, fatal serious adverse events were validated without the requirement of being non–cancer related and termed for this purpose as potential serious adverse events that were fatal. Categorical data were analyzed by agreement analysis and numerical data by paired 2-tailed t tests. A sample size of 100 patients for the validation substudy was expected to be adequate for defining levels of overall agreement with 95% CIs of –10% to +10%.

Ethics approval for this research was obtained from the New Zealand Ministry of Health Northern B Health and Disability Ethics Committee (13/NTB/165/AM02). All procedures will be conducted in accordance with the relevant ethics guidelines (eg, the Declaration of Helsinki). As this research will retrospectively analyze routinely collected data and not involve direct contact with patients, the human participants included in this retrospective cohort study will not be able, or required, to give informed consent. Requirement for informed consent was considered impractical and undesirable by the researchers, ethics committee, and governance groups who approved the study. Patient confidentiality will be maintained by holding any identifying information as password-protected data spreadsheets or as hard copy data in secure facilities only accessible to approved research or clinical personnel whose employment contracts include obligations regarding patient confidentiality.

Validation analyses for categorical data are shown in Multimedia Appendix 1. For the determination of patient eligibility, the comparison of proxy and standard eligibility criteria applied to national electronic health databases and clinical records, respectively, showed overall agreement and κ statistic of 96% and 0.8936, respectively. For the identification of nonfatal serious adverse events, the comparison of national electronic health databases and clinical records showed overall agreement and κ statistic of 93% and 0.8526, respectively. For the identification of potential serious adverse events that were fatal, the comparison of national electronic health databases and clinical records showed overall agreement and κ statistic of 100% and 1.0000, respectively. For the identification of interruptions, reductions, or substitutions of treatment with erlotinib or gefitinib, the comparison of national electronic health databases and clinical records showed overall agreement and κ statistic of 94% and 0.8764, respectively. For the identification of serious adverse drug reactions, the comparison of national electronic health databases and clinical records showed overall agreement and κ statistic of 97% and 0.8529, respectively. For the identification of the use of high-risk concomitant medicines with erlotinib or gefitinib (primary variable), the comparison of national electronic health databases and clinical records showed overall agreement and κ statistic of 95% and 0.8717, respectively. For identifying comorbidities, the comparison of national electronic health databases and clinical records showed overall agreement and κ statistic of 93% and 0.7779, respectively.

Validation analyses for numerical data are shown in Multimedia Appendix 2. To validate national electronic health databases as the source of key dates for determining time to discontinuation of treatment with erlotinib or gefitinib (primary effectiveness outcome) and overall survival, the dates of the first dispensing of erlotinib or gefitinib, last treatment, and death were compared to those in the clinical records. For the dates of the first dispensing of erlotinib or gefitinib, national electronic health databases and clinical records differed on average by approximately 4 days with a nonsignificant P value of .33 and CIs overlapping with zero difference. For the date of last treatment with erlotinib or gefitinib, national electronic health databases and clinical records differed on average by approximately 6 days with a nonsignificant P value of .46 and 95% CIs overlapping with zero difference. For the dates of death, national electronic health databases and clinical records differed on average by <1 day with a nonsignificant P value of .57 and 95% CIs overlapping with zero difference. For the number of comorbidities, national electronic health databases and clinical records differed on average by <1 comorbidity but with a marginally significant P value of .03 and 95% CIs almost overlapping with zero difference.

As of May 2024, the main study is ongoing.

The research project described in this protocol will provide important new information about the everyday impacts, both beneficial and harmful, arising from the introduction of erlotinib and gefitinib into routine clinical use for treating advanced EGFR mutation–positive lung cancer in NZ a little more than 1 decade ago. Previous studies in this area have been limited to evaluations of the impact and uptake of the introduction of EGFR mutation testing in the northern region of NZ [4,30-32]. Currently, few data exist on the effectiveness and safety of erlotinib and gefitinib in the general NZ patient population and on the contributions made by concomitant medications and other factors to adverse outcomes in this setting of routine care. Such new real-world and NZ-specific data are likely to provide deeper insights into the harms profiles of erlotinib and gefitinib and to have important implications for clinical practice and patient-centered care; for example, if the study establishes associations between adverse outcomes and the use of concomitant medicines, an awareness of these data could influence the clinical practices of health care professionals who prescribe or dispense erlotinib, gefitinib, or other medicines for use concomitantly with erlotinib and gefitinib in NZ or elsewhere. The data will also inform new health policies, such as those concerning state subsidies for newer-generation EGFR kinase inhibitor drugs that have gained regulatory approvals for marketing but remain unfunded in NZ since erlotinib and gefitinib were introduced into routine care approximately 1 decade ago.

In the validation substudy, we showed that the methods proposed in the study protocol, when applied to data compiled from national electronic health databases, were valid for determining patient eligibility and identifying the outcomes and variables of interest to this retrospective cohort study. Compared to gold standard data sets compiled by a manual review of clinical records, the results generated by the study methods applied to data compiled from national electronic health databases showed good levels of agreement in categorical data, such as patient eligibility, serious adverse events, and the use of high-risk concomitant medicines, and generally small and statistically insignificant differences in numerical data such as the dates required for the estimation of time to treatment discontinuation. These results support the validity of the methods described in the study protocol and the robustness of the findings that will be generated and eventually reported.

The strengths of the research protocol include its large population-based sample, internal validity, national generalizability, and the unique patient cohort. We envisage recruiting almost all eligible patients from throughout NZ in a whole-of-patient-population cohort, thereby eliminating selection biases and assembling one of the largest national cohorts of this type to be reported to date [33-37]. However, few of these previous studies have evaluated serious adverse events from erlotinib and gefitinib and their association with the use of concomitant medication in real-world settings. The use of national electronic health databases that use consistent data collection methodology will give confidence that the study results will be generalizable to all NZ patients and regions. Previous studies have highlighted striking geographic and ethnic variations in EGFR mutation–positive lung cancer [38,39]. As NZ is a multiethnic society, with the largest global populations of Indigenous Māori and Pacific people, this study will provide important new information of international interest about treatment outcomes in these understudied groups.

The limitations of the study protocol include potential biases from errors inherent in the use of routinely collected data; for example, the safety outcomes for the study protocol assume that the coding of hospital admissions and deaths is always complete and accurate, whereas this is not necessarily the case. Further limitations of the retrospective study design include that, unlike in prospective randomized studies, it will not be possible to assess causality or control for confounding variables. The study design also assumes adherence to the treatment regimens as recorded in dispensing records and provides no practical method for assessing compliance. Another limitation is the historical perspective of the study protocol, which will evaluate the impacts of treatment that occurred between 2 and 12 years ago and may now be of limited relevance to current clinical practice. For the initial assessment of patient eligibility, the study protocol infers EGFR mutation status indirectly from pharmaceutical dispensing records rather than directly from EGFR mutation test results, which could have introduced bias. The findings of the study may not be generalizable outside NZ because of limited external validity and a shift in standard EGFR treatments to targeted therapies that have superseded gefitinib and erlotinib. However, if the study establishes associations between adverse outcomes and the use of concomitant medicines, these findings could have important implications for the use of erlotinib or gefitinib in other countries or for the use of other anticancer kinase inhibitor drugs susceptible to similar medicine interactions [40]. The limitations of the validation study included its small sample size and the resulting wide CIs for the levels of agreement, κ statistic, and other statistical parameters.

In conclusion, a protocol is presented for a planned study that seeks to describe the safety and effectiveness of erlotinib and gefitinib during the first decade of their routine clinical use in NZ for treating EGFR mutation–positive lung cancer in a national whole-of-patient-population retrospective cohort. An exploratory substudy has demonstrated the feasibility and validity of using national electronic health databases and the study methodology developed for determining patient eligibility and identifying the outcomes and variables of interest for this study.