This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
By 2030, prostate cancer will be the most commonly diagnosed cancer in North America. To mitigate this impending challenge, comprehensive support mechanisms for disease- and treatment-specific changes in health and well-being must be proactively designed and thoughtfully implemented for streamlined survivorship care. mHealth apps have been lauded as a promising complement to current outpatient treatment and monitoring strategies, but have not yet been widely used to support prostate cancer survivorship needs. A realist evaluation is needed to examine the impact of such apps on the prostate cancer survivorship experience.
We seek to gain an understanding of how an mHealth app for prostate cancer survivorship care called Ned (No Evident Disease) is adopted and accepted by patients, caregivers, and clinicians. We also aim to determine the effect of Ned on health-related quality of life, satisfaction with cancer care, unmet needs, self-efficacy, and prostate cancer-related levels of anxiety.
The Ned case study is a 12-month mixed-methods embedded single-case study with a nested within-group pre-post comparison of health outcomes. We will give 400 patients, 200 caregivers, and 10 clinicians access to Ned. Participants will be asked to complete study assessments at baseline, 2 months, 6 months, and 12 months. We will conduct 30 semistructured qualitative interviews with patients (n=20) and their caregivers (n=10) poststudy to gain insight into their experience with the app.
We recruited our first survivor in October 2017 and anticipate completing this study by May 2019.
This will, to our knowledge, be the first realist case study to evaluate an app for prostate cancer survivorship care. Prostate cancer survivors are set to increase in number and longevity, heightening the need for integrated survivorship solutions to provide them with optimal and durable outcomes. The knowledge gained from this study will comprehensively inform how and why Ned works, for whom, and in what circumstances. Understanding the impact of digital health interventions such as Ned on how survivors care for themselves is critical to realizing patient-centered care.
By 2030, prostate cancer will be the most commonly diagnosed cancer in North America [
The definition of prostate cancer survivorship has changed with the understanding that the patient experience encompasses far more than just medical treatment [
Three critical problems have been identified in the overall clinical management of prostate cancer survivorship: (1) patients do not know how they are doing and how they compare with other matched patients; (2) clinicians are not optimally informed about patient issues in a systematic and evidenced-based manner; and (3) patients receive fragmented care [
In recent years, digital health interventions have presented new opportunities and challenges for cancer care, with systematic reviews suggesting their overall usefulness and acceptance for cancer patients and their caregivers [
The recent endorsement of mHealth, broadly defined as a health intervention that has been operationalized into an app for patient use and is delivered or supported through the use of wireless devices (eg, mobile phones, tablets, media players, wearables), has been lauded as a promising complement to current outpatient treatment and monitoring strategies for chronic care [
Previous work by our research group has identified a homogeneity in the range of study designs used to evaluate mHealth apps [
This protocol outlines a pragmatic, mixed-methods embedded single-case study with a nested within-group pre-post comparison of health outcomes, guided by Pawson and Tilley’s realist evaluation principles [
We will use the realist evaluation framework to guide the interpretation of this case study [
The Ned case study is a 12-month mixed-methods embedded single-case study with a nested within-group pre-post comparison of health outcomes. We will evaluate the adoption, acceptability, and effectiveness of Ned as perceived by the 400 patients, 200 caregivers, and 10 clinicians who are given access to the app. Our aims are both observational and experimental: we are interested in deriving a deep understanding of how Ned affects survivorship care, but also in establishing parameters of possible clinical change to inform the effect of the app on quality-of-life outcomes. We also seek to identify the process mechanisms required to introduce Ned into a public health network, specifically the resources required to support the platform. This knowledge will inform future innovation diffusion efforts for Ned as we look to scale the platform across Canada. We will employ a combination of semistructured interviews, questionnaires, and qualitative observations to illustrate a rich picture of how Ned is adopted, whether it is accepted, and the effect it has on how prostate cancer survivors and their care team experience survivorship care. We have obtained REB approval from the Trillium Health Partners (THP) hospital network for this research (THP REB ID#826).
Summary of embedded units of analyses for the evaluation of the Ned app.
Embedded unit of analysis | Data sources |
Patient, caregiver, and clinician adoption of Ned | Analytics log data on the number of patients, caregivers, and clinicians who are invited to open a Ned account, and the consequent number of Ned accounts created |
Patient acceptance of Ned | Analytics log data on the frequency, duration, depth, and breadth of patient engagement with the app |
Interviews with 20 patients poststudy | |
Web-based questionnaire assessing acceptability | |
Qualitative observation of patients using Ned to access prostate-specific antigen results and submit patient-reported outcome measures | |
Caregiver acceptance of Ned | Analytics log data on the frequency, duration, depth, and breadth of caregiver engagement with the app |
Interviews with 10 caregivers poststudy | |
Clinician acceptance of Ned | Analytics log data on the frequency, duration, depth, and breadth of clinician engagement with the app |
Wed-based questionnaire assessing system use to be completed by 10 clinicians | |
Clinical effectiveness of Ned | Five validated patient-reported outcome measures (ie, quality of life, treatment satisfaction, unmet needs, self-efficacy, anxiety) administered at baseline, 2 months, 6 months, and 12 months |
Patients must meet the following eligibility criteria to be enrolled into the study: (1) 18 years of age or older, (2) receiving care at the THP Mississauga Hospital or Credit Valley Hospital, (3) pathology report confirming prostate cancer diagnosis via transrectal, transperineal, or transurethral biopsy (standard 12-core template), (4) life expectancy more than 1 year, (5) no concomitant cancer diagnosis, (6) own a device that is compatible with the Ned app and is Web enabled through a data plan or Wi-Fi capabilities, or both (eg, laptop, desktop, tablet, smartphone), (7) able to read, write, and speak English.
Clinicians (ie, urologists, surgical oncologists, radiation oncologists) who are employed by the THP hospital group, practice at either the Mississauga Hospital or Credit Valley Hospital, and are involved in the care of patients with a diagnosis of prostate cancer are eligible to participate in this study; our research group has secured Privacy Impact Assessment and Threat Risk Assessment agreements with both hospitals. Caregivers must be paired with a patient who is enrolled in the study to be eligible for access to Ned. We broadly define caregivers here as a spouse, relative, friend, or formal caregiver of the patient who significantly contributes to their care.
The clinical champion for Ned is a urologist at THP and will initiate clinician recruitment through snowball sampling his colleagues (clinicians from urology, urologic oncology, radiation oncology, and medical oncology departments) from both THP hospitals. Clinicians who express interest in joining the study will provide the clinical champion with their primary contact number. The clinical champion will forward this number to the study coordinator (SC), who will initiate first contact with the clinician through a telephone call. The SC will provide the clinician with information about the study procedures and set a time to meet with the clinician to install Ned in their clinic. The SC will collect informed consent directly from the clinician at their clinic before setting up their Ned account. Once the SC sets up the account, the clinician will enter their medical license number to receive PSA values from OLIS and release them to patients. Clinicians can then invite and add patients from their roster into the study.
Patients will be recruited into the study through an invitation from their clinician during a visit to the clinic. If patients express interest in using Ned as part of their survivorship care resources and meet the study eligibility criteria as assessed by their clinician, their email address will be collected by the clinician and forwarded to the SC. The SC will then initiate first contact with the patient through email to confirm study eligibility, provide a study sheet with a brief description of the study (eg, study purpose and procedures, relevant risks and benefits), and obtain digital consent through a Web-based consent form that can be digitally signed by the patient. Once the patient’s digital consent is verified, they will be prompted to complete a series of demographic and baseline outcome questionnaires. The SC will receive a notification when the patient digitally consents to join the study and completes their baseline assessment, and will email the patient’s clinician to open a Ned account on the patient’s behalf. The clinician will log on to their own Ned account and invite their patient to use Ned. The patient will then have access to Ned and can invite and add their caregiver to use the app.
Caregivers will be invited to use Ned by their partners through a feature in the patient-facing version of Ned that allows them to add a caregiver to view their health data and complete PROMs on their behalf. When the patient uses this feature, the caregiver will receive an email with a link to sign up for Ned. The patient will verbally communicate a secret word to their caregiver, which the caregiver will then enter as part of the account creation process. The caregiver will then have access to Ned.
Ned study flowchart.
Clinicians will consent to participate in the Ned study by digitally signing a Web-hosted consent form provided to them by the SC in clinic. Patients will provide electronic consent by digitally signing a Web-hosted consent form provided to them by the SC through email. Caregivers will be asked to provide digital consent only if they express interest in attending a poststudy interview to share their experience of using Ned. Given that the “Add a caregiver” feature of Ned is controlled by the patient and is a native feature of the app itself, we believe that it is appropriate for patients to consent on behalf of any caregiver they add to use the app. Patients are clearly informed in their consent form that they have the ability to add a caregiver to use Ned; if they consent to using the app for the purposes of this research, they are consenting to be given access to all the features within Ned. Caregivers will be eligible to attend a poststudy interview only if their partner (the patient) has completed an interview themselves.
At study conclusion, we will ask patients and caregivers whether they would like to attend a poststudy interview to share their experience of using Ned. We will then collect a separate consent form from interested patients and caregivers at the interview. Rolling recruitment of patients who have completed the study and their paired caregiver will be done until 20 patients and 10 caregivers have been interviewed.
Patients will receive a Can $5 gift card for every study assessment completed and an additional Can $5 for completing all 4 study assessments, for a total of Can $25 as compensation for their participation in this study. If patients and caregivers choose to attend the 30-minute semistructured interview poststudy with the SC, they will receive a reimbursement of Can $25 for the cost of their parking. Clinicians will not be compensated for participating in this study.
The Ned app facilitates prostate cancer survivorship self-care and ownership of personal health data, and enables survivors to share their care plan with their caregiver and clinician to streamline care. Once patients complete the initial download of the Ned app and create an account, they are able to access all of Ned’s features. Patients can check their PSA values from directly within the app once their clinician has approved and released them. All values are first sent from OLIS to the clinician on their version of the Ned app. The clinician must review and approve these values by physically clicking on a button that confirms their attestation to the validity and appropriateness of the value. Only after the clinician has approved and released the PSA value to their patient does it get sent to the patient, who can then view it on their version of the Ned app.
Patients will have scheduled monthly PROMs, specifically the Expanded Prostate Cancer Index Composite (EPIC-26) [
The app also provides patients with access to Ned’s Notes, a curated feed of educational content coupled with upcoming local social events related to prostate cancer survivorship care. Patients are able to add caregivers to their account; when granted access, caregivers can complete PROMs on behalf of the patient, view PSA results, and take freehand notes on observable health concerns. The centralized hosting of laboratory and PROMs data enables both patients and clinicians to have access to shared data, in the hopes of fostering greater informed decision making. The Ned clinician app and dashboard will provide clinicians with a list of alerts, both for out-of-range PSA and for PROM values. Clinicians can drill down into individual patient profiles and view their PSA values over time. They are also able to view all submitted PROM values and are alerted to outlier trends that require clinical intervention.
Screenshots of the Ned app. (a) Graphical view of prostate-specific antigen (PSA) results; (b) Ned’s Notes; and (c) the Expanded Prostate Cancer Index Composite (EPIC-26) survey.
Our decision to study the adoption of Ned by patients, caregivers, and clinicians was informed by previous pivotal evaluative work on the adoption, nonadoption, and abandonment of a personal electronic health record in the United Kingdom. Greenhalgh et al’s [
To achieve effective health promotion using health technology, it is essential that patients perceive a given treatment to be an acceptable and welcome addition to their care [
Our justifications for these decisions are as follows: hedonic motivation, defined as enjoyment while using technology, is not a relevant construct to evaluate Ned given that the app was designed to deliver PSA results, which are a source of anxiety for many prostate cancer survivors [
While a substantial amount of research has been done to address the functional impairments caused by prostate cancer treatments, less emphasis has been placed on alleviating the psychological and emotional barriers faced by survivors throughout their survivorship experience. There is recognition that, while overall satisfaction with prostate cancer follow-up care is high, the presence of problematic treatment-related side effects is associated with higher psychological morbidity, poorer self-efficacy, greater unmet needs, and poorer overall health status [
Data collection schedule for outcomes measures.
Measure | Data collection method | n | Baseline | 2 months | 6 months | 12 months |
Use data | Log data analytics software | 610 | Throughout study duration | |||
Demographic data | Web-based questionnaire | 400 | x | |||
Clinical data | Chart review | 400 | x | |||
Credibility/Expectancy questionnaire | Web-based questionnaire | 400 | x | |||
Expanded Prostate Cancer Composite | Web-based questionnaire | 400 | x | x | x | x |
Prostate Cancer-Related Quality of Life Scales | Web-based questionnaire | 400 | x | |||
Service Satisfaction Scale for Cancer Care | Web-based questionnaire | 400 | x | x | x | x |
Supportive Care Needs Survey | Web-based questionnaire | 400 | x | x | x | x |
Self-Efficacy for Managing Chronic |
Web-based questionnaire | 400 | x | x | x | x |
Memorial Anxiety Scale for Prostate Cancer | Web-based questionnaire | 400 | x | x | x | x |
Modified UTAUTa survey | Web-based questionnaire | 400 | x | |||
Patient qualitative interviews | (1) Semistructured live interview (2) Qualitative observation of app use | 20 | x | |||
Caregiver qualitative interviews | Semistructured live interview | 10 | x | |||
Clinician System and Use Assessment Survey | Web-based questionnaire | 10 | x |
aUTAUT: unified theory of acceptance and use of technology.
We will assess prostate cancer-specific quality of life and functional recovery after treatment using the EPIC-26 [
We will assess patients’ satisfaction with their treatment outcomes using a cancer care-specific adaptation of the Service Satisfaction Scale [
We will use the 34-item Supportive Care Needs Survey Short Form (SCNS-SF34) [
The Self-Efficacy for Managing Chronic Disease Scale [
We will assess the psychological difficulties faced by prostate cancer survivors using the Memorial Anxiety Scale for Prostate Cancer (MAX-CP) [
In addition to exploring the acceptability of Ned by patients, we will also ask them about their experience using the app to capture emerging themes of how it translates into a real-world setting. We will conduct semistructured interviews with 20 patients at study conclusion, alongside a qualitative observation session where we will ask patients to perform a series of tasks in Ned while observed by an evaluator. The contents of this interview will be modeled after the Greenhalgh interview framework [
The experience of prostate cancer survivorship is often a shared one, with the caregivers of survivors taking on the responsibility of advocating for greater survivorship care [
Future scalability and sustainability efforts for Ned will depend on how clinicians perceive the app and their willingness to champion its use in hospitals and homes. We will use the Canada Health Infoway System and Use Assessment Survey to assess clinician adoption, use, and satisfaction with Ned alongside information and system quality [
The appropriate sample size for single-case research depends on the specific research question being investigated and cannot be calculated in the same way as group designs such as the randomized controlled trial. In his seminal writing on case study designs, Yin recommends abandoning traditional sampling logic in favor of reflecting on the number of replications that are desired to maximize both certainty of research findings and external validity [
The audio recordings for all 30 interviews will be transcribed verbatim. Two members of our research team, (ie, a researcher and research analyst) will first separately code data from 3 patient and 3 caregiver interviews, and record insights and reflections from the data. A conventional qualitative content analysis approach will be used to code qualitative data [
Descriptive statistics will first be conducted on all variables to identify methodological data trends and parameters. We will analyze differences in baseline demographic and clinical variables using Pearson chi-square and nonparametric Wilcoxon rank sum tests. Given that we will be asking the same patients to complete repeated outcome measures across 4 time periods, we will account for autocorrelation effects through the use of linear generalized estimating equations, which are a multivariate analog of linear regression for longitudinal data and recommended for use in longitudinal repeated-measures data analyses [
(1) What is the effect of Ned on health-related outcomes over time? To assess the effect of Ned on quality of life, satisfaction with cancer care, unmet needs, self-efficacy, and prostate cancer-related level of anxiety domains, we will use generalized estimating equation models containing indicators for each study time point to assess whether the average study assessment scores at 2 months, 6 months, and 12 months were significantly different from baseline values.
(2) Is there a relationship between patient engagement with Ned and changes in health-related outcomes over time? We will explore whether patients who engage with the content and functionality of the app experience improvements in their health and well-being by building linear regression models for use and outcomes data.
(3) Is there a relationship between health-related outcomes? To understand whether changes in health-related outcomes share a similar direction and magnitude, we will perform Pearson correlation analyses to test for a correlational relationship between outcome variables.
(4) Do demographic and clinical variables affect changes in health-related outcomes? We will first conduct independent
We will triangulate both quantitative and qualitative data to develop the realist context-mechanism-outcome pattern configurations required to generate a program theory that explains how Ned is experienced by patients, caregivers, and clinicians.
We recruited our first survivor in October 2017 and expect to reach our sample size requirements by March 2018. The anticipated completion date for this work is May 2019, and we aim to disseminate study findings through peer-reviewed publications and presentations starting September 2019.
While there is mounting evidence to support the provision of digital prostate cancer survivorship care [
We have endeavored to design an evaluation that can respectfully explore the nuanced relationship between prostate cancer survivors and a technology designed to improve their survivorship care. To our knowledge, this is the first realist case study evaluation of an mHealth platform for prostate cancer survivorship care. The case study design employed for this work is supported by a mixed-methods approach to data collection [
It is acknowledged that most men with a diagnosis of prostate cancer will die
Prostate cancer survivorship care will be at the forefront of health care resource allocation by 2030. Prostate cancer survivors are set to increase in number and longevity, heightening the need for integrated survivorship solutions to provide them with optimal and durable outcomes. We believe that Ned marks the proactive start of a shift in prostate cancer care innovation. We aim for this research to explore the app’s potential to empower the survivorship experience and inform a new era of prostate cancer survivorship care.
Ned patient acceptance and use of information technology survey.
Ned patient interview questions.
Ned caregiver interview questions.
Expanded Prostate Cancer Index Composite
Memorial Anxiety Scale for Prostate Cancer
minimal clinically important difference
Ontario Laboratories Information System
patient-reported outcome measure
prostate-specific antigen
research ethics board
study coordinator
34-item Supportive Care Needs Survey Short Form
Trillium Health Partners
unified theory of acceptance and use of technology
The authors wish to thank the patients and staff of the Carlo Fidani Regional Cancer Centre at Trillium Health Partners for their participation in this study. We also recognize the Ned product team (project managers, analysts, designers, and software developers) at the Centre for Global eHealth Innovation, University Health Network, Toronto, Canada, for their ingenuity and commitment to delivering the app to prostate cancer survivors and their circle of care. Finally, we acknowledge the financial contributions of Janssen Inc, Astellas Pharma, and the Government of Canada (Federal Economic Development Agency for Southern Ontario) toward the design, development, and evaluation of Ned.
AF and University Health Network jointly own intellectual property rights to the Ned app. Under the respective agreements with their organizations, AF and JAC are entitled to personally benefit from any commercial use of the intellectual property.