RefCheck Maintenance Notice

On Monday, December 3, 2018, from 16:00-18:00 EST, RefCheck will be undergoing maintenance. RefCheck is the process where, during copyediting, all references are extracted from the manuscript file, parsed, matched against various databases (eg, PubMed and CrossRef), and automatically corrected. For more information on RefCheck, please visit our Knowledge Base.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 29.08.18 in Vol 7, No 8 (2018): August

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/8886, first published Sep 01, 2017.

This paper is in the following e-collection/theme issue:

    Original Paper

    Psychosocial Assessment Using Telehealth in Adolescents and Young Adults With Cancer: A Partially Randomized Patient Preference Pilot Study

    1Cancer Centre for Children, Children’s Hospital at Westmead, Westmead, Australia

    2Discipline of Paediatrics, School of Women’s and Children’s Health, UNSW Medicine, University of New South Wales, Sydney, Australia

    3Behavioural Sciences Unit, Kids Cancer Centre, Sydney Children’s Hospital, Randwick, Australia

    4Sydney Youth Cancer Service, Prince of Wales Hospitals, Randwick, Australia

    5Department of Research, Evaluation & Social Policy, CanTeen Australia, Newtown, Australia

    6Cancer Nursing Research Unit, University of Sydney, Sydney, Australia

    7Nelune Comprehensive Cancer Centre, Prince of Wales Hospitals, Randwick, Australia

    8Kids Cancer Centre, Sydney Children’s Hospital, Sydney, Australia

    *these authors contributed equally

    Corresponding Author:

    Ursula Margaret Sansom-Daly, BPsych (Hons), MPsych (Clin), MAPS (Clin), PhD

    Behavioural Sciences Unit

    Kids Cancer Centre

    Sydney Children’s Hospital

    Level 1 South

    High Street

    Randwick, 2031

    Australia

    Phone: 61 (02) 9382 3114

    Fax:61 (02) 9382 3372

    Email:


    ABSTRACT

    Background: Adolescent and young adults with cancer are at increased risk of psychosocial difficulties relative to their healthy peers. Current models of inpatient face-to-face psychosocial care might limit the capacity for clinicians to provide timely and personalized assessment and intervention for this group. Telehealth offers a promising alternative toward increasing access to the provision of evidence-based psychosocial assessment and treatment for adolescent and young adults with cancer.

    Objective: This pilot study aimed to assess the feasibility and acceptability for both patients and clinicians of providing a psychosocial assessment via telehealth to adolescents and young adults currently receiving treatment for cancer, relative to face-to-face delivery.

    Methods: We included patients who were aged 15-25 years, currently receiving treatment, could speak English well, and medically stable. Patients were recruited from oncology clinics or wards from 5 hospitals located across Sydney and Canberra, Australia, and allocated them to receive psychosocial assessment (Adolescent and Young Adult Oncology Psychosocial Assessment Measure) with a clinical psychologist or social worker through face-to-face or telehealth modalities using a partially randomized patient preference model. Patients completed a pre- and postassessment questionnaire comprising validated and purposely designed feasibility and acceptability indices, including the impact of technical difficulties, if patients had their own devices; number of patients who were content with their group allocation; self-reported preference of modality; Treatment Credibility and Expectations Questionnaire; and Working Alliance Inventory. Clinicians also completed a postassessment questionnaire rating their impressions of the acceptability and feasibility of intervention delivery by each modality.

    Results: Of 29 patients approached, 23 consented to participate (response rate: 79%). Participants were partially randomized to either telehealth (8/23, 35%; mean age 16.50 years, range 15-23 years; females: 4/8, 50%) or face-to-face (11/23, 62%; mean age 17 years, range 15-22 years; females: 8/11, 72%) conditions. Four participants withdrew consent because of logistical or medical complications (attrition rate: 17.4%). Most participants (6/8, 75%) in the telehealth group used their computer or iPad (2 were provided with an iPad), with minor technical difficulties occurring in 3 of 8 (37.5%) assessments. Participants in both groups rated high working alliance (Working Alliance Inventory; median patient response in the telehealth group, 74, range 59-84 and face-to-face group, 63, range 51-84) and reported positive beliefs regarding the credibility and expectations of their treatment group. Postassessment preferences between face-to-face or telehealth modalities varied. Most patients in the telehealth group (5/8, 63%) reported no preference, whereas 6 of 11 (55%) in the face-to-face group reported a preference for the face-to-face modality.

    Conclusions: Telehealth is acceptable as patient comfort was comparable across modalities, with no significant technological barriers experienced. However, patients varied in their preferred interview modality, highlighting the need to tailor the treatment to patient preference and circumstances.

    Trial Registration: Australian New Zealand Clinical Trials Registry ACTRN12614001142628; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=366609 (Archived by WebCite at http://www.webcitation.org/721889HpE)

    JMIR Res Protoc 2018;7(8):e168

    doi:10.2196/resprot.8886

    KEYWORDS

    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100



    Introduction

    There has been growing recognition that psychosocial needs of adolescent and young adult (AYA) patients with cancer aged 15-25 years are different from both adults and younger children and warrant specialized services, including timely psychosocial assessment [1,2]. A cancer diagnosis during adolescence has the potential to significantly impact many aspects of normal development, including physical, psychological, social, sexual, educational, and financial domains [3]. In addition, the AYA years are a time during which individuals are at an increased risk of developing mental health disorders, even without the severe stress of a cancer diagnosis [4]. The combination of these factors means that compared with other age groups, young people with cancer often experience more complex psychological distress and social challenges, which might require more long-term, time-intensive psychosocial assessment and intervention [5-10]. Recent studies have reported that clinical-level distress is observed in 23%-27% of AYAs within the first year postdiagnosis [6,8]. Without appropriate assessment and intervention, this might translate into serious long-term mental health risks. A recent study reported that AYAs with cancer are at a significantly higher risk of suicidal behavior (both attempts and completed attempts) in the year after, and up to 5 years postdiagnosis, compared with their peers without cancer [10].

    Besides being associated with a poorer quality of life in its own right [11], elevated distress among AYAs with cancer is also likely to have adverse impact on other clinical factors, such as treatment adherence [12], and their ability to communicate effectively about their symptoms, which might delay or exacerbate treatment complications [13,14]. In addition, distress contributes to delays in the postcancer treatment of AYAs reintegrating into normal life, potentially affecting schooling or work and relationships for years into survivorship [15,16]. Ensuring that AYAs with cancer suffering from elevated distress have access to the services they need (eg, psycho-oncology) at the time they need it is therefore important.

    AYAs with cancer have unique psychosocial needs [17], and the provision of quality care needs to respect their preferences and provide appropriate emotional support, information, and physical comfort. The experience AYAs have with their care affects their functional, emotional, and social adjustment [18]. The provision of timely psychosocial assessment and intervention for AYAs with cancer is another fundamental component of best practice care [3,19] and is recommended as part of gold standard AYA clinical care in several international jurisdictions, including the UK and Australia [20]. However, considerable barriers exist for this group of patients accessing assessment and intervention. Although Australian rural and remote patients with cancer often need to travel long distances for specialist care, at great expense, time, and inconvenience to their jobs and family [21-23], AYAs are at a particular disadvantage. Many hospitals lack specialist AYA services or health professionals with AYA expertise, and the available specialized services are typically located in metropolitan centers [24,25]. AYAs are a dispersed population, and sometimes live a great distance from their treating hospital. These barriers sometimes mean patients avoid traveling for less “urgent” or compulsory aspects of their care altogether (such as receiving psychosocial assessment or intervention), potentially affecting clinical outcomes.

    One method of ensuring higher access to appropriate psychosocial assessment and intervention in AYA patients with cancer is telehealth. Telehealth “involves the use of modern information technology, especially two-way interactive audio/video communications, computers, and telemetry (ability to exchange data), to deliver health (and mental health) services to remote patients” [26]. Previous reviews have highlighted that telehealth might allow for the provision of specialist assessment and intervention to previously inaccessible and remote populations. Telehealth is also valuable in urban areas, especially for those who find it difficult to travel because of logistical or treatment-related constraints. Moreover, providing care using telehealth stands to benefit health providers, as Web-based consultations might reduce clinicians’ travel time costs when caring for patients in diverse locations and increase the efficiency of limited clinician resources, facilitating an increased capacity in service delivery [27]. Finally, AYAs can be difficult to engage in psychosocial services because of barriers such as the stigma of accessing mental health services [28]. Telehealth offers flexibility to the delivery modality of psychosocial care and might assist in enhancing their engagement, by allowing them to talk to a mental health clinician in a truly private setting, and not being seen by a clinician in a ward or an outpatient clinic.

    A key part of providing quality psychosocial care to AYAs is regular psychosocial screening and assessment, which enables members of the health care team to identify patients in distress, as well as those at risk of poor psychosocial outcomes, while detecting specific unmet needs that could be exacerbating patient distress. Ongoing assessment throughout treatment ensures unmet needs are addressed, even when those needs change [2,29]. Telehealth offers a promising avenue to ensure that adequate psychosocial assessment is delivered to AYA patients at appropriate time points throughout their cancer journey and into survivorship.

    At present, no research has specifically examined the feasibility and acceptability of using telehealth to provide psychosocial assessment for AYAs with cancer. However, in the intervention literature, several studies have suggested that AYA patients might be amenable to receiving psychosocial assessment through telehealth [30,31]. The provision of psychosocial support using telehealth in adult patients with cancer has been found to be satisfactory and feasible [32]. Telehealth interventions can achieve therapeutic alliance and rapport (ie, the relational bond between a clinician and a patient) equivalent to face-to-face therapy [33], and telehealth interventions for childhood and AYA cancer survivors implemented by phone, website, and Facebook have demonstrated feasibility and acceptability [34-38]. Finally, telehealth interventions have shown similar efficacy to face-to-face intervention in improving the quality of life of cancer survivors [39]. Despite these promising findings, no previous studies have investigated the feasibility or acceptability of using telehealth technology to provide psychosocial assessments to AYA patients regarding the treatment of cancer; this is critical to establish given that the literature to date has been mixed regarding whether telehealth is an appropriate, acceptable, or feasible modality through which psychologically distressed patients’ needs can (or should) be met [40,41]. As part of one of the first clinical consultations during which a patient’s distress, illness adjustment, and mental health history might be fully explored, establishing the feasibility and acceptability of telehealth for psychological assessment is crucial to future development and expansion of AYAs’ access to these services as part of best practice clinical care.

    This study aimed to evaluate the feasibility and acceptability of telehealth-delivered psychosocial assessment among AYAs undergoing treatment for cancer compared with patients receiving face-to-face assessment. Both patients and clinicians provided feedback for this evaluation. Specifically, the objectives were to assess the feasibility of using telehealth to deliver psychosocial assessments to AYAs during cancer treatment by examining the frequency of technical difficulties experienced in the telehealth group and how many patients in the telehealth group had access to their own devices compared with how many required one. The acceptability of using telehealth to deliver psychosocial assessments to AYAs during cancer treatment was determined by examining (1) how many patients were content with their group allocation and how many patients requested the alternative modality; (2) patients’ subjective frustration with technical difficulties if or when these occurred in the telehealth condition; (3) self-reported preference of modality for all patients and to explore reasons for the stated preference; (4) patient outcome expectations and credibility beliefs about receiving the psychosocial assessment via telehealth or face-to-face; (5) patient-reported levels of working alliance across both telehealth and face-to-face; and (6) clinicians’ impressions of patients’ engagement, comfort, rapport, and openness across both modalities. We hoped that indexing these feasibility and acceptability indices could directly inform the design and planning of a future, larger randomized trial of telehealth-delivered psychosocial assessment for AYA patients with cancer across Australia.


    Methods

    Participants

    This study was approved by the Sydney Children’s Hospitals Network Human Research Ethics Committee. We recruited patients (both inpatients and outpatients) from the following 5 sites in the state of New South Wales (NSW), Australia: Westmead Hospital and The Children’s Hospital at Westmead (Western Sydney), Prince of Wales Hospital and Sydney Children’s Hospital (East Sydney), and Canberra Hospital (Canberra). While the 5 sites are situated in urban centers, they capture both urban and rural patients. Patients were eligible to participate in this study if they had a cancer diagnosis, were on treatment, and were aged 15-25 years. Patients presenting with skin cancer diagnoses were ineligible to participate. Consistent with the New South Wales Health Policy and the clinical care model provided within the AYA services the study recruited from, patients aged <18 years were invited to provide their consent, independent of a legal parent or guardian. The additional inclusion criteria included speaking and being able to read English well and being medically stable. Patients were determined to be medically stable using clinical judgment, liaising with the treating team as appropriate. If patients in the telehealth group did not have access to a necessary device (eg, a computer or an iPad), they were provided one. No compensation was offered for participation in this study. Furthermore, 3 clinicians were involved in delivering the study, one at each site—two clinical psychologists and one social worker.

    Design

    This pilot study was a partially randomized patient preference trial (ACTRN12614001142628) [42-44], which allowed participants to opt out of the randomized allocation and choose their preferred group. This design enables individual differences or biases to be accounted for among those who elect to be randomized, while also being better in assessing the manner in which clinical services are offered and selected by patients in real clinical contexts. We randomized patients who consented to participate in the study to receive their psychosocial assessment via face-to-face or telehealth modalities. If patients were not comfortable being randomized, they were offered the choice to select their preferred assessment modality.

    Procedure

    A Masters-level clinical psychologist or social worker approached patients in person bedside or before or after a clinic appointment for introducing the study and providing the study information form. This necessitated patients having some degree of face-to-face contact with the clinician before the assessment, regardless of the group allocation. The clinicians had limited previous experiences with using telehealth in a clinical setting and were provided training in the use of telehealth and the software before using it with patients. Following informed consent, patients indicated whether they were comfortable being randomized to either condition. Based on their decision, patients were either randomized (using a simple randomization method with random numbers generated by independent personnel) or chose to receive their psychosocial assessment via face-to-face or telehealth. Patients then completed a preassessment questionnaire battery.

    Next, a Masters-level clinical psychologist or social worker assessed patients within 4 weeks using the AYA Oncology Psychosocial Assessment Measure, developed by CanTeen [45]. Patients allocated to telehealth were joined to Web-based videoconferencing software (Cisco Webex), a secure, password-protected videoconferencing application. Immediately following the assessment, care plans were developed collaboratively with patients, as per the AYA Oncology Psychosocial Care Manual [45].

    Then, patients immediately completed a postassessment questionnaire battery. Both pre- and postassessment batteries were completed either on paper or online, depending on the patient’s preference. If a patient was identified to be experiencing elevated distress or was assessed to be at acute risk (eg, experiencing suicidal ideation with intent or plan), the clinician responded as appropriate depending on the level of risk and the modality through which the patient was being assessed. This study utilized the same risk management protocol as reported in other recent Australian trials using videoconferencing, which have been shown to be appropriate in terms of screening for and managing mental health risks [41]. Specifically, if assessed face-to-face, risk management procedures were in keeping with standard clinical care (eg, walking patient to emergency, contacting a general practitioner [GP] or community crisis team etc). In contrast, risk management strategies for patients assessed via telehealth were determined by the level of risk. The protocol included actions such as confirming the AYAs’ location and contacting 000 (Australian emergency phone number), liaising with the AYA’s GP or community crisis team, contacting the AYA’s primary caretaker and treating team, and providing the AYA with crisis telephone numbers to contact if they begin to feel less safe and encourage them to talk to their GP.

    Psychosocial Assessment

    The AYA Oncology Psychosocial Assessment Measure [45] is a modified version of the HEADSS (Home, Education/Employment, peer group Activities, Drugs, Sexuality, and Suicide/depression) assessment [46], a widely used adolescent psychosocial assessment measure administered by semistructured clinical interview, adapted to suit the circumstance and needs of AYA patients with cancer. It includes the assessment of home environment, education or employment status, social history, drug or alcohol use, sexuality, and mental health status (eg, suicidal or depressed) [45]. Furthermore, the Psychosocial Measure informs the development of a care plan with patients, which might involve ongoing psychosocial support or referral to appropriate services [45].

    Measures

    Demographics

    The preassessment questionnaire battery included demographic information, including age, gender, employment status, diagnosis, and treatments received. We included the Youth Satisfaction Questionnaire [47], measuring satisfaction with overall psychosocial care, in the preassessment battery; this measure has adequate internal consistency [47] and has previously been used among AYAs with cancer [48]. Scores range from 3 to 9, with higher scores denoting higher patient satisfaction.

    Feasibility

    Patients were asked to report whether technical difficulties occurred, and if so, how many minutes of interruption these caused. Our benchmark for feasibility was the resolution of any technical difficulties within 5 minutes (assessed in the postassessment battery).

    Patients in the telehealth group also reported whether they had their own device or not. Our benchmark for feasibility was for ≥75% patients to have their own device (assessed in the postassessment battery).

    In terms of recruitment rate and attrition, our benchmark for feasibility was for >50% patients approached to consent and participate in this study (assessed in the postassessment battery).

    Acceptability

    We used the number of patients who were content with their group allocation versus the number of patients who requested an alternative modality as an index of acceptability. Our benchmark for acceptability was for 80% patients to be content with their group allocation (assessed in the postassessment battery).

    Patients were asked to rate how frustrated they were if they experienced technical difficulties on a scale of 1-10, where an average rating of ≤3 was deemed acceptable (assessed in the postassessment battery).

    All patients indicated self-reported preference of modality, or whether they had no preference either way, and were then asked to explain their preference in an open-ended written response. Our benchmark for acceptability was for >80% patients in the telehealth group to prefer their allocation over face-to-face or otherwise have no preference (assessed in the postassessment battery).

    Patient outcome expectations and credibility beliefs about receiving the psychosocial assessment via telehealth or face-to-face were assessed with the Treatment Credibility and Expectations Questionnaire [49] in the preassessment battery. In addition, items were modified for this study as appropriate; for example, “treatment” was replaced with “consultation over the internet.” This measure has high internal consistency and includes 6 items, of which, 4 employ a 9-point Likert scale ranging from 1 to 9 and 2 employ an 11-point Likert scale ranging from 0% to 100%; higher scores reflect higher credibility or expectations. Our benchmark for acceptability was for outcomes on this measure to be comparable across groups (assessed in the preassessment battery).

    Patients reported levels of working alliance across both telehealth and face-to-face assessed using the Working Alliance Inventory (WAI) [50], which measured AYAs’ perceptions of their relationship (ie, rapport and feeling understood) with the clinician who conducted the assessment. This measure employs a 7-point Likert scale ranging from 1 (not at all) to 7 (exactly), where the total score ranges from 12 to 84, with higher scores reflecting a stronger working alliance. Our benchmark for acceptability was for outcomes on this measure to be comparable across groups (assessed in the postassessment battery).

    Upon completion of this study, we asked patients to rate how beneficial and how burdensome the study was respectively, on a scale ranging from 1 (not at all) to 5 (very much; assessed in the preassessment battery).

    Clinicians’ impressions of patients’ engagement, comfort, rapport, and openness were assessed across both modalities, whereby clinicians completed a postassessment questionnaire asking them to rate their impression of patients’ engagement, comfort, rapport, and openness on a 10-point Likert scale ranging from 1 (low) to 10 (high). We defined acceptability as outcomes on this measure being commensurate across groups. Our definitions of feasibility and acceptability are similar to other work assessing feasibility and acceptability of telehealth-delivered psychosocial care [51] (assessed in the postassessment battery).

    Psychosocial State

    The preassessment battery included measures of current psychosocial functioning used to characterize the sample, including the Kessler Psychological Distress Scale 10 [52], measuring anxiety and depression symptoms in the past 4 weeks. Scores in this measure range from 10 to 50, where scores <20 indicate normal functioning, 20-24 indicate mild difficulties, 25-29 indicate moderate difficulties, and >30 indicate severe difficulties [52,53]. In addition, patients completed the Pediatric Quality of Life Inventory for adolescents and young adults [54], measuring AYA cancer-specific quality of life. Scores range from 0 to 100, with higher scores indicating a higher quality of life.

    Statistical Analysis

    Data were analyses using SPSS Version 24.0 (IBM Corp, Armonk, NY, USA). We calculated descriptive statistics for all measures included. When the data is nonnormally distributed, medians as ranges are reported as a measure of central tendency. Between-group quantitative analyses were not conducted because of the small sample size and associated limited power.


    Results

    Participants

    Of the 29 patients approached, 23 (79%) consented to participate and 6 (21%) declined as they were not interested in participating. In addition, 17% (4/23) participants did not complete the study following consent—2 were lost to follow-up, 1 changed mind, and 1 became too unwell and was withdrawn from the study by the investigators (this occurred following consent, but prior to any further participation in the study). Of the 19 remaining participants, 17 were randomized to either telehealth (8/17, 47%) or face-to-face (9/17, 53%) conditions, with 2 participants electing to be assessed face-to-face (Figure 1). The 2 patients who chose the face-to-face modality were collapsed in the data with other face-to-face group participants and not reported separately; this decision was made because of the small number (n=2) of patients who opted out of randomization.

    Table 1 summarizes key patient characteristics. In this study, 19 participating AYAs represented a diverse range of ages, gender, employment status, diagnosis, and treatment. All patients were undergoing treatment for their first diagnosis (ie, had not relapsed). AYAs lived a median distance of 24 (range 13-414) km from their treating hospital. As a group, AYAs’ average levels of distress according to Kessler Psychological Distress Scale 10 were in the “normal” range, although some in each group showed elevated distress (telehealth: 2/8, 25%; face-to-face: 3/11, 27%). In addition, our sample reported health-related quality of life scores in the moderate range, although their emotional or physical functioning was somewhat lower. Overall, AYAs reported very high satisfaction with their psychosocial care to date (median Youth Satisfaction Questionnaire score 8.5, range 6-9).

    Feasibility

    Technical difficulties occurred during a minority of assessments in the telehealth group (3/8, 37.5%); however, in each case, problems were resolved in <5 minutes, meeting our benchmark of feasibility. The technical difficulties that occurred were all because of inconsistent connection speeds in the context of patients’ devices not being connected to Wi-Fi but relying on the 4G mobile connection. Of telehealth participants, 6 of 8 (75%) used their own computer or iPad and the other 2 (25%) were provided with an iPad as their assessment occurred during an inpatient stay, which met our benchmark of feasibility. Of all 29 patients approached, 19 (65.5%) consented to participate and also completed all components of the study, above our feasibility target of 50%.

    Acceptability

    Almost all (17/19, 89%) patients were content with their group allocation, above our acceptability target of 80%. In the telehealth group, all 3 patients who experienced technical difficulties rated their level of frustration because of this as 1 of 10, below our benchmark of 3/10. Table 2 outlines patients’ experiences of the psychosocial assessment. Following the psychosocial assessment, 7 of 8 (87.5%) patients in the telehealth group reported a preference for telehealth or otherwise no preference between face-to-face or telehealth modalities (above our acceptability target of 80%). Evaluating differences in the modality preference across gender, among young men, 3 preferred face-to-face, 3 preferred telehealth, and 1 cited no preference. Among young women, 5 preferred face-to-face, 1 preferred telehealth, and 6 cited no preference. Table 3 outlines qualitative patients’ responses accounting for their reported preference of the assessment modality. Patients reported their perceptions of the ease and comfort associated with each modality, as well as concerns about confidentiality and familiarity with the clinician conducting the interview. Table 4 outlines patients’ expectations of the telehealth or face-to-face modality and their beliefs about the credibility of the modality. Overall, patients had similarly positive expectations for both telehealth and face-to-face modalities and rated high credibility. The median patient response on WAI was 74 (range 59-84) for the telehealth group and 63 (range 51-84) for the face-to-face group. WAI scores for young men (median 67, range 54-81) and young women (median 67, range 51-84) were comparable. In both groups, the median response to whether the study was burdensome was 1 (ie, not at all). The median response to whether the study was beneficial was higher in the telehealth group. Table 5 outlines clinicians’ reports of patients’ engagement, comfort, rapport, and openness. Overall, clinicians tended to report high levels of patient engagement and comfort across both the telehealth and face-to-face groups. Furthermore, clinicians’ ratings of technical difficulties were commensurate with patients’ ratings.

    Figure 1. The Consolidated Standards of Reporting Trials (CONSORT) flowchart of participants. F/U: follow-up.
    View this figure
    Table 1. Patient characteristics.
    View this table
    Table 2. Patients’ experience of psychosocial assessment (assessed in the postassessment battery).
    View this table
    Table 3. Qualitative patient reports explaining their preference of modality for either telehealth, face-to-face, or no preference (assessed in the postassessment battery).
    View this table
    Table 4. Treatment credibility and expectations questionnaire [49] (completed preassessment).
    View this table
    Table 5. Clinicians’ ratings (n=3) of clinicians’ experience of psychosocial assessment (completed postassessment).
    View this table

    Discussion

    Principal Findings

    This study explored whether the administration of a psychosocial assessment would be both feasible and acceptable to AYAs currently receiving cancer treatment if delivered using telehealth. Overall, the delivery of psychosocial assessments via telehealth was found to be feasible. We were able to deliver telehealth-based psychosocial assessments to a broad patient group with a range of cancer diagnoses, ages, and varied distress levels and across both young men and women. All patients randomized to the telehealth group had their own device, which they used to connect to their Web-based assessment (unless they were inpatients, in which case a device was provided). Furthermore, while some patients did experience technical difficulties during their assessment, these were resolved quickly and resulted in minimal frustration, which is in keeping with other recent telehealth studies where technical difficulties and associated dissatisfaction were minimal [51,55]. Clinicians’ scores indicated a high degree of acceptability from their perspective as well. It is important to note that clinicians did not have prior experience in using telehealth as a modality in providing clinical care, highlighting that when provided with appropriate training, telehealth is a feasible tool to implement even in settings where clinicians might not have had previous experience in doing so.

    Psychosocial assessments delivered via telehealth were also found to be acceptable. The ability to effectively deliver an inherently interpersonal service through the medium of technology has been posed as one of the key ethical tensions involved in the expansion of telehealth services [6,56]. Our findings highlighted that both clinicians and patients felt comfortable with the telehealth modality and reported comparably high levels of patients’ engagement and comfort across telehealth and face-to-face modalities. Critically, patients’ perceptions of the therapeutic collaborative relationship with the psychologist or social worker were also comparable across groups. The high proportion of patients who were willing to be randomized to either condition (telehealth or face-to-face) at the outset further emphasizes that future trials will be able to employ this design with confidence. In the context of the partially randomized patient preference design used in this study, AYAs’ openness to both models of care appears to reflect that both models of undertaking psychosocial assessments for AYA patients with cancer might be developmentally and clinically appropriate in different circumstances. Concerning gender, our results suggested that both males and females might be open to telehealth as a model of care; although our sample was quite small, this aligns with previous findings that report no impact of gender on the acceptability of telehealth technology [57,58].

    The qualitative responses showed that patients who preferred telehealth indicated this was largely because of logistical issues, whereby telehealth reduced the burden of travel and increased ease of attending an assessment. Patients who preferred face-to-face assessment indicated it was simply more comfortable and easier to engage with a clinician in person. Although telehealth was the explicit preference among a minority of AYAs, of note, almost three times as many AYAs in the telehealth group expressed having “no preference” of modality. Despite ethical concerns in the literature regarding the potential for confidentiality and security issues in telehealth [40], we found little evidence of AYAs being overly concerned around these confidentiality aspects, and it did not appear to affect our participation rates. Like previous Australian studies, we used a more secure videoconferencing platform, with password-protected sessions, and liaised with patients to ensure that the session was undertaken in a private, confidential location [41,48]. Rather, our findings indicate that while AYAs may have a general tendency to prefer face-to-face interactions with psychosocial clinicians, they may become more open to the possibilities or potential benefits of telehealth once they have tried it for themselves. This notion agrees with previous literature that has found young people exposed to telehealth reported appreciation for the privacy it allows [59], which overcomes barriers to service utilization, such as mental health stigma, and the sense of empowerment and control patients have about terminating sessions [60]. Thus, future studies should investigate whether and how patients’ (and clinicians’) attitude toward telehealth may change with exposure to it. In order to better gauge AYAs’ relative preferences for telehealth versus face-to-face delivered psychosocial services, a future trial employing a crossover design in which AYAs gain exposure to both models of care would be recommended. The variance in patients’ preferences reported in this study (and the high proportion indicating no preference) highlights the importance of clinicians engaging collaboratively with AYAs to ensure that they offer a service that suits each patient’s needs.

    Although participants’ preference for assessment modality varied, the median level of reported comfort talking about personal issues was comparable across groups. In addition, participants’ perceptions of the working relationship with their psychologist appeared equally positive regardless of their gender or how far away they lived from the hospital. These findings echo other recent studies that have also found that young people report positive perceptions of psychological interactions that take place using telehealth [61,62]. For many patients, therefore, concerns about the quality of the rapport or working relationship over telehealth might not be the primary driving factor underlying preferences between the two treatment models.

    Our sample also represented a group of AYAs with lower total health-related quality of life, and in particular lower emotional and physical functioning, relative to previously reported samples of AYA cancer survivors [11]. Telehealth technologies might be particularly helpful for individuals currently experiencing a great symptom burden and for whom additional travel to a hospital site to receive support would be difficult [18,48]. Indeed, satisfaction with telehealth is understandably very high in rural communities where the need to travel for face-to-face care is prohibitive [63]. As AYA psychosocial assessments are increasingly recognized as a key part of the best practice clinical care (Clinical Oncology Society of Australia guidelines [56]), this study shows the potential utility of incorporating telehealth into that delivery model and suggests that this would be feasible and acceptable to both patients and clinicians.

    Strengths

    This study has several strengths. To the best of our knowledge, this is the first study to explore the clinically important issue of whether using telehealth to provide psychosocial assessment is viable among AYAs with cancer. We had a high response rate and level of participation among patients approached to participate, highlighting the appeal and importance of the study to AYAs. Contrary to prior studies where AYAs’ ownership of technology was a requirement for participation [51], we were able to provide AYAs with access to technological equipment if they needed it, ensuring equity of access to AYAs with varied socioeconomic and financial resources. Furthermore, the study demonstrated the feasibility and acceptability of using a telehealth modality to conduct psychosocial assessments among patients of varying levels of psychological distress. Prior work has questioned the appropriateness of using emerging technologies, including telehealth, with distressed populations, citing safety concerns [18,64]. This study provides a first step to supporting the acceptability and feasibility of telehealth psychosocial assessment for vulnerable and distressed groups and is in line with previous work demonstrating that distress can be assessed in clinically and ethically appropriate ways using Web-based telehealth technologies 41.

    Limitations

    This study also has several limitations. First, the small sample size and nonnormal data distribution restricted quantitative between-group analyses. Second, all patients received some degree of face-to-face engagement by the clinicians regardless of group randomization, which may have lessened the differences between the two groups. This occurred in the patient recruitment and consent process and was conducted by the assessing clinician because of resource limitations. However, recent research has reported that emerging technologies, such as telehealth, are likely to be best used as “adjuncts” or supplements to routine, face-to-face clinical care [13,14,18,65-68]. As such, the mix of face-to-face and telehealth care our sample received is likely to be a good approximation of how this might be incorporated into routine practice in clinical settings. In future, studies using a “crossover” design would assist in more rigorously comparing the relative feasibility and acceptability of these two modalities, whereby participants would have exposure to both face-to-face and telehealth modalities before providing feedback on these. In addition, although the psychosocial assessment used is manualized and clinicians engaged in self-reported fidelity checks where they ticked off each component of the assessment for each patient, we did not include a rigorous assessment of clinicians’ fidelity to the AYA psychosocial assessment as one of our outcome measures; consequently, it is possible that the face-to-face and telehealth arms also differed in the delivery of this assessment between clinicians. Finally, as a pilot study, this study has inherent limitations with respect to the generalizability of the results without a future larger scale trial being undertaken.

    Conclusions

    Despite these limitations, this study represents the first attempt to investigate the acceptability and feasibility of telehealth in providing a psychosocial assessment to AYAs on cancer treatment, as well as clinicians’ experience of patients’ response to telehealth. Overall, telehealth was well received, patient comfort was comparable across modalities, and no significant technological barriers were experienced. While some patients will indicate a preference for a face-to-face assessment, a preference that needs to be respected, telehealth offers a feasible and acceptable alternative for patients who prefer it or otherwise would be burdened by accessing face-to-face assessment.

    Acknowledgments

    The authors wish to acknowledge contributions to this study from a number of collaborating AYA clinicians, including Rachel Bailey, Pip Eve, Katie Carlson, Tora Sibbald, Alison Baker, Robert Battisti, Cath O’Dwyer, and Fiona Maguire. We also wish to thank Holly Evans, Sarah Ellis, Brittany McGill, Stephanie Konings, Ellen Yanxiang Gan, and Sophie Champion from The Behavioural Sciences Unit for their assistance in preparing this manuscript for publication. This work is supported by Kids Cancer Alliance, a CINSW Translational Cancer Research Centre (15/TRC/1-04). US-D is supported by an Early Career Fellowship from the Cancer Institute of New South Wales (ID: 14/ECF/1-11) and an Early Career Fellowship from the National Health and Medical Research Council of Australia (APP1111800). The Behavioural Sciences Unit is supported by the Kids with Cancer Foundation and the Kids Cancer Alliance as well as a Cancer Council New South Wales Program Grant PG16-02 with the support of the Estate of the late Harry McPaul.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    CONSORT‐EHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 532KB

    References

    1. Bellacosa A, Genuardi M, Anti M, Viel A, de Leon M. Hereditary nonpolyposis colorectal cancer: Review of clinical, molecular genetics, and counseling aspects. Am. J. Med. Genet 1996 Apr 24;62(4):353-364. [CrossRef] [Medline]
    2. Palmer S, Patterson P, Thompson K. A national approach to improving adolescent and young adult (AYA) oncology psychosocial care: the development of AYA-specific psychosocial assessment and care tools. Palliat Support Care 2014 Jun;12(3):183-188. [CrossRef] [Medline]
    3. Zebrack B, Santacroce SJ, Patterson PA, Gubin A. Adolescents and Young Adults with Cancer: A Biopsychosocial Approach. In: Abrams AN, Muriel AC, Wiener L, editors. Pediatric Psychosocial Oncology: Textbook for Multidisciplinary Care. Switzerland: Springer International Publishing; 2016:199-218.
    4. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005 Jun;62(6):593-602. [CrossRef] [Medline]
    5. Australian Institute of Health and Welfare. Young Australians: Their Health and Wellbeing. -Canberra: -Australian Institute of Health and Welfare; 2011. Young Australians: their health and wellbeing 2011   URL: https:/​/www.​aihw.gov.au/​reports/​children-youth/​young-australians-their-health-and-wellbeing-2011/​contents/​table-of-contents
    6. Kwak M, Zebrack B, Meeske K, Embry L, Aguilar C, Block R, et al. Trajectories of psychological distress in adolescent and young adult patients with cancer: a 1-year longitudinal study. J Clin Oncol 2013 Jun 10;31(17):2160-2166. [CrossRef] [Medline]
    7. McCarthy MC, McNeil R, Drew S, Dunt D, Kosola S, Orme L, et al. Psychological Distress and Posttraumatic Stress Symptoms in Adolescents and Young Adults with Cancer and Their Parents. J Adolesc Young Adult Oncol 2016 Dec;5(4):322-329. [CrossRef] [Medline]
    8. Zebrack BJ, Corbett V, Embry L, Aguilar C, Meeske KA, Hayes-Lattin B, et al. Psychological distress and unsatisfied need for psychosocial support in adolescent and young adult cancer patients during the first year following diagnosis. Psychooncology 2014 Nov;23(11):1267-1275. [CrossRef] [Medline]
    9. Clerici CA, Massimino M, Casanova M, Cefalo G, Terenziani M, Vasquez R, et al. Psychological referral and consultation for adolescents and young adults with cancer treated at pediatric oncology unit. Pediatr Blood Cancer 2008 Jul;51(1):105-109. [CrossRef] [Medline]
    10. Sutherland H, Llewellyn-Thomas H, Lockwood G, Tritchler D, Till J. Cancer patients: their desire for information and participation in treatment decisions. J R Soc Med 1989 May;82(5):260-263 [FREE Full text] [Medline]
    11. Smith A, Bellizzi K, Keegan T, Zebrack B, Chen V, Neale A, Hamilton, et al. Health-related quality of life of adolescent and young adult patients with cancer in the United States: the Adolescent and Young Adult Health Outcomes and Patient Experience study. J Clin Oncol 2013 Jun 10;31(17):2136-2145 [FREE Full text] [CrossRef] [Medline]
    12. Butow P, Palmer S, Pai A, Goodenough B, Luckett T, King M. Review of adherence-related issues in adolescents and young adults with cancer. J Clin Oncol 2010 Nov 10;28(32):4800-4809. [CrossRef] [Medline]
    13. Lin M, Sansom-Daly U, Wakefield C, McGill B, Cohn R. Health Literacy in Adolescents and Young Adults: Perspectives from Australian Cancer Survivors. J Adolesc Young Adult Oncol 2017 Mar;6(1):150-158. [CrossRef] [Medline]
    14. May E, McGill B, Robertson E, Anazodo A, Wakefield C, Sansom-Daly U. Adolescent and Young Adult Cancer Survivors' Experiences of the Healthcare System: A Qualitative Study. J Adolesc Young Adult Oncol 2018 Feb;7(1):88-96. [CrossRef] [Medline]
    15. Sansom-Daly U, Wakefield C. Distress and adjustment among adolescents and young adults with cancer: an empirical and conceptual review. Transl Pediatr 2013 Oct;2(4):167-197 [FREE Full text] [CrossRef] [Medline]
    16. Vetsch J, Wakefield C, McGill B, Cohn R, Ellis S, Stefanic N, et al. Educational and vocational goal disruption in adolescent and young adult cancer survivors. Psychooncology 2018 Feb;27(2):532-538. [CrossRef] [Medline]
    17. Morgan S, Davies S, Palmer S, Plaster M. Sex, drugs, and rock 'n' roll: caring for adolescents and young adults with cancer. J Clin Oncol 2010 Nov 10;28(32):4825-4830. [CrossRef] [Medline]
    18. Skaczkowski G, White V, Thompson K, Bibby H, Coory M, Pinkerton R, et al. Do Australian adolescents' and young adults' experiences of cancer care influence their quality of life? Psychooncology 2018 Mar;27(3):990-997. [CrossRef] [Medline]
    19. CanTeen. Australian Youth Cancer Framework for Adolescents and Young Adults with Cancer. Australia: CanTeen; 2017.   URL: https://www.canteen.org.au/youth-cancer/resources/aycf/
    20. Patterson P, Hardman F, Cheshire J, Sansom-Daly U. Balancing risk with resilience: Using holistic screening and assessment tools effectively with adolescents and young adults with cancer. In: Olsen P, Smith S, editors. Nursing Adolescents and Young Adults with Cancer: Developing Knowledge, Competence and Best Practice. Cham: Springer International Publishing; 2018:95-119.
    21. Wakefield CE, McLoone JK, Evans NT, Ellis SJ, Cohn RJ. It's more than dollars and cents: the impact of childhood cancer on parents' occupational and financial health. J Psychosoc Oncol 2014;32(5):602-621. [CrossRef] [Medline]
    22. Daniel G, Wakefield CE, Ryan B, Fleming CA, Levett N, Cohn RJ. Accommodation in pediatric oncology: parental experiences, preferences and unmet needs. Rural Remote Health 2013;13(2):2005 [FREE Full text] [Medline]
    23. Miedema B, Easley J, Robinson L. Comparing urban and rural young adult cancer survivors' experiences: a qualitative study. Rural Remote Health 2013;13(2):2324 [FREE Full text] [Medline]
    24. Anazodo A, Chard J. Medical and psychosocial challenges in caring for adolescent and young adult patients with cancer. Cancer Forum 2013;37(1):23-26 [FREE Full text]
    25. Albritton KH, Eden T. Access to care. Pediatr Blood Cancer 2008 May;50(5 Suppl):1094-1098. [CrossRef] [Medline]
    26. Darkins AW, Carey MA. Telemedicine and Telehealth: Principles, Policies, Performances and Pitfalls. New York, USA: Springer Publishing Company; 2000.
    27. Jang-Jaccard J, Nepal S, Alem L, Li J. Barriers for delivering telehealth in rural australia: a review based on Australian trials and studies. Telemed J E Health 2014 May;20(5):496-504. [CrossRef] [Medline]
    28. Olsson C, Boyce M, Toumbourou J, Sawyer S. The Role of Peer Support in Facilitating Psychosocial Adjustment to Chronic Illness in Adolescence. Clin Child Psychol Psychiatry 2016 Jul 26;10(1):78-87. [CrossRef]
    29. Zebrack B, Isaacson S. Psychosocial care of adolescent and young adult patients with cancer and survivors. J Clin Oncol 2012 Apr 10;30(11):1221-1226. [CrossRef] [Medline]
    30. Ye X, Bapuji S, Winters S, Struthers A, Raynard M, Metge C, et al. Effectiveness of internet-based interventions for children, youth, and young adults with anxiety and/or depression: a systematic review and meta-analysis. BMC Health Serv Res 2014 Jul 18;14:313 [FREE Full text] [CrossRef]
    31. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual Research Review: Digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry 2017 Apr;58(4):474-503. [CrossRef]
    32. Shepherd L, Goldstein D, Whitford H, Thewes B, Brummell V, Hicks M. The utility of videoconferencing to provide innovative delivery of psychological treatment for rural cancer patients: results of a pilot study. J Pain Symptom Manage 2006 Nov;32(5):453-461. [CrossRef] [Medline]
    33. Andersson G, Paxling B, Wiwe M, Vernmark K, Felix CB, Lundborg L, et al. Therapeutic alliance in guided internet-delivered cognitive behavioural treatment of depression, generalized anxiety disorder and social anxiety disorder. Behav Res Ther 2012 Sep;50(9):544-550. [CrossRef] [Medline]
    34. Judge SS, Asmus K, Kadan-Lottick N, Grey M. Feasibility and preliminary outcomes from a pilot study of coping skills training for adolescent--young adult survivors of childhood cancer and their parents. J Pediatr Oncol Nurs 2010;27(1):10-20. [CrossRef] [Medline]
    35. Rabin C, Pinto B, Fava J. Randomized Trial of a Physical Activity and Meditation Intervention for Young Adult Cancer Survivors. J Adolesc Young Adult Oncol 2016 Mar;5(1):41-47. [CrossRef] [Medline]
    36. Seitz DCM, Knaevelsrud C, Duran G, Waadt S, Goldbeck L. Internet-based psychotherapy in young adult survivors of pediatric cancer: feasibility and participants' satisfaction. Cyberpsychol Behav Soc Netw 2014 Sep;17(9):624-629. [CrossRef] [Medline]
    37. Valle CG, Tate DF, Mayer DK, Allicock M, Cai J. A randomized trial of a Facebook-based physical activity intervention for young adult cancer survivors. J Cancer Surviv 2013 Sep;7(3):355-368 [FREE Full text] [CrossRef] [Medline]
    38. Maurice-Stam H, Scholten L, de GE, van DZR, Conijn B, Last B, et al. Feasibility of an online cognitive behavioral-based group intervention for adolescents treated for cancer: a pilot study. J Psychosoc Oncol 2014;32(3):310-321. [CrossRef]
    39. Freeman LW, White R, Ratcliff CG, Sutton S, Stewart M, Palmer JL, et al. A randomized trial comparing live and telemedicine deliveries of an imagery-based behavioral intervention for breast cancer survivors: reducing symptoms and barriers to care. Psychooncology 2015 Aug;24(8):910-918 [FREE Full text] [CrossRef] [Medline]
    40. Sansom-Daly UM, Wakefield CE, McGill BC, Wilson HL, Patterson P. Consensus Among International Ethical Guidelines for the Provision of Videoconferencing-Based Mental Health Treatments. JMIR Ment Health 2016 May 18;3(2):e17 [FREE Full text] [CrossRef] [Medline]
    41. Sansom-Daly UM, Wakefield CE, McGill BC, Patterson P. Ethical and Clinical Challenges Delivering Group-based Cognitive-Behavioural Therapy to Adolescents and Young Adults with Cancer Using Videoconferencing Technology. Australian Psychologist 2015 Jul 20;50(4):271-278. [CrossRef]
    42. Coward DD. Partial randomization design in a support group intervention study. West J Nurs Res 2002 Jun;24(4):406-421. [CrossRef] [Medline]
    43. Preference Collaborative Review Group. Patients' preferences within randomised trials: systematic review and patient level meta-analysis. BMJ 2008 Oct 31;337:a1864 [FREE Full text] [Medline]
    44. Sedgwick P. What is a patient preference trial? BMJ 2013 Oct 04;347(oct04 2):f5970-f5970. [CrossRef]
    45. CanTeen. Adolescent and Young Adult Oncology Psychosocial Care Manual. Australia: CanTeen; 2011.
    46. Goldenring J, Cohen E. Getting into adolescents heads. Contemporary Pediatrics 1988;5:75-90 [FREE Full text]
    47. Stüntzner-Gibson D, Koren P, DeChillo N. The Youth Satisfaction Questionnaire: What kids think of services. Families in Society 1995;76(10):616-624.
    48. Sansom-Daly U, Wakefield C, Bryant R, Butow P, Sawyer S, Patterson P, et al. Online group-based cognitive-behavioural therapy for adolescents and young adults after cancer treatment: a multicenter randomised controlled trial of Recapture Life-AYA. BMC Cancer 2012 Aug 03;12:339 [FREE Full text] [CrossRef] [Medline]
    49. Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 2000 Jun;31(2):73-86. [Medline]
    50. Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. J Counsel Psychol 1989;36(2):223-233. [CrossRef]
    51. Campo RA, Bluth K, Santacroce SJ, Knapik S, Tan J, Gold S, et al. A mindful self-compassion videoconference intervention for nationally recruited posttreatment young adult cancer survivors: feasibility, acceptability, and psychosocial outcomes. Support Care Cancer 2017 Dec;25(6):1759-1768. [CrossRef] [Medline]
    52. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SLT, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002 Aug;32(6):959-976. [Medline]
    53. Andrews G, Slade T. Interpreting scores on the Kessler Psychological Distress Scale (K10). Aust N Z J Public Health 2001 Dec;25(6):494-497. [Medline]
    54. Ewing JE, King MT, Smith NF. Validation of modified forms of the PedsQL generic core scales and cancer module scales for adolescents and young adults (AYA) with cancer or a blood disorder. Qual Life Res 2009 Mar;18(2):231-244. [CrossRef] [Medline]
    55. Sabesan S, Simcox K, Marr I. Medical oncology clinics through videoconferencing: an acceptable telehealth model for rural patients and health workers. Intern Med J 2012 Jul;42(7):780-785. [CrossRef] [Medline]
    56. Kaplan B, Litewka S. Ethical challenges of telemedicine and telehealth. Cambridge Quarterly of Healthcare Ethics 2008;17(4).
    57. Sørensen JFL. Attitudes toward telehealth use among rural residents: a Danish survey. J Rural Health 2008;24(3):330-335. [CrossRef] [Medline]
    58. Tsan J, Day S. Personality and Gender as Predictors of Online Counseling Use. Journal of Technology in Human Services 2008 Sep 08;25(3):39-55. [CrossRef]
    59. Bischoff R, Hollist C, Smith C, Flack P. Addressing the Mental Health Needs of the Rural Underserved: Findings from a Multiple Case Study of a Behavioral Telehealth Project. Contemporary Family Therapy 2004 Jun;26(2):179-198. [CrossRef]
    60. Grealish A, Hunter A, Glaze R, Potter L. Telemedicine in a child and adolescent mental health service: participants' acceptance and utilization. J Telemed Telecare 2005;11 Suppl 1:53-55. [CrossRef] [Medline]
    61. Lu D, Fall K, Sparén P, Ye W, Adami H, Valdimarsdóttir U, et al. Suicide and suicide attempt after a cancer diagnosis among young individuals. Ann Oncol 2013 Dec;24(12):3112-3117. [CrossRef] [Medline]
    62. Sucala M, Schnur J, Constantino M, Miller S, Brackman E, Montgomery G. The therapeutic relationship in e-therapy for mental health: a systematic review. J Med Internet Res 2012 Aug 02;14(4):e110 [FREE Full text] [CrossRef] [Medline]
    63. Starling J, Rosina R, Nunn K, Dossetor D. Child and Adolescent Telepsychiatry in New South Wales: Moving Beyond Clinical Consultation. Australas Psychiatry 2003 Oct;11(1_suppl):S117-S121. [CrossRef]
    64. Rathert C, Williams E, McCaughey D, Ishqaidef G. Patient perceptions of patient-centred care: empirical test of a theoretical model. Health Expect 2015 Apr;18(2):199-209 [FREE Full text] [CrossRef] [Medline]
    65. Eland-de KP, van OH, Vergouwe-Meijer A, Bruijnzeel-Koomen C, Ros W. A systematic review of the effects of e-health on chronically ill patients. J Clin Nurs 2011 Nov;20(21-22):2997-3010. [CrossRef] [Medline]
    66. Zhai Y, Zhu W, Cai Y, Sun D, Zhao J. Clinical- and cost-effectiveness of telemedicine in type 2 diabetes mellitus: a systematic review and meta-analysis. Medicine (Baltimore) 2014 Dec;93(28):e312 [FREE Full text] [CrossRef] [Medline]
    67. Widmer R, Collins N, Collins C, West C, Lerman L, Lerman A. Digital health interventions for the prevention of cardiovascular disease: a systematic review and meta-analysis. Mayo Clin Proc 2015 Apr;90(4):469-480 [FREE Full text] [CrossRef] [Medline]
    68. Pietrzak E, Cotea C, Pullman S, Nasveld P. Self-management and rehabilitation in osteoarthritis: is there a place for internet-based interventions? Telemed J E Health 2013 Oct;19(10):800-805. [CrossRef] [Medline]


    Abbreviations

    AYA: adolescent and young adult
    WAI: Working Alliance Inventory


    Edited by A Powell; submitted 01.09.17; peer-reviewed by C Valle, C Maheu; comments to author 15.11.17; revised version received 05.03.18; accepted 26.04.18; published 29.08.18

    ©John Alexander Chalmers, Ursula Margaret Sansom-Daly, Pandora Patterson, Geoffrey McCowage, Antoinette Anazodo. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 29.08.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.