This paper is in the following e-collection/theme issue:

Grant Proposals (eHealth, funded) (80) Wearable Devices and Sensors (900) Methods and Feasibility Studies (383) Natural Language Processing (704) Clinical Information and Decision Making (1426)

Published on in Vol 11, No 5 (2022): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/37975, first published .
Exploring the Use of Wearable Sensors and Natural Language Processing Technology to Improve Patient-Clinician Communication: Protocol for a Feasibility Study

Exploring the Use of Wearable Sensors and Natural Language Processing Technology to Improve Patient-Clinician Communication: Protocol for a Feasibility Study

Exploring the Use of Wearable Sensors and Natural Language Processing Technology to Improve Patient-Clinician Communication: Protocol for a Feasibility Study

Authors of this article:

Virginia LeBaron1 Author Orcid Image ;   Mehdi Boukhechba2 Author Orcid Image ;   James Edwards1 Author Orcid Image ;   Tabor Flickinger3 Author Orcid Image ;   David Ling3 Author Orcid Image ;   Laura E Barnes2 Author Orcid Image
Article Authors Cited by (6) Tweetations (5) Metrics

    Protocol

    1School of Nursing, University of Virginia, Charlottesville, VA, United States

    2School of Engineering & Applied Science, University of Virginia, Charlottesville, VA, United States

    3School of Medicine, University of Virginia, Charlottesville, VA, United States

    Corresponding Author:

    Virginia LeBaron, BSN, MS, PhD

    School of Nursing

    University of Virginia

    225 Jeannette Lancaster Way

    Charlottesville, VA, 22908

    United States

    Phone: 1 520 4883649

    Email: vtl6k@virginia.edu


    Background: Effective communication is the bedrock of quality health care, but it continues to be a major problem for patients, family caregivers, health care providers, and organizations. Although progress related to communication skills training for health care providers has been made, clinical practice and research gaps persist, particularly regarding how to best monitor, measure, and evaluate the implementation of communication skills in the actual clinical setting and provide timely feedback about communication effectiveness and quality.

    Objective: Our interdisciplinary team of investigators aims to develop, and pilot test, a novel sensing system and associated natural language processing algorithms (CommSense) that can (1) be used on mobile devices, such as smartwatches; (2) reliably capture patient-clinician interactions in a clinical setting; and (3) process these communications to extract key markers of communication effectiveness and quality. The long-term goal of this research is to use CommSense in a variety of health care contexts to provide real-time feedback to end users to improve communication and patient health outcomes.

    Methods: This is a 1-year pilot study. During Phase I (Aim 1), we will identify feasible metrics of communication to extract from conversations using CommSense. To achieve this, clinical investigators will conduct a thorough review of the recent health care communication and palliative care literature to develop an evidence-based “ideal and optimal” list of communication metrics. This list will be discussed collaboratively within the study team and consensus will be reached regarding the included items. In Phase II (Aim 2), we will develop the CommSense software by sharing the “ideal and optimal” list of communication metrics with engineering investigators to gauge technical feasibility. CommSense will build upon prior work using an existing Android smartwatch platform (SWear) and will include sensing modules that can collect (1) physiological metrics via embedded sensors to measure markers of stress (eg, heart rate variability), (2) gesture data via embedded accelerometer and gyroscope sensors, and (3) voice and ultimately textual features via the embedded microphone. In Phase III (Aim 3), we will pilot test the ability of CommSense to accurately extract identified communication metrics using simulated clinical scenarios with nurse and physician participants.

    Results: Development of the CommSense platform began in November 2021, with participant recruitment expected to begin in summer 2022. We anticipate that preliminary results will be available in fall 2022.

    Conclusions: CommSense is poised to make a valuable contribution to communication science, ubiquitous computing technologies, and natural language processing. We are particularly eager to explore the ability of CommSense to support effective virtual and remote health care interactions and reduce disparities related to patient-clinician communication in the context of serious illness.

    International Registered Report Identifier (IRRID): PRR1-10.2196/37975

    JMIR Res Protoc 2022;11(5):e37975

    doi:10.2196/37975

    Keywords

    communication; technology; ubiquitous computing, natural language processing; cancer; palliative care 


    Background

    Effective communication is the bedrock of quality health care, but it continues to be a major problem for patients, family caregivers, health care providers, and organizations [1-4].The ramifications of poor health care communication are profound and can include medical errors [5], suboptimal symptom management [6-9], decreased quality of life for patients and caregivers [10], health care provider distress and burnout [11,12], and inappropriate health care service usage [2]. Effective communication is especially critical in the context of oncology and palliative care, when patients and their families are coping with the stressors of advanced illness and difficult symptoms, such as pain that affects up to 60%-90% of people with cancer [13-15]. Even more problematic is the reality that poor communication related to symptom management contributes to disturbing and unethical health disparities. For example, research has shown that patients from underrepresented racial and ethnic groups are significantly more likely to suffer with undertreated pain [16], die in the intensive care unit when it is not their preference [17,18], and generally experience poorer communication about their health care issues and needs [17,19-22].

    Although progress related to communication skills training for health care providers has been made, clinical practice and research gaps persist, including the following: (1) whether the effects are sustained over time [23], (2) which communication training programs are most likely to improve patient care outcomes [23], and (3) the lack of a scalable way to monitor, measure, and evaluate the implementation of communication skills in a natural clinical setting and provide real-time feedback about communication effectiveness [2,24-27]. Leaders in the field, including the Strategic Plan from the National Cancer Institute [28], suggest that to advance the science of communication, we must find ways for continuous, scalable, and clinically meaningful measurement methods [25-27,29,30]. Our protocol helps fill this gap by offering a technology that leverages ubiquitous sensing methods, combined with linguistic and paralinguistic feature engineering methods, to create a novel, scalable, and longitudinal framework to measure the impact of communication in the actual clinical setting tied to a relevant patient outcome such as cancer pain.

    Computational methods for processing natural (ie, human) language provide novel opportunities to improve outcomes within health care, including patient-clinician communication. For example, advancements in natural language processing (NLP) technology now make granular analysis of written text and human speech more feasible, allowing us to better parse and understand the dynamics of complex interpersonal interactions [31-34]. This work builds upon prior research regarding NLP analysis of palliative care documentation [31,32,35,36] and evaluation of structured communication skills [37,38], and extends prior work using existing and foundational techniques in NLP and machine learning [39] to implement CommSense. Successful achievement of the aims will establish proof of concept that CommSense can identify relevant verbal (and limited nonverbal) communication signals during patient-clinician interactions, extract relevant metrics of communication performance, and ultimately (long term) provide timely and personalized user feedback to track and evaluate communication performance.

    In summary, this proposed research is timely, relevant, and addresses an urgent problem in health care, namely how to assess and measure patient-clinician communication. Improving communication related to cancer pain management can have profound positive implications, such as decreasing patient and family caregiver suffering [40-46], reducing health disparities related to pain management and cancer care [16,19,47-51], improving health care provider job satisfaction [10,52], and mitigating trips to the hospital or emergency department due to uncontrolled pain [53-56]. It is important to emphasize that we recognize the multiple and complex dimensions for improving communication, and many different types of patient-clinician interactions. However, for this initial pilot research, we aim to address 1 aspect, specifically health care provider communication related to cancer pain in the palliative care context, and to determine if we can make a positive impact by building technology to measure and evaluate key features of these types of conversations that can be assessed in real time. Given the scope and intent of this pilot work and the documented need related to this problem, we believe it is an appropriate place to start. If successful, we envision that the CommSense platform will be applicable to a broad range of health care–related conversations and contexts.

    Preliminary Work

    CommSense will build upon an existing Android smartwatch platform (SWear) developed by coauthors (LB and MB) to collect sensor signals from smartwatches [57]. Prior work using SWear has demonstrated acceptance of the technology, accuracy of the underlying NLP technology, and the ability to successfully use the platform across multiple contexts and study samples [58-64]. Specifically, the SWear platform has been previously used to evaluate communication in socially anxious individuals, and the feasibility and ability of Swear to extract audio features (eg, energy, pitch, Mel-frequency cepstral coefficients and NLP features such as sentence representation from pretrained Roberta/Sentence Transformers, term frequency-inverse document frequency) for predicting different anxious states during natural conversations have been established. Although this study addresses a different clinical problem, the foundational and established NLP methods, and the techniques for collecting, storing, and processing audio data are similar. This research leverages our team’s complementary skillsets related to smartphone-based biomarkers of cognitive states and virtual human training systems for patients, clinicians, and teachers (engineering, LB [65-69]); mobile and ubiquitous computing, NLP technology, machine learning (engineering, MB [66]); patient-provider communication and smart health (nursing, VL [70-73]; medicine, TF [74-77]); informatics (medicine, DL [78,79]); and oncology and pain management (nursing, VL [73,80-82]). Our team possesses the clinical and technical expertise necessary to support the aims of this research.

    Aims

    This is a 1-year (November 2021-November 2022) “proof-of-concept” feasibility study to develop a novel ubiquitous system and associated algorithms for measuring quality of conversations (CommSense) that can (1) be implemented on mobile devices, such as smartwatches; (2) reliably capture patient-clinician interactions in a clinical setting; (3) process communication to extract key markers of communication effectiveness and performance; and (4) ultimately (long term) provide real-time feedback to end users to improve communication and patient health outcomes.


    Specific Aim 1: Establish Feasible Metrics of Communication to Extract From Conversations (Months 1-3)

    Data Collection

    Clinical investigators will conduct a thorough review of the recent health care communication and palliative care literature to develop an evidence-based “ideal and optimal” list of communication metrics. We anticipate that this list will have two categories: (1) general best practices of health care communication (eg, verbal metrics such as the amounts of silence or pauses; speaking turns, interruptions, and overtalking; open-ended versus closed-ended questions; and nonverbal metrics, such as eye contact; arms crossed or open; sitting or standing) and (2) metrics more specific to conversations about cancer pain management (eg, assessment questions related to the severity, onset, and quality of the pain). We will organize this list conceptually around the 6 recommended domains to operationalize patient-centered palliative cancer care communication as detailed by McCormack et al [25,83], including exchanging information, fostering healing relationships, recognizing and responding to emotions, managing uncertainty, making decisions, and enabling patient self-management. We will also organize this list by “need,” “nice,” and “next” to record CommSense features considered essential by the clinical team, including the preferred and future features. This initial list of communication metrics will encompass the “what” (content questions) and the “how” (in what manner the questions are asked). This list will be discussed collaboratively within the study team and consensus about the included items will be reached. The list will also be vetted with other communication experts in the field, with whom investigators of this project have established relationships.

    Specific Aim 2: Develop the CommSense Software (Months 4-7)

    Data Collection

    The “ideal and optimal” list of communication metrics will be shared with engineering investigators to gauge technical feasibility. We anticipate this will be a highly iterative process between the engineering and clinical team investigators to refine our list of desired communication metrics based upon technical capabilities and clinical relevance. As discussed above (see Preliminary Work), CommSense will build upon prior work using an existing Android smartwatch platform (SWear) to collect sensor signals from smartwatches [57]. SWear collects multiple sensor streams such as motion, audio, and physiological data and synchronizes the data to a secure server for further analysis. SWear can also deliver microsurveys (Ecological Momentary Assessments) for collecting self-reported data. SWear has already been validated in multiple studies and is available on the Android play store [57,66]. CommSense will include sensing modules that can collect (1) physiological data via built-in sensors to measure variables such as heart rate, (2) gesture data via accelerometers and gyroscope sensors, and (3) voice data via the embedded microphone (see Figure 1). Although we recognize the importance of nonverbal communication and will thus leverage the existing passive sensing capabilities of CommSense to collect data related to heart rate variability and movement, these markers will be secondary to our primary focus of collecting audio data to analyze verbal and linguistic metrics of patient-provider communication.

    ‎
    Figure 1. CommSense system overview. Data are captured during patient-clinician interactions using smartwatches and synchronized to the secure cloud server to extract metrics characterizing communication quality, such as linguistic and paralinguistic markers (primary focus of the study) and physiological markers (secondary focus of study).
    View this figure

    Specific Aim 3: Pilot Testing the Ability of CommSense to Accurately Extract Identified Communication Metrics (Months 8- 12)

    Data Collection

    CommSense will be piloted with 5 nursing or medical students and 5 experienced oncology/palliative clinicians (n=10) using simulated scenarios to evaluate its accuracy in capturing and extracting the preidentified communication metrics (Table 1). Each participant will work through 2 conversation scenarios (n=20,10 per group [84,85]) and we will collect multiple data points related to paralinguistic and linguistic markers, as well as body language and physiological markers (Figure 1). The primary goal of Aim 3 is to verify the fidelity of the data captured using CommSense by comparing findings to ground truth. Clinical team members will write 2 relevant scripted scenarios (approximately 10-15 minutes in length) that relate to assessing and managing cancer pain in a palliative care context. It is critical to emphasize that although we ultimately aim to advance communication evaluation beyond scripted and simulated scenarios, this pilot study represents the foundational first step to develop technology that can reliably capture and analyze communication data before being implemented “in the wild.” Consistent with the scope of an exploratory pilot, this initial research will not involve real patients with protected health information. Future work with CommSense that involves actual patients will address all relevant privacy measures and the regulations of the Health Insurance Portability and Accountability Act of 1996. Pilot testing will occur in the institution’s Clinical Simulation Labs. After consent and basic demographic data are obtained, participants will wear CommSense and enact the 2 scenarios. The “patient” for our pilot testing may be a voice-capable mannequin, a member of our study team, or an experienced clinician volunteer, depending on what is feasible considering COVID-19 restrictions. The interaction will be recorded by CommSense and by a separate fixed external microphone and recording device to establish ground truth. At the end of the interaction, participants will complete a brief survey to assess the acceptability of using CommSense, provide suggestions for future iterations, state preferences regarding data sharing, and rate their self-perceived communication performance.

    Table 1. Examples of anticipated features to extract and analyze from conversations using CommSense.
    FeatureCommunication goal or rationale
    Audio signal variables

    		SilenceTo allow time to process complex or difficult information

    		Speaking turns and interruptionsTo avoid speech dominance and ensure all participants are heard

    		Prosody, flow, and rhythmTo reduce stress, and increase empathy and clarity
    Natural language variables (primary)

    		Complexity of languageTo avoid medical jargon to decrease confusion

    		Tone or sentimentTo convey empathy, warmth, and openness, and build rapport

    		Open-ended versus close-ended questionsTo allow exploration and promote bidirectional dialogue

    		Language associated with communication best practices related to palliative care and pain management (eg, “I want to be sure I understand…;” “It sounds like you are feeling…;” “Can you tell me more about…”)To use language associated with therapeutic communication related to symptom management in the context of serious illness
    Nonverbal variablesa (secondary)

    		Heart rate, motion or movement, and gesturesTo use nonverbal indicators (such as sitting down and not crossing arms) for establishing rapport, trust, and dialogue between the patient and provider, and heart rate for indicating provider stress level during conversation

    aDue to the capability of the sensing platform and ease of collecting the data, nonverbal physiological and gesture-related variables will be collected, but they are not the primary focus of this study.

    Data Analysis

    CommSense data will be first preprocessed to clean the sensor data and extract markers of communication quality based on the metrics established in Aim 1. This will be achieved by analyzing (1) paralinguistic markers from the audio signals such as pitch, energy, and Mel-frequency cepstral coefficients to characterize features such as tone, silence, and speaking turns [86,87]; and (2) linguistic markers from the audio signals by first parsing the signal to text using Google’s speech-to-text application programming interface. Then, several semantic features will be extracted using NLP methods such as word embedding features that can describe structural organization of words in the conversation (eg, frequency-based methods such as count vector and term frequency-inverse document frequency) and lexical features such as linguistic inquiry and word count, which is one of the most popular lexical feature extraction methods that has been rigorously validated in the context of psychometric analysis of textual data. We will also explore passively collected motion and physiological data, such as motion and heart rate data, and extract features that can characterize stress, such as heart rate variability. Finally, all the extracted linguistic, physiological, and motion-derived features will be analyzed to study how they fluctuate across scenarios and groups using a multilevel analysis given the hierarchical structure of the data.

    Establishing Ground Truth

    Externally recorded conversations will be transcribed, verified, and then compared to the CommSense-generated output for the same conversation. To conduct this comparison, we will proceed in a stepwise manner. First, hard copy transcripts of the audio-recorded conversation will be independently coded using qualitative software by 2 clinical investigators to identify the communication metrics that we expect to be extracted by CommSense. Second, the results obtained by the 2 investigators will be compared to establish interrater reliability. If there is a discrepancy, a third team member will be consulted. Given concerns regarding the use of the Cohen κ [88] to evaluate interrater reliability, we chose verbal discussion to reconcile any potential disagreements between investigators regarding communication metrics identified in the transcripts. Third, we will compare the generated CommSense output with the ground truth investigator review of the same conversation. We will do this by assigning a numerical value based on the concordance between the CommSense output and the investigator review of the transcript. For each identified communication metric (eg, instances of overtalking) we will assign a score of 0 if the CommSense output does not match the investigator review of the hard copy transcript and a score of 1 if it does. For example, with 10 identified metrics, the best possible concordance score would be 10, meaning the CommSense output and the hard copy transcript review achieve 100% concordance. We will then calculate concordance scores for each conversation metric to achieve a composite score for each conversation to explore how accurately the CommSense software is able to extract the desired metrics from palliative care conversations. Descriptive statistics will be used to summarize demographic data and participant responses to the postdeployment survey assessing the acceptability of CommSense.

    Ethics Approval

    Institutional Review Board (University of Virginia Social & Behavioral Sciences IRB, #4985) approval has been granted.


    Work began on Specific Aim 1 in November 2021 and Specific Aim 2 in April 2022. Participant recruitment is expected to begin in summer 2022. We expect preliminary results to be available in fall 2022.


    Potential Applications of CommSense

    We hypothesize that it will be feasible to extract relevant metrics of communication performance with >80% concordance between ground truth audio transcripts and the CommSense-generated output. We also hypothesize that health care providers will consider CommSense acceptable and helpful in improving their communication skills.

    This pioneering idea represents a paradigm shift in health care delivery by leveraging a scalable and novel technology to measure, track, and evaluate patient-provider communication in the clinical setting. Consider the following potential scenario: A team of oncology health care providers from Clinic A complete intensive communication skills training and learn structured techniques to improve their ability to have difficult conversations with patients coping with advanced cancer. During the training, they practice these skills using scripted role plays. After the training, the health care providers in Clinic A are each given a CommSense wristband sensor (eg, smartwatch) to wear as they care for patients with advanced cancer in the actual clinic setting. With consent from all participants, the wristband sensor, equipped with multimodal sensors to capture natural language, is activated during patient-clinician conversations, and on-board processors capture, extract, and summarize preprogrammed metrics of interaction quality. Following the clinic visit, the wrist-worn device uploads summarized communication performance metrics for real-time display to the physician, nurse, and social worker. The physician is shocked to see that her speaking turns occupied 89% of the visit time and that she interrupted the patient 15 times. The nurse is pleased to see that her use of medical jargon has decreased to a single instance; however, she asked 3 times as many closed-ended questions compared to open-ended questions. The social worker realizes that yet again, he was the only health care provider who engaged the caregiver in discussing the pain management plan or addressed safe opioid handling practices at home. The entire team can collectively observe that their communication performance was not as strong as last week, but they are still performing at a higher level than 75% of their peer clinics. Additionally, as the individual and team communication performance in Clinic A improves, patients report lower pain scores. The trainers who led the communication skills training can track the progress of participants over time, assess the retention of communication skills over time, see how skills are being implemented “in the wild,” and based on the results, they can better tailor future training sessions to meet the needs of participants. The hospital administrator is thrilled when she receives a monthly report that clearly shows oncology clinics using CommSense sensors have higher patient and caregiver satisfaction scores, fewer medication errors, and lower readmission rates.

    The above scenario paints the long-range vision and potential impact of the CommSense technology. Although we propose to begin with 1 aspect of cancer care communication (pain management), we believe this model will be generalizable to other health care settings and contexts where quality communication is essential (eg, other types of high-stakes interactions, such as goals-of-care conversations or death notifications). We are eager to test CommSense in populations at high risk for health disparities and communication barriers (eg, patients from underrepresented groups experiencing undertreatment of pain), English as second language patients, and those at high risk for distressing symptoms (eg, patients with metastatic cancer). We also see key opportunities to customize this intervention for different metrics depending on the communication context, relevant outcome measures and communication preferences, and the needs and goals of participants. For example, different communication strategies may be needed for a patient who is an artist with pancreatic cancer, does not speak English, and is being treated in a rural community hospital, compared to the communication strategies needed for a patient who is a physician, speaks English fluently, and is being treated at an academic medical center for routine gallbladder surgery.

    Dissemination Plan

    Results from this research will be presented at relevant technical and clinical academic conferences, as well as published in scholarly peer-reviewed journals. As we are a highly interdisciplinary team, our dissemination plan will aim to reach diverse audiences within nursing, medicine, and engineering domains. We also anticipate sharing findings with other key stakeholders (eg, clinicians, hospital administrators, and cancer advocacy groups) in more informal settings to continue developing the CommSense platform.

    Limitations

    The primary limitation of this study is that it is being conducted in simulated clinical scenarios rather than in real clinical settings. Although this is an important limitation to acknowledge, it is appropriate and essential to validate the functionality and feasibility of CommSense in simulated settings before implementing it in an actual patient care setting. Another important limitation is the inability of CommSense to completely capture the important and complex nuances (eg, subtle nonverbal cues) of patient-clinician communication.

    Conclusions

    This pilot study is poised to make a valuable contribution to communication science, and NLP and wearable computing capabilities. We see particularly exciting options to apply the CommSense technology to support effective virtual and remote patient-clinician interactions, which are likely to become more normative in our post-COVID world. Future steps include leveraging results to (1) implement CommSense in real clinical scenarios with a larger sample of participants to more robustly assess its feasibility and acceptability; (2) test CommSense with diverse high-risk, high-need populations and contexts and explore how to best customize the system using communication metrics tailored to patient, clinician, and organizational needs; (3) evaluate the impact of CommSense on relevant patient-centered and organizational outcomes, such as patient pain, medical errors, staff turnover, goal-concordant care and health care usage; (4) conduct experimental trials to test the effectiveness of CommSense in terms of pre- and postcommunication training versus standard of care; (5) determine how to best capture, integrate, and display nonverbal data relevant to communication such as position, eye contact, and gestures; (6) test multiple concurrent users of CommSense (eg, the clinician, patient, and family caregiver wearing CommSense during a conversation and receiving feedback) to evaluate the interactional aspects of communication; and (7) identify how to most effectively share collected data with key stakeholders (patients, caregivers, health care providers, and organizational units).

    Acknowledgments

    This study was funded by the University of Virginia’s “Engineering in Medicine Seed Pilot Projects” program. We wish to thank Toby Campbell and Lee Ellington for providing feedback regarding the study design and Daniel Wilson for support with our literature search.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Peer-review report by The Center for Engineering in Medicine, University of Virginia - Engineering-in-Medicine Seed Grant Program (Virginia, USA).

    PDF File (Adobe PDF File), 160 KB
    1. Lancaster G, Kolakowsky-Hayner S, Kovacich J, Greer-Williams N. Interdisciplinary communication and collaboration among physicians, nurses, and unlicensed assistive personnel. J Nurs Scholarsh 2015 May;47(3):275-284. [CrossRef] [Medline]
    2. Sanders JJ, Paladino J, Reaves E, Luetke-Stahlman H, Anhang Price R, Lorenz K, et al. Quality measurement of serious illness communication: recommendations for health systems based on findings from a symposium of national experts. J Palliat Med 2020 Jan;23(1):13-21. [CrossRef] [Medline]
    3. Mazor KM, Gaglio B, Nekhlyudov L, Alexander GL, Stark A, Hornbrook MC, et al. Assessing patient-centered communication in cancer care: stakeholder perspectives. J Oncol Pract 2013 Sep;9(5):e186-e193 [FREE Full text] [CrossRef] [Medline]
    4. Galushko M, Romotzky V, Voltz R. Challenges in end-of-life communication. Curr Opin Support Palliat Care 2012 Sep;6(3):355-364. [CrossRef] [Medline]
    5. Lyndon A, Zlatnik MG, Wachter RM. Effective physician-nurse communication: a patient safety essential for labor and delivery. Am J Obstet Gynecol 2011 Aug;205(2):91-96. [CrossRef]
    6. Walling AM, Keating NL, Kahn KL, Dy S, Mack JW, Malin J, et al. Lower patient ratings of physician communication are associated with unmet need for symptom management in patients with lung and colorectal cancer. J Oncol Pract 2016 Jun;12(6):e654-e669. [CrossRef]
    7. Underhill ML, Sheldon LK, Halpenny B, Berry DL. Communication about symptoms and quality of life issues in patients with cancer: provider perceptions. J Canc Educ 2014 Apr;29(4):753-761. [CrossRef]
    8. Canivet D, Delvaux N, Gibon AS, Brancart C, Slachmuylder JL, Razavi D. Improving communication in cancer pain management nursing: a randomized controlled study assessing the efficacy of a communication skills training program. Support Care Cancer 2014 Aug;22(12):3311-3320. [CrossRef]
    9. Berry DL, Blumenstein BA, Halpenny B, Wolpin S, Fann JR, Austin-Seymour M, et al. Enhancing patient-provider communication with the electronic self-report assessment for cancer: a randomized trial. J Clin Oncol 2011 Mar;29(8):1029-1035. [CrossRef]
    10. Thorne SE, Bultz BD, Baile WF. Is there a cost to poor communication in cancer care?: a critical review of the literature. Psychooncology 2005 Oct;14(10):875-884. [CrossRef]
    11. Wittenberg-Lyles E, Goldsmith J, Ferrell B. Oncology nurse communication barriers to patient-centered care. Clin J Oncol Nurs 2013 Mar;17(2):152-158. [CrossRef]
    12. Granek L, Krzyzanowska MK, Tozer R, Mazzotta P. Oncologists' strategies and barriers to effective communication about the end of life. J Oncol Pract 2013 Jul;9(4):e129-e135. [CrossRef]
    13. van den Beuken-van Everdingen MHJ, Hochstenbach LMJ, Joosten EAJ, Tjan-Heijnen VCG, Janssen DJA. Update on prevalence of pain in patients with cancer: systematic review and meta-analysis. J Pain Symptom Manage 2016 Jun;51(6):1070-1090.e9. [CrossRef]
    14. Paice JA, Von Roenn JH. Under- or overtreatment of pain in the patient with cancer: how to achieve proper balance. J Clin Oncol 2014 Jun;32(16):1721-1726. [CrossRef] [Medline]
    15. Deandrea S, Corli O, Consonni D, Villani W, Greco MT, Apolone G. Prevalence of breakthrough cancer pain: a systematic review and a pooled analysis of published literature. J Pain Symptom Manage 2014 Jan;47(1):57-76 [FREE Full text] [CrossRef] [Medline]
    16. Anderson KO, Green CR, Payne R. Racial and ethnic disparities in pain: causes and consequences of unequal care. J Pain 2009 Dec;10(12):1187-1204. [CrossRef]
    17. Elliott AM, Alexander SC, Mescher CA, Mohan D, Barnato AE. Differences in physicians' verbal and nonverbal communication with Black and White Patients at the end of life. J Pain Symptom Manage 2016 Jan;51(1):1-8. [CrossRef]
    18. Mack JW, Paulk ME, Viswanath K, Prigerson HG. Racial disparities in the outcomes of communication on medical care received near death. Arch Intern Med 2010 Sep 27;170(17):1533-1540. [CrossRef]
    19. Palmer NRA, Kent EE, Forsythe LP, Arora NK, Rowland JH, Aziz NM, et al. Racial and ethnic disparities in patient-provider communication, quality-of-care ratings, and patient activation among long-term cancer survivors. J Clin Oncol 2014 Dec;32(36):4087-4094. [CrossRef]
    20. Long AC, Engelberg RA, Downey L, Kross EK, Reinke LF, Cecere Feemster L, et al. Race, income, and education: associations with patient and family ratings of end-of-life care and communication provided by physicians-in-training. J Palliat Med 2014 Apr;17(4):435-447. [CrossRef]
    21. Singh S, Cortez D, Maynard D, Cleary JF, DuBenske L, Campbell TC. Characterizing the nature of scan results discussions: insights into why patients misunderstand their prognosis. J Oncol Pract 2017 Mar;13(3):e231-e239. [CrossRef]
    22. Hong YA, Hossain MM, Chou WS. Digital interventions to facilitate patient‐provider communication in cancer care: a systematic review. Psychooncology 2020 Jan;29(4):591-603. [CrossRef]
    23. Moore PM, Rivera Mercado S, Grez Artigues M, Lawrie TA. Communication skills training for healthcare professionals working with people who have cancer. Cochrane Database Syst Rev 2013 Mar;2013(3):CD003751 [FREE Full text] [CrossRef] [Medline]
    24. Ostherr K, Killoran P, Shegog R, Bruera E. Death in the digital age: a systematic review of information and communication technologies in end-of-life care. J Palliat Med 2016 Apr;19(4):408-420. [CrossRef]
    25. McCormack LA, Treiman K, Rupert D, Williams-Piehota P, Nadler E, Arora NK, et al. Measuring patient-centered communication in cancer care: a literature review and the development of a systematic approach. Soc Sci Med 2011 Apr;72(7):1085-1095. [CrossRef]
    26. Tulsky JA, Beach MC, Butow PN, Hickman SE, Mack JW, Morrison RS, et al. A research agenda for communication between health care professionals and patients living with serious illness. JAMA Intern Med 2017 Sep;177(9):1361-1366. [CrossRef]
    27. Bernacki RE, Block SD, American College of Physicians High Value Care Task Force. Communication about serious illness care goals: a review and synthesis of best practices. JAMA Intern Med 2014 Dec;174(12):1994-2003. [CrossRef]
    28. The NCI strategic plan for leading the nation to eliminate the suffering and death due to cancer. National Cancer Institute. 2006.   URL: https://stacks.stanford.edu/file/druid:vh318mh4510/nci_2007_strategic_plan.pdf [accessed 2022-04-25]
    29. Curtis JR, Back AL, Ford DW, Downey L, Shannon SE, Doorenbos AZ, et al. Effect of communication skills training for residents and nurse practitioners on quality of communication with patients with serious illness. JAMA 2013 Dec;310(21):2271-2281. [CrossRef]
    30. Mazor KM, Beard RL, Alexander GL, Arora NK, Firneno C, Gaglio B, et al. Patients' and family members' views on patient-centered communication during cancer care. Psychooncology 2013 Nov;22(11):2487-2495 [FREE Full text] [CrossRef] [Medline]
    31. Lindvall C, Lilley EJ, Zupanc SN, Chien I, Udelsman BV, Walling A, et al. Natural language processing to assess end-of-life quality indicators in cancer patients receiving palliative surgery. J Palliat Med 2019 Feb;22(2):183-187. [CrossRef]
    32. Udelsman BV, Moseley ET, Sudore RL, Keating NL, Lindvall C. Deep natural language processing identifies variation in care preference documentation. J Pain Symptom Manage 2020 Jun;59(6):1186-1194.e3. [CrossRef]
    33. Yim WW, Yetisgen M, Harris WP, Kwan SW. Natural language processing in oncology: a review. JAMA Oncol 2016 Jun;2(6):797-804. [CrossRef] [Medline]
    34. Lee KC, Udelsman BV, Streid J, Chang DC, Salim A, Livingston DH, et al. Natural language processing accurately measures adherence to best practice guidelines for palliative care in trauma. J Pain Symptom Manage 2020 Feb;59(2):225-232.e2. [CrossRef]
    35. Brizzi K, Zupanc SN, Udelsman BV, Tulsky JA, Wright AA, Poort H, et al. Natural language processing to assess palliative care and end-of-life process measures in patients with breast cancer with leptomeningeal disease. Am J Hosp Palliat Care 2019 Nov;37(5):371-376. [CrossRef]
    36. Lilley EJ, Lindvall C, Lillemoe KD, Tulsky JA, Wiener DC, Cooper Z. Measuring processes of care in palliative surgery: a novel approach using natural language processing. Ann Surg 2018 May;267(5):823-825. [CrossRef] [Medline]
    37. Kruser JM, Nabozny MJ, Steffens NM, Brasel KJ, Campbell TC, Gaines ME, et al. “Best case/worst case”: qualitative evaluation of a novel communication tool for difficult in-the-moment surgical decisions. J Am Geriatr Soc 2015 Sep;63(9):1805-1811. [CrossRef]
    38. Sommovilla J, Kopecky KE, Campbell T. Discussing prognosis and shared decision-making. Surg Clin North Am 2019 Oct;99(5):849-858. [CrossRef]
    39. Lötsch J, Ultsch A. Machine learning in pain research. Pain 2018 Apr;159(4):623-630. [CrossRef]
    40. Meeker MA, Finnell D, Othman AK. Family caregivers and cancer pain management: a review. J Fam Nurs 2011 Feb;17(1):29-60. [CrossRef]
    41. Chi NC, Demiris G. Family caregivers’ pain management in end-of-life care: a systematic review. Am J Hosp Palliat Care 2016 Mar;34(5):470-485. [CrossRef]
    42. Chi NC, Barani E, Fu YK, Nakad L, Gilbertson-White S, Herr K, et al. Interventions to support family caregivers in pain management: a systematic review. J Pain Symptom Manage 2020 Sep;60(3):630-656.e31. [CrossRef]
    43. Mehta A, Cohen SR, Ezer H, Carnevale FA, Ducharme F. Striving to respond to palliative care patients' pain at home: a puzzle for family caregivers. Oncol Nurs Forum 2011 Jan;38(1):E37-E45. [CrossRef]
    44. Mehta A, Chan LS, Cohen SR. Flying blind: sources of distress for family caregivers of palliative cancer patients managing pain at home. J Psychosoc Oncol 2014 Jan;32(1):94-111. [CrossRef]
    45. Lowey SE. Management of severe pain in terminally ill patients at home: an evidence-based strategy. Home Healthc Now 2020 Jan;38(1):8-15 [FREE Full text] [CrossRef]
    46. Wilson E, Caswell G, Turner N, Pollock K. Managing medicines for patients dying at home: a review of family caregivers' experiences. J Pain Symptom Manage 2018 Dec;56(6):962-974. [CrossRef]
    47. Meghani SH, Gallagher RM. Disparity vs inequity: toward reconceptualization of pain treatment disparities. Pain Med 2008 Jul;9(5):613-623. [CrossRef]
    48. Meghani SH, Polomano RC, Tait RC, Vallerand AH, Anderson KO, Gallagher RM. Advancing a national agenda to eliminate disparities in pain care: directions for health policy, education, practice, and research. Pain Med 2012 Jan;13(1):5-28. [CrossRef]
    49. Charlton M, Schlichting J, Chioreso C, Ward M, Vikas P. Challenges of rural cancer care in the United States. Oncology (Williston Park) 2015 Sep;29(9):633-640 [FREE Full text] [Medline]
    50. Perry LM, Walsh LE, Horswell R, Miele L, Chu S, Melancon B, et al. Racial disparities in end-of-life care between Black and White adults with metastatic cancer. J Pain Symptom Manage 2021 Feb;61(2):342-349.e1. [CrossRef]
    51. Stein KD, Alcaraz KI, Kamson C, Fallon EA, Smith TG. Sociodemographic inequalities in barriers to cancer pain management: a report from the American Cancer Society's Study of Cancer Survivors-II (SCS-II). Psychooncology 2016 Aug;25(10):1212-1221. [CrossRef]
    52. Thorne SE, Hislop TG, Armstrong EA, Oglov V. Cancer care communication: The power to harm and the power to heal? Patient Educ Couns 2008 Apr;71(1):34-40. [CrossRef]
    53. Cao T, Johnson A, Coogle J, Zuzelski A, Fitzgerald S, Kapadia V, et al. Incidence and characteristics associated with hospital readmission after discharge to home hospice. J Palliat Med 2020 Feb;23(2):233-239. [CrossRef]
    54. Oh TK, Jo YH, Choi JW. Associated factors and costs of avoidable visits to the emergency department among cancer patients: 1-year experience in a tertiary care hospital in South Korea. Support Care Cancer 2018 May;26(11):3671-3679. [CrossRef]
    55. Delgado-Guay MO, Kim YJ, Shin SH, Chisholm G, Williams J, Allo J, et al. Avoidable and unavoidable visits to the emergency department among patients with advanced cancer receiving outpatient palliative care. J Pain and Symptom Manage 2015 Mar;49(3):497-504. [CrossRef]
    56. Green E, Ward S, Brierley W, Riley B, Sattar H, Harris T. “They shouldn’t be coming to the ED, should they?”: a descriptive service evaluation of why patients with palliative care needs present to the emergency department. Am J Hosp Palliat Care 2017 Jan;34(10):984-990. [CrossRef]
    57. Boukhechba M. SWear sensing on Android smartwatch. Google Play.   URL: https://play.google.com/store/apps/details?id=uva.swear [accessed 2022-04-25]
    58. Tavakoli A, Boukechba M, Heydarian A. Leveraging ubiquitous computing for empathetic routing: a naturalistic data-driven approach. In: CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021 May Presented at: 2021 CHI Conference on Human Factors in Computing Systems; May 8-13, 2021; Yokohama, Japan p. 1-6   URL: https://dl.acm.org/doi/abs/10.1145/3411763.3451693 [CrossRef]
    59. Tavakoli A, Kumar S, Boukhechba M, Heydarian A. Driver state and behavior detection through smart wearables. In: 2021 IEEE Intelligent Vehicles Symposium (IV). 2021 Jul Presented at: 2021 IEEE Intelligent Vehicles Symposium (IV); July 11-17, 2021; Nagoya, Japan p. 559-565   URL: https://arxiv.org/abs/2104.13889 [CrossRef]
    60. Tavakoli A, Boukhechba M, Heydarian A. Personalized driver state profiles: a naturalistic data-driven study. In: Advances in Human Aspects of Transportation. AHFE 2020. Advances in Intelligent Systems and Computing. 2020 Jul Presented at: International Conference on Applied Human Factors and Ergonomics; July 16-20, 2020; United States p. 32-39   URL: https://link.springer.com/chapter/10.1007/978-3-030-50943-9_5 [CrossRef]
    61. Adam S, Coward B, DeBerry G, Glazier C, Magnusson E, Boukhechba M. Investigating novel proximity monitoring techniques using ubiquitous sensor technology. In: 2021 Systems and Information Engineering Design Symposium (SIEDS). 2021 Jul Presented at: 2021 Systems and Information Engineering Design Symposium (SIEDS); April 29-30, 2021; Charlottesville, United States p. 1-6   URL: https://ieeexplore.ieee.org/document/9483795 [CrossRef]
    62. Baglione AN, Gong J, Boukhechba M, Wells KJ, Barnes LE. Leveraging mobile sensing to understand and develop intervention strategies to improve medication adherence. IEEE Pervasive Comput 2020 Jul;19(3):24-36. [CrossRef]
    63. Homdee M, Boukhechba M, Feng YW, Kramer N, Lach J, Barnes LE. Enabling smartphone-based estimation of heart tate. ArXiv Preprint posted online on Dec 18, 2019. [FREE Full text] [CrossRef]
    64. Boukhechba M, Cai L, Wu C, Barnes LE. ActiPPG: using deep neural networks for activity recognition from wrist-worn photoplethysmography (PPG) sensors. Smart Health 2019 Dec;14:100082. [CrossRef]
    65. Chow PI, Fua K, Huang Y, Bonelli W, Xiong H, Barnes LE, et al. Using mobile sensing to test clinical models of depression, social anxiety, state affect, and social isolation among college students. J Med Internet Res 2017 Mar;19(3):e62 [FREE Full text] [CrossRef] [Medline]
    66. Bhoukhechba M, Barnes LE. SWear: sensing using WEARables. Generalized human crowdsensing on smartwatches. In: AHFE 2020: Advances in Usability, User Experience, Wearable and Assistive Technology. 2020 Jan Presented at: IEEE 11th International Conference on Applied Human Factors and Ergonomics; July 16-20, 2020; United States p. 510-516   URL: https://link.springer.com/chapter/10.1007/978-3-030-51828-8_67 [CrossRef]
    67. Datta D, Brashers V, Owen J, White C, Barnes LE. A deep learning methodology for semantic utterance classification in virtual human dialogue systems. In: Part of the Lecture Notes in Computer Science book series (LNAI,volume 10011). 2016 Oct Presented at: International Conference on Intelligent Virtual Agents; September 20-23, 2016; Los Angeles, United States p. 451-455   URL: https://link.springer.com/chapter/10.1007/978-3-319-47665-0_53 [CrossRef]
    68. Datta D, Phillips M, Bywater JP, Chiu J, Watson GS, Barnes L, et al. Virtual pre-service teacher assessment and feedback via conversational agents. In: Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications.: Association for Computational Linguistics; 2021 Apr Presented at: 16th Workshop on Innovative Use of NLP for Building Educational Applications; April 2021; Online p. 185-198   URL: https://aclanthology.org/2021.bea-1.20
    69. Datta D, Phillips M, Chiu J, Watson GS, Bywater JP, Barnes L, et al. Improving classification through weak supervision in context-specific conversational agent development for teacher education. ArXiv Preprint posted online on Oct 23, 2020. [FREE Full text] [CrossRef]
    70. LeBaron V, Hayes J, Gordon K, Alam R, Homdee N, Martinez Y, et al. Leveraging smart health technology to empower patients and family caregivers in managing cancer pain: protocol for a feasibility study. JMIR Res Protoc 2019 Dec;8(12):e16178 [FREE Full text] [CrossRef] [Medline]
    71. LeBaron V, Bennett R, Alam R, Blackhall L, Gordon K, Hayes J, et al. Understanding the experience of cancer pain from the perspective of patients and family caregivers to inform design of an in-home smart health system: multimethod approach. JMIR Form Res 2020 Aug;4(8):e20836. [CrossRef]
    72. LeBaron V, Adhikari A, Bennett R, Chapagain Acharya S, Dhakal M, Elmore CE, et al. A survey of cancer care institutions in Nepal to inform design of a pain management mobile application. BMC Palliat Care 2021 Nov;20(1):171 [FREE Full text] [CrossRef] [Medline]
    73. LeBaron VT, Blonquist TM, Hong F, Halpenny B, Berry DL. Screening for pain in the ambulatory cancer setting: Is 0-10 enough? J Oncol Pract 2015 Nov;11(6):435-441. [CrossRef]
    74. Flickinger TE, DeBolt C, Waldman AL, Reynolds G, Cohn WF, Beach MC, et al. Social support in a virtual community: analysis of a clinic-affiliated online support group for persons living with HIV/AIDS. AIDS Behav 2016 Oct;21(11):3087-3099. [CrossRef]
    75. Flickinger TE, Ingersoll K, Swoger S, Grabowski M, Dillingham R. Secure messaging through PositiveLinks: examination of electronic communication in a clinic-affiliated smartphone app for patients living with HIV. Telemed J E Health 2020 Mar;26(3):359-364. [CrossRef]
    76. Flickinger TE, Saha S, Moore RD, Beach MC. Higher quality communication and relationships are associated with improved patient engagement in HIV care. J Acquir Immune Defic Syndr 2013 Jul;63(3):362-366 [FREE Full text] [CrossRef] [Medline]
    77. Flickinger TE, Saha S, Roter D, Korthuis PT, Sharp V, Cohn J, et al. Clinician empathy is associated with differences in patient–clinician communication behaviors and higher medication self-efficacy in HIV care. Patient Educ Couns 2016 Feb;99(2):220-226. [CrossRef]
    78. Minichiello TA, Ling D, Ucci DK. Breaking bad news: a practical approach for the hospitalist. J Hosp Med 2007 Nov;2(6):415-421. [CrossRef]
    79. Lindenauer PK, Ling D, Pekow PS, Crawford A, Naglieri-Prescod D, Hoople N, et al. Physician characteristics, attitudes, and use of computerized order entry. J Hosp Med 2006 Jul;1(4):221-230. [CrossRef]
    80. Lebaron V, Beck SL, Maurer M, Black F, Palat G. An ethnographic study of barriers to cancer pain management and opioid availability in India. Oncologist 2014 May;19(5):515-522 [FREE Full text] [CrossRef] [Medline]
    81. LeBaron V, Beck SL, Black F, Palat G. Nurse moral distress and cancer pain management: an ethnography of oncology nurses in India. Cancer Nurs 2014 Sep;37(5):331-344. [CrossRef] [Medline]
    82. LeBaron VT, Camacho F, Balkrishnan R, Yao NA, Gilson AM. Opioid epidemic or pain crisis? using the Virginia All Payer Claims Database to describe opioid medication prescribing patterns and potential harms for patients with cancer. J Oncol Pract 2019 Dec;15(12):e997-e1009. [CrossRef]
    83. Epstein RM, Street Jr RL. Patient-Centered Communication in Cancer Care: Promoting Healing and Reducing Suffering.: National Cancer Institute U.S. Department of Health and Human Services. National Institutes of Health; 2007.   URL: https://cancercontrol.cancer.gov/sites/default/files/2020-06/pcc_monograph.pdf [accessed 2022-04-25]
    84. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat 2005 Oct;4(4):287-291. [CrossRef]
    85. Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health 2008 Apr;31(2):180-191. [CrossRef] [Medline]
    86. Wang F. Does speech prosody matter in health communication? Evidence from native and non-native English speaking medical students in a simulated clinical interaction [Dissertation].: University of East Anglia; 2014.   URL: https://ueaeprints.uea.ac.uk/id/eprint/50542/ [accessed 2022-04-25]
    87. Applebaum L, Coppola M, Goldin-Meadow S. Prosody in a communication system developed without a language model. Sign Lang Linguist 2014 Nov;17(2):181-212. [CrossRef]
    88. Pontius RG, Millones M. Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment. Int J Remote Sens 2011 Aug;32(15):4407-4429. [CrossRef]

    ‎
    NLP: natural language processing

    Edited by T Leung; This paper was peer reviewed by The Center for Engineering in Medicine, University of Virginia - Engineering-in-Medicine Seed Grant Program (Virginia, USA). See the Multimedia Appendix for the peer-review report; submitted 14.03.22; accepted 30.03.22; published 20.05.22

    Copyright

    ©Virginia LeBaron, Mehdi Boukhechba, James Edwards, Tabor Flickinger, David Ling, Laura E Barnes. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 20.05.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.