This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Despite research supporting the use of care coordination in chronic obstructive pulmonary disease (COPD), there is relatively little known about the comparative effectiveness of different strategies used to organize care for patients. To investigate the most important COPD care coordination strategies, community-based stakeholder input is needed, especially from medically underserved populations. Web-based platforms are electronic tools now being used to bring together individuals from underrepresented populations to share input and obtain clarification on comparative effectiveness research (CER) ideas, questions, and hypotheses.
Use low computer-literate, collaborative survey technology to evaluate stakeholder priorities for CER in COPD care coordination.
A mixed-method, concurrent triangulation design was used to collect survey data from a virtual advisory board of community-based stakeholders including medically underserved patients with COPD, informal caregivers, clinicians, and research scientists. The eDelphi method was used to conduct 3 iterative rounds of Web-based surveys. In the first 2 survey rounds, panelists viewed a series of “mini research prospectus” YouTube video presentations and rated their level of agreement with the importance of 10 COPD care coordination topics using 7-point Likert scales. In the final third-round survey, panelists ranked (1=most important, 8=least important) and commented on 8 remaining topics that panelists favored most throughout the first 2 survey rounds. Following the third-round survey, panelists were asked to provide feedback on the potential impact of a Web-based stakeholder engagement network dedicated to improving CER in COPD.
Thirty-seven panelists rated the following care coordination topics as most important (lower means indicate greater importance): (1) measurement of quality of care (mean 2.73, SD 1.95); (2) management of COPD with other chronic health issues (mean 2.92, SD 1.67); (3) pulmonary rehabilitation as a model for care (mean 3.72; SD 1.93); (4) quality of care coordination (mean 4.12, SD 2.41); and (5) comprehensive COPD patient education (mean 4.27, SD 2.38). Stakeholder comments on the relative importance of these care coordination topics primarily addressed the importance of comparing strategies for COPD symptom management and evaluating new methods for patient-provider communication. Approximately one half of the virtual panel assembled indicated that a Web-based stakeholder engagement network could enable more online community meetings (n=19/37, 51%) and facilitate more opportunities to suggest, comment on, and vote for new CER ideas in COPD (n=18/37, 49%).
Members of this unique virtual advisory board engaged in a structured Web-based communication process that identified the most important community-specific COPD care coordination research topics and questions. Findings from this study support the need for more CER that evaluates quality of care measures used to assess the delivery of treatments and interventions among medically underserved patients with COPD.
Chronic obstructive pulmonary disease (COPD), a chronic lung condition characterized by progressive airflow limitation, shortness of breath, and productive cough, is the third leading cause of death in the United States [
Inadequate coordination between primary and specialist care and between community and medical settings may be the main reason for poorly managed symptoms observed in patients with long-term conditions such as COPD [
To date, there has been limited involvement from individuals most affected by care coordination processes (ie, patients and informal caregivers) in defining CER topics and evaluating COPD care coordination strategies [
Current barriers to meaningful stakeholder engagement include lack of interest, limited stakeholder education about research topics and health issues, and human/fiscal resource constraints. These barriers can preclude researchers from identifying, engaging, and interacting with stakeholders; however, there are a number of innovative, technology-based methods that help overcome some of these obstacles [
Currently, more CER is needed to help better understand the relative benefits and harms of the many available strategies used to coordinate the delivery of care in COPD. Community-level input on the importance of different strategies can improve the design of CER studies of COPD care coordination approaches in different health care settings. As such, the primary objective of this stakeholder engagement study was to systematically prioritize CER topics related to COPD care coordination using electronic feedback provided by a diverse community-based panel of stakeholders. A secondary objective was to gather stakeholder input that could help inform the development of a Web-based stakeholder engagement network dedicated to evaluating, translating, and disseminating CER findings in COPD care coordination.
A mixed-method concurrent triangulation design [
Compared with the traditional paper-based and snail mail Delphi, the eDelphi offers researchers several other advantages, including the following: (1) Web-based storage, processing, and transmission of secure data; (2) protection of respondent anonymity; (3) rapid feedback to panelists in the form of tables, charts, and statistics; and (4) fewer logistical challenges generally associated with bringing groups of people together for research-related purposes [
The commitment of stakeholders to participate in eDelphi studies is often related to their level of interest with the topic [
Purposive sampling was used to identify patients with COPD registered in a community patient database operated and managed by the CEnR program. All patients were invited to participate by a study navigator working on behalf of the CEnR program. Patients were asked predetermined questions over the telephone to determine whether they met the following inclusion criteria: (1) diagnosis of COPD by a physician or specialist; (2) valid email address on file in the community patient database; (3) minimum 40 years of age; and (4) speak English as a first language. The minimum age of 40 was established because individuals at this age generally begin experiencing COPD symptoms such as excess mucus buildup, wheezing, and productive cough [
Informal caregivers of patients with COPD were recruited using snowball sampling methods that capitalized on patient referrals. Snowball sampling is particularly useful when identifying and recruiting racial/ethnic minorities and individuals with low SES to participate in research [
Practicing clinicians and research scientists with professional experience studying COPD care coordination were identified through searching VIVO, an intrainstitutional semantic-Web-networking platform used to foster cross-disciplinary team science in the university where the research took place [
Three-round eDelphi process.
In the first-round survey, panelists were provided with a brief (3-4 sentences) bulleted description of 10 COPD care coordination topics selected from the literature [
Panelists’ level of agreement with the importance of each of the 10 COPD care coordination topics was evaluated using a 7-point Likert scale (1=strongly disagree, 7=strongly agree). Panelists were also asked to provide demographic information using items adapted from the 2014 Behavioral Risk Factor Surveillance System’s Questionnaire [
Definitions of chronic obstructive pulmonary disease care coordination topics evaluated by stakeholder panelists.
Care coordination topic | Definition | Sample comparative effectiveness research questiona |
Management of chronic obstructive pulmonary disease (COPD) when other chronic health problems are present | Participation among multiple health care professionals in the treatment of COPD and other comorbidities experienced by the patient | Would a standard process for identifying chronic health conditions commonly found in patients with COPD help improve a doctor’s ability to treat patients? |
Pulmonary rehabilitation as a model for care in COPD | Formal program aimed at improving the quality of life for patients with chronic lung disease that includes a broad range of activities such as nutritional counseling, breathing retraining, exercise training, and practicing energy conservation | Which setting for pulmonary rehabilitation is most effective for different patients with COPD? |
Depression and mental health management in patients with COPD | Active screening for depression and the management of mental health provided by trained depression care managers and primary care providers, including antidepressant treatment and/or psychotherapy provided by a supervising psychiatrist | Would patients undergoing regular screening for depression show improvements in mental and physical health outcomes? |
Quality of care coordination | Measuring how effectively patient's needs and preferences are understood and communicated at the right time to the right people, and whether or not this information is used to provide safe, appropriate, and effective care to the patient | What factors are associated with high-quality care coordination in patients with COPD? |
Measurement of quality of care in patients with COPD | The degree to which various strategies used for managing treatment across health care providers and services produces the desired health outcomes and improves the patient’s quality of life | What should be measured while a patient with COPD is following a prescribed treatment plan? |
Comprehensive COPD patient education | Knowledge- and skill-building programs for patients with COPD aimed at improving the patient’s ability to independently self-monitor and manage COPD | What is the comparative effectiveness of different types of educational programs designed to teach patients how to use their respiratory inhaler? |
Cost effectiveness of care | The direct comparison of the costs and health benefits of a treatment or intervention option | What is the cost effectiveness of different models of care coordination such as pulmonary rehabilitation versus comprehensive patient education? |
Case management in COPD | Strategies that involve working with health care providers to coordinate and monitor treatments to meet individual patient needs | What is the effectiveness of case management programs for patients with other chronic conditions such as heart failure and diabetes, compared with regular care in patients with COPD? |
Measuring hospitalization risk in patients with COPD | Evaluating and identifying which patients are at the greatest risk of hospitalization and rehospitalization to inform and modify risk-specific care plans | Is it possible to use a screening tool to identify which patients are at greatest risk of visiting the doctor’s office or emergency room on a more regular basis? |
Patient-centered medical home | Care strategy that uses a team of health care providers to monitor and improve patient care in the most accessible, convenient way for a patient | Does an individual patient with COPD experience better outcomes if he/she participates in a patient-centered medical home as compared to following other COPD treatment plans? |
aAdapted from sample questions suggested by Krishnan and colleagues [
After the first-round survey closed, 2 members of the research team (MS and JA) compiled all panelist ratings and type-written comments from the first-round survey. These data were used in conjunction with other stakeholder education materials [
All consenting panelists who successfully completed the first-round survey were sent an email invitation with an embedded hyperlink to access this second-round survey. In the second-round survey, panelists were asked to review summary results from the first-round survey and watch all 10 care coordination YouTube videos developed by the research team using summary data from round one. Following each video, panelists were invited to enter their own comments into a textbox and rate their level of agreement with the importance of CER questions explained on each video (1=strongly disagree, 7=strongly agree). Like in round one, panelists were given 2 weeks to submit their survey responses (September 10, 2014, to September 25, 2014).
Item sequence during second-round survey administered on Qualtrics.
Following the closure of the second-round survey, 2 researchers compiled all panelists’ ratings and type-written comments from the second-round survey to create a final 6-minute video presentation delivered by the lead investigator. This final video summarized all eDelphi results from the first and second rounds. Specifically, it described the level of panelist convergence around the highest priority care coordination topics from the first to second rounds. The video was embedded onto the Qualtrics survey platform using the YouTube media player. After viewing the final summary video, panelists were asked to rank-order 8 remaining topics that received consensus support in the prior 2 rounds (1=most important, 8=least important). In addition, panelists were asked to comment on why they believed the topic they ranked as “1” was most important to study further.
Panelists were also asked to answer questions related to the potential impact of a Web-based stakeholder engagement network dedicated to fostering and improving CER in COPD. Panelists reported how likely they would be to visit the stakeholder engagement network website on a 6-point Likert scale (1=very unlikely, 6=very likely), and they were asked to indicate how often they would visit the website using a 5-point Likert scale (1=rarely or never, 5=every day or almost every day). Panelists were also asked to rank 5 potential purposes of the website from 1 to 5 (1=most important, 5=least important). Potential purposes of the Web-based stakeholder engagement network included learning from others, accessing research reports online, study recruitment, and building trust. Panelists could also enter in their own desired purpose using a field marked “Other.” Finally, panelists responded to the following open-ended questions: (1) “If you were to join a CER Network website, what roles or responsibilities would you see for yourself?”; (2) “What would you expect to gain from visiting a CER Network website?”; and (3) “What types of resources would you like available on a CER Network website?” Panelists could select from a list of all responses that they felt answered each question. Panelists were given 2 weeks to complete the third-round survey (October 15, 2014, to October 29, 2014), and following completion, each participant was redirected to a secure external website to provide contact information that would enable them to receive a US$ 45 gift card incentive for their participation.
Once duplicate surveys were deleted from the final database and a code number was assigned to each survey, the Internet protocol addresses were deleted to eliminate any linkages between participant names and email addresses. Qualitative and quantitative data were analyzed separately and independently. Two researchers scanned all submitted comments from round one and round two surveys to determine whether any panelists reported confusion related to any of the topic categories. Round-by-round panelist response and attrition rates were computed using frequency statistics. Frequency statistics were also computed to report the sociodemographic characteristics of panelists. Panelists’ ratings on each COPD care coordination topic were summarized in each round using mean (SD) statistics to quantify the collective judgment of respondents [
To determine which COPD care coordination research topics were elaborated on the most by panelists, Braun and Clarke’s [
After analyzing the third-round qualitative data with thematic analysis, both qualitative and quantitative sets of results were merged by directly comparing the quantitative rankings from the third-round survey with the qualitative feedback describing panelists’ reasons for selecting the highest priority COPD care coordination topic to study in future CER. Additional qualitative feedback was used to determine which COPD care coordination topics were elaborated on to the greatest extent. Finally, frequency and descriptive statistics were computed to analyze data on stakeholders’ beliefs regarding the potential impact of a Web-based stakeholder engagement network for advancing CER in COPD.
Round three survey codes, definitions, and kappa values.
Code | Definition | Kappa |
Behaviors/lifestyle | Includes patient health behaviors or lifestyle choices (eg, smoking, healthy eating) | >.99 |
Comorbidity | Mention of having multiple health problems at once (COPDa and asthma) | >.99 |
Coordination | Individuals working together or planning together for a common purpose; includes working as a team | .90 |
Coping | Patient strategies or discussion of ways to deal with health issues caused by or related to COPD | .91 |
Cost | Refers to money needed or spent on the care, management, or treatment of COPD | >.99 |
Diagnosis | Diagnosis of COPD or the health issues (eg, diagnosis of depression) stemming from COPD symptoms | .91 |
Doctor visits | Visits to health care providers that are involved in the treatment of COPD, NOT including emergency room visits | >.99 |
Patient education | References to the need to learn more or increase understanding about COPD, the treatments of COPD, or anything else related to COPD | .93 |
Effectiveness | References to the effectiveness of COPD treatments to manage or prevent health issues; include cost effectiveness; comparison benefits versus negative effects | .95 |
Emergency room visits | Mention of the occurrence or prevention of emergency room visits due to COPD symptoms or complications | >.99 |
Expectation | References to the patient’s expectation of COPD and its treatment | .80 |
Follow-up | Mention of the patient or provider following up on a treatment or issue related to COPD | .80 |
Health effects | Mentions of any health outcome related to COPD or symptoms of COPD (eg, breathing issues) | .94 |
Home | Reference to the in-home care or treatment of COPD | >.99 |
Hospital stay | Mention of having to stay at a hospital for a day or more for a treatment related to COPD; hospital visit outside of a regular doctor visit; not including emergency room visits | >.99 |
Informed | Mention of all individuals involved or affected by COPD having the knowledge to make appropriate or proper decisions about treatment or care | .95 |
Limitations | Reference to the downfalls or possible cons of a treatment or care strategy for COPD | >.99 |
Management | General references to managing COPD | .96 |
Medicine | Mention of any type of medicine taken to treat COPD or health issues related to COPD | >.99 |
Mental health | Mention of mental health concerns, such as depression, stemming from COPD diagnosis, symptoms, or treatment | >.99 |
Monitor | Monitoring the progress or effects of COPD symptoms and treatment | >.99 |
Necessary/helpful | Reference to someone or something being necessary, sufficient, or helpful in the COPD treatment or management | .99 |
Death | Discussion or reference to passing away | >.99 |
Patient-provider communication | Reference to the communication or lack thereof between a patient and the health care providers involved in their treatment | >.99 |
Personal experience | A patient or provider references a specific event that occurred related to diagnosis or treatment of COPD | >.99 |
Personalization | Reference to the unique factors of patients and the need for individualized treatment, care, or consideration | .96 |
Prevention | Reference to the strategies and need for preventing COPD or the complications of COPD | >.99 |
Health-related quality of life | Reference to quality of life, such as the importance or the negative/positive effects that COPD has on quality of life | .86 |
Standardize | Reference to the standardization of treatment options and management of COPD for patients | .75 |
Statistics | References to numerical data related to COPD, such as number of deaths caused by COPD or the number of people living with COPD | .80 |
aChronic obstructive pulmonary disease
eDelphi panelists’ sociodemographic characteristics (n=37).a
Demographic variable |
|
Patients |
Informal caregivers |
Clinicians |
Researchers |
Unidentifiedb
|
|
|
|
|
|
|
|
|
Female | 11 (47.8) | 3 (100.0) | 0 (0.0) | 3 (50.0) | 2 (33.3) |
|
Male | 11 (47.8) | 0 (0.0) | 2 (100.0) | 3 (50.0) | 1 (16.6) |
|
Other | 1 (4.4) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
|
|
|
|
|
|
|
|
White | 15 (65.2) | 3 (100.0) | 1 (50.0) | 5 (83.3) | 2 (33.3) |
|
Black/African American | 6 (26.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 1 (16.6) |
|
Hispanic or Latino | 3 (13.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 1 (16.6) |
|
Other | 3 (13.0) | 0 (0.0) | 1 (50.0) | 1 (16.6) | 1 (16.6) |
|
|
|
|
|
|
|
|
Less than a high-school degree | 2 (8.7) | 0 (0.0) | 0 (0.0) | 1 (16.6) | 2 (8.7) |
|
High-school/general educational development | 5 (21.7) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
|
Some collegec | 16 (69.5) | 3 (100.0) | 2 (100.0) | 6 (100.0) | 1 (16.6) |
|
|
|
|
|
|
|
|
<US$ 25,000 | 18 (78.2) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
|
US$ 25,000-US$ 49,999 | 4 (17.3) | 1 (33.3) | 0 (0.0) | 0 (0.0) | 3 (50.0) |
|
>US$ 50,000 | 0 (0.0) | 1 (33.3) | 2 (100.0) | 6 (100.0) | 0 (0.0) |
|
|
|
|
|
|
|
|
Married or widowed | 7 (30.4) | 1 (33.3) | 2 (100.0) | 5 (83.3) | 2 (33.3) |
|
Divorced | 8 (34.7) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
|
Separated | 4 (17.3) | 0 (0.0) | 0 (0.0) | 1 (16.6) | 0 (0.0) |
|
Never married | 4 (17.3) | 2 (66.6) | 0 (0.0) | 0 (0.0) | 1 (16.6) |
aPanelists identified with more than 1 stakeholder group (n=3).
bThree (n=3) individuals who did not identify their stakeholder role did not provide any demographic information.
cSome college is defined as completing at least 1 year of coursework in the college/university setting.
One-hundred and thirty-seven patients living with COPD in the community met eligibility criteria to serve on the virtual advisory board. Fifty-four patients (39%) agreed to participate; however, less than half of those who agreed to participate (n=23) completed the first-round survey. An equal number of female (n=11) and male (n=11) patients joined the panel (mean age 57.65 years, SD 6.80 years). Most patient panelists identified as being white (n=15, 65%), with fewer identifying as black/African American (n=6, 26%). The majority of patient panelists (n=16, 70%) reported completing at least one year of college; yet over 75% of patients (n=18) reported annual household incomes less than US$ 25,000/year. Almost three quarters of patients were either divorced (n=8), separated (n=4), or never married (n=4).
Only 3 informal caregivers agreed to join the virtual panel. All informal caregiver panelists were women, white, and reported completing at least one year of college. The mean age of informal caregivers was 41 years (SD 9.54 years). Annual household income for each informal caregiver varied, with a panelist reporting an income less than US$ 25,000, another reporting US$ 25,000-US$ 49,999, and another reporting US$ 50,000 or more. Two informal caregivers indicated they had never been married, and 1 was currently married or widowed.
Thirty-seven clinicians working at the university where the research took place were eligible to serve as panelists, but 2 clinicians could not be reached via email. Of the 35 clinicians who were successfully contacted, only 2 (retention rate, RR, 6%) agreed to participate. All clinician panelists were men (mean age 36.50 years, SD 12.02 years). One clinician identified as being white, and the second clinician noted his race as “other.”
Seventeen research scientists from the university where the research took place were eligible to serve as panelists, but 1 researcher could not be reached via email. Of the 16 researchers who were successfully contacted, 6 (RR 38%) agreed to participate. There were an equal number of female (n=3) and male (n=3) research scientists (mean age 38 years, SD 6.48 years). Research scientists identified as being white (n=5, 83%) and non-Hispanic (n=6, 100%).
Three panelists (8%) either did not want to disclose their stakeholder role or did not select one of the available stakeholder group options. Panelists who did not associate with one particular group reported a mean age of 50.33 years (SD 1.53 years), and identified their race/ethnicity as white (n=2, 33%), black or African American (n=1, 17%), Hispanic/Latino (n=1, 17%), other (n=1, 17%), or missing (n=1, 17%). All panelists in this group reported an annual household income less than US$ 25,000.
Eight panelists who completed the first-round survey did not complete the third-round survey. Study drop out was observed among all stakeholder groups: patients (n=2), informal caregivers (n=1), clinicians (n=1), and research scientists (n=2). A greater number of women (n=4) dropped out as compared with men (n=2). The highest dropout rate was observed among those who identified as white (n=5). Panelists reporting 1 year of college education or more (n=5) dropped out more often than panelists with other education levels. Most panelists who dropped out reported annual household incomes less than US$ 25,000 (n=6).
Of the 189 stakeholders representing all groups who were successfully contacted about participating on the virtual panel, 37 completed the first-round survey (RR 20%).
In the first-round survey, all care coordination topics received mean importance ratings of 6 or higher. “Measuring hospitalization risk,” received the highest mean rating (mean 6.55, SD 0.67), with over 90% of panelists (n=35) agreeing that this topic was important to investigate in future CER. Over 90% of the panelists also agreed that 4 other topics were especially important areas for future CER: “management of COPD with other chronic conditions” (n=35, mean 6.54, SD 0.61), “pulmonary rehabilitation as a model for care in COPD” (n=34, mean 6.47, SD 0.60), “quality of care coordination” (n=34, mean 6.54, SD 0.65), and “measurement of quality of care” (n=34, mean 6.43, SD 0.96). The care coordination topic that received the lowest mean rating was “patient-centered medical home” (mean 6.29, SD 1.00). As much as 72% of panelists (n=27) agreed that this topic was important to investigate.
Thirty-five of the 37 original panelists completed the second-round survey (RR 95%). As was the case in the first-round survey, all 10 care coordination topics received mean importance ratings above 6. In the second-round survey, “comprehensive COPD patient education” received the highest mean rating (mean 6.65, SD 0.81), with 91% of panelists (n=32/35) agreeing that this topic was important to investigate in future CER. Two other topics were also highly rated by a majority of panelists (≥88%; n=31/35): “pulmonary rehabilitation as a model for care in COPD” (mean 6.45, SD 1.12) and “quality of care coordination” (mean 6.39, SD 1.36). Interestingly, the topic, “management of COPD with other chronic conditions,” which was highly rated in the first round, scored lowest on perceived importance in the second round (mean 6.14, SD 1.44); however, over 85% of the panelists (n=30/35) agreed that this topic was important to study in future CER. Conversely, less than 80% of panelists agreed that “measuring hospitalization risk” (n=24/35) and “patient-centered medical home” (n=26/35) were important topics. Because consensus agreement was not reached on the importance of these 2 topics, panelists were not asked to evaluate either of these topics in the round-three survey.
First- and second-round survey ratings for chronic obstructive pulmonary disease care coordination topics.
COPD care coordination topica | Round one (N=37) | Round two (N=35) | ||
Mean (SD) | Consensus agreement |
Mean (SD) | Consensus agreement |
|
Management of COPD with other conditions | 6.54 (0.61) | 35 (94) | 6.14 (1.44) | 30 (85) |
Pulmonary rehabilitation | 6.47 (0.60) | 34 (91) | 6.45 (1.12) | 31 (88) |
Depression and mental health management | 6.38 (0.76) | 33 (89) | 6.44 (0.70) | 30 (85) |
Quality of care coordination | 6.54 (0.65) | 34 (91) | 6.39 (1.36) | 30 (85) |
Measurement of quality of care | 6.43 (0.96) | 34 (91) | 6.46 (0.78) | 31 (88) |
Comprehensive COPD patient education | 6.50 (0.77) | 32 (86) | 6.65 (0.81) | 32 (91) |
Cost effectiveness of care | 6.54 (0.78) | 31 (83) | 6.53 (0.71) | 30 (85) |
Case management | 6.34 (0.84) | 31 (83) | 6.36 (1.14) | 29 (82) |
Measuring hospitalization risk | 6.55 (0.67) | 30 (81) | 6.33 (0.89) | 24 (68) |
Patient-centered medical home | 6.29 (1.00) | 27 (72) | 6.18 (1.07) | 26 (74) |
aChronic obstructive pulmonary disease
bConsensus agreement was calculated by reporting the percentage of stakeholders who selected either 6 (agree) or 7 (strongly agree) on the 7-point Likert scale when evaluating each COPD care coordination topic.
Twenty-nine participants completed the final third-round survey (RR 78%).
Mean (SD) rankings of 8 remaining COPD care coordination topics (n=29).a
COPDb care coordination topic | Mean (SD) |
1. Measurement of quality of care | 2.73 (1.95) |
2. Management of COPD with other conditions | 2.92 (1.67) |
3. Pulmonary rehabilitation | 3.73 (1.93) |
4. Quality of care coordination | 4.12 (2.41) |
5. Comprehensive COPD patient education | 4.27 (2.38) |
6. Depression and mental health management | 4.62 (2.04) |
7. Cost effectiveness of care | 5.61 (2.33) |
8. Case management | 6.00 (2.28) |
aTopics were ranked from 1 (most important) to 8 (least important).
bChronic obstructive pulmonary disease
Panelists provided 22 type-written comments that described reasons for their final rankings. Almost all of the comments (n=18/29, 62%) discussed the overall importance of strategies for COPD management. One panelist commented,
Given that comorbid conditions are commonly encountered together, along with a number of other chronic medical conditions and medications, clear management strategies in these types of populations are important.
Patients described difficulty when attempting to manage medications intended to treat multiple health conditions. One patient stated,
I believe that if you manage this condition it will help overall with the other situations that one is faced with. I have thyroid disease along with vertigo and COPD. Having multiple conditions is very hard to manage; taking more than one medication and the different effects that they can have.
Panelists from almost all groups also commented on the need to determine the most effective ways to facilitate effective patient-provider communication. Miscommunication between stakeholder groups was cited frequently by panelists (n=20/29, 69%). One panelist who identified as both an informal caregiver and a clinician noted,
It has been my experience as an RN and family caregiver of my parents who both had COPD that there is a lack of communication between medical disciplines and grasp by the medical team of the overall health of COPD patients.
One research scientist also observed that
There are so many miscommunications between patients [with COPD] and providers.
In addition, panelists noted the difficulty experienced by patients and providers in terms of identifying which patient education topics need to be covered with individual patients dealing with various stages of COPD. One panelist noted,
Although all the topics are very important for the care and results of the treatment of COPD, if a patient doesn't understand the basic importance of them, I don't believe they will be as effective. Especially the procedures.
When panelists were asked about the potential impact of a Web-based stakeholder network devoted to CER in COPD, they noted the importance of accessing results from recent CER studies (mean 2.34, SD 1.19). Patients, informal caregivers, and research scientists all indicated a strong desire to learn from others affected by COPD to build reciprocal trust and communication (mean 2.34, SD 1.19). Most panelists (n=22/29, 76%) also reported that they were likely to visit such a network website at least a few times/month. Panelists perceived their roles/responsibilities for network involvement to be primarily meeting with others interested in COPD (n=19/29, 66%), and suggesting, commenting on, and voting for new CER ideas in COPD (n=18/29, 62%). Finally, more than half of the panel noted the importance of accessing and learning from the following resources: COPD educational videos (n=24/29, 83%), news updates on CER taking place in COPD (n=18/29, 62%), and downloadable CER summaries (n=18/29, 62%). Patients and research scientists primarily requested access to educational videos on COPD care coordination, whereas clinicians preferred having Web access to recent research articles on CER in COPD.
The coordinating capacity of health care settings is often determined by the approaches and coordination activities used to deliver care. Achieving a good fit between coordination needs and coordinating capacity is key for effective and efficient care; however, the adequacy of fit is perceived differently by different stakeholders who are involved in processes of care at different levels. Traditionally, patients, informal caregivers, and health care providers have different perspectives on health care processes [
Because of this shift to patient-centered research designs, the primary purpose of this eDelphi study was to use a Web-based structured communication process to connect community-based stakeholders to discuss, generate, and evaluate potential COPD care coordination research topics that can be studied using CER methods. In this study, a team of faculty and clinical staff, in collaboration with research navigators working on behalf of a CEnR program, helped to organize a virtual advisory board consisting of stakeholders that represented 4 different stakeholder groups (patients, informal caregivers, clinicians, and research scientists). Results suggested that measuring quality of care delivered was the most important care coordination topic perceived by members of the virtual advisory board. Quality of care during the care transition process is especially critical for patients with COPD who are often treated for breathing exacerbations in the hospital or emergency room (ER). Often, these patients must proactively maintain pharmacotherapy and rehabilitation regimens based on changes in respiratory and physical condition [
The virtual advisory board further commented on the importance of disease management, communication, and education as key components to consider when assessing quality of care. For example, comprehensive patient education was noted to be tied to most all other care coordination topics evaluated. Collaborative patient education and stakeholder communication based on shared objectives were identified as critical to high-quality COPD care coordination both inside and outside of the clinical setting. Specifically, panelists commented on the need to evaluate different self-management education tools that patients could benefit from by successfully coping with the day-to-day challenges they encounter. Direct stakeholder feedback on these important needs can now be used as the basis for the development and evaluation of care-quality metrics assessing alternative COPD care coordination approaches used in this patient community.
Finally, to facilitate continued communication regarding research priority setting, a follow-up informal needs assessment was conducted following the main study to evaluate stakeholder interest in belonging, and contributing, to a collaborative Web-based community network for advancing CER in COPD. Findings suggested that the existence of a Web-based community would promote stakeholder understanding, engagement, and shared decision making, while building a new social media channel for generating, evaluating, and disseminating CER in COPD. Most panelists reported that they would be willing to regularly participate on a Web-based stakeholder engagement network to provide feedback on how to improve CER methods in COPD and review research summaries on the latest CER findings in COPD. This feedback will now be used to shape the purpose, scope, and function of the proposed Web-based stakeholder engagement infrastructure.
Previous research suggests that organizations often fail to capitalize on stakeholder engagement opportunities by only fostering one-way communication [
Similar to prior research [
A recent systematic literature review by Moorhead and colleagues [
On the network home page, users can click on videos uploaded using the YouTube media player that describe the 5 most important care coordination topics as rated by panelists in our study. Users are also able to post and respond to comments regarding the video content using textboxes attached to each video.
Screenshot of prototype version of the COPDFlix CER Network home page.
Screenshot of care coordination topic video and user comment box on COPDFlix CER Network page.
Low initial response rate was one limitation of this study, with only 37 of 189 contacted (20%) stakeholders participating on the virtual panel. Reasons for the low response rate included the following: (1) lack of patient access to a computer; (2) low knowledge on how to use a computer; and (3) time constraints. While the use of only 1 CEnR program for recruitment may have limited the total number of respondents, the focus of this study was on gathering opinions from community-based stakeholders who could potentially be involved with the development and evaluation of CER proposals in COPD. Adding a paper-based alternative for the round-by-round questionnaires may have increased the response rate from patients and informal caregivers. In addition, the use of Web-based surveys may have limited the generalizability of findings to only those medically underserved patients with computer access and experience. However, it is important to note that several patients were able to travel to the CEnR program headquarters and complete the Web-based survey in their computer laboratory with assistance from program staff. The low initial response rate for this research priority-setting study highlights the need for future Web-based stakeholder engagement research within low SES populations. This type of research would benefit from incorporating computer and Internet literacy trainings prior to and during all research-related activities.
Few informal caregivers (n=3) were enrolled into the study at any point in time, which we attributed, in part, to the snowball sampling referral method used to recruit stakeholders from this group. Informal caregivers could not be contacted directly by research navigators affiliated with the CEnR program, because of patient privacy risks and the potential for informal caregivers feeling coaxed into participation. In addition, several individuals living with COPD felt that they did not need assistance from a family member or friend to manage their COPD, and therefore, they reported having no informal caregivers. Future studies seeking input from informal caregivers may benefit from using alternative sampling methods and seeking out members from informal caregiver registries operated by local/regional health care agencies and organizations.
Similarly, few eligible research scientists (n=6) agreed to participate on the virtual panel. Initial panel recruitment occurred primarily in the summer months, which may have conflicted with the regular academic calendar (fall/spring) for some researchers. Clinicians were the least represented stakeholder group on the panel, with only 2 of the 35 clinicians agreeing to serve as a panelist. Potential reasons for the very low response rate among clinicians may have included lack of time, interest, and limited incentive for participation (US$ 45 gift card). Because of the nonrandom and very small samples in several stakeholder groups, findings from this study may not be representative and generalizable to the populations they represent. Nevertheless, the purpose of this Web-based stakeholder engagement study was to establish the validity and reliability of using the eDelphi technique to generate CER priorities in a community where previous research has not been conducted, and therefore, our methodological emphasis was appropriately on maximizing internal validity versus establishing external generalizability. Future stakeholder consensus-building effort may benefit from investing more time developing collaborative relationships with key clinician/research scientist gatekeepers (eg, leadership of local medical societies, senior clinicians, chairs of respiratory health departments) prior to study recruitment. Further development of our COPDFlix CER Network is expected to improve stakeholder recruitment effort in future CER studies.
Finally, the expertise of eDelphi panelists (patients, informal caregivers, clinicians, and research scientists) was not uniform. Stakeholders from different groups likely possessed different levels of knowledge regarding CER and COPD care coordination. Some panelists chose not to identify their stakeholder group affiliation, because they may have felt uncomfortable providing this information in this university-based study. There was also potential for intellectual, financial, and clinical conflicts of interest, which may have biased rankings, especially among research scientists and clinicians. To our knowledge, and as acknowledged by other researchers [
Although engaging representatives of diverse stakeholder groups is feasible and can be used to identify support for CER topics in COPD [
Effectively engaging patients with COPD from various racial, ethnic, aged, and low-resourced backgrounds is important during all stages of the CER process (eg, planning, data collection, choice of intervention, analysis of results, and implementation of findings into clinical practice). Findings from this study support the need to conduct more CER that explores whether the use of eHealth and social media represents an acceptable and effective way to engage low-income and racial/ethnic minority populations for CER in COPD. Web-based stakeholder networks have the potential to engage more stakeholders and improve the study of COPD care coordination using CER methods. Future studies would benefit from collecting more robust data from larger, more representative samples of community-dwelling stakeholders interested in improving COPD care.
community engagement and research
comparative effectiveness research
chronic obstructive pulmonary disease
emergency room
information and communication technologies
retention rate
socioeconomic status
This work was supported by the Office of the UF Health Executive Vice President for Research and Education and NIH (NCATS) CTSA awards to the University of Florida (UL1TR000064 and KL2TR000065).
None declared.