This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Complementary and alternative medicine (CAM) practitioners, such as chiropractors, acupuncturists, and massage therapists, are a growing presence in the US health care landscape and already provide health and wellness care to significant numbers of patients who use tobacco. For decades, conventional biomedical practitioners have received training to provide evidence-based tobacco cessation brief interventions (BIs) and referrals to cessation services as part of routine clinical care, whereas CAM practitioners have been largely overlooked for BI training. Web-based training has clear potential to meet large-scale training dissemination needs. However, despite the exploding use of Web-based training for health professionals, Web-based evaluation of clinical skills competency remains underdeveloped.
In pursuit of a long-term goal of helping CAM practitioners integrate evidence-based practices from US Public Health Service Tobacco Dependence Treatment Guideline into routine clinical care, this pilot protocol aims to develop and test a Web-based tobacco cessation training program tailored for CAM practitioners.
In preparation for a larger trial to examine the effect of training on CAM practitioner clinical practice behaviors around tobacco cessation, this developmental study will (1) adapt an existing in-person tobacco cessation BI training program that is specifically tailored for CAM therapists for delivery via the Internet; (2) develop a novel, Web-based tool to assess CAM practitioner competence in tobacco cessation BI skills, and conduct a pilot validation study comparing the competency assessment tool to live video role plays with a standardized patient; (3) pilot test the Web-based training with 120 CAM practitioners (40 acupuncturists, 40 chiropractors, 40 massage therapists) for usability, accessibility, acceptability, and effects on practitioner knowledge, self-efficacy, and competency with tobacco cessation; and (4) conduct qualitative and quantitative formative research on factors influencing practitioner tobacco cessation clinical behaviors (eg, practice environment, peer social influence, and insurance reimbursement).
Web-training and competency assessment tool development and study enrollment and training activities are complete (N=203 practitioners enrolled). Training completion rates were lower than expected (36.9%, 75/203), necessitating over enrollment to ensure a sufficient number of training completers. Follow-up data collection is in progress. Data analysis will begin immediately after data collection is complete.
To realize CAM practitioners’ potential to promote tobacco cessation and use of evidence-based treatments, there is a need to know more about the facilitative and inhibitory factors influencing CAM practitioner tobacco intervention behaviors (eg, social influence and insurance reimbursement). Given marked differences between conventional and CAM practitioners, extant knowledge about factors influencing conventional practitioner adoption of tobacco cessation behaviors cannot be confidently extrapolated to CAM practitioners. The potential impact of this study is to expand tobacco cessation and health promotion infrastructure in a new group of health practitioners who can help combat the continuing epidemic of tobacco use.
Complementary and alternative medicine (CAM) practitioners such as chiropractors, acupuncturists, and massage therapists are a growing presence in the US health care landscape as a significant proportion of Americans report using CAM. The National Health Interview Survey (NHIS) periodically includes additional questions (Adult Alternative Medicine (ALT) supplement) on the use of CAM therapies. Clarke et al [
Tobacco use accounts for nearly 443,000 deaths in the United States each year and is responsible for US $96 billion in health care expenditures annually [
Decades of public health tobacco control efforts have led to steady declines in tobacco use prevalence from 40.3% in 1964 to 19.3% in 2011, and 19.2% in 2013 [
With a few exceptions [
Web-based training has clear potential to meet large-scale training dissemination needs. Yet, despite widespread use of Web-based instruction for training health professionals [
Innovative aims of this study include (1) shift the current public health and clinical practice paradigm in tobacco cessation training, currently focused on conventional biomedical practitioners, to include training CAM practitioners in evidence-based knowledge and skills to promote tobacco cessation and encourage use of effective treatments; (2) develop a unique, Web-based tobacco cessation BI training program for CAM practitioners to improve their competency in helping their clients or patients quit tobacco; (3) develop a new Web-based tool to assess CAM practitioners’ clinical skills competency at providing tobacco BIs; and (4) fill a gap in the literature by collecting new information on CAM providers regarding online BI training, clinical competency testing, and the facilitating and inhibiting role of practice environment, insurance reimbursement, and training availability on adoption of tobacco cessation BI behaviors. Initial components of the study that have already been completed are described in the Results section. We note that the three CAM disciplines participating in the CAM Reach Web (CAMR-Web) study customarily use different terms to refer to persons seeking their care. Chiropractors and acupuncturists usually refer to “patients”, whereas massage therapists usually say “clients”. For simplicity, we will use “patients” throughout this paper.
The CAMR-Web study is a mixed-method (qualitative and quantitative) pilot feasibility study. Theoretical frameworks informing this work are Social Cognitive Theory [
CAM practitioners differ from conventional practitioners in substantive ways such as professional training, scope of practice, philosophies of practice and approaches to healing; practice patterns, and patient or client relationships [
Despite growth of Web-based training of health care practitioners, effective Web-based methodologies to assess clinical competencies lag behind. “Standardized patient” is an evaluation method widely used in medical education for assessing clinical skills competency [
Phase 1 of the study (completed) involved evaluation of the original CAMR study training curriculum for the adaptation necessary for Web-based training delivery of the content, followed by design and production of the training website. Specific methods for accomplishing these aspects are detailed in the following sections.
The curriculum and Web development teams worked collaboratively in the creation of the CAMR-Web training content map and specifications to best suit the curriculum. This team of investigators and staff had extensive experience working together to develop online tobacco cessation trainings. The content map and specifications served as a guide for the design process which began with storyboarding interactive learning activities and course content, followed by developing “screenplay” versions of learning activities and content and identifying needed animation or video assets and/or interactive simulations to reinforce and demonstrate desired instructional objectives within the content map. Database connectivity was designed to facilitate (1) the collection of demographic data from users in a convenient, easy-to-analyze format; (2) the use of open- and closed-ended evaluation measures; (3) the collection and instantaneous scoring of nominal and ordinal evaluation data; (4) the ability to track user progress through curriculum modules and the site as a whole; and (5) the ability to deploy dynamic, customized Web pages created in real-time for each user.
Investigators adapted the existing CAMR in-person, multimedia training to best suit the online learning environment. Continuing the highly successful, inclusive approach to developing the CAMR in-person curriculum, adaptation of the curriculum for Web-based delivery was conducted in consultation with external reviewers from our local and national advisory panels. Upon project startup, local advisors were invited from our pool of completed CAMR study participants. In tandem with this curriculum content conversion, we developed customized clinical cases to assess core competencies addressed by the CAMR Web online curriculum. We then deployed the clinical cases through the DecisionSim simulation development learning platform [
Evaluation instruments including screening, baseline, pretraining and post-training assessments, and follow-up surveys were developed or adapted from existing instruments developed for the CAMR study [
Using DecisionSim, the summative assessment is an online simulation of a case-based practice of communication skills and application of tobacco cessation knowledge. The learner is presented with a tobacco-using patient or client scenario for which an “ideal” interaction path has been previously specified by investigators, in consultation with our advisors. Each response option is associated with a tag (in the back-end database) as being “optimal”, “critical”, or “poor.” To successfully complete the training a learner needs to meet a minimum level of competence as defined by selection of a response path that is within a specified range of the “ideal” interaction path. To evaluate the online competency tool, we also developed a standardized patient “live simulation” role play exercise in which participants role play the exercise scenarios with a standardized patient via Skype video. Live simulation cases reflect a range of patients’ readiness to quit and evaluate the practitioner’s ability to assess the patient or client and adjust their BI behaviors accordingly. Standardized patient cases, scripts, and examiner checklists for the feasibility study were developed in tandem with simulation cases for the competency assessment tool. For both the live and online simulation cases, we developed related evaluation criteria and thresholds for determining competency. Development of evaluation criteria and thresholds were guided by published, evidence-based tobacco cessation behavioral support competencies and guidance documents (eg, those supported by national tobacco cessation programs [
The CAMR website underwent rigorous user and usability testing with local and national panel members as well as with members of the target population of CAM practitioners. Investigators reviewed the initial version of the training program and made modifications to improve usability. The initial version was also thoroughly tested in-house for code errors. It was tested for usability with volunteer CAM practitioners (N=18; 6 chiropractors, 3 massage therapists, 9 acupuncturist) with protocol analysis. Practitioners were given a set of tasks to mimic real-world use (eg, registering on the site, viewing the training, completing the competency assessment). Testers completed a postsurvey on ease of use and attractiveness of screen design [
Near completion, the CAMR website was beta tested with CAM practitioners and advisory panel members (N=12). The goal of beta testing was to confirm that revisions made in response to usability study findings eliminated identified problems and that all site features function as intended. Beta testing also gauged the instructional effectiveness of the curriculum. The process followed protocols suggested by Trollip and Alessi [
The feasibility study design is a single group with participant assessments at (1) baseline (study enrollment) and pretest prior to CAMR-Web training intervention; (2) immediately post-training; and (3) 3 and 6 months post training completion. A subsample of practitioners from each practitioner type was selected based on specific criteria to complete an in-depth, open-ended response follow-up survey on selected factors influencing practitioner adoption of tobacco intervention behaviors (eg, practice environment, peer social influence, and insurance reimbursement). Phase 2 participant enrollment is complete and data collection is still in progress.
Based on literature regarding attrition among online learners [
To participate, practitioners must be ≥18 years of age, have unrestricted license or credentials to practice their CAM discipline, be actively in practice three quarters time to full-time, see at least 10 patients per week, have access to a computer with broadband Internet access and audio output, and be willing to provide consent and participate in the entire study. Practitioners were excluded if they reported participation in formal tobacco cessation training in the past 2 years.
Practitioners arriving at the project website were presented with a project overview. To enroll, practitioners first completed the screening instrument to determine eligibility. Responses to the screening items were processed immediately and the eligibility status displayed to the practitioner. Eligible participants were directed to a link to the consent portal. This portal displayed the consent document in its entirety with a link to download and print the form. To continue with enrollment, the participant was required to click a button indicating that he or she has read and agrees to all parts of the online consent form. Next, the online baseline and pretraining survey were presented for completion. (
Enrolled practitioners experience the CAMR-Web tobacco cessation BI curriculum [
After training, practitioners are asked to complete Web-based questionnaires at 3 and 6 months post training completion (
Measure and addressed domains
Screening
Eligibility criteria
Baseline and pretest assessment
Demographics
Cessation intervention behaviors, attitudes regarding tobacco use, quitting, and intervening with patients
Confidence and self-efficacy in performing intervention behaviors
Training-related knowledge [
Post-training assessment
Cessation intervention behaviors, attitudes regarding tobacco use, quitting and intervening with patients,
Confidence and self-efficacy in performing intervention behaviors,
Training-related knowledge,
Training site usability [
Web-based skills competency assessment
Ability to recognize and select appropriate: (1) Techniques to ask or become aware of patient or client tobacco use; (2) Communication skills to assess tobacco use of client/patient; (3) Advice and encouragement to consider quitting tobacco; (4) Assistance (eg, referral to guideline based services and provision of materials or information); (5) Arrangements to follow up with patient or client regarding tobacco use.
Follow-up assessments at 3 and 6 months
Cessation intervention behavior and use of training materials,
Confidence and self-efficacy in performing intervention behaviors
In-depth, open-ended follow-up survey
Open-ended, in-depth survey with a subsample of participants
This feasibility pilot study is using the University of Arizona College of Medicine’s standard protocols for training standardized patients. To test feasibility for a larger validation study, we recruited a subsample of 10 CAM practitioners who completed the CAMR-Web training (n=3-5 per practitioner type) to participate in the live video standardized patient interviews to assess CAM practitioner skill competency. Eligible participants must have completed training within the prior 2 weeks and have access to bi-directional Internet video capability (eg, Skype) sufficient to participate in a 30-minute, live video, standardized patient role-play session. Each participant is evaluated with two standardized patient cases (1 ready to quit, 1 not ready to quit), and each case is scored separately. Practitioners are not offered feedback on their performance to minimize the impact of the standardized patient on practitioner behavior during the remainder of the follow-up period. Practitioners who complete the role-play are then invited to complete a survey about usefulness of the activity. Their live assessment scores are then compared to their online competency assessment scores.
Approximately 6 months after training completion, practitioners are screened for eligibility to complete an in-depth online survey with open-ended response options. Inclusion criteria into this survey require that (during their 3-month follow-up survey) practitioners report seeing tobacco users in their practice and respond with “always” or “often” on a 4-point scale (always, often, sometimes, never) to at least 3 of the following (1) assessing interest in quitting; (2) identifying reasons to quit; (3) offering materials; (4) discussing medications; (5) treating tobacco addiction; or (6) referring to tobacco treatment outside of their practice.
Open-ended responses from the online survey will be analyzed using qualitative methodology to gain a more detailed understanding of practitioners’ experiences as they apply training in real-world settings [
Data for power analysis came from pretest and post-test assessments of knowledge and confidence (self-efficacy) in the previous in-person CAMR training study. We compared the number of correct answers on a 15-item tobacco cessation knowledge test administered before and after training for 33 practitioners. Scores on the items approximated a normal distribution. A paired
The pretest and post-test confidence question was “I am confident that I can personalize the benefits of quitting with each individual patient”. The possible responses ranged from 1 (“Not at all confident”) to 4 (“Very confident”). The scores on this question approximated a normal distribution. A paired
The process of using the CAMR-Web training website is being evaluated by assigning CAM providers IDs, which mark usage in the backend database, including time spent, pages viewed, and modules completed. The association of use of the website with change in knowledge and confidence will be assessed using regression analysis methods. These analyses will be limited by use not being randomly assigned so potential third variables (eg, demographics, years in practice, patient volume) will be included as covariates.
In so far as knowledge and confidence were measured at baseline, 3, and 6 months (for three practitioner types), changes will be analyzed using mixed-models [
Qualitative analysis of open-ended survey responses will use a coding-categorizing technique [
We have enrolled 203 practitioners (63 chiropractors, 78 acupuncturists and traditional Chinese medicine practitioners, and 62 massage therapists). Initial training completion rates varied by CAM discipline, but overall the average completion rate was lower (36.9%, 75/203) than we expected based on our prior experience with other Web-based training. We therefore overenrolled to meet the original target of 20 practitioners per CAM group completing training and 3-month follow-up. At the time of writing, all training activities have been completed and follow-up survey data collection is in progress. Final data collection will be completed in November 2015. In Phase 3, we will analyze the main outcomes, synthesize results from quantitative and qualitative data, evaluate the feasibility of a larger competency assessment tool validation study, and address the feasibility outcomes in preparation for a larger dissemination study.
Focusing tobacco BI training on conventional practitioners limits BIs’ potential public health impact. Unhealthy lifestyle behaviors are the root cause of the growing burden of chronic disease in the United States [
Factors impacting CAM practitioner tobacco cessation behaviors are not well studied, especially regarding adoption of PHS Guideline-based tobacco dependence treatment practices. Among conventional practitioners, training [
CAM practitioners are an increasingly important presence in the US health care system and already provide health and wellness care to significant numbers of patients who use tobacco and/or have other chronic disease(s) and risk factors. There has been scant research, however, into the potential factors impacting on CAM practitioners’ willingness and ability to adopt evidence-based tobacco cessation behaviors as part of routine clinical practice. These factors include cessation training adapted to their professional discipline, professional scope of practice, and approaches to health and wellness. Other potential factors include practice organization and business models, and the availability of third party insurance reimbursement for tobacco cessation services. The keys to insurance reimbursement are proven competency and credentials, and evidence-based deployment of an effective intervention. To engage CAM practitioners in tobacco cessation, it is critical to know how best to increase incorporation of evidence-based interventions and practices into their routine clinical care. Similarly, research on the effect of insurance reimbursement is critical to develop evidence-based policy for CAM practitioners to provide effective cessation interventions. The potential impact of this study is to build infrastructure in a new group of health practitioners who can help combat the continuing epidemic of tobacco use.
Adult Alternative Medicine
brief interventions
chiropractors, acupuncturists, and massage
CAM Reach
CAM Reach Web
National Health Interview Survey
Objective Structured Clinical Examination
Public Health Service
This work was supported by a grant from National Cancer Institute (R21 CA175450). The authors gratefully acknowledge the members of the CAMR local and national advisory panels for their participation and thoughtful commentary prior to launching the study. We also appreciate the efforts of the research team for their work in implementing the study: Eva Matthews, Lubna Govindarajan, Perlana Howard, and Emily Clifford.
All authors contributed to the study conceptualization and participated in its design and intervention development. MM, AH, and EE helped draft the manuscript. All authors read and edited for significant intellectual content and approved the final manuscript.
None declared.