This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Web-based cessation interventions have been shown to reduce tobacco use, be more efficacious than self-help booklets, be more efficacious if they provide tailored messages, and enhance quit rates in conjunction with nicotine replacement therapy.
The objective of this study was to usability test and pilot test the Tobacco Tactics website for veterans.
Both formative and summative evaluations were used across three small successive studies to develop and test the Tobacco Tactics website for veterans, which was based on a prior face-to-face smoking cessation intervention. Once the website was developed, the research team and Web developers usability tested the website with 5 veteran smokers and former smokers. Feedback from the veterans was collected as they navigated each webpage, then used to revise the website. In pilot study 1, 9 veteran smokers were provided access to the website, and given a baseline and 30-day follow-up survey. In pilot study 2, 18 veteran smokers, who were also motivated to quit smoking, were recruited and randomized to either the Tobacco Tactics website plus nicotine replacement therapy or to the 1-800-QUIT-NOW telephone line.
As a result of usability testing, more than 27 modifications were made to improve the website. In pilot study 1, 50% (3/6) veterans who entered the website had cut down on the number of cigarettes and 83% (5/6) found the website enjoyable, easy to read, easy to navigate, and would recommend the website to others. In pilot study 2, which included only smokers motivated to quit and also offered nicotine replacement therapy, seven-day point prevalence abstinence at 30-day follow-up was 40% (4/10) in the intervention group compared to 13% (1/8) in the control group.
These preliminary results are promising and suggest the need for wider-scale testing of the Tobacco Tactics website for veterans.
While smoking rates among veterans have decreased from 33% in 2001 to 22.2% in 2005, smoking remains especially pronounced among subgroups of veterans, such as returning Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) veterans [
Within the VA, outpatient programs provide the majority of services to smokers, in accordance with the Agency for Healthcare Research and Quality (AHRQ) guidelines [
Social marketing theory, which was used to design and refine the Tobacco Tactics website, is the planning and implementation of programs designed to bring about social change. Social marketing uses concepts from commercial marketing including the “4 Ps”: (1) create an enticing “Product” (ie, the package of benefits associated with the desired action such as the tailored Tobacco Tactics website), (2) minimize the “Price” the target audience believes it must pay in the exchange, such as easy access to the website and cessation medications, (3) make the exchange available in “Places” that reach the audience and fit its lifestyles, such as accessing the website from home or public libraries, and (4) “Promote” the exchange opportunity with creativity and through channels and tactics that maximize desired responses such as a tailored, multimedia, interactive website enhanced with medications and follow-up [
The Tobacco Tactics website was based on the face-to-face Tobacco Tactics intervention (VA IIR 98-500), which was tested among head and neck cancer patients and showed a significant difference in 6-month smoking cessation rates when compared to an enhanced usual care group [
Web-based cessation interventions have been shown to reduce tobacco use [
The Internet is a valuable tool for providing health education and support given its 24-hour accessibility, growing number of Internet users, potential to reach thousands of individuals, and potential cost-effectiveness [
While smoking cessation websites are already available, many Internet users encounter frustrations on sites that are poorly designed. A study of 202 cessation websites found that only 46 provided actual cessation treatment, and only 5 of those received high rankings for usability based on evidence-based guidelines [
Both formative (process) and summative (outcome) evaluation were used across three small successive studies to develop and test the Tobacco Tactics website for veterans. Formative evaluation was conducted with a small group of veterans to “test run” the Tobacco Tactics website, obtain feedback, and then revise the website. Summative evaluation was conducted to determine if the website did what it was designed to do in two small pilot studies. Human studies approval was obtained from the VA Ann Arbor Healthcare System and the University of Michigan.
The sample for all three studies consisted of veteran smokers and former smokers who had access to the Internet. The samples for the usability test and first pilot study were recruited at the VA Ann Arbor Healthcare System and were not necessarily smokers who were interested in quitting (usability testing included former smokers) while pilot study 2 consisted of all motivated smokers.
Based on modules used in our prior studies, the Tobacco Tactics website was built with our website developers, Allen Wayne, LTD. First, the research team and website developers sat individually with 5 VA smokers and former smokers (employees or volunteers from Voluntary Services) for one hour as they navigated each successive webpage and requested feedback on: (1) ability to accomplish tasks, (2) ability to accomplish goals with skill and speed, (3) ability to operate the system, and (4) satisfaction with the website. Then, based on the feedback, the website was revised for pilot testing.
Using the revised website, 9 veteran smokers (patients) who had computer access and were willing to test the website at home were recruited from the smoking shelter. Veterans were given a baseline survey and a 30-day follow-up survey to request information about their smoking habits and other covariates that can influence smoking.
To further test the website among smokers motivated to quit who also received NRT, 18 veteran smokers who had computer access and were interested in quitting were recruited from Operating Engineers Locals 324 Training Center as part of a larger funded study. This is a registered randomized controlled study (clinicaltrials.gov NCT01124110). Veterans were randomized to the Tobacco Tactics website plus mailed NRT (ie, patch, gum, or lozenge) or the 1-800-QUIT-NOW telephone line, which also mailed NRT for those without insurance. The specific type of NRT was given based on the number of cigarettes the veteran smoked as well as his/her previous experience with NRT and personal preferences. Veterans were given a baseline survey and a 30-day follow-up survey to request information about their smoking habits and other covariates.
The questions for the usability testing were open-ended. Examples of the open-ended questions were: Why did you click on that? or Why did you skip that exercise?
The outcome of interest was self-reported seven-day point prevalence abstinence at 30-day follow-up. To be considered as having quit, the patient had to self-report on their 30-day follow-up survey that they had “not used any tobacco products, even a single puff of a cigarette, in the past 7 days.” Variables known to be associated with smoking and cessation were measured on the survey using previously validated tools including: (1) the Fagerström Test for Nicotine Dependence (FTND), (2) the Alcohol Use Disorders Identification Test (AUDIT), (3) the abbreviated Center for Epidemiologic Studies Depression Scale (CESD) [
The Tobacco Tactics website was developed based on prior work and is in keeping with guideline recommendations for treatment of tobacco addiction [
As seen in
Treatment links review the medications available to assist with cessation, including NRT (gum, patch, spray and lozenge), Zyban, and Chantix. An interactive exercise provides a tailored medication protocol and contraindications assessment (see
Sample Tobacco Tactics for veterans home page.
Sample Tobacco Tactics for veterans webpages.
Sample Tobacco Tactics for veterans webpage for pharmacists.
Additional buttons on the left side of the home page include “Goal Setting”, “Handling Thoughts About Smoking”, “Coping With Cravings”, “Coping With Relapses”, “Common Problems in Quitting”, “Life As A Nonsmoker”, and buttons that provide relaxation exercises and resources. Tabs across the top of the home page include “About Us”, “Messages”, “Latest News”, and a tab for an e-community (chat room).
Extensive programming has resulted in the highly interactive exercises with tailored messages that save data to “My Plan” so users can monitor their progress over time. For example, a cost calculator depicts the amount of money that can be saved by quitting smoking. The e-community feature of the website consists of a three-day per week (one morning and two evenings), nurse-moderated forum that allows participants to conduct text-based discussions.
The artwork and logo were developed using social marketing techniques based on prior work by Duffy [
Veterans who use the Tobacco Tactics website received a follow-up telephone call encouraging them to use the website and these calls also provided additional counseling. Proactive/warm transfer calls have been used to introduce patients to interventions in other studies [
As a result of usability testing, more than 27 modifications were made including, but not limited to: (1) moving the user login box to the top of the page, (2) making the play button for the video more visible, (3) changing the font color of selected headings from blue so as not to confuse them with a URL link, (4) making the scrolling list of chemicals slower, (5) enlarging the calendar icon when setting a quit date, and (6) changing the wording to be more veteran-centric (eg, “Smoking causes more deaths than all Americans killed in World War I, World War II, and the Korean and Vietnam wars combined”).
Thirteen veteran smokers were approached to recruit 9 who had computer access and were willing to test the website at home. The 9 smokers were not necessarily motivated to quit nor were they provided any cessation medications. Sixty-seven percent (6/9) of veterans accessed the website with prompting by a follow-up telephone call. Overall, the average age was 48 years, 67% (6/9) were male, and 67% (6/9) were white. Sixty-seven percent (6/9) had some college education and 33% (3/9) had less than a high school degree or a high school diploma/GED. One-third of the sample was never married, 22% (2/9) were married, and the remainder were separated, widowed, or divorced. Forty-four percent (4/9) were unemployed. At 30-day follow-up, none of the 6 responders had quit, however, 1 had attempted to quit. Fifty percent (3/6) had cut down on the number of cigarettes, averaging a reduction of 7 cigarettes per day. Most found the website easy to read (n=5), enjoyable (n=4), and easy to navigate (n=4), rated the website highly on pictures, illustrations, and colors, and indicated that they would recommend the website to others (n=5).
To further pilot test the Tobacco Tactics website with veteran smokers who were motivated to quit, 18 veterans recruited from a population of Operating Engineers (heavy equipment operators) were randomized to the Tobacco Tactics website plus NRT versus the 1-800-QUIT-NOW telephone line. Groups were similar on baseline variables. At 30-day follow-up, 80% (8/10) intervention participants responded, while 63% (5/8) control condition responded. Using an intent-to-treat analysis (assuming non-responders in the control group were smokers and did not use NRT),
Characteristics and smoking quit rates for Veteran Operating Engineers randomized to the Tobacco Tactics website intervention compared to control subjects (N=18).
|
|
Baseline | |
|
|
Intervention (N=10) | Control (N=8) |
|
|
Mean (SD) or |
Mean (SD) or |
|
|||
|
Age | 45.6 (7.5) | 46.3 (8.1) |
|
Male | 10 (100%) | 8 (100%) |
|
White | 10 (100%) | 6 (75%) |
|
High school or less than high school | 7 (70%) | 4 (50%) |
|
Marital status | 7 (70%) | 5 (71%) |
|
Nicotine dependence a | 5.0 (2.6) | 3.9 (3.0) |
|
Baseline Problem Drinking (AUDIT) b | 8.9 (4.6) | 10.6 (8.2) |
|
Baseline Probable Depression (CES-D) c | 12.1 (10.2) | 10.3 (8.5) |
|
|||
|
Quit Rate | 4 (40%) | 1 (13%) |
|
NRT use | 7 (70%) | 1 (13%) |
aThe Fagerström Test for Nicotine Dependence ranges from 0-10; a score ≥ 8 is considered nicotine dependent.
bThe AUDIT ranges from 0-40; a score ≥ 8 is considered problem drinking.
cThe CES-D ranges from 0-60; a score of 16 or higher is indicative of significant or mild depressive symptomatology.
The two pilot studies showed that veterans were able to change their smoking behavior after exposure to the Tobacco Tactics website. The poor quit rate in the first pilot study was not particularly surprising in that none of the veterans were actively seeking cessation services and none received cessation medications. Nonetheless, 17% (1/6) of veterans had attempted to quit and the reduction in cigarettes per day was substantial for 50% (3/6) of the veterans. Moreover, participants rated the website highly. Albeit the study was underpowered, the seven-day point prevalence abstinence at 30-day follow-up in the second pilot study was more than three times as high in the intervention group compared to the control group as it included smokers motivated to quit who were also given NRT along with the website compared to the 1-800-QUIT-NOW telephone line where they may also have obtained NRT. These preliminary results are promising and suggest the need for wider-scale testing of the Tobacco Tactics website for veterans.
Most of the veterans who used the Tobacco Tactics website found it to be enjoyable, easy to read, easy to navigate, rated the website highly on pictures, illustrations, and colors, and indicated that they would recommend the website to others. This is likely due to the extensive usability testing that was conducted and subsequent revisions that were made. These revisions likely enhanced engagement and reduced navigational problems common on other websites [
Similar to other populations [
The major barrier to implementation is that VA data security policies are such that there are several layers of “red tape” in order to house interactive websites on VA servers. Thus, currently the Tobacco Tactics website is housed on a secure University of Michigan server. For the moment, the VA is partnering with another website program [
Sample Tobacco Tactics for Operating Engineers webpage.
The sample sizes for pilot studies 1 and 2 were small which may limit generalizability. Given that these were pilot studies, there was limited time for longer follow-up intervals as well as biochemical verification. Contact between participants and the research team may have influenced participants’ reports about the website. While data on the number of times the participants accessed the website was collected, no information was collected on the number of times they accessed particular exercises. The slightly higher differential drop out in pilot study 2 control subjects has implications for interpretation of the results, and may imply less engagement, as more intervention participants responded. Veterans assigned to the Tobacco Tactics website were given ready access to NRT while those assigned to the 1-800-QUIT-NOW group
The preliminary results show that the Tobacco Tactics website has the potential to be an efficacious intervention for veterans trying to quit smoking. Moreover, the website has the potential to increase the reach of smoking cessation interventions to veterans who may have difficulty accessing services for younger OEF/OIF veterans who prefer the Internet. An efficacious tobacco cessation intervention that has the ability to reach a large number of veterans has the potential to increase smoking cessation rates and decrease morbidity and mortality in the VA.
National Institutes of Health
Nicotine Replacement Therapy
Veterans Affairs
Operation Enduring Freedom
Operation Iraqi Freedom
Agency for Healthcare Research and Quality
Fagerström Test for Nicotine Dependence
Alcohol Use Disorders Identification Test
Center for Epidemiologic Studies Depression Scale
This study was funded by the Department of Veterans Affairs SHP 08-197, Blue Cross Blue Shield of Michigan Foundation N011646–1465.RFP, and the National Institutes of Health (R21 CA152247-01).
None declared.