This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Assessing the frequency and intensity of e-cigarette use presents special challenges beyond those posed by cigarette use. Accurate measurement of e-cigarette consumption, puff duration, and the stability of these measures over time will be informative for estimating the behavioral and health effects of e-cigarette use.
The purpose of this pilot study was to compare the accuracy of self-reported e-cigarette puff counts collected via ecological momentary assessment (EMA) to objective puff count data collected by a Bluetooth-enabled e-cigarette device and to examine the feasibility and acceptability of using a second-generation e-cigarette among adult smokers.
A total of 5 adult smokers were enrolled in a longitudinal parent study assessing how e-cigarette use affects cigarette use among e-cigarette–naïve smokers. Using a text message–based EMA system, participants reported e-cigarette puffs for 2 weeks. Participants were also given a Bluetooth-enabled e-cigarette (Smokio) that passively collected puff counts and puff duration. Comparisons between mean reports of Smokio (device-report) and EMA (self-report) use were evaluated using paired
A total of 5 African American participants, 4 men and 1 woman, who ranged in age from 24 to 59 years completed the study, resulting in 5180 observations (device-report) of e-cigarette use. At baseline, participants reported smoking for 5 to 25 years and consumed a mean of 7 to 13 cigarettes per day (CPD); 4 smoked within 30 minutes of waking. At the 30-day follow-up, CPD range decreased to 1 to 3 cigarettes; 4 participants reported past 7-day e-cigarette use, and 1 participant reported no cigarette smoking in the past 7 days. Over 2 weeks of e-cigarette use, participants took an average of 1074 e-cigarette (SD 779.0) puffs per person as captured by the device reports. Each participant took a mean of 75.0 (SD 58.8) puffs per day, with each puff lasting an average of 3.6 (SD 2.4) seconds. Device reports captured an average of 33.3 (SD 47.8) more puffs per person per day than the self-reported e-cigarette puffs. In 87% of days, participants underestimated the number of puffs they had taken on the Smokio. There was significant moderate correlation (
Compared to self-reported e-cigarette use, the Bluetooth-enabled device captured significantly more e-cigarette use and allowed for examination of puff duration in addition to puff counts. A Bluetooth-enabled e-cigarette is a powerful and feasible tool for objective collection of e-cigarette use behavior in the real world.
There has been a rapid increase in lifetime and past 30-day adult use of e-cigarettes in the United States since their introduction in 2007 [
Much of the scientific literature on e-cigarette use stems from national surveys and laboratory studies, which are limited by recall bias or are not generalizable to e-cigarette use behavior in the real world. Ecological momentary assessment (EMA) is an intensive longitudinal method that samples participant behaviors and experiences to reveal how individual differences and within-person processes interact to produce a behavioral outcome such as quitting smoking [
Accurate measurement of e-cigarette consumption, puff duration, and the stability of these measures over time will be informative for estimating the behavioral and health effects of e-cigarette use. Because EMA has been applied in cigarette smoking research, EMA could also be a powerful tool to understand e-cigarette use in isolation and in combination with other tobacco products (dual use). To date, no studies have employed EMA assessment of e-cigarette use in analyses, and only one study has used daily reports to examine within-person variation in e-cigarette use among smokers; in that study, only the presence or absence of e-cigarette use in that day was assessed, without attention to frequency or intensity of use [
Measurement of e-cigarette use poses challenges beyond those posed by measuring cigarette smoking. The term e-cigarette encompasses an array of products with different performance characteristics. Unlike cigarettes, which have a distinct beginning and end point, an e-cigarette could last several days before it needs to be refilled or discarded. Asking about the number of puffs in an e-cigarette use session may be an adequate measure of e-cigarette use intensity; however, it is unknown whether adult smokers can reliably report e-cigarette puff counts. The purpose of this pilot study was to compare the accuracy of self-reported e-cigarette puff counts collected via EMA to objective puff count data collected by a Bluetooth-enabled e-cigarette device and to examine the feasibility and acceptability of using a second-generation e-cigarette among adult smokers.
Data from this study come from a pilot embedded in a longitudinal study (Mixed Method E-Cigarette [Moment] Study). Details on the study protocol and procedures are available elsewhere [
Eligible individuals were English-speaking adults aged 18 years or older residing in the Washington, DC, metro area who smoked at least 8 cigarettes a day for the past 5 years. To simplify EMA tobacco use reports, we excluded polytobacco users, defined as having smoked a little cigar/cigarillo, large cigar, or hookah more than 5 times in the last 30 days or used smokeless tobacco in the past 30 days. Additional eligibility criteria included (1) no e-cigarette use in the last 30 days, (2) interest in trying an e-cigarette, and (3) report interest in quitting cigarette smoking in the next 30 days at the initial screening. To facilitate EMA data collection, participants were required to use a cell phone daily and have an unlimited text message plan. Participants were recruited via public online postings, paid advertisements, and physical flyers. Recruitment documents directed potential participants to an online screening survey (www.ecigstudy.org).
In-person procedures consisted of 4 office visits. During the baseline visit, participants completed the informed consent process and confirmed current smoking status with an exhaled carbon monoxide test. A research assistant (RA) registered participant phones to receive EMA text messages and trained participants on how to respond to the EMA random texts and self-initiated tobacco use reports. At the second office visit, participants were provided with 2 Smokio batteries (
The Smokio battery/cartomizer combination (top) was given to participants during the study; the Smokio tank (bottom) was provided to participants at the end of the study.
This study employed 2 types of active EMA data collection: (1) participant-initiated cigarette and e-cigarette use reports and (2) system-initiated random prompts to assess mood and craving. Random prompt data are not reported in this manuscript; more information on these methods is available elsewhere [
Both streams of data were collected via text messages on participant personal cell phones. The EMA data collection system returned an error message to prevent participants from skipping items or entering out-of-range values. All EMA entries were stamped with the time, date, and geolocation of the report and were uploaded to Truth Initiative’s secure server via an encrypted representational state transfer application program interface.
Smokio (
An RA conducted in-depth interviews with all participants at week 1, 2, and 3 office visits. Among other topics, the RA asked participants about their sensory and social experience using the e-cigarette and any difficulties they had using the device, such as remembering to keep it charged. Interviews did not exceed 30 minutes. Both the baseline and follow-up surveys were computer-assisted self-interview surveys. Survey questions assessed sociodemographics, tobacco and e-cigarette use history, tobacco and e-cigarette use beliefs and cognitions, tobacco product and e-cigarette harm perceptions, alcohol use, and health status.
Descriptive statistics were used to characterize the sample in terms of demographic characteristics (sex, age, race, and education), baseline factors (years smoked, cigarettes per day, nicotine dependence, and other tobacco use) and follow-up smoking/e-cigarette use (cigarettes per day, e-cigarette use, and point prevalence abstinence). All EMA e-cigarette puff reports (self-reports and device-reports) were aggregated at the day level. Comparisons between day-level mean reports of e-cigarette device-recorded puffs and self-reported puffs were evaluated using paired
After checking the descriptive statistics, we examined the extent to which timing and device-reported daily puffs influenced self-report accuracy. A linear mixed effect model was used to determine the fixed effect of timing and Smokio-reported daily puffs on report accuracy by taking the random effect of each individual into consideration. To find out the best and worst timing under each time window cases, Tukey's test was used for multiple pairwise comparisons. All statistical analyses were performed in R (The R Foundation) and SAS version 9.4 (SAS Institute Inc); figures were created in R, SAS 9.4 and JMP version 10.0.2 (SAS Institute Inc). Statistical significance was set to a
A total of 5 African American participants, 4 men and 1 woman, with an age range of 24 to 59 years completed this pilot study (
Over 2 weeks of e-cigarette use, the Smokio device captured 5180 e-cigarette puff observations. Participants took an average of 1074 (SD 779.0) e-cigarette puffs per person, with a mean of 75.0 (SD 58.8) puffs per day and 536 (SD 377.9) puffs per week, with each puff lasting an average of 3.6 (SD 2.4) seconds (device data). The average number of device- and self-reported puffs per person per day did not vary by week.
Participant characteristics at baseline and 30-day follow-up (n=5).
Characteristics | Number | ||
Female, n | 1 | ||
Age, range | 24-59 | ||
African American, n | 5 | ||
High school or less | 1 | ||
Some college or more | 4 | ||
Years smoked, range | 5-25 | ||
CPDa, range | 7-13 | ||
Smoke within 30 minutes of waking, n | 4 | ||
CPD, range | 1-3 | ||
Past 7-day e-cigarette use, n | 4 | ||
7-day cigarette smoking PPAb, n | 1 | ||
Increase | 0 | ||
No change | 1 | ||
Decrease | 4 | ||
Increase | 1 | ||
No change | 0 | ||
Decrease | 4 |
aCPD: cigarettes per day.
bPPA: point prevalence abstinence.
Descriptive statistics of e-cigarette use by week of study, comparing e-cigarette use captured by the Smokio device to self-reported e-cigarette use.
Puffs per person per day | Device-reported | Self-reported | % difference | |
Overall, mean (SD) | 75.0 (58.8) | 48.0 (32.6) | 56.25 | <.001 |
Week 2, mean (SD) | 69.9 (53.1) | 47.3 (31.0) | 47.78 | <.001 |
Week 3, mean (SD) | 80.1 (64.1) | 48.7 (34.6) | 64.48 | <.001 |
aPaired
Panel plot of e-cigarette puffs per day by participant and study day comparing device-reported puffs (orange line) to self-reported puffs (blue line).
All 5 of the participants felt the device was convenient and acceptable, with none reporting problems with turning the device off/on or keeping it charged. Participants were struck by their ability to use the Smokio in places where they could not smoke a cigarette.
I kind of like the fact that I can kind of stay in the crowd and still get my nicotine without offending other people.
I could smoke it like absolutely anywhere…it didn’t bother anybody.
Participants also positively described the taste of the AVAIL e-liquid.
It tasted menthol-y enough to where it satisfied the nicotine taste, my body's craving.
It was a pretty good experience for first time using. [It gets]…that sweet menthol taste out of it. So it’s pretty good actually.
Participant 1 stated that it “was weird trying to get used to the taste of it.” While Participant 2 reported “holding it was kind of odd because it wasn't the size of a cigarette,” no one disliked the Smokio because of its size or weight. All participants felt that using the device suppressed their urge for a cigarette.
When I did smoke the Smokio, it took away a lot of cravings and urges, made me feel better.
It satisfies my craving when I need it.
Compared to self-initiated e-cigarette puff reports, the Smokio proved to be a far superior method to collect e-cigarette puff data. However, the moderate correlation (
In comparing device-reported to self-reported data, other consistent trends were revealed. First, we observed that participants underestimated the number of e-cigarette puffs consumed in nearly 90% of study days. While missing data is never preferred, a consistent pattern to missing data can be accounted for in the analyses and interpretation of results. Second, despite a burdensome design, participants did not reduce their average self-reported e-cigarette puff counts between weeks, indicating that there was minimal fatigue in EMA e-cigarette reporting. While Smokio captured more e-cigarette puff data than self-reports, the between-person average difference between the two methods remained steady between the second and third weeks of data collection. If objective measurement of e-cigarette use using a device like Smokio is cost prohibitive or otherwise not feasible (eg, participants are provided with “cigalike” e-cigarettes, or e-cigarettes with small batteries that are often disposable), self-reported e-cigarette puffs may be an acceptable alternative as long as the research question and design allow for underreporting of e-cigarette use and variation in the accuracy of reporting between individuals by heaviness of e-cigarette use.
Data from in-depth interviews with our 5 pilot participants demonstrate that the Smokio device and e-liquid choices were acceptable and convenient. Participants reported liking the taste and experience of using the Smokio, although one individual initially found using the device awkward. Participants also commented on the Smokio’s ability to alleviate craving and the appeal of feeling free to use the device in places where they could not smoke cigarettes. None of our participants reported trouble with keeping the device charged nor did they find operation of the device challenging. Continued use of an e-cigarette at follow-up also suggests that the device was acceptable, although we cannot be certain that participants continued to use the Smokio from the study or some other e-cigarette device. At the 30-day follow-up, it is notable that all participants reported that they smoked fewer cigarettes than when they enrolled in the study, with the average CPD dropping from 9 to 1.8.
This pilot study has several limitations. First, as a pilot study, conclusions are based on only 5 individuals. Results are intended to inform measurement of e-cigarette use in EMA studies and should not be interpreted beyond this purpose. Second, we assumed that the device-recorded puffs were the gold standard method for assessing naturalistic e-cigarette use; however, we did not conduct a formal laboratory assessment of the Smokio puff counter or puff duration measurement and thus cannot be certain of the precision of the device. Additionally, as of summer 2016, the Smokio (now called Vap.io) is no longer available for purchase in the United States. Other devices, such as several Joytech products, collect puff count data but do not currently sync data and push to a remote server, which allows real-time monitoring of e-cigarette use behavior and reduces data loss if a participant misses an in-person visit. Future collaboration with private companies or independent developers will be necessary to create a product with capabilities similar to the Smokio. Research funders should consider supporting the development of a range of e-cigarette device types that passively capture user data, including puff counts and puff duration. We also note that Participant 3 was remarkably accurate in her self-reports. We did not collect information that explains why her self-reports outperformed those of the other participants; however, participants did know that the Smokio counted their puffs—that awareness could have encouraged counting. It is also possible that novel users are more aware of their puff patterns than more established users; thus, these conclusions may not transfer to established e-cigarette users. We also did not collect information on the amount of e-liquid consumed by each participant, which would be an additional helpful source of data to compare against the Smokio-reported puffs. Finally, our instruction to participants to initiate an e-cigarette self-report when they “put down the e-cigarette and did not intend to pick it up again in a while” may not translate to established vapers who “graze” on their devices throughout the day. In this population, an EMA coverage approach or, ideally, provision of a Smokio-like device would be the best option to capture puffing data.
Compared to self-reported e-cigarette use, the Bluetooth-enabled device captured significantly more e-cigarette use and allowed for examination of puff duration in addition to puff counts. A Bluetooth-enabled e-cigarette is a powerful and feasible tool for objective collection of e-cigarette use behavior in the real world. As e-cigarette users adopt more sophisticated devices, researchers should consider harnessing the existing capabilities of these devices to aid data collection.
concordance correlation coefficient
cigarettes per day
Center for Tobacco Products
ecological momentary assessment
Food and Drug Administration
research assistant
Research reported in this publication was supported by the National Institute on Drug Abuse at the National Institutes of Health and FDA CTP under grant number 5R21DA036472. J Pearson’s time was also supported by the National Institute on Drug Abuse at the National Institutes of Health and FDA CTP under grant K01DA037950. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the FDA. At the time of the study, S Smiley was a Scholar with the HIV/AIDS, Substance Abuse, and Trauma Training Program at the University of California, Los Angeles, supported by an award from the National Institute on Drug Abuse (R25DA035692). J Pearson conceived the study design and manuscript topic, led the data collection and analysis team, and wrote the first draft of the paper. H Elmasry and Y Zhou conducted the analyses. B Das, S Smiley, L Rubin, T DeAtley, E Harvey, R Niaura, and D Abrams contributed to the analysis plan, interpretation of results, and manuscript writing. S Smiley was project director of the parent study. L Rubin was the research assistant for this pilot.
None declared.