Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, December 24 through Wednesday, December 26 inclusive. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 02.02.17 in Vol 6, No 2 (2017): February

This paper is in the following e-collection/theme issue:

    Original Paper

    Feasibility of Ecological Momentary Assessment of Daily Sexting and Substance Use Among Young Adult African American Gay and Bisexual Men: A Pilot Study

    1Schroeder Institute for Tobacco Research and Policy Studies at Truth Initiative, Washington, DC, United States

    2Case Comprehensive Cancer Center, School of Medicine, Case Western Reserve University, Cleveland, OH, United States

    3Department of Health, Behavior, and Society, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, United States

    4Georgetown University Medical Center, Lombardi Comprehensive Cancer Center, Georgetown University, Washington, DC, United States

    5Department of Psychiatry and Biobehavioral Sciences, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, Los Angeles, CA, United States

    6Veterans Administration Greater Los Angeles Healthcare System, Veterans Administration Center for the Study of Healthcare Innovation, Implementation, and Policy, Los Angeles, CA, United States

    Corresponding Author:

    Sabrina L Smiley, PhD, MPH, MCHES

    Schroeder Institute for Tobacco Research and Policy Studies at Truth Initiative

    900 G Street, NW, Fourth Floor

    Washington, DC,

    United States

    Phone: 1 202 454 5928

    Fax:1 202 454 5599

    Email:


    ABSTRACT

    Background: Recent evidence suggests that sexualized text communication (“sexting”) is associated with substance use and sexual risk behaviors among young adults, yet little is known about this relationship among young adult African American gay and bisexual men, a population disproportionately impacted by HIV in the United States. Rapid advances in mobile phone technology indicate a clear need for research using mobile health (mHealth) methods such as ecological momentary assessment (EMA) to serve as a viable counterpart to retrospective evaluation methods by using real-time data collection to assess sexting and substance use among this population.

    Objective: The objective of this pilot study was to (1) describe the EMA study design and protocol, (2) characterize the study population, and (3) assess the feasibility of a random prompt text message-based thrice-daily EMA over 14 days, as a means of prospectively studying sexting, marijuana, and alcohol use among a sample of young adult African American gay and bisexual men ages 21 to 25.

    Methods: Participants were recruited through flyers and snowball sampling during spring and summer 2015 at a community-based HIV/AIDS prevention, care, and support organization in Washington, DC. Eligible participants were enrolled in a one-time in-person study visit that consisted of informed written consent to participate in the study, a self-administered survey, a semi-structured interview, and enrollment and training in EMA data collection. Commencing the day after the study visit, a random prompt survey was texted to participants on their personal mobile phones 3 times a day over a 14-day data collection period assessing mood, texts sent, texts received, sexts sent, sexts received, marijuana want, marijuana use, and alcohol use.

    Results: EMA feasibility was tested with 25 self-identified African American gay (n=16) and bisexual (n=9) men (mean age of 23.48 years, SD 1.5). Each random prompt survey had 8 questions with responses including yes/no and Likert scale options. There were 104 total days of EMA observation, and the retention rate was 72% (18 out of 25 participants). Participants responded to the random prompt surveys with a 57.3% compliance rate providing a total of 544 completed surveys out of 949 surveys. The overall mean response time to complete a survey was 6.1 minutes. There were significant positive associations between EMA texts sent and received questions (ρ 0.84, P<.001) as well as sexts sent and received queries (ρ 0.72, P<.001).

    Conclusions: The use of an EMA protocol has the potential to be a very useful research tool for understanding episodic behaviors such as sexting and substance use in this relatively understudied and underserved population, and has implications for practice. Additional research is needed on how to maximize survey compliance.

    JMIR Res Protoc 2017;6(2):e9

    doi:10.2196/resprot.6520

    KEYWORDS

    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100



    Introduction

    Despite an overall reduction in new HIV infections in the United States, young African American gay and bisexual men continue to experience a disproportionate burden of HIV [1]. In 2014, among all African American gay and bisexual men diagnosed with HIV in the United States, an estimated 39% (4321) were aged 13-24 [1]. Young adulthood is a critical period of vulnerability for HIV-related risk behaviors including uptake in consumption of mood-altering substances before and during sexual encounters [2,3]. Among young adult African American gay and bisexual men, the most commonly used substances are alcohol and marijuana [4].

    In recent years, researchers have begun to investigate an emerging mobile phone technology trend—sexualized text communication known as “sexting” [5-7]. Sexting describes sending and receiving sexually suggestive photos or text messages via mobile phone [5-7]. Benotsch et al found that sexting among young adults was associated with substance use (eg, alcohol, marijuana, ecstasy, cocaine), multiple sexual partners, unprotected sexual encounters, and sexually transmitted infections [5]. Bauermeister et al found that sexting was more prevalent among young adult gay and bisexual men compared to their heterosexual peers [7]. Although valuable contributions to the literature, both studies [5,7] were limited either by a predominately white, heterosexual or non-heterosexual sample, making it difficult to generalize to young adult African American gay and bisexual men.

    Retrospective self-reports are the dominant method of data collection for assessing sexting and substance use among young adults [5-7]. However, collecting data through self-reporting methods can threaten the reliability and validity of measurement, particularly if individuals under- or overreport their behavior [8-10]. Ecological momentary assessment (EMA), a mobile health (mHealth) method that involves repeated sampling of an individual's behavior in real-time, is an innovative counterpart to traditional research methods that have heavily relied on retrospective self-reported measures [9,10]. EMA can increase the accuracy of self-disclosed, episodic behaviors such as sexting and substance use, and offers an observational longitudinal approach for collecting multiple, real-time assessments of these behaviors over time, and in an individual's environment [9,10]. The use of EMA also offers unique advantages, as it is designed to minimize recall bias, capture time-stamped data, and increase validity in daily life settings [9,10]. Additionally, EMA employs mobile technology data collection tools, including PDAs, Palm Pilots, and mobile phones [9,10]. Rapid advances in mobile phone technology and text messaging have vastly impacted communication and offer a promising approach to developing EMA methods for understanding how mobile phone technology trends, such as sexting, are associated with HIV-related risk behaviors.

    Building on existing literature that has used EMA methods with young adult African American gay and bisexual men, the objective of this pilot study was to (1) describe the EMA study design and protocol, (2) characterize the study population, and (3) assess the feasibility of a text message–based thrice-daily EMA over 14 days, as a means of prospectively studying sexting, marijuana, and alcohol use among a sample of young adult African American gay and bisexual men aged 21 to 25. Feasibility was assessed in terms of random prompt text message survey response compliance, question response compliance, and response times. Retention was also assessed, and recommendations for future EMA research and implications for practice are presented.


    Methods

    Study Participants

    Participants were recruited through flyers and snowball sampling during spring and summer 2015 at a community-based HIV/AIDS prevention, care, and support organization in Washington, DC. Prospective participants were screened by phone by a trained study coordinator to determine whether they met the study’s eligibility criteria, which included self-identifying as black/African American, self-identifying as gay or bisexual, being between the ages of 21 and 25, residing in the Washington, DC, metro area, being English-speaking, having a personal mobile phone and using it daily, having a mobile phone plan with unlimited text messages, and being able to travel to the designated study site.

    Study Procedures

    All eligible participants were screened and enrolled until the target sample size of 25 was met, which is characteristic of EMA pilot studies [11-13]. Participants completed a one-time study visit that consisted of informed written consent, a self-administered survey, a semi-structured interview, and training in EMA data collection. The self-administered survey assessed sociodemographic characteristics (eg, age, education, annual income, and current employment status), social networking application behavior (eg, self-reported use of the social networking applications Facebook and Jack’d), text messaging behavior (eg, self-reported daily texting frequency, sending and receiving sexts), sexual history (eg, self-reported number of male partners in the past 12 months), substance use (alcohol, combustible and noncombustible tobacco products, and marijuana), and psychological state (eg, self-reported anxiety and depression symptoms using the Patient Health Questionnaire for Depression and Anxiety [PHQ-4]) [14].

    Ecological Momentary Assessment Data Collection

    After participants completed the self-administered survey, the primary author registered their phones to receive the random prompt surveys, and participants were trained on how to respond to them via dummy surveys triggered to their phones. The EMA data collection system at the primary author’s institution initiated random prompt surveys 3 times a day for 14 days. On weekdays, surveys were delivered between 6:00 AM and 23:59 PM. On weekends, surveys were delivered between 9:00 AM and 23:59 PM. The EMA data collection system recorded the date and time it took the participant to respond to and complete a random prompt survey, and the date and time a survey expired. The survey expired after 15 minutes of inactivity. The EMA data collection system returned an error message to prevent skipping items or entering out-of-range values. Text messages reminding participants to complete the random prompt surveys were delivered to participants’ phones each day at 8:00 PM, and the primary author was available by office phone, work cell, and email to respond to participants’ inquiries and concerns.

    The random prompt survey consisted of 8 questions with responses including yes/no and Likert scale options. Questions were adapted from similar EMA questionnaires combined with current literature on sexting and substance use [7,15-17]. Figure 1 presents a screenshot of 3 of the 8 questions of the EMA random prompt text message survey. Questions included “How is your mood right now?” (1=very pleasant, 5=very unpleasant), “Since the last time we texted you, how many text messages have you sent today using your cell phone?” (1=0, 6=more than 120), “Since the last time we texted you, how many text messages have you received today to your cell phone?” (1=0, 6=more than 120), “Since the last time we texted you, did you send a sexually explicit message or photo of yourself using your cell phone today?” (1=yes, 2=no), “Since the last time we texted you, did you receive a sexually explicit message or photo of someone by way of cell phone today?” (1=yes, 2=no), “Right now, how much do you want to use marijuana?” (1=not at all, 4=to a great extent), “Since the last time we texted you, have you used marijuana?” (1=yes, 2=no), and “Since the last time we texted you, did you use alcohol, including sips of someone’s drink or your own drink?” (1=yes, 2=no).

    Figure 1. Screenshot of 3 of the 8 ecological momentary assessment (EMA) random prompt text message survey questions.
    View this figure

    Participants received a US $25 Visa gift card at the one-time study visit. Following 14 days of EMA data collection, participants who completed less than 50% (0-20) of the random prompt surveys received a US $25 Visa gift card for their time and cell phone usage. To maximize response compliance, participants who completed 50% or more (21-42) of the random prompt surveys received a US $50 Visa gift card. Additionally, during data collection, the primary author contacted participants via phone call, text message, or email and informed them of their progress toward earning the US $50 Visa gift card. The Chesapeake Institutional Review Board approved all study procedures (Pro00012060) and a Certificate of Confidentiality was obtained through the National Institute on Drug Abuse.

    Ecological Momentary Assessment Compliance Measures

    Retention was defined as the percentage of participants who completed surveys on at least 10 of the possible 14 days of EMA data collection. Survey compliance reflects the proportion of complete surveys out of all surveys texted to the participant’s phone. To count as a complete survey, all 8 survey questions were required to be answered by the participant. Partial survey compliance includes abandoned surveys, where at least 1 survey question was answered but not all 8 survey questions. Question compliance is the proportion of questions answered out of all questions sent. Time to complete a survey reflects the time between the first survey prompt and when participants completed their responses. Participants were prompted 3 times at 5 minute intervals; as such, time to complete a survey reflects not only the actual time to answer all 8 survey questions but also the number of prompts sent before a response was received.

    Data Analysis

    Descriptive statistics were used to characterize the study population in terms of sociodemographic characteristics, childhood experiences (parental smoking status, perceived introduction to adult responsibilities), self-reported health, personal financial status, use of social networks, sexualized text communication, sexual history, substance use, and compliance measures. Differences in survey compliance on baseline characteristics were examined using linear regression. Comparisons between week 1 and week 2 compliance measures were evaluated using paired t tests. Pairwise correlations were assessed for all 8 daily measures (mood, text sent, text received, sext sent, sext received, marijuana want, marijuana use, and alcohol use) using Pearson correlation. Cronbach alpha was also produced for the following related measures: (1) text sent, text received, sext sent, sext received and (2) marijuana want, marijuana use, alcohol use. All statistical analyses were completed in SAS version 9.4 (SAS Institute Inc) and Stata version 13 (StataCorp LP) with figures created in both SAS and JMP version 10.0.02 (SAS Institute Inc).


    Results

    Baseline Characteristics of Participants

    Participants (n=25) were self-identified black/African American gay (n=16) and bisexual (n=9) men who ranged in age from 21 to 25 (mean 23.48, SD 1.5) years. More than half (15/25, 60%) reported annual incomes below US $35,000. The majority of the sample reported full or part-time employment (21/25, 84%) and at least some college education (21/25, 84%). When asked to describe their overall personal financial situation, 48% (12/25) of the sample reported just meeting basic expenses or not meeting expenses. In regard to childhood experiences, 64% (16/25) of the sample reported that one of their parents or guardians smoked cigarettes during their childhood. In terms of taking on adult responsibilities, 76% (19/25) of the sample reported growing up faster than other people their age. Most of the sample (19/25, 76%) reported using their phone multiple times per day to access social networking sites such as Facebook, Twitter, and Instagram. Nearly all (24/25, 96%) reported that they have received a sexually explicit photo of someone to their phone, and most of them (21/25, 84%) reported that they have sent a sexually explicit photo to someone using their phone. Over two-thirds (17/25, 68%) of the sample reported using marijuana every day or some days, and 56% (14/25) reported using little cigars/cigarillos/bidis at least 1 time in the past month. Additionally, 48% (12/25) reported drinking alcohol at least 2 to 3 times per week in the past month, and 36% (9/25) reported no condom use the last time they had anal sex with a man. According to the PHQ-4 [14], 52% (13/25) reported mild, moderate or severe psychological distress. Table 1 summarizes participants’ characteristics.

    Table 1. Participant characteristics.
    View this table

    Feasibility Assessment

    The EMA data collection period was programmed to last 14 days. There were 25 participants who provided 104 days of observation (April 17, 2015, through July 30, 2015). Average number of days of observation was 10.64 (SD 3.2, range 0-14). Due to a system scheduling error, 5 participants responded to the EMA random prompt text message surveys less than 14 days. Additionally, 1 participant’s phone was lost during the data collection period resulting in missing data. Another participant did not complete any surveys during the 14-day EMA data collection period but was included in the final sample since surveys were still sent to his phone. The total retention rate was 72% (n=18). Table 2 summarizes EMA compliance data.

    Table 2. Ecological momentary assessment compliance data.
    View this table

    Table 3 summarizes EMA compliance data overall and by week. A total of 949 random prompt surveys were texted to the sample’s phones and a total of 544 surveys were completed, resulting in an overall compliance rate of 57.3%. Due to the aforementioned system error, fewer surveys were sent out in week 2 compared to week 1. A total of 277/484 (57.2%) surveys were completed over week 1, and 267/465 (57.4%) surveys were completed over week 2. There were 41 surveys that were partially completed resulting in an overall partial survey compliance rate of 4.3%. Over week 1, 22/484 (4.5%) surveys were partial, and over week 2, 19/465 (3.9%) surveys were partial. Overall, 364 (36%) surveys expired due to nonresponse. A total of 4496 survey questions were completed resulting in an overall question compliance rate of 59.22%. Over week 1, 2290 (59.14%) questions were completed, and over week 2, 2206 (59.30%) were completed. Completion time was evaluated as the number of minutes elapsed from initiation of each survey to synchronization with the server. The overall mean response time was 6.06 (SD 4.99) minutes. The mean response time over the first week was 6.1 (SD 5.2) minutes, and over the second week it was 6.0 (SD 4.8) minutes.

    Figure 2 shows the time, in minutes, participants took to complete the survey for each individual response on weekdays versus weekends across study days. Response time did not vary significantly between weekdays and weekends (Mweekday: 6.1 [SD 5.0] minutes; Mweekend: 6.0 [SD 5.0] minutes). Most points on weekdays and weekends lie beneath the 10-minute mark which indicates it took 2 or fewer system prompts to solicit a complete response from participants.

    There were significant positive associations between EMA text sent and received questions (ρ: 0.84, P<.001) as well sexting sent and received queries (ρ: 0.72, P<.001). For text-focused EMA questions (text sent, text received, sext sent, sext received), the resulting Cronbach alpha was 0.68, suggesting acceptable internal consistency.

    Table 3. Ecological momentary assessment compliance data overall and by week (data restricted to weeks 1 and 2 only).
    View this table
    Figure 2. Time-to-complete survey from first prompt by system to last response by participant on weekdays versus weekends across study days.
    View this figure

    Discussion

    Principal Findings

    To our knowledge, this is one of the first pilot studies to examine the feasibility of mobile phone-based text message EMA to prospectively capture sexting and marijuana and alcohol use among a sample of young adult African American gay and bisexual men. Study findings suggest that completing random prompt text message surveys 3 times per day across 14 days is a feasible approach. This study also underscores the importance of partnering with community-based HIV/AIDS prevention, care, and support organizations, as findings demonstrate the feasibility of recruiting, enrolling, and retaining young adult African American gay and bisexual men in an intensive, longitudinal pilot study.

    This study found an overall survey compliance rate of 57.3% (544/949), with a rate of 57.2% (277/484) across the first week and a rate of 57.4% (267/465) across the second week. Previous EMA studies have reported compliance rates from 50% to 90% [12,18,19]. Compared to these studies, an overall survey compliance rate of 57.3% is promising, given that compliance was impacted by 8 days of observation due to the participant whose phone was lost during the data collection period. Additionally, data for the participant who did not complete any surveys—suggesting that he dropped out the day after his study visit—was included since random prompt surveys were still texted to his phone across the 14-day course of the study. Future studies will involve repeated testing of the system software and EMA protocol and procedures, to ensure quality assurance. Particular attention should be given to a lower-burden EMA protocol and procedures that better meet the needs of young adult African American gay and bisexual men. For example, each random prompt survey contained 8 questions. Since most participants (21/25) reported working either full-time or part-time, it is possible that they were less likely to complete a survey when they were engaged in the active duties of their employment. Future research will explore prompting participants to answer fewer than 8 questions in a survey.

    There were also similar response rates in question compliance in the first week (2290/4496, 59.14%) compared to the second week (2206/4496, 59.30%). Future research will include conducting individual interviews or focus groups with members of the study population. These methods of formative research may provide an opportunity to explore why a participant may or may not complete a random prompt survey, thereby informing acceptability and strategies to maximize compliance.

    Similar to other studies [15,20], participants were provided up to 15 minutes to complete a random prompt survey, and the mean time to complete across the 14-day assessment was 6.1 minutes. Findings demonstrate that participants completed surveys in a very timely manner. Additionally, mean response time on weekdays (6.1 minutes) and weekends (6.0 minutes) were all under 10 minutes, indicating that it took 2 or fewer system prompts to solicit a complete response (eg, answered all 8 questions in a survey) from participants. This suggests a high level of comfort with text messaging among participants. Mean time to complete a survey was slightly but not significantly faster on weekends, suggesting that participants may not have been as engaged in the active duties of their employment compared to weekdays.

    Implications for Practice and Future Research

    In this study, nearly all participants reported having received a sexually explicit photo of someone to their phone (24/25, 96%) and having sent a sexually explicit photo to someone using their phone (21/25, 84%). Additionally, 68% (17/25) reported using marijuana every day or some days, 48% (12/25) reported drinking alcohol at least 2 to 3 times a week in the past month, and 36% (9/25) reported no condom use the last time they had anal sex with a man. The sexting, substance use, and sexual risk behavior among this sample of young adult African American gay and bisexual men suggests that EMA data collection tools, including mobile phones and text messaging, have the potential to address mobile phone technology trends such as sexting and reinforce substance use treatment and HIV prevention, care, and support services available through community-based HIV/AIDS organizations serving this population. Future research involves (1) repeated testing of the EMA protocol and data collection system, (2) limiting the number of questions asked in a random prompt text message survey, (3) soliciting buy-in to the study during orientation, (4) increasing study visits, (5) retraining participants during study visits, (6) providing feedback on response compliance, (7) providing a higher bar for incentives, (8) including both random prompts and self-reports to capture events and assess level of substance use, and (9) evaluating protocol acceptability by participants.

    Limitations

    There are limitations to consider when interpreting study findings. While the sample size (n=25) was appropriate for pilot study research [21], the study is limited in generalizability, in that it uses a convenience sample of young adult African American gay and bisexual men who reported full or part-time employment, some college education, access to mobile phones with unlimited data plans, and residence in the Washington, DC, metro area. Although 15/25 participants (60%) reported annual incomes below US $35,000 and 9/25 participants (36%) reported that they just meet basic expenses, this study is unlikely to capture the diversity of viewpoints of all young adult African American gay and bisexual men. Additionally, in an effort to not overburden participants, EMA data collection was limited to 2 weeks. Future research should extend data collection to 3 or 4 weeks to provide more “in the moment” data.

    Conclusions

    Compared to their white and Latino peers, young adult African American gay and bisexual men are arguably at an increased risk for HIV infection [1,22,23]. Mobile phone technology and text messaging present opportunities for novel, innovative research and HIV prevention strategies. This feasibility study highlighted the utility of an EMA approach for advancing knowledge about episodic behaviors, such as sexting and marijuana and alcohol use, to inform research and prevention strategies for this relatively understudied and underserved population. Additional research is needed to customize a fully effective EMA platform.

    Acknowledgments

    At the time of the study, Sabrina L Smiley was a Scholar with the HIV/AIDS, Substance Abuse, and Trauma Training Program at the University of California Los Angeles supported through an award from the National Institute on Drug Abuse (R25DA035692). The research reported in this publication was supported by the National Institute on Drug Abuse under grant number 5R25DA035692. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

    The authors are especially grateful for the men who participated in this study and Travis Wise for his dedication to the outreach, recruitment, and enrollment phase of this study.

    Authors' Contributions

    SLS conceived the study design and manuscript topic, led and conducted data collection and data analysis, and wrote the first draft of the paper. HE conducted data analysis and contributed to manuscript writing. MWH, RSN, ABH, and NGM contributed to interpretation of results and manuscript writing.

    Conflicts of Interest

    None declared.

    References

    1. Centers for Disease Control and Prevention. HIV Among African American Gay and Bisexual Men. 2016.   URL: https://www.cdc.gov/hiv/group/msm/bmsm.html [accessed 2017-01-25] [WebCite Cache]
    2. Halkitis PN, Siconolfi DE, Stults CB, Barton S, Bub K, Kapadia F. Modeling substance use in emerging adult gay, bisexual, and other YMSM across time: the P18 cohort study. Drug Alcohol Depend 2014 Dec 1;145:209-216 [FREE Full text] [CrossRef] [Medline]
    3. Bruce D, Harper GW, Fernandez MI, The Adolescent Medicine Trials Network for HIV AIDS Interventions (ATN). Heavy marijuana use among gay and bisexual male emerging adults living with HIV/AIDS. J HIV AIDS Soc Serv 2013 Jan;12(1):26-48 [FREE Full text] [CrossRef] [Medline]
    4. Morgan E, Skaathun B, Michaels S, Young L, Khanna A, Friedman SR, et al. Marijuana use as a sex-drug is associated with HIV risk among black MSM and their network. AIDS Behav 2016 Mar;20(3):600-607 [FREE Full text] [CrossRef] [Medline]
    5. Benotsch EG, Snipes DJ, Martin AM, Bull SS. Sexting, substance use, and sexual risk behavior in young adults. J Adolesc Health 2013 Mar;52(3):307-313 [FREE Full text] [CrossRef] [Medline]
    6. Gordon-Messer D, Bauermeister JA, Grodzinski A, Zimmerman M. Sexting among young adults. J Adolesc Health 2013 Mar;52(3):301-306 [FREE Full text] [CrossRef] [Medline]
    7. Bauermeister JA, Yeagley E, Meanley S, Pingel ES. Sexting among young men who have sex with men: results from a national survey. J Adolesc Health 2014 May;54(5):606-611 [FREE Full text] [CrossRef] [Medline]
    8. Starosta AJ, Earleywine M. Assessing base rates of sexual behavior using the unmatched count technique. Health Psychol Behav Med 2014 Jan 1;2(1):198-210 [FREE Full text] [CrossRef] [Medline]
    9. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol 2008;4:1-32. [Medline]
    10. Shiffman S. Ecological momentary assessment (EMA) in studies of substance use. Psychol Assess 2009 Dec;21(4):486-497 [FREE Full text] [CrossRef] [Medline]
    11. Cook PF, McElwain CJ, Bradley-Springer LA. Brief report on ecological momentary assessment: everyday states predict HIV prevention behaviors. BMC Res Notes 2016 Jan 04;9:9 [FREE Full text] [CrossRef] [Medline]
    12. Garcia C, Hardeman RR, Kwon G, Lando-King E, Zhang L, Genis T, et al. Teenagers and texting: use of a youth ecological momentary assessment system in trajectory health research with Latina adolescents. JMIR Mhealth Uhealth 2014;2(1):e3 [FREE Full text] [CrossRef] [Medline]
    13. Henry JA, Galvez G, Turbin MB, Thielman EJ, McMillan GP, Istvan JA. Pilot study to evaluate ecological momentary assessment of tinnitus. Ear Hear 2012;33(2):179-290 [FREE Full text] [CrossRef] [Medline]
    14. Kroenke K, Spitzer RL, Williams JB, Löwe B. An ultra-brief screening scale for anxiety and depression: the PHQ-4. Psychosomatics 2009;50(6):613-621. [CrossRef] [Medline]
    15. Pearson JL, Smiley SL, Rubin LF, Anesetti-Rothermel A, Elmasry H, Davis M, et al. The Moment Study: protocol for a mixed method observational cohort study of the Alternative Nicotine Delivery Systems (ANDS) initiation process among adult cigarette smokers. BMJ Open 2016;6(4):e011717 [FREE Full text] [CrossRef] [Medline]
    16. Rice E, Gibbs J, Winetrobe H, Rhoades H, Plant A, Montoya J, et al. Sexting and sexual behavior among middle school students. Pediatrics 2014 Jul;134(1):e21-e28 [FREE Full text] [CrossRef] [Medline]
    17. Phillips MM, Phillips KT, Lalonde TL, Dykema KR. Feasibility of text messaging for ecological momentary assessment of marijuana use in college students. Psychol Assess 2014 Sep;26(3):947-957. [CrossRef] [Medline]
    18. Wray TB, Kahler CW, Monti PM. Using ecological momentary assessment (EMA) to study sex events among very high-risk men who have sex with men (MSM). AIDS Behav 2016 Jan 8:2231-2242. [CrossRef] [Medline]
    19. Yang C, Linas B, Kirk G, Bollinger R, Chang L, Chander G, et al. Feasibility and acceptability of smartphone-based ecological momentary assessment of alcohol use among African American men who have sex With men in Baltimore. JMIR Mhealth Uhealth 2015;3(2):e67 [FREE Full text] [CrossRef] [Medline]
    20. Shrier LA, Walls CE, Kendall AD, Blood EA. The context of desire to use marijuana: momentary assessment of young people who frequently use marijuana. Psychol Addict Behav 2012 Dec;26(4):821-829. [CrossRef] [Medline]
    21. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat 2005 Oct;4(4):287-291. [CrossRef]
    22. Cornelius JB, Dmochowski J, Boyer C, St Lawrence J, Lightfoot M, Moore M. Text-messaging-enhanced HIV intervention for African American adolescents: a feasibility study. J Assoc Nurses AIDS Care 2013;24(3):256-267 [FREE Full text] [CrossRef] [Medline]
    23. Muessig KE, Pike EC, Fowler B, LeGrand S, Parsons JT, Bull SS, et al. Putting prevention in their pockets: developing mobile phone-based HIV interventions for black men who have sex with men. AIDS Patient Care STDS 2013 Apr;27(4):211-222 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    EMA: ecologic momentary assessment
    PHQ-4: Patient Health Questionnaire for Depression and Anxiety


    Edited by G Eysenbach; submitted 19.08.16; peer-reviewed by B Linas, T Wray, P Cook; comments to author 04.10.16; revised version received 30.11.16; accepted 13.12.16; published 02.02.17

    ©Sabrina L Smiley, Hoda Elmasry, Monica Webb Hooper, Raymond S Niaura, Alison B Hamilton, Norweeta G Milburn. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 02.02.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.