Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 11.09.20 in Vol 9, No 9 (2020): September

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/18051, first published Mar 02, 2020.

This paper is in the following e-collection/theme issue:

    Original Paper

    Work2Prevent, an Employment Intervention Program as HIV Prevention for Young Men Who Have Sex With Men and Transgender Youth of Color (Phase 3): Protocol for a Single-Arm Community-Based Trial to Assess Feasibility and Acceptability in a Real-World Setting

    1Planned Parenthood Great Plains, Overland Park, KS, United States

    2Center for Interdisciplinary Inquiry and Innovation in Sexual and Reproductive Health, Department of Obstetrics and Gynecology, University of Chicago, Chicago, IL, United States

    3Guttmacher Institute, New York, NY, United States

    4Division of Adolescent Medicine, Ann & Robert H Lurie Children’s Hospital, Department of Pediatrics, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States

    5Division of Research on Children, Youth, and Families, Children's Hospital Los Angeles, Los Angeles, CA, United States

    6Fenway Health, The Fenway Institute, Boston, MA, United States

    7Collaborative Studies Coordinating Center, Department of Biostatistics, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States

    8Department of Medicine, University of Chicago, Chicago, IL, United States

    Corresponding Author:

    Brandon J Hill, PhD

    Planned Parenthood Great Plains

    4401 W 109th St #100

    Overland Park, KS, 66211

    United States

    Phone: 1 913 345 4624

    Email: brandon.hill@ppgreatplains.org


    ABSTRACT

    Background: In the United States, young cisgender men who have sex with men (YMSM), young transgender women (YTW), and gender nonconforming (GNC) youth face elevated rates of HIV infection. However, racial and ethnic disparities in adolescent HIV infection cannot be attributed to individual-level factors alone and are situated within larger social and structural contexts that marginalize and predispose sexual and gender minority youth of color to HIV. Addressing broader ecological factors that drive transmission requires interventions that focus on the distal drivers of HIV infection, including violence exposure, housing, food insecurity, educational attainment, and employment. Given the ways that economic instability may make YMSM, YTW, and GNC youth of color vulnerable to HIV exposure, this study focuses on employment as an HIV prevention intervention. More specifically, the intervention, called Work2Prevent (W2P), targets economic stability through job readiness and employment as a means of preventing behaviors and factors associated with adolescent and young adult HIV, such as transactional sex work and homelessness. The intervention was adapted from iFOUR, an evidence-based employment program for HIV-positive adults in phase 1 of this study, and pilot tested in a university-based setting in phase 2.

    Objective: This paper aims to describe the protocol for the community-based test phase of W2P. The purpose of this phase was to pilot test a tailored, theoretically informed employment intervention program among YMSM, YTW, and GNC youth of color within a lesbian, gay, bisexual, transgender, and queer (LGBTQ) community setting.

    Methods: The employment intervention was pilot tested using a single-arm pretest-posttest trial design implemented among a sample of vulnerable YMSM, YTW, and GNC youth of color using services within a community-based LGBTQ center. Assessments will examine intervention feasibility, acceptability, and preliminary estimates of efficacy.

    Results: Phase 3 of W2P research activities began in May 2019 and was completed in December 2019. Overall, 41 participants were enrolled in the community-based pilot.

    Conclusions: This study will assess intervention feasibility and acceptability in the target populations and determine preliminary efficacy of the intervention to increase employment and reduce vulnerability to HIV when implemented in a community-based setting serving LGBTQ youth of color. Testing the intervention in a community setting is an opportunity to evaluate how recruitment, retention, and other outcomes are impacted by delivery in a venue akin to where this intervention could eventually be used by nonresearchers. If W2P demonstrates feasibility and acceptability, a larger multisite trial implemented in multiple community settings serving YMSM, YTW, and GNC youth of color is planned.

    Trial Registration: ClinicalTrials.gov NCT03313310; https://clinicaltrials.gov/ct2/show/NCT03313310

    International Registered Report Identifier (IRRID): DERR1-10.2196/18051

    JMIR Res Protoc 2020;9(9):e18051

    doi:10.2196/18051

    KEYWORDS

    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100



    Introduction

    Background

    In the United States, youth of color assigned male at birth who engage in sexual contact with individuals assigned male at birth, including young cisgender men who have sex with men (YMSM), young transgender women (YTW), and gender nonconforming (GNC) youth, face elevated rates of HIV infection [1,2]. In 2018, an estimated 26% of all new HIV diagnoses were among Black men who have sex with men (MSM), while 20% were among Hispanic/Latino MSM [1]. These outcomes are stark, given that Black and Latino individuals comprise 13% and 18% of the US population, respectively [3]. Further, although the Centers for Disease Control and Prevention (CDC) does not provide estimates of HIV infection for transgender and GNC populations due to gaps in public health department data, a previous meta-analysis of US studies found an average HIV prevalence rate of 28% among transgender women, with a higher rate of 56% among Black transgender women specifically [4]. This racial disparity is not solely driven by individual sexual behaviors, but rather a number of interrelated socioecological and structural factors that proximate YTW, GNC youth, and YMSM of color to HIV exposure and disease susceptibility [2,5-11]. In particular, these groups face co-occurring and interconnecting disparities in housing, health care access, social service availability, education, poverty, employment, and violence [12-21]. Economic factors may be especially salient among these populations, as prior research suggests that sexual and gender minority youth of color face hiring bias, job discrimination, low pay, and limited benefits when navigating employment [18,22]. These experiences contribute to a large proportion of sexual and gender minority youth of color living in poverty [18,22-24].

    Additionally, economic marginalization may increase potential for HIV exposure and infection by contributing to reliance on survival sex work, or exchanging sex for money, food, shelter, drugs, or other commodities [25-27]. Survival sex work has also been associated with coping-related substance use [12,13,17,20,21]. Previous studies have found that engagement with survival sex work among YMSM, YTW, and GNC of color is also associated with structural factors, including financial insecurity and socioeconomic disconnection [28-30]. Further, engagement in survival sex work may increase youths’ susceptibility to sexually transmitted infections (STIs) and HIV infection by exposing them to sexual networks with higher STI and HIV prevalence, increasing their numbers of sexual partners, and creating challenges in condom use negotiation [28-30].

    Given the role that economic instability may play in driving HIV susceptibility among YMSM, YTW, and GNC youth of color, structural interventions are needed in order to address HIV inequities at their root cause [31]. Structural-level interventions target underlying social drivers of poor health and promote agency among marginalized groups in order to facilitate health-positive actions that can benefit both the individual and community [31]. More specifically to HIV, structural interventions, including comprehensive sex education, community- and venue-based HIV/STI testing, stable housing programs, increased health care coverage, and needle exchange programs, have greatly reduced HIV infection [32-34]. However, fewer structural interventions that address distal drivers of HIV infection, including decreased criminalization and poverty and increased economic stability, have been developed and implemented, although these factors have the potential to mitigate broader disparities in HIV [33,34]. Accordingly, employment as HIV prevention has the potential to be a scalable intervention that targets the economic drivers of HIV infection among YMSM, YTW, and GNC youth.

    Rationale for Employment as HIV Prevention

    In a recent Chicago-based social network intervention for Black YMSM and YTW aged 18 to 35 years, roughly 45% of participants from the respondent-driven sample reported being unemployed [35]. Across Adolescent Medical Trials Network for HIV/AIDS Interventions (ATN) sites in 17 US cities, 64.6% of adolescents and young adults reported being unemployed, despite 67.8% having completed high school and being a mean working age of 20.4 years [36]. Thus, many youths vulnerable to HIV may benefit from a tailored intervention that specifically addresses job preparedness and workforce engagement [18,22,37]. Additionally, employment interventions may also need to specifically address the experiences of overt and implicit discrimination and mistreatment that sexual and gender minority youth report while job seeking and within the workplace [38-40].

    The objective of the Work2Prevent (W2P) study is to adapt and pilot test Increased Individual Income and Independence (iFOUR), an effective, theoretically-driven employment program for HIV-positive adults [41-44], to the needs of vulnerable YMSM, YTW, and GNC youth of color aged 16 to 24 years. For a full rationale of the overall W2P study, please refer to the W2P phase 2 paper [45].

    Theoretical Framework

    The W2P study is informed by frameworks from the health belief model [46] and positive youth development [47-49]. More specifically, the W2P intervention is intended to help participants identify barriers to obtaining employment and increase positive beliefs regarding the perceived benefits of employment. This is supported by helping young people understand, value, and develop both external and internal assets that will benefit their pursuit and maintenance of stable, formal employment. For a more thorough explanation of the theoretical framework, please refer to the W2P phase 2 paper [45].

    Rationale for Testing the Intervention in a Community Setting

    Phase 3 of the W2P study, the topic of this current paper, focuses on pilot testing the intervention in a community-based social service setting. Implementation science highlights the importance of implementing and evaluating interventions in community settings that more closely reflect the real conditions under which the intervention will be delivered upon being scaled [50-53], given that interventions may perform differently when translated from a university research setting to a community-based organization or clinic [54-56]. Further, community-based interventions are increasingly recognized as essential to achieving the Joint United Nations Programme on HIV/AIDS 90-90-90 targets through their ability to overcome structural barriers to health care access [57]. For example, a community-based implementation of Many Men, Many Voices, a CDC-endorsed evidence-based health intervention for Black MSM, evidenced greater reductions in HIV-related behaviors than in the original randomized trial [58,59]. In translating an intervention from a university setting to a community setting, researchers may identify necessary adaptations and challenges that must be considered to maintain the utility of the intervention. Therefore, in order to ensure that efficacious HIV interventions may be optimally scaled, it is important to implement and evaluate these interventions in community-based clinics and organizations [60].

    Assessing how interventions perform in a community setting may be particularly important for interventions that employ a structural-level focus. Indeed, social and structural outcomes are often interrelated. For instance, a previous study of transgender women of color found that employment status was associated with stable housing [61]. Further, the impact of unemployment and job loss on mental health has been well documented [62], often leading to poor mental health outcomes [63] and maladaptive health behaviors [64]. When structural-level interventions are delivered in a university setting, they are generally delivered in isolation and thus do not directly consider how the intervention may interact with services aimed at addressing other related structural factors. However, when an intervention is integrated into an existing service-providing organization, other services that address related structural needs are generally more readily available to study participants. Thus, evaluation of structural-level interventions in community-based settings allows researchers to better assess how intervention performance may be impacted by the social ecology of existing organizations and their corresponding service apparatuses.

    Evaluating interventions in community settings, particularly those tailored for marginalized groups, may also be necessary because organizational settings can have an impact on participants’ level of trust and engagement. Due to historical events, like the Tuskegee syphilis study, and negative experiences with the health system [65], many people of color are reluctant to take part in health research [66]. Further, university settings are often unknown to study participants prior to enrollment and thus, initial impressions of the organizations delivering the intervention may be less established. In contrast, delivery of interventions in community organizations and clinics familiar to participants leverages existing relationships in order to make the intervention more reputable to the community. Further, the presence of familiar individuals who vouch for the trustworthiness of the researchers has the potential to decrease mistrust and engender greater participation in an intervention [67]. Understanding how this impacts study participants’ engagement and the performance of the intervention may be critical in ensuring potential scalability. This may be particularly important for interventions targeting sexual and gender minority youth of color, who often express high levels of institutional and medical distrust due to longstanding and historical mistreatment [68,69].

    Implementing and evaluating interventions in community settings may also allow researchers to assess how intervention delivery is impacted by real-world logistical concerns [53,70]. In order to implement an intervention in a community setting, it may need to be further tailored to accommodate organizational constraints such as staffing, space, scheduling, technological capacity, and resource supports [53]. Testing an intervention in a community-based organization allows researchers to tailor the intervention to address these real-world constraints and assess how this additional tailoring impacts intervention performance.


    Methods

    Conceptual Model

    The W2P conceptual model in Figure 1 draws on the existing iFOUR theoretical framework to hypothesize the potential relationship between adolescent and young adult employment and HIV. The W2P model proposes that employment and subsequent economic connection and stability serve as a structural-level intervention for adolescents and young adults. Our hypothesis is that the adapted and tailored iFOUR intervention will facilitate increased job self-efficacy and job readiness (path A) and ultimately increase employment placement and maintenance (path B). Further, establishing economic stability will decrease engagement in HIV-related behaviors and increase HIV prevention and care (path C), while also decreasing involvement with known social determinants of HIV, such as sex work and substance use (path D), which are directly linked to HIV transmission and acquisition among YMSM, YTW, and GNC youth of color (paths E and F).

    Figure 1. W2P conceptual model. iFOUR: Increased Individual Income and Independence; LGBTQ: lesbian, gay, bisexual, trangender, and queer; PrEP: pre-exposure prophylaxis; STI: sexually transmitted infection; W2P: Work2Prevent.
    View this figure

    Study Design

    W2P uses a mixed methods design. Phase 1 involved the adaptation of relevant intervention components from the existing evidence-based iFOUR employment program for HIV-positive adults [41-43] to YMSM, YTW, and GNC youth of color. Phase 2 involved a pretest of the intervention and study assessments, followed by pilot testing in a university setting to assess feasibility and acceptability and to provide preliminary estimates of efficacy using pretest-posttest comparisons. Phase 3 consists of further refinement of the intervention and study assessments, as well as pilot testing of the intervention in a community-based setting in order to further assess feasibility and acceptability and to provide preliminary estimates of intervention efficacy under real-world conditions using pretest-posttest comparisons. For the purposes of adaptation to a community setting, a number of changes were made between phase 2 and phase 3, including condensing the survey instruments, workshop sessions, and intervention timeline. In phase 2, the intervention workshops were delivered over 2 weeks, and the participants generally completed the baseline survey during a separate study visit within 3 weeks prior to the first intervention session. In phase 3, the intervention workshops were delivered over the course of 2 days, and the first session was offered immediately after participants complete the baseline assessment. In phase 2, participants completed a follow-up visit at 8 months, whereas the follow-up visit was at 3 months for phase 3. For a complete description of phase 2, please refer to the W2P phase 2 protocol paper [45].

    Ethics, Consent, and Institutional Board Approval

    W2P has been reviewed and approved by the University of Chicago Institutional Review Board (IRB# 16-1152). Informed consent for this study was obtained in person by study staff before any study-related activities took place. The trial was registered in October 2017 at Clinicaltrials.gov (NCT03313310).

    Participants

    Study participants included 41 Black or African American and Hispanic or Latinx YMSM, YTW, and GNC youth. Inclusion criteria included (1) being assigned male at birth, (2) reporting ever having sex with men, (3) identifying as African American or Black or Hispanic or Latinx, (4) aged 16 to 24 years, (5) English speaking, (6) currently unemployed but seeking employment or employed only part-time, defined as working 35 hours or less on average per week, (7) able to attend a 4-session workshop, and (8) did not participate in phase 2 of W2P.

    Study Setting

    All study visits were conducted at a community site, the Village. The Village is a community-based setting that provides drop-in services, support groups, resources, behavioral health counseling, community programs, housing and legal assistance, and HIV and STI testing to sexual and gender minority youth. The Village is affiliated with the Chicago Center for HIV Elimination at University of Chicago Medicine and primarily serves Black YMSM, YTW, and GNC youth. This community space is located on the south side of Chicago, a predominantly Black and African American area of the city. A previous spatial study found this area was lacking in career services tailored to the lesbian, gay, bisexual, transgender, and queer (LGBTQ) community [19]. The Village serves approximately 1000 individuals annually, of which 90% are Black or African American, 9% are Hispanic or Latinx, 74% are MSM, 7% are transgender, and 36% are adolescents and young adults aged 24 and younger.

    Recruitment

    Planned participant recruitment efforts included recruitment from the community site itself, as well as from primary and community clinics serving YMSM, YTW, and GNC youth, such as Howard Brown Health, a Chicago-based LGBTQ-focused Federally Qualified Health Center, during their youth drop-in programs. Interested participants completed a prescreen survey to assess eligibility. Eligible participants were then scheduled for study visits and workshop sessions.

    Incentives

    Study participants were offered compensation for their time. Participants could receive up to US $290 total for complete participation in the form of cash or Visa gift card equivalents. Participants received US $30 for each study visit completed at baseline, postintervention, and 3-month follow-up, up to US $40 for biological specimens at baseline and 3-month follow-up, if provided, and US $30 for each workshop session attended.

    Visit Schedule and Data Collection

    W2P consisted of data collection across 3 time points, which occurred at baseline, postintervention, and 3-month follow-up, as referenced in Figure 2. A 4-session intervention workshop series occurred over a 2-day period, with workshop sessions 1 and 2 being conducted on the same day as the baseline visit and sessions 3 and 4 on the same day of the postintervention visit.

    Figure 2. Study procedures. ACASI: audio computer-assisted self-interview.
    View this figure
    Baseline

    Participants completed informed consent and then confirmed eligibility. Subsequently, participants complete an audio computer-assisted self-interview (ACASI) survey using an iPad (Apple Inc). Survey items included questions pertaining to demographics, sexual behaviors, HIV risk behaviors, relationships, employment, income, substance use, and other structural variables such as homelessness, food insecurity, and health care use. Optional biologic samples were collected from participants who consented to them. These samples included a finger stick for rapid HIV testing using the Determine HIV-1/2 Ag/Ab Combo (Alere Inc), a urine sample for drug screening and chlamydia and gonorrhea testing, and anal and oral swabs for chlamydia and gonorrhea testing.

    Intervention

    Participants completed a 4-session intervention workshop adapted from the existing iFOUR program [41-43] and piloted in phase 2 of W2P. In phase 3, workshop sessions occurred over a 2-day period, with 2 sessions on each day. Session 1 focused on goal setting and identifying strengths; session 2, on communication, networking, and job searching; session 3, on balancing work with health and wellness; and session 4, on preparing job application materials and interview preparation. For a full description of the process for determining these intervention components, please refer to the W2P phase 1 protocol paper [71].

    Workshop sessions were delivered by 2 facilitators in groups of 6 to 12 participants across the course of 2 days, with 2 sessions per day. The W2P Career Readiness Workbook was used as a guide for all workshop sessions and given to all study participants at the first session. Facilitators used an annotated W2P Facilitator Guide that provides detailed instruction on delivery of the intervention curriculum. During each session, facilitators completed a fidelity assessment to help ensure fidelity to the W2P Career Readiness Workbook, and after each session, they completed a workshop debriefing form to capture any workshop notes or comments.

    Postintervention

    Once participants completed the workshop sessions, they completed a postintervention ACASI survey using an iPad. Survey items included questions on workshop evaluation, job-seeking self-efficacy, and pre-exposure prophylaxis and HIV testing use.

    3-Month Follow-up

    The final study visit occurred 3 months after the intervention was completed. During this visit, participants completed the baseline ACASI survey using an iPad and provided repeat biologic samples, if they consented to them.

    Outcomes

    Primary Outcomes

    The primary outcomes of this study are (1) information systems success model (ISSM) score, (2) workshop completion, (3) change in job-seeking self-efficacy scale score, and (4) change in protean career attitudes (PCA) scale score.

    First, the ISSM will be used to assess for intervention acceptability and satisfaction. The 21-item scale yields a total score and measures 4 subdomains: information quality, handbook quality, perceived usefulness, and overall satisfaction. This scale has been adapted from Horvath et al [72].

    Second, workshop completion will be used to assess intervention feasibility. Workshop or intervention completion is defined as having attended at least two of the 4 workshop sessions and is measured by tracking participant attendance.

    Third, job-seeking self-efficacy is defined as one’s perceived ability and confidence to perform job search and application activities. The 12-item job-seeking self-efficacy scale by Barlow et al [73] yields a total score in which higher values indicate higher self-efficacy. Job-seeking self-efficacy was previously found to be associated with employment in a previous study of transgender women of color [61].

    Fourth, PCAs are defined as self-direction in the pursuit of success in one’s work. Protean career attitudes have previously been found to be associated with positive career satisfaction and self-perceived success [74]. The validated 7-item scale by Porter et al [75] yields a total score and measures 2 subdomains: self-directed attitudes and values-driven attitudes.

    Secondary Outcomes

    Secondary outcomes include (1) change in self-reported hours worked per week, (2) change in self-reported sexual risk behaviors, (3) change in chlamydia test result, (4) change in gonorrhea test result, and (5) reactive HIV test.

    First, hours worked per week was self-reported at baseline and at the 3-month follow-up visit. Change in hours worked per week from the baseline to the 3-month follow-up will be used to assess change in employment status.

    Second, sexual risk behaviors are defined as self-reported engagement in the following behaviors during the previous 3 months [76]: (1) condomless anal intercourse (receptive or insertive) with cisgender male partner of unknown HIV status, (2) anal intercourse (receptive or insertive) with at least 3 cisgender males, (3) sex with cisgender male partner with an STI, (4) condomless anal intercourse (receptive or insertive) with HIV-positive cisgender male partner, (5) anal intercourse (receptive or insertive) with condom failure, and (6) transactional sex work involvement. The previous 3 months refers to the 3 months prior to the baseline visit for the first assessment and the 3 months prior to the 3-month follow-up visit for the second assessment. Change in sexual risk behaviors is defined as the change in self-reported behaviors from baseline to the 3-month follow-up.

    Third, prevalence of chlamydia infections was assessed at baseline and 3-month follow-up using oral, anal, and urine samples. Each of the 3 tests yields a positive or negative result. Change in chlamydia test result is defined as the change from baseline to the 3-month follow-up. Oral, anal, and urine tests are treated as separate outcomes.

    Fourth, prevalence of gonorrhea infections was assessed at baseline and 3-month follow-up using oral, anal, and urine samples. Each of the 3 tests yields a positive or negative result. Change in gonorrhea test result is defined as the change from baseline to the 3-month follow-up. Oral, anal, and urine tests are treated as separate outcomes.

    Fifth, testing for reactive or nonreactive HIV was assessed at baseline and 3-month follow-up. The reactive HIV test outcome uses the 3-month follow-up result.

    Power

    Given the exploratory nature of this study and limited access to this population, the analyses are not designed to have a specified level of statistical power. A repeated measures pretest-posttest design will be used to reduce the variability in the estimate of the treatment effect.

    Statistical Analysis

    The analytic plan will estimate preliminary efficacy of the intervention by comparing preassessments and postassessments of employment and sexual risk behaviors. Descriptive statistics will be used to analyze the proportions and central tendencies for participant sociodemographic characteristics collected in the surveys. We will first generate frequencies, means, and other measures of central tendency as appropriate to describe our sample and outcomes at each of the 3 time points: baseline, postintervention, and 3-month follow-up.

    All participants who were enrolled at baseline and completed the baseline ACASI will be included in the primary and secondary analyses as applicable. Analysis population participants will be included in all primary and secondary analyses for which their data for the specified outcome are not missing. Participants who did not attend any workshop sessions will not be included in analyses involving workshop evaluation. Primary analyses will assess intervention acceptability, satisfaction, and feasibility, as well as change in job-seeking self-efficacy and PCA score. Secondary analyses will evaluate the intervention by comparing preintervention and postintervention employment and sexual risk behaviors.

    Changes in primary and secondary outcomes between baseline and 3-month follow-up will be assessed using paired 2-tailed t tests for continuous variables (eg, ISSM, job-seeking self-efficacy, and PCA scores) and the McNemar test for matched categorical variables (eg, STI results). We will use standard diagnostic tools to assess the appropriateness of the normality assumption and, if approximate normality of the residuals is not tenable, a nonparametric test for continuous paired data (ie, Wilcoxon signed rank test) will be used. All hypothesis testing will be performed at an α level of .1, given the exploratory nature of the study. To the extent that data allow, multivariable analyses will adjust for sociodemographic characteristics, workshop attendance, baseline employment status, and study completeness. Analytical models will include linear regression or generalized linear models for continuous outcomes and logistic regression for binary outcomes.

    Analysis of the primary and secondary outcomes are described in detail within the statistical analysis plan, which will be accessible on ClinicalTrials.gov once study results have been entered.


    Results

    Phase 3 of W2P research activities began in May 2019 and was completed in December 2019. Overall, 41 participants were enrolled in the community-based pilot.


    Discussion

    Interventions that address the complex socioecological factors that make YMSM, YTW, and GNC youth of color vulnerable to HIV are necessary to curb the epidemic in this population. Given the role of social determinants of health in HIV infections experienced by these young people, interventions must explicitly address factors such as unemployment, homelessness, and survival sex work in order to maximize the impacts of individual-level behavior changes intended to mitigate vulnerability for adolescent and young adult HIV.

    In the first 2 phases of W2P, the intervention was adapted from iFOUR, tailored to YMSM, YTW, and GNC youth of color, and then evaluated in a university setting. The goal of phase 3 of W2P is to pilot test this structural-level employment intervention in a community-based setting that serves the target population. Testing the intervention in a community setting is an opportunity to evaluate how recruitment, retention, and other outcomes are impacted by delivery in a venue more akin to where this intervention could eventually be used by nonresearchers. Given recent findings that identify the south side of Chicago as largely lacking services targeting LGBTQ communities’ specific needs [18], the selected site approximates well the most likely venue where this kind of intervention could be received by members of the target population. Further, by virtue of the study recruiting from and delivering the intervention in a community-based clinic that provides a range of supportive services, such as housing assistance, counseling, and HIV/STI testing, participants are in closer proximity to resources targeting a range of social determinants of health.

    Given that the intervention was delivered to YMSM, YTW, and GNC youth of color in an urban LGBTQ community setting, deployment of the intervention in suburban or rural communities may require additional adaptation and refinement. Although implementing the employment intervention in an LGBTQ community–based setting offers an opportunity for direct recruitment and enrollment, the intervention may not reach YMSM, YTW, and GNC youth of color who are not connected with LGBTQ community and social services. Thus, additional approaches may be needed to reach the most vulnerable youth who are experiencing socioeconomic hardship.

    If W2P demonstrates feasibility and acceptability when delivered in a community-based setting, we plan to test the efficacy in a multicity longitudinal trial across the ATN study sites. Phase 3 of W2P tested the intervention in one community venue. A scale-up of the project will allow the intervention to be tested in a range of community venues that may differ in ways that are relevant to outcomes, such as space, staff experience with intervention delivery, or availability of other resources. This will be an opportunity to further assess efficacy and identify implementation challenges and opportunities. If W2P demonstrates efficacy in the multicity trial, the intervention could be an asset for community organizations invested in addressing the role of employment in the HIV epidemic. Further, this intervention, which has been tailored to the target community and evaluated in relevant community-based settings, will provide YMSM, YTW, and GNC youth of color a way to gain employment skills needed to improve their economic situation and reduce HIV transmission.

    Acknowledgments

    This study is supported by the ATN from the National Institutes of Health (5U24HD089880-02) through the Eunice Kennedy Shriver National Institute of Child Health and Human Development (B Kapogiannis and S Lee), National Institute on Minority Health and Health Disparities, National Institute of Mental Health, and National Institute on Drug Abuse. Network operations and data management are supported through the ATN Coordinating Center at the University of North Carolina at Chapel Hill.

    The content in this manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

    Author AV’s affiliation is included for informational purposes only; this work was not conducted under the auspices of the Guttmacher Institute. The views expressed herein are those of the authors and do not necessarily reflect the views of the Guttmacher Institute.

    Conflicts of Interest

    None declared.

    References

    1. HIV Surveillance Report.: Centers for Disease Control and Prevention; 2017 Nov.   URL: https://www.cdc.gov/hiv/pdf/library/reports/surveillance/cdc-hiv-surveillance-report-2017-vol-29.pdf [accessed 2020-02-14]
    2. Baral SD, Poteat T, Strömdahl S, Wirtz AL, Guadamuz TE, Beyrer C. Worldwide burden of HIV in transgender women: a systematic review and meta-analysis. Lancet Infect Dis 2013 Mar;13(3):214-222. [CrossRef] [Medline]
    3. American Community Survey.: United States Census Bureau; 2018.   URL: https://data.census.gov/cedsci [accessed 2020-02-26]
    4. Herbst JH, Jacobs ED, Finlayson TJ, McKleroy VS, Neumann MS, Crepaz N, HIV/AIDS Prevention Research Synthesis Team. Estimating HIV prevalence and risk behaviors of transgender persons in the United States: a systematic review. AIDS Behav 2008 Jan;12(1):1-17. [CrossRef] [Medline]
    5. Bauer GR, Travers R, Scanlon K, Coleman TA. High heterogeneity of HIV-related sexual risk among transgender people in Ontario, Canada: a province-wide respondent-driven sampling survey. BMC Public Health 2012 Apr 20;12:292 [FREE Full text] [CrossRef] [Medline]
    6. Brennan J, Kuhns LM, Johnson AK, Belzer M, Wilson EC, Garofalo R, Adolescent Medicine Trials Network for HIV/AIDS Interventions. Syndemic theory and HIV-related risk among young transgender women: the role of multiple, co-occurring health problems and social marginalization. Am J Public Health 2012 Sep;102(9):1751-1757 [FREE Full text] [CrossRef] [Medline]
    7. Garofalo R, Osmer E, Sullivan C, Doll M, Harper G. Environmental, Psychosocial, and Individual Correlates of HIV Risk in Ethnic Minority Male-to-Female Transgender Youth. Journal of HIV/AIDS Prevention in Children & Youth 2007 May 15;7(2):89-104. [CrossRef]
    8. Maulsby C, Millett G, Lindsey K, Kelley R, Johnson K, Montoya D, et al. HIV among Black men who have sex with men (MSM) in the United States: a review of the literature. AIDS Behav 2014 Jan;18(1):10-25. [CrossRef] [Medline]
    9. Institute of Medicine Committee on Lesbian, Gay, Bisexual, and Transgender Health Issues and Research Gaps and Opportunities. The Health of Lesbian, Gay, Bisexual, and Transgender People: Building a Foundation for Better Understanding. Washington, DC: The National Academies Press; 2011.
    10. Millett GA, Flores SA, Peterson JL, Bakeman R. Explaining disparities in HIV infection among black and white men who have sex with men: a meta-analysis of HIV risk behaviors. AIDS 2007 Oct 1;21(15):2083-2091. [CrossRef] [Medline]
    11. Operario D, Nemoto T. HIV in transgender communities: syndemic dynamics and a need for multicomponent interventions. J Acquir Immune Defic Syndr 2010 Dec;55 Suppl 2:S91-S93 [FREE Full text] [CrossRef] [Medline]
    12. Bradford J, Reisner SL, Honnold JA, Xavier J. Experiences of transgender-related discrimination and implications for health: results from the Virginia Transgender Health Initiative Study. Am J Public Health 2013 Oct;103(10):1820-1829. [CrossRef] [Medline]
    13. Gayles TA, Kuhns LM, Kwon S, Mustanski B, Garofalo R. Socioeconomic Disconnection as a Risk Factor for Increased HIV Infection in Young Men Who Have Sex with Men. LGBT Health 2016 Jun;3(3):219-224 [FREE Full text] [CrossRef] [Medline]
    14. Kosciw JG, Palmer NA, Kull RM, Greytak EA. The Effect of Negative School Climate on Academic Outcomes for LGBT Youth and the Role of In-School Supports. Journal of School Violence 2012 Oct 03;12(1):45-63. [CrossRef]
    15. Melendez RM, Exner TA, Ehrhardt AA, Dodge B, Remien RH, Rotheram-Borus M, et al. Health and health care among male-to-female transgender persons who are HIV positive. Am J Public Health 2006 Jun;96(6):1034-1037. [CrossRef] [Medline]
    16. Mena L, Crosby RA, Geter A. A novel measure of poverty and its association with elevated sexual risk behavior among young Black MSM. Int J STD AIDS 2017 May;28(6):602-607. [CrossRef] [Medline]
    17. Meyer IH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychol Bull 2003 Sep;129(5):674-697 [FREE Full text] [CrossRef] [Medline]
    18. Pizer J, Sears B, Mallory C, Hunter N. Evidence of Persistent and Pervasive Workplace Discrimination Against LGBT People: The Need for Federal Legislation Prohibiting Discrimination and Providing for Equal Employment Benefits. Loyola of Los Angeles Law Review 2012 Jan;45(3):715 [FREE Full text]
    19. Rosentel K, VandeVusse A, Hill BJ. Racial and Socioeconomic Inequity in the Spatial Distribution of LGBTQ Human Services: an Exploratory Analysis of LGBTQ Services in Chicago. Sex Res Soc Policy 2020 Mar;17(1):87-103. [CrossRef]
    20. Santos G, Wilson EC, Rapues J, Macias O, Packer T, Raymond HF. HIV treatment cascade among transgender women in a San Francisco respondent driven sampling study. Sex Transm Infect 2014 Aug;90(5):430-433. [CrossRef] [Medline]
    21. Sevelius JM, Patouhas E, Keatley JG, Johnson MO. Barriers and Facilitators to Engagement and Retention in Care among Transgender Women Living with Human Immunodeficiency Virus. Ann Behav Med 2014 Feb;47(1):5-16. [CrossRef] [Medline]
    22. Movement Advancement Project, Center for American Progress, Freedom to Work, Human Right Campaign, National Black Justice Coalition. A Broken Bargain for LGBT Workers of Color. Boulder, CO: Movement Advancement Project; 2013.   URL: https://www.lgbtmap.org/file/a-broken-bargain-for-lgbt-workers-of-color.pdf [accessed 2020-02-17]
    23. Badgett M, Durso L, Schneebaum A. New patterns of poverty in the lesbian, gay, and bisexual community. The Williams Institute. 2013.   URL: https://escholarship.org/uc/item/8dq9d947 [accessed 2020-08-20]
    24. Forge N, Ream G. Homeless lesbian, gay, bisexual and transgender (LGBT) youth in New York City: Insights from the field. Child Welfare 2014;93(2):7-22 [FREE Full text]
    25. McMillan K, Worth H, Rawstorne P. Usage of the Terms Prostitution, Sex Work, Transactional Sex, and Survival Sex: Their Utility in HIV Prevention Research. Arch Sex Behav 2018 Jul;47(5):1517-1527. [CrossRef] [Medline]
    26. Marshall BDL, Shannon K, Kerr T, Zhang R, Wood E. Homeless lesbian, gay, bisexual and transgender (LGBT) youth in New York City: Insights from the field. J Acquir Immune Defic Syndr 2010 Apr;53(5):661-664 [FREE Full text] [CrossRef] [Medline]
    27. Greene JM, Ennett ST, Ringwalt CL. Prevalence and correlates of survival sex among runaway and homeless youth. Am J Public Health 1999 Sep;89(9):1406-1409. [Medline]
    28. Operario D, Soma T, Underhill K. Sex work and HIV status among transgender women: systematic review and meta-analysis. J Acquir Immune Defic Syndr 2008 May 01;48(1):97-103. [CrossRef] [Medline]
    29. Poteat T, Wirtz AL, Radix A, Borquez A, Silva-Santisteban A, Deutsch MB, et al. HIV risk and preventive interventions in transgender women sex workers. Lancet 2015 Jan 17;385(9964):274-286 [FREE Full text] [CrossRef] [Medline]
    30. Williams ML, Atkinson J, Klovdahl A, Ross MW, Timpson S. Spatial bridging in a network of drug-using male sex workers. J Urban Health 2005 Mar;82(1 Suppl 1):i35-i42 [FREE Full text] [CrossRef] [Medline]
    31. Auerbach JD, Parkhurst JO, Cáceres CF. Addressing social drivers of HIV/AIDS for the long-term response: conceptual and methodological considerations. Glob Public Health 2011;6 Suppl 3:S293-S309. [CrossRef] [Medline]
    32. Poteat T, Malik M, Scheim A, Elliott A. HIV Prevention Among Transgender Populations: Knowledge Gaps and Evidence for Action. Curr HIV/AIDS Rep 2017 Aug;14(4):141-152 [FREE Full text] [CrossRef] [Medline]
    33. Sipe TA, Barham TL, Johnson WD, Joseph HA, Tungol-Ashmon ML, O'Leary A. Structural Interventions in HIV Prevention: A Taxonomy and Descriptive Systematic Review. AIDS Behav 2017 Dec;21(12):3366-3430 [FREE Full text] [CrossRef] [Medline]
    34. Adimora AA, Auerbach JD. Structural interventions for HIV prevention in the United States. J Acquir Immune Defic Syndr 2010 Dec;55 Suppl 2:S132-S135 [FREE Full text] [CrossRef] [Medline]
    35. Young LE, Schumm P, Alon L, Bouris A, Ferreira M, Hill B, et al. PrEP Chicago: A randomized controlled peer change agent intervention to promote the adoption of pre-exposure prophylaxis for HIV prevention among young Black men who have sex with men. Clin Trials 2018 Feb;15(1):44-52 [FREE Full text] [CrossRef] [Medline]
    36. van den Berg JJ, Isabel Fernández M, Fava JL, Operario D, Rudy BJ, Wilson PA, ATN 086/106 Protocol Teams for the Adolescent Medicine Trials Network for HIV/ADS Interventions. Using Syndemics Theory to Investigate Risk and Protective Factors Associated with Condomless Sex Among Youth Living with HIV in 17 U.S. Cities. AIDS Behav 2017 Mar;21(3):833-844 [FREE Full text] [CrossRef] [Medline]
    37. Hill BJ, Rosentel K, Bak T, Silverman M, Crosby R, Salazar L, et al. Exploring Individual and Structural Factors Associated with Employment Among Young Transgender Women of Color Using a No-Cost Transgender Legal Resource Center. Transgend Health 2017 Mar;2(1):29-34 [FREE Full text] [CrossRef] [Medline]
    38. Dabrowski J. The Exception that Doesn't Prove the Rule: Why Congress Should Narrow ENDA's Religious Exemption to Protect the Rights of LGBT Employees. Am University Law Rev 2014;63(6) [FREE Full text]
    39. Leppel K. The labor force status of transgender men and women. International Journal of Transgenderism 2016 Oct 13;17(3-4):155-164. [CrossRef]
    40. Raiford JL, Hall GJ, Taylor RD, Bimbi DS, Parsons JT. Erratum to: The Role of Structural Barriers in Risky Sexual Behavior, Victimization and Readiness to Change HIV/STI-Related Risk Behavior Among Transgender Women. AIDS Behav 2017 Jul;21(7):2215. [CrossRef] [Medline]
    41. Colaric S, editor. Employment Best Practices & Support Systems. In: Presentation at AIDS Foundation of Chicago. 2012 Presented at: AIDS Foundation of Chicago; 2012; Chicago, IL.
    42. US Department of Housing and Urban Development. Implementing the National HIV/AIDS Strategy: A Report to the White House Office of National AIDS Policy.: US Department of Health and Human Services; 2011 Feb.   URL: https://files.hiv.gov/s3fs-public/nhas-operational-plan-hud.pdf [accessed 2020-02-20]
    43. Perloff J, Potts C. iFOUR employment program: Increase individual income and independence. 2011 Presented at: Positive Futures Pre-Conference Institute for the United States Conference on AIDS; July 25, 2019; Chicago, IL.
    44. Razzano LA, Hamilton MM, Yost C, Pashka NJ, Perloff JK. Employment Outcomes for Individuals With HIV/AIDS and Co-Occurring Mental Health Factors. American Journal of Psychiatric Rehabilitation 2015;18(1):19-41. [CrossRef]
    45. Hill B, Motley D, Rosentel K, VandeVusse A, Garofalo R, Schneider J, et al. An Employment Intervention Program (Work2Prevent) for Young Men Who Have Sex With Men and Transgender Youth of Color (Phase 2): Protocol for a Single-Arm Mixed Methods Pilot Test to Assess Feasibility and Acceptability. JMIR Res Protoc 2020 Aug 10;9(8):e16401. [CrossRef] [Medline]
    46. Becker M. The Health belief model personal health behavior. Health Education Monographs 1974;2:324-473.
    47. Bernat DH, Resnick MD. Healthy youth development: science and strategies. J Public Health Manag Pract 2006 Nov;Suppl:S10-S16. [CrossRef] [Medline]
    48. Bowers EP, von Eye A, Lerner JV, Arbeit MR, Weiner MB, Chase P, et al. The role of ecological assets in positive and problematic developmental trajectories. J Adolesc 2011 Dec;34(6):1151-1165. [CrossRef] [Medline]
    49. Gavin LE, Catalano RF, David-Ferdon C, Gloppen KM, Markham CM. A review of positive youth development programs that promote adolescent sexual and reproductive health. J Adolesc Health 2010 Mar;46(3 Suppl):S75-S91. [CrossRef] [Medline]
    50. El-Sadr WM, Philip NM, Justman J. Letting HIV transform academia--embracing implementation science. N Engl J Med 2014 May 01;370(18):1679-1681. [CrossRef] [Medline]
    51. Glasgow RE, Eckstein ET, Elzarrad MK. Implementation science perspectives and opportunities for HIV/AIDS research: integrating science, practice, and policy. J Acquir Immune Defic Syndr 2013 Jun 01;63 Suppl 1:S26-S31. [CrossRef] [Medline]
    52. Schackman BR. Implementation science for the prevention and treatment of HIV/AIDS. J Acquir Immune Defic Syndr 2010 Dec;55 Suppl 1:S27-S31 [FREE Full text] [CrossRef] [Medline]
    53. Veniegas RC, Kao UH, Rosales R. Adapting HIV prevention evidence-based interventions in practice settings: an interview study. Implement Sci 2009 Nov 23;4:76 [FREE Full text] [CrossRef] [Medline]
    54. Collins CB, Sapiano TN. Lessons Learned From Dissemination of Evidence-Based Interventions for HIV Prevention. Am J Prev Med 2016 Oct;51(4 Suppl 2):S140-S147 [FREE Full text] [CrossRef] [Medline]
    55. Cunningham SD, Card JJ. Realities of replication: implementation of evidence-based interventions for HIV prevention in real-world settings. Implement Sci 2014 Jan 06;9(5) [FREE Full text] [CrossRef] [Medline]
    56. Kelly JA, Heckman TG, Stevenson LY, Williams PN, Ertl T, Hays RB, et al. Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation. AIDS Educ Prev 2000;12(5 Suppl):87-98. [Medline]
    57. Mukherjee JS, Barry D, Weatherford RD, Desai IK, Farmer PE. Community-Based ART Programs: Sustaining Adherence and Follow-up. Curr HIV/AIDS Rep 2016 Dec;13(6):359-366 [FREE Full text] [CrossRef] [Medline]
    58. Stein R, Shapatava E, Williams W, Griffin T, Bell K, Lyons B, et al. Reduced Sexual Risk Behaviors Among Young Men of Color Who Have Sex with Men: Findings from the Community-Based Organization Behavioral Outcomes of Many Men, Many Voices (CBOP-3MV) Project. Prev Sci 2015 Nov;16(8):1147-1158. [CrossRef] [Medline]
    59. Wilton L, Herbst JH, Coury-Doniger P, Painter TM, English G, Alvarez ME, et al. Efficacy of an HIV/STI prevention intervention for black men who have sex with men: findings from the Many Men, Many Voices (3MV) project. AIDS Behav 2009 Jun;13(3):532-544. [CrossRef] [Medline]
    60. Klesges LM, Estabrooks PA, Dzewaltowski DA, Bull SS, Glasgow RE. Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination. Ann Behav Med 2005 Apr;29 Suppl:66-75. [CrossRef] [Medline]
    61. Hill BJ, Rosentel K, Bak T, Silverman M, Crosby R, Salazar L, et al. Exploring Individual and Structural Factors Associated with Employment Among Young Transgender Women of Color Using a No-Cost Transgender Legal Resource Center. Transgend Health 2017 Mar;2(1):29-34 [FREE Full text] [CrossRef] [Medline]
    62. Rueda S, Raboud J, Mustard C, Bayoumi A, Lavis JN, Rourke SB. Employment status is associated with both physical and mental health quality of life in people living with HIV. AIDS Care 2011 Jan 25;23(4):435-443. [CrossRef]
    63. Price R, Choi J, Vinokur A. Links in the chain of adversity following job loss: How financial strain and loss of personal control lead to depression, impaired functioning, and poor health. Journal of Occupational Health Psychology 2002 Apr;7(4):302-312. [CrossRef] [Medline]
    64. Hammarström A, Janlert U. Early unemployment can contribute to adult health problems: results from a longitudinal study of school leavers. J Epidemiol Community Health 2002 Aug;56(8):624-630 [FREE Full text] [CrossRef] [Medline]
    65. Eaton LA, Driffin DD, Kegler C, Smith H, Conway-Washington C, White D, et al. The role of stigma and medical mistrust in the routine health care engagement of black men who have sex with men. Am J Public Health 2015 Feb;105(2):e75-e82 [FREE Full text] [CrossRef] [Medline]
    66. Scharff DP, Mathews KJ, Jackson P, Hoffsuemmer J, Martin E, Edwards D. More than Tuskegee: understanding mistrust about research participation. J Health Care Poor Underserved 2010 Aug;21(3):879-897 [FREE Full text] [CrossRef] [Medline]
    67. Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q 2012 Jun;90(2):311-346 [FREE Full text] [CrossRef] [Medline]
    68. Malebranche DJ, Peterson JL, Fullilove RE, Stackhouse RW. Race and sexual identity: perceptions about medical culture and healthcare among Black men who have sex with men. J Natl Med Assoc 2004 Mar;96(1):97-107. [Medline]
    69. Peterson M, Nowotny K, Dauria E, Arnold T, Brinkley-Rubinstein L. Institutional distrust among gay, bisexual, and other men who have sex with men as a barrier to accessing pre-exposure prophylaxis (PrEP). AIDS Care 2019 Mar;31(3):364-369. [CrossRef] [Medline]
    70. Kegeles SM, Rebchook G, Tebbetts S, Arnold E, TRIP Team. Facilitators and barriers to effective scale-up of an evidence-based multilevel HIV prevention intervention. Implement Sci 2015 Apr 17;10(1):50 [FREE Full text] [CrossRef] [Medline]
    71. Hill B, Motley DN, Rosentel K, VandeVusse A, Garofalo R, Schneider JA, et al. An Employment Intervention Program (Work2Prevent) for Young Men Who Have Sex With Men and Transgender Youth of Color (Phase 1): Protocol for Determining Essential Intervention Components Using Qualitative Interviews and Focus Groups. JMIR Res Protoc 2020 Aug 10;9(8):e16384. [CrossRef] [Medline]
    72. Horvath KJ, Oakes JM, Rosser BRS, Danilenko G, Vezina H, Amico KR, et al. Feasibility, acceptability and preliminary efficacy of an online peer-to-peer social support ART adherence intervention. AIDS Behav 2013 Jul;17(6):2031-2044 [FREE Full text] [CrossRef] [Medline]
    73. Barlow J, Wright C, Cullen L. A job-seeking self-efficacy scale for people with physical disabilities: Preliminary development and psychometric testing. British Journal of Guidance & Counselling 2002 Feb;30(1):37-53. [CrossRef]
    74. Volmer J, Spurk D. Protean and boundaryless career attitudes: relationships with subjective and objective career success. ZAF 2011;43(3):207-218. [CrossRef]
    75. Porter C, Woo SE, Tak J. Developing and Validating Short Form Protean and Boundaryless Career Attitudes Scales. Journal of Career Assessment 2016;24(1):162-181. [CrossRef]
    76. Hosek SG, Rudy B, Landovitz R, Kapogiannis B, Siberry G, Rutledge B, Adolescent Trials Network (ATN) for HIVAIDS Interventions. An HIV Preexposure Prophylaxis Demonstration Project and Safety Study for Young MSM. J Acquir Immune Defic Syndr 2017 Jan 01;74(1):21-29 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    ACASI: audio computer-assisted self-interview
    ATN: Adolescent Medicine Trials Network for HIV/AIDS Interventions
    CDC: Centers for Disease Control and Prevention
    GNC: gender nonconforming
    iFOUR: Increased Individual Income and Independence
    ISSM: information systems success model
    LGBTQ: lesbian, gay bisexual, transgender, and queer
    MSM: men who have sex with men
    PCA: protean career attitudes
    STI: sexually transmitted infection
    W2P: Work2Prevent
    YMSM: young men who have sex with men
    YTW: young transgender women


    Edited by G Eysenbach; submitted 02.03.20; peer-reviewed by R Sanders, L Young; comments to author 11.06.20; revised version received 25.06.20; accepted 25.06.20; published 11.09.20

    ©Brandon J Hill, Darnell N Motley, Kris Rosentel, Alicia VandeVusse, Robert Garofalo, Lisa M Kuhns, Michele D Kipke, Sari Reisner, Betty Rupp, Rachel West Goolsby, Micah McCumber, Laura Renshaw, John A Schneider. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 11.09.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.