This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
There has been little progress in adolescent health outcomes in recent decades. Researchers and youth-serving organizations struggle to accurately elicit youth voice and translate youth perspectives into health care policy.
Our aim is to describe the protocol of the MyVoice Project, a longitudinal mixed methods study designed to engage youth, particularly those not typically included in research. Text messaging surveys are collected, analyzed, and disseminated in real time to leverage youth perspectives to impact policy.
Youth aged 14 to 24 years are recruited to receive weekly text message surveys on a variety of policy and health topics. The research team, including academic researchers, methodologists, and youth, develop questions through an iterative writing and piloting process. Question topics are elicited from community organizations, researchers, and policy makers to inform salient policies. A youth-centered interactive platform has been developed that automatically sends confidential weekly surveys and incentives to participants. Parental consent is not required because the survey is of minimal risk to participants. Recruitment occurs online (eg, Facebook, Instagram, university health research website) and in person at community events. Weekly surveys collect both quantitative and qualitative data. Quantitative data are analyzed using descriptive statistics. Qualitative data are quickly analyzed using natural language processing and traditional qualitative methods. Mixed methods integration and analysis supports a more in-depth understanding of the research questions.
We are currently recruiting and enrolling participants through in-person and online strategies. Question development, weekly data collection, data analysis, and dissemination are in progress.
MyVoice quickly ascertains the thoughts and opinions of youth in real time using a widespread, readily available technology—text messaging. Results are disseminated to researchers, policy makers, and youth-serving organizations through a variety of methods. Policy makers and organizations also share their priority areas with the research team to develop additional question sets to inform important policy decisions. Youth-serving organizations can use results to make decisions to promote youth well-being.
Youth are central to every major health problem today [
Previous efforts to gather the perspectives of youth to investigate adolescent health have employed more traditional quantitative and qualitative data collection methods. For example, studies that have employed quantitative strategies, such as surveys, are challenged by a long turnaround time for collection, analysis, and dissemination [
One promising method for engaging youth in research is through text messaging. Studies that employ text messaging support the use of this novel strategy for data collection, concluding that the method is effective, easy to implement, and preferable among low-income communities [
The use of text messaging can also address the problems with more traditional approaches (eg, interviews, surveys, school-based samples) to adolescent research described previously [
The MyVoice study was designed in response to calls for “strategic science,” defined as research that addresses gaps in knowledge important to policy decisions. Strategic science is research derived from the reciprocal flow of information between researchers and policy makers, and communicated not only in scholarly publications but also in forms relevant to policymakers [
MyVoice collects real-time data from youth via text messaging. The purpose of MyVoice is to harness youth perspectives to connect youth attitudes, beliefs, and behaviors to researchers and policymakers who can bridge the divide between youth and youth-centered policies. The aim of this paper is to describe the research protocol used to engage youth in an ongoing longitudinal research study to investigate youth perspectives on health and well-being. In addition, plans for dissemination of findings to inform policies that impact youth are described.
MyVoice uses a longitudinal mixed methods research design [
MyVoice research study design.
We assembled an interdisciplinary team of clinicians, academic investigators, methodologists, policymakers, and youth to bring together clinical, topical, methodological, and lived experiences relevant to youth health. The research team includes high school and college students to ensure that youth perspectives are incorporated into the full research process [
The MyVoice study has been iteratively developed through a yearlong pilot study from June 2016 to June 2017. The first phase of pilot testing was used to understand how to recruit and maintain a racially and economically diverse sample, and how to maximize the user experience for participants. The first phase was tested with 114 individuals and revealed key insights that were applied to improve the project. For example, we collected quantitative and qualitative evaluation data about recruitment and consenting procedures, participant preferences for style and content of questions, and technical glitches encountered while participating in the pilot phase. Although the full findings of the pilot evaluation are beyond the scope of this protocol, results were used to make the following changes to the study design: (1) focusing questions on political topics and current events, (2) waiving parental consent for participants younger than age 18 years, (3) addressing technical issues with the text messaging platform, and (4) using targeted advertisement to meet sampling quotas for demographics.
The next phase aims to enroll a national population-based sample of approximately 1000 participants. To date, MyVoice has enrolled approximately 800 participants and is nearing quota goals for all demographics.
Youth are recruited in person, at community and youth-centered events, and online via Facebook and Instagram advertisements. Recruiting on Facebook and Instagram is an effective way to reach populations not traditionally reached through research because Facebook is used by 71% of adolescents and more likely to be used by teens that are from disadvantaged households [
Quota sampling is used to achieve a population-based sample of youth participants. Recruited samples are matched with demographic characteristics of a national sample of youth aged 14 to 24 years. Facebook advertisements are created to target specific demographic characteristics, including age, gender, race/ethnicity, education, family income, and region of the country using weighted samples from the most recent American Community Survey. Creating a national sample of youth requires regular monitoring of participant demographics and tailoring online advertisements to meet predetermined quotas. Specifically, the study team is (1) opening recruitment on Facebook and Instagram and conducting weekly checks of participant demographics currently enrolled; (2) adjusting Facebook and Instagram recruitment advertisements to meet benchmarks, such as pausing advertisements targeting groups that have met the benchmark while increasing advertisements targeting populations that have not met the benchmark; and (3) continuing recruitment until all benchmarks are met and the sample is adequately powered for quantitative calculations.
Modest incentives are provided for participation throughout the study. Participants receive a one-time US $5 incentive for completing the online demographic survey on the study website following enrollment in the study. For the weekly surveys, participants receive US $1 for each completed survey. Every 12 weeks, participants also receive a US $3 bonus if all 12 weekly surveys are completed (total incentive of US $15 every 12 weeks for completed surveys). MyVoice also offers occasional “bonus” questions to elicit perspectives on time-sensitive policy issues. Participants receive an additional US $1 for completion of each bonus survey.
Based on our iterative pilot phases, we developed three primary strategies beyond the incentives described previously to promote participation and reduce attrition. First, the study design focuses on the perspectives of youth and asks youth to share their experiences. Second, with a goal to better understand and improve the health of youth, we actively generate weekly question sets that address topics that have real-world implications. Third, we regularly disseminate results to the youth participants, through infographics, summaries of findings, or select quotations (see Dissemination for more detail).
In weekly MyVoice research team meetings, upcoming topics are identified and discussed. A research question is identified to structure each week’s question set. Each week of questions focuses on one topic or issue identified by the MyVoice team, external youth-serving organizations, researchers, or policy makers. Topics are selected that align with upcoming policy priorities or timely policy concerns specific to the health of youth. In pilot testing, topics that have been fielded included stress, weight, nutrition, substance use, health insurance, relationships, and sexual health education.
Weekly text message surveys consist of three to five questions. Although topics and question structure varied, we include closed- and open-ended questions that assess the knowledge, attitudes, and beliefs of study participants. During team meetings, we revise questions to maximize comprehension, appropriateness for youth, and adherence to the research question. The team also discusses how to structure the questions to optimize the use of the text messaging platform.
Once written and edited by the research team, weekly surveys are sent to a pilot team of youth, community members, survey experts, methodologists, and topical experts. Participants in the pilot-testing sample provide feedback regarding topics, sentence structure, phrasing, and word choice. The research team makes modifications to weekly surveys in response to pilot feedback before data collection.
After participants provide consent/assent via the study website, they receive a text message tutorial that briefly describes the study process (see
Questions are created in 12-week blocks. Textizen is the Web-based platform used to deliver surveys via text message. One-week delays are placed between each weekly question set. Weekly surveys are sent out to participants via Textizen on a consistent day and time.
Each week’s questions are rated by respondents on a scale of 1 to 5 “stars.” Ratings for questions are used to inform future surveys.
All responses are downloaded as a comma-separated values file. Data analysis depends on the nature of the questions from each weekly survey. Closed-ended and categorical (quantitative) responses are cleaned for case mismatch, spelling variants, and typographic errors to prepare for analysis using descriptive statistics.
MyVoice demographic survey completed on enrollment.
Question | Response options | ||
What is your date of birth? Please enter it in the following format: mm/dd/yyyy. | Open-ended | ||
What is your cell phone number? Please enter in the following format: 1 555 555 5555 | Open-ended | ||
What is your gender? | Male; female; transgender (FTM); transgender (MTF); nonbinary; other (please specify) | ||
What is your race? Check all that apply. | American Indian or Alaska Native; Asian; black or African American; Native Hawaiian or other Pacific Islander; white or Caucasian; other (please describe) | ||
Are you Hispanic or Latino? | Yes; no | ||
What zip code do you live in? | Open-ended | ||
What is the highest level of education you have achieved? | Eighth grade or less; some high school; high school graduate; some vocational/technical training; completed vocational/technical training; some college; completed an associate’s degree; completed a bachelor’s degree; some graduate school; completed a master’s degree; some graduate training beyond a master’s degree; completed a doctoral degree | ||
What is the highest level of education any parent/guardian has achieved? | Eighth grade or less; some high school; high school graduate; some vocational/technical training; completed vocational/technical training; some college; completed an associate’s degree; completed a bachelor’s degree; some graduate school; completed a master’s degree; some graduate training beyond a master’s degree; completed a doctoral degree | ||
Who do you live with most of the time? | My parent(s)/guardian(s); my aunt/uncle; my kids; in a dorm; in apartment or house with other people, not family; in a fraternity/sorority; I live alone; other; my spouse, partner, or significant other | ||
How many people are in your immediate family? (Include you, any parents/guardians, siblings, step-siblings, etc) | 1-3; 4-6; 7-10; ≥10 | ||
What is your parent(s)/guardians’ current marital status? | Married; together but not married; separated; divorced; widowed; unsure | ||
How did you hear about MyVoice? | From a family member; from a friend; Facebook; Instagram; other | ||
Thinking about the house you live in at the moment, do your parents own it or rent it? (If they have a mortgage, tick “they own it”) | They own it; they rent it; I don’t know | ||
Do you have a car or van at home? | Yes, one car or van; yes, more than one car or van; no, we don’t own a car or van | ||
Which of the following Internet technology devices do you have at home? | Desktop computer; laptop computer; iPad or other tablet; other (please specify) | ||
When you were in middle/high school, did you receive free or reduced price school lunch? | Yes; no | ||
What is your annual household income? (Just an estimate of the total amount of everyone in your household) | Open-ended |
Screenshot of tutorial sent to new participants.
MyVoice dissemination model.
For open-ended (qualitative) responses, natural language processing (NLP) techniques are applied to cluster responses. Using NLP, responses are parsed and synonymous words are first grouped into word clusters. These word clusters are further grouped using word similarity measures resulting in semantic word clusters. Our initial analysis of these semantic word clusters shows sufficient agreement with traditional qualitative methods based on manual review and coding responses. Effectiveness of this approach will be reported in a future paper.
Results may be stratified by demographic characteristics to better understand differences and meet the needs of specific stakeholder organizations. For example, responses to surveys may be stratified by age to distinguish between younger (age 14-18 years) and older (age 19-24 years) respondents.
Results will be disseminated to multiple audiences, including academics, policymakers, and youth (see
For youth, findings are shared via text message as #WeHearU messages (see
Example of #WeHearU sent to participants.
Postcard distributed to lawmakers.
We have launched the national study and are currently recruiting and enrolling participants through in-person and online strategies. The MyVoice website is live and enrollment (including consent) and demographic questionnaires are embedded (see
Through the first two phases of pilot testing, we have collected 56 weeks of data from 114 participants. See
Screenshot of home page for MyVoice website.
Embedded enrollment (including consent) form in MyVoice website.
Intro:
Q1:
Q2:
Q3:
Q4:
Q5:
Outro:
Intro:
Q1:
Q2:
Q3:
Q4:
Outro:
Intro:
Q1:
Q2:
Q3:
Q4:
Q5:
Outro:
The MyVoice sample is not nationally representative; however, quota sampling allows MyVoice to recruit using a variety of methods while meeting national benchmarks for important demographic characteristics.
Question sets typically only include three to five questions per week and may require repeat longitudinal sampling to get in-depth knowledge (ie, fielding several weeks of questions on the same topic). However mixed methods techniques that integrate qualitative and quantitative data allow us to collect data with a deep understanding of context not attainable in traditional surveys of youth and are collected in a manner that may elicit more truthful responses [
This study represents a novel example of strategic science that collects youth thoughts and opinions in real time with a goal of disseminating to appropriate change agents at the pace that policy decisions are being made [
This study protocol describes a youth-centered method of gathering the beliefs, attitudes, and behaviors of youth in their own words. We know that sustainability of interventions, programs, and policies is dependent on the buy-in of its users. Understanding youth perspectives can allow for the development of youth-centered policies that build on the needs, priorities, and recommendations of those who will be directly affected.
The Department of Health and Human Services Office of Adolescent Health currently lists no ongoing longitudinal qualitative or mixed methods study of youth [
Second, we have developed a strategy to engage youth over time. Including youth in longitudinal research has been difficult, yet essential, to understanding the health problems we face today [
Third, our use of text messaging and NLP allows for real-time data collection, analysis, and dissemination of research findings. The timeline of research (months to years) is typically misaligned with the timeline of policy decision making (weeks to months) [
Finally, our project adds to the growing evidence that participation of advisory groups, peer leaders, and community researchers enhances the quality and reach of research [
natural language processing
This research was supported by the University of Michigan MCubed program and the Department of Family Medicine Building Blocks grant. We would like to thank Christina Czuhajewski for contributing design and visual communication expertise and Katie Grode for providing administrative support for this study.
None declared.