Published on in Vol 10, No 9 (2021): September

Preprints (earlier versions) of this paper are available at, first published .
Adapting to the Pandemic: Protocol of a Web-Based Perinatal Health Study to Improve Maternal and Infant Outcomes

Adapting to the Pandemic: Protocol of a Web-Based Perinatal Health Study to Improve Maternal and Infant Outcomes

Adapting to the Pandemic: Protocol of a Web-Based Perinatal Health Study to Improve Maternal and Infant Outcomes


1Department of Family Medicine, University of Michigan Medical School, Ann Arbor, MI, United States

2Department of Obstetrics and Gynecology, University of Michigan Medical School, Ann Arbor, MI, United States

3Department of Health Behavior and Health Education, University of Michigan School of Public Health, Ann Arbor, MI, United States

4Butler Center for Research, Hazelden Betty Ford Foundation, Center City, MN, United States

5Department of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Providence, RI, United States

6Department of Medicine, Women and Infants Hospital, Providence, RI, United States

7Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa

Corresponding Author:

Golfo Tzilos Wernette, PhD

Department of Family Medicine

University of Michigan Medical School

1018 Fuller Street

Ann Arbor, MI, 48104

United States

Phone: 1 17349987120


Background: The identification of interconnected health risks during the perinatal period offers an opportunity to prevent negative maternal and infant health outcomes. Marijuana, opioid, and other substance use during pregnancy is a rapidly growing public health concern with significant and costly health consequences for the woman and the developing fetus. Pregnant persons who misuse substances are disproportionately more likely to engage in risky sexual behaviors resulting in sexually transmitted infections (STIs), which are on the rise in this population and can lead to adverse effects on maternal health and on fetal development.

Objective: Our goal is to continue testing an innovative and low-cost technology-delivered intervention, the Health Check-Up for Expectant Moms (HCEM), which simultaneously targets alcohol and drug use and STI risk during pregnancy, both of which are on the rise during the COVID-19 pandemic.

Methods: We describe the ways in which we have adapted the web-based HCEM intervention to continue recruitment and study enrollment during the pandemic.

Results: Study recruitment, visits, and participant safety assessments were all successfully modified during the initial year of the COVID-19 pandemic. Compared to in-person recruitment that occurred prepandemic, remote recruitment yielded a greater proportion of women enrolled in the study (83/136, 61.0% vs 43/52, 83%) in a shorter period (12 months vs 7 months).

Conclusions: Despite study challenges related to the pandemic, including time and effort adapting to a remote protocol, remote recruitment and visits for this study were found to constitute a successful approach.

Trial Registration: NCT03826342;

International Registered Report Identifier (IRRID): DERR1-10.2196/30367

JMIR Res Protoc 2021;10(9):e30367




Alcohol, marijuana, opioid, and other substance use during pregnancy is a rapidly growing public health concern with significant and costly health consequences for the woman and developing fetus [1]. Women who misuse substances are disproportionately more likely to engage in risky sexual behaviors that can result in sexually transmitted infections (STIs). Pregnant persons are a scientifically complex group as national STI prevalence rates are on the rise among this population, leading to adverse effects on maternal health and on fetal development [2].

COVID-19 and Women’s Health

The impact of the COVID-19 pandemic on women’s health has been significant. Women have reported experiencing more severe stress than men resulting in greater health impact, and pregnant women have reported more health-related worry and high levels of anxiety directly related to the COVID-19 pandemic [3,4]. Pregnant women are particularly impacted by the pandemic as they are at increased risk for severe illness compared to nonpregnant women, and they may be at higher risk for preterm birth [5]. Since the onset of the COVID-19 pandemic, there are numerous reports that women’s alcohol and other drug use has been rising in the United States. The US Centers for Disease Control and Prevention reported that approximately 12% of adult women reported either beginning or increasing their substance use to cope with pandemic-related stress [6]. Frequency of binge drinking—for women, defined as four or more drinks on one occasion—has increased substantially (over 40%) during this time, and cases of alcohol-related liver diseases have increased, especially among young women [7]. Moreover, marijuana use continues to escalate among pregnant women, with the most commonly cited reasons for use cited as relief of stress or anxiety, nausea or vomiting, and pain [8]. Pregnant women with opioid use disorders have faced unique challenges to care during the pandemic due to their complex health care needs (eg, clinic travel to receive medication and stigma) [9].

The co-occurrence of alcohol and substance use and sexual risk-taking contribute significantly to STI acquisition. With respect to the impact of the COVID-19 pandemic on sexual health, STIs were already at record highs before the pandemic and climbing, especially for childbearing women. Recent reports (2019) reveal increases from the previous year in the prevalence of gonorrhea, chlamydia, and syphilis of 5%, 3%, and 14%, respectively; among women of childbearing age, there was a 36% increase in syphilis cases [2]. Furthermore, there was a 40% increase in congenital syphilis cases, and an alarming 22% increase in newborn deaths related to congenital syphilis during the same time period [2]. Access to STI screening and treatment has been limited during the pandemic due to restrictions [10], likely leading to continued health consequences among this group.

The perinatal period has been identified as an urgent time to address and prevent these co-occurring risks [11], and technology-delivered interventions are ideally suited given their low cost and potential reach [12]. We are currently testing an innovative and low-cost technology-delivered intervention, the Health Check-Up for Expectant Moms (HCEM) (see study protocol in Tzilos Wernette et al [13]), which is theoretically grounded, consistent with motivational interviewing, and informed by the Information-Motivation-Behavior model, simultaneously targeting alcohol and drug use risk and risky sexual behavior during pregnancy. The HCEM is a 60-minute intervention that is guided by a narrator; provides information, including short video testimonials, highlighting the bidirectional relationships among these risk factors; and provides motivational strategies to enhance behavioral skills, including male and female condom use. In this paper, we highlight the ways in which our study team has adapted our research protocol and study procedures so that we can continue recruitment, assessment, and intervention remotely during the COVID-19 pandemic. Remote study participation may have potential advantages for participants, including lessening the burden of travel time, costs, and inconvenience associated with in-person visits [14].

Our study uses a two-group, randomized controlled design with a baseline session (prior to 22 weeks pregnant), plus two brief booster sessions within 1 month of study enrollment. We conduct three follow-up assessments at 2 and 6 months from baseline, and a postpartum assessment at 6 weeks postpartum. All sessions are asynchronously delivered via technology using the Computerized Intervention Authoring Software [15]. Additionally, each session includes the assessment of risk behaviors (eg, risky sexual behavior and alcohol and drug use) using the “timeline follow-back” interview method, a calendar-assisted structured interview [16]. Study recruitment prior to the pandemic was conducted exclusively in person at obstetric and primary care clinics. During the pandemic, recruitment efforts shifted to remote (eg, phone, text messaging, and online). Study participants include 250 pregnant women, aged 18 years or older; participants endorse the following risk factors in order to be eligible for the study: (1) unprotected sex in the past 30 days in addition to having more than one male partner in the last 6 months and/or having uncertainty about current sexual partner’s monogamy and (2) current alcohol and drug use risk [17,18]. The study protocol was approved by the University of Michigan Medical Institutional Review Board (HUM00143896) and registered at (NCT03826342).


All in-person behavioral research studies university-wide were paused in March 2020 due to the COVID-19 pandemic. This impacted our study in three important ways. Primarily, we could no longer recruit new participants or conduct any study visits, including assessments and viewing of the intervention or control conditions. Next, STI testing and biological (eg, hair and urine) sample collection for the assessment of drug use was suspended since this was no longer done in person during a perinatal study visit. Furthermore, because we were no longer seeing women in person, we had to modify our guidelines around assessing for participant safety, including the remote assessment of suicidality. Finally, the following modifications to our technology-based intervention and protocol were made in order to continue during the pandemic. The changes we implemented are detailed below in Table 1 [19], by category; for each point, we describe the original protocol followed by modifications (ie, “current protocol”).

Table 1. Modifications to the Health Check-Up for Expectant Moms protocol in response to the COVID-19 pandemic.
Study aspects and protocolsDetails
Study visits

Baseline visits

Original protocolBaseline visits were conducted in person with research staff via interviews and accessing the CIASa platform using a study iPad.

Current protocolBaseline visits were adapted to be completed over the telephone and online through an individualized weblink to the CIAS platform.

Delivery of follow-up assessments

Original protocolDelivery of follow-up assessments (2 and 6 months from baseline and 6 weeks postpartum) was conducted in person with research staff via interviews and accessing the CIAS platform using a study iPad.

Current protocolDelivery of all follow-up assessments was modified to be completed over the phone and through an individualized weblink to the CIAS platform.

Collection of biological samples

Original protocolCollection of biological samples (urine and hair) for drug use assessment was conducted in person at clinic by research staff.

Current protocolCollection of biological samples is temporarily suspended.

Incentive payments

Original protocolIncentive payments (US $200 in cash) were made in person at study visits.

Current protocolIncentive payments (US $200) are made via electronic gift card (emailed) or check (mailed).

Modified COVID-19 assessmentb

Current protocolA modified COVID-19 assessment was added to every study visit: the COVID-19 Family Stress Screener [19] was adapted for our study.

Intervention arm content

Original protocolIntervention arm content included references to booklet given in person and to the male and female condom demonstrators, which was part of the in-person intervention.

Current protocolIntervention arm content was updated to reflect remote nature of visits (eg, no longer refers to booklet given in person or to male and female condom demonstrators).

Informed consent

Original protocolSigned informed consent was obtained by research staff in person at the clinic.

Current protocolInformed consent process is conducted over the phone (copy of consent emailed to participants) and the research staff obtains consent verbally.

Safe sex kits

Original protocolSafe sex kits (eg, male and female condoms and dental dams) and study booklets were offered to the participants in person during the intervention study visit.

Current protocolSafe sex kits and study booklets are mailed to interested intervention participants with their permission.

Informational handouts

Original protocolAll informational handouts (intervention and control) and local resources were provided to the participants in person at study visits.

Current protocolAll informational handouts and list of local resources are emailed, including the study booklet for participants in the intervention arm.

Provider-ordered sexually transmitted infection (STI) tests

Original protocolSTI test results (urine samples) were collected at our clinic labs at each study visit.

Current protocolNew data are collected of all provider-ordered STI tests during participant’s pregnancy and postpartum period (not part of original protocol, in which we only collected study STI test results).

Potential participants

Original protocolPotential participants were identified from clinic schedules and approached by research staff in person at clinics.

Current protocolPotential participants are still identified from clinic schedules, but recruitment resumes remotely via telephone calls, text messages, and email messages.

Call patientsb

Current protocolPatients are not called until after their second obstetrics visit to help ensure active pregnancy (modified during COVID-19 pandemic, given remote procedures).

Interested patients

Original protocolInterested patients met with research staff to complete the screening survey in person at the clinic using the study iPad.

Current protocolInterested patients are emailed a weblink to complete the screening survey online.

Screening consent form

Original protocolThe screening consent form was offered to the patient in person prior to completing the survey questions.

Current protocolThe screening consent form appears on the webpage before any survey questions can be viewed.


Original protocolAll prospective study participants (eligible and ineligible) were physically given a US $5 gift card for their time by the research staff upon completing the screening survey at the clinic.

Current protocolAll prospective study participants are emailed US $5 electronic gift card for their time upon completing the screening survey.

Eligible patients

Original protocolEligible patients were scheduled for the baseline visit and consented in person at the clinic.

Current protocolEligible patients are contacted by the research assistant via telephone, where more detailed study information is given; a copy of the consent form is emailed; and a baseline visit is scheduled for interested patients.
Participant safety, suicide risk assessment, and child abuse and neglect

Modified procedures

Original protocolWe did not include questions explicitly asking about suicide risk or child abuse and neglect. Participants that spontaneously indicated any distress from the study were given a list of referral options that included information on how to contact a social worker within our health system and/or were assessed by clinic social worker in person during the study visit.

Current protocolWe modified our procedures for suicide risk assessment to reflect remote nature of visits (ie, research staff can no longer go to clinic social worker with participant to assess severity in person, as was done in the original protocol). We also modified our protocol for steps to take if our study team becomes aware of child abuse and neglect remotely, including procedures for how to report.

List of local resources

Original protocolA list of local resources was provided to all participants at the baseline study visit.

Current protocolAn updated list of local resources is emailed and includes online resources (eg, Alcoholics Anonymous and Narcotics Anonymous remote meetings) during COVID-19 and changes to hours and/or procedures for all other local resources (eg, food pantries, counseling, and STI testing).

aCIAS: Computerized Intervention Authoring Software.

bThis study aspect only came about as a result of the COVID-19 pandemic and only includes a current protocol.

Participant Recruitment

While the pandemic caused significant initial delays to our study recruitment, since adapting to remote recruitment, we have exceeded the total number of patients screened compared to prepandemic recruitment. Remote recruitment over a 7-month period, from August 2020 to March 2021, has yielded 1122 contacts, with 305 women being screened. In the previous 12 months, from April 2019 to March 2020, we approached 892 women in the clinics and screened 232 women. Furthermore, of all women who were screened and were eligible prior to the pandemic, only 61.0% (83/136) went on to enroll in the study, whereas during remote recruitment, a greater proportion (43/52, 83%) of women enrolled in the study.

Principal Findings

In this paper, we provide an overview of the HCEM intervention and study methodology. The COVID-19 pandemic has been disruptive to most facets of life globally, including research efforts. We highlight the ways in which we successfully adapted our study protocol to continue recruitment and intervention efforts with pregnant women during the pandemic. Study recruitment, visits, and participant safety assessments were all modified during the initial year of the COVID-19 pandemic.

Despite the challenges of the pandemic on women’s health and intervention efforts, there have been silver linings. Clinically, reduced health care visits and more telehealth visits have had some positive impacts on pregnant women [20-22]. Across the country, clinical treatment programs for substance use have ramped up telehealth utilization, offering care through a mix of telephone and/or video visits, with limited in-person visits to reduce COVID-19 exposure risk. This has been particularly beneficial to perinatal women seeking care for substance use disorder, including opioid use disorder, with relaxed regulations around travel to crowded clinics to receive treatment medications (eg, methadone) and psychotherapy, which help to overcome known treatment barriers (ie, stigma, health care access, transportation, and childcare) in this population [9,20-22]. Our own research has also seen positive impacts as a result of the COVID-19 pandemic, particularly in terms of study recruitment. The ability to offer “contact-free” visits alleviates anxiety surrounding the risks of in-person visits, and the convenience of being able to take the study screener and assessments from home seems to have increased interest in study participation.

Remote recruitment yielded a larger percentage of women screened in a shorter time frame compared to in-person recruitment. Furthermore, a large percentage of eligible women went on to enroll in the study. Study procedures and intervention content had to be adapted to reflect the remote delivery, but because the groundwork for the study was primarily technology based, it was a feasible and successful transition that took place over the course of 3 months. Our study team had to carefully consider potential issues related to participant safety, given that we no longer had in-person access to clinical care providers in the event of an emergency. This resulted in revisions to our procedures for how to respond to the disclosure of suicide risk and child abuse and neglect reporting. Our study assessments—both the original and modified versions—do not include questions explicitly asking about suicide risk. Rather, if a participant spontaneously shares a desire or plan to hurt themselves and/or suicide or self-harm is explicitly mentioned, we have in place an adapted protocol of action steps to remotely assess suicidality risk (ie, low, moderate, or high). For example, if a participant reports any suicidal intent, but indicates that she has no likelihood of acting on these thoughts in the near future, our staff provides her with resources (eg, National Suicide Prevention Lifeline and local resources, including social work) and offers to notify their primary care physician or mental health provider. If the participant discloses that they are somewhat or very likely to harm themselves, our research staff expresses concern and recommends that they speak to a professional as soon as possible. They also contact the principal investigator (PI) or clinical backup of the study who would plan to follow up with them by phone immediately. The research staff would provide numbers for local crisis and emergency mental health services within our health system that are offered. The PI would take appropriate action depending on the level of risk (eg, high risk would require a referral to the emergency department closest to the study participant). To date, we have not had a participant disclose suicidality.


Despite study challenges related to the pandemic, including time and effort adapting to a remote protocol, remote recruitment and remote visits for this study were found to constitute a successful approach. We also recognize the potential advantages to participants in accessing our study remotely, including lessening the burden of time, costs, and inconvenience associated with traveling for in-person visits at the clinic [14]. It is important to note, however, that there are limitations to remote delivery of study visits, which may pose challenges for generalizing to other research studies or clinical settings that may not offer remote care. First, because study participation requires access to a telephone and a device with a reliable internet connection, it is possible that we could exclude low-income and marginalized women who might not have access to the required technology, and who are also the most vulnerable to negative health outcomes. Second, in our remote protocol, we are no longer able to collect biological (ie, hair and urine) samples as a secondary measure for drug use. Third, as this study is ongoing, our outcomes are pending. Notwithstanding these unknowns, just as COVID-19 has likely permanently changed the way we seek and utilize health care, it has also altered the way we conduct clinical behavioral research. In both instances, there are many benefits to embrace and maintain as we move forward.


This research was supported by funding from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (grant R01HD093611; PI: Tzilos Wernette). We would like to thank all of the women who participated in this study. We are grateful to our study coinvestigators, Drs Ananda Sen and Lisa Prosser, and consultant, Dr Chris Kahler, for their support. We also thank the members of our Data and Safety Monitoring Board for their guidance, especially during the pandemic. We would also like to acknowledge the contributions of Amy Frauhammer, Cailey Rullman, and Michelle Scully.

Authors' Contributions

GTW, KC, OM, QMN, and CZ all contributed to the design of the study adaptations. KC contributed to participant recruitment and data entry. GTW, KC, OM, QMN, and CZ all contributed to the analysis, interpretation, and writing. All authors contributed to the editing, review, and approval of the manuscript.

Conflicts of Interest

None declared.

  1. Substance Abuse and Mental Health Services Administration. Key Substance Use and Mental Health Indicators in the United States: Results From the 2019 National Survey on Drug Use and Health (HHS Publication No. PEP20-07-01-001, NSDUH Series H-55). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; 2020.   URL: https:/​/www.​​data/​sites/​default/​files/​reports/​rpt29393/​2019NSDUHFFRPDFWHTML/​2019NSDUHFFR090120.​htm [accessed 2021-08-03]
  2. New CDC report: STDs continue to rise in the US. Centers for Disease Control and Prevention. 2019 Oct 08.   URL: [accessed 2021-08-03]
  3. Connor J, Madhavan S, Mokashi M, Amanuel H, Johnson NR, Pace LE, et al. Health risks and outcomes that disproportionately affect women during the COVID-19 pandemic: A review. Soc Sci Med 2020 Dec;266:113364 [FREE Full text] [CrossRef] [Medline]
  4. Naurin E, Markstedt E, Stolle D, Enström D, Wallin A, Andreasson I, et al. Pregnant under the pressure of a pandemic: A large-scale longitudinal survey before and during the COVID-19 outbreak. Eur J Public Health 2021 Feb 01;31(1):7-13 [FREE Full text] [CrossRef] [Medline]
  5. Pregnant and recently pregnant people: At increased risk for severe illness from COVID-19. Centers for Disease Control and Prevention. 2021 Mar 05.   URL: https:/​/www.​​coronavirus/​2019-ncov/​need-extra-precautions/​pregnant-people.​html#anchor_1614966846976 [accessed 2021-08-03]
  6. Czeisler MÉ, Lane RI, Petrosky E, Wiley JF, Christensen A, Njai R, et al. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic - United States, June 24-30, 2020. MMWR Morb Mortal Wkly Rep 2020 Aug 14;69(32):1049-1057 [FREE Full text] [CrossRef] [Medline]
  7. Pollard MS, Tucker JS, Green HD. Changes in adult alcohol use and consequences during the COVID-19 pandemic in the US. JAMA Netw Open 2020 Sep 01;3(9):e2022942 [FREE Full text] [CrossRef] [Medline]
  8. Ko JY, Coy KC, Haight SC, Haegerich TM, Williams L, Cox S, et al. Characteristics of marijuana use during pregnancy - Eight states, Pregnancy Risk Assessment Monitoring System, 2017. MMWR Morb Mortal Wkly Rep 2020 Aug 14;69(32):1058-1063 [FREE Full text] [CrossRef] [Medline]
  9. McCarthy JJ, Jones HE, Terplan M, Rudolf VP, von Klimo MC. Changing outdated methadone regulations that harm pregnant patients. J Addict Med 2021 Apr 01;15(2):93-95. [CrossRef] [Medline]
  10. Mmeje OO, Coleman JS, Chang T. Unintended consequences of the COVID-19 pandemic on the sexual and reproductive health of youth. J Adolesc Health 2020 Sep;67(3):326-327 [FREE Full text] [CrossRef] [Medline]
  11. ACOG Practice Bulletin No. 117: Gynecologic care for women with human immunodeficiency virus. Obstet Gynecol 2010;116(6):1492-1509. [CrossRef] [Medline]
  12. Ondersma SJ, Svikis DS, Thacker LR, Beatty JR, Lockhart N. Computer-delivered screening and brief intervention (e-SBI) for postpartum drug use: A randomized trial. J Subst Abuse Treat 2014 Jan;46(1):52-59 [FREE Full text] [CrossRef] [Medline]
  13. Tzilos Wernette G, Plegue M, Mmeje O, Sen A, Countryman K, Ngo Q, et al. Reducing sexual health risks and substance use in the prenatal setting: A study protocol for a randomized controlled trial. Contemp Clin Trials 2019 Sep;84:105827 [FREE Full text] [CrossRef] [Medline]
  14. McDermott MM, Newman AB. Remote research and clinical trial integrity during and after the coronavirus pandemic. JAMA 2021 May 18;325(19):1935-1936. [CrossRef] [Medline]
  15. Ondersma SJ, Chase SK, Svikis DS, Schuster CR. Computer-based brief motivational intervention for perinatal drug use. J Subst Abuse Treat 2005 Jul;28(4):305-312 [FREE Full text] [CrossRef] [Medline]
  16. Sobell LC, Sobell MB. Timeline follow-back. In: Litten RZ, Allen JP, editors. Measuring Alcohol Consumption. Totowa, NJ: Humana Press; 1992:41-72.
  17. Sokol RJ, Martier SS, Ager JW. The T-ACE questions: Practical prenatal detection of risk-drinking. Am J Obstet Gynecol 1989 May;160(4):863-868; discussion 868. [CrossRef] [Medline]
  18. Yonkers K, Gotman N, Kershaw T, Forray A, Howell H, Rounsaveille B. Screening for prenatal substance use: Development of the Substance Use Risk Profile-Pregnancy scale. Obstet Gynecol 2010;116(4):827-833. [CrossRef]
  19. Huth-Bocks A. COVID-19 Family Stress Screener. 2020.   URL: https:/​/commons.​​HigherLogic/​System/​DownloadDocumentFile.​ashx?DocumentFileKey=4d647e65-0459-4c72-a051-c671d42eb9ab [accessed 2021-08-31]
  20. Moreland A, Guille C, McCauley JL. Increased availability of telehealth mental health and substance abuse treatment for peripartum and postpartum women: A unique opportunity to increase telehealth treatment. J Subst Abuse Treat 2021 Apr;123:108268. [CrossRef] [Medline]
  21. Patton EW, Saia K, Stein MD. Integrated substance use and prenatal care delivery in the era of COVID-19. J Subst Abuse Treat 2021 May;124:108273 [FREE Full text] [CrossRef] [Medline]
  22. Sadicario JS, Parlier-Ahmad AB, Brechbiel JK, Islam LZ, Martin CE. Caring for women with substance use disorders through pregnancy and postpartum during the COVID-19 pandemic: Lessons learned from psychology trainees in an integrated OBGYN/substance use disorder outpatient treatment program. J Subst Abuse Treat 2021 Mar;122:108200 [FREE Full text] [CrossRef] [Medline]

HCEM: Health Check-Up for Expectant Moms
PI: principal investigator
STI: sexually transmitted infection

Edited by G Eysenbach; submitted 18.05.21; peer-reviewed by K McDonnell, R Krukowski; comments to author 26.07.21; revised version received 30.07.21; accepted 03.08.21; published 10.09.21


©Golfo Tzilos Wernette, Kristina Countryman, Okeoma Mmeje, Quyen M Ngo, Caron Zlotnick. Originally published in JMIR Research Protocols (, 10.09.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.