Prior to this study, CDC researchers and collaborators completed a pilot project to inform the current larger study [7]. The protocol used in the CDC pilot study served as a framework for the funding announcement. Because Utah and Arizona submitted separate applications in response to the funding announcement, a unified protocol was developed by both sites in the first year of the study. The two sites were tasked with identification, location, and recruitment of English- or Spanish-speaking parents with children with SB (International Classification of Diseases, Ninth Revision, Clinical Modification, codes 741.0, spina bifida with hydrocephalus and 741.9, spina bifida without hydrocephalus) born between September 1, 2004, and August 31, 2009, residing in one of the two catchment areas. Individuals living in nearby states were also recruited, but only if they attended an SB clinic at the participating sites. The birth date range was selected to ensure children were between 3 and 6 years at the time of enrollment in the study, which allowed the collection of extensive baseline data on health status, social, and cognitive development, of the children prior to entering school. Each child’s parent was required to be over 18 years old and able and willing to sign the consent forms. Institutional review board (IRB) approvals were obtained separately at the two sites. Recruitment began in May 2011 and ended in September 2013.

Children with birth defects were first identified from surveillance systems from both sites. IRBs at each site differed in how they allowed potential participants to be identified, contacted, and recruited; therefore, the recruitment processes varied between the two sites. In Utah, children with SB were identified using population-based statewide surveillance data from the Utah Birth Defect Network. Surveillance data included demographic and diagnostic information to determine case eligibility. All parents of eligible children were sent a recruitment letter from the Utah Department of Health introducing them to the study and inviting them to participate. Follow-up phone calls were made by the study coordinator approximately 7 days after the recruitment letter was sent to assess the parents’ interest in participating in the study. Parents of eligible children with SB who were attending the SB clinic during the study period were also invited by the SB clinic staff to participate. Occasionally, eligible children with SB and their parents traveling from Utah’s surrounding states (eg, Idaho, Wyoming, and Nevada) to attend the SB clinic in Salt Lake City were invited to participate. Staff in Utah were not restricted in contact attempts or recruitment by their local IRB.

In Arizona, eligible children with SB were identified through the birth defects monitoring program, hospital discharge databases, SB multispecialty clinics, and the primary children’s hospitals in Tucson and Phoenix to assemble the eligible population, including health and demographic characteristics of the children. The IRB for this site did not permit research staff to contact any patients without expressed consent; therefore, recruitment letters, emails, and Web announcements were sent advertising the study and providing contact information for self-enrollment through the Arizona Spina Bifida Association and from the Children’s Rehabilitation Services program (Medicaid-funded program for children with physical disabilities). Staff at the Phoenix and Tucson multispecialty SB clinics were allowed to make phone calls to parents of active patients informing them about the study and requesting permission for research staff to contact them. Active recruitment was also performed during scheduled visits to clinics at the Tucson location. However, the IRB at the Phoenix location with the largest SB population only permitted making flyers available to potential participants and did not permit any direct recruitment. The primary IRB approval limited the contact with any family to three phone calls at each stage in the recruitment process.

Parents were given two options to participate: (1) an in-person clinic visit with the child and family to complete neuropsychological assessments and parent surveys (ie, Full Protocol), or (2) a phone survey with mailed questionnaires to the parents (ie, Partial Protocol). Parents also had the option of a mailed questionnaire that could be completed and returned instead of either the in-person assessment or the phone interview. In all options, parents were asked consent to release their child’s medical records for medical record abstraction. Medical record abstraction was performed at both sites to obtain detailed data on clinic visits and hospitalizations, surgeries, growth, and comorbid conditions. Medical record data at each site were stripped of personal identifiers and transmitted to the CDC for pooling into a central dataset.

The in-person visits to the clinics were conducted in either English or Spanish and took 2-3 hours to complete; the phone survey lasted approximately 30 minutes. Because of the length of time associated with the in-person clinic visit, study appointments had to be scheduled on a day other than the child’s regular SB clinic visit. Gift cards of US $50 were given to parents who participated in the full protocol as compensation for travel and time, and US $25 gift cards were given to those who participated in the phone survey.

The survey included 120 items and covered topics regarding the child’s medical issues, development and learning, nutrition and physical growth, mobility and functioning, general health, and family demographic information. Although most of the survey items were created in the pilot project completed prior to the current study (with modifications for this specific study), many of the more generic items in the survey have previously been used in large national surveys, such as the Youth Risk Behavior Survey and the National Early Intervention Longitudinal Study. Parents who chose a phone interview were mailed the consent forms and medical records release form as well as five of the six self-administered questionnaires and were asked to complete them at home and send them back in the postage-paid envelope provided. One of the parent surveys (the Pediatric Evaluation Disability Index [PEDI]) was not mailed to the participants because the investigators considered it too difficult for parents to complete on their own. After the consent forms and surveys were completed and returned to the study coordinator, parents were contacted to schedule a time to participate in the phone survey.

At the beginning of each in-person study visit, the consent and parental permission forms and a medical record release form were presented to the parent and any questions or concerns were addressed. After the parent reviewed and signed these documents, the pediatric psychologist escorted the child into a separate room for neuropsychological testing. Parents were typically not in the exam room unless parental attendance was warranted according to the professional judgement of the clinician. The psychologist administered five assessments: the Bracken Basic Concept Scale‒Receptive (BBCS-R) [19], the Differential Abilities Scale 2^nd Edition (DAS-2) [20], the NEuroPSYchological Assessment (NEPSY) II [21], the Peabody Picture Vocabulary Test, 4^th Edition [22], and the Wide Range Assessment of Visual Motor Abilities (WRAVMA) [23]. For Spanish-speaking participants, all documents, including the consent, parental permission, and medical records release forms, parent interview, parent surveys, and three of the five child assessments (DAS-2, BBCS-R, and WRAVMA) were conducted in Spanish. In Utah, a translator was available for appointments with Spanish speakers to assist in the completion of the study documents, the parent interview, and the parent surveys and administering the battery of neuropsychological assessments used in the study. In Arizona, which has a large Hispanic population, all appointments were conducted by bilingual professionals. While the child was completing testing, the study coordinator administered the study survey to the parents, which was the same questionnaire used in the phone survey. Parents were asked to complete six self-administered questionnaires addressing family functioning, child behaviors and personality, self-care, mobility, and functioning, and health and well-being: the Adaptive Behavior Assessment System, 2^nd Edition [24]; the Behavior Assessment System for Children, 2^nd Edition [25]; the Behavior Rating Inventory [26]; the Child Health Questionnaire [27]; McMaster Family Assessment [28]; and PEDI [29]. The battery of assessments and questionnaires is provided in Table 1.

The Arizona IRB did not require consent for medical record abstraction of eligible cases, whereas the Utah IRB did. The medical records of consented children in Utah, and eligible children in Arizona with or without consent, were abstracted at each site. Prior to the start of the project, the two sites and CDC agreed on a number of data elements to collect from the medical records. This included birth and mortality data (if applicable), as well as demographic information (ie, insurance status at birth, maternal and paternal ages at birth, race/ethnicity, education, gravidity, plurality, marital status, and occupation status), newborn hearing evaluation results, SB level of lesion and type, visual acuity measurements, conditions secondary to SB, and growth parameters. Additionally, information was collected regarding clinic visits (ie, clinic type, provider, and visit reason), hospitalizations (ie, admit/discharge dates, hospitalization reason, and discharge diagnosis codes/text), and surgical history (ie, dates, surgery type, reason, and procedural and diagnosis codes). Medical record data from both sites were entered into an Access database created by the Arizona team specifically for the project. An abstraction manual was created to ensure uniformity and consistency among abstractors at both sites.

The aim of this project was to explore methods piloted by CDC to collect health, cognitive, and social development information for children with spina bifida by identifying eligible children in Arizona and Utah, and collecting similar data as in the pilot. In total, 814 children with SB, aged 3-6 years, were identified, of whom 366 (44.9%) were eligible to participate instead of 368 (45.2%). Medical records were abstracted on 188 (51.4%) of eligible participants. There were unexpected methodological challenges that arose due to differences in the sampling plan between sites, primarily due to differences in IRB permissions for study selection and to some participants’ perception of the potential for adverse outcomes as a result of participation.

Differences in the approved sampling design between Utah and Arizona sites presented variation in study methodology. First, the sampling frame in Utah was a more comprehensive list of the eligible families than that of Arizona. Eligible participants in Utah were derived from a population-based state surveillance system, whereas those in Arizona were identified through varying sources that included a birth defects monitoring program, multispecialty clinics, and hospital databases. Therefore, Utah had greater target population representation in their sample and findings from their site may have greater generalizability and relevance to the interested Utah population. Second, Arizona was not allowed to actively recruit from their list of eligible participants. Although a common study methodology was agreed on between sites to evaluate parents and their children with SB, the difference in the processes permitted by the IRBs for the identification, ascertainment, and recruitment of eligible children with SB impacted Arizona’s participation rate. Utah’s population-based surveillance program within the Utah Department of Health was approved by the IRB to identify eligible children with SB and invite the parents directly to participate in this study. The IRB in Arizona permitted access to medical records of children whose parents were not contacted to participate; however, a challenge for the Arizona site was the inability to mail letters or directly recruit parents and their children from the SB specialty clinic. Thus, one lesson learned from this multisite study was that variance in sampling frame can impact study methodology and participant selection. In this research, the difference in permissible methods of contacting eligible participants affected the response rate in Arizona. The external validity of findings from Arizona is limited since the proportion of eligible participants who participated was low.

There are unique challenges to participating in research for individuals who have complex conditions. SB requires multidisciplinary care and services, which can be time-consuming and cumbersome for parents. Parents must devote significant time and resources to finding and utilizing health and educational services needed by their children. These demanding tasks may reduce the opportunities for participation in research, which could be seen as having no immediate or long-term benefit for their child. Yet having a child with a severe impairment—or the time constraint that might result—does not explain the differential recruitment in the current study because this factor is not likely to differ between sites or states. In addition, those who found the in-person assessment to be too time-consuming had the option of participating by phone survey, which required less time commitment, and this option was offered at both sites. Given that the study was designed to be population-based, the different outcomes in recruitment in the two states may be the result of both the contrasting interpretations by the local IRBs of rules created to protect human subjects and the differences in the racial/ethnic composition and geographic distribution of the population between the two states.

Utah’s population is relatively homogeneous, with non-Hispanic whites representing 79% and Hispanics 13% of the population [30]. Geographically, 75.5% of Utah’s population lives within 50 miles north and south of Salt Lake City (known as the Wasatch Front) where the tertiary and subspecialty pediatric clinical services are located [31]. With medical care for children and adolescents centrally located within the state and the majority of residents living along the Wasatch Front, parents may be more likely to participate in studies that are based where this care is provided.

In Arizona, 90% of the population lives in urban areas, primarily in and around Phoenix and Tucson. Arizona has a unique racial/ethnic distribution of 57% non-Hispanic white, 31% Hispanic, and 4% Native American. Most pediatric specialty services can be accessed only at facilities in the Phoenix and Tucson metro areas. Since the population in Arizona lives primarily in the urban areas where the specialty clinics are located, it is more likely that there may be a cultural explanation to the low participation rate in Arizona. The principal investigator in Arizona observed that some Hispanic parents, when approached to participate by study recruiters, were reluctant to test their children because they felt that their children did not need another medical or cognitive label (author SR, personal communication, September 20, 2012). The fear of a stigmatizing label that could arise from poor performance on cognitive testing was likely a deterrent for these families. Additionally, if the principal investigator, a person trusted and known by the patients, explained the benefits and risks of the study, they were more likely to agree to participate. Feelings of discomfort and fear of loss of privacy have been recognized in other studies as reasons for low participation [32]. Some of these issues may apply to the population in this study.

Recruiting participants to engage in research can be demanding and some of the challenges are highlighted here. Difficulties recruiting and retaining individuals with specific conditions or diseases to participate, for example, in clinical trials and behavioral interventions have been discussed elsewhere [32-36]. For a study that is dependent on the successful recruitment of representative samples, considering the challenges of recruitment at participating sites early in the planning stage may have a positive impact. The researchers may be aware of the specific factors and contexts that may challenge their recruitment efforts and address what can be done to counteract these. Considering the potential for variation in project interpretation by local IRBs early in the development phase may also contribute to a representative picture of the populations in the sites that participate.

A total of 101 children aged 3-6 years and their families participated in this project in Arizona and Utah. Parents completed a survey that inquired about their child’s medical status, development and learning, nutrition and physical activity, mobility, general health, and family functioning. Medical records were abstracted for demographics, clinical characteristics, inpatient and outpatient encounters, and surgical history. Children were assessed in the areas of social and cognitive development and visual/motor skills. Additionally, families were assessed in the areas of family functioning, child behavior and personality, self-care, mobility and functioning, and health and well-being. It is expected that findings from these assessments will highlight areas of deficit that may impact the development of the child and their success in school. Knowledge of these deficits and development of plans to address them may support a more developmentally appropriate trajectory for the population of children affected by SB.