The Achievement SleepHealth study was implemented using a Web-based platform to screen, consent, and enroll eligible individuals from across the United States to participate in this 1-year long observational study (Achievement Studies, Evidation Health Inc). Participants could access the Web-based study platform at any time from any Web-enabled device to track progress and complete various study-required procedures. While all participants took part in the study remotely, research staff were available by phone or email to answer questions and otherwise provide support at any time during and after the study period.

Prospective participants were recruited over a 5-day period using a Web-based strategy within the Achievement community. Members of Achievement can connect their activity trackers and fitness and health apps to the program; as members log activities and use their activity trackers, they accumulate points that are redeemable for monetary rewards. Members can also receive targeted offers to participate in various research opportunities and studies. Members with relevant connected activity trackers were invited through email to participate in the Achievement SleepHealth study. Participants were prompted to visit a website to learn more about the study and verify their eligibility by answering a few Web-based screening questions. If eligible, they completed an electronic informed consent (eConsent) process to enroll in the study. The eConsent document specified that the only data that would be included in the study dataset from their personal device would be from smartphone apps that they specifically permissioned as part of study procedures (eg, Fitbit data). They then went on to complete baseline questionnaires and were invited to connect a wearable activity tracker account if they had not already done so. Participants who completed the baseline questionnaires were subsequently sent a single-item Sleepiness Checker on a daily basis for 7 consecutive days.

After the initial study procedures described above, participants were asked to track daily alertness levels for 7 consecutive days every 3 months and track sleep and activity with their wearable devices throughout the 1-year study period.

Study questionnaires and the Sleepiness Checker activity from the SleepHealth Mobile App Study were replicated on the Achievement Studies platform. Most questions were multiple-choice type, with a small number of numeric entry questions (eg, participant age, weight, caffeine, and alcohol consumption).

This paper focuses on baseline characteristics, but future work will share results from longitudinal data and trends over the course of the 1-year study. All analyses were performed in SPSS Statistics v23 (IBM, Armonk, NY, USA). Descriptive statistics were performed on the remaining valid data. All data reported as mean (SD; range) unless indicated otherwise. All enrollment time data are in Coordinated Universal Time (UTC).

Participants were first recruited by email around 9:00 am per Pacific Time Zone on March 24, 2017 (16:00 pm as per UTC). The study was closed to further enrollment on March 29, 2017. The first 100 participants were enrolled in less than 20 minutes. Enrollment reached 1000 participants within about 9 hours. Figure 2 provides a detailed bar graph of enrollment by date and time interval; blue bars represent the number of participants recruited at each time interval, and the red line indicates cumulative study enrollment. During the first 1 hour, the enrollment rate was 6.6 participants per minute (394 per hour). In the first 24 hours, the enrollment rate was 0.8 participants per minute (47 participants per hour).

In total, 1156 participants were enrolled in the study within the recruitment window, but 1 withdrew shortly after enrollment, yielding 1155 participants. Complete study flow with the total number of participants at each step is documented in Figure 3. White boxes indicate steps of the enrollment process, with number and percentage of total valid participants; gray boxes indicate the numbers lost between each step.

In addition, the Web-based recruitment strategy attracted participants from a wide geographic distribution. Using telephone area codes as a proxy for location (which does not guarantee that participants live in these locations), at least 1 participant from all 50 states was enrolled in this study. A graphical representation of proxy geographic location is shown in Figure 4, with point size scaled by the number of participants per area code (range 1-12) and points located at the corresponding latitude and longitude centroid for each represented area code. Alaska and Hawaii area codes corresponded to 6 and 9 participants, respectively, but are not depicted. Furthermore, 12.0% (138/1153) of the participants had area codes from the 15 most rural states, defined by having >50% of the population residing in designated US Census Bureau rural settings [28].

Overall, 1132 participants completed the baseline questionnaires (1132/1156, 97.9%). Relevant participant demographic information is summarized in Table 1. Additional participant data are presented in Multimedia Appendix 1.

Mean participant age was 34.6 (9.4; 18-67) years. The mean participant weight was 183.4 (51.8; 94-425) pounds. The majority of the participants were women (1053/1132, 93.0%). Note that while Hispanic individuals were not specifically assessed in this study, participants had the opportunity to select “Other” and enter their race; 18 participants reported that they were Hispanic individuals.

In this study, 1047 participants completed the baseline Sleepiness Checker activity (1047/1156, 90.5%). Mean days of the Sleepiness Checker completed per participant were 5.7 (1.7; 1-7). Overall, 51.8% (542/1047) of those who completed the baseline Sleepiness Checker completed it on all 7 days. A complete breakdown of the number of days that participants completed the Sleepiness Checker activity is outlined in Table 2.

Overall, 1000 participants provided activity-specific Fitbit data (1000/1156, 86.5%), and 982 (982/1156, 84.9%) participants provided sleep-specific Fitbit data. Participants shared data from their Fitbits, and these data were used to provide objective sleep and activity metrics for the study.

This study resulted in several key findings in speeding up research enrollment while maintaining a high rate of engagement with the study protocol. The study enrolled 1156 participants within a 5-day window, with the majority being enrolled within the first day. Of the original 1156, only 1 withdrew from the study within the first week, indicating that the study was able to achieve a very low rate of initial dropout, which is important in conducting Web-based, eConsent studies. Finally, the rate of initial data collection was quite high for this type of research study as well, with 97.9% (1132/1156) of participants completing initial baseline questionnaires, 90.6% (1047/1156) completing the daytime Sleepiness Checker, and 84.9% (982/1156) sharing their wearable data.

Recruitment through the Achievement Studies platform yielded a remarkably high rate of enrollment, which can be attributed to (1) its Web-based methods and (2) its broad potential patient population. Compared with traditional cohort studies of sleep, which in line with previously introduced literature may take many months to enroll fully, Web-based methods of recruitment and enrollment can accelerate this process by orders of magnitude. The emerging Web-based study literature has many examples of this benefit in different patient populations. For instance, Web-based mental health studies with a similar number of participants reached targets in 5 months [29], 3 months [30], and 4 days [31]. A review of 12 studies utilizing Web-based recruitment methods calculated an average of 5 months spent on recruiting, ranging from 7 weeks to 7 months [32]. In comparison, our recruitment approach reached these targets within hours of the first email solicitation. Importantly, this study of sleep health had minimal inclusion and exclusion criteria and barriers to entry, as the study’s research focus is relevant to all individuals regardless of demographics or comorbidities. Such broad relevance, in addition to the convenient and predominantly passive means of data collection, additionally affects the observed recruitment timeframe.

It should be noted that one of the original ResearchKit studies (Health eHeart) reportedly enrolled over 10,000 new participants in its first 24 hours of study launch. However, several unique characteristics to its study launch were as follows: (1) the study team was able to work on the open-source platform for nearly a year before launch; (2) the launch was coordinated with the release of 4 other ResearchKit studies; and (3) it was announced by Apple at a major event.

While there are clear advantages to this method of participant recruitment and enrollment, some authors assert that significant drawbacks are involved in Web-based studies such as low levels of engagement and retention [32,33]. Long-term retention and engagement metrics for this study will be reported in subsequent publications; however, results demonstrate promisingly high levels of daily participant engagement, and other studies run on the Achievement Studies platform have yielded high completion rates [14,15]. Furthermore, Web-based recruitment methods are not subject to significant geographic restrictions and are limited only by access to the internet.

One limitation of this study is that it was conducted only with members of a Web-based community. While similar studies on the Achievement Studies platform can extend recruitment beyond just existing Achievement community members, to supplement with recruitment on other Web-based channels, on the occasion that the Achievement community is not deemed representative of the desired population, conducting research in this manner does require participants to have internet access. Additionally, the data sources required for this study limited participants to those who owned and used wearable devices. Another limitation is the sample’s limited demographic representativeness of the overall population. Participants were predominantly white women (largest subset married with children, educated, income >50k). Screening for the study was not implemented toward specific demographic targets; the inclusion criteria were those aged 18 years and above, residing in the United States, and able to read and understand English. A more diverse sample could have been quickly recruited through the Web by proportionally targeting subgroups using specific criteria, and this approach has been successfully used in other studies on the Achievement Studies platform. Furthermore, it is understood that objective data provided by consumer wearable devices may not accurately agree with clinically acquired measures, such as those obtained using actigraphy and polysomnography, although there is a growing body of recent literature pursuing validation efforts against gold standard, suggesting that consumer wearables may provide useful information about sleep and activity, especially with regards to intrauser trends [34,35]. Clearly, more research is required in the domain of consumer wearables [36-39]. Additional limitations involved in carrying out research in this manner are selection biases, the potential for low retention, and the possibility of privacy and security issues [40,41]. Moreover, participant geographical location was obtained through a proxy measure (telephone area code). This may not always reflect the geographic location where participants currently reside.

The Achievement Studies platform is a useful framework for carrying out longitudinal research from which a large number of participants with wearable and other behavioral data can be rapidly recruited for research studies. In addition, initial participant retention and engagement appear promising. Carrying out community-based research in which individuals can be targeted for studies using specific criteria is an innovative approach to carrying out research. As is the case with large datasets, the data quality will need to be carefully examined. In addition, the representativeness of the sample of the overall population will need to be evaluated and long-term retention and engagement remains to be seen.