Background: Younger adults are difficult to engage in preventive health, yet in Australia they are gaining more weight and increasing in waist circumference faster than middle-to-older adults. A further challenge to engaging 18- to 35-year-olds in interventions is the limited reporting of outcomes of recruitment strategies.
Objective: This paper describes the outcomes of strategies used to recruit young adults to a randomized controlled trial (RCT), healthy lifestyle mHealth program, TXT2BFiT, for prevention of weight gain. The progression from enquiry through eligibility check to randomization into the trial and the costs of recruitment strategies are reported. Factors associated with nonparticipation are explored.
Methods: Participants were recruited either via letters of invitation from general practitioners (GPs) or via electronic or print advertisements, including Facebook and Google—social media and advertising—university electronic newsletters, printed posters, mailbox drops, and newspapers. Participants recruited from GP invitation letters had an appointment booked with their GP for eligibility screening. Those recruited from other methods were sent an information pack to seek approval to participate from their own GP. The total number and source of enquiries were categorized according to eligibility and subsequent completion of steps to enrolment. Cost data and details of recruitment strategies were recorded.
Results: From 1181 enquiries in total from all strategies, 250 (21.17%) participants were randomized. A total of 5311 invitation letters were sent from 12 GP practices—16 participating GPs. A total of 131 patients enquired with 68 participants randomized (68/74 of those eligible, 92%). The other recruitment methods yielded the remaining 182 randomized participants. Enrolment from print media was 26% of enquiries, from electronic media was 20%, and from other methods was 3%. Across all strategies the average cost of recruitment was Australian Dollar (AUD) $139 per person. The least expensive modality was electronic (AUD $37), largely due to a free feature story on one university Web home page, despite Facebook advertising costing AUD $945 per enrolment. The most expensive was print media at AUD $213 and GP letters at AUD $145 per enrolment.
Conclusions: The research indicated that free electronic media was the most cost-effective strategy, with GP letters the least expensive of the paid strategies in comparison to the other strategies. This study is an important contribution for future research into efficacy, translation, and implementation of cost-effective programs for the prevention of weight gain in young adults. Procedural frameworks for recruitment protocols are required, along with systematic reporting of recruitment strategies to reduce unnecessary expenditure and allow for valuable public health prevention programs to go beyond the research setting.
Trial Registration: Australian New Zealand Clinical Trials Registry (ANZCTR): ACTRN12612000924853; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=362872 (Archived by WebCite at http://www.webcitation.org/6YpNfv1gI).
Younger adults in Australia are gaining more weight and increasing their waist circumference faster than older adults . Research targeting this population of 18- to 35-year-olds has been an emerging area with several recent interventions having been completed [ - ] or underway [ - ], with demonstrated effectiveness at preventing weight gain in the short term. However, engaging young adults in public health research, particularly interventions aimed at the prevention of unhealthy weight gain, remains challenging. There is limited information reported on recruitment strategies, timelines, costs, and alterations to original recruitment protocols in response to any challenges encountered, and limited advice on the application of the recruitment techniques to community settings [ ]. Current recruitment evidence is skewed toward older age groups and there is a limited representation of young adults in the literature as they are highly mobile, complicating recruitment efforts [ ].
Recruitment for young adults into interventions is often timed with commencement of life events, such as starting tertiary education, moving out of the parental home, or the postpartum period. Interventions are often conducted within tertiary settings and target tertiary students . This suggests that the type of young people engaging in weight gain prevention interventions may not be representative of the young adult population at large, which may reduce the external validity and translation potential [ ].
Recruitment strategies reported mainly include multiple strategies, and it is not described whether the multitude of strategies is in response to low uptake from the strategy originally planned or if a combination of strategies is more effective. Traditional strategies in combination, such as posters, flyers, advertisements, email blasts, and/or information stands, are the most commonly reported . In efficacy and effectiveness reporting, inadequate detailed information is provided on the recruitment materials, advertising messages, detail of location of recruitment material placement, quantity and time frame, cost of strategies, and/or the original number of people invited or making initial enquiries to participate who do not proceed to eligibility check [ ].
In light of this, current research is recognizing the need for in-depth evaluation of the recruitment process for obesity prevention programs and the implications of this for translation and scalability. Research is emerging on the use of new recruitment avenues, including social media and social media advertising. New studies using Facebook advertising have been shown to be effective in recruiting young adults, particularly young women. The studies are mainly online, lifestyle, weight gain prevention programs and/or evaluations [, ]. They show promise in recruiting a representative sample of the target population [ ] and underrepresented groups [ , ].
Furthermore, little is known about eligible participants who do not engage with prevention research, and the reasons for nonparticipation . There is considerable financial investment in recruiting individuals who do not complete research studies [ ]. Systematic reporting of cost and effectiveness of recruitment strategies will enable researchers to select the most appropriate strategies for recruiting participants into health research studies [ ]. With limited recruitment information currently reported and the large heterogeneity of studies, research interventions are not easily generalizable [ ].
This paper describes the strategies used to recruit young adults to a randomized controlled trial (RCT) of an mHealth program, TXT2BFiT. The progression from enquiry through eligibility check to randomization into the trial and the costs of recruitment strategies are reported. Factors associated with nonparticipation in TXT2BFiT are explored.
Participants and Eligibility Criteria
The eligibility criteria for the RCT of the TXT2BFiT program included being a young adult aged 18 to 35 years . Furthermore, participants had to meet the following conditions: (1) have a body mass index (BMI) of 25.0 to 31.9 kg/m2, or 23.0 to 24.9 kg/m2 with reported weight gain of more than 2 kg over the past 12 months, (2) have a fruit intake of less than two servings per day, a vegetable intake of less than five servings per day, sugar-sweetened beverage intake of at least 1 L per week, energy-dense takeout meals more than once per week, and/or engage in moderate-intensity physical activity of less than 60 minutes per day, (3) own a mobile phone capable of receiving text messages, and (4) have access to the Internet at least once a week. Exclusion criteria included (1) being pregnant or planning to fall pregnant within the next 9 months, (2) enrolled in an alternate weight loss program, (3) had lost more than 10 kg voluntarily in the past 3 months, (4) taking medications that have caused more than 2 kg of weight gain, (5) medical condition that precludes following dietary or physical recommendations, (6) history of disordered eating, and/or (7) does not speak English. The detailed eligibility and study protocol is available elsewhere [ ].
The participant information statement informed participants that both groups would receive free advice on diet and physical activity to help them achieve and maintain a healthy weight, and that they would be compensated for their participation by receiving Australian Dollar (AUD) $10 vouchers for completing surveys and attending an in-person weigh-in (ie, a total of AUD $30 for completion of all measures).
The original protocol was to enroll 354 participants, based on detecting a mean difference of 2.0 kg with P<.05 and 80% power, that assumed the standard deviation was 10 kg and the correlation between baseline and final weight was .8. A total of 284 participants were required—142 per arm—and accounting for a 20% dropout rate, an additional 70 participants would be needed. Two phases of recruitment were employed and are detailed below.
Recruitment Phase 1: General Practitioner Letters
The first phase of recruitment involved personal letter invitations (see) to young adult patients of participating general practitioners (GPs) recruited from Medicare Locals within the Greater Sydney Area. In July 2011, Australian primary health care services were restructured into independent entities called Medicare Locals, which are responsible for coordinating primary health care over a specified geographic area. GPs can only be recruited for study participation through the assistance of Medicare Locals.
Recruitment Phase 1 commenced in November 2012 (see) with three of the seven Medicare Locals in the Greater Sydney Area invited and willing to participate. Our original research and calculations indicated that 3% of patients in a GP practice would be eligible, which meant there would be 60 eligible patients in an average practice. Based on previous research [ , ], if 25% of patients accepted an invitation, it meant that 24 practices would be needed and recruitment was expected to last for 18 months. From these Medicare Locals, 16 GPs from 12 practices—14 (14/352, 4.0%) from Medicare Local A, 2 (2/672, 0.3%) from Medicare Local B, and 0 (0%) from Medicare Local C—agreed to join the study using the latest available GP numbers for each area [ - ]. A total of 5311 letters of invitation were sent to young adult patients. GPs do not routinely collect anthropometric data, including weight and height, and therefore all young adults in the required age range were eligible to receive a letter of invitation, regardless of BMI.
Screening for Eligibility: Phase 1, General Practitioner Letters
The invitation letter directed prospective participants to an online survey to screen for eligibility. Questions in the screening survey were structured such that ineligible participants were redirected to a national social marketing website for healthy eating and physical activity promotion [, ]. Eligible participants reaching the end of the survey were able to nominate dates and times to attend an appointment with the GP—paid for by the study—who had invited them via the letter (see and ). This 10-minute appointment was booked on the participant’s behalf and details sent to the patient by research staff in a confirmatory short message service (SMS) text message. At the appointment, the GP measured the participant’s weight (kg) and height (cm), approved their participation, and collected the participant’s signed, written informed consent to enter into the trial. Signed consent forms were returned to the researchers with the participants' anthropometric data.
Recruitment Phase 2: Other Forms of Recruitment
Phase 2 of recruitment ran from July 2013 (see) until July 2014, involving two main avenues of electronic and print media (see ) because recruitment from Phase 1 slowed. This range of other recruitment strategies and materials using a variety of modalities is described below (see ). Advertising messages were brief, using simple language. Furthermore, advertisements were accompanied by a TXT2BFiT logo, along with healthy and/or unhealthy food images, and/or positive physical activity images (see ).
Electronic Media Recruitment
Electronic media utilized Facebook and Gumtree advertisements, social media through the use of a TXT2BFiT Facebook page, university e-newsletter, university Web home page news story, and a consistent listing on two university research volunteer pages for the duration of recruitment (see). Paid Facebook and Google advertising was used three and four times, respectively. To generate these advertisements, the target population was defined, along with a specific budget and time frame. After the duration of the advertisement listing, advertising data were downloaded and interpreted. Free advertising on the Gumtree website—a network of free online classifieds and community websites—was updated 16 times and included two different low-cost advertisements. The TXT2BFiT Facebook page status was updated weekly for the duration of the recruitment period. Anyone interested "requested to be a friend" of the page. Friends were predominantly those of the research staff and their friends and family. Once saturation was reached (ie, no new research staff members to share the page with their "friends"), this avenue provided no further enquiries. The research study was featured in the e-newsletters of two universities, which were sent to all enrolled undergraduate and postgraduate students and staff, and was the topic of a feature story on the Web home page of one university for approximately one week. For the duration of recruitment, the study was listed on two separate universities' research volunteer Web pages and briefly mentioned in lectures at a third university.
Print Media Recruitment
Print media consisted of advertising posters, brochures, commuter and local newspaper advertisements, and university student magazines. Over the 12-month time period (excluding semester breaks), posters were displayed on two university campuses at popular locations and replaced weekly (see). Posters were also displayed at various community locations and at Technical and Further Education (TAFE) institutions at the beginning of two semesters. Brochures were mostly delivered by research staff and students (n=35,002), but 20,000 brochures were delivered by a distribution company. Suburbs within the Greater Sydney Area with a high percentage of young adults based on census data were targeted. Newspaper and magazine paid advertising was conducted in five local district newspapers and magazines and one newspaper, widely distributed to commuters for free at train stations, on six occasions.
|Modality||Recruitment strategy||Recruitment strategy detail|
|GPa||GP letter||5311 letters sent from 12 participating practices (16 participating GPs)|
Personally addressed letter with GP letterhead, TXT2BFiT and university logo, and instructions provided on how to access screener survey
AUDb $500 GP fee per practice paid for time spent to access to their database for potential participants
Additional AUD $3827.22 for postage and printing
|Paid advertisements over 22 days|
Link with image accompanying website ( ) on the right-hand advertising column of Facebook—targeted to location and age demographics
See reach data in Results, Effectiveness and Cost of Different Recruitment Strategies section
|Paid advertisements over 7 days|
Top-of-page and side-advertisement text
See reach data in Results, Effectiveness and Cost of Different Recruitment Strategies section
|16 free advertisements|
Two paid advertisements
|Social media (Facebook)||TXT2BFiT Facebook page|
Weekly updates for 12 months
Status update examples accompanied by a photo ( )
|Three featured newsletters sent to all students at three universities|
|One banner news story on the home page of one university|
volunteer Web page
|Listed for the duration of the study recruitment (21 months)|
|Advertising poster||Placed at poster locations at two university campuses during semester (14 months), including a brief advertisement at the end of PowerPoint lectures at one university|
Posters placed at various community locations
Posters placed at TAFEc institutes at the beginning of two semesters
|Brochures||54,872 delivered in total|
19,870 company delivered and 35,002 delivered by research staff
Handed out during peak hours, 2:30-7:00 PM, on weekdays for train commuters at CBDd locations
663,000 readers, predominantly 18-39 years, in three major capitals in Australia (only advertised in Sydney)e and 73.5% of readers live outside inner-city Sydney
|Two advertisements in two different local newspapers|
|Two advertisements in two different university magazines|
|Word of mouth||Friend/family||Heard about the study through friends or family|
|Other||Unknown||Participant did not respond to email and/or could not recall where they heard about the study|
aGeneral practitioner (GP).
bAustralian Dollar (AUD).
cTechnical and Further Education (TAFE).
dCentral business district (CBD).
eObtained from the mX website .
Screening for Eligibility: Phase 2, Other Forms of Recruitment
Potential participants registered their interest via email or SMS text message. Researchers screened participants to assess if their age, BMI, and the location of their GP made them eligible. Subjects reported how they heard about the research study (ie, recruitment source). Ineligible participants exited the survey and were redirected to national social marketing sites as detailed above for Phase 1 participants. Eligible participants reaching the end of the survey provided their postal address and received a pack containing a letter explaining the study to the participant, a consent form, a participant information sheet, and a letter explaining the study to the GP with an approval form for the GP to sign as detailed above. However, in this case the participant booked their own appointment, which was paid for by the study (see).
Data Collection Procedures
All participant enquiries were recorded in a database. The online survey website, SurveyMonkey , collected data on demographics, including gender, postcode—for categorizing socioeconomic data [ ]—recruitment modality and strategy, and the eligibility criteria. Detailed data were also collected on the number of GPs in each Medicare Local [ - ]; number of participating GPs; number of GP letters sent and the associated cost; paid advertising costs; number, location, and time frame of brochure deliveries; and time frame of advertising poster distribution and social media updates. All data were recorded in a database.
Descriptive statistics for continuous measures, including counts and percentages for total number of enquiries, total eligible participants, and total participants randomized, are provided for each recruitment modality and strategy. Total costs (AUD $) are reported per recruitment modality and strategy, with the average cost calculated per participant randomized and per eligible participant. Results from each recruitment method are discussed in comparison to each other, as there were not standards or targets defined in the literature for recruitment methodology.
Logistic regression was used to assess any differences in baseline characteristics between eligible participants who were or were not randomized into the study. Characteristics included gender, BMI, postcode—for categorizing socioeconomic data —and recruitment modality and strategy.
Materials and methods of the TXT2BFiT RCT were approved by the University of Sydney Human Research Ethics Committee in September 2012 (Approval Number 15226). The trial is registered with the Australian New Zealand Clinical Trials Registry (ACTRN 12612000924853).
Flow of Participants From Recruitment to Randomization
A total of 1181 people enquired from Phase 1 and 2—24.64% (291/1181) male, 53.18% (628/1181) female, and 22.18% (262/1181) remaining unknown (see). Of the 1181 people, 349 (29.55%) did not enquire further from their initial enquiry and 118 (9.99%) people were ineligible primarily due to their BMI being below 23 kg/m2. The remaining 714 (60.46%) were sent links to the screener surveys as detailed above. Of the 714 remaining participants, 119 (16.7%) were ineligible on completion of the screener survey and a further 198 (27.7%) people did not complete it—167 of the 198 (84.3%) did not even attempt the survey. GP appointments were made for 137 people from phase 1, of whom 13.9% (19/137) did not attend. GP information packs were sent to 260 people from Phase 2, of whom nearly half (113/260, 43.5%) did not see a GP to complete screening. A total of 250 out of 1181 (21.17%) participants were randomized, with over half resulting from recruitment methods other than GP letters (see ).
|Modality||Recruitment strategy||Total enquiries, n (%)||Total eligible,|
n (% per enquiry)
|Total randomized, n (% per eligible)||Total costa,|
|Cost per participant randomizedc, AUD $|
|All modalities||All recruitment methods||1181 (100)||390 (33.02)||250 (64.1)||34,638.96||138.56|
|General practice||General practice letter||131 (11.09)||74 (56.5)||68 (92)||9827.22||144.52|
|All electronic media||335 (28.37)||118 (35.2)||68 (57.6)||2498.06||36.74|
|Facebook advertisement||13 (1.10)||10 (77)||2 (20)||1890.66||945.33|
|Google advertisement||4 (0.34)||3 (75)||1 (33)||571.45||571.45|
|Gumtree advertisement||50 (4.23)||10 (20)||3 (30)||35.95||11.98|
|Social media (Facebook)||7 (0.59)||7 (100)||3 (43)||No cost||N/Ad|
|76 (6.44)||43 (57)||23 (53)||No cost||N/A|
|164 (13.89)||35 (21.3)||28 (80)||No cost||N/A|
|University research volunteer page||21 (1.78)||10 (48)||8 (80)||No cost||N/A|
|All print media||410 (34.72)||180 (43.9)||105 (58.3)||22,313.68||212.51|
|Advertising poster||109 (9.23)||48 (44.0)||29 (60)||No cost||N/A|
|Brochures||135 (11.43)||67 (49.6)||41 (61)||13,631.01||332.46|
|Commuter newspaper advertisements||163 (13.80)||63 (38.7)||34 (54)||6875.00||202.21|
|Local newspaper advertisements||0 (0)||0 (0)||0 (0)||1067.67||No one randomized|
|University students\' magazines||3 (0.25)||2 (67)||1 (50)||740.00||740.00|
|Word of mouth||Friend/family||30 (2.54)||8 (27)||5 (63)||No cost||N/A|
|Other||Unknown||275 (23.29)||10 (3.6)||4 (40)||N/A||N/A|
aResearch staff costs were not included.
bAustralian Dollar (AUD).
cAll randomized participants had a AUD $55 general practitioner visit paid for that was not included in this analysis.
dNot applicable (N/A).
Effectiveness and Cost of Different Recruitment Strategies
shows the number of enquiries and eligible participants, the number of subjects enrolled and randomized, and their respective costs stratified by modality type and the strategy subcategories. Letters sent by GPs resulted in 131 enquiries out of a total of 1181 (11.09%), with 74 eligible out of 131 (56.5%). Electronic media resulted in 335 enquiries out of a total of 1181 (28.37%), with 118 eligible out of 335 (35.2%). Print media achieved 410 enquiries out of a total of 1181 (34.72%), with 180 eligible out of 410 (43.9%). The conversion of eligible enquiries into enrolments and randomization indicates that GPs (68/74 eligible per strategy, 92%) were highest, followed by print media (68/118 eligible per strategy, 57.6%) and electronic media (105/180 eligible, 58.3%). A total of AUD $34,638.96 was spent, which meant an average of AUD $138.56 was spent per participant enrolled and randomized. This was AUD $144.52 for GPs, AUD $36.74 for electronic media, and AUD $221.51 for print media. When the substrategies within the major modalities were examined, it was found that the university newsletters yielded the second-most enquiries with more than half eligible, and just over half of these enrolled and randomized, and there was no direct cost. For print media, the brochures gave the second-most enquiries with most eligibility and highest enrolment, but the cost was high at AUD $332.46 per person.
Electronic media had a wide range of costs per strategy—AUD $0 to AUD $944 per participant randomized. Targeted paid advertising on Facebook reached 953,007 people (seeand ), yet only attracted 13 enquiries and made this strategy the most expensive, costing AUD $945 per participant randomized. Likewise, Google advertising was served to 605,504 people over 7 days (see and ), at a cost of AUD $571 with four enquiries and one participant randomized.
Print media was the most expensive modality per participant randomized, at AUD $213 (see) and was the most time-consuming for research staff (ie, brochure distribution and poster placement). It resulted in the most enquiries (410/1181, 34.72%) and provided the greatest proportion of participants randomized (105/250, 42.0%). All print media strategies had similar enrolment rates from eligible participants, despite the advertising materials varying considerably in the information provided (see ). There were no enquires from paid company-delivered brochures (see ). Brochures delivered individually by the research staff resulted in 136 enquiries. Word-of-mouth (ie, family/friend) recruitment only accounted for 2.0% (5/250) of the total participants randomized.
|Advertisement data||Year and duration|
n, %, or AUDb$
n, %, or AUD $
n, %, or AUD $
n, %, or AUD $
|Unique clicks, n||45||132||34||91||27||105||0|
|Spent, AUD $||51.68||148.32||27.03||72.97||21.89||78.11||1490.70|
|Cost per 1000 people reached, AUD $||0.71||1.24||0.64||0.92||0.45||0.91||N/Ag|
|CPCh, AUD $||1.12||1.04||0.80||0.78||0.78||0.72||1.40|
|Cost per unique click, AUD $||1.15||1.12||0.80||0.80||0.81||0.74||N/A|
|People taking action, n||72,979||119,661||42,310||79,731||48,525||86,086||N/A|
aDownloaded from Facebook Ads Reporting.
bAustralian Dollar (AUD).
cFrequency is the average number of times the advertisement was served to each person.
dClick-through rate (CTR).
eUnique click-through rate (uCTR).
fCost per 1000 impressions (CPM).
gNot applicable (N/A).
hCost per click (CPC).
|Campaign, placement||A group of advertisement sets that share the same objective, where the advertisement was served on Facebook and Google|
|Duration||Length of the advertising|
|Reach||The number of people the advertisement was served to|
|Frequency||The average number of times the advertisement was served to each person|
|Impressions||The number of times the advertising was served. On mobile apps, an advertisement is counted as served the first time it is viewed. On other Facebook interfaces, an advertisement is served the first time it is placed in a person\'s News Feed or each time it is placed in the right-hand column.|
|Clicks||The total number of clicks on the advertisement. Depending on what is being promoted, this can include page likes, event responses, or app installs.|
|Unique clicks||The total number of unique people who have clicked on the advertisement. For example, if 3 people click on the same advertisement 5 times, it will count as 3 unique clicks.|
|CTRb||The number of clicks received divided by the number of impressions|
|uCTRc||The number of people who clicked on the advertisement divided by the number of people you reached. For example, if you received 20 unique clicks and your advertisement was served to 1000 unique people, your unique click-through rate would be 2%.|
|Spent/cost||The total amount spent so far|
|CPMd||The average cost paid to have 1000 impressions on the advertisement|
|Cost per 1000 people reached||The average amount paid to have the advertisement served to 1000 unique people|
|CPCe||The average cost per click for the advertisements, calculated as the amount spent divided by the number of clicks received|
|Cost per unique click||The average cost per unique click for the advertisements, calculated as the amount spent divided by the number of unique clicks received|
|Actions||The number of actions taken on the advertisement—page, app, or event—after the advertisement was served to someone, even if they didn’t click on it. Actions include page likes, app installs, conversions, event responses, and more. For example, 2 page likes and 2 comments would be counted as 4 actions.|
|People taking action||The number of unique people who took action such as liking the page or installing the app as a result of the advertisement. For example, if the same person likes and comments on a post, they will be counted as 1 unique person.|
|Average position||Average position of where the advertisement ranks compared to other ads|
aDefinitions available from Facebook Ads Reporting and Google Ads Reporting.
bClick-through rate (CTR).
cUnique click-through rate (uCTR).
dCost per 1000 impressions (CPM).
eCost per click (CPC).
|Advertisement data||Year 2013|
|Cost, AUD $||581.95|
|Average position, rank||2.3|
aDownloaded from Google Ads Reporting.
bClick-through rate (CTR).
cCost per click (CPC).
dAustralian Dollar (AUD).
Eligible Nonrandomized Participants Versus Randomized Participants
During recruitment, 138 potential participants were identified that were not randomized into the study. Reasons included failure to return their consent form, nonattendance at a GP appointment booked on their behalf, changing their mind, and/or their GP did not recommend the study (not for medical reasons) (see). Logistic regression models demonstrated that females were less likely to go on to randomization compared to males—odds ratio (OR) 0.64 (95% CI 0.41-1.00) (see ). Eligible participants recruited through a GP letter were more likely to be randomized than those recruited through all other recruitment modalities—OR 1.8 (95% CI 1.4-2.4) (see ).
|Characteristic, modality, or strategy||Total eligible not randomized,|
|Male||40 (29.0)||97 (38.8)|
|Female||98 (71.0)||153 (61.2)|
|0-60%c||10 (7.2)||15 (6.0)|
|61-80%||29 (21.0)||45 (18.0)|
|81-100% (highest)||99 (71.7)||189 (75.6)|
|GPd letter||6 (4.3)||68 (27.2)|
|Electronic media||50 (36.2)||69 (27.6)|
|Print media||73 (52.9)||104 (41.6)|
|Other||9 (6.5)||9 (3.6)|
|GP letter||6 (4.3)||68 (27.2)|
|Facebook advertisement||8 (5.8)||2 (0.8)|
|Google advertisement||2 (1.4)||1 (0.4)|
|Gumtree advertisement||7 (5.1)||3 (1.2)|
|Social media (Facebook)||4 (2.9)||3 (1.2)|
|University e-newsletter||20 (14.5)||23 (9.2)|
|University Web home page||7 (5.1)||28 (11.2)|
|University research volunteer page||2 (1.4)||8 (3.2)|
|Advertising poster||18 (13.0)||29 (11.6)|
|Brochures||25 (18.1)||41 (16.4)|
|Commuter newspaper advertisements||29 (21.0)||34 (13.6)|
|Local newspaper advertisements||0 (0)||0 (0)|
|University students\' magazines||1 (0.7)||1 (0.4)|
|Word of mouth||3 (2.2)||5 (2.0)|
|Unknown||6 (4.3)||4 (1.6)|
|23.0-24.9||31 (22.5)||58 (23.2)|
|25.0-29.9||87 (63.0)||156 (62.4)|
|30.0-32.0||20 (14.5)||36 (14.4)|
aAll data obtained from screener survey.
bSocioeconomic status (SES) by population percentile for Socio-Economic Indexes for Areas (SEIFA) Index of Relative Socio-economic Advantage and Disadvantage (IRSAD) (Australian Bureau of Statistics, 2008).
cCombined bottom three quintiles.
dGeneral practioner (GP).
eBody mass index (BMI).
The TXT2BFiT mHealth study, aimed at preventing weight gain in 18- to 35-year-olds, recruited 250 participants over an 18-month time period, with 21% of those expressing interest randomized into the study. The recruitment protocol originally planned to enroll 354 participants from GP letters (Phase 1) but the inability of two Medicare Locals to fully engage potential participants, and lower than expected response from patients, lead to a second recruitment phase using other means. Free or low-cost electronic media appeared to be the most cost-effective and time-efficient strategy to recruit young adults. However, electronic strategies that had a greater reach (ie, Facebook and Google advertising) achieved low numbers of enquiries. GP letters were a more effective recruitment strategy than print media in terms of cost and eligibility from enquiries. Both GPs and paid print media strategies, including brochures and commuter newspaper advertisements, potentially reached a more diverse population. Men were more likely than women to follow through with enrolment into the study.
Recruitment into a face-to-face, group weight gain prevention intervention in the United States for 18- to 35-year-olds, with a BMI between 21 and 30 kg/m2 and similar recruitment time frame (19 months), reported 10% of total enquiries were randomized, costing US $233 per participant randomized, which excluded research staff costs . In Australia, a face-to-face individual weight management study recruited 50 overweight or obese (BMI ≥ 27.5 kg/m2) young women 18 to 25 years over a 2-year period, and cost AUD $308 per person randomized, however, this included research staff costs. If staff costs were removed, this would be reduced to AUD $62 per person randomized [ ]. In this study, one full-time research staff member was employed at a cost of AUD $100,000 per annum, but it is estimated that no more than 30% of the time over the 18 months was spent on recruitment, as they also were involved with intervention delivery. In addition, a research student spent 1 day per week on recruitment. Taking this into account would mean recruitment per participant would be estimated at AUD $319. This mHealth study utilized low-cost recruitment strategies with no budget for mass media. Recruitment for young adults to the previously mentioned weight gain prevention program had limited success with mass media television advertising, costing over US $1000 per person randomized, and having a low percentage of the total randomized [ ]. However, mass media campaigns have been shown to be an effective method of promoting a telephone-based, state-wide lifestyle program, particularly targeting socioeconomically disadvantaged and overweight participants [ ], although no cost data were presented for this program and results included a wide age range of participants. Process evaluation indicated that when developing mass media communications, preference should be given to specifically designed and tailored messaging that explains, models, and displays the relevant contact details for as long as possible to facilitate contact to the program [ ]. Secondary referral recruitment, such as GP referral, was recommended as a supplement to the mass media campaigns.
Using Medicare Locals to invite GPs to join in with participant recruitment was included as a feasible method for recruitment in this study due to the reported success in other prevention interventions, although they focused on older adults with existing metabolic risk factors . Targeted recruitment and high enquiry rate (30.6%) was possible in older age groups as anthropometric and metabolic risk factors were documented in a patient’s medical records [ ]. A fee of AUD $500 per practice for study participants was considered a worthwhile investment by researchers. Young adults had a lower-than-expected enquiry rate to the GP letters (2.5%), with approximately 5.7 participants per practice randomized (range 0 to 18). The study would have required an additional 50 practices—50% greater than anticipated, 62 in total—at the rate observed, costing over AUD $50,000 to randomize the original target of 354 participants. Weight and height are rarely recorded for young adult patients in general practice, and this limited the targeting of invitation letters to patients at risk of weight gain. The transient nature of young adults may suggest that having a long-term GP is less likely, and a proportion of the GP letters may not have been reached by the recipient. For the young adults deemed eligible to participate in Phase 2 of recruitment, the required paid visit to the GP was for ethical reasons, however, this may have been a barrier to enrolment into the RCT.
GPs also showed low interest in engaging young adults into the study, despite being compensated AUD $500 for allowing access to their patient database. Reasons for lack of interest require further research. Primary health care (ie, Medicare Locals) were undergoing extensive restructuring which negatively impacted on cooperation of the Medicare Locals and participation by GPs. Only one of three Medicare Locals recruited the anticipated number of GPs—eight per Local—and one Medicare Local took 12 months to be sufficiently organized to participate and then failed to recruit any GPs. The Medicare Local network has yet again been dissembled with a change of government. This avenue appears to have some degree of instability and may have hindered recruitment efforts. In addition, there is some evidence to suggest GPs believe lifestyle interventions are ineffective . No published RCT aiming to prevent weight gain in young adults utilized the general practice setting to recruit participants. Recruitment strategies are generally poorly reported as we have previously published [ ]. Brief descriptions of recruitment methods were reported in only 62% of studies [ ], which included using existing databases, mailings, posters, flyers, advertisements, email blasts, and information stands. The effectiveness and cost of traditional strategies in this population have only been reported in two interventions from Western countries [ , ] and other populations at risk of weight gain, such as young families [ ]. Formative research into recruitment strategies is providing valuable evidence that may assist in efficient and systematic recruitment processes [ , ].
Among the possible reasons why enquiries were low, focus groups of overweight young adults show health, social image, and self-confidence were reasons identified for pursuing weight loss . However, young adult men, particularly those with reported weight gain and with an overweight BMI (≥ 25.0 kg/m2), reported needing to gain greater than 6 kg before becoming concerned [ ]. This is consistent with Australian data showing that overweight men recognize a growing societal concern with many health-related implications with weight gain, but do not feel this was something that affected them personally at their current life stage [ ]. It is unclear whether identifying poor behavioral choices associated with weight gain, such as inadequate fruit and vegetable consumption, high intake of sugary soft drinks, increased frequency of takeout meals, and low levels of physical activity, were reasons for young adults to engage in prevention interventions. Thus, advertising for the TXT2BFiT study used images focused on weight gain, depicting an overweight man with central adiposity, as well as scales. Formative research into advertising materials suggested young adults would avoid advertising focusing on images of scales [ ], but this was not published at the time TXT2BFiT materials were designed. Now our slogan “gained a few kilos” may be more powerful if the cumulative effect of excess weight gain over time was advertised.
Young adults reported they would be unlikely to click on paid adverting for recruitment to a weight gain prevention program on social networking sites such as Facebook . Facebook paid advertising proved ineffective in the current study, costing approximately seven times the average cost per participant randomized, and had a low enquiry rate despite the high reach. Australian research that has demonstrated recruitment through free advertising on a university Facebook page has been shown to be effective for recruiting young women, 18 to 30 years, to an online weight management survey [ ], and was comparable to the e-newsletter strategy used in this study. However, this may limit the representativeness of the sample as the target audience is restricted to the university population. Facebook paid advertising has been shown to be a cost-effective recruitment strategy—US $20 per compliant participant—for online health surveys in 18- to 25-year-old women [ ]. Moreover, this has been shown to reach a more representative sample of the population, with success in recruiting nonurban and low-income women [ , ]. Facebook paid advertising was compared to social networking and social marketing for parents of adolescent children and was shown to recruit nearly three times as many participants in less time and at less cost—204 participants over 2 months at AUD $5.94 per participant versus 74 participants over 8 months at AUD $58.70 per participant, respectively [ ]. Traditional survey methods for young women such as mailings are becoming costly, with a recent study reporting a three-fold increase—from AUD $30 to just over AUD $100—in the cost to recruit young women aged 18 to 23 years for a national Australian survey [ ].
Recent CONSORT (2010) guidelines recommend clearly displaying the flow of participants throughout a study and that studies report the number of eligible participants prior to randomization, yet they do not insist on the need to report the original overall number of responders invited to participate (prior to eligibility) . Despite identifying recruitment as part of their framework, the CONSORT guidelines do not define the actions needed to identify and recruit potential populations of participants. There is an absence of conceptual frameworks for recruitment to intervention studies and also a lack of procedural models. There is a need to identify what factors are effective in engaging eligible participants to improve the external validity of the research study and to establish recruitment goals based on the target population to engage with population subgroups.
The cost data reported from this research study is limited to cost per strategy and does not include research staff time. Time spent on recruitment was difficult to calculate due to research staff having multiple roles within the research study. This study had one full-time staff member or less employed at any one time. This needs to be accounted for with future cost analysis. For future translation potential, the recruitment process for this study has areas where improvements can be implemented to attract only eligible participants. No formative research was conducted to inform the development of the recruitment materials nor any focus group discussions of the advertisements employed. In future, it is recommended that formative research be conducted prior to scale-up. The recruitment materials can contain a quick response code for mobile phones, which can provide the potential participant with an instant and direct link to the program website explaining eligibility, which will eliminate the need for prior email correspondence. Self-reported measurements have been shown to accurately identify overweight and/or obesity in young people . The requirement for a GP visit resulted in a large dropout of eligible participants prior to attending an appointment. The visit may have been a potential barrier for participants, and the necessity of this step will be investigated further before translation and scale-up in the wider community. Reasons for eligible participants not participating in research studies requires further exploration. Considerable cost is invested in recruiting participants who drop out and researching the reasons for nonparticipation may lead to future cost saving in population obesity prevention programs. Finally, the recruitment strategies resulted in a sample skewed toward a higher SES advantage, but evidence is lacking and this requires further investigation. Targeted recruitment for socially disadvantaged and minority groups needs to be established for future effectiveness research.
This study is an important contribution for future research into efficacy, translation, and implementation of cost-effective programs for the prevention of weight gain in young adults in general, and in using eHealth. The research indicated that free electronic media was the most cost-effective strategy, with GP letters the most effective of the paid strategies. The results provide guidance for future research, as currently there is limited published research available on the cost and effectiveness of recruitment strategies. The large heterogeneity between published studies shows conflicting information on the best strategies to engage young adults. Procedural frameworks for recruitment protocols are required, along with systematic reporting of recruitment strategies to reduce unnecessary expenditure and allow for valuable public health prevention programs to go beyond the research setting.
We wish to thank Ms Sally Jo-Hsin Juan from the School of Molecular Bioscience for her contribution to the recruitment efforts in the study, the partnering general practices, and Medicare Local offices for their time and support in this research.
Funding to conduct this trial was received from the Hospitals Contribution Fund (HCF) Health and Medical Research Foundation (Reference number MAUsyd1008201111), listed on the Australian Competitive Research Grants Register Category 1. This work was also supported by the Commonwealth Government of Australia via an Australian Postgraduate Award Scholarship to SRP, and LH was supported by a National Health and Medical Research Council Scholarship between 2011 and 2013.
SRP made substantial contributions to the preparation and revision of the manuscript, as well as significant contributions to recruitment for the study, data collection, and analysis. KB, AW, LH, and MAF contributed significantly to recruitment for the study. KB, AW, LH, EDW, MH, PP, and AB were involved in the revision of the manuscript for important intellectual content. KM and MAF were involved in data analysis, and drafting and critical revision of the manuscript for important intellectual content. All authors have contributed and approved the final version of the manuscript.
Conflicts of Interest
Multimedia Appendix 1
Examples of the recruitment materials for each modality.PDF File (Adobe PDF File), 1MB
- Tanamas SK, Magliano DJ, Lynch B, Sethi P, Willenberg L, Polkinghorne K, et al. AusDiab 2012: The Australian Diabetes, Obesity and Lifestyle Study. Melbourne, Australia: Baker IDI Heart and Diabetes Institute; 2013. Chapter 3: Obesity URL: https://www.bakeridi.edu.au/Assets/Files/Baker%20IDI%20Ausdiab%20Report_interactive_FINAL.pdf [accessed 2015-05-31] [WebCite Cache]
- Hebden L, Chey T, Allman-Farinelli M. Lifestyle intervention for preventing weight gain in young adults: a systematic review and meta-analysis of RCTs. Obes Rev 2012 Aug;13(8):692-710. [CrossRef] [Medline]
- Poobalan AS, Aucott LS, Precious E, Crombie IK, Smith WC. Weight loss interventions in young people (18 to 25 year olds): a systematic review. Obes Rev 2010 Aug;11(8):580-592. [CrossRef] [Medline]
- Hutchesson MJ, Hulst J, Collins CE. Weight management interventions targeting young women: a systematic review. J Acad Nutr Diet 2013 Jun;113(6):795-802. [CrossRef] [Medline]
- Laska MN, Pelletier JE, Larson NI, Story M. Interventions for weight gain prevention during the transition to young adulthood: a review of the literature. J Adolesc Health 2012 Apr;50(4):324-333 [FREE Full text] [CrossRef] [Medline]
- Batch BC, Tyson C, Bagwell J, Corsino L, Intille S, Lin PH, et al. Weight loss intervention for young adults using mobile technology: design and rationale of a randomized controlled trial - Cell Phone Intervention for You (CITY). Contemp Clin Trials 2014 Mar;37(2):333-341 [FREE Full text] [CrossRef] [Medline]
- Patrick K, Marshall SJ, Davila EP, Kolodziejczyk JK, Fowler JH, Calfas KJ, et al. Design and implementation of a randomized controlled social and mobile weight loss trial for young adults (project SMART). Contemp Clin Trials 2014 Jan;37(1):10-18 [FREE Full text] [CrossRef] [Medline]
- Wing RR, Tate D, Espeland M, Gorin A, LaRose JG, Robichaud EF, et al. Weight gain prevention in young adults: design of the study of novel approaches to weight gain prevention (SNAP) randomized controlled trial. BMC Public Health 2013;13:300 [FREE Full text] [CrossRef] [Medline]
- Kattelmann KK, White AA, Greene GW, Byrd-Bredbenner C, Hoerr SL, Horacek TM, et al. Development of Young Adults Eating and Active for Health (YEAH) internet-based intervention via a community-based participatory research model. J Nutr Educ Behav 2014;46(2):S10-S25. [CrossRef] [Medline]
- Treweek S, Lockhart P, Pitkethly M, Cook JA, Kjeldstrøm M, Johansen M, et al. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open 2013;3(2) [FREE Full text] [CrossRef] [Medline]
- Gokee-LaRose J, Gorin AA, Raynor HA, Laska MN, Jeffery RW, Levy RL, et al. Are standard behavioral weight loss programs effective for young adults? Int J Obes (Lond) 2009 Dec;33(12):1374-1380 [FREE Full text] [CrossRef] [Medline]
- Partridge SR, Juan SJ, McGeechan K, Bauman A, Allman-Farinelli M. Poor quality of external validity reporting limits generalizability of overweight and/or obesity lifestyle prevention interventions in young adults: a systematic review. Obes Rev 2015 Jan;16(1):13-31. [CrossRef] [Medline]
- Leonard A, Hutchesson M, Patterson A, Chalmers K, Collins C. Recruitment and retention of young women into nutrition research studies: practical considerations. Trials 2014;15:23 [FREE Full text] [CrossRef] [Medline]
- Gilligan C, Kypri K, Bourke J. Social networking versus facebook advertising to recruit survey respondents: a quasi-experimental study. JMIR Res Protoc 2014;3(3):e48 [FREE Full text] [CrossRef] [Medline]
- Fenner Y, Garland SM, Moore EE, Jayasinghe Y, Fletcher A, Tabrizi SN, et al. Web-based recruiting for health research using a social networking site: an exploratory study. J Med Internet Res 2012;14(1):e20 [FREE Full text] [CrossRef] [Medline]
- Lohse B. Facebook is an effective strategy to recruit low-income women to online nutrition education. J Nutr Educ Behav 2013;45(1):69-76. [CrossRef] [Medline]
- Lohse B, Wamboldt P. Purposive facebook recruitment endows cost-effective nutrition education program evaluation. JMIR Res Protoc 2013;2(2):e27 [FREE Full text] [CrossRef] [Medline]
- Brintnall-Karabelas J, Sung S, Cadman ME, Squires C, Whorton K, Pao M. Improving recruitment in clinical trials: why eligible participants decline. J Empir Res Hum Res Ethics 2011 Mar;6(1):69-74 [FREE Full text] [CrossRef] [Medline]
- Huynh L, Johns B, Liu SH, Vedula SS, Li T, Puhan MA. Cost-effectiveness of health research study participant recruitment strategies: a systematic review. Clin Trials 2014 Oct;11(5):576-583. [CrossRef] [Medline]
- Mapstone J, Elbourne D, Roberts I. Strategies to improve recruitment to research studies. Cochrane Database Syst Rev 2007(2):MR000013. [CrossRef] [Medline]
- Hebden L, Balestracci K, McGeechan K, Denney-Wilson E, Harris M, Bauman A, et al. 'TXT2BFiT' a mobile phone-based healthy lifestyle program for preventing unhealthy weight gain in young adults: study protocol for a randomized controlled trial. Trials 2013;14:75 [FREE Full text] [CrossRef] [Medline]
- Fanaian M, Laws RA, Passey M, McKenzie S, Wan Q, Davies GP, et al. Health improvement and prevention study (HIPS) - evaluation of an intervention to prevent vascular disease in general practice. BMC Fam Pract 2010;11:57 [FREE Full text] [CrossRef] [Medline]
- Passey ME, Laws RA, Jayasinghe UW, Fanaian M, McKenzie S, Powell-Davies G, et al. Predictors of primary care referrals to a vascular disease prevention lifestyle program among participants in a cluster randomised trial. BMC Health Serv Res 2012;12:234 [FREE Full text] [CrossRef] [Medline]
- Eastern Sydney Medicare Local. 2015. GP service directory URL: http://www.esml.org.au/i-m-a-community-member/directories/gp-service-directory [accessed 2015-01-14] [WebCite Cache]
- Inner West Sydney Medicare Local. 2014. Health service directory - search results URL: https://www.iwsml.org.au/directories-search-form?keywords=&type=provider_general_practitioner&postcode=0&suburb=0&gender=&language=0&ataps=&ataps_specialisation=&option=com_hsd&view=searchresults [WebCite Cache]
- Northern Sydney Medicare Local: Working with General Practice. Thornleigh, Australia: Northern Sydney Medicare Local; 2014 Aug. URL: http://www.nsml.com.au/about-us/general-practice-lores_20140827143636.pdf [accessed 2015-01-14] [WebCite Cache]
- The University of Western Australia and the Centre for Health Promotion and Research Sydney. National Physical Activity Guidelines for Australians. Canberra, Australia: Department of Health and Ageing; 1999. URL: http://fulltext.ausport.gov.au/fulltext/1999/feddep/physguide.pdf [accessed 2014-12-09] [WebCite Cache]
- Australian Dietary Guidelines: Incorporating the Australian Guide to Healthy Eating. Providing the Scientific Evidence for Healthier Australian Diets. Canberra, Australia: National Health and Medical Research Council; 2011 Dec. URL: http://consultations.nhmrc.gov.au/files/consultations/n55draftaustraliandietaryguidelinesconsultation111212.pdf [accessed 2014-08-24] [WebCite Cache]
- mx.net.au. 2015. mX app URL: http://www.mxnet.com.au/ [accessed 2015-01-20] [WebCite Cache]
- SurveyMonkey. 2015. URL: https://www.surveymonkey.com/ [accessed 2015-01-20] [WebCite Cache]
- Pink B, Australian Bureau of Statistics. Socio-Economic Indexes for Areas (SEIFA) 2011. Canberra, Australia: Commonwealth of Australia; 2013. URL: http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/22CEDA8038AF7A0DCA257B3B00116E34/$File/2033.0.55.001%20seifa%202011%20technical%20paper.pdf [accessed 2014-09-02] [WebCite Cache]
- Tate DF, LaRose JG, Griffin LP, Erickson KE, Robichaud EF, Perdue L, et al. Recruitment of young adults into a randomized controlled trial of weight gain prevention: message development, methods, and cost. Trials 2014;15:326 [FREE Full text] [CrossRef] [Medline]
- Griffin HJ, O'Connor HT, Rooney KB, Steinbeck KS. Effectiveness of strategies for recruiting overweight and obese Generation Y women to a clinical weight management trial. Asia Pac J Clin Nutr 2013;22(2):235-240 [FREE Full text] [Medline]
- O'Hara BJ, Phongsavan P, King L, Develin E, Milat AJ, Eggins D, et al. 'Translational formative evaluation': critical in up-scaling public health programmes. Health Promot Int 2014 Mar;29(1):38-46 [FREE Full text] [CrossRef] [Medline]
- O'Hara BJ, Bauman A, King EL, Phongsavan P. Process evaluation of the advertising campaign for the NSW Get Healthy Information and Coaching Service. Health Promot J Austr 2011 Apr;22(1):68-71. [Medline]
- Ampt AJ, Amoroso C, Harris MF, McKenzie SH, Rose VK, Taggart JR. Attitudes, norms and controls influencing lifestyle risk factor management in general practice. BMC Fam Pract 2009;10:59 [FREE Full text] [CrossRef] [Medline]
- Ghai NR, Reynolds KD, Xiang AH, Massie K, Rosetti S, Blanco L, et al. Recruitment results among families contacted for an obesity prevention intervention: the Obesity Prevention Tailored for Health Study. Trials 2014;15:463 [FREE Full text] [CrossRef] [Medline]
- Corsino L, Lin PH, Batch BC, Intille S, Grambow SC, Bosworth HB, et al. Recruiting young adults into a weight loss trial: report of protocol development and recruitment results. Contemp Clin Trials 2013 Jul;35(2):1-7 [FREE Full text] [CrossRef] [Medline]
- Gokee LaRose J, Gorin AA, Clarke MM, Wing RR. Beliefs about weight gain among young adults: potential challenges to prevention. Obesity (Silver Spring) 2011 Sep;19(9):1901-1904 [FREE Full text] [CrossRef] [Medline]
- Mendis K, Forster T, Paxton K, Hyland K, Yelverton J, McLean R, et al. Large and forgotten in rural Australia: assessment, attitudes and possible approaches to losing weight in young adult males. BMC Public Health 2014;14:243 [FREE Full text] [CrossRef] [Medline]
- Harris ML, Herbert D, Loxton D, Dobson A, Wigginton B, Lucke JC. Recruiting young women for health surveys: traditional random sampling methods are not cost-effective. Aust N Z J Public Health 2014 Oct;38(5):495. [CrossRef] [Medline]
- Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Int J Surg 2011;9(8):672-677 [FREE Full text] [CrossRef] [Medline]
- Bowring AL, Peeters A, Freak-Poli R, Lim MS, Gouillou M, Hellard M. Measuring the accuracy of self-reported height and weight in a community-based sample of young people. BMC Med Res Methodol 2012;12:175 [FREE Full text] [CrossRef] [Medline]
- Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
|AUD: Australian Dollar|
|BMI: body mass index|
|CBD: central business district|
|CPC: cost per click|
|CPM: cost per 1000 impressions|
|CTR: click-through rate|
|GP: general practitioner|
|HCF: Hospitals Contribution Fund|
|IRSAD: Index of Relative Socio-economic Advantage and Disadvantage|
|OR: odds ratio|
|RCT: randomized controlled trial|
|SEIFA: Socio-Economic Indexes for Areas|
|SES: socioeconomic status|
|SMS: short message service|
|TAFE: Technical and Further Education|
|uCTR: unique click-through rate|
Edited by G Eysenbach; submitted 21.01.15; peer-reviewed by D Kerr, K Volkova; comments to author 31.03.15; revised version received 10.04.15; accepted 23.04.15; published 05.06.15Copyright
©Stephanie R Partridge, Kate Balestracci, Annette TY Wong, Lana Hebden, Kevin McGeechan, Elizabeth Denney-Wilson, Mark F Harris, Philayrath Phongsavan, Adrian Bauman, Margaret Allman-Farinelli. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 05.06.2015.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.