This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Digital questionnaire delivery offers many advantages to investigators and participants alike; however, evidence supporting digital questionnaire delivery via touchscreen device in the older adult population is lacking.
The objective of this study was to compare the use of tablet computer-delivered and printed questionnaires as vehicles for the collection of psychosocial data from older adults to determine whether this digital platform would be readily adopted by the sample, and to identify whether tablet delivery influences the content of data received.
The participants completed three questionnaires using both delivery methods, followed by a brief evaluation.
A nonparametric one-sample binomial test indicated a significantly greater proportion of individuals preferred the tablet-delivered questionnaires (
This study provides preliminary evidence suggesting that questionnaires delivered to older adults using contemporary tablet computers may be acceptable and do not substantively influence the content of the collected data.
The population of the United States is aging rapidly, and the number of adults 65 years of age or greater has increased more than 15% since the year 2000 [
The assessment of HRQL and health-related behavior is typically done by providing participants with paper and pencil questionnaires. For smaller sample sizes, this method is relatively inexpensive and can allow researchers to examine theoretical constructs that may underlie behavior change brought about by intervention. However, as sample sizes increase, this delivery method can become expensive, time and labor intensive, and the data are exposed at many points to the potential for human error [
Technological advances continue to generate new devices that, when used properly, may increase the efficiency and accuracy of questionnaire data collection. By delivering questionnaires digitally, researchers pay fixed costs initially (eg, development costs, equipment costs), and unlike paper-based questionnaires, these costs increase little with expanding sample sizes [
The advantages of digital questionnaires over paper-based questionnaires are of little value if the intended audience reacts negatively to their use. Indeed, the rapid evolution of technology, particularly with regard to input styles and techniques, can make it difficult for some individuals to adopt new devices [
Results from the few interventions using tablet devices to positively influence health behavior in older adults have been promising [
These studies suggest that tablet computers can be used to effectively deliver content to older adults. However, the apparent feasibility of tablet-based questionnaire delivery within this population, and the potential advantages of doing so, do not provide a sufficient basis for the adoption of the technology in the research context. Similar concerns were expressed when Web-based, PC-delivered questionnaires initially increased in popularity. For example, several researchers examined whether mode effects existed for Web-based versus paper-based delivery of questionnaires [
Because tablet computers are able to offer unique, minimally cluttered interfaces, and because they provide a unique method for interaction, it is important to determine whether these features result in improved or diminished experiences for potential users. It is also important to determine whether these features influence the content of the data collected. The purpose of this pilot study was to compare tablet computer and printed questionnaires as vehicles for the collection of psychosocial data from older adults to determine whether this digital platform would be readily adopted by the sample, and to identify whether mode effects are present. It was hypothesized that a significantly greater proportion of an older adult sample would prefer the tablet-delivered questionnaires, and it was also hypothesized that the collected data would not vary due to mode of delivery.
The present study implemented a proprietary Web-based software package designed for the Apple iPad 2, with an interface that was customized for older adults (eg, large font sizes; high contrast between text, selected answer, and background; minimization of visual clutter) [
The QuIET study software package was developed as a Web app with the use of hypertext markup language (HTML), cascading style sheets (CSS), JavaScript, and Perl programming languages. The package was designed aesthetically to resemble a pad of paper, and all navigation features were removed to simplify the interface. For short answer questions, participants used the iPad’s digital keyboard to enter responses, and for Likert-type questions, participants were instructed to use a finger or stylus to touch their answer, which highlighted in response. Unlike printed questionnaire completion, for which users are able to gauge progress based on the number of pages completed or remaining, digital questionnaire completion offers no such physical indication of progress through the set. To account for this, a progress bar was included at the top of each page to indicate the portion of the total questionnaire battery completed, and a small motivational prompt indicating percent completion was provided between questionnaires. Though basic aesthetic elements were stylized to enhance clarity, the general layout and content of each questionnaire was the same in the print and digital versions.
To further enhance clarity, each questionnaire displayed only appropriate information when presenting conditional items. For example, when asking a question about the frequency with which participants engaged in walking behaviors, possible choices included: (1) Never (Skip to next question), (2) Seldom (1-2 Days), (3) Sometimes (3-4 days), or (4) Often (5-7 days). While the question remained unanswered or if the participant chose “Never”, the questionnaire only displayed the next question (
To improve accuracy, user input was validated upon submission of each questionnaire. This validation was accomplished with JavaScript and Perl. In the case that a question was left unanswered, a prompt was given to the participant that alerted them to the specific question missed. They were given the option to answer the question or to skip it if they were unable to provide an answer. If a question was answered and a required follow-up question was skipped (eg, “How many hours did you do this activity?”), the user was alerted to the missed item and was unable to proceed until it had been answered. All numeric free-response items were also checked for plausibility. For instance, for an item inquiring about number of hours of sleep per night, if a participant entered a number greater than 24, they were prompted to revisit the question and edit their answer before being allowed to continue. After all data were validated, the participant was allowed to proceed to the next questionnaire.
Sample printed questionnaire.
Sample questionnaire without follow-up items.
Sample questionnaire with follow-up items.
Due to the nature of the primary research question (ie, will a significantly greater proportion of a sample of older adults prefer digitally delivered questionnaires), an estimated minimum of 38 participants was needed to detect whether 75% of participants preferring the digital questionnaire was significantly different from 50% preferring each method. This estimate was calculated at a 5% level of significance, 80% power (two-sided test), and with a dropout rate of 25%.
A number of methods were used to recruit community dwelling older adults. Short recruitment talks were given to local older adult philanthropy groups, life-long learning program participants, and senior exercise group participants. Additionally, emails were sent to individuals in existing study databases that agreed to participate in future research. Eligible individuals were English speaking, free from cognitive impairment as assessed via the Modified Telephone Interview for Cognitive Status [
Depending on individual preference, participant screening was conducted on the Internet or by telephone. During the initial screening process, demographics as well as information pertaining to current computer and mobile device use habits were collected. This included the type of mobile devices used (ie, smartphone, tablet, e-reader), number of hours the computer and mobile devices were typically used, and the reason for using these devices. After this screening process, participants registered for a one hour appointment at the research center. Upon recruitment closure, participants were randomly assigned to one of two groups: (1) the iPad-first group that received the tablet-based questionnaires prior to receiving the paper-based questionnaire packet, or (2) the paper-first group that received the printed questionnaires first. Due to the small number of questionnaires provided, participants received and completed their second set of questionnaires upon completion of the first.
At the end of the testing session, an evaluation of the process of questionnaire completion was given to each participant. Using a four point scale, participants were asked to rate ease of use, as well as their perceived level of arousal for each delivery type. They were asked to comment on strengths, weaknesses, and features to change for each medium. Finally, they were asked to indicate which method they preferred. All data were compiled and analyzed using SPSS version 21 for Windows [
A total of 56 individuals responded to recruitment efforts. There were two individuals that were too young to participate, one was unable to attend a session at the study center, and four qualified to participate, but did not attend their scheduled session. The participants who completed the study (N=49; median age 64; interquartile range [IQR]=57-71) were mostly female (36/49, 74%), caucasian (39/49, 80%), and well educated (21/49, 43% with a graduate degree; see
A nonparametric one-sample binomial test indicated a significantly greater proportion of individuals preferred the tablet-delivered questionnaires to the traditional pen and paper method (
The association between preferred delivery method and daily mobile device use approached significance (
With regard to strengths of the tablet delivery, participants most frequently noted improved speed of use (n
Regarding paper delivery, commonly noted strengths related to readability (n=14; eg, “Can see everything at the same time”) and navigation (n=7; eg, “Can skip ahead, look back”). The participants most commonly stated that the paper packet was time consuming or cumbersome (n=8; eg, “Seemed more time consuming and longer”) and was not dynamic (n=6; eg, “Not clear that you could skip questions if you hadn't answered initial question”).
Sample demographics.
Variable | PFa, n | IFb, n | χ2 | Total, N=49 | dfc |
|
|
|
|
|
.004 | 49 | 1 | .947 | |
Male | 7 | 6 |
|
|
|
|
|
Female | 19 | 17 |
|
|
|
|
|
|
|
|
0.3 | 42 | 1 | .607 | |
Caucasian | 19 | 20 |
|
|
|
|
|
African American | 1 | 2 |
|
|
|
|
|
Not reported | 6 | 1 |
|
|
|
|
|
|
|
|
0.9 | 42 | 1 | .335 | |
Hispanic or Latino | 0 | 1 |
|
|
|
|
|
Not Hispanic or Latino | 20 | 21 |
|
|
|
|
|
Not reported | 6 | 1 |
|
|
|
|
|
|
|
|
2.4 | 42 | 1 | .448 | |
<High school graduate | 1 | 0 |
|
|
|
|
|
High school graduate | 0 | 1 |
|
|
|
|
|
College or vocational school degree | 8 | 11 |
|
|
|
|
|
Graduate level degree | 11 | 10 |
|
|
|
|
|
Not reported | 6 | 1 |
|
|
|
|
|
|
|
|
3.9 | 42 | 6 | .701 | |
< US $40,000 per year | 4 | 4 |
|
|
|
|
|
> US $40,000 per year | 9 | 14 |
|
|
|
|
|
Prefer not to answer | 7 | 4 |
|
|
|
|
|
Not reported | 6 | 1 |
|
|
|
|
aPF=Paper-first group
bIF=iPad-first group
cdf=degrees of freedom
Sample device use.
Variable | PFa, n | IFb, n | χ2 | Total, N=49 | dfc |
|
|
|
|
|
0.2 | 49 | 1 | .626 | |
Yes | 24 | 22 |
|
|
|
|
|
No | 2 | 1 |
|
|
|
|
|
|
|
|
3.5 | 49 | 1 | .062 | |
Yes | 10 | 15 |
|
|
|
|
|
No | 16 | 8 |
|
|
|
|
|
|
|
|
0.3 | 49 | 1 | .560 | |
Yes | 6 | 7 |
|
|
|
|
|
No | 20 | 16 |
|
|
|
|
|
|
|
|
0.8 | 49 | 1 | .357 | |
Yes | 8 | 10 |
|
|
|
|
|
No | 18 | 13 |
|
|
|
|
aPF=Paper-first group
bIF=iPad-first group
cdf=degrees of freedom
Scale scores for digital and print questionnaires.
Scale score | Digital, mean (SD) | Print, mean (SD) | N | 95% CIa for mean difference |
|
dfb |
|
PASE total score | 139.43 |
140.50 (68.33) | 49 | -7.15, 5.00 | -0.356 | 48 | .724 |
BARSE total score | 58.95 |
59.18 |
49 | -1.66, 1.19 | -0.332 | 48 | .741 |
aCI=confidence interval
bdf=degrees of freedom
cStudent’s
Scale scores for digital and print questionnaires.
Scale score | Digital, median (IQR) | Print, median (IQR) | N |
|
|
PSQI global score | 6.00(3.00-8.00) | 5.00 |
49 | 1.096 | .273 |
aWilcoxon sign-rank test
This study provides preliminary evidence in support of the use of contemporary tablet computer devices for collecting psychosocial questionnaire data. The results indicated that a significant majority of older adult participants preferred tablet delivery of the questionnaires. Participants did not respond differently to questionnaire items based on the method of delivery, a finding that is in line with previous research [
Importantly, findings from this pilot study indicate that older adults may respond positively to and indeed prefer completing digital questionnaires on tablet devices equipped with software like the package designed for this study. Such dynamic and interactive user environments might benefit participants tasked with completing lengthy questionnaire batteries. For example, hiding unneeded follow-up questions on conditional items resulted in a questionnaire that appeared shorter, a finding reflected in user testimonials. Customized and informative prompts can provide motivation and information between questionnaires [
Researchers also benefit from such computerized methods of data collection. Digital data collection removes the need for research staff to manually enter data, and real time validation ensures that collected data are accurate. Although Web-based data collection conducted via PC can take advantage of some of these same features, the interactive nature of touchscreen devices and the ability to create a simple, clutter-free interface allow researchers to deliver a user experience that is likely more comfortable for many older adults. Finally, because Web apps are cross-platform compatible, and because older adults are increasingly purchasing tablet computers, the ability to deliver questionnaires in a digital format may make it easier for researchers to collect data from broader and more diverse populations.
We believe that this study possesses several strengths. It provides preliminary evidence that with the use of a population-specific interface, older adults may find tablet computer-delivered questionnaires to be acceptable, and perhaps preferable to traditional printed methods. Additionally, this study successfully implemented a tablet-based Web app to collect psychosocial questionnaire data. In the context of questionnaire delivery, we believe that the use of a Web app provides several important advantages to the researcher. First, it can be readily designed to be cross-platform compatible, allowing owners of a variety of tablet computer platforms to access study materials. Further, because Web apps do not require that users install software on their device, researchers need only provide study participants a hyperlink to access questionnaires. This may allow researchers to recruit from a broader geographic area without requiring that participants visit the research center.
This study also provides early evidence to suggest that data collected via tablet computer do not statistically differ from those collected with printed questionnaires. Due to the unique characteristics of the platform, this finding is an important first step in establishing the utility of the device in the research context.
It is important, however, to recognize the limitations of this study. First, the sample was primarily female (36/49, 74%). There are, however, proportionately more women than men in the older adult population, and this gender makeup is similar to that seen in many health-related randomized controlled trials [
Regarding program design, the decision to require individuals to complete any question should not be taken lightly, as individuals may have valid reasons to leave a question unanswered. In the context of the current study, only questions which provided clarification for an initial question (eg, the number of hours spent in an activity) were required in order to avoid such conflicts, while all other items allowed participants to explicitly state their intent to leave the item unanswered. Finally, it is possible that the progress bar and short motivational messages could bias participant responses. Follow-up research may benefit by providing questionnaires with and without these features to examine whether differences are present.
The findings from this pilot study indicate that psychosocial questionnaires, when designed for older adults and delivered via touchscreen enabled tablet computers, may improve efficiency of data collection and may provide more accurate data for the researcher. Importantly, tablet computer-based questionnaire delivery does not appear to influence the content of the data collected. With the aid of additional research, these digitally delivered questionnaires may prove beneficial to the study of HRQL in older adults.
Barriers Self-Efficacy Scale
health-related quality of life
interquartile range
Physical Activity Scale for the Elderly
personal computer
Pittsburgh Sleep Quality Index
questionnaire and inventory evaluation via tablets
None declared.