This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Males experience a shorter life expectancy and higher rates of chronic diseases compared to their female counterparts. To improve health outcomes among males, interventions specifically developed for males that target their health behaviors are needed. Information technology (IT)-based interventions may be a promising intervention approach in this population group, however, little is known about how to maximize engagement and retention in Web-based programs.
The current study sought to explore attributes hypothesized to influence user engagement among a subsample of participants from the ManUp study, a randomized controlled trial testing the efficacy of an interactive Web-based intervention for promoting physical activity and nutrition among middle-aged males.
Semistructured interviews were conducted and audiotaped with 20 of the ManUp participants. Interview questions were based on a conceptual model of engagement and centered on why participants took part in the study, what they liked and did not like about the intervention they received, and how they think the intervention could be improved. Interview recordings were transcribed and coded into themes.
There were five themes that were identified in the study. These themes were: (1) users’ motives, (2) users’ desired outcomes, (3) users’ positive experiences, (4) users’ negative emotions, and (5) attributes desired by user.
There is little research in the field that has explored user experiences in human-computer interactions and how such experiences may relate to engagement, especially among males. Although not conclusive, the current study provides some insight into what personal attributes of middle-aged males (such as their key motives and goals for participating) and attributes of the intervention materials (such as usability, control, and interactivity) may impact on user engagement in this group. These findings will be helpful for informing the design and implementation of future health behavior interventions for males.
Australian New Zealand Clinical Trials Registry: ACTRN12611000081910; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?ACTRN=12611000081910 (Archived by WebCite at http://www.webcitation.org/6M4lBlvCA).
Despite strong evidence that physical inactivity and poor nutrition are associated with an increased risk of chronic diseases and mortality [
There is a clear gap in the research literature addressing the physical activity and nutritional behaviors of males. Little male-specific research has been undertaken, and best practice approaches have not been identified [
The purpose of the ManUp study [
Despite the comprehensive and systematic approach to intervention development, issues with engagement and retention of participants were experienced. Few participants logged on to the website regularly (median number of log-ins over the 9 month intervention period was 2 per participant) (interquartile range-IQR=6), and engagement in key behavior change components was low overall (median number of self-monitoring entries was 1 per participant) (IQR=20); median number of challenges initiated was 1 per participant (IQR=3). The dropout was high across both intervention groups, but was significantly higher in the Web-based intervention arm (52/96, 54%, had dropped out at the 9 month follow-up) compared to the print-based positive intervention control arm (96/205, 46.8%, had dropped out at the 9 month follow-up). This is in line with findings from other Web-based-delivered health behavior intervention studies [
Prior research examining user experiences in human-computer interactions suggests that engagement consists of four distinct stages: (1) the point of engagement, (2) a period of sustained engagement, (3) disengagement, and (4) (possibly) reengagement [
The aim of the current study was to explore participant engagement in the ManUp intervention using this framework and to generate directions for future research in the field. The behavioral outcomes of the ManUp randomized trial, relating to changes in diet and physical activity behavior, are forthcoming and will be published in a separate manuscript.
A detailed description of the ManUp randomized controlled trials (RCT) protocol and study sample has been published previously [
Both interventions comprised three main components: (1) educational materials that were designed to enhance health literacy by clearly communicating the health benefits associated with physical activity and a healthy diet, show the risks associated with inactivity and an unhealthy diet, and show the amount or type of physical activity and nutrition behaviors required to achieve health benefit; (2) ManUp physical activity, nutrition personal, and group “challenges” constructed to provide participants with specific, measureable, and time-based goals and to encourage self-monitoring behaviors, and; (3) self-monitoring resources, providing participants with the ability to record progress and keep themselves informed of progress towards completing these challenges.
Participants in the print-delivered intervention arm received these components via a hard copy booklet that contained a series of log sheets that could be used to monitor their progress and/or successful completion of any of the ManUp physical activity or nutrition challenges. Participants in the IT-delivered intervention arm were given access to a password protected website containing these components across six separate sections which participants could navigate (ie, My Profile, My Progress, My Mates, My Groups, My Weight, and Information Center). The website contained additional components that reflected the ability of the IT-based intervention to deliver automated feedback on challenge progress. Participants were also able to record and receive feedback on their body weight (kg), Body Mass Index (BMI) (kg/m2), and waist circumference (cm), as well as view summaries of all data recorded, schedule activities, search for “mates,” write on “mates” profile pages, and take part in group challenges. Additionally, participants with Internet access on their mobile phones were given access to a mobile phone application, facilitating self-monitoring behaviors by allowing them to quickly and conveniently start a new ManUp physical activity or healthy eating challenge, record progress, and view progress towards completing challenges.
Screenshot of ManUP intervention website (participant profile).
Before approaching ManUp participants an estimate of the required sample size to reach saturation was conducted [
Semistructured telephone interviews were conducted and audiotaped by research assistants working on the ManUp project. During each interview, written notes were taken in as much detail as possible to aid in the interpretation of the recordings. Each interview ran for approximately 10 minutes. Telephone interviews were chosen over other interview methods due to the geographical dispersion of participants and evidence that this method can provide rich data [
Prior to conducting interviews with participants, training interviews were conducted on a convenience sample (n=5) to provide the interviewers with an opportunity to practice interview skills and refine the interview materials if necessary. Feedback was provided to the interviewers by the lead investigator (MJD) and necessary changes were made to the interview protocol prior to data collection.
Interview questions were designed to explore attributes associated with participant engagement and disengagement in the intervention materials. The development of the open-ended questions was guided by O’Brien and Toms’ [
Hypothesized attributes influencing engagement and related interview questions.
Phase | Hypothesised attributes influencing engagement | Interview questions |
Point of engagement | Aesthetic appeal | What was the reason you participated in the study? |
|
Novel presentation | What did you expect to get out of the program? |
|
Users’ motivations | What did you like about the program? |
|
Users’ desired outcome/goal for interacting with the application |
|
|
Users’ ability |
|
|
Users’ perception that there is sufficient time to use the application |
|
Engagement | Usability of the interface | Did the program meet your expectations? |
|
Feeling of connectedness to the technology (influenced by physical, social and cognitive interactivity) | What did you like about the program? |
|
Feeling of control |
|
|
Positive/negative emotions associated with how well the application features match the users’ motivation/goal, and need for sensory appeal, novelty and challenge |
|
Disengagement | Usability |
What didn’t you like about the program? |
|
Distractions in the user’s environment | What would you suggest to improve the materials? |
|
Positive emotions (user’s needs are satisfied) |
|
|
Negative emotions (frustration, overwhelmed by challenges or information, boredom, loss of motivation or interest). |
|
Data collection was conducted from April 2012 to June 2012. Data extraction was conducted between January and February 2013. Interview recordings were transcribed verbatim by a research assistant and analyzed thematically using a theory-driven code [
Twenty out of the 60 individuals contacted took part in the study. A summary of participant characteristics is presented in
Themes from the theory-driven analysis are summarized here. Representative quotes relating to each theme are provided in
Participant characteristics.
|
Print (N=7) | IT-based (N=13) | |
|
|
|
|
|
Age (years; x, SD) | 43.42 (6.02) | 42.08 (4.25) |
|
University education (N) | 6 | 4 |
|
|
||
|
Professional | 4 | 9 |
|
White collar | 2 | 4 |
|
Blue collar | 1 | 0 |
BMI | 27.69 (2.81) | 31.33 (7.56) | |
|
|
|
|
|
Nutrition literacy scale (x, SD) | 25.85 (1.46) | 25.58 (1.67) |
|
Physical activity literacy scale (x, SD) | 2.57 (1.81) | 3.66 (1.30) |
|
|
|
|
|
Meeting the physical activity guideline (N) | 3 | 7 |
|
Moderate/vigorous physical activity (mins/wk; x, SD) | 42.86 |
123.33 |
|
Serves of vegetables/day (x, SD) | 3 (1.91) | 2 (1.34) |
|
Serves of fruit/day (x, SD) | 2.71 (2.62) | 2.41 (2.27) |
|
Frequency of eating red meat last week (x, SD) | 6 (0.81) | 5.41 (1.62) |
|
|
|
|
|
Median log-ins (1-36 weeks) and interquartile range | N/A | 13.5 (interquartile range=26) |
The most common motivating factors among participants were weight loss and gaining the necessary tools and incentives to self-manage one’s health. In particular, participants expressed a desire to “gauge” their health and fitness levels and to be provided with feedback and advice on how to control their weight and maintain healthy lifestyle habits. For a few participants, participation was driven by external factors, such as pressure to participate from their partner.
Overall, the outcomes people anticipated from engaging in the program largely reflected their motives for participating. Most were expecting to receive guidance and counseling from the project team to help them enhance their diet and participation in physical activity. For some, this guidance and advice were expected to be specific, such as a prescriptive diet plan to follow or support for their particular sport and activities. For others, the type of advice and support expected were described more generally, such as “tips and suggestions” to live a healthier life. Strategies to help participants stay disciplined and to take action were also expected, such as the provision of materials to record diet and physical activity behaviors. A few participants also expressed that they expected to improve their lifestyle behaviors and/or weight status as a result of participating in the study.
Participants from both intervention groups liked that the information they were provided with was easy to read, use, and had an appropriate tone (ie, not derogatory). Participants from both groups liked that the materials could be used as a benchmark and reference tool when thinking about their own health. Participants who received the IT-based intervention liked the ability to record and view a visual summary of their progress.
Some participants who received the printed information found that the booklet was too long and that some of the text was long-winded. Furthermore, there were a few participants who expressed disappointment with the level of interaction and feedback provided and felt they would have done better with the “Web-based stuff.” Among those who received the IT-based intervention, some participants expressed that they would have liked functional aspects of the website to be improved, such as the ability to enter and keep track of different types of activities, the ability to enlarge text, and the progress calendar. Furthermore, a few participants raised sustaining self-monitoring as an issue, especially when personal physical activity routines did not change (ie, self-monitoring via the website was considered less useful) or when in out-of-service areas (ie, when self-monitoring could not be done immediately and conveniently due to a lack of Internet connection). A few participants also expressed disappointment with the intervention content, with some participants reporting that the physical activity and nutrition content was not prescriptive enough, and others reporting that they would have liked to have received more personalized information and feedback about how their changes in health behavior were likely to impact on their health.
Suggestions on how to improve the print-based intervention included providing more tips and helpful hints that are based on the experiences of their peers, and transferring the intervention onto a Web or mobile phone platform to make it more interesting and accessible. Suggestions on how to improve the IT-based intervention were more varied and included both suggestions on how to improve the website usability and for improving intervention content. Specific functional components requested by participants included a facility to report IT-based issues, reminder emails offering direct links to participant profiles (without logging in), the capacity to use the mobile app when there is no Internet connection, and to sync the data with the website at a later date. Suggestions on how to improve intervention content included providing links to nutrition and physical activity information on a separate page of the website, allowing participants to set their own challenge metrics, providing more detailed and iterative feedback, and providing access to other useful tools, such as a calorie conversion calculator. Some of these suggestions, namely a facility to report IT-based issues and links providing further lifestyle information, were actually included on the website.
Representative quotes from participants relating to each theme.
Theme | All participants |
Users’ motivations | I was looking to lose a bit of weight |
|
I was hoping you could give me some sort of insight into how to control my weight |
|
My wife told me I had to |
|
Just to gauge my fitness |
|
To keep track that I’m doing the right thing and a bit more of an incentive |
Users’ desired outcomes | To gain a bit more knowledge on my body and how I can better manage my health |
|
Guidance to make sure that I was doing the right thing as far as exercising a bit more and eating properly |
|
To record what I was doing and then talk to your consultant and actually see ways of improving either fitness or health |
|
A solution to weight loss |
|
Documenting what I was doing/motivation to continue |
Representative quotes from participants relating to each theme.
|
IT-based (Web + mobile) | |
Users’ positive emotions | I didn’t know too much about what I was doing each day so that helped me out | The program made me focus more on my physical activity and diet after actually seeing the data |
|
I went in with an open mind and it was pretty much what I thought it would be | Overall I think it’s great |
|
It was easy to read | I got a little bit out of it. It actually encouraged me to start walking a lot more. In my particular job I'm out of town a lot so there are not a lot of regular exercise programs I can actually sign up for. Whereas, I ended up buying myself a pedometer and I have been walking |
|
Gives you a good benchmark on how to lose weight | It was easy to read |
|
I used print material as a reference tool |
I really liked seeing the visual record of my progress |
|
They were well written and It wasn’t too derogatory or you didn’t feel like you were being taught a kindergarten lesson but by the same token, it was quite readable and achievable for anybody with limited literacy skills | I like the idea of being able to use the calendar to record progress |
|
It was all pretty good relevant information | I’m not high tech minded but I could still use it |
Users’ negative emotions | I was expecting more feedback and interaction | I wanted to be told what the outcome would be if I did specific amounts of activity |
|
I think I would have done better with the Web-based stuff, more motivation that way | I wanted to count cross training exercise but didn’t think that it was really designed for this |
|
I was never going to be able to fill the whole book in and I suppose you’ve written the book with that in mind, so I suppose the expectation might have been a little bit high for me to fill something in every week | I didn’t like that you couldn’t enlarge the calendar |
|
They were a little bit long winded | It wasn’t prescriptive enough |
|
|
It didn’t give me the outcome I was looking for |
|
|
I think I may have let the system down by not following through as much as I should. Initially I was recording my activity weekly but it was a routine that didn’t change much so I sort of fell off on the updating sort of sense |
Attributes desired by users | Tips, helpful hints | I would have liked reminder emails with link to direct entry without log-in |
|
Encouragement, people to give you ideas | Capacity to enter data when no reception is available and then sync when phone has reception |
|
I think one method to address some of my habitual failings, is if it was on either a mobile phone app or an Internet version. Cause I’m more of a technology orientated person than I am paper orientated. I probably would have addressed and achieved more of the challenges purely because, you know, if it’s on the phone or on my computer it’s more ah, it’s more interesting and more accessible for me | Post challenge-report that detailed progress over time period of the entire challenge |
|
|
I would have liked to have been able to measure total physical activity across different types of activity |
|
|
I’d have liked a calorie conversion option on my progress chart |
|
|
I would have liked to have been able to set my own challenge metrics and timeframes |
|
|
I’d like to be able to pause my challenge or reset for when I am sick or away |
Poor user engagement in Web-based health behavior interventions is consistently reported in the literature [
The desirable intervention components reported in this study are similar to those reported in previous (atheoretical) qualitative studies exploring participant’s experiences with Web-based interventions [
The findings of this study provide some insight into how we may improve engagement among males in Web-based behavior change interventions targeting physical activity and nutrition. First, as weight management appears to be a key motivator for many males participating in Web-based programs, participant engagement (in terms of recruitment and content) may be higher if weight loss is also highlighted as an intervention outcome. Findings from the SHED-IT trial showed that how the weight loss intervention is framed is important and suggest that programs that are framed as nonintrusive and flexible are most appealing [
A major strength of this study is that it provides a greater understanding of what males want in terms of Web-based health behavior interventions. Males are a hard-to-reach population [
Compared to their female counterparts, males have higher age-specific rates of chronic disease (such as heart disease, Type 2 diabetes, obesity, and mental health issues) [
body mass index
interquartile range
information technology
randomized controlled trials
self-help, exercise, diet, and information technology
Queensland Health provided funding to conduct the ManUp project. CV is supported by a National Health and Medical Research Council of Australia (#519778) and a National Heart Foundation of Australia (#PH 07B 3303) postdoctoral research fellowship. CES and MJD are supported by fellowships provided by the Institute of Health and Social Science Research, CQUniversity. MWD was funded by a student internship provided by the Institute of Health and Social Science Research, CQUniversity.
None declared.