This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Type 2 diabetes is a major challenge for Canadian public health authorities, and regular physical activity is a key factor in the management of this disease. Given that fewer than half of people with type 2 diabetes in Canada are sufficiently active to meet the recommendations, effective programs targeting the adoption of regular physical activity (PA) are in demand for this population. Many researchers argue that Web-based, tailored interventions targeting PA are a promising and effective avenue for sedentary populations like Canadians with type 2 diabetes, but few have described the detailed development of this kind of intervention.
This paper aims to describe the systematic development of the Web-based, tailored intervention, Diabète en Forme, promoting regular aerobic PA among adult Canadian francophones with type 2 diabetes. This paper can be used as a reference for health professionals interested in developing similar interventions. We also explored the integration of theoretical components derived from the I-Change Model, Self-Determination Theory, and Motivational Interviewing, which is a potential path for enhancing the effectiveness of tailored interventions on PA adoption and maintenance.
The intervention development was based on the program-planning model for tailored interventions of Kreuter et al. An additional step was added to the model to evaluate the intervention’s usability prior to the implementation phase. An 8-week intervention was developed. The key components of the intervention include a self-monitoring tool for PA behavior, a weekly action planning tool, and eight tailored motivational sessions based on attitude, self-efficacy, intention, type of motivation, PA behavior, and other constructs and techniques. Usability evaluation, a step added to the program-planning model, helped to make several improvements to the intervention prior to the implementation phase.
The intervention development cost was about CDN $59,700 and took approximately 54 full-time weeks. The intervention officially started on September 29, 2014. Out of 2300 potential participants targeted for the tailored intervention, approximately 530 people visited the website, 170 people completed the registration process, and 83 corresponded to the selection criteria and were enrolled in the intervention.
Usability evaluation is an essential step in the development of a Web-based tailored intervention in order to make pre-implementation improvements. The effectiveness and relevance of the theoretical framework used for the intervention will be analyzed following the process and impact evaluation. Implications for future research are discussed.
Type 2 diabetes is a chronic metabolic disease characterized by poor regulation of blood glucose, which can lead to many health complications. It is by far the most common type of diabetes and the most widespread worldwide [
In addition to pharmacological treatment, lifestyle modifications are a major component of programs aiming to reduce cardiovascular risk factors associated with diabetes, which mainly include the promotion of regular physical activity (PA) and the control of energy intake [
The use of information technologies like the Internet is seen as a promising avenue for the development of large scale interventions with behavior change purposes [
As mentioned in previous literature, scientific reporting regarding how health behavior change interventions work is currently lacking when compared to publications examining the effectiveness of these interventions, as incomplete information is provided on their theoretical basis, behavior change techniques employed, and other elements forming the core components of interventions [
Using recent reporting standards in the fields of health behavior change and tailoring [
We will also explore the use of concepts derived from Self-Determination Theory (SDT), a theory focusing on types rather than just amount of motivation [
An adaptation of the program-planning model developed by Kreuter et al [
We felt it necessary to adapt the model to the needs of our Web-based intervention because it appeared more logical for our team to follow the original steps in a different order (see
In addition to changing the order of the steps, we divided Step 9 (Kreuter’s), evaluation of the program, into two different steps: (1) usability evaluation, which comes prior to implementing the intervention, and (2) process, impact, and outcome evaluation, which is subsequent to the implementation of the intervention. Usability evaluation, a new step added to the model, will be described and discussed later in this paper. The adapted model is presented in
Program-planning model for the DEF Web-based tailored intervention.
Steps | Step description |
1. Analyzing the health problem | Learn as much as possible about the population and the health outcome. |
2. Developing a program framework | An outline is created to describe all parts of the Web-based tailored intervention. |
3. Developing tailoring assessment | Assessment questionnaires that will be used to collect information are created. |
4. Creating tailored algorithms (order changed) | Logic statements and decision rules are created. These specify which messages should be given to which participants under which circumstances. |
5. Writing tailored messages | All intervention messages are created based on logic statements and decision rules specified in Step 4. |
6. Designing feedback (order changed) | Decisions are made about how the tailored messages will look and how they will be presented in the intervention. This step includes designing all templates for the DEF website. |
7. Automating the tailoring process and creating the website | The algorithms and messages are translated into a computer program that automatically allows participants to receive appropriate feedback in regard to their individual profile. The tailored intervention website is created. |
8. Usability evaluation (step added) | The intervention is tested with a small group of participants and final improvements are made to the intervention prior to its implementation. |
9. Implementing the program | The intervention is put in use and becomes available to the official and larger group of participants recruited for the DEF intervention. |
10. Evaluating the process, impact, and outcome | The intervention is analyzed and evaluated on these three aspects. |
According to Kreuter et al, three main strategies can serve to gain knowledge and understanding of aerobic PA behavior within our target population: (1) reviewing applicable theories and models, (2) reviewing previous research, and (3) collecting original data. The first strategy aims to identify which model or theory explaining health-related behavior would best suit our main objective: the adoption and maintenance of regular aerobic PA among Canadian francophone adults with type 2 diabetes. The second strategy serves to gain additional knowledge by reviewing the work others have done addressing the same topic or health problem, which can help refine the model to be used to build the intervention and gain extra knowledge on other related topics. Finally, in combination with the first two strategies, collecting original data will help fill the remaining knowledge gap to have an optimal overview of the health problem.
A wide range of theories and models have been used in behavior change interventions. Among them, Social Cognitive Theory [
Although it is well accepted that tailoring is an effective population approach to the promotion of regular PA, it has been mentioned that future research should focus on establishing larger effect sizes and sustained effects on PA behavior [
In order to develop a more autonomous motivation toward PA in an individual, SDT also explains that the social environment surrounding the individual in a PA context must support three basic psychological needs: (1) autonomy, (2) competence, and (3) relatedness (see
Overview of MI-SDT concept application to the DEF intervention: SDT needs and related need-supportive strategies.
Strategies | Integration to the DEF intervention | ||
|
|||
|
|
||
|
|
Let the client make decisions about what and how to change [ |
The weekly action plan tool to which participants have access allows them to select their objectives and activities by themselves. The program explicitly recognizes that the participant is the best person to make decisions about what and how to change. |
|
|
Roll with resistance [ |
The program has specific feedback for participants with low scores on intention, self-efficacy, and attitude. Reflections of content are provided when participants adopt resistant behavior (eg, do not elaborate on their beliefs, choose to do no PA for a certain week). |
|
|
Explore options [ |
The program asks participants open and multiple choice questions during motivational sessions. [ |
|
|
Encourage Change-Talk [ |
Motivational sessions use MI interviewing skills (OARS; [ |
|
|
||
|
|
Provide a menu of effective options for change [ |
The action planning tool proposed preferred physical activities and solutions to common barriers of people with type 2 diabetes. |
|
|
Provide a rationale for information given [ |
The rationale of each motivational session and tool is explained either through a video or short sentence on the first page of each intervention component. |
|
|
Supporting patients’ choices and initiatives [ |
Positive feedback is provided to participants when they make even the smallest commitment toward change. The program does not judge participants who provide answers that are not in keeping with change. |
|
|||
|
|
||
|
|
Present clear and neutral information about behavior and outcomes [ |
Participants can receive information about the risks associated with physical inactivity and the benefits of regular physical activity during the first two motivational sessions. Participants are told explicitly that they are the only experts about what and how to change. |
|
|
Help the client develop appropriate goals [ |
The action plan tool encourages participants to set realistic weekly behavioral goals for themselves. |
|
|
Provide positive feedback [ |
Participants are frequently valorised for their participation in the motivational sessions and efforts toward change. Participants are also valorised for their strengths and values in motivational sessions 3 and 5. |
|
|
Support self-efficacy [ |
The program affirms the strengths of participants and unconditionally recognizes their capacity and ability to change. |
|
|
||
|
|
Help skills building and problem solving [ |
The program gives participants information and tools on how to calculate and self-monitor their PA level. It also helps participant identify effective solutions to their barriers and provides information on how to practice PA safely. |
|
|||
|
|
||
|
|
Express empathy [ |
Messages are written in a way that shows participants that their opinions matter. |
|
|
Explore client’s concern [ |
The first motivational session explores worries participants may have toward their insufficient PA level in an empathic way. A tab is dedicated to participants who want to share their opinion on the website or express concerns about it. |
|
|
Demonstrate understanding of the client’s position [ |
The reflections and summary are used throughout the motivational sessions to try to understand what participants think or feel. |
|
|
Avoid judgment or blame [ |
Messages to participants are written in a neutral or rewarding style. |
|
|
||
|
|
Provide unconditional positive regard [ |
Messages to participants are never judgmental. The program unconditionally recognizes that participants are able to change their PA behavior. |
|
|
Provide a consistently warm interpersonal environment [ |
The introduction videos at the start of each motivational session use an enthusiastic but calm tone. We have tried to make the website and tools inviting and warm for participants. |
In the context of this intervention, four distinct research fields were reviewed to complement the knowledge acquired with the first research strategy: (1) determinants of aerobic PA adoption and maintenance among people with type 2 diabetes, (2) factors contributing to the effectiveness of tailored interventions targeting PA behavior, (3) factors contributing to the effectiveness of Web-based interventions targeting PA behavior, and (4) effective behavior change techniques used in interventions targeting PA behavior.
For each of the fields, our team searched for articles published before December 29, 2013, in PsycINFO, Medline (PubMed), and Embase databases. Search strings used for each field are described in
Research strings used for reviewing articles of all four research fields related to the DEF intervention.
Topic | Search strings |
Determinants of PA adoption and maintenance among people with type 2 diabetes | Determinants: (correlate OR determinant OR mediator OR moderator OR predictor) |
Physical activity: (“physical activity” OR exercise) | |
Type 2 diabetes: diabetes AND (“type 2” OR “type II”) | |
Factors contributing to the effectiveness of tailored interventions targeting PA behaviora | Tailoring: tailor* AND (feedback* OR intervention* OR individualized OR program*) |
Physical activity: (“physical activity” OR exercise) | |
Factors contributing to the effectiveness of Web-based interventions targeting PAa | Internet: (Internet OR web OR “e-health” OR eHealth OR online) AND (intervention* OR individualized OR program OR platform OR service) |
Physical activity: (“physical activity” OR exercise) | |
Effective behavior change techniques used in interventions targeting PA behaviora | Behavior change techniques: (“behaviour change” OR “behavior change”) AND techni* |
Physical activity: (“physical activity” OR exercise) |
aOnly systematic reviews and meta-analysis papers have been selected for those topics.
Nine articles identifying potential determinants of aerobic PA in adults with type 2 diabetes were found in our qualitative review [
At this stage, we believed we had all the necessary information to choose the intervention model. That is, in order to take into account all the knowledge gained from our research, our team decided to develop its own model adapted from MI, SDT, and the I-Change model, on which the DEF tailored intervention would be based (see
Like the I-Change Model original framework, our model assumes there are three phases in the PA behavior change process: (1) to increase a person’s awareness about the importance of making a PA behavior change, (2) to motivate the person in favor of the adoption of regular PA, and (3) to transfer this motivation toward the adoption of regular PA [
New constructs added to the I-Change Model are (1) Type of Motivation (SDT related), (2) Importance ruler (MI related), (3) Confidence ruler (MI related), and (4) MI-SDT concept application. First, Type of Motivation will help to identify which type of motivation (autonomous vs controlled) regulates the participants’ practice of PA, to give evaluative feedback on participants’ type of motivation, and to analyze whether the intervention has succeeded at developing autonomous motivation toward PA in participants. Second, Importance ruler and Confidence ruler, which are constructs closely related to Attitude and Self-efficacy [
Although our model resembles previous attempts that integrated more than one model for the prediction of PA behavior change [
Theoretical framework of the DEF tailored intervention.
Constructs of the DEF intervention model and related evidence-based or theory-based behavior change techniques.
Construct & behavior change techniques | DEF examples | ||
|
|||
|
|
||
|
|
Reduce negative outcomes [ |
The program provides tailored evaluative feedbackaon self-efficacy that reassures participants with low self-efficacy scores. |
|
|
Self-talk [ |
At the end of an action-planning activity, the program encourages participants to persevere no matter what the final outcomes of the week’s plan are. |
|
|
Social support (unspecified)b[ |
An MI-like session is provided after feedback on self-efficacy, allowing participants to select strengths that could help them change. An elaboration exercise follows where participants conduct a more in-depth evaluation of how those strengths could help them change concretely. |
|
|
Vicarious experiences [ |
Video testimonies of 2 regularly active people with type 2 diabetes are provided during the program’s 6th tailored motivational session. |
|
|
Imaginary reward [ |
During the 6th tailored motivational session, participants are invited to explore what their ideal PA practice would be like 1 year after the program. They are also asked what they would gain from this accomplishment and how proud they would feel if they succeed at being regularly active within a year after the program. |
|
|
Social reward [ |
During motivational sessions 4 and 8, participants are valorized by the program if they took part in more PA during the program than when they started. The self-monitoring tool praises participants with evaluative feedbackaevery time they participate in a new PA session. |
|
|
Self-reward [ |
If the participant’s current PA level is higher than their initial PA level during motivational sessions 4 and 8 (when PA level is reassessed), an evaluative feedbackais provided advising participants to be proud of themselves. |
|
|
||
|
|
Social support (unspecified) [ |
Tailored comparative progress feedbackais provided during motivational session 8; it compares the initial confidence ruler score with the current one. The feedback is accompanied by an MI question eliciting change-talk tailored to their current confidence ruler score. |
|
|
||
|
|
Information about health consequences [ |
Participants are informed on the consequences of an insufficient PA level during motivational session 1 and on the benefits of an optimal PA level during motivational session 2. |
|
|
Reduce negative outcomes [ |
The program provides tailored evaluative feedbackabased on the overall attitude score that reassures participants when the score is low, and congratulates them when the score is high. |
|
|
Social support (unspecified) [ |
Two MI-like sessions are provided aiming at changing attitude of participants. Motivational session 1 allows participants to select two health-risks associated with type 2 diabetes that they want to avoid, which they later elaborate on to analyze what the concrete impacts of those risks in their lives would be. Similar activities are repeated during motivational session 2. This time, participants identify two PA benefits that are important to them instead of two health risks. |
|
|
||
|
|
Social support (unspecified) [ |
Tailored comparative progress feedbackais provided during motivational session 4. This feedback compares the initial importance ruler score with the current one and is accompanied by an MI-based question eliciting change-talk tailored to their current importance ruler score. |
|
|
||
|
|
Social support (unspecified) [ |
Tailored comparative progress feedbackais provided during motivational sessions 4 and 8. It compares the initial intention score with the current one and is accompanied by an MI-based question eliciting change-talk. |
|
|
Reduce negative outcomes [ |
The program provides tailored evaluative feedbackaon the intention score; it reassures participants with a low score and congratulates participants with a high one. |
|
|
||
|
|
Feedback on behavior [ |
During motivational session 1, tailored comparative-normative feedbackacomparing the participants’ current PA level with the PA recommendations of the Canadian Diabetes Association is provided to participants. |
|
|
Review behavioral goal(s) [ |
During their action plan activities, participants are encouraged to choose a higher PA level than their current one for their weekly goals; however, we recommend setting a higher goal only if they feel confident enough to reach it. |
|
|
Social comparison [ |
During motivational session 1, participants are informed about the percentage of Canadians with type 2 diabetes meeting PA recommendations versus those not meeting them. |
|
|
Instruction on how to perform the behavior [ |
Participants are advised about what PA intensity to aim for. In addition, a tab is dedicated to safety advice for people with type 2 diabetes practicing PA. |
|
|
Commitment [ |
At the end of the action plan tool, participants can click on a commitment check box that reads “I, (name of participant), commit to executing my action plan for the week”. |
|
|
Self-monitoring of behavior [ |
A specific tool on the website is dedicated to participants who wish to monitor their behavior. |
|
|
||
|
|
Social support (unspecified) [ |
During motivational session 3, the program provides descriptive feedbackaon type of motivation whether participants possess more controlled or autonomous forms of motivation. During the same motivational session, participants are asked to identify their most important values to see how regular PA would help them be more “congruent” with these values in their daily lives. |
|
|||
|
|
||
|
|
Feedback on behavior [ |
See Feedback on behavior for PA behavior. |
|
|
Information about health consequences [ |
See Information about health consequences for Attitude. |
|
|
Prompt/cues [ |
When participants complete an action plan, they receive an email advising them that it is now time to execute it. In their action plan, participants also select the days that they will practice their physical activities. |
|
|
||
|
|
Social support (unspecified) [ |
Social support is provided through our MI-SDT concept application construct throughout the program activities (see |
|
|
Social comparison [ |
See Social comparison for PA behavior. |
|
|
||
|
|
Goal setting (behavior) [ |
Participants are asked to set a behavioral goal (ie, a PA level they want to reach for a specific week) each time they build an action plan. |
|
|
Problem-solving [ |
During their action-planning activities, participants can select which barrier is more likely to prevent them from being active and then select a solution to overcome this barrier accordingly. |
|
|
Action planning [ |
A specific tool on the website serves to build action plan for each week of the program. |
|
|
||
|
|
Content is personalized for each participant [ |
All emails and messages of the motivational sessions are adapted by gender (Contextualizationa) and use the participants’ names (Identificationa). Participants are also explicitly advised that the program will give them feedback based on their answers to the tailoring assessment questionnaire (Raising expectations of customizationa). |
|
|
||
|
|
Tailored messages adapted to participants’ scores of the DEF model constructs [ |
Tailored feedback is provided during each motivational session for constructs included in the tailoring assessment questionnaire. |
|
|
||
|
|
Use of the Internet and emails [ |
Only website pages and emails are used to deliver messages to participants. |
|
|
||
|
|
Credible source [ |
The team responsible for the DEF intervention is composed of exercise and diabetes experts. Each motivational session starts with a video featuring an exercise specialist who explains the purpose of each session. |
aThe terminology used for the types of tailored feedback and other tailored components is based on the Harrington & Noar reporting standards for tailored interventions [
bMotivational Interviewing techniques used as part of the DEF intervention have been coded as
Although many reviews have been published in recent years in the field of tailoring, few aimed to describe factors contributing to the effectiveness of such interventions. Noar et al discovered that tailored interventions targeting health behavior change are more effective when (1) there are multiple contacts with participants, (2) they include theoretical, behavioral, and demographic constructs, (3) they use more than three constructs for tailored content, and (4) they target constructs such as self-efficacy, stages of change, social support, and processes of change [
Regarding Web-based interventions targeting PA, many reviews mentioned factors associated with the effectiveness of these interventions. The contributing factors are (1) intervention targeting only one behavior [
In recent years, many reviewers also focused on effective behavior change techniques targeting physical activity [
Prior to the development of the intervention, a research team member (FB) conducted a study using an extended version of the Theory of Planned Behavior as the theoretical framework to examine the key determinants predicting the practice of regular PA among individuals with type 2 diabetes [
To conclude Step 1, it is also important to note that our literature review included three other papers not mentioned so far, which provided insights into common beliefs related to aerobic PA among people with type 2 diabetes [
Program objectives were based on the 2013 Canadian Diabetes Association guidelines related to PA for adults between 18 and 65 years old [
Inclusion criteria to be eligible to participate in the intervention are having type 2 diabetes, not meeting the Canadian Diabetes Association guidelines related to aerobic PA, being able to understand French, being able to navigate on the Internet, being between 18 and 65 years of age, and not having medical indications limiting the practice of PA.
The program development was limited to CDN $60,000 from a $232,000 project budget and a 3-year study timeline. The tailored content of the website was developed by the research team with a software offering advanced possibilities called TailorBuilder [
Other parts of the intervention were developed with external human resources and expertise. First, along with the MI workshop participation of 2 researchers on the team (MM, FB), we benefited from consulting with an MI expert who provided guidance on how to best implement MI strategies into our fully automated, Web-based intervention. Second, website development and implementation of the tailored content into the website were performed entirely by AlphaZero, an Internet solution company [
Based on the evidence identified in our literature review, program constraints, and the research team’s expertise, a general framework was developed. A detailed graphic overview of the DEF intervention timeline and components is presented in
An 8-week Web-based, tailored intervention was conceived with the information gathered. The intervention contained a self-monitoring tool, an action planning tool, a motivational session for each week of the program, multiple uses of email, and extra tabs providing safety tips, answers to frequently asked questions, and research team technical support.
The tailored motivational sessions constitute the core of the intervention. The order in which each motivational session is positioned was established in such a way as to correspond to the theoretical framework developed in Step 1. First, motivational session 1 aims to raise awareness among participants on how their insufficient PA level can impact their health. This corresponds to the first phase of the I-Change Model: increasing the importance of change in participants. Second, all other sessions aim to motivate participants toward increasing their PA level over the course of the intervention. This corresponds to the second phase of the I-Change Model, which is to develop motivation toward making the desired change among participants. Third, all motivational sessions are built to end by asking participants if they wish to build an action plan for the week, which aims to help them take action and thus increase their PA level. This last step is the final phase of the I-Change Model where motivation is transferred into the adoption of the desired behavior.
Regarding the integration of SDT and MI concepts, the research team attempted to implement the strategies outlined in
Ethics approval was obtained from the Ethics Committee of Research with Humans from the Université du Québec à Trois-Rivières on June 30, 2011.
Written informed consent was obtained from participants who participated in the usability evaluation and online informed consent was obtained from participants in the DEF intervention for publication of this report.
The questionnaire was developed using the constructs selected for the tailored content of the intervention. In addition to these constructs, sociodemographic and anthropometric data (self-reported) were collected including height, weight, age, occupation, origins, and marital status. Items related to each tailored construct are presented in
Description of the variables assessed for the tailored content of the intervention.
Construct | Items, n | Questions and possible answers |
PA behavior | 6 | During a typical 7-day week, how many times do you practice the following activities for at least 10 consecutive minutes? (adapted version of GLTEQ) [ |
Intention | 3 | I intend to be regularly active in the next month [ |
My plans are to practice physical activities regularly in the next month [ |
||
I estimate that my chances of practicing physical activities over the next month are [ |
||
Attitude | 4 | I think that practicing physical activities regularly in the next month would be [ |
Self-efficacy | 3 | I feel capable of practicing physical activities regularly in the next month [ |
For me, practicing physical activities regularly in the next month would be [ |
||
How much control do you feel you have over the fact of practicing physical activities regularly [ |
||
Type of motivation | 23 | I will practice physical activities in the next month because… [ |
Importance ruler | 1 | How important is it for you to practice physical activities regularly [ |
Confidence ruler | 1 | How confident are you that you can practice physical activities regularly [ |
Gender | 1 | Select your gender (n/a): male or female |
First name | 1 | What is your first name? (n/a) (open question) |
This tailoring assessment questionnaire will be incorporated into the website, within the registration process, which participants will have to complete to officially be part of the intervention. This registration process can be completed during the 2-week enrollment period and includes five main steps: (1) creating a website account, (2) completing the selection criteria phase, (3) reading the detailed information about the project, (4) giving informed consent, and (5) completing the tailoring assessment questionnaire.
Moreover, some constructs were reassessed during motivational session 4 and motivational session 8 to provide a dynamic tailoring experience and comparative-progress feedback to participants. The constructs that were reassessed during motivational session 4 are PA behavior, intention, and importance ruler. During motivational session 8, the reassessed constructs were PA behavior, intention, and confidence ruler. There are two major reasons that confidence ruler and importance ruler were used instead of self-efficacy and attitude to provide comparative-progress feedbacks. The first reason is that fewer items are necessary to measure MI-constructs compared to those of the I-Change Model, which eases the task of answering questions for participants and may make the motivational sessions more enjoyable. The other reason is that MI constructs were developed specifically for intervention purposes. In other words, scores obtained on importance ruler and confidence ruler allow the program to provide an appropriate tailored feedback aimed at eliciting change-talk and therefore better foster autonomous motivation toward change in participants.
Tailoring algorithms are the logic statements or decision rules that specify which messages should be given to which participants under which circumstances [
A flowchart can be viewed as a type of diagram that uses symbols to graphically represent the processes, decisions, operations, and other concepts of a program [
With the flowchart representations completed, the next phase in officially creating the algorithms is to transform the representations into real computation formulas. This phase was executed in Step 7, automating the tailoring process, with the TailorBuilder software.
A message booklet was developed for the tailored and non-tailored components of the intervention. The parts of the booklet related to the tailored components were developed based on the flowcharts created in Step 4. For these tailored components, a variable name was assigned to each possible message block to facilitate the later automation of the tailoring process (Step 7). For the non-tailored components, messages were written for the website home page, main menu (when participants are logged in to the website), all emails that participants will receive, the self-monitoring tool, and Safety Tips, FAQs, and Contact tabs. Because of many updates made during the development of the intervention, no official message booklet can be offered as a reference in this paper. However, a sample of the message booklet is provided in
Considerable efforts were also made to write every intervention message in a needs-supportive style of communication to correctly apply the SDT and MI concepts outlined in
The development of the intervention components design was executed in two parts. First, non-tailored components were designed by a member of the research group (MM) with LucidChart. Web pages corresponding to each of these components were drawn for a first time by MM and then discussed and revised with the Web solution company and FB to refine them. All mock-up templates created with the software can be seen in
It is also important to mention that in addition to the discussions held with the Web solution company, efforts were made to respect some of the general design principles described in the program-planning of Kreuter et al [
Examples of the application of the general design principles proposed by Kreuter et al.
Design principles | Example of application |
Preferably have one objective per Web page | All intervention components are separated into different tabs on the website. Each page of the tailored motivational sessions and action-plan tool usually only has one specific goal, maximum two. |
Choose a single feel for the tailored feedback provided | Efforts were made to ensure that the intervention content is written in a needs-supportive and MI style of communication. |
Don’t overfill pages with content and leave empty space | Participants can view the content of a page involving tailored components without having to scroll. Messages are separated in short paragraphs. |
Be consistent with illustration or photographic style | Because our population varies between 18-65 years of age, we chose not to integrate any images in the website in order to be as inclusive as possible. The only images on the website appear in the introduction videos of each tailored motivational session. The age of the people in these images varies extensively. |
Limit the design to a few font styles | All content is written in Arial, with a font size of 16 points or more. |
Use bold effects and color sparingly | Generally speaking, only logo colors are used on the website. Bold effects are used for the title of the Web pages and occasionally to emphasize an important message. |
The Web solution company created most of the non-tailored intervention components using the previously designed mock-up templates of Step 6. These non-tailored components were thus transformed into functional Web pages and include the following: (1) website home page, (2) account creation page, (3) a more info page, (4) a forgotten password page, (5) safety tips tab, (6) FAQs tab, (7) contact tab, and (8) a self-monitoring tool.
All other components were generated into functional Web pages using TailorBuilder and include the following: (1) tailored motivational sessions, (2) action-planning tool, (3) main menu when participants are logged in, (4) follow-up questionnaires, (5) pre-programmed emails, and (6) an enrollment session including selection criteria, detailed information about the project, consent of participants, and the tailoring assessment questionnaire. These components were generated in two major steps. First, messages and questions related to each component were linked with their appropriate coding variable and integrated into their HTML/CSS format. Second, by using the variable codes assigned to the entire set of messages and questions, all the algorithms were translated into their computer format, which is mainly composed of decision rule statements (eg, IF…ELSE…THAN…) implemented with a simplified programming language integrated in TailorBuilder. An overview of all the programming code written with TailorBuilder is presented in
Once the two parties involved in this step finished creating their Web pages, the Web solution company merged all the Web pages into one website, thus creating the complete DEF Web platform ready for Step 8, evaluating usability.
Our usability evaluation not only assessed the users’ ability to perform tasks successfully on the DEF website but also examined the users’ broader interaction with the website comprising their thoughts, feelings, and perceptions [
In order to receive guidance on the best way to evaluate our website, we first recruited a field expert who assisted the team in the development of an evaluation framework, but not for the realization of the evaluation itself. Concretely, by examining the project’s needs and constraints for the project, this expert provided an interview guide, concrete guidelines to follow when running this kind of evaluation, and advice on tools we can use to facilitate the evaluation. Since we also recruited an external interviewer for the evaluation, some guidelines were discussed extensively with this professional to standardize the interview procedures.
Participants for the usability evaluation were recruited from a diabetes education center located in Trois-Rivières. A small sample of the study population (n=11) was recruited. Some characteristics of the participants were (1) 6 females, (2) average age of 53.5 years, (3) all but one not meeting PA recommendations, with a group average of 50 minutes per week of moderate aerobic PA, (4) average years with type 2 diabetes diagnosis was 8.11, (5) all but one using the Internet at least daily, for an average of 1 hour per session, (6) 5 participants with elementary school as their highest education level, 2 participants with high school, and 4 with a university degree or higher, and (7) all participants followed by at least one health professional. Although small, this sample seems representative of the population targeted and the larger population of Canadian adults with type 2 diabetes [
Participants were invited to the facilities of the Université du Québec à Trois-Rivières, where one-on-one interviews were conducted in a research lab by the external interviewer (a clinical nurse) using a think-aloud protocol and a software allowing for sound, video, and computer screen recording. During the interview, participants were asked to execute some predefined tasks on the website, while all of their facial expressions, comments and actions on the website were recorded. While participants executed the tasks, the interviewer used the interview guide to make sure that the participants voiced their thoughts and feelings to be able to measure their comprehension and enjoyment of specific parts of the website. The data collected during the recorded interviews was used for analysis and usability evaluation on various metrics including time on task, number, and type of errors made, bugs, and relevant issues [
It is also important to mention that the evaluation was conducted iteratively. That is, a first evaluation period took place about 5 weeks before the intervention was implemented. During this first period, six interviews were conducted over a 2-day period after which a first set of improvements was made to the intervention. Four weeks prior to the intervention implementation, a second evaluation period took place during which five interviews were again conducted over a 2-day period. After this final evaluation period, a month was granted for final improvements before implementing the intervention. Due to the project’s time restrictions and because we felt quite confident about launching the intervention with the data collected during the two evaluation periods, no other usability testing was conducted. See
Following this step, numerous improvements (ie, more than 40) were made, ranging from minor (eg, changing a word that was hard to understand on a page to an easier one) to major (eg, redesigning a whole page). Here are some of the major improvements made to the website: (1) reduction of the number of pages that must be viewed to complete the enrollment process (ie, 5 pages fewer, from 20 to 15 pages), (2) replacement of the neutral background of videos to backgrounds inspiring PA behavior, (3) redesign and reorganization of the information on the self-monitoring tool page, (4) reduction of the length of the information messages and summary messages by about 50% for almost every tailored motivational session, (5) significant reduction in the number of choices for MI-based multiple choice questions in tailored motivational sessions 3 and 5, (6) redesign of the emails sent to recruit participants, and (7) reduction in the number of times participants have to scroll to almost zero.
Iterations of the usability evaluation of the DEF intervention.
In the Web-based context of the DEF intervention, data from participants are handled automatically through interactions between the website and TailorBuilder to provide appropriate feedback. This step therefore involves only establishing (1) how participants will be recruited and (2) how the data will be collected and exported to data-analysis software.
For the recruitment phase, the DEF tailored intervention was implemented in partnership with Diabète Québec, a recognized organization for Canadians with type 2 diabetes (particularly in the province of Quebec). Consequently, following a joint agreement between both parties, Diabète Québec sent two invitation emails over 2 weeks to nearly 2300 Canadian francophone adults with type 2 diabetes, who were members of its newsletter.
Sociodemographics and data related to each construct of our theoretical model were collected automatically by TailorBuilder. All data collection questionnaires related to the intervention were automated, so all answers given to each question of the questionnaires were automatically saved under their appropriate variable code and assigned to the appropriate participant. Data exportation was executed easily with TailorBuilder, which allows for the creation of Statistical Package for the Social Sciences (SPSS) files that merge all the data collected in a ready-for-analysis format.
According to Kreuter’s program-planning model, this step serves to determine how effective the program was in meeting its objectives and how the program could be modified for greater performance in the future. Kreuter et al propose their own questions to properly evaluate a program. Our team decided to keep only some of these questions and to add others to better suit the context of our intervention. Questions that the research team will seek to answer through this step are (1) through which processes did behavior change occur in participants? (2) how effective was the program in meeting its behavioral objectives? (3) how effective was the program compared to another type of Web-based intervention using a more general approach (ie, non-tailored) combined with peer-support strategies? (4) was the intervention content well received by the participants? and (5) how can the intervention be improved in the future regarding every dimensions of the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework [
In order to address these questions, it is important to note that this intervention is part of a larger project led by one of the authors (FB). This research project will specifically seek to compare the effectiveness of two types of Web-based interventions having the same behavior change purpose: (1) the intervention described in this paper and (2) a Web-based intervention offering similar but non-tailored content combined with peer-support strategies (ie, a Facebook discussion group).
Readers can access all videos of the intervention in
The DEF intervention development cost was about CDN $59,700 and took approximately 54 full-time weeks to develop. Details on the costs and duration of each step are provided in
Cost per step and total cost for the DEF intervention development.
Component | Cost, $CAD | |
|
||
|
PsyMontreal Inc – guidance on MI applications for the DEF intervention | $230 |
|
Three MI workshop participations | $1000 |
|
$0 | |
|
$0 | |
|
|
|
|
LucidChart software license | $100 |
|
$0 | |
|
$0 | |
|
|
|
|
AlphaZero – website development | $4500 |
|
Point Bleu – media productions | $8500 |
|
TailorBuilder software license | $5000 |
|
Vimeo Plus membership | $70 |
|
||
|
Usability evaluation expert guidance | $1000 |
|
Usability evaluation interviews | $1300 |
|
||
|
Financial compensation paid to Diabète Québec | $3000 |
Researcher salary (MM) for the coordination of the intervention development (54 full-time weeks) | $35,000 | |
Total cost for intervention development including Steps 1-9 | $59,700 |
The enrollment period started on September 15, 2014, while the intervention officially started on September 29, 2014. Out of 2300 potential participants targeted for the DEF tailored intervention, approximately 530 people visited the website, 170 people completed the registration process, and 83 corresponded to the selection criteria and were enrolled in the intervention. Four weeks after the intervention started, no dropouts were reported. Similar enrollment results have been obtained for the two other groups of the study (ie, peer-support group and control group). Prior to recruiting participants, a statistical power analysis to determine the required sample size was conducted based on using repeated-measure analysis of variance to detect change in PA. Thus, according to a mixed plan 3 (ie, Web-based tailored intervention group, Web-based peer-support intervention group and control group) x 3 (ie, baseline, 1- and 6-month follow-up), assuming a statistical power of 0.80 and alpha=.05, a total sample size of 204 participants is necessary to detect a conservative effect size of
Through the experience of developing the DEF Web-based tailored intervention, the research team learned several lessons that will help guide future health professionals interested in developing such an intervention.
First, we confirm that the development of a tailored intervention can be quite a complex and time-consuming process, as has been highlighted by previous tailored intervention developers [
Further adding to the complexity of the development, the use of multiple behavior change theories can also require additional efforts, as researchers must be sure to understand all theories and models underlying the intervention and ensure that the integration of these provides a coherent framework for later development steps. We chose to integrate the I-Change Model, SDT, and MI to further explore potential factors that will add to the current effectiveness of tailored interventions targeting PA behavior, as it has been suggested as an area for future research [
Regarding the development method used for the intervention, the program-planning model developed by Kreuter et al was very useful for building an evidence-based tailored intervention. As we mentioned, very small changes were made to the initial framework in order to make it fit with our logic, context, and needs. However, one interesting area for future research could focus on developing new strategies to accelerate the process through which evidence-based tailored interventions are developed, given that this is generally a time-consuming task. For example, in our case, all steps were executed in a sequential manner (eg, once Step 1 was completed, Step 2 was executed), but future developers could examine the feasibility of executing certain steps simultaneously to reduce development time. Or, researchers could also try to determine which systems are faster for automating the tailoring process of interventions, whether it be TailorBuilder [
For the automation of the tailoring process and the creation of the website, TailorBuilder has been a very important tool for Web pages including tailored components. This software allowed us to make these tailored Web pages a reality, even if the research team had minimal computer programming background. However, it is important to note that one member possessed some basic skills with the client-side programming languages CSS, HTML, and JavaScript, which we highly recommend acquiring before developing a Web-based tailored intervention, unless one plans on using specialized human resources for the entire website construction.
As noted earlier, the step added to the program-planning model of Kreuter et al, usability evaluation, was found to be very useful during the development of the intervention, as it allowed for several important website improvements. However, more iterations during the usability evaluation phase could have led to more insights on how to improve the website. Also, we might have gained more insights within the same amount of time by using a slightly different method such as the Rapid Iterative Testing and Evaluation method (RITE) [
Additionally, our literature review covered a wide range of fields and provided a strong corpus of evidence on which to base the intervention. However, it is important to note that other fields of research that are less explored in the tailoring field could have further contributed to our evidence basis, such as fields related to message design (eg, the “look and feel” of messages) and message structure (eg, emotional appeals, narrative vs statistical, gain vs loss framing) [
Finally, to our knowledge, this paper presents one of the most exhaustive development descriptions of a Web-based, tailored intervention published to date. However, although we believe this work can help health professionals adopt a deeper view of what the task of developing such an intervention truly represents, we are also of the opinion that greater efforts could be made in future papers to be even more educational, contributory, and reproducible on many levels (eg, clarity of the programming code presented, completeness of message booklet and demo website, use of open source software to facilitate reproducibility). We encourage future developers to embrace this challenge.
Two main limitations related to the development of the intervention are worth mentioning at this stage of the project. First, the assessment of development costs was not performed according to a standardized method [
Another limitation is that behavior maintenance may currently be minimally addressed by the intervention (ie, as a reference, interventions promoting better maintenance normally last more than 24 weeks [
This paper aimed to fill a gap in the field of Web-based intervention research, which is in need of more in-depth descriptions of the development process of such interventions. It provides a detailed example of a fully automated, Web-based, tailored intervention development for researchers interested in developing similar interventions. Also, as it represents a potential avenue to enhance the effectiveness of tailored interventions, this paper discussed the integration of the I-Change Model, Self-Determination Theory, and Motivational Interviewing as the theoretical framework of the intervention.
Finally, we reemphasize that usability evaluation, a step added to the program-planning model of Kreuter et al, is strongly encouraged prior to the implementation phase in order to better optimize an intervention before disseminating it on a larger scale.
DEF program framework.
DEF algorithms.
DEF message booklet.
DEF design.
DEF TailorBuilder code.
DEF videos.
DEF demo website.
Diabète en Forme
Motivational Interviewing
physical activity
Self-Determination Theory
This study was made possible through a grant awarded to FB by the Fonds de Recherche en Santé du Québec (FRQS-22855). We would also like to thank all the main external contributors to the intervention development including Gilles Tanguay from AlphaZero Inc, Mark Lardinois from Overnite Software Europe, John William Kayser from PsyMontreal Inc, and Sylvie Croteau from R.A. Malatest & Associates Ltd. Finally, we thank Hein de Vries from Maastricht University and two of his graduate students (Daniela Schulz and Michel Walthouwer) for their contribution to the development of the intervention.
MM was primarily in charge of developing the intervention and drafting the manuscript, with relevant contributions by FB and MPG. FB is the original author of the study design, which was complemented with input by MM and MPG. All authors critically evaluated and approved the final version of the paper.
None declared.