This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
The distribution of printed materials is the most frequently used strategy to disseminate and implement clinical practice guidelines, although several studies have shown that the effectiveness of this approach is modest at best. Nevertheless, there is insufficient evidence to support the use of other strategies. Recent research has shown that the use of computerized decision support presents a promising approach to address some aspects of this problem.
The aim of this study is to provide qualitative evidence on the potential effect of mobile decision support systems to facilitate the implementation of evidence-based recommendations included in clinical practice guidelines.
We will conduct a qualitative study with two arms to compare the experience of primary care physicians while they try to implement an evidence-based recommendation in their clinical practice. In the first arm, we will provide participants with a printout of the guideline article containing the recommendation, while in the second arm, we will provide participants with a mobile app developed after formalizing the recommendation text into a clinical algorithm. Data will be collected using semistructured and open interviews to explore aspects of behavioral change and technology acceptance involved in the implementation process. The analysis will be comprised of two phases. During the first phase, we will conduct a template analysis to identify barriers and facilitators in each scenario. Then, during the second phase, we will contrast the findings from each arm to propose hypotheses about the potential impact of the system.
We have formalized the narrative in the recommendation into a clinical algorithm and have developed a mobile app. Data collection is expected to occur during 2018, with the first phase of analysis running in parallel. The second phase is scheduled to conclude in July 2019.
Our study will further the understanding of the role of mobile decision support systems in the implementation of clinical practice guidelines. Furthermore, we will provide qualitative evidence to aid decisions made by low- and middle-income countries’ ministries of health about investments in these technologies.
Successful implementation of clinical practice guidelines (CPGs) has the potential to reduce health care costs and improve the quality of care by promoting the use of cost-effective, evidence-based interventions [
Distribution of printed materials has been the predominant CPG dissemination and implementation (D&I) strategy [
Some evidence-based recommendations ask the clinicians to implement clinical algorithms. In these cases, decision support systems (DSSs) provide an approach that facilitates the understanding of the recommendation among the intended users and its integration into their daily routine [
Recent research has shown that it is feasible to use DSSs to support the implementation of clinical algorithms, even in resource-constrained environments [
The first of these challenges is that CPG recommendations are often unclear, ambiguous or incomplete [
The second challenge is that in many cases, DSSs rely on their integration into other clinical information systems, or at least on the availability of personal computers. This dependency on informatics infrastructure represents a barrier to the broad application of these systems as a D&I strategy. Most care centers in low-income countries do not have personal computers, let alone clinical information systems. Even in the case of middle-income countries, rural areas frequently present the same general shortage of computer infrastructure. Furthermore, although it is typical that urban care centers in middle-income countries and other technology-rich environments have electronic health records, these are often homegrown or produced by many different vendors. Consequently, the broad integration of DSSs into these systems would imply enormous investments, requiring the modification of many programs and coordination among many parties.
Recent evidence suggests that mobile technologies could provide an approach to address these barriers, allowing DSSs to be implemented without requiring personal computers or its integration into other clinical information systems [
Our goal is to provide qualitative evidence on the potential effect of mobile DSSs to facilitate the implementation of evidence-based recommendations, as well as the determinants of their adoption. To achieve this aim, we will compare the experience of primary care physicians while they try to implement a recommendation in their clinical practice using either a printout of the journal article containing the recommendation or a mobile DSS. This comparison will consider aspects related to the behavioral change intended and those related to the acceptance of each technology.
As shown in
Arm assignment will be by center. Therefore, all participants from the same center will receive the same intervention. We will include a minimum of eight centers, half of them located in rural areas. We will assign centers to each arm iteratively as they enter the study, ensuring that half of the urban centers, as well as half of the rural centers, will be assigned to each arm.
Data will be collected through qualitative interviews, starting 1 month after the subjects received the article or the mobile app. During this time, the subjects will try to implement the screening recommendation during their daily practice as primary care physicians.
Analysis will focus on identifying barriers and facilitators for the implementation and contrasting these findings between the two arms.
Study design. DSS: decision support system.
We will ask participants to try to implement the case finding recommendation included in the Colombian clinical practice guideline for chronic obstructive pulmonary disease. This recommendation instructs the physician to check a series of risk factors, signs, and symptoms in adults, 40 years or older. When the suspicion is established, the guide recommends ordering a spirometry test to confirm or rule-out the diagnosis [
The selection of this recommendation was arbitrary. The only criteria considered was that it included a clinical algorithm (in this case, the decision about ordering a spirometry test based on a series of risk factors, signs, and symptoms) and that it would be part of a Colombian CPG.
Participants from centers assigned to arm A will be provided with a printout of the journal article in which the guideline was published [
During the first stage, we will conduct a series of meetings with the guideline developers. Throughout these sessions, we will apply the cognitive analysis proposed by Patel et al [
During the second stage, we will implement a mobile app that will support the participants in the implementation of the clinical algorithm. The app will be responsible for asking the user for information related to risk factors, signs, and symptoms, making calculations, and explaining the algorithm’s end points. To improve the chances of acceptance, the app will be able to run on Android, iOS, and Windows Mobile devices, and will operate autonomously, without the need for internet connectivity. The latter requirement will ensure that participating in the study will not generate new or unexpected costs for our subjects.
During the third stage, we will inspect the usability of the app prototype by conducting a cognitive walkthrough [
Our study will include a minimum of eight health centers, and a minimum of three participants per center, for a minimum sample of 24 participants. Half of the centers will be assigned to each arm.
We will conduct the study in Colombia. To explore the influence of the availability of informatics resources, half of the centers will be in Bogotá, while the other half will be in rural areas. The Bogotá sites will operate in a technology-rich environment that includes the use of clinical information systems and access to the internet during patient visits. In contrast, the rural sites will fulfill the criteria of having neither clinical information systems nor access to the internet during patient visits.
Stages of the system development.
Our subjects will be general practitioners operating a regular primary care practice with adult patients.
At each site, the study will start with a short talk with the subjects. During this meeting, the principal investigator will outline the study procedures, and 1 of the guideline developers will present the screening recommendation, including a summary of the literature supporting it. At the end of the meeting, we will provide the participants assigned to arm A with a printout of the journal article in which the practice guideline was published [
To gain access to the sites, we will arrange meetings with their directors. During the meeting, we will present the project, ask for permission to recruit participants among the primary care physicians, ask for the director’s help in the recruitment process, and collect information about previous initiatives to implement CPGs, and the number of potential participants and their schedule.
The general practitioners from each center will be invited to participate by email (if possible, from the director of the respective site). The email will contain a description of the research goals, an outline of the study procedures (eg, the participant’s attempt to implement the screening recommendation), the informed consent document, and contact information. The email will also ask the potential participant to respond, either by email or phone, to express their decision to participate or not. Two weeks after sending the email, we will visit potential participants who have not responded, to invite them personally.
To incentivize participation, potential subjects will be offered with two COP $100,000 prepaid cards from a local supermarket (worth approximately US $35).
Our data collection and analysis will explore factors influencing the behavioral change intended by the recommendation, as well as those that regulate the system’s acceptance among the intended users. To achieve this, we will develop a conceptual framework that will harmonize the Theoretical Domains Framework (TDF) [
To harmonize the contributing frameworks, we will compare the constructs’ descriptions. Based on these comparisons, we will define the harmonized constructs using the following strategies:
We will unify the name of constructs that refer to very similar concepts. Example: both the TDF and the UTAUT consider the extent to which the context provides support and resources to accomplish the behavior or to use the system effectively. However, the TDF calls this construct “environmental context and resources,” while the UTAUT calls it “facilitating conditions.”
We will include constructs that we consider relevant to understand the determinants of behavioral change and technology acceptance, but are not included in all contributing frameworks. Example: a recent literature review from Khong et al [
We will map constructs that we consider too specific to more abstract constructs included in other frameworks. Example: the same review from Khong et al [
The decisions about using strategies 2 and 3 will consider the usefulness of the resulting constructs in developing questions for data collection and their relevance in the context of the project’s scope.
The TDF was proposed by Michie et al [
In 2003, Venkatesh et al [
UTAUT’s constructs are general enough to be applied to any technology. However, to focus our research, we will also include specific concepts that have been identified as influencing the acceptance of mobile and decision support systems.
In 2016, Gagnon et al [
Finally, in 2015, Khong et al [
Data will be collected through qualitative interviews, starting 1 month after the intervention delivery. To leverage the conceptual framework, we will conduct semistructured interviews, which have a loose structure based on a set of open-ended questions that define the area to be explored, but allows divergence to explore new ideas or gather more detail [
Each participant will be interviewed twice, the first interview concerning aspects of behavioral change, while the second, focused on aspects of technology acceptance. Before each interview round, the participants from the respective site will be assigned at random to attend a semistructured or in-depth interview. All interviews will be recorded and transcribed verbatim.
The analysis will be comprised of two phases. During phase one, we will perform a template analysis [
Coding will be conducted independently by 2 researchers. We will use NVivo version 11 (QSR International) to support analysis and data management. Consensus meetings will be held after coding every 4 transcripts. Before each meeting, the intercoder agreement for each transcript will be assessed using the Coding Comparison tool provided by NVivo. During the meeting, the researcher will discuss changes to the coding template and review transcripts with kappa coefficients below .7.
During the second phase, we will assess the potential impact of the system by contrasting the beliefs of the participants from the two arms, across the constructs in the conceptual framework and the emerging categories identified during the first phase. This comparison will allow us to propose hypotheses about the underlying mechanisms that operate over the implementation process, as well as the effects of using each tool. Specifically, those barriers and facilitators that are not present in one scenario or to which the subjects refer to with contrasting emphasis will form the basis of hypotheses about the impact of the DSS in comparison with the printed materials. Barriers and facilitators that seem to have the same emphasis in both scenarios will form the basis of hypotheses concerning determinants not related to the implementation tool.
Additionally, the analysis in both phases will consider the type of center the subject belongs to (ie, urban or rural), the number of years the subject has used smartphones, and whether the subject was provided with a smartphone to participate in the study.
We do not expect that participating in the study will put the subjects at a greater risk of harm or discomfort than what they encounter in their daily working life. Therefore, under the Federal Policy for the Protection of Human Subjects [
The process of informed consent to participate will start before collecting any data and will continue through the entire study. Before being interviewed, the participants will have time to read the consent document and ask questions about the study goals, the procedures in which they will be involved, and the measures undertaken to protect their rights as human subjects. At any time during the study, the participants will be free to withdraw their consent to participate and leave the study, stop the recording, order the destruction of any audio records they have participated in, and terminate or reschedule an interview.
The interviews will be scheduled at the participants’ convenience, with special attention given to not disrupting clinical practice. Audio records and transcripts will be securely stored and will not be shared with persons other than the 2 researchers involved in the analysis. Additionally, no publication will mention the participants’ names or the names of the study sites. Although we will offer an economic incentive for participation, its value is small in comparison to the participants’ monthly income. Therefore, we do not expect the compensation to compel subjects to participate. Finally, this research protocol has been approved by the institutional review boards of the Pontificia Universidad Javeriana in Bogotá, Colombia and the University of Pittsburgh in Pittsburgh, PA, USA.
We designed and implemented the mobile DSS, following the process presented in
Screenshots of the app.
During stage two, we implemented the algorithm in the form of a mobile app. To fulfill the requirement of being cross-platform (ie, being able to run on iOS, Android, and Windows Mobile), while reducing the development effort, we programmed the app using the IONIC 2 platform [
As presented in
The first two sections contain checklists. The first captures the information related to risk factors, while the second concerns signs and symptoms. Some items include second level questions. For instance, when the user indicates that the patient has smoked (in Spanish: “Ha fumado”), the app asks how many cigarettes and for how many years. Finally, the third section presents the algorithm end points depending on the information entered. In the example, due to patient’s age, his smoking history and the presence of dyspnea, the app recommends the user to order a spirometry test.
We are in the process of harmonizing the concepts from the contributing conceptual frameworks [
Dissemination and implementation of CPGs is recognized as a challenging problem [
Our study will provide qualitative evidence about the potential effects of mobile DSSs in the context of a middle-income country. Furthermore, since the study design considers environments with constraints in informatics resources, our results may be used to inform decisions in low-income countries.
Finally, several authors have highlighted the need for theory-based research about the determinants of the effectiveness and adoption of these interventions [
clinical practice guideline
dissemination and implementation
decision support system
mobile health
Theoretical Domains Framework
Unified Theory of Acceptance and Use of Technology
This research was funded in part by the US National Institute on Aging (K01AG044433). We thank Dario Londoño, MD, MS, and Olga Milena Garcia, MD, for their support and insights during the guideline formalization. We also thank Rebecca Crowley Jacobson, MD, MS, for comments and insights that greatly improved this protocol.
None declared.