Background: Hip and knee osteoarthritis is common and leads to pain, stiffness, and disability. Clinical practice guidelines provide recommendations based on the best available evidence to assist health care professionals and patients in clinical decision-making. However, several studies have reported a gap between guideline recommendations and clinical practice in physiotherapy. Improved implementation strategies and the removal of existing barriers may facilitate the transfer of evidence into clinical practice and contribute to optimized quality of care.
Objective: This paper presents the protocol for a study that aims to describe the current physiotherapy practice in patients with hip and knee osteoarthritis and to investigate physiotherapists’ adherence to clinical practice guidelines, to identify and specify barriers to and facilitators of guideline use and implementation, and to develop and pilot test a theory-based tailored implementation intervention aiming to increase guideline use in osteoarthritis care.
Methods: The research project is divided into 4 parts. During the first part, we will conduct a nationwide web-based survey among German physiotherapists to evaluate the current management of hip and knee osteoarthritis and to evaluate whether treatment aligns with guideline recommendations. Subsequently, semistructured interviews will be conducted to specify barriers to and facilitators of guideline use and implementation among physiotherapists (part 2). On the basis of these findings, in part 3, we will develop a theory-driven implementation intervention based on the Theoretical Domains Framework and the Behavior Change Wheel, which will be evaluated in a controlled pilot study in terms of effectiveness, feasibility, and acceptability (part 4).
Results: Data collection of the web-based survey among German physiotherapists (part 1) was completed in December 2021. The semistructured interviews (part 2) were conducted between January and September 2023. Recruitment of physiotherapy practices to participate in the development of the implementation intervention is expected to start in January 2024.
Conclusions: This research project aims to develop a theory-driven implementation intervention to facilitate the transfer of evidence from hip and knee osteoarthritis guidelines in physiotherapy practice. We hypothesize that successful implementation will lead to increased guideline adherence in physiotherapists, which in turn will improve the quality of care. The results from our project will provide valuable knowledge concerning the development process and effectiveness of tailored implementation interventions.
International Registered Report Identifier (IRRID): DERR1-10.2196/47834
Osteoarthritis is the most common chronic joint disease and is mainly present in the lower extremity joints such as hips and knees . In Germany, the prevalence of outpatient diagnosed hip and knee osteoarthritis in older patients is 21.8%, whereas osteoarthritis is more common in women than in men and its prevalence increases with age [ ]. Osteoarthritis is characterized by inflammation and degenerative changes in the joints, causing pain, stiffness, functional disability, and reduced quality of life [ ]. In addition to its effects on individuals, osteoarthritis is associated with an enormous burden on the health care system. The average annual costs per patient living with osteoarthritis in the lower extremities worldwide were estimated to be €11,100 (US $11,722) [ ]. In Germany, the direct costs for osteoarthritis (International Classification of Diseases, 10th Revision: M15-M19) amounted to €7.62 (US $8.05) billion in 2008. Data on indirect costs are not available, but with 114,975 cases of work incapacity owing to knee osteoarthritis and 44,637 cases owing to hip osteoarthritis (2011), an enormous burden on the national economy can be assumed [ ].
There are numerous treatment options available including conservative (eg, physiotherapy) and operative methods to manage hip and knee osteoarthritis, which vary with respect to efficacy and costs. To facilitate the implementation of research findings in clinical practice, clinical practice guidelines (CPGs) summarizing study results have been developed. CPGs are described as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” .
In recent CPGs [- ], conservative nonpharmacological methods, particularly education and therapeutic exercise (and weight loss if overweight or obese), are widely recommended and are seen as first-line treatment for patients with hip and knee osteoarthritis. Other interventions (eg, pain medication or gait aids) can be used to relieve pain and to support first-line treatment. Joint replacement surgery is usually considered when osteoarthritis symptoms become intolerable and significantly affect quality of life, and other nonsurgical therapies have been ineffective or unsuitable [ ].
Physiotherapeutic interventions play a key role in the conservative management of hip and knee osteoarthritis and have been shown to be effective in improving pain, quality of life, and physical function . On the basis of the German health insurance data, Postler et al [ ] identified that approximately 63.4% of patients (aged ≥60 y) diagnosed with hip or knee osteoarthritis receive pain medication (44.1% of which are nonsteroidal anti-inflammatory drugs), whereas physiotherapy is prescribed only to 43.1% of the patients, with decreasing numbers with increasing age.
However, deficits in osteoarthritis management are not only evident in physicians’ prescribing behavior. Recent studies involving physiotherapists from Italy  and Belgium [ ] indicated a gap between clinical practice and research findings, as treatment of hip and knee osteoarthritis often is not in line with current guideline recommendations. Instead of providing first-line treatments, many physiotherapists include interventions that do not contribute to high-quality care. This is consistent with findings from a systematic review in which Zadro et al [ ] found that the median percentage of physiotherapists who chose treatments recommended in CPGs (eg, advice to stay active, aerobic, and strengthening exercises) was 58%, whereas 98% of the physiotherapists provided treatments with low or conflicting evidence in the management of knee osteoarthritis (eg, exercise that is neither aerobic nor strengthening). With regard to the nonpharmacological treatment of people with osteoarthritis, Hagen et al [ ] noted that there is substantial room for improvement in the care of individual patients.
The limited use of evidence in clinical practice is a well-known problem, and the reasons for not using CPGs are manifold. Important barriers to the implementation of CPGs in physiotherapy mentioned in the literature are, for example, lack of time, lack of generalizability of research findings to individual patients, conflicting patient preferences, and lack of support in the workplace [- ]. In addition, physiotherapists are often unaware of current CPGs [ , ]. Owing to the characteristics of the German health care system and the education of most physiotherapists in Germany at vocational schools instead of at institutions of higher education, it can be assumed that there are further barriers specific to the German context.
Publication and dissemination of guidelines alone is not sufficient to change clinical practice. It requires organizational change and behavioral change in individual health care professionals toward their use of guideline recommendations. Therefore, CPGs need to be accompanied by evidence-based implementation strategies . There is evidence that tailored implementation programs targeting specific barriers and facilitators are more likely to improve professional practice than nontailored interventions or passive guideline dissemination [ , ]. It is recommended to use theories and frameworks to guide the implementation process and facilitate the identification of determining factors in health care professionals’ behavior [ ]. Moreover, using theories allows for a better understanding of the generalizability and replicability of implementation interventions [ ].
A promising approach to developing a tailored, theory-driven implementation intervention might be the involvement of the Theoretical Domains Framework (TDF) and the Behavior Change Wheel (BCW). As one of the most commonly used theories in health care settings , the TDF synthesizes different theories of behavior change into one framework and is commonly used to explore factors that influence clinical behavior and guideline adherence. The 14 domains of the TDF include knowledge; skills; social or professional role and identity; beliefs about capabilities; optimism; beliefs about consequences; reinforcement; intentions; goals; memory, attention, and decision processes; environmental context and resources; social influences; emotion; and behavioral regulation [ ]. The BCW is a synthesis of 19 theoretical frameworks of behavior change and provides a systematic approach to developing a tailored guideline implementation intervention. The barriers and facilitators identified through the TDF can be used to expand the Capability, Opportunity, Motivation–Behavior components (COM-B), the core of the BCW, which indicates that behavior change occurs when there is a change in an individual’s capability, opportunity, and motivation [ ].
To date, little is known regarding the current physiotherapy practice for hip and knee osteoarthritis in Germany and whether its management is in line with guideline recommendations. We hypothesize that guideline adherence among physiotherapists is low, as in Germany, an academic level of physiotherapy education is not mandatory and only about 3% of the physiotherapists have graduated from higher educational institutions . Most physiotherapists in Germany are trained at vocational schools where skills and knowledge regarding the implementation of guidelines and evidence-based practice are not an integral part of the curriculum. Given the burden of the disease and the high costs associated with osteoarthritis, as well as the importance of physiotherapy interventions in the treatment of hip and knee osteoarthritis, it is important to address this lack of research. Developing a tailored implementation intervention can increase evidence uptake in clinical practice and thus, improve quality of health care.
The objectives of the research project are (1) to describe the current physiotherapy practice in patients with hip and knee osteoarthritis in Germany and to investigate physiotherapists’ adherence to CPG recommendations; (2) to identify and specify barriers to and facilitators of guideline use and implementation; and (3) to develop and (4) pilot test a theory-based tailored implementation intervention that aims to increase guideline use in the physiotherapy care of patients with hip and knee osteoarthritis in the German context.
The study was divided into 4 parts (). In the first part, we will evaluate the current physiotherapy practice regarding the management of patients with hip and knee osteoarthritis. Subsequently, we will identify and specify barriers to and facilitators of guideline use among physiotherapists using a qualitative approach. On the basis of these findings, in part 3, we will develop a tailored implementation intervention to increase guideline use in clinical practice, which will be evaluated in a pilot study in part 4.
The evaluation and implementation of guideline-adherent care will be based on the German guidelines for hip and knee osteoarthritis [, ]. These guidelines are aimed specifically at physicians working in musculoskeletal care and serve as information for general practitioners, physiotherapists, and occupational therapists, for example.
Part 1: Evaluation of Current Physiotherapy Practice and Guideline Adherence
A nationwide web-based survey among physiotherapists will be conducted to evaluate the current management of patients with hip and knee osteoarthritis and to explore adherence from physiotherapists to guideline recommendations. The reporting of the study will follow the Strengthening the Reporting of Observational Studies in Epidemiology statement  and the Consensus-Based Checklist for Reporting of Survey Studies [ ].
A self-administered questionnaire will be developed based on the recommendations regarding conservative nonpharmacological management listed in the German CPGs for hip and knee osteoarthritis [, ]. The questionnaire will be divided into three sections to assess (1) the demographic and working characteristics of the participants, (2) the current physiotherapy practice in patients with hip and knee osteoarthritis, and (3) perceived barriers to the implementation and clinical use of guidelines. We will use a mix of multiple choice questions and yes or no questions. To evaluate the current clinical practice, study participants will be asked to rate different treatment modalities for osteoarthritis on a 4-point Likert-type scale (1=never, 2=sometimes, 3=mostly, and 4=always). Specific barriers and facilitators regarding the German guidelines for hip and knee osteoarthritis will be assessed using the translated and cultural adapted version of “The Barriers and Facilitators Assessment Instrument” [ ]. For participants who reject using CPGs in clinical practice, reasons against guideline use will be explored based on barriers identified from literature. Adaptive questioning will be used to minimize the response burden and to reduce complexity of the questions. The web-based survey will be conducted using LimeSurvey, a free and open source web-based tool for research projects. The survey will be accessible without restrictions (password or registration) via an internet link to the LimeSurvey platform with no incentives offered. Before initiating the survey, the study participants have to read and accept written informed consent and are required to verify self-reported eligibility criteria. Participation will be voluntary and anonymous. To improve the quality and understanding of the questionnaire, a pretest of the web-based survey with 10 physiotherapists representative of the physiotherapists targeted with the survey will be conducted.
We will include physiotherapists mainly working in Germany who treat patients with hip or knee osteoarthritis in clinical practice and who are able to read and speak the German language. Only physiotherapists with a professional degree will be eligible; therefore, students will be excluded. The recruitment of participants will be initiated through announcements and calls on different physiotherapeutic networks, social media, and personal contacts of the study team. According to the snowball sampling approach, study participants will be invited to further distribute the participation invitation. As this study has an explorative character, no sample size calculation will be performed. There are approximately 203,000 physiotherapists working in Germany , but the number of physiotherapists involved in osteoarthritis care is unclear. We will target a total sample size of 1000 participants to allow regression analyses with subgroups of sufficient sample size. As survey participation is anonymous and different recruitment strategies will be used to maximize the response rate, there is a risk of duplicate responses. Therefore, we will systematically review all the responses to identify any surveys with matching demographics and answers.
Data analysis will be performed using the R software (version 4.0.3; R Foundation for Statistical Computing). Participants’ characteristics and current clinical practice will be analyzed using descriptive statistics. Guideline adherence will be defined as accordance between recommendations retrieved from the German guidelines for knee and hip osteoarthritis [, ] and the therapists’ treatment choices. As there is no standardized method to evaluate guideline adherence, we will use 2 different approaches that have been recently used in previous studies. In the first approach, guideline adherence will be determined using a points system. The participant’s treatment choices will be compared with the current guideline recommendations, and in case of adherent behavior, each will be scored with one point. The benchmarks for good adherence will be set at ≥80% of the maximally achievable points [ ]. Assessing guideline adherence on the basis of the methods of Battista et al [ ], participants will be considered as delivering, partially delivering, and nondelivering depending on their chosen treatment. For example, a participant will be classified delivering if all recommended interventions have been chosen without considering nonrecommended treatments. To assess possible determinants of guideline adherence, regression analyses will be conducted. The potential determinants will be selected based on the literature. The level of statistical significance will be set at P<.05.
Part 2: Identification and Specification of Barriers to and Facilitators of Guideline Use
A qualitative approach including semistructured interviews will be used to identify and specify barriers and facilitators to guideline use in the management of hip and knee osteoarthritis specifically for the German context. The reporting of this study will be in accordance with the Consolidated Criteria for Reporting Qualitative Research .
Purposive sampling will be used to gain a wide range of information. We aim to recruit a diverse sample of physiotherapists based on age, gender, work experience, and professional education level. Only physiotherapists mainly working in Germany who treat patients with hip or knee osteoarthritis in clinical practice and who are able to read and speak the German language will be included. The invitation to participate in one-to-one semistructured interviews will be distributed through announcements and calls on different physiotherapeutic networks, social media, and personal contacts of the study team. Recruitment will continue until data saturation has been reached (a minimum of 10 participants). Data saturation will be defined as the point when 3 consecutive interviews had been conducted with no additional information on barriers and facilitators emerging .
An interview guide will be developed based on the TDF . Questions will be informed by key findings from the previous survey and information on barriers and facilitators to guideline use identified through literature. The guide will explore the domains of the TDF to understand the factors that influence guideline use in clinical practice in the management of hip or knee osteoarthritis. The interview guide will be pilot tested with 2 physiotherapists before use. Feedback on comprehension and content will subsequently be incorporated into the interview guide after discussion with the study team.
The interviews are anticipated to last approximately 45 to 60 minutes and will be conducted via telephone or video call. Before entering the study, the participants will receive a written informed consent. Interviews will be audio-recorded and transcribed verbatim. Data collection and storage will be done pseudonymously. To give participants the opportunity to review the transcript and to make any modifications, the transcripts will be sent to the participants for member checking.
Thematic analysis following the 6 stages from Braun and Clarke  will be used to analyze the interviews. The first transcript from the semistructured interview will be pilot coded independently by 2 researchers to develop a coding strategy. The remaining interviews will be analyzed iteratively by the first author who will discuss any difficulties with a second researcher. Finally, the barriers and facilitators will be mapped onto relevant domains from the TDF. Data will be analyzed using MAXQDA (version 22; VERBI GmbH) software.
Part 3: Development of a Tailored Implementation Intervention
In this step, 6 outpatient physiotherapy practices will be recruited, 3 of which will be randomly selected to participate in the development process of the tailored intervention. In these physiotherapy practices, the intervention will be implemented (study part 4), whereas the remaining practices will serve as a control group. We will include physiotherapy practices in the region of Brandenburg and Saxony (Germany) with >5 physiotherapists who treat patients with hip or knee osteoarthritis in clinical practice.
The development of the implementation intervention will be guided by the BCW and follows 3 stages, as outlined by Michie et al : understand the target behavior, identify intervention options, and identify content and implementation options. These can be further divided into key substeps as illustrated in [ ].
Stage 1: Understand the Target Behavior
The key barriers identified from the semistructured interviews (part 2) will be used to define the problem in behavioral terms and to specify the behavior and target population (step 1). From a list of all potential candidate behaviors, the study team will choose 1 or 2 core behaviors that appear to be promising in increasing guideline adherence in physiotherapists (step 2). The target behavior will be selected considering each potential impact, likelihood of change, spillover effect, and ease of measurement. After specifying the target behavior in detail and its context (who, what, when, where, how often, and with whom; step 3), focus groups will be conducted to identify what needs to change in the individual or the environment to achieve the desired behavior change (step 4).
The reporting of the focus groups will be guided by the Consolidated Criteria for Reporting Qualitative Research checklist . One to 2 focus groups will be conducted at each participating physiotherapy practice. The focus groups will take place at the participants’ worksite and will be moderated by 2 members of the research team. Mixed groups of 6 to 10 participants recruited from each physiotherapy practice representing different stakeholders (eg, physiotherapists, patients, management, administration, and other health professionals) will be invited to participate to promote diversity and to encourage a wide range of information on what needs to change regarding the target behavior. All participants will receive and approve a written informed consent before initiating the study. The topic guide will be developed using the TDF. The standardized guide will be used across all focus groups with some flexibility through open questions to allow further discussion between participants. Before the focus groups, basic demographics (eg, age, gender, job type, and working hours) will be collected from each participant. The focus group discussions will be audio-recorded while simultaneously handwritten field notes capturing relevant nonverbal responses or dynamics of the group will be documented. Data from the focus groups will be transcribed verbatim and will be analyzed following the same process used for the semistructured interviews described in the Part 2: Identification and Specification of Barriers to and Facilitators of Guideline Use section.
Stage 2: Identify Intervention Options
On the basis of the findings from the focus groups, relevant intervention functions will be identified using a matrix that systematically maps TDF domains to intervention functions (eg, education, training, persuasion; step 5). As changes at the policy level will be beyond the scope and time constraints of our project, and intervention programs targeting individual level and local barriers have been shown to be effective , policy categories (eg, regulation, communication or marketing, and service provision) provided by the BCW guidance will not be considered in this study (step 6). To select the most appropriate and effective intervention functions for our implementation intervention, the affordability, practicability, effectiveness and cost effectiveness, acceptability, side effects or safety, and equity criteria will be used [ ].
Stage 3: Identify Content and Implementation Options
On the basis of the intervention functions selected in step 5, behavior change techniques (BCTs) will be identified (step 7). We will use BCT taxonomy (version 1; BCTTV1)  to generate a list of BCTs mapped to our intervention functions. Regarding the affordability, practicability, effectiveness and cost effectiveness, acceptability, side effects or safety, and equity criteria, the research team will work in close collaboration with the stakeholders from the participating physiotherapy practices and discuss the included BCTs as well as possible modes of delivery for the intervention (step 8). Those that are likely to be the most effective and feasible to increase guideline adherence will be selected. Reporting of the intervention developed will be consistent with the recommendations for specifying and reporting implementation strategies in research studies proposed by Proctor et al [ ] and the Template for Intervention Description and Replication checklist [ ].
Part 4: Pilot Test of the Implementation Intervention
A controlled pilot study will be conducted to evaluate the effectiveness, feasibility, and acceptability of the tailored intervention implementation. The pilot study will simultaneously test a clinical intervention and an implementation intervention or strategy and thus will be a hybrid study . The Standards for Reporting Implementation Studies statement [ ] and the CONSORT (Consolidated Standards of Reporting Trials) statement [ ] for pilot and feasibility trials will be used to guide the reporting of study methods and results.
The previously designed intervention will be implemented and evaluated at the 3 physiotherapy practices recruited in part 3 that participated in the development process, whereas the other 3 recruited practices will serve as the control group.
All physiotherapists involved in osteoarthritis management will be invited to participate. Patients will be eligible if they (1) are aged ≥18 years, (2) are primarily diagnosed with hip or knee osteoarthritis, and (3) have no restraint to receive physiotherapy. Participation will be voluntary, and informed consent will be obtained before the recruitment of any participant into the study.
To explore effectiveness at the patient level, the assessment of primary patient-relevant outcomes will follow the Outcome Measures in Rheumatology-Osteoarthritis Research Society International core domain set for clinical trials in hip and knee osteoarthritis and will include 5 domains: pain, physical function, quality of life, patient’s global assessment of the target joint, and adverse events . Patient outcomes will be assessed at baseline, at the end of physiotherapy treatment, and after 6 months. Physiotherapists’ adherence to the osteoarthritis guidelines will be evaluated using a documentation questionnaire after each physiotherapy session. The implementation process will be evaluated using the Acceptability of Intervention Measure, Intervention Appropriateness Measure, and the Feasibility of Intervention Measure [ , ]. Additional outcomes will be the recruitment and retention rate [ ]. Results from the pilot study will be used to inform sample size calculation for a subsequent study.
Patient and Public Involvement Statement
Patients and the public were not directly involved in the design or conduct, reporting, or dissemination plans of this research.
Ethics approval for each part of the study will be sought from the ethics committee of the Brandenburg University of Technology Cottbus-Senftenberg, Germany. Permission for the nationwide web-based survey (study part 1) and the semistructured interviews (study part 2) has already been obtained from the ethics committee of the Brandenburg University of Technology Cottbus-Senftenberg, Germany (EK2021-10, EK2022-15). Throughout the research project, the principles established in the Declaration of Helsinki  will be strictly followed. Participants who are eligible for any part of the study will be informed about the respective objectives, content, and procedures of the study. Written informed consent will be obtained from all patients before enrollment. Participation will be voluntary, and participants are free to withdraw from the study at any moment.
The findings from this research project will be disseminated through open-access publications in international peer-reviewed journals and presentations at national and international scientific meetings. In addition, we will use social media (eg, X, formerly known as Twitter) to spread our findings to the public.
In December 2021, the survey among German physiotherapists was completed, and results on the management of patients with hip and knee osteoarthritis and guideline adherence were published  (part 1 of the research project). Ethics approval for the second part of the research project was obtained in August 2022, and the semistructured interviews were conducted between January and September 2023. Recruitment of physiotherapy practices to participate in the development of the implementation intervention is expected to start in January 2024.
The described research project aims to evaluate and improve the implementation of hip and knee osteoarthritis guidelines in physiotherapy in Germany, which to our knowledge has not been performed to date. Considering the burden of disease and the importance of physiotherapy interventions in the management of osteoarthritis, as well as findings from other countries indicating gaps between clinical practice and research findings [, ], it seems important to address this lack of research.
The implementation of research findings into clinical practice is considered relevant for improving quality of care but is often hindered by various barriers . Although guidelines are considered helpful in translating research findings into clinical practice by providing recommendations that can be used by health care professionals, patients, policy makers, and other stakeholders, many guidelines include no or a variety of guideline implementation tools for different stakeholders [ ]. There are few studies on how to choose implementation strategies that address guideline-specific barriers, but education for clinicians or patients and printed materials were the most commonly used strategies for translating guidelines into practice [ ]. Holm et al [ ] reported that the limited use of CPGs may be because of insufficient knowledge of current evidence or that clinicians and patients may find it difficult to break old habits and abandon existing low-value treatment and also because the increasing number of frameworks and checklists for guideline implementation makes implementation strategies too comprehensive and complicated.
For the management of people with osteoarthritis, pragmatic approaches to implementing osteoarthritis guidelines exist. For example, Good Life with osteoArthritis in Denmark (GLA:D) or Active with osteoArthritis (ActiveA) have been implemented in several countries . GLA:D, which is currently being implemented in Germany, and ActiveA are both effective in reducing pain intensity and increasing quality of life, physical activity, or physical function [ , - ]. Østerås et al [ ] showed that a structured implementation of ActiveA in primary care resulted in a higher quality of osteoarthritis care as compared with usual care. However, it is unclear if such approaches can be implemented as standard care in the German health care system, as GLA:D does not fit into the German catalog of remedies, which defines the number and type of physiotherapeutic treatments, and is therefore only available based on special contracts with health insurance companies or must be paid for privately by patients. Furthermore, not all patients can (eg, owing to limited access to physiotherapy practices) or want to (eg, because they do not want to train in a group) participate in GLA:D. Therefore, it is necessary to evaluate how to increase guideline use in osteoarthritis care and thus improve the quality of care in addition to the ongoing implementation of GLA:D in Germany. With regard to physiotherapists in Germany, it was recently shown that the implementation of evidence-based practice in clinical practice is insufficient [ ]. In a recent nationwide survey on physiotherapy management of low back pain, less than one-third of the participating physiotherapists reported being aware of the National Disease Management Guideline for “non-specific low back pain” or have dealt with its recommendations [ ]. Similar results were shown by another survey, in which only 47% of the responding physiotherapists reported being aware of the German guidelines on stroke rehabilitation [ ]. One reason for the low awareness of CPGs could be that the entry level for German physiotherapists is vocational school education. Furthermore, physiotherapists report limited access to guidelines [ ] and often do not know where to find them [ ]. We hypothesize that the physiotherapy management of persons with hip or knee osteoarthritis will most likely not be guideline adherent and therefore needs to be improved, which is the aim of this study.
There are numerous frameworks, models, and theories that provide guidance in developing strategies to facilitate guideline implementation . However, many guideline implementation studies do not report explicit theory use or theory is not used consistently and transparently [ , ]. Choosing the most appropriate theory-driven approach is challenging as such models can be complex, that is, difficult for both researchers and health care practitioners to understand and operationalize, or lack comprehensiveness [ ]. Although the TDF and BCW are comparatively new implementation frameworks, they have already been used in several studies to understand and promote the uptake of evidence in clinical practice [ - ], including in the field of physiotherapy [ ]. For this research project, we decided to use the TDF as it has proven to be a comprehensive and accessible way to understand the barriers and facilitators to best practice in primary and secondary care in Western countries [ ]. There is evidence that TDF-based interview schedules achieve a broader range of responses than when using a nontheoretical approach [ ]. Therefore, the TDF is described as an “inclusive” rather than a “selective” approach [ ]. The TDF works within the BCW framework as it provides a more detailed understanding of the behavior components of capability, opportunity, and motivation. The BCW provides a systematic and pragmatic approach to developing implementation interventions. It encourages intervention designers to comprehensively consider the full range of options and select those that are most promising for the context. In addition, it allows for a detailed specification of the content using agreed labels and definitions [ ].
The results from our project will provide valuable knowledge concerning the development process and effectiveness of an intervention to implement hip or knee osteoarthritis guidelines in physiotherapy practice. We aim to develop an implementation intervention that can also be transferred to other health conditions, with modifications, if necessary. We hypothesize that successful implementation will lead to increased guideline adherence in physiotherapists, which in turn will improve the quality of care.
This research was funded by an intramural grant from the Brandenburg University of Technology, Cottbus-Senftenberg (award or grant number is not applicable). The funder had no role in the study design, data collection, analysis and interpretation, decision to publish, or preparation of the manuscript.
The data sets generated during the research project will be available on the Open Science Framework: 10.17605/OSF.IO/A7P3W.
CB, BS, and CK conceptualized the design of the research project and drafted the manuscript. All authors contributed to the development of this manuscript and have read and approved the final version for publication.
Conflicts of Interest
- Hunter DJ, Bierma-Zeinstra S. Osteoarthritis. Lancet 2019 Apr 27;393(10182):1745-1759 [CrossRef] [Medline]
- Postler A, Ramos AL, Goronzy J, Günther KP, Lange T, Schmitt J, et al. Prevalence and treatment of hip and knee osteoarthritis in people aged 60 years or older in Germany: an analysis based on health insurance claims data. Clin Interv Aging 2018 Nov 14;13:2339-2349 [https://europepmc.org/abstract/MED/30532524] [CrossRef] [Medline]
- Hunter DJ, McDougall JJ, Keefe FJ. The symptoms of osteoarthritis and the genesis of pain. Rheum Dis Clin North Am 2008 Aug;34(3):623-643 [https://europepmc.org/abstract/MED/18687276] [CrossRef] [Medline]
- Salmon JH, Rat AC, Sellam J, Michel M, Eschard JP, Guillemin F, et al. Economic impact of lower-limb osteoarthritis worldwide: a systematic review of cost-of-illness studies. Osteoarthritis Cartilage 2016 Sep;24(9):1500-1508 [https://linkinghub.elsevier.com/retrieve/pii/S1063-4584(16)01073-6] [CrossRef] [Medline]
- Rabenberg M. Themenheft 54 "Arthrose". Robert Koch-Institut. 2013. URL: https://link.springer.com/article/10.1007/BF03044519 [accessed 2021-12-28]
- Institute of Medicine. Clinical Practice Guidelines We Can Trust. Washington, DC: The National Academies Press; 2011.
- Bannuru RR, Osani MC, Vaysbrot EE, Arden NK, Bennell K, Bierma-Zeinstra SM, et al. OARSI guidelines for the non-surgical management of knee, hip, and polyarticular osteoarthritis. Osteoarthritis Cartilage 2019 Nov;27(11):1578-1589 [https://linkinghub.elsevier.com/retrieve/pii/S1063-4584(19)31116-1] [CrossRef] [Medline]
- Fernandes L, Hagen KB, Bijlsma JW, Andreassen O, Christensen P, Conaghan PG, et al. EULAR recommendations for the non-pharmacological core management of hip and knee osteoarthritis. Ann Rheum Dis 2013 Jul;72(7):1125-1135 [https://ard.bmj.com/lookup/pmidlookup?view=long&pmid=23595142] [CrossRef] [Medline]
- Osteoarthritis in over 16s: diagnosis and management. National Institute for Health and Care Excellence. 2022. URL: https://www.nice.org.uk/guidance/ng226 [accessed 2022-12-21]
- Skou ST, Roos EM. Good Life with osteoArthritis in Denmark (GLA:D™): evidence-based education and supervised neuromuscular exercise delivered by certified physiotherapists nationwide. BMC Musculoskelet Disord 2017 Feb 07;18(1):72 [https://bmcmusculoskeletdisord.biomedcentral.com/articles/10.1186/s12891-017-1439-y] [CrossRef] [Medline]
- Battista S, Salvioli S, Millotti S, Testa M, Dell'Isola A. Italian physiotherapists' knowledge of and adherence to osteoarthritis clinical practice guidelines: a cross-sectional study. BMC Musculoskelet Disord 2021 Apr 23;22(1):380 [https://bmcmusculoskeletdisord.biomedcentral.com/articles/10.1186/s12891-021-04250-4] [CrossRef] [Medline]
- Spitaels D, Hermens R, Van Assche D, Verschueren S, Luyten F, Vankrunkelsven P. Are physiotherapists adhering to quality indicators for the management of knee osteoarthritis? An observational study. Musculoskelet Sci Pract 2017 Feb;27:112-123 [CrossRef] [Medline]
- Zadro J, O'Keeffe M, Maher C. Do physical therapists follow evidence-based guidelines when managing musculoskeletal conditions? Systematic review. BMJ Open 2019 Oct 07;9(10):e032329 [https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=31591090] [CrossRef] [Medline]
- Hagen KB, Smedslund G, Østerås N, Jamtvedt G. Quality of community-based osteoarthritis care: a systematic review and meta-analysis. Arthritis Care Res (Hoboken) 2016 Oct;68(10):1443-1452 [https://onlinelibrary.wiley.com/doi/10.1002/acr.22891] [CrossRef] [Medline]
- Sorondo D, Delpierre C, Côté P, Salmi LR, Cedraschi C, Taylor-Vaisey A, et al. Determinants of clinical practice guidelines' utilization for the management of musculoskeletal disorders: a scoping review. BMC Musculoskelet Disord 2021 Jun 01;22(1):507 [https://bmcmusculoskeletdisord.biomedcentral.com/articles/10.1186/s12891-021-04204-w] [CrossRef] [Medline]
- da Silva TM, Costa Lda C, Garcia AN, Costa LO. What do physical therapists think about evidence-based practice? A systematic review. Man Ther 2015 Jun;20(3):388-401 [CrossRef] [Medline]
- Fischer F, Lange K, Klose K, Greiner W, Kraemer A. Barriers and strategies in guideline implementation-a scoping review. Healthcare (Basel) 2016 Jun 29;4(3):36 [https://www.mdpi.com/resolver?pii=healthcare4030036] [CrossRef] [Medline]
- Ferreira de Meneses S, Rannou F, Hunter DJ. Osteoarthritis guidelines: barriers to implementation and solutions. Ann Phys Rehabil Med 2016 Jun;59(3):170-173 [https://linkinghub.elsevier.com/retrieve/pii/S1877-0657(16)00032-4] [CrossRef] [Medline]
- Paci M, Faedda G, Ugolini A, Pellicciari L. Barriers to evidence-based practice implementation in physiotherapy: a systematic review and meta-analysis. Int J Qual Health Care 2021 Jun 26;33(2):mzab093 [CrossRef] [Medline]
- Bahns C, Happe L, Thiel C, Kopkow C. Physical therapy for patients with low back pain in Germany: a survey of current practice. BMC Musculoskelet Disord 2021 Jun 19;22(1):563 [https://bmcmusculoskeletdisord.biomedcentral.com/articles/10.1186/s12891-021-04422-2] [CrossRef] [Medline]
- Derghazarian T, Simmonds MJ. Management of low back pain by physical therapists in quebec: how are we doing? Physiother Can 2011;63(4):464-473 [https://europepmc.org/abstract/MED/22942525] [CrossRef] [Medline]
- Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2010 Mar 17(3):CD005470 [https://europepmc.org/abstract/MED/20238340] [CrossRef] [Medline]
- Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev 2015 Apr 29;2015(4):CD005470 [https://europepmc.org/abstract/MED/25923419] [CrossRef] [Medline]
- Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol 2005 Feb;58(2):107-112 [CrossRef] [Medline]
- Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG). Designing theoretically-informed implementation interventions. Implement Sci 2006 Feb 23;1:4 [https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-1-4] [CrossRef] [Medline]
- Liang L, Bernhardsson S, Vernooij RW, Armstrong MJ, Bussières A, Brouwers MC, et al. Members of the Guidelines International Network Implementation Working Group. Use of theory to plan or evaluate guideline implementation among physicians: a scoping review. Implement Sci 2017 Feb 27;12(1):26 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0557-0] [CrossRef] [Medline]
- Atkins L, Francis J, Islam R, O'Connor D, Patey A, Ivers N, et al. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implement Sci 2017 Jun 21;12(1):77 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0605-9] [CrossRef] [Medline]
- Michie S, Atkins L, West R. The Behaviour Change Wheel: A Guide to Designing Interventions. London, United Kingdom: Silverback Publishing; 2014.
- Zahlen, Daten, Fakten zur Physiotherapie. PhysioDeutschland Deutscher Verband für Physiotherapie. 2022 Jan 14. URL: https://www.physio-deutschland.de/fileadmin/data/bund/Dateien_oeffentlich/Beruf_und_Bildung/Zahlen__Daten__Fakten/Zahlen_Daten_Fakten14.1.22.pdf [accessed 2022-11-01]
- Deutsche Gesellschaft für Orthopädie und Orthopädische Chirurgie, Koxarthrose. AWMF online. 2019. URL: https://register.awmf.org/assets/guidelines/033-001l_S2k_Koxarthrose_2019-07_1_01.pdf [accessed 2021-01-18]
- Deutsche Gesellschaft für Orthopädie und Orthopädische Chirurgie, Gonarthrose. AWMF online. 2018. URL: https://register.awmf.org/assets/guidelines/033-004l_S2k_Gonarthrose_2018-01_1-verlaengert_01.pdf [accessed 2021-01-18]
- von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol 2008 Apr;61(4):344-349 [CrossRef] [Medline]
- Sharma A, Minh Duc NT, Luu Lam Thang T, Nam NH, Ng SJ, Abbas KS, et al. A consensus-based checklist for reporting of survey studies (CROSS). J Gen Intern Med 2021 Oct;36(10):3179-3187 [https://europepmc.org/abstract/MED/33886027] [CrossRef] [Medline]
- Harmsen M, Peters M, Wensing M. Barriers and facilitators assessment instrument: introduction, instructions and instrument. IQ Scientific Institute for Quality of Healthcare. 2005. URL: https://www.iqhealthcare.nl/media/124628/barriers-and-facilitators-questionnaire_2018.pdf [accessed 2022-01-06]
- Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357 [CrossRef] [Medline]
- Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health 2010 Dec;25(10):1229-1245 [CrossRef] [Medline]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101 [https://www.tandfonline.com/doi/abs/10.1191/1478088706qp063oa] [CrossRef]
- Stander J, Grimmer K, Brink Y. Training programmes to improve evidence uptake and utilisation by physiotherapists: a systematic scoping review. BMC Med Educ 2018 Jan 15;18(1):14 [https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-018-1121-6] [CrossRef] [Medline]
- Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95 [https://core.ac.uk/reader/191129821?utm_source=linkout] [CrossRef] [Medline]
- Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013 Dec 01;8:139 [https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-139] [CrossRef] [Medline]
- Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014 Mar 07;348(mar07 3):g1687 [https://core.ac.uk/reader/19913708?utm_source=linkout] [CrossRef] [Medline]
- Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012 Mar;50(3):217-226 [https://europepmc.org/abstract/MED/22310560] [CrossRef] [Medline]
- Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. StaRI Group. Standards for Reporting Implementation Studies (StaRI) statement. BMJ 2017 Mar 06;356:i6795 [http://www.bmj.com/lookup/pmidlookup?view=long&pmid=28264797] [CrossRef] [Medline]
- Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ 2016 Oct 24;355:i5239 [http://www.bmj.com/lookup/pmidlookup?view=long&pmid=27777223] [CrossRef] [Medline]
- Smith TO, Hawker GA, Hunter DJ, March LM, Boers M, Shea BJ, et al. The OMERACT-OARSI core domain set for measurement in clinical trials of hip and/or knee osteoarthritis. J Rheumatol 2019 Aug 15;46(8):981-989 [https://eprints.whiterose.ac.uk/139523/] [CrossRef] [Medline]
- Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci 2017 Aug 29;12(1):108 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0635-3] [CrossRef] [Medline]
- Kien C, Griebler U, Schultes MT, Thaler KJ, Stamm T. Psychometric testing of the German versions of three implementation outcome measures. Glob Implement Res Appl 2021 Aug 23;1(3):183-194 [https://link.springer.com/article/10.1007/s43477-021-00019-y] [CrossRef]
- Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010 Jan 06;10:1 [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-10-1] [CrossRef] [Medline]
- World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA 2013 Nov 27;310(20):2191-2194 [CrossRef] [Medline]
- Bahns C, Kopkow C. Physiotherapy for patients with hip and knee osteoarthritis in Germany: a survey of current practice. BMC Musculoskelet Disord 2023 May 26;24(1):424 [https://bmcmusculoskeletdisord.biomedcentral.com/articles/10.1186/s12891-023-06464-0] [CrossRef] [Medline]
- Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet 2003 Oct 11;362(9391):1225-1230 [CrossRef] [Medline]
- Liang L, Abi Safi J, Gagliardi AR, Members of the Guidelines International Network Implementation Working Group. Number and type of guideline implementation tools varies by guideline, clinical condition, country of origin, and type of developer organization: content analysis of guidelines. Implement Sci 2017 Nov 15;12(1):136 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0668-7] [CrossRef] [Medline]
- Gagliardi AR, Alhabib S, Members of Guidelines International Network Implementation Working Group. Trends in guideline implementation: a scoping systematic review. Implement Sci 2015 Apr 21;10(1):54 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-015-0247-8] [CrossRef] [Medline]
- Holm I, Risberg MA, Roos EM, Skou ST. A pragmatic approach to the implementation of osteoarthritis guidelines has fewer potential barriers than recommended implementation frameworks. J Orthop Sports Phys Ther 2019 Jan;49(1):1-4 [CrossRef] [Medline]
- Barton CJ, Kemp JL, Roos EM, Skou ST, Dundules K, Pazzinatto MF, et al. Program evaluation of GLA:D® Australia: physiotherapist training outcomes and effectiveness of implementation for people with knee osteoarthritis. Osteoarthr Cartil Open 2021 Sep;3(3):100175 [https://linkinghub.elsevier.com/retrieve/pii/S2665-9131(21)00038-8] [CrossRef] [Medline]
- Roos EM, Grønne DT, Skou ST, Zywiel MG, McGlasson R, Barton CJ, et al. Immediate outcomes following the GLA:D® program in Denmark, Canada and Australia. A longitudinal analysis including 28,370 patients with symptomatic knee or hip osteoarthritis. Osteoarthritis Cartilage 2021 Apr;29(4):502-506 [https://linkinghub.elsevier.com/retrieve/pii/S1063-4584(21)00033-9] [CrossRef] [Medline]
- Roos EM, Barton CJ, Davis AM, McGlasson R, Kemp JL, Crossley KM, et al. GLA:D to have a high-value option for patients with knee and hip arthritis across four continents: good life with osteoArthritis from Denmark. Br J Sports Med 2018 Dec 07;52(24):1544-1545 [CrossRef] [Medline]
- Holm I, Pripp AH, Risberg MA. The active with OsteoArthritis (AktivA) physiotherapy implementation model: a patient education, supervised exercise and self-management program for patients with mild to moderate osteoarthritis of the knee or hip joint. A national register study with a two-year follow-up. J Clin Med 2020 Sep 26;9(10):3112 [https://www.mdpi.com/resolver?pii=jcm9103112] [CrossRef] [Medline]
- Østerås N, Moseng T, van Bodegom-Vos L, Dziedzic K, Mdala I, Natvig B, et al. Implementing a structured model for osteoarthritis care in primary healthcare: a stepped-wedge cluster-randomised trial. PLoS Med 2019 Oct 15;16(10):e1002949 [https://dx.plos.org/10.1371/journal.pmed.1002949] [CrossRef] [Medline]
- Braun T, Ehrenbrusthoff K, Bahns C, Happe L, Kopkow C. [Adherence to and influencing factors of evidence-based practice in physiotherapeutic care in Germany: a cross-sectional study]. Z Evid Fortbild Qual Gesundhwes 2022 Feb;168:8-20 [https://linkinghub.elsevier.com/retrieve/pii/S1865-9217(21)00200-2] [CrossRef] [Medline]
- Scheffler B, Schimböck F, Schöler A, Rösner K, Spallek J, Kopkow C. Current physical therapy practice and implementation factors regarding the evidence-based 'Rehabilitation of Mobility after Stroke (ReMoS)' guideline in Germany: a cross-sectional online survey. BMC Neurol 2022 Jul 30;22(1):284 [https://bmcneurol.biomedcentral.com/articles/10.1186/s12883-022-02780-5] [CrossRef] [Medline]
- Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015 Apr 21;10:53 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-015-0242-0] [CrossRef] [Medline]
- Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci 2010 Feb 09;5:14 [https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-5-14] [CrossRef] [Medline]
- Dyson J, Cowdell F. How is the theoretical domains framework applied in designing interventions to support healthcare practitioner behaviour change? A systematic review. Int J Qual Health Care 2021 Jul 28;33(3):mzab106 [CrossRef] [Medline]
- Bérubé M, Albert M, Chauny JM, Contandriopoulos D, DuSablon A, Lacroix S, et al. Development of theory-based knowledge translation interventions to facilitate the implementation of evidence-based guidelines on the early management of adults with traumatic spinal cord injury. J Eval Clin Pract 2015 Dec 31;21(6):1157-1168 [CrossRef] [Medline]
- Kourouche S, Buckley T, Van C, Munroe B, Curtis K. Designing strategies to implement a blunt chest injury care bundle using the behaviour change wheel: a multi-site mixed methods study. BMC Health Serv Res 2019 Jul 08;19(1):461 [https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-019-4177-z] [CrossRef] [Medline]
- Mackay HJ, Campbell KL, van der Meij BS, Wilkinson SA. Establishing an evidenced-based dietetic model of care in haemodialysis using implementation science. Nutr Diet 2019 Apr;76(2):150-157 [CrossRef] [Medline]
- Thomas S, Mackintosh S. Use of the theoretical domains framework to develop an intervention to improve physical therapist management of the risk of falls after discharge. Phys Ther 2014 Nov;94(11):1660-1675 [CrossRef] [Medline]
- Dyson J, Lawton R, Jackson C, Cheater F. Does the use of a theoretical approach tell us more about hand hygiene behaviour? The barriers and levers to hand hygiene. J Infect Prev 2010 Oct 08;12(1):17-24 [https://journals.sagepub.com/doi/10.1177/1757177410384300] [CrossRef]
- Francis JJ, O'Connor D, Curran J. Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci 2012 Apr 24;7(1):35 [https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-7-35] [CrossRef] [Medline]
|ActiveA: Active with osteoArthritis|
|BCT: behavior change technique|
|BCW: Behavior Change Wheel|
|CONSORT: Consolidated Standards of Reporting Trials|
|CPG: clinical practice guideline|
|GLA:D: Good Life with osteoArthritis in Denmark|
|TDF: Theoretical Domains Framework|
Edited by A Mavragani; submitted 05.04.23; peer-reviewed by L Baumbach, C Varsi; comments to author 20.06.23; revised version received 07.07.23; accepted 25.07.23; published 16.11.23Copyright
©Carolin Bahns, Bettina Scheffler, Christian Kopkow. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 16.11.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.