This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
In Belgium, the construction of a national electronic point-of-care information service, EBMPracticeNet, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The collaboration of the government, health care providers, evidence-based medicine (EBM) partners, and vendors of electronic health records (EHR) is unique to this project. All Belgian health care professionals get free access to an up-to-date database of validated Belgian and nearly 1000 international guidelines, incorporated in a portal that also provides EBM information from other sources than guidelines, including computerized clinical decision support that is integrated in the EHRs.
The objective of this paper was to describe the development strategy, the overall content, and the management of EBMPracticeNet which may be of relevance to other health organizations creating national or regional electronic point-of-care information services.
Several candidate providers of comprehensive guideline solutions were evaluated and one database was selected. Translation of the guidelines to Dutch and French was done with translation software, post-editing by translators and medical proofreading. A strategy is determined to adapt the guideline content to the Belgian context. Acceptance of the computerized clinical decision support tool has been tested and a randomized controlled trial is planned to evaluate the effect on process and patient outcomes.
Currently, EBMPracticeNet is in "work in progress" state. Reference is made to the results of a pilot study and to further planned research including a randomized controlled trial.
The collaboration of government, health care providers, EBM partners, and vendors of EHRs is unique. The potential value of the project is great. The link between all the EHRs from different vendors and a national database held on a single platform that is controlled by all EBM organizations in Belgium are the strengths of EBMPracticeNet.
There has been an explosive growth in scientific evidence, with 75 trials and 11 systematic reviews being published a day [
Properly designed information retrieval and clinical decision support systems are now being promoted as a Global Positioning System to prevent health care providers from getting lost in clinical practice [
Structured clinical guidelines and computerized clinical decision support systems have the potential for improving the quality of care [
In Belgium, the construction of a national electronic point-of-care information service, EBMPracticeNet, was initiated in September 2011 to optimize quality of care by promoting evidence-based decision-making [
The aim of this paper is to describe the development strategy, the overall content, and the management of EBMPracticeNet.
EBMPracticeNet was officially founded as a non-profit organization in 2011, and originated from grass roots gathering of EBM-producing organizations. This integrative cooperation project was inspired by CEBAM, the Belgian Centre for EBM and the Belgian branch of the Dutch Cochrane Collaboration. EBMPracticeNet is open to four types of organizations, namely independent producers of EBM information, disseminator organizations, user organizations, and governmental public health departments. Funding comes from the national health insurance institute (National Institute for Health and Disability Insurance, INAMI-RIZIV). The project fits within the broader range of eHealth initiatives in Belgium. The goal is to share national or adapted international EBM guidelines on one certified and secured platform (eHealth), easily accessible to different vendors of EHRs. The information structure of EBMPracticeNet is presented in
The guidelines database is a mix of national and international guidelines. First of all it entails about 50 Belgian guidelines, regularly updated by local Belgian Guideline producers. We supplemented this with a comprehensive database of international clinical guidelines, with the intention to adapt the content to the Belgian context. Several candidate providers of comprehensive guideline solutions were evaluated on the basis of a published review [
EBMPracticeNet information structure.
Duodecim EBM Guidelines comprise of nearly 1000 clinical guidelines and one million words. Translating this comprehensive set and adapting it to the Belgian context posed a huge challenge for EBMPracticeNet. Our first step in this process involved the translation from English to Dutch and French, two official languages of Belgium. This translation project was undertaken in cooperation with our technical partner, Iscientia IVS, a broker company for scientific information, and provider of technical platforms for scientific bibliographic information. The process was supervised by an academic institution of Applied Language Studies (Hogeschool Gent). First, the translation software, SDL Trados Studio, produced a machine translation, which was then post-edited by a human translator, a medical proofreader and a validator [
Next is the adaptation of the Duodecim EBM Guidelines to the Belgian context. This process was guided by a preliminary prioritization effort. First, an inventory was made of 50 validated Belgian guidelines. The content of these Belgian guidelines was transformed in the format of the Duodecim EBM guidelines to replace the International guidelines. This involved a two-step transformation with first the production of a structured summary, and then legacy conversion into the EBM Guideline Extensible Markup Language format. Second, an additional priority list of 50 clinical topics was drafted by taking into account clinical information need (based on user surveys and epidemiological reason-for-encounter data in sentinel practice networks) and priority areas indicated by key stakeholders. The relevant guidelines for this list of priority clinical topics are now screened in close consultation with the core community of experts from the Belgian producers of EBM information [
For the screening of the larger set of low-priority guidelines, we have invited stakeholder organizations and volunteers to screen the remaining guidelines only with regard to their compliance with the Belgian context. This process of screening and adaptation has to be completed by the end of 2015. Meanwhile, all the guidelines will be published with an explicit indication that adaptation to the Belgian context is under way. This indication will be removed when the screening (and if necessary the adaptation) is performed.
The Duodecim EBM Guidelines are revised continuously at a rate of 80 updated guidelines each trimester. This means that we need to keep up that pace of translating the updated information and screening when, and if, updated recommendations need adapting.
The user can search for information with a search engine or can browse for information using a navigation menu based on the conditions included in the database. The content and search engine are organized in such a way that they can be used during the patient encounter in a minimum of time. Implementation of direct access to the EBMPracticeNet is now a criterion in the accreditation of EHR software in Belgium.
The information in the original Duodecim EBM Guidelines database is indexed with the International Classification of Primary Care, Second edition (ICPC-2) and the International Classification of Diseases, Tenth revision (ICD-10) codes and Medical Subject Headings (MeSH) terms. The available ICD-10 and ICPC-2 codes were submitted to scrutiny and validated for the diagnostic part of the guidelines. However, additional coding is needed for process and outcome aspects. An available translation of MeSH terms in Dutch and French will be used to translate the English MeSH terms [
Based on the 1000 clinical guidelines, we will build a multilingual terminology database. Selected words and phrases will be attributed to each guideline to create an effective search engine to search the database in Dutch, French, and English.
In addition to rapid access to practical recommendations at the point-of-care, this portal also organizes the flow of the clinical information in a chain of evidences that allows users with specific clinical questions to move efficiently from guidelines to systematic reviews and primary studies. As the time invested in these searches increase, they will typically be accessed outside the patient consultation.
With the Duodecim EBM guidelines comes a collection of more than 4000 evidence summaries. These evidence summaries are graded statements with a short description of systematic reviews or original research [
Each guideline in the database will be linked to specific information on the websites of Belgian EBM producers. Although the Belgian EBM information is scattered across various websites, this will make it possible for the user to surf the relevant links on the site of the producers, through the EBMPracticeNet. Likewise, the guidelines will be linked to information from INAMI-RIZIV.
Integration with the CEBAM Digital Library for Health enables the users to move from the bibliographic information provided on EBMPracticeNet to the full text of original research or systematic reviews, either in the Cochrane Library or in the large collection of scientific journals, subscribed to by the Digital Library [
The computerized clinical decision support component uses the Evidence Linker technology and the Evidence-based Medicine electronic Decision Support (EBMeDS) system. The Evidence Linker is a new tool developed by the CEBAM with two General Practice trainees supervised at the KU Leuven (by BA), that provides a direct link between patient data from the EHR and guidelines for general practitioners [
The EBMeDS system was developed by Duodecim and its content development process is also accredited as such by the United Kingdom NHS [
To promote use of the EBMPracticeNet services, we adopt a multifaceted strategy [
The building and management of EBMPracticeNet is coordinated by a project leader, an editor-in-chief, five editors, and a secretary, all working part-time on this project and representing two full-time equivalents. The processes are being implemented in collaboration with the Belgian EBM producers, technical experts, and volunteers. Finding competent volunteers that are motivated to participate in these processes is a key factor in the sustainability of this project. The use of volunteers can include taking on the responsibility for one or several guidelines to ensure that the recommendations and their updates are in accordance with the Belgian context.
The project involves many working processes such as: information collection, processing and validation, publication and updating of the published information, and usage monitoring. An important initial effort was to describe all the key processes in Business Process Model and Notation (BPMN). The BPMN is a standard graphical notation that describes working processes in flowcharts and enables the management team to clarify and optimize processes for all stakeholders, and ensure that processes are easily transferable within the team and to the partners. We developed these flowcharts in the Open Source software Bizagi, which allows exportation to automated work flows and task lists on our editorial platform (Microsoft Sharepoint). This proved to be vital to increase the manageability of the different working processes for a small project team, collaborating with a large group of partners and volunteers.
Currently, EBMPracticeNet is still in a “work in progress” state. The use of EBMPracticeNet will be monitored in order to better meet the needs of the users and for research purposes. For this purpose, routine statistical information will be collected, such as:
Mechanism of information retrieval: search engine, navigation system, Evidence-Linker, EBMeDS scripts.
Type of information: search terms used, documents opened,
User profile: type of user (health care providers, general public), language group.
Time of use: hour and day of use, time spent in a resource.
This information is general in nature and collected anonymously, in respect of privacy regulations.
A research agenda needs to be developed to evaluate the impact of EBMPracticeNet on the care provided by Belgian health care providers. The research will comprise user-centered evaluations to analyze factors associated with failure or success; content-centered evaluation to evaluate the EBM quality; and quality and safety of care evaluation.
Preliminary results include the pilot implementation of the EBMeDS system in the EHRs of a small group of Belgian general practitioners and a quantitative and qualitative assessment of acceptance of the system has already been performed. The early adopters that responded to this survey reported a positive attitude toward this system and definitely intended to continue using it [
The prerequisite for the functioning of seamless information flows is accurate and sophisticated recording of data in the EHRs, with structured data entry, facilitated by interface terminology systems, to bridge the gap between every day medical communication and international nomenclatures and classification systems [
The LMF is used for the unilingual end user terminology and Terminological Markup Framework (TMF) is used for the multilingual reference terminology. The two resources could be managed with a Web-based Semantic Media Wiki Application, and published in Linked Open Data. Correct medical registration will optimize the functioning of automated decision support alert systems by providing both correct triggering of alerts and comments, only when necessary, and not when known exceptions are present. The coding behind correct medical registration can also provide the pathway to focused clinical questions, when practice problems surface, which halt the routine flow of the consultation process. The answer to these clinical questions can then be seamlessly provided on the guideline platform.
To facilitate information retrieval for the users, we also planned additional tools such as:
The development of a Patient-Intervention- Comparison-Outcome interface and the complex indexing of specific practice recommendations to make the database searchable for specific patient problems [
The further elaboration of the navigation menu according to taxonomy of generic clinical questions and organization of the content as a strategy to route general user questions to more specific clinical questions and focused recommendations [
Specialty-specific indexing of information for several groups of allied health personnel and medical specialists.
The Institute of Medicine defines Health Care Quality as the extent to which health services provided to individuals and patient populations improve desired health outcomes [
Professional behavior of caregivers consists of evidence-based practice, reflecting on their own performance, accountability, and continuous professional education. Information and communication technology (ICT) plays an important supporting role in improving the quality of care. The ICT can increase efficiency through the efficient management of resources and administrative simplification. But ICT also plays a crucial role in the effective use of treatments and patient safety (evidence-based practice) in promoting the participation of the patient and may ensure better continuous professional development and education of the caregiver.
While EBMPracticeNet is currently in “work in progress” state, the potential value of the project is great. The link between all the EHRs from different vendors with a national database held on a single platform and controlled by all EBM organizations in Belgium is the strength of EBMPracticeNet. As yet, we are not aware of an identical project in the world. The collaboration of government, health care providers, EBM partners, and vendors of EHRs is unique. With the help of national leadership in standardization and the collaboration of medical software vendors, standards can be set to facilitate the integration of different types of evidence-based and clinical information. This collaboration stems from the free delivery of independent content by the government and EBM providers, and the creativity of software vendors in creating applications for this content. A mechanism for gradual improvement of the resulting systems is the accreditation process of medical software for EHRs in which the Belgian eHealth authorities verify if the EHR fulfills the certification criteria for EHR technology.
Since Belgian EBM organizations are now formally united in EBMPracticeNet, the potential for collaboration increases. This will help reduce duplication in efforts during the development of EBM information. International collaboration on evidence synthesis and guideline development methodology, standardization of data structures, and ontologies (terminologies and their relationships) for evidence, clinical questions, recommendations and decision support, facilitated sharing of knowledge resources, and tools for staying informed about evidence will further enhance the impact of EBMPracticeNet. In addition to the EBM guidelines and EBMeDS editorial teams in Finland and Austria, the collaborative network consists of the Cochrane Collaboration, Guidelines International Network, and the GRADE Working Group.
To sustain funding for this project the impact on the quality of care will need to be demonstrated, if possible on patient outcome. The development of a research agenda is needed to verify the impact on changing clinical practice.
Business Process Model and Notation
Belgian Centre for EBM
evidence-based medicine
Evidence-Based Medicine electronic Decision Support
electronic health records
International Classification of Diseases, Tenth revision
International Classification of Primary Care, Second edition
information and communication technology
National Institute for Health and Disability Insurance
Lexical Markup Framework
Medical Subject Headings
National Health Service
randomized controlled trial
Terminological Markup Framework
The authors are grateful for the funding support received from the national health insurance institute (INAMI-RIZIV).
The authors also wish to acknowledge the editorial team and board, the medical proofreaders and validators, the technical experts, and the following organizations for their vision, expertise, and contributions to EBMPracticeNet: AFMPS-FAGG, BAPCOC, BCFI-CBIP, CEBAM Digital Library for Health, CEBAM, CIPIQ-S, Departments of General Practice KULeuven , VUB , UA , UGent , ULG , ULB, UCL, EHEALTH, Faculty of Applied Language Studies Hogeschool Gent, FARMAKA, Federal public service Health, Food Chain Safety and Environment, INAMI-RIZIV, ISCIENTIA IVS, KCE, MINERVA, NVKVV, PRISCI, SMALS, SSMG, VEV, WVVK, and the Working group of Belgian producers of EBM information.
RVS is the salaried Project Leader, SVDV is the salaried Editor-in-Chief, and AH is a salaried Editor for EBMPracticeNet. BA and SG are founding members of EBMPracticeNet. BF is president of the EBMPracticeNet board and is salaried editor of EBMPracticeNet. DR is board member of CEBAM. IK is the salaried Chief Editor of EBM Guidelines and EBMeDS decision support service.