The Karma system is currently undergoing maintenance (Monday, January 29, 2018).
The maintenance period has been extended to 8PM EST.

Karma Credits will not be available for redeeming during maintenance.

JMIR Research Protocols


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 10.07.13 in Vol 2, No 2 (2013): Jul-Dec

This paper is in the following e-collection/theme issue:


    EBMPracticeNet: A Bilingual National Electronic Point-Of-Care Project for Retrieval of Evidence-Based Clinical Guideline Information and Decision Support

    1EBMPracticeNet, Leuven, Belgium

    2Belgian Centre for Evidence Based Medicine (CEBAM), Leuven, Belgium

    3CEBAM Digital Library for Health, Leuven, Belgium

    4Academic Center for General Practice, Department of Public Health and Primary Care, Katholieke Universiteit Leuven, Leuven, Belgium

    5ZNA Hospital Network Antwerp, Antwerp, Belgium

    6Duodecim, Helsinki, Finland

    *all authors contributed equally

    Corresponding Author:

    Stijn Van de Velde, PhD


    Blok J bus 7001 300

    Kapucijnenvoer 33

    Leuven, 3000


    Phone: 32 016332697

    Fax:32 016337480



    Background: In Belgium, the construction of a national electronic point-of-care information service, EBMPracticeNet, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The collaboration of the government, health care providers, evidence-based medicine (EBM) partners, and vendors of electronic health records (EHR) is unique to this project. All Belgian health care professionals get free access to an up-to-date database of validated Belgian and nearly 1000 international guidelines, incorporated in a portal that also provides EBM information from other sources than guidelines, including computerized clinical decision support that is integrated in the EHRs.

    Objective: The objective of this paper was to describe the development strategy, the overall content, and the management of EBMPracticeNet which may be of relevance to other health organizations creating national or regional electronic point-of-care information services.

    Methods: Several candidate providers of comprehensive guideline solutions were evaluated and one database was selected. Translation of the guidelines to Dutch and French was done with translation software, post-editing by translators and medical proofreading. A strategy is determined to adapt the guideline content to the Belgian context. Acceptance of the computerized clinical decision support tool has been tested and a randomized controlled trial is planned to evaluate the effect on process and patient outcomes.

    Results: Currently, EBMPracticeNet is in "work in progress" state. Reference is made to the results of a pilot study and to further planned research including a randomized controlled trial.

    Conclusions: The collaboration of government, health care providers, EBM partners, and vendors of EHRs is unique. The potential value of the project is great. The link between all the EHRs from different vendors and a national database held on a single platform that is controlled by all EBM organizations in Belgium are the strengths of EBMPracticeNet.

    JMIR Res Protoc 2013;2(2):e23



    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100


    There has been an explosive growth in scientific evidence, with 75 trials and 11 systematic reviews being published a day [1]. However, the use of literature remains suboptimal because health care providers do not have the time to search actively for information or have difficulty finding the relevant evidence [2]. Too often, clinical decisions are based only upon experience, unsubstantiated routine, and opinions of experts [3,4].

    Properly designed information retrieval and clinical decision support systems are now being promoted as a Global Positioning System to prevent health care providers from getting lost in clinical practice [4-6]. Such systems either use technologies, where users can pull clinical information from a database or use services that push information through reminders or alerts [7,8].

    Structured clinical guidelines and computerized clinical decision support systems have the potential for improving the quality of care [9-13]. However, most of these evaluative studies focused on physician behavior or process of care rather than on the evaluation of effects on patient outcomes. Despite modern technology, it remains an important challenge to implement such systems effectively [6,14]. To be successful, it is essential that systems make clinical decision-making easier by integrating it in the clinician’s workflow the moment the clinician meets the patient. Recommendations should be generated on the fly and be action-oriented rather than mere assessments [14]. Above all, alert fatigue has to be avoided [15].

    In Belgium, the construction of a national electronic point-of-care information service, EBMPracticeNet, was initiated in September 2011 to optimize quality of care by promoting evidence-based decision-making [16]. All Belgian health care professionals get free access to an up-to-date database of validated Belgian and international guidelines incorporated in a portal that also provides Evidence-based medicine (EBM) information from other sources than guidelines, including a computerized clinical decision support linked to the electronic health record (EHR). The primary focus is on general practitioners at this moment. In the second phase, there will also be a multidisciplinary focus on allied health personnel and specialist physicians. The platform is also available to patients, albeit for now not in layman’s language.

    The aim of this paper is to describe the development strategy, the overall content, and the management of EBMPracticeNet.



    EBMPracticeNet was officially founded as a non-profit organization in 2011, and originated from grass roots gathering of EBM-producing organizations. This integrative cooperation project was inspired by CEBAM, the Belgian Centre for EBM and the Belgian branch of the Dutch Cochrane Collaboration. EBMPracticeNet is open to four types of organizations, namely independent producers of EBM information, disseminator organizations, user organizations, and governmental public health departments. Funding comes from the national health insurance institute (National Institute for Health and Disability Insurance, INAMI-RIZIV). The project fits within the broader range of eHealth initiatives in Belgium. The goal is to share national or adapted international EBM guidelines on one certified and secured platform (eHealth), easily accessible to different vendors of EHRs. The information structure of EBMPracticeNet is presented in Figure 1.

    The guidelines database is a mix of national and international guidelines. First of all it entails about 50 Belgian guidelines, regularly updated by local Belgian Guideline producers. We supplemented this with a comprehensive database of international clinical guidelines, with the intention to adapt the content to the Belgian context. Several candidate providers of comprehensive guideline solutions were evaluated on the basis of a published review [17]. The EBM Guidelines of Duodecim Medical Publications was selected [18]. The main advantages include the strong EBM methodology, the large number of guidelines, the quality of keywords indexation, the focus on the first line level of health care, the good editorial quality, and efforts to keep the database up-to-date [17,19]. An additional argument was the formal accreditation by the UK National Health Service (NHS) of the Duodecim approach to the production of guidelines, after a formal evaluation based on the AGREE criteria [20-22]. The latter is important to demonstrate that the recommendations are trustworthy [23,24]. For the same reason, all Belgian guidelines need to be formally validated by the Belgian Centre for EBM (CEBAM), before they can be published in the database.

    Figure 1. EBMPracticeNet information structure.
    View this figure

    Translation and Adaptation to Belgian Context

    Duodecim EBM Guidelines comprise of nearly 1000 clinical guidelines and one million words. Translating this comprehensive set and adapting it to the Belgian context posed a huge challenge for EBMPracticeNet. Our first step in this process involved the translation from English to Dutch and French, two official languages of Belgium. This translation project was undertaken in cooperation with our technical partner, Iscientia IVS, a broker company for scientific information, and provider of technical platforms for scientific bibliographic information. The process was supervised by an academic institution of Applied Language Studies (Hogeschool Gent). First, the translation software, SDL Trados Studio, produced a machine translation, which was then post-edited by a human translator, a medical proofreader and a validator [25]. Machine translation was supported by a translation memory database and a terminology management system to ensure the consistent use of terms. The terminology management system used multi-term files, based on MeSH translations developed within the faculty of Applied Language Studies and based on medical glossaries available with the faculty [26]. Apart from the terminology system, the translators also consulted the InterActive Terminology for Europe multilingual term base from the European Union [27]. The post-editing of the human translators was captured in the translation memory, which increased the efficiency throughout the process. It took approximately 2000 translation hours, 500 proofreading hours and 200 validation hours per language, spread over 15 months, to accomplish the translation of the full set of guidelines. The validated versions of the translated guidelines were re-entered in the translation memory database, which will improve the quality of the translation when future updates of the international guidelines have to be translated.

    Next is the adaptation of the Duodecim EBM Guidelines to the Belgian context. This process was guided by a preliminary prioritization effort. First, an inventory was made of 50 validated Belgian guidelines. The content of these Belgian guidelines was transformed in the format of the Duodecim EBM guidelines to replace the International guidelines. This involved a two-step transformation with first the production of a structured summary, and then legacy conversion into the EBM Guideline Extensible Markup Language format. Second, an additional priority list of 50 clinical topics was drafted by taking into account clinical information need (based on user surveys and epidemiological reason-for-encounter data in sentinel practice networks) and priority areas indicated by key stakeholders. The relevant guidelines for this list of priority clinical topics are now screened in close consultation with the core community of experts from the Belgian producers of EBM information [28]. The guidelines are categorized in three groups: no need for adaptation for guidelines that are trustworthy and in accordance with the Belgian context, and guidelines requiring minor or major adaptation. The guidelines requiring contextual adaptation will undergo a tailored ADAPTE procedure in collaboration with the Belgian experts [29]. Content-related remarks are followed up by Duodecim experts and editors so that the Belgian process of adaptation is synchronized with the Finnish updating cycle. We also reached out to the Belgian University Centers for Primary Care, responsible for Vocational training in General Practice, to involve a substantial group of General Practice trainees in the screening and adaptation process of additional guidelines as part of their thesis.

    For the screening of the larger set of low-priority guidelines, we have invited stakeholder organizations and volunteers to screen the remaining guidelines only with regard to their compliance with the Belgian context. This process of screening and adaptation has to be completed by the end of 2015. Meanwhile, all the guidelines will be published with an explicit indication that adaptation to the Belgian context is under way. This indication will be removed when the screening (and if necessary the adaptation) is performed.

    The Duodecim EBM Guidelines are revised continuously at a rate of 80 updated guidelines each trimester. This means that we need to keep up that pace of translating the updated information and screening when, and if, updated recommendations need adapting.

    Content Organization and Navigation

    The user can search for information with a search engine or can browse for information using a navigation menu based on the conditions included in the database. The content and search engine are organized in such a way that they can be used during the patient encounter in a minimum of time. Implementation of direct access to the EBMPracticeNet is now a criterion in the accreditation of EHR software in Belgium.

    The information in the original Duodecim EBM Guidelines database is indexed with the International Classification of Primary Care, Second edition (ICPC-2) and the International Classification of Diseases, Tenth revision (ICD-10) codes and Medical Subject Headings (MeSH) terms. The available ICD-10 and ICPC-2 codes were submitted to scrutiny and validated for the diagnostic part of the guidelines. However, additional coding is needed for process and outcome aspects. An available translation of MeSH terms in Dutch and French will be used to translate the English MeSH terms [30,31].

    Based on the 1000 clinical guidelines, we will build a multilingual terminology database. Selected words and phrases will be attributed to each guideline to create an effective search engine to search the database in Dutch, French, and English.

    Portal for Other Evidence-Based Medicine Information

    In addition to rapid access to practical recommendations at the point-of-care, this portal also organizes the flow of the clinical information in a chain of evidences that allows users with specific clinical questions to move efficiently from guidelines to systematic reviews and primary studies. As the time invested in these searches increase, they will typically be accessed outside the patient consultation.

    • With the Duodecim EBM guidelines comes a collection of more than 4000 evidence summaries. These evidence summaries are graded statements with a short description of systematic reviews or original research [18]. In addition, the EBM guidelines information corpus includes images and videos, which are helpful in making diagnoses and carrying out procedures.
    • E