Public health knowledge management aims to systematically manage tasks and support decision making, which view implicit and explicit knowledge as key strategic resources [24]. Knowledge management needs storage, retrieval, and utilization of public health knowledge. We propose the ontology-driven knowledge management approach as shown in Figure 1, which decomposes public health documents to elements of knowledge and stores them in an ontology, namely, the Public Health Document Ontology (OntoPH). OntoPH was developed using ontology competency questions as guidance. Grüninger and Fox [25] state that an ontology should answer competency questions proposed based on the motivation of the ontology. Competency questions define the terminology and specify the definitions and constraints of the terminology. Knowledge is modeled using the terminology and retrieved via semantic rules. An inference engine accesses knowledge models and assembles and manipulates elements of knowledge in the ontology to draw conclusions about EID preparedness and response.

Public health knowledge is mainly preserved in public health documents, which include guidelines, procedures, and academic publications. They are the most important media to share, store, and manage knowledge because they are vetted, high-quality, generated by an authoritative content source, verifiable by a trusted source, and up to date and regularly updated [5]. In order to support decision making, OntoPH’s corpus should meet at least 2 requirements: breadth and depth. Breadth means the corpus should cover many, if not all, fields that are involved in public health decision making. Depth means the corpus should contain not only global-level guidelines but also local-level procedures.

The first task is to represent knowledge preserved in public health documents. Effective knowledge storage and retrieval requires a knowledge representation, which addresses both hierarchical complexity and semantic heterogeneity. The hierarchical complexity of public health knowledge is rooted in the multiple layers of public health activities. Public health practitioners need different chunks of knowledge in various contexts to prepare for and respond to EIDs. Health workers in the clinic, for example, demand knowledge about disease diagnosis, whereas the department of health wants to know how to manage and coordinate. Knowledge always serves some purposes. The health workers’ knowledge leads to accurate diagnoses. The department of health’s knowledge achieves effective emergency response. Multiple layers of public health activities are linked via their purposes. For example, to better respond to emergencies, departments of health require the health workers to diagnose diseases more effectively.

Semantic heterogeneity, on the other hand, is the result of the cross-reference of public health knowledge, which is a mixture of various fields such as medical science, epidemiology, biology, and engineering [8]. For instance, the knowledge of physician training lies in the intersection of medical science (ie, what skills to train) and management science (ie, how to train). Nonetheless, the 2 aspects share the same purpose (ie, training physicians for better EID preparedness). A recent study by Venkatasubramanian and Zhang [26] finds that complex system activities usually have 4 common purposes: communication, decision making, processing, and sensing. Training, as part of education, is an important type of implementation activities.

One can resolve both hierarchical complexity and semantic heterogeneity by identifying the purpose of knowledge, for a piece of knowledge could serve different purposes under different conditions. Venkatasubramanian and Zhang [26] identify the importance of means-end relation in complex system risk management and propose a systems engineering framework to explicate the relation. Adopting this idea, our approach models elements of knowledge based on their means-end relations. We use teleological functions to represent the purposes of knowledge elements. Unlike mathematical functions that map a set of inputs onto a set of permissible outputs, teleological functions emphasize the means to realize a goal by indicating the common purpose between 2 connected entities. The 4 common purposes induce 4 types of teleological functions. A function-based knowledge representation has been used in many fields including engineering [27-30] and data science [31].

To develop such a function-based knowledge representation, we first classify public health documents into 2 categories, general documents that contain general public health principles and specific documents that store evidence-based procedures. There exists a gap between the 2 types of documents: general documents are usually too general to implement, whereas specific documents are mostly event-specific thereby limiting their usefulness for new events. We organize knowledge of general documents as a teleological function of that of specific documents: knowledge_{general doc}= f(knowledge_{specific doc1}, knowledge_{specific doc2},...), where f is a teleological function. Specific activities expand a general guideline with specific recommendations. For example, since the 2009 influenza A H1N1 pandemic, many specific documents have discussed vaccination preparedness and distribution [32,33]. The World Health Organization (WHO) also has issued general guidelines for vaccination preparation during the pandemic [34]. The function vaccination describes activities related to vaccination preparedness and distribution. Therefore, the equation can be rewritten as knowledge_[34] = vaccination (knowledge_[32], knowledge_[33]), meaning that WHO guidelines about vaccination can be expanded with specific activities and, hence, bridge the gap. The function-based knowledge representation is depicted as a tree structure shown in Figure 2. The root of the tree is a public health document, and the leaves are the event-based procedures. A general document (eg, g1) contains general knowledge expressions (eg, ge1.1 and ge1.2). A general knowledge expression specifies a teleological function. For instance, the WHO guideline [34] points out roles of the health and nonhealth sectors in vaccination sharing and distribution activities. We can label this knowledge expression with a function vaccination (eg, f2). Specific guidelines (eg, s2) elaborate the teleological functions and define many specific knowledge expressions (eg, se1.2). Specific knowledge expressions can further indicate subfunctions (eg, sf1.2), which include detailed procedures and instructions. Unlike specific procedures, teleological functions are event independent. The same functions can apply to different events with similar fundamental lessons. The tree structure demonstrates how general documents and specific documents are linked via teleological functions. The function-based knowledge representation handles the hierarchical complexity through the tree structure of documents and manages the semantic heterogeneity by grouping distinct activities under the same function. Teleological functions define the scope and intention of the specific documents. They let a specific document elaborate a general document by adding actionable items.

The corpus, with 135,946 words in total, consists of the US Code [45], federal level regulations [32,33,46,47], international health regulations [34,44,48,49], and pandemic evaluations of outbreak responses [50,51]. They cover all types of public health documents aforementioned. The US Code is the generic legal document, which ensures that the ontology aligns with laws. The federal regulations and the international health regulations are guidelines regarding surveillance, transportation, and preparedness. The evaluations are chosen per disease. H1N1 and West Nile Virus are 2 specific diseases chosen for illustration. These 2 cases were selected because they are well-studied recent emerging diseases with an impact on health resources both locally and globally. In addition, their impacts on health and geographical coverage are both significant. We wanted to evaluate case examples where the primary infection risk is associated with different infection transmission routes in order to evaluate the potential for having a unified framework for EIDs.

We implemented a hybrid concept extraction approach. NLP methods are used to preprocess the corpus. By removing stop words and tagging the parts of speech, one can extract meaningful and most frequent terms and relations using text mining tools like KHCoder [52]. The classification work is done manually with 2 domain experts reviewing every term and relation and deciding their descriptions and constraints. OntoPH is built upon these terms and relations. Domain experts and ontology engineers work collaboratively to select and annotate documents. Such a team-based method has been used extensively in many scientific studies and applications, such as hazard and operability analysis in chemical engineering [53]. Such a team should be as small as possible while maintaining sufficient expertise. In a series of meetings, team members work together to select documents. Conflicts must be resolved before the list of documents is finalized. Each domain expert annotates a part of the corpus and reviews others’ annotations. This practice, therefore, keeps the corpus and annotation as objective as possible.

The quality of ontology is critical. It affects not only the quality of reasoning results but also the effectiveness of the application. Ontology can be evaluated on many aspects, namely, vocabulary, syntax, structure, semantics, representation, and context [24]. Extensive research has been conducted to formally evaluate the quality of ontologies [24,54-57]. Among these methods, we follow the Ontology Quality Evaluation Framework (OQuaRE) approach [55], which adapts the International Organization for Standardization standards for Software Quality Requirements and Evaluation. OQuaRE assesses 6 characteristics and 39 subcharacteristics of an ontology using quality metrics. Quality metrics are composed of primitive and derived measurements. Primitive measurements are metrics that can be measured directly on the ontology, such as number of classes, number of relations, etc. Derived measurements are combinations of some primitive ones [55]. With a scale of 1 to 5 (1=not acceptable and 5=exceeds the requirement), it rates every aspect of an ontology. The final score is the arithmetic average of individual scores of all characteristics. The details of this method can be found in Duque-Ramos et al [55]. We include 30 out of the 39 subcharacteristics in our evaluation. The other 9 subcharacteristics, which require expert subjective assessment, are excluded. The evaluation results of the OntoPH core ontology are presented in Table 1. The evaluation indicates that the OntoPH core ontology is satisfactory, with an average score of 4. Problems have been found on redundancy and controlled vocabulary, mainly due to the relatively small corpus size.

The possibility of using ontology and semantic reasoning in public health decision making has been recognized in literature [58]. In this work, we adapt this idea and our previous experience in knowledge management in the pharmaceutical industry [59] to derive a detailed methodology on how to develop such a tool. We introduce the systems engineering–inspired ontology-driven framework for public health knowledge management. We demonstrate how complex and heterogeneous public health knowledge can be modeled and stored in an ontology. Previous work has focused on local activities, such as activities within a health care network [60]. OntoPH extends the scope from local level to global/national level by focusing on general documents.

OntoPH’s strength is threefold. First, it stores public health documents knowledge as classes, relations, and instances. Public health documents, including guidelines, procedures, and academic publications, are important sources of knowledge. Even though medical records, geographic information system data, and disease information have been studied and stored in ontologies [60,61], to our knowledge, there is no ontology for public health documents. OntoPH provides this missing piece of public health knowledge management. Second, we present a flexible knowledge management framework. OntoPH implements a modularized structure, which ensures its extensibility. For example, the space-time module can be extended using time ontologies [60,62] and World Wide Web Consortium spatial ontologies [63]. It is also possible to add new modules. If disease information is needed, we can create a new disease module, which inherits the disease ontology [61]. This modularized structure makes OntoPH a potential generic public health knowledge center. Third, OntoPH can manage the hierarchical complexity and heterogeneity of public health knowledge. Elements of knowledge are effectively organized by the teleological functions that highlight the means-end relations.

This framework is most useful in low- and middle-income countries (LMICs). Lack of resources and public health experts in LMICs usually makes a knowledge management system difficult to implement. Nonetheless, OntoPH’s general knowledge is widely applicable. By expanding the data sources to include LMIC-specific knowledge [64] and connecting with other ontologies [61-63,65], OntoPH would become a useful tool to help LMICs respond to an outbreak quickly, both at the national and local levels.

Potentially, OntoPH can support decision making by answering user queries. For example, given an outbreak scenario, a user could list questions regarding disease identification, transmission prevention, disease control, and risk mitigation. With enough prestored knowledge, OntoPH could answer the list of questions by producing logical assertions with respect to each question.

At this stage, however, there are some limitations. First, the training document corpus is relatively small. Only 5 general documents and 7 specific documents are prestored due to the manual annotation constraint. It will require a concerted effort to annotate and develop a more extensive public health knowledge base for widespread application. Nonetheless, the current corpus is comprehensive enough for proof of concept. Second, the selection of documents is subjective. When the corpus size is small, the accuracy of reasoning results is dependent on the document selection rather than the knowledge base. Increasing the size of the corpus and precise query statement will improve reasoning accuracy in general. In addition, rule-based reasoning has its intrinsic limitations—semantic rules are subjective. SWRL rules rarely allow ternary relations, and that limits the power of the SWRL representation. Third, the current framework is restricted to public health documents, which lack information from various data sources, such as geographic information system data, news articles, social media feeds, etc. This limits OntoPH’s real-time usage. Moreover, current knowledge representation would not be able to capture knowledge in research articles that do not fit in the knowledge model. However, the basic and domain ontologies, such as space-time, resource, role, and agent modules, contain fundamental public health knowledge, therefore, making the knowledge framework extendable to cover research articles. This, of course, requires further study of new knowledge representation. Potentially, a research article knowledge expression module could be developed and incorporated into OntoPH.

Future work will address the limitations and evaluate OntoPH’s reasoning capacity. Adopting artificial intelligence techniques would significantly reduce the human effort, and, thus, get rid of many of the limitations. Specifically, a term extraction module implementing NLP techniques such as topic modeling would enable automated concept classification of public health documents, reducing the amount of work required for annotation. Enriching data sources will improve OntoPH’s ability for real-time response. We plan to expand the corpus incorporating expert opinions. A survey for eliciting expert feedback on what to include in the corpus will be conducted. A systematic literature review on effectiveness of policy and interventions could help us determine what documents to include.

To evaluate this method, we will collect a list of general queries regarding general EID preparedness and response from public health experts and practitioners. Moreover, we will test OntoPH’s reasoning capacity on hypothetical outbreaks. These full-scale case studies will provide us with valuable information on how to improve the usage and accuracy of OntoPH decision support.

In recent decades, many EID outbreaks and epidemics have resulted in considerable human disability and mortality, in part due to ineffective coordination or slow response at the start of the outbreak. Responding to EID outbreaks is intrinsically challenging due to the uncertainties associated with EIDs, specifically level of risk and potential for impact of its spread in a population. During an outbreak, evidence-based public health policies developed by public health authorities, legislators, and other government officials facilitate the implementation of a strong public health response. However, there are structural and political forces that prevent decision makers from making evidence-based policies in response to outbreaks. Therefore, it is necessary to have in place a mechanism to easily identify evidence in order to evaluate the consequences of public health or policy actions recommended to address these public health emergencies. An ontology framework for public health outbreak response will cut the time spent aggregating expert opinions during the initial stages of an outbreak. It would also assist public health administrators and government officials on next steps based on individual- and systems-level factors associated with the outbreak.

Our approach is a potentially effective methodology for EID preparedness and response. It manages complex knowledge via a function-based knowledge representation. It introduces a systematic way of storing, retrieving, and using public health knowledge. Accuracy and comprehensiveness of the ontology can be improved as more knowledge is stored. We advocate the public health community work toward the goal of developing a Gene Ontology-like [66] knowledge sharing standard. OntoPH demonstrates the possibility of knowledge management for EID emergency preparedness and response.