Degenerative cervical myelopathy (DCM) arises when degenerative changes of spinal structures cause myelopathy of the cervical spinal cord [1]. These degenerative changes include spondylosis, disc prolapse, hypertrophy, calcification, and ossification of the posterior longitudinal ligament and ligamentum flavum [1]. Ultimately, this results in stenosis of the spinal canal leading to cord compression, mechanical stretch, repetitive microtrauma, and chronic reduction in cord blood flow [1,2]. A complex pathological cascade follows with neuroinflammation, demyelination, neurodegeneration, and gliosis, resulting in the clinical entity we know as myelopathy [1,3,4].

The prevalence of DCM has proven difficult to ascertain owing to the novel umbrella term, difficulty in diagnosis, and the relative paucity of data [1]. Nevertheless, it is widely accepted as the most common cause of adult myelopathy worldwide [5]. DCM is not only very prevalent but also quite disabling, with the quality of life scores (36-Item Short Form Health Survey) in patients with DCM being lower than those in patients with most other common conditions, with heart failure and sciatica being identified as the only 2 conditions with lower scores [6].

Despite this, there remains no specific term or diagnostic criteria in the International Classification of Diseases 11th Revision (ICD-11), which encompasses the related and often coexisting conditions covered under the DCM umbrella [7]. Similarly, there exists no Medical Subject Headings (MeSH) term for PubMed (or an equivalent grouping index term for other databases) for DCM or its constituent terms (Textbox 1) [7]. MeSH terms improve the precision and efficiency of literature searches [8]. The hierarchical structure arrangement of MeSH trees allows for narrower terms to fall under the MeSH term heading and for search engines to consider other terms as MeSH synonyms [9]. This has proven particularly useful in medical terms that follow umbrella structures [9], for example, the MeSH Spinal Cord Injuries encompasses terms such as spinal cord transection, traumatic myelopathy, and spinal cord contusions and includes useful subheadings such as etiology, diagnosis, and surgery. This structure is likely to be useful in DCM terminology, which encompasses terms such as cervical spondylotic myelopathy and ossification of the posterior longitudinal ligament [1]. The relative novelty of the term DCM does not entirely explain its lack of MeSH terms. In 2019, 421 new MeSH terms were added to the Medical Literature Analysis and Retrieval System Online (MEDLINE)/PubMed database, some of which are in reference to other equally novel terms, such as emerging monoclonal antibody-based therapies and small molecule inhibitors [10]. Indeed, the use of free-text searches alone of DCM returns imprecision in the form of overlapping terms, including the subjects of noncervical myelopathy, noncervical spine degenerative changes, and gynecological cervix [7].

The lack of a consistent index term and MeSH term for DCM makes searching the literature imprecise and inefficient. This is particularly crucial when considering the importance of thorough systematic reviews and meta-analyses in reducing research wastage [11]. Indeed, in 2010, an estimated US $240 billion was spent on biomedical research, with an estimated 85% of this research wasted, resulting in no clinical translation or benefit [11,12]. Reviews of the literature increase research efficiency by preventing research duplication and directing future primary research [11]. However, previous surveys have indicated that over half of the clinical trial designers may be unaware of all the existing major reviews relevant to their study design [13]. The omission of this crucial step in informing trials has led to countless numbers of trials with inappropriate design, with one series highlighting that up to 75% of trials without the mention of systematic reviews or meta-analyses informing their protocols had trial designs that were considered inadequate [14,15]. In addition, systematic reviews and meta-analyses are important in preventing duplication of existing knowledge and in putting the results of trials into the context of existing literature so that the clinical relevance of findings is more interpretable [15].

Currently, the process of conducting systematic reviews and meta-analyses in many fields is laborious and inefficient [15,16]. This study aimed to illustrate the difficulties encountered when attempting to carry out a comprehensive and accurate evidence search in the field of DCM by identifying the key sources of imprecision and quantifying their impact.

Studies concerning DCM within the last 5 years were initially identified using a search filter/hedge, which has been previously validated for DCM and has returned a 100% sensitivity in DCM datasets [7]. Search strategies used in these systematic reviews and meta-analyses were compared with the search strategies of the validated hedge. The validated strategy was combined with the strategies that have been actively used by authors, forming our example search strategy (Textbox 2). Effectively, this resulted in the addition of the terms Degenerative Cervical Myelopathy and Ossification of Posterior Longitudinal Ligament to the strategy. This approach, rather than the exclusive use of the search filter, was chosen because it most closely aligned with the current search practices in DCM.

The outcomes measured during Phase 1 and Phase 2 searches were as follows:

This search strategy returned 3439 results, refined to 175 results using the above filters and with 1 duplicate being subsequently removed. The categorization of the remaining 174 results is summarized in Multimedia Appendix 1. Of note, 18.4% (32/116) results fitted the inclusion criteria, totaled in the DCM category of Multimedia Appendix 1. Of these, 9.8% (17/116) DCM studies were systematic reviews or meta-analyses, with the 15 other studies consisting of case reports and narrative reviews.

With regard to the overlapping terms when considering non-DCM search results relating to the spine (116/174, 66.6% of results), the most common categories involved were spinal neoplasms (25/174, 14.4%), spinal cord injury (15/174, 8.6%), and infectious diseases of the spine and central nervous system (CNS; 15/174, 8.6%). When considering nonspinal categories (26/174, 14.9% of results), diseases of the nervous system—cerebral diseases (9/174, 5.7%)—and disorders of the urological tract and male genital tract (3/174, 1.7%) were the most commonly encountered.

As stated above, this Phase 1 search strategy failed to identify our chosen index articles [1,21]. Resultantly, this search was refined via removal of the adults (>19 years old) and human filters and thereafter termed the Phase 2 strategy (Textbox 3), searched on January 17, 2019.

This search strategy returned 24,166 results, refined to 537 results using filters. Of these, 2 duplicates and 1 letter to editor publication were subsequently removed (Figure 1). The remaining 534 studies are categorized in Table 1. We found that 32.9% (176/534) of results fitted into the DCM category of Table 1. Of these, 18.2% (97/534) DCM studies were systematic reviews or meta-analyses, with the 79 other studies consisting predominantly of case reports and narrative reviews.

Of note, our search strategy was formed from an amalgamation of strategies used by previous authors in the field and validated PubMed hedge. Resultantly, the search terms used included 2 additional terms (DCM/Degenerative Cervical Myelopathy and OPLL/Ossification of Posterior Longitudinal Ligament) to the original validated hedge by Davies et al [7]. The addition of these terms made no difference to the search results during Phase 1 or Phase 2.

Our study served to replicate an important step in research synthesis and evidence-based clinical practice, identifying systematic reviews or meta-analyses in the field. It was hypothesized that, without an agreed index term, MeSH, or an equivalent and accurate indexing of articles, this attempt at comprehensively searching the literature would be inefficient and imprecise. Specifically, only 18.2% (97/534) of search results concerned DCM systematic reviews or meta-analyses and commonly used PubMed search filters (such as adult) stratified studies incorrectly. Moreover, it is clear that expanding the search with related articles functions and searching additional databases will identify additional relevant studies. These results taken together indicate that systematic search in DCM is currently extremely labor-intensive.

The PubMed filters of systematic reviews, reviews, and meta-analyses, with proven satisfactory sensitivity and specificity [19,20], were applied to our search terms. The standard against which imprecision was judged was the percentage and number of DCM systematic reviews or meta-analyses found. This aimed to replicate common occurrence in clinical practice and the initial steps of research synthesis. It also served as a pragmatic approach to assessing imprecision. DCM systematic reviews and meta-analyses were 18.2% (97/534) of studies returned in our search (Phase 2), with principal categories of imprecision including spinal cord injury and spinal neoplasms.

The need to exercise caution when applying other generic filters in the field of DCM is illustrated in the results of the Phase 1 search. This included the additional filters of adult and human and a search that was less sensitive or specific than Phase 2 (searching without these filters applied). However, the addition of the English filter did not affect the number of DCM systematic reviews or meta-analyses found in our search. This is unfortunate as these filters have the potential to make this already labor-intensive process more efficient but fail to do so in DCM PubMed literature.

The related articles function on PubMed’s displayed utility in the setting of both low- and high-sensitivity searches. In Phase 1, it was used to find 5 additional DCM systematic review and meta-analysis studies after searching the original 32 articles—1 new DCM systematic review or meta-analysis was found per 6.4 articles examined. In Phase 2, 6 extra articles were found via this extended search, equating to 1 new DCM systematic review or meta-analysis found per 1.67 articles examined. Thus, the common practice of using this function for the literature search is justified and recommended. In addition, it is important to note that the EMBASE database composed 14% (16/114) of the total EMBASE plus MEDLINE DCM systematic reviews and meta-analysis yield of our search and 9.2% (18/195) of the total EMBASE plus MEDLINE DCM category results. This reiterates the potential value of searching multiple literature databases while performing systematic reviews or meta-analyses in the field of DCM, and DCM reviewers should be cognoscente of this [11].

The findings of our search reflect previous studies discussing systematic review or meta-analysis retrieval in other fields. Specifically, they mirror the search precision generated by a DCM hedge. Overlapping terms and general imprecision have spurred the creation of search hedges, aiming to increase search efficiency when performing comprehensive literature retrieval [23]. However, there is a broad range of sensitivities and specifies that is achieved using these hedges [23,24]. This is compounded by the inaccuracy of generic search filters, for example, the cross-sectional studies filter [25]. There is an agreement that root problems to such search inefficiencies included interindexer inconsistency when labeling studies and a lack of natural language processing terms such as MeSH terms [26,27]. Regardless, searching of multiple databases and using related articles-type functions are widely accepted for their utility [28]. Importantly, the choice of databases must be carefully considered. For example, Google Scholar is another option commonly considered for literature retrieval. It holds advantages in its simplicity, familiarity, and ability to search a broader area of the literature (including multiple medical libraries and preprint articles) [29]. However, it has been criticized for being less comprehensive, less precise, and less sophisticated (in terms of advanced search functions and controlled vocabulary) [29,30]. Therefore, we elected dedicated literature search databases for the purposes of identifying imprecision.

As the rate of our primary research synthesis exceeds our ability to review it [31], it is imperative that our methods for systematically reviewing and analyzing data emphasize efficiency. Over the last decade, an average of 700,000 to 850,000 articles per year were published in MEDLINE [32], whereas 2500 systematic reviews are published yearly [12]. It is estimated that 10,000 Cochrane systematic reviews would be needed to sufficiently synthesize the information from 300,000 trials in the Cochrane Central Register of Controlled Trials literature database [33]. This was thought by Cochrane to be achievable by 2010 to 2015, but to date, this figure stands at approximately 7900 [33]. Furthermore, although Cochrane aspires to update these reviews regularly with new studies and analyses, it struggles to do so [12]. The suggested reason for this is the inefficiency of the systematic review or meta-analysis process [2]. Proposed methods to ameliorate this issue include standardization (eg, Preferred Reporting Items for Systematic Reviews and Meta-Analyses and International Prospective Register of Systematic Reviews) and availing of technology to streamline the process [15].

Although technology such as meta-search engines, machine learning platforms, and automated information extraction systems continue to develop, these solutions remain largely experimental [34-37]. Filter or hedge development has been proposed as one solution to this problem. However, as demonstrated in this study, this can have varying degrees of accuracy. When developing a DCM search filter with 100% sensitivity, this returned <20% precision values, and efforts to optimize its specificity using NOT functions reduced the sensitivity [7].

A more comprehensive change would be the development of an index term/ICD category with a paired MeSH term. This could deliver immediate search efficiencies. MeSH terms have been developed as natural language processing tools [38], streamlining the current literature search process, and will likely prove integral to a future machine-assisted and/or machine-led review of the literature [37]. Indeed, MeSH tags have the potential to solve the identified issues in DCM literature of heterogenous synonyms and overlapping terms with non-DCM literature. The hierarchal tree structuring of MeSH tags will allow encompassing the various index terms that exist for DCM without the inclusion of such a large body of the literature, which is unrelated but shares isolated overlapping words or phrases. The MeSH labeling process, once a MeSH term is created, has moved from a human-only process to a machine-assisted process, saving cost and time for literature libraries [27]. Each article is currently processed by using the Medical Text Indexer technology, suggesting MeSH labels (on the basis of the title, abstract, and related articles’ labels) to human indexers [27]. However, the rate of comprehensive and accurate indexing struggles to keep up with high rates of research synthesis [27]. However, novel fully automatic MeSH indexing technologies (eg, MeSHLabeler and DeepMeSH) employ machine learning algorithms to make large-scale MeSH indexing cheaper, more efficient, and more accurate. Employing such technology should motivate us to aim for a fully indexed body of DCM literature [27,39].

Index terms are equally important in standardizing our language in both research and clinical practice. In addition to search inefficiencies, inconsistencies within the definition of DCM have prevented all retrieved studies being pooled for analysis [40]. Development of a universally agreed definition has been successfully done via consensus processes for other diseases, more specifically via a modified Delphi process [41]. Our group aims to establish this index term for DCM as part of our Research objectives and Common Data Elements for DCM study. For creation of an ICD entry, a proposal can be made via the World Health Organization web-based ICD-11 platform to be reviewed by a Topic Advisory Group and Revision Steering Group [42].

This illustrative search excluded the vast majority of primary DCM research by using the systematic review, review, and meta-analysis filters. This was done to make the illustration of imprecision, a process that requires hand searching of articles, more pragmatic. Although this reduced the number of articles retrieved, given that the objective of this study was to consider the sources of imprecision, we do not feel that this would have limited our findings. Moreover, the practice of limiting research synthesis is reflective of day-to-day search practices.

In addition, a small number of results covered multiple categories of imprecision. In these cases, a review of the article’s full text for the primary area of discussion was undertaken, followed by allocation to that imprecision category. We acknowledge that this is a relatively subjective process. However, imprecision because of overlapping terms was still identified, regardless of categorization, and thus, the primary aims of the study were fulfilled. Finally, we searched only 1 additional database, and only 10% of articles had their related articles function tested. Again, this served to elucidate their known utility in a practical fashion, with further database evaluation not required.

This paper illustrates the difficulties encountered by past, current, and future reviewers of DCM literature. Overlapping and heterogenous search terms and inaccurate article indexing lead to an imprecise and wasteful process. Researchers in the field of DCM must be aware of the adverse effects that sensitivity and specificity common search functions (eg, humans and adult) may have on the retrieval of results. However, the common practice of using related articles functions and searching multiple databases is recommended in DCM literature. Looking forward, MeSH labeling, a standardized DCM definition, and comprehensive indexing of DCM literature will be crucial steps in ameliorating these hurdles.