Knowledge synthesis, also called evidence synthesis, is the process of identifying, evaluating, and critically integrating all available evidence on a topic [1]. This process is a foundational practice for the development of evidence-based decision-making [1]. Effective and comprehensive search strategies are one of the core components of knowledge synthesis [2]. To support robust search strategies, search hedges are increasingly used. A search hedge consists of pre-established search strings designed to comprehensively retrieve references in a topic area from one or multiple bibliographic databases [3]. A search hedge is typically developed by librarians working with subject matter experts and should be validated for its performance [4]. Validated search hedges can help ensure all relevant literature is retrieved, reduce the number of irrelevant articles screened, and increase the efficiency of resource use in the knowledge synthesis process.

In addition to hedges with a specific subject matter focus, more recent publications have emphasized other search attributes, such as geographic, methodological, or population-specific searches [5-7]. While search hedges focused on combined sexual and gender minority groups are available, there are currently no validated and published search hedges that focus on transgender and gender diverse (TGD) communities and topics [8,9]. In this protocol, we use the term TGD to describe people who do not identify with their assigned gender at birth. This definition has the potential to also describe intersex people and people with differences of sex development (DSDs), and there are indeed certain overlapping experiences between TGD and intersex people. However, we follow the recommendations of organizations such as interACT: Advocates for Intersex Youth who make plain that being intersex and being TGD are distinct categories of identity and experience. Thus, we will not include articles studying DSDs or intersex communities unless it is clear within the article that the study sample includes people who are both intersex and transgender to respect these identities and experiences as distinct from those of TGD communities.

Additionally, we recognize that TGD can be understood as an anglophone export or a category imposed onto autochthonous sex and gender systems that have been devalued, erased, or even criminalized through imperialism and settler colonialism [10,11]. Thus, our search hedges incorporate terminology beyond that centered within the medical literature of the anglophone world and Global North. This decision to incorporate culturally specific terms, while intended to be inclusive, also runs the risk of reproducing the centrality of not only English-language terms but also medicalized models of understanding gender nonconformity and gender diversity.

TGD people face systematic marginalization and oppression unique from those of the cisgender heterosexual population and their sexual minority peers [12,13]. Misconceptions and disinformation regarding the experiences, quality of life, and health of TGD people further fuel the discrimination against this population [14-16]. Rigorous and transparent knowledge synthesis processes, starting with high-quality search hedges, can help inform and equip community members, clinicians, policymakers, and other key decision-makers with scientifically sound evidence.

Precision and sensitivity are 2 measurements commonly used in the validation process of search hedges and together offer quantifiable reference points in the performance of a hedge. Precision is the ability of a search hedge to minimize the retrieval of nonrelevant articles [21,22]. Precision is calculated as the proportion of relevant over total retrieved articles [22]. Sensitivity, also called relative recall, is the ability of a search hedge to retrieve all relevant articles [21,22]. Sensitivity is calculated as the proportion of articles from a gold-standard set that are retrieved by a search hedge [22]. The most common method of establishing a gold-standard set for the validation of a search hedge is through manually searching a journal, a set of journals, or a database [22]. However, the main disadvantage of this method is its high time consumption and low article yield [22]. Another approach to finding a gold standard is using a previously assembled collection [22]. In this study, the gold-standard set consists of 144 TGD articles from Knowsy (search year: 2019; for a full list of articles in the gold-standard set, see Multimedia Appendix 2). Knowsy is an online dataset of scoping reviews, systematic reviews, and structured reviews focused on 2SLGBTQIA+ topics [23,24]. Articles in Knowsy were identified by systematically searching 4 academic databases (MEDLINE, APA PsycInfo, Embase, and CINAHL) and selected in accordance with the standard 2-step screening process performed by 2 independent, trained reviewers [23,24]. Each reference in Knowsy has been labeled with standardized category tags for topics, populations, and type of knowledge synthesis by a trained reviewer, with independent validation by a second reviewer [23,24].

Special consideration also needs to be paid to ensure ethical and responsible research when working with a marginalized community, such as gender minority communities. Accountability and transparency in the search and research process are 2 key components to ensure that researchers do not further alienate already marginalized people. One of the best practices to accomplish these goals is the publication of a protocol at the start of the research process [25]. A protocol offers detailed background information on the search and screening process and increases the reproducibility of the results. It also provides an accountability mechanism as deviations from the protocol need to be identified when reporting study findings. As validations of search hedges are still uncommon, protocols for the validation of these hedges are even more limited [26-28]. The scant literature regarding search hedge validation methods could further discourage this type of research from being conducted as lack of established methods can increase the amount of time and resources dedicated to this process. Thus, publishing a search hedge validation protocol that includes details about the methods used aids in knowledge and resource sharing, as well as facilitating the peer review process and increasing research rigor. In this paper, we present a protocol for the validation of 2 TGD search hedges using a novel validation structure.

This protocol describes the methods for a search hedge validation. The objectives are to (1) validate 2 interdisciplinary and cross-cultural TGD search hedges designed for retrieving references in MEDLINE and APA PsycInfo, (2) determine the usage pattern of TGD polysemic terms in the literature, and (3) assess the usage pattern of TGD terms in 2SLGBTQIA+ literature.

Ethics approval will not be sought for this study because it will only include analysis of published articles. This study will not involve human participants, medical records, patient information, observations of public behaviors, or secondary data analyses.

To develop and validate the 2 search hedges, we followed a 2-stage process: a preliminary search and a finalized search.

While a number of validated and unvalidated search hedges focused on 2SLGBTQIA+ populations exist, to the best of our knowledge, this protocol is the first to detail the process of evaluating 2 search hedges that exclusively focus on the TGD population for MEDLINE and APA PsycInfo [8,9,33,34]. In this paper, we developed a novel screening structure for terminologies with dynamic meanings, with full-text screening only for references where further confirmation was required. We anticipate that both the MEDLINE and APA PsycInfo search hedges will achieve a similar sensitivity of 100% to that of their pilot hedges. We also anticipate that the hedges will achieve higher precision than their pilot counterparts due to the fine-tuning conducted in between. The precision, sensitivity, and search term insight results will provide guidance for librarians and researchers to customize the hedges to best fit their research questions.

While various published and validated search filters exist, most of these hedges focus on retrieving articles based on geographic locations, type of study, and treatment [5,26,27,35]. Validated search hedges focusing on populations, specifically marginalized communities, are particularly scant. Schilperoort et al [34] documented the development and validation process of search filters for PubMed focusing on queer women. Wafford et al [36] documented the development and validation process of a search filter for PubMed focusing on immigrant populations. Both of these population filters relied on the relative recall method to create their respective gold-standard sets. Relative recall is the process of retrieving structured reviews, such as systematic reviews, scoping reviews, and evidence gap maps, related to the specified population and searching through the citations of each review to further identify relevant articles. Schilperoort et al [34] used an internal validation approach in which the gold-standard set was used to generate the search terms as well as the standard for testing the sensitivity of the filters. On the other hand, Wafford et al [36] used an external validation approach in which the team followed the same process of relative recall but partitioned the retrieved articles into 2 separate sets: a development set from which to derive search terms and a validation set (occupying the same role of a gold standard) to test the filter. External validation offers more rigorous results as it mimics a testing environment closer to how a filter will be deployed in a real-world setting [35]. In this protocol, we are able to use an external validation approach due to the existence of the TGD subset in Knowsy. The Knowsy dataset is entirely independent in development and screening to the 2 current search hedges, allowing for a more stringent validation process.

Our validation method further diverged from the methods used by Schilperoort et al [34] and Wafford et al [36]. Both groups validated the precision of their filters against known sets established through the relative recall process, whereas the existence of the Knowsy dataset as our gold standard allows us to test and calculate the precisions and sensitivities of our hedges against unknown sets, offering a closer resemblance to real-world settings [34,36]. While Wafford et al [36] also used external validation, the team only accessed articles at the title and abstract level. For a large portion of our retrieved articles, the titles and abstracts do not provide sufficient information regarding our population of interest, a similar problem that Schilperoort et al [34] also encountered with a search filter focused on queer women. Thus, without the need to generate a gold-standard or validation set, our screening process can explore the semantics and context of terms through our Mixed_Topics and LGB_Maybe_T categories while minimizing the number of articles needing to be screened in full text.

Once validated, the search hedges will be submitted to the InterTASC Information Specialists’ Sub-Group Search Filter Resource, the McMaster Health Knowledge Refinery, and the University of Alberta Library’s health sciences search filter database [37-39]. We plan to periodically update the 2 hedges to keep abreast with indexing changes in their respective databases. Additionally, future plans will include development and adaptation of the hedges for other bibliographic databases, such as Embase, PubMed, CINAHL, and Scopus. Other knowledge mobilization plans include presentations at relevant research and knowledge synthesis conferences such as Moving Trans History Forward and the Global Evidence Summit [40,41].

One primary limitation of the hedges is the limited semantic search. This is related to the standard search mechanisms in MEDLINE, APA PsycInfo, and other databases. The default search mechanism for these databases is the keyword search, which relies on exact match or known variants of the search strings [42]. For example, if a search string specifies the term transgender, the keyword search will retrieve only items with the exact match. Truncation operators can increase recall by looking for variants based on the word root (eg, plurals) but not semantic meaning. However, a keyword search does not take into account the context in which the term occurred, thus retrieving results with false semantic matches. In this example, articles with phrases such as not transgender would still be retrieved by the hedges. We anticipate that this search mechanism will decrease the precision rates of the hedges.

Another consideration when using the search hedges is the trade-off between sensitivity and precision. The primary user audience for the search hedges are information professionals supporting evidence syntheses, and thus, the hedges were designed with sensitivity as the top priority. The trade-off of this prioritization is precision, and certain types of reviews with time and resource constraints, such as rapid reviews, should keep this in mind. By examining specific search terms that are anticipated to retrieve more nonrelevant articles, we hope to provide librarians and other readers with additional information to make guided adaptations of the hedges that best fit individual research questions and purposes.

This protocol provides a blueprint for future studies to respectfully and comprehensively search literature related to self-described identities used by TGD communities. This protocol, along with other published articles of its kind, will contribute to the openness of science and the practice of sharing resources within the research community.