The US Public Law 115-435—The Evidence Act—requires the Department of Veterans Affairs (VA) to support appropriation requests with evidence and evaluation [1]. The Veterans Health Administration, Office of Research and Development, Quality Enhancement Research Initiative (QUERI), developed the Evidence-based Policy Evaluation Center Award mechanism to develop an infrastructure of centers that specialize in rigorous, independent evaluations focused on national priorities that are informed by the Foundations for Evidence-Based Policymaking Act (Evidence Act, US PL 115-435) of 2018 (Multimedia Appendix 1).

The current protocol presents the proposed StrAtegic PoLicy EvIdence-Based Evaluation CeNTer (SALIENT), with the goal to conduct evaluations that support the adoption of programs and policies aligned with VA priorities and VA’s goal to be a high-reliability organization and learning health system. SALIENT will generate, disseminate, and implement evidence-based policy recommendations and best practices in accordance with the Evidence Act. To accomplish this, we propose 4 objectives. For objective 1, the SALIENT team will develop and conduct comprehensive evaluations using a wide range of approaches (quantitative, qualitative, mixed methods, policy analysis, economic, clinical informatics, and implementation science) specifically tailored to VA’s strategic mission goals and priority areas. For objective 2, the SALIENT team will develop and deploy knowledge translation resources such as communication briefs, playbooks, and other materials for key stakeholders to accelerate the implementation of evidence-based practices and impact. For objective 3, the SALIENT team will use evaluation methods to develop best practices in policy evaluation and implementation. For objective 4, the SALIENT team will train diverse evaluation scientists to democratize evaluation, implementation, and knowledge translation expertise.

In 2022, to align with 2021-2025 QUERI strategic goals and accelerate rapid knowledge translation in the pursuit of transforming VA into to a learning health system, QUERI launched an initiative to include a funding mechanism for evidence-based policy evaluation centers to support innovation in policy evaluation [2,3]. SALIENT was selected to support the enterprise-wide effort in policy planning and evaluation, due to the team’s capability to foster advancements in VA priority areas.

Our interdisciplinary team has broad and deep expertise across VA evaluation priorities. Table 1 illustrates our expertise in these priority areas and details resources of our multisite center, including leadership from the Informatics, Decision Enhancement, & Analytics Sciences Center and James A. Haley Veterans’ Hospital, and collaborators from the Center of Innovation for Complex Chronic Healthcare (Hines VA), the Center for Innovation for Veteran-Centered and Value-Driven Care, the Center for Care Delivery and Outcomes Research, and the War Related Illness and Injury Study Center. This expertise uniquely positions us to address 6 national evaluation priorities, identified by QUERI’s multistakeholder process, that are linked to Veterans Health Administration Performance Plan metrics (eg, Strategic Analytics for Improvement and Learning).

SALIENT investigators are uniquely positioned to evaluate environmental exposures [4], including airborne hazards [5], which are high priorities of the President and Congress [6]. Our team conducted foundational studies to document the impact of military exposures and improve health care, demonstrating our operational partnerships and “big data” resources. This includes pioneering the use of the electronic health record to examine exposure concerns, that is, chronic multisymptom illness (AFM, LMM, and MHS), and conducting the first hybrid effectiveness implementation trial for exposure concerns: chronic multisymptom illness (LMM) [7-11]. VA’s Health Outcomes Military Exposure program office called on us to lead the use of the electronic health record and individual longitudinal exposure record to connect phenotypes with military exposures (MJP), evaluate Health Outcomes Military Exposure/ War Related Illness and Injury Study Center initiatives and clinical services (AFM and LMM), and lead and support educational efforts (LMM and SCH).

SALIENT is uniquely positioned to evaluate and optimize care for veterans with complex conditions that require integration of care across providers. Dr Gordon leads the Vulnerable Veteran Innovative Patient-Aligned Care Team Service & Research, Veteran Innovative Patient-Aligned Care Team Initiative, which has developed and implemented integrative care models for vulnerable veterans, including the implementation of medication treatment for opioid use disorder in 36 VA facilities across the nation (SCOUTT). Dr Brooke implemented a multidisciplinary program, the Transitional Pain Service, in Veterans Integrated Services Networks-19 coordinating pain management for patients at risk for opioid use disorder at the time of surgery [12,13]. This includes implementation for primary care providers to identify and refer high-risk patients to Transitional Pain Service. Our team is also a leader in integrating care for exposures into primary care. Dr Hunt is the national director of the postdeployment integrated care initiative, which is the office of primary care initiative to integrate deployment and exposure care into primary care, and Dr McAndrew is conducting multiple evaluations of education and consultation services to support primary care in addressing these concerns.

National efforts have focused on rebalancing long-term support services to promote care for veterans in their homes rather than institutional settings. This priority area is aligned with our expertise leading the Data and Policy Core of the Elizabeth Dole Center of Excellence for Veteran and Caregiver Research (Dole CoE). The Dole CoE helps veterans age in place. The Dole CoE team works with operations partners in Geriatric Extended Care to identify health services trajectories associated with quality home care and those predicting institutionalization using Geriatric Extended Care–Data Analysis Center VA Medicare and national survey data. The data resources (Multimedia Appendix 2) and expertise of SALIENT investigators in evaluation and geriatrics [14] are a strong foundation to address this VA priority.

SALIENT investigators have played an instrumental role in the COVID-19 response for multiple federal agencies and local and state health departments [15,16]. Our contributions to efforts to mitigate the adverse impacts of COVID-19 highlight our expertise in epidemiology and control of infectious diseases. We developed new data resources for case ascertainment, clinical classification, and assessment of outcomes [17-19]. Combining clinical expertise with advanced analytical techniques, we have addressed key questions around short- and long-term management of COVID-19 infection [20]. We lead studies funded by the Food and Drug Administration and the Centers for Disease Control and Prevention on veteran COVID-19–related health inequities. In VA Health Systems Research (HSR), QUERI, and Department of Defense funded studies, we use mixed methods to evaluate problems of vaccine hesitancy and trust, and the impact of COVID-19 on veterans’ health and well-being [21]. Building on our HSR-funded work on colon cancer screening, we evaluated the impact of the pandemic on veteran access to endoscopy [22,23]. Working with our VA operational partners, the SALIENT team proposed the COVID Post-Exposure Evaluation and Symptomatology Center, which addresses critical knowledge gaps concerning the post–acute sequelae of COVID-19 (ie, post–COVID-19 condition) and underscores our focus on advancing COVID-19 research and improving care for veterans living with COVID-19.

SALIENT is uniquely positioned to evaluate quality and cost of VA-purchased community care across a broad set of domains, including mental health and disparities. Dr Vanneman is a VA HSR Career Development Award recipient and a national expert in measuring access, quality, and cost of VA-purchased community care, with an emphasis on access, patient experience, and behavioral health [24-31]. Dr Vanneman is a multiple principal investigator (MPI) of the Access and Community Care Engagement Network Team COnsortium of REsearch. Dr Haun leads research and operations-based projects on implementation and evaluation of digital care resources to improve VA employee workflows and care coordination for veterans and their informal caregivers. Her work has driven the implementation and redesign of My HealtheVet, Secure Messaging [32], and the Veterans’ Delegation tool, and extended integration of digital resources in primary care [32-38].

The MISSION Act (Section 1703C of title 38, Section 104) required VA to establish standards for quality for VA and VA-purchased community care health care services [39]. Published studies, including those by Dr Vanneman and collaborators, show similar or better quality of care at the national level for VA-delivered versus VA-purchased community care and are consistent with findings from prior studies that have compared VA and non-VA care (eg, Medicare). As Dr Vanneman is one of the few HSR investigators with extensive expertise in policy and analysis of community care data, SALIENT is well positioned to evaluate MISSION Act standards of care and impacts on quality improvement and policy.

Figure 1 shows the interrelationships of VA operations and leadership, QUERI, the Partnered Evidence-Based Policy Resource Center (PEPReC), and SALIENT stakeholders.

We will work with these partners and other evaluation centers to conduct evaluations that support VA programs and policies. We will continually communicate research findings to other VA evaluators and synthesize learning across SALIENT and other groups to enhance the quality of evaluations conducted by the VA more broadly and develop best practices in policy implementation, impact, and evaluation.

SALIENT will conduct evaluations that support the adoption of programs and policies aligned with VA priorities and its goal to be a national learning health system. Our evaluation approach is anchored by key tenets of implementation science, quality improvement, and learning health systems (Figure 2).

Following the steps outlined by the QUERI Implementation Roadmap [40], and ACTION (Alignment with multilevel priorities, Commitment from operational partners, Tailoring to the local context, Informing the field, Observing health care changes, and generating New questions or projects) Impact Framework [41], SALIENT researchers will (1) identify problems and solutions, (2) engage stakeholders, (3) develop an implementation research logic model, (4) help implement relevant interventions as needed for a given project, (5) monitor data, and (6) perform ongoing reflection and evaluation. Through this process, we will develop best practices in policy evaluation (Figure 2). The theoretical frameworks guiding our project evaluations include the Consolidated Framework for Implementation Research (CFIR) and the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM). CFIR organizes factors that influence implementation outcomes across 5 domains—intervention, outer setting, inner setting, individuals, and implementation process [42]. The RE-AIM framework focuses explicitly on issues and steps in the implementation process that may improve or impede desired impact [43,44]. The principles of Lean Six Sigma and the Learning Health Systems Knowledge to Action Framework ensure a continual feedback loop of lessons learned [45].

This policy center was funded by a QUERI quality improvement center award mechanism and as such, the center and projects are to be conducted as quality improvement nonresearch. In alignment with regulatory guidelines, this center application was reviewed by the VA Salt Lake City Human Research Protection Office and received approval as quality improvement. It is notable that QUERI projects and centers are funded as quality improvement; research-regulated projects are funded by other services within the Veterans Health Administration, Office of Research and Development. Participants’ identities are kept confidential, and data are stored behind VA fire walls. All methods are performed in accordance with relevant guidelines and regulations. Informed consents, compensation for participants, and institutional review board oversight are not applicable for quality improvement projects funded as part of SALIENT.

Consent has been provided by coinvestigators and project leads to use names in text, tables, and figures within this protocol.

After PEPReC assigns evaluations [46], the Operations Core (Lead Cochran) will coordinate work using Lean methodologies in addition to managing contracts and external reporting (eg, QUERI, PEPReC, other VA stakeholders, and the Office of Management and Budget). The Methods Core (Leads Vanneman and Zickmund) will ensure that appropriate methodologies are used for each evaluation (objective 1). The Knowledge Translation (KT) Core (Lead Fagerlin) will use state-of-the-art translation of evaluation results into action and policy. KT Core team integration will enhance impact through guidance as evaluations are developed and implemented and products such as executive summaries and playbooks are produced (objective 2). All cores will prioritize the development of best practices in evaluation science (objective 3) and training of diverse implementation scientists (objective 4).

We propose a multilayered theoretical approach, using rigorous mixed methods that relies on implementation science frameworks, Lean Six processes and a strategic process across SALIENT project conceptualization, data collection, analysis, implementation strategy development, reporting, synthesis, and dissemination over time. RE-AIM, CFIR, evidence-based implementation strategies, and the Implementation Research Logic Model process will inform our conceptual lens across projects to standardize our approach to policy evaluation and implementation over time.

Consistent with Lean Six [48] the Operations Core is charged with continuous and sustained improvement of center operations to ensure that performance is meeting stakeholders’ expectations and the SALIENT team improves work processes and optimizes efficiencies during and after each evaluation. The Operations Core will facilitate the feedback loop with stakeholders, cores, and ETs using periodic reflections [54], throughout evaluations, allowing rapid feedback from stakeholders on aspects of design and execution that will inform the direction of the project, modifications, and development of a learning health system (Analyze and Improve phases). We will facilitate a continuous feedback loop to avoid misalignment in goals and expectations, allow rapid and strategic course corrections throughout the evaluation, and ensure that the end user can optimize use of evaluation results.

To develop best practices in policy evaluation and implementation across projects over time, we will develop a SALIENT Policy Evaluation Catalog of projects to document project factors (ie, policy type, stakeholders, constructs, implementation strategies, communication and dissemination strategies, and mechanisms). We also propose to collaborate with other centers to collect these data on their evaluations to inform large-scale synthesis of policy evaluation and implementation best practices. Data from our inventory can be used by SALIENT, other centers, and the coordinating center for cross-project observations and learning, which is a broader opportunity for overall QUERI development. To accomplish this, the Operations Core will conduct after-action evaluations including operations partners (see left side of Figure 3), investigators, and end users of the evaluation to discuss successes or concerns observed during the evaluation documented by detailed field notes and recordings. A quantitative postevaluation assessment (Multimedia Appendix 5) will capture operations successes and barriers.

Using the SALIENT Evaluation Policy Catalog data we will brainstorm alternatives, overcome barriers experienced, and inform future evaluation development and processes. These data will be used to create Lean Six tools, such as Value Stream Maps (schematics used to identify activities that have value to stakeholders as well as waste, delays, and inefficacies), to identify strengths and weaknesses of SALIENT activities. Examples of wastes, delays, or inefficiencies could include team members from one core unnecessarily waiting for completion of entire components of evaluations (batches) instead of working continually as smaller components are completed (single-piece workflow); a highly inefficient approach to collaborative work processes. These schematics and proposed priorities will be discussed with center and core leads. The group will come to consensus on priorities and their rank order. Sustaining improvements are ensured by this type of regular review and use of audit checklists for continued monitoring. These analyses will synergize findings and processes across cores and evaluations to develop best practices in policy evaluation and implementation.

The goal of this evaluation center is to support the successful identification and implementation of VA high-priority operational evaluation projects that are aligned with VA population and system needs. This protocol illustrates a multilevel theoretical and Lean Sprint methodology to conduct rapid rigorous operations-based policy evaluation in a learning health system. To our knowledge, this protocol is unique in that it informs a center-wide Lean Sprint methodology to conducting quality improvement to evaluate priority policies. Our impact will be to accelerate the generation, dissemination, and implementation of evidence-based policy recommendations pursuant to the Evidence Act, develop best practices in policy evaluation and implementation, and train diverse evaluation and implementation scientists.

This center protocol contributes to the field in four distinct ways by (1) using a Lean Sprint methodology (iterative, incremental, rule-governed approach to clearly defined, and time-boxed work) to develop our evaluation plans collaboratively with operational partners and key stakeholders including veterans, policy experts, and clinicians; (2) using a multicore approach (operations, methods, and knowledge translation) to conduct evaluations and rapidly disseminate findings to inform knowledge translation; (3) developing a center infrastructure to train diverse evaluation scientists to democratize evaluation, implementation, and knowledge translation expertise; and (4) identifying best practices in policy evaluation and implementation to inform learning health systems.

Potential weaknesses of the center should also be considered. First, given geographic diversity, communication breakdowns across sites are a risk; however, a comprehensive center management plan and digital communication methods ensure continuity in communication and center-site interactions. Second, given the size of the center and rapid nature of projects, there is a risk of diffuse efforts or mismanagement of resources; however, the multicore approach of the center ensures infrastructural, administrative, and resource support from an administrative, operational, and methodological perspective. Third, with shifting topical and funding priorities, continuity in resources and staffing could pose challenges across projects over time; however, SALIENT leadership is engaged with QUERI, other QUERI Evidence Act evaluation centers, and operational leadership to anticipate and respond to the changing climate to allow for symbiotic reciprocity. Furthermore, SALIENT’s Lean Sprint systems redesign approach maximizes resources and sustained efficiency of projects. Fourth, often operations-based projects require data sets and resources that may not be readily available, particularly on the rapid timeline of high-priority projects. SALIENT subject matter experts have a highly developed network of resources and collaborations within and beyond VA to ensure currency and continuity across data resources.

In summary, the SALIENT environment has exceptional evaluation and training resources; has established infrastructure with a proven track record of rapid evaluation and dissemination of results; and strong, lasting operational partnerships and collaborations. The SALIENT team is poised to rapidly conduct state-of-the-science evaluations to support the broad VA response to the Evidence Act and high-priority legislative requirements related to environmental hazards and airborne exposures. SALIENT evaluations will contribute to (1) optimized policies and programs for veterans; (2) improved outcomes regarding health, equity, cost, and provider well-being; (3) advances in the science of policy evaluation and knowledge translation by synthesizing learning across SALIENT and other center evaluations; and (4) expansion of the implementation and dissemination science workforce.