ResProt JMIR Res Protoc JMIR Research Protocols 1929-0748 JMIR Publications Toronto, Canada v8i7e13801 31274111 10.2196/13801 Protocol Protocol Community-Directed Bacterial Sexually Transmitted Infection Testing Interventions Among Men Who Have Sex With Men: Protocol for an E-Delphi Study in Toronto, Canada Eysenbach Gunther Millson Peggy van Gemert Caroline Zlotorzynska Maria Risling Tracie Burchell Ann N PhD 1
Centre for Urban Health Solutions Li Ka Shing Knowledge Institute St Michael's Hospital 209 Victoria St Toronto, ON, M5B 1T8 Canada 1 4168646060 ext 77498 burchella@smh.ca
2 http://orcid.org/0000-0003-2822-2169 Lisk Ryan BA 3 http://orcid.org/0000-0001-7171-4295 Yeung Anna PhD 1 http://orcid.org/0000-0003-4301-8473 Rana Jayoti MPH 1 http://orcid.org/0000-0003-4294-3229 Bacon Jean BA 4 http://orcid.org/0000-0001-6729-4830 Brunetta Jason MD 5 http://orcid.org/0000-0002-0506-8616 Gilbert Mark MD 6 http://orcid.org/0000-0001-5978-6843 Gesink Dionne PhD 7 http://orcid.org/0000-0002-8874-0470 Grewal Ramandip MPH 1 http://orcid.org/0000-0002-1922-2442 Guiang Charlie B MD 2 8 http://orcid.org/0000-0003-0545-854X Kwag Michael BA 9 http://orcid.org/0000-0003-0444-9043 Logie Carmen H PhD 10 http://orcid.org/0000-0002-8035-433X Mitterni Leo BSW 8 http://orcid.org/0000-0002-8791-0853 Shahin Rita MD 11 http://orcid.org/0000-0001-9478-8000 Tan Darrell HS PhD 1 http://orcid.org/0000-0002-3069-2875 Centre for Urban Health Solutions Li Ka Shing Knowledge Institute St Michael's Hospital Toronto, ON Canada Department of Family and Community Medicine St Michael's Hospital Toronto, ON Canada ACT Toronto, ON Canada Ontario HIV Treatment Network Toronto, ON Canada Maple Leaf Medical Clinic Toronto, ON Canada BC Centre for Disease Control Vancouver, BC Canada Dalla Lana School of Public Health University of Toronto Toronto, ON Canada Hassle Free Clinic Toronto, ON Canada Community-Based Research Centre Vancouver, BC Canada Factor-Inwentash Faculty of Social Work University of Toronto Toronto, ON Canada Toronto Public Health Toronto, ON Canada Corresponding Author: Ann N Burchell burchella@smh.ca 072019 04 07 2019 8 7 e13801 28 2 2019 11 5 2019 28 5 2019 29 5 2019 ©Ann N Burchell, Ryan Lisk, Anna Yeung, Jayoti Rana, Jean Bacon, Jason Brunetta, Mark Gilbert, Dionne Gesink, Ramandip Grewal, Charlie B Guiang, Michael Kwag, Carmen H Logie, Leo Mitterni, Rita Shahin, Darrell HS Tan. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 04.07.2019. 2019

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.

 Background

HIV-positive and HIV-negative (gay, bisexual, and other) men who have sex with men (MSM) have experienced a dramatic increase in bacterial sexually transmitted infections (STIs)—syphilis, gonorrhea, and chlamydia. STI testing and treatment mitigate adverse health outcomes and substantially reduce transmission; yet, testing rates remain below recommended levels. Innovation is needed to produce the required increases in testing levels, frequency, and the use of appropriate testing technologies in ways that are engaging, nonstigmatizing, and acceptable to men.

 Objective

The aim of this study is to build consensus with regard to interventions with the greatest potential for improving local STI testing services for MSM communities in Toronto, Canada.

 Methods

Following a literature review of evidence regarding the effectiveness of novel testing interventions, and focus groups, and surveys to describe local barriers and facilitators of testing among MSM, we will conduct a Web-based, modified Delphi study (e-Delphi). We will form expert panels of community members and STI test providers. Panelists will rate potential interventions in terms of their priority, using a 7-point Likert scale from definitely not a priority to definitely a priority. They will also rank their preferences by selecting their top 3 preferred interventions. Surveys will be distributed in 3 rounds, with feedback on the distribution of responses from preceding rounds provided in rounds 2 and 3. We will define consensus as having ≥60% (18/30) members indicate a preference within 2 adjacent response points. Qualitative data on disagreements will be obtained using open-ended text responses to explain for ratings and rankings that are different from the majority.

 Results

On the basis of a literature review and identification of barriers and facilitators to STI testing among community members and test providers in Toronto, we have selected 8 potential interventions for inclusion in the e-Delphi panel surveys. These include 4 interventions that streamline STI testing for asymptomatic individuals, 2 interventions that are targeted at clients and 2 interventions that are targeted at providers.

 Conclusions

Findings will provide community direction for informed decision making regarding the implementation of STI testing interventions in this setting. They will characterize the intervention climate for innovation to STI testing services, including perceived needs for changes to test delivery, relative priorities for change, and readiness for implementation. These methods may be transferable to other urban jurisdictions experiencing similar epidemics and for other contexts where stakeholder input is needed to manage sensitive areas of concern.

 International Registered Report Identifier (IRRID)

PRR1-10.2196/13801

 sexual and gender minorities sexually transmitted diseases community-based research mass screening patient acceptance of health care Introduction

Delphi Studies

Delphi studies are a valuable approach for building consensus around an issue where little knowledge or agreement previously existed [1]. They use “structured anonymous communication between experts…to gather consensus perspectives about an issue or topic that can then be…used to inform decision making” [2]. Traditionally applied using an in person format, this design is increasingly being adapted for Web-based environments. Briefly, the Web-based modified Delphi study (e-Delphi) involves rounds of Web-based questionnaires in which experts are asked to provide their opinion on particular topics [1,3]. Initially this is done independently, but in subsequent rounds, experts are made aware of the opinions of the group when making their decisions, with the goal of reaching consensus. The key features of the e-Delphi methods are that they are iterative and anonymous, which are particularly beneficial for community-based and patient-oriented research [2,4]. Anonymity and the Web-based format encourage opinion sharing from all panel members, thus preventing dominant individuals from controlling discussion; this is important within hierarchical environments involving the health care system [2,4].

Bacterial Sexually Transmitted Infections

We will adapt the e-Delphi method to learn community perspectives to address a pressing health care system issue in our setting: the rise of bacterial sexually transmitted infections (STIs)—specifically syphilis, gonorrhea, and chlamydia. These infections pose a heavy burden on population health, with most cases occurring among HIV-positive and HIV-negative gay, bisexual, and other men who have sex with men (MSM) [5-9]. Untreated syphilis may progress to neurosyphilis, in which symptoms such as meningitis or dementia may develop [10]. Globally, public health agencies are pressing for increased vigilance of antibiotic-resistant gonorrhea strains [11,12]. Certain serovars of Chlamydia trachomatis may cause Lymphogranuloma venereum with painful proctitis and rectal bleeding. Unlike HIV, these STIs can easily transmit via oral sex [13]. In 2014, there were 109,263 chlamydia, 16,285 gonorrhea, and 2357 syphilis cases reported in Canada, much greater than a decade earlier [14]. The true counts are even higher, as many cases are asymptomatic and go unreported. Gonorrhea and syphilis rates have dramatically increased among males in the province of Ontario, with nearly all syphilis cases and approximately 40% of gonorrhea cases among MSM and >40% of syphilis cases among HIV-positive MSM [15,16]. We have documented that 23% of HIV-positive MSM have had syphilis, and new infections occur at minimum rates of 1 gonorrhea, 1 chlamydia, and 4 syphilis cases per 100 person years [5-8]. Most cases occur in the city of Toronto, with no signs of a decline [17-19]. Within Toronto, the syphilis epidemic is mature and not restricted to a core sociodemographic group among MSM [8,20], requiring broadscale approaches for control.

STI testing and treatment could mitigate adverse health outcomes and substantially reduce population-level transmission among MSM [21]. However, innovation is needed to produce the required increases in testing levels, frequency, and the use of appropriate testing technologies in ways that are engaging, nonstigmatizing, and acceptable to men. Canadian STI Guidelines recommend annual screening for bacterial STIs among sexually active MSM and as frequently as every 3 months for individuals at ongoing risk for STIs [13]. Unfortunately, there are suboptimal levels of STI testing and frequency of testing among MSM in Toronto. STI testing patterns are best known for HIV-positive MSM. In 2009, 55% had tested for syphilis, on average, once per year [8]. As of 2013, we observed only a modest increase to 64% being tested annually, with a few testing more frequently than once per year [22]. Testing rates for chlamydia and gonorrhea are lower than those for syphilis [7,23]; from 2010 to 2013, only 25% of HIV-positive MSM tested annually for genital infection using urine-based tests. Few MSM undergo extragenital testing for gonorrhea and chlamydia, despite Canadian and international guidelines [7,12,23-27]. Without rectal and pharyngeal tests, 71% to 100% of cases will be missed [28,29].

We describe herein our plans to conduct an e-Delphi study as part of a larger mixed-methods study that aims to identify bacterial STI testing interventions for implementation and evaluation among MSM in Toronto. We will assemble 2 expert panels: the first with community members with lived experience as MSM seeking STI testing, and the second with health care providers and public health professionals with expertise in providing STI testing for MSM communities in our setting. Our objective is to build consensus regarding intervention(s) with the greatest potential for improving local STI testing services.

 Methods

All procedures have been reviewed and approved by the research ethics boards of St Michael’s Hospital, Toronto, and the University of Toronto.

Setting

Toronto is a metropolitan city with a population size of 2.71 million in the province of Ontario [30]. All residents with citizen, permanent resident, refugee, and refugee claimant status have access to provincial or federal health insurance for medically necessary services. STI testing services are available from a variety of sources, including primary care practices, specialist services, or dedicated sexual health clinics.

 Knowledge Synthesis to Select Candidate Interventions

To select STI testing interventions for primary inclusion in the e-Delphi panel, we undertook a review of the published literature [31]. To further refine the interventions, we conducted focus groups with MSM STI testing clients [32] and surveyed health care providers [33] in Toronto. Briefly, the focus groups were conducted with HIV-positive, HIV-negative, and trans-identified men (of any HIV serostatus) to identify barriers and facilitators to bacterial STI testing. Health care providers were surveyed about their current practices, barriers, and attitudes to improve bacterial STI testing rates. Manuscripts for these findings are in preparation.

Literature Review

For our literature review, systematic reviews published in 2016 were used as a baseline and updated. These reviews summarized evidence for the effectiveness of STI control interventions, including screening in and outside clinic-based settings published in 2000 or after [24,34]. In addition to repeating the 2016 searches, we expanded literature searches in MEDLINE up to April 2017 using the following keywords: sexually transmitted diseases/STI, chlamydia, gonorrhea, or syphilis. Inclusion criteria for our search were that the article described an intervention aimed at increasing bacterial STI testing; used high-income country settings in urban or semiurban cities; had a study population that included men; and used a study design that was either a trial with a comparison group (controlled, uncontrolled, or pre-post historical controls) or an observational design if it was set in Canada, focused on MSM, or described a Web-based STI testing service. Publications were ineligible if they included only women or heterosexual couples or if they were a study protocol.

Next, we classified the interventions into 3 categories: (1) streamlined testing for asymptomatic individuals, (2) interventions targeted toward clients, and (3) interventions targeted toward providers (Table 1). We use the term clients to refer to users of STI testing services, whether or not they are experiencing signs or symptoms of an STI. Using the same strategy as Taylor et al [34], examining outcomes in increasing the proportion tested or increasing frequency of testing, interventions with a comparison group were categorized as very effective (absolute difference (AD) ≥20% or relative difference (RD) ≥100%), moderately effective (AD 5%-19% or RD 10%-99%), or ineffective (AD <5% or RD <10%). Classifications and categorizations were done by JR and verified by ANB. A complete list of the publications used for the final selection of interventions can be found in Multimedia Appendix 1.

The investigators then reviewed the above findings in a series of meetings and selected promising interventions for the Toronto setting to be included in the Delphi panel exercise. Our selection focused on novel approaches for testing rather than efforts that would reinforce existing STI test practices (eg, patient or provider education alone). To minimize respondent burden for panelists, choices are limited to 6 (for community panelists) or 8 (for provider panelists) intervention options.

Categories of interventions.

Category Definition
Streamlined STIa testing for asymptomatic individuals Interventions that focus on testing asymptomatic individuals with a focus on collection of specimens and reducing the time patients spend in clinics
Client-targeted STI testing interventions Interventions that are targeted at clients to increase client engagement in STI testing
Provider-targeted STI testing interventions Interventions that are targeted at health care providers to increase provision of STI testing

aSTI: sexually transmitted infection.

 Recruitment of E-Delphi Panelists

In our application of the e-Delphi method, the term expert is meant to include persons with lived experience alongside health care professionals. We will form 2 panels: the first with community members with lived experience as MSM, seeking STI testing in Toronto (Community Experts), and the second with health care providers and public health professionals with expertise in providing STI testing for MSM communities in Toronto (Provider Experts). We opted to recruit these 2 panels separately, rather than combined, as it was of interest to identify differences in prioritized interventions between the 2 groups, if these exist, rather than forcing consensus between community and provider experts.

To be eligible for the Community Panel, candidates (1) must be a cis- or trans-identified man aged 18 years and older, living in Toronto, and who has sex with men in the preceding 18 months and (2) must have sought and/or underwent STI testing in Toronto in the preceding 18 months.

To be eligible for the Provider Panel, candidates must have a minimum of 1-year experience providing STI testing and management care in Toronto.

Flowchart for recruitment of community and provider expert panel. Left: To be eligible for the Community Panel, candidates (1) must be a cis- or trans-identified man aged 18 years and older, living in Toronto, and who has sex with men in the preceding 18 months and (2) must have sought and/or underwent sexually transmitted infection testing in Toronto in the preceding 18 months. Right: To be eligible for the Provider Panel, candidates must have a minimum of 1-year experience providing sexually transmitted infection testing and management care in Toronto.

The choice of experts for an initial invitation will be informed by our team’s community and professional networks. Experts who agree to participate will be encouraged to refer other eligible experts, mitigating potential bias from our team’s selection of members.

Using the approach shown in Figure 1, invitations will be informed by our team’s community and professional networks. Community participants will also be recruited via the existing social media channels (eg, Facebook and Twitter) of our community-based partner and through paid banner advertisements on popular gay dating apps (eg, Grindr). In addition, targeted emails will be sent to other organizations that serve the MSM community, including AIDS Service Organizations that cater to specific ethnoracial groups. These methods were successfully used in the recruitment of MSM for the focus groups and other studies conducted by our community-based partner [32]. Providers will be recruited using targeted emails to health care organizations known to serve large MSM patient populations, as we have done previously in our provider survey [33]. We aim to recruit a minimum of 30 experts with diverse backgrounds (including ethnoracial identity, gender identity, sexual orientation, and age) for each panel—feasible and sufficient for a Delphi study [2-4,35]. For community panelists, we have set target goals to recruit a minimum of 40% to be men aged ≤40 years and 40% to identify as non-white race/ethnicity.

Recruitment email invitations, social media, and dating app advertisements will include a link to an eligibility survey. Interested participants will need to complete the eligibility survey to identify those who meet the inclusion criteria. This step serves to minimize false participation. Eligible participants will then be sent a consent form to provide an email address or a phone number to receive the e-Delphi surveys. This information is not linked to the survey responses.

 E-Delphi Methods

The online surveys for each round of the e-Delphi will be delivered through Qualtrics (Provo, United States). Qualtrics is a secure Web-based survey platform and allows for anonymous participation. All data collected in Qualtrics will be stored in Canada and are protected with high-end firewalls and are treated confidentially. We will own and manage all the data collected via Qualtrics. All identified and interested members of the expert panel will be sent a personalized link to fill out each round of the survey. Although a personalized link will be used to access the survey, personal information will not be stored, and contact details will be removed in the completed survey.

Rounds

In the first round, panelists will review and consider the selected STI testing interventions. The preamble for each intervention will include a brief description and a list of considerations; panelists will also be given the opportunity to provide their opinion in an open text field. Panelists will be asked to rate each bacterial STI intervention on a 7-point Likert scale: 1=definitely not a priority, 2=not a priority, 3=somewhat not a priority, 4=undecided, 5=somewhat of a priority, 6=a priority, 7=definitely a priority. An open text field will be available for panelists to explain their priority choice. Finally, panelists will be invited to suggest an alternative STI testing intervention that was not listed but that they believe to be important. Panelists will also report their sociodemographic characteristics, specifically age, race/ethnicity, transgender identity, sexual orientation, and HIV serostatus (optional).

In the second round, panelists will be asked to prioritize the same interventions they considered in round 1. However, this time they will see the distribution of responses from the previous round (eg, the proportions of persons selecting each of the Likert scale options), as well as a summary of the rationale for prioritizing that particular intervention. Those who select a priority rating that does not agree with the majority will be asked to provide details for their choice with an open text field question, such as the following example: “Most guys chose an Online App for Booking Bacterial STI testing as ‘A priority’. Why did you not prioritize this option?”. Panelists will also be asked to rank their top 3 interventions that they consider the highest priority. If consensus is achieved after round 2, then that intervention option will be removed for round 3 prioritization (although they would still be included as options for respondents’ top 3 ranked interventions).

In round 3, panelists will again rate and rank the interventions alongside summaries of the prioritization and ranking responses from round 2, that is, they will have a third chance to rate interventions and a second chance to rank them. Those who rank a bacterial intervention component different from the majority will be asked to provide details for their choice with an open text field question: “One or more of your responses is a different priority than the other experts, please explain why you chose your response.”

 Compensation

Each survey round will be accessible for 2 weeks, with 1-week breaks to conduct the analyses and provide response summaries for the subsequent round. To encourage retention throughout, we will provide increasing incentives at rates of Can $25, $35, and $40 for completion of rounds 1, 2, and 3 (total Can $100 for all 3 rounds), respectively. To receive this compensation, panelists will be provided with a link at the end of their survey which will take them to a reimbursement form to fill in contact information. The contact information will be collected and stored separately from study data and is asked for the purposes of reimbursement only.

 Analysis

The analysis of responses from each round will occur iteratively and independently for the Community and Provider panels. The primary purpose is to achieve consensus within each of the panels to identify which subset of the proposed 8 interventions have the greatest potential for increasing testing levels among MSM in Toronto. As there is no standard definition of consensus for Delphi studies [35], we will define consensus as having ≥60% members (≥18/30) indicate a preference within 2 adjacent response points (+/−1) on a 7-point Likert scale. We will supplement the quantitative analyses with a thematic analysis of open-ended text data [36,37] to better understand disagreements within and between panels, should this occur. The top 3 ranked interventions will be determined based on frequency counts.

 Results

Progress to date includes knowledge synthesis and selection of candidate interventions for the e-Delphi surveys. In our updated literature review, we identified 246 publications, of which 88 were in the original published systematic reviews [24,34]. After applying our inclusion and exclusion criteria, 203 publications were excluded because of the following reasons: (1) the article did not describe an intervention aimed at increasing bacterial STI testing (n=176), (2) the intervention was implemented in a rural setting (n=1), (3) the study population included only women or heterosexual participants (n=22), and (4) study protocol of intervention (n=4).

In our final review, we included 43 publications describing 49 interventions. The largest number of publications were from Australia (n=15). Only 2 publications were from Canada. Effectiveness was categorized for these 49 interventions (Multimedia Appendix 1 ). A total of 37 interventions were deemed effective, with 24 moderately effective and 13 very effective.

In the category of streamlined testing among asymptomatic individuals, routine testing was the predominant intervention, with all 9 effective, followed by Web-based or home-based testing, with 6 out of 7 effective. A total of 8 effective interventions in this category incorporated testing of extragenital sites for chlamydia and gonorrhea, with 7 employing self-collection of anal swabs.

In the category of the client-targeted interventions, the most common intervention was client reminders, with 6 of 8 being effective, followed by 3 effective client counseling interventions. Both client incentive interventions (n=2) were ineffective.

In the category of the provider-targeted interventions, audit and feedback (n=2) and provider alerts (n=2) were effective. The effectiveness of provider education interventions was variable with 1 very effective and 1 ineffective study.

On the basis of the above evidence for effective interventions and emerging findings from our focus groups and provider survey, we selected the following interventions and their rationales for inclusion in the e-Delphi surveys (Table 2).

Descriptions and rationale for interventions.

Category and intervention Description Rationale for inclusion Summary of effectiveness
Streamlined testing among asymptomatic individuals
Routine testing Clients are tested at every visit using standing orders. Routine STIa testing was effective in improving STI testing rates in all 9 studies identified by reducing stigma and normalizing testing. Very effective: 5/10 studies [38-42]; Moderately effective: 4/10 studies [43-46]; Unknown effectiveness: 1/10 studies [47]
Web-based/home-based testing STI tests are ordered on the Web, client can opt for in-person lab testing or mailed self-testing kits. Web-based or home testing was effective in improving STI testing rates in most studies, identified by increasing convenience and reducing the need to see a health care provider. Very effective: 2/11 studies [48,49]; Moderately effective: 4/11 studies [50-53]; Ineffective: 1/11 studies [54]; Unknown: 4/11 studies [55-58]
Nurse/nonphysician-led testing A health care provider who is not a doctor (such as a nurse) collects information on a client’s sexual history and symptoms and collects samples. A total of 2 identified studies demonstrated that having nurses provide testing is effective in improving STI testing rates with reducing the need to see a doctor and increased convenience. Moderately effective: 2/2 studies [59]
Express testing at clinics with self-collection of sample On the basis of a self-completed questionnaire on sexual history and symptoms, clients are directed to self-collected testing if asymptomatic. Express testing was effective in improving STI testing rates in 1 study by increasing convenience and reducing the need to see a health care provider. Moderately effective: 1/2 studies (express clinic with self-collection of some specimens) [60], 1/2 studies (self-collection of samples in clinic) [61]
Client-targeted
Client reminders Client gives permission to clinic to receive reminders via short message service text message, email, or mailed letter. Client reminders were effective in improving STI testing rates in most studies identified. Clients are notified to test, and it becomes part of the health care routine. Very effective: 4/9 studies [62-65]; Moderately effective: 3/9 studies [66-68]; Ineffective: 1/9 studies [69]; Unknown effectiveness: 1/9 studies [70]
Web-based educational and testing booking app Clients find information about bacterial STIs on an app/website and use it to book an appointment at a clinic. A Web-based personally controlled health system manager was effective in improving STI testing rates by increasing knowledge and convenience. Moderately effective: 1/1 study [71]
Provider-targeted
Provider audit and feedback Providers receive a report on their own STI testing practices. Providing feedback reports on STI testing rates was effective in improving STI testing rates by identifying good performance and areas to improve. Very effective: 1/2 studies [72]; Moderately effective: 1/2 studies [73]
Provider reminders Providers receive alerts through electronic medical record systems to prompt an offer of STI testing. Provider reminders to test clients at increased risk of STI acquisition were effective in improving STI testing rates by notifying provider to offer STI testing. Moderately effective: 2/2 studies [74,75]

aSTI: sexually transmitted infection.

 Discussion Overview

By conducting an e-Delphi exercise with community members and providers in Toronto, Canada, we will produce evidence to allow for community-directed, informed choices regarding the implementation of novel STI testing interventions for MSM. To maximize the chances for successful implementation, we first need to better understand the barriers to access testing and the intervention contexts in other settings, then work with community partners to determine which candidate intervention(s) would best overcome these barriers and how they may need to be adapted for the local context. Interventions must be acceptable to members of communities that they intend to serve [76]. Our choice of the e-Delphi method to prioritize potential interventions allows community members to have an equal voice alongside professional stakeholders.

Our plan is not without potential pitfalls. One challenge was selecting interventions for consideration by panelists. Our choices were based on an extensive literature review and qualitative and quantitative data on local patient and provider barriers and facilitators for STI testing. Nevertheless, it is possible that we overlooked or excluded interventions that could be effective in our setting. A second challenge is ensuring diversity in representation among members of the Community and Provider panels, as MSM communities are particularly heterogeneous in large urban cities, such as Toronto. We will seek out as representative a sample as possible to identify diverse perspectives but acknowledge that the opinions of panelists are unlikely to capture all possible views within a small sample size. Motivated panelists are crucial to ensure carefully considered ratings and high response and retention throughout the rounds. We will maximize input by limiting the number of questions asked and providing increasing incentives for completing each round. The potential for false participation is a concern (eg, participation by individuals pretending to meet the inclusion criteria), particularly for the establishment of the Community Panel. Procedures will minimize false participation including study promotion and direct invitations via established MSM community channels, an eligibility questionnaire step as we form the panel (without compensation), and a sliding scale of compensation, such that the highest amount is provided for completion of the third and final questionnaire. Finally, consensus may not be reached at the end of the 3 rounds within and between each expert panel. However, in conducting the Delphi panels, we will gain a better understanding of the interventions with the greatest potential for improving local STI testing services for MSM in Toronto and be better positioned to anticipate potential roadblocks to implementation.

 Conclusions

Innovative approaches to health care delivery are needed to produce the required increases in bacterial STI testing levels, frequency, and the use of appropriate testing technologies in ways that are engaging, nonstigmatizing, and acceptable for MSM [21]. Many community- and clinic-based bacterial STI test interventions have demonstrated effectiveness in the international literature [24,34] and/or are being attempted as pilot projects in Canada [22,55]. Yet the choice of intervention to implement can be daunting without local evidence regarding the best fit. The results of the proposed e-Delphi will characterize the intervention climate including perceived needs for changes to test delivery, relative priorities for change, and readiness for implementation [77]. Our approach may be transferable to other settings where stakeholder input is needed to manage sensitive areas of concern.

 Multimedia Appendix 1

Effectiveness of 49 interventions from 43 articles.

 Multimedia Appendix 2

Peer-reviewer report from the Canadian Institutes of Health Research.

 Abbreviations AD

absolute difference

 CIHR

Canadian Institutes of Health Research

 MSM

men who have sex with men

 RD

relative difference

 STI

sexually transmitted infection

This study is funded by an HIV/AIDS community–based catalyst grant from the Canadian Institutes of Health Research (CIHR, FRN 150082) and a CIHR Foundation Award to ANB (FDN 148432).

None declared.

 Cole ZD Donohoe HM Stellefson ML Internet-based Delphi research: case based discussion Environ Manage 2013 03 51 3 511 23 10.1007/s00267-012-0005-5 23288149 PMC3581739 Brady SR Jason LA Glenwick DS The Delphi method Handbook Of Methodological Approaches To Community-Based Research: Qualitative, Quantitative, And Mixed Methods 2016 New York City Oxford University Press 61 8 Khodyakov D Hempel S Rubenstein L Shekelle P Foy R Salem-Schatz S O'Neill S Danz M Dalal S Conducting online expert panels: a feasibility and experimental replicability study BMC Med Res Methodol 2011 11 174 10.1186/1471-2288-11-174 22196011 1471-2288-11-174 PMC3313865 Coulter I Elfenbaum P Jain S Jonas W SEaRCH™ expert panel process: streamlining the link between evidence and practice BMC Res Notes 2016 01 7 9 16 10.1186/s13104-015-1802-8 26744077 10.1186/s13104-015-1802-8 PMC4704387 Burchell AN Allen VG Gardner SL Moravan V Tan DH Grewal R Raboud J Bayoumi AM Kaul R Mazzulli T McGee F Rourke SB OHTN Cohort Study Team High incidence of diagnosis with syphilis co-infection among men who have sex with men in an HIV cohort in Ontario, Canada BMC Infect Dis 2015 08 20 15 356 10.1186/s12879-015-1098-2 26289937 10.1186/s12879-015-1098-2 PMC4546079 Burchell AN Gardner SL Mazzulli T Manno M Raboud J Allen VG Bayoumi AM Kaul R McGee F Millson P Remis RS Wobeser W Cooper C Rourke SB Hepatitis C virus seroconversion among HIV-positive men who have sex with men with no history of injection drug use: results from a clinical HIV cohort Can J Infect Dis Med Microbiol 2015 26 1 17 22 10.1155/2015/689671 25798149 PMC4353264 Burchell AN Grewal R Allen VG Gardner SL Moravan V Bayoumi AM Kaul R McGee F Millson MP Remis RS Raboud J Mazzulli T Rourke SB OHTN Cohort Study Team Modest rise in chlamydia and gonorrhoea testing did not increase case detection in a clinical HIV cohort in Ontario, Canada Sex Transm Infect 2014 12 90 8 608 14 10.1136/sextrans-2014-051647 25178285 sextrans-2014-051647 PMC4251188 Burchell AN Allen VG Moravan V Gardner SL Raboud J Tan DH Bayoumi AM Kaul R Mazzulli T McGee F Millson P Remis RS Rourke SB OHTN Cohort Study Research Team Patterns of syphilis testing in a large cohort of HIV patients in Ontario, Canada, 2000-2009 BMC Infect Dis 2013 05 28 13 246 10.1186/1471-2334-13-246 23710699 1471-2334-13-246 PMC3668135 Wilton J Gay Men's Sexual Health Alliance 2015 The (Re)emergence of STIs Among MSM http://www.gmsh.ca/about-the-gmsh/news/~15-The-re-emergence-of-STIs-among-MSM; Taylor MM Aynalem G Olea LM He P Smith LV Kerndt PR A consequence of the syphilis epidemic among men who have sex with men (MSM): neurosyphilis in Los Angeles, 2001-2004 Sex Transm Dis 2008 05 35 5 430 4 10.1097/OLQ.0b013e3181644b5e 18446083 00007435-200805000-00002 Government of Canada 2011 Important Notice - Public Health Information Update on the Treatment for Gonococcal Infection https://www.canada.ca/en/public-health/services/infectious-diseases/sexual-health-sexually-transmitted-infections/canadian-guidelines/alerts/2011/important-notice-public-health-information-update-on-treatment-gonococcal-infection.html Guerra F Yu Y Macdonald L Menon S Pritchard J Whelan M Allen V Public Health Ontario 2018 Ontario Gonorrhea Testing and Treatment Guide, 2nd Edition https://www.publichealthontario.ca/-/media/documents/guide-gonorrhea-testing-treatment.pdf?la=en Government of Canada 2016 Canadian Guidelines on Sexually Transmitted Infections https://www.canada.ca/en/public-health/services/infectious-diseases/sexual-health-sexually-transmitted-infections/canadian-guidelines.html Public Health Agency of Canada Centre for Communicable Diseases and Infection Control, Infectious Disease Prevention and Control Branch, Public Health Agency of Canada 2017 Report on Sexually Transmitted Infections in Canada: 2013-2014https://www.canada.ca/en/public-health/services/publications/diseases-conditions/report-sexually-transmitted-infections-canada-2013-14.html Public Health Ontario 2017 01 23 Public Health Ontario Rounds: STI Series – Session 1 Overview of bacterial sexually transmitted infectionshttps://www.publichealthontario.ca/-/media/documents/sti-series-session1.pdf?la=en Public Health Ontario 2017 03 20 Public Health Ontario Rounds: STI Series – Session 3 Infectious Syphilishttps://www.publichealthontario.ca/-/media/documents/sti-series-session3.pdf?la=en Public Health Ontario 2018 Reportable Disease Trends in Ontario: Chlamydia, 2005-2017, Males, Toronto [web tool]https://www.publichealthontario.ca/en/data-and-analysis/infectious-disease/reportable-disease-trends-annually#/11 Public Health Ontario 2018 Reportable Disease Trends in Ontario: Gonorrhea, 2005-2017, Males, Toronto [web tool]https://www.publichealthontario.ca/en/data-and-analysis/infectious-disease/reportable-disease-trends-annually#/18 Public Health Ontario 2018 Reportable Disease Trends in Ontario: Syphilis, 2005-2017, Males, Toronto [web tool]https://www.publichealthontario.ca/en/data-and-analysis/infectious-disease/reportable-disease-trends-annually#/54 Gesink D Wang S Norwood T Sullivan A Al-Bargash D Shahin R Spatial epidemiology of the syphilis epidemic in Toronto, Canada Sex Transm Dis 2014 11 41 11 637 48 10.1097/OLQ.0000000000000196 25299409 00007435-201411000-00001 Fairley CK Law M Chen MY Eradicating syphilis, hepatitis C and HIV in MSM through frequent testing strategies Curr Opin Infect Dis 2014 02 27 1 56 61 10.1097/QCO.0000000000000020 24275695 Burchell AN Allen VG Grewal R MacPherson PA Rachlis A Walmsley S Mishra S Gardner SL Raboud J Cooper C Gough K Rourke SB Rousseau R Salit I Tan DH Enhanced syphilis screening among HIV-positive men (ESSAHM): a study protocol for a clinic-randomized trial with stepped wedge design Implement Sci 2016 01 16 11 8 10.1186/s13012-016-0371-0 26772390 10.1186/s13012-016-0371-0 PMC4715363 Grewal R Allen VG Gardner S Moravan V Tan DH Raboud J Bayoumi AM Kaul R Mazzulli T McGee F Rourke SB Burchell AN OHTN Cohort Study Research Team Serosorting and recreational drug use are risk factors for diagnosis of genital infection with chlamydia and gonorrhoea among HIV-positive men who have sex with men: results from a clinical cohort in Ontario, Canada Sex Transm Infect 2017 02 93 1 71 5 10.1136/sextrans-2015-052500 27154185 sextrans-2015-052500 PMC5293859 Bernstein KT Chow JM Pathela P Gift TL Bacterial sexually transmitted disease screening outside the clinic--implications for the modern sexually transmitted disease program Sex Transm Dis 2016 02 43 2 Suppl 1 S42 52 10.1097/OLQ.0000000000000343 26779687 00007435-201602001-00007 PMC5583631 British Association for Sexual Health and HIV 2018 2018-11-19 BASHH Guidelines https://www.bashh.org/guidelines Australian STI Management Guidelines 2018 2018-11-19 MSM - Men who have sex with men http://www.sti.guidelines.org.au/populations-and-situations/msm#testing-advice; Centers for Disease Control and Prevention 2018 2018-11-19 2015 Sexually Transmitted Diseases Treatment Guidelines https://www.cdc.gov/std/tg2015/specialpops.htm#MSM; Marcus JL Bernstein KT Kohn RP Liska S Philip SS Infections missed by urethral-only screening for chlamydia or gonorrhea detection among men who have sex with men Sex Transm Dis 2011 10 38 10 922 4 10.1097/OLQ.0b013e31822a2b2e 21934565 00007435-201110000-00009 Golub S STI Data From Community-Based PrEP Implementation Suggest Changes to CDC Guidelines It's Complicated: Renal Function and STIs in PrEP Users 2016 Conference on Retroviruses and Opportunistic Infections February 22-25, 2016 Boston, Massachusetts Statistics Canada 2017 Toronto, C [Census subdivision], Ontario and Toronto, CDR [Census division], Ontario (table). Census Profile. 2016 Census. Statistics Canada Catalogue no. 98-316-X2016001 https://tinyurl.com/y2adxhzd Rana J Grewal R Reinhard R Burchell A Narrative Review of Interventions to Improve Bacterial STI Testing in Men, Particularly Gay, Bisexual and Other Men Who Have Sex (gbMSM) 27th Annual Canadian Conference on HIV/AIDS Research 2018 CAHR'18 April 26-29, 2018 Vancouver, Canada 149 Burchell AN Gesink D Logie C Nelson L Rana J Wang S Lisk R Community Perspectives on Ideal Bacterial STI Testing Services for Gay, Bisexual and Other Men Who Have Sex With Men (MSM) in Toronto, Canada 28th Annual Canadian Conference on HIV/AIDS Research 2019 CAHR'19 May 9-12, 2019 Saskatoon, Canada 231 Burchell AN Rana J Guiang C Shahin R Brunetta J Mitterni L Tan D Bacon J Gilbert M Gesink D Grewal R Logie C Yeung A Lisk R Providers' Views on Barriers and Facilitators of Bacterial STI Testing Among Gay, Bisexual and Other Men Who Have Sex With Men (MSM) Who are Living With or at Risk of HIV 28th Annual Canadian Conference on HIV/AIDS Research 2019 CAHR'19 May 9-12, 2019 Saskatoon, Canada 183 Taylor MM Frasure-Williams J Burnett P Park IU Interventions to improve sexually transmitted disease screening in clinic-based settings Sex Transm Dis 2016 02 43 2 Suppl 1 S28 41 10.1097/OLQ.0000000000000294 26779685 00007435-201602001-00006 von der Gracht HA Consensus measurement in Delphi studies: review and implications for future quality assurance Technol Forecast Soc Change 2012 10 79 8 1525 36 10.1016/j.techfore.2012.04.013 Braun V Clarke V Using thematic analysis in psychology Qual Res Psychol 2006 01 3 2 77 101 10.1191/1478088706qp063oa Attride-Stirling J Thematic networks: an analytic tool for qualitative research Qual Res 2001 12 1 3 385 405 10.1177/146879410100100307 Bissessor M Fairley CK Leslie D Howley K Chen MY Frequent screening for syphilis as part of HIV monitoring increases the detection of early asymptomatic syphilis among HIV-positive homosexual men J Acquir Immune Defic Syndr 2010 10 55 2 211 6 10.1097/QAI.0b013e3181e583bf 20585261 Callander D Baker D Chen M Guy R Including syphilis testing as part of standard HIV management checks and improved syphilis screening in primary care Sex Transm Dis 2013 04 40 4 338 40 10.1097/OLQ.0b013e31828052c5 23486501 00007435-201304000-00017 Cohen CE Winston A Asboe D Boag F Mandalia S Azadian B Hawkins DA Increasing detection of asymptomatic syphilis in HIV patients Sex Transm Infect 2005 06 81 3 217 9 10.1136/sti.2004.012187 15923288 81/3/217 PMC1744980 Andersen B Eidner PO Hagensen D Lomborg S Hoff G Opportunistic screening of young men for urogenital chlamydia trachomatis infection in general practice Scand J Infect Dis 2005 37 1 35 9 10.1080/00365540510026418 15764188 GMWW2L28KN3N8760 Tebb KP Pantell RH Wibbelsman CJ Neuhaus JM Tipton AC Pecson SC Pai-Dhungat M Ko TH Shafer MB Screening sexually active adolescents for chlamydia trachomatis: what about the boys? Am J Public Health 2005 10 95 10 1806 10 10.2105/AJPH.2003.037507 16186459 95/10/1806 PMC1449440 Botes LP McAllister J Ribbons E Jin F Hillman RJ Significant increase in testing rates for sexually transmissible infections following the introduction of an anal cytological screening program, targeting HIV-positive men who have sex with men Sex Health 2011 03 8 1 76 8 10.1071/SH10027 21371387 SH10027 Lawton BA Rose SB Elley CR Bromhead C MacDonald EJ Baker MG Increasing the uptake of opportunistic chlamydia screening: a pilot study in general practice J Prim Health Care 2010 09 2 3 199 207 10.1071/hc10199 21069115 Graham S Guy RJ Wand HC Kaldor JM Donovan B Knox J McCowen D Bullen P Booker J O'Brien C Garrett K Ward JS A sexual health quality improvement program (SHIMMER) triples chlamydia and gonorrhoea testing rates among young people attending aboriginal primary health care services in Australia BMC Infect Dis 2015 09 2 15 370 10.1186/s12879-015-1107-5 26329123 10.1186/s12879-015-1107-5 PMC4557217 Patton ME Kirkcaldy RD Chang DC Markman S Yellowman M Petrosky E Adams L Robinson C Gupta A Taylor MM Increased gonorrhea screening and case finding after implementation of expanded screening criteria-urban Indian health service facility in Phoenix, Arizona, 2011-2013 Sex Transm Dis 2016 12 43 6 396 401 10.1097/OLQ.0000000000000457 27200523 00007435-201606000-00012 Ritchie S Henley R Hilton J Handy R Ingram J Mundt S Nisbet M Thomas M Briggs S Uptake, yield and resource requirements of screening for asymptomatic sexually transmissible infections among HIV-positive people attending a hospital outpatient clinic Sex Health 2014 03 11 1 67 72 10.1071/SH13167 24618022 SH13167 Andersen B Olesen F Møller JK Østergaard L Population-based strategies for outreach screening of urogenital chlamydia trachomatis infections: a randomized, controlled trial J Infect Dis 2002 01 15 185 2 252 8 10.1086/338268 11807700 JID010602 Reagan MM Xu H Shih SL Secura GM Peipert JF A randomized trial of home versus clinic-based sexually transmitted disease screening among men Sex Transm Dis 2012 11 39 11 842 7 10.1097/OLQ.0b013e3182649165 23064532 00007435-201211000-00003 PMC3476063 Scholes D Heidrich FE Yarbro P Lindenbaum JE Marrazzo JM Population-based outreach for chlamydia screening in men: results from a randomized trial Sex Transm Dis 2007 11 34 11 837 9 10.1097/OLQ.0b013e31805ba860 17538514 Kløvstad H Natås O Tverdal A Aavitsland P Systematic screening with information and home sampling for genital chlamydia trachomatis infections in young men and women in Norway: a randomized controlled trial BMC Infect Dis 2013 01 23 13 30 10.1186/1471-2334-13-30 23343391 1471-2334-13-30 PMC3558461 Smith KS Hocking JS Chen MY Fairley CK McNulty AM Read P Bradshaw CS Tabrizi SN Wand H Saville M Rawlinson W Garland SM Donovan B Kaldor JM Guy RJ Dual intervention to increase chlamydia retesting: a randomized controlled trial in three populations Am J Prev Med 2015 07 49 1 1 11 10.1016/j.amepre.2015.01.014 26094224 S0749-3797(15)00034-3 Kersaudy-Rahib D Lydié N Leroy C March L Bébéar C Arwidson P de Barbeyrac B Chlamyweb study II: a randomised controlled trial (RCT) of an online offer of home-based sampling in France Sex Transm Infect 2017 12 93 3 188 95 10.1136/sextrans-2015-052510 28377422 sextrans-2015-052510 van den Broek IV van Bergen JE Brouwers EE Fennema JS Götz HM Hoebe CJ Koekenbier RH Kretzschmar M Over EA Schmid BV Pars LL van Ravesteijn SM van der Sande MA de Wit GA Low N Op de Coul EL Effectiveness of yearly, register based screening for chlamydia in the Netherlands: controlled trial with randomised stepped wedge implementation Br Med J 2012 07 5 345 e4316 10.1136/bmj.e4316 22767614 PMC3390168 Gilbert M Salway T Haag D Fairley CK Wong J Grennan T Uddin Z Buchner CS Wong T Krajden M Tyndall M Shoveller J Ogilvie G Use of GetCheckedOnline, a comprehensive web-based testing service for sexually transmitted and blood-borne infections J Med Internet Res 2017 12 20 19 3 e81 10.2196/jmir.7097 28320690 v19i3e81 PMC5379018 Mann TA Uddin Z Hendriks AM Bouchard CJ Etches VG Get tested why not? A novel approach to internet-based chlamydia and gonorrhea testing in Canada Can J Public Health 2013 03 7 104 3 e205 9 10.17269/cjph.104.3684 23823883 Kwan KS Jachimowicz EA Bastian L Marshall L Mak DB Online chlamydia testing: an innovative approach that appeals to young people Med J Aust 2012 09 3 197 5 287 90 10.5694/mja11.11517 22938127 10.5694/mja11.11517 Götz HM van den Broek IV Hoebe CJ Brouwers EE Pars LL Fennema JS Koekenbier RH van Ravesteijn S Op de Coul EL van Bergen J High yield of reinfections by home-based automatic rescreening of chlamydia positives in a large-scale register-based screening programme and determinants of repeat infections Sex Transm Infect 2013 02 89 1 63 9 10.1136/sextrans-2011-050455 22728910 sextrans-2011-050455 Snow AF Vodstrcil LA Fairley CK El-Hayek C Cummings R Owen L Roth N Hellard ME Chen MY Introduction of a sexual health practice nurse is associated with increased STI testing of men who have sex with men in primary care BMC Infect Dis 2013 07 1 13 298 10.1186/1471-2334-13-298 23815371 1471-2334-13-298 PMC3702429 Knight V Ryder N Guy R Lu H Wand H McNulty A New xpress sexually transmissible infection screening clinic improves patient journey and clinic capacity at a large sexual health clinic Sex Transm Dis 2013 01 40 1 75 80 10.1097/OLQ.0b013e3182793700 23250305 00007435-201301000-00016 Barbee LA Tat S Dhanireddy S Marrazzo JM Implementation and operational research: effectiveness and patient acceptability of a sexually transmitted infection self-testing program in an HIV care setting J Acquir Immune Defic Syndr 2016 12 1 72 2 e26 31 10.1097/QAI.0000000000000979 26959189 PMC4868654 Malotte CK Ledsky R Hogben M Larro M Middlestadt S Lawrence JS Olthoff G Settlage RH van Devanter NL GCAP Study Group Comparison of methods to increase repeat testing in persons treated for gonorrhea and/or chlamydia at public sexually transmitted disease clinics Sex Transm Dis 2004 11 31 11 637 42 10.1097/01.olq.0000143083.38684.9d 15502669 00007435-200411000-00001 Downing SG Cashman C McNamee H Penney D Russell DB Hellard ME Increasing chlamydia test of re-infection rates using SMS reminders and incentives Sex Transm Infect 2013 02 89 1 16 9 10.1136/sextrans-2011-050454 22728911 sextrans-2011-050454 Bourne C Knight V Guy R Wand H Lu H McNulty A Short message service reminder intervention doubles sexually transmitted infection/HIV re-testing rates among men who have sex with men Sex Transm Infect 2011 04 87 3 229 31 10.1136/sti.2010.048397 21296796 sti.2010.048397 Nyatsanza F McSorley J Murphy S Brook G 'It's all in the message': the utility of personalised short message service (SMS) texts to remind patients at higher risk of STIs and HIV to reattend for testing-a repeat before and after study Sex Transm Infect 2016 12 92 5 393 5 10.1136/sextrans-2015-052216 26670912 sextrans-2015-052216 Zou H Fairley CK Guy R Bilardi J Bradshaw CS Garland SM Sze JK Afrizal A Chen MY Automated, computer generated reminders and increased detection of gonorrhoea, chlamydia and syphilis in men who have sex with men PLoS One 2013 8 4 e61972 10.1371/journal.pone.0061972 23613989 PONE-D-12-29017 PMC3629129 Paneth-Pollak R Klingler EJ Blank S Schillinger JA The elephant never forgets; piloting a chlamydia and gonorrhea retesting reminder postcard in an STD clinic setting Sex Transm Dis 2010 06 37 6 365 8 10.1097/OLQ.0b013e3181cab281 20473247 Fernando KA Fowler T Harding J Flew S Caley M Phattey J Ross JD Detecting re-infection in patients after an initial diagnosis of gonorrhoea: is routine recall for re-screening useful? Int J STD AIDS 2015 08 26 9 640 7 10.1177/0956462414548905 25161175 0956462414548905 Burton J Brook G McSorley J Murphy S The utility of short message service (SMS) texts to remind patients at higher risk of STIs and HIV to reattend for testing: a controlled before and after study Sex Transm Infect 2014 02 90 1 11 3 10.1136/sextrans-2013-051228 24064987 sextrans-2013-051228 Harte D Mercey D Jarman J Benn P Is the recall of men who have sex with men (MSM) diagnosed as having bacterial sexually transmitted infections (STIs) for re-screening a feasible and effective strategy? Sex Transm Infect 2011 12 87 7 577 82 10.1136/sextrans-2011-050144 21965470 sextrans-2011-050144 Mortimer NJ Rhee J Guy R Hayen A Lau AY A web-based personally controlled health management system increases sexually transmitted infection screening rates in young people: a randomized controlled trial J Am Med Inform Assoc 2015 07 22 4 805 14 10.1093/jamia/ocu052 25773130 ocu052 Kalwij S French S Mugezi R Baraitser P Using educational outreach and a financial incentive to increase general practices' contribution to chlamydia screening in south-east London 2003-2011 BMC Public Health 2012 09 18 12 802 10.1186/1471-2458-12-802 22984897 1471-2458-12-802 PMC3524034 Merritt TD Durrheim DN Hope K Byron P General practice intervention to increase opportunistic screening for chlamydia Sex Health 2007 12 4 4 249 51 10.1071/SH07033 18082068 SH07033 Lister NA Smith A Fairley CK Introduction of screening guidelines for men who have sex with men at an STD clinic, the Melbourne sexual health centre, Australia Sex Health 2005 2 4 241 4 10.1071/SH05006 16402672 Bissessor M Fairley CK Leslie D Chen MY Use of a computer alert increases detection of early, asymptomatic syphilis among higher-risk men who have sex with men Clin Infect Dis 2011 07 1 53 1 57 8 10.1093/cid/cir271 21653303 cir271 Israel BA Schulz AJ Parker EA Becker AB Review of community-based research: assessing partnership approaches to improve public health Annu Rev Public Health 1998 19 173 202 10.1146/annurev.publhealth.19.1.173 9611617 Damschroder LJ Aron DC Keith RE Kirsh SR Alexander JA Lowery JC Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science Implement Sci 2009 08 7 4 50 10.1186/1748-5908-4-50 19664226 1748-5908-4-50 PMC2736161