Published on in Vol 5, No 3 (2016): Jul-Sept

Get Checked… Where? The Development of a Comprehensive, Integrated Internet-Based Testing Program for Sexually Transmitted and Blood-Borne Infections in British Columbia, Canada

Get Checked… Where? The Development of a Comprehensive, Integrated Internet-Based Testing Program for Sexually Transmitted and Blood-Borne Infections in British Columbia, Canada

Get Checked… Where? The Development of a Comprehensive, Integrated Internet-Based Testing Program for Sexually Transmitted and Blood-Borne Infections in British Columbia, Canada

Original Paper

1British Columbia Centre for Disease Control, Vancouver, BC, Canada

2School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada

3College of Registered Nurses of British Columbia, Vancouver, BC, Canada

4Provincial Health Services Authority, Vancouver, BC, Canada

5British Columbia Centre for Excellence in HIV/AIDS, Vancouver, BC, Canada

6Children and Women’s Health Centre of British Columbia, Vancouver, BC, Canada

Corresponding Author:

Mark Gilbert, MD MHSc FRCPC

British Columbia Centre for Disease Control

655 West 12th Ave

Vancouver, BC, V5Z 4R4


Phone: 1 604 707 2400

Fax:1 604 707 2401


Background: Testing for sexually transmitted and blood-borne infections (STBBI) is an effective public health strategy that can promote personal control of one’s health and prevent the spread of these infections. Multiple barriers deter access to testing including fear of stigmatization, inaccurate health care provider perceptions of risk, and reduced availability of clinic services and infrastructure. Concurrent increases in sexually transmitted infection (STI) rates and demands on existing clinical services make this an even more pressing concern. Web-based testing offers several advantages that may alleviate existing clinical pressures and facilitate appropriate testing access.

Objective: This paper describes the planning, development, and usability testing of a novel Web-based testing service, GetCheckedOnline (GCO), as a complementary testing option integrated within existing sexual health services within British Columbia (BC).

Methods: From 2009 to 2014, we engaged a multidisciplinary team in the design and development of GCO. We conducted 3 initial research studies to ascertain the opinions of youth, men who have sex with men (MSM), and STI clinic clients regarding Web-based testing and elicited perspectives of sexual health care providers through focus groups. We developed an informed consent process, risk assessment questions, and test recommendations based on provincial and national guidelines and evaluated these through consultations with clinical and community stakeholders. We also conducted a preliminary health equity impact assessment whose findings also informed the GCO program mode. Finally, from April 2011 to December 2012 we gathered qualitative data from 25 participants on the functionality and usability of a GCO prototype and incorporated their recommendations into a final model.

Results: GCO launched in the fall of 2014 across 6 pilot sites in Vancouver, BC. The service involves 3 main steps: (1) create an account, complete an assessment, and print a laboratory requisition, (2) provide blood and urine specimens at participating laboratory locations, and (3) receive test results on the Internet or by phone. During this pilot phase, we promoted GCO to existing STI clinic clients and MSM in the Greater Vancouver region. A rigorous mixed-method evaluation of GCO’s uptake, acceptability, and health system impacts is currently underway.

Conclusions: GCO is the first comprehensive Web-based STBBI testing program in Canada that is integrated with existing sexual health services, with the potential to reduce pressures on existing clinical services and reach populations facing the greatest barriers to testing. Our experience highlights the facilitators and challenges of developing and implementing novel complex eHealth interventions within the health care system, and underscores the importance of considering broader implementation contexts.

JMIR Res Protoc 2016;5(3):e186



In 1999, an outbreak of syphilis among gay, bisexual, and other men who have sex with men (MSM) in San Francisco who were users of online chat room heralded the beginning of a “new era” for prevention of sexually transmitted infections (STI) [1]. The recognition that the Internet posed both a new risk environment for STI and offered unique opportunities to reach populations affected by STI has since led to the development of a wide array of Web-based interventions, including partner notification programs, tailored educational interventions, and Web-based outreach [2,3]. Around the same time in British Columbia (BC) as elsewhere, community surveys and STI clinic records demonstrated increasing use of the Internet to find sex partners by MSM and other populations at higher risk of infection [4]. As a result of these behavioral and intervention paradigm shifts, the British Columbia Centre for Disease Control (BCCDC) prioritized the development of Web-based sexual health services, starting in 2004 with the first nursing-led cyber-outreach program in Canada [5]. Following further consultation with international experts in this nascent field and assessment of local service gaps, development of a Web-based testing program for sexually transmitted and blood-borne infections (STBBI) was prioritized in 2008.

While promoting testing is a longstanding cornerstone of public health strategies for control of STBBI, multilevel barriers to appropriate testing persist in BC as in most jurisdictions. These include individual (eg, risk perception, privacy concerns, fear of disclosing sexual behavior, knowledge of testing locations, discomfort with health care professionals), provider (eg, discomfort with questions about sexual orientation, inaccurate perceptions of risk), and clinic barriers (eg, travel requirements, limited clinic hours, wait times), and these can be particularly pronounced in rural areas [6-9]. BC is also similar to other jurisdictions by having increasing STI rates and demands on STI clinical services that coincide with overall reductions in clinical capacity and infrastructure [10,11]. Web-based testing services may overcome some of these barriers and demands, as these services promote a patient-centered approach and allow clients to access testing without presenting to a provider or clinic.

Typically, Web-based testing involves visiting a website to request a home self-collection kit or to print a laboratory form to take to a laboratory to provide specimens, with results being provided through a website, text messaging, or by phone. Web-based testing programs take a variety of forms, the most common being population-based screening for a single infection (usually chlamydia by self-sampling) [12,13]. Comprehensive models testing for multiple STBBI are less common [14,15]. Programs can be stand-alone programs or be fully integrated with clinical services [15-19]. Evidence of acceptability, reach, and satisfaction with these programs is robust, confirming that Web-based STBBI testing is in high demand, particularly among youth and MSM. Some programs have demonstrated uptake of Web-based testing in high-risk or target groups and found high positivity rates, including clients from diverse socioeconomic positions and races, clients with a history of STI diagnosis, and clients with behavioral markers of STI risk [20-27]. A small number of studies have suggested Web-based testing is cost effective [25,28-30].

Recognizing the potential of Web-based testing to reduce inequities in access to testing among populations with higher infection rates, in 2009 the Provincial Health Services Authority (PHSA) awarded the BCCDC (an agency within the PHSA) 5 years of funding to develop an Online Sexual Health Services Program. This included the development of an Internet-based testing program with 3 objectives: (1) to improve sexual health by increasing the uptake and frequency of STBBI testing and earlier diagnosis, (2) to reach populations with a greater prevalence of infection and barriers to access testing, and (3) to increase the capacity of STI clinic services and improve the use of clinician resources. MSM, youth younger than 25 years, and people living in rural areas were named as initial priority populations for this service, given high infection rates and Web-based sex-seeking behaviors [11,31], barriers to accessing confidential, culturally sensitive and appropriate care [6,8], and demonstrated acceptance of Web-based sexual health interventions [32-35]. In this article, we will describe the development of this Web-based testing service for STBBI – branded GetCheckedOnline (GCO) – and plans for its evaluation. In so doing, we hope that we will also provide helpful insights for other researchers and service providers interested in developing Web-based sexual health services within complex health care systems.

Theoretical Framework

Our approach to developing GCO involved the same steps and activities typically recommended for the development of eHealth interventions. These included: using a multidisciplinary approach (with involvement of different research and provider disciplines on the development team), involvement of stakeholders including potential users throughout the process, conducting continuous and systematic evaluation throughout all phases of development, and use of robust, mixed-methods for formative and summative evaluation [36]. Our primary assumption and rationale was that GCO would reduce barriers to STBBI testing among individuals who are already motivated to test (and was not to change testing behavior among unmotivated individuals per se). As our focus was on adoption of GCO, we used elements of diffusion of innovation theory to inform the development and evaluation of GCO; for example, by considering the relative advantages and disadvantages of GCO from the perspective of potential users, identifying the characteristics of potential adopters, and considering the health system contexts in which GCO is implemented [37,38]. Finally, an important theoretical underpinning of GCO was our positioning of the intervention as being complementary to (not replacing), and fully integrated with, existing public health and clinic-based sexual health services in BC.

Planning Phase: 2009-2011

Establishing a Multidisciplinary Team

The development of GCO was led by the BCCDC Online Sexual Health Services (OSHS) program, which consisted of a medical lead, program manager, business analyst, and epidemiologist. At the outset, a program of research was established between the OSHS and the Youth Sexual Health Team, a research unit at the University of British Columbia, based on an integrated knowledge translation model (ie, where knowledge translation principles are applied to the entire research process, with involvement of knowledge users as equal partners) [39], and drawing on a range of qualitative and quantitative research disciplines including public health and clinical research, epidemiology, and social sciences.

Identifying, Engaging, and Consulting With Stakeholders

We identified 3 groups of stakeholders to engage in the development of GCO either on an ad hoc or continuous basis: internal stakeholders within the BCCDC; stakeholders within other PHSA agencies (eg, public health laboratory, privacy, information technology); and external stakeholders (Table 1).

We conducted a stakeholder analysis and developed a communication strategy for engaging stakeholders, including the development of fact sheets and standard presentations. Initially, we met with each stakeholder to provide an orientation to the concept of Web-based STBBI testing and GCO, and to discuss the nature of their involvement with GCO development. We provided ongoing updates about GCO development through a team blog and email bulletins, as well as seeking out opportunities to disseminate information about GCO through community agency networks, publications, and events [40-42]. We then established 3 stakeholder committees to guide the activities of, and provide updates about, GCO development:

1. Clinical Integration Committee (CIC): An internal decision-making group that met monthly to determine how GCO should be integrated with the provincial STI clinic at BCCDC, which would be responsible for clinical and public health follow-up of test results. The CIC was comprised of medical and nursing leads for the clinic, education and outreach programs at BCCDC.

2. Community Consultation Working Group (CCWG): An external advisory group that met biannually to ensure GCO would be a useful resource to community organizations and tailored to meet the needs of the populations they served. The CCWG was comprised of community organizations working in sexual health and STBBI prevention (including with youth and MSM).

3. Internet Services Committee (ISC): An advisory group that met monthly to receive updates about GCO and to guide its development, comprised of BCCDC, PHSA, and external stakeholders.

Reviewing the Literature and Expert Consultation

We reviewed published and gray literature to identify Web-based STBBI testing programs and evaluations of their impact, and to summarize barriers and facilitators of STBBI testing that are potentially mediated through Web-based testing. We then contacted program experts involved with comprehensive Web-based testing programs integrated with clinical services in San Francisco and Amsterdam, to learn about their experience with setting up these programs [14,15].

Based on this information and consultation, we developed a high level model for GCO to use in future consultations. This model described the program’s 3 main steps: create an account, complete a risk assessment, and print a laboratory requisition; provide blood and urine specimens (only; no oral, vaginal, or rectal swabs); receive test results (with paths described if negative, positive, or if there is a problem with the result) (Figure 1). This model did not include mailing of home test kits for human immunodeficiency virus (HIV) as these are not yet licensed for use in Canada. We also did not propose including self-collection of specimens at home as sending collected specimens potentially containing infectious agents by general mail is not permitted in Canada. In this high level model, positive results are delivered by a health care provider by phone, which is consistent with current practice at the provincial STI clinic in order to provide appropriate posttest counseling and ensure treatment and appropriate follow-up. In contrast, all negative test results would be viewed on the Internet, as is the case for many Web-based testing programs in order to reduce barriers to accessing test results (ie, by eliminating requirements to contact the clinic).

Table 1. Key stakeholders involved in the development of GetCheckedOnline.
StakeholderRole in relation to GCOaInvolvement with GCO development

Provincial sexually transmitted infection clinicResponsible for clinical aspects of GCO implementation (authority for ordering tests, review and management of results, entering results into app)Continuous; part of ISC, CIC, GWGc; key knowledge user involved in research activities

Education and outreach programsProvide support for or lead other Web-based/clinic-based sexual health services with which GCO is integratedContinuous; part of ISC

CommunicationsProvide support for media and public communicationsContinuous; part of ISC

ExecutiveResponsible for overall strategic direction and operations of BCCDC including the Division responsible for GCOAd hoc; key knowledge user involved in research activities
PHSAd (government agency within which BCCDC is located)

BC Public Health LaboratoryResponsible for conducting all tests ordered through GCOContinuous; part of ISC, GWG; key knowledge user involved in research activities

PrivacyProvides privacy-related advice on GCOContinuous; part of ISC

Risk management and legalProvides legal advice regarding risk management of GCOAd hoc

Information management/IMITSeResponsible for approving the technical specifications and final applicationContinuous; part of TWCf

ExecutiveResponsible for overall strategic direction and administration of PHSA (including BCCDC, BC Public Health Laboratory, privacy, risk management, and IMITSAd hoc; key knowledge user involved in research activities
External stakeholders

Health care providers conducting STBBIgInteract with users of GCO (eg, refer clients to GCO)Consulted during planning phase

Community organizations working with youth or men who have sex with men, and/or in sexual healthInteract with users of GCO (eg, refer clients to GCO). Promotion of GCO as part of education or outreach programs to clients.Continuous; part of ICS, CICc, CCWGh; key knowledge user involved in research activities

LifeLabsPrivate laboratory company that operates the specimen collection sites for GCOConsulted during development, testing, and implementation planning

Public Health programs in the other 6 regional or provincial health authorities in BCOversee regional public health testing initiatives with which GCO must be alignedAd hoc

Ministry of HealthSets provincial strategies for STBBI testing and oversight for provincial testing initiatives with which GCO must be alignedContinuous; part of ISC; key knowledge user involved in research activities

Professional practice regulatory bodies (College of Physicians and Surgeons of BC; College of Registered Nurses of BC)Determines acceptable scope of practice for physicians and registered nurses involved with GCOAd hoc


bBritish Columbia Centre for Disease Control.

cInternet Services Committee (ISC); Clinical Integration Committee (CIC); GetCheckedOnline Working Group (GWG).

dProvincial Health Services Authority (PHSA).

eInformation technology services (IMTS).

fTechnical working group (TWC).

gSexually transmitted and blood-borne infections testing (STBBI).

hCommunity consultation working group (CCWG).

Figure 1. High level overview of GetCheckedOnline used during formative research.
View this figure
Consulting With Potential Users

Given the importance of end-user input at early stages in the development of novel eHealth interventions, we conducted 3 research studies to ascertain the opinions of youth, MSM, and STI clinic clients about Web-based sexual health services and our Web-based testing model [32,34,43]. Overall, GCO was perceived as private, convenient, and providing greater control over testing. We elicited a number of concerns, including privacy of data and security of the app, reliance on outdated technologies (eg, printing), and anxiety at receiving a positive result on the Internet. In these studies, the participants proposed mitigation strategies for these concerns, many of which were incorporated into the GCO model (Table 2).

Determining Regulatory and Practice Requirements

As a new testing paradigm, Web-based testing raised a number of questions regarding its relation to existing regulatory and practice requirements for physicians and nurses in BC (eg, can a physician order tests recommended through a Web-based app, and if so how, and is any liability assumed?). To answer these questions, we consulted with provincial practice regulatory bodies for physicians and nurses, and the national medical protection insurance agency. We established that GCO’s model, where a BCCDC STI specialist physician is the ordering physician with results reported to and followed up by provincial STI certified registered nurses and managed according to standard protocols, is an extension of acceptable clinical practice. We also developed policies to address identified clinical nursing practice gaps, including guidelines for providing nursing services on the Internet and for emailing clients.

Table 2. Key findings from potential users on the acceptability and perceptions of Web-based sexual health services/testing and how these influenced the design of GetCheckedOnline.
ActivityKey findingsInfluence on GCOa design
Interviews and focus groups with youth to determine their perceptions of sexual health websites [32]For sexual health–related websites youth preferred practical information, professional approaches to design and content (vs colloquial or explicit language or images)Adopted professional tone using every day, noncolloquial language and select use of imagery

Interviews and focus groups with youth, MSMb, and clinic clients to determine their perceptions of Web-based testing in general and GCO specifically [43]Web-based testing perceived as convenient, offering immediate access to testing, greater privacy, reduced anxiety compared with face-to-face testing, and greater control over the testing process
Concerns about providing personal information via the Internet, potential for abuse (eg, if an account was created using email belong to someone else), distrust of security of data provided via the Internet, lack of comprehensive pretest information, lack of support for individuals receiving a positive result.
Expectations that Web-based testing would be professional, adhere to standard guidelines (and advise when different, such as lack of swabs), be fully on the Internet (eg, from booking appointments for specimen collection, to electronic ordering of tests, to getting results and prescriptions), ability to control how and when they receive notifications
Minimum data is collected with rationale for questions provided
Account creation requires email validation
Explicit privacy policy and terms of use developed to explain how data is collected, stored, and used
Advice provided for additional privacy measures (eg, clearing cache)
Detailed pretest information provided
No positive results provided via the Internet, only by phone, with links to other services accessible throughout the website
Testing reminders can be turned off
Tailored educational information is provided for other sexually transmitted and blood-borne infections testing or prevention strategies not available through GCO (eg, emergency contraception, throat, and rectal swabs)
GCO clearly identified as a program of the British Columbia Centre for Disease Control
Wording on the GCO website and promotions emphasize privacy and convenience of the service
Web-based national survey of Canadian MSM to determine intention to use Web-based testing [34]Overall intention to use Web-based testing was 72%, with little variation by participant characteristics.
Greatest perceived benefits were privacy, convenience, and testing any time.
Greatest drawbacks were inability to see a doctor or nurse, wanting to talk to someone about results, not wanting Web-based results, and low trust in the service

aGCO: GetCheckedOnline.

bMSM: men who have sex with men.

Consulting With Sexual Health Care Providers

As GCO is implemented in an existing sexual health care system where Web-based testing clients may also be accessing sexual health services at other clinics not operated by BCCDC, we conducted focus groups with local sexual health care providers to understand their opinions of GCO [44]. Providers perceived GCO as an inevitable evolution within the current system of care, with perceived benefits including shifting the locus of control from providers to patients, addressing testing barriers (eg, privacy concerns, clinic hours of operation), facilitating increased engagement in sexual health care (eg, including reminders for pap testing), and freeing up provider time and ability to see more complex patients. Providers also considered that these benefits may be offset by perpetuating existing inequities in populations GCO is trying to reach (eg, youth who do not have access to a private printer, MSM requiring swabs for diagnosis of oral or rectal STI) or predominantly being used by individuals who already have the resources necessary to access testing (eg, tech “savvy,” higher income). A number of potential personal or clinical harms were identified, such as anxiety at receipt of result notifications, repeated use by individuals at lower risk (“worried well”), misunderstanding test limitations, such as window periods, inadequate pre- and posttest counseling, and missed opportunities for education and prevention such as contraception. As with potential users, mitigation strategies for these harms were also proposed and incorporated into GCO (Table 3). At the same time, providers recognized that many of these harms could also occur within face-to-face clinical testing encounters.

Table 3. Potential harms and mitigation strategies recommended by sexual health care providers, and how these were addressed in the design of GetCheckedOnline.
Potential harmRecommended mitigation strategyHow addressed
Anxiety related to viewing email notification or retrieving voicemail (if positive) outside of clinic hoursProvide after-hours support, send notifications early in the day
Notification emails should be generic and not include results
Links to BCCDCa sexual health website and provincial after-hours support services
Generic wording used for notification emails

Not addressing underlying anxiety of repeat tests by the “worried well”Ability to monitor and intervene if appropriate (eg, refer to clinic for care)Monitored during the pilot evaluation
Clinic protocol developed to handle this scenario
Misunderstanding information on the website, such as window periods, symptomsEnsure appropriate educational content on website related to test limitations and symptomsInformation accessible throughout the site related to test limitations and window periods
Links to British Columbia Centre for Disease Control sexual health website for more information about symptoms
Inadequate pre- and posttest counselingProvide equivalent information on website, with some mandatory information
Include clear consent process and disclaimer regarding limitations of Web-based testing
Content from provincial pre/posttest guidelines incorporated, with mandatory and optional content
Consent page including acknowledgement of limitations as final step before printing requisition
Missed opportunities for education and prevention that can be elicited during clinical testing encountersInclude information and referrals for pap testing, human papilloma virus vaccineTailored recommendations for sexually transmitted and blood-borne infections prevention provided based on assessment responses, including vaccines, oral and rectal swabs, emergency contraception, HIV postexposure prophylaxis
Does not include all potentially relevant tests (eg, Hepatitis C, swabs)Include Hepatitis C testing
Have clear referrals to clinics for other tests
On assessment include question about specific sexual acts (oral, vaginal, anal) and recommend swabs if appropriate
Explain why certain tests are not offered
Hepatitis C testing included for men who have sex with men, or history of injection drug use
Swabs prioritized for inclusion after implementation
Not answering assessment questions accurately and inappropriate tests recommended (or not)Give option to skip assessment and recommend all tests
Encourage clients to provide accurate information (through disclaimer, encourage to select “prefer not to answer” option)
Importance of providing accurate information emphasized
Clients have option of deselecting any recommended tests
Positive results not followed up because of client providing fake contact informationEncourage use of real name and phone numberImportance of using real name or consistent pseudonym, and providing telephone number emphasized

aBCCDC: British Columbia Centre for Disease Control.

Development, Usability Testing, and Revision: 2011-2014

Stakeholder and end user consultation continued throughout the development, usability testing, and revision stages (details shown in Figure 2).

We established a GCO Working Group including the core GCO development team and representatives from the BCCDC clinic (nursing and clerical) and the PHSA Public Health Laboratory (which conducts the testing for GCO specimens). The working group met approximately every 2 weeks to develop the detailed requirements (ie, blueprint) for the GCO app and the final GCO service. In addition, we established a GCO Technical Working Group, comprised of the GCO development team and representatives from different program areas in Information Technology (IT) (eg, databases, servers, network security). The group met every 2 weeks to develop the architecture, hosting, and other technical requirements for the app. The final model was informed by findings from the planning phase as well as the additional activities below.

Figure 2. Key players involved in the development, testing, and revision of the GetCheckedOnline app.
View this figure
Assessing GetCheckedOnline’s Impact on Health Equity

The “techno-optimism” with which Web-based health interventions are viewed is tempered by the reality that their adoption is patterned along social gradients (eg, digital divides) [45,46]. New testing technologies such as GCO may reinforce or reproduce the relationship between social position and health status if only taken up by individuals who already have the resources needed to access clinic-based STBBI testing (eg, social capital, education) [47]. We conducted a preliminary health equity impact assessment (HEIA; screening, scoping) consisting of a literature review and expert consultations. In so doing, we identified ways in which GCO was likely to reinforce or circumvent health inequities in sexual health for historically underserved and marginalized populations with a higher burden of STBBI (including: youth; MSM; people from ethnocultural minorities; intersex, transgender or gender variant populations; Indigenous people; residents of rural areas) [48]. HEIA recommendations that were incorporated into GCO design included: collecting information on ethnicity and gender identity; avoiding normative and stigmatizing language and images; expanding testing options to include hepatitis C.

Meeting Requirements for Pretest Counseling and Informed Consent

As flagged during consultations with end-users and sexual health care providers, we needed to determine how provider-delivered pretest counseling and the obtaining of informed consent could be translated to a Web-based app. We did so by: (1) reviewing the published literature to determine the effectiveness of alternate models of providing pretest counseling (eg, videos, written information), (2) reviewing national and provincial testing policies and procedures, and (3) consulting with a clinical ethicist, privacy advisor, legal counsel, and provincial nursing practice leads to determine the necessary app requirements in order to obtain informed consent on the Internet (eg, mandatory vs optional steps). We included a specific consent step on GCO that must be completed by users prior to printing a laboratory test requisition, and conducted individual interviews with end-users following their participation in usability testing of the GCO app to probe specifically about their perceptions of the consent webpage (which overall were favorable) [49].

Determining Risk Assessment Questions and Test Recommendations

As GCO was conceptualized as an extension of clinical STBBI testing services offered by BCCDC, we intended the assessment step and testing recommendations to mirror routine clinical practice as much as possible. We reviewed national and provincial guidelines for STBBI testing, treatment and scope of practice, and the BC epidemiology of STBBI in our target populations. To identify assessment questions that could be used for recommending specific tests and tailored educational messages, we reviewed existing Web-based risk assessment tools and the published literature to identify models or variables that were predictive of STI infection [50,51].

Our final set of assessment questions, testing recommendations, and tailored messages were evaluated through usability testing and consultation with clinical and community stakeholders. These included routine recommendation for all clients of urine testing for chlamydia and gonorrhea, and serum testing for HIV and syphilis (as these are generally routinely recommended for BCCDC STI clinic clients and MSM); serum testing for hepatitis C was also recommended for MSM (optional) and individuals with a history of injection drug use. Clients can opt out (deselect) any of the recommended tests. Assessment questions were also designed to (1) provide tailored recommendations for additional testing or prevention interventions that may be indicated (including HIV postexposure prophylaxis, emergency contraception, and need for oral and/or rectal swabs for chlamydia and gonorrhea testing), and (2) to determine the recommended frequency of testing in order to set up testing reminders that are sent by email to GCO clients (ie, 3, 6, or 12 months).

Usability Testing and Revision

As the incorporation of user feedback is a key component of app development, we conducted testing of the website with potential end-users at 3 separate points during the development process. All usability testing was done in-person with participants recruited through Web-based advertising and from attendees of the Provincial STI Clinic at BCCDC. The first round of testing, performed on a prototype with 10 participants, was designed to observe how users interacted with the app and to validate the risk assessment; outcomes informed numerous changes to the app user interface. The second round of testing, with 8 participants, was performed on a functional but incomplete version of the website to test the overall functioning and content of the app; results led to a redesign of the homepage and changes to the website information architecture. The final round of usability testing, with 14 participants, was completed on a fully functional version of the website to test different options for presenting information via the homepage and the informed consent page; feedback informed the final homepage design and validated the informed consent process.

Final Model

The final model and flow diagram for GCO [52] is shown in Figure 3. The app screenshots and a video walk-through are included in Multimedia Appendices 1 and 2, respectively.

In brief, potential users (clients) of GCO can create an account if they have an access code (included with promotional materials, or provided by the BCCDC STI clinic or a local sexual health care provider) or have provided an email address, which is then used by BCCDC staff to send an invitation to create an account. During the account creation process, the client provides an email address to serve as their login (which is subsequently verified) and chooses a password. Both mandatory (name, date of birth, sex; not verified) and optional information (phone number, first 3 digits of postal code, ethnicity) are collected. Clients are advised to use their real name or initials although without verification pseudonyms are possible, a model consistent with low-threshold clinic-based testing services offered through BCCDC in order to reduce barriers for clients with privacy concerns. Clients are asked to provide consent to be contacted for research purposes, and must indicate their understanding of the terms of use and privacy policy. While there is no minimum age requirement for use of GCO, individuals less than 19 years are defined as children in BC legislation and can consent to their own medical care if capable. All clients indicating an age less than 19 years of age are recommended to seek testing at an STI clinic in order that capacity to provide consent can be assessed. However, as the majority of clients using GCO less than 19 years of age are expected to be capable of providing consent they are not barred from proceeding.

Clients then complete the first part of the assessment, where the questions are used to identify clients who have symptoms or are a contact to someone with an STBBI. These clients are subsequently recommended not to test through GCO but to seek standard clinical STBBI testing as other tests or immediate treatment may be indicated; clients do have the option to acknowledge the recommendation and continue with Web-based testing. The second part of the assessment includes questions used to recommend hepatitis C testing and provide tailored educational messages. Next, the GCO app provides test recommendations and educational messages as described above. Clients can deselect tests if desired, proceed to the consent page to indicate their understanding of key pretest counseling messages, and then print their laboratory test requisition (which is saved for later printing if needed).

Clients present their laboratory test requisitions, on which their name has been replaced with a unique GCO client code, at a private laboratory specimen collection site and provide urine and/or blood specimens. Specimens are shipped to the PHSA Public Health Laboratory for testing, with results reported to the BCCDC STI clinic for appropriate management. All results are entered into the GCO app by clerical staff who then trigger the app to send a notification email to the client that their results are ready to view. If all results are negative, the client can see their results in their GCO account and there is no interaction with clinic staff. If any of the results are invalid (for example, a problem occurred with the specimen), clients will view any negative test results and see a notification to call the BCCDC clinic to arrange for retesting for the invalid result. If any of the results are positive, the client can see only a message directing them to call the BCCDC clinic for their results; at the same time, if the client has provided a phone number a BCCDC clinic nurse will attempt to contact the client directly. For each client, a testing history is maintained on the app with dates of testing and test types; test results are not retained within GCO after 1 month as a privacy and security precaution. Testing reminders are set for all clients at 3 or 12 months based on the degree of sexual risk reported on the assessment questions; clients can opt out or change these settings.

Figure 3. GetCheckedOnline program model demonstrating interactions between clients, clinicians, laboratories, and the GetCheckedOnline app.
View this figure
Preparation for Implementation

Preparing for implementation of the GCO service involved numerous activities related to privacy and security, IT support, operational protocols, reporting, communications, and final validation. We collaborated closely with PHSA Privacy, Risk Management, and IT Security teams to conduct a thorough privacy impact assessment and security threat and risk assessment that included testing of the app’s security model by an external vendor (penetration testing); recommendations from these assessments were incorporated into the final app. A privacy policy and terms of use specific to the program were developed, along with administrative policies around access control, auditing and data reporting, and all were vetted with PHSA Privacy and IT Security. A new software vendor was contracted to provide ongoing app support in partnership with PHSA IT, and extensive documentation was completed to establish internal IT support around client services, network, hardware, and software maintenance. Clinical procedures for the management of GCO clients were developed in consultation with clinical and clerical team leads, and multiple training sessions were conducted to familiarize staff with procedures and the app itself. A series of reports were developed to routinely monitor testing volumes, uptake of the service and drop-off of clients at different points in the testing process. Promotional materials (ie, sign-up sheets, posters, wallet cards, and brochures) were designed and a communications package was created, including email templates for the launch announcement, sample content for social media, a one-page overview of the program, and frequently asked questions. The last step prior to the official launch of the service was a comprehensive final validation (beta-testing) of the app, specimen collection and transport, specimen testing, and clinical procedures using actual clients; where possible, we used feedback from the testing to improve the overall user experience.

Funding Model

The pilot phase of GCO is fully funded by the PHSA, with costs shared between the BCCDC (costs of program operation, app revision, specimen collection) and the BC Public Health Laboratory (BCPHL) (costs of laboratory testing).

GetCheckedOnline Pilot Phase

GCO [52] went live in September 2014, with 6 participating private laboratory specimen collection sites in Vancouver, BC and is now sustained through ongoing operational funding. During this pilot phase, we promoted GCO to existing BCCDC STI clinic clients and subsequently to MSM in the Vancouver region in April 2015. This pilot phase lasted until December 2015 and an evaluation of this pilot phase is underway (eg, number of accounts created, number of specimens submitted, positivity rates, and treatment outcomes).

How GetCheckedOnline Will be Evaluated

As recommended for the evaluation of eHealth interventions, we will use mixed-methods to evaluate the impact of GCO at individual, population, and health service delivery levels [53,54] (Table 4). Our objectives and outcomes of interest were identified from stakeholder consultations and by reviewing the eHealth implementation literature (eg, acceptability, mitigation of testing barriers, risk behavior, HIV knowledge, treatment and follow-up, uptake, reach). These will be evaluated using 5 methods (Textbox 1) funded through research grants obtained from the Canadian Institutes of Health Research [Gilbert et al, unpublished data 2011 and 2014].

Five methods to evaluate objectives and outcomes.

Web-based survey of GetCheckedOnline (GCO) and British Columbia Centre for Disease Control (BCCDC) sexually transmitted infection (STI) clinic clients following a Web- or clinic-based testing encounter (baseline) and 3 months later

•To measure acceptability of GCO; identify characteristics of GCO compared with clinic clients; compare baseline and short-term HIV knowledge, and risk behavior.

Web-based surveys and community intercept surveys of men who have sex with men (MSM)

•To measure acceptability of GCO; measure awareness and diffusion of GCO among networks; identify characteristics associated with uptake; assess reach to MSM most at-risk of infection.

Interviews with individuals testing through GCO

•To measure acceptability of GCO; identify if and how GCO mitigates testing barriers.

An administrative data cohort using retrospective and prospective longitudinal testing data for GCO and BCCDC STI clinic clients

•To measure acceptability (repeated use) of GCO; identify differences in testing, treatment and partner notification, test frequency, and infection rates.

Analysis of GCO and BCCDC STI clinic health services data (eg, tests conducted, number of clinic and drop-in visits, estimates of physician and nursing time)

•To determine changes in STI clinic staff configuration, staff tasks, overall clinic capacity, and laboratory testing volumes following GCO implementation.

Textbox 1. Five methods to evaluate objectives and outcomes.
Table 4. Evaluation matrix showing level of potential impact, objectives, data collection methods, and metrics.
Level of impactObjective to determineData collection method(s)Outcome measures

The acceptability of GCOa (among both clients using the service and prospective clients)Virtual cohortPercentage and characteristics of clients who repeat-test

Web-based client surveySelf-reported satisfaction and willingness to refer a friend

Web-based community surveyIntention to use GCO (prospective clients)

Client interviewsQualitative analysis of comments on experience with GCO

How GCO mitigates existing barriers to accessing STIb/HIV testingClient interviewsAnalysis of self-described factors which facilitate or limit clients’ opportunities to access in-clinic or Web-based STI/HIV testing

If GCO clients have any short-term differences in risk behavior and posttest HIV knowledge in comparison to clinic-based clients receiving traditional in-person pre/posttest counselingWeb-based client survey
(0 vs 3 months)
Risk behavior measures; 5-point true/false scale including items related to HIV transmission, risk reduction, testing, and public health follow-up

If outcomes differ for clients testing positive via GCO (ie, are less likely to access STI treatment, or to be reached by public health for follow-up including partner notification)Virtual cohortPercent of those who test positive who access treatment and public health follow-up

The diffusion of GCO into priority populations (ie, men who have sex with men in Phase 1)Web-based community surveyPercent of respondents who have heard of GCO, used GCO, and seen promotional materials

The client characteristics associated with uptake and nonuptake of GCOWeb-based community surveyEthnicity, education, income, STI/HIV testing history, sexual risk behaviors, perceptions of GCO, use of other health services and Web-based services

Whether GCO reaches individuals who are most at-risk of infectionWeb-based client surveyMeasures of sexual risk behavior

Web-based community surveyMeasures of sexual risk behavior

Whether GCO clients have higher rates of infection than those testing in-clinicVirtual cohortIncidence of infection (HIV, chlamydia, gonorrhea, syphilis)

Web-based community surveyPercent reporting recent STI or HIV diagnosis

If GCO results in increased test frequency and earlier diagnosis among individuals most at-risk of infectionVirtual cohortPercent of clients who repeat-test and intertest intervals (including interval between positive test and last negative test)
Health services delivery

What changes in staff configuration and tasks will occur as GCO is integrated with existing clinic sexual health servicesSexual health systems dataEstimates of total/aggregate clerical and clinical staff time spent entering test results into system, seeing asymptomatic clients in-clinic, delivering test results, and following-up with positive cases; number of episodes and estimated clerical time spent on GCO user support

If the introduction of GCO increases the capacity of existing clinic-based sexual health services
Number of drop-in appointments and turn-aways ; number and types of STI/HIV tests conducted

The impact on laboratory testing volume as a result of introducing GCO
Number and types of STI/HIV tests conducted

aGCO: GetCheckedOnline.

bSTI: sexually transmitted infection.

Model Changes Following Implementation

We made 2 major changes to the GCO model following implementation, reflecting further refinement based on the findings of the planning and development phases:

In the fall of 2015, we revised the risk assessment questions in tandem with an update of our preliminary HEIA in order to improve their appropriateness for clients of diverse gender identities. We also made revisions to prospectively collect the necessary variables to validate clinical prediction rules (CPR) developed by our team for urine testing for chlamydia and gonorrhea, and blood testing for HIV. These CPR were developed using test results from over 30,000 asymptomatic clients at STI clinics across BC [55,56]. Once these are further validated using prospective GCO data, we aim to include these CPRs prior to promoting GCO outside of our target populations, where our current model for recommending tests may not be appropriate.

In February 2016, we added self-collected rectal and throat swabs for chlamydia and gonorrhea testing. Swabs are recommended for MSM clients if reporting giving oral or receiving anal sex during the assessment step, women if reporting receptive anal sex (as per routine clinical practice). While always intended for future versions of GCO, we accelerated the inclusion of swabs as their absence was flagged as a clinical risk during provider consultations and a potential exacerbation of health inequities for MSM. Clients are given self-collection kits at the private laboratory collection sites when providing urine and/or blood samples, and clients can either self-swab on-site or take home for self-collection and return. Self-collection instruction guides (Multimedia Appendix 3) were developed through reviewing examples found on the Internet with 2 focus groups of potential users, and pilot tested with 12 users who found the guides easy to understand, sensitive to various genders and sexual identities, and conducive to successful self-collection [57].

A Unique Opportunity

GCO is the first comprehensive Web-based STBBI testing program in Canada, with few global counterparts, and is integrated with existing sexual health services. It represents a new paradigm for offering testing services that we believe has great potential to reach populations in BC that have high rates of infection and face the greatest barriers to accessing testing. With other health authorities recognizing this potential, and GCO’s alignment with recent provincial strategies and funding for expanding HIV testing services in BC [58], planning for scale-up to other regions began in early 2014, with the first specimen collection sites coming on board in March 2016. We have a unique opportunity to comprehensively study the implementation, impact, and transferability of a Web-based health service intervention within the Canadian health care system and across diverse populations and settings in BC. We will be conducting the research necessary to determine if we have successfully achieved our objectives both in Vancouver and in subsequent scale-up, paying particular attention to examining whether GCO improves or exacerbates existing health inequities (ie, whether the rhetoric of eHealth interventions matches the reality of their implementation) [45].

While we did not adopt a specific theoretical framework, our approach to the development of GCO was most consistent with van Gemert-Pijnen and colleagues’ [36] principles for a holistic approach to developing eHealth technologies by: using a participatory process with involvement of stakeholders throughout, including potential users; applying continuous evaluation cycles that are iterative, flexible, and dynamic; considering conditions necessary for implementation from the outset; recognizing that GCO will change the organization of health care; use of persuasive design techniques; and use of advanced methods to assess impact. As others have recommended for successful development of eHealth apps, we also believe that a critical factor in the development of GCO was the skill set of our multidisciplinary team, including early establishing of research partnerships [59]. We would also particularly emphasize the iterative, flexible, and dynamic nature of the formative evaluation and work needed to develop GCO, which we have described as a sequential series of discrete steps out of necessity. In reality, these activities were overlapping, interconnected, and mutually reinforcing of our final program model.

Challenges With Developing GetCheckedOnline

An astute reader will have noted that while funded in 2009, GCO was not implemented until 2014, which was longer than we had anticipated. As a complex health system intervention involving multiple sectors, the development of GCO was dependent on the capacity and competing priorities of key stakeholders outside (and beyond the control of) the BCCDC. For example, the PHSA and other health authorities in the greater Vancouver region underwent a reorganization and consolidation of technical and support services during this time period, leading to substantial delays in engagement of stakeholders needed to provide the internal technical support essential to developing a Web-based intervention. Characteristics of GCO were also ground-breaking within PHSA, including automated use of email notifications and allowing patients direct Web-based access to their own personal health information. Developing the policy and technical infrastructure to support these novel aspects of GCO did take time to address, yet this has paved the way for the implementation of other eHealth apps within these agencies. However, institutional barriers remain that prevent the inclusion of features that our formative research indicates are desired by potential users of GCO. Most notably, our requirement for GCO clients to print a laboratory requisition was widely regarded as a barrier to using the service, yet receipt and storage of all laboratory requisitions is a laboratory accreditation requirement. Electronic ordering of laboratory tests is not yet widespread in Canada outside of specific electronic medical record systems such as in hospitals. Our private laboratory partner, which carries out the specimen collection is in the process of establishing electronic ordering of laboratory tests that we anticipate will be in place in the next year. This will permit barcode scanning from a client’s smartphone, which we are planning to include in GCO once this is possible.

Implications for Development of eHealth Interventions

Our experience speaks to the challenges of developing and implementing novel, complex eHealth interventions, and adds weight to recommendations to expand technology adoption models to consider the role of broader implementation contexts that both facilitate and challenge the development and uptake of Web-based/digital health services [60,61]. The organizational context is particularly important; for example, in our experience, the commitment of ongoing operational funds, and a health agency environment that seeks to foster innovations in health care, have been critical to the successful implementation of GCO.

It is striking that while developing and advancing eHealth interventions is widely prioritized, there is relatively little practical guidance on their implementation. We hope that our detailed description of the steps taken to plan and develop GCO will be helpful not just to other jurisdictions developing similar Web-based testing programs, but more broadly to developers of similarly complex interventions that are integrated within health care systems.


All authors have contributed to and reviewed this manuscript. We also would like to thank Kimberly Thomson for her assistance in preparing this manuscript. Funding for the research activities described in this manuscript were provided by the Canadian Institutes of Health Research Partnerships for Health System Improvement grants (PHE-114129; PHE-318068) and Operating Grant (MOP-106440) awarded to JS, MG, and GO.

Authors' Contributions

All authors contributed to the development of the GetCheckedOnline app and writing of this manuscript. MG, DH and TH conducted the initial review of Web-based testing interventions; MG and TS led the Web-based survey analysis; JS, CC, TH, JF led the interviews and focus groups; AB led the informed consent evaluation; JF led the health equity impact evaluation. MG, DH, TH, and MB documented the development, implementation, and usability process and results. MG, GO, and JS lead the ongoing program evaluation.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Screenshots of the GCO application.

PDF File (Adobe PDF File), 19MB

Multimedia Appendix 2

Video walk-through of the GCO application.

MP4 File (MP4 Video), 145MB

Multimedia Appendix 3

Self-collection instruction guides for oral and rectal swabs.

PDF File (Adobe PDF File), 2MB

  1. Klausner JD, Wolf W, Fischer-Ponce L, Zolt I, Katz MH. Tracing a syphilis outbreak through cyberspace. JAMA 2000;284:447-449. [Medline]
  2. McFarlane M, Bull SS, Rietmeijer CA. The Internet as a newly emerging risk environment for sexually transmitted diseases. JAMA 2000;284:443-446. [Medline]
  3. McFarlane M, Bull S. Behavioral Interventions for Prevention and Control of Sexually Transmitted Diseases. Springer US.). New York: Springer; 2007.
  4. Ogilvie GS, Taylor DL, Trussler T, Marchand R, Gilbert M, Moniruzzaman A, et al. Seeking sexual partners on the internet: a marker for risky sexual behaviour in men who have sex with men. Can J Public Health 2008;99:185-188. [Medline]
  5. Sandstra IL, Gold F, Jones E, Harris P, Taylor D. Cyber outreach: ST/HIV education online. Can Nurse 2008 Jun;104:24-28, 30. [Medline]
  6. Goldenberg S, Shoveller J, Koehoorn M, Ostry A. Barriers to STI testing among youth in a Canadian oil and gas community. Health Place 2008;14:718-729. [CrossRef] [Medline]
  7. Goldenberg SM, Shoveller JA, Ostry AC, Koehoorn M. Sexually transmitted infection (STI) testing among young oil and gas workers: the need for innovative, place-based approaches to STI control. Can J Public Health 2008;99:350-354. [Medline]
  8. Shoveller JA, Knight R, Johnson J, Oliffe JL, Goldenberg S. 'Not the swab!' Young men's experiences with STI testing. Sociol Health Illn 2010;32:57-73 [FREE Full text] [CrossRef] [Medline]
  9. Rusch M, Shoveller J, Burgess S, Stancer K, Patrick D, Tyndall M. Association of sexually transmitted disease-related stigma with sexual health care among women attending a community clinic program. Sex Transm Dis 2008;35:553-557. [CrossRef] [Medline]
  10. Wong W, Miller S, Rabins C, Bertrand T, Thompson K, Aubin M, et al. STD program capacity and preparedness in the United States: results of a national survey. : National STD Prevention Conference; 2009 Presented at: National STD Prevention Conference; March 8-11, 2010; Atlanta p. 8-11.
  11. British Columbia Centre for Disease Control. STI in British Columbia: Annual Surveillance Report 2014. Vancouver BC; 2015.   URL: [accessed 2016-06-23] [WebCite Cache]
  12. Gaydos CA, Dwyer K, Barnes M, Rizzo-Price PA, Wood BJ, Flemming T, et al. Internet-based screening for Chlamydia trachomatis to reach non-clinic populations with mailed self-administered vaginal swabs. Sex Transm Dis 2006;33:451-457. [CrossRef] [Medline]
  13. Woodhall SC, Sile B, Talebi A, Nardone A, Baraitser P. Internet testing for Chlamydia trachomatis in England, 2006 to 2010. BMC Public Health 2012;12:1095 [FREE Full text] [CrossRef] [Medline]
  14. Koekenbier R, Fennerna H, Van Leent E, Davidovich U. The STI outpatient clinic online: the use and efficacy of online Syphilis testing and the development of an full-scale testing application online. 2007 Presented at: Mednet 2007; October 7-10, 2007; Leipzig, Germany.
  15. Levine D. Changemakers. 2007. - Free, Accessible, Online STD/HIV Testing   URL: [accessed 2016-06-23] [WebCite Cache]
  16. Gaydos C, Barnes M, Aumakhan B, Quinn N, Agreda P, Whittle P, et al. Characteristics of users of the internet who submit self-obtained urogenital samples through the mail for testing for sexually transmitted infections. 2008 Presented at: National STD Prevention Conference; March 10-13, 2008; Atlanta, Georgia p. 10-13.
  17. Koekenbier RH, Davidovich U, van Leent E, Thiesbrummel H, Fennema H. Online-mediated syphilis testing: feasibility, efficacy, and usage. Sex Transm Dis 2008;35:764-749. [CrossRef] [Medline]
  18. Mann TA, Uddin Z, Hendriks AM, Bouchard CJ, Etches VG. Get Tested Why Not? A novel approach to internet-based chlamydia and gonorrhea testing in Canada. Can J Public Health 2013;104:e205-e209. [Medline]
  19. van den Broek I, van Bergen J, Brouwers E, Fennema J, Götz HM, Hoebe C, et al. Effectiveness of yearly, register based screening for chlamydia in the Netherlands: controlled trial with randomised stepped wedge implementation. BMJ 2012;345:e4316 [FREE Full text] [Medline]
  20. Gaydos CA, Barnes M, Aumakhan B, Quinn N, Agreda P, Whittle P, et al. Can e-technology through the Internet be used as a new tool to address the Chlamydia trachomatis epidemic by home sampling and vaginal swabs? Sex Transm Dis 2009;36:577-580 [FREE Full text] [CrossRef] [Medline]
  21. Gaydos CA, Barnes M, Jett-Goheen M, Quinn N, Whittle P, Hogan T, et al. Characteristics and predictors of women who obtain rescreening for sexually transmitted infections using the screening programme. Int J STD AIDS 2013;24:736-744 [FREE Full text] [CrossRef] [Medline]
  22. Novak D, Novak M. Use of the Internet for home testing for Chlamydia trachomatis in Sweden: who are the users? Int J STD AIDS 2012;23:83-87. [CrossRef] [Medline]
  23. Op de Coul E, Götz HM, van Bergen J, Fennema J, Hoebe C, Koekenbier RH, et al. Who participates in the Dutch Chlamydia screening? A study on demographic and behavioral correlates of participation and positivity. Sex Transm Dis 2012;39:97-103. [CrossRef] [Medline]
  24. Götz HM, van den Broek I, Hoebe C, Brouwers E, Pars LL, Fennema J, et al. High yield of reinfections by home-based automatic rescreening of Chlamydia positives in a large-scale register-based screening programme and determinants of repeat infections. Sex Transm Infect 2013;89:63-69. [CrossRef] [Medline]
  25. Bracebridge S, Bachmann MO, Ramkhelawon K, Woolnough A. Evaluation of a systematic postal screening and treatment service for genital Chlamydia trachomatis, with remote clinic access via the internet: a cross-sectional study, East of England. Sex Transm Infect 2012;88:375-381. [CrossRef] [Medline]
  26. Dokkum NF, Koekenbier RH, van den Broek I, van Bergen J, Brouwers E, Fennema J, et al. Keeping participants on board: increasing uptake by automated respondent reminders in an Internet-based chlamydia screening in the Netherlands. BMC Public Health 2012;12:176 [FREE Full text] [CrossRef] [Medline]
  27. Ladd J, Hsieh Y, Barnes M, Quinn N, Jett-Goheen M, Gaydos CA. Female users of internet-based screening for rectal STIs: descriptive statistics and correlates of positivity. Sex Transm Infect 2014;90:485-490 [FREE Full text] [CrossRef] [Medline]
  28. Jenkins W, Rabins C, Barnes M, Agreda P, Gaydos C. Use of the internet and self-collected samples as a sexually transmissible infection intervention in rural Illinois communities. Sex Health 2011;8:79-85. [Medline]
  29. Spielberg F, Levy V, Lensing S, Chattopadhyay I, Venkatasubramanian L, Acevedo N, et al. Fully integrated e-services for prevention, diagnosis, and treatment of sexually transmitted infections: results of a 4-county study in California. Am J Public Health 2014;104:2313-2320. [CrossRef] [Medline]
  30. Huang W, Gaydos C, Barnes M, Jett-Goheen M, Blake D. Cost-effectiveness analysis of Chlamydia trachomatis screening via internet-based self-collected swabs compared with clinic-based sample collection. Sex Transm Dis 2011;38:815-820. [Medline]
  31. Ling D, Janjua N, Wong S, Krajden M, Hoang L, Morshed M, et al. Sexually transmitted infection trends among gay or bisexual men from a clinic-based sentinel surveillance system in British Columbia, Canada. Sex Transm Dis 2015;42:153-159. [Medline]
  32. Shoveller J, Knight R, Davis W, Gilbert M, Ogilvie G. Online sexual health services: examining youth's perspectives. Can J Public Health 2012;103:14-18. [Medline]
  33. Flicker S, Goldberg E, Read S, Veinot T, McClelland A, Saulnier P, et al. HIV-positive youth's perspectives on the Internet and e-health. J Med Internet Res 2004 Sep 29;6:e32 [FREE Full text] [CrossRef] [Medline]
  34. Gilbert M, Hottes TS, Kerr T, Taylor D, Fairley CK, Lester R, et al. Factors associated with intention to use internet-based testing for sexually transmitted infections among men who have sex with men. J Med Internet Res 2013;15:e254 [FREE Full text] [CrossRef] [Medline]
  35. Kubicek K, Carpineto J, McDavitt B, Weiss G, Kipke MD. Use and perceptions of the internet for sexual information and partners: a study of young men who have sex with men. Arch Sex Behav 2011;40:803-816 [FREE Full text] [CrossRef] [Medline]
  36. van Gemert-Pijnen J, Nijland N, van LM, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res 2011;13:e111 [FREE Full text] [CrossRef] [Medline]
  37. Berwick DM. Disseminating innovations in health care. JAMA 2003;289:1969-1975. [CrossRef] [Medline]
  38. Haider M, Kreps G. Forty years of diffusion of innovations: utility and value in public health. J Health Commun Suppl 2004;9:1-11. [Medline]
  39. Canadian Institutes of Health Research. Guide to Knowledge Translation Planning at CIHR: Integrated End-of-Grant Approaches. Ottawa: Canadian Institutes of Health Research; 2015 Mar 19.   URL: [accessed 2016-06-23] [WebCite Cache]
  40. Lovebytes: Researching Digital Interventions to Improve Sexual Health.   URL: [accessed 2016-06-23] [WebCite Cache]
  41. Hottes T, Gilbert M, Haag D. From waiting rooms to websites: online testing for sexually transmitted infections is being developed for BC. Living Positive 2011;71:25.
  42. Gilbert M, Hottes T. Pacific AIDS Network. 2012. Guest Blog: On the Internet, Nobody Knows Who You Snog   URL: [accessed 2016-06-23] [WebCite Cache]
  43. Hottes TS, Farrell J, Bondyra M, Haag D, Shoveller J, Gilbert M. Internet-based HIV and sexually transmitted infection testing in British Columbia, Canada: opinions and expectations of prospective clients. J Med Internet Res 2012;14:e41 [FREE Full text] [CrossRef] [Medline]
  44. Gilbert M, Hottes T, Chabot C, Haag D, Shoveller J, Ogilvie G. “There are a million scenarios to consider”: health care provider perspectives on internet-based testing for sexually transmitted infections, HIV and hepatitis C in British Columbia. Vienna; 2013 Presented at: STI & AIDS World Congress; July 14-17, 2013; Vienna.
  45. Newman L, Frank O. The rhetoric and reality of e-health: a critical assessment of the benefits of e-health in primary health care. Aust J Prim Health 2013;19:265-269. [CrossRef] [Medline]
  46. Baur C. An analysis of factors underlying e-health disparities. Camb Q Healthc Ethics 2008;17:417-428. [CrossRef] [Medline]
  47. Phelan JC, Link BG, Tehranifar P. Social conditions as fundamental causes of health inequalities: theory, evidence, and policy implications. J Health Soc Behav 2010;Suppl:S28-S40. [CrossRef] [Medline]
  48. Farrell J, Hottes T, Haag D, Bondyra M, Gilbert M. Health equity and British Columbia's GetCheckedOnline program: how can we make an online testing service for sexually transmitted infections that works for everyone? 2015 Presented at: Gay and Lesbian Medical Association 33rd Annual Conference; Sept 24-26, 2015; Portland   URL:
  49. Bonnell A, Haag D, Bondyra M, Farrell J, Achen M, Elliot E, et al. “Click yes to consent”: can we achieve informed consent for online testing for sexually transmitted infections? Paper presented at. 2015 Presented at: World STI&HIV Congress; September 13-16, 2015; Brisbane, Australia   URL:
  50. Falasinnu T, Gustafson P, Hottes TS, Gilbert M, Ogilvie G, Shoveller J. A critical appraisal of risk models for predicting sexually transmitted infections. Sex Transm Dis 2014;41:321-330. [CrossRef] [Medline]
  51. Falasinnu T, Gilbert M, Hottes TS, Gustafson P, Ogilvie G, Shoveller J. Predictors identifying those at increased risk for STDs: a theory-guided review of empirical literature and clinical guidelines. Int J STD AIDS 2015;26:839-851. [CrossRef] [Medline]
  52. GetCheckedOnline: A Free and Confidential Way to Test for Sexually Transmitted Infections in Vancouver.   URL: [accessed 2016-06-23] [WebCite Cache]
  53. Catwell L, Sheikh A. Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS Med 2009;6:e1000126 [FREE Full text] [CrossRef] [Medline]
  54. Lilford RJ, Foster J, Pringle M. Evaluating eHealth: how to make evaluation more methodologically robust. PLoS Med 2009;6:e1000186 [FREE Full text] [CrossRef] [Medline]
  55. Falasinnu T, Gilbert M, Gustafson P, Shoveller J. A validation study of a clinical prediction rule for screening asymptomatic chlamydia and gonorrhoea infections among heterosexuals in British Columbia. Sex Transm Infect 2016;92:12-18. [CrossRef] [Medline]
  56. Falasinnu T, Gustafson P, Wong S, Haag D, Wong J, Ogilvie G, et al. Validation of the Denver HIV risk score for targeting HIV screening in Vancouver, British Columbia. 2015 Presented at: International AIDS Society Conference on HIV Pathogenesis, Treatment and Prevention; July; July 19-22, 2015; Vancouver, British Columbia.
  57. Farrell J, Haag D, Bondyra M, Thomson K, Gilbert M. Should your visuals be wearing blue, pink or yellow pants (or no pants at all)? Developing specimen self-collection instructions that are more sensitive to the needs of traditionally marginalized and underserved client populations. 2015 Presented at: Gay & Lesbian Medical Association 33rd Annual Conference; September 24-26, 2015; Portland, Oregon   URL: http:/​/lovebytesresearch.​ca/​wp-content/​uploads/​2016/​01/​Janine_Self-collection-poster_GLMA-2015_Final.​pdf
  58. British Columbia Ministry of Health. From Hope to Health: Towards an AIDS-free Generation. Victoria, BC: British Columbia Ministry of Health; 2012.   URL: [WebCite Cache]
  59. Horvath KJ, Ecklund AM, Hunt SL, Nelson TF, Toomey TL. Developing Internet-based health interventions: a guide for public health researchers and practitioners. J Med Internet Res 2015;17:e28 [FREE Full text] [CrossRef] [Medline]
  60. Schoville RR, Titler MG. Guiding healthcare technology implementation: a new integrated technology implementation model. Comput Inform Nurs 2015;33:99-107; quiz E1. [CrossRef] [Medline]
  61. Abbott PA, Foster J, Marin H, Dykes PC. Complexity and the science of implementation in health IT--knowledge gaps and future visions. Int J Med Inform 2014;83:e12-e22. [CrossRef] [Medline]

BC: British Columbia
BCCDC: British Columbia Centre for Disease Control
BCPHL: BC public health laboratory
CCWG: community consultation working group
CIC: clinical integration committee
CPR: clinical prediction rules
GCO: GetCheckedOnline
GWG: GetCheckedOnline Working Group
HEIA: health equity impact assessment
HIV: human immunodeficiency virus
ISC: Internet Services Committee
IT: information technology
MSM: men who have sex with men
PHSA: Provincial Health Services Authority
STBBI: sexually transmitted and blood-borne infections
STI: sexually transmitted infection
TWC: technical working group

Edited by G Eysenbach; submitted 29.06.16; peer-reviewed by J Mitchell, T Schopf; comments to author 13.08.16; revised version received 14.08.16; accepted 21.08.16; published 20.09.16


©Mark Gilbert, Devon Haag, Travis Salway Hottes, Mark Bondyra, Elizabeth Elliot, Cathy Chabot, Janine Farrell, Amanda Bonnell, Shannon Kopp, John Andruschak, Jean Shoveller, Gina Ogilvie. Originally published in JMIR Research Protocols (, 20.09.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.