Background: As SARS-CoV-2, the virus that causes COVID-19, spread rapidly across the United States in the spring of 2020, institutions of higher education faced numerous challenges associated with minimizing risk of exposure to COVID-19 among their students, faculty, staff, and surrounding communities. This paper describes the protocol, South Carolina (SC) Safer Together, developed by Clemson University (Clemson) to design, deploy, and evaluate multi-level communication and dissemination and implementation (D&I) strategies in line with recommendations from governmental and educational agencies to mitigate the risk of exposure to COVID-19. Safer Together was enhanced by the addition of the Google/Apple Exposure Notification app, an alternative strategy to support a recommendation of COVID-19 testing outcomes: contact tracing, isolation, and quarantine.
Objective: This study aimed to (1) describe the content and intended audiences of D&I strategies used to deploy recommended COVID-19 mitigation strategies on a major college campus; (2) determine the reach, acceptability, adoption, and use of D&I strategies among target audiences among university students, faculty, and staff; and (3) characterize barriers and facilitators to the implementation and use of recommended mitigation strategies.
Methods: The study team incorporated elements of the Health Belief Model, the Technology Acceptance Model, communication and social marketing models, and the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to identify and develop appropriate constructs and specific outcomes for inclusion in our approach to evaluate the communication, dissemination and implementation processes related to deployment of Safer Together at Clemson. A parallel convergent mixed methods design was used to (1) inform implementation strategies used to launch the program and (2) evaluate program reach, acceptability, adoption, and use guided by the RE-AIM framework. Data collection tools include surveys, data analytics–tracking, and focus groups or interviews with key stakeholders (students, employees, and university leadership).
Results: Rigorously studying both the dissemination and implementation of Safer Together in a national public university setting is expected to yield lessons that will be valuable at many organizational and governmental settings. On a local level, broad adoption and use of the Safer Together may help reduce COVID-19 transmission and keep the university “open.” On a larger scale, lessons learned on how to influence student and employee behavior with respect to the use of a public health outbreak prevention tool including Safer Together may be applicable in future pandemic and outbreak situations.
Conclusions: This study proposes a structured, theory-driven approach to evaluate dissemination and implementation strategies associated with the deployment of Safer Together in a university setting from the viewpoint of students, employees, and university leadership. Our results will inform future implementation of apps such as Safer Together at major state universities in SC.
International Registered Report Identifier (IRRID): DERR1-10.2196/32567
As SARS-CoV-2, the virus that causes COVID-19, spread rapidly across the United States in the spring of 2020, institutions of higher education (IHEs) faced numerous challenges associated with minimizing the risk of exposure to COVID-19 among their students, faculty, staff, and surrounding communities [- ].
This paper describes the protocol used by Clemson University (hereinafter referred to as “Clemson”) to design, deploy, and evaluate multi-level communication and dissemination and implementation (D&I) strategies in line with recommendations from governmental and educational agencies to mitigate the risk of exposure to COVID-19 and to assess D&I outcomes (reach, acceptability, adoption, and use), barriers, and facilitators encountered during this deployment from the perspective of multiple stakeholders.
Published guidelines of the Centers for Disease Control and Prevention (CDC) [- ], the Equal Employment Opportunity Commission [ ], the US Department of Education on the Family Educational Rights and Privacy Act [ ], the Health Insurance Portability and Accountability Act [ ], state and local health agencies and local governments, and various educational organizations such as the Chronicle of Higher Education [ ] included these specific strategies for consideration by IHEs:
- Opening in various formats (eg, virtual only or a hybrid of virtual and in-person classes) during the spring and fall semesters of 2020
- Requiring entry screening prior to the beginning of each term
- Implementing a universal screening testing strategy based on whether community SARS-CoV-2 was deemed moderate, substantial, or high
- Assuring the availability of sufficient testing capacity
- Consulting with local public health authorities
- Implementing actions to support testing outcomes of contact tracing, isolation, and quarantine
- Providing options to immediately separate students with COVID-19 and their close contacts by providing virtual learning options and self-isolation and self-quarantine rooms in residence halls or other housing facilities.
- Providing support to students to manage COVID-19 symptoms, including medical care as necessary, as well as support managing emotional issues related to isolation or quarantine
- Provision of alternative food service arrangements for students living on campus
- Offering alternative teaching and work-at-home options for faculty, instructors, and staff who have COVID-19 or have been identified as a close contact, provided that they are well enough to continue working remotely, and
- Considering the implementation of flexible sick leave and supportive policies and practices.
Clemson developed and implemented a plan that addressed each of these recommended strategies, in some form, in spring 2020.
Contact tracing is one typical and vital public health response to identify and isolate exposed close contacts (ECC) of known COVID-19 cases. The infectivity characteristics of SARS-CoV-2, the large proportion of very severe or deadly cases, and combined lack of specific treatment options for SARS-CoV-2 make it imperative to quickly isolate infected individuals and identify and track their ECC to reduce the odds of continued spread . Moreover, for SARS-CoV-2, the incubation period is short (ie, 3-5 days); hence, chains of transmission must be recognized quickly in order to have an impact [ , ].
Implementing a timely contact tracing process is relatively slow, requires multiple calls to track down described contacts for interviews, and is labor-intensive. Simulation studies suggest the process is too slow to meaningfully limit COVID-19 spread . In actual practice, reaching out to identified contacts is often unsuccessful [ , ]. Furthermore, the resulting data are flawed by memory biases, and many “contacts” never know that they were exposed.
To circumvent these issues, conventional contact tracing includes asking “cases” about where they were (ie, settings of potential exposure). While this information can be helpful in identifying ECC, information regarding the distance between persons and the total time of exposure gathered in this process often results in some individuals being considered as ECC when they actually had very limited or no exposure to an infected individual.
The time delays and personal recall limitations inherent in conventional contact tracing as well as the epidemiology and pathogenesis of SARS-CoV-2 provide a rationale for investigating novel approaches to control infection via contact tracing such as mobile apps .
To address the recommended strategy of implementing actions to support testing outcomes such as contact tracing, isolation, and quarantine, Clemson also took advantage of a novel partnership among the state public health authority for South Carolina (SC), the SC Department of Health and Environmental Control (DHEC), Google, and Apple.
In 2020, Google and Apple created the Google/Apple Exposure Notification system (GAEN), a novel mobile app to help governments and the global community accelerate the process of contact tracing . Mutual agreements among the DHEC, Google, Apple, Clemson Computing and Information Technology (CCIT), and the Medical University of South Carolina (MUSC) Biomedical Informatics Center branded GAEN as the SC Safer Together App (hereinafter referred to as “Safer Together”). Safer Together became part of an existing informational ecosystem at Clemson, which links test-ordering, test result–reporting, and case referral and management with exposure notification. Combined with deployment of other recommended strategies for COVID-19 mitigation, Safer Together was adopted as an efficient and privacy-protected approach to contact tracing ( ). This system of notification, sharing of test results, and rapid access to testing or medical care when notified of exposure was expected to minimize contact tracing time and conserve campus resources.
Safer Together interfaces with public health, clinical providers, and clinical laboratory systems as the source of official laboratory findings. Case managers at Clemson Redfern Health Center use an automated function to send the text message, which allows the Key Authorization Server to deliver a digital certificate to the user’s cellphone. The cellphone user chooses whether they upload their test result to the exposure server.
To assure user confidentiality, each phone advertises itself using multiple keys that rotate over time to protect from malfeasants. If a match is found with another phone, the duration of exposure (total of 15 minutes at less than 6 feet over 24 hours) and the signal strength are evaluated by functions built into Safer Together and aligned with revised CDC guidelines for significant exposures [, ].
When an exposure notification is triggered, the exposed individual can use the “get into care” tab for information on what to do next. Compared to conventional contact tracing based on “case” recollection, cellphone-based exposure notifications have the potential to inform individuals of exposures more quickly, to identify a more complete set of true contacts, and to better assess extent of exposure . Additionally, privacy-preserving Bluetooth technology promptly provides contacts with information regarding the frequency and intensity of lower-grade (shorter time, further distant) exposures, thereby empowering users to proactively measure their risks and take actions, which might safeguard their own health and public health, such as obtaining a test to determine their COVID-19 status or voluntarily quarantining.
Clemson operational units had concerns about whether their campus had sufficient beds and capacity to manage anticipated increases in quarantine facility requirements, based on Safer Together identification of ECC. A compromise plan was developed to conduct a limited pilot deployment of Safer Together for approximately 380 on-campus students living in 2 dormitories.
The first round of student messaging regarding recommended strategies and the use of Safer Together was initiated through Clemson Student Housing on October 28, 2020, to students in the two dormitories. On January 9, 2021, Safer Together was messaged and launched with all Clemson students in 2 waves, the first with students on campus and the second with all students regardless of residence.
Messaging for students was sent via email to explain Safer Together and its capacity to let students know if they had been exposed to someone with confirmed COVID-19, encourage them to download the app, link the app to the Healthy Clemson United as Tigers app, and understand what to do should she/he be notified of potential exposure. Students were also encouraged to participate in efforts to evaluate deployment strategies and learn about their acceptance and use of Safer Together ().
Given the small size of this pilot with students in only 2 dormitories and that on-campus isolation was not a burden to the university employees, Clemson’s administration disseminated messaging about Safer Together to all Clemson employees on November 20, 2020, via an email asking them to download and activate Safer Together and advising them of the effort to evaluate Safer Together deployment strategies ().
Evaluation Process Infrastructure
The D&I team is composed of individual representatives of Clemson Public Health Sciences, Clemson Human Resources, Clemson Student Affairs, Clemson Housing, Clemson Strategic Communications, MUSC College of Medicine Department of Public Health Sciences, Clemson Redfern Health Center, and Clemson Strategic Communications. This team was responsible for developing the study protocol, gaining institutional review board (IRB) approvals at both Clemson and MUSC, developing surveys and focus group and interview guides, and coordinating all data collection with Clemson Human Resources, Clemson Redfern Health Center, and Clemson Strategic Communications.
The Clemson-MUSC Safer Together Implementation team was divided into three groups: D&I Research Group (11 members), Technology Design and Enablement (14 members), and Data Analysis (2 members).
The D&I Research Group includes 4 faculty members with expertise in D&I science and is responsible for all aspects of the evaluation study design.
The Technology Design and Enablement Group is composed of members of the MUSC Biomedical Informatics Center and Clemson CCIT. This team was responsible for developing Safer Together and coordinating all design components with systems at DHEC, Clemson Redfern Health Services, and relevant university service components.
The Data Analysis Group includes individuals from the Department of Clemson Public Health Sciences and the Department of Public Health Sciences at MUSC. This group was charged with creating and maintaining study-related databases and conducting analyses.
All teams met together weekly and, as needed, by team or group.
Specific Aims and Objectives
The purpose of this mixed methods study is to (1) to characterize and evaluate communication and D&I strategies used to promote and support the use of Safer Together and (2) to examine implementation outcomes (reach, acceptability, adoption, and use), barriers, and facilitators encountered from the perspective of multiple stakeholders.
The study objectives are to:
- Describe the content and intended audiences of multi-level dissemination and implementation strategies used to deploy Safer Together and other recommended mitigation strategies
- Determine the reach, acceptability, adoption, and use of Safer Together among targeted audiences of university students, faculty, and staff, and
- Characterize barriers and facilitators to implementation and use of Safer Together and recommended other mitigation strategies.
Our D&I Research Group incorporated elements of the Health Belief Model (HBM) , the Technology Acceptance Model (TAM) [ ], communication and social marketing models [ ], and the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework [ ] to identify and develop appropriate constructs and specific outcomes for inclusion in our approach to evaluating the communication and D&I processes related to the deployment of Safer Together at Clemson.
The HBM was used create questions for use in surveys and focus groups about HBM constructs such as perceived susceptibility, perceived severity, perceived benefits, perceived barriers, and self-efficacy .
The TAM measures intent to use a new technology by assessing perceived usefulness, perceived ease of use, compatibility, and self-efficacy of mobile health care systems .
Communication and social marketing models were used to create messaging and expected outcomes and to influence behavior by offering target market members (in this case students and employees) an attractive package of benefits and by reducing barriers that would otherwise encourage (or discourage) them from engaging in the behavior .
The RE-AIM model served as the guiding framework for measuring the D&I of Safer Together in our research setting . The RE-AIM framework allows multiple measures at various setting levels while focusing on the measurement of “real world” D&I outcomes and processes. provides a conceptual overview of the RE-AIM framework as applied to the evaluation of Safer Together.
|Outcomes||Number, proportion, and representativeness of students and employees participating in Safer Together||Number, proportion, and representativeness of settings and agents willing to initiate the Safer Together system||Extent to which the Safer Together system is launched, implemented, and used as planned|
|Data collection methods||Surveys and focus groups||Focus groups||Surveys, focus groups, and Data analytics tracking|
A parallel convergent mixed methods design will be used to (1) inform implementation strategies (ie, marketing, distribution, and education) used to launch the program and (2) evaluate program reach, acceptability, adoption, and use guided by the RE-AIM framework . The three main data collection tools include surveys, data analytics–tracking, and focus groups or interviews with key stakeholders (students, employees, and university leadership). contains a more comprehensive description of the RE-AIM framework elements, key questions to be addressed, and data to be collected.
|Framework domain||Students or employees||Health care providers||University leaders|
|What proportion of eligible individuals were offered and then downloaded the Safer Together intervention?||✓||✓|
|What percent of individuals seeking a COVID-19 test report actively using Safer Together?||✓||✓|
|How did Safer Together use vary by individuals’ demographic characteristics (eg, age, race, gender, student vs employee, faculty vs staff, and student residence location)?||✓||✓|
|What number and percent of individuals seeking a COVID-19 test report receiving a risk exposure notification on Safer Together?||✓|
|Were there differences in the time of the risk exposure notification between Safer Together and formal COVID-19 contact tracing methods?||✓||✓|
|What was the effect of Safer Together on health care system outcomes (eg, work processes, organizational change, and interdisciplinary collaboration)?||✓||✓|
|What are the characteristics of setting and adopting Safer Together?||✓||✓|
|How well did Safer together fit with the values and expectations of stakeholders?||✓||✓||✓|
|How did the Safer Together system help Clemson University achieve their educational and practice missions during the pandemic?||✓||✓||✓|
|Was there sufficient leadership support and buy-in for the Safer Together system (predisposing)?||✓||✓||✓|
|What were the potential barriers to successful Safer Together implementation and use and how were they addressed (enabling)?||✓||✓||✓|
|What workflow adjustments were needed to streamline Safer Together into routines of daily life at Clemson University (enabling)?||✓||✓|
|What measures were needed to create readiness for Safer Together adoption, commitment, and buy-in by stakeholders (enabling)?||✓||✓|
|Did users perceive Safer Together as easy-to-use and useful (predisposing)?||✓||✓||✓|
|What were the confidentiality and data security concerns when adopting the Google/Apple Exposure Notification app and how were they addressed (enabling)?||✓||✓|
|How did Safer Together use by stakeholders evolve over time?||✓||✓||✓|
|What efforts were needed to maintain the app participation rate and effectiveness?||✓||✓|
|Was the use of Safer Together sustained over time?||✓||✓|
Clemson University, located in Upstate SC, is one of two land-grant universities in SC. Clemson was founded in 1889 and currently has 20,868 undergraduate students and 5538 graduate students. Clemson offers over 80 major and 90 minor areas of study and more than 130 graduate programs and maintains a 16:1 student to faculty ratio. As a major research university, Clemson was awarded US $106.3 million in research support in 2019-2020. A robust workforce of staff employees (n=4611), faculty employees (n=1742), and emeritus faculty (n=654), totaling to 6007 employees, supports all major initiatives and activities undertaken on and off campus.
A convenience sample of Clemson students, employees, and leaders involved in implementing or facilitating the use of Safer Together will be presented the opportunity to participate in this study. The inclusion and exclusion criteria for this study are shown in.
Study inclusion and exclusion criteria.
- All registered students and all employees (faculty and staff) of Clemson University
- Being a nonemployee (eg, contractor)
- Not meeting the aforementioned inclusion criterion
Participant Recruitment, Enrollment, and Consent
Recruitment strategies for each group using social marketing and targeted messaging are detailed below. University representatives from the Student Affairs, Human Resources, and Strategic Communications units will participate as consultants on the D&I team to ensure coordination of efforts.
Social marketing and targeted messaging strategies will be deployed throughout the duration of the study, with specific emphasis on three time points: initial return of students to campus, start of the spring semester (and flu season), and late spring before completion of the academic year.
Given the “opt-in” nature of these user choices, and the fact that identifiable data are not available to the research team, the research team received a waiver of informed consent.
Members of the D&I research team and the Administration department at Clemson will not know whether any user activates or allows deidentified sharing of COVID-19 test results.
With respect to student and employee survey participation, anonymous surveys will be undertaken to obtain information related to student and employee use and perceived usefulness of Safer Together. Accordingly, the D&I Team received a waiver of informed consent for the anonymous surveys to be administered through the academic year.
With respect to focus group discussions addressed below, the D&I Team received a waiver of written informed consent. Verbal consent will be obtained from all participants after a study investigator provides a description of the purpose, procedures, and risks and benefits of the study.
Potential participants include university students, employees, and leaders. Potential participants will be approached through targeted communication avenues (eg, student housing listservs and student and employee newsletters). Our D&I Team will verify that participants meet the study eligibility criteria and approach them via electronic media or cellphone-based methods. Participants will be considered to have been enrolled when verbal consent is obtained in focus groups and acknowledged by moving forward to complete the electronic survey.
Implementation strategies are designed to increase the uptake and use of the Safer Together App by students and employees.
Social Marketing and Targeted Messaging
Social marketing seeks to influence behavior by offering target market members (in this case students and employees) an attractive package of benefits and by reducing barriers that would otherwise encourage (or discourage) them from engaging in the behavior.
Key elements of social marketing include mutual fulfillment of self-interests, consumer orientation, segmentation (marketing to various subsets of the organization), and a marketing mix (product, price, place, and promotion) . The study D&I team (including consulting representatives of Clemson’s Strategic Communications, Student Affairs, and Human Resources units) will create targeted messaging based on social marketing principles to be delivered throughout the academic year 2020-2021.
The three major study time periods were the following: original back-to-school messaging in September-October 2020 timeframe, at the transition of semesters (and flu season) in the January-February 2021 timeframe, and spring break to the end of the academic year between March and May 2021.
The overall messaging campaign focused on (1) building a Clemson community partnership around COVID-19 infection control, (2) raising awareness about how exposure notifications can facilitate and supplement other public health efforts, (3) addressing privacy protections and security concerns, and (4) highlighting ease of use and voluntary participation.
Four main strategies were used for the campaign: (1) flyers were posted strategically across campus at the start of the semester; (2) presentations were made at key student, employee, and university leadership meetings; (3) email and social media messaging were used to provide strategic messages over the course of the planning, early implementation, and maintenance phases, and (4) university-level messaging was delivered electronically via selected venues.
Branding for all materials and messages allows consumers to identify and connect with messages. The Basic Communication Model  elements were used to guide the selection of optimal communication channels, sources, and messages to fit the target audiences of students and employees. Messages were distributed through highly visible channels and delivered by reputable sources (eg, Clemson leaders, staff, and student champions). provides further details about each of these strategies.
Messaging was designed to influence student and employee behavior and prompt use of Safer Together. Specific behaviors include downloading the app, activating the app, sharing test results on the app, and pursuing COVID-19 testing if a user receives a Safer Together exposure notification.
|Planned schedule of marketing activities||Content||Purpose|
|Flyers posted strategically on campus||Highlight the Clemson University community partnership around COVID-19 infection control; brief overview of the app and how to use it; topics will cover benefits of contact tracing, how exposure notification can help, privacy protection, and how the app works||To increase awareness and acceptance of the app, address potential user concerns, and inform users how to use it|
|Presentations at key web-based employee and student meetings||Highlight the Clemson University community partnership around COVID-19 infection control; brief overview of the app and how to use it; topics will cover benefits of contact tracing, how exposure notification can help, privacy protection, and how the app works||To increase awareness and acceptance of the app, address potential user concerns, and inform users how to use it|
|Email/social media blasts||Highlight awareness at multiple levels of the university, colleges, and departments; garner student support and enthusiasm||To increase awareness and commitment at the university and department levels|
|University messaging 1 (via selected venues)||Highlight the importance of university commitment to curtail the spread of COVID-19; expand the readership to a variety of stakeholders||Increase awareness to a broad range of stakeholders; convey commitment from university leadership|
|University messaging 2||Brief overview of the app and how to use it; topics will cover the importance of the Clemson university community partnership for COVID-19 infection control; the benefits of contact tracing, how exposure notification can help, privacy protection, and how the app works||To increase awareness and acceptance of the app, address potential user concerns, and inform users how to use it|
|University messaging 3||Reminder about Clemson University community partnership for COVID-19 infection control and frequently asked questions about the app, and who to contact with questions about installing and using the app if an exposure occurs||To address problems users may be having in using the app via a series of frequently asked questions|
|University messaging 4||Brief stories of how the app has been successful when used in other places||To increase awareness of public health benefits of using the app|
|University messaging 5||Brief update on Clemson University’s experience using the app and how it may help control the pandemic on campus||To increase buy-in regarding the app|
|University messaging 6||Reminder about Clemson University community partnership for COVID-19 infection control and engagement with the app again, repeating the brief overview||To provide a booster educational session to encourage continued use|
|University messaging 7||Messaging about not letting one’s guard down during the spring break—continue protections and engagement with the app (spring break starts on March 15)||To increase buy-in regarding the app during vacation|
|University messaging 8||Message of thanks for participating in the app, with a brief summary of Clemson University’s experience using the app||To debrief and increase buy-in|
Study Flow and Data Collection Processes
Closer to the date of student return to the Clemson campus, the university initiated a series of messages to students and employees about Safer Together and this study. Messaging encouraged downloading of Safer Together to student and employee cellphones, individual consent to activate Safer Together, and individual consent to share COVID-19 test results.
Students and employees selected through random sampling for COVID-19 testing, who present at the COVID-19 testing site at Clemson Redfern Health Center, will be asked questions as part of their registration process related to Safer Together. One question asks if the individual has downloaded and activated Safer Together. A second question relates to whether the individual presented for COVID-19 testing primarily because of a Safer Together exposure notification alert.
Students and employees not selected through random sampling but presenting for COVID-19 testing at the COVID-19 testing site will be asked questions as part of their registration process related to Safer Together. One question asks if the individual has downloaded and activated Safer Together. A second question relates to whether the individual presented for COVID-19 testing primarily because of a Safer Together exposure notification alert.
Students and employees were asked to complete a one-time electronic anonymous survey administered by the D&I team to address framework measures. Key questions included assessments of the individual’s decision to download and activate Safer Together, consent to record COVID-19 test results in Safer Together, and report whether they took some action in response to an exposure notification (eg, additional COVID-19 testing and quarantine). Additional questions assessed user experiences with the app and whether they found the app acceptable and easy to use. Among those who did not download the app, questions focused on factors influencing one’s decision to not download Safer Together, including perceived barriers to downloading the app. All participants were asked questions about testing behaviors, perceived susceptibility to and severity of COVID-19, and individual sociodemographic characteristics.
At key points during the study, focus groups facilitated by the D&I team will be conducted with key stakeholders including students and employees (health care providers and university leaders) to address framework measures. Questions will predominantly address measures within the Adoption, Implementation, and Maintenance domains of the RE-AIM framework by using a structured interview guide. Focus groups will be completed virtually (on Zoom), last approximately 45-60 minutes, and be audiotaped for analysis.
Key leaders involved in developing, planning, and launching the Safer Together initiative were selected to complete individual interviews using a structured interview guide. Participants will include representatives from the information technology office, the Clemson Redfern Health Center, the communications office, the Healthy Clemson: United as Tigers initiative, dormitory housing, and the public health strategy team. Participants will be invited to participate in a 30-minute, audiotaped interview.
Using a team-developed implementation tracking log (), the D&I Team monitored the completion, timing, and barriers faced in accomplishing planning, engagement, technology development, and messaging milestones. The log was competed at routine team meetings to assess completion of steps (with “yes” or “no” responses) and whether delays (eg, information technology, communication, university leadership, and engagement) were encountered.
Primary Outcomes of Effectiveness
In this study, limited data are available for extraction by the research team from Safer Together, so most study measures are captured by alternative approaches. The four primary outcome measures are downloading the Safer Together app, activating Safer Together, activating sharing of COVID-19 test results in Safer Together, and responsive behavior to Safer Together exposure notification.
Participants Downloading Safer Together
The number and proportion of research participants who seek testing and those who complete the survey and download Safer Together will be delivered via the DHEC to Clemson CCIT. We will record this statistic on a daily basis and aggregate for periods prior to and after our social marketing targeted messaging.
Participants Activating Safer Together
The number and proportion of research participants who present for COVID-19 testing and claim that they have downloaded Safer Together and activated its exposure notification alert will be recorded.
Participants who consent to sharing COVID-19 results within Safer Together and the number and proportion of those who declare that they have authorized the sharing of COVID-19 results on Safer Together will be recorded. These data are not available from Safer Together and can only be assessed through surveys and focus groups.
Effect of Safer Together Exposure Notification Alert on Participant Behavior
The number and proportion of research participants who present for COVID-19 testing and claim that the primary reason for presenting for COVID-19 testing was a Safer Together exposure notification alert will be recorded.
The survey was administered to a random stratified sample of 20% of Clemson employees (n=802) and students (n=4998), for a total sample size of 5790 individuals.
The analysis of our mixed methods approach will follow approaches for assessing and integrating findings from both a quantitative and qualitative perspective .
Safer Together overall use will be measured by a binary variable (“yes” or “no”). Univariate analyses for differences in app use via student demographics will be compared using chi-square tests and independent 2-sample t tests for categorical and continuous variables, respectively. Multivariate analyses will be performed using logistic regressions. Exposure notification time will be defined as the average of the time between an individual’s recorded COVID-19 test (if positive) and notification of all contacts. Differences in exposure time between the two methods will be assessed using Kaplan-Meier estimators and Cox proportional hazard models.
The reach of each method will be measured as the number of contacts informed; differences will be assessed using independent 2-sample t tests. Upon presentation to health services, students will be asked whether they were notified of exposure through the Safer Together app, formal contact tracing, or other. Comparisons between Safer Together app tracing and formal contact tracing will be conducted using chi-square tests.
Transcriptions of digital recordings of focus groups and field notes will be analyzed to characterize app use experiences and barriers and facilitators to program delivery, adoption, and implementation. Analysis will be conducted using manual coding to identify, categorize, and contextualize patterns. We will use an initial codebook derived from the RE-AIM framework also allowing additional themes to arise directly from the data. Two independent coders will read and reread transcripts, outlining and organizing themes and subthemes; discrepancies will be resolved in team meetings. After completing qualitative and quantitative data analyses independently, we will use graphical matrix configurations (“joint displays”) to integrate survey findings with qualitative data for data triangulation . Qualitative themes will be supplemented by patterns identified in quantitative results guided by the RE-AIM framework.
We will use a team-based approach with individual and dual review of data and data findings with, as appropriate or needed, adjudication by a third party to interpret and translate results and determine D&I implications of our findings.
The protocol was approved by both the Clemson University IRB and the IRB-II-Medical University of South Carolina on September 11, 2020.
Rigorous evaluation of both the dissemination and implementation of Safer Together in a national public university setting is expected to yield insights that will be valuable at many organizational and governmental settings. On a local level, broad adoption and use of the Safer Together may help reduce transmission of COVID-19 and keep the university “open.” On a larger scale, lessons learned on how to influence student and employee behavior regarding the use of a public health outbreak prevention tool like Safer Together may be applicable in future pandemic and outbreak situations.
This study proposes a structured approach based on the RE-AIM framework, to evaluate dissemination and implementation strategies associated with deployment of the Safer Together in a university setting from the viewpoint of students, employees and university leadership. The results of this study will inform future implementation of apps such as Safer Together at major state universities.
This study was supported, in part, by the National Center for Advancing Translational Sciences of the National Institutes of Health (grant UL1 TR001450). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
CLM is the primary author and senior advisor. KRS is the secondary author and D&I advisor, and led quantitative tool development and analysis. RG is the Chair of the D&I Research Team and liaison with Clemson and MUSC information technology groups and with the Clemson leadership. LL is the lead bioinformatics team member. KBC is the senior author, developed the analysis plan, and is the lead qualitative analyst. The SC Safer Together Team includes Latoya Daniels, Kristin Donnelly, Leasa Evinger, Lesslie Graves, Corey Kalbaugh, Lior Rennert, Tara Romanella, and Heidi Williams.
Conflicts of Interest
- Walke HT, Honein MA, Redfield RR. Preventing and Responding to COVID-19 on College Campuses. JAMA 2020 Nov 03;324(17):1727-1728. [CrossRef] [Medline]
- COVID-19 Protocols and Policies. University of Notre Dame. URL: https://here.nd.edu/health-safety/daily-health-check/ [accessed 2022-01-05]
- Fox MD, Bailey DC, Seamon MD, Miranda ML. Response to a COVID-19 Outbreak on a University Campus - Indiana, August 2020. MMWR Morb Mortal Wkly Rep 2021 Jan 29;70(4):118-122 [FREE Full text] [CrossRef] [Medline]
- Guidance for Institutions of Higher Education (IHEs). Centers for Disease Control and Prevention. URL: https://www.cdc.gov/coronavirus/2019-ncov/community/colleges-universities/considerations.html [accessed 2022-01-05]
- Testing, Screening, and Outbreak Response for Institutions of Higher Education (IHEs). Centers for Disease Control and Prevention. 2020. URL: https://pesquisa.bvsalud.org/global-literature-on-novel-coronavirus-2019-ncov/resource/pt/grc-747131 [accessed 2022-01-05]
- Discontinuation of Isolation for Persons with COVID -19 Not in Healthcare Settings. Centers for Disease Control and Prevention. URL: https://www.munsonhealthcare.org/media/file/Physician%20Services/Discontinuation%20of%20Isolation%20for%20Persons%20with%20COVID-19.pdf [accessed 2022-01-05]
- Interim Guidance for Case Investigation and Contact Tracing in Institutions of Higher Education (IHEs). Centers for Disease Control and Prevention. URL: https://extranet.who.int/goarn/interim-guidance-case-investigation-and-contact-tracing-institutions-higher-education-ihes [accessed 2022-01-05]
- Coronavirus and COVID-19. U.S. Equal Employment Opportunity Commission. URL: https://www.eeoc.gov/coronavirus [accessed 2022-01-05]
- FERPA and the Coronavirus Disease 2019 (COVID-19). U.S Department of Education. 2020. URL: https://studentprivacy.ed.gov/resources/ferpa-and-coronavirus-disease-2019-covid-19 [accessed 2022-01-05]
- Reddy A, Vance A. Student Privacy During the COVID-19 Pandemic. AASA, The School Superintendents Association. 2020 Mar 20. URL: https://studentprivacycompass.org/wp-content/uploads/2020/03/COVID-19-Student-Privacy-FAQs-03-20-2020-1.pdf [accessed 2022-01-05]
- McKenzie L. Ramping up for remote instruction. Inside Higher Ed. 2020 May 27. URL: www.insidehighered.com/news/2020/05/27/new-resources-help-support-faculty-quality-online-instruction [accessed 2022-01-05]
- Ferretti L, Wymant C, Kendall M, Zhao L, Nurtay A, Abeler-Dörner L, et al. Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 2020 May 08;368(6491):eabb6936 [FREE Full text] [CrossRef] [Medline]
- Gandhi M, Yokoe DS, Havlir D. Asymptomatic Transmission, the Achilles' Heel of Current Strategies to Control Covid-19. N Engl J Med 2020 May 28;382(22):2158-2160 [FREE Full text] [CrossRef] [Medline]
- Otterman S. N.Y.C. Hired 3,000 Workers for Contact Tracing. It’s Off to a Slow Start. The New York Times. 2020 Jun 21. URL: https://www.nytimes.com/2020/06/21/nyregion/nyc-contact-tracing.html [accessed 2022-01-05]
- Servick K. Can phone apps slow the spread of the coronavirus? Science 2020 Jun 19;368(6497):1296-1297. [CrossRef] [Medline]
- Exposure Notifications: Using technology to help public health authorities fight COVID‑19. Google. COVID-19 Information & Resources. URL: https://www.google.com/covid19/exposurenotifications/ [accessed 2022-01-05]
- Hochbaum GM. Public Participation in Medical Screening Programs: A Socio-psychological Study. Washington, DC: U.S. Department of Health, Education, and Welfare, Public Health Service, Bureau of State Services, Division of Special Health Services, Tuberculosis Program; 1958.
- Brennan L, Binney W, Parker L, Aleti T, Nguyen D. Multi-theory models. In: Social Marketing and Behaviour Change: Models, Theory and Applications. Glos: Edward Elgar Publishing Inc; Dec 26, 2014:181-198.
- Lee Y, Kozar KA, Larsen KR. The Technology Acceptance Model: Past, Present, and Future. CAIS 2003;12. [CrossRef]
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [CrossRef] [Medline]
- Guetterman TC, Fetters MD, Creswell JW. Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays. Ann Fam Med 2015 Nov;13(6):554-561 [FREE Full text] [CrossRef] [Medline]
|CCIT: Clemson Computing and Information Technology|
|CDC: Centers for Disease Control and Prevention|
|DHEC: Department of Health and Environmental Control|
|D&I: dissemination and implementation|
|ECC: exposed close contacts|
|GAEN: Google/Apple Exposure Notification|
|HBM: Health Belief Model|
|IHE: institution of higher education|
|IRB: institutional review board|
|MUSC: Medical University of South Carolina|
|RE-AIM: Reach, Effectiveness, Adoption, Implementation, and Maintenance|
|SC: South Carolina|
|TAM: Technology Assessment Model|
Edited by G Eysenbach; submitted 03.08.21; peer-reviewed by Y Strekalova; comments to author 26.08.21; revised version received 07.12.21; accepted 31.12.21; published 19.01.22Copyright
©Cathy Lee Melvin, Katherine Regan Sterba, Ron Gimbel, Leslie Andrew Lenert, Kathleen B Cartmell, SC Safer Together Team. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 19.01.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.