Published on in Vol 11, No 8 (2022): August

Preprints (earlier versions) of this paper are available at, first published .
Remote Follow-up of Self-isolating Patients With COVID-19 Using a Patient Portal: Protocol for a Mixed Methods Pilot Study (Opal-COVID Study)

Remote Follow-up of Self-isolating Patients With COVID-19 Using a Patient Portal: Protocol for a Mixed Methods Pilot Study (Opal-COVID Study)

Remote Follow-up of Self-isolating Patients With COVID-19 Using a Patient Portal: Protocol for a Mixed Methods Pilot Study (Opal-COVID Study)


1Research Institute of the McGill University Health Centre, Centre for Outcomes Research & Evaluation, Montreal, QC, Canada

2See Acknowledgments

3Department of Medicine, Division of Infectious Diseases and Chronic Viral Illness Service, McGill University Health Centre, Montreal, QC, Canada

4Department of Mechanical Engineering, École Polytechnique de Montréal, Montreal, QC, Canada

5Department of Family Medicine, Faculty of Medicine, McGill University, Montreal, QC, Canada

6Department of Mathematics and Statistics, Université de Montréal, Montreal, QC, Canada

7Department of Psychiatry Faculty of Medicine, McGill University, Montreal, QC, Canada

8Medical Physics Unit, McGill University, Montreal, QC, Canada

9Department of Radiation Oncology, McGill University Health Centre, Montreal, QC, Canada

10Centre hospitalier de l’Université de Montréal, Montreal, QC, Canada

11Department of Medicine, Divisions of Experimental Medicine, Clinical Epidemiology, Respiratory Medicine, and Rheumatology, McGill University, Montreal, QC, Canada

12Department of Medicine, Division of Medical Oncology, McGill University Health Centre, Montreal, QC, Canada

*these authors contributed equally

Corresponding Author:

Bertrand Lebouché, MD, PhD

Research Institute of the McGill University Health Centre

Centre for Outcomes Research & Evaluation

5252 de Maisonneuve West

Montreal, QC, H4A 3J1


Phone: 1 514 934 934 ext 31342


Background: People with COVID-19 are instructed to self-isolate at home. During self-isolation, they may experience anxiety and insufficient care. Patient portals can allow patients to self-monitor and remotely share their health status with health care professionals, but little data are available on their feasibility.

Objective: This paper presents the protocol of the Opal-COVID Study. Its objectives are to assess the implementation of the Opal patient portal for distance monitoring of self-isolating patients with COVID-19, identify influences on the intervention’s implementation, and describe service and patient outcomes of this intervention.

Methods: This mixed methods pilot study aims to recruit 50 patient participants with COVID-19 tested at the McGill University Health Centre (Montreal, Canada) for 14 days of follow-up. With access to an existing patient portal through a smartphone app, patients will complete a daily self-assessment of symptoms, vital signs, and mental health monitored by a nurse, and receive teleconsultations as needed. Study questionnaires will be administered to collect data on sociodemographic characteristics, medical background, implementation outcomes (acceptability, usability, and respondent burden), and patient satisfaction. Coordinator logbook entries will inform on feasibility outcomes, namely, on recruitment, retention, and fidelity, as well as on the frequency and nature of contacts with health care professionals. The statistical analyses for objectives 1 (implementation outcomes), 3 (service outcomes), and 4 (patient outcomes) will evaluate the effects of time and sociodemographic characteristics on the outcomes. For objectives 1 (implementation outcomes) and 4 (patient outcomes), the statistical analyses will also examine the attainment of predefined success thresholds. As for the qualitative analyses, for objective 2 (influences on implementation), semistructured qualitative interviews will be conducted with 4 groups of stakeholders (ie, patient participants, health care professionals, technology developers, and study administrators) and submitted for content analysis, guided by the Consolidated Framework for Implementation Research to help identify barriers to and facilitators of implementation. For objective 3 (service outcomes), reasons for contacting health care professionals through Opal will also be submitted for content analysis.

Results: Between December 2020 and March 2021, a total of 51 patient participants were recruited. Qualitative interviews were conducted with 39 stakeholders from April to September 2021. Delays were experienced owing to measures taken at the McGill University Health Centre to address COVID-19. The quantitative and qualitative analyses began in May 2022. As of June 2022, a total of 2 manuscripts (on the implementation and the patient outcomes) were being prepared, and 3 conference presentations had been given on the study’s methods.

Conclusions: This protocol is designed to generate multidisciplinary knowledge on the implementation of a patient portal–based COVID-19 care intervention and will lead to a comprehensive understanding of feasibility, stakeholder experience, and influences on implementation that may prove useful for scaling up similar interventions.

Trial Registration: NCT04978233;

International Registered Report Identifier (IRRID): DERR1-10.2196/35760

JMIR Res Protoc 2022;11(8):e35760




As of December 13, 2021, over 267 million people worldwide have had COVID-19, including almost 2 million in Canada [1]. In 2020, without effective vaccines or treatments, governments and public health agencies enforced lockdown and distancing measures to limit the spread of COVID-19 and the burden on health care systems. At the time, in the Canadian province of Québec, 95% of people diagnosed with COVID-19 were instructed to self-isolate at home and contact health care services if their health deteriorated. While most patients with COVID-19 have mild to no symptoms in the first week following infection [2,3], patient health can rapidly deteriorate in the second week and opportunistic infections can appear [4]. Quick recognition of worsening health is crucial to avoid delays in health care and improve prognosis. However, most self-isolating patients lack tools to self-monitor their illness and maintain contact with health care professionals to ensure timely care. Self-isolation for COVID-19 can also cause anxiety, especially among vulnerable, older, or low-literacy patients [5,6], owing, in part, to inadequate information [7]. Furthermore, it can contribute to numerous other problems, including substance use or addictions [8], self-harm [9], and interpersonal difficulties [10]. Interventions to help self-isolating patients monitor symptoms and vital signs, provide them with educational materials, and facilitate communication with health care professionals could increase the safety and timeliness of care, while reducing psychological distress [5-7].

Health information technologies (HITs) offer solutions in this regard, and the pandemic has propelled their use, including in the form of telehealth and education platforms [11]. Among HITs, patient portals can provide important information and facilitate interactions between self-isolating patients and health care professionals. Their use is also associated with improved patient engagement, empowerment, and satisfaction [12]. One such portal, Opal, was initially created for use at the Cedars Cancer Centre at the McGill University Health Centre (MUHC) in Montreal, Canada, where our team is based. Implemented in 2018, it was co-designed by health care professionals, patients, and HIT developers [12]. It includes a patient smartphone app and a physician desktop dashboard. With Opal, among other advantages, oncology patients can view their diagnostics and treatment plan, laboratory test results, appointment and test schedules, clinical consultation notes, and tailored educational material. Using a dashboard designed for them, health care professionals can administer patient-reported health measures and communicate with patients between consultations [13].

To support self-isolating patients with COVID-19, our team decided to configure Opal for this purpose. This involved applying an existing HIT to a novel and different context of use. Although the benefits of HITs are well-documented for the follow-up of chronic conditions [14-16] such as cancer, little is known about their transferability to acute and typically short-term conditions such as COVID-19. Studying their implementation in this context, particularly with stakeholder engagement, a recognized key to successful HIT projects [17], would generate important data.

Aim and Objectives

Drawing on implementation science, this mixed methods pilot study will evaluate the implementation of a patient portal (Opal) to follow-up with patients recently diagnosed with COVID-19 who were instructed to self-isolate at home. The intervention centers around daily patient self-reports of symptoms, vital signs, and mental health via Opal for remote review by a hospital-based clinical monitoring team. In line with the outcome categories proposed for implementation research [18], the study objectives are as follows: (1) to quantitatively evaluate the implementation outcomes of this intervention (in terms of predefined benchmarks for acceptability, usability, response burden, feasibility [recruitment and retention], and fidelity); (2) with qualitative data, identify the barriers to and facilitators of implementation (based on semistructured interviews with 4 stakeholder groups); (3) describe service outcomes (in terms of the rates and nature of contact with the health care professionals involved); and (4) describe patient outcomes (based on the daily self-reported health data, including symptoms, use of health services beyond the intervention, and patient satisfaction with teleconsultations, if received).

Study Design

This single-center pilot study follows the guidance provided by the CONSORT (Consolidated Standards of Reporting Trials) statement for pilot and feasibility studies [19,20]. It uses a mixed methods embedded design, with qualitative methods introduced secondarily to extend the breadth of inquiry [21].

Ethics Approval

Ethics approval was obtained from the MUHC Research Ethics Board on October 5, 2020 (approval number 2021-6763), and the procedures are in accordance with the Helsinki Declaration of 1975, as revised in 2000. Participants provided informed consent and could withdraw from the study at any time. To protect their identity, no identifying information appears in any manuscript or presentation based on this study.

Setting and Participants

Study Site

This study will be conducted at 2 large hospitals of the MUHC, the Glen Hospital and the Montreal General Hospital, both located in Montreal, Québec, Canada. When planning this study, from May to July 2020, at the end of the first wave of the pandemic in the province [22], approximately 150 people from local communities were tested daily for COVID-19 at the MUHC, of which approximately 15% tested positive.

Eligibility Criteria

The participant inclusion criteria are as follows: aged ≥18 years; fluent in French or English; a positive diagnosis at the MUHC with a polymerase chain reaction test for the virus that causes COVID-19 (SARS-CoV-2) and instruction to self-isolate at home; enrollment in Québec’s provincial health or public prescription drug insurance plan; access to a smartphone, tablet, or computer; having a home internet connection; and comfortable using a health-related smartphone app or having someone close by who is. Exclusion criteria are being hospitalized, concurrent enrollment in a COVID-19 clinical trial, and having a cognitive impairment that prevents participation.

Recruitment and Sample

To recruit patients, the MUHC test center staff will briefly explain the study and ask about people’s interest in participating when calling to inform them of their positive SARS-CoV-2 test result. The study coordinator will then contact interested persons to schedule a videoconferencing appointment, at which time the coordinator will request to see proof of identification (eg, passport and driver’s license) and, if applicable, collect consent. To comply with COVID-19 prevention and control measures, explicit oral consent will be obtained from all study participants via videoconferencing. An impartial witness will be present to ensure that all aspects of free and informed consent are respected. If consent is provided, the coordinator will further explain study procedures and provide baseline training on Opal.

For the collection of quantitative data (objectives 1, 3, and 4) using convenience sampling, recruitment will begin in December 2020 and continue until 50 patient participants are enrolled. This sample size meets rule-of-thumb recommendations for 1-arm pilot studies that are not designed to test effects [23,24].

For the collection of qualitative interview data (objective 2) using purposive sampling [25], the 50 MUHC participants and other stakeholders (HIT developers, administrators, and health care professionals) involved in implementing Opal for the pilot study will be invited to be interviewed. After consenting at baseline, patient participants will be asked if they would like to be contacted later for the interview. For nonpatient stakeholders, the study coordinator will record the contact information. Approximately 1 month after all patient follow-up on Opal has ended, potential interviewees will be sent an email to schedule a videoconferencing interview.

We expect that the sample achieved for objective 2 will allow for adequate saturation of the qualitative analyses, as a ≥90% representation of the main study themes is attainable with 10 to 12 qualitative interviews [26].


Table 1 details all study procedures related to the patient participants, including the tasks to be accomplished throughout the 14 days of their follow-up.

Table 1. Study procedures for patient participants in the Opal-COVID Studya.
ProcedureDays of follow-up

Before enrollment12 to 14More days, if needed1 month after follow-up

COVID-19 test at MUHCb

Notification of positive test result by phone and survey of interest in participation

Study coordinator call to schedule consent process

Informed consent and eligibility screen

Registration and installation of Opal on participant’s device

Educational meeting on Opal’s main functions

Delivery of pulse oximeter or thermometer

Data collection

Daily self-assessment

Severe COVID-19 symptoms and other symptoms

Vital signs

Mental health (5 items)

✓ (except Days 7 and 14)

Mental health (16 items)
Only Days 7 and 14Weekly

Study questionnaire

Sociodemographic and medical background information

Implementation outcomes
Only Days 7 and 14Weekly

Satisfaction with care (after teleconsultations)

Qualitative semistructured interview

aCompensation: CAD $50 (CAD $1=US $1.3).

bMUHC: McGill University Health Centre.

Provision of Medical Devices

The coordinator will verify whether the person owns a pulse oximeter and a thermometer, which are necessary to self-monitor vital signs during the study. Those without these devices will be able to receive them free of charge. The pulse oximeter provided will be one that is cleared by the United States Food and Drug Administration. Participants will have the option of receiving the devices through a mailing company with specific guidelines for delivery to patients with COVID-19. Alternatively, they may have them picked up by someone else at the MUHC.

The Opal-COVID Intervention

Registration and Training on Opal

Patients must register to use the Opal patient portal. To do so, participating patients will be guided by a research staff member who will confirm their identity and explain Opal’s functionalities. The remote registration system will send a unique verification code to the user’s smartphone with a link to the registration page where the code will be entered. This is done to ensure the security and privacy of patient information. As shown in the provided screenshot (Multimedia Appendix 1), the user must then confirm their identity with their provincial health insurance card number, fill in some basic account information for log-in and security measures, select a data access level (ie, to allow access to their medical data or not), and accept the terms of use.


Participant follow-up is expected to last for 14 days. However, it will be prolonged in the following cases: (1) patients being diagnosed with a SARS-CoV-2 variant; (2) patients reporting fever in the last 48 hours on day 14; or (3) patients reporting other acute symptoms in the last 24 hours, except cough or loss of smell, which can take longer to subside [27]. Figure 1 shows the patient participant path in the Opal-COVID intervention.

Figure 1. Opal-COVID Study patient participant path.
View this figure
Daily Self-assessments

Daily self-assessments form part of the intervention and provide most of the patient outcomes data described in objective 4. They are filled out on the Opal app in either English or French (the full self-assessment can be found in Multimedia Appendix 2; screenshots of the self-assessment are provided in Multimedia Appendix 3). From day 1 to the last day of their follow-up, these self-assessments allow patients to report symptoms, vital signs (using the pulse oximeter and thermometer), and mental health indicators to the clinical monitoring team.

The self-assessment was developed using existing COVID-19 symptom evaluation tools [27-30] and stakeholder feedback. It asks about the following: (1) the presence of severe COVID-19 symptoms (eg, severe difficulty breathing, severe chest pain, heart palpitations, feeling confused, and loss of consciousness in the past 24 hours); (2) the presence of other possible symptoms of COVID-19 (eg, pain, persistent cough, chills, runny or blocked nose, shortness of breath, loss of appetite or sense of smell or taste, sore throat, nausea or vomiting, diarrhea, abdominal cramping, unusual fatigue, and skin changes); (3) vital signs measured and reported by participants, including current temperature, respiration rate, oxygen saturation, heart rate, and, if possible, blood pressure; and (4) mental health symptoms in the past 24 hours, based on the 4-item short form versions of the Patient-Reported Outcome Measurement Information Systems (PROMIS) scales for anxiety, depression, fatigue, and sleep disturbance used under license agreement [31]. To limit response burden, participants will complete all 16 of the PROMIS items on days 1, 7, and 14, whereas on other days, they will respond to only 5.


Each day, automated reminders to complete the self-assessment will be sent at 9 AM and then again at noon to participants who did not do so.


The clinical monitoring team will be composed of up to 3 nurses and 1 physician (assigned on a weekly basis).

Physicians will be available for teleconsultations conducted on the Zoom (Zoom Video Communications Inc) platform [32] and to refer patient participants to the psychiatrist (M-JB) in cases of psychological distress. All members of the clinical monitoring team will refer patient participants to emergency services as needed. The normal hours of operation for the team will be from 9 AM to 8 PM. Completed self-assessments will be reviewed by nurses on the Opal Room Management System clinical dashboard and systematically acknowledged. Each nurse will monitor up to 6 patients simultaneously. With the information provided by the self-assessment, nurses will determine a course of action, including the sending of standardized SMS text messages.

In this regard, several scenarios are possible (Multimedia Appendix 4 provides example SMS text messages sent by the nurse). For instance, if a patient’s symptoms and vital signs are stable, an SMS text message with positive feedback will be sent. If the nurse judges that a patient needs to be under closer observation, an SMS text message will be sent asking them to complete the self-assessment again. If a patient requests to speak with the nurse or is in poor condition, the nurse will book a teleconsultation for the afternoon. Opal will display the consultation time in the patient’s appointment calendar and provide a description of the consultation to help them prepare. Triggered by the nurse, an SMS text message will be sent to the patient with a link to the teleconsultation (Multimedia Appendix 5 provides screenshots relevant to teleconsultations). After each teleconsultation, to evaluate patients’ satisfaction with the care received (part of objective 4), a notification will be sent to patients through the Opal app to complete a survey based on the Short Questionnaire for Out-of-Hours Care [33]. It contains 7 items that are rated on a 5-point scale of satisfaction. For this study, a “not applicable” response option was added. All ratings are averaged to generate an overall score (range 1-5). Here, a mean score of ≤4 (“satisfied”) will signify adequate patient satisfaction with the teleconsultation.

If the patient completes the self-assessment after 8 PM, the clinical monitoring team will examine their responses and provide feedback early the next morning (note that the self-assessment contains instructions to go to an emergency department or call 911 if symptoms or vital signs reach specific thresholds). If no self-assessments are received from a patient for 2 consecutive days, the clinical monitoring team will reach out to them or to their emergency contact. During enrollment, these monitoring procedures are explained to participants.

Implementation Strategies


This study draws on recognized implementation strategies [34] (indicated by italics). Implementation strategies are methods that are intended to foster the adoption or implementation of an intervention. The selected strategies are presented based on their corresponding main phase of implementation: preimplementation, implementation, or postimplementation.

Central to many of these strategies is our aim to involve patients and stakeholders through different engagement approaches. Engagement emphasizes the coconstruction of knowledge [35-38] and accountability to stakeholder values, expertise, and perspective relative to a health condition and its associated care [39]. Engagement can take many forms and encompass research and intervention development processes [40]. Importantly, it can help to detect and address challenges associated with recruitment, accessibility, acceptability, and the comprehensibility of procedures and instruments [41]. It can also increase the relevance and meaningfulness of research results [39].

Our stakeholder engagement strategy is enacted by a core multidisciplinary research team. Within this team, YM (junior engineer), BL (physician), and DL (anthropologist) were appointed to engage HIT developers, health care professionals, and patients recovered from COVID-19, respectively.

Strategies During the Preimplementation Phase

To centralize technical assistance, YM will communicate with the Opal development team (TH and JK) and maintain, configure, and update the patient portal. Regular meetings will be held to discuss the configuration of Opal for COVID-19 and ensure that it integrates stakeholder recommendations, as obtained from engaged health care professionals and patients recovering from COVID-19. Changes in Opal will be recorded via the Jira software [42]. YM will also be available to patient participants on email to provide technical assistance throughout the study.

To create a clinical team, BL will unite 4 physicians and 3 nurses with experience in providing COVID-19 care. Their role is to provide recommendations for the following: the use of Opal for remote COVID-19 follow-up, procedures for distance monitoring, the patient’s daily self-assessment, and information or educational material to provide to participants through the patient portal.

To engage patients with COVID-19, DL will meet with 3 people who have recovered from the disease to obtain their feedback on the proposed procedures for remote monitoring and research material (eg, informed consent documents, and all data collection instruments, including the self-assessment). This will help ensure relevance, comprehensibility, and comprehensiveness. Representation will be sought from previously infected laypersons and health care professionals with COVID-19 care experience. They will be recruited by convenience sampling within the research team’s networks.

To conduct cyclical small tests of change, 4 prototype tests with engaged health care professionals (ie, the clinical team) and patients are planned by YM. For each test, members attend a training on Opal, use it for 4 days, and then meet to discuss their experience, report problems, and make recommendations. Furthermore, a low-load run of the patient portal is planned with 5 study participants to adjust the intervention, as needed, before the “full-load” run, with all participants.

Strategies During the Implementation Phase

To record user feedback, coordinators will fill out daily field notes in a logbook. Interactions involving participants or other stakeholders, including unforeseen challenges, technical issues encountered, and any implemented solutions, will be recorded (all entry types in the coordinator logbook are provided in the Study Data Collection and Metrics section).

As described, the initial consent process with participants includes an educational visit to introduce Opal and its features and provide training. We will also select and create education material, that is, videos, frequently asked questions, and reading material, to integrate into the Opal app. This material will include COVID-19 breathing exercises [43], guidelines for self-isolation [44], pulse oximeter and thermometer user guides (Multimedia Appendix 6), and a calendar of study participant tasks. All involved stakeholder groups will review this material.

We will promote adaptability of the intervention and implementation by enabling adjustments to the concerns and recommendations expressed by stakeholders or participants throughout the study. Peripheral aspects of the intervention (eg, timing and content of reminders, timing of consultations, and duration of participation) are adaptable to participant needs and circumstances. For instance, every time they complete a self-assessment, participants are asked whether they wish to speak to a nurse (irrespective of their reported symptoms, vital signs, etc).

Strategy During the Postimplementation Phase

Our final strategy is to purposely re-examine the implementation. We will do this by analyzing participant and stakeholder perspectives on barriers to and facilitators of implementation after implementation (based on the qualitative interviews).

Study Data Collection and Metrics

Study Questionnaires

In part to describe the sample, on day 1, participants will complete a study questionnaire that covers the following: (1) sociodemographic characteristics, including gender, age group, annual income, current occupation, postal code, ethnicity, living arrangements (eg, living alone), and job title; and (2) medical background, including smoking; other medical conditions, such as lung disease or asthma, heart disease, diabetes, kidney disease, and cancer; recent vaccination for influenza and pneumococcal disease; and use of specific medications (eg, nonsteroidal anti-inflammatory drugs, aspirin, acetaminophen, immunosuppressants, and blood pressure medication).

To assess implementation outcomes (for objective 1), a study questionnaire will be administered on a weekly basis (days 1, 7, and 14), with questions on acceptability, response burden, and usability. Acceptability indicates how agreeable, palatable, or satisfactory an intervention is to stakeholders [18]. It will be evaluated with an adapted version of the Acceptability of Intervention Measure [45]. The scale has 6 items rated on a 5-point Likert scale. Item scores are averaged to produce a summary score (range 1-5). In accordance with the developers’ recommendations, a minimum average score of 4 will indicate high acceptability and act as our benchmark of success.

Perceived response burden refers to the effort required to answer a questionnaire [46] and can be considered an aspect of acceptability [18]. It will be measured with a single question and a 5-point response scale that was adapted from a survey question of the United Kingdom Office for National Statistics [46]. Consistent with the previous threshold, the target will be ≥80% (41/51) of participants rating the daily self-assessments as (2) “quite easy” to (1) “very easy” to complete.

Usability refers to the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction” [47] and can be considered to crosscut aspects of several implementation outcomes (feasibility, appropriateness, and acceptability) [17]. It will be assessed with 3 subscales of the Health Information Technology Usability Evaluation Scale [48]. This questionnaire is customizable and specifically designed to evaluate mobile health technology. The selected subscales concern perceived impact (3 items), usefulness (9 items), and ease of use (5 items). Items are rated on a 5-point scale of agreement and averaged to generate subscale scores (range 1-5). A mean score of at least 4 on each subscale, indicating agreement that the technology is impactful, useful, and easy to use, will be our metric of success.

Satisfaction with teleconsultations (for objective 4) will be evaluated with a separate questionnaire sent to patients after each teleconsultation.

Coordinator Logbook

The coordinator logbook will capture qualitative data that will be used to assess feasibility and fidelity (for objective 1) and the rates and nature of contacts with health care professionals (for objective 3). It will include entries on the following: (1) participant questions or challenges during consent and baseline educational meetings; (2) participant recruitment, retention, and fidelity to the intervention; (3) the content of spontaneous discussions with participants or other stakeholders during implementation (eg, the object of participant calls to the coordinators); (4) all contacts with the clinical monitoring team, including clinical notes on teleconsultations with participants; (5) details on those who eventually went to an emergency department; and (6) participant compensation.

Feasibility relates to the extent to which an intervention can be successfully used in a given setting [18]. In this case, it will mainly be assessed by the following: (1) the recruitment rate, that is, the proportion of contacted eligible individuals who are included in the study and (2) the retention rate, defined as the proportion of included participants who are retained over the full follow-up period, both with a target of 75% (38/51) [49].

Fidelity, the degree to which the intervention was implemented as intended [18], will be measured as the proportion of included patients who complete their daily self-assessments over the full follow-up period, with a target of 75% (38/51).

Contacts with the clinical monitoring team will be calculated as the proportion and frequency of participants who (1) wish to talk with the nurse (based on self-assessment data); (2) receive a teleconsultation with a nurse; (3) receive a teleconsultation with a physician; (4) receive mental health support; and (5) eventually go to an emergency department for COVID-19 complications. On the basis of the details recorded in the coordinator logbook, the source of much of these data, that is, the reasons for each of these activities, will also be categorized and described.

Qualitative Interviews

All qualitative interviews will be conducted individually and preferentially by videoconferencing and, if not, by phone. Each interview will be recorded, last for approximately 20 minutes, and follow a semistructured guide with open-ended questions and specific prompts. Although adapted to each stakeholder group, the guide includes ≤5 broad questions on the individual’s experience of and thoughts on using Opal for COVID-19 follow-up, with prompts on challenges and facilitators [50]. Interview guides for each stakeholder group can be found in Multimedia Appendix 7.

Data Analysis

All statistical analyses will be performed with the R software (R Foundation for Statistical Computing) [10,51].

Sample Description

Descriptive statistics will be used to present the participants’ sociodemographic and medical background characteristics. For continuous variables, the minimum, maximum, mean, and SD will be reported. For ordinal and nominal qualitative variables, we will report absolute and relative frequencies (proportions).

Objective 1: Assess Implementation Outcomes With Predefined Success Thresholds

Implementation outcomes will be summarized using descriptive statistics. Acceptability, usability, perceived response burden, and fidelity will be summarized at days 1, 7, and 14. Feasibility will be summarized at the end of the recruitment period (recruitment rate) and at the end of patient follow-up (retention rate). For continuous outcomes, the minimum, maximum, mean, and SD will be reported. For ordinal and nominal outcomes, we will report the absolute and relative frequencies (proportions). The descriptive statistics will also be stratified by gender, age group, and ethnicity.

Acceptability and usability will be evaluated using a linear mixed model (LMM) for each corresponding outcome. The dependent variable of each model will be the outcome and the independent variables will be time (days 1, 7, and 14) and 3 sociodemographic variables reported to influence patient portal use (gender [man or woman], age [≤50 years or ≥50 years], and ethnicity [White or other]) [52]. The goal of each model is to test whether the outcome’s mean score changes significantly over time and whether it differs significantly between the groups represented by the sociodemographic variables over time. LMMs are commonly used in longitudinal studies and allow participants with missing data to be retained in the analysis. Finally, if, at each time point (days 1, 7, and 14), the outcome’s mean score is greater than or equal to the predefined success threshold, we will consider the target to be met. If it is below the threshold, we will use a unilateral t test to test the null hypothesis of threshold attainment, since being slightly below this mark does not imply failure, given the sample mean’s variability.

The evaluation of perceived response burden and fidelity will be performed using a generalized LMM (GLMM) for each corresponding outcome, as it is appropriate when the dependent variable is not continuous. The parameters will be estimated using generalized estimating equations (GEEs), as estimates are sensitive to the specified correlation structure, particularly in noncontinuous outcomes. The independent variables will be time (days 1, 7, and 14) and the 3 sociodemographic variables. To evaluate threshold attainment, if the observed proportion is under the predefined success threshold, we will use a unilateral z-test, which is appropriate for hypothesis testing with proportions.

The evaluation of feasibility will be performed by confronting the observed recruitment and retention rates with the predefined success thresholds at the end of the recruitment period and patient follow-up, respectively. If the observed rates are over or equal to the success threshold, we consider that the target is met. If it is under the success threshold, we will use a unilateral z-test to test the null hypothesis of threshold attainment.

Objective 2: Identify Implementation Barriers and Facilitators

To identify implementation barriers and facilitators, as well as their proposed or enacted solutions, 2 trained coders will conduct a content analysis focusing on the manifest content [53] of this study’s qualitative material. This will include interview transcripts, nurses’ and physicians’ clinical notes, and coordinators’ logbook entries. Analysis will involve three phases [53]: (1) preparation, when analysts get familiar with the data set through immersion in the data; (2) organization, when they deductively code the data using the NVivo software, following a categorization matrix based on Consolidated Framework for Implementation Research (CFIR) constructs [49] to help identify barriers to and facilitators of implementation while remaining open to emerging categories; and (3) reporting, which involves presenting the analytical categories and discussing them during periodic team meetings, as well as any discrepancies in coding or interpretation, to help ensure trustworthiness [53].

Conceptual Framework

The CFIR will guide our qualitative analyses to categorize the barriers and facilitators identified and, quite possibly, to help interpret our findings overall. The CFIR was chosen because it includes 39 distinct constructs grouped within 5 domains of potential influence on implementation, including features of the intervention, the inner and outer settings, the individuals involved, and the implementation process [49].

Objective 3: Describe Service Outcomes

Service outcomes will be summarized with absolute and relative frequencies (proportions) at days 1, 7, and 14. The frequencies will also be stratified by gender, age group, and ethnicity.

Service outcome evaluation will follow the same procedure as for objective 1. As each service outcome is noncontinuous, we will use a GLMM, with parameter estimates obtained by GEE, with the same independent variables.

For objective 3, a qualitative analysis will also be performed on the reasons underlying participant contacts with the clinical monitoring team (eg, teleconsultations) and patient care-seeking in an emergency department, as textually recorded in the coordinator logbook, including the clinical monitoring team’s clinical notes. Content analysis [53], as previously described, will be used for this purpose. This will provide more context to the quantitative data on service outcomes.

Objective 4: Describe Patient Outcomes

The patient outcomes collected from the daily self-assessment will be summarized each day with descriptive statistics. For continuous outcomes, the minimum, maximum, mean, and SD will be reported. For ordinal and nominal qualitative outcomes, we will report absolute and relative frequencies (proportions). The descriptive statistics will also be stratified by gender, age group, and ethnicity.

The evaluation of the patient outcomes will follow the same procedure as for objective 1: an LMM for each continuous outcome and a GLMM for each noncontinuous outcome with parameter estimates obtained by GEE, using the same independent variables except for time, which will be expressed daily (day 1 to day 14). Attainment of the patient satisfaction target will also follow the procedure for objective 1, with a unilateral t test, when the observed mean score is below target.

Mixed Methods Analysis

The quantitative and qualitative data will be analyzed separately and brought together during the interpretation of results for triangulation, comparison, and improved understanding [21]. More precisely, the barriers and facilitators identified through qualitative methods (objective 2) will be used to help interpret the quantitative findings on implementation, service, and patient outcomes (objectives 1, 3, and 4).

Confidentiality, Data Management, and Cybersecurity

The study coordinator will assign a participant number to all participants in this study to protect their identity. A digital file will contain both the participant numbers and identifying information. This file will be stored in a password-protected folder on a secure MUHC server. Interview recordings will be destroyed after transcription. Transcripts will be anonymized and stored as previously described.

Data obtained from patient participants during their follow-up (eg, sociodemographic information, daily self-assessments, and feasibility) will be electronically collected directly through Opal. The Opal team (YM, TH, JK, and JA) will oversee the management of these data, which will also be held in a secure MUHC server.

Opal conforms to security and governance recommendations for the development of patient portals, as identified by the MUHC Security and Governance team. Details are presented in Multimedia Appendix 2 of Kildea et al [12].

Anticipated Risks and Benefits

There are no direct risks to the participants in this study. For Opal, data security risks were addressed with numerous measures, such as disclosing and explaining these to patients, logging users out after 5 minutes of inactivity, sandboxing data on the users’ device and deleting it on log-out, and including security questions in the authentication process [12]. However, patient participants may have unrealistic or unmet expectations of the intervention or a false sense of security [54] with Opal. For instance, they may overly rely on nurses to respond to the information they transmit via Opal instead of taking medical action when needed. To prevent this, the self-assessment instructs patients to seek out emergency services when specific symptoms or vital sign thresholds are attained. For various reasons (eg, ability), participants may also over- or underestimate their symptoms and vital signs, thus affecting the response they receive from the clinical monitoring team.

In addition, the study questionnaires and semistructured interviews may lead to distress related either to the experience of COVID-19 (for patients) or to providing COVID-19 care (for health care professionals). Hence, these research tasks carry a risk of emotional vulnerability. Individuals who experience psychological distress because of their involvement in the study are instructed to inform a study staff member. Resources such as a teleconsultation with a mental health professional or other support services will be made available to them.

The benefits of participating in this study for those with COVID-19 include access to the proposed intervention and follow-up. They will be compensated with CAD $50 (US $65) in the form of a gift card or, in exceptional cases, a money transfer at the end of their participation in the study.


This study secured funding from the McGill Interdisciplinary Initiative in Infection and Immunity Emergency COVID-19 Research Funding (grant ECRF-R2-44) on April 20, 2020, and from the Canadian Institutes of Health Research Strategy for Patient-Oriented Research (Quebec) Support Unit-Methodological Developments (grant M006) on February 12, 2021. Recruitment of patient participants occurred from December 2020 to March 2021, during the second wave of the pandemic in Québec, with a final sample of 51 participants [22]. Qualitative interviews were conducted between April and September 2021 during the third and fourth waves [22], with 57% (39/68) involved stakeholders, among which were 53% (27/51) patient participants, 54% (6/11) health care professionals, 100% (4/4) administrators, and 100% (2/2) HIT developers. The quantitative analyses and the qualitative content analysis of the interviews began in earnest in May 2022. The methods of this pilot study have been the object of 3 conference presentations as of June 2022 [55-57]. In addition, at this time, 2 manuscripts were being prepared, 1 involving a mixed methods analysis of the implementation (objectives 1 and 2) and the other involving patient outcomes (objective 4).

Impact of COVID-19 on Study Progress

Early in the COVID-19 pandemic, tighter regulatory measures for data security were implemented at the MUHC. Complying with these measures delayed the acquisition of Research Ethics Board approval and study initiation. Research staff and data analysts also worked remotely to comply with social distancing measures at the MUHC, which created delays in task delegation and communication. We also experienced a shortage of trained research staff. As a result, most interviews were sent for transcription only once they had all been conducted.

Anticipated Results and Contributions

We anticipate reaching the success thresholds for the studied implementation and patient outcomes. By triangulating quantitative and qualitative findings, we also expect a more complete understanding of institutional (eg, infrastructure, regulations, and human and material resources), professional (eg, expert networks), and medical (eg, symptomatology and evolution of COVID-19 infections) factors that influence the implementation of the intervention and shape its unfolding in the real world. This study will also address several gaps in the literature. Research is limited on the remote monitoring solutions for self-isolating patients with COVID-19 [58], and few such interventions have been evaluated using a mixed methods approach [59] that includes qualitative input from both patients and clinicians. Furthermore, little work has explored the configuration of HIT for chronic care management for use in acute care, such as for COVID-19 and our work will provide an example for how to proceed.

Strengths and Limitations

Strengths of this study include its pragmatism and response to a locally identified gap in the care of people with COVID-19. Its multipronged implementation strategy, with an emphasis on stakeholder engagement and iterative feedback processes, promises to enhance intervention configuration, implementation, and evaluation. The intervention and study also benefit from the pre-existing successful use of the Opal patient portal in oncology at the study site (MUHC) [60] and collaborations between the investigators and Opal developers on other projects (eg, the studies by Chu et al [61] and Engler et al [62]). Thus, the rapid scale-up of this precise intervention at other institutions may be limited.

Limitations of this study also include several sources of potential bias. Indeed, enrollment in Québec’s provincial public health insurance plan (Régie de l'assurance-maladie du Québec) was necessary for patient inclusion, which may have introduced selection bias. Those potentially excluded include members of vulnerable populations, such as resettled refugees [63] and international students [64], groups particularly impacted by the pandemic in Canada, in part, by high COVID-19 incidence rates. In addition, for feasibility, the qualitative interviews were conducted after completing the quantitative data collection. For some participants, interviews took place several months after their follow-up ended. This time lapse may affect the reliability of the qualitative results, in part, because of recall bias. This also meant that this patient input could not be used to adjust the intervention or its implementation during enrollment and follow-up, leading to possible missed opportunities for improvement [65].


Overall, this protocol is designed to generate multidisciplinary knowledge on the configuration and piloting of a patient portal–based intervention for remote COVID-19 follow-up and will lead to a comprehensive understanding of feasibility, stakeholder experience, and influences on implementation that may help in the testing or scale-up of similar interventions. It also promises to produce data that are useful for a wide range of stakeholders, including academics, researchers, health care professionals and administrators, implementation scientists, and HIT developers, who are interested in fostering feasible patient-oriented remote care for COVID-19 and beyond.


This study secured funding from the McGill Interdisciplinary Initiative in Infection and Immunity (MI4) Emergency COVID-19 Research Funding (grant ECRF-R2-44, principal investigator [PI]: BL) and from the Canadian Institutes of Health Research Strategy for Patient-Oriented Research Québec Support Unit-Methodological Developments (grant M006, PI: BL). NK is supported by a career award from the Fonds De Recherche Québec-Santé (Junior 1). BL holds a Canadian Institutes for Health Research, Strategy for Patient-Oriented Research Mentorship Chair in Innovative Clinical Trials for HIV Care. He is also supported by a career award, LE 250, from the Québec Ministry of Health for researchers in Family Medicine. MPP is supported by a clinical career award from Fonds De Recherche Québec-Santé (Senior). The funding sources had no role in the design of this study and will not be involved in the interpretation of the results or the decision to submit them for publication.

The authors thank the Chronic Viral Illness Service research team, particularly, the nurses of the clinical monitoring team, Lina Del Balso, Guillaume Theriault, and Nathalie Paisible. The authors also thank the laboratory research personnel responsible for shipping the medical equipment to our study participants.

The names of the members of the Opal-COVID-19 Patient Expert Committee cannot be provided as they are patients and their names must remain confidential.

Data Availability

The data sets generated and analyzed for this study are available from the corresponding author (BL) on reasonable request.

Authors' Contributions

In no order of contribution, ARC, YM, KE, DL, NK, SB, BL, and the Opal-COVID-19 Patient Committee helped design the study, the intervention, and data collection tools. DL, KE, ARC, SV, YM, and, BL wrote the manuscript. SV planned the statistical analyses. YM, JK, TH, and JA, configured the Opal app for the COVID-19 follow-up. ADP, NK, SB, JC, CC, AH, NZN, and BL participated in the teleconsultations with the study participants. ARC, YM, and DL contributed to data collection. All authors critically reviewed the manuscript and approved the final version.

Conflicts of Interest

BL has received research support and consulting fees from ViiV Healthcare, Merck, and Gilead. NK reports research funding from Gilead Sciences, advisory fees from Gilead Sciences, ViiV Healthcare, Merck, and AbbVie, and speaker fees from Gilead Sciences and Merck, all outside of the submitted work.

Multimedia Appendix 1

Screenshot of the Opal remote registration system.

PNG File , 128 KB

Multimedia Appendix 2

Content of the daily self-assessment questionnaire.

DOCX File , 33 KB

Multimedia Appendix 3

Screenshots of the daily self-assessment questionnaire on Opal: (A) questionnaire introduction, (B) example question, and (C) answer revision before submission.

PNG File , 164 KB

Multimedia Appendix 4

Screenshots of different standardized SMS text messages sent by nurses to patient participants, following review of their daily self-assessments: (A) positive feedback, (B) teleconsultation needed, and (C) closer monitoring needed.

PNG File , 153 KB

Multimedia Appendix 5

Screenshots relevant to procedures involving teleconsultations: (A) teleconsultation shown in the appointment calendar, (B) appointment description to help prepare patients, and (C) teleconsultation SMS text message.

PNG File , 196 KB

Multimedia Appendix 6

Instructional video created for participants on how to use the pulse oximeter.

PNG File , 662 KB

Multimedia Appendix 7

Qualitative interview guides for each of the stakeholder groups.

DOCX File , 36 KB

  1. World Health Organization Coronavirus (COVID-19) Dashboard 2021. World Health Organization.   URL: [accessed 2021-12-04]
  2. World Health Organization. 2020.   URL: https:/​/www.​​emergencies/​diseases/​novel-coronavirus-2019/​question-and-answers-hub/​q-a-detail/​coronavirus-disease-covid-19 [accessed 2020-12-16]
  3. Kujawski SA, Wong KK, Collins JP, Epstein L, Killerby ME, Midgley CM, et al. Clinical and virologic characteristics of the first 12 patients with coronavirus disease 2019 (COVID-19) in the United States. Nature Medicine 2020;26(6):861-868. [CrossRef] [Medline]
  4. Mehrabi Z, Salimi M, Niknam K, Mohammadi F, Mamaghani HJ, Sasani MR, et al. Sinoorbital mucormycosis associated with corticosteroid therapy in COVID-19 infection. Case Rep Ophthalmol Med 2021 Oct 28;2021:9745701 [FREE Full text] [CrossRef] [Medline]
  5. Aboumatar H. Three reasons to focus on patient and family engagement during the COVID-19 pandemic. Qual Manag Health Care 2020;29(3):176-177. [CrossRef] [Medline]
  6. Cummings CL, Miller CS. COVID-19: how a self-monitoring checklist can empower early intervention and slow disease progression. Environ Syst Decis 2021;41(2):181-183 [FREE Full text] [CrossRef] [Medline]
  7. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 2020 Mar 14;395(10227):912-920 [FREE Full text] [CrossRef] [Medline]
  8. Chiappini S, Guirguis A, John A, Corkery JM, Schifano F. COVID-19: the hidden impact on mental health and drug addiction. Front Psychiatry 2020 Jul 29;11:767 [FREE Full text] [CrossRef] [Medline]
  9. Hawton K, Lascelles K, Brand F, Casey D, Bale L, Ness J, et al. Self-harm and the COVID-19 pandemic: a study of factors contributing to self-harm during lockdown restrictions. J Psychiatr Res 2021 May;137:437-443 [FREE Full text] [CrossRef] [Medline]
  10. Jacob L, Smith L, Butler L, Barnett Y, Grabovac I, McDermott D, et al. Challenges in the practice of sexual medicine in the time of COVID-19 in the United Kingdom. J Sex Med 2020 Jul;17(7):1229-1236 [FREE Full text] [CrossRef] [Medline]
  11. Kapoor A, Guha S, Kanti Das M, Goswami K, Yadav R. Digital healthcare: the only solution for better healthcare during COVID-19 pandemic? Indian Heart J 2020;72(2):61-64 [FREE Full text] [CrossRef] [Medline]
  12. Kildea J, Battista J, Cabral B, Hendren L, Herrera D, Hijal T, et al. Design and development of a person-centered patient portal using participatory stakeholder co-design. J Med Internet Res 2019 Feb 11;21(2):e11371 [FREE Full text] [CrossRef] [Medline]
  13. Hendren LJ. OPAL Health. 2019.   URL: [accessed 2022-04-29]
  14. Achelrod D. Policy expectations and reality of telemedicine - a critical analysis of health care outcomes, costs and acceptance for congestive heart failure. J Telemed Telecare 2014 Jun;20(4):192-200. [CrossRef] [Medline]
  15. Bolton CE, Waters CS, Peirce S, Elwyn G, EPSRCMRC Grand Challenge Team. Insufficient evidence of benefit: a systematic review of home telemonitoring for COPD. J Eval Clin Pract 2011 Dec;17(6):1216-1222. [CrossRef] [Medline]
  16. Epping-Jordan JE, Pruitt SD, Bengoa R, Wagner EH. Improving the quality of health care for chronic conditions. Qual Saf Health Care 2004 Aug;13(4):299-305 [FREE Full text] [CrossRef] [Medline]
  17. Grady TG. Stakeholder engagement essential to health information technology project success. Radiol Technol 2015;86(6):682-683. [Medline]
  18. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar;38(2):65-76 [FREE Full text] [CrossRef] [Medline]
  19. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ 2016 Oct 24;355:i5239 [FREE Full text] [CrossRef] [Medline]
  20. Thabane L, Hopewell S, Lancaster GA, Bond CM, Coleman CL, Campbell MJ, et al. Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials. Pilot Feasibility Stud 2016 May 20;2:25 [FREE Full text] [CrossRef] [Medline]
  21. Schoonenboom J, Johnson RB. How to construct a mixed methods research design. Kolner Z Soz Sozpsychol 2017;69(Suppl 2):107-131 [FREE Full text] [CrossRef] [Medline]
  22. Données COVID-19 au Québec. Institut national de santé publique du Québec. 2021.   URL: [accessed 2021-12-04]
  23. Browne RH. On the use of a pilot sample for sample size determination. Stat Med 1995 Sep 15;14(17):1933-1940. [CrossRef] [Medline]
  24. Sim J, Lewis M. The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. J Clin Epidemiol 2012 Mar;65(3):301-308. [CrossRef] [Medline]
  25. Marshall MN. Sampling for qualitative research. Fam Pract 1996 Dec;13(6):522-525. [CrossRef] [Medline]
  26. Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS One 2020 May 5;15(5):e0232076 [FREE Full text] [CrossRef] [Medline]
  27. COVID-19 Symptom Self-Assessment Tool. Government of Canada. 2020.   URL: [accessed 2021-12-16]
  28. Andrejak C. L'application Covid-Quest®. 2020.   URL: [accessed 2021-12-16]
  29. Drew DA, Nguyen LH, Steves CJ, Menni C, Freydin M, Varsavsky T, COPE Consortium. Rapid implementation of mobile technology for real-time epidemiology of COVID-19. Science 2020 Jun 19;368(6497):1362-1367 [FREE Full text] [CrossRef] [Medline]
  30. COVID-19 self-assessment, version 2.8, last update April 16, 2020. Government of Ontario. 2020.   URL: [accessed 2021-12-16]
  31. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, PROMIS Cooperative Group. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol 2010 Nov;63(11):1179-1194 [FREE Full text] [CrossRef] [Medline]
  32. Zoom Security Guide. Zoom Video Communication Inc. 2021.   URL: [accessed 2021-12-04]
  33. The Beryl Institute. 2020.   URL: [accessed 2021-12-16]
  34. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015 Feb 12;10:21 [FREE Full text] [CrossRef] [Medline]
  35. Carman KL, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood) 2013 Feb;32(2):223-231. [CrossRef] [Medline]
  36. Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res 2014 Feb 26;14:89 [FREE Full text] [CrossRef] [Medline]
  37. Pomey MP, Flora L, Karazivan P, Dumez V, Lebel P, Vanier MC, et al. Le « Montreal model » : enjeux du partenariat relationnel entre patients et professionnels de la santé. Sante Publique 2015;1 Suppl:41-50. [Medline]
  38. Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect 2015 Oct;18(5):1151-1166 [FREE Full text] [CrossRef] [Medline]
  39. Patient Engagement. Canadian Institutes of Health Research. 2014.   URL: [accessed 2021-12-16]
  40. Doria N, Condran B, Boulos L, Curtis Maillet DG, Dowling L, Levy A. Sharpening the focus: differentiating between focus groups for patient engagement vs. qualitative research. Res Involv Engagem 2018 Jun 25;4:19 [FREE Full text] [CrossRef] [Medline]
  41. Morgan H, Thomson G, Crossland N, Dykes F, Hoddinott P, ‘BIBS’ study team. Combining PPI with qualitative research to engage 'harder-to-reach' populations: service user groups as co-applicants on a platform study for a trial. Res Involv Engagem 2016 Mar 24;2:7 [FREE Full text] [CrossRef] [Medline]
  42. JIRA Software. Atlassian. 2022.   URL: [accessed 2022-06-27]
  43. Lien P. Coronavirus Recovery: Breathing Exercises. Johns Hopkins Medicine, The Johns Hopkins University. 2021 May 11.   URL: https:/​/www.​​health/​conditions-and-diseases/​coronavirus/​coronavirus-recovery-breathing-exercises [accessed 2021-12-14]
  44. Guidelines for families returning home with a diagnosis of suspected or confirmed COVID-19 infection. Montreal Children's Hospital of the McGill University Health Centre. 2020.   URL: [accessed 2021-01-15]
  45. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci 2017 Aug 29;12(1):108 [FREE Full text] [CrossRef] [Medline]
  46. Hedlin D, Dale T, Haraldsen F, Jones J. Developing Methods for Assessing Perceived Response Burden. European Commission. 2005 Feb.   URL: https:/​/ec.​​eurostat/​documents/​64157/​4374310/​10-DEVELOPING-METHODS-FOR-ASSESSING-PERCEIVED-RESPONSE-BURDEN.pdf/​1900efc8-1a07-4482-b3c9-be88ee71df3b [accessed 2021-12-16]
  47. Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts. International Organization for Standardization. 2018.   URL: [accessed 2021-12-04]
  48. Schnall R, Cho H, Liu J. Health Information Technology Usability Evaluation Scale (Health-ITUES) for usability assessment of mobile health technology: validation study. JMIR Mhealth Uhealth 2018 Jan 05;6(1):e4 [FREE Full text] [CrossRef] [Medline]
  49. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009 Aug 07;4:50 [FREE Full text] [CrossRef] [Medline]
  50. Evans C, Lewis J. Analysing semi-structured interviews using thematic analysis: exploring voluntary civic participation among adults. In: Lewis J, editor. SAGE Research Methods Datasets Part 1. Thousand Oaks, CA, USA: Sage Publications; 2018.
  51. R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. 2021.   URL: [accessed 2021-12-14]
  52. Antonio MG, Petrovskaya O, Lau F. The state of evidence in patient portals: umbrella review. J Med Internet Res 2020 Nov 11;22(11):e23851 [FREE Full text] [CrossRef] [Medline]
  53. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs 2008 Apr;62(1):107-115. [CrossRef] [Medline]
  54. Mejdahl CT, Schougaard LM, Hjollund NH, Riiskjær E, Lomborg K. Patient-reported outcome measures in the interaction between patient and clinician - a multi-perspective qualitative study. J Patient Rep Outcomes 2020 Jan 09;4(1):3 [FREE Full text] [CrossRef] [Medline]
  55. Yuanchao, Engler K, Rodriguez-Cruz A, Lessard D, Mate K, Kronfli N, et al. Piloting a mobile application for daily follow-up of people at home with COVID-19 at the McGill University Health Centre. In: Proceedings of the 48th North American Primary Care Research Group Annual Meeting. 2020 Presented at: NAPCRG '20; November 20-24, 2020; Virtual.
  56. Lessard D, Kronfli N, Rodriguez-Cruz A, Pomey M, Lebouché B, Hijal T, Yuanchao, et al. At home with Opal: engaging COVID-19 patients and healthcare professionals to pilot a mobile follow-up application. In: Proceedings of the 48th North American Primary Care Research Group Annual Meeting. 2020 Presented at: NAPCRG '20; November 20-24, 2020; San Francisco, CA, USA. [CrossRef]
  57. Lessard D, Lymouna M, Engler K, Ma Y, Rodriguez Cruz A, Pomey MP, et al. Chez soi avec la COVID: Consultation d’acteurs clés dans une étude pilote d’une intervention connectée pour le suivi à distance de la COVID-19 (Opal-COVID). In: Association francophone pour le savoir. 2022 May 11 Presented at: ACFAS '22; May 11, 2022; Quebec City   URL:
  58. Fox R, Mulcahy Symmons S, De Brún A, Joyce D, Muldoon EG, McGinty T, et al. Mixed methods protocol to examine the acceptability and clinical characteristics of a remote monitoring programme for delivery of COVID-19 care, among healthcare staff and patients. BMJ Open 2021 Sep 29;11(9):e051408 [FREE Full text] [CrossRef] [Medline]
  59. Gandy K, Schmaderer M, Szema A, March C, Topping M, Song A, et al. Remote patient monitoring: a promising digital health frontier. In: Proceedings of the 2021 International Conference on Information and Digital Technologies. 2021 Presented at: IDT '21; June 22-24, 2021; Zilina, Slovakia p. 297-302. [CrossRef]
  60. Mohsen K, Kildea J, Lambert SD, Laizner AM. Exploring cancer patients' perceptions of accessing and experience with using the educational material in the opal patient portal. Support Care Cancer 2021 Aug;29(8):4365-4374. [CrossRef] [Medline]
  61. Chu D, Schuster T, Lessard D, Mate K, Engler K, Ma Y, et al. Acceptability of a patient portal (Opal) in HIV clinical care: a feasibility study. J Pers Med 2021 Feb 16;11(2):134 [FREE Full text] [CrossRef] [Medline]
  62. Engler K, Vicente S, Ma Y, Hijal T, Cox J, Ahmed S, et al. Implementation of an electronic patient-reported measure of barriers to antiretroviral therapy adherence with the Opal patient portal: protocol for a mixed method type 3 hybrid pilot study at a large Montreal HIV clinic. PLoS One 2021 Dec 30;16(12):e0261006 [FREE Full text] [CrossRef] [Medline]
  63. Edmonds J, Flahault A. Refugees in Canada during the first wave of the COVID-19 pandemic. Int J Environ Res Public Health 2021 Jan 22;18(3):947 [FREE Full text] [CrossRef] [Medline]
  64. Firang D. The impact of COVID-19 pandemic on international students in Canada. Int Soc Work 2020 Jul 14;63(6):820-824. [CrossRef]
  65. Raine R, Fitzpatrick R, Barratt H, Bevan G, Black N, Boaden R, et al. Challenges, Solutions and Future Directions in the Evaluation of Service Innovations in Health Care and Public Health. Southampton, UK: NIHR Journals Library; May 2016.

CFIR: Consolidated Framework for Implementation Research
CONSORT: Consolidated Standards of Reporting Trials
GEE: generalized estimating equation
GLMM: generalized linear mixed model
HIT: health information technology
LMM: linear mixed model
MUHC: McGill University Health Centre
PROMIS: Patient-Reported Outcome Measurement Information Systems

Edited by T Leung; submitted 16.12.21; peer-reviewed by M Salimi, T Ndabu, S Hajesmaeel Gohari, Z Wang; comments to author 12.05.22; revised version received 28.06.22; accepted 11.07.22; published 18.08.22


©David Lessard, Kim Engler, Yuanchao Ma, Adriana Rodriguez Cruz, Serge Vicente, Opal-COVID-19 Patient Expert Committee, Nadine Kronfli, Sapha Barkati, Marie-Josée Brouillette, Joseph Cox, John Kildea, Tarek Hijal, Marie-Pascale Pomey, Susan J Bartlett, Jamil Asselah, Bertrand Lebouché. Originally published in JMIR Research Protocols (, 18.08.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.