This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
Adherence to lifesaving antiretroviral therapy (ART) for HIV infection remains a challenge for many patients. Routine screening for barriers to ART adherence could help make HIV care more patient-centered and prevent virologic rebound or failure. Our team is currently developing a new HIV-specific patient-reported outcome measure (PROM) of these barriers for use in Canada and France along with a digital app for its electronic administration. In our previous work, we developed the PROM’s multidimensional conceptual framework and generated 100 English items, which have been translated to French.
This study aims to use a Web-based Delphi to help validate and select the content of this new HIV-specific PROM, based on the perspective of anglophone and francophone patients and providers in Canada and France. Here, we present the proposal for this Delphi.
This modified Delphi will involve a diverse panel of patients (n=32) and providers (n=52) recruited especially from the 9 sites of the PROM development study (site locations in Canada: Montreal, Toronto, Vancouver; in France: Paris, Nantes, Clermont-Ferrand, Saint-Martin, Cayenne). Overall, 2 rounds of Web-based questionnaires will be conducted. The threshold for consensus is set at 60% and will determine which items are carried forward to the second round. Per item, 3 aspects will be rated: importance as a barrier to ART adherence, relevance for HIV care, and clarity. In both rounds, space will be available for free text comments. Overall comprehensiveness will be assessed in the second round.
This study has undergone a methodological review by experts in patient-oriented research. It has received approval from a research ethics board of the McGill University Health Centre. It is financially supported, in part, by the Canadian Institutes of Health Research’s Strategy for Patient-Oriented Research-Quebec Support Unit (M006). As of May 21, 2019, 15 people living with HIV and 25 providers completed the first round of the Delphi (24 from Canada and 16 from France).
To our knowledge, this is the first Delphi to seek consensus on the most relevant and clinically actionable barriers to ART adherence, collecting opinions on an extensive list of barriers. Drawing on a relatively large and diverse panel of HIV patients and providers, it essentially engages key stakeholders in decision making about the PROM’s final content, helping to ensure its utility and adoption.
PRR1-10.2196/12836
The success of antiretroviral therapy (ART) for the treatment of HIV currently depends on adequate daily adherence to suppress replication of the virus. Both people living with HIV (PLHIV) and providers agree that adherence is among the top priority areas of HIV clinical care [
The patient-reported outcome measure’s conceptual framework of barriers to antiretroviral therapy adherence (revised from Engler et al, 2018, following the item generation and translation processes). ART: antiretroviral therapy.
When developing a PROM, it is important to measure domains that are relevant to the target patients [
From these subdomains, 100 English items were drafted, virtually all referring to a specific barrier mentioned in at least 4 (10%) of the studies included in the synthesis [
Following guidelines for the translation and cultural adaptation of PROMs [
The French version of the PROM was not subjected to readability testing equivalent to that performed on the English version, before translation. However, readability was considered in the translation process. We also expect the Delphi and subsequent steps in our measure’s development (eg, cognitive interviews in both Canada and France) to allow further checks on the items’ readability.
When considering PROM use in clinical care, it is important to consult both patients and providers [
The general objective of this study is to use a Web-based Delphi to help validate and select the content of a new HIV-specific PROM, based on the perspective of patients and providers in Canada and France. Specific objectives are to evaluate the relevance, actionability, comprehensibility, comprehensiveness, and crosscultural equivalence of the instrument’s items (eg, French-Canada vs French-France).
Here, we outline the proposal for this Delphi.
The main research questions addressed and, in parentheses, the type of evidence to be examined are as follows. In the stakeholders’ experience: (1) What items reflect important barriers to ART adherence? (relevance and crosscultural equivalence), (2) What items are relevant to HIV care? that is provide useful information for medical decision making (relevance, actionability, and crosscultural equivalence), (3) Are the items clear? (comprehensibility), and (4) Do the items address all relevant barriers? (comprehensiveness).
The proposed study will employ Delphi survey techniques, which are consensus-building methods [
A Delphi is a useful and recognized method for ensuring the content validity of new measures [
The panel will contain 2 broad stakeholder groups, HIV patients and providers, acknowledging potential differences in opinion [
Although variable, Delphi sample sizes of 15 to 20 participants are common [
A Delphi is usually conducted with panelists who possess subject matter expertise on the given topic [
HIV providers will mainly be recruited within the 9 participating sites of the PROM development study in Canada and France, which is described elsewhere [
Gatekeepers will help identify eligible HIV providers [
In the participating sites, HIV patients will be approached by health care providers or research staff and referred to a designated staff member, if interested in the study. Eligibility and adequate inclusion of women will be verified among those who wish to participate. As with providers, eligible individuals, who accept, will be directed to a secure website where they will be guided through the consent process [
The planned Delphi structure will involve only 2 rounds of data collection with Web-based questionnaires to limit costs, respondent burden, and attrition. To develop all participant questionnaires and acquire informed consent, we will use the SurveyMonkey software (SurveyMonkey Inc, San Mateo, California, USA). Before its use, the round 1 questionnaire, available in French or English, will be piloted with at least 2 clinicians and 2 patients. Once an individual has accepted to participate, has consented, and completed a brief survey on their characteristics, they may begin the Delphi’s first round, which involves completing the associated questionnaire within 2 weeks. Within 2 weeks of receiving the panel’s full data, feedback will be given to the panelists in the form of a report, detailing areas of consensus and disagreement. Any specific instructions for round 2 will be sent with the round 1 report, requesting panelist responses, again, within 2 weeks. Within 2 weeks of receiving round 2’s data, a second report will be provided to panelists, describing the final results.
Although ultimately under participant control, maintaining involvement is important to a Delphi’s rigor and reminders can be used to enhance response rates [
We will ensure the anonymity of panelists to each other but not to members of the research team. This
Attrition is particularly concerning in Delphi studies, because of their multiple rounds [
Following the Web-based consent process, both patients and providers will complete a brief survey to allow description of the panel. Both groups will provide information on their sociodemographic characteristics, whereas providers will answer additional questions on their HIV clinical practice and patients, on their HIV treatment (eg, whether currently on ART). For both Delphi rounds, patients and providers will complete identical Web-based questionnaires and receive instructions in simple language.
Round 1 will be the most intensive for participants. The respondents’ tasks will be, for each item proposed, to rate its (1) importance as a barrier to ART adherence (ie, Is this an important barrier to properly taking ART?), (2) relevance for HIV care (ie, Is this useful information for HIV care?), and (3) clarity (ie, Is the item clearly written? Does it make sense?). They will also provide comments, as needed (eg, suggested corrections and new items). For the list of items evaluated, contact the corresponding author.
After considering the feedback provided in the round 1 report, respondents’ main tasks for round 2 will be to review and rate the contested items on 1 to all of the same 3 aspects: (1) importance as a barrier, (2) relevance for HIV care, and (3) clarity (following item modification, if applicable). The measure’s overall comprehensiveness will also be rated as will be any respondent-proposed items, if included.
Relevance and importance will be measured with a slightly adapted frequently used four-point ordinal scale, appropriate for this purpose [
Responses collected for each patient-reported outcome measure item during round 1 of the Delphi.
Item | Answers | ||||
Is this item... | |||||
|
an important barrier? | No | Somewhat | Quite | Very |
relevant for HIV care? | No | Somewhat | Quite | Very | |
clear? | No | Somewhat | Quite | Very | |
Comments (optional): |
In the absence of standards for determining consensus in Delphi studies [
For these reasons, we determined
For example, if at least 60% of one group or of all participants agree that an item is important (ie, a score of 3 or 4 on the 4-point rating scale), consensus will be considered reached, as for relevance. For clarity, if minimum 60% of at least one stakeholder group or of all participants agree an item is not or only
Items demonstrating consensus on importance and relevance, without clarity problems (as defined above), will not be carried forward into round 2. Any remaining items not meeting this condition after round 2 will be reevaluated. Specifically, given the complexity of the results generated by this Delphi and the panel’s diversity, a multidisciplinary committee formed of investigators, providers, patients, psychometricians, and other experts will be constituted to review the final Delphi results and many potentially relevant comparisons (eg, by country and sex). The committee will make decisions about item removal or inclusion for the PROM and about which results to prioritize in the process.
Consensus, as defined earlier, will determine the inclusion or exclusion of questionnaire items in round 2. However, for informative purposes, the content validity of each item will be calculated with the item-content validity index (I-CVI) [
Crosscultural equivalence will be examined based on
At each round, proportions endorsing each response option and measures of central tendency will be calculated (ie, medians and modes) [
For descriptive purposes, statistical analyses of group comparisons (eg, patients vs providers) with nonparametric tests, appropriate with ordinal data (eg, Chi square and Fisher exact test with dichotomized variables) [
If respondents provide sufficient written comments, they will be submitted to content analysis [
If new items are suggested at round 1 in the comments, they will be compared with the existing items and discussed by the research team to decide if they should be evaluated by panelists in round 2.
In line with recommendations and to stimulate consensus [
This Delphi received methodological review by the Canadian Institutes of Health Research (CIHR), Strategy for Patient-Oriented Research (SPOR)-Quebec Support Unit, between March and May 2018, from Delphi experts on the Method Development team. The McGill University Health Center REB evaluated it as an amendment to the previously approved e-PROM development study. It granted approval on November 30, 2018.
In November 2018, the Delphi Study was presented for feedback to our Montreal-based Knowledge Users Committee, which includes a strong representation of CBOs. As a result, it was decided to include CBO staff on the expert panel. In December 2018, the round 1 questionnaire was piloted in Montreal with 2 providers (a francophone pharmacist and an anglophone nurse) and 2 female patients (1 francophone and 1 anglophone), leading to more specific instructions to patients. To help limit the impact of computer literacy and internet access on patient participation, in partnership with a CBO, AIDS Community Care Montreal, a computer workshop and terminals were available to potential participants, leading to the inclusion of 5 participants in April 2018. There are also plans through our partnership with this CBO to include an incarcerated individual who would complete the questionnaire on paper.
On May 21, 2019, of 58 individuals who were sent the invitation and survey link, 47 opened the invitation, 10 did not, and 1 opted out. Among these, 40 (69%) provided complete data, which represents 48% of our recruitment goal (40/84). The median time to complete the round 1 questionnaire was 1 h 55 min.
Respondents with complete data included 15 PLHIV and 25 providers, with 24 individuals from Canada and 16 from France. Among providers, there were 18 women and 7 men. Those in Canada were from Montreal (n=11), Toronto (n=4), and Vancouver (n=1). Those in France were from Paris (n=8) and Clermont Ferrand (n=1). The provider categories represented were pharmacist (n=8), nurse (n=7), physician (n=6), psychiatrist or psychologist (n=2), social worker (n=1), and CBO staff (n=1). Their year of birth ranged from 1955 to 1988. Over half had 15 or more years of experience treating PLHIV (13/25) and worked exclusively in a hospital setting (14/25). Among PLHIV, there were 7 women, 7 men, and 1 transgender person. They were from Montreal, Canada (n=9), and Clermont Ferrand (n=4) and Paris, France (n=2). Their year of birth ranged from 1947 to 1999. Approximately half had immigrated to their country of residence (7/15) and described their sexual orientation as heterosexual (8/15). Over a quarter (4/15) had ever injected drugs. All were currently on ART, with a quarter (3/14) reporting not being satisfied with their latest ART regimen.
From the scale up of recruitment (April 2019), the Delphi is expected to be led over 6 months. Study findings will be communicated through peer-reviewed publications, conference presentations, and other forms of knowledge dissemination (eg, academic rounds and Web-based reports in partnership with CBOs).
To our knowledge, this is the first Delphi to seek consensus on the most important and clinically actionable barriers to ART adherence, drawing on a relatively large and diverse panel of HIV patients and providers. For the e-PROM’s intended use in routine HIV care in Canada and France, the Delphi will serve to identify items that should be accorded priority or that require revision for clarity. Country and language group differences in ratings will also provide indications of the crosscultural validity of the measure items. Essentially, this Delphi will engage important stakeholders in decision making about the measure’s final content, helping to ensure its utility and adoption.
Routine e-PROM collection in HIV health service provision is in its infancy, despite notable initiatives [
Untitled.
antiretroviral therapy
community-based organization
Canadian Institutes of Health Research
electronic PROM
item-content validity index
people living with HIV
patient-reported outcome measure
research ethics board
Strategy for Patient-Oriented Research
KE, DL, and IT are funded in part by a SPOR Mentorship Chair in Innovative Clinical Trials awarded by the CIHR to BL (grant number 383427). The Delphi is funded, in part, by the Quebec SPOR Support Unit-Method Development component (M006). The Delphi is a subproject of the PROM development study, the I-Score study, which is supported in part by the CIHR HIV Clinical Trials Network (CTN 283), Mitacs-Accelerate (grant number IT05857), and by a research grant from the Investigator Initiated Studies Program of Merck Canada Inc (grant number IISP-53538). The opinions expressed in this study are those of the authors and do not necessarily represent those of Merck Canada Inc or its affiliates or related companies. The authors thank Patrick Keeler from AIDS Community Care Montreal for his valuable input and work on this study. They also thank Rosalie Ponte for coordinating the study and Leo Wong for assistance with
None declared.