This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
The quality of care for tuberculosis (TB) is deficient in high-burden countries and urgently needs improvement. However, comprehensively identifying the required improvements is challenging. Providing high-quality TB care is an important step toward improving patients’ quality of life and decreasing TB morbidity and mortality. Effective tools for assessing the quality of TB services using international standards and guidelines can identify existing gaps in services and inform improvements to ensure high-quality inpatient TB services.
This study aimed to develop evaluation instruments for defining the quality of provision of TB services.
To assess quality of services in the largest TB hospital in Armenia, we developed instruments based on the Joint Commission International Accreditation Standards for Hospitals, International Standards for TB Care, TB Laboratories Bio-Safety Standards, and the World Health Organization framework for conducting TB program reviews. A mixed methods approach was utilized, triangulating quantitative (checklists) and qualitative (in-depth interviews) results. A scoring system and strengths, weaknesses, opportunities, and treats analysis was applied to detail results for each of the 122 standards assessed. A scaling approach was used to present overall performances of inpatient services for eight patient-centered functions and five organization management functions.
Overall, 40 in-depth interviews and 91 checklists (21 observations, 16 policy papers, 20 staff qualification documents, and 34 medical records) were developed, utilized, and analyzed to explore practices of health care professionals, assess inpatient treatment experience of patients and their family members, evaluate facility environmental conditions, and define the degree of compliance to standards.
The effective comprehensive evaluation instruments and methods developed in this study for quality of inpatient TB services support the implementation of similar effective assessments in other countries. It may also become a platform to develop similar approaches for assessing ambulatory TB services in resource-limited countries.
DERR1-10.2196/13903
In 2014, the 67th World Health Assembly set a goal for the year 2035: The number of tuberculosis (TB) deaths will reduce by 95%, the TB incidence rates will reduce by 90% as compared with those in the year 2015, and no family will “face catastrophic costs due to tuberculosis” [
To achieve the goal of reducing the burden of TB worldwide, the special challenges of MDR-TB in high-burden resource-limited countries such as Armenia have provided impetus for new recommendations, moving from a focus on expanding coverage of free TB diagnostic and treatment services to a focus on quality of services [
The internationally adopted Joint Commission International (JCI) Accreditation Standards for Hospitals are evidence-based standards to measure the quality of services provided in hospitals in order to improve performance and outcomes of hospitals [
The Quality Improvement Handbook for TB and MDR-TB Programs identifies three perspectives on the quality of TB care: the perspectives of patients, service providers, and health facility managers [
Assessments of quality of TB health care have typically relied heavily on quantitative assessment methods, including survey instruments, checklists using simulations, checklists for direct observations in the health care facility, and chart reviews or audits [
The most comprehensive understanding of the quality of health care and underlying factors influencing quality requires a combination of both quantitative and qualitative assessment methods, a
For a larger, more comprehensive, valid evaluation of the quality of diagnostic and treatment services of the largest TB hospital of the National Tuberculosis Control Center (NTCC) in Armenia, we used a mixed methods study design. We integrated qualitative and quantitative methods by triangulating results from all data collection instruments and methods [
We developed the study instruments based on suitable JCI Standards for Hospital Accreditation, International Standards for TB Care, TB Laboratories Bio-Safety Standards, and WHO Framework for Conducting TB Programs Review [
The study instruments include document review checklists for policy review, staff qualification review, and medical records; observation checklists for TB patients’ admission, access and continuity of care, laboratory services, medication storage management and use, infection prevention and control, and kitchen and food storage; and in-depth interview guides for 11 groups of key informants (
The Institutional Review Board of the American University of Armenia approved the study for compliance with locally and internationally accepted ethical standards (protocol number: AUA-2016-002).
All participants were informed of their rights; all those who chose to participate provided verbal informed consent. Audio recording and observations were possible only with permission of participants; if a participant did not want to be audio recorded, only written notes were taken.
Flowchart for data management and analyses. JCI: Joint Commission International; NTCC: National Tuberculosis Control Center; SWOT: strengths, weaknesses, opportunities, and threats; TB: tuberculosis; WHO: World Health Organization.
Policy documents were reviewed to understand the formal documents that regulate the hospital’s daily practices in relation to TB treatment and diagnosis. Reviewed documents included organizational charter of the NTCC, internal disciplinary rules of the organization, and internal regulations of different structural units. We also reviewed several guidelines and national regulations, such as hand-hygiene guidelines, waste and expired materials’ disposal guidelines, methodological guides for TB infection control, and the national standards for TB treatment and diagnosis.
The staff’s qualification document review was designed to evaluate compliance of relevant professional experience, qualifications, and credentials required for job duties and responsibilities of the staff.
Medical records review included reviews of medical records and TB treatment cards of patients with TB admitted for TB inpatient treatment at the NTCC 2 months before the assessment. We selected 2 months before the assessment to have adequate time to review the full range of medical records per patient (medical history, TB treatment card, and MDR-TB treatment files). After discharge, patients’ TB treatment cards are transferred to TB outpatient centers. All clinical departments that did not admit new TB patients during the data collection period were asked to provide all records they had from 2016.
Observations were designed to explore several environmental conditions and daily practices of health care providers in TB treatment and diagnosis using standardized checklists. To evaluate the quality of diagnostic laboratory services, we observed daily practices in both bacteriological or microbiological and clinical laboratories of the hospital, considering the degree of compliance with the WHO biosafety standards of TB laboratories [
The study team identified key informants from clinical departments and administrative units (based on experience and expertise on inpatient care) by using purposive sampling to optimize information acquisition and convenience sampling for those willing to participate, given the optimal utilization of available resources. All stakeholders of inpatient TB services were included to ensure validity. In-depth interviews included 11 key informant groups or sampling units to protect their confidentiality and provide
The quantitative assessment checklists (legal and staff qualification documents review, medical records, and observations) data were entered in a Microsoft Excel 2013 worksheet for further analysis.
The qualitative in-depth interview data were analyzed utilizing deductive content analysis with a structured matrix [
To integrate qualitative and quantitative results to measure compliance to the standards, we developed a single checklist comprising all assessed standards with their measurable elements. Using this checklist, we applied integrated
Next, we developed a scoring system, setting the maximum score for each standard to 10. Applying weighted scores to measurable elements within each standard, we calculated the score of the assessed standards by summing the scores of their measurable elements. The number of measurable elements for each standard ranged from 2 to 10 (average=4), depending on the standards’ complexity. We based our evaluation of compliance to the NTCC’s daily practices on the obtained scores (scored from 0 to 10) for each assessed standard.
After developing the coding scheme and the scoring system, we conducted a strengths, weaknesses, opportunities, and threats (SWOT) analysis, grouping all the findings into SWOT for each of the 122 standards [
Furthermore, to measure the overall performance of inpatient services of the NTCC in meeting the 13 assessed functions, we calculated the
Overall, 16 different internal policy papers and national regulations were reviewed to complete the policy review checklist. We used the policy review checklist to assess the presence or absence of certain policies and procedures that are recommended internationally. The staff’s qualification documents review utilized 20 personnel files, including descriptions of positions of all employees from all departments and the staffing plan of the organization for which standardized checklists were utilized. The medical records review included 34 medical records and TB treatment cards of patients with TB and utilized standardized checklists (
Applying the standardized checklists, we conducted 21 observations in the hospital and in its vicinity (
Overall, 40 in-depth interviews of key informants were conducted: NTCC managers (n=2), heads of departments (n=5), TB physicians and nurses (n=21), TB patients and their family members (n=4), laboratory staff (n=2), pharmacist (n=1), infection prevention and control staff (n=1), radiologist (n=1), monitoring and evaluation staff (n=1), continuing education staff (n=1), and human resource management staff (n=1;
The calculated scores of the 122 assessed standards and their SWOT analysis contributed to understanding the details and the extent to which inpatient TB services in Armenia comply with local and international standards. The scaling approach, which was applied to evaluate the overall performance of inpatient services for its 13 functions, helped visualize existing gaps in patient-centered and organization management levels of the system and share findings with a wider audience.
Improving the quality of TB health services is possible through adherence to international standards adapted for the local resource-limited context. Modern mechanisms of patient safety and quality assurance in inpatient and diagnostic facilities will result in sustained improvements in operations and improved quality of care provision and will create a safer environment for patients with TB including those with drug-resistant TB. The suggested protocol for quality assessment could help identify gaps in quality of care and patient safety; addressing those gaps could strengthen
National TB programs in other countries (beside Armenia) can use similar innovative mixed methods and instruments to determine compliance of their TB care systems with the internal policies and procedures and national and international guidelines to improve TB care.
Moreover, this approach of inpatient assessment of TB services can be applied for developing and adopting mechanisms for ambulatory assessment of TB services, providing resource-limited national TB programs with a tool to comprehensively measure compliance of TB services with the international standards.
Standards of patient-centered and health care organization management functions to evaluate the quality of inpatient diagnostic and treatment services.
Examples of strengths, weaknesses, opportunities, and threats and overall performance analyses.
Joint Commission International
multidrug resistant
National Tuberculosis Control Center
strengths, weaknesses, opportunities, and threats
tuberculosis
World Health Organization
We are grateful to the Armenian Medical Fund for the financial support. We are thankful to the administration of the National TB Control Center of the Ministry of Health of the Republic of Armenia and the inpatient center clinical staff, who provided their continuous support throughout the study. This work was financially supported by the Armenian Medical Fund, United States.
NT, ZG, LM, and VP designed the study. NT, ZG, and LM implemented the study activities. NT and BC drafted the manuscript. NT, ZG, LM, BC, and VP reviewed, commented on, and approved the manuscript for publication.
None declared.