This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.
There is an increasing amount of electronic data sitting within the health system. These data have untapped potential to improve clinical practice if extracted efficiently and harnessed to change the behavior of health professionals. Furthermore, there is an increasing expectation from the government and peak bodies that both individual health professionals and health care organizations will use electronic data for a range of applications, including improving health service delivery and informing clinical practice and professional accreditation.
The aim of this research program is to make eHealth data captured within tertiary health care organizations more actionable to health professionals for use in practice reflection, professional development, and other quality improvement activities.
A multidisciplinary approach was used to connect academic experts from core disciplines of health and medicine, education and learning sciences, and engineering and information communication technology with government and health service partners to identify key problems preventing the health care industry from using electronic data to support health professional learning. This multidisciplinary approach was used to design a large-scale research program to solve the problem of making eHealth data more accessible to health professionals for practice reflection. The program will be delivered over 5 years by doctoral candidates undertaking research projects with discrete aims that run in parallel to achieving this program’s objectives.
The process used to develop the research program identified 7 doctoral research projects to answer the program objectives, split across 3 streams.
This research program has the potential to successfully unpack electronic data siloed within clinical sites and enable health professionals to use them to reflect on their practice and deliver informed and improved care. The program will contribute to current practices by fostering stronger connections between industry and academia, interlinking doctoral research projects to solve complex problems, and creating new knowledge for clinical sites on how data can be used to understand and improve performance. Furthermore, the program aims to affect policy by developing insights on how professional development programs may be strengthened to enhance their alignment with clinical practice. The key contributions of this paper include the introduction of a new conceptualized research program, Practice Analytics in Health care, by describing the foundational academic disciplines that the program is formed of and presenting scientific methods for its design and development.
PRR1-10.2196/27984
Emerging digital technologies for collecting and using eHealth data have the potential to make data more accessible to individual clinicians, clinical teams, organizations, and the general public. The increasing accessibility of eHealth data provides opportunities for their use in a wide range of applications, including quality improvement activities [
CPD is a cornerstone of health education and has a recognized place in maintaining high-quality care [
Although electronic data are a rich source of information for health professionals’ practice, health professionals currently have limited access to their data for this purpose [
The difference between the use of eHealth data for quality improvement and health professional learning.
Thus far, a notable gap in health professionals’ education and practice reflection is the use of eHealth data contained in clinical data sources such as electronic health records (EHRs), patient administrative systems, and registries. The value of using electronic data, such as that collected by EHRs for training and education, has been acknowledged in the literature [
Despite the growing emphasis on the use of eHealth data to enable practice reflection, there are a number of challenges that need to be considered. A notable challenge relates to data management and ensuring the quality of data used while also considering privacy and security concerns [
Another challenge the health sector faces in harnessing the full potential of eHealth data is making the data actionable to health professionals in ways that can lead to improvements in quality, outcomes, or cost of care [
Engineering and information communication technologies:
Computer science
Human–computer interaction
Learning analytics
Visualization
Data analytics
Machine learning
Data mining
Medicine and health:
Epidemiology
Public health
Implementation science
Quality improvement
Ethics and law
Data privacy and security
Learning sciences and education:
Medical education
Health professions education
Learning analytics
The program described in this paper seeks to understand how health professionals can be supported by digital technology to make effective use of eHealth data to support practice reflection. This will be achieved through a comprehensive research program delivered through a number of discrete doctoral research projects.
The aim of this research program is to make eHealth data captured within tertiary health care organizations more actionable to health professionals for use in practice reflection, professional development, and other quality improvement activities, with the ultimate aim of improving patient outcomes.
The research program described in this protocol is a problem-driven multidisciplinary research program between academic partners and the Digital Health Cooperative Research Centre (DHCRC), and industry partners made up of government and health service organizations. The research program is funded by the DHCRC [
The multidisciplinary team guiding the delivery of the research program and supporting the project includes academic institutions (2/6, 33%), health care organizations (3/6, 50%), and peak bodies in the Australian health sector (1/6, 17%). The team was engaged early during the problem identification and research formulation stage, aligned with recommendations in the literature on how to undertake multidisciplinary research [
The research program incorporates scientific theories from implementation science and action research. Action research is based on action, evaluation, and critical analysis of data to drive improvements, and it is commonly used to improve practices in various health care environments [
The research program was co-designed by an academic and industry team supported by a program manager. Academic team members (n=5) brought expertise from a range of specialized research fields from the disciplines of medicine and health, engineering and information communication technology, and education and learning sciences. Industry team members (n=6) included representatives from private tertiary hospitals and representatives from peak national bodies.
To develop the study design, industry representatives who may have interest or expertise in using eHealth data with health professionals were identified from the DHCRC network. A one-on-one meeting with each industry representative and the lead investigator for the research program was conducted to identify key points and research priorities between May and July 2019. Meeting notes were synthesized into a preliminary research program outline by 2 researchers (TS and AJ) familiar with the area. This outline was then circulated to each industry representative who was interested in the research program for feedback.
Industry representatives who indicated continued interest in the direction of the research program participated in a planning workshop in August 2019. This workshop involved an open discussion between academic and industry attendees (n=14) to agree on the final objectives of the program. The workshop also identified the outline of the doctoral projects that would be achievable over the course of full-time candidature and that would meet the priorities of the industry representatives. Feedback from the planning workshop was reviewed and used to turn the research outline into a comprehensive research program road map. The research program was built around 3 streams. The protocol also described 6 objectives for the research program.
Overview of the 3 research streams in the Practice Analytics in Health care program.
Research stream and objectives | Doctoral project | |
|
||
|
Understand the readily available data that is most likely to be useful in performance feedback and continuous practice enhancement and explore the ethical considerations of using patient-reported outcomes and the barriers and enablers to health professionals using these data in practice |
Defining clinical practice indicators Optimizing the actionability of patient-reported experience and outcome measures (future project) |
|
||
|
Determine the acceptability of different tools for feeding back performance data to individual clinicians and health care teams, understand the processes medical practitioners use to make sense of the data presented to them, and understand the ethical and policy implications for organizations and individual clinicians when using eHealth data for reflective practice |
Visualizing performance data Data sensemaking Ethical, medico-legal, and policy implications of Practice Analytics (future project) |
|
||
|
Explore how performance data can be linked to professional development requirements, clinical governance, and hospital accreditation standards and understand how transition points in clinical careers influence the quality and usability of electronic data for practice reflection |
Understanding the role of performance data in formal and informal professional education Exploring lifelong learning and career transition points in Practice Analytics (future project) |
A final full-day workshop was held with academic and industry representatives (n=17) in February 2020 to review the final scope of the research program and confirm the doctoral projects and other logistical considerations. On the basis of the conclusion of the second workshop, there was a consensus on the 3 streams that would make up the research program. The scope for each of the 4 initial doctoral projects that would be embedded in the research program was also confirmed along with a road map for 3 future doctoral projects.
The research program will be enabled via an integrated set of doctoral research projects over a 5-year period. Ensuring the privacy and confidentiality of data is a key consideration for the research program. The research program procedures are in place with considerations for the research data being collected and the health data being used. To ensure that all research data are managed appropriately, each doctoral candidate will obtain appropriate human research ethics committee approval, which will require compliance with the National Statement on Ethical Conduct in Human Research [
Regarding data about health professionals, individual doctoral candidates may use identifiable data. However, the process for accessing and using these data for each doctoral candidate will be approved by the relevant human research ethics committee. No identifiable data will be extracted or used without prior written consent. Finally, because of the importance of data privacy and ethical and legal considerations in the PAH space, the research program has prioritized a specific doctoral project to research these issues (
Each doctoral research project is supported by a team comprising, at minimum, a primary academic supervisor with expertise in the research area, an industry supervisor with clinical or technological domain knowledge, and a postdoctoral fellow funded through the program.
The collaborative methodology used to develop the Practice Analytics in Health Care research program.
The process used to develop the research program identified 7 doctoral research projects to answer the program objectives, split across 3 streams. The following section presents a brief description of each doctoral research project and shows their alignment with the 3 streams of the research program.
This research project is designed to understand how to extract and structure data within health care organizations so that it can be used effectively for performance feedback. It is currently challenging to use eHealth data collected in health care organizations, such as data from patient administrative systems and electronic medical record (EMR), as the systems were not designed to capture data for this purpose. Currently, these systems are not designed with a primary focus on supporting health professional training and practice reflection. As such, it is necessary to explore how to repurpose these data for this project. The project will do this by identifying clinical indicators for individual diagnoses or procedures across different specialties that have clinical validity for the performance profiling of individual clinicians. Once the clinical indicators have been agreed upon by key stakeholders, a data audit will be undertaken to determine whether the quality and completeness of data within an individual health care organization are sufficient to populate each indicator by using routinely collected data.
The project will explore the value of a generic set of patient-reported experience and outcome measures to use in informing clinical practice across key disciplines. The project will also explore the value of discipline-specific patient-reported experience and outcome measures and patient survey data sets for informing clinical practice.
This research project will explore how performance feedback tools are designed to enable health professionals to engage with their data. Research on the design of these platforms for use by health professionals is currently limited. This research project will address this problem by exploring the extent to which a one-size-fits-all solution is viable for feeding back performance data and contrasting it to more personalized approaches. When designing performance feedback tools, personalized approaches to visualizing data could account for factors such as health professional data literacy, different information needs, history of use of the data, and context-specific data use. It will also explore how performance feedback tools can support health professionals who desire a high level of engagement with their data rather than just supporting entry-level needs.
The overarching aim of the project is to understand how health professionals make sense of data related to their performance. Specifically, the project seeks to investigate how EMR and other health administration data are currently used by health professionals across different health care organizations. Currently, there is little research exploring the processes that health professionals use to make sense of their data, which is a problem because it is a barrier to the more widespread use of electronic data by this cohort. This project will address this problem by exploring the process of data sensemaking across different disciplines and organizational contexts. The project also aims to understand the factors that inform health professionals make sense of information such as emotion and previous experiences. Finally, the project will study how sensemaking informs further action in terms of both CPD and practice.
This research project is designed to understand the ethical and policy implications for organizations and individual clinicians when using eHealth data for reflective practice. The project will likely explore questions such as how organizations, individuals, and teams respond when data show an individual outlier that needs support and what information must be disclosed outside the organizations. The project will explore the way ethico-legal implications of using data for reflective practice affect engagement by specialists and other health professionals. This includes the nexus between using performance data from performance improvement rather than performance management and in potential litigation.
This research project seeks to understand how medical regulatory bodies, education providers, and health care organizations can use eHealth data to personalize training programs. This is a challenge for health professionals, medical regulatory bodies, education providers, and other key stakeholders because there is an increasing expectation that health professionals will engage with their practice data for lifelong learning and other professional development activities. The project will address this problem by exploring how eHealth data from health care organizations can be used to better understand the clinical and professional practices of health professionals. Finally, the project will also investigate the attitudes of health professionals to the use of their data for professional development and reflective practice.
This research project is designed to understand how transition points in clinical careers influence the quality and usability of electronic data for practice reflection. The project will look specifically at the experiences of health professionals as they transition from specialist training to early-career specialty fellowships. It will also look at how data are presented and reported to health professionals in a way that captures knowledge, skills, and professional growth that occurs throughout a clinical career.
This protocol paper provides a description of the term PAH and describes the academic disciplines that contribute to this multidisciplinary field. Furthermore, it presents an overview of the methodology for designing a PAH research program. The research program is the first to bring together academic disciplines combined with industry partners to identify and solve problems they face, increasing the actionability of eHealth data by health professionals for reflective practice and improvement. Finally, the protocol paper outlines how the aims of the research program will be achieved over the course of the program through an intervention that interlinked doctoral projects undertaken by doctoral candidates with diverse skill sets. At the conclusion of the research program, the intervention will have fostered strong partnerships between academic and industry partners; developed new knowledge in the field of PAH to address gaps in the research regarding the extraction, integration, and use of electronic data to change behavior; and translated key findings from the research program to clinical and regulatory settings to change practice.
A problem-driven multidisciplinary approach was foundational in designing the research program described in this protocol. To develop the PAH research program, experts with different perspectives and expertise worked collaboratively to refine the problem to focus on how electronic data within health care organizations could be used to support individual learning and strengthen professional development. The important role of multidisciplinary approaches involving academic and industry experts throughout the process can be challenging; however, through a guided process of discovery, ideation, and development, it can give voice to both health professionals and academic specialists to drive necessary changes [
A key component of the research program is harnessing individual research projects to deliver an intervention that meets the needs of both academic and industry stakeholders. Each individual doctoral candidate brings knowledge and skills from their own discipline and applies them to answer a specific research question in service of the aims of the Practice Analytics research program. By harnessing principles from action research [
To ensure that individual research projects are aligned with the goals of the PAH program, a number of processes have been implemented. These processes include having touch points with representatives of all industry partners supporting the Practice Analytics program, regular meetings between the academic supervisor team for each candidate, regular meetings between the academic supervisor team and the doctoral candidates, and regular meetings between the doctoral candidates and the postdoctoral fellows. These meetings are augmented with shared web-based repositories of information and serve the dual purpose of acting as coordination tools for the project and facilitating shared communication between all parties. Establishing an overall program management and coordination strategy is a recommended practice for successful multidisciplinary research programs [
This research program will need to draw on key learnings from different disciplines to address problems in the PAH field. An example of how this research program leverages research knowledge from diverse disciplines relates to a key challenge in the health sector: making eHealth data more actionable. To date, the health care industry has not fully achieved the potential benefits of using eHealth data to support health professional learning [
Finally, because of the multidisciplinary nature of the research program, it is anticipated that outputs will have an effect not only on health care but also across a range of sectors. The research program will also incorporate established theories from implementation science [
The use of a multidisciplinary research program built around core objectives that align with the priorities of both industry and academic stakeholders is a unique approach that aims to unpack the potential of eHealth data to support learning, practice reflection, and other training activities of health professionals, improving the quality of care and patient outcomes. It is anticipated that this program will be extended to specific translational outputs informed by the research findings.
continuing professional development
Digital Health Cooperative Research Centre
electronic health record
electronic medical record
Practice Analytics in Health care
The research described in this protocol is funded through a project grant from the Digital Health Cooperative Research Centre (DHCRC). The DHCRC is established and supported by the Australian Government’s Cooperative Research Centres Program. The DHCRC project primarily grants funding scholarships for higher-degree research candidates and postdoctoral research fellows. Author AJ is undertaking a postdoctoral research fellowship that is fully funded by the DHCRC.
Author TS is the Director of Research at Digital Health Cooperative Research Centre. Author AJ is undertaking a postdoctoral research fellowship that is fully funded by the Digital Health Cooperative Research Centre. Authors ST, JK, and DG declare that they have no conflicts of interest.