This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.
The prevalence of information technology (IT) in society is a foundation for new modes of interaction between patients and health specialists. IT plays an important role in the renewal of care. Several countries have incorporated eHealth plans into their national health strategies. Part of the eHealth evolution concerns Internet psychological treatment and psychosocial care. These interventions are complex to design and evaluate due to legal, ethical, organizational, technical, and methodological challenges.
The objective of our study was to seek to make explicit contributions to the understanding of ideals in eHealth research, and illuminate their implications for establishing an effective research environment. Our analysis draws from three years of experience in establishing an eHealth research environment, and the literature.
We worked inductively to characterize challenging research ideals, and their origins, in our environment. Thereafter, we made a selective search of the literature to scrutinize and illuminate each ideal and it’s implications.
In this work, we propose a structured approach to address ideals in eHealth research. The scrutinized ideals are accountability, innovation, rigor, relevance, and sustainability. The approach supports researchers to systematically understand the ideals, their origin, and to manage their implications within an eHealth research environment.
The complexity of eHealth research causes a need for sustainable, multi-disciplinary research environments. There is a need for a structured approach to organize eHealth research. The proposed approach helps to systematically scrutinize ideals, thus promoting high quality research.
The prevalence of information technology (IT) in society is a foundation for new modes of interaction between patients and health specialists. While the field of eHealth is still in its infancy, it is clearly conceived of as an important strategy for the future of health care. This is signified by the status of eHealth as a key area in the Digital Agenda for Europe and the Innovation Union, both major parts of the Europe 2020 strategy presented by the European Commission in 2010.
Part of the eHealth evolution is the growth of Web-based psychological treatment and psychosocial care. Web-based self help is effective for psychiatric disorders and promotion of health behaviors [
Web-based interventions are complex to develop and evaluate, and substantial investments are required [
The challenges with eHealth research at least partly originate from the environment in which the research takes place. Researchers need to relate to ideals put forward by numerous stakeholders including, but not limited to, the research community, legislative bodies, and the media. Ideals are in flux, continually reinforced and challenged by stakeholder groups [
The research setting at hand is a multi-disciplinary research environment at Uppsala University, Uppsala, Sweden, between 2010 and 2014, and the development of the Uppsala University Psychosocial Care (U-CARE) program. The program was established to support three randomized controlled trials on Internet-based treatment of depression and anxiety for patients with somatic disease. The development may be characterized as an entangled design of the research environment, trials, interventions, and software. The U-CARE program involves academics from psychology, medicine, information systems, caring sciences, and economics, as well as health practitioners. A set of organizational decisions was made at the inception of the program. A scientific advisory board was established, including a set of scholars with expertise in their respective academic fields. The scientific advisory board meets with U-CARE staff annually to provide feedback on the U-CARE research activities. In addition, coordination groups, including researchers from different disciplines, were established to coordinate the work between and within various studies. Information systems researchers were contributing practically with software development, as well as doing research on information systems issues.
U-CARE is oriented toward eHealth services for patients. The overall aim of the U-CARE program is to prevent and reduce emotional distress in persons struck by a somatic disease. Being struck by a potentially life threatening disease such as cancer can cause, for example, depression and anxiety [
The eHealth software at hand was designed to be configurable in a number of ways to facilitate the diverging needs within the three initial trials. An example of this flexibility is the option to either compose new interventions that are to be evaluated in additional trials, these possibly based on the intervention content in the original trials, or to reuse the original content. In the same manner, it is possible to use the questionnaires in the original trials, or to use new ones as well as to use the original inclusion and randomization logic, or to develop a new logic for inclusion and randomization. The software and the interventions have already attracted a number of research groups who will perform observational or intervention studies via the software. At this point, nine research groups are in different stages of planning and starting studies. These studies are both benefitting from and contributing to knowledge and technology within the U-CARE environment. In addition, the ethical approval required for each trial increased the experience of ethical considerations in online trials among U-CARE staff.
Our research approach follows the pragmatist assumption that design and intervention in a real world setting are effective ways to understand social mechanisms [
The domain at hand is the evolution of a research environment. Given the complexity of the environment, aiming at researching psychosocial support for people with somatic disease, through collaboration across disciplines and including multiple hospitals, we conceive of the establishment of the research environment as a rich source for reflections on performing eHealth research. During the evolution of the U-CARE environment, a number of situations occurred that were not easily remedied. The situations typically included design complexities that were not anticipated, for example, the uncertainty on how to manage details of IT-reliant communication between patients and staff. Over time, the awareness emerged that these challenges needed to be more systematically addressed to avoid “bottlenecks” in the design process. The experiences connected to these situations were interpreted in the light of relevant literature and abstracted into the results presented in this paper, consisting of five identified ideals that affect eHealth research, and an approach to scrutinizing such ideals. The process was initially inductive, for example, it was based on the prevalent challenges that were encountered during the design of research protocols, patient treatment, and software. Through literature studies, the approach was iteratively refined in an interpretive and hermeneutic process. The resulting approach is thus an abstraction from a single case study [
In this section, we provide a structure to scrutinize ideals based on: (1) a practice perspective on organizations, and (2) a stakeholder centric approach. Since the objective of this paper is to discuss the challenges in building an eHealth research environment, there is a need for reflection about the definition of a “research environment”. Numerous theories can be used to explain and analyze organizations. A contemporary view is that organizations may be studied using a “practice lens”. Social science researchers have elaborated on the concept of practice for a long time, and there is no unified view of what it means. We here subscribe to Schatzki’s view on practice [
A research environment operates in a context continually influenced by and adapting to external parties, such as academic journals, ethical approval boards, funding agencies, and legislative bodies. An implication of adopting the practice view on the research environment is that a successful research environment needs to adapt to their stakeholders, who define the preconditions for research practice, and evaluate its outcome.
A structure to scrutinize ideals in eHealth research.
Aspect | Explanatory question |
Theoretical discussion and definition | What is the meaning of the ideal and why is it important? |
Stakeholders | Who reinforces the ideal? |
Impact on the U-CARE environment | How, when, and where did the ideal affect the environment? |
Managerial implications | What are the managerial implications for the research environment? |
An approach to make sense of qualitative data is to appropriate the six interrogatives: (1) what, (2) who, (3) why, (4) how, (5) when, and (6) where [
In the remainder of this section, we adopt the approach to scrutinizing ideals to provide an account of each ideal and its impact on the research environment.
Accountability is a core concern in health care. The meaning of information accountability is that “[...] use of information should be transparent so it is possible to determine whether a particular use is appropriate under a given set of rules, and that the system enables individuals and institutions to be held accountable for misuse” [
Accountability builds on human rights and legislation, and differs between nations. Researchers need to account for the way they manage patient information to research funders, ethical approval boards, journals, and government agencies, and not least the citizens.
Accountability needs to be addressed using multiple competencies, including law, health, and information technology expertise. While substantial resources are required to find solutions to accountability issues, it is imperative to systematically reuse knowledge and technology to efficiently set up and execute new projects within the eHealth area.
We adhere to the view of innovation as “...the multi-stage process whereby organizations transform ideas into new/improved products, services or processes, in order to advance, compete, and differentiate themselves successfully in their marketplace” [
Innovation capability may affect research funding, thus this capability is crucial for the survival of the research environment. If research results do not affect health care practice, their value is questionable. Following the innovation ideal, new treatments that prove effective should be implemented into practice.
Innovation is desired by research funders, no matter if funding is commercial, governmental, or comes from nonprofit organizations. Innovation is important both for public health and industrial growth. In addition, in many cases researchers desire that their results be implemented due to commercial or “altruistic” reasons.
A research environment aiming at innovation needs a strategy to handle IP rights, preferably from the inception of a project. Innovation requires collaboration with, for example, software companies, legal experts, and health care providers. The research environment needs to collaborate with experts in related areas such as implementation science, service management, and business administration to develop business models supporting the translation of results into practice.
Researchers should be able to explain the societal relevance of their work. Applied research is expected to contribute to society, and research legitimacy is demonstrated in terms of practical relevance [
Stakeholders related to the relevance ideal are similar to those related to innovation. To promote relevance, there is a need to center the design process on health care practice, and include those who will receive the interventions, for example, the patients, the significant others, and those who will provide them, for example, the health care specialists. Relevance builds on an understanding of societal needs. Such needs are periodically investigated and reported by government agencies and the European Union, important stakeholders with regard to relevance.
Research should proactively adopt a stakeholder-centric design process, including a broad range of stakeholders. The experiences from U-CARE support an iterative approach to development research, interventions, and software. In addition, relevance highlights the need for a continuous monitoring of knowledge gaps and improvement opportunities in the health sector. The research environment needs to engage in intelligence work to understand societal needs in order to maintain and improve the relevance of research.
Rigor concerns the effective use of knowledge, including both the theoretical foundations and the research methodology throughout the research process [
In research on online interventions, it has been argued that trials should lead to an increased understanding of the processes and mechanisms that make treatment effective [
Rigor is important for researchers who aim for high impact publications. It is equally important for journals to maintain and improve their credibility in the academic community. Arguably, publication in high impact outlets strengthens the research environment, and its capability to have an impact on health care practice. Thus, rigor plays an important role as a foundation for research and its meaning for health care practice.
Senior researchers play an important role in promoting rigor in the research environment. Knowledge management strategies, including formal routines and informal discussions, need to be applied to support all coworkers to continually reflect on the three aspects of rigor outlined above. The elaborate reuse of software and interventions enhances rigor by providing new projects with well tested practices.
Something is sustainable if it ‘‘...meets the needs of the present without compromising the ability of future generations to meet their own needs” [
The European Union Horizon 2020 program, along with some other funding agencies, reinforces sustainability ideals. Social sustainability—promoting equal health care for all—concerns all citizens. However, commercial and public organizations are affected by the trend toward openness, and it’s meaning for organizing and making profits on a market.
Sustainability adds complexity to organizing research, since it enhances the need for competences in social and ethical issues, IP rights, and the design of technology that benefits society outside the scope of the ongoing trials.
During 2010-2014, the following five ideals repeatedly occurred while planning and executing research. These ideals constitute a comprehensive rather than complete list: (1) accountability, (2) innovation, (3) relevance, (4) rigor, and (5) sustainability. The challenges regarding each issue of research have been reviewed for their impact on the U-CARE environment.
Accountability was a challenging aspect in the setup of the U-CARE environment and had an impact on how work was organized, and how interventions and software were designed. As psychological treatment is provided within the original trials, Uppsala University (hosting the U-CARE program) registered as a caregiver. A health care organization was set up within the research environment. These organizational implications of accountability were not anticipated initially. The legal and ethical aspects of the management of participant data were continuous concerns radically affecting the software design. Some examples include the way participant data is logged and accessed, the use of double authentication for participants, role-based privileges to access information, and organizational and technical solutions to protect the information. Electronic health record legislation adds further complexity, and health records are at this point managed manually. While accountability issues need to be addressed in the design process, they also need to be aligned with research goals, development, and evaluation of interventions etc.
In order to promote innovation, the U-CARE environment strives toward open sourcing of the software and the interventions. Other research groups may utilize software and interventions developed within U-CARE. Interventions are released under a Creative Commons license, allowing anyone to use them for noncommercial purposes. Research groups associated with U-CARE contribute with new interventions for future reuse. The licensing of software is not yet determined, but the intention is to make it open source. Swedish legislation provides researchers with the IP rights of their innovations and results. Licensing thus becomes subject to a negotiation between contributing researchers. The challenge is to agree about licensing in a way that supports an effective implementation of the software and the interventions, while at the same time resonating with the interests of researchers who are legible IP rights holders. The issue of “effective implementation” cannot be solved without an understanding of stakeholders’, such as IT companies and caregiving organizations having incentives to use and further develop research deliverables. At this point, the U-CARE research environment operates as a service provider. The software is hosted at Uppsala University, providing associated researchers with the opportunity to conduct their studies via the eHealth software. It is a temporary solution, since the university should focus on research, rather than service provisions like IT hosting, software development, and IT support. The university hosts a unit to support innovation, which is periodically consulted by the researchers.
The relevance of the research to the patients is important. Initially, the work was centered on developing interventions and flexible software. In the first feasibility study, directed to adolescents with cancer, the software and the intervention, for example, a self-help program consisting of cognitive behavioral therapy, information, and interactive support, was not received well. Recruitment was difficult and retention was low. In order to improve recruitment, participation, and retention, a group of adolescents with lived experience of cancer was involved in research activities. Patient representatives were also involved in the development of self-help programs for adults with cancer, and adults having had a myocardial infarct. The group of adolescents with lived experience of cancer provided important feedback on various issues, for example, the content of the self-help program, the software user interface, the inclusion criteria, and inclusion procedures. Ideally patient representatives should have been involved in all U-CARE activities from the very start. In the multi-disciplinary setting, there is also exploratory research. As an example, information systems researchers identified relevant research questions during the design process, such as management and design issues concerning privacy and accountability in eHealth.
The U-CARE environment emphasized methodological rigor from the very start. Theoretical foundations have been discussed extensively, but an emphasis has been placed on methodological rigor. However, in response to internal discussions and input from the scientific advisory board, the environment has increasingly paid attention to theoretical foundations and potential theoretical contributions. There are several arenas within U-CARE where methodology and theory is discussed; for example, it is discussed at research seminars, and study coordination group meetings, with the purpose of improving rigor. Rigor is also tightly connected to software design. Well recognized and extensively used instruments are available for reuse in the software, as well as features to improve adherence, for example, rule-based email reminders to participants. In addition, the software has been equipped with extensive, theory-based, logging functionality to improve post hoc scrutiny of patient behaviors. It is believed that such logging is important for rigorous development of theory and for accountability purposes. In essence, rigor is implemented in the organizational routines and in the software to support research.
The aspect of “openness” in U-CARE has been illustrated in the presentation of the innovation ideal. Open sourcing, however, is also a matter of sustainability. In relation to the digital divide, it is clear that eHealth interventions target only part of the population. In the Swedish context, interventions such as those developed and evaluated within U-CARE would make psychological treatment and psychosocial care available for larger groups than they are available for today. First, the online mode of treatment expands the geographical reach of support. Second, at least in the Swedish context, the U-CARE online treatment protocols facilitate support to a group of patients not regularly offered any support today. However, while overall access to psychosocial support and psychological interventions in society is improved, the new form of support may add to the digital divide, due to the dependency on technical equipment and proficiency in using IT. Even though sustainability has affected the work so far, we believe that the research environment would benefit from a more systematic approach to address economic and social sustainability.
Some research has emphasized the operational aspects of online intervention research, for example, the guidelines for conducting online trials [
We suggest that a research environment benefits from systematically scrutinizing ideals that govern research, and their origin in terms of stakeholders. We have identified five ideals and addressed them in a novel way by discussing their implications for managing a research environment. A specific ideal, accountability, originates from the ethics of privacy, and the need to hold people accountable of their actions in case of information misuse or maltreatment. Another two of the discussed ideals, relevance and rigor, are established scientific ideals. Health research is concerned with ethics and the long-term effects of new policies and practices. These concerns are manifested in the ideals sustainability and innovation. Depending on the research context, the ideals may be more or less important to factor into the research design. Given our approach to identify ideals based on a single research environment, it should also be emphasized that a different research context might be influenced by other ideals than those included here. Research management needs to incorporate mechanisms into the environment to continually address ideals, for example, integrating discussions about the proposed ideals into research planning, execution, and reporting.
The result emphasizes the need for different skills, which fuels the argument that an eHealth research environment benefits from a multi-disciplinary collaboration. We have pointed out skills that are of importance to meet each ideal noted in the paper, and the challenges associated with each of these ideals. The need for multiple skills—as well as reuse of knowledge and technology—underscores the complexity of eHealth research. We argue that strong, sustainable, multi-disciplinary research environments are required in order to conduct eHealth research that appropriately addresses the complexity that follows from the five ideals. We have given practical examples of the implications of these ideals by giving examples of the impact of the five identified ideals in the U-CARE research environment.
Finally, ideals are sometimes conflicting. Within the U-CARE program, there is an ongoing discussion about whether open sourcing is an effective way to disseminate results into practice. A commercialization of software and interventions developed within U-CARE might be considered less sustainable, even though commercialization may promote innovation. In addition, research tends to require a trade-off between relevance and rigor. Our goal here is not to provide prescriptions for such decisions; we merely suggest an approach to systematically scrutinize ideals. Such scrutiny informs decisions, and contributes to well reflected multi-disciplinary eHealth research.
intellectual property
information technology
randomized controlled trial
Uppsala University Psychosocial Care Programme
None declared.