Published on 02.04.14 in Vol 3, No 2 (2014): Apr-Jun
A Web-Based Clinical Decision Support Tool for Primary Health Care Management of Back Pain: Development and Mixed Methods Evaluation
Background: Many patients with back pain do not receive health care in accordance with best practice recommendations. Implementation trials to address this issue have had limited success. Despite the known effectiveness of clinical decision support systems (CDSS), none of these are available for back pain management.
Objective: The objective of our study was to develop a Web-based CDSS to support Australian general practitioners (GPs) to diagnose and manage back pain according to guidelines.
Methods: Asking a panel of international experts to review recommendations for sixteen clinical vignettes validated the tool. It was then launched nationally as part of National Pain Week and promoted to GPs via a media release and clinic based visits. Following this, a mixed methods evaluation was conducted to determine tool feasibility, acceptability, and utility. The 12 month usage data were analyzed, and in-depth, semistructured interviews with 20 GPs were conducted to identify barriers and enablers to uptake.
Results: The tool had acceptable face validity when reviewed by experts. Over a 12 month period there were 7125 website visits with 4503 (63.20%) unique users. Assuming most unique users are GPs, around one quarter of the country’s GPs may have used the tool at least once. Although usage was high, GP interviews highlighted the sometimes complex nature of management where the tool may not influence care. Conversely, several “touch-points”, whereby the tool may exert its influence, were identified, most notably patient engagement.
Conclusions: A novel CDSS tool has the potential to assist with evidence-based management of back pain. A clinical trial is required to determine its impact on practitioner and patient outcomes.
JMIR Res Protoc 2014;3(2):e17
Global Disease Burden From Back Pain
Back pain is the number one ranked disabling health condition worldwide . Across 21 regions, the lowest ranking of burden was 4th of 291 diseases in both the Caribbean and Southern sub-Saharan Africa. This makes a compelling case for increased efforts to improve health care for this condition.
Back pain is best conceptualized as a chronic disease. Although symptoms usually improve quickly after onset, pain and disability frequently persist at low to moderate levels . We found that nearly one third of the people with reported acute back pain still experienced pain one year later [ ]. Around one quarter of those who recovered had a recurrence within one year [ ], and the majority of those with a prolonged initial course of pain (> 3 months) reported persisting pain 12 months later [ ]. Similar findings were noted in a UK study [ ]. Thus, back pain is commonly a long term health condition with an unpredictable pattern of acute episodes, remission, and recurrence [ , ].
Evidence-Practice Gaps in Management of Back Pain
Despite high health care expenditure for back pain, clear improvements in outcomes do not necessarily follow. In the United States, back pain expenditure increased by 65% (inflation adjusted) in the period 1997-2005, yet the health status for people with back pain fell . Further, 16.7% of visits for back pain include inappropriate imaging tests, resulting in over $175 million per year of unnecessary expenditure [ ]; the principal driver for this is that people with back pain are not receiving the care endorsed in guidelines. Our large survey of Australian primary health care providers revealed that few patients received the care recommended in the nationally endorsed guideline [ ]. Less than 10% of the patients received appropriate analgesia, and just 20% were provided with appropriate advice. Additionally, over one quarter of the patients were referred for diagnostic imaging. Crucially, this pattern of care was essentially the same in the periods before and after the guideline was released in 2004 [ ]. Similar evidence-practice gaps have been reported in other countries [ - ].
Health Care Intervention Studies
Several health care intervention studies to improve back pain management have resulted in minimal or no improvements in mainly process outcome measures [- ]. Clinical decision support systems (CDSS) are one of the most promising interventions to improve uptake of guideline recommendations. In two recent systematic reviews, around two thirds of CDSS trials demonstrated improvements [ , ]. In the broader eHealth arena, there is now growing evidence that interactive Internet and mobile interventions improve outcomes from chronic conditions. A Cochrane review of 124 studies concluded that computer based “Interactive Health Communication Applications” could improve cognitive and social support outcomes in patients with chronic conditions [ ]. Despite this, we are not aware of any CDSS that have been trialed for back pain management.
In this paper, we outline the development of a Web-based CDSS for back pain management. We also discuss the development and validation of an algorithm, in addition to a feasibility study examining the acceptability and utility of the tool following national deployment for use by Australian general practitioners (GP). Our objective was to determine whether the tool had the potential to support GPs to diagnose and manage back pain in accordance with guideline recommendations, and to identify barriers and enablers to uptake. The study was designed to inform a future large scale trial evaluation.
First, our recent systematic search of relevant guidelines was drawn on to inform development of the decision support algorithm . In this review, we searched for guidelines in MEDLINE, PEDro, the US Agency for Health Care Research and Quality National Guideline Clearinghouse, and the UK National Institute for Health and Clinical Excellence, and used backward and forward citation tracking, retrieving 15 eligible guidelines. A key finding from the review was that there was a high level of consistency in recommendations from all 15 guidelines. These recommendations were converted to plain language rules, hardcoded into an algorithm, and a prototype tool was developed. illustrates the key variables in the algorithm and treatment recommendations provided.
Second, the algorithm was validated as follows. There were 16 hypothetical cases representing all permutations of the algorithm that were entered into the tool to generate management recommendations. The output was compared to an expert's clinical decisions for each case based on their interpretation of the guidelines. Additionally, a panel of four experts who developed the UK, US, and Australian guidelines [- ] reviewed the prototype and commented on the recommendations. Any misinterpretations were reviewed and inconsistencies resolved by a consensus process.
Third, a user interface was designed, resulting in a four step process of excluding serious pathology, clinical assessment, establishing treatment options, and building a personalized, patient information sheet (- show this interface). Technical user acceptance testing was conducted to ensure the tool functioned as specified.
The tool was published on the Internet in June 2012, accompanied by a media release issued to coincide with Australian National Pain Week . The tool was subsequently mentioned in several leading GP focused periodicals, the Royal Australian College of General Practitioners electronic newsletter, sent to over 20,000 GPs, the Australian Medicare Local Alliance newsletter, and circulated to all 61 Medicare Locals, which are government funded, middle tier primary health care organizations. The tool was also directly promoted via a NPS MedicineWise “visiting topic”, in which physiotherapist educators conducted small clinic based workshops to support GPs in improved health care quality.
Following this, a convenience sample of twenty GPs in the Sydney area were invited to use the tool over a four week period in order to provide further insights into its utility, barriers, and enablers to uptake.
A mixed methods evaluation was conducted of the tool. Usage data were collected using Google Analytics over a 12 month period. Data included visit frequency, number of unique visitors, number of page views, and average visit duration. The qualitative component involved in-depth, semistructured interviews with all 20 GPs who had trialed the tool. A GP interviewer who was not involved in tool development conducted the semistructured interviews (seefor the interview guide). The interviews covered four key domains: (1) opinions on back pain management, (2) a “live walk-through” of the website, (3) clinical and patient perspectives on utility, and (4) recommendations for improvements. The interviews were digitally recorded, transcribed, and imported for analysis into NVivo 10 (QSR International). An inductive approach was taken, with thematic content analysis conducted contemporaneously with data collection. A preliminary coding framework was collaboratively derived from the analysis of four interviews, discussed by the broader project team, and revised accordingly. There were two researchers that then independently coded the remaining interviews (eight each), and discussed new themes not previously identified. Saturation of themes occurred after around 15 interviews. At the end of the interview coding, final findings were again discussed by the project team and key themes were further refined.
The University of Sydney Human Research Ethics Committee approved the study.
Website Usage Data
shows the usage data from June 2012 to May 2013 of the tool. The peak in early August 2012 coincides with the media release; subsequent to this usage, patterns remained relatively constant. The average time spent per visit was just over four minutes and the average number of pages viewed per visit was six pages. Almost all visitors clicked through to at least the second page of the back pain assessment.
outlines the characteristics of the GPs interviewed that had trialed use of the tool.
|English primary language spoken at home||14 (70)|
|Australian university graduate||15 (75)|
|Vocationally registered||20 (100)|
|Number of sessions worked per week|
|Participates in research often or very often||6 (30)|
|Conducts own research||8 (40)|
|Daily use of Internet or email for work purposes||20 (100)|
|Number of patients registered|
|Number of doctors|
|Number of nurses|
|Practices with a practice manager||17 (85)|
|GP registrar placement||4 (20)|
aOne response missing
There were three interrelated themes on the management of back pain and the role of tools that were identified. An additional theme, focused on the functional aspects of the tool, also yielded important information on recommendations for improvement.shows the key thematic domains and related subthemes.
Theme 1: Low Back Pain Is a Common Condition
Our motivation for developing the CDSS tool was to improve adherence to guideline recommendations. Many of the GPs interviewed, however, considered back pain diagnosis and management to be common and relatively straightforward. It was viewed as integral to being a competent practitioner, and, therefore, the notion that GPs might need a tool to assist them seemed to indicate that “something was wrong” with their skills.
You can’t have a tool for every presentation in general practice...this is something that should be up there (pointing to his head) rather than “Oh wait a second, let me use this tool...”
Similarly, one GP was worried that the use of such a tool during a consultation would make him look “a bit strange” in the eyes of his patient, while another felt it was condescending and that he did not need “a step-by-step approach on how to suck an egg.” These accounts highlight the tension between a perceived, sound knowledge of best practice and the published data demonstrating large evidence-practice gaps.
Theme 2: Low Back Pain Is a Complex, Chronic Condition
Juxtaposed with the notion that back pain is common, was a more nuanced appreciation that it may have a complex, chronic course. This course can be characterized by relapse and setbacks, psychosocial and work related repercussions, and fragmented health care experiences. Most GPs acknowledged that under these circumstances back pain management was challenging and required considerable effort to ensure satisfactory health care. These challenges are particularly pronounced with work related back pain, where health care costs are subsidized in Australia by the government worker’s compensation schemes.
For one patient injured at work...he seems to be using it to get back at his employer...It also looks as though his relationship (with his partner) is going to break down...and he fosters a sense of entitlement through his back pain...
In such circumstances, GPs may be more likely to conduct imaging tests or prescribe complex pain relief medications, while cognizant that these are not routinely indicated.
I tried to get (him) managing the pain, but he was also a very anxious man. He went to my colleague, had a CT scan, which then showed something he fixated on. It’s been years to get him to a point where he’s not obsessing about every twinge in his back.
Theme 3: Tools to Engage Patients
We came to appreciate that the tool was situated amid highly varied clinical contexts, ranging from where GPs required minimal additional support, through to complex situations where new resources were desirable. This helps to explain why at times the tool was seen as superfluous, and at other times it had the potential to assist with management dilemmas.
The tool was perceived to be the most useful in situations where patient reassurance and avoidance of complex tests or medicines were recommended. The GPs used the tool as a source of authoritative advice to support their recommendations. The patient information sheet was a particularly useful component, as it synthesized the key messages that would otherwise take some time to explain.
Well, I suppose we tended to make the patients feel happy if they want x-rays...Now at least we can say, “This is the evidence. You’ve got to wait for a while...”
Similarly, in the context of a work related injury, one GP used the tool to communicate with the patient’s employer.
I used the tool for the employer because the patient refused to go back to work...So I copied a few things from the tool and emailed it to the employer and they were quite happy.
Theme 4: Recommendations to Improve Tool Content
While the tool had the potential to be of value, some GPs found that it was not sufficiently dynamic to be of use in the long term. Once the core components became clear, there was perceived to be little variation in the content. This was not necessarily an adverse finding, as many GPs felt that over time the recommendations would be “incorporated as part of their normal skills.” Some GPs, however, wanted more detailed advice for complex pain management and indications for referral, whereas others wanted more autonomy in customizing the patient information.
The main influence to future tool uptake appeared to be integration into routine workflow. Integration with clinical software systems and the ability to rapidly navigate to the parts of immediate interest were frequently recommended by the GPs. They were also concerned with the growing number of tools that compete for their attention. Many described a process by which they establish their own “tool box” of resources, often compiled over many years. Tool developers therefore need to make new tools that complement rather than compete with existing tools.
This study examined the development, uptake, and acceptability of an Australian GP Web-based CDSS for the management of back pain. There were three key findings that were observed. First, the algorithm underpinning the tool appeared to have acceptable face validity when reviewed by experts in the field and when implemented among GPs. This reflects consistency in guideline recommendations over several years. Second, we identified an effective implementation mechanism for rapid dissemination of the tool, drawing on a multi-pronged approach of media releases in target periodicals, alignment with National Pain Week, and promotion via an existing national network of facilitators who conduct face-to-face GP visits. Assuming that most unique visits were GPs, and that there are 20,360 GPs in Australia , then, over a 12 month period, around one quarter of the GPs in the country used the tool. Moreover, the ratio of new to returning users has remained constant, indicating the tool has not yet reached full market saturation. Third, and perhaps most importantly, the qualitative component revealed the complexities of managing low back pain and the “touch-points”, whereby a CDSS tool may exert its influence.
This study was exploratory in nature and did not assess the clinical effectiveness of the tool. The focus was to look at usability factors, identify enhancements needed, and further refine the tool. Further, although the usage data are useful, we were unable to analyze use by provider type, and to examine in more detail which parts of the tool were most/least popular. A final limitation was that we did not examine the perspectives of patients and other health care providers, such as physiotherapists and specialists.
Comparison With Prior Work
To our knowledge, this is the first CDSS for back pain to be evaluated. Although uptake and general satisfaction was encouraging, ongoing usage patterns are likely to be highly variable. Our findings resonate with our recent study of a cardiovascular CDSS tool and outline that GPs adopt guideline recommendations judiciously, depending on multiple social and environmental factors [, ]. This may partly explain why interventions, including Internet interventions to improve back pain management, have thus far yielded small effects at best [ , ]. The notion that GPs establish their own “tool box” to conduct their work is useful in understanding how a new tool will be perceived. For common conditions like back pain, this compendium of existing tools tends to be well established, and, consequently, a tool that covers only the basics is likely to be less effective. Conversely, GPs clearly experience challenges in effectively managing back pain for their patients. If a CDSS tool can support GPs in these domains, especially complex pain management, psychosocial impact, and work related issues, then such a tool is likely to be concordant with their needs. Tool customization was of particular importance to the GPs. In a model akin to a mobile “app store”, GPs could select and modify their preferred “apps” on the basis of personal interest, user feedback, or endorsement by trusted organizations.
Despite the existence of clear and consistent recommendations from national and international guidelines for the management of back pain, large evidence-practice gaps exist. Our findings suggest that decision support tools have the potential to serve as an effective dissemination mechanism for the implementation of guidelines. This study represents the initial stage of a research translation program to improve health care quality for back pain. Several tool features were identified to be useful, but, importantly, a number of contextual factors affecting how back pain is managed were also noted. These factors may require additional tools with a different focus. Further work with care providers and consumers is planned to develop and test these enhancements. The system will then be assessed for its effectiveness in improving health care for the world’s most disabling health condition.
The George Institute for Global Health and NPS MedicineWise internally funded the study. We gratefully acknowledge the support of the GPs that participated in this study. We also thank the following NPS MedicineWise staff for their support with the project - Mallory Allsopp, Chris Hayward, Aine Heaney, Paul Heasley, Damon Judges, Ben Smyth, Andrew Sung-Hoon Jun, and Clare Weston. A National Health and Medical Research Council Fellowship (ID 1054754) supports DP. An Australian Research Council Fellowship supports CM.
DP, CM, and CW designed the study. RH, RL, and JR led the implementation of the intervention and were responsible for the acquisition of quantitative data. AD was responsible for the acquisition of qualitative data. DP wrote the first draft of the study manuscript. All authors critically reviewed the manuscript and reviewed the final draft prior to submission.
Conflicts of Interest
Multimedia Appendix 1
Interview guide.PDF File (Adobe PDF File), 109KB
- Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990-2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012 Dec 15;380(9859):2163-2196. [CrossRef] [Medline]
- da C Menezes Costa L, Maher CG, Hancock MJ, McAuley JH, Herbert RD, Costa LO. The prognosis of acute and persistent low-back pain: A meta-analysis. CMAJ 2012 Aug 7;184(11):E613-E624 [FREE Full text] [CrossRef] [Medline]
- Henschke N, Maher CG, Refshauge KM, Herbert RD, Cumming RG, Bleasel J, et al. Prognosis in patients with recent onset low back pain in Australian primary care: Inception cohort study. BMJ 2008;337:a171 [FREE Full text] [Medline]
- Stanton TR, Henschke N, Maher CG, Refshauge KM, Latimer J, McAuley JH. After an episode of acute low back pain, recurrence is unpredictable and not as common as previously thought. Spine (Phila Pa 1976) 2008 Dec 15;33(26):2923-2928. [CrossRef] [Medline]
- Costa Lda C, Maher CG, McAuley JH, Hancock MJ, Herbert RD, Refshauge KM, et al. Prognosis for patients with chronic low back pain: Inception cohort study. BMJ 2009;339:b3829 [FREE Full text] [Medline]
- Dunn KM, Jordan K, Croft PR. Characterizing the course of low back pain: A latent class analysis. Am J Epidemiol 2006 Apr 15;163(8):754-761 [FREE Full text] [CrossRef] [Medline]
- Foster NE. Barriers and progress in the treatment of low back pain. BMC Med 2011;9:108 [FREE Full text] [CrossRef] [Medline]
- Hay EM, Dunn KM. Prognosis of low back pain in primary care. BMJ 2009;339:b3694. [Medline]
- Martin BI, Deyo RA, Mirza SK, Turner JA, Comstock BA, Hollingworth W, et al. Expenditures and health status among adults with back and neck problems. JAMA 2008 Feb 13;299(6):656-664. [CrossRef] [Medline]
- Kale MS, Bishop TF, Federman AD, Keyhani S. "Top 5" lists top $5 billion. Arch Intern Med 2011 Nov 14;171(20):1856-1858. [CrossRef] [Medline]
- Williams CM, Maher CG, Hancock MJ, McAuley JH, McLachlan AJ, Britt H, et al. Low back pain and best practice care: A survey of general practice physicians. Arch Intern Med 2010 Feb 8;170(3):271-277. [CrossRef] [Medline]
- Jackson JL, Browning R. Impact of national low back pain guidelines on clinical practice. South Med J 2005 Feb;98(2):139-143. [Medline]
- Schers H, Braspenning J, Drijver R, Wensing M, Grol R. Low back pain in general practice: Reported management and reasons for not adhering to the guidelines in The Netherlands. Br J Gen Pract 2000 Aug;50(457):640-644 [FREE Full text] [Medline]
- Tacci JA, Webster BS, Hashemi L, Christiani DC. Clinical practices in the management of new-onset, uncomplicated, low back workers' compensation disability claims. J Occup Environ Med 1999 May;41(5):397-404. [Medline]
- Becker A, Leonhardt C, Kochen MM, Keller S, Wegscheider K, Baum E, et al. Effects of two guideline implementation strategies on patient outcomes in primary care: A cluster randomized controlled trial. Spine (Phila Pa 1976) 2008 Mar 1;33(5):473-480. [CrossRef] [Medline]
- Bishop PB, Wing PC. Knowledge transfer in family physicians managing patients with acute low back pain: A prospective randomized control trial. Spine J 2006;6(3):282-288. [CrossRef] [Medline]
- Dey P, Simpson CW, Collins SI, Hodgson G, Dowrick CF, Simison AJ, et al. Implementation of RCGP guidelines for acute low back pain: A cluster randomised controlled trial. Br J Gen Pract 2004 Jan;54(498):33-37 [FREE Full text] [Medline]
- Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: A randomised trial. Lancet 2001 May 5;357(9266):1406-1409. [CrossRef] [Medline]
- Engers AJ, Wensing M, van Tulder MW, Timmermans A, Oostendorp RA, Koes BW, et al. Implementation of the Dutch low back pain guideline for general practitioners: A cluster randomized controlled trial. Spine (Phila Pa 1976) 2005 Mar 15;30(6):559-600. [Medline]
- Schectman JM, Schroth WS, Verme D, Voss JD. Randomized controlled trial of education and feedback for implementation of guidelines for acute low back pain. J Gen Intern Med 2003 Oct;18(10):773-780 [FREE Full text] [Medline]
- Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. BMJ 2005 Apr 2;330(7494):765 [FREE Full text] [CrossRef] [Medline]
- Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev 2009(3):CD001096. [CrossRef] [Medline]
- Murray E, Burns J, See TS, Lai R, Nazareth I. Interactive health communication applications for people with chronic disease. Cochrane Database Syst Rev 2005(4):CD004274. [CrossRef] [Medline]
- Koes BW, van Tulder M, Lin CW, Macedo LG, McAuley J, Maher C. An updated overview of clinical guidelines for the management of non-specific low back pain in primary care. Eur Spine J 2010 Dec;19(12):2075-2094 [FREE Full text] [CrossRef] [Medline]
- Chou R, Qaseem A, Snow V, Casey D, Cross JT, Shekelle P, Clinical Efficacy Assessment Subcommittee of the American College of Physicians, American College of Physicians, American Pain Society Low Back Pain Guidelines Panel. Diagnosis and treatment of low back pain: A joint clinical practice guideline from the American College of Physicians and the American Pain Society. Ann Intern Med 2007 Oct 2;147(7):478-491. [Medline]
- Australian Acute Musculoskeletal Pain Guidelines Group. Canberra: National Health and Medical Research Council. 2003 Evidence-based management of acute musculoskeletal pain URL: http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp94_evidence_based_management_acute_musculoskeletal_pain_131223.pdf [accessed 2014-02-11] [WebCite Cache]
- National Institute for Health and Care Excellence. Low back pain: Early management of persistent non-specific low back pain. 2009 URL: http://www.nice.org.uk/nicemedia/live/11887/44343/44343.pdf [accessed 2014-02-11] [WebCite Cache]
- NPS MedicineWise. NPS MedicineWise. 2012. Back pain choices URL: http://www.nps.org.au/health-professionals/resources-and-tools/decision-and-management-tools/back-pain-choices [accessed 2013-10-30] [WebCite Cache]
- Britt H, Miller GC, Henderson J, Charles J, Valenti L, Harrison C, et al. The University of Sydney. 2011. General practice activity in Australia 2011-2012 URL: http://ses.library.usyd.edu.au//bitstream/2123/8675/4/9781743320198_ONLINE.pdf [accessed 2013-10-31] [WebCite Cache]
- Peiris D, Usherwood T, Weeramanthri T, Cass A, Patel A. New tools for an old trade: A socio-technical appraisal of how electronic decision support is used by primary care practitioners. Sociol Health Illn 2011 Nov;33(7):1002-1018. [CrossRef] [Medline]
- Peiris DP, Joshi R, Webster RJ, Groenestein P, Usherwood TP, Heeley E, et al. An electronic clinical decision support tool to assist primary care providers in cardiovascular disease risk management: Development and mixed methods evaluation. J Med Internet Res 2009;11(4):e51 [FREE Full text] [CrossRef] [Medline]
- French SD, McKenzie JE, O'Connor DA, Grimshaw JM, Mortimer D, Francis JJ, et al. Evaluation of a theory-informed implementation intervention for the management of acute low back pain in general medical practice: The IMPLEMENT cluster randomised trial. PLoS One 2013;8(6):e65471 [FREE Full text] [CrossRef] [Medline]
- Krein SL, Kadri R, Hughes M, Kerr EA, Piette JD, Holleman R, et al. Pedometer-based internet-mediated intervention for adults with chronic low back pain: Randomized controlled trial. J Med Internet Res 2013;15(8):e181 [FREE Full text] [CrossRef] [Medline]
|CDSS: clinical decision support system|
|GP: general practitioner|
Edited by G Eysenbach; submitted 04.11.13; peer-reviewed by J Lurie, A Bussieres; comments to author 19.01.14; revised version received 11.02.14; accepted 11.02.14; published 02.04.14
©David Peiris, Christopher Williams, Rachel Holbrook, Robyn Lindner, James Reeve, Anurina Das, Christopher Maher. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 02.04.2014.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.