ResProt JMIR Res Protoc JMIR Research Protocols 1929-0748 JMIR Publications Inc. Toronto, Canada v3i4e78 25532218 10.2196/resprot.4039 Protocol Protocol Development and Validation of a Personalized, Web-Based Decision Aid for Lung Cancer Screening Using Mixed Methods: A Study Protocol Eysenbach Gunther Volk Robert Lau Yan Kwan MPH 1 http://orcid.org/0000-0002-1612-9912 Caverly Tanner J MD, MPH 2 3 http://orcid.org/0000-0002-6560-0547 Cherng Sarah T MPH 1 http://orcid.org/0000-0002-9000-4202 Cao Pianpian BS 1 http://orcid.org/0000-0001-8886-9672 West Mindy BS 1 http://orcid.org/0000-0002-6501-613X Arenberg Douglas MD 4 http://orcid.org/0000-0001-7707-5092 Meza Rafael PhD 1
Department of Epidemiology School of Public Health University of Michigan 1415 Washington Heights SPH-II 5533 Ann Arbor, MI, 48109-2029 United States 1 734 763 1946 1 734 764 5436 rmeza@umich.edu
http://orcid.org/0000-0002-1076-5037 1 Department of Epidemiology School of Public Health University of Michigan Ann Arbor, MI United States 2 Division of General Internal Medicine Department of Internal Medicine University of Michigan Medical School Ann Arbor, MI United States 3 Center for Clinical Management Research Veterans Affairs Ann Arbor Healthcare System Ann Arbor, MI United States 4 Division of Pulmonary & Critical Medicine Department of Internal Medicine University of Michigan Medical School Ann Arbor, MI United States Corresponding Author: Rafael Meza rmeza@umich.edu Oct-Dec 2014 19 12 2014 3 4 e78 14 11 2014 24 11 2014 ©Yan Kwan Lau, Tanner J Caverly, Sarah T Cherng, Pianpian Cao, Mindy West, Douglas Arenberg, Rafael Meza. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 19.12.2014. 2014

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.

 Background

The National Lung Screening Trial demonstrated that low-dose computed tomography (LDCT) screening could be an effective way to reduce lung cancer mortality. Informed decision-making in the context of lung cancer screening requires that potential screening subjects accurately recognize their own lung cancer risk, as well as the harms and benefits associated with screening, while taking into account their personal values and preferences.

 Objective

Our objective is to develop a Web-based decision aid in accordance with the qualifying and certification criteria in the International Patient Decision Aid Standards instrument version 4.0 that will assist patients in making informed decisions with regard to lung cancer screening.

 Methods

In “alpha” testing, a prototype of the decision aid was tested for usability with 10 potential screening participants in focus groups. Feedback was also sought from public health and health risk communication experts external to the study. Following that, improvements to the prototype were made accordingly, and “beta” testing was done in the form of a quasi-experimental design—a before-after study—with a group of 60 participants. Outcomes tested were knowledge, risk perception of lung cancer and lung cancer screening, decisional conflict, and acceptability of the decision aid as determined by means of a self-administered electronic survey. Focus groups of a subsample of survey participants will be conducted to gain further insight into usability issues.

 Results

Alpha testing is completed. Beta testing is currently being carried out. As of 2014 December 7, 60 participants had completed the before-after study. We expect to have results by 2015 January 31. Qualitative data collection and analysis are expected to be completed by 2015 May 31.

 Conclusions

We hypothesize that this Web-based, interactive decision aid containing personalized, graphical, and contextual information on the benefits and harms of LDCT screening will increase knowledge, reduce decisional conflict, and improve concordance between patient preferences and the current US Preventive Services Task Force’s screening guidelines.

 informed decision-making lung cancer screening patient decision aid patient education Introduction

Lung cancer remains the leading cause of cancer death in the United States [1]. The National Lung Screening Trial demonstrated that lung cancer screening with low-dose computed tomography (LDCT) has the potential to significantly reduce lung cancer mortality [2]. On the basis of this and other evidence, the US Preventive Services Task Force (USPSTF) gave a B recommendation for LDCT screening [3,4], the same strength of recommendation associated with mammography screening. However, real-world success in lung cancer screening will be conditional on identifying and screening those at highest risk for lung cancer while discouraging screening in those at low risk. Lung cancer screening presents a challenge, because it is the first population-wide screening modality with eligibility criteria based not only on age but also on a lifestyle behavior (at least 30 pack-years of tobacco use and smoking within the past 15 years). Identifying those at risk and helping them understand the benefits of screening and how to reduce their risk (e.g., tobacco cessation) is paramount for an effective implementation of population-wide lung cancer screening.

Implementing lung cancer screening in an environment where patients do not have the tools or information to understand disease risks and the harm-benefit balance of screening will most likely be counterproductive. In addition to providing information for individuals regarding lung cancer screening that allows them to weigh the potential harms of LDCT in accordance with the benefits, we also recognize that the decision to be screened is preference-sensitive. In light of this, there is a need to assist individuals with making informed decisions regarding lung cancer screening in which their personal values are also taken into account.

The USPSTF defines informed decision-making as “an individual’s overall process of gathering relevant health information from both his or her clinician and from other clinical and nonclinical sources, with or without independent clarification of values” ( [5], p. 59). In particular, a patient decision aid’s functions are (1) to provide facts about an individual’s condition and the options available and their characteristics, (2) to help individuals clarify values and personal preferences, and (3) to assist these individuals to discuss their values and preferences with health professionals [6]. It is in this context that our goal is defined: to create a decision aid that improves the knowledge of LDCT screening, decreases decisional conflict, and improves concordance of screening preferences between the official recommendations and the individual (i.e., to assist individuals with informed decision-making about whether or not to screen). Concordance with official recommendations is important, because this will ensure that the resulting population of screened individuals is consistent with that for which lung cancer screening is deemed effective.

Evidence shows that decision aids can improve decision quality as a result of better knowledge of options and their associated harms and benefits; decrease decisional conflict; and reduce the overuse and increase the underuse of screening options [7-10]. In particular, Jimbo and colleagues [9] focused on cancer screening and outlined recommendations for the evaluation of decision aids, the most important of which was to base the decision aid on a theoretical framework so that relevant outcomes are measured. The development of the decision aid in our study uses the Ottawa Decision Support Framework, a widely used theoretical framework that applies theories from psychology, social psychology, economics, and social support [11].

Whereas numerous decision aids exist for prostate, colon, and breast cancers, there are only a handful of tools that fulfill the functions of a patient decision aid, either partially or fully, with regard to LDCT screening [12-16]. We are aware of only one that has formally evaluated measures of the effect of decision aids established in the theories of behavioral research in a peer-reviewed journal [16]. However, this decision aid [16] is in the format of a video that provides average risks and benefits of LDCT screening. We know that risk varies greatly among smokers [17]; therefore, in our study, we propose to develop an alternative format: a Web-based, interactive decision aid that takes into account a more accurate depiction of an individual’s personal lung cancer risk. The decision aid will comply with the qualifying and certification criteria set out in the International Patient Decision Aid Standards instrument (IPDASi) version 4.0 [18]. The rationale for a Web-based tool is that the Web is increasingly becoming an important source of cancer information [19]. Another advantage of the Web-based format is that the content of decision aids could easily be expanded and customized for different audiences. Studies have also confirmed the feasibility of using the Web to deliver the Web-based decision aids [20].

 Methods Prototype Development

A prototype was developed based on the most recent clinical guidelines provided by the USPSTF [5], the IPDASi version 4.0 checklist [18], and recommendations in terms of risk communication [21-23]. The decision aid includes information on personalized lung cancer incidence risk calculated by using an established risk model [24], risk factors of lung cancer reported in the literature, harms and benefits of LDCT screening, and an explicit values clarification exercise. We also used a currently existent print-based decision aid developed by the Veterans Affairs Healthcare System as a reference [13,14].

The distinguishing factor of the Web-based version is that it allows individuals to compute their individualized lung cancer risk using an established risk model. Although all cancer screening is based on some risk factor identification, for the most common evidence-based cancer screening (e.g., colorectal, breast, and cervical cancer screening), risk factor identification involves little more than noting the relevant age and sex data. Lung cancer screening differs in very important ways in that proper application of current recommendations involves measuring risk by also identifying the individual’s cumulative tobacco exposure measured in pack-years and the timing of tobacco use. Whereas smoking history accurately identifies risk on a population basis, it is less useful for individuals because there is such great variability of lung cancer risk, even among smokers with similar exposure. Current models that more accurately quantify individual lung cancer risk incorporate up to 10 clinical and demographic variables that provide a more accurate, though complex, determination of risk. This requires sophisticated lung cancer risk prediction models [24].

In addition to accurately characterizing individual lung cancer risk, there is evidence that individuals may prefer to have such tailored information, which, in turn, may affect health-care-seeking decisions [10,25]. To this end, personalized risk in absolute terms (as a percentage) and individualized benefits in the form of icon arrays are generated by the decision aid. We also provide average harms as an estimation of individual harms (see Multimedia Appendix 1 for the current iteration of the results page).

We believe that putting LDCT screening in context with other common screening recommendations in terms of reduction of disease-specific mortality (i.e., reduction of breast cancer death by screening with mammography) will allow the individual to gauge how LDCT screening compares to other widely accepted screening practices. This has not been done with the available decision aids for LDCT lung cancer screening [13,14,16].

 Participants

Our target population comprised of potential users of LDCT screening. The specific inclusion criteria are given in Textbox 1.

These criteria applied to all 3 phases of the study detailed below: Phase I alpha testing focus groups, Phase IIa beta testing before-after survey, and Phase IIb beta testing postsurvey focus groups.

Inclusion criteria.

Current and former smokers

Aged 45-80 years

No history of lung cancer

No previous chest computed tomographic scan in the past year

 Recruitment Procedure

A combination of passive and active recruitment was done to form 2 convenience samples of participants: 10 for alpha testing and 60 for beta testing. An advertisement was placed on the University of Michigan’s (UM’s) online portal for volunteers of clinical studies [26], along with flyers at all of the clinics in the University of Michigan Health System (UMHS) and district libraries in Ann Arbor. Active recruitment involved in-person recruitment at a pulmonary clinic and general internal medicine clinic in the UMHS. Also, a list of participants and their contact details was generated by the Honest Broker Office of UM Medical School from medical records who fulfilled the inclusion criteria as listed above. The project coordinator verified the eligibility of all potential participants over the phone.

 Phase 1: Alpha Testing Overview

We solicited feedback from public health and health risk communication experts with regard to the content and wording and how risk is expressed in the prototype. After incorporating their suggestions, we conducted focus groups with potential users of LDCT screening to test the usability of the prototype as part of the decision tool’s iterative development process. Specifically, we pretested our tool for comprehension of the content, as well as the acceptability of the design, layout, and messages conveyed. A total of 10 people participated in 1 of 2 focus groups. The eligible and willing participants were sent a Web link to work through the Web-based decision aid prototype via e-mail. All participants were asked to participate in the focus group within the week they reviewed the decision aid. Version 1 was the product of alpha testing.

 Data Collection

The focus group was conducted by a trained external facilitator and a study team member using an interview guide developed by study team members with expertise in qualitative research and lung cancer screening decision-making (see Multimedia Appendix 2). Focus groups were held at a venue at the UM School of Public Health. At the focus group, individuals were asked to complete a survey to document the demographic makeup of the focus group participants (see Multimedia Appendix 3). Thereafter, all participants accessed the decision aid again on an iPad individually to refresh their impression. After all participants in the focus group were done reviewing the tool, the external facilitator asked questions pertaining to the usability of the decision aid. All sessions were audiotaped, and field notes were taken. Ten minutes before the scheduled end of the focus group, the participants were asked to fill out an exit survey with 4 questions (see Multimedia Appendix 4).

 Analysis

Audio recordings were transcribed verbatim. Data analysis took place simultaneously with data collection, which, in turn, assisted in the iterative development of both the interview guide and decision aid. Using the transcriptions and field notes, a brief report was given by the study team member present at the focus groups to the rest of the study team. The study team then decided what changes to incorporate into the decision aid, forming version 1.

 Phase IIa: Beta Testing: Self-Administered Electronic Survey Overview

Following alpha testing, we conducted a pilot study of version 1 of our decision aid with 60 individuals using a quasi-experimental design: a before-after study. This study design is consistent with the development of a decision aid and is an accepted method to test the effectiveness of decision aid tools under “real-life” conditions [16,27,28]. The before-after study will be followed by focus groups to gain further insight into the outcomes of interest and other spontaneous feedback about the tool in general.

 Data Collection

Following a successful screen for eligibility, a participant was invited to come to UM to complete a series of surveys administered on a computer by Qualtrics, an online survey tool. In particular, a participant began with the knowledge survey (see Multimedia Appendix 5). This was followed by the pretest demographic, lung cancer risk, and prior screening experiences survey (see Multimedia Appendix 6); risk perception of lung cancer and lung cancer screening (see Multimedia Appendix 7); and the decision conflict scale survey (see Multimedia Appendix 8). This formed the “before” survey, and the participant was automatically redirected to the website upon clicking “submit.” The participant was instructed to explore the website for 10-15 minutes and on the final page, he or she would click on a link at the bottom that would redirect them to the “after” survey. The “after” survey comprised the knowledge, risk perception of lung cancer, and lung cancer screening; the values clarification questionnaire (see Multimedia Appendix 9); the decision conflict scale; and the acceptability survey (see Multimedia Appendix 10). The whole process lasted approximately 45 minutes.

 Analysis

The outcomes measured were adapted from the Ottawa Decision Support Framework: knowledge of the benefits and risks of lung cancer screening, acceptability [29], decisional conflict [30], and concordance between the USPSTF’s recommendation and an individual’s preference. Concordance is a binary variable defined as “Yes” = 1, where an individual prefers to get screened (or not to get screened) and is eligible (or ineligible) and “No” = 0 if an individual prefers not to get screened (or to get screened) but is eligible (or ineligible). We also measured a participant’s risk perception as recommended by relevant literature [9]. To compare before-after outcomes, we will use the Wilcoxon signed-rank test for continuous outcomes (knowledge and decisional conflict) and McNemar’s test for binary outcomes (concordance and risk perception indicators). Frequency statistics will be computed for the acceptability items, as these will be measured only in the “after” survey. The software Stata 13 will be used to carry out all analyses [31].

 Phase IIb: Beta Testing: Focus Group Overview

Following completion of the survey, participants were asked if they would like to participate in a focus group to allow them to give the study team more feedback about the decision aid they just viewed. If the answer was affirmative, the participant was told that he or she could be contacted within the month to make an appointment. They were also asked whether the decision aid had indicated that they were eligible for screening (based on USPSTF guidelines) and what the risk of dying due to lung cancer was as computed by the calculator.

Focus groups will be stratified by self-reported eligibility for LDCT screening. Given that one of the main goals of the tool is to increase concordance with USPSTF eligibility guidelines, it will be useful to have specific focus groups consisting of screen-(in)eligible individuals entirely. The aim is to have 4-8 individuals per focus group. Where possible, focus groups will also be stratified by sex and age, given eligibility for LDCT screening.

 Data Collection

The same steps will be followed for beta testing as those used for the focus groups conducted for alpha testing. The current version of the interview guide can be seen in Multimedia Appendix 11.

 Analysis

Thematic analysis will be done on the data yielded from the focus group sessions. Two study team members with qualitative analysis expertise will code the data separately and compare codes to establish the themes to be explored. A report will be given to the rest of the team, and all members will decide what changes to include for the next iteration of the decision aid.

 Research Ethics

All study participants will be asked to complete consent forms. Specific consent for audiotaping of focus groups will also be sought. This study was approved by the University of Michigan Medical School Institutional Review Board on 2014 June 18 (Study ID: HUM00088232).

 Results

Alpha testing is completed. Beta testing is currently being carried out. As of 2014 December 7, 60 participants had completed the before-after study. We expect to have results by 2015 January 31. Qualitative data collection and analysis are expected to be completed by 2015 May 31. The current iteration of the results page with personalized risk generated by the decision aid can be seen at Multimedia Appendix 1.

 Discussion

Preliminary results from the before-after study indicate that the decision aid improves knowledge about lung cancer screening, decreases decisional conflict, and increases concordance between USPSTF recommendations and the screening option preferred by the user. Therefore, we anticipate that the decision aid will be helpful to individuals in making informed decisions about lung cancer screening.

 Multimedia Appendix 1

An example of the current iteration of the results page.

 Multimedia Appendix 2

Focus group interview guide - alpha testing.

 Multimedia Appendix 3

Focus group participant survey.

 Multimedia Appendix 4

Focus group exit survey.

 Multimedia Appendix 5

Before-after survey: knowledge.

 Multimedia Appendix 6

Before-after survey: demographics and risk factors.

 Multimedia Appendix 7

Before-after survey: risk perception.

 Multimedia Appendix 8

Before-after survey: decisional conflict.

 Multimedia Appendix 9

Before-after survey: values clarification.

 Multimedia Appendix 10

Before-after survey: acceptability.

 Multimedia Appendix 11

Focus group interview guide – revised for beta testing.

 Abbreviations IPDASi

International Patient Decision Aid Standards instrument

 LDCT

low-dose computed tomography

 NLST

National Lung Screening Trial

 UM

University of Michigan

 UMHS

University of Michigan Health System

 USPSTF

US Preventive Services Task Force

YKL, TJC, DA, and RM conceived of the study and sought funding and ethical approval. YKL and RM are responsible for the management of the alpha and beta testing phases and planned the statistical analysis. STC and PC assisted in the development of the prototype and subsequent iterations of the decision tool. PC and MW have assisted with data collection in both phases of testing. All authors have been involved in drafting and revising the manuscript and approved the final version.

This work has been supported by the Elizabeth A Crary Fund of the University of Michigan Comprehensive Cancer Center.

None declared.

 American Cancer Society Cancer facts & figures 2013 2013 2014-12-07 Atlanta, GA American Cancer Society http://www.cancer.org/acs/groups/content/@epidemiologysurveilance/documents/document/acspc-036845.pdf 6UeHzsZjI National Lung Screening Trial Research Team Church TR Black WC Aberle DR Berg CD Clingan KL Duan F Fagerstrom RM Gareen IF Gierada DS Jones GC Mahon I Marcus PM Sicks JD Jain A Baum S Results of initial low-dose computed tomographic screening for lung cancer N Engl J Med 2013 368 21 1980 91 10.1056/NEJMoa1209120 23697514 PMC3762603 Moyer VA US Preventive Services Task Force Screening for lung cancer: U.S. Preventive Services Task Force recommendation statement Ann Intern Med 2014 160 5 330 8 10.7326/M13-2771 24378917 1809422 de Koning HJ Meza R Plevritis SK ten Haaf K Munshi VN Jeon J Erdogan SA Kong CY Han SS van Rosmalen J Choi SE Pinsky PF Berrington de Gonzalez A Berg CD Black WC Tammemägi MC Hazelton WD Feuer EJ McMahon PM Benefits and harms of computed tomography lung cancer screening strategies: a comparative modeling study for the U.S. Preventive Services Task Force Ann Intern Med 2014 160 5 311 20 10.7326/M13-2316 24379002 1809424 PMC4116741 Sheridan SL Harris RP Woolf SH Shared Decision-Making Workgroup of the US Preventive Services Task Force Shared decision making about screening and chemoprevention: a suggested approach from the U.S. Preventive Services Task Force Am J Prev Med 2004 26 1 56 66 14700714 S0749379703002873 Elwyn G O'Connor A Stacey D Volk R Edwards A Coulter A Thomson R Barratt A Barry M Bernstein S Butow P Clarke A Entwistle V Feldman-Stewart D Holmes-Rovner M Llewellyn-Thomas H Moumjid N Mulley A Ruland C Sepucha K Sykes A Whelan T International Patient Decision Aids Standards (IPDAS) Collaboration Developing a quality criteria framework for patient decision aids: online international Delphi consensus process BMJ 2006 333 7565 417 10.1136/bmj.38926.629329.AE 16908462 bmj.38926.629329.AE PMC1553508 O'Connor AM Bennett C Stacey D Barry MJ Col NF Eden KB Entwistle V Fiset V Holmes-Rovner M Khangura S Llewellyn-Thomas H Rovner DR Do patient decision aids meet effectiveness criteria of the international patient decision aid standards collaboration? A systematic review and meta-analysis Med Decis Making 2007 27 5 554 74 10.1177/0272989X07307319 17873255 0272989X07307319 Stacey D Samant R Bennett C Decision making in oncology: a review of patient decision aids to support patient participation CA Cancer J Clin 2008 58 5 293 304 10.3322/CA.2008.0006 18755939 CA.2008.0006 Jimbo M Rana GK Hawley S Holmes-Rovner M Kelly-Blake K Nease DE Ruffin MT What is lacking in current decision aids on cancer screening? CA Cancer J Clin 2013 63 3 193 214 10.3322/caac.21180 23504675 PMC3644368 Stacey D Légaré F Col NF Bennett CL Barry MJ Eden KB Holmes-Rovner M Llewellyn-Thomas H Lyddiatt A Thomson R Trevena L Wu JH Decision aids for people facing health treatment or screening decisions Cochrane Database Syst Rev 2014 1 CD001431 10.1002/14651858.CD001431.pub4 24470076 O’Connor AM Stacey D Jacobsen MJ Ottawa Hospital Research Institute 2011 2014-11-26 Ottawa, Ontario Ottawa decision support tutorial: improving practitioners' decision support skillshttps://decisionaid.ohri.ca/ODST/pdfs/ODST.pdf 6UNe9B9L9 Memorial Sloan Kettering Cancer Center 2014-11-14 Lung cancer screening decision toolhttp://nomograms.mskcc.org/Lung/Screening.aspx 6U5RgfIDW Veterans Health Administration My lung cancer screening did not show lung cancer: now what? 2014-11-26 Washington, DC Veterans Health Administration http://www.prevention.va.gov/docs/NowWhat.pdf 6UNe3Yn2g Veterans Health Administration Screening for lung cancer pamphlet 2014-11-26 Washington, DC Veterans Health Administration http://www.prevention.va.gov/docs/LungCancerScreeningHandout.pdf 6UNe4oKXm Chen Y Marcus MW Niaz A Duffy SW Field JK MyLungRisk: a user-friendly, web-based calculator for risk assessment of lung cancer based on the validated Liverpool Lung Project risk prediction model Int J Health Promot Educ 2014 52 144 152 10.1080/14635240.2014.888814 Volk RJ Linder SK Leal VB Rabius V Cinciripini PM Kamath GR Munden RF Bevers TB Feasibility of a patient decision aid about lung cancer screening with low-dose computed tomography Prev Med 2014 62 60 3 10.1016/j.ypmed.2014.02.006 24518006 S0091-7435(14)00071-1 Bach PB Kattan MW Thornquist MD Kris MG Tate RC Barnett MJ Hsieh LJ Begg CB Variations in lung cancer risk among smokers J Natl Cancer Inst 2003 95 6 470 8 12644540 Joseph-Williams N Newcombe R Politi M Durand MA Sivell S Stacey D O'Connor A Volk RJ Edwards A Bennett C Pignone M Thomson R Elwyn G Toward minimum standards for certifying patient decision aids: a modified Delphi consensus process Med Decis Making 2014 34 6 699 710 10.1177/0272989X13501721 23963501 0272989X13501721 Viswanath K Science and society: the communications revolution and cancer control Nat Rev Cancer 2005 5 10 828 35 10.1038/nrc1718 16195753 nrc1718 Dorfman CS Williams RM Kassan EC Red SN Dawson DL Tuong W Parker ER Ohene-Frempong J Davis KM Krist AH Woolf SH Schwartz MD Fishman MB Cole C Taylor KL The development of a web- and a print-based decision aid for prostate cancer screening BMC Med Inform Decis Mak 2010 10 12 10.1186/1472-6947-10-12 20199680 1472-6947-10-12 PMC2845091 Fagerlin A Zikmund-Fisher BJ Ubel PA Helping patients decide: ten steps to better risk communication J Natl Cancer Inst 2011 103 19 1436 43 10.1093/jnci/djr318 21931068 djr318 PMC3218625 Zikmund-Fisher BJ Witteman HO Dickson M Fuhrel-Forbis A Kahn VC Exe NL Valerio M Holtzman LG Scherer LD Fagerlin A Blocks, ovals, or people? Icon type affects risk perceptions and recall of pictographs Med Decis Making 2014 34 4 443 53 10.1177/0272989X13511706 24246564 0272989X13511706 PMC3991751 Trevena LJ Zikmund-Fisher BJ Edwards A Gaissmaier W Galesic M Han PK King J Lawson ML Linder SK Lipkus I Ozanne E Peters E Timmermans D Woloshin S Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers BMC Med Inform Decis Mak 2013 13 Suppl 2 S7 10.1186/1472-6947-13-S2-S7 24625237 1472-6947-13-S2-S7 PMC4045391 Tammemägi MC Katki HA Hocking WG Church TR Caporaso N Kvale PA Chaturvedi AK Silvestri GA Riley TL Commins J Berg CD Selection criteria for lung-cancer screening N Engl J Med 2013 368 8 728 36 10.1056/NEJMoa1211776 23425165 PMC3929969 Holbrook AM Pullenayegum EM Troyan SM Nikitovic M Crowther MA Personalized benefit-harm information influences patient decisions regarding warfarin J Popul Ther Clin Pharmacol 2013 20 3 e406 15 University of Michigan Clinical Studies 2014-11-26 Ann Arbor, MI Michigan Institute for Clinical & Health Research https://umclinicalstudies.org/ 6UNAJQIjV Coulter A Stilwell D Kryworuchko J Mullen PD Ng CJ van der Weijden T A systematic development process for patient decision aids BMC Med Inform Decis Mak 2013 13 Suppl 2 S2 10.1186/1472-6947-13-S2-S2 24625093 1472-6947-13-S2-S2 PMC4044159 Schonberg MA Hamel MB Davis RB Griggs MC Wee CC Fagerlin A Marcantonio ER Development and evaluation of a decision aid on mammography screening for women 75 years and older JAMA Intern Med 2014 174 3 417 24 10.1001/jamainternmed.2013.13639 24378846 1792914 PMC4017368 O’Connor AM Cranney A User manual – acceptability [modified 2002] 1996 2014-11-26 Ottawa, Ontario Ottawa Hospital Research Institute https://decisionaid.ohri.ca/docs/develop/User_Manuals/UM_Acceptability.pdf 6UNCdbnoG O’Connor AM Decisional conflict scale [updated 2010] 1993 2014-11-26 Ottawa, Ontario Ottawa Hospital Research Institute https://decisionaid.ohri.ca/docs/develop/Tools/DCS_LowLiteracy_English.pdf 6UNCFAFki StataCorp STATA 13 2013 2014-12-07 College Station, TX StataCorp LP http://www.stata.com/stata13/ 6UeK3vopG