Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 13.05.20 in Vol 9, No 5 (2020): May

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/16103, first published Sep 02, 2019.

This paper is in the following e-collection/theme issue:

    Protocol

    Use of Simulation Modeling to Inform Decision Making for Health Care Systems and Policy in Colorectal Cancer Screening: Protocol for a Systematic Review

    1Telfer School of Management, University of Ottawa, Ottawa, ON, Canada

    2Division of Colorectal Surgery, The Ottawa Hospital, Department of Surgery, Ottawa, ON, Canada

    3Office of Research Services, MacEwan University, Edmonton, AB, Canada

    Corresponding Author:

    Heather Smith, BSc, MD

    Telfer School of Management

    University of Ottawa

    55 Laurier Ave East

    DMS6132

    Ottawa, ON, K1N6N5

    Canada

    Phone: 1 6135625884

    Email: hsmit037@uottawa.ca


    ABSTRACT

    Background: Simulation modeling has frequently been used to assess interventions in complex aspects of health care, such as colorectal cancer (CRC) screening, where clinical trials are not feasible. Simulation models provide estimates of outcomes, unintended consequences, and costs of an intervention; thus offering an invaluable decision aid for policy makers and health care leaders. However, the contribution that simulation models have made to policy and health system decisions is unknown.

    Objective: This study aims to assess if simulation modeling has supported evidence-informed decision making in CRC screening.

    Methods: A preliminary literature search and pilot screening of 100 references were conducted by three independent reviewers to define and refine the inclusion criteria of this systematic review. Using the developed inclusion criteria, a search of the academic and gray literature published between January 1, 2008, and March 1, 2019, will be conducted to identify studies that developed a simulation model focusing on the delivery of CRC screening of average-risk individuals. The three independent reviewers will assess the validation process and the extent to which the study contributed evidence toward informed decision making (both reported and potential). Validation will be assessed based on adherence to the best practice recommendations described by the International Society for Pharmacoeconomics and Outcomes Research-Society for Medical Decision Making (ISPOR-SMDM). Criteria for potential contribution to decision making will be defined as outlined in the internationally recognized Grading of Recommendations Assessment, Development and Evaluation Evidence to Decision (GRADE EtD) framework. These criteria outline information that the health system and policy decision makers should consider when making an evidence-informed decision including an intervention’s resource utilization, cost-effectiveness, impact on health equity, and feasibility. Subgroup analysis of articles based on their GRADE EtD criteria will be conducted to identify methods associated with decision support capacity (ie, participatory, quantitative, or mixed methods).

    Results: A database search of the literature yielded 484 references to screen for inclusion in the systematic review. We anticipate that this systematic review will provide an insight into the contribution of simulation modeling methods to informed decision making in CRC screening delivery and discuss methods that may be associated with a stronger impact on decision making. The project was funded in May 2019. Data collection took place from January 2008 to March 2019. Data analysis was completed in November 2019, and are expected to be published in spring 2020.

    Conclusions: Our findings will help guide researchers and health care leaders to mobilize the potential for simulation modeling to inform evidence-informed decisions in CRC screening delivery. The methods of this study may also be replicated to assess the utility of simulation modeling in other areas of complex health care decision making.

    International Registered Report Identifier (IRRID): DERR1-10.2196/16103

    Trial Registration: PROSPERO no. 130823; https://www.crd.york.ac.uk/PROSPERO

    JMIR Res Protoc 2020;9(5):e16103

    doi:10.2196/16103

    KEYWORDS

    Crowdfunding campaign to support this specific research

    We help JMIR researchers to raise funds to pursue their research and development aimed at tackling important health and technology challenges. If you would like to show your support for this author, please donate using the button below. The funds raised will directly benefit the corresponding author of this article (minus 8% admin fees). Your donations will help this author to continue publishing open access papers in JMIR journals. Donations of over $100 may also be acknowledged in future publications.

    keyboard with crowdfunding key instead of enter key

    Suggested contribution levels: $20/$50/$100



    Introduction

    The benefits of colorectal cancer (CRC) screening have been well cited including a reduced incidence of CRC, earlier stage of presentation, improved outcomes for patients with detected malignancy, and reduced CRC-associated mortality [1-4]. Screening uses fecal tests, diagnostic imaging, and endoscopic examination to assess the possiblity of CRC occurring among asymptomatic individuals at increased risk of developing CRC. Participation of eligible individuals is voluntary and remains low in many regions across our country [5]. Determining the best screening modality, delivery method, resource allocation, and follow-up for screening is complex because health policy and system decision makers must weigh the cost and benefits of screening, taking into account the sensitivity, specificity, and accessibility of various screening modalities. Furthermore, such interventions often cannot be tested in clinical trials because of multiple environmental, sociocultural, and health system factors that negate the feasibility and safety of a trial [6]. Researchers and decision makers have increasingly relied on simulation models to evaluate interventions in CRC screening [7,8].

    A simulation model is a computer-generated representation of a real-world system or process that can be used to analyze the evolving behavior of a system over time, or modified to predict results of a variety of “what-if” scenarios [9]. Simulation models have been applied to a broad range of areas in health care to predict outcomes, unintended consequences, and costs of proposed interventions, thereby offering an invaluable decision aid for policy makers and health care leaders [7,10-12]. The purpose of simulation models is well defined, that is, to provide decision makers with evidence to facilitate decision making; however, the extent to which simulation modeling has fulfilled this purpose in CRC screening is unknown [13]. Simulation modeling has the potential to provide strong evidence for multiple aspects of informed decision making at the policy and health system level, including a proposed intervention’s resource utilization, cost-effectiveness, feasibility, sustainability, potential impact, and acceptance among stakeholders [14,15]. For instance, by simulating what-if scenarios informed by clinical trials and observational data, the model can help to identify the appropriate age to initiate screening, superiority of one screening modality over another, or the most cost-effective frequency of screening a particular population in the long term. However, the extent to which simulation modeling has realized this potential impact in CRC screening is unknown.

    For simulation models to be useful for decision makers, models must be sufficiently accurate and valid for application [13]. There have been concerns with model credibility in health care and reporting of model conceptualization, parameterization, and validation is not consistent in the literature [16-18]. For this reason, in this systematic review, each study will be assessed for adherence to best practice recommendations in model validation as detailed in the Methods section of this proposal [16].

    From our review of the literature, we identified two gaps that we plan to address with this systematic review: (1) no systematic review has examined the application of simulation in CRC screening within the last 10 years and (2) no systematic review has specifically addressed the impact of simulation modeling on decision making in health care. The most recent systematic review that had examined CRC screening only included articles until 2007 inclusively [19]. Since that time, systematic reviews have been conducted examining the quality and cost-effectiveness of simulation modeling in breast cancer screening, but not for CRC screening [20,21]. For instance, a systematic review by Sobolev and colleagues [22] looked at the reported “utility” of simulation models in surgical patient flow and reinforced the need for evaluating the impact of models on decision making, but did not formally evaluate this in their review. Therefore, this study aims to address these knowledge gaps by assessing the validity and impact of simulation modeling on health system and policy decision making in CRC screening delivery. Publication of this protocol will allow for critical peer review of the aims and methods outlined for the intended study. This will help strengthen the rigor by which it will be conducted and validate its utility.


    Methods

    Protocol and Registration

    This systematic review will be conducted in accordance with the Cochrane Library systematic reviews guide and the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) [23,24]. This protocol is reported in accordance with the PRISMA-Protocol checklist (Multimedia Appendix 1). The protocol has been submitted for registration in PROSPERO, and any amendments will be filed with PROSPERO (no. 130823).

    Search Strategy and Eligibility Criteria

    Studies will be selected by searching the academic and gray literature published between January 1, 2008, and March 1, 2019, conducted to identify articles that include (1) simulation modeling methods and (2) a focus on CRC screening. Only full-text articles available in English will be included. Studies will be identified from academic databases (Medline, Embase, Cochrane Central, Scopus, Web of Science, IEEEXplore, ACM Digital Library, Econolit, National Health Service Economic Evaluation Database, Health Assessment Database, and Cost-Effective Analysis Registry) using controlled vocabulary (Medical Subject Heading) and text word search terms selected by the author HS and an experienced librarian Alexandra Davis (Textbox 1).

    Terms were selected to capture the most commonly used types of simulation models in health care (discrete event simulation, Markov chain model, Monte-Carlo simulation, agent-based model, and system dynamics model) [11]. This will be supplemented by hand searching of the gray literature and conference proceedings as well as citation searches of selected articles.


    Textbox 1. Preliminary search strategies.
    View this box

    Simulation modeling can be used in a broad scope of applications in cancer screening. To help refine the appropriate inclusion criteria and feasibility of this review, a preliminary search of the literature was conducted. A search of the literature published between January 1, 1946, and March 1, 2019, was conducted, and a pilot screening of 100 abstracts was performed by three independent reviewers (HS, PV, and CK) using the following inclusion criteria: (1) simulation model use and (2) CRC screening. This yielded 56 of 100 included abstracts after 24 conflicts were resolved through extensive discussion among all authors (Figure 1). To reduce conflicts, the inclusion and exclusion criteria were further refined to include only original articles describing a simulation model derived from clinical data focused on the delivery of CRC screening individuals with average CRC risk using one or all of the following modalities of screening recommended by the Canadian guidelines within the last 10 years: fecal occult blood test, fecal immunohistochemical testing, flexible sigmoidoscopy, and colonoscopy [25-27]. Excluded articles are those describing other screening modalities not recommended in the Canadian screening guidelines as identified above, commentary or review articles, simulation models that include screening of other cancers, or articles that have no mention of screening delivery. The time frame was also further restricted to only include articles published after 2008 because a systematic review on the use of simulation modeling in health care, including CRC screening, was identified and had included articles published until the end of 2007 [19]. A second pilot screening was conducted using the revised criteria, which yielded fewer conflicts, and the revised inclusion and exclusion criteria were adopted for this systematic review.

    Figure 1. Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) flow diagram of pilot search and reference screening.
    View this figure

    Selection of Studies for the Review

    All article titles and abstracts will be screened by three independent reviewers (HS, PV, and CK) using the abstract screening program Abstrakr (Brown University) followed by screening of selected full-text studies for compliance with the eligibility criteria as mentioned above using DistillerSR, version 2019 (Evidence Partners) [28,29]. Articles or reports of the same study will be linked together. Authors of the articles will be contacted to clarify study eligibility, where appropriate. Conflicts will be resolved through discussion between reviewers or consultation with author (RB), as needed.

    Data Extraction

    All included studies will be reviewed by three independent reviewers (HS, CK, and PV). Using DistillerSR version 2019, data will be extracted regarding the study and model description, and model validation as outlined in Table 1 [13]. Discrepancies will be identified and resolved through discussion, and missing data will be requested from the study authors as needed.

    The validation of a simulation model is an important determinant of the risk of bias and applicability of a simulation model. All models will be assessed in accordance with the guidelines of the International Society for Pharmacoeconomics and Outcomes Research-Society for Medical Decision Making (ISPOR-SMDM) report [13]. From the literature review, several tools were identified to assist model developers and users with model validation [13,16,17,30-33]. The ISPOR-SMDM Taskforce has developed good practice guidelines for modeling in health care including recommendations on conceptualization, parameterization, and validation. Of the validation tools identified in the literature, the taskforce report had the broadest scope and most rigorous development process; therefore, it was selected to guide validation assessment in this review [16]. Authors (HS, CK, and PV) will individually assess whether authors report or conduct face validity (wherein experts evaluate model structure, data sources, assumptions, and results), verification or internal validity (check accuracy of coding), cross validity (comparison of results with other models analyzing the same problem), external validity (comparing model results with real-world results), and predictive validity (comparing model results with prospectively observed events), as outlined in Table 1 [13].

    Table 1. Model characteristics and validation.
    View this table

    Studies will then be assessed for the extent to which the study has or could potentially have contributed evidence toward informed decision making. For reported contribution, each article will be searched in its entirety for statements referring to the simulation model results informing decision making. If not clearly stated in the publication, the information will be requested from study authors.

    Recognizing that the impact of a simulation model on a specific decision may not be communicated at the time of publication, we plan to also assess the potential contribution a simulation model could have made to evidence-informed decision making based on whether the results align with important factors for making an informed decision [34]. We will assess articles to determine whether they include evidence considered to be important for decisions, as outlined in the internationally recognized Grading of Recommendations Assessment, Development and Evaluation Evidence to Decision (GRADE EtD) framework [14]. The GRADE EtD framework has been developed as part of the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence (DECIDE) project in collaboration with researchers in the health system and public health internationally. It outlines a set of important factors that decision makers should consider and address with research evidence to guide their decisions in health policy or systems [35]. These criteria include information on an intervention’s resource utilization, cost-effectiveness, impact on health equity, and feasibility (Table 2). We will assess whether the study results apply to the GRADE EtD criteria. Subgroup analysis of articles based on their GRADE EtD criteria will be conducted to identify methods associated with decision support capacity (ie, quantitative or mixed methods).

    Table 2. GRADE EtD (Grading of Recommendations Assessment, Development and Evaluation Evidence to Decision) criteria of decision making for health system and public health decisions [14].
    View this table

    Results

    A preliminary search of the literature published between January 1, 1946, and March 1, 2019, was conducted, yielding 617 references and a pilot screening of 100 randomly selected abstracts was performed by three independent reviewers (HS, PV, and CK) using the following inclusion criteria: (1) simulation model use and (2) CRC screening (Figure 1). This resulted in inclusion of 56 of 100 (56%) abstracts after 24 conflicts were resolved through extensive discussion among all authors, leading to revision and clarification of the inclusion criteria as described in the section “Methods”. The revised search yielded 484 references to review. A repeated pilot screening resulted in the inclusion of 8 of 100 (8%) abstracts after 16 conflicts were resolved with minimal discussion. The publication of this article was funded by University of Ottawa Telfer School of Management Research Grant and a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada.

    The project was funded in May 2019. Data collection took place from January 2008 to March 2019. Data analysis was completed in November 2019, and are expected to be published in spring 2020.


    Discussion

    The purpose of simulation modeling in health care is generally to inform a decision [13,33,36]. The extent to which simulation modeling has fulfilled this purpose has not been assessed. We anticipate that this systematic review will help address this knowledge gap by assessing the contribution simulation modeling has made to informed decision making in an area of health care where it has been frequently used: CRC screening delivery. We will use the GRADE EtD framework to structure our analysis of potential decision impact of the models. This includes the model’s contribution to determining the feasibility of screening, acceptability of proposed screening strategies by stakeholders, and sustainability of screening over the long term. This analysis will help guide researchers by identifying methods in simulation modeling that have been associated with a greater success in decision support, such as mixed methods, participatory simulation model development, and group model building, which has been reported as beneficial in other applications of simulation modeling in health care [37,38]. It will assist decision makers and model users to identify areas where simulation modeling has proven to be useful, such as for identifying resource requirements or conducting cost-benefit analysis.

    The dataset search yielded 484 references, which suggests that the body of literature on this topic is fairly robust. We anticipate there will be an adequate number of relevant models to conduct an informative systematic review on this topic.

    We foresee several potential limitations to this study. The heterogeneity of articles may make it challenging to evaluate studies using a uniform framework from validation and decision-making criteria. Furthermore, the impact and decision support that a study provide are difficult to quantify and therefore will be subject to both authors’ and reviewers’ bias. We aim to mitigate this by using the GRADE EtD framework and by having reviewers with clinical (HS), health informatics (CK), and simulation (PV) expertise review each included study. Finally, our assessment of model validity will be limited by a lack of validation standards in the literature and reporting by authors on their validation process and outcomes [18].

    In conclusion, the proposed systematic review will provide an insight into the contribution and validity of simulation modeling in CRC screening. The results have the potential to inform researchers, health care leaders, and policy makers to develop valid, informative simulation models that will support decision making. This analysis could be expanded to assess the use of simulation modeling in other areas of health care.

    Acknowledgments

    We thank Alexandra Davis from The Ottawa Hospital Library for her assistance in conducting the search.

    Authors' Contributions

    HS is the guarantor of this proposed review. HS, CK, and PV performed the pilot screening of abstracts outlined in this manuscript. All authors participated in the conceptualization and design of this review, and have read and approved the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) checklist.

    PDF File (Adobe PDF File), 83 KB

    References

    1. NCCN Colorectal Cancer Screening [Internet].   URL: https://www.nccn.org/professionals/physician_gls/pdf/colorectal_screening.pdf [accessed 2018-11-23]
    2. Swartz AW, Eberth JM, Strayer SM. Preventing colorectal cancer or early diagnosis: Which is best? A re-analysis of the US Preventive Services Task Force Evidence Report. Prev Med 2018 24;118:104-112. [CrossRef] [Medline]
    3. Rabeneck L, Rumble RB, Thompson F, Mills M, Oleschuk C, Whibley A, et al. Fecal immunochemical tests compared with guaiac fecal occult blood tests for population-based colorectal cancer screening. Can J Gastroenterol 2012 ;26(3):131-147 [FREE Full text] [CrossRef] [Medline]
    4. Meester RGS, Doubeni CA, Lansdorp-Vogelaar I, Goede SL, Levin TR, Quinn VP, et al. Colorectal cancer deaths attributable to nonuse of screening in the United States. Ann Epidemiol 2015 ;25(3):208-213.e1 [FREE Full text] [CrossRef] [Medline]
    5. Canadian-Cancer-Statistics-2018-EN.pdf [Internet].   URL: http:/​/www.​cancer.ca/​~/​media/​cancer.ca/​CW/​cancer%20information/​cancer%20101/​Canadian%20cancer%20statistics/​Canadian-Cancer-Statistics-2018-EN.​pdf?la=en [accessed 2018-11-8]
    6. Canadian Partners Against Cancer. 2017. Colorectal cancer screening in Canada: environmental scan [Internet]. cancerview.ca   URL: http://www.cancerview.ca/preventionandscreening/colorectalcancerscreeningpage/ [accessed 2018-11-18]
    7. Fone D, Hollinghurst S, Temple M, Round A, Lester N, Weightman A, et al. Systematic review of the use and value of computer simulation modelling in population health and health care delivery. J Public Health 2003 1;25(4):325-335 [FREE Full text] [CrossRef] [Medline]
    8. Gauvreau CL, Fitzgerald NR, Memon S, Flanagan WM, Nadeau C, Asakawa K, et al. The OncoSim model: development and use for better decision-making in Canadian cancer control. Curr Oncol 2017 ;24(6):401-406 [FREE Full text] [CrossRef] [Medline]
    9. Banks J. Discrete-Event System Simulation, 5th edition, International versionth edition. Upper Saddle River, NJ: Pearson Education; 2010. ISBN:978-0-13-815037-2.
    10. Xue H, Slivka L, Igusa T, Huang TT, Wang Y. Applications of systems modelling in obesity research. Obes Rev 2018;19(9):1293-1308 [FREE Full text] [CrossRef] [Medline]
    11. Salleh S, Thokala P, Brennan A, Hughes R, Booth A. Simulation modelling in healthcare: an umbrella review of systematic literature reviews. PharmacoEconomics 2017 1;35(9):937-949 [FREE Full text] [CrossRef] [Medline]
    12. Freebairn L, Atkinson J, Kelly P, McDonnell G, Rychetnik L. Simulation modelling as a tool for knowledge mobilisation in health policy settings: a case study protocol. Health Res Policy Syst 2016 21;14(1):71 [FREE Full text] [CrossRef] [Medline]
    13. Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM modeling good research practices task force-7. Value Health 2012 1;15(6):843-850 [FREE Full text] [CrossRef] [Medline]
    14. Moberg J, Oxman AD, Rosenbaum S, Schünemann HJ, Guyatt G, Flottorp S, for the GRADE Working Group. The GRADE Evidence to Decision (EtD) framework for health system and public health decisions. Health Res Policy Syst 2018 29;16(1):45. [CrossRef] [Medline]
    15. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst 2003 13;1(1):2 [FREE Full text] [CrossRef] [Medline]
    16. Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices—overview: a report of the ISPOR-SMDM modeling good research practices task force-1. Value Health 2012 ;15(6):796-803 [FREE Full text] [CrossRef]
    17. Lorscheid I, Heine BO, Meyer M. Opening the ‘black box’ of simulations: increased transparency and effective communication through the systematic design of experiments. Comput Math Organ Theory 2012 1;18(1):22-62 [FREE Full text] [CrossRef]
    18. de Boer PT, Frederix GWJ, Feenstra TL, Vemer P. Unremarked or unperformed? Systematic review on reporting of validation efforts of health economic decision models in seasonal influenza and early breast cancer. PharmacoEconomics 2016 1;34(9):833-845 [FREE Full text] [CrossRef] [Medline]
    19. Katsaliaki K, Mustafee N. Applications of simulation within the healthcare context. J Oper Res Soc 2011 1;62(8):1431-1451 [FREE Full text] [CrossRef]
    20. Koleva-Kolarova RG, Zhan Z, Greuter MJW, Feenstra TL, De Bock GH. Simulation models in population breast cancer screening: a systematic review. Breast 2015 1;24(4):354-363 [FREE Full text] [CrossRef]
    21. Schiller-Frühwirth IC, Jahn B, Arvandi M, Siebert U. Cost-effectiveness models in breast cancer screening in the general population: a systematic review. Appl Health Econ Health Policy 2017 1;15(3):333-351 [FREE Full text] [CrossRef] [Medline]
    22. Sobolev B. Linking operations and health services research. Clin Invest Med Ott 2005 ;28(6):305-307. [Medline]
    23. PRISMA [Internet].   URL: http://www.prisma-statement.org/ [accessed 2018-12-10] [WebCite Cache]
    24. Cochrane Handbook for Systematic Reviews of Interventions [Internet].   URL: http://handbook-5-1.cochrane.org/ [accessed 2018-12-10] [WebCite Cache]
    25. Telford JJ. Canadian guidelines for colorectal cancer screening. Can J Gastroenterol 2011 ;25(9):479-481 [FREE Full text] [CrossRef] [Medline]
    26. Canadian Task Force on Preventive Health Care. Colorectal cancer screening: recommendation statement from the Canadian Task Force on Preventive Health Care. CMAJ 2001 24;165(2):206-208. [Medline]
    27. Care CTF on PH. Recommendations on screening for colorectal cancer in primary care. CMAJ 2016 15;188(5):340-348. [CrossRef] [Medline]
    28. Home [Internet]. Systematic review and literature review software by evidence partners   URL: https://www.evidencepartners.com/ [accessed 2019-4-4]
    29. Wallace BC, Small K, Brodley CE, Lau J, Trikalinos TA. Deploying an interactive machine learning system in an evidence-based practice center: Abstrackr. New York, NY: ACM; 2012 Presented at: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium 2012; 2018-11-16 p. 819-824. [CrossRef]
    30. Law AM. How to build valid and credible simulation models. 2008 Presented at: 2008 Winter Simulation Conference; Miami, FL p. 39-47. [CrossRef]
    31. Sargent RG. Validation and verification of simulation models. Washington, DC: Winter Simulation Conference; 2004 Presented at: Proceedings of the 36th Conference on Winter Simulation Conference [Internet]; 2019-1-28 p. 17-28   URL: http://dl.acm.org/citation.cfm?id=1161734.1161742
    32. Vemer P, Corro Ramos I, van Voorn GAK, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. PharmacoEconomics 2016 1;34(4):349-361 [FREE Full text] [CrossRef] [Medline]
    33. Kopec JA, Finès P, Manuel DG, Buckeridge DL, Flanagan WM, Oderkirk J, et al. Validation of population-based disease simulation models: a review of concepts and methods. BMC Public Health 2010 18;10(1):710 [FREE Full text] [CrossRef] [Medline]
    34. Wheeler SB, Leeman J, Hassmiller Lich K, Tangka FKL, Davis MM, Richardson LC. Data-powered participatory decision making: leveraging systems thinking and simulation to guide selection and implementation of evidence-based colorectal cancer screening interventions. Cancer J 2018;24(3):136-143 [FREE Full text] [CrossRef] [Medline]
    35. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Group the GW. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 2016 28;353:i2016 [FREE Full text] [CrossRef] [Medline]
    36. Weinstein MC, Toy EL, Sandberg EA, Neumann PJ, Evans JS, Kuntz KM, et al. Modeling for health care and other policy decisions: uses, roles, and validity. Value Health 2001;4(5):348-361 [FREE Full text] [CrossRef] [Medline]
    37. Weeks MR, Li J, Lounsbury D, Green HD, Abbott M, Berman M, et al. Using participatory system dynamics modeling to examine the local HIV test and treatment care continuum in order to reduce community viral load. Am J Community Psychol 2017 ;60(3-4):584-598 [FREE Full text] [CrossRef] [Medline]
    38. Rouwette EAJA, Vennix JAM, van Mullekom T. Group model building effectiveness: a review of assessment studies. Syst Dyn Rev 2002;18(1):5-45 [FREE Full text] [CrossRef]


    Abbreviations

    CRC: colorectal cancer
    GRADE EtD: Grading of Recommendations Assessment, Development and Evaluation Evidence to Decision
    ISPOR-SMDM: International Society for Pharmacoeconomics and Outcomes Research-Society for Medical Decision Making


    Edited by G Eysenbach; submitted 02.09.19; peer-reviewed by T V, A Young; comments to author 18.10.19; revised version received 09.11.19; accepted 26.11.19; published 13.05.20

    ©Heather Smith, Peyman Varshoei, Robin Boushey, Craig Kuziemsky. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 13.05.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.