ResProt JMIR Res Protoc JMIR Research Protocols 1929-0748 JMIR Publications Toronto, Canada v8i1e12006 30622097 10.2196/12006 Protocol Protocol Development of the Gambling Disorder Identification Test (G-DIT): Protocol for a Delphi Method Study Kuter Nicola Luquiens Amandine Bond Kathy Molander Olof MSc 1
Centre for Psychiatry Research Department of Clinical Neuroscience Karolinska Institutet Norra Stationsgatan 69, 7th floor Stockholm, 113 64 Sweden 46 700011241 olof.molander@ki.se
2 http://orcid.org/0000-0001-5348-051X Volberg Rachel PhD 3 http://orcid.org/0000-0003-2959-0748 Sundqvist Kristina PhD 4 http://orcid.org/0000-0002-5923-0092 Wennberg Peter PhD 4 http://orcid.org/0000-0002-0013-2965 Månsson Viktor MSc 1 2 http://orcid.org/0000-0001-8954-6875 Berman Anne H PhD 1 2 4 5 http://orcid.org/0000-0002-7709-0230 Centre for Psychiatry Research Department of Clinical Neuroscience Karolinska Institutet Stockholm Sweden Stockholm Health Care Services Stockholm County Council Stockholm Sweden School of Public Health and Health Sciences University of Massachusetts Amherst, MA United States Department of Public Health Sciences Stockholm University Stockholm Sweden Stockholm Center for Dependency Disorders Stockholm Sweden Corresponding Author: Olof Molander olof.molander@ki.se 012019 08 01 2019 8 1 e12006 22 8 2018 11 9 2018 5 10 2018 8 10 2018 ©Olof Molander, Rachel Volberg, Kristina Sundqvist, Peter Wennberg, Viktor Månsson, Anne H Berman. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 08.01.2019. 2019

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.

 Background

Research on the identification and treatment of problem gambling has been characterized by a wide range of outcome measures and instruments. However, a single instrument measuring gambling behavior, severity, and specific deleterious effects is lacking.

 Objective

This protocol describes the development of the Gambling Disorder Identification Test (G-DIT), which is a 9- to 12-item multiple-choice scale with three domains: gambling consumption, symptom severity, and negative consequences. The scale is analogous to the widely used Alcohol Use Disorders Identification Test (AUDIT) and the Drug Use Disorders Identification Test (DUDIT).

 Methods

The G-DIT is developed in four steps: (1) identification of items eligible for the G-DIT from a pool of existing gambling measures; (2) presentation of items proposed for evaluation by invited expert researchers through an online Delphi process and subsequent consensus meetings; (3) pilot testing of a draft of the 9- to 12-item version in a small group of participants with problem gambling behavior (n=12); and (4) evaluation of the psychometric properties of the final G-DIT measure in relation to the existing instruments and self-reported criteria of the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5), among individuals with problem gambling and nonproblematic recreational gambling behaviors (n=600). This protocol article summarizes step 1 and describes steps 2 and 3 in detail.

 Results

As of October 2018, steps 1-3 are complete, and step 4 is underway.

 Conclusions

Implementation of this online Delphi study early in the psychometric development process will contribute to the face and construct validity of the G-DIT. We believe the G-DIT will be useful as a standard outcome measure in the field of problem gambling research and serve as a problem-identification tool in clinical settings.

 International Registered Report Identifier (IRRID)

RR1-10.2196/12006

 consensus methods Delphi technique DSM-5 gambling Gambling Disorder Identification Test measurement psychometrics screening

Protocol

Introduction Overview

Gambling is the only addiction without any psychopharmacological substance use that has been recognized as a diagnosis by the American Psychiatric Association in the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5) [1]. Problem gambling is associated with poor mental and physical well-being in individuals with gambling problems [2]; in addition, their partners, parents, and children are negatively affected [3]. Problem gambling leads to severe negative consequences in important life domains such as finance, well-being, health, and relationships [1] and is associated with high rates of suicide ideation and attempts [4]. The clinical diagnostic criteria for pathological gambling were revised in 2013 and termed Gambling Disorder (GD) in the DSM-5 [1]. GD is part of the Substance-Related and Addictive Disorder category in DSM-5, in contrast to the Impulse Disorder category in DSM, 4th edition (DSM-IV) [5,6]. Other updates in the DSM-5 include removal of a previous criterion, illegal acts to finance gambling, and specification of disorder severity. Currently, fulfillment of 4-5 diagnostic criteria leads to a diagnosis of mild GD, 6-7 symptoms are diagnosed as moderate GD, and 8-9 symptoms are diagnosed as severe GD.

As a research field, problem gambling is still in its infancy and is 20-30 years behind research on substance use disorders [7]. Research on the identification and treatment of problem gambling has been characterized by a wide range of outcome measures and instruments [8], leading to difficulties in comparing the effectiveness of different treatments [9]. An additional current challenge for clinical assessment and research outcome measures is that only a few existing instruments have been validated using the relatively new DSM-5 diagnostic criteria for GD. Furthermore, measuring problem gambling from a treatment-oriented perspective is a challenge, as current screening instruments adopt a public health perspective and generally focus on consumption behaviors, symptoms, or negative consequences, but do not encompass all three domains.

To address the issue of variation in outcome measures, an expert panel of researchers convened in 2006 and agreed upon a set of characteristics that should define measures of problem gambling in future treatment studies; these characteristics are collectively known as the Banff consensus agreement [8]. Regarding the issue of including DSM-5 criteria in measures for identification of GD, researchers have proposed some specific DSM-5 criteria such as “chasing losses,” “repeated unsuccessful efforts to stop,” “tolerance,” “loss of control,” and “jeopardized/lost relationships/job” as important gambling measures, because they can be used from a psychometric perspective to better differentiate among various gambling groups as compared to the other GD diagnostic criteria [10-12].

In response to the Banff consensus agreement and the discussion regarding inclusion of specific DSM-5 criteria and with a goal of optimizing a treatment-oriented screening measure, our team is developing the Gambling Disorder Identification Test (G-DIT). We aim to establish a problem gambling-screening test analogous to the Alcohol Use Disorders Identification Test (AUDIT) [13] and the Drug Use Disorders Identification Test (DUDIT) [14]. Our test will include items in three domains: gambling consumption, symptom severity, and negative consequences. The development and validation of the G-DIT is part of the ongoing 6-year Responding to and Reducing Gambling Problems research program in Sweden.

The G-DIT is under development in four steps: (1) identification of items eligible for the G-DIT from a pool of existing gambling measures; (2) presentation of proposed items for evaluation by the authors of this article in a pilot Delphi round, followed by presentation of the proposed items for evaluation by a larger group of invited international expert researchers in a formal Delphi process, and finally, an international expert consensus meeting followed by additional smaller consensus meetings to resolve issues tabled at the international meeting; (3) pilot testing of a draft 9- to 12-item version in a small group of participants with problem gambling behavior (n=12); and (4) evaluation of psychometric properties of the final G-DIT measure in relation to existing instruments and self-reported DSM-5 criteria in individuals with problem gambling and nonproblematic recreational gambling behaviors (n=600). This article summarizes step 1 and describes steps 2 and 3 in detail; the results of steps 2 and 3 will be described in an upcoming publication, and an additional publication will detail step 4.

 Aims and Research Questions <p>The research questions are as follows:</p> <list list-type="bullet"> <list-item> <p>Which of the presented items should have the highest priority?</p> </list-item> <list-item> <p>What are the potential problems of the proposed G-DIT?</p> </list-item> <list-item> <p>How is the face validity of the G-DIT perceived?</p> </list-item> <list-item> <p>What psychometric findings could be of additional importance?</p> </list-item> </list> </sec> </sec> <sec sec-type="methods"> <title>Methods Study Approval and Consent

This study was approved by the Regional Ethics Board of Stockholm, Sweden (ref. no. 2017/1479-31). Approval was granted for the Delphi procedure and evaluation of the instrument in individuals with problem gambling behavior, individuals from gambling self-help groups, and individuals with recreational gambling behavior from a population sample. Informed consent was obtained from all stakeholders in the Delphi process as well as all participants with problem gambling behavior in the “think aloud” interviews. Participants were approached or volunteered via the methods outlined below. Individual Delphi stakeholders were sent a short email introducing the study, and more information on the study and consent forms were made available online. Individual responses were analyzed and presented anonymously in both the Delphi process and “think aloud” procedure. All participants provided consent for publication.

 Analysis of Existing Measures

In step 1, we aimed to identify the maximum number of existing gambling measures. We conducted an extensive literature search of review articles on gambling measures [15-17] and a prior unpublished collection of gambling measures compiled by local colleagues (A Nilsson and K Magnusson, personal communication, February 2017), which resulted in a list of 47 gambling measures (Table 1) [12,18-63]. Items from the measures were gathered in an item pool. Items with the same meaning were identified as doublets between instruments but classified as unique items within an instrument (eg, items in subscales). The final item pool consisted of 726 items, of which 583 were deemed unique items and 143 were deemed doublets; the latter were excluded from the item pool.

The first author categorized all items based on their content into four main categories and 27 subcategories: Gambling Consumption (Type of Game, Time Gambled, Sums, and Gambling Behavior); DSM-5 Criteria (Preoccupation, Tolerance, Loss of Control, Abstinence Symptoms, Escape, Chasing Losses, Lies, Social Consequences, and Relies on Other); Negative Consequences (General Problem, Health, Financial, Critique from Others, Illegal, and Other Negative Consequences); and Other (Motives for Gambling; Self-Efficacy; Situations or Relapse; Cognitive Distortions or Beliefs; Motivation; Anxiety, Depression, or Negative Effect; Alcohol or Drugs; and Other or Miscellaneous). The Other main item category was excluded, as it was not relevant to the G-DIT domains. Thereafter, three additional authors (blind to the original categorization) individually recategorized each item in the three remaining main categories (Gambling Consumption, DSM-5 Criteria, and Negative Consequences) and the predefined subcategories. Interrater reliability was calculated on the basis of the item-categorization agreement for all items, items per subcategory, and items per main category. Statistical analysis using Fleiss kappa [64] for 4 raters in R [65] showed that the interrater reliability ranged from fair to moderate (k=0.42 for all items and k=0.24, k=0.51, and k=0.51 for the relevant main item categories of Gambling Consumption, DSM-5 Criteria, and Negative Consequences, respectively).

 The Delphi Study

We chose the Delphi method to collect feedback from expert researchers. The Delphi method is an iterative technique, comprising sequential questionnaires that are answered anonymously by many relevant stakeholders [66]. To prepare for the formal Delphi process in step 2, we conducted a pilot Delphi procedure in two rounds with the authors of the present study. In the preparation rounds, we evaluated 15 candidate items based on the interrater analysis in step 1. The criteria for selection were 75% agreement on the categorization and importance of these items. These two preparation rounds clarified the variation in expert evaluation of the items and led to a decision to increase the number of candidate items to 30 for the next formal Delphi rounds. The selection of these items was based on interrater agreement of items relevant to the G-DIT domains, previous psychometric findings regarding problem gambling, and the recommendations of the Banff consensus agreement [8]. An overview of the item categories is presented in Figure 1.

Panel Size and Recruitment

There are no accepted guidelines for the panel size in a Delphi analysis. Therefore, we determined our panel size on the basis of the practicality, scope, and time available, similar to previous studies [67,68]. Stakeholders were identified through contacts via our research group and team members of the ongoing research project “Responding to and Reducing Gambling Problems - Studies in Help-Seeking, Measurement, Comorbidity and Policy Impacts” (REGAPS) and through published research in the gambling field. We invited the following stakeholders to participate in the Delphi rounds and requested them to forward the invitation to other researchers in their network (snowball sampling): all authors of the Banff consensus [8] and previous psychometric research targeting specific DSM-5 symptoms [10-12]; presenters at the Alberta Gambling Research Institute’s 17th Annual Conference, 2018, which is the annual independent gambling conference in Banff (these individuals were identified as key influential gambling researchers for the international consensus meeting); all authors of reviews of gambling measures identified in our extended literature search [15-17]; corresponding and first and last authors of published articles or reports of the gambling measures identified in our extended literature search (Table 1); trial investigators including corresponding and first and last authors of reports of randomized trials in the field identified in published systematic reviews [9,69,70]; members of the REGAPS network; and members of the Gambling Research Network, which is a Swedish network for gambling research.

We addressed the potential for attrition between rounds through a personalized invitation, email reminders (every 5 days, but no more than two reminders in total), and provision of an easy interface, which minimized the time required to complete each round [67]. The Delphi-process questionnaire was uploaded on the online SurveyXact platform [71].

Gambling measures (n=47) identified in the literature search.

Measure Reference
The Brief Biosocial Gambling Screen Gebauer et al, 2010 [41]
The Canadian Problem Gambling Index Ferris et al, 2001 [23]
The Case-finding and Help Assessment Tool Goodyear-Smith et al, 2008 [36]
The Consumption Screen for Problematic Gambling Rockloff et al, 2012 [44]
The Control of Pathological Gambling Questionnaire Saiz-Ruiz et al, 2005 [33]
The Cumulative Clinical Signs Method Volberg et al, 1990 [55]
The Early Intervention Gambling Health Test Sullivan, 2007 [50]
The Gamblers Self-Efficacy Questionnaire May et al, 2003 [27]
The Gamblers’ Belief Questionnaire Steenbergh et al, 2002 [56]
The Gambling Abstinence Self-Efficacy Scale Hodgins et al, 2004 [29]
The Gambling Activity Measurement Tool Jacksson et al, 2013 [57]
The Gambling Anonymous Twenty Questions Toneatto et al, 2008 [53]
The Gambling Attitudes and Beliefs Survey Breen et al, 1999 [49]
The Gambling Cognitions Inventory McInnes et al, 2014 [58]
The Gambling Craving Scale Young et al, 2009 [40]
The Gambling Follow-Up Scale de Castro et al, 2005 [34]
The Gambling Motives Questionnaire Stewart et al, 2008 [52]
The Gambling Motives Questionnaire Financial Schellenberg et al, 2015 [47]
The Gambling Passion Scale Rousseau et al, 2002 [25]
The Gambling Pathways Questionnaire Nower et al, 2016 [48]
The Gambling Problem Index Neighbors et al, 2002 [24]
The Gambling Quantity and Perceived Norms Neighbors et al, 2002 [24]
The Gambling Readiness to Change Questionnaire Raylu et al, 2004 [30]
The Gambling Refusal Self-Efficacy Questionnaire Casey et al, 2008 [35]
The Gambling Symptom Assessment Scale Kim et al, 2009 [38]
The Gambling Urge Scale Raylu et al, 2004 [59]
The Gambling-Related Cognition Scale Raylu et al, 2004 [30]
The Inventory of Gambling Situations Turner et al, 2013 [45]
The Lie/Bet Johnson et al, 1997 [21]
The Maroondah Assessment Profile for Problem Gamblinga Shek et al, 2009 [60]
The Massachusetts Gambling Screen Shaffer et al, 1994 [19]
The NODSb-CLIPc Volberg et al, 2011 [12]
The NODS-PERCd Volberg et al, 2011 [12]
The NORCe Diagnostic Screen for Gambling Problems Gerstein et al, 1999 [22]
The NORC Diagnostic Screen for Gambling Problems Self-Administered Gerstein et al, 1999 [22]
The Pathological Gambling Behavioral Self-Report Scale Myrseth et al, 2011 [61]
The Problem and Pathological Gambling Measure Willimas et al, 2013 [46]
The Problem Gamble Research and Treatment Centre Screen —f
The Problem Gambling Severity Index Ferris et al, 2001 [23]
The Scale of Gambling Choices Baron et al, 1995 [20]
The South Oaks Gambling Screen Lesieur et al, 1987 [18]
The South Oaks Gambling Screen Short Room et al, 1999 [62]
The South Oaks Gambling Screen-Revised Abbott et al, 1990 [63]
The Sydney Laval Gambling Scale Blaszczynski et al, 2008 [51]
The Temptations for Gambling Questionnaire Holub et al, 2005 [31]
The Victorian Gambling Screen Tolchard et al, 2010 [42]
The Yale Brown Obsessive Compulsive Scale adapted for Pathological Gambling Pallanti et al, 2005 [32]

aThis measure was excluded from the item pool because it was not possible to obtain the instrument.

bNODS: National Opinion Research Center Diagnostic Screen for Gambling Problems.

cCLIP: Loss of Control, Lying, and Preoccupation. 

dPERC: The National Opinion Research Center Diagnostic Screen for Gambling Problems - Preoccupation, Escape, Risked Relationships, and Chasing. 

eNORC: National Opinion Research Center. 

fPublished reference not found.

Item categorization and item selection for the Gambling Disorder Identification Test (G-DIT). The number of items is provided within parentheses. *Five items were lost in the initial categorization. **Interrater recategorization. ***Main Delphi. ****Time gambled and gambling behavior were merged to fit the G-DIT domains. DSM-5: Diagnostic and Statistical Manual of Mental Disorders, 5th edition.

 The Delphi Questionnaire and Rounds

Stakeholders were instructed to log on to the online questionnaire where they first read information about the study and electronically signed an informed consent form and to provide data on demographic characteristics including gender, country, number of years engaged in gambling-related work, and profession. Thereafter, the stakeholders viewed the proposed items in the measure. The items were listed randomly to avoid assigning any order of importance to the items. For each item, the stakeholders were instructed to provide feedback on the psychometric relevance and accuracy, semantic structure, and multiple-choice alternatives. In addition, the stakeholders were asked to rate each item on a scale of 1-9, where scores of 1-3 were considered “not important for inclusion,” 4-6 were considered “important but not critical,” and 7-9 were considered “critical for inclusion.” Further, an open-text field was provided with each item, through which the stakeholders could provide additional feedback or information; for example, important psychometric findings that were previously not noted by our research group. A rationale for each item shown from a psychometric perspective was presented; for example, “Item 5. How often do you gamble to win back money you lost? Never, Less than monthly, Monthly, Weekly, or Daily or almost daily.” The rationale for inclusion of this item is that “Chasing losses” is a key symptom in the diagnostic criteria of GD. A recent latent class analysis of data found that “the main diagnostic item serving to discriminate recreational from problem gamblers was endorsement of ‘chasing losses’” [10].

The Delphi survey was repeated in a second round. The importance of completing both rounds was emphasized to the stakeholders in the study information. After completion of Round 1, all stakeholders were invited to Round 2, where they were asked to respond to the questionnaire again. In addition to the previously described content, the stakeholders were presented with an anonymous summary of the other stakeholders’ responses. Using this information, each expert was asked to reflect on their own rating in relation to the overall group rating and rate each item again. After Round 2, the results of the Delphi analysis were summarized.

 Consensus Meeting

After the end of the Delphi rounds, a consensus meeting was held with a subgroup of international researchers attending the Alberta Gambling Research Institute’s 17th Annual Conference. The results from the Delphi were presented and discussed, and a consensus was reached to determine the final G-DIT item structure. To review the results and adjust the G-DIT measure accordingly, subsequent consensus meetings were held on issues tabled at the international consensus meeting. Participants at these meetings were the authors of the present article and two Swedish participants of the international consensus meeting. At the end of the consensus process, the G-DIT was also translated into Swedish using a back-translation procedure [72].

 Think Aloud Procedure

Swedish individuals (n=12) with problem gambling behavior were recruited from treatment-seeking and self-help groups. The inclusion criteria were willingness to participate in the study and personal experience of gambling problems. The participants provided feedback according to the “think aloud” procedure [73,74]. They were instructed in advance to think aloud “as if alone in the room.” First, the participants practiced the procedure when presented with an instruction text. Subsequently, they were presented with each item in the draft version of the Swedish G-DIT. Their comments were noted by the interviewer, who otherwise did not intervene, except to provide reminders to think aloud. The results of the interviews were analyzed using content analysis. Thereafter, the G-DIT was adjusted further to increase face validity of the measure.

 Psychometric Evaluation in Treatment-Seeking and Population Cohorts

In the final step of the study protocol, the psychometric properties of the G-DIT will be evaluated in relation to the DSM-5 diagnostic criteria for GD [1] and other gambling instruments through survey data and clinical interviews. Data will be collected from treatment-seeking and self-help group samples as well as population samples including people with recreational gambling behavior in Sweden (n=600). The inclusion criteria for treatment-seeking and self-help group participants will be a total score of ≥3 on the Problem Gambling Severity Index (PGSI) [23], 18-85 years of age, ability to read and write Swedish, and not fulfilling the criteria for a manic episode. The inclusion criteria for the population sample will be 18-85 years of age and the ability to read and write Swedish. The procedure will first be piloted with a cohort of participants seeking treatment for problem gambling (n=80), after which additional adjustment of the G-DIT, such as further reduction of items, may be performed.

 Results

Funding sources for the G-DIT project include the Swedish Research Council for Health, Working Life and Welfare (Grant no. 2016-07091), covering a 6-year program grant entitled REGAPS, and development funds from the Stockholm Health Care Services, Stockholm County Council, for identification and treatment of problem gambling. As of November 2018, steps 1-3 have been completed, and step 4 is underway.

 Discussion

This article describes a study protocol to develop a new measure for the assessment of problem gambling. We describe methods for item generation, instrument development, and procedures for testing the face and construct validity by collecting feedback from expert researchers and participants with problem gambling behavior. This study will set the foundation for a subsequent psychometric study that will aim to evaluate the psychometric properties of the G-DIT in relation to existing instruments, clinical interviews, and self-reported DSM-5 criteria among Swedish individuals with problem gambling behavior from treatment-seeking and self-help groups samples as well as population samples including people with recreational gambling behaviors.

This study protocol has several strengths. First, our extensive literature search identified a large number of existing gambling measures. Our overview indicated that no single existing measure seemed to adequately fulfill the recommendations of the Banff consensus. Second, only a few measures have been validated by the DSM-5 diagnostic criteria for GD. Third, many existing measures include item responses with generalized multiple or dichotomous “yes” or “no” response options rather than specific behavior or time frequencies. Fourth, the use of digital platforms in this study facilitates broad national and international collaborations in emerging research fields such as problem gambling. Our scope for recruiting expert researchers was wide. Implementation of a Delphi study early in the psychometric development process will contribute to the face and construct validity of the final measure. Through the Delphi process, several key problematic issues for measuring gambling-related content were identified and will be discussed in the forthcoming publication. Our systematic procedure will contribute to the establishment of public health guidelines for gambling behavior, similar to the guidelines for alcohol consumption currently available in many countries.

The final G-DIT will consist of three domains: gambling consumption, symptom severity, and negative consequences. In addition, an appendix on expenditure and gambling types will be included. We believe the G-DIT will complement existing screening scales in upcoming intervention trials among community and treatment-seeking groups and prove useful as a standard outcome measure for change in problem gambling behavior. An additional potential area of use is the identification of problem gambling in clinical settings.

 Abbreviations CLIP

Loss of Control, Lying, and Preoccupation 

 DSM-5

Diagnostic and Statistical Manual of Mental Disorders, 5th edition

 DSM-IV

Diagnostic and Statistical Manual of Mental Disorders, 4th edition

 G-DIT

Gambling Disorder Identification Test

 NODS

National Opinion Research Center Diagnostic Screen for Gambling Problems

 NORC

National Opinion Research Center 

 PERC

The National Opinion Research Center Diagnostic Screen for Gambling Problems - Preoccupation, Escape, Risked Relationships, and Chasing 

 REGAPS

Responding to and Reducing Gambling Problems - Studies in Help-Seeking, Measurement, Comorbidity, and Policy Impacts

None declared.

AHB, OM, RV, KS, PW, and VM conceived the study. OM compiled and categorized the first item pool. AHB, KS, and VM recategorized the relevant G-DIT items. OM wrote the first manuscript draft, and AHB revised the second draft. RV provided expert guidance on the methodology as an experienced gambling researcher, developer of existing gambling measures, and member of the REGAPS research program. All authors participated in the Delphi pilot rounds. All authors edited and contributed to subsequent manuscript drafts.

 American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders, 5th Edition: DSM-5 2013 Washington American Psychiatric Association Dowling NA Merkouris SS Lorains FK Interventions for comorbid problem gambling and psychiatric disorders: Advancing a developing field of research Addict Behav 2016 07 58 21 30 10.1016/j.addbeh.2016.02.012 26900888 S0306-4603(16)30037-5 Rychtarik RG McGillicuddy NB Preliminary evaluation of a coping skills training program for those with a pathological-gambling partner J Gambl Stud 2006 06 22 2 165 78 10.1007/s10899-006-9008-6 16841184 Newman SC Thompson AH A population-based study of the association between pathological gambling and attempted suicide Suicide Life Threat Behav 2003 33 1 80 7 12710543 American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders DSM-IV 1994 Washington American Psychiatric Press Inc Hasin DS O'Brien CP Auriacombe M Borges G Bucholz K Budney A Compton WM Crowley T Ling W Petry NM Schuckit M Grant BF DSM-5 criteria for substance use disorders: recommendations and rationale Am J Psychiatry 2013 08 170 8 834 51 10.1176/appi.ajp.2013.12060782 23903334 1722068 PMC3767415 Gooding P Tarrier N A systematic review and meta-analysis of cognitive-behavioural interventions to reduce problem gambling: hedging our bets? Behav Res Ther 2009 07 47 7 592 607 10.1016/j.brat.2009.04.002 19446287 S0005-7967(09)00087-4 Walker M Toneatto T Potenza MN Petry N Ladouceur R Hodgins DC el-Guebaly N Echeburua E Blaszczynski A A framework for reporting outcomes in problem gambling treatment research: the Banff, Alberta Consensus Addiction 2006 04 101 4 504 11 10.1111/j.1360-0443.2005.01341.x 16548930 ADD1341 Pallesen S Mitsem M Kvale G Johnsen B Molde H Outcome of psychological treatments of pathological gambling: a review and meta-analysis Addiction 2005 10 100 10 1412 22 10.1111/j.1360-0443.2005.01204.x 16185203 ADD1204 Chamberlain SR Stochl J Redden SA Odlaug BL Grant JE Latent class analysis of gambling subtypes and impulsive/compulsive associations: Time to rethink diagnostic boundaries for gambling disorder? Addict Behav 2017 12 72 79 85 10.1016/j.addbeh.2017.03.020 28384607 S0306-4603(17)30131-4 PMC5457805 Stinchfield R McCready J Turner NE Jimenez-Murcia S Petry NM Grant J Welte J Chapman H Winters KC Reliability, Validity, and Classification Accuracy of the DSM-5 Diagnostic Criteria for Gambling Disorder and Comparison to DSM-IV J Gambl Stud 2016 09 32 3 905 22 10.1007/s10899-015-9573-7 26408026 10.1007/s10899-015-9573-7 PMC4993799 Volberg R Williams R University of Lethbridge 2011 2018-11-24 Developing a Brief Problem Gambling Screen Using Clinically Validated Samples of At-Risk, Problem and Pathological Gamblers https://opus.uleth.ca/handle/10133/2561 Saunders JB Aasland OG Babor TF de la Fuente JR Grant M Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption--II Addiction 1993 06 88 6 791 804 8329970 Berman AH Bergman H Palmstierna T Schlyter F Evaluation of the Drug Use Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample Eur Addict Res 2005 11 1 22 31 10.1159/000081413 15608468 81413 Caler K Garcia J Nower L Assessing Problem Gambling: a Review of Classic and Specialized Measures Curr Addict Rep 2016 10 5 3 4 437 444 10.1007/s40429-016-0118-7 Dowling NA Merkouris SS Manning V Volberg R Lee SJ Rodda SN Lubman DI Screening for problem gambling within mental health services: a comparison of the classification accuracy of brief instruments Addiction 2018 06 113 6 1088 1104 10.1111/add.14150 29274182 Pickering D Keen B Entwistle G Blaszczynski A Measuring treatment outcomes in gambling disorders: a systematic review Addiction 2018 03 113 3 411 426 10.1111/add.13968 28891116 PMC5836978 Lesieur HR Blume SB The South Oaks Gambling Screen (SOGS): a new instrument for the identification of pathological gamblers Am J Psychiatry 1987 09 144 9 1184 8 10.1176/ajp.144.9.1184 3631315 Shaffer HJ Labrie R Scanlan KM Cummings TN Pathological gambling among adolescents: Massachusetts Gambling Screen (MAGS) J Gambl Stud 1994 12 10 4 339 62 10.1007/BF02104901 24234969 Baron E Dickerson M Blaszczynski A O'Connor J The Scale of Gambling Choices: Preliminary Development of an Instrument to Measure Impaired Control of Gambling Behaviour High stakes in the nineties 1995 National Association of Gambling Studies Curtin University Press 153 68 Johnson EE Hamer R Nora RM Tan B Eisenstein N Engelhart C The Lie/Bet Questionnaire for screening pathological gamblers Psychol Rep 1997 02 80 1 83 8 10.2466/pr0.1997.80.1.83 9122356 Gerstein D Volberg R Toce M Harwood H Johnson R Buie T Christiansen E Chuchro L Cummings W Engelman L National Opinion Research Center, University of Chicago 1999 2018-11-23 Gambling impact and behavior study: Report to the national gambling impact study commission http://www.norc.org/PDFs/publications/GIBSFinalReportApril1999.pdf Ferris J Wynne H The Canadian Problem Gambling Index: Final Report 2018-11-27 http://www.ccgr.ca/en/projects/resources/CPGI-Final-Report-English.pdf Neighbors C Lostutter TW Larimer ME Takushi RY Measuring gambling outcomes among college students J Gambl Stud 2002 18 4 339 60 12514914 453970 PMC1797803 Rousseau FL Vallerand RJ Ratelle CF Mageau GA Provencher PJ Passion and gambling: on the validation of the Gambling Passion Scale (GPS) J Gambl Stud 2002 18 1 45 66 12050847 Hodgins DC Makarchuk K Trusting problem gamblers: reliability and validity of self-reported gambling behavior Psychol Addict Behav 2003 09 17 3 244 8 10.1037/0893-164X.17.3.244 14498819 2003-07737-008 May RK Whelan JP Steenbergh TA Meyers AW The gambling self-efficacy questionnaire: an initial psychometric evaluation J Gambl Stud 2003 19 4 339 57 14634297 470976 Hodgins DC Using the NORC DSM Screen for Gambling Problems as an outcome measure for pathological gambling: psychometric evaluation Addict Behav 2004 11 29 8 1685 90 10.1016/j.addbeh.2004.03.017 15451138 S0306460304000735 Hodgins D Peden N Makarchuk K Self-efficacy in pathological gambling treatment outcome: development of a gambling abstinence self-efficacy scale (GASS) International Gambling Studies 2004 11 4 2 99 108 10.1080/14459790412331296947 Raylu N Oei TPS The Gambling Related Cognitions Scale (GRCS): development, confirmatory factor validation and psychometric properties Addiction 2004 06 99 6 757 69 10.1111/j.1360-0443.2004.00753.x 15139874 ADD753 Holub A Hodgins D Peden N Development of the temptations for gambling questionnaire: A measure of temptation in recently quit gamblers Addiction Research & Theory 2009 07 11 13 2 179 191 10.1080/16066350412331314902 Pallanti S DeCaria CM Grant JE Urpe M Hollander E Reliability and validity of the pathological gambling adaptation of the Yale-Brown Obsessive-Compulsive Scale (PG-YBOCS) J Gambl Stud 2005 21 4 431 43 10.1007/s10899-005-5557-3 16311876 Saiz-Ruiz J Blanco C Ibáñez A Masramon X Gómez MM Madrigal M Díez T Sertraline treatment of pathological gambling: a pilot study J Clin Psychiatry 2005 01 66 1 28 33 15669885 de Castro V Fuentes D Tavares H The gambling follow-up scale: development and reliability testing of a scale for pathological gamblers under treatment Can J Psychiatry 2005 02 50 2 81 6 10.1177/070674370505000202 15807223 Casey LM Oei TPS Melville KM Bourke E Newcombe PA Measuring self-efficacy in gambling: the Gambling Refusal Self-Efficacy Questionnaire J Gambl Stud 2008 06 24 2 229 46 10.1007/s10899-007-9076-2 17849178 Goodyear-Smith F Coupe NM Arroll B Elley CR Sullivan S McGill A Case finding of lifestyle and mental health disorders in primary care: validation of the 'CHAT' tool Br J Gen Pract 2008 01 58 546 26 31 10.3399/bjgp08X263785 18186993 PMC2148235 Wickwire EM Burke RS Brown SA Parker JD May RK Psychometric evaluation of the National Opinion Research Center DSM-IV Screen for Gambling Problems (NODS) Am J Addict 2008 17 5 392 5 10.1080/10550490802268934 18770081 902145949 Kim SW Grant JE Potenza MN Blanco C Hollander E The Gambling Symptom Assessment Scale (G-SAS): a reliability and validity study Psychiatry Res 2009 03 31 166 1 76 84 10.1016/j.psychres.2007.11.008 19200607 S0165-1781(07)00402-7 PMC3641525 Toce-Gerstein M Gerstein DR Volberg RA The NODS-CLiP: a rapid screen for adult pathological and problem gambling J Gambl Stud 2009 12 25 4 541 55 10.1007/s10899-009-9135-y 19603259 PMC3670681 Young MM Wohl MJA The Gambling Craving Scale: Psychometric validation and behavioral outcomes Psychol Addict Behav 2009 09 23 3 512 22 10.1037/a0015043 19769435 2009-14441-012 Gebauer L LaBrie R Shaffer HJ Optimizing DSM-IV-TR classification accuracy: a brief biosocial screen for detecting current gambling disorders among gamblers in the general household population Can J Psychiatry 2010 02 55 2 82 90 10.1177/070674371005500204 20181303 Tolchard B Battersby MW The Victorian Gambling Screen: reliability and validation in a clinical population J Gambl Stud 2010 12 26 4 623 38 10.1007/s10899-009-9172-6 20035440 Volberg RA Munck IM Petry NM A quick and simple screening method for pathological and problem gamblers in addiction programs and practices Am J Addict 2011 20 3 220 7 10.1111/j.1521-0391.2011.00118.x 21477050 PMC3076109 Rockloff MJ Validation of the Consumption Screen for Problem Gambling (CSPG) J Gambl Stud 2012 06 28 2 207 16 10.1007/s10899-011-9260-2 21830133 Turner NE Littman-Sharp N Toneatto T Liu E Ferentzy P Centre for Addiction and Mental Health Inventory of Gambling Situations: Evaluation of the Factor Structure, Reliability, and External Correlations Int J Ment Health Addict 2013 11 526 545 10.1007/s11469-013-9446-1 24707240 9446 PMC3971445 Williams RJ Volberg RA The classification accuracy of four problem gambling assessment instruments in population research International Gambling Studies 2013 10 09 14 1 15 28 10.1080/14459795.2013.839731 Schellenberg B McGrath D Dechant K The Gambling Motives Questionnaire financial: factor structure, measurement invariance, and relationships with gambling behaviour International Gambling Studies 2015 11 02 16 1 1 16 10.1080/14459795.2015.1088559 Nower L Blaszczynski A Development and validation of the Gambling Pathways Questionnaire (GPQ) Psychol Addict Behav 2017 02 31 1 95 109 10.1037/adb0000234 27936817 2016-59850-001 Breen R Zuckerman M `Chasing' in gambling behavior: personality and cognitive determinants Personality and Individual Differences 1999 12 27 6 1097 1111 10.1016/S0191-8869(99)00052-5 Sullivan S Don’t Let an Opportunity Go by: Validation of the EIGHT Gambling Screen Int J Ment Health Addiction 2007 4 17 5 4 381 389 10.1007/s11469-007-9064-x Blaszczynski A Ladouceur R Moodie C The Sydney Laval Universities Gambling Screen: Preliminary data Addiction Research & Theory 2009 07 11 16 4 401 411 10.1080/16066350701699031 Stewart SH Zack M Development and psychometric evaluation of a three-dimensional Gambling Motives Questionnaire Addiction 2008 07 103 7 1110 7 10.1111/j.1360-0443.2008.02235.x 18554344 ADD2235 Toneatto T Reliability and Validity of the Gamblers Anonymous Twenty Questions J Psychopathol Behav Assess 2008 1 5 30 1 71 78 10.1007/s10862-007-9070-0 Williams RJ Volberg RA Stevens RM Williams LA Arthur JN University of Lethbridge Research Repository: OPUS 2017 2018-11-23 The definition, dimensionalization, and assessment of gambling participation: Report prepared for the Canadian Consortium for Gambling Research https://opus.uleth.ca/bitstream/handle/10133/4838/Williams%20the%20defn,%20dimensionalization%20and%20assessment.pdf Volberg RA Banks SM A review of two measures of pathological gambling in the United States J Gambl Stud 1990 06 6 2 153 63 10.1007/BF01013495 24242856 Steenbergh TA Meyers AW May RK Whelan JP Development and validation of the Gamblers' Beliefs Questionnaire Psychol Addict Behav 2002 06 16 2 143 9 12079253 Jackson AC Francis KL Byrne G Christensen DR Leisure Substitution and Problem Gambling: Report of a Proof of Concept Group Intervention Int J Ment Health Addiction 2012 8 17 11 1 64 74 10.1007/s11469-012-9399-9 McInnes A Hodgins D Holub A The Gambling Cognitions Inventory: scale development and psychometric validation with problem and pathological gamblers International Gambling Studies 2014 06 27 14 3 410 431 10.1080/14459795.2014.923483 Raylu N Oei TPS The gambling urge scale: development, confirmatory factor validation, and psychometric properties Psychol Addict Behav 2004 06 18 2 100 5 10.1037/0893-164X.18.2.100 15238051 2004-15666-002 Shek DTL Chan EML Assessment of problem gambling in a Chinese context: the Chinese G-MAP ScientificWorldJournal 2009 07 02 9 548 56 10.1100/tsw.2009.42 19578712 PMC5823089 Myrseth H Molde H Støylen I Johnsen B Holsten F Pallesen S A pilot study of CBT versus escitalopram combined with CBT in the treatment of pathological gamblers International Gambling Studies 2011 04 11 1 121 141 10.1080/14459795.2011.559647 Room R Turner NE Ialomiteanu A Community effects of the opening of the Niagara casino Addiction 1999 10 94 10 1449 66 10790898 Abbott MW Volberg R Department of Internal Affairs 1990 2018-11-24 A Report on Phase Two of the 1999 National Prevalence Survey: Problem and Non-Problem Gamblers in New Zealand https://www.dia.govt.nz/diawebsite.nsf/wpg_URL/Resource-material-Our-Research-and-Reports-New-Zealand-Gaming-Survey?OpenDocument Fleiss J Measuring nominal scale agreement among many raters Psychological Bulletin 1971 76 5 378 382 10.1037/h0031619 R Core Team R: A language and environment for statistical computing 2013 2018-11-24 Vienna, Austria R Foundation for Statistical Computing https://www.r-project.org/ Sinha IP Smyth RL Williamson PR Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies PLoS Med 2011 01 25 8 1 e1000393 10.1371/journal.pmed.1000393 21283604 PMC3026691 Blackwood B Ringrow S Clarke M Marshall J Rose L Williamson P McAuley D Core Outcomes in Ventilation Trials (COVenT): protocol for a core outcome set using a Delphi survey with a nested randomised trial and observational cohort study Trials 2015 08 20 16 368 10.1186/s13063-015-0905-9 26289560 10.1186/s13063-015-0905-9 PMC4545990 Shorter GW Heather N Bray JW Giles EL Holloway A Barbosa C Berman AH O'Donnell AJ Clarke M Stockdale KJ Newbury-Birch D The 'Outcome Reporting in Brief Intervention Trials: Alcohol' (ORBITAL) framework: protocol to determine a core outcome set for efficacy and effectiveness trials of alcohol screening and brief intervention Trials 2017 12 22 18 1 611 10.1186/s13063-017-2335-3 29273070 10.1186/s13063-017-2335-3 PMC5741954 Cowlishaw S Merkouris S Dowling N Anderson C Jackson A Thomas S Psychological therapies for pathological and problem gambling Cochrane Database Syst Rev 2012 11 14 11 CD008937 10.1002/14651858.CD008937.pub2 23152266 Petry NM Ginley MK Rash CJ A systematic review of treatments for problem gambling Psychol Addict Behav 2017 12 31 8 951 961 10.1037/adb0000290 28639817 2017-27176-001 PMC5714688 SurveyXact 2018 2018-11-27 Ramböll Management Consulting https://www.surveyxact.se/ KuliÅ› D Whittaker C Greimel E Bottomley A Koller M EORTC Quality of Life Group Reviewing back translation reports of questionnaires: the EORTC conceptual framework and experience Expert Rev Pharmacoecon Outcomes Res 2017 12 17 6 523 530 10.1080/14737167.2017.1384316 28974101 Ericsson K Simon H Verbal reports as data Psychological Review 1980 87 3 215 251 10.1037/0033-295X.87.3.215 Boren T Ramey J Thinking aloud: reconciling theory and practice IEEE Trans. Profess. Commun 2000 43 3 261 278 10.1109/47.867942