FULL-LENGTH REPORT Journal of Behavioral Addictions 7(4), pp.

939–952 (2018)
DOI: 10.1556/2006.7.2018.111
First published online November 14, 2018

Treatment efficacy of a specialized psychotherapy program for Internet

Gaming Disorder
ALEXANDRA TORRES-RODRÍGUEZ1*, MARK D. GRIFFITHS2, XAVIER CARBONELL1 and URSULA OBERST1
1
Psychology Department, FPCEE Blanquerna, Universitat Ramon Llull, Barcelona, Spain
2
International Gaming Research Unit, Psychology Department, Nottingham Trent University, Nottingham, UK

(Received: May 10, 2018; revised manuscript received: July 26, 2018; second revised manuscript received: October 10, 2018;
accepted: October 14, 2018)

Background and aims: Internet Gaming Disorder (IGD) has become health concern around the world, and specialized
health services for the treatment of IGD are emerging. Despite the increase in such services, few studies have
examined the efficacy of psychological treatments for IGD. The primary aim of this study was to assess the efficacy of
a specialized psychotherapy program for adolescents with IGD [i.e., the “Programa Individualizado Psicoterapéutico
para la Adicción a las Tecnologías de la Información y la Comunicación” (PIPATIC) program]. Methods: The sample
comprised 31 adolescents (aged 12–18 years) from two public mental health centers who were assigned to either the
(a) PIPATIC intervention experimental group or (b) standard cognitive-behavioral therapy (CBT) control group. The
interventions were assessed at pre-, middle-, and post-treatment phases, as well as a 3-month assessment was carried
out after completing the interventions. Results: No significant differences between either group in the pre-treatment
phase were found. Relating to the different interventions examined, significant differences were found at pre-test and
post-test on the following variables: comorbid disorders, intrapersonal and interpersonal abilities, family relation-
ships, and therapists’ measures. Both groups experienced a significant reduction of IGD symptoms, although the
PIPATIC group experienced higher significant improvements in the remainder of the variables examined. Discussion
and conclusions: The findings suggest that PIPATIC program is effective in the treatment of IGD and its comorbid
disorders/symptoms, alongside the improvement of intra- and interpersonal abilities and family relationships.
However, it should also be noted that standard CBT was also effective in the treatment of IGD. Changing the
focus of treatment and applying an integrative focus (including the addiction, the comorbid symptoms, intra- and
interpersonal abilities, and family psychotherapy) appear to be more effective in facilitating adolescent behavior
change than CBT focusing only on the IGD itself.

Keywords: Internet Gaming Disorder, adolescence, video game, gaming disorder treatment, cognitive-behavioral
therapy

INTRODUCTION Although the study of Internet Gaming Disorder (IGD)

has grown markedly in recent years, few studies have
The excessive and problematic use of technology has led to examined the efficacy of psychological treatments and
increasing public health concerns around the world (World pharmacological interventions for IGD (Griffiths, 2008;
Health Organization, 2016). Consequently, specialized King, Delfabbro, Griffiths, & Gradisar, 2011; King et al.,
health services have emerged with outpatient treatment 2017). Most of the studies, to date, have been carried out in
for various technological addictions (Martín-Fernández, Asian countries where the prevalence of IGD appears to be
Matalí, García-Sánchez, Pardo, & Castellano-Tejedor, higher than other areas of the world (Du, Jiang, & Vance,
2016; Young, 2007). There has also been recognition 2010; Kim, 2008; King et al., 2017). Furthermore, system-
that excessive maladaptive use of online video games atic reviews related to IGD treatments have expanded the
can lead to associated psychological problems for a nomenclature to include internet addiction, since this term is
small minority of individuals, particularly adolescents commonly used by Asian countries where most of the
(e.g., Ferguson, Coulson, & Barnett, 2011; Kuss & treatment studies carried out (King et al., 2017; Winkler,
Griffiths, 2012). This has led to the introduction of Dörsing, Rief, Shen, & Glombiewski, 2013). However, very
treatment services for problems related to video game
playing among adolescents as a consequence of the risks * Corresponding author: Alexandra Torres-Rodríguez; Psychology
and vulnerabilities related to this life stage (e.g., Kuss & Department, FPCEE Blanquerna, Universitat Ramon Llull, 34 Císter
Griffiths, 2012; Schneider, King, & Delfabbro, 2017; Street, Barcelona 08022, Spain; Phone: +34 93 253 30 00; Fax: +34
Torres-Rodríguez & Carbonell, 2017). 93 253 30 32; E-mail: [email protected]

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License,
which permits unrestricted use, distribution, and reproduction in any medium for non-commercial purposes, provided the original author and
source are credited, a link to the CC License is provided, and changes – if any – are indicated.

ISSN 2062-5871 © 2018 The Author(s)

 Torres-Rodríguez et al.

few treatment programs have been developed in the USA or different diagnostic tool, (f) the randomization and presence
European countries (King et al., 2017; Thorens et al., 2014; of control groups were scarce, and (g) many studies focused
Wölfling, Beutel, Dreier, & Muller, 2014; Young, 2007, on the assessment of gaming symptoms leaving aside the
2013). Furthermore, there has been little evaluation con- diagnostic changes and/or the comorbid symptoms. The few
cerning the effects of different psychological interventions published studies present many limitations (King et al.,
with children and adolescents (King, Delfabbro, & Griffiths, 2017) and comprise many challenges that hinder the rigor-
2013; King & Delfabbro, 2017). Consequently, there is an ous application of CONSORT guidelines guaranteeing the
evident need to develop and evaluate IGD treatments for quality of clinical trials (Schulz, Altman, & Moher, 2010).
European youth. Consequently, there is an evident need to develop and
Based on the peer-reviewed literature, cognitive-behavior evaluate comprehensive and specialized treatments for IGD
therapy (CBT) appears to be the most commonly applied among European children and adolescents. This study
treatment for online addictions including IGD (Greenfield, contributes to such scientific need by evaluating a compre-
1999; Griffiths & Meredith, 2009; Kaptsis, King, Delfabbro, hensive and specialized IGD treatment program applied to a
& Gradisar, 2016; King, Delfabbro, & Griffiths, 2010; King western youth population. This study also compared the
et al., 2011; Young, 2007, 2013). Along these lines, most of treatment efficacy of two psychological treatments among a
the therapeutic recommendations of CBTs for online sample of treatment-seeking adolescents: a specialized psy-
addictions, such as IGD, are based on substance abuse chotherapy program for adolescents with IGD (i.e., the
treatment (Huang, Li, & Tao, 2010; King et al., 2011), PIPATIC program: “Programa Individualizado Psicotera-
including stimulus control, learning appropriate coping péutico para la Adicción a las Tecnologías de la Información
responses, self-monitoring strategies, cognitive restructuring, y la Comunicación”) and standard CBT. It was expected that
problem solving related to addiction, and withdrawal regula- PIPATIC program would lead to improvement in both
tion techniques with exposure (Griffiths & Meredith, 2009; psychotherapeutically focused areas and reduced symptoms
King et al., 2010; Young, 2007). Previous studies have of IGD, whereas the CBT program would only lead to
suggested an integrative approach for specialized treatments reduction in IGD symptoms. The study provides useful
of IGD due to the high presence of comorbid disorders and empirical and clinical data about the effects and efficacy
associated problems, as well as interventions that address low of a newly developed IGD treatment program and attempts
self-esteem, poor social skills, low emotional intelligence, to overcome some of the limitations in previously reported
and family dysfunction (among others) in order to address the IGD treatment studies. Compared to previous reports of
disorder more holistically. In particular, previous IGD studies IGD treatment, this study included: (a) a detailed description
have reported psychological problems including affective of the treatment program (outlined in previous papers by the
instability, low self-esteem, insecure personality, shyness, present authors; see Torres-Rodríguez & Carbonell, 2017;
loneliness, limited leisure activities, family deficits, maladap- Torres-Rodríguez, Griffiths, & Carbonell, 2017); (b) a
tive coping styles, lower social competence, and lower school clinical sample of European adolescents; (c) an intervention
performance (e.g., Gentile et al., 2011; Kim, Namkoong, Ku, control group (standard CBT); (d) comparison of symptoms
& Kim, 2008; King & Delfabbro, 2017; Kuss, van Rooij, across four different assessment points (pre-, middle-, post-
Shorter, Griffiths, & van de Mheen, 2013; Lemmens, treatment, and follow-up); (e) the use of clinical interviews,
Valkenburg, & Peter, 2011; Liebert, Lo, Ph, Wang, & Fang, alongside validated and reliable instruments for use with
2005; Rehbein, Psych, Kleimann, Mediasci, & Mößle, 2010; participants, relatives, and therapists; and (f) assessments
Schneider et al., 2017; Tejeiro, Gómez-Vallecillo, Pelegrina, and evaluations by trained clinical psychologists comprising
Wallace, & Emberley, 2012). Other disorders associated with clinical interviews, administering of reliable psychometric
symptoms of IGD include anxiety disorders, depression, instruments, and rigorous assessment of comorbid symp-
suicidal ideation, behavioral disorders, social phobia, autism toms and problems associated with IGD.
spectrum disorder, attention-deficit hyperactivity disorder, The primary goal of the newly developed PIPATIC
obsessive–compulsive disorder, and personality disorders program (see Torres-Rodríguez et al., 2017 for an in-depth
(e.g., Andreassen et al., 2016; Chan & Rabinowitz, 2006; description) is to offer specialized psychotherapy for adoles-
Ferguson et al., 2011; Gentile et al., 2011; Han, Lee, Shi, & cents with symptoms of IGD and comorbid disorders. The
Renshaw, 2014; Kelleci & Inal, 2010; Kim et al., 2006; program comprises six therapeutic work modules, in turn
Ko et al., 2006; Shapira, Goldsmith, Keck, Khosla, & made up of more specific subobjectives. Following previous
McElroy, 2000). studies (Hansen & Lambert, 2003; Kadera, Lambert, &
A recent systematic review of 30 IGD treatment studies Andrews, 1996; Lambert & Bergin, 1994) – and in order
(King et al., 2017) suggested that CBT treatment had a large to ensure therapeutic changes in patients – the duration of the
empirical base compared to other interventions. Neverthe- program was 6 months (22 sessions of approximately 45-min
less, the review reported a number of limitations regarding duration). The intervention, based on a CBT approach,
the studies evaluated. More specifically, (a) one-third of the employed crosscutting techniques and resources commonly
studies did not employ control groups, (b) there was a lack used in psychotherapy (Hofmann & Barlow, 2014; Kleinke,
of sample size justification and information about recruit- 1994; Laska, Gurman, & Wampold, 2014). The design of the
ment and intervention, (c) there were inconsistencies in PIPATIC program integrates several areas of intervention
assessment of treatment outcomes and a lack of follow-up structured into six modules: (a) psychoeducation and
in several studies, (d) most of the psychological interven- motivation, (b) addiction treatment as usual (TAU) adapted
tions focused on CBT programs often lacking detail in the to IGD, (c) intrapersonal, (d) interpersonal, (e) family inter-
descriptions of the treatments, (e) many studies employed vention, and (f) development of new lifestyle.

940 | Journal of Behavioral Addictions 7(4), pp. 939–952 (2018)

 Internet Gaming Disorder treatment efficacy

METHODS by asking about the approximate number of hours that were

spent gaming during weekdays and the weekend (holiday
Participants periods were excluded).
Ability to stop gaming: Participants and their families
The sample originally comprised 58 adolescents who vol- self-reported the ability to stop gaming using a simple Likert
untarily sought treatment for their problematic video game scale (1–5). More specifically, they were asked how difficult
playing in two public mental health centers in the Barcelona it was to stop gaming to do more important activities where
metropolitan area (Spain) during the 18-month period when 1 was “never having difficulties to stop their gaming” and
the study was carried out. Of these, 12 participants were 5 was “always having problems to stop gaming.”
considered as lost (because they did not return to the Self-awareness of engagement in gaming: Participants
treatment center after a first visit) and 12 more participants and their families were asked to what extent they were
were excluded for not meeting the inclusion criteria of this engaged in gaming using a simple Likert scale (1–10),
study (i.e., four participants did not meet the inclusion where 10 was the maximum engagement (i.e., totally
criteria (a) and (b) below; one was under 12 years; two addicted to gaming).
participants presented with a severe mental disorder where Internet Gaming Disorder Test (IGD-20 Test; Pontes
the primary disorder needed treating as opposed to the IGD; et al., 2014): To assess IGD, the validated Spanish version
and five participants declined to participate in the study). Of of 20-item IGD Test was used (Fuster et al., 2016). The
these treatment-seekers, 34 met the inclusion criteria and 31 scale comprises six dimensions: salience (e.g., “I often lose
participants (aged 12–18 years) completed the treatment and sleep because of long gaming sessions”), mood modifica-
completed follow-up measures (Figure 1). One participant tion (e.g., “I never play games in order to feel better”),
dropped out the PIPATIC treatment and two participants tolerance (e.g., “I have significantly increased the amount
dropped out the standard treatment. Participants did not of time I play games over last year”), withdrawal symptoms
report any other current psychotherapy treatment. The in- (e.g., “When I am not gaming I feel more irritable”),
clusion criteria were: (a) endorsing at least five or more of conflict (e.g., “I have lost interest in other hobbies because
the nine IGD criteria according to DSM-5 (American of my gaming”), and relapse (e.g., “I would like to cut down
Psychiatric Association [APA], 2013); (b) scoring 71 or my gaming time but it is difficult to do”). All items are
more on Internet Gaming Disorder Test (IGD-20 Test; answered using a simple Likert scale (1–5, “strongly dis-
Pontes, Király, Demetrovics, & Griffiths, 2014) adapted to agree,” “disagree,” “neither agree,” “agree,” and “strongly
Spanish population (Fuster, Carbonell, Pontes, & Griffiths, agree”). The minimum and maximum scores are 20 and
2016); (c) being aged 12–18 years; (d) not having a severe 100, respectively, and those scoring 71 or more are classed
mental disorder (i.e., schizophrenia, schizoaffective disor- as having IGD. Cronbach’s α for the IDG-20 Test in this
der, and bipolar disorder) or intellectual disability; and study was .87.
(e) understanding the Spanish language. Thus, the final Comorbid symptoms. Comorbid symptoms were
sample comprised 31 male adolescents diagnosed with IGD. assessed from both family and patient perspectives. To
assess comorbid disorders as well as the behavioral and
Measures emotional functioning of the patients, the two scales from
the Achenbach System of Empirically Based Assessment
Video game habits and IGD were used. These were the Youth Self-Report for Ages 11–18
Weekly hours spent gaming: This measure was obtained Years (YSR/11-18) and the Child Behavior Checklist for
through self-reports from the participants and their relatives Ages 6–18 Years (CBCL/6-18) in their Spanish validated

Figure 1. Schematic diagram of the recruitment and the methodological process

Journal of Behavioral Addictions 7(4), pp. 939–952 (2018) | 941

 Torres-Rodríguez et al.

versions (Achenbach & Rescorla, 2001). The YSR/11-18 is assess the interpersonal abilities (e.g., “I sometimes avoid
a 112-item self-report scale completed by the adolescents, making questions because of my fear to appear stupid”).
and the CBCL/6-18 is the version for their parents. The first Scale items are assessed using a self-report 4-point Likert
part of both instruments assesses the psychosocial compe- scale that can be completed by both adolescents and adults.
tencies of adolescents across four subscales (7 items; This instrument has adequate validity and high internal
e.g., “Please list the sports you most like to take part in”), consistency (with a Cronbach’s α of .88). The young male
and the second part assesses behavioral and emotional scoring guidelines described in the manual were used for
symptoms across eight subscales (113 items; Table 5; this study (Gismero, 2000).
e.g., “I argue a lot”). For the scoring, ADM v.910 Family relationships. To assess the impact of IGD
School-Age Module for CBCL and YSR was used. Both treatment on the participant’s family relationships, the
scales have been validated for the Spanish population, and Family Discord (G) scale (e.g., “I like my home”) from
both obtaining high validity and internal consistency. For the Spanish MACI test was used. This scale is part of the
example, the internalizing and externalizing problem scales Expressed Concerns Scales of MACI test, and assesses the
have been reported as having a Cronbach’s α of .80 (Lemos, adolescent’s personal perceptions regarding family
Fidalgo, Calvo, & Menéndez, 1992). conflicts.
Intrapersonal and interpersonal abilities. The Expressed Therapist measures. To assess clinical severity and
Concern Scales of the Millon Adolescent Clinical Inventory change over time of each participant, the Spanish validated
(MACI; Millon, 1994) in its Spanish validated version were version of the Clinical Global Impression Scale – Severity of
used to assess intrapersonal abilities. More specifically, Illness (CGI-SI) was used (Busner & Targum, 2007)
(a) “identity diffusion” (e.g., “I often feel with no direction (e.g., “Considering your total clinical experience with this
in mind”), (b) “self-devaluation” (e.g., “I don’t like being particular population, how mentally ill is the patient at this
the person I have become”), and (c) “body disapproval” time?”). The severity was assessed on a scale of 1–7, with 1
(e.g., “I think I have a good body”). The MACI is a widely being normal (shows no signs of illness) and 7 being the
used validated and standardized instrument to assess ado- most extremely ill of patients. To assess the changes,
lescent personality patterns (12 subscales), expressed con- the Clinical Global Impression Scale – Global Improvement
cerns (8 subscales), and clinical syndromes (7 subscales), in (CGI-GI) was used (Busner & Targum, 2007) (e.g.,
addition to four validity (modifying) scales. This study used “Compared to your patient’s condition at time of first
the Spanish version of MACI and comprised 160 items. assessment, how much has s/he changed?”), with 1 being
Possible answers were either “true” or “false.” The stan- very much improved and 7 being very much worse. To
dardized base rate (BR) scores were used in this study; BR assess the global functioning activity, the Spanish version of
scores of 0 and 115 were selected to represent the minimum the 1-item Global Assessment of Functioning (GAF) scale
and maximum possible on each scale. This study followed was used, extracted from DSM-IV-TR (APA, 2002). This
the scoring guidelines described in the Spanish manual one item considers the psychological, social, and occupa-
(Millon, 2004). To assess the intrapersonal abilities, the tional functioning on a hypothetical continuum of mental
Expressed Concern Scales were used. Cronbach’s α reli- health illness [e.g., “Consider psychological, social and
abilities of MACI scales ranged from .73 to .91. occupational functioning on a hypothetical continuum of
The Trait Meta-Mood Scale (TMMS-24; Salovey, mental health-illness. Do not include impairment in func-
Mayer, Goldman, Turvey, & Palfai, 1995) was used to tioning due to physical (or environmental) limitations”] and
assess intrapersonal abilities. The TMMS-24 is a 24-item comes with a description of what constitutes high and low
instrument and uses a 5-point Likert scale to assess per- scores. Scores range from 0 to 100, with 0 being the
ceived emotional intelligence. The Spanish version of the persistent danger of severely hurting oneself or others and
TMMS-24 was used (Fernandez-Berrocal, Extremera, & 100 being superior functioning in a wide range of activities.
Ramos, 2004). The TMMS-24 is widely used in adolescents For instance, scores of 91–100 are described as being:
and adults, and comprises three subscales: (a) attention to “Superior functioning in a wide range of activities, life’s
emotion (participants’ self-perception of the degree to which problems never seem to get out of hand, is sought out by
they pay attention to their own moods and emotions; e.g., others because of his or her many positive qualities. No
“I pay much attention to my feelings”), (b) clarity (partici- symptoms,” whereas scores of 1–10 are described as being:
pants’ self-perception of the degree to which they under- “Persistent danger of severely hurting self or others (e.g.,
stand their own emotions; e.g., “I am usually very clear with recurrent violence) OR persistent inability to maintain
my feelings”), and (c) repair of emotion (participants’ self- minimal personal hygiene OR serious suicidal act with
perception of the degree to which they are able to modify clear expectation of death.”
their own emotions; e.g., “When I am upset, I think of all the Satisfaction with the treatment. The Working Alliance
pleasure of life”). The Spanish TMMS-24 has psychometric Theory of Change Inventory (WATOCI; Horvath &
characteristics similar to the original version with an internal Greenberg, 1989) is a 17-item scale answered by individuals
consistency (Cronbach’s α) of .90, .90, and .86 for attention, and used to evaluate aspects, such as therapeutic alliance
clarity, and repair, respectively. For this study, the bench- and patient satisfaction with the treatment (e.g., “I think that
marks for males described in the Spanish version were used the things I do in therapy help me to get the changes that
(Fernandez-Berrocal et al., 2004). I want”). It has been validated for Spanish population
The Escala de Habilidades Sociales (EHS – Social Skills (Corbella & Botella, 2004) with a high internal consistency
Scale; Gismero, 2000) is a 32-item Spanish scale used to (with a Cronbach’s α of .93).

942 | Journal of Behavioral Addictions 7(4), pp. 939–952 (2018)

 Internet Gaming Disorder treatment efficacy

Procedure patients, the scheduled duration of the program is 6 months

(22 sessions of around 45-min weekly sessions). The
Data acquisition. Before the treatment program was intervention, based on a cognitive-behavioral approach,
launched, a pilot study had been implemented to assess employs crosscutting techniques and resources common in
the operationalization of the intervention design and to psychotherapy (Hofmann & Barlow, 2014; Kleinke, 1994;
identify any potential problems regarding the intervention Laska et al., 2014). The design and content of PIPATIC
(Torres-Rodríguez & Carbonell, 2015). Following this, spe- has previously been described in detail (i.e., Torres-
cialized training for health teams in the collaborating public Rodríguez & Carbonell, 2017; Torres-Rodríguez et al.,
mental health institutions was carried out to provide informa- 2017), providing the in-depth clinical and methodological
tion about the study along with the inclusion/exclusion aspects. The experimental group received the PIPATIC
criteria, and to train individuals to carry out clinical inter- specialized treatment.
views to assess IGD symptoms and other comorbid disorders. The control group also received psychological attention,
The training aim was to provide treatment strategies to the because the use of the waiting list was considered unethical
health teams (comprising psychiatrists, clinical psychologists, according to the following considerations: (a) the partici-
general practitioners, and nurses) to identify the problem in pants were minors and were in a stage of increased vulnera-
child and adolescent populations. Data collection comprised bility, (b) the participants presented with a high level of
clinical interviews and data from the administration of diag- psychological symptomatology and distress, and (c) it was
nostic instruments. The clinical interviews were conducted by necessary to attend to the needs and demands of the family.
clinical psychologists, who also applied the diagnostic tests. For that reason, it was decided to apply a standard CBT
The participants carried out repeated measurements during (or TAU) intervention for addiction. One of the most
the treatment process: pre-treatment (T1), post-treatment commonly adapted CBT approaches for gaming addiction
(T3), and 3-month follow-up (T4). In addition, participants was used (Greenfield, 1999; Griffiths & Meredith, 2009;
completed a brief measurement during the middle of the Kaptsis et al., 2016; King et al., 2010, 2017; Winkler et al.,
program (T2, 11th session) to assess the change process 2013; Young, 2007, 2013). The standard CBT intervention
during the interventions. The parents of the participants were was extracted from the second module of PIPATIC (Table 1)
included in all of these stages and completed their own (Torres-Rodríguez et al., 2017) and was applied to the
instruments [(a) weekly hours spent gaming, (b) perceived control group across 22 sessions with greater depth of
ability to stop gaming, (c) self-awareness of engagement in addiction psychotherapeutic work. This standard CBT treat-
gaming, and (d) the CBCL/6-18]. The therapists also com- ment comprised five modules: (a) addiction stimulus con-
pleted their own measures in each assessment (CGI-SI, CGI- trol, (b) coping responses, (c) cognitive restructuring,
GI, and GAF). There was no significant data loss during this (d) problem solving related to addiction, and (e) exposition
process. The trained psychologists tried to ensure the highest (for more in-depth information regarding these modules, see
quality of data collection in each repeated measurement Torres-Rodríguez et al., 2017). The level of families’ par-
providing specific instructions to each participant and their ticipation in the standard CBT and in PIPATIC program
relatives. intervention was different. In both interventions, the rela-
Interventions: Individualized psychotherapy treatment tives acted as co-therapists in working with the gaming
for IGD (PIPATIC program) and TAU. The primary goal addiction. However, in the PIPATIC program, the relatives
of the PIPATIC (Torres-Rodríguez & Carbonell, 2017; participated in a specific therapeutic module of family
Torres-Rodríguez et al., 2017) was to offer specialized therapy. The relatives attended all sessions in both groups
psychotherapy for adolescents with symptoms of IGD and (i.e., in each specific module that required their involvement
comorbid disorders. This program comprises six therapeutic they completed the therapeutic tasks and recorded the
work modules, in turn made up of more specific subobjec- adolescents’ gaming). In the PIPATIC program, the
tives, in order to address different life areas and not just relatives were involved in two sessions of the psychoeduca-
addictive behaviors (Table 1). Following previous studies tional module, and in two sessions of standard CBT
(Hansen & Lambert, 2003; Kadera et al., 1996; Lambert & module. In the family module, they assisted in the totality
Bergin, 1994), in order to ensure therapeutic changes in of the sessions and were involved in the totality of the

Table 1. Summary of the psychotherapeutic modules of the PIPATIC program

1. Psychoeducational module: individual and family psychoeducation, motivational interviewing, choosing goals and objectives
(three sessions)
2. Standard CBT addiction intervention module: stimulus control, learning appropriate coping responses, cognitive restructuration,
problem solving related to addiction, exposure : : : (five sessions)
3. Intrapersonal module: psychotherapeutic work on identity, self-esteem, self-control, emotional-intelligence, and anxiety control
(five sessions)
4. Interpersonal module: encouraging adaptive communication skills, assertiveness, and increasing communication skills (two sessions)
5. Familiar module: family communication, limits, and affect (three sessions)
6. Development of a new lifestyle module: self-observation of improvement, alternative activities, and relapse prevention (two sessions)
Note. The PIPATIC program includes two floating sessions that can be incorporated into the module that the therapist chooses, according to the
needs of the patient. In this way, the set program offers some flexibility (Carroll & Nuro, 2002; Therien, Lavarenne, & Lecomte, 2014).

Journal of Behavioral Addictions 7(4), pp. 939–952 (2018) | 943

 Torres-Rodríguez et al.

development of a new lifestyle. In the standard CBT group, anonymized. The study procedures were carried out in
the relatives were involved in the same sessions apart from accordance with the Declaration of Helsinki.
the family module.
The participants were assigned to the groups in order of
arrival at the centers, and the assignment was blinded for the RESULTS
participants and the families. The treatments were carried
out by a clinical psychologist, extensively trained in the The participants were all males aged between 12 and 18
treatment of behavioral addictions, with the supervision of years. The experimental group had a mean age of 15.19
the mental health teams of the collaborating centers and the years (SD = 1.9), whereas the control group had 14.73 years
authors of the study. A comparison between patients com- (SD = 1.58). All participants were Spanish, and all but two
pleting the PIPATIC program (n = 16) and patients com- were students during the treatment. None of the participants
pleting TAU (n = 15) (i.e., pure CBT) was carried out. The reported any serious physical health problem, although one
first intervention (PIPATIC program) focused the problem participant was currently receiving antidepressant medica-
in an integrative way and addressed different psychological tion. The psychological characteristics of the sample have
areas and not only the addiction. The second intervention been described in detail elsewhere (Torres-Rodríguez,
(TAU) focused on the addiction as a primary problem. Griffiths, Carbonell, & Oberst, 2018). The participants
mostly reported problematic use of the online video games
Statistical analyses with three participants reporting problematic use of offline
video games. The most popular type of online video games
All analyses were carried out using SPSS software version in the sample was: massively multiplayer online role-
24. Due to the non-normality of the data, non-parametric tests playing games (51.6%), role-playing games (32.3%),
were utilized, and the non-parametric Mann–Whitney U test multiplayer online battle arena games (64.5%), shooter
for two independent samples was used to compare the results games (64.5%), sports games (35.5%), and others (2.3%).
of the treatment between the experimental group and the A small proportion of the sample also reported problematic
control group (i.e., the CBT group). In order to compare the use of the internet (19.4%) and smartphones (9.7%).
changes across the four different points of assessment, non-
parametric Friedman tests for repeated measures were used. Comparison of experimental and control groups
The effect sizes statistical were calculated. The range for
small effects is 0.20–0.50, for medium effects is 0.50–0.80, Regarding the outcome evaluations in both groups, changes
and for large effects is ≥0.8 (Cohen, 1988). The Wilcoxon relating to the different interventions were examined with
test was used in a post-hoc Friedman analysis to calculate the reference to the pre-test and post-test scores in the dependent
effect sizes regarding the changes via the temporal stages variables listed above. Before examining the efficacy of
(pre- and post-measures). To correct for multiple compar- each intervention, the measures at T1 were compared
isons, the Bonferroni procedure was applied. between the experimental and control groups and no signif-
icant differences were detected, indicating that the two
Ethics groups are at the same or similar level of clinical measures
at the baseline of the study (Tables 2–4). Features related to
The study was approved by the ethics committees of the IGD are reported in Table 2. Both patients and their families
mental health centers that participated in the studies (Centro reported similar perceptions. There were no significant
de Salud Mental Infanto Juvenil Joan Obiols of Barcelona, differences between either group in the pre-treatment phase.
and Consorci Sanitari del Maresme, Mataró) and the re- However, in the post-test, the PIPATIC group (compared to
search team’s ethics committee. The participants and their the control group) dedicated fewer hours to gaming, and had
legal guardians signed consent forms. All the information lower scores in being able to stop gaming, subjective scores
that could have been used to identify the patients was relating to engagement/addiction, and IGD-20 scores.

Table 2. Medians and standard deviations (in brackets) of measures regarding video game use and IGD for treatment condition and pre- and
post-assessment
Pre-test Post-test

EG CG p EG CG p r
Week gaming hours (P) 51.40 (19.21) 43.36 (15.51) .384 11.15 (7.07) 31.80 (15.63) .0001 .707
Week gaming hours (F) 48.87 (17.45) 50.06 (19.34) .968 14.84 (9.16) 32.83 (17.93) .001 .586
Postponement (P) 4.19 (0.65) 4.07 (1.03) .966 1.81 (0.75) 3.27 (0.79) .0001 .702
Postponement (F) 4.44 (0.62) 4.60 (0.63) .402 2.19 (0.91) 3.67 (0.97) .0001 .644
Subjective addiction (P) 7.75 (1.00) 8.27 (1.16) .239 3.13 (1.40) 6.47 (1.88) .0001 .702
Subjective addiction (F) 8.88 (0.88) 9.00 (1.00) .663 3.69 (1.81) 7.33 (1.79) .0001 .708
IGD-20 79.75 (5.77) 76.13 (5.37) .126 32.19 (7.51) 51.93 (16.9) .001 .554
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). Mann–Whitney U test was used to compare
the results between experimental group (EG) and control group (CG). IGD: Internet Gaming Disorder.

944 | Journal of Behavioral Addictions 7(4), pp. 939–952 (2018)

 Internet Gaming Disorder treatment efficacy

Table 3. Medians and standard deviations (in brackets) of all YSR subscales and total CBCL subscales for treatment condition and pre- and
post-assessment
Pre-test Post-test

EG CG p EG CG p r
Competence scales
Activities 30.31 (6.92) 29.66 (5.32) .953 36.56 (14.22) 34.00 (6.21) .405
Social 37.25 (18.07) 32.00 (9.72) .403 37.75 (12.66) 34.86 (7.45) .183
Schoola
Total competence scales 26.92 (6.50) 28.93 (11.25) .913 33.28 (13.74) 30.40 (6.40) .394
Syndrome scales
I – Anxious/depressed 62.38 (8.05) 60.07 (9.34) .393 53.69 (3.96) 58.00 (8.08) .230
II – Withdrawn/depressed 66.69 (7.54) 62.47 (11.86) .087 56.94 (7.33) 59.13 (8.91) .631
III – Somatic complaints 61.75 (10.02) 55.07 (5.62) .027 51.81 (2.92) 54.40 (6.53) .549
IV – Social problems 61.56 (8.28) 61.33 (9.75) .827 53.19 (3.56) 59.67 (8.22) .016 .239
V – Thought problems 56.63 (5.62) 55.87 (6.04) .645 52.00 (2.94) 53.87 (4.502) .295
VI – Attention problems 63.50 (8.60) 66.80 (11.59) .633 53.25 (6.19) 64.33 (14.74) .003 .530
VII – Rule-breaking behavior 55.81 (5.63) 57.33 (5.97) .402 53.69 (4.33) 55.87 (5.95) .143
VIII – Aggressive behavior 58.19 (9.88) 62.60 (10.83) .147 52.69 (5.22) 59.80 (8.66) .005 .503
Total scales
Internalizing problems 65.38 (7.05) 58.27 (11.32) .085 51.19 (6.53) 54.60 (12.12) .736
Externalizing problems 55.88 (7.50) 59.80 (8.81) .166 47.13 (9.04) 57.40 (8.63) .005 .509
Total problems 62.50 (5.30) 61.80 (7.36) .781 48.75 (7.46) 58.73 (9.27) .004 .515
Total CBCL scales
Internalizing problems 70.00 (6.78) 69.27 (11.64) .692 59.69 (6.86) 64.47 (11.44) .190
Externalizing problems 59.38 (8.15) 64.33 (7.97) .052 52.56 (8.45) 61.53 (7.76) .003 .537
Total problems 66.94 (5.63) 69.33 (8.04) .351 56.06 (7.60) 64.40 (8.24) .005 .501
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). Mann–Whitney U test was used to compare the
results between experimental group (EG) and control group (CG). YSR: Youth Self-Report; CBCL: Child Behavior Checklist.aSystem missing.

Table 4. Medians and standard deviations (in brackets) of MACI, TMMS subscales, and social abilities scale for treatment condition and pre-
and post-assessment
Pre-test Post-test

EG CG p EG CG p r
MACI’s scales
Identity diffusion (A) 61.60 (22.8) 65.30 (14.5) .782 41.10 (20.2) 50.30 (26.3) .323
Self-devaluation (B) 54.30 (19.2) 58.80 (22.4) .752 42.40 (14.4) 57.00 (23.6) .044
Body disapproval (C) 52.70 (18.5) 52.70 (24.1) .922 50.90 (16.0) 57.07 (28.3) .678
Emotional intelligence
Attention to feelings 19.60 (7.48) 17.20 (6.77) .406 24.50 (7.59) 19.13 (6.99) .050
Clarity of feelings 22.30 (5.16) 21.06 (6.91) .329 26.44 (8.38) 20.33 (3.43) .020 .416
Mood repair 21.31 (6.23) 21.66 (3.53) .874 28.80 (8.47) 23.20 (5.28) .045
Social abilities
Global scale 28.81 (28.5) 35.80 (28.9) .405 68.50 (30.74) 41.00 (37.02) .042
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). To increase confidence in the results,
Bonferroni corrections were used. Nonetheless, there are multiple values smaller than .05 alpha that should be considered for future research.
Mann–Whitney U test was used to compare the results between experimental group (EG) and control group (CG). MACI: Millon Adolescent
Clinical Inventory; TMMS: Trait Meta-Mood Scale.

There were also significant differences between the In relation to the CBCL results, no differences between
two groups regarding comorbid disorders, as well as the groups were found at baseline treatment. However, at
behavioral and emotional functioning of the participants post-treatment, there were statistically significant
from both patients (YSR test) and their relatives (CBCL differences between the two groups including: activities
test). Table 3 demonstrates that all group comparisons were (MEG = 39.56; MCG = 30.26; U = 48.5; p < .01), total com-
non-significant. However, in the post-test, the experimental petence scale (MEG = 36.23; MCG = 27.2; U = 46.5;
group had significantly lower scores in several important p < .05), rule-breaking behavior (MEG = 54; MCG = 58.6;
areas (social problems, attention problems, aggressive behav- U = 56.5; p < .05), aggressive behavior scales (MEG = 55.81;
ior scales, externalizing problems, and total problem scales). MCG = 63.8; U = 45; p < .01), externalizing problems

Journal of Behavioral Addictions 7(4), pp. 939–952 (2018) | 945

 Torres-Rodríguez et al.

(MEG = 52.56; MCG = 61.53; U = 44.5; p < .01), and total scale that needs to be applied once the treatment has been
problem scales (MEG = 56.06; MCG = 64.40; U = 49.5; completed. The results demonstrated statistically significant
p < .01) (Table 3). In addition, the PIPATIC program differences between both groups at the following WATOCI
generally had lower median scores even if they were scales (Table 5): tasks, bond, goals, theory of change, and
not significantly different. In addition, it is noteworthy that total score. On all these measures, the PIPATIC group had
in the MACI’s scale “suicidal tendency” (GG) showed a better scores than the control group on these scales demon-
statistically significant difference in both groups at post- strating that the PIPATIC treatment was more effective than
treatment (MEG = 32.8; MCG = 48.7; U = 58; p < .01). CBT alone. The only subscale where there was no signifi-
The findings in relation to the intrapersonal and interper- cant difference was the bond between therapist and the
sonal abilities are presented in Table 4. Regarding the patient (i.e., patients in both groups bonded equally well
intrapersonal abilities assessed by three MACI subscales and with their therapists).
the TMMS test for emotional intelligence, significant group
differences were found in the post-test for clarity of feelings Effects of the IGD treatment process
(subscale of TMMS test). Nonetheless, self-devaluation (B)
scale and the two of three emotional intelligence scales As shown in Table 6, all participants in the experimental
(attention to feelings and mood repair) presented smaller group demonstrated a decrease over time in all measures
values than 0.05. With respect to the interpersonal and social related to gaming and addiction. This effect was significant,
abilities, no significant group differences were found at post- and maintained stability in the follow-up assessment. The
treatment assessment because of the Bonferroni correction. control group also presented significant changes in postpone-
Concerning family discord (G), no significant differences ment, patient’s subjective addiction score, family’s subjective
were found at T1 (U = 96.5; p > .05). However, the results addiction score, and IGD-20 results. The analysis was carried
demonstrated there was a significant group difference in out examining the comorbid symptoms plus the contribution
family discord (U = 49; p < .01) at post-assessment, with an of the interventions regarding behavioral and emotional
improvement in PIPATIC group. functioning from both patients’ and relatives’ perspective
Finally, in relation to the therapists’ measures, no signif- (Table 7 for patients and Table 8 for families). For the
icant differences were found in CGI (U = 120; p > .05) and patients, the PIPATIC group demonstrated a significant
GAF (U = 93; p > .05) between both groups at baseline. At decrease in all the YSR scales (apart from the social compe-
post-treatment, significant differences were found between tence scale), as did their relatives (CBCL). The parents of the
groups for CGI (U = 18.5; p < .001) and GAF (U = 10; control group had perceptions similar to parents of the
p < .001) with the PIPATIC group demonstrating better PIPATIC group, and reported that the patients had improved
scores than the control group. The WATOCI variables were with respect to some areas (i.e., anxious/depressed, attention
analyzed only at post-treatment assessment, because it is a problems, aggressive behavior, and total problems).

Table 5. Medians and standard deviations (in brackets) of WATOCI dependent variables for treatment condition at post-assessment
EG CG p r
Tasks 25.68 (2.44) 19.26 (1.98) .0001 .788
Bond 25.50 (3.32) 25.13 (2.58) .545
Goals 25.06 (2.08) 19.60 (3.20) .0001 .703
Theory of change 32.00 (3.52) 26.40 (3.64) .0001 .625
Total 108.25 (10.47) 90.40 (7.07) .0001 .650
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). Mann–Whitney U test was used to compare
the results between experimental group (EG) and control group (CG). WATOCI: Working Alliance Theory of Change Inventory.

Table 6. Medians of variables regarding video game use and IGD for treatment condition
EG CG

T1 T2 T3 T4 p r T1 T2 T3 T4 p r
Week gaming hours (P) 51.40 19.03 11.15 11.40 .0001 .879 44.63 32.50 29.22 34.13 .013
Week gaming hours (F) 48.87 20.25 14.84 11.75 .0001 .879 52.63 36.09 31.86 46.59 .032
Postponement (P) 4.19 2.50 1.81 1.81 .0001 .900 4.07 3.27 3.27 3.13 .030
Postponement (F) 4.44 2.44 2.19 2.25 .0001 .891 3.40 2.47 2.17 1.97 .002 .644
Subjective addiction (P) 7.75 5.06 3.13 2.44 .0001 .886 8.27 7.07 6.47 6.07 .001 .670
Subjective addiction (F) 8.88 6.06 3.69 2.94 .0001 .883 9.00 7.53 7.33 6.73 .0001 .679
IGD-20 79.75 46.31 32.19 30.69 .0001 .879 76.13 62.60 51.93 56.40 .0001 .879
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). The effect sizes were statistically significant
using T1 and T3 measures (pre- and post-). To increase confidence in the results, Bonferroni corrections were used. Nonetheless, there are
multiple values regarding the control group smaller than .05 alpha that should be considered for future research. Friedman test for repeated
measures was used for comparison between pre-, middle-, and post-treatment and 3-month follow-up assessment. EG: experimental group;
CG: control group; IGD: Internet Gaming Disorder.

946 | Journal of Behavioral Addictions 7(4), pp. 939–952 (2018)

 Internet Gaming Disorder treatment efficacy

Table 7. Medians of YSR subscales for treatment condition

EG CG

T1 T3 T4 p r T1 T3 T4 p
Competence scales
Activities 30.31 36.56 43.81 .0001 .426 29.21 33.64 30.14 .233
Social 37.25 37.75 40.06 .024 30.57 34.57 32.71 .030
Total competence scales 27.00 35.84 40.15 .003 .550 28.14 30.07 27.35 .316
Syndrome scales
I – Anxious/depressed 62.38 53.69 52.69 .0001 .778 60.07 58.00 59.73 .571
II – Withdrawn/depressed 66.69 56.94 55.13 .0001 .815 62.47 59.13 60.73 .141
III – Somatic complaints 61.75 51.81 51.69 .0001 .781 55.07 54.40 55.47 .607
IV – Social problems 61.56 53.19 51.82 .0001 .767 61.33 59.67 59.67 .498
V – Thought problems 56.63 52.00 50.13 .0001 .718 55.87 53.87 53.53 .038
VI – Attention problems 63.50 53.25 53.00 .0001 .802 66.80 64.33 64.53 .484
VII – Rule-breaking behavior 5.81 53.69 52.38 .002 .455 57.33 55.87 57.93 .334
VIII – Aggressive behavior 58.19 52.69 52.56 .0001 .796 62.60 59.80 61.67 .793
Total scales
Internalizing problems 65.38 51.19 49.13 .0001 .853 58.27 54.60 57.87 .262
Externalizing problems 55.88 47.13 45.94 .0001 .854 59.80 57.40 60.07 .368
Total problems 62.50 48.75 46.25 .0001 .879 61.80 58.70 60.07 .395
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). The effect sizes statistically significant using
T1 and T3 measures (pre- and post-). Friedman test for repeated measures was used for comparison between pre- and post-treatment and
3-month follow-up assessment. EG: experimental group; CG: control group; YSR: Youth Self-Report.

Table 8. Medians of CBCL subscales for treatment condition

EG CG

T1 T3 T4 p r T1 T3 T4 p r
Competence scales
Activities 25.68 39.56 44.56 .0001 .866 28.85 29.64 29.07 .486
Social 29.40 39.53 43.53 .0001 .835 31.85 34.28 34.78 .620
School 36.61 42.69 44.92 .001 .658 37.35 38.71 39.85 .358
Total competence scales 22.23 36.23 42.00 .0001 .780 24.92 26.64 27.07 .678
Syndrome scales
I – Anxious/depressed 65.44 56.94 53.75 .0001 .745 66.80 60.67 60.33 .006 .616
II – Withdrawn/depressed 78.88 63.69 61.56 .0001 .834 77.33 71.87 68.20 .034
III – Somatic complaints 64.25 56.06 55.44 .002 .630 62.73 61.13 63.27 .059
IV – Social problems 30.25 55.13 54.69 .0001 .770 64.53 61.20 58.67 .140
V – Thought problems 64.88 55.69 54.38 .0001 .796 65.00 60.33 58.73 .184
VI – Attention problems 61.81 56.44 55.06 .0001 .696 70.20 64.33 62.27 .007 .604
VII – Rule-breaking behavior 58.06 54.00 53.44 .0001 .605 61.13 58.60 60.27 .058
VIII – Aggressive behavior 61.06 55.81 54.19 .006 .566 67.20 63.80 61.67 .009 .414
Total scales
Internalizing problems 70.00 59.69 57.19 .0001 .841 69.27 64.47 64.67 .071
Externalizing problems 59.38 52.56 49.38 .0001 .608 64.33 61.53 60.87 .059
Total problems 66.94 56.06 53.56 .0001 .808 69.33 64.40 63.87 .017 .657
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). The effect sizes were statistically significant
using T1 and T3 measures (pre- and post-). Friedman test for repeated measures was used comparison between pre- and post-treatment and
3-month follow-up assessment. EG: experimental group; CG: control group; CBCL: Child Behavior Checklist.

With reference to intrapersonal abilities, the significant χ2 = 1.32; p > .05), but was significant in the PIPATIC
differences between both groups were observed (Table 9). group (MT1 = 52.19; MT3 = 43.37; MT4 = 36.88; χ2 = 8.66;
Participants in the experimental group showed an improve- p < .05). Finally, in relation to the therapists’ measures,
ment regarding identity, self-esteem, emotional intelligence, significant changes across the assessment period were
and social abilities. The control group only showed a found. The PIPATIC group (MT1 = 5; MT3 = 1.88; MT4 =
significantly higher attention to their feelings. Concerning 1.25; χ2 = 45.92; p < .001) and the control group (MT1 = 5;
family discord (G), no significant change was found in the MT3 = 3.87; MT4 = 3.47; χ2 = 28.73; p < .001) demonstrat-
control group (MT1 = 58.73; MT3 = 63.73; MT4 = 66.53; ed a significant reduction of the mental illness severity

Journal of Behavioral Addictions 7(4), pp. 939–952 (2018) | 947

 Torres-Rodríguez et al.

Table 9. Medians of MACI, Emotional Intelligence, and Social Abilities subscales for treatment condition
EG CG

T1 T3 T4 p r T1 T3 T4 p
MACI’s scales
Identity diffusion (A) 61.63 41.19 37.38 .030 .695 65.33 50.33 56.60 .175
Self-devaluation (B) 54.31 42.44 32.94 .001 .532 58.87 57.00 59.07 .859
Body disapproval (C) 52.75 50.94 38.19 .084 52.73 57.07 54.20 .155
Emotional intelligence
Attention to feelings 19.68 24.50 27.68 .002 .533 17.20 19.13 20.80 .026
Clarity of feelings 22.37 26.44 31.31 .001 .355 21.06 20.33 20.60 .748
Mood repair 21.31 28.81 30.62 .013 .660 21.66 23.27 23.00 .516
Social abilities
Global scale 28.81 68.50 75.75 .0001 .852 35.8 41.60 35.20 .250
Note. Bold values indicate significance at p < .025 level (obtained with Bonferroni correction). The effect sizes were statistically significant
using T1 and T3 measures (pre- and post-). Friedman test for repeated measures was used comparison between pre- and post-treatment and 3-
month follow-up assessment. EG: experimental group; CG: control group; MACI: Millon Adolescent Clinical Inventory.

(CGI-SI) of the participants. In relation to the global activity The differences between both interventions can be sum-
(GAF), both groups experienced an improvement: PIPATIC marized in the following aspects: (a) no significant differ-
group (MT1 = 47.38; MT3 = 82.81; MT4 = 86.69; χ2 = 45.07; ences were found between the two groups concerning the
p < .001) and control group (MT1 = 42.87; MT3 = 62.8; IGD variables at baseline treatment; (b) the results at post-
MT4 = 63.13; χ2 = 36.41; p < .001). treatment demonstrated significant differences between the
two groups in the number of weekly gaming hours, IGD
symptoms, comorbidity disorders, externalizing problems,
DISCUSSION overall total problems, emotional intelligence, and the fam-
ily relationships (with those in the PIPATIC program dem-
This study evaluated the effects of the PIPATIC program on onstrating more improved scores on these aspects compared
a number of key variables and compared these with the to those given standard CBT); (c) apart from the bond
effects of a standard CBT (TAU) control group. The vari- between patient and therapist, those undergoing the PIPA-
ables were assessed at baseline, during the middle of the TIC treatment (compared to the control group) found the
treatment, immediately after treatment, and at a treatment more satisfying; (d) significant differences be-
3-month follow-up. The measures were completed by par- tween groups were found in measures completed by the
ticipants, their relatives, and their therapists in an effort to therapists in the post-treatment phase (CGI and GAF
triangulate the findings. More specifically, the study evalu- scales); and (e) the PIPATIC program demonstrated a
ated the effects and changes regarding IGD symptoms, greater reduction than CBT treatment in IGD symptoms
psychopathological and comorbid symptoms, emotional and improvement of abilities.
intelligence, self-esteem, social skills, family environment, This study described the effects of a practical clinical trial
therapeutic alliance, and change perceptions, by comparing of psychotherapeutic approaches for adolescents with IGD.
standard CBT with the newly developed PIPATIC program. The aim of PIPATIC program is to offer specialized psy-
The main findings of the comparative evaluation can be chotherapy for adolescents with symptoms of IGD and
summarized as follows: (a) both groups experienced a accompanying comorbid disorders. The other aims of the
significant reduction of symptoms regarding IGD, but those program were to help improve interpersonal and intraper-
individuals in the PIPATIC group demonstrated more sta- sonal abilities and apply family therapy. Its program seeks to
tistically significant changes than control group; (b) the reestablish the adolescent’s well-being and to reintegrate the
PIPATIC group demonstrated significant reductions in co- individual back into a normal life including the controlled
morbid symptoms as reported by the patients and their use of video games and internet. It is noteworthy that this is
relatives. Moreover, the treatment program improved their an intervention model based on an integration of previous
identity diffusion, self-devaluation, emotional intelligence, research findings.
social abilities, and reduced family conflict. In contrast, the The findings of this study corroborate the importance of
control group experienced positive significant changes in extending psychotherapeutic work into comorbid disorders
anxiety, attention problems, aggressive behavior, and over- in addition to addressing IGD itself. Previous research has
all problems reported by relatives (CBCL). However, the consistently found an association between high levels of
improvements (based on the effect sizes) were less than that distress and online addictions (Mentzoni et al., 2011; Yan,
of the PIPATIC group; (c) most of the PIPATIC patients Li, & Sui, 2014), and high rates of comorbid psychiatric
experienced a decrease of negative symptoms during the disorders (Andreassen et al., 2016; Bozkurt, Coskun,
middle of the treatment (T2) at the 11th session; and (d) the Ayaydin, Adak, & Zoroglu, 2013; Ferguson et al., 2011;
changes achieved with the PIPATIC program and for those Müller, Beutel, Egloff, & Wölfling, 2014). The findings
undergoing standard CBT demonstrated continued stability regarding the participation of the family are warranted and
3 months after the end of the respective treatment. according to the idea that intervention programs for

948 | Journal of Behavioral Addictions 7(4), pp. 939–952 (2018)

 Internet Gaming Disorder treatment efficacy

adolescents need to include parents actively in the therapy also be noted that standard CBT also demonstrated positive
(Liu et al., 2015; Young, 2009). This is especially relevant significant effects in the treatment of IGD, but reductions
given that family conflict and poor family relationships co- regarding the most of the comorbid disorders and improve-
occur in adolescent IGD (Bonnaire & Phan, 2017). In this ments in other areas of life functioning were not statistically
study, as in others (Winkler et al., 2013), a CBT approach significant. Changing the focus of treatment and applying
was effective in treating IGD symptoms (although more so the integrative focus of the PIPATIC treatment (including
when individuals were assigned to the PIPATIC program). the addiction, the comorbid symptoms, intrapersonal and
Finally, some limitations of this study should be noted interpersonal abilities, and family psychotherapy) appeared
and therefore generalization of the present results should be to generate greater improvements for facilitating adolescent
considered with caution. First, the self-report data included behavior change than the therapy focusing only on the IGD
potential sources of error (e.g., social desirability bias and itself (i.e., CBT).
memory recall bias). Moreover, the multiple self-assessment
tools used have the potential to lead to a type-I error (given
how many different variables were evaluated). Nevertheless,
the data collection was supervised by a trained psychologist Funding sources: This study was supported by personal
to ensure the highest quality of data that was also based on Blanquerna Research Grant (BRB) to AT-R.
clinical interviews. Moreover, Bonferroni test was used to
overcome type-I errors increasing confidence in results Authors’ contribution: AT-R: study concept and design,
obtained. The psychometric tools used demonstrated ade- access to the sample, analysis and interpretation of data,
quate psychometric properties and they had all been writing, and review. MDG and XC: study concept and
previously psychometrically validated using Spanish popu- design, writing, and review. UO: analysis and interpretation
lations. In addition, the inclusion of a follow-up provided an of data, writing, and review. All authors had full access to all
opportunity to explore the effects and stability of the therapy data in the study and take responsibility for the integrity of
observed months after the treatment. On the other hand, all the data and the accuracy of the data analysis.
the data were self-report, although the inclusion of corrob-
orating data from relatives increased the veracity of the data Conflict of interest: The authors report no financial or other
collected. Second, the sample size is relatively small. How- relationship relevant to the subject of this article.
ever, many other treatment studies report a similar (or even
smaller) sample size (Dell’Osso et al., 2008; Kim, 2008; Acknowledgements: The authors would like to thank all the
Pallesen, Lorvik, Bu, & Molde, 2015; Wölfling et al., 2014; participants and their families.
Yao et al., 2017; Zhang et al., 2016). Third, there was no
randomization of participants being assigned to treatment
groups, because all participants were assigned to one of the
two groups in order of arrival of seeking help for IGD at the REFERENCES
treatment centers. Fourth, the lack of a traditional control
group without treatment (waiting list) was not utilized, Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the
because it was unethical to deny treatment to those who ASEBA school-age forms & profiles. Burlington, VT: ASEBA.
sought treatment for IGD at the treatment centers. In future, American Psychiatric Association [APA]. (2002). Diagnostic and
and if deemed ethical, studies should include a wait-list statistical manual of mental disorders (DSM-IV-TR) (4th ed.,
control group without any intervention. However, the use of text rev.). Barcelona, Spain: Masson.
a TAU control group is a valuable element of this study that American Psychiatric Association [APA]. (2013). Diagnostic and
allowed statistical comparisons and explored the effects of statistical manual of mental disorders (DSM-5) (5th ed.).
the both interventions. Fifth, the participants were all Spanish Washington, DC: American Psychiatric Association.
male adolescents; therefore, it is not known whether the Andreassen, C. S., Billieux, J., Griffiths, M. D., Kuss, D. J.,
PIPATIC treatment would be equally effective for females, Demetrovics, Z., Mazzoni, E., & Pallesen, S. (2016). The
adults, and/or individuals from other countries and cultures. relationship between addictive use of social media and video
Due to the study design, it was not possible to determine the games and symptoms of psychiatric disorders: A large-scale
efficacy of each intervention compared to no treatment, cross-sectional study. Psychology of Addictive Behaviors,
although as noted earlier in the paper, previous studies have 30(2), 252–262. doi:10.1037/adb0000160
shown CBT to be effective compared to non-treatment Bonnaire, C., & Phan, O. (2017). Relationships between parental
(Greenfield, 1999; Griffiths & Meredith, 2009; Kaptsis attitudes, family functioning and Internet gaming disorder in
et al., 2016; King et al., 2010, 2011; Young, 2007, 2013). adolescents attending school. Psychiatry Research, 255,
104–110. doi:10.1016/j.psychres.2017.05.030
Bozkurt, H., Coskun, M., Ayaydin, H., Adak, I., & Zoroglu, S. S.
CONCLUSIONS (2013). Prevalence and patterns of psychiatric disorders in
referred adolescents with Internet addiction. Psychiatry and
Based on the findings of this study, the PIPATIC program Clinical Neurosciences, 67(5), 352–359. doi:10.1111/pcn.
presented significant positive effects regarding the treatment 12065
of the IGD and its comorbid disorders/symptoms, alongside Busner, J., & Targum, S. (2007). The Clinical Global Impressions
statistically significant improvements of intra- and interper- Scale: Applying a research tool in clinical practice. Psychiatry
sonal abilities and family relationships. However, it should (Edgmont), 4(7), 28–37.

Journal of Behavioral Addictions 7(4), pp. 939–952 (2018) | 949

 Torres-Rodríguez et al.

Carroll, K. M., & Nuro, K. F. (2002). One size cannot fit all: A stage Hofmann, S. G., & Barlow, D. H. (2014). Evidence-based psy-
model for psychotherapy manual development. Clinical Psy- chological interventions and the common factors approach:
chology: Science and Practice, 9(4), 396–406. doi:10.1093/ The beginnings of a rapprochement? Psychotherapy, 51(4),
clipsy.9.4.396 510–513. doi:10.1037/a0037045
Chan, P. A., & Rabinowitz, T. (2006). A cross-sectional analysis of Horvath, A. O., & Greenberg, L. S. (1989). Development and
video games and attention deficit hyperactivity disorder validation of the Working Alliance Inventory. Journal of
symptoms in adolescents. Annals of General Psychiatry, Counseling Psychology, 36(2), 223–233. doi:10.1037/0022-
5(1), 16–26. doi:10.1186/1744-859X-5-16 0167.36.2.223
Cohen, J. (1988). Statistical power analysis for the behavioral Huang, X., Li, M., & Tao, R. (2010). Treatment of Internet
sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum addiction. Current Psychiatry Reports, 12(5), 462–470. doi:10.
Associates. 1007/s11920-010-0147-1
Corbella, S., & Botella, L. (2004). Psychometric properties of the Kadera, S. W., Lambert, M. J., & Andrews, A. A. (1996). How
Spanish version of the Working Alliance Theory of Change much therapy is really enough?: A session-by-session analysis
Inventory (WATOCI). Psicothema, 16(4), 702–705. of the psychotherapy dose-effect relationship. Journal of Psy-
Dell’Osso, B., Hadley, S., Allen, A., Baker, B., Chaplin, W. F., & chotherapy Practice and Research, 5(2), 132–151.
Hollander, E. (2008). Escitalopram in the treatment of Kaptsis, D., King, D. L., Delfabbro, P. H., & Gradisar, M. (2016).
impulsive-compulsive Internet usage disorder: An open-label Withdrawal symptoms in Internet gaming disorder: A system-
trial followed by a double-blind discontinuation phase. Journal atic review. Clinical Psychology Review, 43, 58–66.
of Clinical Psychiatry, 69(3), 452–456. doi:10.4088/JCP. doi:10.1016/j.cpr.2015.11.006
v69n0316 Kelleci, M., & Inal, S. (2010). Psychiatric symptoms in adolescents
Du, Y., Jiang, W., & Vance, A. (2010). Longer term effect of with Internet use: Comparison without Internet use. Cyberp-
randomized, controlled group cognitive behavioral therapy for sychology, Behavior, and Social Networking, 13(2), 191–194.
Internet addiction in adolescent students in Shanghai. Austra- doi:10.1089/cyber.2009.0026
lian and New Zealand Journal of Psychiatry, 44(2), 129–134. Kim, E. J., Namkoong, K., Ku, T., & Kim, S. J. (2008). The
doi:10.3109/00048670903282725 relationship between online game addiction and aggression,
Ferguson, C., Coulson, M., & Barnett, J. (2011). A meta-analysis self-control and narcissistic personality traits. European Psy-
of pathological gaming prevalence and comorbidity with men- chiatry, 23(3), 212–218. doi:10.1016/j.eurpsy.2007.10.010
tal health, academic and social problems. Journal of Psychiat- Kim, J. U. (2008). The effect of a R/T group counseling program
ric Research, 45(12), 1573–1578. doi:10.1016/j.jpsychires. on the Internet addiction level and self-esteem of Internet
2011.09.005 addiction university students. International Journal of Reality
Fernandez-Berrocal, P., Extremera, N., & Ramos, N. (2004). Therapy, 27(2), 4–12.
Validity and reliability of the Spanish modified version of the Kim, K., Ryu, E., Chon, M. Y., Yeun, E. J., Choi, S. Y., Seo, J. S.,
Trait Meta-Mood Scale. Psychological Reports, 94(3), & Nam, B. W. (2006). Internet addiction in Korean adolescents
751–755. doi:10.2466/pr0.94.3.751-755 and its relation to depression and suicidal ideation: A ques-
Fuster, H., Carbonell, X., Pontes, H. M., & Griffiths, M. D. (2016). tionnaire survey. International Journal of Nursing Studies,
Spanish validation of the Internet Gaming Disorder 43(2), 185–192. doi:10.1016/j.ijnurstu.2005.02.005
(IGD-20) Test. Computers in Human Behavior, 56, 215–224. King, D., Delfabbro, P., & Griffiths, M. (2010). Cognitive behav-
doi:10.1016/j.chb.2015.11.050 ioral therapy for problematic video game players: Conceptual
Gentile, D., Choo, H., Liau, A., Sim, T., Li, D., Fung, D., & Khoo, considerations and practice issues. Journal of Cyber Therapy
A. (2011). Pathological video game use among youths: A two- and Rehabilitation, 3(3), 261–273.
year longitudinal study. Pediatrics, 127(2), e319–e329. King, D., Delfabbro, P., Griffiths, M., & Gradisar, M. (2011).
doi:10.1542/peds.2010-1353 Assessing clinical trials of Internet addiction treatment: A
Gismero, E. (2000). EHS Escala de Habilidades Sociales [EHS systematic review and CONSORT evaluation. Clinical Psy-
Social Skills Scale]. Madrid, Spain: TEA. chology Review, 31(7), 1110–1116. doi:10.1016/j.cpr.2011.
Greenfield, D. N. (1999). Virtual addiction: Help for netheads, 06.009
cyberfreaks, and those who love them. Oakland, CA: King, D. L., & Delfabbro, P. H. (2017). Features of parent-child
New Harbin. relationships in adolescents with Internet gaming disorder.
Griffiths, M. D. (2008). Videogame addiction: Further thoughts International Journal of Mental Health and Addiction,
and observations. International Journal of Mental Health and 15(6), 1270–1283. doi:10.1007/s11469-016-9699-6
Addiction, 6(2), 182–185. doi:10.1007/s11469-007-9128-y King, D. L., Delfabbro, P. H., & Griffiths, M. D. (2013). Trajectories
Griffiths, M. D., & Meredith, A. (2009). Videogame addiction and of problem video gaming among adult regular gamers:
its treatment. Journal of Contemporary Psychotherapy, 39(4), An 18-month longitudinal study. Cyberpsychology, Behavior
247–253. doi:10.1007/s10879-009-9118-4 and Social Networking, 16(1), 72–76. doi:10.1089/cyber.
Han, D. H., Lee, Y. S., Shi, X., & Renshaw, P. F. (2014). Proton 2012.0062
magnetic resonance spectroscopy (MRS) in on-line game King, D. L., Delfabbro, P. H., Wu, A. M. S., Doh, Y. Y., Kuss, D. J.,
addiction. Journal of Psychiatric Research, 58, 63–68. Pallesen, S., Mentzoni, R., Carragher, N., & Sakuma, H. (2017).
doi:10.1016/j.jpsychires.2014.07.007 Treatment of Internet gaming disorder: An international system-
Hansen, N. B., & Lambert, M. J. (2003). An evaluation of the dose- atic review and CONSORT evaluation. Clinical Psychology
response relationship in naturalistic treatment settings using Review, 54, 123–133. doi:10.1016/j.cpr.2017.04.002
survival analysis. Mental Health Services Research, 5(1), Kleinke, C. L. (1994). Common principles of psychotherapy.
1–12. doi:10.1023/A:1021751307358 Pacific Grove, CA: Brooks/Cole Pub Co.

950 | Journal of Behavioral Addictions 7(4), pp. 939–952 (2018)

 Internet Gaming Disorder treatment efficacy

Ko, C. H., Yen, J. Y., Chen, C. C., Chen, S. H., Wu, K., & Yen, Mental & Physical Health, 117(2), 490–495. doi:10.2466/
C. F. (2006). Tridimensional personality of adolescents with 02.PR0.117c14z9
Internet addiction and substance use experience. Canadian Pontes, H. M., Király, O., Demetrovics, Z., & Griffiths, M. D.
Journal of Psychiatry, 51(14), 887–894. doi:10.1177/ (2014). The conceptualisation and measurement of DSM-5
070674370605101404 Internet gaming disorder: The development of the IGD-20
Kuss, D. J., & Griffiths, M. D. (2012). Online gaming addiction in Test. PLoS One, 9(10), e110137. doi:10.1371/journal.pone.
children and adolescents: A review of empirical research. 0110137
Journal of Behavioral Addictions, 1(1), 3–22. doi:10.1556/ Rehbein, F., Psych, G., Kleimann, M., Mediasci, G., & Mößle, T.
JBA.1.2012.1.1 (2010). Prevalence and risk factors of video game dependency
Kuss, D., van Rooij, A., Shorter, G. W., Griffiths, M. D., & van de in adolescence: Results of a German nationwide survey.
Mheen, D. (2013). Internet addiction in adolescents: Preva- Cyberpsychology, Behavior, and Social Networking, 13(3),
lence and risk factors. Computers in Human Behavior, 29(5), 269–277. doi:10.1089/cyber.2009.0227
1987–1996. doi:10.1016/j.chb.2013.04.002 Salovey, P., Mayer, J., Goldman, S. L., Turvey, C., & Palfai, T. P.
Lambert, M. J., & Bergin, A. E. (1994). The effectiveness of (1995). Emotional attention, clarity and repair Exploring
psychotherapy. Oxford, UK: John Wiley. emotional intelligence using the Trait Meta-Mood Scale. In
Laska, K. M., Gurman, A. S., & Wampold, B. E. (2014). Expand- J. W. Pennebaker (Ed.), Emotion, disclosure, and health
ing the lens of evidence-based practice in psychotherapy: A (pp. 125–154). Washington, DC: American Psychiatric
common factors perspective. Psychotherapy, 51(4), 467–481. Association.
doi:10.1037/a0034332 Schneider, L. A., King, D. L., & Delfabbro, P. H. (2017). Maladap-
Lemmens, J. S., Valkenburg, P. M., & Peter, J. (2011). Psychoso- tive coping styles in adolescents with Internet gaming disorder
cial causes and consequences of pathological gaming. Com- symptoms. International Journal of Mental Health and Addic-
puters in Human Behavior, 27(1), 144–152. doi:10.1016/j. tion, 16(4), 905–916. doi:10.1007/s11469-017-9756-9
chb.2010.07.015 Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT
Lemos, S., Fidalgo, A. M., Calvo, P., & Menéndez, P. (1992). 2010 statement: Updated guidelines for reporting parallel
Validación de la escala de psicopatología infanto-juvenil YSR group randomized trials. Annals of Internal Medicine,
[YSR Infant-Juvenile Psychopathology Scale validation]. Clin- 152(11), 726–732. doi:10.7326/0003-4819-152-11-201006
ica y Salud, 3(2), 183–194. 010-00232
Liebert, M. A., Lo, S., Ph, D., Wang, C., & Fang, W. (2005). Shapira, N. A., Goldsmith, T. D., Keck, P. E., Jr., Khosla, U. M., &
Physical interpersonal relationships and social anxiety. McElroy, S. L. (2000). Psychiatric features of individuals with
CyberPsychology & Behavior, 8(1), 15–20. doi:10.1089/ problematic Internet use. Journal of Affective Disorders,
cpb.2005.8.15 57(1–3), 267–272. doi:10.1016/S0165-0327(99)00107-X
Liu, Q. X., Fang, X. Y., Yan, N., Zhou, Z. K., Yuan, X. J., Lan, J., & Tejeiro, R. A., Gómez-Vallecillo, J. L., Pelegrina, M., Wallace, A.,
Liu, C. Y. (2015). Multi-family group therapy for adolescent & Emberley, E. (2012). Risk factors associated with the abuse
Internet addiction: Exploring the underlying mechanisms. Ad- of video games in adolescents. Scientific Research, 3(4),
dictive Behaviors, 42, 1–8. doi:10.1016/j.addbeh.2014.10.021 310–314. doi:10.4236/psych.2012.34044
Martín-Fernández, M., Matalí, J. L., García-Sánchez, S., Pardo, Therien, P., Lavarenne, S. A., & Lecomte, T. (2014). The treatment
M., & Castellano-Tejedor, C. (2016). Adolescentes con Tras- of complex dual disorders: Clinicians’ and service users’
torno por juego en Internet (IGD): Perfiles y respuesta al perspectives. Journal of Addiction Research and Therapy,
tratamiento [Adolescents with Internet gaming disorder: Pro- 10(6), 2–7. doi:10.4172/2155-6105.S10-006
files and treatment response]. Adicciones, 29(2), 125–133. Thorens, G., Achab, S., Billieux, J., Khazaal, Y., Khan, R., Pivin,
doi:10.20882/adicciones.890 E., Gupta, V., & Zullino, D. (2014). Characteristics and
Mentzoni, R. A., Brunborg, G. S., Molde, H., Myrseth, H., treatment response of self-identified problematic Internet users
Skouverøe, K. J. M., Hetland, J., & Pallesen, S. (2011). in a behavioral addiction outpatient clinic. Journal of Behav-
Problematic video game use: Estimated prevalence and asso- ioral Addictions, 3(1), 78–81. doi:10.1556/JBA.3.2014.008
ciations with mental and physical health. Cyberpsychology, Torres-Rodríguez, A., & Carbonell, X. (2015). Adicción a los
Behavior, and Social Networking, 14(10), 591–596. videojuegos en línea: Tratamiento mediante el programa
doi:10.1089/cyber.2010.0260 PIPATIC [Online video game addiction: Treatment through
Millon, T. (1994). Millon Adolescent Clinical Inventory (MACI). the PIPATIC program]. Aloma, 33(2), 67–75.
Minneapolis, MN: Nacional Computers Systems. Torres-Rodríguez, A., & Carbonell, X. (2017). Update and pro-
Millon, T. (2004). MACI: Inventario Clínico para Adolescente posal of treatment for Internet gaming disorder: PIPATIC
[MACI Clinical Inventory for Adolescents]. Madrid, Spain: program. Revista de Psicoterapia, 28(107), 317–336.
TEA Ediciones. Torres-Rodríguez, A., Griffiths, M. D., & Carbonell, X. (2017).
Müller, K. W., Beutel, M. E., Egloff, B., & Wölfling, K. (2014). The treatment of Internet gaming disorder: A brief overview of
Investigating risk factors for Internet gaming disorder: A the PIPATIC program. International Journal of Mental Health
comparison of patients with addictive gaming, pathological and Addiction, 16(4), 1000–1015. doi:10.1007/s11469-017-
gamblers and healthy controls regarding the big five personali- 9825-0
ty traits. European Addiction Research, 20(3), 129–136. Torres-Rodríguez, A., Griffiths, M. D., Carbonell, X., &
doi:10.1159/000355832 Oberst, U. (2018). Internet gaming disorder in adolescence:
Pallesen, S., Lorvik, I. M., Bu, E. H., & Molde, H. (2015). An Psychological characteristics of a clinical sample. Journal of
exploratory study investigating the effects of treatment Behavioral Addictions, 7(3), 707–718. doi:10.1556/2006.7.
manual for video game addiction. Psychological Reports: 2018.75

Journal of Behavioral Addictions 7(4), pp. 939–952 (2018) | 951

 Torres-Rodríguez et al.

Winkler, A., Dörsing, B., Rief, W., Shen, Y., & Glombiewski, J. A. reality therapy and mindfulness meditation decrease intertem-
(2013). Treatment of Internet addiction: A meta-analysis. poral decisional impulsivity in young adults with Internet
Clinical Psychology Review, 33(2), 317–329. doi:10.1016/j. gaming disorder. Computers in Human Behavior, 68,
cpr.2012.12.005 210–216. doi:10.1016/j.chb.2016.11.038
Wölfling, K., Beutel, M. E., Dreier, M., & Muller, K. W. (2014). Young, K. (2009). Understanding online gaming addiction
Treatment outcomes in patients with Internet addiction: A treatment issues for adolescents. American Journal of Family
clinical pilot study on the effects of a cognitive-behavioral Therapy, 37(5), 355–372. doi:10.1080/01926180902942191
therapy program. BioMed Research International, 2014, Young, K. S. (2007). Cognitive behavior therapy with Internet
425924. doi:10.1155/2014/425924 addicts: Treatment outcomes and implications. CyberPsy-
World Health Organization. (2016). Gaming disorder. ICD-11 chology & Behavior, 10(5), 671–679. doi:10.1089/cpb.
beta draft. Retrieved January 7, 2018, from https://icd. 2007.9971
who.int/dev11/l-m/en#/https%3A%2F%2Ffanyv88.com%3A443%2Fhttp%2Fid.who.int%2Ficd Young, K. S. (2013). Treatment outcomes using CBT-IA with
%2Fentity%2F1448597234 Internet-addicted patients. Journal of Behavioral Addictions,
Yan, W., Li, Y., & Sui, N. (2014). The relationship between recent 2(4), 209–215. doi:10.1556/JBA.2.2013.4.3
stressful life events, personality traits, perceived family func- Zhang, J.-T., Yao, Y.-W., Potenza, M. N., Xia, C.-C., Lan, J., Liu,
tioning and Internet addiction among college students. Stress L., Wang, L. J., Liu, B., Ma, S. S., & Fang, X.-Y. (2016).
and Health, 30(1), 3–11. doi:10.1002/smi.2490 Altered resting-state neural activity and changes following a
Yao, Y. W., Chen, P. R., Li, C.-s. R., Hare, T. A., Li, S., Zhang, craving behavioral intervention for Internet gaming disorder.
J. T., Liu, L., Ma, S.-S., & Fang, X. Y. (2017). Combined Scientific Reports, 6(1), 28109. doi:10.1038/srep28109

952 | Journal of Behavioral Addictions 7(4), pp. 939–952 (2018)