Accepted Manuscript

Game development from a software and creative product perspective a quanti-

tative literature review approach

Henrik Engström, Björn Berg Marklund, Per Backlund, Marcus Toftedahl

PII: S1875-9521(17)30092-7
DOI: https://doi.org/10.1016/j.entcom.2018.02.008
Reference: ENTCOM 249

To appear in: Entertainment Computing

Received Date: 15 August 2017

Revised Date: 23 January 2018
Accepted Date: 22 February 2018

Please cite this article as: H. Engström, B. Berg Marklund, P. Backlund, M. Toftedahl, Game development from a
software and creative product perspective a quantitative literature review approach, Entertainment Computing
(2018), doi: https://doi.org/10.1016/j.entcom.2018.02.008

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers
we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting proof before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Title page

Title: Game development from a software and creative product perspective - a

quantitative literature review approach

Author names and affiliations:

Henrik Engström
PhD, Assistant Professor
School of Informatics
University of Skövde
Box 408
541 28 Skövde
Sweden
[email protected]

Björn Berg Marklund

PhD, Assistant Professor
School of Informatics
University of Skövde
Box 408
541 28 Skövde
Sweden
[email protected]

Per Backlund
PhD, Associate Professor
School of Informatics
University of Skövde
Box 408
541 28 Skövde
Sweden
[email protected]

Marcus Toftedahl
PhD Student
School of Informatics
University of Skövde
Box 408
541 28 Skövde
Sweden
[email protected]

Corresponding author:
Henrik Engström
PhD, Assistant Professor
School of Informatics

1
University of Skövde
Box 408
541 28 Skövde
Sweden
[email protected]
Funding
This
-Kattegat-Skagerrak European Regional Development Fund. Project id: NYPS
20200428

2
GAME DEVELOPMENT FROM A SOFTWARE AND CREATIVE
PRODUCT PERSPECTIVE
A QUANTITATIVE LITERATURE REVIEW APPROACH

Abstract
This article presents the methodology and initial analysis of a systematic literature
review that aims to explore how the craft and processes of game development have
been studied in previous research. In particular, the review focuses on how previous
research treats the inherent duality of video game development, since it both involves
computer software development and creative production. Researchers are often in a
position where they need to emphasize game development’s relation to one of these
disciplines, and it is not unusual for game development to be treated as a direct
offspring of one field with some mild influences from another. Employing a more all-
encompassing review approach, that includes research conducted from the
perspectives of both computer science and the arts and humanities equally, makes the
presented study different from previous literature reviews. The results show that there
is a tendency that the management of software development has a negative correlation
with the management of creativity in the studied material. The heterogenity of the
fields and the limited amount of studies that focus on the duality of game development
suggest that there is a need for a deeper analysis of the individual components and to
synthesize results from disparate fields.

Keywords: Video game; Creative product; Software development; Creativity

management; Systematic literature review

3
1 Introduction
Video games are simultaneously advanced software products and complex works of
creativity and art. This merger of disciplines makes video game production an
interesting process to study from many different perspectives, but it also poses several
challenges for the game development community. While game developers can borrow
influences and find guidance in processes and methods used in both software and
creative production, knowing how to intertwine methods from these two fields in
projects simultaneously is a complicated task. A game development project that adheres
too strongly to either software or creative production methods will likely suffer in one
area or the other. Currently, there is no clear consensus of how software and creativity
can be intertwined during game development, and to understand this challenge better
this paper presents a review of how software and creative production is treated and
understood in games-related research.

1.1 Video games as software

Video game development’ are arguably ingrained deepest in the fields of
software development and computer sciences. As they have their origins in computer
, w “ l” w
sciences, and have followed a similar path of gradual emancipation from rigidity. When
w l l ‘50 ‘60 , w l x
new technologies and their use contexts created challenges in the planning,
management, and execution of software projects. As a means to deal with this
complexity, software engineering started evolving by becoming increasingly influenced
by methods found in applied sciences, mechanical and industrial engineering, and
management research [1]. Although many principles from these fields were found to be
highly applicable and useful in software development, there were some fundamental
mismatches between the theoretically correct way of developing software and actually
developing commercial software while dealing with real-world constraints. While the
academic world embraced the structured and formal approach to software
development, many practitioners in the field struggled to combine structured
development models (e.g., waterfall planning) with the sometimes chaotic and
entangled nature of customer requirements, prototyping, and feature creeping.

This mismatch led to practitioners creating their own alternative, more flexible,
approaches to software development, where the Agile Manifesto [2] in particular
constituted a paradigm shift within the software development community. The Agile
Manifesto presents four core values that constitute the ethos of flexible development,
which all echo strongly in the game development industry:
 individuals and interactions over processes and tools;
 working software over comprehensive documentation;
 customer collaboration over contract negotiation;
 responding to change over following a plan.

The software research community has since started adhering to the practitioner
guidelines established in the manifesto, and Agile development is now widely
considered an integral part of software development practices. The modern software

4
 l l k w l“ l”
engineering and now constitutes its own field: "The nature of software itself thereby
raises the question of how much guidance one may expect from trying to emulate the
development patterns of those engineering disciplines." [1, p. 17]

It is important to mention here that video game developers were perhaps particularly
strongly affected by the mismatch of structured software development and real-world
constraints. Game developers were early in experimenting with less structured means
of software development and have been using Agile approaches long before the Agile
manifesto had been devised [3]. The Agile Manifesto, however, popularized a unifying
term for these types of flexible development methods, and game development is
nowadays seen as being inextricably linked with Agile development methods. As
Murphy-Hill, Zimmermann, & Nagappan stated in a comparison between game
l “ l” software development, [3, p. 6] "It appears that the
unpredictability in games is what makes Agile a good fit".

Game development has always been present in the evolution of software development,
and has both been influenced by, and contributed to, new practices. The game industry
is an interesting part of the software industry for many reasons as it has been driving in
the development of hardware and software for 3D rendering. Games are played on a
wide range of platforms, ranging from simple handheld devices for solitary play to
massive online gaming online communities, and games also constitute a digital market
that is capable of creating great revenue. Video game development thus influences the
practices in the field of software development, while also borrowing many of its work
processes and standards from it. This intertwined relationship is, for example,
described by Ampatzoglou & Stamelos [4, p. 888] who state that "software engineering
for computer games is a field that embraces many techniques and methods from
conventional software engineering".

During the last few decades there has been a growing research interest in game
development processes, and several literature reviews on game development have been
conducted where it has been studied from a software development perspective (e.g.,
[4,5]). While game development has largely been seen as synonymous with Agile
development, an increasingly nuanced picture is starting to emerge from new research
and shortcomings of Agile, in particular its accommodation for creative processes, are
starting to become increasingly apparent.

1.2 Video games as creative products

Video games can not solely be seen and understood as software products, as they can
also be characterized as creative products, cultural expressions, or even artwork, and
the discrepancies within development teams are symptoms of this ambivalence. The
processes of creating a play, a book, or a painting are very different from those involved
in creating inventory systems, security measures, or business-oriented software
products. Agile software development is l “ l ” it is
presented as an alternative with the previously used structured engineering processes.
For artists accustomed with other processes, however, this agility may seem strict by
comparison; for a video game artist, even Agile methods may appear rigid and
creativity-killing [6].

5
The management of creativity in this context is not unproblematic. There is an inherent
contradiction between control and creativity, which makes management of creativity
different from traditional management [7,8]. Malecki [7], for example, present two
different faces of creativity: the managed kind and the unpredictable and
unmanageable. Management of creativity and creative industries has received
significant research interest during the last few decades [9]. The creative industries
have been defined as "those industries which have their origin in individual creativity,
skill and talent and which have a potential for wealth and job creation through the
x l ll l ” [10].

In the creative industry, video games play a much less central and influencing role as
compared to their role in software development. Whereas video game development and
software development have existed within a fairly similar timeframe alongside one
another, most creative disciplines have traditions that span centuries or millennia, as
literature, painting, sculpting, theatre, and music are often considered to be as old as
civilization. Even a comparative youngster such as cinema, a creative discipline that
video games are perhaps most often compared to, has more than 100 years of tradition
in creating artistically interesting pieces of work and throughout its lifespan the
processes of producing films have been continuously refined. Compared to these
creative fields, video game development has only just started to be credited with artistic
interest and still carries comparatively little weight as a creative endeavour. In contrast
to its more equal relationship with software engineering, video game development is
not seen as a discipline that influences creative production but rather one that follows
(or, at times, disrupts) long pre-established traditions. While the situation of video
’ , The National Endowment for the
Arts in the US ‘Games’ category in its grants programs in 2011 [11], games
have not made as much headway in other countries. In Sweden, Norway, and Denmark
for example, there are still no government support programs specifically targeting game
development, and games often need to apply for the same funds as other creative
industries or turn to private beneficiaries.

Game development is often compared to movie production. Movie production has

strong similarities with game production, and was used as an inspiration in the early
days when game studios established their production processes [12]. The production
process for movies is, however, much more standardized and specialised [13] and most
importantly it does not have to accommodate for the challenges associated with
interactivity that are an inherent part of video games. One key difference is that the
authorial control enjoyed by movie makers does not exist in the same way in games. In
game design (whether the game is analogue or digital), the gaming experience and the
meaning a game conveys cannot be totally controlled by the ’ designer. It
materializes in the interaction between the player and the game. This makes game
design a second-order design problem [14] whereas many creative products, such as
movies, are first order design problems. These characteristics give video games
inherent complexities that are not present in most forms of linear media.

On top of these creative challenges, video games also inherit many of the complexities of
software development. There exist several reports outside the field of game
development that discuss and highlight how software development process need to be

6
managed and adapted to accommodate the rhythms of game development [15,16].
When it comes to the management of creativity, however, game development is often
listed alongside other types of creative productions without much regards to how the
creative process is impacted or complicated by the complex nature of software
development (e.g., [17,18]).

1.3 Bridging the gap

In summary, video games constitute a highly interdisciplinary field, and it is difficult to
define the boundaries for which methods or research should or should not be
co “ - l ”. While there are methods and models that attempt to cater
to software developers as well as graphic artists simultaneously, few seem to be
considered useful in real-world development situations. For example, similarly to how
problems emerged when strict engineering principles were applied to the
comparatively flexible field of software development, fundamental mismatches also
appear when Agile software development principles are applied to game development.
For example, Hodgson & Briand [6] report, based on a study of a medium-sized
Canadian game studio, that it was hard to get artists and animators to work with Agile
methods as they experienced them as being too constricting to their creative crafts. As a
result, management introduced alternative approaches to monitor the work of these
groups and that accommodated for the particularities of creative processes and their
comparatively unstructured nature. This exemplifies that even a method such as Agile,
which by many is considered to be a model that accounts for rapid iterative “l ”
development processes, does not meet the requirements and preferences of creative
producers on game development projects.

The systematic literature review presented in this article intends to bridge this gap
between perspectives. The aim is to include and contrast research on game
development seen from both a software and a creative product perspective. We aim to
highlight if and how this dichotomy has been discussed and problematized in previous
research. This incommensurability is something that needs to be strongly considered
when conducting a broad literature review regarding games, as a strict disciplinary
approach has the apparent risk that certain characteristics of game development will be
emphasized on the expense of others. The principle of “if all you have is a hammer,
everything looks like a nail” applies: if all you have is a computer science perspective,
video games look like a software development challenge. A software engineer will focus
on the software characteristics of games, just as a media researcher will focus on its
narrative expression.

Conducting an interdisciplinary review of this nature requires that the review method
accommodates for differences in approaches, terminologies, and research validity
criteria. A large co ’ w
disciplines has been to create such a review method, so the contributions of this paper
is both a description of the method itself as well as a subsequent quantitative analysis of
the review outcomes.

7
2 Review Methodology
Systematic literature reviews and meta-analyses play an important role in medicine,
where clinical results from a number of published studies are often aggregated and
compared to ensure medicine efficacy and understand potential risks of certain
treatments. The structure and methodology used for systematic literature reviews in
medicine can however be applied in other areas [19]. We follow the same proposed
steps, but adjust the execution to the special circumstances of this study. The primary
consideration being that our study includes a range of fields and research traditions,
which makes it very different from evidence-based medicine. Okoli &Schabram [20]
suggest the following eight major steps for a systematic literature review:
1. purpose of the literature review;
2. protocol and training;
3. searching for the literature;
4. practical screen;
5. quality appraisal;
6. data extraction;
7. synthesis of studies;
8. writing the review.
In this section we present the methodological considerations taken in steps 1-7, and in
the subsequent sections we will present the outcomes of the study.

2.1 Purpose of the literature review

The purpose of this study is to explore the process of video game development from a
broad perspective, from which video games are seen both as software and a creative
product. This includes state of the art practices used in the video game industry; how
established software engineering methods are applied; how creativity is managed in
creative industries; and how the duality of video games as both software and creative
products is addressed in the development process.

To manage this broad multidisciplinary scope we have, following the methodological

guidelines stated by [21], formulated four research questions that serve as a
fundamental framework for the review process:
 RQ1: What are the current development practices used in the video games industry?
 RQ2: What formal methods and approaches are used to handle the complexities
related to the software development part of video game development?
 RQ3: What methods and approaches are used to handle the complexities related to
creativity and the development of creative products?
 RQ4: How is the duality of video games as both software and creative products
problematized and addressed in the literature?

Figure 1 illustrates how these questions relate to each other and how they relate to the
research on software development, creative industry and game development
respectively.

8
 Figure 1 The relation between research questions and their position in relation to
software development and creative industry research

The first research question is the broadest and aims to capture game development
practices, as they have been reported in the literature. The focus here is studies that
report on the actual practice used in companies, rather than practices proposed by
researchers. The question is deliberately open. The term "development practices" may
include aspects of team structure, roles and responsibilities, process and activities,
documentation & management culture.

The second research question refines the first one, and focuses on how methods and
approaches from the software development community are used in game development
or studied by game researchers.

The third research question is similar to the second one but with a focus on methods
and approaches from creative industries. As illustrated in Figure 1, the third research
question has a scope that extends further away from game development compared to
the other two. The reason for this is to be able to capture aspects of creative work that
may not yet have been identified in a video game development context, but still may be
relevant. As previously mentioned, software development and game development
already have a deeply intertwined history, and have been around for comparable
timelines. The connections between software development and game development are
thus perhaps more explicit and easier to identify than the connection between game
development and creative production processes. The engineering side of the spectrum
has strong traditions in formulating and adhering to methods and procedures, which we
suspect are less articulated on the art side. With this in mind, our requirements on
papers relating to RQ3 were less strict when it came to how significant their links were
to video game development. The creative industry field is a heterogeneous field [22]
and video game is a relatively young member. The aim of RQ3 is to capture studies on
management of creativity and creative industries. Results from the creative industry
research could thus be applicable to video games, even if the study is not explicitly
pursuing or drawing conclusions regarding these connections. The lessened
requirements on video-game-related conclusions or statements in papers were, in
essence, meant to counteract the already strong bias towards the software development
and computer science perspectives in video game research.

Finally, the fourth research question once again narrows the scope down by specifically
focusing on how the duality of video games has been problematized and addressed in
previous literature. While this question is the main focus of this literature study, the
other three questions are important in its framing. By first describing how processes
related to software and creative production apply to game development or are treated
in game research, the intersection between these two perspectives can be understood in
a more nuanced way.

9
2.2 Protocol and training
A review protocol was developed with a number of areas that together frame the
research questions:
 publication information (year, forum, type of publication);
 research rigour (clearly stated question, method and results);
 regional information (continent, country, focus on regional issues);
 video game focus (empirics from the industry, type of industry, size of company etc.);
 creative industry focus (empirics from creative industry, type of industry);
 management of software development (focus on management issues, use of
methods);
 management of creativity (focus on creativity management issues, use of methods);
 duality of games (is the duality of games as software and creative product discussed
and problematized);
 relevance for this study (overall, relevance for specific research questions);
 additional notes.

The protocol was created as an Excel sheet (see Figure 2) where each area contained a
structured classification part with predefined scales in drop-down lists and free text
sections where the reviewer could give more nuanced descriptions.

Figure 2 An extract of the Excel protocol used to document the review of an article

The protocol was developed collectively in the group of reviewers (4 individuals). After
a first version of the protocol had been devised, a trial evaluation review was conducted
using two articles that represented two ends of the software-art spectrum:

10
  article 1 - A study on how video game development differs from software
development. This article was published at a software engineering conference [3];
 article 2 - A comparison of the creative processes in the Renaissance age with the
digital age. This article was published in a journal on technological forecasting and
social change [23].

None of the reviewers were familiar with the papers beforehand, and independently
read and reviewed both articles using the review protocol. After the reviews were
concluded, the inter-rater reliability ’ l was
analysed. All Likert scaled data for each article was used. The inter-rater reliability
(IRR) among reviewers was assessed using a two-way mixed, consistency, average-
measures intra-class correlation (ICC). The Cronbach's alpha for article 1 was 0.943,
which indicates excellent agreement, and for article 2 it was 0.867, which indicates good
agreement. There were, however, some differences in judgement between reviewers,
which were analysed and discussed. If the protocol itself was used inconsistently, or if
’ tocol questions were incongruent, the
final review results would be unusable. To eliminate differences in protocol
interpretation and paper review methods between authors as much as possible,
variations between author judgements were brought up and discussed. The protocol
was subsequently adjusted to clarify areas in which the protocol seemed to make room
for misunderstandings, and this revised version of the protocol was used in the data
extraction step. By reviewing the same papers together as a group, the process also
allowed the authors to form a more unified approach to reviewing papers and
interpreting their connections to game development. The process was thus not only
l , w “ ” w ll.

2.3 Search process

When it came to devising the algorithms for finding and filtering papers that were
relevant to video game development, the interdisciplinary nature of the presented
literature review made it difficult to adapt conventional procedures of selecting
databases and search terms.

Different research disciplines have different traditions and publication cultures.

Computer science and informatics-oriented research is typically disseminated in
scientific journals and conferences, which are all well indexed in research databases. In
media studies and humanities there is a stronger tradition to publish research in
monographs and books [24]. To create a superset of papers that include both
perspectives equally, the search process thus needed to cast a wide net when it came to
the types of publication venues. The search terms could also not be easily delimited to
just a handful of terms as the terminology differs heavily between disciplines, and many
relevant studies may have been excluded if, for example, we would have assumed that
they would uniformly use the term "development process". To handle this challenge, a
five-phase search process was devised:
 phase 1. Organic identification of a reference set of relevant articles;
 phase 2. Identification of databases that index articles in the reference set;
 phase 3. Formulation of search queries that would return all articles in the reference
set;

11
  phase 4. The databases were queried to collect a list of potential articles;
 phase 5. The resulting list was reduced, based on title and abstract, by removing
articles that met a set of exclusion criteria. Exclusion was based on title and abstract
contents, and served to eliminate papers that matched the search query but were
clearly unrelated to game development (e.g., “game” and “development” could result
in papers of children’s cognitive development while playing games).

In phase 1, a broad explorative organic search was conducted using a large number of
keywords using Google Scholar and a forward- and backward snowballing process: once
an interesting article was found it was scanned and related articles were identified
using both forward and backward referencing, which in turn generated new search
terms. From this initial organic search, 30 articles were found that were identified as
highly relevant for one or more of the four research questions. This set of “ ll
” articles was then used as a reference set in phases 2 and 3 to establish an initial
list of search terms and databases.
In phase 2, databases were selected so that all articles identified in phase 1 would be
represented. In this phase, reliable research databases were queried for each of the
articles in the reference set, and databases were added until all articles had been found
in at least one database (duplicates were removed in the fourth step). The database that
returned the most results in this phase was Scopus, which returned 22 of the 30 articles
in the reference set. This is not particularly surprising given its prominence [25]. Of the
remaining 8 articles, 6 were published by either ACM or Springer, which led to the
inclusion of the ACM digital library and Springer Link database in subsequent searches.
The two articles that were still unaccounted for were published by DiGRA (the Digital
Games Research Association). The final set of databases that was used for the search
was thus Scopus, the ACM Digital Library, Springer Link, and the DiGRA Digital Library.

In phase 3, search queries were formulated with a requirement that they would return
all articles in the reference set on a title-abstract-keyword search in the identified
databases. The very broad scope of this study and the nature of the search process (the
reference set) made this phase more complex than just formulating a single list of key-
words. The initial set of search terms was little more than various synonyms of the
“ ” “ l ”. ll 30 w
not returned with these searches, the set was extended to include terms that were
identified in the abstracts of articles that were not returned. This process was repeated
until a search vernacular that was broad enough to return all reference papers had been
established. The final query was a conjunction of two blocks of disjunctions. The first
(10 )w “ ”, “ ”, “ w
” (16 ) “ l
” ( . ., , , ction etc.). The full
query is shown in Figure 3.

12
 Figure 3 The search query used

In phase 4, the databases selected in phase 2 were queried using the queries that were
formulated in phase 3. The results from these searches were combined into a single list
of articles where duplicates were eliminated. As a final delimitation to the paper search,
the publication year was included in search queries to exclude publications predating
2006 (this is further explained in Section 2.4). The search query was posted to the four
selected databases and it resulted in w’ l 2278 l .

In phase 5, a manual reduction based on a read-through of titles and abstracts was

conducted by the main author of this article. In this phase, the exclusion erred on the
side of caution by being inclusive rather than exclusive. Articles were excluded only
when it was apparent from the title or abstract that they, beyond a shadow of a doubt,
did not meet the inclusion criteria (as described in Section 2.4). The 488 articles that
remained were then evenly distributed among the four reviewers who conducted a
careful review where each article was read in full, reviewed once again in terms of the
inclusion and exclusion criteria described in Section 2.4, and rated using the protocol
described in Section 2.2.

Table 1 shows the number of articles remaining from each database in each review step.
Note that Scopus was used as the primary source which means that duplicates were
removed from the other three databases.

13
 Table 1 Number of articles from the used databases after each step in the review
process
Database Initial Remaining after Remaining after exclusion Remaining after review
search duplicate removal criteria applied to title and (excluding articles with low
based on DOI abstract or very low overall relevance)
Scopus 1012 1012 242 81
Springer 10001 954 211 129
DiGRA 14 7 2 2
ACM 252 175 33 16
Total 2278 2148 488 228

2.4 Inclusion and exclusion criteria

After the search and initial filtration of papers had been conducted, the next part of the
process involved in- l ’ , l
of their quality, and an analysis of l w’ q .

Previous studies [5,26] have shown that published peer-reviewed research on game
development is seldom closely connected to real-world industry practices, and that
there is an over-representation of research conducted in academic development
contexts. Game industry experiences and praxis are more often discussed and reported
in "grey literature" (i.e., non-peer-reviewed books, blogs, or articles written by
practitioners) instead of in academic publication venues. This poses a severe limitation
’ understanding of real-world game development as
academic game projects have scopes, purposes, production environments,
requirements, and value chains that are unrepresentative of the ones developers face in
the games industry. Due to differences in requirements (e.g., testing a hypothesis rather
than having to meet revenue goals), the majority of academic game projects can be
conducted in an environment where only a fraction of the competences present in a
game company are represented. Based on almost two decades of previous research in
the video games field, we have identified a pattern that for studies conducted in
academic settings, emphasis is often placed on making the game just functional enough
to fulfil the requirements of a study. The games are often created at computer science
departments, where less emphasis is placed on “ - l” production values,
aesthetics, and game dynamics that would require the presence of skilled graphic
artists, sound designers, or experienced game designers. Even so, many studies
conducted in these types of contexts make statements or claims regarding how various
work processes, engineering practices, or management styles should be used in or
adapted to the games industry.

A ’ l “ l”
non-representative development contexts is another complicating factor for the
purposes of this study. As this literature review was conducted to better understand
industry practices and the relationship between creativity and software in game
development, basing the review on studies that are disconnected from the industry
would make the epistemological foundation of the work severely flawed. For this

1 This is the maximum number of titles returned by the Springer system.

14
reason, additional exclusion criteria were employed to filter out papers that did not take
industry realities into consideration.

The following inclusion criteria was used in the study:

 studies that target the development process of games (irrespective of whether it is
conventional, educational or serious games);
 studies containing empirics from the game industry;
 studies that focus on other parts of the creative industry and highlight creative
processes and/or production processes that relate to the creative content;
 studies that highlight management of creativity.

The exclusion criteria used were:

 research published before 2006. The reason for this is mainly based on the fact that
game development is a young business and that it goes through a rapid development
which makes studies based on the conditions in the 1990s less relevant;
 the research is solely reporting conclusions and experiences gained from game
development projects conducted in a school or university context;
 non-English articles. The vast majority of research on games is published in English
and the reviewers are unable to review in any other major language.

While these criteria helped by limiting the study to papers that studied topics and
utilized research approaches that matched well w w’ q ,
final filtration based on paper quality was also necessary. As previously mentioned, a
common principle for systematic literature reviews is to appraise the quality of
reviewed research based on the clarity of the research question, method, result, and
quality of the publication process [20]. As this review includes multiple research fields
and paradigms, it is especially important to note that qualitative and quantitative
research approaches have different quality metrics and evaluation principles from one
another. While the selection of search databases automatically provides some level of
quality assurance since most of them only index peer-reviewed publications, or for
databases that include grey literature (e.g., Springer) they have likely gone through
some editorial processes, a round of quality appraisal helps ensure that the papers that
are from widely different publication venues and disciplines all adhere to a unifying
level of quality.

With these things in mind, the initial quality appraisal of the papers that dictated their
inclusion or exclusion in the final review focused on evaluating their research quality in
terms of:
 research question - is the research question clearly formulated;
 method - is the method to address the question clearly presented;
 results - are the results from the conducted study, using the presented method,
clearly presented.

Another main discriminator used in this review is also the overall relevance of the
article for the aim of this study. To ultimately create a pool of papers that were both
clear in their research questions, methodologies, and outcomes, and actually connected
to the games industry, the papers were ultimately judged based on their relevance to
the four main research questions posed by the review. All articles that were rated to

15
have either low or very low relevance when it came to game development, management
of software engineering processes, management of creativity, and the intersection
between these aspects of game development, were excluded from the analysis.

2.5 Data extraction and analysis

To ultimately conduct a thorough review of the contents on the articles that survived
the search filters and quality appraisals, the protocols of all remaining reviewed articles
were aggregated to a single repository structured according to the protocol format. The
Likert scales and nominal data were collated and prepared for statistical analysis.

While compiling and contrasting quantitative data from the protocols is a relatively
straight-forward task, the analysis of the extracted qualitative data from the protocols
( . ., ’ ,
development processes, inclusion of relevant quotes, etc.) is a far more extensive task
that will be presented in future publications focusing on the individual research
questions. The quantitative analysis of reviews is presented in this article. This analysis
has been conducted using R version 3.3.3.

16
3 Results
The literature review was concluded by three senior researchers and one PhD student.
The PhD student, however, has several years of experience from teaching, researching,
and developing video games, and was thus deemed to have the expertise necessary to
be a valid reviewer for this study. The review of papers was carried out over a period of
five months.

3.1 Relevance
At the end of the search and article filtering process presented in Section 2.3, a total of
488 articles were ultimately left to be carefully reviewed. One of those articles was
excluded as it was written in Spanish. This article managed to remain to this stage as it
was indexed in Scopus as being written in English, and the paper also had an English
abstract. Of the remaining 487 articles, 175 were rated in the thorough manual reviews
to be of very low relevance (36%), 84 were judged to be of low relevance (17%), 94 of
intermediate relevance (19%), 68 of high relevance (14%), and 66 to be of very high
relevance (14%). In the analysis presented below, articles with very low or low overall
relevance have been excluded. The remaining 228 articles have a very wide distribution
on the ratings in relation to the four main research questions pursued in this review
(see Section 2.1). That is to say, the different research questions were all represented in
the set of papers, but naturally the papers related to the research questions in different
ways.

The ways in which the papers relate to the four research questions reveal some
interesting indicators as to how the disciplines of software development and creative
production relate to one another and how they relate to research on video game
development. As can be seen in Figure 4, at least 35% of the 228 articles that were
included in the review have low or very low relevance to some research question.

Figure 4 Relevance ratings on individual research questions (n=228)

17
The patterns in Figure 4 alone, however, do not reveal any of the correlations between
the different research questions in terms of their relevance ratings (e.g., there is no
indication that papers rated high in RQ3 are consistently rated in specific ways on other
RQs). By conducting a Spearman's rank correlation, however, this can be examined. A
S ’ k shows a weak positive correlation between RQ1 and RQ2 (ρ=0.497,
p<0.001), RQ1 and RQ4 (ρ=0.381, p<0.001), RQ2 and RQ4 (ρ=0.379, p<0.001), and
between RQ3 and RQ4 (ρ=0.316, p<0.001). No correlation could be observed between
RQ1 and RQ3 (ρ=-0.084, p=0.206). The latter can likely be attributed to the lessened
requirements of video game connections in papers on creative production. What is
more surprising, however, is that there is a weak, significant negative correlation
between RQ2 and RQ3 (ρ=-0.239, p<0.001). This implies that there is a tendency in our
material that articles rated to be relevant with respect to software development are
likely to be less relevant with respect to creativity. The inverse also applies: articles that
are rated to be relevant with respect to creativity are likely to be less relevant with
respect to software development. This negative correlation is maybe not particularly
surprising, but it is an indicator that there is indeed a noteworthy dichotomy between
how software and creativity are addressed in research relating to game development.

3.2 Research Fields

One part of the review protocol involved categorizing which discipline the papers could
be considered to belong to. The classification was based on author affiliations and the
publication venue (e.g., journal title).

The protocol has six proposed categories - art/humanities; software engineering;

management; sociology/anthropology; and user experience. These were identified
based on the outcome of the organic search. In addition to the predefined categories,
there was also an "other" category where the reviewers could provide a free text
description. The free text description revealed that handfuls of articles were described
as being either information system research, media studies, or as a mix of two or more
of the predefined categories.

Table 2 shows the median rating of articles, divided per field. As can be seen, all fields
have a clear representation, although software engineering and management categories
were clearly the biggest (with 56 and 57 articles respectively).

18
 Table 2 The fields of the selected studies and the median rating per field
Field Number Median Median Median RQ2 Median Median
of Overall RQ1 (video (software RQ3 (creative RQ4 (duality
articles Rating2 games) dev.) industry) software-
creativity)
Art/humanities 19 High Low Very Low Intermediate Low
Business 24 Intermediate Intermediate Low Intermediate Low
Software 56 High Intermediate High Low Low
engineering
Management 57 Very High High Intermediate High Intermediate
Sociology/ 14 High Intermediate Low High Low/
anthropology Intermediate
User 20 High Intermediate Intermediate Low/ Low
experience Intermediate
Other 38 High Intermediate Low Intermediate Low
Total 228 High Intermediate Low/ Intermediate Low
Intermediate

Table 2 shows the median rating of articles for overall relevance and relevance for
individual research questions. There are some notable differences between the fields
with respect to the relevance criteria. The management field has a significant higher
overall rank compared to the other fields (p<0.001, using a Wilcoxon rank sum test).
The business field has a significant (p=0.042) lower overall rating compared to the
other fields.

It appears that the duality between software development and creativity is best
represented in the management field. It has a significantly higher ranking on RQ3
(p<0.001) and RQ4 (p<0.001) and is on par in RQ1 & RQ2. This is, however, not to say
that individual articles cover the duality particularly well. The median ranking on RQ4 is
intermediate in the management field and this is the highest value in any field.

The art/humanities studies have a significant lower rating on both RQ1 (p<0.001) and
RQ2 (p<0.001) compared to the other fields. These studies appear to have been
included mainly based on their relevance on RQ3, although the median relevance on
this question is only intermediate. As a contrast, the software engineering field has a
significantly higher rating on RQ2 (p<0.001) and a significant lower rating on RQ3
(p<0.001). The sociology/anthropology field has a high rating on RQ3 but it is not
significant.

To conclude, the management field has a high overall relevance and deals with the
duality of creativity and software development more than research from other
disciplines. Just as indicated by the relevance analysis in the previous section, the
categorization of papers also reveals the presence of a dichotomy between software and
creativity in game development research. The software development perspective (RQ2)

2The overall rating was used to select articles, so the range for selected articles was intermediate to very
high. For RQ1-RQ4 the range was from very low to very high.

19
is highly ranked only in the software engineering field, and management of creativity
(RQ3) is highly ranked only in the management and sociology/anthropology fields. In
other words, there seems to be a tendency of research isolating itself to one specific
component of game development to the exclusion of other perspectives, which might be
problematic in a field that is supposedly as interdisciplinary as video game
development.

3.3 Type of publications and publication year

The type of publications venues of the selected articles was almost evenly distributed
among book chapters (36%), conference papers (35%) and journal articles (27%).
Differences in research traditions are reflected in the type of publications that
dominates in the different fields. Book chapters are dominating in the art/humanities
(47%) and business fields (58%). Conference papers are dominating in the software
engineering (52%) and user experience (60%) fields while journal articles dominate in
sociology/anthropology (50%). Management has a relatively even distribution in
publication forms - 33% book chapters, 30% conference papers, and 35% articles.

There is no clear trend when it comes to number of publications per year, as can be seen
in Figure 5.

Figure 5 Number of articles for each year (n=228). Note that only the first quarter
of 2016 is based on definitive numbers from the review process.

The dip in 2016 is explained by the fact that the search was conducted in March 2016
which means that the majority of articles published in 2016 are excluded. Applying the
“ l ” w w od, an estimate of relevant
publications published in 2016 can be produced. According to this estimate, 2016
reaches numbers of relevant publications in line with 2010 and 2013. However, 2016
will not be included in in-depth quantitative discussions of yearly publications. The
average number of articles in the period 2006-2015 was 22. A slight increase can be
seen if we compare the first half of this decade (average of 20 articles 2006-2010) with
the second half (average of 24 articles 2011-2015). The difference is, however, quite
small and there is not enough data to say anything conclusive regarding changes in
publication rates from this observation alone.

20
3.4 Regions
The origin of articles was classified based on the home university of the first author.
Table 3 shows the percentages of articles overall ranking separated on continent. All
continents, except Africa were represented in all cases. There is a very strong
dominance of Europe followed by North America. Only 15% of the articles come from
Asia, Australia or South America.

Table 3 Percentage of articles per region and overall rating

Continent Intermediate High Very High Total
Africa 0.0 0.0 0.0 0.0
Asia 3.5 3.1 0.4 7.0
Australia 1.3 0.4 0.9 2.6
Europe 20.2 17.1 18.0 55.3
North America 14.0 8.3 7.5 29.8
South America 2.2 0.9 2.2 5.3

The vast majority of articles relating to game development took little to no

consideration of regional aspects in their research approaches. That is to say, most
papers seemed to work under an axiom that game development is a fairly globalized
practice and regional aspects that might affect development are seldom discussed (e.g.,
cultural context, legislation, technology infrastructures, etc.). In 84% of the articles, no
consideration was given to regional aspects, 8.8% of articles very briefly mentioned
regional considerations, 4.8% had a strong focus, and only 2.6% had a main focus on
regional aspects.

3.5 Focus of articles

When examining the general areas of game development that the articles either focused
on or had briefer discussions about, some patterns regarding which topics of
conversation were more popular in the reviewed papers emerge. This distribution is
based on the judgement on the extent to which articles discussed video games, software
development management, creative production and management, or the duality that
emerges when the latter two subjects intersect in game development projects. As Figure
6 shows, video games were, naturally, discussed in a large majority of the papers to
some extent, whereas software development and creativity have a fairly even
representation in the total article set. Finally, the software/creativity duality was found
to be mentioned in papers quite often, with 52.6% of papers at the very least brushing
the subject. However, just under 20% of papers had what could be considered a
discussion on the topic.

The relatively even representation of the different perspectives can be seen as an

indicator that the final set of articles maintained even inclusivity of perspectives, while
also remaining anchored and relevant to game development. Video games were given
significant emphasis in many of the papers, but not in an overwhelming majority that
overshadowed other perspectives.

21
 Figure 6 Percentage distribution of how frequently, and to what extent, different
research foci were represented in the reviewed articles.

l “ ” w l
rooted in, or where it starts its research and line of argumentation from. For an article
that is written from the perspective of video game development, and then studies how
various creative management processes can be applied to improve development
, l ’ , w, “V ”. In this
x l , w l “ l ”
and discussed, while not being the main focus.

In the remainder of this section we will discuss how these different aspects of game
development were represented in the reviewed articles more in-depth, and how the
different research perspectives tended to problematize and analyse video game
development.

3.5.1 Video game focus

As can be seen in both Figure 6 and Table 4, slightly less than half of the reviewed
articles had video games as their main focus (i.e., they explicitly studied video games
first and foremost). Only 12.3% of the reviewed articles did not address video games at
all, leaving 87.7% of the reviewed papers addressing them to some extent. A common
type of example of papers not focusing on video games but addressing them would
usually be papers written from a software or creative perspective, which problematized
how these perspectives needed to adapt to better suit video game development.

Table 4 Focus on video-game-related issues, percentage of articles

Question No Partly Strongly Main Focus
Are video games addressed? 12.3 21.1 20.6 46.1
Is the video game industry addressed? 27.6 25.9 21.9 24.6
Are empirics from the video game industry 47.4 20.2 14.5 18.0
included?

It is notable that there is a big difference in the amount of articles that focus on
videogames (46.1%) and the amount of articles that aim to address the video game
industry (24.6%). This reflects observations from the field of software engineering

22
where a lot of research is being done in academic settings with little connection to real-
world industry contexts [5].

More than half of the studied articles have some form of empirics from the game
industry. These articles (n=120) were further analysed on the type of industry that was
studied. In one third of the cases the empirics came from AAA game studios, in 11% of
the articles it came from Indie game developers, and in 6% it was from serious games
developers. In the remaining 50% of the cases the type of game studies was either not
clearly presented (37%) or could not be matched with any of the predefined categories
(15%).

In conclusion, the literature in the review has a very strong representation of studies of
video games in general. However, this literature review has a specific focus on
examining how practices used in the video game industry is researched and described,
and this type of research turns out to be relatively rare as most video game research is
conducted with little connection to real-world industry practice. About a third of the
articles have clear empirics (strong or main focus) from the video game industry. This
resonance with the rating on RQ1, shown in Figure 4, where 39% of the articles got a
high or very high rating.

3.5.2 Management of software development

Two thirds of the reviewed articles address, to some degree, how software development
could be handled in video game development (as shown in Table 5). In general, these
articles tend to discuss various software development methods and models and
examine whether they are a good match for game development, or which challenges a
development team might encounter trying to use them.

Table 5 Focus on management of software development, percentage of articles

Question No Partly Strongly Main Focus
Is management of software development addressed? 34.6 32.5 14.0 18.9
Are any particular development methods mentioned? 54.8 20.2 12.7 12.3

Among the methods that are discussed in these articles, Agile methods stand out as a
strong majority, and Scrum stands out as the far most common sub-category of Agile
development. This ratifies previous research that has reported similar findings
’
software development in the video game sector [3,27]. Requirements engineering is
often brought up as one aspect of software development that is difficult to translate to
video game development contexts, mainly due to the fact that the requirements that
games need to fulfil are highly subjective and difficult to define, for example stating that
"representing or specifying non-functional requirements ll ” [28, p. 299].

Similar discussions of other software development methods and their incongruity with
video game development were frequently encountered in the reviewed papers, which
further indicates devel ’ l create software frameworks for
games while simultaneously accommodating for the fluid nature of creativity. For
example, the issues surrounding requirements engineering also seem to apply to the
evaluation phases of software and game development projects. Several papers included
in the review discussed how testing and evaluation processes differ between software

23
and game development processes (e.g., [16,29,30]), and point out that the loose and
subjective requirements that drive video game development also affect evaluation
processes. Primarily, the loosely defined requirements lead game developers to use
explorative testing to iteratively fine tune user experiences, rather than employing pre-
planned approaches with a set list of goals and improvements [16,30]. In software
development, testing is comparatively structured, and distinguishing whether or not a
requirement has been met is a less ambiguous process. This is not to say that technical
q ’ game development, but as stated in [16, p. 9],
“ ll l ,w
of the technical software l k .” T
the game development process is thus also present in the evaluative processes carried
w ’ , w
technically oriented responsibilities.

3.5.3 Creative industries and management of creativity

A majority of the total set of articles emphasized creativity in some way, with only
34.6% of the paper circumventing the topic entirely. Table 6 shows the result of the
review with respect to creativity and creative industries.

Table 6 Focus on creative industry, percentage of articles

Question No Partly Strongly Main Focus
Is creativity addressed? 34.6 28.5 18.4 18.4
Is the creative industry addressed?3 54.8 20.6 12.3 12.3
Are empirics from creative industries included? 73.2 13.6 3.9 9.2

Most of the discussions surrounding creativity were anchored in a video game context
in some way or other. As discussed in Section 3.5.1, however, there were some articles
that did not consider video games but had a focus on other types of creative industries.
These were articles that had a main focus on the following areas: movie; theatre;
perfumery; haute cuisine; architecture; interior design; music; food ideation;
magazines; animation industry; fashion industry; and visual effects.

The management of creativity (irrespective of whether it is specifically focused on video

games) was not addressed in 40% of the articles, as seen in Table 7. This is a slightly
higher number compared to software development management, shown in Table 5.

Table 7 Focus on management of creativity, percentage of articles

Question No Partly Strongly Main Focus
Is management of creativity addressed? 39.9 36.0 14.0 10.1
Are any particular methods mentioned? 67.5 17.5 10.5 4.4

In general, there was less focus on the management of creativity compared to software
management. As can be seen in Table 7, 10.1% of the articles had a main focus on
management of creativity but only 4.4% had a main focus on creativity methods. The
corresponding figures for software development are 18.9% and 12.3% (Table 5). It
appears that methods are less important when creativity is in focus. The methods that
are mentioned include a large number of methods from the software development field

3 Video games were excluded from consideration in questions regarding the creative industry.

24
(Scrum and other Agile methods, emotional requirements engineering etc.). In some
cases, it is explicit adaptations of software methods to be more flexible and
accommodating for creative work. This indicates that many of these studies approach
creativity from a software development direction, rather than from a perspective where
the creative process is the baseline.

When it comes to describing methods and models that guide design processes in game
development, Design Patterns are mentioned in a few cases. This is an interesting
example of a method that has its origin in the creative industry (architecture) but has
been widely adopted in the software industry, and also transferred to a game design
context. It is, however, debatable to which extent the use of Design Patterns can be seen
as a method to manage creativity. McGee [31, p. 1], for example, describes the use of
Patterns in software and game development as primarily being a documentation of
l k w w k: “ l w k
formalizing the necessary elements and stylistics of Pattern creation, such work is often
done in terms of a desired result rather than particular techniques for achieving those
results.“ So, Patterns are primarily being used to describe designs that are known to
work well, but do not necessarily discuss the processes that produce them. While
Patterns can have some positive impact on creative work by giving developers a target
to strive for, describing them as creative management methods might be
misrepresenting their current use.

3.5.4 Duality of games

The main aim with this study is to analyse how the duality of game development, as
being both software and a creative product, is addressed in the literature. Table 8 shows
the percentage of articles that explicitly focused on understanding this duality.

Table 8 Focus the duality of games, percentage of articles

Question No Partly Strongly Main Focus
Is the duality of video games as both software and 47.4 32.9 13.6 6.1
creative product addressed?

As can be seen, 47.4% of the reviewed articles did not focus on discussing or
researching this duality. In 32.9% of the articles there was some partial discussion
regarding these aspects, much along the lines discussed previously where the issue was
casually mentioned. For example, research that mainly focused on software
development processes sometimes have brief mentions of how they could potentially
accommodate for creative processes. In 13.6% of the articles there were a stronger
focus and more extensive debates regarding how software development processes and
creative work intertwine in game development, and in 6.1% there was a clear main
focus to better understand this duality and its ramifications for game development.

It is thus apparent that game development and the challenges involved in developing
products that are born from both creative work and software development are
addressed in the literature. These 6.1% constitute a total of 14 articles that focus on the
duality of games. Interestingly, they represent five out of the total seven research fields
identified in this study, excluding only business and user experience. There is hence a
small and disparate research interest in this particular area of video game development.

25
3.6 Highly Relevant Articles with Research Rigor
Approximately 15% of the articles did not have a clear research question that could be
identified by the reviewer. The majority of these were articles that appeared as book
chapters, and software engineering research in particular seems to be slightly over-
represented in this regard. 25% of the articles did not have a clearly presented research
method, and this was once again mainly the case for book chapters. Finally, 14% of the
articles did not have clearly described results.

It may seem contradictory that articles without clear results were considered relevant
in the overall assessment. These articles have been included as they present an
insightful background, reports interesting observations, highlight relevant questions,
and otherwise discuss matters that are relevant to video game development. These
discussions were thus deemed important even though they were done outside a clearly
specified question-method-result framework. In total 62 out of 228 articles (27%) had a
clear research question, method and clear results.

Of the 62 articles that had a clear question, method and result, 31 articles were
considered to have very high overall relevance. Interestingly, all research fields are
represented in these 31 articles. The largest field was management with 14 articles
(45%). Software engineering was the second largest field with 7 articles (23%) followed
by other with 4 articles (13%). The remaining fields only had one or two articles.

If we look at articles with a main focus on the software/creativity duality (discussed in

Section 3.6.5) and how they meet a requirement on research rigor, we can conclude that
the already small set of articles can be further reduced. Only eight articles that focus on
the software/creativity duality of video games meet the requirement of high research
rigor. If a very strict literature review methodology is applied (where we adhere to
quality appraisal as a requirement of article inclusion), we thus end with the eight
articles listed in Table 9.

Table 9 Articles with very high relevance and research rigor that focus on the duality
of games as both software and creative products.
Title Reference
Is requirements engineering useless in game development? [32]
What do game developers test in their products? [16]
Games are not convergence: The lost promise of digital production and convergence [33]
Towards an understanding of game software development processes: A case study [34]
State-of-the-Art: Design of the Semantics Acquisition Games [35]
Opening up to agile games development [36]
Collaboration in serious game development: a case study [37]
Software Architectures and the Creative Processes in Game Development [38]

These eight articles form a basis for answering RQ4. The large heterogeneity observed
in research disciplines and their focus on various parts of the research questions make
it important to be inclusive when studying individual research questions in order to
understand the main question. The articles in Table 9 are all, except for one, from the
management and software engineering fields. There is thus a risk that aspects of
creative work will be lost if a more rigid filtering of articles is applied. Furthermore, this

26
indicates that creativity, in the context of this study, is seldom discussed on equal terms
as management and software development. Rather, creativity is often discussed through
l l ’ l ,w
strongly informs the ways in which creativity is considered. Often, it is seen as a
mystical intractable force that just needs to be dealt with through an increase in
flexibility within software development practices. This leads to creativity being
understood primarily in contrast or in opposition to management and software
practices. Thus, even when creativity is being discussed, it is done through the lens of
these disciplines.

3.7 Limitations
The conducted review has a very broad scope as it includes several different research
disciplines and paradigms. The reviewers will have a bias affected by their own
background, when approaching these articles. This background varies for the individual
reviewers, ranging from computer science, informatics, and game design. All have many
years of experience from video game research and have been working in
interdisciplinary projects including both quantitative and qualitative studies as well as
pure artistic work. The background of reviewers will still, to some extent, affect the
result of the review.

The presented work excludes publications published prior to 2006. This is a limitation
motivated mainly by the rapid development in the game industry. It is still possible that
some highly relevant articles have been excluded in the presented review. It should be
noted that the amount of publications prior to 2006 is clearly less than after. In Scopus,
for example, 234 additional papers would be added if the exclusion criteria would not
be applied (less than 20% of the total number of returned articles).

The presented study shows that the majority of the research comes from Europe. It is
not surprising that Europe plays an important role, but a clear dominance is somewhat
surprising. It is important to note, however, that the presented study was conducted in
Europe by European researchers. Differences between regions in traditions and culture
may affect the outcome and it is possible that a more diversified group of reviewers
would have given a slightly different result. Naturally, only including articles written in
English strongly affects the regional distribution of articles as well.

The analysed review data are ordinal and comparisons have been made accordingly
using ordinal measures. The scales used have had few steps and the median is a quite
blunt measure. One should therefore be careful with both observed similarities and
differences. The statistical tests presented are, however, applicable.

27
4 Conclusions
This article presents a systematic literature review which addresses video game
development processes from a broad perspective, acknowledging that these processes
both have highly structured components, which it shares with other types of software
development, as well as highly creative components which it shares with other forms of
creative products. The study formulates four research questions which cover video
games development in general, the software development aspects of it, aspects
regarding development of creative products, and finally how the duality of video games
as both software and creative products has been studied.

An initial set of 2148 articles was narrowed down to 488 articles that were reviewed
using a standardised protocol. Almost half of these articles (228) were found to have at
least intermediate relevance for the overall aim of the study. The result from analysis of
the quantitative elements in the review protocol for these articles has been presented in
this article. The broad scope has led us to adopt a methodology that extends a
traditional keyword search approach to conduct a more thorough mining process that
aims to return a set of relevant articles identified through an organic search process.

The study shows that highly relevant research has been published in many different
research fields ranging from software engineering to art/humanities. The management
field is the one with most relevant research and that also seems to have the best balance
between perspectives. The software engineering field has a stronger relevance
regarding software development aspects, but has a much lower relevance on
management of creativity. There is a great variation in how the different research
questions are addressed. One thing that our result indicates is that most research that
address the double nature of games comes from a software engineering or management
perspective. The arts/humanities field is represented but at a much lower scale. One
example of this is when it comes to methodological approaches to manage creativity.
The most commonly mentioned methods were adaptations of methods from the
software development tradition. It appears that even if guidelines and principles for
creative production exist, they are much looser and disparate than is the case for
software development. This is most likely in the nature of the field of video games but
unless care is taken, the structured and established approaches will overshadow the
subtler ones.

An awareness of challenges in software development is almost not present at all in

articles that come from an art/humanities or sociological perspective. In fact, the article
’D ll [33] is the only publication in Table 9 that is not from the software
engineering or management field. This may be problematic as the artistic and creative
dimensions may be lost when they are framed by an engineering or management mind-
set. On the other hand, game projects in the arts/humanities field may encounter huge
problems if the inherent complexities of software development are not sufficiently
acknowledged.

The Agile Software Development Manifesto was declared by a number of practitioners

in early 2000 as a reaction to the engineering-dominated practices proposed by
managers and researchers. The Agile principles have been embraced by game

28
companies and are widely accepted. Game production is not run by software developers
alone however. Composers, animators, graphic artists, game writers and game
designers are all integral parts of the production. A strict software-oriented application
of Agile methods has a risk to introduce the same tensions that gave rise to the Agile
Software Development Manifesto. At the same time, the production practices used in
other creative industries, such as movies & theatre, may not at all address the inherent
complexities of software development and the challenges of including the audience to
interact with the product. This study shows that the awareness of challenges regarding
software development is very low in the arts/humanities field. The awareness of
challenges regarding creative work is low in the software engineering field. There is
much that suggests that game development should be handled as a craft in its own right,
where software development is combined with the creative processes and where the
user/audience/player has an important, interactive role.

Our analysis also reveals a worrying negative correlation in ranking between articles
with respect to their relevance from a software development perspective compared to
their relevance from a management of creativity perspective. This begs for further
analyses of game development processes that carefully try to avoid this negative trend.
The presented literature review forms a basis to find a deeper understanding of how to
handle this dichotomy. In future studies we will conduct a thorough analysis of the
individual research questions which will enable a more qualitative comparison of the
similarities and differences between game development and other creative products.

Acknowledgment
[Removed for review]

29
References
[1] M.S. Mahoney, Finding a history for software engineering, IEEE Ann. Hist. Comput.
26 (2004) 8–19. doi:10.1109/MAHC.2004.1278847.

[2] Agile Alliance, Agile manifesto, 2001. http://www.agilemanifesto.com.

[3] E. Murphy-Hill, T. Zimmermann, N. Nagappan, Cowboys, ankle sprains, and keepers

of quality: how is video game development different from software development?, in:
Proc. 36th Int. Conf. Softw. Eng., ACM, 2568226, 2014: pp. 1–11.
doi:10.1145/2568225.2568226.

[4] A. Ampatzoglou, I. Stamelos, Software engineering research for computer games: A

systematic review, Inf. Softw. Technol. 52 (2010) 888–901.
doi:10.1016/j.infsof.2010.05.004.

[5] A. Osborne O’Hagan, R. V O’Connor, Towards an understanding of game software

development processes: A case study, in: Eur. Conf. Softw. Process Improv., Springer,
2015: pp. 3–16.

[6] D. Hodgson, L. Briand, Controlling the uncontrollable: “Agile” teams and illusions of
autonomy in creative work, Work. Employ. Soc. 27 (2013) 308–325.
doi:10.1177/0950017012460315.

[7] E.J. Malecki, Creativity: Who, How, Where?, in: P. Meusburger, J. Glückler, M. el
Meskioui (Eds.), Knowl. Econ., Springer Netherlands, Dordrecht, 2013: pp. 79–93.
doi:10.1007/978-94-007-6131-5_5.

[8] B. Townley, N. Beech, A. McKinlay, Managing in the creative industries: Managing the
motley crew, Hum. Relations. 62 (2009) 939–962. doi:10.1177/0018726709335542.

[9] M. Hutter, Creative Production in the Creative Industries, in: G. Buenstorf, U.

Cantner, H. Hanusch, M. Hutter, H.-W. Lorenz, F. Rahmeyer (Eds.), Two Sides Innov.
Creat. Destr. Evol. Capital. Econ., Springer International Publishing, Cham, 2013: pp.
159–165. doi:10.1007/978-3-319-01496-8_8.

[10] DCMS, Creative Industries Mapping Document, Creat. Ind. Mapp. Doc. (2001).
https://www.gov.uk/government/publications/creative-industries-mapping-
documents-2001.

[11] NEA, Media Arts Fact Sheet, 2016.

https://www.arts.gov/sites/default/files/Media_Arts_fact_sheet_nov2016.pdf.

[12] M. Ramsay, Trip Hawkins, in: M. Ramsay (Ed.), Gamers Work Stories Behind Games
People Play, Apress, Berkeley, CA, 2012: pp. 1–15. doi:10.1007/978-1-4302-3352-7_1.

[13] M. Pankow, Explorative Case Study Analysis, in: M. Pankow (Ed.), Component-Based
Digit. Movie Prod. A Ref. Model an Integr. Prod. Syst., Gabler, Wiesbaden, 2008: pp.
31–78. doi:10.1007/978-3-8350-5589-6_4.

[14] K. Salen, E. Zimmerman, Rules of Play: Game Design Fundamentals, MIT Press,
2004.

[15] F. Petrillo, M. Pimenta, Is agility out there? Agile practices in game development, in:
SIGDOC 2010 - Proc. 28th ACM Int. Conf. Des. Commun., 2010: pp. 9–15.
doi:10.1145/1878450.1878453.

[16] J. Kasurinen, K. Smolander, What do game developers test in their products?, in: Int.

30
 Symp. Empir. Softw. Eng. Meas., 2014. doi:10.1145/2652524.2652525.

[17] C.A. Scolari, Transmedia Storytelling: Implicit Consumers, Narrative Worlds, and
Branding in Contemporary Media Production, Int. J. Commun. 3 (2009) 586–606.

[18] A. Marcos, N. Zagalo, Instantiating the creation process in digital art for serious
games design, Entertain. Comput. 2 (2011) 143–148.
doi:10.1016/j.entcom.2010.12.003.

[19] B. Kitchenham, O.P. Brereton, D. Budgen, M. Turner, J. Bailey, S. Linkman,

Systematic literature reviews in software engineering–a systematic literature review,
Inf. Softw. Technol. 51 (2009) 7–15. doi:10.1016/j.infsof.2008.09.009.

[20] C. Okoli, K. Schabram, A Guide to Conducting a Systematic Literature Review of

Information Systems Research, Work. Pap. Inf. Syst. 10 (2010) 1–49.

[21] P. Brereton, B.A. Kitchenham, D. Budgen, M. Turner, M. Khalil, Lessons from

applying the systematic literature review process within the software engineering
domain, J. Syst. Softw. 80 (2007) 571–583. doi:10.1016/j.jss.2006.07.009.

[22] M. Peltoniemi, Cultural Industries: Product-Market Characteristics, Management

Challenges and Industry Dynamics, Int. J. Manag. Rev. 17 (2014) 41–68.
doi:10.1111/ijmr.12036.

[23] J. Sapsed, F.T. Tschang, Art is long, innovation is short: Lessons from the Renaissance
and the digital age, Technol. Forecast. Soc. Change. 83 (2014) 127–141.
doi:10.1016/j.techfore.2013.09.014.

[24] D. Torres-Salinas, N. Robinson-Garcia, J.M. Campanario, E.D. López-Cózar,

Coverage, field specialisation and the impact of scientific publishers indexed in the
Book Citation Index, Online Inf. Rev. 38 (2014) 24–42. doi:10.1108/OIR-10-2012-
0169.

[25] J.F. Burnham, Scopus database: a review, Biomed. Digit. Libr. 3 (2006) 1–8.
doi:10.1186/1742-5581-3-1.

[26] D. Callele, E. Neufeld, K. Schneider, A report on select research opportunities in

requirements engineering for videogame development, in: 2011 4th Int. Work.
Multimed. Enjoyable Requir. Eng. - Beyond Mere Descr. with More Fun Games,
MERE’11 - Co-Located with 19th IEEE Int. Requir. Eng. Conf., RE’11, 2011: pp. 26–33.
doi:10.1109/MERE.2011.6043942.

[27] J. Koutonen, M. Leppänen, How Are Agile Methods and Practices Deployed in Video
Game Development?, in: Agil. Process. Softw. Eng. Extrem. Program. XP 2013,
Springer, 2013: pp. 135–149. doi:10.1007/978-3-642-38314-4_10.

[28] D. Callele, E. Neufeld, K. Schneider, Emotional Requirements in Video Games, in:

14th IEEE Int. Requir. Eng. Conf., 2006: pp. 299–302. doi:10.1109/RE.2006.19.

[29] J. Kasurinen, J.P. Strandén, K. Smolander, What do game developers expect from
development and design tools?, in: ACM Int. Conf. Proceeding Ser., 2013: pp. 36–41.
doi:10.1145/2460999.2461004.

[30] M. Daneva, Play-Testing and Requirements Engineering: Implications for Research

and Teaching, in: Proc. - 2\textsuperscript{nd} Int. Work. Requir. Eng. Testing, RET
2015, 2015: pp. 9–12. doi:10.1109/RET.2015.10.

[31] K. McGee, Patterns and computer game design innovation, in: Proc. 4th Australas.

31
 Conf. Interact. Entertain., RMIT University, 1367972, 2007: pp. 1–8.

[32] J. Kasurinen, A. Maglyas, K. Smolander, Is requirements engineering useless in game

development?, in: Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell.
Lect. Notes Bioinformatics), 2014: pp. 1–16. doi:10.1007/978-3-319-05843-6_1.

[33] C. O’Donnell, Games are not convergence: The lost promise of digital production and
convergence, Convergence. 17 (2011) 271–286. doi:10.1177/1354856511405766.

[34] A.O. O’Hagan, R. V O’Connor, Towards an understanding of game software

development processes: A case study, in: Commun. Comput. Inf. Sci., Springer, 2015.
doi:10.1007/978-3-319-24647-5_1.

[35] . im o, M. Bieli ov , State-of-the-Art: Design of the Semantics Acquisition Games,

in: Semant. Acquis. Games, Springer International Publishing, 2014: pp. 95–117.
doi:10.1007/978-3-319-06115-3_7.

[36] P. Stacey, J. Nandhakumar, Opening up to agile games development, Commun. ACM.

51 (2008) 143–146. doi:10.1145/1409360.1409387.

[37] M.Q. Tran, R. Biddle, Collaboration in serious game development: a case study, in:
Proc. 2008 Conf. Futur. Play Res. Play. Share, ACM, 1496993, 2008: pp. 49–56.
doi:10.1145/1496984.1496993.

[38] A.I. Wang, N. Nordmark, Software Architecture and the Creative Process in Game
Development, in: Entertain. Comput. - ICEC 2015, Springer, 2015: pp. 272–285.
doi:10.1007/978-3-319-24589-8_21.

32
Highlights
 An extensive literature review approach is presented, which target video game
development.
 Research relevant for game development can be found across a large number of
fields.
 Found a negative correlation between software development and creativity
management.
 Only a few articles address both software development and creativity management.

33