A Shift In Artistic Practices through

Artificial Intelligence

Authors

Kıvanç Tatar* Petter Ericson* Kelsey Cotton*

Chalmers University of Umeå University Chalmers University of
Technology Sweden Technology
Gothenburg, Sweden [email protected] Gothenburg, Sweden
[email protected] [email protected]

Paola Torres Núñez del Roser Batlle-Roca Beatriz Cabrero-Daniel

Prado Universitat Pompeu Fabra, University of Gothenburg
Stockholm University of the Spain Gothenburg, Sweden
Arts [email protected] beatriz.cabrero-daniel@gu
Stockholm, Sweden .se
[email protected]

Sara Ljungblad Georgios Diapoulis Jabbar Hussain

University of Gothenburg Chalmers University of University of Gothenburg
Sweden Technology Sweden
[email protected] Gothenburg, Sweden [email protected]
[email protected]

* These authors share equal contribution as the first authors.

Abstract

The explosion of content generated by Artificial Intelligence models has initiated

a cultural shift in arts, music, and media, where roles are changing, values are
shifting, and conventions are challenged. The readily available, vast dataset of
the internet has created an environment for AI models to be trained on any
content on the web. With AI models shared openly, and used by many, globally, how
does this new paradigm shift challenge the status quo in artistic practices? What
kind of changes will AI technology bring into music, arts, and new media?
Current AI techniques in Artistic Practices
Increasing interest and accessibility in Artificial Intelligence
technologies have moved the societal discussion of AI to the mainstream. Ease in
accessing computation resources, open-access AI knowledge, open-source AI
software, and vast datasets in different modalities enable artists to explore new
possibilities. The recent advancements in AI architectures have been applied to
all artistic domains such as visual arts, video art, music and sound art, dance
and performance arts, literature, and interdisciplinary arts.

AI technologies for artistic image and video generation [1] have had to contend
with building an entirely new vocabulary for describing images or frames in
video. Whilst image generation with autonomous painting was established decades
ago (such as the Aaron system by Harold Cohen), it is only relatively recently
that AI in visual domain has become accessible to vast communities thanks to open
source, open-access, and browser-based tools that require minimal local
computation resources, such as StableDiffusion, and Generative Adversarial
Networks such as BigGAN and StyleGAN.

In music and sound arts, four main tracks of AI applications appear [2] and
interactive music systems [3]: approaches working with symbolic music; systems
using audio signals directly; systems that map symbolic music domain to musical
audio or vice versa, and approaches that connect musical domains to other domains
such as bodily movement.

In performing arts, movement generation [4] has been one research track of AI.
The advancements in this domain have created a ripple effect in generative dance
and animation where AI systems have been used as a tool for assistance,
choreography support, performance collaboration, and movement generation.

Text generation and conversational bots [5] have initiated recent discussions on
how omnipresent Large Language Models (LLMs) such as ChatGPT can change all
aspects of society. Additionally, LLMs have been incorporated into
interdisciplinary domains, interactive artworks with chatbots, as well as tool
generation for other artistic domains such as audio synthesizer code or creative
coding script generation examples.

There is currently no consensus in the definition and coverage of the term

Artificial Intelligence in the literature [6]. Even though we mainly cover Machine
Learning approaches utilizing training datasets above, these are often embedded
in complex systems with sensing and action capabilities. These approaches are
alternatively referred to in the literature as Multi-agent Systems [7]. In the
remainder of the paper, we use AI as an encapsulating term to include systems
based on ML approaches, which may or may not have sensing and action
capabilities. We will call those systems as Applied Artificial Intelligence
technologies– or simply, AI–in the remainder of the paper.

Infinite Content in Indefinite Internet

There is a long history of digital art that generates infinite content in
various guises [8]. New AI approaches, such as Stable Diffusion and MusicLM, have
gained widespread interest in public discussions due to their aesthetic
possibilities, when combined with LLMs. These AI models are trained, fine-tuned,
and parameterized through a process involving many actors such as data creators,
AI developers, and model architecture researchers. Artists have been joining all
processes of AI technology, including data creation, developing novel AI
architectures, training specific models, and utilizing readily-available models.
As the technical skill barrier in using AI technology has decreased due to
ease-of-access for known and recent models, all digital platforms have been
receiving a new wave of AI generated content.

As a result of the increased accessibility to AI tools, the values in artistic

production have been shifting from manually-made content to automatically
generated content. AI systems such as Stable Diffusion have been challenging the
conventional values in visual arts practices by generating infinite amounts of
content. In some known cases, the training datasets of widely used AI content
generators were gathered by scraping the content from the internet, even when the
data was held within copyright. A recent Vox short documentary [9] mentions how
the American illustrator James Gurney has become a common style prompt entry in
AI image generation. The artist raises the issue of consent within infinite
content generators. Gurney states,

“I think it is only fair to people looking at this work that they should
know what the prompt was and also what software was used. I think the
artists should be allowed to opt-in or opt-out of having their work that
they worked so hard on by hand be used as a dataset for creating this other
artwork.”

Gurney’s comment highlights the current status quo where AI technology developers
overlook the explicit consent from artists and content owners, and how current
regulations fall short on empowering artists in decisions on the use of their
content. Furthermore, massive content generation in the style of a particular
artist could devalue it, as in the case of James Gurney, or increase it by social
networking, as in the case of Holly Herndon [10] who is an artist shared their
voice as a Machine Learning model (vocal deepfake) for other artist to perform
with her voice. This shift in capital value in cultural practices calls for a new
take in enacted ethics of artistic practices in the era of ubiquitous AI.

Labor in Artistic Data

The increasing requirement of large datasets to mass-generate high fidelity
content with AI has triggered a notable shift in enacted internet ethics of
copying, appropriating, and distributing art and music. Currently, the datasets
that are used to train AI models such as DALL-E, Stable Diffusion, or Github
CoPilot, have been scraping data from the web while ignoring the licences and
intellectual property of data. This is akin to an enclosure of cultural commons
in the digital era, internet, and AI.

The case of James Gurney is one example of how the creators of these widely used,
easily accessible AI models for content generation have been gathering data for
their training datasets: by overlooking the consent of data creators. This
directly exploits the artist’s labor that is put into creating the original
content [11]. The data that is publicly shared on the internet is approached as
“free to take” and transformed into a capital commodity through the process of
training Machine Learning models, exhibiting similarities to historical practises
of exploitatively ignoring cultural context and consent, within the structures of
colonialism and its enclosures. This current status quo in data gathering can be
addressed through empowering the artist by incorporating their explicit consent
into current structures. The consent can be manifested by the artist, as in the
case of Stability.ai, where artists can choose an option so that their work is
not used in the training of Stable Diffusion models. However, it is not clear to
what extent this pledge can be honored, considering the multitude of remixes and
format shifts of popular artists' works on the internet. Other licenses in that
discussion, such as Creative Commons, emphasize the importance of fair-use for
public good [12]. While both proposals highlight valuable aspects, we envisage
that a future of copyright in artistic practices requires platforms and
structures for artistic data sovereignty.

Although existing copyright licences are legally recognised mechanisms for

protecting intellectual property, the current gatekeeping mechanisms in accessing
online platforms have implicit consent structures related to artistic work- such
as terms-of-service agreements and cookie mechanisms. Those agreements are often
produced without participation of artists, thereby benefiting the industry that is
gatekeeping the access to the digital platforms. Whether the copyright
permissions of artists are respected through explicit consent, is an issue of
traceability, accountability, and regulation. Many practitioners in artistic
domains are individuals or small business owners which do not have the power to
counter “big-tech” such as OPEN-AI, in imposing the consent decisions. We still
need tools of traceability, third-party non-profit organizations for proposing
regulations, and public structures for accountability. All three aspects of
traceability, regulations, and accountability require inclusive and participatory
discussions of ethics for the foreseen future with AI.

A Shift in Artistic Practices

The emergence of accessible AI tools for artistic production caused a
natural shift in artistic practices. The notion of “artist as genius”, where a
single person produces a masterpiece, has been shifting towards communal
production where several actors are involved in the artistic production.
Nonprofits such as EleutherAI and for-profits such as Stability.AI, with
involvement of other developers, have been pursuing open-sourcing the
ground-breaking AI architectures, which has been benefiting artistic practices by
initiating public discussions.

Even though AI technology has conventions of open-source and open-access tools,

it is still a discussion if the power structures within these technologies are
truly democratized. The majority of tools for AI development are provided,
hosted, or maintained, by a few technology companies such as Google (Tensorflow,
Collab), OpenAI (DALLE-2, GPT-3), Facebook (Pytorch), and Microsoft (Github).
Although there is a shift toward communal production in artistic practices where
different actors have varying levels of contributions to the artwork production,
these technology companies still hold power and capital over the AI tools for
artistic practices, and their computational resources. The decision-making in AI
technology creation of tools for artistic production have yet to be democratized.

An artwork is in close relation with its medium. Along with ubiquitous AI, the
evolution of new art markets such as social media, streaming platforms, and NFTs
initiated a cultural and monetary value reassignation by aggregating status as
major channels of artwork “storage” and distribution. The value of an artwork is
typically affected by the ranking or curation mechanisms of the medium. Those
mechanisms are directly related to the artwork’s visibility and thereby the value
of the artwork. These platforms now have substantial influence in reshaping
current social conceptions around economic and cultural capital. For example, the
notion of added value through engagement appears in Holly Herndon’s AI system for
voice synthesis, titled Holly+. Herndon trained the AI architecture on their own
voice recordings, and Holly+ is the AI model where users can synthesize recorded
audio in the style of Herndon’s personal voice. Herndon deploys a Decentralized
Autonomous Organization (DAO) blockchain to vote on the minting of artworks made
using Holly+, and they distribute tokens [13] to members of the DAO (and the
creator of minted artworks) to share in any profits from the usage of Holly+
recorded on the relevant blockchain. Here, Herndon acknowledges the value put
into Holly+ by each user when they engage with the AI system. By giving away DAO
tokens, Herndon creates a platform for users to acquire both a voting stake in
the Holly+DAO and to fiscally benefit from the capital value produced by artistic
work made with Holly+. This semi-decentralized governing structure for Holly+ is
centred on connecting value to engagement processes: the more people who ‘engage’
with art, the more valuable the art itself is perceived to be.

Sustainability, Ethics, Accessibility, and Inclusion

A discussion on societal aspects of AI is entangled with the concepts of

sustainability, ethics and values. The broader framing of these issues, focused
on the evolution of culture, and impact upon culture industries, has implications
on the environmental and climate impact of leveraging AI technologies within
artistic practices. Within discussions around sustainability are also broader
concerns with social sustainability of artists, and the sustainability of the
role of art within society. A core concern in this regard is how advancements in
AI technologies deployed within artistic contexts may adversely impact the
cultural economy of artists in society, and further how their role in society is
implicated by the wider usage of AI tools.

The appreciation of art is in the eyes of the beholder. A larger discussion in

this regard is centered on the intrinsic nature of AI tool usage as originating
from a desire to “be better”, and thereby enable “better” artistic work. This
highlights questions regarding what the nature of “better” actually is, who is
the decider of this, and if this continual pursuit of “better” is an enacted
artistic Darwinism. Whilst conclusive decisions on how power, relevancy, and
“better” are yet to be navigated, it is plainly apparent that the positioning of
interdisciplinary borders are significant in shaping the value systems in the role
AI occupies within cultural industries. To this end, the rigidity (or fuzziness)
of these boundaries need to facilitate equitable exchange, promote artistic
diversity and encourage an artistic culture of mindful progress.

Figure 1 - Countries with at least one organization (e.g., government,

university) issuing AI guidelines for responsible and ethical AI development and
implementation (not necessarily laws, as of 2019) are Australia, Canada, EU,
Finland, France, Germany, Iceland, India, Italy, Japan, Netherlands, Norway,
Singapore, South Korea, Spain, UAE, UK, and USA. Populations of these countries
(as of 2021): 2796.76 millions compared to the total world population of 7888
millions. Therefore, these organizations make recommendations for 35% of the
world population [14]. (Creative Commons 0 license – Public domain.)

A world with more widely available AI art creation tools and globalized
platforms, giving more people than ever the ability to create and share art,
could lead to a greater chance to find a wider audience for previously unknown
and/or marginalized creators, styles, and art forms. Currently, however, the same
world and those same platforms are more often built to entrench the dominance of
the hegemonic culture and existing artists. There is still more to be done for
improving inclusion [15]. Geographical borders still matter in accessibility of
the internet, and protection of rights in AI applications (Fig 1). Therefore, the
community must take action to address the active participation of
underrepresented cultures beyond the passive appropriation of their imaginaries.
Cultivating diversity requires ethical considerations in all processes of all
stakeholders and actors of AI; while reducing bias, and analyzing stereotypes and
representation of individuals and their creations.

Some ubiquitous AI models have been shown to reproduce problematic stereotypes in

the data [16], which also may appear in artworks that are generated using these
models. It is often hard for a single user or artist to become aware of those
stereotypes in AI models since they become apparent after many output generations
on similar prompts, and using statistical analysis on the generated content [17].
In the case of tools such as DALL-E/Midjourney/Stable Diffusion, it is worth
noting the difference in the quality of the images generated when the prompted
text used for generation refers to images, symbols or art that belongs to the
hegemonic culture (undoubtedly of easier access) when compared to the outputs of
prompts that relate to more “obscure”, underrepresented, or non-hegemonic
cultural manifestations. Cultural hierarchies could indeed be extrapolated from
how much detail, quality or “realism” (or lack thereof) the generated images
finally contain.

A Better Future for Artistic Practices with Artificial

Intelligence
From data production and curation to model design, implementation, training,
configuration and final use; a human is involved at every point of AI technologies.
Ethical considerations emerge in the decisions of all stakeholders and actors, in
addition to the artists. The conceptualizations and values of all involved
parties leave residues behind that influence the artworks and the culture that
they situate in. The AI technology, with an immense power of shifting the culture
and society globally, goes beyond proprietary software rights of a single
for-profit company. Public discussions on what would be a future society with AI
are critical.

Primarily, a fundamental change in power and the distribution of power is

necessary for inclusivity in regard to the decision-making of all AI system
designs. In the case of recommendation systems, inclusive decision-making
processes can have significance in the invisibilizing and hypervisibilizing of
artworks and artists, especially those from underrepresented and marginalized
communities, cultures and regions. Additionally, a crossover between art and
technology within pedagogical contexts is beneficial so that artists acquire the
knowledge, skills, and access to make artistic use of new AI technologies, and
that technologists are given time, training, and opportunity to explore artistic
pursuits and their requirements.

Copyright is an important aspect of artistic practice in a digital world that is

further highlighted and complicated by the introduction of widely available and
powerful AI technologies. Exact future steps on consent, traceability,
regulations, and accountability are not clear yet. However, the discussions
within e.g. various pirate parties, digital rights groups, hacktivist
organizations, open-access global initiatives of technology replications (such as
GPT-NEO, GPT-J, StableDiffusion and DALL-E 2 pytorch replication) are
significantly more instructive than the various rear-guard actions fought by
various other industry organizations. It is evident that there is an immense need
for reform of the current regime of “might makes right”, wherein large
corporations can infringe copyright with absolute impunity, shedding light on
problematic power relations. The discrepancy between corporate copyright
transgressions in the creation of enormous datasets as input to AI systems with
no knowledge or consent from the copyright holders and the automated “copyright”
takedowns of private individuals’ meticulously researched fair-use remix and
commentary works on media platforms highlights the differences in power and
punishment across private versus industrial use. Legislations, communal
guidelines, and ethical dimensions of AI technologies for artistic practices are
ongoing societal discussions. At the time of writing of this paper, Italy has
banned the popular chatGPT, citing “security concerns” for Italian citizens in a
move that may be followed by other countries, and will not allow the access to
this new technology until their data protection to-do list is implemented [18],
whereas countries such as Japan are considering softer regulations in which the
content usage for AI model training is widely allowed [19]. As an alternative to
draconian measures, it is long overdue and the perfect moment in time to include
artists and practitioners proactively in these discussions. Accommodating and
fusing different voices and knowledge is a must for the reformation of equity,
equality, and justice in AI technology creation. Art is for everyone, and the
tools we use to make art, especially AI tools, should enable and empower just and
equitable creation.

Acknowledgement
The first, second, and third authors share equal contributions as the first
authors. This work was partially supported by the Wallenberg AI, Autonomous
Systems and Software Program – Humanities and Socie-ty (WASP-HS) funded by the
Marianne and Marcus Wallenberg Foundation and the Marcus and Amalia Wallenberg
Foundation.

References

[1] F.-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah, “Diffusion Models in

Vision: A Survey.” arXiv, (2022), doi: 10.48550/arXiv.2209.04747.

[2] F. Carnovalini and A. Rodà, “Computational Creativity and Music Generation

Systems: An Introduction to the State of the Art,” Frontiers Artificial
Intelligence 3, p. 14, Apr. 2020, doi: 10.3389/frai.2020.00014.

[3] K. Tatar and P. Pasquier, “Musical agents: A typology and state of the art
towards Musical Metacreation,” Journal of New Music Research 48, no. 1, pp.
56–105, (2019), doi: 10.1080/09298215.2018.1511736.

[4] D. Bisig, “Generative Dance - a Taxonomy and Survey,” in Proceedings of the

8th International Conference on Movement and Computing, pp. 1–10, (2022), doi:
10.1145/3537972.3537978.

[5] T. Iqbal and S. Qureshi, “The survey: Text generation models in deep
learning,” Journal of King Saud University - Computer and Information Sciences,
34, No. 6, Part A, pp. 2515–2528, (2022), doi: 10.1016/j.jksuci.2020.04.001.

[6] M. Haataja and J. J. Bryson, “Reflections on the EU’s AI act and how we could
make it even better,” TechREG™ Chronicle, March 2022.

[7] Michael Wooldridge, An Introduction to MultiAgent Systems (West Sussex,

England: John Wiley & Sons, 2009).

[8] Boden, M. A., and Edmonds, E. A., “What is generative art?” Digital
Creativity 20, no. 1, pp. 21–46, (2009), doi:10.1080/14626260902867915.

[9] Joss Fong, “The text-to-image revolution, explained,” Vox, 1 June 2022, <
https://www.youtube.com/watch?v=SVcsDDABEkM >, accessed 16 April 2023.

[10] Holly Herndon, “Holly+ 👭 🗣 🦾,” 13 July 2021,

<https://holly.mirror.xyz/54ds2IiOnvthjGFkokFCoaI4EabytH9xjAYy1irHy94>, accessed
12 April 2023.
[11] I. Arrieta-Ibarra, L. Goff, D. Jiménez-Hernández, J. Lanier, and E. G. Weyl,
“Should We Treat Data as Labor? Moving beyond ‘Free,’” AEA Papers and Proceedings
108, pp. 38–42. (May 2018). doi: 10.1257/pandp.20181003.

[12] Stephen Wolfson, “Fair Use: Training Generative AI,” Creative Commons, 17
February 2023,
<https://creativecommons.org/2023/02/17/fair-use-training-generative-ai/>,
accessed 12 April 2023.

[13] Scott Fitsimones, “The DAO Handbook: How Internet Strangers Are Building
Collective Movements,” The DAO Handbook, 19 January 2023,
<https://www.daohandbook.xyz/>, accessed 12 April 2023.

[14] Jobin, A., Ienca, M., & Vayena, E., “The global landscape of AI ethics
guidelines,” Nature Machine Intelligence, Volume 1, Issue 9, Article 9, (2019),
doi: 10.1038/s42256-019-0088-2

[15] Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel,
I., & Mohamed, S., “Power to the People? Opportunities and Challenges for
Participatory AI,” in Equity and Access in Algorithms, Mechanisms, and
Optimization, Article No. 6, 1–8, Arlington VA USA: ACM, (2022) doi:
10.1145/3551624.3555290.

[16] Luccioni, A. S., Akiki, C., Mitchell, M., & Jernite, Y., “Stable Bias:
Analyzing Societal Representations in Diffusion Models”, arXiv:2303.11408. (2023)
doi: 10.48550/arXiv.2303.11408

[17] Lorena O’Neil, “These Women Tried to Warn Us About AI”, Rolling Stone. 12
August, 2023.
<https://www.rollingstone.com/culture/culture-features/women-warnings-ai-danger-r
isk-before-chatgpt-1234804367/>.

[18] Elvira Pollina, “ChatGPT can resume in Italy if meets data watchdog’s
demands.”, Reuters, 13 April 2023,
<https://www.reuters.com/technology/italy-lift-curbs-chatgpt-if-openai-meets-dema
nds-by-end-april-data-protection-2023-04-12/>, accessed 16 April 2023.
[19] Yomiuri Y. Shimbun. “Intellectual Property Plan Signals Reversal on AI
Policy.” June 10 2023,
<https://japannews.yomiuri.co.jp/politics/politics-government/20230610-115423/>,
accessed 14 August 2023.

Bios

Kıvanç Tatar is an artist, technologist, and researcher who works in the

intersection of Machine Learning, Artificial Intelligence, Music, Interactive
Arts, Design, and Human-Computer Interaction. The computational approaches
developed through that interdisciplinary research have been integrated into
musical and audiovisual performances, interactive artworks, immersive
envi-ronments including virtual reality. Kıvanç is currently an Assistant
Professor and a WASP-HS fellow at Chalmers University of Technology, starting a
new research group connecting Art, Technology, and Artificial Intelligence.

Petter Ericson is a postdoctoral fellow in the research group for Responsible AI

at Umeå University in Sweden, working on graph problems and formal descriptions
of structured data, with a strong interest in ethics, music and society. His
recent research interests include anti-capitalist artificial intelligence and
circumventing political and structural barriers that bar AI from being used to
support democratic and egalitarian values. Outside of academia, his musical
interests have led him to everything from seedy late-night jam sessions at
Copenhagen jazz clubs to organizing 24-hour hackathons around producing
elec-tronic music from ESA data.

Kelsey Cotton is a vocalist-artist-mover working with experimental music, Musical

Artificial Intelligence and Human-Computer Interaction. Passionate about somatic
interaction, the potential for intersomatic experiences between fleshy and
synthetic bodies, and first-person feminist perspectives of musical AI, Kelsey is
pursuing doctoral studies in Interactive Music and AI at Chalmers University of
Technology.

Paola Torres Núñez del Prado is pursuing doctoral studies at the Stockholm
University of the Arts focusing on researching and developing hybrid interactive
textile sound interfaces that include the use of A.I. systems.
Roser Batlle-Roca is pursuing doctoral studies in Transparent AI & Music
Generation at Universitat Pompeu Fabra in Spain, in collaboration with JRC-EC and
Sony.

Beatriz Cabrero-Daniel is a postdoctoral fellow at Gothenburg University in

Sweden, currently researching trustworthy AI.

Sara Ljungblad is a researcher and senior lecturer at Gothenburg University and

Chalmers University of Technology in Sweden, doing critical robotics, studying
people's experiences and use of robotic products and autonomous systems in
everyday settings in the field of Human-Robot Interaction.

Georgios Diapoulis is pursuing doctoral studies in gestural interaction with

generative algorithms for machine musicianship at Chalmers University of
Technology and University of Gothenburg, Gothenburg, Sweden.

Jabbar Hussain is pursuing doctoral studies at Gothenburg University in Sweden in

Informatics in the area of trustworthy/responsible AI.