INSIGHTS | P E R S P E C T I V E S

ARTIFICIAL INTELLIGENCE

Art and the science of generative AI

Understanding shifts in creative work will help guide AI’s impact on the media ecosystem

By Ziv Epstein1, Aaron Hertzmann2,3, of art’s demise, but rather is a new medium is needed to understand how perceptions of
and the Investigators of Human Creativity* with its own distinct affordances. As a suite the generative process affect attitudes toward
of tools used by human creators, generative outputs and authors. This could facilitate the

T
he capabilities of a new class of tools, AI is positioned to upend many sectors of design of systems that disclose the generative
colloquially known as generative ar- the creative industry and beyond—threat- process and avoid misleading interpretations.
tificial intelligence (AI), is a topic of ening existing jobs and labor models in the Generative AI’s specific affordances in turn
much debate. One prominent appli- short term, while ultimately enabling new give rise to new aesthetics that may have a
cation thus far is the production of models of creative labor and reconfiguring long-term effect on art and culture. As these
high-quality artistic media for visual the media ecosystem. tools become more widespread, and their use
arts, concept art, music, and literature, as Unlike past disruptions, however, gen- becomes commonplace (as with photogra-
well as video and animation. For example, erative AI relies on training data made by phy a century ago), it remains an open ques-
diffusion models can synthesize high-qual- people. The models “learn” to generate art tion how the aesthetics of their outputs will
ity images (1), and large language models by extracting statistical patterns from exist- affect artistic outputs. A low barrier to entry
(LLMs) can produce sensible-sounding and ing artistic media. This reliance on train- for generative AI could increase the overall

Downloaded from https://www.science.org on June 20, 2023

impressive prose and verse in a wide range ing data raises new issues—such as where diversity of artistic outputs by expanding the
of contexts (2). The generative capabilities the data is sourced, how it influences the set of creators who engage with artistic prac-
of these tools are likely to fundamentally al- outputs, and how to determine authorship. tice. At the same time, aesthetic and cultural
ter the creative processes by which creators By leveraging existing work to automate norms and biases embedded in the training
formulate ideas and put them into produc- aspects of the creative process, generative data might be captured, reflected, and even
tion. As creativity is reimagined, so too may amplified—thereby decreasing diversity (7).
be many sectors of society. Understanding AI-generated content may also feed future
the impact of generative AI—and making “...generative AI is not the models, creating a self-referential aesthetic
policy decisions around it—requires new
interdisciplinary scientific inquiry into cul-
harbinger of art’s demise, but flywheel that could perpetuate AI-driven cul-
tural norms. Future research should explore
ture, economics, law, algorithms, and the rather is a new medium ways to quantify and increase output diver-
interaction of technology and creativity. sity and study how generative AI tools may
Generative AI tools, at first glance, seem with its own distinct affordances.” influence aesthetics and aesthetic diversity.
to fully automate artistic production—an The opaque, engagement-maximizing
impression that mirrors past instances AI challenges conventional definitions of recommender algorithms of social media
when traditionalists viewed new technolo- authorship, ownership, creative inspiration, platforms could further reinforce aesthetic
gies as threatening “art itself.” In fact, these sampling, and remixing and thus compli- norms through feedback loops (8) that pro-
moments of technological change did not cates existing conceptions of media produc- duce sensational and shareable content. As
indicate the “end of art,” but had much tion. It is therefore important to consider algorithm and content creators try to maxi-
more complex effects, recasting the roles generative AI’s impacts on aesthetics and mize engagement, this may further homog-
and practices of creators and shifting the culture, legal questions of ownership and enize content. However, some preliminary
aesthetics of contemporary media (3). For credit, the future of the creative work, and experiments (9) suggest that incorporating
example, some 19th-century artists saw the impacts on the contemporary media ecosys- engagement metrics when curating AI-
advent of photography as a threat to paint- tem. Across these themes, there are key re- generated content can, in some cases, diver-
ing. Instead of replacing painting, however, search questions to inform policy and ben- sify content. It remains an open question
photography eventually liberated it from eficial uses of this technology (4). what styles are amplified by recommender
realism, giving rise to Impressionism and To properly study these themes, it is first algorithms, and how that prioritization af-
the Modern Art movement. By contrast, necessary to understand how the language fects the types of content creators make
portrait photography did largely replace used to describe AI affects perceptions of the and share. Future work must explore the
portrait painting. Similarly, the digitization technology. The very term “artificial intelli- complex, dynamic systems formed by the
of music production (e.g., digital sampling gence” might misleadingly imply that these interplay between generative models, rec-
and sound synthesis) was decried as “the systems exhibit human-like intent, agency, ommender algorithms, and social media
end of music.” Instead, it altered the ways or even self-awareness. Natural language– platforms, and their resulting impact on
people produce and listen to music, and based interfaces now accompany generative aesthetics and conceptual diversity.
helped spawn new genres, including hip AI models, including chat interfaces that Generative AI’s reliance on training data
hop and drum’n’bass. Like these historical use the “I” pronoun, which may give users a to automate aspects of creation raises legal
analogs, generative AI is not the harbinger sense of human-like interaction and agency. and ethical challenges regarding authorship
These perceptions can undermine credit to and thus should prompt technical research
1
Massachusetts Institute of Technology, Cambridge, MA, the creators whose labor underlies the sys- into the nature of these systems. Copyright
USA. 2Adobe Research, San Francisco, CA, USA. 3University tem’s outputs (5) and deflect responsibility law must balance the benefits to creators,
of Washington, Seattle, WA, USA. Corresponding author:
[email protected] *The Investigators of Human Creativity from developers and decision-makers when users of generative AI tools, and society at
coauthors and affliations are listed at the end of this paper. these systems cause harm (6). Future work large. Laws could treat the use of training

1110 16 JUNE 2023 • VOL 380 ISSUE 6650 science.org SCIENCE

data as noninfringing if protected works characterize the specific steps of the creative in real media, such as unedited journalis-
are not directly copied; fair use if training process, which of those steps might be af- tic photographs? As content production in-
involves a substantial transformation of the fected by generative AI tools, and the effects creases, collective attention spans may de-
underlying data; allowed only if creators on workplace requirements and activities of crease (16). The explosion of AI-generated
give an explicit license; or subject to statu- varying cognitive occupations (11). content may in turn hamper society’s ability
tory compulsory licensing that allows data Although these tools may threaten some to collectively discuss and act in important
to be used for training provided that cre- occupations, they could increase the pro- arenas such as climate and democracy.
ators are compensated. Much of copyright ductivity of others and perhaps create new Every artistic medium mirrors and com-
law relies on judicial interpretations, so it ones. For example, historically, music au- ments on the issues of its time, and the
is not yet clear if collecting third-party data tomation technologies enabled more musi- debates surrounding contemporary AI-
for training or mimicking an artist’s style cians to create, even as earnings skewed (12). generated art reflect present issues sur-
would violate copyright. Legal and technical Generative AI systems can create hundreds rounding automation, corporate control,
issues are entwined: Do models directly copy of outputs per minute, which may accelerate and the attention economy. Ultimately,
elements from the training data, or produce the creative process through rapid ideation. we express our humanity through art, so
entirely new works? Even when models do However, this acceleration might also under- understanding and shaping the impact of
not directly copy from existing works, it is mine aspects of creativity by removing the AI on creative expression is at the center
not clear whether and how artists’ individ- initial period of prototyping associated with of broader questions about its impact on
ual styles should be protected. What mecha- a tabula rasa. In either case, production time society. New research into generative AI
nisms could protect and compensate artists and costs will likely fall. The production of should inform policy and beneficial uses of
whose work is used for training, or even per- creative goods may become more efficient, the technology while engaging with critical
mit them to opt out, while still allowing new leading to the same amount of output with stakeholders, particularly artists and cre-
cultural contributions to be made with gen- fewer workers. In turn, demand for creative ative laborers themselves, many of whom

Downloaded from https://www.science.org on June 20, 2023

erative AI models? Answering these ques- work may increase. However, the production actively engage with difficult questions at
tions and determining how copyright law of creative goods may become more efficient, the vanguard of societal change. j
should treat training data require substan- leading to the same amount of output with
REF ERENCES AND NOTES
tial technical research to develop and under- fewer workers. Furthermore, many work-for-
1. R. Rombach et al., Proceedings of the IEEE/CVF
stand the AI systems, social science research hire occupations using conventional tools, Conference on Computer Vision and Pattern Recognition
to understand perceptions of similarity, and like illustration or stock photography, could (2022). pp. 10684–10695.
legal research to apply existing precedents to be displaced. Several historical examples 2. A. Vaswani et al., “Attention is all you need” in Advances
in Neural Information Processing Systems 30 (2017).
new technology. Of course, these points rep- bear this out. Most notably, the Industrial 3. A. Hertzmann, Arts. 7 (no. 2) (2018).
resent only an American legal perspective. Revolution enabled mass production of tra- 4. Z. Epstein et al., Art and the science of generative AI: A
A distinct legal question concerns who ditionally artisanal crafts (e.g., ceramics, tex- deeper dive. arXiv2306.04141 [cs.CY] (2023).
5. Z. Epstein et al., iScience 23, 101515 (2020).
can claim ownership over model outputs. tiles, and steelmaking) with the labor of non- 6. M. C. Elish, Engag. Sci. Technol. Soc. 5, 40 (2019).
Answering this requires understanding the artisans; hand-made goods became specialty 7. S. U. Noble, Algorithms of Oppression: How Search
creative contributions of a system’s users ver- items. Similarly, photography replaced por- Engines Reinforce Racism (New York Univ. Press, 2018).
8. A. J. B. Chaney et al., in Proceedings of the 12th ACM
sus other stakeholders, such as the system’s trait painting. Digitization of music removed Conference on Recommender Systems (2018),
developers and creators of the training data. constraints of learning to physically manipu- pp. 224–232.
AI developers could claim ownership over late instruments and enabled more complex 9. Z. Epstein, M. Groh, A. Dubey, A. Pentland, Proc. ACM
Hum. Comput. Interact. 5 (CSCW2), 1 (2021).
outputs through terms of use. By contrast, if arrangements with more contributors. These
10. C. B. Frey, M. A. Osborne, Technol. Forecast. Soc. Change
users of the system have engaged in a mean- tools may change who can work as an artist, 114, 254 (2017).
ingfully creative way (e.g., the process is not in which case artists’ employment may rise 11. M. R. Frank et al., Proc. Natl. Acad. Sci. U.S.A. 116, 6531
fully automated, or does not emulate specific even as average wages fall. (2019).
12. D. Hesmondhalgh et al., “Music creators’ earnings in the
works), then they might be considered as As these tools affect creative labor, they digital era.” Intellectual Property Office Research Paper
the default copyright holders. But how sub- also introduce potential downstream harms (2021).
stantial must users’ creative influence be for to the broader media ecosystem. As the 13. J. A. Goldstein et al., arXiv 2301.04246 [cs.CY] (2023).
14. B. Chesney, D. Citron, Calif. Law Rev. 107, 1753 (2019).
them to claim ownership? These questions cost and time to produce media at scale de- 15. H. Farid, J. Online Trust Safety 10.54501/jots.v1i4.56
involve studying the creative process of us- creases, the media ecosystem may become (2022).
ing AI-based tools and may become more vulnerable to AI-generated misinformation 16. P. Lorenz-Spreen et al., Nat. Commun. 10, 1759 (2019).
complex if users gain more direct control. through the creation of synthetic media, ACKNOWL EDGMENTS
Regardless of legal outcomes, genera- particularly media that provides proba- Z.E. and M.G. helped assess DALL-E 2 and M.R.F. helped
tive AI tools are likely to transform creative tive evidence for claims (13). These new assess ChatGPT for OpenAI. A.H. and L.H. work for Adobe,
work and employment. Prevailing economic possibilities for generating photorealistic which makes generative AI tools; the ideas expressed here do
not represent those of the company.
theory [i.e., skill-biased technological change synthetic media may undermine trust in
(SBTC)] assumes that cognitive and creative authentically captured media through the Investigators of Human Creativity Memo Akten1, Hany
Farid2, Jessica Fjeld3, Morgan R. Frank4, Matthew Groh5,
workers face less labor disruption from au- so-called “liar’s dividend” (fake content Laura Herman6,7, Neil Leach8, Robert Mahari3,5, Alex “Sandy”
tomation because creativity is not readily benefits liars by undermining trust in the Pentland5, Olga Russakovsky9, Hope Schroeder5, Amy Smith10
encodable into concrete rules (i.e., Polanyi’s truth) (14) and also increase threats of fraud 1
University of California, San Diego, San Diego, CA, USA.
2
paradox) (10). Yet, new tools have sparked and nonconsensual sexual imagery. This University of California, Berkeley, Berkeley, CA, USA. 3Harvard
Law School, Cambridge, MA, USA. 4University of Pittsburgh,
employment concerns for creative occupa- raises important research questions: What Pittsburgh, PA, USA. 5Massachusetts Institute of Technology,
tions such as composers, graphic designers, is the role of platform interventions such as Cambridge, MA, USA. 6University of Oxford, Oxford, UK. 7Adobe,
and writers. This conflict arises because tracking source provenance and detecting Inc., London, UK. 8Florida International University, Miami, FL,
USA. 9Princeton University, Princeton, NJ, USA. 10Queen Mary
SBTC fails to differentiate between cognitive synthetic media downstream in governance University of London, London, UK.
activities such as analytical work and cre- and building trust (15)? And how does the
ative ideation. A new framework is needed to proliferation of synthetic media affect trust 10.1126/science.adh4451

SCIENCE science.org 16 JUNE 2023 • VOL 380 ISSUE 6650 1111

 Art and the science of generative AI
Ziv Epstein, Aaron Hertzmann, the Investigators of Human Creativity, Memo Akten, Hany Farid, Jessica Fjeld, Morgan
R. Frank, Matthew Groh, Laura Herman, Neil Leach, Robert Mahari, Alex Sandy Pentland, Olga Russakovsky, Hope
Schroeder, and Amy Smith

Science, 380 (6650), .

DOI: 10.1126/science.adh4451

Downloaded from https://www.science.org on June 20, 2023

View the article online
https://www.science.org/doi/10.1126/science.adh4451
Permissions
https://www.science.org/help/reprints-and-permissions

Use of this article is subject to the Terms of service

Science (ISSN 1095-9203) is published by the American Association for the Advancement of Science. 1200 New York Avenue NW,
Washington, DC 20005. The title Science is a registered trademark of AAAS.
Copyright © 2023 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim
to original U.S. Government Works