Qualitative Study of Text-To-Image AI Generators and Their Relationship With NFTs
Qualitative Study of Text-To-Image AI Generators and Their Relationship With NFTs
Abstract— The evolution of technology and science has likely to engage in more creative activities such as making art
brought groundbreaking developments to the current visual arts, and composing music, while those that do not produce art
and the application of digital technology has brought significant themselves will have more time to enjoy the created art [4].
changes to the creation and aesthetic taste of traditional art. This Humans now not only have more time to generate art but also
study, therefore, investigates the current state-of-the-art artificial can generate and view tmore art due to the automation provided
intelligence (AI) technologies and applications in generating visual by Artificial Intelligence (AI) [5]. By using the power of AI
art while giving a brief history of the intersection of AI and art, almost anything can be created employing existing mediums [6].
including the milestone advancements in neural networks. Fifteen These developments demonstrate that AI not only can create art
interviews were conducted with technical artists who use text-to-
but also change the way humans consume art [7]. New forms of
image AI generators to gather data. Based on the findings from
the interviews, the state-of-the-art applications were reviewed and
classic art, conceptual art, modern art, abstract art, and pop art
analyzed in six categories: Accessibility, Barrier to Entry, Novelty, will start to be exhibited in galleries. Additionally, the growing
Ethics and Morality, Control, Non-fungible tokens (NFT) and demand for AI art will attract a selection of new collectors. By
Monetization which were widely discussed along with their success following major auction houses like Sotheby’s and Christie’s, it
and limitations. The research concludes with three main findings; is observed that AI art prices and demand are slowly growing
(a) monetization of digital media through NFTs that has a direct [5].
impact on the advancement of art generating AI applications, (b)
Technological advancements, particularly AI, have been
there is a significant change in the traditional creative process with
the integration of AI applications, and AI is not just a tool but it’s
significantly changing the nature of creative processes [8, 9]. As
a creative agent that artists collaborate with (c) art generating AI machines become more capable and faster, forms of artificial
applications can generate limitless possibilities within the same intelligence reinforce their presence in the center stage of the
aesthetics as a result revolutionize the way humans create and creative process, becoming the main drivers of creativity and
interact with art. innovation [10]. AI has demonstrated rapid improvement and
the possibility of outperforming humans in domains such as
Keywords—Artificial Intelligence, NFT, Text-to-image, deep traditional art and artistic creativity [6]. Neural networks can
learning, generative art explore the patterns in the strokes, colors, and shading of a
particular art piece. It can transfer the style from the original
artwork into a new image based on the analysis and do it faster
I. INTRODUCTION than humans. This has important implications as art, especially
painting, has been regarded as the pinnacle of human creativity
Historically, technology has dramatically expanded creative for a thousand years [11]. In the West, painting has been
and professional opportunities for artists by providing newer and associated with religious symbolism and has been typically seen
more powerful tools. The emergence of new technologies often as representing humanity’s most pure and artistic expression.
causes fears of displacement among traditional artists. These Advances in AI artwork thus naturally complicate contemporary
new tools eventually help to create new artistic styles and feed understandings of creativity and aesthetic beauty in the arts [12].
vitality into art forms that might otherwise become stale. In
addition, new tools also make art more accessible to broader In recent years, amid the most significant advances in
segments of society, both as creators and as viewers [1]. Since Machine Learning are the introduction of pre-trained, text-to-
1965, leisure time has increased by about 4 to 8 hours per week image AI generators that create visual media. These models are
[2]. Researchers predict that this will increase as automation trained on large data sets and can generate novel images [13]
assumes time-consuming, repetitive tasks [3]. Some people are that is indistinguishable in quality from human-generated
241
Authorized licensed use limited to: ST ANDREWS UNIVERSITY. Downloaded on February 22,2024 at 15:09:09 UTC from IEEE Xplore. Restrictions apply.
create an AI NFT [15]. Using AI algorithms, an AI art generator There is very limited academic literature in the field of text-
analyzes endless pieces of art and creates its own images as to-image AI generators in regard to art and creativity. However,
unique visual interpretations of the original texts. This allows AI today, anyone who has access to these applications can generate
NFT application users to produce new pieces of art that are art without expert technical knowledge [36]. This, along with
based on chosen texts or images. The AI-generated work can the rapidly growing AI-art community emphasizes the urgent
then be minted directly to an NFT marketplace such as OpenSea need for more research in this area. Table 1 describes the
via the AI-NFT generator. Fotor [15] and SketchAR [15] are themes that emerged from the analysis of the interviews
state-of-the-art AI based NFT art generators. The value of AI conducted with fifteen technical artists. The advancement of
NFT applications is to streamline the AI art creation, minting numerous text-to-image AI generators and their accessibility
and selling the created art on blockchain with minimum steps.
have led to an increased number of AI-generated artworks [37].
Although NFTs can be generated using artificial Convenience of these platforms lowered the barrier to entry for
intelligence, there is also another trend and that is the integration the non-technical artist from out of STEM networks by
of Artificial Intelligence within blockchain. This means the allowing them to experiment without requiring technical skills
creation of dynamic and intelligent experiences that are not [38]. The plethora of AI-generated art that is created in a short
possible to create with other techniques. AI-generated NFTs can period of time reinforced the discussion of novelty and the
be used to create unique and personalized experiences for the limitations around the ability to fully control the platforms [39].
audience. Alethea AI was the first to create an intelligent NFT In addition, training datasets, consisting of hundreds of millions
called “Alice” [20]. This NFT has strong self-learning of images, raise widespread ethical concerns regarding
capabilities, as it gains new knowledge as it interacts with more copyright and ownership. Nonetheless, fueled by the ever-
people. This smart NFT was auctioned at Sotheby’s for almost
increasing demand for NFTs there is a growing interest in AI
half a million USD. In other words, an AI NFT is a non-fungible
token embedded with an AI model prompt as part of its smart based art generating tools [40].
contract. This type of intelligent NFT is not only intelligent but A. Accessibility and Barrier to Entry
comes with other properties like animation, interactivity and
many other generative capabilities still emerging [33]. AI gives Most of the interviewees aligned on the importance of
the blockchain the ability to build an NFT metaverse where Accessibility and low barrier to entry as the leading catalysts that
digital objects interact and evolve with each other [34]. The idea contribute to art generating AI applications becoming
of AI and NFT collaboration is to turn works of art into NFTs mainstream. Since a large number of tasks require a high volume
and the rest of the NFT industry into agents that interact with of data sets, this naturally tends to increase the computational
each other and with the surrounding environment. In addition to cost [41]. Pre-trained models allow the user to generate art
providing users with more meaningful ways to interact with their without needing a GPU and keep the computational cost to
NFT, AI NFTs give the NFT the ability to learn and evolve over minimum making it more accessible for the user [42]. Prior to
time. These AI-embedded NFTs are touted as appreciating in the emergence of the AI applications that generate visual art, the
value not just in response to the market, but also thanks to how only artists who experiment with AI art were from technical
far along the NFT is in its journey of evolution, learning, and background [16]. The only option to generate novel art was
self-amendment [35]. through gathering a large dataset and training the model long
IV. QUALITATIVE ANALYSIS AND FINDINGS enough to get artistically meaningful results [37]. This required
a GPU and long hours to gather a dataset and train the AI
Fifteen interviews were conducted were conducted and algorithm to form a model that can generate images [43]. Today,
analyzed through an inductive grounded theory approach. as all the publicly available state-of-the-art text-to-image AI
Interview questions are open ended to enable the interviewee to generators use a pre-trained model [16,15,17], any artist without
share their experience without any constraints. Analysis of the needing technical skills, can generate art. However, this comes
responses from successive interviews has driven the review and with one major downside to creativity which is the ability to
modification of questions for subsequent respondents. This control the outcome. While increased accessibility and need for
interview and review process continued until the point of control contradict each other because of the pre-trained model,
theoretical saturation which has been reached with the fifteenth not requiring advanced technical knowledge attracted and
interview. Recordings of the interviews were put into welcomed more artists from out of STEM networks. This
transcription software. The Nvivo software system was chosen encouraged the formation of a community around digital media,
for this research for its versatility and support provided for AI art and NFTs [37]. What was in the monopoly of very few
qualitative research efforts. Nvivo is used to determine codes technical artists before is now accessible to everyone without
and for analyzing the information. Codes from the transcripts advanced technical training.
were analyzed to determine the evaluation criteria that is
described in Table 1.
TABLE I. EMERGING THEMES FROM THE ANALYSIS OF THE INTERVIEWS CONDUCTED WITH FIFTEEN TECHNICAL ARTISTS
Evaluation Criteria Description of Evaluation Criteria
Accessibility Ability to avoid high computational cost and energy therefore can be accessed simply through internet
connection.
242
Authorized licensed use limited to: ST ANDREWS UNIVERSITY. Downloaded on February 22,2024 at 15:09:09 UTC from IEEE Xplore. Restrictions apply.
Barrier to Entry Ability to navigate and utilize the platform without expert technical knowledge. Making new technology
accessible to users and getting people from diverse disciplines interested.
Control Ability to give the user level of control towards the planned outcome leaving minimum work to post
processing.
Novelty Ability to facilitate and reinforce production of novel outcomes giving the user the capability to create
their own style.
Ethics and Morality Ethics and moral considerations regarding the dataset used to train the AI model and its environmental
impact.
NFTs and Monetization Impact of monetization through NFTs on the advancement of text-to-image AI generators.
243
Authorized licensed use limited to: ST ANDREWS UNIVERSITY. Downloaded on February 22,2024 at 15:09:09 UTC from IEEE Xplore. Restrictions apply.
power the hardware for training such models is significant is found that there is a significant gap in the academic
considering that training happens over weeks or months [53]. literature regarding the relationship between AI applications
Previous research states that even though it is possible to and NFTs. Possible future research includes speech to image
procure a portion of the required energy from renewable AI art applications, copyright, and ownership of intelligent
resources, the high energy demands of these models are still NFTs, mobile applications of text-to-image AI generators and
causing concerning challenges. The main reason being the the evolution of human creative process with the introduction
majority of the energy is not currently derived from of collaborative machine creativity.
renewable sources in many locations, or if renewable energy
is available, it is still limited to the equipment that is produced REFERENCES
to store it [51]. Even though NFTs have faced major criticism [1] Hertzmann, A. (2018). Can computers create art? In Arts (Vol. 7, No.
2, p. 18). Multidisciplinary Digital Publishing Institute.
concerning their impact on the environment, they do not
[2] Peng, H., Chou, C., & Chang, C. Y. (2007). From the virtual to physical
cause any environmental impact on their own, however their environments: Exploring interactivity in ubiquitous-learning systems.
impact on the environment is directly linked to how they are In Second International Conference on Innovative Computing,
produced. Most NFTs to emerge during the initial 2021 boom Information and Control (ICICIC 2007) (pp. 162-162).
were minted on Ethereum when it was using Proof-of-Work, [3] Ortiz-Ospina, E. Giattino, C. & Roser, M. (2020) - "Time Use".
an energy-intensive consensus mechanism that also secures Published online at OurWorldInData.org.
Bitcoin [54]. This prompted an argument regarding [4] Brinson, S. (2019). How Will Artificial Intelligence Impact the Art
World?
blockchain’s environmental impact. While Ethereum’s
[5] Alexander, D. (2020). Artificial Intelligence Is Changing the Way We
energy use climbed from 2021 through early 2022, it dropped Experience Art, and It's Spectacular
around 99.95% when they completed the move to Proof-of- [6] He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into
Stake in 2022 [41]. While concerns over the environmental rectifiers: Surpassing human-level performance on imagenet
impact of any technology are valid, it is worth mentioning classification. In Proceedings of the IEEE international conference on
that YouTube consumes more electricity than Ethereum, but computer vision (pp. 1026-1034).
it doesn’t face as much pressure to go green. [7] Vincent, J. (2022). Anyone can use this AI art generator - that's the risk.
[8] Nadini, M., Alessandretti, L., Di Giacinto, F., Martino, M., Aiello, L.
M., & Baronchelli, A. (2021). Mapping the NFT revolution: market
E. NFTs and Monetization trends, trade networks, and visual features. Scientific reports, 11(1), 1-
11.
With the introduction of NFTs, any type of intellectual [9] Kundu, R. (2022). AI-Generated Art: From Text to ImageText-to-
property can be monetized. AI-generated art is one of the image s & Beyond [Examples]
fastest-growing NFTs [54]. Rapid monetization caused by the [10] Hristov, K. (2016). Artificial intelligence and the copyright dilemma.
emergence of NFTs is an important catalyst driving the IDEA, 57, 431.
attention to the AI generated art. Text-to-image AI generators [11] Mazzone, M., & Elgammal, A. (2019). Art, creativity, and the potential
of artificial intelligence. In Arts (Vol. 8, No. 1, p. 26). Multidisciplinary
allow users to create a collection of NFTs in just a few Digital Publishing Institute.
minutes by simply typing in words as prompt. Many NFT [12] Hong, J. W., & Curran, N. M. (2019). Artificial intelligence, artists,
collections include thousands of unique images [55]. As such, and art: attitudes toward artwork produced by humans vs. artificial
generating NFTs via AI applications like DALL-E or intelligence. ACM Transactions on Multimedia Computing,
Midjourney is an expected approach to maximizing Communications, and Applications (TOMM), 15(2s), 1-16.
efficiency. It can be argued that not all the images that are [13] Oh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018, April). I
lead, you help but only with enough details: Understanding user
generated on these platforms are considered art, however the experience of co-creation with artificial intelligence. In Proceedings of
ability to generate thousands of images without any artistic or the 2018 CHI Conference on Human Factors in Computing Systems
technical skills provides a unique creative outlet for users. (pp. 1-13).
[14] Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., &
V. CONCLUSION Smith, N. A. (2021). All that's' human'is not gold: Evaluating human
evaluation of generated text. arXiv preprint arXiv:2107.00061.
Reaching widespread notoriety within the last two years, [15] Oppenlaender, J. (2022). The Creativity of Text-based Generative Art.
NFTs have been since challenging the traditional norms of [16] Navarro, et. Al. (2021). Risk of bias in studies on prediction models
the art world. Providing a new commercial use for the developed using supervised machine learning techniques: systematic
creative outputs, NFTs created a strong motivation for artists review. bmj, 375., Chicago
globally. This global trend created an opportunity for [17] Raj, M. M., & Ganesan, M. D. CRYPTO AI: DIGITAL NOSTALGIC
generative art and specifically text-to-image AI generators ART GENERATION USING GAN AND CREATION OF NFT
USING BLOCKCHAIN.
which create visual art with just a simple text prompt. Hence,
[18] Lee, L. H., Lin, Z., Hu, R., Gong, Z., Kumar, A., Li, T., ... & Hui, P.
this paper finds that the monetization of digital media through (2021). When creators meet the metaverse: A survey on computational
NFTs has a direct impact on the advancement of text-to- arts. arXiv preprint arXiv:2111.13486.
image AI generators. These applications can generate [19] Marcos, A. (2007). Digital Art: When artistic and cultural muse and
limitless possibilities within the same aesthetics as a result computer technology merge. IEEE Computer Graphics and
Applications.
revolutionize the way humans create and interact with art. As
[20] Notaro, A. (2022). All that is solid melts in the Ethereum: the brave
a result, there is a significant change in the traditional creative new (art) world of NFTs. Journal of Visual Art Practice, 1-24.
process with the integration of AI applications, AI is not just
a tool but it’s a creative agent that artists collaborate with. It
244
Authorized licensed use limited to: ST ANDREWS UNIVERSITY. Downloaded on February 22,2024 at 15:09:09 UTC from IEEE Xplore. Restrictions apply.
[21] Hutzler, G. (1997). The garden of chances: An integrated approach to [47] Ramey, V. (2007). How Much has Leisure Really Increased Since
abstract painting and reactive DAI. 1965? University of California at San Diego Working Paper.
[22] Lopez de Mantaras, R. (2016). Artificial intelligence and the arts: [48] Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., &
Toward computational creativity. Cohen-Or, D. (2022). Prompt-to-prompt image editing with cross
[23] Luddecke, T., & Ecker, A. (2022). Image Segmentation Using Text and attention control. arXiv preprint arXiv:2208.01626.
Image Prompts. In Proceedings of the IEEE/CVF Conference on [49] Abbaschian, B. J., Sierra-Sosa, D., & Elmaghraby, A. (2021). Deep
Computer Vision and Pattern Recognition (pp. 7086-7096). learning techniques for speech emotion recognition, from databases to
[24] Little-Tetteh, K., & Shchyhelska, H. (2019). Artificial intelligence models. Sensors, 21(4), 1249.
painting: is it art, really? [50] Benhamou, Y. (2022). The protection of AI-generated photographs
[25] Marcos, A. F., Branco, P., & Carvalho, J. AÃÅ. (2009). The computer under copyright law (Doctoral dissertation, University of Geneva).
medium in digital art's creative process. In Handbook of Research on [51] Romero, A. (2022). DALL·E 2, Explained: The Promise and
Computational Arts and Creative Informatics (pp. 1-25). Limitations of a Revolutionary AI
[26] Cohen, H. (1988). How to Draw Three People in a Botanical Garden. [52] Strubell, et al. (2019): Energy and Policy Considerations for Deep
In AAAI (Vol. 89, pp. 846-855). Learning in NLP.
[27] Colton, et al., (2015). The Painting Fool Sees! New Projects with the [53] Morris, M. R., Cai, C. J., Holbrook, J. S., Kulkarni, C., & Terry, M.
Automated Painter. In ICCC (pp. 189-196). (2022). The Design Space of Pre-Trained Models.
[28] Colton, S. (2012). The painting fool: Stories from building an [54] Khelifi, H., Luo, S., Nour, B., Sellami, A., Moungla, H., Ahmed, S. H.,
automated painter. In Computers and creativity (pp. 3-38). Springer, & Guizani, M. (2018). Bringing deep learning at the edge of
Berlin, Heidelberg. information-centric internet of things. IEEE Communications Letters,
[29] Charnley, J.; Pease, A.; and Colton, S. 2012. On the notion of framing 23(1), 52-55.
in computational creativity. In Proceedings of the 3rd ICCC [55] McFarland, M. (2016). What AlphaGo's sly move says about machine
[30] van Wynsberghe, A. (2021). Sustainable AI: AI for sustainability and creativity. The Washington Post, 15.
the sustainability of AI. AI and Ethics, 1(3), 213-218.
[31] Liu, X., Liu, Y., & Wei, Z. (2020). A Rational Survey of Art and
Technology: From Traditional Painting to Intelligent Painting. In 6th
International Conference on Education, Language, Art and Inter-
cultural Communication (ICELAIC 2019) (pp. 722-728). Atlantis
Press.
[32] Shah, et al. (2022). DC‚ÄêGAN‚Äêbased synthetic X‚Äêray images
augmentation for increasing the performance of EfficientNet for
COVID‚Äê19 detection. Expert Systems, 39(3), e12823.
[33] Kirkwood, J. W. (2022). From Work to Proof of Work: Meaning and
Value after Blockchain. Critical Inquiry, 48(2), 360-380.
[34] Jeon, H. J., Youn, H. C., Ko, S. M., & Kim, T. H. (2022). Blockchain
and AI Meet in the Metaverse. Advances in the Convergence of
Blockchain and Artificial Intelligence, 73.
[35] Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., ... &
Norouzi, M. (2022). Photorealistic Text-to-Image Diffusion Models
with Deep Language Understanding.
[36] Tugan, A. Liberation of The Medium: Decentralization of Dynamic
Generative Art Creations by NFT Marketplaces.
[37] Buraga, A. P. (2022). The Emergence of the Type-Generated AI Art
Community: A Netnographic and Content Analysis Approach.
[38] Berner, S. (2022). The Best Online Free AI Image Generators for
Converting Text to ImageText-to-image s 2022
[39] Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022).
Hierarchical text-conditional image generation with clip latents.
[40] Exmundo,J. (2022). The Next Big Thing in NFTs? Artificial
Intelligence.
[41] Kapengut, E., & Mizrach, B. (2022). An Event Study of the Ethereum
Transition to Proof-of-Stake. arXiv preprint arXiv:2210.13655.
[42] Vermillion, J. (2022). Iterating the Design Process Using AI Diffusion
Models.
[43] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B.
(2022). High-resolution image synthesis with latent diffusion models.
In Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition (pp. 10684-10695).
[44] McCorduck, P. (1991). Aarons Code. W.H. Freeman & Co Ltd.
[45] Caramiaux, B., & Fdili Alaoui, S. (2022). " Explorers of Unknown
Planets" Practices and Politics of Artificial Intelligence in Visual Arts.
Proceedings of the ACM on Human-Computer Interaction, 6(CSCW2),
1-24.
[46] Valeonti, F., Bikakis, A., Terras, M., Speed, C., Hudson-Smith, A., &
Chalkias, K. (2021). Crypto collectibles, museum funding and
OpenGLAM: challenges, opportunities and the potential of Non-
Fungible Tokens (NFTs). Applied Sciences, 11(21), 9931.
245
Authorized licensed use limited to: ST ANDREWS UNIVERSITY. Downloaded on February 22,2024 at 15:09:09 UTC from IEEE Xplore. Restrictions apply.