Art Style Classification From Art Images

Using Convolutional Neural Network

SURYA K S
2022510022

Abstract - Categorizing art images is an increasingly To this end, our project aims to automate the
relevant and challenging task, especially as the volume of categorization of over 9,000 high-resolution art images
digital art collections continues to grow. Our work builds in the Kaggle Art Images Dataset into one of five
on recent advances in deep learning techniques, which distinct categories: drawings, engravings, paintings,
have enabled automated image classification with high iconography, and sculptures.
accuracy and efficiency. Our approach involves using a
pre-trained CNN, we can leverage its ability to Our approach to automated art classification involves
distinguish fine-grained visual features such as edges, the use of convolutional neural networks (CNNs) and
textures, and shapes. To enhance the performance of our transfer learning. By utilizing transfer learning
model, we have also employed a variety of data techniques, our model can leverage pre-trained models
augmentation techniques in our pre-processing step. on similar problems and improve its accuracy.
These techniques help to create artificial variations in the Furthermore, we have employed data augmentation
original training dataset, making our model more robust techniques to enhance the size and generalizability of
to variations in image size, orientation, and color. our dataset, enabling our model to perform better on
Techniques like random flipping, rotation, and zooming, smaller, less varied datasets.
along with Gaussian noise injection, have been shown
to work effectively in many areas of computer vision, Through rigorous training of our custom models using
including art classification. Our model achieved high cross-entropy loss, we can classify art images based on
accuracy on both the training and validation sets, their visual features and style more accurately. Our
indicating that it can generalize well to new, previously models can significantly aid art historians, museum
unseen images. However, we acknowledge that some curators, and art enthusiasts alike, providing them with
degree of bias and variability may be present in the an efficient and reliable tool that accurately categorizes
dataset due to differences in artist styles, time periods, or art into five classes. The outcome of our project
image quality. Nonetheless, our model can provide highlights the power of CNNs and transfer learning to
valuable insights into the visual features and automate art classification and demonstrates the
characteristics that distinguish one art style from potential for these technologies to unlock new insights
another. Overall, our work contributes to the growing into art's diverse and captivating forms of expression.
field of art classification and demonstrates the
effectiveness of modern deep learning techniques in this
area. By categorizing art images into five distinct groups,
we can help art historians and enthusiasts to explore
art collections more efficiently, discover new trends
and connections between different styles, and gain a
deeper appreciation and understanding of the rich
diversity of artistic expression throughout history.

I. INTRODUCTION

Art has been one of the most enduring forms of human

expression throughout history, encompassing a vast
array of styles, techniques, and materials. With the rise
of digital art collections, the need for automatic
classification tools to organize and catalog artworks
has also risen.
FIG 1 :- SAMPLE IMAGES
 II. EXISTING WORKS multiple layers including ReLU, convolution and
pooling layer. CNN model can achieve this
An improved random forest classifier is suggested by automatically using its hidden layers. Inspired by their
Baoxun Xu and Yunming Ye[2]. Their approach is approach we built the CNN model ourselves first to
particularly to solve classification problems for classify the style and then compared the result with the
multiple classes, which is useful for artwork style ResNet-18 and ResNet-v2 model.
recognition because each painting may be labelled with
several styles. In their random forest model, they III. PROPOSED FRAMEWORK
introduced a novel feature weighting method and tree
selection method to make the RF frame which is good This project revolves around building an image
at classifying data with a large number of object classification system using a Convolutional Neural
categories. The feature weighted method and tree Network (CNN). The framework employs Keras and
selection method are taken into account simultaneously TensorFlow, encompassing data loading,
The feature-weighted method improves the preprocessing, model design, training, and evaluation.
classification power of individual trees and the detected By leveraging data augmentation and deep learning
bad trees are excluded to prevent their negative impacts techniques, the goal is to create a robust model capable
from the performance. We hope to use their proposed of accurately classifying images. The code includes
random forest frame and the feature extracted to visualization of training progress, along with
predict the multiple styles of a given art painting. comprehensive evaluation metrics such as ROC curves
and precision-recall curves.
Support vector machines are also a potentially capable
classifier for our style classification. SVM is a set of Now, let's delve into the details of each component in
supervised learning methods that are effective in higher the proposed framework.
dimensional spaces. SVM is also versatile because
different kernel functions can be applied for the
decision and even custom kernels. Jiuzhen Liang
proposed a support vector machine for the multi
classification problem[3]. He focuses on the direct
design of the multi-classifier of SVM and its
application to Web page classification, which is quite
similar to style classification and thus could be an
inspiration.

We draw the inspiration of style classification from

the papers suggesting approaches of image style
transformation. In these papers[1][4][5], transferring
style from one image on to another 1 is considered as FIG 2 :- BASIC CNN ARCHITECTURE
a texture transfer. They synthesise a texture from the
source image while constraining the texture synthesis in
order to preserve the semantic content of a target Data Loading and Preprocessing:
image.
The project starts by specifying the paths for the
In paper[1], Gatys and Ecker use CNN to learn a training, testing, and validation datasets using
generic feature representation of painting style and TrainingImagePath, TestingImagePath, and
use these features to manipulate the image content to ValidationImagePath.
convert the style. Their approach constrains a texture
synthesis method by feature representations from a ImageDataGenerator from Keras is used for data
state-of-theart CNN and generates New images by augmentation. It rescales the pixel values to the range
performing a pre-image search to match feature [0, 1] and applies transformations like shear, zoom, and
representations of the example image of a certain target horizontal flip on the training data to increase its
style. In general, their approach for this feature diversity.
extraction is using a convolutional neural network with
Neural Network Architecture: Performance Visualization:

The CNN architecture is defined using the Keras The code includes the visualization of training and
Sequential API. The first layer is a convolutional layer. validation accuracy and loss over epochs using
This is followed by a max-pooling layer. The output of matplotlib. Two subplots are created: one for accuracy
the max-pooling layer is flattened and connected to a and one for loss.
dense layer. The final layer is a dense layer with the
number of neurons equal to the number of output ResultMap and Pickle:
classes (determined by OutputNeurons) and softmax
activation, making it suitable for multi-class ResultMap is a dictionary that maps class indices to
classification. class names. This mapping is created based on the class
indices obtained from the training set. The mapping is
stored in a file named 'ResultMap.pkl' using the pickle
module. Pickle is a serialization module in Python.

Testing on a Sample Image:

An example image path (testImg) is provided, and the

image is loaded and preprocessed. The model makes
predictions on the sample image, and the result is
displayed along with the image using matplotlib.

Evaluation Metrics Visualization:

The code calculates and visualizes several evaluation

metrics:

ROC curve: Visualizes the trade-off between true

positive rate and false positive rate for different
ean su c p t nisl el tellus varius is. classification thresholds. Confusion Matrix: Shows the
Suspendisse ac sodales tis in nisl in
count of true positive, true negative, false positive, and
sollicitudin. Ut erat
false negative predictions. Classification Report:
FIG 3 :- CNN LAYERS OF MODEL1
Provides precision, recall, and F1-score for each class.
Model Compilation: Separate Precision-Recall Curves: Displays precision-
recall curves for each class separately.
The model is compiled using categorical cross-entropy
as the loss function, RMSprop as the optimizer, and
accuracy as the evaluation metric.

Training the Model:

The model is trained using the fit method. The training

data is provided using the training_set generator, and
validation data is provided using the valid_set
generator. The training process runs for 10 epochs.

Saving the Model:

The trained model is saved to a file named

'DSLABLMODEL1.h5' using the save method. This
file can be loaded later to make predictions on new
data.
FIG 4 :- SAMPLE PREDICTIONS
 IV. RESULTS

The results of the image classification project provide a

comprehensive evaluation of the trained Convolutional
Neural Network (CNN). Through visualizations and
metrics, the project gauges the model's effectiveness in
categorizing images into predefined classes. Training
and validation accuracy/loss plots offer insights into
the learning dynamics of the model, highlighting its
ability to generalize to new data. Mapping class indices
to class names facilitates the interpretation of model
predictions. ROC curves and AUC values quantify the
model's discrimination ability, while the confusion
matrix delves into specific instances of correct and FIG 6 :- ACCURACY AND LOSS PLOTS FOR
incorrect predictions for each class. The classification MODEL 3 AND 4
report further dissects precision, recall, and F1-score, ROC Curve:
offering a nuanced assessment of the model's
performance across classes. Precision-recall curves, The Receiver Operating Characteristic (ROC) curve is
along with AUC scores, provide a detailed analysis of plotted to visualize the trade-off between true positive
the model's precision and recall trade-offs for rate and false positive rate at different classification
individual classes. Lastly, sample predictions showcase thresholds. The Area Under the Curve (AUC) is
the model in action, offering a tangible glimpse into its calculated for each class, providing a single-value
classification prowess on real-world images. Together, summary of the model's discrimination ability.
these results empower a holistic understanding of the
model's strengths and areas for improvement, crucial
for refining its performance in image classification
tasks.

Training and Validation Accuracy/Loss Visualization:

The code includes visualizations of training and

validation accuracy as well as training and validation
loss over epochs. These plots give insights into the
model's learning progress. High training accuracy with
a corresponding high validation accuracy indicates that
the model is learning well and generalizing to unseen
data.

FIG 5 :- ACCURACY AND LOSS PLOTS FOR

MODEL 1 AND 2
FIG 7 :- ROC CURVE FOR MODEL 1 AND
2
 FIG 8 :- ROC CURVE FOR MODEL 3 AND 4
Confusion Matrix:

The confusion matrix is calculated and visualized. It

provides a detailed breakdown of correct and incorrect
predictions for each class. Diagonal elements represent
the true positives, and off-diagonal elements represent
misclassifications.

FIG 9 :- CONFUSION MATRICES FOR ALL THE

MODELS
Classification Report: Dataset:

The classification report is generated, presenting Context:

precision, recall, and F1-score for each class. Precision
is the ratio of correctly predicted positive observations This dataset is curated for the purpose of classifying
to the total predicted positives. Recall is the ratio of various styles of art, encompassing a diverse range of
correctly predicted positive observations to the total artistic expressions. It is tailored to recognize five
actual positives. F1-score is the harmonic mean of primary categories of art: Drawings and
precision and recall. Watercolours, Works of Painting, Sculpture, Graphic
Art, and Iconography (Old Russian Art). The aim is
to develop a model capable of accurately categorizing
images into these distinct artistic styles.

Content:

The dataset comprises approximately 9,000 images

sourced from various platforms, including Google
Images, Yandex Images, and other relevant sources.
Each image is associated with one of the five main
categories mentioned above. The diversity in art styles
and mediums allows for a robust training and
validation set, ensuring the model's capacity to
generalize across a broad spectrum of artistic
expressions.

Potential Use Cases:

Art Style Recognition: The dataset can be utilized to

train models capable of recognizing and classifying
diverse art styles.

Educational Tools: It can serve as a valuable resource

for educational tools and platforms focused on art
history and appreciation.

Art Curation Apps: Applications that curate and

categorize artworks based on their styles and genres
could benefit from this dataset.

Note:

The success of the model trained on this dataset

depends on factors such as the quality of labeled
data, model architecture, and training parameters.
Regular updates and refinements to the dataset may
enhance its utility for future applications.

FIG 10 :- CLASSFICATION REPORTS OF ALL

THE MODELS
Comparison of models: interpretation. The precision, recall, and F1-score
metrics, akin to the critique of an astute art historian,
provide a comprehensive evaluation of its discernment.
Model
Accua Precisi Rec
Name cy on all As we gaze upon the sample predictions, the model
unfolds its portfolio, showcasing its proficiency in
attributing the right label to each piece. The
CNN visualizations encapsulate the learning journey—the
1 81.52 0.848 0.78
% 2 12 rise in accuracy, the dance between loss and validation
loss—a journey that transforms a model into a digital
curator of the arts.
CNN
2 84.06 0.944 0.90
% 0 62 In conclusion, this model stands not just as a
technological artifact but as an ode to the synergy
between data science and art. It embodies the beauty of
CNN
3 82.14 0.755 0.78 classification, where the elegance of algorithms
harmonizes with the richness of artistic expressions. As
V. CONCLUSION the model rests in its saved state, it invites us to ponder
the boundless possibilities when AI and artistry
In the realm of image classification, this model, coalesce—a symphony of pixels, algorithms, and the
sculpted with precision and trained on a canvas of timeless essence of human creativity.
artistic diversity, emerges as a testament to the
intersection of technology and creativity. The journey VI. REFERENCES
through the dataset, spanning drawings, paintings,
sculptures, graphic art, and the enigmatic realm of 1. Deep Learning for Art Style Classification:
old Russian iconography, has imbued the model with
a profound understanding of the nuanced tapestry of Gatys, L. A., Ecker, A. S., & Bethge, M. (2015). A
artistic expressions. neural algorithm of artistic style. arXiv preprint
arXiv:1508.06576.
As the layers of the Convolutional Neural Network
unfold, each convolution and pooling operation Johnson, J., Alahi, A., & Fei-Fei, L. (2015). Perceptual
delicately extracts the essence of diverse art forms. The losses for real-time style transfer and super-resolution.
meticulous separation of data into training and In European Conference on Computer Vision (pp. 694-
validation sets has nurtured a model capable of not 710). Springer, Cham.
merely memorizing, but truly comprehending the
intrinsic features that distinguish one art style from Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2017).
another. Improved texture transfer. In International
Conference on Computer Vision (pp. 692-701).
The ROC curves, precision-recall curves, and the IEEE.
captivating confusion matrix unveil the model's
prowess in discerning the subtleties of art. The AUC Li, R., Liu, W., Xu, D., & Shum, H.-Y. (2020).
values, like strokes of mastery, quantify its ability to Universal style transfer via feature constellations. In
discriminate between different artistic genres. Through Proceedings of the AAAI Conference on Artificial
each epoch of training, the model has refined its Intelligence (Vol. 34, pp. 4856-4867).
brushstrokes, ensuring it doesn't merely recognize but
appreciates the soul of art encapsulated in every image. 2. Datasets for Art Style Classification:

This model is not just a classifier; it is an art UUC-ImageNet-StyleDB:

connoisseur—a digital curator of visual masterpieces. https://github.com/funnyzhou/C2L_MICCAI2020
Its ability to predict, with finesse, the category of art
captured in an image resonates with the spirit of artistic LAA DATASET:
https://faculty.sites.uci.edu/mdlee/similarity-data/
WikiArt dataset: 4. Additional Resources:
https://www.kaggle.com/datasets/ipythonx/wikiart-
gangogh-creating-art-gan A blog post on art style classification with TensorFlow:
https://www.tensorflow.org/lite/examples/style_transfer
Oxford-IIIT dataset: /overview
https://www.robots.ox.ac.uk/~vgg/data/pets/
A tutorial on building an art style classifier with Keras:
https://machinelearningmastery.com/machine-learning-
with-python/

A survey paper on art image processing:

https://arxiv.org/pdf/2210.15300