Scribd
Upload
9 views

Reference

This document summarizes a research paper that proposes a wine label image recognition method using convolutional neural networks (CNN) with augmented data. The method builds a dataset of 130 wine label images, applies data augmentation techniques like cropping, rotation and hue modification to enlarge the dataset, designs a CNN model for recognition, and achieves 98% accuracy when combining all augmentation techniques, compared to only 40% accuracy without augmentation. Experimental results demonstrate that data augmentation improves the performance of CNNs for wine label image recognition.

Uploaded by

Huong Nguyen Thi
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
9 views

Reference

This document summarizes a research paper that proposes a wine label image recognition method using convolutional neural networks (CNN) with augmented data. The method builds a dataset of 130 wine label images, applies data augmentation techniques like cropping, rotation and hue modification to enlarge the dataset, designs a CNN model for recognition, and achieves 98% accuracy when combining all augmentation techniques, compared to only 40% accuracy without augmentation. Experimental results demonstrate that data augmentation improves the performance of CNNs for wine label image recognition.

Uploaded by

Huong Nguyen Thi
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

International Conferences Interfaces and Human Computer Interaction 2019; Game and Entertainment Technologies 2019;

and Computer Graphics, Visualization, Computer Vision and Image Processing 2019

WINE LABEL IMAGE RECOGNITION USING


CONVOLUTIONAL NEURAL NETWORK WITH
AUGMENTED DATA

Mei-Yi Wu
Graduate Institute of Food Culture and Innovation, National Kaohsiung University of Hospitality and Tourism, Taiwan

ABSTRACT
Wine label information was found to have a positive effect on consumer choice. However, it is not easy for consumers to
read the information on wine labels since these texts are usually not print in English. Wine label information system with
visual recognition function seems to be important as an auxiliary reading tool for customers. In this paper, we proposed
an efficient wine image label recognition method using convolutional neural network(CNN) algorithms. Data
augmentation techniques for machine learning with different image manipulation such as cropping, rotation and hue
modification were applied to enlarge the dataset. Experimental results show a high accuracy on 130 different wine label
images using the proposed CNN model with mixing different image manipulation.

KEYWORDS
Wine Label Image Recognition, Convolutional Neural Network, Augmented Data, Deep Learning

1. INTRODUCTION
In order to encourage consumption in a country without a strong culture of wine drinking, the expanding
wine producers put their efforts on the design of wine labels. Some wine marketing studies make reference to
the importance of wine labels and the information they contain. The information content of wine labels can
be grouped under seven information positioning statements: namely, parentage, nonpareil, manufacture,
attributes, endorsements, end user and end use. Nested within some of these statements is other information
commonly associated with wine labels (Thomas et al., 2003,). However, it is not easy for consumers to read
the information on wine labels since these texts are usually not print in English. A simple approach to solve
this problem is to develop a mobile wine label image recognition system. Customers can take a shot of wine
label photograph with mobile phone camera and the related information about the wine label will be
displayed on the mobile phone.
Two different approaches, traditional image recognition and deep learning algorithms, are generally
applied in the implementation of image recognition systems. Traditional image recognition schemes are
based on specific handcrafted feature extraction. On the contrary, deep learning schemes optimize the
features that are extracted. The major difference between deep learning and conventional methods is that
deep learning automatically learns features from big data, instead of adopting handcrafted features, which
mainly depends on priori knowledge of designers and is highly impossible to take the advantage of big data.
Deep learning can automatically learn feature representation from big data, including millions of parameters
which largely explains why these deep learning perform better (Liang et al., 2017). Although great
achievements and perspectives, deep learning algorithms have some relevant challenges to tackle. The most
frequently mentioned problem in the field of machine learning is the lack of sufficient amount of the training
data within the datasets (Mikołajczyk et al., 2018,). In this paper, we have compared and analyzed multiple
methods of data augmentation like cropping, rotation and hue modification to enlarge the dataset. More
details about the proposed method and experimental results can be found in the next session.

479
ISBN: 978-989-8533-91-3 © 2019

2. PROPOSED METHOD
The implementation of proposed wine label image recognition system can be divided into four steps (1) Build
a small-size wine label image dataset (2) data augmentation (3) Design CNN algorithms (4) Wine label
recognition. The flowchart of proposed system is illustrated in Figure 1.

Build a small-size wine Data augmentation


label image dataset

wine label Convolutional Wine label


image dataset Neural Network recognition

Figure 1. The flowchart of proposed wine label image recognition system

2.1 Build a Small-Size Wine Label Image Dataset


To fit the need of user’s convenience, camera of mobile phones is preferred in proposed system to capture
wine label photos. In building the wine label image dataset, we have selected 130 different wine bottles from
wine retailers and taken 5 photos in different views for each wine bottle. Figure 2 shows the process to build
the small-size wine label image dataset.

(c)
Figure 2. Build a small-size wine label image dataset. (a) a wine label image shot by mobile phone camera (b) an wine
label image example (c) 5 images captured in different views for each wine bottle

2.2 Data Augmentation


Three different image manipulations were used to enlarge the small-size dataset (including cropping, rotation
and hue modification). Figure 3 shows an example for image manipulation with cropping, rotation and hue
modification, respectively. In this study, each image sample in small-size dataset was modified in 2 factors.

Figure 3. An example for image manipulation with cropping, rotation and hue modification

480
International Conferences Interfaces and Human Computer Interaction 2019; Game and Entertainment Technologies 2019;
and Computer Graphics, Visualization, Computer Vision and Image Processing 2019

2.3 Convolutional Neural Network and Experimental Results


The CNN offers a state-of-the-art technique for image recognition (Zhang et al. 2018). In this study, images
were resized into 64 × 64. The proposed CNN model comprises 4 convolution layers and 2 pooling layers,
with a fully connected layer to produce the final result of the task. In image classification, each unit of the
final layer indicates the class probability.
Three different image manipulations were used to enlarge the small-size dataset(including cropping,
rotation and hue modification). There are totally 130 classes of wine label images in the original dataset.
Each class has 5 different view images. In this study, each type of image manipulation will generate two
times of “similar” images. We performed the proposed CNN model to train the dataset with different image
manipulations. The original method without performing any data augmentation obtained 0.4 test accuracy
rate. The test accuracy rates are 0.96, 0.83 and 0.88 with hue modification, rotation and cropping,
respectively. And the best performance is obtained by mixing all types of augmentation operations and the
accuracy rate can achieve 0.98. Figure 4 shows these experimental results.

Original: accuracy 0.4 Hue modification: accuracy 0.96 Rotation: accuracy 0.83

Cropping: accuracy 0.88 Mixed all: accuracy 0.98


Figure 4. Test accuracy using different manipulations

ACKNOWLEDGEMENT
We would like to thank the Ministry of Science and Technology of ROC for supporting this work in part
under Contract MOST 107-2637-E-328-001.

REFERENCES
Thomas, A., & Pickering, G., 2003, The importance of wine label information. International journal of wine marketing,
15(2), pp. 58-74.
Liang, H., Sun, X., Sun, Y., & Gao, Y., 2017, Text feature extraction based on deep learning: a review. EURASIP journal
on wireless communications and networking, 2017-211, pp.1-12.
Mikołajczyk, A., & Grochowski, M., 2018, Data augmentation for improving deep learning in image classification
problem. In 2018 international interdisciplinary PhD workshop (IIPhDW), pp. 117-122.
Zhang, X., Xv, C., Shen, M., He, X., & Du, W., 2018, Survey of Convolutional Neural Network. In 2018 International
Conference on Network, Communication, Computer Engineering (NCCE 2018).

481

You might also like