See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/350334147

A Deep Learning Approach for Efficient Palm Reading

Conference Paper · December 2020

DOI: 10.1109/TAAI51410.2020.00039

CITATIONS READS
0 372

3 authors, including:

Suvajit Acharjee Sirapop Nuannimnoi

National Chiao Tung University National Chiao Tung University
1 PUBLICATION   0 CITATIONS    6 PUBLICATIONS   17 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Semiconductor processes + Machine Learning View project

All content following this page was uploaded by Sirapop Nuannimnoi on 12 August 2021.

The user has requested enhancement of the downloaded file.

 2020 International Conference on Technologies and Applications of Artificial Intelligence (TAAI)

A Deep Learning Approach for

Efficient Palm Reading
2020 International Conference on Technologies and Applications of Artificial Intelligence (TAAI) | 978-1-6654-0380-1/20/$31.00 ©2020 IEEE | DOI: 10.1109/TAAI51410.2020.00039

Suvajit Acharjee Sirapop Nuannimnoi Ching-Yao Huang

EECS International Graduate Program EECS International Graduate Program Electronics Engineering
National Chiao Tung University National Chiao Tung University National Chiao Tung University
Hsinchu, Taiwan Hsinchu, Taiwan Hsinchu, Taiwan
[email protected] [email protected] [email protected]

Abstract— Palmistry or palm reading is the art of foretelling finger lengths, the outlines of palms extracted were analyzed
and characterizing persons through the study of palm lines and with a ratio system to characterize persons into 4 groups,
patterns. However, this field is still not much technically including Jupiter-ruled, Sun-ruled, Saturn-ruled, and
developed and a palmist has to analyze palms personally and Mercury-ruled persons. Leung and Law [2] proposed an
manually. In this paper, we have proposed a deep learning adaptive thresholding for segmentation of the palm image to
approach to automatically detect patterns inside palm images. separate foreground (palm) from the background in order to
Our proposed automated palm reader can effectively detect and extract fingers, and the three principal palm lines. A regression
classify a user’s palm according to our predefined labels. model was applied for producing connected and continuous
Keywords—Palmistry; Palm reading; Deep Learning; Object
palm lines. Then, based on 2D:4D ratio principle according to
detection; Convolutional Neural Network; Multi-label traditional Chinese “Feng Shui”, they determined the persons’
classification personalities and health. Tin [3] also developed a palm reading
system with principal lines detection using Canny Edge
I. INTRODUCTION detection algorithm and Hough transform.
Palmistry is an art to forecast people’s future regarding their As mentioned earlier, these existing solutions rely on obsolete
behavior, characteristic, career, wealth etc. This field is also image processing techniques, which are not effective for
called as Chiromancy also popularly known as Palmist. It is automated palm reading. Some patterns cannot be recognized
said from the ancient scriptures that it has been originated and differentiated through edge detection and Hough
from India and then it took a rise in interest and spreading transform techniques. The accurate analysis of the present and
through other Asian and Western countries mostly China, the future with a high accuracy is also not found in those
Tibet etc. There are many famous palmists in this modern era. current applications. In this work, we explore the potential of
This practice tells us many things from the lines on the palm deep learning algorithms to recognize such patterns in palm
like the head/mind line (the logical and intelligence), heart line reading and provide more accurate prediction results.
(the emotions and love), life line (native will be healthy
enough or not), Sun line (success, career, luck), some extra III. OUR PROPOSED METHOD
ordinary signs and symbols on one’s palm. The lines, shape of We propose a unique deep learning framework for automated
the hands, the size of the fingers and the mounts can describe palm reading using deep learning algorithms. Our proposed
one’s individual characteristics and show the graph of one’s solution aims to provide more detailed and accurate results
unconscious mind. Although it is a controversial topic, it is a than existing solutions and applications.
science in which it predicts the past, present and the future of
the native from the unconscious mind and the lines and Deep Learning (DL) is a useful subset of machine learning
mounts of a person. methods based on artificial neural networks. DL architectures
such as deep neural networks (DNN) and convolutional neural
There are many automated palms reading, but most of them networks [7 - 9] (CNN) have been applied heavily in the field
are not efficient enough. The existing solutions cannot provide of computer vision and machine vision with outstanding
the accurate analysis like a palmist can do. In this paper, we performance. CNNs are similar to ordinary Neural Networks
have proposed a deep learning-based palm reading application in the fact that they consist of many layers of neurons with
which can analyze the palm of an individual and tell about his some learnable weights and biases. A CNN consists of two
/ her native accurately. important building blocks, which are convolutional layers and
The rest of this paper are as follows. Section II reviews some fully-connected layers. They can classify images, detect
recent automated palm reading techniques based on image objects in images and perform object segmentation. They also
processing algorithms. Section III describes our proposed have abilities to process images of various sizes. These are the
method. Section IV explains the experimental setups and reasons why we intend to apply these useful DL architectures
performance metrics used in this study. Section V shows in our work.
results and discussion. Finally, Section VI concludes this There are three phases in our approach of deep learning-based
paper with some possible future works. automated palm reading:
II. LITERATURE REVIEW A. Semantic Segmentation of Palm images
Over the past decade, there have been a few attempts to use First, palms needed to be taken out from the input images.
computer vision and image processing techniques to perform We scaled all images to 1024 pixels × 2048 pixels, and used
automated palm reading. Vishwaratana et al. [1] used Canny labelme [4] to prepare segmentation masks for all images. In
edge detector and Hough Transform to detect palm outlines order to segment the foreground palm image from the
inside palm image samples. On the basis of their palm and background, we trained our palm image dataset using a

978-1-6654-0380-1/20/$31.00 ©2020 IEEE 171

DOI 10.1109/TAAI51410.2020.00039

Authorized licensed use limited to: National Chiao Tung Univ.. Downloaded on August 12,2021 at 07:59:38 UTC from IEEE Xplore. Restrictions apply.
 semantic segmentation technique. Semantic segmentation [5],
also known as scene parsing, is a group of machine learning
tasks whose goals are to give each pixel of an image a proper
object category label. Fast-SCNN [6] or fast segmentation
convolutional neural network is a real-time semantic
segmentation model on high resolution image data (1024 ×
2048 pixels) suitable for efficient computation on devices with
low memory. The algorithm was claimed to achieve at least
68 % of mean Intersection over Union (IoU) in real-time
segmentation of Cityscapes dataset. In addition, Fast-SCNN
does not require large pre-training. The overall architecture of Fig. 3. Grid cells defined in each palm image (5 x 5 grids)
Fast-SCNN is as shown in Fig. 1. In the first region, class 0 depicts that a person will be
wealthy, very famous, and successful because the line as
shown and marked in red in Fig. 4 (left) represents the sign
of success without issues or obstacles. This line appears very
long in this region, which represents long-term success. Class
1 is similar to class 0. The wealth line, as marked in red, is
shown but the blue diagonal line is also shown which
signifies struggle at that particular age. This is a sign of
defects, which represents a person’s needs to overcome some
Fig. 1. Fast-SCNN architecture obstacles. The pattern of class 2 shows a lot of issues in life
because more defects are obviously found in this pattern than
Fast-SCNN, as shown above, is constructed using 4 major class 1. For class 3, a person has many more of red-marked
building blocks including Learning to Down-sample, Global lines for success and wealth as well as vertical blue-marked
Feature Extractor, Feature Fusion, and finally the classifier. lines. This shows the division of good and bad energies in an
All building blocks are built using depth-wise separable individual. His / her life will be considered “normal”, which
convolution. In the first building block, low level features such means he / she will not be very successful, famous, and
as edges and corners from the image are extracted by deep wealthy.
CNNs. After feature fusion, two depth-wise Separable
convolutional layers followed by one Point-wise
convolutional layer are introduced. At the end of each layer,
both normalization layers and ReLu activations are applied.
In this work, we labeled a palm in each input image as class 1,
and the rest part of the image or background as class 0. The
labeling tool, labelme, converted these label files into PNG
files for our semantic segmentation model. An example of
these resulting files is shown in Fig. 2.

Fig. 4. Region 1 (left) and Region 2 (right)

Region detail
Region
ID Grid
Class Description
list
Will be wealthy, successful, and have a
0
good name
Fig. 2. Palm image segmentation: (a) an original sample, (b) a labeled Native will earn good name and will be
segmentation mask, and (c) the expected ending result before applying grids 1
successful but with great difficulty
in the next step 1, 6, 2,
1 2 Will not a good wealth, failure in life
7
B. Palm regions with Multiple-Grid Approach 3 Normal wealth and success
Second, new palm images with white background were 4 Have good impressive, creative skills
divided by 5 x 5 grids, as shown in Fig. 3 below. The order Will be wealthy, successful, and have a
of these grids is from the top to the bottom, and from the left 0
good married life
to the right. The index starts from 0 to 24. These grids were 1
Will be wealthy but not able to keep
11, 16, stability of his/her income
put together to make each region, which would be fed into 2
12, 17
each convolutional neural network for further classification 2 Unsuccessful
of personalities, lifestyles and future directions in life. Each 3
Problem in family, stressful, obstacles
of these regions is treated as an image, which is re-scaled will be there
depending on how many grids it includes. 0 Good fate

These regions along with their descriptions and list of class 3 7, 8, 9 1 Average
labels are summarized in Table 1. 2 Bad luck
Table I List of grid-based palm regions

172

Authorized licensed use limited to: National Chiao Tung Univ.. Downloaded on August 12,2021 at 07:59:38 UTC from IEEE Xplore. Restrictions apply.
 In the second region, class 0 shows the person’s good IV. EXPERIMENTATION
marriage life and wealth due to the appearance of a prominent We collected our own private palm datasets by providing
mount shown in Fig. 4 (right). Class 1, as seen on the mount, actual palm reading services. With permission from our
some blue-marked lines appear as defects. These are not good sample groups, we collected 553 palm images in total for this
for retaining wealth as there will be more expenses despite experiment. 85 % of this dataset was used for training
good incomes. Class 2 shows the weak ambition and life semantic segmentation model and multi-class convolutional
goals as the mount appeared is not prominent enough to draw neural networks. The rest was used for testing the performance
success and express leadership qualities. Class 3 shows more of the trained models. We set the training epochs of both
defects which signify relationship stress, unhappy married models at 100 iterations. In this study, we focus on right
life and tension of career. palms.
The order of our proposed approach to study the performance,
mean Intersection-over-Union (IoU, Jaccard Index) is used as
performance metrics for semantic segmentation of palm
images. The calculations of these metrics are given as
| ∩ |
= (∑ )/ (1)
| ∪ |

where X is the predicted pixel set values and Y is the

ground truth pixel set values.
Accuracy scores and F1 scores are used as performance
metrics for multi-label multi-class classification models. The
Fig. 5. Region 3
calculations of these metrics are given as
= (2)
Finally, in the last region as shown in Fig. 5, a pattern for
class 0 shows that the person has a clear fate line, as marked = (3)
in red. There is no defect found in this pattern, therefore a
person will be very lucky. If this line is missing (in case of = (4)
class 2), the person will not have good lucks in life and will
struggle a lot. For class 1, although the fate line, marked in 1= 2∙
×
(5)
blue, is found on the palm region, he / she will still face some
issues due to the break inside the line. where TP, TN, FP, and FN denote true positive, true
negative, false positive, and false positive, respectively.
C. Multi-Class classification
Third, each region as defined above by our proposed grid To evaluate the performance of our deep CNN algorithm for
system was fed into a convolutional neural network for palm reading, we compare its classification performance with
training. The architecture detail of our deep CNN is described well-known complex CNN architectures including AlexNet
as follows. [8], and ResNet [10]. These two sophisticated architectures
were successfully used for image classification tasks such as
Our deep CNN receives the input of fixed size of (512, 512, MNIST and CIFAR. We keep every hyperparameter of both
3), which represents the shape of an image, which in this case models by default.
it is a cropped region from a full palm image, and its
corresponding RGB values. For any images whose sizes are V. RESULTS AND DISCUSSION
bigger than 512 x 512 pixels, they are all scaled down using a Table II and Table III report the performance of the semantic
high-quality down-sampling filter. For any images whose segmentation and classification models used in this study,
sizes are smaller than 512 x 512 pixels, we pad them with respectively.
white pixels. The first convolutional block consists of 25 2D Dataset Mean IoU
filters of size 5 x 5 pixels, followed by a ReLu activation and
a max pooling layer of size 2 x 2. The second convolutional Train set 72.8 %
block consists of 50 2D filters of the same size. In order to Test set 68.0 %
stabilize the learning process and accelerate the training of this
architecture, a batch normalization is applied before the output Table II Performance of segmentation algorithm on our dataset
of this block enters the next convolutional block. The third As shown in Table II above, Fast S-CNN still delivers good
convolutional block is similar to the second one, but it has 70 enough mean IoU in the task of semantic segmentation on our
filters of only 3 x 3 pixels. After all the features from an input dataset. An IoU of more than 0.5 is normally considered a
image are extracted by the convolutional layers, the resulting “good” prediction. However, it will be nicer if our predictions
output is flattened into a 1D vectors which are then fed into of mask areas are averagely closer to 1. Therefore, we could
two layers of fully-connected layers of 100 hidden units. avoid any mistakes when we crop the palm out of each image.
Finally, the output layer varies based on the number of classes This will eventually lead to our solution becomes applicable.
specified for each region. Softmax activation function is Table III illustrates the classification performance of our
applied to calculate the class probabilities. The optimization CNN. The accuracy scores and F1 scores for the three regions
algorithm used in the models is Adam. have been obtained from the models. We obtained 98.29% and
93.97% accuracy score from region 1 from the train set and
test set, whereas from region 2 accuracy score of 99.57% and

173

Authorized licensed use limited to: National Chiao Tung Univ.. Downloaded on August 12,2021 at 07:59:38 UTC from IEEE Xplore. Restrictions apply.
 97.59% obtained from train set and test set, respectively. The performance of Fast-SCNN is acceptable but it will be better
region 3 has less accuracy score in both the train set and the to achieve mean IoU closer to 1. For image classification task
test set which is 94.88% and 83.14%, respectively. like this, model complexity and amount of data samples used
for training also play significant roles in improving predictive
The less accuracy scores and F1 scores in the region 3 may be performance. Our results suggest that we need more amount
resulted from unclear patterns of the region found in the of training and test data samples to make our proposed
dataset. It is also possible that more training images or better solution more reliable.
AI architecture could improve the predictive performance of
this region. Possible directions for future works include the increase of
data samples in order to avoid overfitting, improvement and
Classification Performance
Region
development of palm segmentation algorithms and life
F1-score
ID
Accuracy scores direction and human personality classification algorithms,
Train Test Train Test more comprehensive comparison study with other detection
1 98.29 % 93.97 % 0.9156 0.9253 algorithms and classification algorithms, and the deployment
on mobile platforms. Other directions such as end-to-end
2 99.57 % 97.59 % 0.9759 0.9786
classification methods for automated palm reading should
3 94.88 % 83.14 % 0.8313 0.8252 also be considered. Last but not least, this approach could
Table III Classification performance of our CNN on each region also be applied on the left palm images in our future work as
well. A combination of prediction results from both palms
Table IV and V shows the classification performances of might also enhance the foretelling of a person.
AlexNet and ResNet on each region. ResNet shows very low
accuracy scores on validation sets, compared to our more REFERENCES
simple model. It is very likely that ResNet model is overfit to [1] N. Vishwaratna, Y. Divakar, and T. Manisk, “A Novel Approach for
our training set as well. AlexNet does not seem to converge Hand Analysis Using Image Processing Techniques,”International
after a long time training. Journal of Computer Science and Information Security. May, 2010.
[2] K. Leung and N. F. Law, “An efficient automatic palm reading
AlexNet’s Performance algorithm and its mobile applications development,” 2016 IEEE
Region F1-score International Conference on Signal Processing, Communications and
Accuracy scores
ID Computing (ICSPCC), Hong Kong, 2016, pp. 1-6, doi:
Train Test Train Test 10.1109/ICSPCC.2016.7753706.
1 53.47 % 34.93 % 0.5132 0.3411 [3] K. H. H. Tin, “An effective method of a person’s character or future
using the palmprint images,” International Journal of Research and
2 55.82 % 36.14 % 0.5253 0.3646 Scientific Innovation. IV. 09-12. Dec, 2016.
3 56.06 % 33.37 % 0.5337 0.3605 [4] K. Wada, “labelme: Image Polygonal Annotation with Python,” 2016.
Table IV Classification performance of AlexNet on each region [5] N. Atif, M. Bhuyan and S. Ahamed, “A Review on Semantic
Segmentation from a Modern Perspective,” 2019 International
ResNet’s Performance Conference on Electrical, Electronics and Computer Engineering
Region
Accuracy scores F1-score (UPCON), ALIGRAPH, India, 2019, pp. 1-6, doi:
ID
Train Test Train Test 10.1109/UPCON47278.2019.8980189.
[6] R. PK Poudel, S. Liwiki, and R. Cipolla, “Fast-SCNN: Fast Semantic
1 91.47 % 54.22 % 0.9059 0.5479
Segmentation Network,” in 30th British Machine Vision Conference,
2 97.65 % 74.70 % 0.9569 0.7484 BMVC 2019, Cardiff, Welsh, UK, September 10-12, 2019.
[7] Ezat, W., Dessouky, M. and Ismail, N., 2020. Multi-class Image
3 88.91 % 79.52 % 0.8137 0.7477 Classification Using Deep Learning Algorithm. Journal of Physics:
Table V Classification performance of ResNet on each region Conference Series, 1447, p.012021.
[8] Krizhevsky, A., Sutskever, I. and Hinton, G., “ImageNet classification
with deep convolutional neural networks,” 2017 Communications of
VI. CONCLUSION AND FUTURE WORKS the ACM, 60(6), pp. 84-90.
This research work is the first one to develop a deep learning- [9] F. Sultana, A. Sufian and P. Dutta, “Advancements in Image
based palm reading technique that could segment the palm Classification using Convolutional Neural Network,” 2018 Fourth
from the background, and predict future life directions of International Conference on Research in Computational Intelligence
and Communication Networks (ICRCICN), Kolkata, India, 2018, pp.
individuals. The proposed solution seems to have potential to 122-129, doi:10.1109/ICRCICN.2018.8718718.
be used with acceptable accuracy on mobile applications in [10] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for
the future. However, there are still some important problems Image Recognition," 2016 IEEE Conference on Computer Vision and
to address. Our algorithm still needs more improvement in Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 770-778, doi:
terms of predictive performance. Semantic segmentation 10.1109/CVPR.2016.90.

174

Authorized licensed use limited to: National Chiao Tung Univ.. Downloaded on August 12,2021 at 07:59:38 UTC from IEEE Xplore. Restrictions apply.
View publication stats