Artificial Intelligence Algorithms To Predict Customer Satisfaction: A Comparative Study
Artificial Intelligence Algorithms To Predict Customer Satisfaction: A Comparative Study
Corresponding Author:
Othman Berrada Chakour
RITM Laboratory/ESTC, CED ENSEM, Hassan II University
Casablanca, Morocco
Email: [email protected]
1. INTRODUCTION
Customer satisfaction is a term widely used in marketing industry, and represents a service or product
evaluation indicator, that needs to meet the client expectations. This assessment considers all the criteria that
matter to the customer and that he will experience once the product/service is delivered. Customer satisfaction
could have a positive influence on the retention rate and loyalty, also reduces churn rate and protects customers
from other competitors especially since the competition has never been so fierce, thanks to the internet. Client
satisfaction determines whether your business has a positive or negative brand image over the long term.
Furthermore, consumer expectations are constantly changing. Faced with the considerable and
limitless growth for data exchanged, customer specification in terms of responsiveness and efficiency became
denser. Faced with such a need for responsiveness and efficiency, it is obvious that the link with technologies
was quickly established. The previous researches results show that customer satisfaction prediction can be
analyzed using various types of machine learning (ML) and deep learning (DL) algorithms. Due to different
data formats (such as review data or survey data), different models are recommended for different application
domains. Artificial intelligence (AI), ML and DL have become vital to remain competitive for financial
services companies around the globe.
The use of AI algorithms helps predict customer satisfaction and future behavior regarding the
purchase of a particular product again. Having a good customer satisfaction prediction model gives insight into
the future, which can help companies choose and adjust the strategies to implement and the right decisions to
make to achieve a high level of customer satisfaction. AI algorithms are used today in several sectors and
industries. In particular, the sub-domains of AI: ML and DL algorithms are used in banking sector to predict
customer churn [1]‒[3] and customer satisfaction [4], [5]. Therefore, they are used in the e-commerce industry
to predict the online shopper’s behavior to know if a customer will make a new purchase or not on the site
[6]‒[8]. Hence, ML and DL algorithms are applied in the telecommunication sector to predict customer churn
[9], in airlines domain to predict the satisfaction of plane passengers [10], [11], in transportation sector for
predicting car accidents [12], in agriculture field to predict demand and the products pricing [13], in healthcare
and smart home to detect falls [14], in smart manufacturing applications for predicting tool wear [15], in
entertainment domain for correcting postures of piano players [16], and several other fields.
Compared with other studies, our paper not only finds different relationships between factors
affecting customer satisfaction prediction, but also includes comparing the performance of various ML and
DL methods. This paper shows the comparative study of research domains and applications of AI models in
predicting customer satisfaction. This study began by defining the methods based on the prediction process.
In this section, we focused on the popular AI algorithms. In the second part of this article, we presented
various researches that uses ML and DL algorithms to predict customer satisfaction. Finally, we presented an
analysis of these researches through a benchmark that details for each article, the algorithms used, their types,
the date of publication of the article, the metrics evaluated and the optimal algorithm in terms of performance
based on the score of the different metrics.
2. METHOD
AI refers to attempts to build machines able to challenge the human on its intelligence. ML and DL
represent two particularly important steps in the evolution of AI. We define ML as a subset of AI developed
to imitate human intelligence. While DL is, a subset of ML based on artificial neural networks (ANNs).
Figure 1 represents the relationship between AI, ML and DL.
To predict customer satisfaction, it was necessary to study the different algorithms to define the
most efficient method. In this section, several popular ML algorithms such as logistic regression (LR),
decision tree (DT), random forest (RF), gradient boost (GB), k-nearest neighbors (KNN), support vector
machine (SVM), naive Bayes (NB), and DL algorithms such as ANN, convolutional neural network (CNN),
long short-term memory (LSTM), CNN-LSTM are discussed. Through this section, we will define the
previous mentioned algorithms and share examples of their application domain and results.
the discriminating characteristics of this population. It facilitates the read of a complex situation and it helps
to quickly execute the decision made [20]. DT is utilized in several application domains. The best
performances are showed in the subsequent researches. DT was used to assess the problem of predicting
credit card churn in banks [3], predict satisfaction of online banking customer in Bangladesh [4], predict
e-commerce customer satisfaction [7] and analyze the propensities of South Korea airlines customers for
predicting churn and customer satisfaction [16].
algorithm is employed in different application domains. The best results are represented in the following papers.
The ANN was used to predict bank customer satisfaction [5], assess the problem of predicting credit card churn in
banks [3], predict customer churn based on behavioral model analysis in e-commerce domain [8], calculate the
bank customer churn prediction [1], and predict churn prediction of supermarket customer in retail industry [27].
2.3.1. Accuracy
Accuracy is an important and intuitive metric as it calculates the proportion of correct predictions
out of the total number of predictions [34]. It measures how often a ML model correctly predicts the
outcome. This metric is simple to calculate and understand. Accuracy can be calculated by (1).
∑ 𝑇𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒+∑ 𝑇𝑟𝑢𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = × 100 (1)
∑ 𝑇𝑜𝑡𝑎𝑙 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛
2.3.2. Precision
Precision appears to be an important metric in monitoring model performance, as it generally
calculates the ratio between the best predicted results and the effective results [34]. It measures the frequency
of ML predicting the positive cases. Precision can be calculated by (2).
∑ 𝑇𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = × 100 (2)
∑ 𝐹𝑎𝑙𝑠𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+ ∑ 𝑇𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒
2.3.3. Recall
Recall is the ratio of the number of correctly predicted positive instances (TP) to the total number of
actual positive instances [34]. In other words, it measures the completeness of the ML model when it
identifies positive cases. Recall can be calculated by (3).
∑ 𝑇𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒
𝑅𝑒𝑐𝑎𝑙𝑙 = × 100 (3)
∑ 𝑇𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+ ∑ 𝐹𝑎𝑙𝑠𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒
2.3.4. F1 score
F1 score provides a single metric that encapsulates both dimensions of a model’s accuracy. It is a
weighted comparison of average precision and recall [34]. This metric ensures that precision and recall
contribute equally to the score. The F1 score is particularly insightful when you need a single measure to
balance precision and recall. F1 score can be calculated by (4).
∑ 𝑅𝑒𝑐𝑎𝑙𝑙∗ ∑ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛
𝐹1 𝑆𝑐𝑜𝑟𝑒 = 2 ∗ × 100 (4)
∑ 𝑅𝑒𝑐𝑎𝑙𝑙+∑ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛
are in the telecommunications sector and 5% in the retail domain. Figure 4 shows that the most used metrics
include accuracy first, then F1-score, recall, and precision.
Figure 5 indicates that among the most used ML algorithms, RF obtains the first position, then
SVM, GB, and DT. RF and GB give the best performance. Figure 6 shows that the most used DL algorithms
are CNN, LSTM, and the combination of this 2 algorithms CNN-LSTM. The optimal algorithm in terms of
performance is CNN-LSTM. We can also notice that DL models exceed ML models in most cases if these 2
types of AI algorithms are used in the same paper.
12 5
10
4
8
Frequency
Frequency
3
6
2
4
2 1
0 0
LR DT RF SVM KNN GB NB ANN CNN LSTM CNN-LSTM
ML Algorithms DL Algorithms
Used Best frequency Used Best frequency
Figure 5. Paper distribution by used ML algorithms Figure 6. Paper distribution by used DL algorithms
In our study, we discovered that many studies on predicting customer satisfaction highlight issues
with data availability and question the reliability of findings. Therefore, customer satisfaction is considered
subjective, relative, dynamic, and influenced by individual perceptions and subject to change over time,
making its definition and measurement challenging. However, further, and in-depth studies are warranted to
explore alternative data sources that could mitigates the limitations explained before, such as collecting
qualitative data from client feedback sessions or integrating data from emerging technologies like sentiment
analysis of social media posts. Future studies may explore to gather real-world data on customer satisfaction
and implement regular satisfaction surveys focusing on emotional factors. Based on our study we concluded
that the customer satisfaction’s prediction is applicable on a multitude domain. Certainly, there are some
limitations that we already listed previously in this paper, but following implications mentioned also for
future research should focus on developing robust methodologies for data cleaning and preprocessing to
enhance the reliability of predictions.
4. CONCLUSION
In summary, this paper explains the importance of customer satisfaction for any business, which
pushes decision makers to be proactive by implementing strategies based on correct predictions and close to
reality. To predict customer satisfaction, it was necessary to study the different prediction algorithms to
define the most efficient one in terms of accuracy and precision. This paper aims to find the most accurate
prediction model for customer satisfaction. The study’s results showed that RF and GB algorithms in ML and
CNN-LSTM in DL provide the best performance. In addition, DL algorithms outperform ML in most cases.
Over this study, it turns out that most articles related to the prediction of customer satisfaction report the
unavailability of data and question their reliability. Therefore, customer satisfaction is subjective, relative,
and evolving, because it depends on the individual perception of the client and can vary over time.
Satisfaction is never purely rational; it is also a matter of emotions. This is what makes it so difficult to
define and measure. As next step, we intend to collect real data related to customer satisfaction, also set up
frequent satisfaction surveys considering the factors related to the customer emotions.
REFERENCES
[1] S. C. K. Tékouabou, Ș. C. Gherghina, H. Toulni, P. N. Mata, and J. M. Martins, “Towards explainable machine learning for bank
churn prediction using data balancing and ensemble-based methods,” Mathematics, vol. 10, no. 14, Jul. 2022, doi:
10.3390/math10142379.
[2] P. Chen, N. Liu, and B. Wang, “Evaluation of customer behaviour with machine learning for churn prediction: The case of bank
customer churn in Europe,” in Proceedings of the International Conference on Financial Innovation, FinTech and Information
Technology, FFIT 2022, Shenzhen, China, EAI, 2023, doi: 10.4108/eai.28-10-2022.2328450.
[3] A. Muneer, R. F. Ali, A. Alghamdi, S. M. Taib, A. Almaghthawi, and E. A. A. Ghaleb, “Predicting customers churning in
banking industry: A machine learning approach,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 26,
no. 1, pp. 539-549, Apr. 2022, doi: 10.11591/ijeecs.v26.i1.pp539-549.
[4] S. F. Shetu, I. Jahan, M. M. Islam, R. A. Hossain, N. N. Moon, and F. N. Nur, “Predicting satisfaction of online banking system in
Bangladesh by machine learning,” in 2021 International Conference on Artificial Intelligence and Computer Science Technology
(ICAICST), IEEE, Jun. 2021, pp. 223–228, doi: 10.1109/ICAICST53116.2021.9497796.
[5] N. Zeinalizadeh, A. A. Shojaie, and M. Shariatmadari, “Modeling and analysis of bank customer satisfaction using neural networks
approach,” International Journal of Bank Marketing, vol. 33, no. 6, pp. 717–732, Sep. 2015, doi: 10.1108/IJBM-06-2014-0070.
[6] V. Parihar and S. Yadav, “Comparative analysis of different machine learning algorithms to predict online shoppers’ behaviour ,”
International Journal of Advanced Networking and Applications, vol. 13, no. 06, pp. 5169–5182, 2022, doi:
10.35444/IJANA.2022.13603.
[7] A.-N. Wong and B. P. Marikannan, “Optimising e-commerce customer satisfaction with machine learning,” Journal of Physics:
Conference Series, vol. 1712, no. 1, Dec. 2020, doi: 10.1088/1742-6596/1712/1/012044.
[8] S. Agrawal, A. Das, A. Gaikwad, and S. Dhage, “Customer churn prediction modelling based on behavioural patterns analysis
using deep learning,” in 2018 International Conference on Smart Computing and Electronic Enterprise (ICSCEE), IEEE, Jul.
2018, pp. 1–6, doi: 10.1109/ICSCEE.2018.8538420.
[9] J. Maan and H. Maan, “Customer churn prediction model using explainable machine learning,” International Journal of
Computer Science Trends and Technology (IJCST), vol. 11, no. 1, pp. 33-38, Mar. 2023.
[10] S. S. Roy, D. Kaul, R. Roy, C. Barna, S. Mehta, and A. Misra, “Prediction of customer satisfaction using naive bayes, multiclass classifier,
K-star and IBK,” in Advances in Intelligent Systems and Computing, vol. 634, 2018, pp. 153–161, doi: 10.1007/978-3-319-62524-9_12.
[11] S.-H. Park, M.-Y. Kim, Y.-J. Kim, and Y.-H. Park, “A deep learning approach to analyze airline customer propensities: the case
of South Korea,” Applied Sciences, vol. 12, no. 4, Feb. 2022, doi: 10.3390/app12041916.
[12] B. K. Mohanta, D. Jena, N. Mohapatra, S. Ramasubbareddy, and B. S. Rawal, “Machine learning based accident prediction in
secure IoT enable transportation system,” Journal of Intelligent & Fuzzy Systems, vol. 42, no. 2, pp. 713–725, Jan. 2022, doi:
10.3233/JIFS-189743.
[13] T. Chuluunsaikhan, G. A. Ryu, K. H. Yoo, H. Rah, and A. Nasridinov, “Incorporating deep learning and news topic modeling for
forecasting pork prices: The case of South Korea,” Agriculture, vol. 10, no. 11, pp. 1–22, Oct. 2020, doi: 10.3390/agriculture10110513.
[14] N. Thakur and C. Y. Han, “A study of fall detection in assisted living: Identifying and improving the optimal machine learning
method,” Journal of Sensor and Actuator Networks, vol. 10, no. 3, Jun. 2021, doi: 10.3390/jsan10030039.
[15] R. Xie and D. Wu, “Optimal transport-based transfer learning for smart manufacturing: Tool wear prediction using out-of-domain
data,” Manufacturing Letters, vol. 29, pp. 104–107, Aug. 2021, doi: 10.1016/j.mfglet.2021.08.004.
[16] S. H. Park and Y. H. Park, “Audio-visual tensor fusion network for piano player posture classification,” Applied Sciences, vol. 10,
no. 19, pp. 1–15, Sep. 2020, doi: 10.3390/app10196857.
[17] C. Bhatt, I. Kumar, V. Vijayakumar, K. U. Singh, and A. Kumar, “The state of the art of deep learning models in medical science
and their challenges,” Multimedia Systems, vol. 27, no. 4, pp. 599–613, 2021, doi: 10.1007/s00530-020-00694-1.
[18] C. Lawson and D. C. Montgomery, “Logistic regression analysis of customer satisfaction data,” Quality and Reliability
Engineering International, vol. 22, no. 8, pp. 971–984, Dec. 2006, doi: 10.1002/qre.775.
[19] P. K. Jain, V. Saravanan, and R. Pamula, “A hybrid CNN-LSTM: A deep learning approach for consumer sentiment analysis
using qualitative user-generated contents,” ACM Transactions on Asian and Low-Resource Language Information Processing,
vol. 20, no. 5, pp. 1–15, Sep. 2021, doi: 10.1145/3457206.
[20] H. Gou, L. Su, G. Zhang, H. Li, and Z. Li, “Research on telecom customer satisfaction prediction strategy,” in 2023 8th
International Conference on Cloud Computing and Big Data Analytics, ICCCBDA 2023, IEEE, Apr. 2023, pp. 593–598, doi:
10.1109/ICCCBDA56900.2023.10154738.
[21] S. Ray, “A quick review of machine learning algorithms,” in 2019 International Conference on Machine Learning, Big Data,
Cloud and Parallel Computing (COMITCon), IEEE, Feb. 2019, pp. 35–39, doi: 10.1109/COMITCon.2019.8862451.
[22] A. -All-Tanvir, I. A. Khandokar, A. K. M. M. Islam, S. Islam, and S. Shatabda, “A gradient boosting classifier for purchase
intention prediction of online shoppers,” Heliyon, vol. 9, no. 4, Apr. 2023, doi: 10.1016/j.heliyon.2023.e15163.
[23] C. Zhu, M. Wang, and C. Su, “Prediction of consumer repurchase behavior based on LSTM neural network model,” International
Journal of System Assurance Engineering and Management, vol. 13, no. S3, pp. 1042–1053, 2022, doi: 10.1007/s13198-021-
01270-0.
[24] A. Mawaddah, M. H. Dar, and G. J. Yanris, “Analysis of the SVM method to determine the level of online shopping satisfaction
in the community,” SinkrOn, vol. 8, no. 2, pp. 838–855, Apr. 2023, doi: 10.33395/sinkron.v8i2.12261.
[25] M. Rahman and V. Kumar, “Machine learning based customer churn prediction in banking,” in Proceedings of the 4th
International Conference on Electronics, Communication and Aerospace Technology, ICECA 2020, IEEE, Nov. 2020,
pp. 1196–1201, doi: 10.1109/ICECA49313.2020.9297529.
[26] A. Nurdina and A. B. I. Puspita, “Naive Bayes and KNN for airline passenger satisfaction classification: comparative analysis,”
Journal of Information System Exploration and Research, vol. 1, no. 2, Jul. 2023, doi: 10.52465/joiser.v1i2.167.
[27] O. F. Seymen, E. Ölmez, O. Doğan, O. Er, and K. Hiziroğlu, “Customer churn prediction using ordinary artificial neural network
and convolutional neural network algorithms: a comparative performance assessment,” Gazi University Journal of Science,
vol. 36, no. 2, pp. 720–733, Jun. 2023, doi: 10.35378/gujs.992738.
[28] S. Pouyanfar et al., “A survey on deep learning: algorithms, techniques, and applications,” ACM Computing Surveys, vol. 51,
no. 5, pp. 1–36, Sep. 2018, doi: 10.1145/3234150.
[29] E. M. Elgohary, M. Galal, A. Mosa, and G. A. Elshabrawy, “Smart evaluation for deep learning model: churn prediction as a
product case study,” Bulletin of Electrical Engineering and Informatics, vol. 12, no. 2, pp. 1219–1225, Apr. 2023, doi:
10.11591/eei.v12i2.4180.
[30] A. Boukhalfa, A. Abdellaoui, N. Hmina, and H. Chaoui, “LSTM deep learning method for network intrusion detection system,”
International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 3, pp. 3315-3322, Jun. 2020, doi:
10.11591/ijece.v10i3.pp3315-3322.
[31] M. A. Alshamari, “Evaluating user satisfaction using deep-learning-based sentiment analysis for social media data in Saudi
Arabia’s telecommunication sector,” Computers, vol. 12, no. 9, Aug. 2023, doi: 10.3390/computers12090170.
[32] S. Beltozar-Clemente, O. Iparraguirre-Villanueva, F. Pucuhuayla-Revatta, J. Zapata-Paulini, and M. Cabanillas-Carbonell,
“Predicting customer abandonment in recurrent neural networks using short-term memory,” Journal of Open Innovation:
Technology, Market, and Complexity, vol. 10, no. 1, Mar. 2024, doi: 10.1016/j.joitmc.2024.100237.
[33] R. Mutegeki and D. S. Han, “A CNN-LSTM approach to human activity recognition,” in 2020 International Conference on Artificial
Intelligence in Information and Communication, ICAIIC 2020, Feb. 2020, pp. 362–366, doi: 10.1109/ICAIIC48513.2020.9065078.
[34] E. Ismanto, H. Ab. Ghani, and N. I. B. Md Saleh, “A comparative study of machine learning algorithms for virtual learning
environment performance prediction,” IAES International Journal of Artificial Intelligence (IJ-AI), vol. 12, no. 4, pp. 1677-1686,
Dec. 2023, doi: 10.11591/ijai.v12.i4.pp1677-1686.
BIOGRAPHIES OF AUTHORS