Association for Information Systems

AIS Electronic Library (AISeL)

Pacific Asia Conference on Information Systems
PACIS 2017 Proceedings
(PACIS)

Summer 7-19-2017

A Proposed Analytical Customer Satisfaction

Prediction Model for Mobile Internet Networks
Ayisat W. Yusuf-Asaju
Universiti Utara Malaysia, [email protected]

Zulkhairi Dahalin
Universiti Utara Malaysia, [email protected]

Azman Ta’a
Universiti Utara Malaysia, [email protected]

Follow this and additional works at: http://aisel.aisnet.org/pacis2017

Recommended Citation
Yusuf-Asaju, Ayisat W.; Dahalin, Zulkhairi; and Ta’a, Azman, "A Proposed Analytical Customer Satisfaction Prediction Model for
Mobile Internet Networks" (2017). PACIS 2017 Proceedings. 102.
http://aisel.aisnet.org/pacis2017/102

This material is brought to you by the Pacific Asia Conference on Information Systems (PACIS) at AIS Electronic Library (AISeL). It has been
accepted for inclusion in PACIS 2017 Proceedings by an authorized administrator of AIS Electronic Library (AISeL). For more information, please
contact [email protected].
 A Proposed Analytical Customer
Satisfaction Prediction Model for Mobile
Internet Networks
Research-in-Progress

Ayisat W. Yusuf-Asaju Zulkhairi B. Dahalin

School of Computing School of Computing
Universiti Utara Malaysia Universiti Utara Malaysia
Keddah, Malaysia Keddah, Malaysia
[email protected] [email protected]

Azman Ta’a
School of Computing
Universiti Utara Malaysia
Keddah, Malaysia
[email protected]

Abstract
Subjective method (such as survey, interview, etc.) has been the most common and reliable method
used in analyzing customer satisfaction. However, the subjective method is expensive, time
consuming, lacks repeatability in real-time and may not capture the technical aspect of the telecoms
network service performance in telecommunication industry. As a result, perceived quality of
experience (QoE) has been traditionally used to evaluate the satisfaction of telecommunication
services from both Internet service providers and customer’s perspective. However, the result of
perceived QoE in relation to mean opinion score found not suitable enough to quantify customer
satisfaction, and it eliminates the diversity of customer assessment while quantifying satisfaction.
Therefore, this paper proposed an analytical customer satisfaction prediction model based on
perceived QoE, perceived QoE influence factors, perceived QoE measurements and perceived QoE
estimations to overcome the limitations of the subjective measurement. The paper presents how the
mean opinion score can be used to quantify customer satisfaction by ensuring the diversity of
customer’s assessment is not eliminated.
Keywords: Quality of Experience (QoE), Mean Opinion Score, Internet, Mobile Network Operators
(MNOs), QoE measurement, analytics, prediction, Mobile Networks

Introduction
Telecommunications (Telecoms) industry considered customer satisfaction as a pivotal indicator used
to determine the extent at which the Mobile Network Operators (MNOs) are successful in providing
mobile internet services to their customers. In most cases, MNOs mainly focused on the monitoring of
technical constructs consisting of terminals, network, and service infrastructure to aid the provision of
internet services. However, the competitive nature of the telecoms market made MNOs to realize the
need not to only consider the technical aspect of the quality provided, but also customer expectations
and experiences with the aim of satisfying the customers.
The technical aspect of the network quality provided by the MNOs constitutes quality of service (QoS)
parameters of the network services. Among the QoS parameters are throughput, loss, delay,
bandwidth and jitter, these parameters are usually measured on network nodes of the MNOs
instruments or machines (Andrews et al. 2006). However, customers are more interested in the
experiences perceived from the service performance typically in the form of subjective and non
technical terms. Overall customer experiences reffered to as quality of experience (QoE), often used to
estimate the customer perception of the network level performance. Several studies assumed QoE to
be a measure of customer perception in relation to the QoS parameters, context, expectations and
other factors, which influence the customer perception to determine the degree of satisfied or
dissatisfied customers using a specific service or application (Alreshoodi and Woods 2013; Le Callet et
al. 2012). On the other hand, customer expectation is the ideal standards commonly in the form of
service level agreement (SLA), which is an agreement between the customers and the MNOs about the
service characteristics provided by the MNOs (Gozdecki et al. 2003).
Generally, expectation and other QoE influence factors are used to estimate perceived QoE, which is
represented using mean opinion score (MOS) (Alreshoodi and Woods 2013; Demirbilek and Gregoire
2016). MOS is determined through the prediction of the subjective measurement from the objective
measurement. However, recent studies argues that MOS is not sufficient enough to quantify the
satisfaction of customers, because MOS eliminates the diversity of customer assessment while
quantifying customer satisfaction (Hoßfeld et al. 2016). Therefore, this article proposed an analytical
customer satisfaction (ACSAT) predition model, which consists of QoE Influence factors, perceived
QoE and perceived QoE maximization to determine the diversity of satisfied and dissatisfied
customers of internet networks. The remainder of this article discusses perceived QoE, perceived QoE
influence factors, perceived QoE measurements, ACSAT prediction model and conclusions.
Perceived QoE
Qualinet described QoE “as the degree of delight or annoyance of the user of an application or
service. It results from the fulfilment of his or her expectations with respect to the utility and / or
enjoyment of the application or service in the light of the user’s personality and current state” (Le
Callet et al. 2012). Delight of the user of a service can be influenced by context, network, device
application and context of use depending on the application domain. Generally, MNOs need to
understand traffic characteristics, most especially the geographical and dynamic nature of the network
traffic commonly found in the mobile network. Therefore, to appropriately determine the user
perceived QoE, it is important to understand various influence factors of perceived QoE from both
users and network perspectives.
Perceived QoE influence Factors
Influence factors is any characteristics consisting of user, system, service application context, whose
actual state or settings may have influence on the user perceived QoE (Le Callet et al. 2012). Perceived
QoE influence factors can be classified into diiferent dimensions as depicted in table 1. However, the
most common dimension is human, context and system influence factors (Barakovic and Skorin-
Kapov 2013; Le Callet et al. 2012). Human factors describes the demographic, socio-economic
backgroud, physical and users emotional state. System influence factors consititutes the technical
properties of application or service used by the user. Context influence factor involves the user’s
environment in relation to physical, temporal, social economic and techincal charateristics of the
service or application used by the user. Basic understanding of all the influence factors would enable a
better analysis of the perceived QoE of the users in relation to the specific service used by users.
Table 1. Perceived QoE Influence Factors
Author Dimensions Elements
Baraković et al. (2010) Technology performance Application/service, server, network and
device.
Usability Behavioural usability, ease of use, device
features, emotions and feelings.
Expectations Application type, usage history, gender,
brand and personality
context Environment, personal/ social context,
technological context and cultural context
subjective evaluation Service, network and device
DeMoor et al. (2010) QoS parameters Delay, jitter, loss, throughput and
bandwidth.
Context, Prior experiences,
Expectations
User Factors Personalisation and emotions
Stankiewicz and Jajszczyk (2011) QoS factors, Grade of Service Terminals, type of content, application
(GoS), Quality of Resilience specific features,
(QoR)
Customer profiles, Emotions,occupation, education level
environmental, psychological and age
and sociological aspects.
Pricing policies Prepaid or Postpaid.
Skorin-Kapov and Varela (2012) Application Application configuration-related factors.
Resource space Delay, jiter, loss, throughput and system-
related factors)
Context customer location, time, and application-
related factors
User space Demographics, customer preferences,
requirements, expectations, prior
knowledge, behaviour and motivations
Le Callet et al. (2012) Human factors Age, education background, emotions,
gender and user visual aid
System factors Bandwidth, delay, loss, throughput,
security, display size and resolution
Context factor Location, movement, time of day, costs,
subscription type and privacy

Perceived QoE Measurement

Perceived QoE measurement is classified into subjective and objective measurements (ITU-T
Recommendation G.1030 2014). The subjective measurements is based on customer perception of the
services delivered to the customers, while objective measurement is the means of estimating
subjective quality solely from the measurement obtained from the network traffic (Barakovic and
Skorin-Kapov 2013). The subjective method is performed using Mean Opinion Score (MOS). The
MOS is an opinion score on five-point category-judgement scales (scores such as Excellent = 5; Good
= 4; Fair = 3; Poor = 2; Bad = 1), mostly used in many applications such as audio, video, and web
browsing to estimate the perceived QoE. (Demirbilek and Gregoire 2016).

In contrast to the subjective measure that focuses on customer perception through surveys or
experiments, the objective measurement is associated with quality estimation models usually in the
form of mathematical and/ or comparative methods that generate the quantitative measure of the
perceived QoE (Alreshoodi and Woods 2013). Several studies have used objective measurement to
estimate MOS (perceived QoE) using machine learning algorithms to interprete customer satisfaction
(Alreshoodi and Woods 2013; Anchuen et al. 2016). However, recent studies have argued that, it is
necessary to go beyond MOS calculations to allow for better understanding of the customer
population experiencing a satisfactory level of perceived QoE, because the MOS eliminates the
diversity of customer assessment (Hoßfeld et al. 2016).

In addition, Streijl et al (2016) highlighted that, MOS does not clarified what threshold values should
be appointed to identify the problems or acceptability of perceived QoE. Because the MOS measures
in most studies measures the amount of perceived QoE satisfaction rather than using the
maximization of perceived QoE to determine the satisfactory level of the service in relation to satisfied
and dissatisfied users or customers. The authors point out the MNOs might decide to maximize the
QoE for different reasons. Thus, maximizing the overall perceived QoE for multiple customers in the
network to allocate network resources, maximizing the QoE of a certain individual customers or
groups to increase the number of satisfied customers (Barakovic and Skorin-Kapov 2013; Streijl et al.
2016). In this respect, concentrating on the overall MOS alone can lead to unfairness among
customers if the MOS is not properly used (Streijl et al. 2016). Therefore, maximization of estimated
MOS can be achieved using standard deviation of MOS (SOS hypothesis), distribution and quantiles to
determine the satisfied and dissatisfied customers to guarantee certain level of customer fairness.

Proposed Analytical Customer Satisfaction Prediction Model

In the telecoms industry, customer satisfaction is a yardstick to determine the extent at which the
MNOs are successful in providing mobile internet services to their customers. Ideally, if the MNOs
could measure customer satisfaction at any point in time and identify the cause of poor QoE, it would
be easier for the MNOs to address quality issues promptly before such issues deteriorate into large
amount of mobile internet customer dissatisfaction (Diaz-Aviles, et al., 2015). Anderson and Sullivan
(1993) developed customer satisfaction model based on assumption of expectancy disconfirmation
theory. The customer satisfaction model was improved by Xiao and Boutaba (2007) to an analytical
customer satisfaction (CSAT) model used in examining the satisfaction of the mobile internet
customers through simulation method. The CSAT model consist of service utility (represents
measureable set of services performance such as network QoS, network availability and customer
care), expectation and disconfirmation constructs (Xiao and Boutaba 2007). The service utility was
used to analysed the objective perceived utility of the customers, while the results of the perceived
utility and expectation was used to evaluate disconfirmation in order to determine customer
satisfaction. The CSAT model has been confirmed very useful in examining customer satisfaction of
the next-generation networks (Djogatovic et al. 2014; Ibarrola et al. 2014). These studies found that
overall customer satisfaction is closely linked to other contextual parameters like customers previous
experiences. In particular, Djogatovic et al. (2014) modified and improved the CSAT model with
service content and security conditions which are among the perceived QoE system influence factors.
Based on all the stated evidences, this study proposed an Analytical Customer Satisfaction (ACSAT)
prediction model by improving and modifying the CSAT model developed by Xiao and Boutaba
(2007) using perceived QoE influence factors, perceived QoE measurements and estimations.
In contrast to the CSAT model, the ACSAT prediction model consist of perceived QoE influence
factors that are in relation to system factors (such as the QoS parameters), human factors (like
customer demographics) and context factors (such as customer’s location). The three QoE influence
factors (system, customer and context) was considered in the ACSAT prediction model to enable
adequate estimation of the perceived QoE in relation to mobility in relation to time and location (Le
Callet et al. 2012). Hence, instead of estimating perceived utilities as used in CSAT model, ACSAT
prediction model tends to estimate the overall customer perceived QoE.

In CSAT model, expectation was used to analyze perceived utility. On the contrary, ACSAT prediction
model used expectation in two different forms, to estimate the perceived QoE and for expectation
update. Expectation as stated in the SLA along with the QoE influence factors would aid the analysis
of perceived QoE to identify maximum and minimum expected variable values along with variable
weights to determine adequate MOS through the machine learning algorithms (Tsiaras and Stiller
2014). Because, estimation of overall perceived QoE would inform the MNOs about issues causing
deterioration in the network performance, this will assist the MNOs to update customer’s expectation
appropriately.

Customer care support for service utilities in the CSAT model was used along with pricing policies and
bandwidth to aid the maximization of perceived to enable the diversity of satisfied and dissatisfied
customers (Barakovic and Skorin-Kapov 2013). Customer care support most especially the response
time has been confirmed in previous to studies to have a great impact on overall customer experience
(Diaz-Aviles, et al., 2015). As a result, customer care support in this study is for the purpose of
analyzing the reported cases and challenges experienced by the customer while using the mobile
internet services and the response time taken by the customer care support to rectify such issues. In
addition the use of pricing policies focused on different types of customer’s mobile internet
subscriptions that can be used for the maximization of perceived QoE to determine the diversity of
satisfied and dissatisfied customers.
Similar to CSAT model is the use of customer satisfaction result for market segmentation, market
growth and revenue and profit generation. However, customer satisfaction result can also assist the
MNOs in allocating appropriate network resources in location where high dissatisfaction is
experienced as depicted in figure 1.

Service Utilities
System Perceived Pricing Policies,
QoE Customer care Support
Perceived QoE and Bandwidth
Human
Maximization
Customer Satisfaction
Context
QoE Expectation Market
Influence Customer
Behaviour Dynamic
Factors
Expectation Market
Market
Update Growth
Segmentation

Allocation
of
Revenue Network
and Profit Resources

Customer
Behaviour

Figure 1. Analytical Customer Satisfaction Prediction Model

Methodological Instances for Implementation of ACSAT Prediction Model

To implement ACSAT prediction model, this article proposed the use of network data, network billing
data, customer care data and expectation data. The network data comprises of user throughput and
payload metrics for different users. Network billing data constitutes data bundles and price of the data
bundles subscribed by different internet users. Customer care data comprises of the different
customer complaints reported to the customer care agents. Expectation data consists of SLA that will
be used as a benchmark for the modelling of perceived QoE at the modelling phase.
The experimental design for this article comprises of four different phases (data collection, data
preparation, data modelling and maximization of perceived QoE to determine customer satisfaction).
The first phase entails the collection of data from the network traffic. Data gathered from the network
traffic would aid the extraction of the perceived QoE influence factors (such as system, human and
context factors), which can be used for modelling estimated MOS (Perceived QoE).

The second phase is data preparation, which involves different methods such as data exploratory
analysis (EDA), data pre-processing, clustering techniques, feature selection and extraction.
Generally, EDA aids understanding of the interdependencies among the data attributes to become
familiar with the content of the data along with its quality and limitations. Data pre-processing
involves data cleaning, data integration and data transformation. Data cleaning is concerned with the
process of filling in the missing values, smoothing noisy data, identifying, or removing outlier as well
as, resolving inconsistencies and imbalances in the data. Data integration is the process of combining
datasets residing in different sources and providing a unified view of the datasets. Data
transformation involves transforming the raw data obtained from the data sources into the form that
would be appropriate for the analytical modelling stage. Feature selection and extraction process
enables the selection and extraction of the perceived QoE influence factors from the dataset obtained
from the network traffic. These processes allow better understanding of the underlying process that
generate the data to be used for the estimation of perceived QoE, improve the prediction performance
of the predictors, provide a fast and cost-effective predictor to be used for the estimation of perceived
QoE. Clustering techniques allows the grouping of a set of objects together in such a way the objects in
the same group are more similar to each other than those in the other groups. Specifically, use of
expectation maximization clustering algorithm (EMCA) that is suitable for continous and categorical
variables can aid in grouping the selected and extracted variables into different clusters in relation to
Pricing policies, Bandwidth and customer care response. This phase would aid the preparation of the
dataset for the perceived QoE modelling phase.

The third phase involves the modelling of the perceived QoE, which is an abstract representation of
data and its relationship within the dataset. This phase allows the splitting of dataset into training and
testing set usually in the ratio of 70:30. This stage can be carried out by using machine learning
algorithms such as decision trees, random forest, support vector machine, K-nearest, and artificial
neural network (in case of large data set) or data mining algorithms. The modelling of perceived QoE
using machine learning algorithms would map the combination of input parameters to a class value to
build an efficient model that classify extracted features with the maximum precision through the
perceived QoE function described as QoE≔f (User, Service, Variable) (Tsiaras and Stiller 2014). After,
the estimation of the perceived QoE, the next phase is the maximazation of perceived QoE to
determine satisfied and dissatisfied customers.

The last phase which is the maximization of perceived QoE to analyze customer satisfaction is
concerned with using the identified clusters in the second phase to determine the diversity of satisfied
and dissatisfied customers. This can be executed firstly, by predicting expected MOS (mean of the
random variable which represents the quality ratings) and SOS as a function of MOS for each cluster.
Secondly, by determining quartiles for both discrete and continuous with either probability mass
function or probability density function respectively through the q-quantiles for the ratings,
estimated as (Hoßfeld et al. 2016). Thirdly, analysing
acceptability , which can be determined in relation to the probability of the MOS above a certain
threshold , where U is the random variable for quality ratings (MOS). This would enable
the estimation of acceptance in relation to the quality ratings defined as Where 0 and 1 is
accepted and not accepted; satisfied or dissatisfied respectively and is analysed through percentage
good or bad (%GoB) and percentage poor or worse (%PoW). The estimation for the %GoB is defined
as while %PoW is defined as (Hoßfeld et al. 2016). This phase will enable
the estimation of satisfied and dissatisfied customers.

Conclusion
This study proposed an analytical customer satisfaction prediction model, to predict the customer
satisfaction based on the the previous customers experience to assist the MNOs understand the trends
of the network traffic, and make intelligent decisions that would enable them to improve their network
performance by allocating approriate network resources to enhance their service provisioning. This
study contributes to the literature by demonstrating the conceptual view of the ACSAT prediction
model by incorporating the perceived QoE influence factors, measurement, and estimations in
customer analysis process. The proposed model would assist the MNOs to predict the customer
satisfaction ahead before the customer would perceive the rendered services Further research of this
study is to employ the use of big data obtained from the telecoms network traffic for the
implementaion of the ACSAT prediction model. In addition, the study would design a prototype to
enable experimental study through focus group method.
References
Alreshoodi, M., & Woods, J. 2013. “Survey on QOE\QOS corelation models for multimedia services,”
International Journal of Distributed and Parallel Systems (IJDPS) 4:3, pp. 53-72.
Anchuen, P., Uthansakul, P., and Uthansakul, M. 2016. “QOE Model in Cellular Networks Based on
QOS Measurements Using Neural Network Approach,” in proceeding of the 13th
International Conference on Electrical Engineering/Electronics, Computer,
Telecommunications and Information Technology (ECTI-CON), Chiang Mai, pp. 1-5.
Anderson, E. W., and Sullivan, M. W. 1993. “The Antecedents and Consequences of Customer
Satisfaction for Firms,” Marketing Science 12:2 , pp. 125-143.
Andrews, M., Cao, J., and McGowan, J. 2006. “Measuring Human Satisfaction in Data Networks,” in
Proceedings of the 25th IEEE International conference on Computer Communications
Barcelona, Spain, pp. 1-12.
Barakovic, S., and Skorin-Kapov, L. 2013. “Survey and challeges of QoE management issues in
wireless networks,” Journal of conputer networks and communciations, pp. 1-28.
Baraković, S., Baraković, J., and Bajrić, H. 2010. “QoE Dimensions and QoE Measurement of NGN
Services,” in 18th Telecommunications forum TELFOR, Serbia, Belgrade, pp. 15-18.
Demirbilek, E., and Gregoire, J. C. (2016). “Towards Reduced Reference Parametric Models for
Estimating Audiovisual Quality in Multimedia Services,” in proceeding of IEEE International
Conference on Communications (ICC), Kuala Lumpur, pp. 1-5.
DeMoor, K., Ketyko, I., Joseph, W., Deryckere, T., De Marez, L., Martens, L., & Verleye, G. 2010.
“Proposed framework for evaluating quality of experience in a mobile testbed- oriented living
lab setting,” Mobile Networks and Applications, 15:3, pp. 378-391.
Diaz-Aviles, E., Pinelli, F., Lynch, K., Nabi, Z., Gkoufas, Y., Bouillet, E., Calabrese, F., Coughlan, E.,
Holland, P., and Salzwedel, J., 2015. “Towards real-time customer experience prediction for
telecommunication operators,” in proceeding of IEEE International Conference on Big Data
(Big Data), pp. 1063–1072.
Djogatovic, V. R., Kostic-Ljubisavljevic, A., Stojanovic, M., and Mikavica, B. 2014. “Quality of
experience in telecommunication,” in proceeding of the 8th International Quality
Conference, Kragujevac, pp. 899-904.
Gozdecki, J., Jajszczyk, A., and Stankiewicz, R. 2003. “Quality of service terminology in IP networks,”
IEEE Communication Magazine, 41:3, pp. 153-159.
Hoßfeld, T., Heegaard, P. E., Varela, M., and Moller, S. 2016. “QoE beyond the MOS: an in-depth look
at QoE via better metrics and their relation to MOS,” Quality User Experience 1:2 , pp. 1-23.
Ibarrola, E., Saiz, E., Zabala, L., and Cristobo, L. 2014. “A new global qulaity of service model:
QoXphere,” IEEE Communications magazine, 52:1, pp. 193-199.
ITU-T Recommendation G.1030, 2014. “Estimating end-to-end performance in IP networks for data
applications,” in International Telecommunication Union Recommendations, Switzerland
Geneva, pp. 1-22.
Le Callet, P., Möller, S., and Perkis, A. 2012. “Definitions of Quality of Experience (QoE)” in Qualinet
White Output version of the Dagstuhl seminar 12181, Dagstuhl, pp. 1-22.
Skorin-Kapov, L., and Varela, M. 2012. “A multi-dimensional view of QoE: the ARCU model,” in
Proceedings of the 35th International Convention MIPRO, Opatija, pp. 662-666.
Stankiewicz, R., Cholda, P., & Jajszczyk, A. (2011). QoX: What is it Really? IEEE Communications
Magazine , 148-158.
Streijl, R. C., Winkler, S., and Hands, D. S. 2016. Mean opinion score (MOS) revisited: methods and
applications, limitations and alternatives. Multimedia System, 22:2, pp. 213-227.
Tsiaras, C., and Stiller, B. 2014. “A deterministic QoE formalization of user satisfaction demand
(DQX),” in Proceedings of Annual IEEE conference on local computer networks, Canada,
pp.227-235.
Xiao, J., and Boutaba, R. 2007. “Assessing network service profitability:Modeling from market science
perspective,” IEEE/ACM Transactions on networking, 15:6, pp. 1307-1320.