2016 23rd International Conference on Telecommunications (ICT)

MeFoRE: QoE based Resource Estimation at Fog to

Enhance QoS in IoT
Mohammad Aazam, Marc St-Hilaire, Chung-Horng Lung, Ioannis Lambadaris
Department of Systems and Computer Engineering
Carleton University, Ottawa, Canada
[email protected], [email protected], [email protected], [email protected]

Abstract— Internet of Things (IoT) is now transitioning from cloud. Specially, with mobile devices and other IoT nodes
theory to practice. This means that a lot of data will be generated which do not have a reliable connectivity behavior, the cost
and the management of this data is going to be a big challenge. To considerably increases when it comes to resource allocation and
transform IoT into reality and build upon realistic and more a lot of resources go underutilized because of unexpected and
useful services, better resource management is required at the
unreliable behavior of the Cloud Service Customer (CSC).
perception layer. In this regard, Fog computing plays a very vital
role. With the advent of Vehicular Ad hoc Networks (VANET) Furthermore, mission critical and latency sensitive IoT services
and remote healthcare and monitoring, quick response time and require very quick response and processing. In that case, it is not
latency minimization are required. However, the receiving nodes feasible to communicate through the distant cloud, over the
have a very fluctuating behavior in resource consumption Internet. Fog computing plays a very vital role in this regard [3].
especially if they are mobile. Fog, a localized cloud placed close to
the underlying IoTs, provides the means to cater such issues by The concept of Fog computing is to bring networking
analyzing the behavior of the nodes and estimating resources resources near the perception layer. Fog resources lie between
accordingly. Similarly, Service Level Agreement (SLA)
the perception layer and the cloud layer. Fog computing is an
management and meeting the Quality of Service (QoS)
requirements also become issues. In this paper, we devise a
extension of the traditional cloud computing paradigm to the
methodology, referred to as MEdia FOg Resource Estimation edge of the network, helping to create more refined and context-
(MeFoRE), to provide resource estimation on the basis of service aware services [4]. Fog provides low latency and high quality
give-up ratio, also called Relinquish Rate (RR), and enhance QoS streaming to mobile nodes including moving vehicles through
on the basis of previous Quality of Experience (QoE) and Net proxies and access points located accordingly, like along
Promoter Score (NPS) records. The algorithms are implemented highways and tracks. Likewise, resource and power constrained
using CloudSim and applied on real IoT traces on the basis of individual nodes, Wireless Sensor Networks (WSNs) and
Amazon EC2 resource pricing. Virtual Sensor Networks (VSNs) would be able to take
advantage from the presence of Fogs. Since Fog is localized, it
Index Terms—IoT; Cloud of Things; Fog computing; Edge
computing; Amazon EC2; resource management. suits the services related to video streaming, emergency and
disaster management, gaming, healthcare, augmented reality,
I. INTRODUCTION graph/data mining [5] etc.
IoT-based services are becoming popular rapidly. The
In our previous work [6], we proposed a basic mathematical
number of connected devices has reached 9 billion and by 2020,
model for resource estimation in Fog. In the current paper, we
they are expected to grow further up to 24 billion [1]. With such
provide a methodology for historical record-based resource
rate of increase in the number of heterogeneous devices being
estimation to mitigate resource underutilization as well as
part of IoT and generating data, it is not possible anymore for
enhance service quality for multimedia IoTs. We named it
an IoT to efficiently manage the data, power, and bandwidth [2].
MEdia FOg Resource Estimation (MeFoRE). The contributions
Therefore, a lot of service oriented tasks would be performed in
of this work are two folds. First, we extend the model to propose
the cloud, creating amalgamation of IoT and cloud computing
resource management of IoTs at Fog on the basis of historical
[3]. In this case, a localized micro datacenter would be present
records of CSC, which can help in efficient, effective, and fair
close to the underlying nodes for the purpose of offloading the
management of resources. Second, to enhance QoS, we work on
tasks and preprocessing the raw data. That micro datacenter is
previous QoE records, specifically on the basis of Net Promoter
known as Fog or Edge. Fog also helps in minimizing delays and
Score (NPS) to allocate resources accordingly in such a way that
increase service quality by incorporating better responsiveness,
the required QoS is achieved and reliable service is provided.
making it inevitable for multimedia streaming and other delay
NPS is a QoE feedback given by the CSC on a scale of 0-10.
sensitive services. There comes a situation when the cloud is
Where CSCs with 0-6 NPS are known as distractors, 7-8 are
connected with IoT that generates multimedia data. Video on
passive, and 9-10 are promoters. Therefore, the default case
Demand (VoD), Visual Sensor Network or CCTV connected to
would neither be positive (promoters) or negative (distractors).
cloud are examples of such a scenario. Since multimedia content
It would be the mean of passive, which is 7.5. The model
consumes more processing power, storage space, and
presented in this paper has been validated with partial
scheduling resources, it becomes important to manage them
implementation on a real IoT Crawdad trace [7]. In addition to
effectively to perform efficient resource management in the

978-1-5090-1990-8/16/$31.00 ©2016 IEEE

 2016 23rd International Conference on Telecommunications (ICT)

that, the model is implemented based on the Amazon EC21 according to the previous service relinquish records as well as
pricing to get realistic outcome. Netflix, Dropbox, NASDAQ, the QoE (NPS in this model). Resources are increased whenever
iCloud, Bitcasa, etc. all use Amazon’s service for cloud storage. the SLA was not met in the previous case so that better service
is provided and CSC's loyalty is gained. Resource estimation is
II. RELATED WORK done in the following way:
Fog computing is still a very new concept, due to which,
there is not a lot of literature available on it. Most of the works ∑ + (Ʋ ∗ ( − ) ∗ (Ѻ )), =0
still focus mainly on cloud resource management. The scenario
ℜ= ∑ + (Ʋ ∗ ( − ) ∗ (1 − x̄ Ѻ )), =0
of Fog computing or Cloud of Things (CoT) has not been
deliberated by most of the past works. Below are a few ∑ ∑ ( + (Ʋ ∗ ѡ) ∗ (1 − x̄ Ѻ ))
examples of relevant papers on IoT resource management. (1)
Abu-Elkheir et al. [8] elaborate on the organization of data
in IoT. The authors indicate how distinctive design parameters ѡ = (1 − x̄ ( ( | )) − (2)
would work for management of the data. But how that data and
IoT nodes are going to be handled at the cloud layer and how ℜ∊{ , , , , ℎ}
resources are to be managed for the generated data are not part
of this study. Cubo et al. [9] present their work on the integration ∑ , =0
of heterogeneous devices accessible via cloud. The presented ⎧
work, however, does not deal with the key issue of management ⎪
⎪ ∑ , x̄ >
of resources for such devices in the cloud. Ning and Wang
= (3)
discuss in [10] the potentials of IoTs and the volume of data it ⎨∑ ∑ , x̄ ≥ x̄
is going to produce. The authors also underscore efficient ⎪
management of resources for the future Internet, in which ⎪∑ ∑ , x̄ ≥ x̄
heterogeneous IoTs would be a vital part. Chatterjee and Misra ⎩
[11] provide a mapping of sensors to their respective targets
through a sensor-cloud infrastructure. But how every node or = − (4)
sensor is allocated with resources in a dynamic fashion is not
part of this study. Sammarco and Iera [12] analyze Constrained x̄ (∑ ( | )) > 0,
Application Protocol (CoAP) for IoTs and discuss service ( | ) = (5)
0.3 =0
management and method for exploiting resources of IoT nodes.
Tei and Gurgen [13] emphasize on the significance of the
integration of cloud-IoT. They discuss preliminary outcomes of = ∗∑ ( − x̄ ) (6)
a project in this domain. In [14], Rakpong et al. consider
resource allocation in mobile cloud computing environment. Where ℜ represents the required resources, is the
They deliberate on communication/radio resources and NPS ratio generated on the basis of the overall NPS ( ) and
computing resources, but their work only focuses on decision currently requesting CSC’s recorded NPS ( ). Ʋ is the
making for coalition of resources to increase service provider’s basic price of the requested service. is the default RR, which
revenue. Distefano et al. [15] contribute in presenting an outline is applied on a new CSC when previous RR does not exist. Its
for the integration of the underlying IoT nodes with the cloud. value is 0.3, which is the average of low relinquish probability
However, the challenge of dynamic and node-based resource (0.1 to 0.5, from the complete range of 0-1). ѡ is the loyalty
management is not a focus of this study. Bonomi et al. [4] ratio which comes from the Service oriented Relinquish rate
present a basic architecture for Fog computing, which does not (SR) of a particular customer of giving up the same resource
include its practical implications and resource management for that is being currently requested, subtracted by variance. x̄
IoTs. Similarly, Stolfo et al. [16] present data protection through ( ( | ) ), which is the average SR, has its value between 0-
Fog computing, but do not discuss resource management and 1. has four cases as presented in (3). In case 1, when the
associated matters. CSC is new and no previous NPS record exists, then default
NPS ( ) is applied, which is 7.5. However, there is a
III. MEFORE MODEL
possibility that the currently required service may have a better
The underlying IoT nodes, whether they are part of a overall NPS ( ) than the default value (7.5). In that case,
VANET, an emergency or rescue related ad hoc network, online more priority is given to the overall NPS since it is opined by
gaming, or VoD, CSCs contact the relevant Fog to get the more users. This is where case 2 is applied. In case 3, when
required service. Fog performs SLA negotiation with the CSCs. , which is the NPS given by CSC for number of
Fog is responsible to provide the best possible service to the
occasions, is available and greater than the overall NPS, then it
CSCs but also has to ensure that resources are not underutilized.
has a higher priority because it would eventually lead to
In this regard, Fog looks for the reliability in the resource
increase the overall NPS of the service and attains CSC’s
consumption by the CSCs. Hence, Fog estimates resources

1 http://aws.amazon.com/ec2
 2016 23rd International Conference on Telecommunications (ICT)

loyalty. Case 4 is applied for all other scenarios. All types of We have considered different parameters to estimate the
NPS’ are calculated through (4), in which represents required resources for different types of users. TABLE I shows
promotors and represents distractors. the setting for the basic parameters.
TABLE I: KEY PARAMETERS’ SETTING FOR EVALUATION
0 < ≤ 0.5 , 0.5 < ≤ 1 (7) Parameters Range
x̄ (x̄ (∑ ( | ) )), ( | ) ) > 0, Default SR ( ) 0.3
Ѻ = (8)
0.3 =0 Default OR (Ѻ ) 0.3
Default NPS ( ) 7.5
( | ) represents RR for a particular service for Relinquish rate (P) 0~1
number of recorded occasions. For a service being Service price (Ʋ ) USD 85.3 ~ USD 1000
requested for the first time with no previous records, the Variance range 0 ~ 0.16
default value of 0.3 is applied. is the variance of the SR. Minimum VRV 3.89
CSCs, especially mobile users, can have a fluctuating behavior A. Resource Estimation for New CSCs
in utilizing resources, which may lead to deception while
making judgement about resource allocation. That is why, in When different CSCs with diverse behaviors make a request
our model, we have taken into account variance of RR, which for a service, the Fog has to decide the amount of resources that
helps determining the actual behavior of each customer. has to be allocated for that service, based on the previous records
of service utilization, QoS, and QoE. CSCs with more loyal
behavior get more resources. Similarly, CSCs who experienced
Ѻ represents the Overall Relinquish rate (OR), not specific
bad QoS previously would be allocated with more resources, so
to any particular service. Here, it should be noted that ( | )
that loyalty of such CSCs is gained and more reliable and
determines the RR of the particular service that the customer is
profitable business process is maintained. For new customers
currently requesting, while Ѻ is the RR which includes all the
(i.e. the Fog has no past record for them), the default RR value
activities/services a particular customer has been
(0.3) is applied. This is done by anticipating the new CSC to be
doing/requesting. The last activity of the user in this regard tells
somewhat reliable. Considering the fact that datacenter
about his most recent RR. That is why, it is given more
resources are precious, therefore not all the possible resources
importance and the average is taken again. In case of a new
are assigned to an unknown new CSC. Instead, the default rate
customer, when there is no historical data for that user, the
is applied, which is the average of low RR, as explained earlier
default value is set to a low RR of 0.3.
with the model. Fig. 1 presents the unit of estimated resources,
IV. SETUP, IMPLICATIONS, AND OUTCOME termed as Virtual Resource Value (VRV) in our model. The
resources are estimated according to different services. The
In this section, we present the implementation results of our
actual mapping of VRV is done by the Cloud Service Provider
service model, along with the discussion on each result. We
(CSP) keeping in view the type of service and the amount of
defined our service model through algorithms to evaluate the
resources required accordingly. The first instance of CSC
effectiveness in CoT business. Our main objective is to observe
requesting for Amazon EC2 service priced at USD 85.3 gets
the influence of the performance factors on the system and test
VRV 8 without considering QoE or NPS. Since the CSCs are
the feasibility of our method on the basis of actual Crawdad
new here and they have never provided any NPS before for this
trace [7] which was partially applicable for RR through the link
service, the overall NPS ( ) is applied (given in TABLE II)
strength and quality parameters such as the Received Signal
with the default value (7.5). This way, additional resources are
Strength Indicator (RSSI) and the Link Quality Indicator (LQI).
estimated as the value of VRVNPS is 10. Same is the case with
RSSI is a measure of Radio Frequency (RF) power of the
the other instances. This is how the proposed algorithm works
channel, coming from WiFi, Bluetooth, or other IEEE 802.15.4
on the basis of previous RR, NPS, and QoS and eventually
transmitters. LQI determines the quality of the link. It is a
service quality is enhanced and SLA violation is mitigated.
cumulative value used in multi-hop networks. Especially for
Personal Area Networks (PANs) and WSNs, such measure is
deemed important where the user is mobile and link quality
fluctuates. We have used the QoS measures from the trace in
our model as well, other than the user generated feedback,
which was generated through Random Number Generator
(RNG) algorithm. Degradation in quality becomes a reason for
service relinquishing. Crawdad’s trace is a packet delivery
performance over an 802.15.4 link under different stack
parameter configurations for more than 6 months. “The data set
consists of measurements of the data delivery performance of a
WSN link with nearly 200 million packets in the data set” [7].

Figure 1. Resource estimation for new CSCs, for different Amazon EC2
services.
 2016 23rd International Conference on Telecommunications (ICT)

TABLE II: OVERALL NPS FOR NEW CSC CASE. previous estimation of 23.03 VRV. Similarly, considering CSC
Service 1 2 3 4 5 6 7 6, where NPS is 1 (~worst), the estimated resources are
NPSo 4 7 4 9 5 6 3
increased with a higher ratio (10.24 to 17.24). Here, it should
also be noted that CSCs' previous RR also has an effect on the
TABLE III shows an illustrative scenario of how mapping resource estimation. More reliable customers get more
can be performed by the CSP, according to its resource pool and resources.
the type of service being provided. For a VoD service, S1, VRV
8 is mapped to corresponding Resource Pool Level (RPL).
TABLE III: ILLUSTRATIVE SCENARIO OF VRV MAPPING.
VRV RPL Service Resource Pool
CPU: 10%, GPU: 12%, Memory: 15%,
8 L1 S1: VoD Storage: 0%, Bandwidth: 200Kbps
Guarantee: 70%
S2: CPU: 8%, GPU: 6%, Memory: 8%
8 L1 HealthMo Storage: 12%, Bandwidth: 100Kbps
nitoring Guarantee: 80%
CPU: 20%, GPU: 24%, Memory: 30%
16 L2 S1: VoD Storage: 0%, Bandwidth: 400Kbps
Guarantee: 85%

Then, according to the type of service being provided, the

mapping is performed to the actual resource pool. Among the
available resources for service 1, CSP allocates 10% of CPU, Figure 2. Res. estim. for existing CSC with variable NPSc and fixed NPSo.
12% of GPU, 15% of memory, and data rate of 200Kbps.
Storage is not required for this service, therefore, it is 0%. The
guarantee of allocation of these resources is 70%, which means C. Resource Estimation when >
that at least 70% of the resources are guaranteed. Resources are Continuing the previous case with the other scenario when
increased or decreased accordingly as per the requirements of NPSc is higher than NPSo, resource estimation is different. Case
the service. This is only an example. This mapping would vary 4 of (2) is applied here. NPSo is fixed at 6 in this case.
according to the type of service and available resource pool at
the CSP.
B. Resource Estimation for an Existing CSCs, Requiring
Service (Amazon EC2 $85.3, 1X800 SSD)
In the case when CSC is returning and its previous record
exists but requiring service S for the first time, then the Fog can
make use of the available overall record. This way, a more
customized resource allocation can be made possible. But there
is a possibility that the NPS given by this CSC for the current
service is lower than the overall NPS, which is given by the rest
of the customers. In this case, since the majority of users have
a better experience with this service, more priority is given to
that so that biasness is avoided. Case 3 of (3) is applied here.
We have also used automatic NPS in this analysis, which we Figure 3. Res. estim. for existing CSC when NPSc is higher than NPSo.
get from our Crawdad trace (RSSI and LQI) based on software
agent’s feedback, residing in the recipient’s device. Otherwise, In Fig. 3, CSC 1 has OR=0.1 and NPSc=7. This means that
the CSCs will always give low NPS to get more resources next better resource provisioning is required this time to enhance the
time. In this scenario, NPSo was fixed at 7. Here the result is quality and meet the SLA. VRVNPS is 24.2. Since the CSC has
presented for the Amazon EC2 1X800 SSD storage service, given better NPS as compared to the rest of the customers
priced at USD 85.3 per 100 hours. The unit is greater for loyal (NPSo) which was 6, therefore, it is important to keep the loyalty
(L) customers, while it is smaller for disloyal (H) customers, and provide better service this time. This will positively affect
because of their behavior. Since there are more chances of an the NPSo and eventually more customers are attracted towards
H customer to relinquish the service, hence, more priority and the service because of being popular and better in quality.
quality is provided to the more loyal customer, having low RR. Comparing CSC 1 with CSC 3, both having the same NPSc, but
Fig. 2 shows the estimation of resources on the basis of OR, still different VRVNPS, because of having different RR. More
keeping SR to default 0.3. In case of CSC 1, OR is 0.1, which reliable customers get more resources. This way, the resource
means the customer is very loyal. In this case, the previous allocation is done keeping in view NPSo, NPSc, SR, and OR.
overall NPS is fixed at 7, which means that more resources
would be required this time to enhance the QoS. Hence,
VRVNPS is 24.03, which is increased as compared to the
 2016 23rd International Conference on Telecommunications (ICT)

D. Resource Estimation with Variable OR Variance delay sensitive services, QoS has also been included in this
As mentioned earlier, the service RR is very fluctuating in work. We have made use of automatic and user provided QoE
the case of IoT devices and mobile nodes. Due to this, the in the form of NPS to enhance QoS and incorporate more
variance in OR is also included in the user characteristics while reliable service delivery. The presented method helps determine
determining resources. Similarly, the NPSr, which comes from resources according to the behavior and historical record of the
NPSo and NPSc, also has its impact on the amount of estimated customer as well as the required service quality, minimizing
resources. NPSr of 1 means that NPSc and NPSo are the same. resource underutilization and enhancing QoS. The
The higher the value of NPSr above 1, the bigger the difference implementation is based on real IoT traces and Amazon EC2
between the experiences of customers. Specifically, this would service.
be more important when NPSc is lower than NPSo, since it REFERENCES
means that the current CSC had a bad experience of the service [1] Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M., Internet of Things
before and gaining its loyalty is important now to enhance the (IoT): A vision, architectural elements, and future directions. Future
overall impact of the service. This section presents the effect of Generation Computer Systems, 29(7), 1645-1660, 2013.
the variability in OR variance (TABLE IV) as well as NPSr. In [2] Shaukat, U., Ahmed, E., Anwar, Z., & Xia, F. Cloudlet deployment in
local wireless networks: Motivation, architectures, applications, and open
this part, in Fig. 4, comparing CSC 1 with 3 (where CSC 3 has challenges. Journal of Network and Computer Applications, 62, 18-40,
an unreliable OR of 0.7), we can clearly see that CSC 3 gets 2016.
significantly less resources. Similarly, CSC 1 and 2 do not have [3] Aazam, M., & Huh, E. N. Fog computing and smart gateway based
a big difference in their respective reliability behaviors and communication for cloud of things. In Future Internet of Things and
therefore get slightly different amount of resources on the basis Cloud (FiCloud), 2014 International Conference on, 464-470, IEEE,
August 2014.
of their previous QoE. Thus, NPSr is having its effect here.
[4] Bonomi, F., Milito, R., Zhu, J., & Addepalli, S. Fog computing and its
TABLE IV: VARIANCE IN OR. role in the internet of things. In Proceedings of the first edition of the
CSC 1 2 3 4 5 MCC workshop on Mobile cloud computing, 13-16, ACM, August 2012.
[5] Nawaz, W., Khan, K. U., Lee, Y. K., & Lee, S., Intra graph clustering
Variance 0.16 0.013 0.0033 0.04 0.053
using collaborative similarity measure. Distributed and Parallel
Databases, 33(4), 583-603, 2015.
[6] Aazam, M., & Huh, E. N., Dynamic resource provisioning through Fog
micro datacenter. In Pervasive Computing and Communication
Workshops (PerCom Workshops), 2015 IEEE International Conference
on, pp. 105-110. IEEE, March 2015.
[7] Songwei F., Yan Z., CRAWDAD dataset due/packet̻delivery (v.
2015̻04̻01), traceset: packet̻metadata, downloaded from
http://crawdad.org/due/packet̻delivery/20150401/packet̻metadata,
doi:10.15783/C7NP4Z, April 2015.
[8] Abu-Elkheir, M., Hayajneh, M., & Ali, N. A. Data management for the
internet of things: Design primitives and solution. Sensors, 13(11),
15582-15612, 2013.
[9] Cubo, J., Nieto, A., & Pimentel, E., A cloud-based Internet of Things
platform for ambient assisted living. Sensors, 14(8), 14070-14105, 2014.
[10] Ning, H., & Wang, Z., Future internet of things architecture: like mankind
neural system or social organization framework? Communications
Figure 4. Effect of variance on overall resource estimation. Letters, IEEE, 15(4), 461-463, 2011.
[11] Chatterjee, S., & Misra, S., Target tracking using sensor-cloud: Sensor-
V. CONCLUSION AND FUTURE WORK target mapping in presence of overlapping coverage. Communications
Rapidly increasing IoT-based services have triggered the Letters, IEEE, 18(8), 1435-1438, 2014.
need for more sophisticated ways to handle heterogeneous [12] Sammarco, C., & Iera, A., Improving service management in the internet
devices, fluctuating connectivity, and data generating behaviors. of things. Sensors, 12(9), 11888-11909, 2014.
Energy and resource constrained IoT nodes require their [13] Tei, K., & Gurgen, L., Clout: Cloud of things for empowering the citizen
clout in smart cities. In Internet of Things (WF-IoT), 2014 IEEE World
computation tasks to be offloaded. Furthermore, healthcare, Forum on, 369-370, IEEE, March 2014.
emergency, and multimedia services require quick response [14] Kaewpuang, R., Niyato, D., Wang, P., & Hossain, E., A framework for
with minimum latency. With IoT-Cloud communications, it cooperative resource management in mobile cloud computing. Selected
becomes difficult to achieve that, having cloud reachable Areas in Communications, IEEE Journal on, 31(12), 2685-2700, 2013.
through a shared, unreliable core network. Fog computing [15] Distefano, S., Merlino, G., & Puliafito, A., Enabling the cloud of things.
provides the solution by bringing cloud resources to the edge of In Innovative Mobile and Internet Services in Ubiquitous Computing
(IMIS), 2012 Sixth International Conference on, 858-863, IEEE, July
the underlying IoTs and other end nodes. However, with 2012.
heterogeneous devices being part of IoTs, it is difficult to predict [16] Stolfo, S. J., Salem, M. B., & Keromytis, A. D., Fog computing:
how much resources will be consumed and whether the Mitigating insider data theft attacks in the cloud. In Security and Privacy
requesting nodes, devices, or sensors are going to fully utilize Workshops (SPW), 2012 IEEE Symposium on, 125-128, IEEE, May
the resources they have requested. Due to this uncertainty, the 2012.
probability of resource utilization is incorporated while
performing resource estimation. Moreover, since the focus is on