Ieee Coy
Ieee Coy
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 1
Abstract—The prevalent smartphones have become the major entrance to accessing services on the Internet. On smartphones, users
can have two options as the clients, i.e., native apps and Web apps. There have been several debates about native apps and Web
apps. However, major service providers such as Google, Amazon, and Facebook provide both native apps and Web apps to
end-users. Essentially, the performance differences between these two types of apps haven’t been addressed. Indeed, the
performance differences make non-trivial impacts on apps development, deployment, and distribution. In this article, we conduct a
measurement study on the performance of native apps and Web apps on Android smartphones. Specifically, we want to explore given
the same functionalities, do Web apps always perform poorly compared to native apps. We select 328 services from some popular
providers, covering various domains such as e-commerce, map, social networking, and entertainment. With HTTP-level trace analysis,
we demystify the workflows on how native apps and Web apps deliver services on mobile devices, respectively. Then we characterize
the performance differences between native apps and Web apps with the metrics including the number of requests, response time,
data drain, and energy consumption. We find that the performance of Web apps is better than native apps in more than 31% cases.
Our derived knowledge can suggest some recommendations to improve the performance for mobile apps.
1 I NTRODUCTION
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 2
more poorly than the native apps? The answer to this number of HTTP(S) requests, response time, traffic
question is non-trivial and meaningful in two folds. First, volume, and energy consumption. In about 31% of
if the performance of Web apps is close to that of native the situations, Web apps perform much better than
apps, service providers can reduce the maintenance cost of native apps. We then analyze the possible reasons
multiple versions, and end-users can get rid of the updates. that can lead to such differences.
Second, even if the performance of Web apps is poor com- • We draw some implications from our empirical study
pared to that of native apps, demystifying the underlying and propose recommendations on how to improve
technical causes can provide insightful knowledge for the the performance of mobile apps from various aspects
service providers and Web app developers, who can then such as network protocols, cache, and API semantics.
find potential solutions to optimize the Web apps. In particular, we attempt to make Web apps provide
In this article, we break down the central problem by comparable performance levels and user experiences
studying three uncovered problems: (1) how native apps to native apps.
and Web apps access Web services on mobile devices;
The remainder of this article is organized as follows.
(2) to what extent the performance of native apps and
Section 2 introduces some background knowledge. Section
Web apps varies; (3) what kind of knowledge developers
3 describes the measurement methodology and how we
can leverage to develop and optimize the apps. To this
conduct our experiments. Section 4 anatomizes the archi-
end, we consider four quantitative metrics to characterize
tecture of native apps and Web apps on using Web services,
the performance: the number of HTTP(S) requests, traffic
respectively. Section 5 analyzes the performance differences
volume, response time, and energy. We select 328 distinct
of native apps and Web apps, given the same functionality.
services from 12 popular service providers, i.e., Facebook,
Section 6 summarizes our findings according to the architec-
Amazon Shopping, Google Map, Baidu Map, Sina Weibo, Di-
tural differences. Section 7 provides implications to service
anping, Gewala Movie, Jd, Maoyan Movie, Meituan, Zhihu, and
providers and developers. Section 8 discusses the threats to
Douban, which develop both native apps and Web apps with
validity of our approach. Section 9 presents related work,
the same or similar functionalities for mobile devices. We
and Section 10 concludes this article with future outlook.
choose Android as the target platform of native apps as
Android is the most popular mobile OS nowadays. In the
rest of this article, the native apps refer to Android apps if 2 BACKGROUND
not specified. By using two Android smartphones with the
Traditionally, there are two types of apps to deliver Web
same device model under the same network condition, we
services on Smartphones.
conduct our measurement study and find that in about 31%
•Native apps. A native app is developed for a specific OS
of the situations, Web apps perform much better than the
platform, i.e., iOS, Android, or BlackBerry. Native apps are
native apps, when providing the same functionality. Based
usually developed in an imperative programming language.
on the analysis of various aspects such as network protocol,
For example, Android apps are developed in Java, and iOS
third-party libraries, and cache efficiency, we suggest some
apps are developed in Objective-C or Swift. Native apps can
possible recommendations that may help to make the per-
take advantage of the particular features of mobile devices,
formance of Web apps comparable to that of native apps.
such as various sensors. However, most parts of native
This article is the extension of our previous ICWS 2015 apps cannot be executed across different platforms, whereby
paper [7] that made the first study of performance differ- developers have to maintain various versions for different
ences of Web services delivered through native apps and platforms. End-users usually need to download native apps
Web apps. Compared to the conference version, this article from app stores, such as Apple AppStore, Google Play, and so
significantly extends the scale of Web services (from 52 to on. In addition, end-users have to manually keep updating
328) and adds a new metric of energy consumption. Besides of native apps.
obtaining the almost consistent findings of the conference •Web apps. A Web app is developed for running within
version, our results further demonstrate some imperfect Web browsers or browser-based runtime such as WebKit
designs that can lead to unsatisfactory performance and and Gecko. It is developed by Web technologies, such as
user experiences on smartphones. We also provide some HTML5, CSS3, and JavaScript. End-users enter the URL
useful knowledge and suggestions to service providers, in the address bar or click through the hyperlinks, and
Web/native app developers, and end-users. The main con- then navigate to the latest version of Web apps. Web apps
tributions made in this article can be summarized as follows: have advantages in terms of cross-platform compatibility,
• We establish an experimental platform2 to collect the or the so-called “write once, run anywhere” feature. With
network-level traces of requesting Web services on the advances of HTML5, Web apps are claimed to provide
smartphones, and propose a trace-based approach to comparable user experiences to native apps. However, due
characterizing the usage of Web services of native to the limitation of Web technologies, standard Web apps
apps and Web apps, respectively. Based on the trace cannot take full advantage of the devices’ hardware capabil-
analysis, we demystify how Web services are used ities, and their performance is often complained to be poorer
from an architectural style perspective. than native apps in practice. Sometimes, users can perceive
• We design a series of experiments to measure the quite obvious latency when loading a Web app.
performance of native apps and Web apps, given There is another type of apps, called Hybrid apps, that
by the same functionality. Our metrics include the could be regarded as an intermediate type between native
apps and Web apps. Similar to the native apps, the hybrid
apps also need to be downloaded from app stores. Although
2. We have released the whole dataset along with the work pub-
lished. Please contact the corresponding author, Xuanzhe Liu, via the hybrid apps wear the “shell” of native apps, the core
[email protected], for more details logic of hybrid apps heavily relies on Web technologies. As
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 3
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 4
Fig. 1. Trace Logs of the Native App and the Web App of Amazon Shopping Generated by the Same Feature
and the end time of the last response “/images/...” within Chrome Browser of version 40.0.2214.89 on the other
•M ETRIC 4 (E NERGY C ONSUMPTION) is the total amount smartphone. We manually use the native and Web app to
of energy consumed from the time of the first issued network access each selected feature. For each feature, we repeat
request to the time of the last received response in a trace. We three times to avoid the biased measurements. In order to
employ a physical Monsoon Power Monitor4 to record make the comparison fair enough, we concurrently access
the power consumption of the connected smartphones and the same feature of both types of apps with two people
compute the energy consumption when requesting each using the two smartphones simultaneously.
service. By applying the V-Edge energy model proposed
in our previous work [18], we can accurately compute the
energy consumption as follows: 3.4 Tracing Web Services Usage
Z end time
energy consumption = V-Edge(x) (1) We deploy a proxy server which runs both the WireShark
start time and the popular HTTP/HTTPS-level proxy called Fiddler5
to intercept all network traffic between the smartphones and
where the start time means the start time of a trace, end time
the remote Web services. In this way, we can explore the
means the end time of a trace, and x means the time between
network details more accurately by investigating the trace
start time and end time used in the V-Edge power model.
logs from both smartphones and the proxy server. Then, we
Due to page limit, the details of V-Edge can be referred from
compare the network traces of native apps and Web apps,
previous paper [18].
given the same feature. For example, we compare the traces
of searching_a_person feature offered by Facebook, by
3.3 Experiment Setup using the same keyword query on the Web app and the
Next, we introduce the infrastructure of our experimental native app.
platforms and how we conduct our experiments. As the cache mechanism is widely used in both native
• Hardware. We use two smartphones of the same device apps and Web apps [20], we also request each feature of the
model. The selected device model is Samsung Galaxy S4 targeted Web services in two modes, Without Cache and With
with 2 GB RAM and Android 4.2 OS (directly downloaded Cache.
from Google rather than using the manufacturer customized
ROM). The two smartphones are connected to the same • Without Cache. To disable the cache of native apps,
Wi-Fi network, and are “rooted” with the system-level we manually uninstall a native app, and remove all
privilege. We uninstall all other pre-installed apps, stop files in its folder. Then we re-install and revisit the
background activities such as OS auto-update and auto-sync app for the same feature. To disable the cache of
services. In this way, we can prevent other apps from con- Web apps, we manually clear the browser cache of
necting the network and significantly reduce the possible a Web app and reload the app. At each time we
noise. To inspect the details of network activities, we use record the network traces and energy information,
the popular Wireshark tool [19] to inspect the traffic to/from and calculate the number of HTTP(S) requests, traffic
our smartphones. Although the WireShark tool can possibly volume, response time, and energy consumption.
impact the results of energy measurement, previous studies • With Cache. We test the same feature of the two
show the impact is too marginal and could be neglected [18]. types of apps, with the default cache mechanisms
• Software. For each service provider, we install and access turned on. We record the same information and
the native app on one smartphone, and open the Web app calculate the metrics as well.
4. https://www.msoon.com/LabEquipment/PowerMonitor/ 5. http://www.telerik.com/fiddler
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 5
3.5 Research Questions Statement 4.2 Architecture of Native Apps on Using Web Services
Based on the dataset and hardware/software configura- Figure 2 shows the architecture on how a native app uses
tions, our empirical study aims to measure how the native Web services. Once a native app is installed on the de-
apps and Web apps perform, given the same functionality. vice, the native code, including the data parsing, network
More specifically, we aim to answer the following research management, and layout rendering, are maintained locally
questions: on the smartphone. In addition, some resources such as
icons and images can be configured by the developers and
• RQ1: How do native apps and Web apps access Web locally stored. When a user launches the native app and
services on smartphones, respectively? clicks some buttons to issue interaction requests, the pro-
• RQ2: How do native apps and Web apps perform differ- cessing logics of native code are executed. The native code
ently when providing the same functionality? may trigger requests to the target Web services, and send
• RQ3: What kind of knowledge can developers leverage to synchronous/asynchronous requests to fetch data. When
optimize the apps? requesting Web services, the native apps come up with lots
of preloaded elements and fetch only the “required” data
from the Web server rather than the contents of the entire
application. The native app can create a new thread to issue
4 ARCHITECTURE ANATOMY an asynchronous request when a resource is required. When
We first focus on RQ1, i.e., how do native apps and Web apps the data is downloaded, the native code can parse the data,
access Web services, respectively? Answering such a question process the data with the preloaded logic, and re-render the
can help us have the basic insight of how the two types of new user interface.
apps request a Web service. Based on such knowledge, we
can then make in-depth performance analysis. 4.3 Architecture of Web Apps on Using Web Services
In contrast, the Web apps are accessed in the smartphone’s
Web browser such as Chrome, FireFox, and Safari, and
4.1 Web Service Identification update themselves without user intervention. Web apps
are built by standard technologies such as HTML5, CSS,
For better illustration, we take a popular service provider,
JavaScript, and so on. Figure 3 shows the architectural
the Amazon Shopping, as our illustrating example. Figure 1
workflow on how a Web app requests a Web service. The
shows some traces of Amazon Shopping on both the na-
Web app has to download all resources such as page files,
tive app (at left side) and the Web app (at right side).
images, JavaScript files, and CSS files to render the user
When users fill in the information and click the button,
interface in the mobile browser. When a user first loads
some consequent requests and respective responses will
the Web app, the browser initiates a request to fetch the
be triggered. We list traces of three features, i.e., the
structural information of the main page. Then the browser
search_keywords feature, the add_goods_to_cart fea-
interprets the HTML, JavaScript, and CSS files to build up
ture, and the delete_goods_from_cart feature.
user interfaces. There are two forms of user interactions over
All of the 328 Web services in our dataset, are provided
the Web apps. The first one is to input some text fields or
in the RESTful fashion. Such an observation evidences that
submit some forms, then the Web apps jump from one page
SOAP-based Web services are not widely adopted in the
to another page. The other one is that the Web app generates
context of mobile Internet. It can be found that the contents
an AJAX (Asynchronous JavaScript and XML) request, and
of service responses are formatted with JSON in both native
deals with the callback methods without page jumping. As
apps and Web apps. Although these two types of apps
the Web apps fetch the latest resources of a Web service, the
have similar user interfaces, the requested URLs of the same
user is always displayed by the latest version of the app.
feature are entirely different. For example, when we add a
good to the cart, the native app sends a request to the URL
of “/aw/detail/ajax/add-to-cart/”, while the Web app sends a 4.4 Architectural Differences
request to the URL of “/aw/c/”. By exploring all of the Based on the above analysis, we can have an observation
328 Web services in our dataset, it is quite interesting that that native apps and Web apps have quite different work-
the target URLs of Web services requested by native apps flows and behaviors of requesting Web services. Besides the
and Web apps are always different for the same feature. different implementation languages and different RESTful
It implies that Web service providers maintain different API semantics, two differences need to be mentioned.
interfaces for native apps and Web apps. In addition, it is The first difference is the cache management of re-
interesting to find that native apps and Web apps may use sources. Compared to the Web apps that always present the
different IP addresses to access the same services. Therefore, up-to-date version, the native apps are updated manually
we can infer that the server-side service implementations of when the users explicitly launch the Update Manager that
the same feature are separately deployed for native apps is responsible for checking updates and downloading the
and Web apps, although the two types of apps still access latest version. Rather than relying on the browsers that
the same back-end database. manage the resources to be fetched from the remote server,
Based on the traces, we then try to figure out the developers of native apps can programmatically configure
workflow of how native apps and Web apps request a the need-to-update resources. As a result, the “stable”
Web service, respectively. We examine the response of each resources such as icons and images are permanently stored
request and infer its function in the workflow from the in the local storage, and the update manager fetches only
header fields and some keywords in the content such as the “delta” resources. Such a mechanism is quite effective
update, version, etc. to better reuse the already-downloaded resources, and thus
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 6
Fig. 2. The Architecture of Native App Fig. 3. The Architecture of Mobile Web App
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 7
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 8
5.3 GET
fetches the latest collected shops list after deleting a shop. In the semantics of RESTful Web services, GET refers to
The native app of Amazon Shopping transfers data by HTTPS, the request that retrieves a representation of a resource
while the Web app transfers data by HTTP. We also find from the Web server. We examine seven typical features
that the native app of Amazon Shopping adds extra cookie that need GET operations, i.e., the search_keywords
fields in the request header, such as device info, application feature in Weibo, the shanghai_page feature in Dian-
version, etc. For the feature of delete_cart in Jd, the Web Ping, the goods_detail feature in Amazon Shopping, the
service returns a list of recommended items to the native movie_detail feature in Gewala movie, the box_office
app. In contrast, for the unfollow_shop feature in Jd and feature in Maoyan movie, the search feature in Meituan,
the cancel_collection feature in Gewala movie, the Web and the friendlist feature in Zhihu. Figure 12, Figure 13,
app includes more cookie fields in the request header than Figure 14, and Figure 15 show the number of HTTP(S)
the native app to keep states with the server. requests, the traffic volume, the response time, and the
energy consumption in GET operations, respectively.
Response Time. Usually, the DELETE operation can be
Number of Requests. In the GET operations, the Web
executed very fast in both types of apps. An exception is
apps usually initiate more HTTP(S) requests than the native
the native app of Amazon Shopping that transfers data by
apps for the same feature. It is because the Web app needs
HTTPS protocol, and spends non-negligible time on HTTPS
to download all the needed resources, including JavaScript
handshaking.
files, CSS files, and other resources. However, for the
Energy Consumption. Although it is observed that the search_keywords feature in Weibo, the movie_detail
two features in Amazon Shopping consume more traffic vol- feature in Gewala movie, etc., the native apps initiate more
ume and have more response time in its native app, the requests than the Web apps. For example, we find that the
native app consumes less energy. It indicates that we cannot native app receives the friends’ information as well as the
determine the energy consumption of an application by only avatars according to the key words in Weibo, and fetches
the data traffic and response time. pictures of higher quality to provide better user experience
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 9
Fig. 12. HTTP(S) Requests in GET Fig. 14. Response Time in GET
Fig. 13. Traffic Volume in GET Fig. 15. Energy Consumption in GET
in Gewala movie. Such a difference is also caused by the to find that the performance of some Web apps may be
different semantics of targeted Web services. As described better than the native ones. The Web app of Maoyan movie
in Figure 12, the Web app of Zhihu fetches all the friends’ takes advantage of AppCache [23], an advanced feature
avatars, but the native app fetches only a part of friends’ provided by HTML5 allowing the developers to configure
avatars. In addition, apps with cache can save up to 80% stable resources rather than relying on only the browsers. In
requests. With cache enabled, the number of HTTP(S) re- a sense, the resources are configured at finer granularity so
quests in both native apps and Web apps can be significantly a lot of duplicate transfers are avoided.
reduced. Response Time. Compared to the other three operations,
Traffic Volume. The native app of Weibo consumes much the response time of GET operations should be more com-
more network traffic than the Web app because it requests plex, as the smartphone needs to fetch a number of resources
to download extra users’ avatars. We have mentioned that from the server and thus requires more network bandwidth.
the Web app of Zhihu fetches all friends’ avatars. However, The Web apps rely on the browser, where only limited num-
we are surprised to find that the Web app consumes less ber of simultaneous connections are available. For example,
network traffic than the native app without cache. We in FireFox and Chrome browser, only 6 concurrent connec-
explore the raw traffic data, and find that the native app tions are allowed per domain [15]. In contrast, the native
fetches avatars of high-quality portraits. In contrast, the Web apps can fully use the TCP connection pool with the sup-
app fetches avatars of low-quality portraits. Generally, the ports of the underlying OS (e.g., by AndroidHttpClient
cache mechanism in GET operations is quite efficient. For API) or other third-party libraries such as Volley [22].
most features, both native apps and Web apps with cache The performance of “with-cache” is always better than that
enabled can consume less network traffic. The native app of “without cache”. In addition, sometimes the performance
of Google Map can save as much as 90.9% traffic volume of Web apps with cache can be quite close to that of
when the cache is enabled. However, the native app of native apps when the cache is enabled. In particular, the
Baidu Map disables cache for most resources, so the traffic Web app of Maoyan movie (with cache) reduces a lot of
volume reduces quite marginally. When we consider the response time, and performs better than the native app for
network traffic in the with-cache situation, it is surprising the box_office feature (almost 85% improvement).
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 10
Energy Consumption. On the whole, native apps con- 6.2 Impact of Network Performance
sume less energy than Web apps for most features, but the Generally, the native apps expectedly perform better than
differences become smaller when cache is enabled. With the Web apps, but proportional exceptions cannot be ig-
cache enabled. Both native apps and Web apps send less nored. Indeed, the performance variety can significantly
requests, and reduce the cost of parsing resources, resulting derive from the programming languages (i.e., Java V.S.
in less energy consumption. For example, some pictures HTML/JavaScript), but it is out of the topic of this arti-
are loaded into the memory rather than decoded just-in- cle. Instead, we focus on the factors that may impact the
time. In the meantime, there are some features where native network performance, i.e., the connection management, the
apps consume more energy than Web apps. For example, libraries, and the protocols.
the native app of DianPing frequently records the state of
smartphones and uploads data to the remote servers. Such 6.2.1 Impact of Connection Management
a process would consume more energy. Another example First, we focus on the network connections management of
is the goods_detail feature in Amazon Shopping. The the smartphone, as it determines how many resources can
native app employs HTTPS rather than HTTP as the com- be used by the apps, and could be a main factor affecting
munication protocol, which introduces extra handshaking performance. We employ the techniques proposed in [16]
phase and data decryption, thus leading to more energy to analyze the TCP connections from the WireShark logs.
consumption [20]. As mentioned above, the Web apps can use at most
6 concurrent TCP connections per domain, while the
6 E XPLORING THE F INDINGS native apps can fully explore the TCP connections to
The preceding results evidence that Web apps have a con- reach better performance. For example, by using the
siderable proportion of performance advantages than native AndroidHttpClient API and other system-level libraries
apps. We need to find out why or why not the Web apps’ such as android.net.http, org.apache.http, and
performance can reach or even exceed native apps. Consid- java.net.Socket, or using third-party libraries such
ering the architectural comparison, we organize the findings as Volley [22] (20 concurrent connections at maximum)
and explore the possible reasons from the two aspects of the and android-async-http library [24] (10 concurrent
smartphone, cache performance and network performance. connections at maiximum), developers can configure the
“Connection:Keep-Alive” in request header or set
6.1 Impact of Cache Performance “setReuseAddress(true)” to allocate more sockets. In
It is clear that cache mechanism can improve the perfor- practice, we observe that the number of concurrent TCP
mance of both native apps and Web apps, for the GET connections of native apps can be quite flexible (varying
operations. from 10 to 20), but always significantly more than the
The native apps usually transfer less data than Web apps, browsers.
as the number and size of downloaded resources can be However, although the TCP connections cannot be ex-
programmatically controlled by developers. The native app panded due to the browser constraints, it is argued that the
developers can configure which resources should be cached reuse of connections can be a promising option [14]. For
locally by themselves, and define when the resources need some features provided by Facebook and Google Map, the
to be updated. A lot of resources such as icons, pictures, WireShark traces indicate that the Web apps can reuse exist-
and layout configurations, are usually stored at the client ing TCP connections to reduce the latency of fetching new
side. We observe that the cache mechanisms are disabled resource objects, rather than initializing a new connection. In
for many resources in Baidu, as the app intends to show the this way, the performance such as response time and energy
latest data. could be improved. However, it requires the server-side
The cache mechanism of Web apps completely re- protocol support to enable the reuse of TCP connections,
lies on the browser. However, according to our previous e.g., both Google and Facebook enable SPDY [25].
work [6], the browser cache mechanism is far from satis-
fying for mobile Web apps, due to conservative expiration 6.2.2 Impact of Transfer Protocols
time, redundant transfer, and same content with differ- From our observation, the choice of protocol can be quite
ent URLs that are dynamically generated by JavaScript. diverse among service providers in their apps. Three
We examine the cache configurations such as Cache- providers, i.e., Facebook, Google Map, and Douban, use HTTPS
Control field, Expires field, Etag field of the Web apps. in both native apps and Web apps. The providers such as
Most of the Web apps configure the HTML page with Weibo, Dianping use the HTTP in both native apps and Web
“Cache-Control:no-cache(store)”. Such configura- apps. In contrast, the Amazon and Zhihu use HTTPS in their
tion may be too coarse-grained, as all resources in the main native apps, but choose HTTP in the Web apps. Hence, we
HTML body have to be re-fetched even though they are still focus on these two service providers.
valid on the client side. When the cache policies are care- We find that the Web apps can beat the native apps
fully configured at finer granularity, e.g., for different texts for these two service providers in almost 36.6% genres.
and images, the performance of Web apps can surprisingly More specifically, the Web apps can beat native apps in
reach or even exceed that of native apps. For example, the almost all PUT, POST, and DELETE operations. It implies
search_a_person feature in Facebook, the Good_Detail that the HTTPS can introduce overhead in some cases. When
feature in Amazon Shopping, and the View_checkboard exploring the traces, we find that the size of HTTP messages
feature in DianPing configure different expiration time for is usually not very large, i.e., less than 50 KB. From the trace
the icons, small images, and main HTML body, respectively. of Fiddler and WireShark, such overhead mainly comes
In this way, only the need-to-update resources are down- from the SSL handshake phase. The overhead ranging from
loaded when the Web apps are revisited. 4.32 KB to 11 KB (mean: 8.11 KB, median: 7.4 KB) depends
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 11
on several factors including the certificate size. In addition, the browser can reuse TCP connections of previously
HTTPS cost extra energy for message encryption and de- requested resources to avoid the initialization cost for a
cryption. In this way, we can infer that, for small-size new request resource, and the performance seems to be
message, the HTTPS can introduce more network payload, improved. Recently emerging protocols such as SPDY [25]
especially the handshake-phase overhead. and HTTP/2 [30] support the reuse of existing connections.
In addition, we are aware that a lot of popular Web service
providers and browsers (Chrome, FireFox, and Safari) begin
7 I MPLICATIONS AND S UGGESTIONS to employ these new protocols.
Based on the findings and analysis, we then move to RQ3,
i.e., What implications can be leveraged to make Web apps • Avoiding unnecessary complexities. We observe that
and native apps provide comparable performance? We make sometimes the design and implementation of both kinds
analysis to understand the bottlenecks and suggest some of apps are not perfect enough so that both could be
recommendations to improve the performance of mobile improved. For some POST, PUT, and DELETE opera-
apps on smartphones, e.g., making the Web apps reach tions, the native apps may perform more poorly than
comparable performance with native apps. Web apps, e.g., the collect_goods feature in Ama-
zon, the delete_timeline feature in Weibo, and the
•Supporting fine-granularity cache management. delete_cart feature in Jd. The reason is that Web ser-
Undoubtedly, the size of resources fetched from the vices providers enforce their APIs to return extra data for
Web servers significantly impacts the performance of client- refreshing the user interfaces. However, such response data
side apps. The service providers and the app developers may not be always necessary. Web service providers should
need to determine more carefully which resource objects give the Web developers alternatives to programmatically
should be fetched when the app is revisited. The native filter the response data. Besides, in the Web app of Jd, the
apps provide developers very flexible options to configure add_goods_to_cart feature receives the same data in the
the cached resources, while the Web apps heavily rely “cart” and “cartJson” fields. Such unnecessary redundancy
on the browser. However, the cache management of Web should be carefully avoided.
apps is not satisfactory when requesting Web services [26], •Unifying RESTful API semantics. In some cases, it
due to the too coarse-grained configurations of the whole is observed that RESTful Web services have different
HTML body (Cache-Control:no-cache(store)). For semantics for native apps and Web apps given the same
the cases when Web apps beat the native apps (i.e., the feature. It indicates that service providers have to maintain
Maoyan), we find that the cache policies of resources different server-side support for the two client-side apps,
are finely configured at the object level. A substantial although the apps access the same back-end logic and
proportion of resources is rather static, while quite few vary data. When the app developers need to implement both
through different visits. Such a fine-grained configuration clients, they have to understand different server-side API
is somewhat similar to the native apps. To support such semantics, respectively. Hence, service providers had better
fine-grained configuration, both Web service providers and uniformly organize the RESTful APIs to reduce the cost of
Web apps developers should focus on the cache mechanism, development and maintenance for developers.
since better cache performance requires the cooperation
of client-side code, browser runtime, and server-side
support. Some recent features such as HTML5 AppCache
and localstorage can be leveraged to configure the
8 T HREATS TO VALIDITY
expiration time for stable resources. Besides the new We conducted our empirical analysis of only 328 distinct
features, another solution that can be applied to existing Web services from 12 typical service providers. The se-
Web apps is to separate the HTML into two parts, in order lected service providers cover various domains such as Map,
to load the changing resources dynamically and add them Social Network, e-Commerce, and Entertainment. In
to the page on demand. We have made several efforts on addition, these service providers come from both global (i.e.,
improving the cache management of mobile Web apps. [27], Google, Facebook, Amazon) and local (i.e., Baidu, Jd, Zhihu, and
[28] Dianping). In this way, the diversity of service providers can
reduce the selection bias of our empirical study. Our es-
•Reducing HTTPS overhead. Although the HTTPS tablished experimental framework and trace-based analysis
protocol becomes popular, the additional overhead of approach can be generalized to other Web services.
data traffic should be addressed, especially for those Nevertheless, considerable care and attention should be
small-sized HTTP requests. To make the tradeoff between addressed to ensure the rigor of this measurement study.
security and performance, some possible approaches can However, as with any chosen research methodology, it is
be leveraged to mitigate the overhead. We suggest that hardly without limitations. In this section, we discuss the
the providers should use lightweight SSL handshakes. potential threats for validating our results.
Specifically, Session Identifier and Session Ticket can be
used to resuming previously terminated SSL sessions
with low handshake overheads [29] [20]. Session ticket is 8.1 Performance and User Experience
preferred because the SSL state is stored at the client side. Our measurement study in this article mainly focuses on the
performance at the network level and energy consumption
• Exploring connection reuse for Web apps. The Web of native apps and Web apps, given the same requested
apps may suffer from the limited number of concurrent functionality. We employ the number of requests, traffic
TCP connections. However, in some cases, we find that volume, and response time to quantify the performance.
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 12
Indeed, the performance of apps has much broader mean- it is reported that the HTTP processing is much better on iOS
ings that are usually related to user experience, including platform than that on other platforms [16]. Our approach
look and feel, interaction patterns, context awareness, per- relies on a proxy intercepting HTTP-level traces for Android
sonalization, and so on. However, many aspects of user devices. However, this tool cannot work for iOS devices.
experience cannot be directly characterized by quantitative The selected browser used in our experiment is only
metrics. Therefore, the principle of our study is to compare Google Chrome, as Chrome is quite popular and its kernel
the relatively objective metrics when the native apps and takes into account cross-platform issues. It is quite possible
Web apps executing the same user-perceived functionalities that different browsers may perform variously for the same
that have exactly the same contents and similar UIs dis- Web app. To overcome this threat, we can reproduce our
played to users on native apps and Web apps. experiments on the combinations of different versions of
In addition, native apps may implement a certain smart- Android platforms and different Web browsers such as
ness such as context awareness and energy awareness, Chrome, FireFox, and Opera.
making the comparison unfair. To alleviate such an issue,
we ensure a stable context for the smartphones which are
connected to a high-speed WiFi and the battery energy 9 RELATED WORK
is always at 100% level (with adaptor charging). Such a In this article, we study the performance of native apps
stable context could disable the smartness of native apps and Web apps considering the same functionalities deliv-
to some extent but may not totally remove it. We plan to ered through Web services. Web services research on desk-
comprehensively study the smartness of native apps and top computing has been well surveyed by a lot of litera-
their influences on user experience in our future work. tures [31] [32]. With the wide adoption of mobile devices
such as smartphones and tablet computers, the application
8.2 Generalization of Findings context of Web services has evolved from the desktop
computing to the mobile computing. However, research on
The most considerable concern of our empirical study is the
the usage of Web services on mobile devices is just at the
validity of findings on the native and Web apps from other
beginning.
service providers that are not covered in this article. It is
true that the vendor-specific implementation of apps can Srirama et al. [33] and Wagh et al. [34] focused on the
have impacts on the performance. However, our results can Web services provisioning techniques for mobile devices.
be meaningful for end-users to select the proper channel Guinard et al. [35] suggested using Device Profile for Web
for accessing the services, and for developers to fix the Services (DPWS) and REST to easily integrate physical
potential imperfect designs or even performance bugs, such devices into existing enterprise information systems as Web
as redundant response, coarse-grained cache configurations, service providers. However, it is much more common for
and so on. mobile devices acting as Web services consumers instead of
Web services providers.
Additionally, some of our results can be generalized
to the service providers when they design their apps. For Although Tekli et al. [36] focused on SOAP performance
example, we observe that the fine-granularity cache config- enhancement, RESTful Web services are considered to be
uration of objects in Web apps can significantly improve a more promising solution on mobile devices that have
the performance, especially for reducing the payloads of relatively limited computation resources. In practice, it is
large-size but rather stable resources such as HTML body reported that most major service providers have adopted
and images. Instead of using the default heuristic cache RESTful fashion [34] [37]. Wagh et al. [34] made a compar-
configuration, the Web apps distinguish the configurations ison study on the performance evaluation on handheld de-
of the images of goods and those of logos, footers, and icons. vices. Arroqui et al. [38] showed that RESTful Web Services
improve the efficiency of data transfer for mobile devices.
Hamad et al. [39] showed that RESTful Web services con-
8.3 Versions of Apps sumed less message size and response time than SOAP Web
Another threat for reproducing our results is the changing services for mobile devices. Fokaefs et al. [40], KamalDldin
versions for apps. To overcome this threat, we download et al. [41], and Mizouni et al. [42] focused on the performance
the .apk files of the versions of native apps evaluated in of Web services on mobile platforms, and demonstrated that
our article. Practitioners could use these files to reproduce the RESTful pattern is much more suitable for resources-
the experiments described in this article. However, the Web constrained mobile devices.
apps cannot be treated in the same way. It is known that Due to the benefit of cross-platform accessibility by
the Web apps always deliver the latest version to end-users. Web apps, some efforts have been made to improve the
Although we can download the .html files of Web apps performance. Jamal et al. [43] used a cloud-based proxy and
and request them from our server, some dynamic resources cache mechanism to enhance the performance of RESTful
generated by JavaScript cannot be fully covered. In this way, Web services for mobile devices. Agabaov et al. [44] showed
it can be hard to completely reproduce the results of Web that Google presented Flywheel, an HTTP proxy service, to
apps. reduce the page size and response time to speed up Web
apps. Wang et al. [45] and our recent work [6] focused on
the cache mechanism to improve the performance of Web
8.4 Heterogeneous Platforms apps. Butkiewicz et al. [46] presented KLOTSKI, a system
What is worth mentioning is the platform heterogeneity. that prioritized the content to improve the user experiences.
Our experiments were made on only the Android platform. Compared to previous efforts, our research is driven
However, different platforms vary in network performance by an entirely different motivation. Either the native app
by using different libraries for the native apps. For example, and Web app has its own advantages and disadvantages.
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 13
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 14
[28] X. Liu, Y. Ma, X. Wang, Y. Liu, T. Xie, and G. Huang, “Swarovsky: Yun Ma is now a postdoctor in the School of
Optimizing resource loading for mobile web browsing,” IEEE Software of Tsinghua University, Beijing ,China.
Transactions on Mobile Computing, accepted to appear. He received his Ph.D degree in the School of
[29] “The ietf tls protocol version 1.0 .” https://www.ietf.org/rfc/ Electronics Engineering and Computer Science
of Peking University at 2017. His research inter-
rfc2246.txt. ests include Web engineering and mobile com-
[30] “Http/2, what is http/2?” http://http2.github.io/. puting.
[31] S. Wang, W. A. Higashino, M. Hayes, and M. A. Capretz, “Service
evolution patterns,” in Web Services (ICWS), 2014 IEEE International
Conference on. IEEE, 2014, pp. 201–208.
[32] O. Kondratyeva, A. Cavalli, N. Kushik, and N. Yevtushenko,
“Evaluating quality of web services: A short survey,” in Web
Services (ICWS), 2013 IEEE 20th International Conference on. IEEE,
2013, pp. 587–594.
[33] S. N. Srirama, M. Jarke, and W. Prinz, “Mobile web service provi-
sioning,” in Telecommunications, 2006. AICT-ICIW’06. International
Conference on Internet and Web Applications and Services/Advanced
International Conference on. IEEE, 2006, pp. 120–120.
[34] K. Wagh and R. Thool, “A comparative study of soap vs rest Xuanzhe Liu is an associate professor in the
web services provisioning techniques for mobile host,” Journal of School of Electronics Engineering and Com-
Information Engineering and Applications, vol. 2, no. 5, pp. 12–16, puter Science, Peking University, Beijing, China.
2012. He received his Ph.D degree in Peking Univer-
sity at 2009. His research interests are in the
[35] D. Guinard, V. Trifa, S. Karnouskos, P. Spiess, and D. Savio, area of services computing, mobile computing,
“Interacting with the soa-based internet of things: Discovery, and Web engineering.
query, selection, and on-demand provisioning of web services,”
IEEE T. Services Computing, vol. 3, no. 3, pp. 223–235, 2010.
[Online]. Available: http://doi.ieeecomputersociety.org/10.1109/
TSC.2010.3
[36] J. Tekli, E. Damiani, R. Chbeir, and G. Gianini, “SOAP
processing performance and enhancement,” IEEE T. Services
Computing, vol. 5, no. 3, pp. 387–403, 2012. [Online]. Available:
http://doi.ieeecomputersociety.org/10.1109/TSC.2011.11
[37] J. Huang, Q. Xu, B. Tiwana, Z. M. Mao, M. Zhang, and P. Bahl,
“Anatomizing application performance differences on smart-
phones,” in Proceedings of the 8th international conference on Mobile
systems, applications, and services. ACM, 2010, pp. 165–178. Yi Liu is now a Ph.D student in the School of
[38] M. Arroqui, C. Mateos, C. Machado, and A. Zunino, “Restful Electronics Engineering and Computer Science
web services improve the efficiency of data transfer of a whole- of Peking University, Beijing, China. His research
interests include mobile computing, network sys-
farm simulator accessed by android smartphones,” Computers and tems, and Web engineering.
Electronics in Agriculture, vol. 87, pp. 14–18, 2012.
[39] H. Hamad, M. Saad, and R. Abed, “Performance evaluation of
restful web services for mobile devices.” Int. Arab J. e-Technol.,
vol. 1, no. 3, pp. 72–78, 2010.
[40] M. Fokaefs, R. Mikhaiel, N. Tsantalis, E. Stroulia, and A. Lau, “An
empirical study on web service evolution,” in Web Services (ICWS),
2011 IEEE International Conference on. IEEE, 2011, pp. 49–56.
[41] K. Mohamed and D. Wijesekera, “Performance analysis of web
services on mobile devices,” in Proceedings of the 3rd International
Conference on Ambient Systems, Networks and Technologies (ANT
2012), the 9th International Conference on Mobile Web Information
Systems (MobiWIS-2012), Niagara Falls, Ontario, Canada, August 27-
29, 2012, 2012, pp. 744–751.
[42] R. Mizouni, M. A. Serhani, R. Dssouli, A. Benharref, and I. Taleb, Yunxin Liu is a lead researcher in Microsoft
Research Asia. He received his Ph.D. in com-
“Performance evaluation of mobile web services,” in Web Services puter science from Shanghai Jiao Tong Univer-
(ECOWS), 2011 Ninth IEEE European Conference on. IEEE, 2011, sity in 2011 (through the SJTU-MSRA joint PhD
pp. 184–191. program). His current research interests include
[43] S. Jamal and R. Deters, “Using a cloud-hosted proxy to support mobile systems and networking.
mobile consumers of restful services,” Procedia Computer Science,
vol. 5, pp. 625–632, 2011.
[44] V. Agababov, M. Buettner, V. Chudnovsky, M. Cogan, B. Green-
stein, S. McDaniel, M. Piatek, C. Scott, M. Welsh, and B. Yin,
“Flywheel: Google’s data compression proxy for the mobile web,”
in 12th USENIX Symposium on Networked Systems Design and Im-
plementation, NSDI 15, Oakland, CA, USA, May 4-6, 2015, 2015, pp.
367–380.
[45] X. S. Wang, A. Krishnamurthy, and D. Wetherall, “How much can
we micro-cache web pages?” in Proceedings of the 2014 Conference
on Internet Measurement Conference. ACM, 2014, pp. 249–256.
[46] M. Butkiewicz, D. Wang, Z. Wu, H. V. Madhyastha, and V. Sekar,
Gang Huang received his Ph.D degree from
“Klotski: Reprioritizing web content to improve user experience the School of Electronics Engineering and Com-
on mobile devices,” in 12th USENIX Symposium on Networked puter Science of Peking University, Beijing,
Systems Design and Implementation, NSDI 15, Oakland, CA, USA, China, 2003. He is now a full professor in Insti-
May 4-6, 2015, 2015, pp. 439–453. tute of Software, Peking University. His research
[47] Z. Hemel and E. Visser, “Declaratively programming the mobile interests are in the area of middleware of cloud
web with mobl,” ACM SIGPLAN Notices, vol. 46, no. 10, pp. 695– computing and mobile computing. He is a mem-
712, 2011. ber of IEEE.
[48] “The localstorage attribute of html5,” https://html.spec.whatwg.
org/#the-localstorage-attribute.
1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.