0% found this document useful (0 votes)
20 views14 pages

Ieee Coy

This study analyzes and compares the performance of native mobile apps versus mobile web apps on Android devices. The researchers selected 328 popular mobile services and analyzed HTTP traces to understand how each type of app delivers services. They then characterized performance differences based on metrics like number of requests, response time, data usage, and energy consumption. The findings show that for more than 31% of services, web apps actually performed better than native apps in terms of these metrics. This suggests that with advances in technologies like HTML5, the performance of web apps is becoming more comparable to native apps.

Uploaded by

Almer 124
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
20 views14 pages

Ieee Coy

This study analyzes and compares the performance of native mobile apps versus mobile web apps on Android devices. The researchers selected 328 popular mobile services and analyzed HTTP traces to understand how each type of app delivers services. They then characterized performance differences based on metrics like number of requests, response time, data usage, and energy consumption. The findings show that for more than 31% of services, web apps actually performed better than native apps in terms of these metrics. This suggests that with advances in technologies like HTML5, the performance of web apps is becoming more comparable to native apps.

Uploaded by

Almer 124
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 1

A Tale of Two Fashions: An Empirical Study on


the Performance of Native Apps and Web Apps
on Android
Yun Ma, Student Member, IEEE, Xuanzhe Liu, Member, IEEE, Yi Liu, Yunxin Liu, Member, IEEE, and
Gang Huang, Member, IEEE

Abstract—The prevalent smartphones have become the major entrance to accessing services on the Internet. On smartphones, users
can have two options as the clients, i.e., native apps and Web apps. There have been several debates about native apps and Web
apps. However, major service providers such as Google, Amazon, and Facebook provide both native apps and Web apps to
end-users. Essentially, the performance differences between these two types of apps haven’t been addressed. Indeed, the
performance differences make non-trivial impacts on apps development, deployment, and distribution. In this article, we conduct a
measurement study on the performance of native apps and Web apps on Android smartphones. Specifically, we want to explore given
the same functionalities, do Web apps always perform poorly compared to native apps. We select 328 services from some popular
providers, covering various domains such as e-commerce, map, social networking, and entertainment. With HTTP-level trace analysis,
we demystify the workflows on how native apps and Web apps deliver services on mobile devices, respectively. Then we characterize
the performance differences between native apps and Web apps with the metrics including the number of requests, response time,
data drain, and energy consumption. We find that the performance of Web apps is better than native apps in more than 31% cases.
Our derived knowledge can suggest some recommendations to improve the performance for mobile apps.

Index Terms—Services, Mobile Applications, Web, Smartphones

1 I NTRODUCTION

A FTER the announcement of iPhone in 2007, the sales


and popularity of mobile devices such as smartphones
and tablet computers keep tremendously increasing in the
functionalities, there have been several debates between
them [2]. From a technical perspective, the native apps are
claimed to better leverage hardware capabilities of devices
past decade. Mobile devices have become the main entrance and are supposed to provide better performance. However,
to the Web services. Mobile users use mobile applications developing native apps requires a lot of efforts to deal with
(a.k.a., apps) to access these services [1], e.g., weather query various platform-specific features. For example, iOS apps
by Yahoo!, Web search by Google, shopping by Amazon, and are developed in Objective-C or Swift, and Android apps are
social networking by Facebook. developed in Java, which increases their development and
There are two types of mobile apps from which users maintenance cost. Additionally, the package-based software
can access Web services, i.e., native apps and Web apps [2]. release model results in various versions of native apps,
Native apps are developed in imperative languages (e.g., while the end-users have complaints as they have to always
Java and Objective-C) and released as software packages keep updating to the latest versions [4]. In contrast, the
that can be published on and downloaded from online app Web apps are developed by standard HTML, JavaScript,
stores. Web apps are developed with Web technologies (such and CSS, and can be opened in popular Web browsers such
as HTML, JavaScript and CSS) and directly accessed via as Chrome, Safari, and Firefox on any OS. Such a feature
browsers. Recently, a medium app called “hybrid app” can reduces the development and maintenance cost. Users can
be viewed as a special type between native apps and Web access the “always up-to-date” Web apps without having to
apps. In practice, hybrid apps are packaged and released as manually update them. Our recent study over millions of
native apps, and most of their key features are developed Android users shows that the Web browser is still the major
by Web technologies [3]. The behavior of hybrid apps is a app through which Android users access Web services [5].
combination of native apps and Web apps. Although Web apps are reported to perform more slowly
Conceptually, both native apps and Web apps can per- than native apps and the user experience is not satisfactory
form specific tasks and provide features to the end-users. enough [2], a recent study found that 153 out of the top
For example, we can view the timeline through either the 200 websites in Alexa1 still provide the Web apps that are
native app or the Web app provided by Facebook. Although optimized for mobile platforms [6].
these two types of apps can provide similar or even the same With the advances of HTML5, Web apps are supposed
to have comparable performance and experiences of native
• Yun Ma, Xuanzhe Liu, Yi Liu, and Gang Huang, are with the Key apps. However, to the best of our knowledge, very few
Laboratory of High Confidence Software Technologies (Peking Univer-
sity), Ministry of Education, Beijing, China, 100871. Email: {mayun, previous efforts demonstrate whether and to what extent
liuxuanzhe, liuyi14, hg}@pku.edu.cn. Xuanzhe Liu is the corresponding Web apps can be comparable to native apps. In this article,
author of this work. we are interested in answering a central problem: given the
• Yunxin Liu is with Microsoft Research. Beijing, China, 100084. Email:
same or similar functionality, do Web apps always perform
[email protected]
Manuscript received xxx, 2016 1. http://www.alexa.com

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 2

more poorly than the native apps? The answer to this number of HTTP(S) requests, response time, traffic
question is non-trivial and meaningful in two folds. First, volume, and energy consumption. In about 31% of
if the performance of Web apps is close to that of native the situations, Web apps perform much better than
apps, service providers can reduce the maintenance cost of native apps. We then analyze the possible reasons
multiple versions, and end-users can get rid of the updates. that can lead to such differences.
Second, even if the performance of Web apps is poor com- • We draw some implications from our empirical study
pared to that of native apps, demystifying the underlying and propose recommendations on how to improve
technical causes can provide insightful knowledge for the the performance of mobile apps from various aspects
service providers and Web app developers, who can then such as network protocols, cache, and API semantics.
find potential solutions to optimize the Web apps. In particular, we attempt to make Web apps provide
In this article, we break down the central problem by comparable performance levels and user experiences
studying three uncovered problems: (1) how native apps to native apps.
and Web apps access Web services on mobile devices;
The remainder of this article is organized as follows.
(2) to what extent the performance of native apps and
Section 2 introduces some background knowledge. Section
Web apps varies; (3) what kind of knowledge developers
3 describes the measurement methodology and how we
can leverage to develop and optimize the apps. To this
conduct our experiments. Section 4 anatomizes the archi-
end, we consider four quantitative metrics to characterize
tecture of native apps and Web apps on using Web services,
the performance: the number of HTTP(S) requests, traffic
respectively. Section 5 analyzes the performance differences
volume, response time, and energy. We select 328 distinct
of native apps and Web apps, given the same functionality.
services from 12 popular service providers, i.e., Facebook,
Section 6 summarizes our findings according to the architec-
Amazon Shopping, Google Map, Baidu Map, Sina Weibo, Di-
tural differences. Section 7 provides implications to service
anping, Gewala Movie, Jd, Maoyan Movie, Meituan, Zhihu, and
providers and developers. Section 8 discusses the threats to
Douban, which develop both native apps and Web apps with
validity of our approach. Section 9 presents related work,
the same or similar functionalities for mobile devices. We
and Section 10 concludes this article with future outlook.
choose Android as the target platform of native apps as
Android is the most popular mobile OS nowadays. In the
rest of this article, the native apps refer to Android apps if 2 BACKGROUND
not specified. By using two Android smartphones with the
Traditionally, there are two types of apps to deliver Web
same device model under the same network condition, we
services on Smartphones.
conduct our measurement study and find that in about 31%
•Native apps. A native app is developed for a specific OS
of the situations, Web apps perform much better than the
platform, i.e., iOS, Android, or BlackBerry. Native apps are
native apps, when providing the same functionality. Based
usually developed in an imperative programming language.
on the analysis of various aspects such as network protocol,
For example, Android apps are developed in Java, and iOS
third-party libraries, and cache efficiency, we suggest some
apps are developed in Objective-C or Swift. Native apps can
possible recommendations that may help to make the per-
take advantage of the particular features of mobile devices,
formance of Web apps comparable to that of native apps.
such as various sensors. However, most parts of native
This article is the extension of our previous ICWS 2015 apps cannot be executed across different platforms, whereby
paper [7] that made the first study of performance differ- developers have to maintain various versions for different
ences of Web services delivered through native apps and platforms. End-users usually need to download native apps
Web apps. Compared to the conference version, this article from app stores, such as Apple AppStore, Google Play, and so
significantly extends the scale of Web services (from 52 to on. In addition, end-users have to manually keep updating
328) and adds a new metric of energy consumption. Besides of native apps.
obtaining the almost consistent findings of the conference •Web apps. A Web app is developed for running within
version, our results further demonstrate some imperfect Web browsers or browser-based runtime such as WebKit
designs that can lead to unsatisfactory performance and and Gecko. It is developed by Web technologies, such as
user experiences on smartphones. We also provide some HTML5, CSS3, and JavaScript. End-users enter the URL
useful knowledge and suggestions to service providers, in the address bar or click through the hyperlinks, and
Web/native app developers, and end-users. The main con- then navigate to the latest version of Web apps. Web apps
tributions made in this article can be summarized as follows: have advantages in terms of cross-platform compatibility,
• We establish an experimental platform2 to collect the or the so-called “write once, run anywhere” feature. With
network-level traces of requesting Web services on the advances of HTML5, Web apps are claimed to provide
smartphones, and propose a trace-based approach to comparable user experiences to native apps. However, due
characterizing the usage of Web services of native to the limitation of Web technologies, standard Web apps
apps and Web apps, respectively. Based on the trace cannot take full advantage of the devices’ hardware capabil-
analysis, we demystify how Web services are used ities, and their performance is often complained to be poorer
from an architectural style perspective. than native apps in practice. Sometimes, users can perceive
• We design a series of experiments to measure the quite obvious latency when loading a Web app.
performance of native apps and Web apps, given There is another type of apps, called Hybrid apps, that
by the same functionality. Our metrics include the could be regarded as an intermediate type between native
apps and Web apps. Similar to the native apps, the hybrid
apps also need to be downloaded from app stores. Although
2. We have released the whole dataset along with the work pub-
lished. Please contact the corresponding author, Xuanzhe Liu, via the hybrid apps wear the “shell” of native apps, the core
[email protected], for more details logic of hybrid apps heavily relies on Web technologies. As

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 3

a result, hybrid apps can support cross-platform portability TABLE 1


and reduce development costs. Many tools and frameworks Selected Apps of Targeted Web Services
could help to develop hybrid apps, such as Apache Cor-
dova [8], manifoldJS [9], and React Native [10]. The behavior APP Name Description # of Features # of Services
Facebook Social Networking 31 33
of hybrid apps is a combination of native apps and Web Sina Weibo Social Networking 19 25
apps. In order to make a clear comparison, in this article, Amazon e-Commerce 29 34
we do not take hybrid apps into account and leave them as Jd e-Commerce 32 39
Google Map Map 16 22
future work. Baidu Map Map 16 29
Nowadays, service providers deliver their functionalities Dianping Lifestyle 22 31
via Web services. Web services were initially proposed as Douban movie Entertainment 19 23
a way that a software system can support interoperable Gewala movie Entertainment 18 25
Maoyan movie Entertainment 14 17
machine-to-machine interactions over a network [11]. Com- Meituan Groupon Shopping 20 24
monly, there are two main types of Web services, the SOAP- Zhihu Online Q& A 24 26
based ones and the RESTful ones. There have been a lot of
debates on comparing these two types of Web services [12].
In the context of mobile computing, it is observed that with the two apps. Finally, we choose 260 features involving
SOAP-based Web services are hardly applied due to the 328 distinct Web services3 . We list the number of selected
non-negligible performance overheads and restrained com- features and Web services by each provider in Table 1.
putation resources on mobile devices. The RESTful Web
services are more adequate to the heterogeneous mobile
networks and resources-limited mobile devices. Many main- 3.2 Definitions and Metrics
stream service providers, including Facebook, Google, and We first define some terms used to describe our approach.
Amazon, have deprecated or abandoned SOAP-based inter- • D EFINITION 1 (S ERVICE). A service is an atomic functional-
faces. RESTful Web services allow developers to use HTTP ity implemented by the service providers. For example, adding
methods explicitly in a way that follows the protocol as goods to cart and deleting goods from cart in Amazon
defined by RFC 2616 [13]. Four basic HTTP methods can Shopping should be viewed as a service.
operate resources deployed on the Web server: (1) GET refers • D EFINITION 2 (F EATURE). A feature is a functionality that
to retrieve a resource from the server; (2) POST refers to may request a specific (set of) services, such as login, checking
create a sub-resource on the server; (3) PUT refers to create a out the shopping cart, and so on. A feature can comprise
new resource or update the state of an existing resource on more than one service, or, a composite service. For example,
the server; (4) DELETE refers to remove or delete a resource when users log on the Facebook, the client program needs to
on the server. request three services including retrieving the user profile,
updating the status of friends, and checking the timeline.
• D EFINITION 3 (T RACE). A trace is a sequence of
3 MEASUREMENT METHODOLOGY
HTTP/HTTPS requests and responses when using a feature.
In this section, we describe the rationale of our trace-based Figure 1 shows three traces of Amazon Shopping, including
measurement methodology. We provide details of how our the features of searching keywords, adding goods to cart, and
empirical study is designed, including the hardware and deleting goods from cart, respectively.
software setup, metrics and their definitions, and the mea- Based on the definitions above, we then define the met-
surement workflow. We then propose three research ques- rics used to measure the performance in our study. Learning
tions that are to be answered by our empirical study. experiences from previous work [16] [15], we focus on four
metrics.
3.1 The Dataset •M ETRIC 1 ( N UMBER OF HTTP(S) R EQUESTS) refers to the
number of HTTP(S) connections sent by the app when using a
Trace-based performance measurement is widely used in
feature. It can reflect how many HTTP resources are needed
analyzing network performance [14] [15] [16] [17]. We make
to accomplish a specific feature. For example, for the feature
a trace-based analysis to anatomize the workflows of native
of adding goods to cart in Amazon Shopping, there is only
apps and Web apps by requesting the same functionality. We
one request “/aw/detail/ajax/add-to-cart/” sent from the native
choose apps from categories that heavily rely on network,
app, as show in Figure 1.
including social networking, e-commerce, entertainment,
and so on. For each category, we select some popular service
•M ETRIC 2 (T RAFFIC V OLUME) means the total data volume
involved in a trace, including the size of bytes sent and received
providers who provide both native apps and Web apps.
via the app.
Finally, our dataset consists of native apps and Web apps
from 12 popular service providers, including Facebook, Ama-
•M ETRIC 3 (R ESPONSE T IME) is the interval from the time
when the first network request is issued to the time when the
zon Shopping, Google Map, Baidu Map, Sina Weibo, Dianping,
last response is received in a trace. The response time is not
Gewala Movie, Jd, Maoyan Movie, Meituan, Zhihu, and Douban,
equivalent to the user-perceived time when using a feature.
as shown in Table 1. It can be observed that the chosen
Instead, it comprehensively indicates how long a feature can
service providers cover both international and local (China)
be accomplished at the system level. As shown in Figure 1,
ones. Such a diverse and representative dataset can promise
the response time of searching keywords feature means the pe-
our measurements with little bias. For each service provider,
riod between the start time of the first request “s/k=iphone...”
we choose the features that have the same user-perceived
functionalities between the native app and Web app. To this
3. We have released the dataset on our website http://sei.pku.
end, we manually examine every single feature to ensure edu.cn/∼mayun11/NativeWeb, including the details of features and
that the displayed contents on the UI are exactly the same measurement results

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 4

Fig. 1. Trace Logs of the Native App and the Web App of Amazon Shopping Generated by the Same Feature

and the end time of the last response “/images/...” within Chrome Browser of version 40.0.2214.89 on the other
•M ETRIC 4 (E NERGY C ONSUMPTION) is the total amount smartphone. We manually use the native and Web app to
of energy consumed from the time of the first issued network access each selected feature. For each feature, we repeat
request to the time of the last received response in a trace. We three times to avoid the biased measurements. In order to
employ a physical Monsoon Power Monitor4 to record make the comparison fair enough, we concurrently access
the power consumption of the connected smartphones and the same feature of both types of apps with two people
compute the energy consumption when requesting each using the two smartphones simultaneously.
service. By applying the V-Edge energy model proposed
in our previous work [18], we can accurately compute the
energy consumption as follows: 3.4 Tracing Web Services Usage
Z end time
energy consumption = V-Edge(x) (1) We deploy a proxy server which runs both the WireShark
start time and the popular HTTP/HTTPS-level proxy called Fiddler5
to intercept all network traffic between the smartphones and
where the start time means the start time of a trace, end time
the remote Web services. In this way, we can explore the
means the end time of a trace, and x means the time between
network details more accurately by investigating the trace
start time and end time used in the V-Edge power model.
logs from both smartphones and the proxy server. Then, we
Due to page limit, the details of V-Edge can be referred from
compare the network traces of native apps and Web apps,
previous paper [18].
given the same feature. For example, we compare the traces
of searching_a_person feature offered by Facebook, by
3.3 Experiment Setup using the same keyword query on the Web app and the
Next, we introduce the infrastructure of our experimental native app.
platforms and how we conduct our experiments. As the cache mechanism is widely used in both native
• Hardware. We use two smartphones of the same device apps and Web apps [20], we also request each feature of the
model. The selected device model is Samsung Galaxy S4 targeted Web services in two modes, Without Cache and With
with 2 GB RAM and Android 4.2 OS (directly downloaded Cache.
from Google rather than using the manufacturer customized
ROM). The two smartphones are connected to the same • Without Cache. To disable the cache of native apps,
Wi-Fi network, and are “rooted” with the system-level we manually uninstall a native app, and remove all
privilege. We uninstall all other pre-installed apps, stop files in its folder. Then we re-install and revisit the
background activities such as OS auto-update and auto-sync app for the same feature. To disable the cache of
services. In this way, we can prevent other apps from con- Web apps, we manually clear the browser cache of
necting the network and significantly reduce the possible a Web app and reload the app. At each time we
noise. To inspect the details of network activities, we use record the network traces and energy information,
the popular Wireshark tool [19] to inspect the traffic to/from and calculate the number of HTTP(S) requests, traffic
our smartphones. Although the WireShark tool can possibly volume, response time, and energy consumption.
impact the results of energy measurement, previous studies • With Cache. We test the same feature of the two
show the impact is too marginal and could be neglected [18]. types of apps, with the default cache mechanisms
• Software. For each service provider, we install and access turned on. We record the same information and
the native app on one smartphone, and open the Web app calculate the metrics as well.

4. https://www.msoon.com/LabEquipment/PowerMonitor/ 5. http://www.telerik.com/fiddler

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 5

3.5 Research Questions Statement 4.2 Architecture of Native Apps on Using Web Services
Based on the dataset and hardware/software configura- Figure 2 shows the architecture on how a native app uses
tions, our empirical study aims to measure how the native Web services. Once a native app is installed on the de-
apps and Web apps perform, given the same functionality. vice, the native code, including the data parsing, network
More specifically, we aim to answer the following research management, and layout rendering, are maintained locally
questions: on the smartphone. In addition, some resources such as
icons and images can be configured by the developers and
• RQ1: How do native apps and Web apps access Web locally stored. When a user launches the native app and
services on smartphones, respectively? clicks some buttons to issue interaction requests, the pro-
• RQ2: How do native apps and Web apps perform differ- cessing logics of native code are executed. The native code
ently when providing the same functionality? may trigger requests to the target Web services, and send
• RQ3: What kind of knowledge can developers leverage to synchronous/asynchronous requests to fetch data. When
optimize the apps? requesting Web services, the native apps come up with lots
of preloaded elements and fetch only the “required” data
from the Web server rather than the contents of the entire
application. The native app can create a new thread to issue
4 ARCHITECTURE ANATOMY an asynchronous request when a resource is required. When
We first focus on RQ1, i.e., how do native apps and Web apps the data is downloaded, the native code can parse the data,
access Web services, respectively? Answering such a question process the data with the preloaded logic, and re-render the
can help us have the basic insight of how the two types of new user interface.
apps request a Web service. Based on such knowledge, we
can then make in-depth performance analysis. 4.3 Architecture of Web Apps on Using Web Services
In contrast, the Web apps are accessed in the smartphone’s
Web browser such as Chrome, FireFox, and Safari, and
4.1 Web Service Identification update themselves without user intervention. Web apps
are built by standard technologies such as HTML5, CSS,
For better illustration, we take a popular service provider,
JavaScript, and so on. Figure 3 shows the architectural
the Amazon Shopping, as our illustrating example. Figure 1
workflow on how a Web app requests a Web service. The
shows some traces of Amazon Shopping on both the na-
Web app has to download all resources such as page files,
tive app (at left side) and the Web app (at right side).
images, JavaScript files, and CSS files to render the user
When users fill in the information and click the button,
interface in the mobile browser. When a user first loads
some consequent requests and respective responses will
the Web app, the browser initiates a request to fetch the
be triggered. We list traces of three features, i.e., the
structural information of the main page. Then the browser
search_keywords feature, the add_goods_to_cart fea-
interprets the HTML, JavaScript, and CSS files to build up
ture, and the delete_goods_from_cart feature.
user interfaces. There are two forms of user interactions over
All of the 328 Web services in our dataset, are provided
the Web apps. The first one is to input some text fields or
in the RESTful fashion. Such an observation evidences that
submit some forms, then the Web apps jump from one page
SOAP-based Web services are not widely adopted in the
to another page. The other one is that the Web app generates
context of mobile Internet. It can be found that the contents
an AJAX (Asynchronous JavaScript and XML) request, and
of service responses are formatted with JSON in both native
deals with the callback methods without page jumping. As
apps and Web apps. Although these two types of apps
the Web apps fetch the latest resources of a Web service, the
have similar user interfaces, the requested URLs of the same
user is always displayed by the latest version of the app.
feature are entirely different. For example, when we add a
good to the cart, the native app sends a request to the URL
of “/aw/detail/ajax/add-to-cart/”, while the Web app sends a 4.4 Architectural Differences
request to the URL of “/aw/c/”. By exploring all of the Based on the above analysis, we can have an observation
328 Web services in our dataset, it is quite interesting that that native apps and Web apps have quite different work-
the target URLs of Web services requested by native apps flows and behaviors of requesting Web services. Besides the
and Web apps are always different for the same feature. different implementation languages and different RESTful
It implies that Web service providers maintain different API semantics, two differences need to be mentioned.
interfaces for native apps and Web apps. In addition, it is The first difference is the cache management of re-
interesting to find that native apps and Web apps may use sources. Compared to the Web apps that always present the
different IP addresses to access the same services. Therefore, up-to-date version, the native apps are updated manually
we can infer that the server-side service implementations of when the users explicitly launch the Update Manager that
the same feature are separately deployed for native apps is responsible for checking updates and downloading the
and Web apps, although the two types of apps still access latest version. Rather than relying on the browsers that
the same back-end database. manage the resources to be fetched from the remote server,
Based on the traces, we then try to figure out the developers of native apps can programmatically configure
workflow of how native apps and Web apps request a the need-to-update resources. As a result, the “stable”
Web service, respectively. We examine the response of each resources such as icons and images are permanently stored
request and infer its function in the workflow from the in the local storage, and the update manager fetches only
header fields and some keywords in the content such as the “delta” resources. Such a mechanism is quite effective
update, version, etc. to better reuse the already-downloaded resources, and thus

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 6

Fig. 2. The Architecture of Native App Fig. 3. The Architecture of Mobile Web App

reduces the unnecessary data transfer. In contrast, the cache TABLE 2


management of Web apps usually depends on the browser The Percentage of Genres Where Web Apps Beat Native Apps
and some coarse-grained policies [6].
The other difference is the network resources manage- HTTP(S) Traffic Response
Operation Energy
Requests Volume Time
ment, which can be inferred from trace logs of WireShark PUT&POST 22.61% 28.26% 36.96% 36.96%
and Fiddler. We employ the existing techniques [16] to DELETE 20.24% 23.81% 45.24% 42.86%
analyze the traces of WireShark and synthesize the TCP GET 42.97% 24.22% 37.50% 41.41%
connections. In fact, both native apps and Web apps need
to setup TCP connections to the RESTful Web services. It is
interesting to find that, due to the constraints of browsers and Web apps, we categorize all the tested features into
such as Chrome and FireFox [15], the Web apps usually three groups according to the type of RESTful operations,
occupy at most 6 concurrent TCP connections per domain. i.e. GET, DELETE, PUT & POST. We aggregate the PUT
In contrast, there are more concurrent TCP connections in and POST operations into one group, as they both result
native apps, as the developers can configure the number in resources updating on the server. We then report and
of connections by themselves. More specifically, the widely- analyze the performance of each category, respectively.
adopted AndroidHttpClient API and other third-party
libraries such as okhttp [21] and Volley [22] usually 5.1 PUT&POST
support a developer-defined number of TCP connections.
In the semantics of RESTful Web services, the PUT refers to
These two reasons can preliminarily reflect some fun-
the request that creates a new resource or modifies the exist-
damental architectural differences between native apps and
ing resource, and the POST refers to the request that sends
Web apps. In the following empirical studies, we move on
a representation of a new data entity to the server so that
to exploring the performance differences and correlate such
it will be stored as a new subordinate of the resource. We
differences to the architectural style.
choose seven typical features that need PUT& POST opera-
tions where the Web apps provide the comparable or even
5 M EASUREMENT R ESULTS better performance than native apps: the post_timeline
In total, we have 260 distinct features that consist of 328 Web feature in Facebook, the add_comment feature in Dian-
services: 66 PUT & POST operations, 64 DELETE operations, Ping, the collect_goods feature in Amazon Shopping, the
and 198 GET operations. As we focus on 4 metrics and add_goods_to_cart feature and collect_goods fea-
evaluate each metric with/without enabling the cache on ture in Jd, the collect_shop feature in Meituan, and the
two types of apps, finally we have 260*4*2=2,080 genres of add_review feature in Douban. Figure 4, Figure 5, Figure 6,
data. and Figure 7 show the number of HTTP(S) requests, the traf-
We now focus on RQ2, i.e., how do native apps and Web fic volume, the response time, and the energy consumption
apps perform differently when providing the same functionality? of the above PUT&POST operations, respectively.
Given the same features, it is intuitively assumed that the Number of Requests. For PUT&POST operations, we find
performance of Web apps is usually less satisfactory than that the number of HTTP(S) requests does not vary very
native apps. However, it is surprising to find a substan- much. Both the native app and Web app initiate almost the
tial percentage (about 31%) of exception where the Web same number of requests with or without cache. Such an
apps perform better than the native apps, i.e., in 645 out observation is not surprising as the PUT&POST operations
of our 2,080 genres, as shown in Table 2. It implies that usually do not need local access. However, some exceptions
there are some possible reasons to make the Web apps are observed. For example, for the collect_shop feature
have comparable performance with native apps. Due to of Meituan, the native app initiates more network requests.
space limit, this article reports only the cases where The reason is that such a feature needs to get the latest list
Web apps beat the corresponding native apps. The full of all collected shops after the POST operation is finished,
experimental evaluation results can be found at our lab and triggers one additional request. However, in fact such
server via http://sei.pku.edu.cn/∼mayun11/NativeWeb. redundant transfer is not necessary and the two requests
Web services of different RESTful operations represent can be merged.
different kinds of functionalities and their behaviors are Traffic Volume. Unlike the number of HTTP(S) requests,
potentially different. In order to clearly investigate differ- the traffic volume differs between the two types of apps.
ent reasons leading to the different performance of native Generally, the Web apps need more traffic volume than

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 7

Fig. 4. Number of Requests in PUT&POST Fig. 6. Response Time in PUT&POST

Fig. 5. Traffic Volume in PUT&POST Fig. 7. Energy Consumption in PUT&POST

transmission can result in more energy consumption than


native apps for the same PUT&POST operation. The reason
Web apps.
is that Web apps usually send more information in POST &
PUT, such as more cookies and longer User-Agent, to the
servers. However, some exceptions need to be mentioned. 5.2 DELETE
For the Facebook’s post_timeline feature, the Web app In the semantics of RESTful Web services, DELETE refers
saves traffic than the native app, no matter the cache is en- to the request that deletes an existing resource on the Web
abled or not (29.3% saving and 29.9% saving, respectively). server. We show seven typical features that correspond
It is because that the native app refreshes the whole timeline to DELETE operations, i.e., the cancel_collection fea-
after the POST action is triggered, instead of refreshing ture in DianPing, the remove_goods_from_cart feature
only the latest status. Other exceptions can be found in the and the cancel_collection feature in Amazon Shop-
add_review feature of Douban (about 55.4% saving), the ping, the cancel_collection feature in Gewala movie, the
collect feature of Amazon Shopping (about 14.5% saving), delete_cart feature and unfollow_shop feature in Jd,
and the collect_shop feature of Meituan (about 43.6% and the delete_shop feature in Meituan. Figure 8, Figure 9,
saving). In addition, we can also find that the cache mecha- Figure 10, and Figure 11 show the number of HTTP(S)
nism marginally works in the PUT&POST operations. requests, the traffic volume, the response time, and the
Response Time. In most cases, more traffic volume energy consumption of DELETE operations, respectively.
means longer response time. However, we can find that Number of Requests. In most cases, both the native
sometimes the Web app has shorter response time than the apps and Web apps initiate the same number of requests to
native one. For example, the add_goods_to_cart feature convey the deletion command. In term of the delete_cart
and the collect feature of Jd’s Web app consume more feature in Jd, the native app initiates an additional request to
traffic volume than those of Jd’s native app, but the Web fetch some recommended items for customers after clearing
app’s response time is shorter (more than 50% faster) than the shopping cart. In terms of the delete_shop feature
the native app’s. in Meituan, the native app initiates a new request to fetch
Energy Consumption. Overall, the Web apps perform all collected shops after deleting a shop. These additional
more poorly than the native apps in terms of energy requests are not sent in the Web apps. To reduce the cost
consumption. However, the preceding mentioned features of network resources, we shall recommend that the service
with shorter response time for Web apps still save energy provider should merge these requests.
compared to native apps. For example, the native apps need Traffic Volume. Similar to the PUT&POST operations, the
to record the states of the device, write to the local disk, and native apps of Meituan and Amazon Shopping consume more
communicate with the remote servers. The I/O and network traffic volume than the Web app. The native app of Meituan

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 8

Fig. 8. Number of Requests in DELETE Fig. 10. Response Time in DELETE

Fig. 11. Energy Consumption in DELETE


Fig. 9. Traffic Volume in DELETE

5.3 GET

fetches the latest collected shops list after deleting a shop. In the semantics of RESTful Web services, GET refers to
The native app of Amazon Shopping transfers data by HTTPS, the request that retrieves a representation of a resource
while the Web app transfers data by HTTP. We also find from the Web server. We examine seven typical features
that the native app of Amazon Shopping adds extra cookie that need GET operations, i.e., the search_keywords
fields in the request header, such as device info, application feature in Weibo, the shanghai_page feature in Dian-
version, etc. For the feature of delete_cart in Jd, the Web Ping, the goods_detail feature in Amazon Shopping, the
service returns a list of recommended items to the native movie_detail feature in Gewala movie, the box_office
app. In contrast, for the unfollow_shop feature in Jd and feature in Maoyan movie, the search feature in Meituan,
the cancel_collection feature in Gewala movie, the Web and the friendlist feature in Zhihu. Figure 12, Figure 13,
app includes more cookie fields in the request header than Figure 14, and Figure 15 show the number of HTTP(S)
the native app to keep states with the server. requests, the traffic volume, the response time, and the
energy consumption in GET operations, respectively.
Response Time. Usually, the DELETE operation can be
Number of Requests. In the GET operations, the Web
executed very fast in both types of apps. An exception is
apps usually initiate more HTTP(S) requests than the native
the native app of Amazon Shopping that transfers data by
apps for the same feature. It is because the Web app needs
HTTPS protocol, and spends non-negligible time on HTTPS
to download all the needed resources, including JavaScript
handshaking.
files, CSS files, and other resources. However, for the
Energy Consumption. Although it is observed that the search_keywords feature in Weibo, the movie_detail
two features in Amazon Shopping consume more traffic vol- feature in Gewala movie, etc., the native apps initiate more
ume and have more response time in its native app, the requests than the Web apps. For example, we find that the
native app consumes less energy. It indicates that we cannot native app receives the friends’ information as well as the
determine the energy consumption of an application by only avatars according to the key words in Weibo, and fetches
the data traffic and response time. pictures of higher quality to provide better user experience

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 9

Fig. 12. HTTP(S) Requests in GET Fig. 14. Response Time in GET

Fig. 13. Traffic Volume in GET Fig. 15. Energy Consumption in GET

in Gewala movie. Such a difference is also caused by the to find that the performance of some Web apps may be
different semantics of targeted Web services. As described better than the native ones. The Web app of Maoyan movie
in Figure 12, the Web app of Zhihu fetches all the friends’ takes advantage of AppCache [23], an advanced feature
avatars, but the native app fetches only a part of friends’ provided by HTML5 allowing the developers to configure
avatars. In addition, apps with cache can save up to 80% stable resources rather than relying on only the browsers. In
requests. With cache enabled, the number of HTTP(S) re- a sense, the resources are configured at finer granularity so
quests in both native apps and Web apps can be significantly a lot of duplicate transfers are avoided.
reduced. Response Time. Compared to the other three operations,
Traffic Volume. The native app of Weibo consumes much the response time of GET operations should be more com-
more network traffic than the Web app because it requests plex, as the smartphone needs to fetch a number of resources
to download extra users’ avatars. We have mentioned that from the server and thus requires more network bandwidth.
the Web app of Zhihu fetches all friends’ avatars. However, The Web apps rely on the browser, where only limited num-
we are surprised to find that the Web app consumes less ber of simultaneous connections are available. For example,
network traffic than the native app without cache. We in FireFox and Chrome browser, only 6 concurrent connec-
explore the raw traffic data, and find that the native app tions are allowed per domain [15]. In contrast, the native
fetches avatars of high-quality portraits. In contrast, the Web apps can fully use the TCP connection pool with the sup-
app fetches avatars of low-quality portraits. Generally, the ports of the underlying OS (e.g., by AndroidHttpClient
cache mechanism in GET operations is quite efficient. For API) or other third-party libraries such as Volley [22].
most features, both native apps and Web apps with cache The performance of “with-cache” is always better than that
enabled can consume less network traffic. The native app of “without cache”. In addition, sometimes the performance
of Google Map can save as much as 90.9% traffic volume of Web apps with cache can be quite close to that of
when the cache is enabled. However, the native app of native apps when the cache is enabled. In particular, the
Baidu Map disables cache for most resources, so the traffic Web app of Maoyan movie (with cache) reduces a lot of
volume reduces quite marginally. When we consider the response time, and performs better than the native app for
network traffic in the with-cache situation, it is surprising the box_office feature (almost 85% improvement).

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 10

Energy Consumption. On the whole, native apps con- 6.2 Impact of Network Performance
sume less energy than Web apps for most features, but the Generally, the native apps expectedly perform better than
differences become smaller when cache is enabled. With the Web apps, but proportional exceptions cannot be ig-
cache enabled. Both native apps and Web apps send less nored. Indeed, the performance variety can significantly
requests, and reduce the cost of parsing resources, resulting derive from the programming languages (i.e., Java V.S.
in less energy consumption. For example, some pictures HTML/JavaScript), but it is out of the topic of this arti-
are loaded into the memory rather than decoded just-in- cle. Instead, we focus on the factors that may impact the
time. In the meantime, there are some features where native network performance, i.e., the connection management, the
apps consume more energy than Web apps. For example, libraries, and the protocols.
the native app of DianPing frequently records the state of
smartphones and uploads data to the remote servers. Such 6.2.1 Impact of Connection Management
a process would consume more energy. Another example First, we focus on the network connections management of
is the goods_detail feature in Amazon Shopping. The the smartphone, as it determines how many resources can
native app employs HTTPS rather than HTTP as the com- be used by the apps, and could be a main factor affecting
munication protocol, which introduces extra handshaking performance. We employ the techniques proposed in [16]
phase and data decryption, thus leading to more energy to analyze the TCP connections from the WireShark logs.
consumption [20]. As mentioned above, the Web apps can use at most
6 concurrent TCP connections per domain, while the
6 E XPLORING THE F INDINGS native apps can fully explore the TCP connections to
The preceding results evidence that Web apps have a con- reach better performance. For example, by using the
siderable proportion of performance advantages than native AndroidHttpClient API and other system-level libraries
apps. We need to find out why or why not the Web apps’ such as android.net.http, org.apache.http, and
performance can reach or even exceed native apps. Consid- java.net.Socket, or using third-party libraries such
ering the architectural comparison, we organize the findings as Volley [22] (20 concurrent connections at maximum)
and explore the possible reasons from the two aspects of the and android-async-http library [24] (10 concurrent
smartphone, cache performance and network performance. connections at maiximum), developers can configure the
“Connection:Keep-Alive” in request header or set
6.1 Impact of Cache Performance “setReuseAddress(true)” to allocate more sockets. In
It is clear that cache mechanism can improve the perfor- practice, we observe that the number of concurrent TCP
mance of both native apps and Web apps, for the GET connections of native apps can be quite flexible (varying
operations. from 10 to 20), but always significantly more than the
The native apps usually transfer less data than Web apps, browsers.
as the number and size of downloaded resources can be However, although the TCP connections cannot be ex-
programmatically controlled by developers. The native app panded due to the browser constraints, it is argued that the
developers can configure which resources should be cached reuse of connections can be a promising option [14]. For
locally by themselves, and define when the resources need some features provided by Facebook and Google Map, the
to be updated. A lot of resources such as icons, pictures, WireShark traces indicate that the Web apps can reuse exist-
and layout configurations, are usually stored at the client ing TCP connections to reduce the latency of fetching new
side. We observe that the cache mechanisms are disabled resource objects, rather than initializing a new connection. In
for many resources in Baidu, as the app intends to show the this way, the performance such as response time and energy
latest data. could be improved. However, it requires the server-side
The cache mechanism of Web apps completely re- protocol support to enable the reuse of TCP connections,
lies on the browser. However, according to our previous e.g., both Google and Facebook enable SPDY [25].
work [6], the browser cache mechanism is far from satis-
fying for mobile Web apps, due to conservative expiration 6.2.2 Impact of Transfer Protocols
time, redundant transfer, and same content with differ- From our observation, the choice of protocol can be quite
ent URLs that are dynamically generated by JavaScript. diverse among service providers in their apps. Three
We examine the cache configurations such as Cache- providers, i.e., Facebook, Google Map, and Douban, use HTTPS
Control field, Expires field, Etag field of the Web apps. in both native apps and Web apps. The providers such as
Most of the Web apps configure the HTML page with Weibo, Dianping use the HTTP in both native apps and Web
“Cache-Control:no-cache(store)”. Such configura- apps. In contrast, the Amazon and Zhihu use HTTPS in their
tion may be too coarse-grained, as all resources in the main native apps, but choose HTTP in the Web apps. Hence, we
HTML body have to be re-fetched even though they are still focus on these two service providers.
valid on the client side. When the cache policies are care- We find that the Web apps can beat the native apps
fully configured at finer granularity, e.g., for different texts for these two service providers in almost 36.6% genres.
and images, the performance of Web apps can surprisingly More specifically, the Web apps can beat native apps in
reach or even exceed that of native apps. For example, the almost all PUT, POST, and DELETE operations. It implies
search_a_person feature in Facebook, the Good_Detail that the HTTPS can introduce overhead in some cases. When
feature in Amazon Shopping, and the View_checkboard exploring the traces, we find that the size of HTTP messages
feature in DianPing configure different expiration time for is usually not very large, i.e., less than 50 KB. From the trace
the icons, small images, and main HTML body, respectively. of Fiddler and WireShark, such overhead mainly comes
In this way, only the need-to-update resources are down- from the SSL handshake phase. The overhead ranging from
loaded when the Web apps are revisited. 4.32 KB to 11 KB (mean: 8.11 KB, median: 7.4 KB) depends

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 11

on several factors including the certificate size. In addition, the browser can reuse TCP connections of previously
HTTPS cost extra energy for message encryption and de- requested resources to avoid the initialization cost for a
cryption. In this way, we can infer that, for small-size new request resource, and the performance seems to be
message, the HTTPS can introduce more network payload, improved. Recently emerging protocols such as SPDY [25]
especially the handshake-phase overhead. and HTTP/2 [30] support the reuse of existing connections.
In addition, we are aware that a lot of popular Web service
providers and browsers (Chrome, FireFox, and Safari) begin
7 I MPLICATIONS AND S UGGESTIONS to employ these new protocols.
Based on the findings and analysis, we then move to RQ3,
i.e., What implications can be leveraged to make Web apps • Avoiding unnecessary complexities. We observe that
and native apps provide comparable performance? We make sometimes the design and implementation of both kinds
analysis to understand the bottlenecks and suggest some of apps are not perfect enough so that both could be
recommendations to improve the performance of mobile improved. For some POST, PUT, and DELETE opera-
apps on smartphones, e.g., making the Web apps reach tions, the native apps may perform more poorly than
comparable performance with native apps. Web apps, e.g., the collect_goods feature in Ama-
zon, the delete_timeline feature in Weibo, and the
•Supporting fine-granularity cache management. delete_cart feature in Jd. The reason is that Web ser-
Undoubtedly, the size of resources fetched from the vices providers enforce their APIs to return extra data for
Web servers significantly impacts the performance of client- refreshing the user interfaces. However, such response data
side apps. The service providers and the app developers may not be always necessary. Web service providers should
need to determine more carefully which resource objects give the Web developers alternatives to programmatically
should be fetched when the app is revisited. The native filter the response data. Besides, in the Web app of Jd, the
apps provide developers very flexible options to configure add_goods_to_cart feature receives the same data in the
the cached resources, while the Web apps heavily rely “cart” and “cartJson” fields. Such unnecessary redundancy
on the browser. However, the cache management of Web should be carefully avoided.
apps is not satisfactory when requesting Web services [26], •Unifying RESTful API semantics. In some cases, it
due to the too coarse-grained configurations of the whole is observed that RESTful Web services have different
HTML body (Cache-Control:no-cache(store)). For semantics for native apps and Web apps given the same
the cases when Web apps beat the native apps (i.e., the feature. It indicates that service providers have to maintain
Maoyan), we find that the cache policies of resources different server-side support for the two client-side apps,
are finely configured at the object level. A substantial although the apps access the same back-end logic and
proportion of resources is rather static, while quite few vary data. When the app developers need to implement both
through different visits. Such a fine-grained configuration clients, they have to understand different server-side API
is somewhat similar to the native apps. To support such semantics, respectively. Hence, service providers had better
fine-grained configuration, both Web service providers and uniformly organize the RESTful APIs to reduce the cost of
Web apps developers should focus on the cache mechanism, development and maintenance for developers.
since better cache performance requires the cooperation
of client-side code, browser runtime, and server-side
support. Some recent features such as HTML5 AppCache
and localstorage can be leveraged to configure the
8 T HREATS TO VALIDITY
expiration time for stable resources. Besides the new We conducted our empirical analysis of only 328 distinct
features, another solution that can be applied to existing Web services from 12 typical service providers. The se-
Web apps is to separate the HTML into two parts, in order lected service providers cover various domains such as Map,
to load the changing resources dynamically and add them Social Network, e-Commerce, and Entertainment. In
to the page on demand. We have made several efforts on addition, these service providers come from both global (i.e.,
improving the cache management of mobile Web apps. [27], Google, Facebook, Amazon) and local (i.e., Baidu, Jd, Zhihu, and
[28] Dianping). In this way, the diversity of service providers can
reduce the selection bias of our empirical study. Our es-
•Reducing HTTPS overhead. Although the HTTPS tablished experimental framework and trace-based analysis
protocol becomes popular, the additional overhead of approach can be generalized to other Web services.
data traffic should be addressed, especially for those Nevertheless, considerable care and attention should be
small-sized HTTP requests. To make the tradeoff between addressed to ensure the rigor of this measurement study.
security and performance, some possible approaches can However, as with any chosen research methodology, it is
be leveraged to mitigate the overhead. We suggest that hardly without limitations. In this section, we discuss the
the providers should use lightweight SSL handshakes. potential threats for validating our results.
Specifically, Session Identifier and Session Ticket can be
used to resuming previously terminated SSL sessions
with low handshake overheads [29] [20]. Session ticket is 8.1 Performance and User Experience
preferred because the SSL state is stored at the client side. Our measurement study in this article mainly focuses on the
performance at the network level and energy consumption
• Exploring connection reuse for Web apps. The Web of native apps and Web apps, given the same requested
apps may suffer from the limited number of concurrent functionality. We employ the number of requests, traffic
TCP connections. However, in some cases, we find that volume, and response time to quantify the performance.

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 12

Indeed, the performance of apps has much broader mean- it is reported that the HTTP processing is much better on iOS
ings that are usually related to user experience, including platform than that on other platforms [16]. Our approach
look and feel, interaction patterns, context awareness, per- relies on a proxy intercepting HTTP-level traces for Android
sonalization, and so on. However, many aspects of user devices. However, this tool cannot work for iOS devices.
experience cannot be directly characterized by quantitative The selected browser used in our experiment is only
metrics. Therefore, the principle of our study is to compare Google Chrome, as Chrome is quite popular and its kernel
the relatively objective metrics when the native apps and takes into account cross-platform issues. It is quite possible
Web apps executing the same user-perceived functionalities that different browsers may perform variously for the same
that have exactly the same contents and similar UIs dis- Web app. To overcome this threat, we can reproduce our
played to users on native apps and Web apps. experiments on the combinations of different versions of
In addition, native apps may implement a certain smart- Android platforms and different Web browsers such as
ness such as context awareness and energy awareness, Chrome, FireFox, and Opera.
making the comparison unfair. To alleviate such an issue,
we ensure a stable context for the smartphones which are
connected to a high-speed WiFi and the battery energy 9 RELATED WORK
is always at 100% level (with adaptor charging). Such a In this article, we study the performance of native apps
stable context could disable the smartness of native apps and Web apps considering the same functionalities deliv-
to some extent but may not totally remove it. We plan to ered through Web services. Web services research on desk-
comprehensively study the smartness of native apps and top computing has been well surveyed by a lot of litera-
their influences on user experience in our future work. tures [31] [32]. With the wide adoption of mobile devices
such as smartphones and tablet computers, the application
8.2 Generalization of Findings context of Web services has evolved from the desktop
computing to the mobile computing. However, research on
The most considerable concern of our empirical study is the
the usage of Web services on mobile devices is just at the
validity of findings on the native and Web apps from other
beginning.
service providers that are not covered in this article. It is
true that the vendor-specific implementation of apps can Srirama et al. [33] and Wagh et al. [34] focused on the
have impacts on the performance. However, our results can Web services provisioning techniques for mobile devices.
be meaningful for end-users to select the proper channel Guinard et al. [35] suggested using Device Profile for Web
for accessing the services, and for developers to fix the Services (DPWS) and REST to easily integrate physical
potential imperfect designs or even performance bugs, such devices into existing enterprise information systems as Web
as redundant response, coarse-grained cache configurations, service providers. However, it is much more common for
and so on. mobile devices acting as Web services consumers instead of
Web services providers.
Additionally, some of our results can be generalized
to the service providers when they design their apps. For Although Tekli et al. [36] focused on SOAP performance
example, we observe that the fine-granularity cache config- enhancement, RESTful Web services are considered to be
uration of objects in Web apps can significantly improve a more promising solution on mobile devices that have
the performance, especially for reducing the payloads of relatively limited computation resources. In practice, it is
large-size but rather stable resources such as HTML body reported that most major service providers have adopted
and images. Instead of using the default heuristic cache RESTful fashion [34] [37]. Wagh et al. [34] made a compar-
configuration, the Web apps distinguish the configurations ison study on the performance evaluation on handheld de-
of the images of goods and those of logos, footers, and icons. vices. Arroqui et al. [38] showed that RESTful Web Services
improve the efficiency of data transfer for mobile devices.
Hamad et al. [39] showed that RESTful Web services con-
8.3 Versions of Apps sumed less message size and response time than SOAP Web
Another threat for reproducing our results is the changing services for mobile devices. Fokaefs et al. [40], KamalDldin
versions for apps. To overcome this threat, we download et al. [41], and Mizouni et al. [42] focused on the performance
the .apk files of the versions of native apps evaluated in of Web services on mobile platforms, and demonstrated that
our article. Practitioners could use these files to reproduce the RESTful pattern is much more suitable for resources-
the experiments described in this article. However, the Web constrained mobile devices.
apps cannot be treated in the same way. It is known that Due to the benefit of cross-platform accessibility by
the Web apps always deliver the latest version to end-users. Web apps, some efforts have been made to improve the
Although we can download the .html files of Web apps performance. Jamal et al. [43] used a cloud-based proxy and
and request them from our server, some dynamic resources cache mechanism to enhance the performance of RESTful
generated by JavaScript cannot be fully covered. In this way, Web services for mobile devices. Agabaov et al. [44] showed
it can be hard to completely reproduce the results of Web that Google presented Flywheel, an HTTP proxy service, to
apps. reduce the page size and response time to speed up Web
apps. Wang et al. [45] and our recent work [6] focused on
the cache mechanism to improve the performance of Web
8.4 Heterogeneous Platforms apps. Butkiewicz et al. [46] presented KLOTSKI, a system
What is worth mentioning is the platform heterogeneity. that prioritized the content to improve the user experiences.
Our experiments were made on only the Android platform. Compared to previous efforts, our research is driven
However, different platforms vary in network performance by an entirely different motivation. Either the native app
by using different libraries for the native apps. For example, and Web app has its own advantages and disadvantages.

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 13

Although there have been a long debate on native app R EFERENCES


and Web app on mobile devices [47], very few efforts have [1] T. Petsas, A. Papadogiannakis, M. Polychronakis, E. P. Markatos,
been made to understand the differences on using Web and T. Karagiannis, “Rise of the planet of the apps: a systematic
services. For the research community of Services computing, study of the mobile app ecosystem,” in Proceedings of the 2013
it is worth characterizing and understanding the different Internet Measurement Conference, IMC 2013, 2013, pp. 277–290.
[2] N. Serrano, J. Hernantes, and G. Gallardo, “Mobile web apps,”
resource usages of native apps and Web apps for consuming IEEE Software, vol. 30, no. 5, pp. 22–27, 2013.
given the same functionality. [3] “Hybrid apps,” https://en.wikipedia.org/wiki/Multi-channel
To the best of our knowledge, we make the first step app development.
in characterizing the performance of Web services on smart- [4] I. J. M. Ruiz, M. Nagappan, B. Adams, T. Berger, S. Dienst, and
A. E. Hassan, “Analyzing ad library updates in android apps,”
phones by taking into account different client-side apps. Our IEEE Software, vol. 33, no. 2, pp. 74–80, 2016.
derived knowledge on the cache management and network [5] H. Li, X. Lu, X. Liu, T. Xie, K. Bian, F. X. Lin, Q. Mei, and
optimization can help developers take full advantages of F. Feng, “Characterizing smartphone usage patterns from millions
of android users,” in Proceedings of the 2015 Internet Measurement
prior technologies to enhance the performance and user Conference, IMC 15, 2015, pp. 459–472.
experience of Web services on smartphones, for both native [6] Y. Ma, X. Liu, S. Zhang, R. Xiang, Y. Liu, and T. Xie, “Measurement
apps and Web apps. We can further help Web service and analysis of mobile web cache performance,” in Proceedings of
providers to make the tradeoffs of which types of apps to the 24th International Conference on World Wide Web, WWW 2015,
2015, pp. 691–701.
develop, and help the end-users select which type of apps as [7] Y. Liu, X. Liu, Y. Ma, Y. Liu, Z. Zheng, G. Huang, and M. B.
the access channel. In particular, we believe that Web service Blake, “Characterizing restful web services usage on smartphones:
providers can leverage our findings and explore the benefits A tale of native apps and web apps,” in Proceedings of 2015 IEEE
International Conference on Web Services, ICWS 2015, 2015, pp. 337–
gained from Web apps such as platform-neutrality, ease of 344.
deployment, and revenue by existing advertisements. [8] “Apache cordova, apache cordova is a platform for building
native mobile applications using html, css and javascript,” https:
//cordova.apache.org/.
10 C ONCLUSION [9] “manifoldjs, the simplest way to create hosted apps across plat-
forms and devices,” http://www.manifoldjs.com/.
This article has conducted the first comparison study on [10] “React native, a framework for building native apps using react,”
the performance of native apps and Web apps from the https://facebook.github.io/react-native/.
perspective of Web services, in terms of HTTP(S) requests, [11] “W3c, web services activity,” http://www.w3.org/2002/ws/.
traffic, round-trip time, and energy consumption. Although [12] C. Pautasso, O. Zimmermann, and F. Leymann, “Restful web
services vs. big’web services: making the right architectural de-
native apps usually performed better than Web apps, some cision,” in Proceedings of the 17th international conference on World
exceptional scenarios were observed. We further explored Wide Web. ACM, 2008, pp. 805–814.
the potential reasons and implications, so that developers [13] “Rfc 2616, hypertext transfer protocol,” http://www.w3.org/
can optimize their apps. Protocols/rfc2616/rfc2616.html.
[14] X. S. Wang, A. Balasubramanian, A. Krishnamurthy, and
In the future, there are two main ongoing efforts. First, D. Wetherall, “How speedy is SPDY?” in Proc. of NSDI’14, 2014,
we plan to make “What-If” measurement over the existing pp. 387–399.
dataset. We plan to compare the different protocols and [15] ——, “Demystifying page load performance with wprof,” in
Proceedings of the 10th USENIX Symposium on Networked Systems
cache management to explore the ideal upper bound of Design and Implementation, NSDI 2013, Lombard, IL, USA, April 2-5,
performance optimization. Such a measurement study can 2013, 2013, pp. 473–486.
help have more insights of Web services utilization on [16] A. A. Sani, Z. Tan, P. Washington, M. Chen, S. Agarwal, L. Zhong,
smartphones. and M. Zhang, “The wireless data drain of users, apps, & plat-
forms,” Mobile Computing and Communications Review, vol. 17,
Second, we plan to provide some browser-level opti- no. 4, pp. 15–28, 2013.
mizations for improving Web apps. There are three al- [17] F. Qian, Z. Wang, A. Gerber, Z. M. Mao, S. Sen, and O. Spatscheck,
ternatives to try against. The first one is made at the “Characterizing radio resource allocation for 3G networks,” in
Proc. of IMC’10, 2010, pp. 137–150.
resource-level for reducing data transfer of Web services. We [18] F. Xu, Y. Liu, Q. Li, and Y. Zhang, “V-edge: Fast self-constructive
aim to optimize cache configurations based on experiences power modeling of smartphones based on battery voltage dynam-
from our previous study [6], by reducing the redundant ics,” in Proceedings of the 10th USENIX Symposium on Networked
transferred resources and same-content resources. The sec- Systems Design and Implementation, NSDI 2013, Lombard, IL, USA,
April 2-5, 2013, 2013, pp. 43–55.
ond one is at the application-level. We aim to leverage [19] “Wireshark,” https://www.wireshark.org/.
the localStorage [48] and application caching mechanism [20] F. Qian, S. Sen, and O. Spatscheck, “Characterizing resource usage
(AppCache) [23], which take effect at fine granularity of for mobile web browsing,” in Proc. of MobiSys’14, 2014, pp. 218–
resource items rather than complete list of resources, hence 231.
[21] “Okhttp,” http://square.github.io/okhttp/.
some data could be loaded locally instead of transferred by [22] “Volley, an http library that makes networking for android apps
Web services. The last one is at the protocol-level. We also easier and most importantly, faster.” https://developer.android.
plan to evaluate the latest HTTP/2 [30] protocol in terms of com/training/volley/index.html.
[23] “Using the application cache,” https://developer.mozilla.org/
data compression and reuse of single TCP for the dataset of en-US/docs/Web/HTML/Using the application cache.
this article. [24] “Android asynchronous http client, a callback-based http client
library for android,” http://loopj.com/android-async-http/.
[25] “SPDY protocol - draft 3.” [Online]. Available: http://www.
ACKNOWLEDGMENT chromium.org/spdy/spdy-protocol/spdy-protocol-draft3
This work was supported by the National Key Re- [26] X. Liu, Y. Ma, Y. Liu, T. Xie, and G. Huang, “Demystifying the
search and Development Program of China under Grant imperfect client-side cache performance of mobile web browsing,”
IEEE Transactions on Mobile Computing, vol. 15, no. 9, pp. 2206–
No.2016YFB1000105, the Natural Science Foundation of China 2220, 2016.
(Grant No. 61370020 and 61421091), and the Microsoft-PKU [27] X. Liu, Y. Ma, S. Dong, Y. Liu, T. Xie, and G. Huang, “Rewap:
Joint Research Program. Yun Ma and Xuanzhe Liu are co- Reducing redundant transfers for mobile web browsing via app-
primary authors of this work. Xuanzhe Liu is the corresponding specific resource packaging,” IEEE Transactions on Mobile Comput-
author. ing, accepted to appear.

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2017.2756633, IEEE
Transactions on Mobile Computing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, XXXX 201X 14

[28] X. Liu, Y. Ma, X. Wang, Y. Liu, T. Xie, and G. Huang, “Swarovsky: Yun Ma is now a postdoctor in the School of
Optimizing resource loading for mobile web browsing,” IEEE Software of Tsinghua University, Beijing ,China.
Transactions on Mobile Computing, accepted to appear. He received his Ph.D degree in the School of
[29] “The ietf tls protocol version 1.0 .” https://www.ietf.org/rfc/ Electronics Engineering and Computer Science
of Peking University at 2017. His research inter-
rfc2246.txt. ests include Web engineering and mobile com-
[30] “Http/2, what is http/2?” http://http2.github.io/. puting.
[31] S. Wang, W. A. Higashino, M. Hayes, and M. A. Capretz, “Service
evolution patterns,” in Web Services (ICWS), 2014 IEEE International
Conference on. IEEE, 2014, pp. 201–208.
[32] O. Kondratyeva, A. Cavalli, N. Kushik, and N. Yevtushenko,
“Evaluating quality of web services: A short survey,” in Web
Services (ICWS), 2013 IEEE 20th International Conference on. IEEE,
2013, pp. 587–594.
[33] S. N. Srirama, M. Jarke, and W. Prinz, “Mobile web service provi-
sioning,” in Telecommunications, 2006. AICT-ICIW’06. International
Conference on Internet and Web Applications and Services/Advanced
International Conference on. IEEE, 2006, pp. 120–120.
[34] K. Wagh and R. Thool, “A comparative study of soap vs rest Xuanzhe Liu is an associate professor in the
web services provisioning techniques for mobile host,” Journal of School of Electronics Engineering and Com-
Information Engineering and Applications, vol. 2, no. 5, pp. 12–16, puter Science, Peking University, Beijing, China.
2012. He received his Ph.D degree in Peking Univer-
sity at 2009. His research interests are in the
[35] D. Guinard, V. Trifa, S. Karnouskos, P. Spiess, and D. Savio, area of services computing, mobile computing,
“Interacting with the soa-based internet of things: Discovery, and Web engineering.
query, selection, and on-demand provisioning of web services,”
IEEE T. Services Computing, vol. 3, no. 3, pp. 223–235, 2010.
[Online]. Available: http://doi.ieeecomputersociety.org/10.1109/
TSC.2010.3
[36] J. Tekli, E. Damiani, R. Chbeir, and G. Gianini, “SOAP
processing performance and enhancement,” IEEE T. Services
Computing, vol. 5, no. 3, pp. 387–403, 2012. [Online]. Available:
http://doi.ieeecomputersociety.org/10.1109/TSC.2011.11
[37] J. Huang, Q. Xu, B. Tiwana, Z. M. Mao, M. Zhang, and P. Bahl,
“Anatomizing application performance differences on smart-
phones,” in Proceedings of the 8th international conference on Mobile
systems, applications, and services. ACM, 2010, pp. 165–178. Yi Liu is now a Ph.D student in the School of
[38] M. Arroqui, C. Mateos, C. Machado, and A. Zunino, “Restful Electronics Engineering and Computer Science
web services improve the efficiency of data transfer of a whole- of Peking University, Beijing, China. His research
interests include mobile computing, network sys-
farm simulator accessed by android smartphones,” Computers and tems, and Web engineering.
Electronics in Agriculture, vol. 87, pp. 14–18, 2012.
[39] H. Hamad, M. Saad, and R. Abed, “Performance evaluation of
restful web services for mobile devices.” Int. Arab J. e-Technol.,
vol. 1, no. 3, pp. 72–78, 2010.
[40] M. Fokaefs, R. Mikhaiel, N. Tsantalis, E. Stroulia, and A. Lau, “An
empirical study on web service evolution,” in Web Services (ICWS),
2011 IEEE International Conference on. IEEE, 2011, pp. 49–56.
[41] K. Mohamed and D. Wijesekera, “Performance analysis of web
services on mobile devices,” in Proceedings of the 3rd International
Conference on Ambient Systems, Networks and Technologies (ANT
2012), the 9th International Conference on Mobile Web Information
Systems (MobiWIS-2012), Niagara Falls, Ontario, Canada, August 27-
29, 2012, 2012, pp. 744–751.
[42] R. Mizouni, M. A. Serhani, R. Dssouli, A. Benharref, and I. Taleb, Yunxin Liu is a lead researcher in Microsoft
Research Asia. He received his Ph.D. in com-
“Performance evaluation of mobile web services,” in Web Services puter science from Shanghai Jiao Tong Univer-
(ECOWS), 2011 Ninth IEEE European Conference on. IEEE, 2011, sity in 2011 (through the SJTU-MSRA joint PhD
pp. 184–191. program). His current research interests include
[43] S. Jamal and R. Deters, “Using a cloud-hosted proxy to support mobile systems and networking.
mobile consumers of restful services,” Procedia Computer Science,
vol. 5, pp. 625–632, 2011.
[44] V. Agababov, M. Buettner, V. Chudnovsky, M. Cogan, B. Green-
stein, S. McDaniel, M. Piatek, C. Scott, M. Welsh, and B. Yin,
“Flywheel: Google’s data compression proxy for the mobile web,”
in 12th USENIX Symposium on Networked Systems Design and Im-
plementation, NSDI 15, Oakland, CA, USA, May 4-6, 2015, 2015, pp.
367–380.
[45] X. S. Wang, A. Krishnamurthy, and D. Wetherall, “How much can
we micro-cache web pages?” in Proceedings of the 2014 Conference
on Internet Measurement Conference. ACM, 2014, pp. 249–256.
[46] M. Butkiewicz, D. Wang, Z. Wu, H. V. Madhyastha, and V. Sekar,
Gang Huang received his Ph.D degree from
“Klotski: Reprioritizing web content to improve user experience the School of Electronics Engineering and Com-
on mobile devices,” in 12th USENIX Symposium on Networked puter Science of Peking University, Beijing,
Systems Design and Implementation, NSDI 15, Oakland, CA, USA, China, 2003. He is now a full professor in Insti-
May 4-6, 2015, 2015, pp. 439–453. tute of Software, Peking University. His research
[47] Z. Hemel and E. Visser, “Declaratively programming the mobile interests are in the area of middleware of cloud
web with mobl,” ACM SIGPLAN Notices, vol. 46, no. 10, pp. 695– computing and mobile computing. He is a mem-
712, 2011. ber of IEEE.
[48] “The localstorage attribute of html5,” https://html.spec.whatwg.
org/#the-localstorage-attribute.

1536-1233 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

You might also like