0% found this document useful (0 votes)
122 views7 pages

A Fully Automated Deep Packet Inspection Verification System With Machine Learning

The document proposes a fully automated system for periodic verification and updating of Deep Packet Inspection (DPI) signatures using machine learning techniques. It describes automating mobile application traffic generation, signature verification, detection of undetected application flows, and suggestion of new signatures from the undetected flows.

Uploaded by

RAMU
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
122 views7 pages

A Fully Automated Deep Packet Inspection Verification System With Machine Learning

The document proposes a fully automated system for periodic verification and updating of Deep Packet Inspection (DPI) signatures using machine learning techniques. It describes automating mobile application traffic generation, signature verification, detection of undetected application flows, and suggestion of new signatures from the undetected flows.

Uploaded by

RAMU
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
You are on page 1/ 7

A Fully Automated Deep Packet Inspection

Verification System with Machine Learning


Uday Trivedi Munal Patel
Samsung R&D Institute, Bangalore (SRIB) Samsung R&D Institute, Bangalore (SRIB)
Bangalore, India Bangalore, India
[email protected] [email protected]

Abstract— Deep Packet Inspection (DPI) technique has to detect. Therefore, it is essential to frequently analyze and
become very important for traffic detection and resource update identification mechanism to detect flows of newer
management in core networks. Applications frequently update
versions of applications.
their version to add new features and/or to bypass firewall/DPI
systems. Thus, an accurate DPI system needs to periodically However, signature identification process is very rigorous
verify existing signatures and update them if required. The process [1]. For newer application versions, same procedure
manual task of application traffic generation and verification on must be repeated to detect and verify new signatures. Such a
multiple platforms is very tedious and error-prone. We propose a painstaking manual approach will not scale if it has to be
fully automated DPI verification system with machine learning applied individually to the growing range and number of
techniques for periodic DPI signature verification and update. diverse applications [2]. With growing number of mobile apps
Automated mobile application traffic generation is achieved by
open source tools like GUITAR and Appium. Signature
coming with local versions and their agile development life
verification and undetected application flow search is achieved by cycle, automation testing is the only way to go. This paper
AutoIT scripts. New signature patterns from undetected flows proposes a method which automates the entire process of
are suggested by Auto Signature module, completing full application traffic generation and signature verification.
signature verification and update cycle. Initial test results show
that our solution saves lot of man hours and detects signature B. Related Work
update in shortest possible time. Automation testing is widely used to perform effective and
Keywords-component; DPI, Deep Packet Inspection, GUI
fast testing of software products that have a long maintenance
Automation Testing, Automated Signature Generation life. Many proprietary and paid test automation tools provide
record and playback features that allow testers to interactively
I. INTRODUCTION record user actions and replay them back while testing,
The advent of 4G LTE is changing the landscape of data comparing actual results to those expected. This method
usage. LTE offers the promise of a pure, all-IP converged core requires little or no test script development. However, this
network with high data rate. With “all-IP” networks, lot of method may pose major reliability and maintainability
traffic and resource management issues like managing P2P problems. Relabeling a button or changing location of window
application traffic load, providing high QoS for priority flows may require the test to be re-recorded. For testing of web sites,
Headless browsers or solutions based on Selenium Web Driver
etc. can crop up. Network operators need to prioritize their
are normally used.
resources and adjust resource allocation schedule based on
network congestion and application traffic. Thus, they must Researchers [3] state that to cope with frequent upgrades
find effective ways to identify application traffic and apply of mobile devices and technologies, an elastic infrastructure to
different policies based on various parameters like network support large-scale test automation is needed. Research work
[4] on integrated test automation frameworks has proposed
status, application traffic, subscriber profile, time of day etc.
research tools at the system level that support mobile testing
Deep packet inspection (DPI) is a network packet on multiple heterogeneous platforms.
inspection system which examines packet payloads to identify
Mobile application test automation domain has seen lot of
traffic flows. DPI system uses application specific unique research. However, all of the solutions are mainly focused on
signatures to identify application traffic. The working DPI finding GUI or application specific development issues. As
solution must identify applications in real time. Thus, the DPI per our knowledge, this is the first time an attempt is made to
solution must to be fast, accurate and easy to upgrade. automate entire DPI signature verification process using
A. Need For Automation mobile app automation.
The mobile application development world is a dynamic Automatic signature generation has been a thoroughly
world. Developers frequently update their applications with researched domain. Want et al [5] used k-common substrings
encryption, obfuscation, anomalous use of well-known ports and multiple sequence alignment algorithm to generate regular
expressions with good accuracy. Researchers Wang et al., [6]
etc. Every new version makes them more stealthy and difficult
have proposed to use supervised machine learning models
automatically learned from the data sets as application requires rooting of device
signatures. We have attempted very simple signature We had following criteria for selecting automation tool:
suggestion algorithm which provides signature suggestion for  Automation support for Android/iOS/Both
HTTP, HTTPS, TCP and UDP flows.  Open source/proprietary/paid tool
II. DPI TEST AUTOMATION  Coordinate /image/Object based automation
We analyzed multiple tools for their strength and
Since DPI system can have hundreds of application limitations. Table 1 lists popular automation tools with these
signatures to be tested, we devised a way to fast-track the test details. We selected Appium and GUITAR for Android and
of those applications which have updated versions from iOS automation respectively.
previous test.
1) Automated App Traffic Generation on Android
We have written a python script which reads application
page on google play store and iOS app store and saves app Appium is an open source test automation tool for mobile
version information. In next run, if it finds change in app applications. It allows users to test all the three types of mobile
version, automated test is run immediately for that application applications: native, hybrid and mobile web. It also allows
for possible signature update. running the automated tests on actual devices, emulators and
simulators. Appium doesn’t modify app or need to even
We target following manual tasks to be automated: recompile the app on Android platform.
 Generating mobile application traffic for Android and
iOS platforms. Appium works on elements present on screen which can be
accessed with APIs provided by respective OS. Once an
 Generating application website traffic on mobile
element is selected, we can perform various gestures on it.
browser.
Appium works as a server between test case script and
 Verifying existing signatures and finding undetected
application running on mobile device. Initially test case should
application flows.
set capabilities of application so that server can launch
 Suggesting new signatures from undetected flows and
application in specific context. Appium explores capabilities
verifying those signatures with application traffic.
of Android to access view hierarchy represented with DOM
Our contribution in this paper is to automate the task of (Document Object Model). So while designing test cases, IDs
generating application traffic using open source tools, devising from DOM can be used and operations can be performed on it.
algorithms for verifying existing DPI signatures using Appium supports all major operation on touch screen like
machine learning techniques and suggesting new signatures touch, swipe, zoom, long press, etc.
from undetected flows. Following sub-sections discuss each of
Appium has few limitations as well. Appium does not do
these tasks in detail.
image comparison on mobile application GUI for performing
A. Automated Application Traffic Generation gestures. Appium requires developer image for iOS app
Test automation on mobile device poses many challenges. automation. Since DPI task involves working on 3rd party
Platforms: Android, iOS, Windows mobile, Tizen etc. No apps, we used GUITAR for iOS test automation.
single tool supports all platforms. 2) Automated App Traffic Generation on iOS
Screen size and resolution (Depth per Inch): Each mobile GUITAR is a GUI driven mobile test automation
device comes with different screen size and resolution. The framework. It performs test cases based on Image search.
automation script should be able to work with all of them. GUITAR framework searches images on attached mirrored
Table 1: Comparison between mobile automation tools mobile device window and performs gestures on it. GUITAR
Tool Strengths Limitations scripts are written with associated images. GUITAR provides
Monkey Coordinate based Supports only Android. image clipping tool to cut images while developing scripts.
Runner automation. Easy to Requires difference scripts As GUITAR works on application window on screen, we
setup. for different devices which need to mirror iOS mobile device to test system with control
can break with slight capabilities. iOS does not provide any means to control device
change in UI.
connected to a system. So we need to root iOS device and then
UI Integrated with Android Supports only Android.
Automator development IDE. Object Doesn’t provide support use desktop sharing tools like VNC to control device. As VNC
based automation for web-view, long press. with mobile data or WiFi creates significant delay in rendering
Monkey Open source, eclipse Limited features in free images, communication of VNC data is done through USB.
Talk based IDE. Supports iOS version. GUITAR supports multiple mobile devices at the same time so
& Android apps. that we can run VoIP apps on both devices for VoIP call/chat.
Appium Element based iOS Apps requires Many Signup and transactional flows require users to
[7] Navigation, Large Open source/debug binaries, No
recognize and enter captcha text to make sure that humans and
source community image based navigation
GUITAR Open Source, Supports Small open source
not automation scripts are operating. Out system is designed to
[8] iOS & Android apps, community, No Element use Tesseract library to fill captcha details by converting
Image based navigation based navigation, iOS image to text (OCR) and then using that text. Further, there are
some application flows where image to text is required while
checking results. For chat apps, when a message is sent to are categorized as relevant or irrelevant flow. Flows with SSL
other device, we can check if other device received same renegotiation do not have any signature pattern in payload.
message using image to text conversion. This case is handled by retrieving SSL session id from all
relevant SSL flows and marking SSL flows having same
B. Generating Application WebTraffic on Mobile Browser
session id as relevant flow. For UDP flows, port number in
Many services do not have any mobile applications and “include” and “exclude” list is used to mark relevant flows.
require browsers to operate. For many services, users may All remaining TCP flows are checked for flow size and
prefer browser based interface rather than mobile app as they marked as relevant flow if it is above threshold size.
don’t want to download/install multiple applications. The script then generates list of relevant flows. The
Considering these facts, it is necessary to include verification verification system uses a utility which reads detection
for mobile browser based traffic as well. As browser based database and provides DPI detection result. The verification
web interface tend to change a lot, conventional image or system gives list of false negative cases if relevant application
element based navigation may not provide complete coverage. flow is marked as HTTP/HTTPS/Undetected and not with the
To solve this problem, we came up with an approach where tested application id A. Similarly, verification system gives list
we utilize features of chrome browser along with some third of false positive cases if relevant application flow is marked as
party extensions and python script to access all links from any another application id B, instead of tested application id
webpage till nth level depth. This way we can generate traffic A. Relevant flows which are not detected properly are
with different scenarios. forwarded to automated signature generation module. Auto
AutoIt [9] is a scripting language designed for automating signature suggestion module gives unique application pattern
the Windows GUI and general scripting. It uses a combination suggestion for HTTP, HTTPS, TCP and UDP flows. For
of simulated keystrokes, mouse movement and window/ HTTP and HTTPS flows, it checks standard headers like
control manipulation. The AutoIT script sets chrome browser “Host”, “Referer”, “Client Hello server name” etc. For TCP
to emulate particular mobile device browser. After that, the and UDP flows, it finds unique byte pattern(s) by comparing
script opens base URL on browser. The browser opens base multiple similar flows.
URL web page. After the page is loaded, a JavaScript fetches Following fig. 1 shows complete architecture of DPI
all other sub-URLs from that page. The script hits few random automated verification system. CLI is used to run some
URLs from sub-URL list. There is a configuration option to configuration command for DPI module and to get detection
ignore certain URLs based on keywords. This process runs till result in terms of detection count for each application.
nth level of URL depth or for certain period of time.
Each flow from the PCAP files is classified into four
C. Automated Application Signature Verification categories: HTTP, SSL, TCP (other than HTTP and SSL) and
Signature verification task is very complex task due to the UDP. Each flow from these categories is processed separately
fact that an application/web page can have number of to identify relevant flows.
“application unrelated” links to 3rd party ad/stats sites. Also, 1. HTTP: HTTP flows can have unique signatures in HTTP
application can have 3rd party CDN links from which it may “Host”, “Referer”, “User-Agent” header and/or inside URI
fetch audio/video/ images related to its service. The string. The script checks “include list” and marks all flows
verification system must classify application specific relevant with matching pattern from “include list” in HTTP request
flows and check detection result for only those flows. If that packet as relevant flows. The script checks “exclude list” as
task is not done properly, the verification system can check well and marks all flows with matching pattern from “exclude
detection result for non-relevant flows and generate many list” as non-relevant flows. From relevant flows, each server
false “false positive“ alarms. For ex, many sites have option of IP is selected and all HTTP flows having that server IP are
sharing video to Facebook, Twitter etc. Many sites use scripts also considered as relevant flows.
from 3rd party apps for running site smoothly. Such flows are
irrelevant flows while testing an application signature.
We solve this problem by keeping “exclude list” and
“include list”. The script parses signature database and copies
unique application specific patterns into “include list”.
“Exclude list” has patterns from unrelated 3rd party
CDN/generic/Adv./stats sites.
Before running GUI test case, verification script starts
capturing PCAP file using tshark. Tshark is command line
extension of Wireshark network protocol analyzer. GUI script
opens test application on mobile and generates application
traffic by performing gestures on various features. After GUI
run, the script extracts relevant flows from saved PCAP file
using “include” and “exclude” lists. Based on pattern
matching from these lists, HTTP, SSL, TCP and UDP flows
doubleclick.com 30 3 (id x,y,z) 1
stats.com 6 1 0.066
We compute probability of a pattern being part of non-
relevant flow by following equation:

(1)
Where FlowsDetected is total flows detected with given
pattern, AppsDetected is total apps detected with given
pattern, FlowThreshold is flow threshold for marking the
pattern as non-relevant pattern (around 30) and AppThreshold
is total matching application threshold for marking the pattern
as non-relevant pattern (around 3).
As noted, unique pattern(s) related to an application can
be found from certain headers of HTTP/HTTPS packets. A
Figure 1: DPI Automated Verification System Architecture non-relevant HTTP/HTTPS flow from application X will be
2. SSL: SSL flows can have unique signature in “Client Hello” marked as either HTTP or HTTPS if detection for application
and “Server Hello” Packets. The patterns in include and X is not supported by system. This is obvious as system does
exclude list are used to mark flows as relevant and non- not have signature to detect flows of app X. If app X detection
relevant flows like HTTP. SSL renegotiation case is handled is supported, that non-relevant flow will be marked as
by retrieving session id from all relevant flows and marking application X. In both cases, verification system parses
flows having same session id as relevant flow. HTTP/HTTPS packet of such non-relevant flows and retrieves
3. UDP: For UDP flows, port number is used in “include list” patterns from standard headers and adds it in non-relevant
and “exclude list”. By default, port number <=1024 are flow table. If that same pattern is repeated in multiple
excluded from relevant flows as they are used for standard application traffic, the system concludes that it is generic flow
protocol related flows. If any port <= 1024 is required for and marks it as non-relevant flow.
particular application, it can be added in “include list”. UDP Initially, only few manual entries are added in non-
flows above minimum size are considered as relevant flows. relevant flow table. With more tests, this table is updated with
4. TCP (Other than SSL and UDP): All remaining TCP flows more patterns. Once flow and app count reaches given
are checked for flow size and marked as relevant flow if it is threshold, given pattern is marked as non-relevant pattern and
above threshold size. then onwards, flows having this pattern are marked as non-
relevant flows. It should be noted that Google, Facebook,
1) Machine Learning: Twitter traffic is also found in majority of application traffic.
Proposed verification system uses machine learning However, they are already added in “exclude list” and thus,
algorithms to learn more about possible weak signatures, false such traffic is not processed in this module.
positive/negative match occurrence etc. Following machine For ex. “doubleclick.com” pattern getting detected in
learning techniques are employed: multiple apps suggests that this pattern is part of a generic
a) Building non-relevant pattern database: flow and should be part of “exclude list”. With each
How does verification system know that a particular application test run, non-relevant pattern database grows and
undetected flow is relevant flow or advertisement/ statistics/ subsequent flow classification accuracy goes up.
Content Delivery Network (CDN) related flow? To correctly b) Generating signature rule priority change suggestion
classify relevant and non-relevant flows, a robust non-relevant for multiple rule matching scenario
pattern database is required. However, one can not build such For a given packet, there could be multiple patterns from
database manually as there are millions of such links and different rules matching at different offsets. DPI system has to
applications keep on adding new statistical/advertisement/ select one rule as matching rule among multiple matches. To
CDN links in newer versions. select one rule among many, each rule is assigned one priority
With machine learning technique, we build non-relevant value which represents the confidence of that rule.
pattern database from scratch by finding non-relevant For ex. Rule A has pattern “abc.com”. Rule B has pattern
application traffic getting detected in multiple application “xyz.com”. Both rules have same priority value. “Include list”
traffic. The system keeps track relative frequency of patterns for testing app A has pattern “abc.com”.
from HTTP/HTTPS packets and keeps updating it to conclude HTTP packet payload from app A is:
about a given undetected flow. Below table 2 shows sample GET \version HTTP 1.0\r\n
database for non-relevant flows. Host: abc.com\r\n
Referer: xyz.com\r\n\r\n
Table 2: Non-relevant pattern database
Pattern Flows Apps detected Prob. of non- In above example, patterns from both rules A and B will
detected relevant match at different offset. Since both rules have same priority,
DPI system may select rule A or B as final matching rule. session id/session ticket value, retrieves server name from
Suppose rule B is selected as final rule. Here, even though rule that flow and suggests server name as potential signature.
A’s pattern matches with pattern in “include list”, rule B was 3. TCP/UDP: For TCP/UDP flows on non-standard port,
wrongly selected as matching rule. Our algorithm will detect system needs to check multiple similar flows to find
and correct such cases. common bytes. For that, flow categorization is done as
If the flow of application A is identified as application B, first phase and then common patterns are found for each
and pattern of app A is found in “include list”, then the flow category. The system also gives useful information
algorithm suggests that we need to change rule priority in our like multiple UDP flows having same port, multiple
signature database such that when both patterns from rule of TCP/UDP flows having same destination IP and port etc.
app A and B are matched, priority should be given to With multiple UDP flows with same local port, if initial
application A. One such example of multiple rule matching flow is detected correctly, all subsequent flows can easily
involves 3rd party CDN, statistics and advertisement related be detected by using simple 2-tuple (Destination IP +
sites. Such sites are generic and found in flows of multiple Port) table match.
applications. Similarly, many apps have links to share post on 1) Flow categorization
Facebook, Twitter etc. Such flows have HTTP Referer as
The target application may generate one or more types of
testing application domain name (Ex. “xyz.com”) and HTTP
flows for various services. For ex., VoIP app can generate
Host as “facebook.com”. Patterns for such generic
flows for login, User search/add, chat, voice and video calls
applications should have lower priority than actual application
among other services. We would need to analyze all such
related patterns.
flows separately. If we try to compare a chat flow with a video
c) Finding almost matching patterns call flow, it is obvious that we can not find any potential
For certain application patterns, only 1 or 2 bytes might signature as those two flows will be having totally different
have changed in newer app version. Our algorithm finds such characteristics.
occurrences by finding almost matching pattern with longest To overcome this obstacle, one solution is to categorize the
subsequence matching algorithm. For ex., we have initial flows based on flow characteristics which would result in the
signature pattern: ”ABCDE” at offset X. During n th test, the formation of correct categories, wherein each category
system notices that pattern at offset X is “AFCDE” with corresponds to one type of the flow. Then after, flows in each
change at X+1 offset (‘F’ instead of ‘B’). The system marks category can be analyzed separately for signatures. Effective
this offset as vulnerable offset and suggests regular expression categorization would ensure correct signature suggestion.
“A[BF]CDE” as signature.
We have used statistical analysis for flow categorization.
During subsequent tests, if the flow is not detected, system However, instead of calculating statistical vector for known
checks bytes at vulnerable offset and finds longest matching protocols, we calculate statistical vector for all application
subsequence as new signature pattern “A*CDE”. It recognizes flows to put them into different categories which are created
that last time there was a change in this pattern/parameter so on-the-fly. For statistical vector generation, flow properties
there could be a further change at the same place. like average packet size of whole flow/uplink/downlink,
D. Automated Signature suggestion and Reverification standard deviation of packet size of whole
flow/uplink/downlink, Shannon entropy are used. K-means
Once AutoIT scripts find out relevant flows which are not
clustering algorithm is used for classifying the flow in
detected by DPI system, such flow information is forwarded to
available clusters. Once flows are categorized correctly, we
next phase where automated signature generation module is
can examine multiple flows from similar category to find
running. This module examines payload of such flows and
unique byte patterns among them.
suggests unique byte pattern as possible signatures.
Auto signature suggestion module has following sub modules 2) Finding unique byte pattern
for various flow classes. Byte sequences which appear at a constant offset in packets
1. HTTP: The script gets ”Host”, “Referer” and “User- across different flows are candidate for the signature set.
Agent” header values and removes patterns which are part To detect such signature, we line up flows of same category.
of “exclude list”. Remaining patterns are shown as Then we try to match bytes of 1 st to ith packet of each flow. If
possible signature patterns for given flows. there is a common byte pattern among the flows, the result
2. SSL/HTTPS: The script gets server name from SSL would be that pattern. If no common pattern is found, result
“Client Hello” packet and suggests that pattern as would be no match. Fig. 2 shows process of finding unique
potential signature. If “Client Hello” packet has no server byte pattern in given flows. Here, F n denotes flow number and
name and session id or session ticket value is non-zero, it Pn denotes packet number. For patterns on non-fixed offset, we
is SSL renegotiation case. In SSL renegotiation case, SSL use longest common substring algorithm.
flow does not have SSL negotiation parameters like server Verification test is run for failed applications again with
name etc. as those parameters were exchanged in some updated signatures to check detection result of previously
linked flow earlier. The script finds the linked flow using undetected relevant flows. If the test finds few more relevant
flows going undetected, same process is repeated again to get
new signature suggestion for detecting such flows. This process automation testing saves lot of testing cost. Table 4 shows few
is repeated until we get all test cases getting passed. auto signature suggests that we got for various applications.
III. TEST SETUP AND RESULT Table 3: Automated Verification System Performance
This section discusses our experimental test setup and results Application Issues Signature Relevant flow
for proposed method. Fig. 3 shows experimental test setup for Tested Found Suggested Accuracy
Month 1 40 12 6 80%
DPI automation verification testing. We run DPI engine on Month 2 75 15 5 83%
Cavium Octeon 68XX Multi-Core MIPS64 processor which Month 3 115 13 7 89%
runs 32 cores on a single chip to accelerate packet processing. Month 4 130 9 5 92%

Figure 2: Finding unique byte pattern for given category Figure 4: Cost for Manual Vs Automated test for multiple apps
Table 4: Auto Signature suggestions
HTTP bmimages, fishtank, isapi_solutions, jquery
HTTPS Mzstatic, shopify, thisisaim, truste, disqus
TCP “BitTorrent Protocol”, 0xd13a[ad]
UDP 0x000100242112a442 ,0x00060020

IV. CONCLUSION
An average application signature test can capture hundreds
of HTTP, HTTPS, TCP and UDP flows. Manually checking
detection result for all these flows in detection database is very
slow and error-prone. With automation along with machine
learning algorithms, we save lot of time in regression tests and
Figure 3: Test setup for DPI testing
find few false positive/negative cases which we tend to
All application specific signatures are loaded on DPI overlook in manual task. With every test run, our “exclude
engine. One Windows 7 system with 4 GB RAM and 2.0 GHz list” database grows and relevant flow accuracy increases.
clock was used as test system. Android/iOS Test mobiles were
connected to a Wi-Fi router which was connected to test Previous mobile test automation attempts were mostly
system. All packets to/from test mobile pass through test confined to generating mobile application traffic and testing
system. The test system was connected to DPI engine which the app by generating multiple gestures. We extend that idea
examined all packets and forwarded the traffic to destination. If to verify DPI signatures and suggest new signatures for
DPI engine detected any known signature, it saved flow undetected flows. We use many machine learning algorithms
information in detection table. which keep building non-relevant pattern database after
We have developed GUITAR test automation scripts for multiple runs, suggest signature priority change for multiple
300+ applications and Appium scripts for 20 apps. The matches, and suggest signature change for almost matching
verification system was tested for apps from categories like patterns. With that, the signature verification system can
streaming, P2P, VoIP etc. Table 3 shows verification system identify issues in existing signatures and update new signature
performance for various parameters. As the results indicate, in short time of new application version release.
verification system found multiple false positive/negative We successfully demonstrated that manual, error-prone DPI
detection issues in signature. The system also suggested signature verification task can be fully automated. This
HTTP, HTTPS signatures for undetected flows. The accuracy verification system can identify signature update cases quite
to find relevant flow kept increasing as more generic/3 rd party early and suggest changes. In future work, we would add more
app patterns were added in “Exclude list” by our algorithm. intelligence in machine learning algorithms, tighten detection
One important analysis is cost analysis - cost of testing of all false positive and false negative cases and increase the
(man months) vs. cost of script development and running the accuracy of automated verification system.
test. With multiple apps, manual testing time grows very fast. REFERENCES
However, automation testing requires only one time script
[1] U. Trivedi, "A self-learning stateful application identification method for
development for those apps and then running the test multiple Deep Packet Inspection," Computing Technology and Information
times. Fig. 4 shows number of apps vs. total time for both Management (ICCM), 2012 8th International Conference on, Seoul,
manual and automated testing. The results clearly indicate that 2012, pp. 416-421.
[2] ACAS: Automated Construction of Application Signatures - Patrick
Haffner, Subhabrata Sen, Oliver Spatscheck, Dongmei Wang
SIGCOMM’05 Workshops, August 22–26, 2005, Philadelphia, PA,
USA.
[3] Jerry Gao, Xiaoying Bai, Wei-Tek Tsai, Tadahiro Uehara, "Mobile
Application Testing: A Tutorial", Computer, vol.47, no. 2, pp. 46-55,
Feb. 2014, doi:10.1109/MC.2013.445
[4] H. Song et al., “An Integrated Test Automation Framework for Testing
on Heterogeneous Mobile Platforms,” Proc. 1st ACIS Int’l Symp.
Software and Network Eng., 2011, pp.141–145
[5] WangY,et al. Generating regular expression signatures for network
traffic classification in trusted network management.JNetwork Comput
Appl (2011), doi:10.1016/j. jnca.2011.03.017
[6] Wang Y, Xiang Y, Yu. SZ. An automatic application signature
construction system for unknown traffic. Concurrency and
Computation–Practice and Experience 2010;22(13):1927–44
[7] appium.io
[8] http://dev.naver.com/projects/guitar
[9] https://www.autoitscript.com/site/

You might also like