Downloaded from http://iranpaper.

ir
http://translate68.ir

Malware Detection and Classification Based on

Extraction of API Sequences

Dolly Uppal, Rakhi Sinha, Vishakha Mehra and Vinesh Jain

Department of Computer Engineering and Information Technology
Government Engineering College
Ajmer, India
[email protected], [email protected], [email protected], [email protected]

Abstract— With the substantial growth of IT sector in the the concept of pattern (byte code-signature) recognition. Anti-
21st century, the need for system security has also become Virus Scanners (AVS) follow traditional signature based
inevitable. While the developments in the IT sector have detection method to identify malware. Signatures are the
innumerable advantages but attacks on websites and computer sequences of bytes existent in the database. Although
systems are also increasing relatively. One such attack is zero day signature based examination can discern malware by yielding
malware attack which poses a great challenge for the security low false positive rates but this technique turns out to be
testers. The malware pen testers can use bypass techniques like inefficient for malware whose signature are not listed in the
Compression, Code obfuscation and Encryption techniques to database. Also, once should be well versed with the minutiae
easily deceive present day Antivirus Scanners. This paper
of software for static analysis, which is not usually possible.
elucidates a novel malware identification approach based on
Malware detection using static analysis can be easily
extracting unique aspects of API sequences. The proposed
feature selection method based on N grams and odds ratio circumvented by malware writers by making use of the
selection, capture unique and distinct API sequences from the obfuscation techniques. Thus, signature based detection fails
extracted API calls thereby increasing classification accuracy. to uncover zero day malware attack.
Next a model is built by the classification algorithms using active Dynamic Analysis is defined as the method of monitoring
machine learning techniques to categorize malicious and benign the working of an application by analyzing its runtime
files. performance. Day by day Malware are becoming more
advanced and vigilant enough so as to halt their execution as
Keywords— Malware, API sequence, API call gram
soon as they detect that their execution is simulated for
security based analysis using emulators or virtual
I. INTRODUCTION environment. In this way they can easily elude from the
MALWARE is a contraction of two words Malicious malware detection setups. Dynamic analysis can be preferred
Software which cause substantial security threats in computer over static analysis because this technique does not require a
world. Viruses, Worms, Trojans, Spyware, Adware and other deep technical understanding of the software. In addition to
similar attack mechanisms, all fall under the broad term this, dynamic analysis is also proficient in detecting malware
MALWARE. “Malware, in general speaking, are the set of whose signatures are not known. In present scenario, dynamic
instructions inserted in the application software, with the analysis is most commonly used to detect malware but it is not
intention to threaten the security of the system, without the adequate because there are some negative aspects of dynamic
knowledge of its owner.” Nowadays, whenever software analysis like some malware exhibit malevolent characteristics
(application program) runs, it is not sure that it will generate of self modification and this type of behavioral aspects cannot
the desired output. It may produce some harmful results or can be unearthed using this runtime analysis technique. In addition
damage the system. In other words, the system has been to this, malware which depend on certain trigger conditions
attacked by some malicious activity. According to Panda for their execution and can alter their behavior when these
Security Annual Report 2013[1], an average of 82000 new conditions are achieved, are not exposed by dynamic analysis
malware has been reported per day. Also, Kaspersky Lab because all possible execution paths cannot be probed in a
detects 200000 malicious programs every day in 2012[2]. The single pass.
statistics is large when it comes to system security.
A. Motivation
Malware attackers have made windows portable
executable (PE) as their most vulnerable prey for carrying out Zero day malware attacks by malware assailant are
malware based attacks. Therefore, our research work is increasing rapidly. Moreover, malware attackers have made
focused on analyzing portable executables. Malware analysis windows portable executable as their eminent prey. Code
can be accomplished in two ways- static analysis and dynamic obfuscation techniques, simply frauds antivirus scanners,
analysis. Static analysis can be defined as the method for thereby, decreasing the obfuscated malware detection rate.
examining software without executing it. This technique uses Therefore, our research work has focused on analyzing
portable executables by extracting aspects of API calls.

978-1-4799-3080-7/14/$31.00 2014
c IEEE 2337
Downloaded from http://iranpaper.ir
http://translate68.ir

Silvio Cesare and Yang Xiang in their work [9], has

B. Offering of the paper given a method to identify polymorphic malware through
This paper contributes in following spheres of research: flow-graphs. The isomorphism of graphs is calculated using
previously defined graph isomorphism graph matching
• A new feature selection method based on N grams
algorithms. Using these isomorphic flow-graphs, the
and odds ratio selection is used.
similarities among programs are calculated. If the similarity
• A model is built by the classification algorithms is high between given input program and malware, then the
using active machine learning techniques. variant of malware is identified. This method encompasses
performance rate on real time use like end host email gate-
• Dataset is collected from other sources, also which way etc. The proposed approaches by authors succeed in
are not openly accessible. identifying a huge variant of real polymorphic malware.
The paper is ordered as follows: Section II briefly The authors in [10] proposed a purely dynamic approach
discusses background and related work. Section III to classify benign and malicious samples. The traces of API
elaborates the proposed work followed by Section IV calls of portable executables are analyzed by executing them
mentioning the experimental results. Section V concludes on virtual machine. Tracing tools used here are HookMe
our paper. and Microsoft Detours. The analysis result gives accuracy of
above 97%.
II. BACKGROUND AND RELATED WORK
In the research of Jain and Nair [11], the emulated
Malware has been a challenge for malware defenders environment has been used for extracting API calls. They
since 1980’s. Initially there was no method to defend used QEMU for analysis. The samples are classified using
malware. Gradually, two techniques- signature based pattern recognition method.
detection and anomaly or behavior based detection have
been designed for malware detection [3]. Likewise, authors in [12], first disassembled the portable
executables for generating a control flow graph. Control
Signature based detection uses the technique of pattern flow graphs are used to generate basic blocks, among which
recognition in search of predefined signatures stored in a the relation is the show. In order to find the similarity in the
database. Signature based detection technique is commonly executables, the author uses graph isomorphism.
used up by AVS as their basic block. This technique does
not stand on its hundred percent as it is unable to detect In the study in 2012 [13], the authors worked for
malware whose signature is not present in its database developing a new approach against packers. Packing is a
record. On the other hand, behavior based detection technique for covering the malicious code in software
mechanism, as its name suggests, keeps an eye on the through obfuscation technique, which makes the software
behavior of suspected application for detection of malicious harmful. To identify the packer tool used for making the
activity. If something unusual happens, the suspected file is software malicious, the authors proposed a technique that
headed as malicious. For detection of unknown malware this generates the signature of every packed malware. The
detection mechanism stands good. dataset for analysis is divided into two parts. First part is
used for constructing the signature for the packer and
A thorough study by Robin Sharp [4], illustrated that determining the threshold while second part is created
day by day new variants of malware are being created which manually through packing each available data set. The first
has been a tough test for malicious code detectors. New part is the training dataset and second part is the testing data
categories of malware- encryption, oligomorphic, set.
polymorphic and metamorphic malwares are taken into
account [5]. The first method developed by malware Heejo, Kyoochang and Jeong in their research [14], does
attackers was encryption. Encrypted malware consists of not disassembled the executables. They used semantic
two parts- main body and decryptor. The main body is invariant approach. For differentiating malware and clean
attacked by the malware, but is no use without its decryptor. files, the author uses the concept of code graph, a
These types of malware are easily detectable by antivirus topological graph. The code graph is created by using call
engines. Then comes the oligomorphic malware- they are and jump non-sequential decision making technique, which
the advancement of encrypted malware, such that, only the results in the 67% accuracy of clean files.
decryptor is made different for each malware, which makes Authors in [15], presented a technique for analyzing a
the job of AVS a bit difficult. Polymorphic malware and malware visually by generating the image metrics out of
metamorphic malware are the outcomes of code obfuscation binary of a malware. This approach classifies the malware
techniques, which are harder to detect, as they easily bypass according to their families.
the AVS.
A handful of work is done in [16], for detection of
Study in [6][7] shows that, at present, Portable malicious code by giving the relation between malware
Executables of Windows are the prime target of malware. semantics and their API calls. For the complete malware
The structure of Win32 Portable Executable File Format [8] family the author defined a core signature. This signature
has been fully illustrated in an article by Matt Pietrek. The can define the unknown malware and also the advanced one
analysis of PE can be done through Application that lies under the same class of malware family.
Programming Interface (API) calls.

2338 2014 International Conference on Advances in Computing,Communications and Informatics (ICACCI)

Downloaded from http://iranpaper.ir
http://translate68.ir

Benign Malicious Machine learning

Dataset Dataset techniques and Data
Mining Methods
Executable Dataset

Monitor API Calls Training data

Benign
Apply proposed feature extraction
algorithm to capture distinct API Classify
sequences
Malware

Stage 1 Stage 2

Fig.1 Proposed Malware Identification Framework

The drawback of this approach was only that it does not

work for packed malware. B. Select the distinct API sequences
This step is accomplished in two parts; generate the call
In this research we proposed a novel approach to grams and calculate the odds ratio of each gram and
determine maliciousness in an input sample. First, a pre- generate feature vector.
modeled program is used to keep track of the execution of
the input sample to capture the API calls. Then the next step The API sequences received in the above step cannot be
is to extract the API sequences from these API calls. We used for classification purpose. So we proposed feature
proposed an algorithm based on call grams and odds ratio selection algorithm based on call grams and odds ratio
selection to extract the distinct API sequences which are selection algorithm.
then fed as input to the classifiers to effectively categorize
1) Generate the call grams: First, we generate the calls-
the malicious and benign samples. Classification model is
built by making use of machine learning algorithms: Naïve grams from the API calls retrieved in step A. 1 API-gram is
Bayes, Random Forest, Decision Tree, Support Vector generated by using API call sequences obtained in step A
machine (SVM). and by repeatedly performing sliding window operation we
can get two, three and up to six API-grams.. Our
III. PROPOSED WORK experimental analysis shows that 2 API-gram give better
accuracy than 1API-gram and further improvement in this
This section discusses the proposed methodology
accuracy is provided by 3 API-gram. Overall, 4 API-gram
comprising of two stages as delineated in Fig.1. Stage1
comprises of two steps; capture the API calls from benign outperform among all the API-grams.
and malicious samples and apply the feature selection 2) Calculate the Odds ratio of each call gram and
technique to API sequences to get distinct API sequences. generate feature vector: Next step is to select distinct and
Stage2 describes the use of machine learning algorithms and useful call-grams as we retrieved thousands of call-grams in
classification algorithms to determine whether an input
sample is malicious or benign. above step. Every obtained API Call-gram is considered as
a feature Xj and odds ratio for a feature X can be calculated
A. Monitor API Calls as
A pre-modeled program [17] is used to keep track of the Pr( X | c)(1 − Pr( X | c))
OR ( X ) = log
execution of input samples and capture the API calls from Pr( X | c)(1 − Pr( X | c)
these samples to get the API call sequences.

2014 International Conference on Advances in Computing,Communications and Informatics (ICACCI) 2339

Downloaded from http://iranpaper.ir
http://translate68.ir

Where IV. EXPERIMENTAL RESULTS

Pr(X|c) is the probability of feature X appearing in class c.
− A. Dataset
Pr( X | c ) is the probability of feature X appearing in c . Experimental investigation was conducted on the 120
c and c represent two classes, i.e. benign and malicious. malicious PE files and 150 benign samples. Benign dataset
includes various Application softwares like Decision
Next step is to rank the features according to the making software, Programming language software, Internet
calculated odds ratio and select the leading n features to Browser etc. Malicious dataset (refer TABLE II) was
form feature vector. collected from [18] including Trojans, Viruses, Rootkits,
and Worms.
C. Machine Learning Algorithms for classification TABLE I. Benign Dataset
Various machine learning algorithms were used to Content of Benign samples
construct the proposed model for classification, some of Program Category Number of samples
Utility-software 40
them being Naïve Bayes (the concept of Bayes conditional Application software 50
probability is used in this classification scheme), Random System software 35
Forest (multiple trees are used to obtain results which are Device Drivers 25
combined together for classification), Decision Tree
(training data is used to generate decision trees according to
attributes) and SVM (the nominal attributes are converted to TABLE II. Malicious Dataset
binary ones using this algorithm). Content of Malicious samples

Distinct API sequences were drawn out using proposed Program Category Number of samples
feature selection algorithm and were fed as input to all these Trojan 35
classification algorithms. These classifiers necessitate the Worm 30
use of supervised learning in order to train them, so we have Virus 30
used K-fold cross validation with all these classification Rootkit 25
algorithms for examining and producing independent
dataset. In K-fold cross validation, thorough training and
testing was performed for different values of K and it can be B. Evaluation Metrics
deduced from the findings that error estimation is best in
Evaluation Metrics can also be constructed by finding
case of K=10 that is, for 10 folds(implies that out of the
TP (True Positive), TN (True Negative), FP (False Positive)
total data, one tenth data is utilized for testing and nine-
and FN (False Negative) where True Positives is all the
tenth data is utilized for training). The performance of all
malware samples recognized correctly as malware, True
the applied classification algorithms after applying K-fold
Negatives is all the benign samples recognized correctly as
cross validation method is shown in Fig 2 and the essential
benign, False Positives is all the benign samples recognized
inference is that SVM provides best accuracy among all the
incorrectly as malware and False Negatives is all the
remaining classifiers.
malware samples recognized incorrectly as benign.
Accuracy (Acc) is the extent to which malware and
benign samples are recognized correctly and can be
calculated as
Acc = (TP + TN) / (TP + TN + FP + FN)
Accuracy of discussed algorithms is shown in Fig.3

C. Analysis Results
In this experimental research, designed program captures
the API calls from the input sample to get the API
sequences. Next, the proposed feature selection algorithm
extracts the distinct set of API sequences. Then, these API
sequences fed as input to classification algorithms to
categorize malicious samples from benign samples. Our
experimental analysis inferred that SVM output the best
results among all classification algorithms.
A comparison of the accuracy of proposed method to
other malware detection methods is listed in Table III.
Fig.2 Performance of four different algorithms using k
cross validation (k=2 to 10)

2340 2014 International Conference on Advances in Computing,Communications and Informatics (ICACCI)

Downloaded from http://iranpaper.ir
http://translate68.ir

TABLE III. Comparison of Various Malware Detection

Methods
Authors Approach Aspects ACC, DR,
TPR
Ammar Ahmed Combine static Combine static Not specified
E. Elhadi et and dynamic and dynamic
al.[19] analysis analysis with
approaches behavior and
static based
methods

Mojtaba Control Flow Highlight the ACC-97.77%

Eskandari et Graph based problem related DR-97.53%
al.[20] approach to large no. of
edges in CFG or
large no. of
aspects in vector
space model

Kyoochang Static Analysis Detection of 67%

Jeong et al.[14] malwares using
code graphs Fig.3 Bar graph showing highest achieved classification
accuracy of SMO in comparison to all the discussed
Faraz Ahmed et Dynamic Uses spatial and ACC-97% algorithms
al.[21] Analysis temporal aspects
of API calls
REFERENCES
Ronghua Tian Behavioral Examine the ACC-97% [1] Panda Labs-pandasecurity. www.pandalabs.pandasecurity.com/ .
et al.[10] Analysis behavioral [2] Kaspersky Security Lab Bulletin. www.kaspersky.in/ 2012.
aspects of API
[3] Vinod P. et al., “Survey on Malware Detection”,Workshop on
calls to
Comput. and Internet Security,2009,pp.74-79.
differentiate
malicious and [4] Robin sharp, “An Introduction to Malware”,Spring 2007.
benign files [5] B.B.Rad et al., “Camouflage in Malware: from Encryption to
Metamorphism”,Int. J. of Comput. Science and Network Security,
Fatemeh Graph mining Behavior graph DR-96.6% Volume 12, no. 8, August 2012.
Karbalaie et techniques for malware FP-3.4% [6] Mamoun Alazab et al., “Towards Understanding Malware Behaviour
al.[22] detection by the Extraction of API Calls”, Second Cybercrime and Trustworthy
Computing Workshop, 2010,pp.52-59.
Yoshiro Behavioral Behavior based DR-60%
Fukushima et Analysis identification of FP-0% [7] Parvez Faruki et al., “Mining Control Flow Graphs as API Call-
al.[23] suspicious Grams to Detect Portable Executable Malware”, In Proc. of the 5th
process International Conference on Security of Information and Networks ,
2012, pp.130-137.
Proposed Behavioral Malware ACC-98.5% [8] Matt Pietrek,” An In-Depth Look into the Win32 Portable Executable
method Analysis detection based File Format.” March 2002, issue of MSDN magazine.
on extraction of [9] S.Cesare and Y.Xiang, “A Fast Flowgraph Based Classification
distinct set of API System for Packed and Polymorphic Malware on the Endhost”, In
sequences Proc. of the 24th IEEE Int. Conf. on Advanced Information
Networking and Application ,IEEE,2010,pp.721-728.
[10] Ronghua Tian et al.,” Differentiating Malware from Cleanware Using
Behavioural Analysis”, In Proc. of the 3rd Int. Conf. on Security of
Inform. and Networks, SIN’10, IEEE, March 2010, pp. 23-30.
V. CONCLUSION [11] V. P. Nair et al.,”MEDUSA: MEtamorphic Malware Dynamic
In this research work, we proposed a novel approach for analysis Using Signature from API”, in 5th Int. Conf. on malicious
malware detection method with increased accuracy. We and unwanted software,ACM, 2010,pp.263-269.
proposed an algorithm based on call grams and odds ratio [12] R.T. Dullien and B. Universitaet, “Graph Based Comparison of
Executable Objects”, University of Technology, Florida, 2005.
selection to extract the distinct API sequences which are
[13] Smita Naval et al., “SPADE: Signature Based Packer Detection”, In
then fed as input to the classifiers to effectively categorize Proc. of the 1st Int. Conf. on Security of Internet of Things, 2012,
the malicious and benign samples. After analyzing the pp.96-101.
obtained results it can be concluded that SVM classification [14] Kyoochang et al., “Code Graph for Malware Detection”, In Int. Conf.
algorithm yields the best results among remaining on Inform. Netorking, ICOIN’08, 2008, pp 1-5.
classifiers. [15] KyoungSoo Han et al., “Malware Analysis Method using
Visualization of Binary Files”, In Proc. of the 2013 Research in
Adaptive and Convergent Systems, ACM, 2013,pp.317-321.
[16] V. Sathyanarayan et al., “Signature generation and detection of
malware families”, In Proc. Of the 13th Australasian Conf. on
Information Security and privacy, 2008,pp.336-349.

2014 International Conference on Advances in Computing,Communications and Informatics (ICACCI) 2341

Downloaded from http://iranpaper.ir
http://translate68.ir

[17] APIMonitorCorp.Win32APIMonitor[OL].2011.http://www.APImonit
or.com/
[18] VX-Heavens. Virus Collections (VXheavens)
http://vl.netlux.org/vl.php/
[19] Ammar Ahmed E. Elhadi et al.,” Malware Detection Based on Hybrid
Signature Behaviour Applicat. Programming Interface Call Graph”,
American J. of Appl. Sci., 2012, pp. 283-288.
[20] Mojtaba Eskandari and Sattar Hashemi, “Metamorphic Malware
Detection using Control Flow Graph Mining”, Int. J. of Comput. Sci.
and Network Security, IJCSNS, vol. 11,12, Dec. 2011.
[21] Faraz Ahmed, et al.,” Using Spatio-Temporal Information in API
Calls with Machine Learning Algorithms for Malware Detection”, In
Proc. of the 2nd ACM workshop on Security and artificial intelligence,
ACM, March 2009, pp. 55-62.
[22] Fatemeh Karbalaie et al., “Semantic Malware Detection by
Deploying Graph Mining”, In Proc. of the Student Int. Conf. for IT
Security for the next generation, University of HongKong and
Kasperky Academy,2010, pp. 1020-1025.
[23] Yoshiro Fukushima et al.,” A Behavior Based Malware Detection
Scheme for Avoiding False Positive”, In Proc. of 6th IEEE ICNP
Workshop on Secure Network Protocols, (NPSec), Oct., 2010, pp.79-
84.

2342 2014 International Conference on Advances in Computing,Communications and Informatics (ICACCI)