Machine Learning Approach to Predict Computer

Operating Systems Vulnerabilities

1st Freeh Alenezia,b 2nd Chris P. Tsokosb
a
Department of Mathematics Distinguished University Professor
Majmaah University b
Department of Mathematics and Statistics
Majmaah, 11952, Saudi Arabia University of South Florida
[email protected] Tampa, 33647, USA
[email protected]

Abstract—Information security is everyone’s concern. Com- vendors, the Common Vulnerability Scoring System (CVSS)
puter systems are used to store sensitive data. Any weakness is the commonly used scoring system. CVSS is under the
in their reliability and security makes them vulnerable. The custodial care of the Forum of Incident Response and Security
Common Vulnerability Scoring System (CVSS) is a commonly
used scoring system, which helps in knowing the severity of Teams (FIRST) [1]. CVSS provides software developers and
a software vulnerability. In this research, we show the effec- end-users with a framework to disseminate publicly published
tiveness of common machine learning algorithms in predicting software vulnerabilities and thereby assist them in prioritizing
the computer operating systems security using the published their efforts related to the most significant impacts [2]. There
vulnerability data in Common Vulnerabilities and Exposures are three versions of the CVSS, namely versions 1.0, 2.0, and
and National Vulnerability Database repositories. The Random
Forest algorithm has the best performance, compared to other 3.0., referred as CVSS v1, v2, and v3.
algorithms, in predicting the computer operating system vulner- The impact of security breaches as a result of OSes
ability severity levels based on precision, recall, and F-measure vulnerability exploitation could be unpredictably large. For
evaluation metrics. In addition, a predictive model was developed example, the exploitation of the Microsoft vulnerability (SMB
to predict whether a newly discovered computer operating system MS17–010 in May 2017), forced several UK health-care
vulnerability would allow attackers to cause denial of service to
the subject system. providers to cancel doctors’ appointments, including surgeries,
and FedEx international shippers were also hit, among others
Index Terms—computer, operating system, security, machine [3]. This attack is known as WannaCry which made Microsoft
leaning, data mining publish security updates for Windows XP, Windows 8, and
Windows Server 2003 [4].
I. I NTRODUCTION This research focuses on the numerical and textual details
Information security is a major concern of not only soft- of computer operating systems vulnerabilities. Therefore, the
ware developers but also end-users. Software reliability and primary contributions of this research are to:
security are important components requiring evaluation during 1) Develop a machine learning based predictive model to
the development process, and monitoring for as long as the classify computer operating systems vulnerabilities using
software is publicly available for users. Our high dependency their associated factors in the National Vulnerability
on software packages for managing simple daily routines, such Database (NVD). Five methods are employed to select
as storing our personal and financial information, require us to the most accurate analytical model,namely: Logistic Re-
trust such software. Small and large businesses use software gression, K nearest neighbors, Gaussian Naive Bayes,
packages to operate and store their data as well. Random forest, and Adaptive Boosting.
Software security breaches mostly happen when hackers 2) Develop a predictive model to classify whether a given
identify and exploit known and unknown software vulnera- computer operating system vulnerability would allow
bilities. The National Institute of Standards and Technology attackers to cause Denial of Service to the subject system.
(NIST), under the U.S. Department of Commerce, defines The paper is organized as follows, Section II describes the
system vulnerability as, "Weakness in an information system, related research. Section III presents the data and methodol-
system security procedures, internal controls, or implementa- ogy. Section IV presents results and discussion. Section V are
tion that could be exploited or triggered by a threat source". the conclusions and future work.
Efforts have been made to evaluate the vulnerabilities of
II. R ELATED R ESEARCH
systems, such as computer operating systems (OSes), to help
establish and maintain security policies, one of which is Recent efforts utilized CVSS to develop probabilistic and
developing a scoring system to classify each system vul- statistical models to predict network security. Kaluarachchi,
nerability. While scoring systems have been developed by Tsokos, and Rajasooriya, [5], proposed a stochastic model,
using a small schematic network system and structuring an
978-1-7281-4213-5/20/$31.00 © 2020 IEEE attack graph, that helps to measure the network security.

Authorized licensed use limited to: University College London. Downloaded on May 25,2020 at 13:36:47 UTC from IEEE Xplore. Restrictions apply.
The authors also proposed a statistical model to predict the III. DATA AND M ETHODOLOGY
expected path length and estimate, with a probability of 1, A. CVSS v2
the minimum number of steps to reach the target software in
the network. Pokhrel and Tsokos, [6], proposed a stochastic According to FIRST, the CVSS-SIG and others have iden-
predictive model, based on the Markovian process, to evaluate tified major issues with the CVSS v1. One of the main issue
the risk to the entire computer network. Pokhrel, Rodrigo, and is that it did not go through mass peer review across multiple
Tsokos, [7], proposed a time series based predictive model, organizations and industries, which led the team to develop
using linear and non-linear approaches, to predict the number and publish the CVSS v2, [16].
of vulnerabilities of a given operating system and thereby The vulnerability score under the CVSS v2 is the scope
assist vendors in monitoring their OSes. of this research
Operating Systems Reliability & Security
because it is easily accessible to all of
Alenezi & Tsokos
Moreover, several authors [8]–[12] showed the usefulness the disclosed vulnerabilities via CVEdetails, [17]. CVEdetails
and validity of machine learning techniques in the evaluation was developed by Serkan Özkan. It automatically collects
and Voelker study [7] because the OSVDB and the extracted large number of variables were not
of software vulnerabilities, such as predicting the number of vulnerability data from various sources as the NVD is the
vulnerabilities and the time length until exploitation, if they main source, and others as additional sources such as CWE,
available to them.

are exploitable. Exploit Database [18], Microsoft Security Bulletin, Metasploit,

Bozorgi, Saul, Savage, and Voelker [8] used the 4 OpenData among others, [19]. The Beautiful Soup and Scrapy, [20], [21],
and Methodology
Source Vulnerability Database (OSVDB) and the CVE libraries in Python were used to scrap the vulnerabilities of the
database to find how likely a vulnerability could be exploited Microsoft, Apple, and Linux OSes from 1999 to 2019.
4.1 CVSS v2
and how soon. They extracted a very large number of variables, The following schematic diagram shows the computation of
many of which are binary variables derived using a bag-of-The the
following vulnerability
schematic diagram shows base the score based on
computation of theCVSS v2. base score based on CVSS
vulnerability

words model representing whether certain words exist inv2. each

text field (i.e. name of attack types). Their data showed that Common Vulnerability Scoring System (CVSS)
73% of the vulnerabilities were classified as "exploited" which
contradicts the data collected in [11], [13], [14]. By using Base Temporal Environmental
linear support vector machines (SVMs), they were able to metrics metrics metrics

reach a false positive rate of 12.5% and approximately 90% =((0.6 · IM) + (0.4 · EM) - 1.5)· f(IM))
accuracy. where:
f(IM) = 0 if IM=0, 1.176 otherwise
Zhang, Caragea, and Ou [15] carried out some machine
learning algorithms, among other methods, using NVD to Exploitabilty metrics (EM) Impact metrics (IM)
= 20 · AV · AC · AU =10.41·(1–(1–CI)·(1–II)·(1–AI))
develop a predictive model that predicts time to next vulner-
ability. The software considered in their study were Linux Access Vector (AV): Confidentiality Impact (CI):
Network (N)= 1.0 Complete (C): 0.660
and Linux OSes and Mozilla web browser. The machine Adjacent network (A)= 0.646 Partial (P): 0.275
Requires local access (L)= 0.395 None (N): 0.0
learning algorithms were radial basis function (RBF) network,
Access Complexity (AC): Integrity Impact (II):
sequential minimal optimization (SMO), multilayer perceptron Low (L)= 0.71 Complete (C): 0.660
Medium (M)= 0.61 Partial (P): 0.275
(MLP), and simple logistic. The raw input variables were High (H)= 0.35 None (N): 0.0

software’s name, vulnerability’s published time, software’s Authentication (AU): Availability Impact (AI):
version, and software’s CVSS. Despite their claim that better Not required (N): 0.704
Single instance (S): 0.56
Complete (C): 0.660
Partial (P): 0.275
prediction capability can be achieved by transforming the Multiple instances (M): 0.45 None (N): 0.0

software’s version variable to be the distance between two Figure 1: The calculation of the vulnerability base score under the CVSS v2
Fig. 1: The calculation of the vulnerability
framework where it is eventually rounded to one decimal place
different versions, the false positive rates of their predictive base score under the CVSS v2 framework.
models are high (about 40%) with approximately accuracy
70% using SMO which is their best performance. According to FIRST, the CVSS-SIG and others have identified major issues with the CVSS v1.

Edkrantz and Said [11] studied the vulnerabilities One fromof the B.
mainComputer
issue is that operating systems:
it did not go through massMicrosoft, Apple,
peer review across and organizations
multiple Linux and
NVD up to May 2015. They applied Support Vectorindustries, Ma- which According
led the team totothe NetMarketShare,
develop and publish the CVSS [22],v2,as[51].
of March 2019,
chines (SVM), K-Nearest-Neighbors (KNN), Naive Bayes, and the operating system market shares are 87.45%, 9.73%, and
The VS under the CVSS v2 is the scope of this research because it is easily accessible to all
Random Forests ML algorithms to predict exploit status of 2.16% for Microsoft, Apple, and Linux OSes, respectively.
of the disclosed vulnerabilities via CVEdetails, [55]. CVEdetails was developed by Serkan Özkan. It
vulnerabilities.The variable space considered includes CVSS These represent 99.34% of the existing OSes. Therefore, they
scores and parameters, CWE variables, length of theautomatically
text arecollects vulnerability data from various sources as the NVD is the main source, and others
selected for this research. The following table shows
summary, words, vendors, and references. Considering the the versions
as additional sources such asofCWE, the Exploit
Microsoft,Database Apple, and Linux
[45], Microsoft Security OSes thatMetasploit,
Bulletin,
binary classification, the SVM with Radial Basis Function areincluded
among others, in this Soup
[35]. The Beautiful research.
and Scrapy, [39, 24], libraries in Python were used to scrap
(RBF) kernel has the best performance, among others, with Table I shows that Microsoft has the largest number of
prediction accuracy, precision, and recall of nearly 83%. They OSes compares with the others.8 Linux family has various OSes
also commented that their prediction accuracy is less than that were developed by different developers as Linux is an
the accuracy presented in Bozorgi, Saul, Savage, and Voelker open source tool that allows anyone to use it for operating
study [8] because the OSVDB and the extracted large number system development. On the other hand, Apple has the smallest
of variables were not available to them. number of OSes due to its early existence compared to others.

Authorized licensed use limited to: University College London. Downloaded on May 25,2020 at 13:36:47 UTC from IEEE Xplore. Restrictions apply.
 TABLE I: Computer operating systems considered in this research. TABLE II: The collected and computer factors for the vulnerability
CVE-2018-0751.
OS Vendor Family Factor Value Variable Value
Mac Os X Apple Apple CVE ID Number CVE-2018-0751 Days 299
Mac Os X Server Apple Apple Family Microsoft Vulnerability level low
Mac Os Apple Apple CVSS Score 3.6 Year 2018
Confidentiality Impact Partial Polarity 0.125
Debian Linux Debian Linux
Integrity Impact Partial Subjectivity 0.6875
Linux Kernel Linux Linux Availibility Impact None Frequency 1
Ubuntu Linux Canonical Linux Access Complexity Low Denial Of Service 0
Opensuse Opensuse Linux Authentication Not required Gained Access None
Enterprise Linux Redhat Linux NumAttacks 0 – –
Solaris SUN Linux
Fedora Fedoraproject Linux
Kernel Linux Linux The vulnerability (CVE-2018-0751) is a weakness in Mi-
Linux Kernel-rt Linux Linux crosoft products; Windows 10 and Windows 8.1 computer
Windows 7 Microsoft Microsoft operating systems. It was published with a summary text; "The
Windows Vista Microsoft Microsoft Windows Kernel API in Windows 8.1 and RT 8.1, Windows
Windows 10 Microsoft Microsoft Server 2012 and R2, Windows 10 Gold, 1511, 1607, 1703
Windows Xp Microsoft Microsoft and 1709, Windows Server 2016 and Windows Server, version
Windows 8.1 Microsoft Microsoft 1709 allows an elevation of privilege vulnerability due to
Windows 2000 Microsoft Microsoft the way the Kernel API enforces permissions, aka "Windows
Windows Microsoft Microsoft Elevation of Privilege Vulnerability". This CVE ID is unique
Windows 8 Microsoft Microsoft from CVE-2018-0752."
Windows 98 Microsoft Microsoft D. Machine Learning (ML)
Windows 95 Microsoft Microsoft
Due to the fact that NVD provides a venerability severity
levels, we select Logistic Regression (LR), K nearest neigh-
bors (kNN), Gaussian Naive Bayes (GNB), Random forest
C. Research Data
(RF), and Adaptive Boosting (AdaBoost) ML algorithms to
The research data is comprised of original data collected estimate computer operating system vulnerability level given
from the CVEdetails site. Additionally, we include the follow- its vulnerability description. Next, an overview of each method
ing data that are computed and collected using the research is presented.
original data: 1) Logistic Regression (LR): LR is a class determinate,
• Year: The year that the OSes vulnerability was published. which means each variable should have a single value for each
• Days: Number of days between the publish date and the class, where variables’ dependency doesn’t prevent applying it.
most recent update date (up to March 15, 2019) of the Logistic sigmoid function is used to ensure the model outcome
vulnerability. is bounded between zero and one. The analytical form of LR
• Frequency: An indicator variable; whether the vulnera- is given by:
bility affects single or multiple OSes.
• NumAttacks: Number of vulnerability types defined in
P (x)
Log = βT · x . (1)
the CVEdetails site. 1 − P (x)
• Family: The family of each OS (Table I) 2) K nearest neighbors (KNN): KNN is considered a simple
• Polarity: The polarity of the textual summary of the and intuitive algorithm. It stores all available cases from the
vulnerability; a score that ranges from [−1, 1]. training set, then for each testing data point with a predeter-
• Subjectivity: The subjectivity of the textual summary of mined K value, say K=k, for now, it chooses the k nearest data
the vulnerability; a score that ranges from [0, 1]. points from the training set, and classifies this data point to
Natural Language Processing (NLP) was used to compute the class that the majority of data points belong to. To avoid
the polarity and subjectivity of the summary text of each the problem of ties, it is practically recommended to choose
vulnerability. Note that we assume the polarity score is nearly an odd number for k, [23]. Therefore, KNN is a classification
0, where any score with negative or positive signs indicates method based on similarity measure (i.e. distance functions).
the negativity and positivity toward the description of the 3) Gaussian Naive Bayes (GNB): GNB is a classification
vulnerability. Also, we assume the subjectivity score is nearly algorithm that is based on Bayes rule (P (A|B) = P P(A∩B)
(B)
, for
0, where any score near 0 or 1 indicates the objectivity and any A and B events). GNB has a strong assumption that
subjectivity toward the description of the vulnerability. variables are independent given class label, and is given by:
For example, for the vulnerability (CVE-2018-0751) that n
was published in January 4, 2018, and its description is given
Y
P (xj |Cc ) = P (xji |Cc ) . (2)
by Table II. j=1

Authorized licensed use limited to: University College London. Downloaded on May 25,2020 at 13:36:47 UTC from IEEE Xplore. Restrictions apply.
 The general Bayes theorem in this regard is given by: TABLE III: Analytical form for each evaluation metric, where c is
Qn the total number of classes.
(i) (i) j=1 P (xji |c) · P (c)
P (c|x1 , x2 , ..., x(i)
n ) = (i) (i) (i)
. (3) Metric Analytical form
P (x1 , x2 , ..., xn )
Pc tpi +tni
In addition, GNB assumes normality for each continuous i=1 tpi +f pi +f ni +tni
variable which is hard to maintain in real data (i.e. vulnera- Average Accuracy (AA) c
bility), and therefore makes it has lower performance. Pc tpi
i=1 tpi +f pi
4) Random forest (RF): Breiman defined Random Forests Average Precision (AP ) c
as, "A random forest is a classifier consisting of a collection Pc tpi
i=1 tpi +f ni
of tree-structured classifiers {h(x, Θk ), k = 1, ...} where the Average Recall (AR) c
{Θk } are independent identically distributed random vectors AP ·AR
and each tree casts a unit vote for the most popular class Average F –score (AF –score) (1 + β 2 ) · (β 2 ·AP )+AR
at input x". By bootstrapping from the data, RF is a group
of classification trees, where the final decision is made after
considering the results from these trees. The following is the F–measure ( or F–score) is the harmonic average of precision
process of RF algorithm [24]–[26]: and recall where it balances between them, and considers
• Repeatedly, form a subset of the training data by drawing them as perfect values when it reaches 1. It is defined by
randomly with replacement (i.e. bootstrapping). (1 + β 2 ) · (β 2P·P·R . When β 2 = 1, it means the precision and
)+R
• Grow a classification tree out of each subset. recall are given the same weight when computing the F–score
• Randomly, sample a group of the risk variables at each and would be called F1 –score. These evaluation metrics can be
node of the tree, and use it to find the best split. generalized for multi-class classification except for the ROC
• The results are combined to form a final decision: ma- curve which is only for binary classification, [28], [29].
jority vote for a classification problem. In addition, the time (T ) that ML classification algorithms
5) Adaptive Boosting (AdaBoost): Boosting in machine consume to train and test the predictive models are included
learning is a way of combining several weak classifiers into in the previous metrics.
one strong classifier. AdaBoost was introduced by Freund and
Schapire, [27] in 1996. It is considered an iterative algorithm, IV. R ESULTS AND D ISCUSSIONS
where it starts from a weak classifier (i.e. low classification The CVSS v2 base score is classified into three severity
performance). Then apply it several times where each time the levels, namely Low (0-3.9), Medium (4-6.9), and High (7-10).
training data points receive a certain weigh. The final classifier The high level means the vulnerability has highest severity
(H(x)) consists of the linear combination of t weak classifiers, status. There were 16, 164 published vulnerabilities for the
given by: considered OSes (shown in Table I) by March 15, 2019. The
T
X vulnerability levels for each OS are presented in Fig. 2.
H(x) = Sign( αt ht (x)) . (4)
t=1

Each weak classifier will be given a weight, and will be given

a positive sign when it is accurate, minus sign when it is not
accurate (i.e. less than 50%), and zero when its accuracy is
50%.
E. Evaluation metrics
There are useful metrics using the confusion matrix, namely
precision, recall, receiver operating characteristic (ROC) curve,
and F–measure. Table III has the analytical forms of the
evaluation metrics considering multi-class classification.
Precision (P ) is defined as the number of correctly positive
attacks that are classified as positive attacks (tp) over the
number of correctly positive and negative attacks that are
classified as positive (tp + fb). Recall (R) is defined as the
number of correctly positive attacks that are classified as Fig. 2: Frequency of each vulnerability level within
positive attacks (tp) over the number of correctly positive that computer operating systems families
are classified as positive and negative (tp + fn), it provides an
insight of the effectiveness of the ML algorithm in correctly The Microsoft OSes have the largest number of high
classifying the positive attacks. The ROC curve is defined by vulnerabilities levels, followed by Linux and Apple OSes,
2 ( tp+f n + tp+f n ). It is used to evaluate the ML algorithm; respectively. The Linux OSes have the largest number of
1 tn tn

ability to avoid false positive and false negative classifications. medium venerabilities, followed by Microsoft and Apple

Authorized licensed use limited to: University College London. Downloaded on May 25,2020 at 13:36:47 UTC from IEEE Xplore. Restrictions apply.
OSes, respectively. The Apple OSes have the lowest number algorithm. Although the training and testing times for all
of low vulnerabilities compared to Microsoft and Linux OSes. ML classification algorithms are very small, the Adaboost
algorithm consumed more time than the others.
A. Vulnerability Classification
The 10-fold cross validation was also applied to the ML
The ML classification algorithms were ran to build predic- classification algorithms to investigate whether the classifier
tive models that classify a new vulnerability level based on overestimated the vulnerability levels. The box-plots of the
the vulnerability descriptions (risk factors). results appear in Fig. 3.
Newton-Raphson method was used to obtain approximate
maximum likelihood estimates (MLEs) of the coefficients (β̂j )
of the LR model in equation 1.
The Minkowski distance function and k=3 neighbors were
chosen in the KNN algorithm after an extensive search to
maximize the accuracy rate for classifying an OS vulnerability
level.
GNB assumes the variables in the present study are inde-
pendently and identically distributed by Normal distribution.
The MLEs of the model parameters are used to calculate the Fig. 3: 10-fold cross validation of the the ML classification algorithms.
conditional probabilities and class assignment is based on the
highest probability, [30]. It can be seen that LR, KNN, GNV, and RM have main-
RF algorithm was applied with T = 200 trees after an tained similar performance in classifying the vulnerability
extensive search to maximize the accuracy rate for classifying levels, whereas Adaboost steadily worsened during the 10-fold
severity level of an OS vulnerability. than randomly splitting the training and testing datasets. The
AdaBoost algorithm was applied with RF as a weak classi- box-plots of Fig. 3 do not indicate any outliers and all results
fier, which reveals a better performance with the vulnerability for each algorithm were around the average accuracy values.
data comparing to other weak classifiers such as Decision Tree The Adaboost classification algorithm clearly was over-fitting
and GNB algorithms. AdaBoost analytical model is given by in classifying the vulnerability levels. The average accuracy
equation (4) of the RF classification algorithm is nearly 94%, which is
Table IV presents the performance of the predictive models the highest among all algorithms, which suggests that it is
based on the precision, recall, F1 –score, and accuracy. the best algorithm to predict vulnerability levels among other
algorithms in the present study.
TABLE IV: Performance of ML algorithms in classifying
the OSes vulnerabilities’ levels.
B. Denial of Service Attack
Classifier Precision Recall F1 –score Accuracy The exploitability of any newly discovered OS vulnerability
Logistic Regression 0.71 0.71 0.7 0.71 is an important aspect that specialists try to discover in order
K nearest neighbors 0.77 0.77 0.77 0.77 to prepare a remediation plan. Denial of service attack is
Gaussian Naive Bayes 0.69 0.67 0.68 0.67 a commonly known attack and building a predictive model
Random Forest 0.93 0.93 0.93 0.93 would assist in preventing it. CVEdetails provides information
Adaptive Boosting 0.92 0.92 0.92 0.92 about whether the published OS vulnerability allows attackers
to cause denial of service. Using this information, the RF
The RF and Adaboost classification algorithms outperform classification algorithm was applied. The precision, recall,
the LR, KNN, and GNB, where their evaluation metrics values F1 –score, and accuracy were 93% in predicting whether the
were stable at 93% and 92%, respectively. In addition, Table newly discovered OS vulnerability would allow attackers to
V shows the number of mis-classified vulnerabilities, and cause denial of service to the system.
training and testing times.
V. C ONCLUSION
TABLE V: Performance of ML algorithms in classifying the Today, information is stored in systems where their secu-
OSes vulnerabilities’ levels based on the Miss-Classified,
training and testing times.
rity is an important component, along with their reliability
and availability. Computer operating systems (OSes) such as
Classifier Miss-Classified Training Time Testing Time
Microsoft, Apple, and Linux are the focus of this research.
Logistic Regression 1414 0:00:02.409 0:00:00.000
K nearest neighbors 1114 0:00:00.018 0:00:00.104
Specifically, the vulnerabilities of the OSes were collected
Gaussian Naive Bayes 1579 0:00:00.004 0:00:00.001 from January 1999 to March 2019, and utilized to achieve
Random Forest 328 0:00:01.343 0:00:00.150 the research objectives.
Adaptive Boosting 367 0:00:06.126 0:00:00.571 Using the vulnerability level, predictive models were devel-
oped using the vulnerabilities details along with our introduced
The RF has the highest accuracy and therefore smallest factors Polarity, Subjectivity, Frequency, NumAttacks, and
number of miss-classified vulnerabilities. The second highest Days which were significant to the models’ performance.
was the Adaboost algorithm with the influence of the RF The Random Forest machine learning algorithm had the best

Authorized licensed use limited to: University College London. Downloaded on May 25,2020 at 13:36:47 UTC from IEEE Xplore. Restrictions apply.
performance in predicting the vulnerability level and how [10] B. L. Bullough, A. K. Yanchenko, C. L. Smith, and J. R. Zipkin,
likely it would be for attackers to cause Denial of Service “Predicting exploitation of disclosed software vulnerabilities using
open-source data,” in Proceedings of the 3rd ACM on International
to the system, with 93% precision, recall, F1 –score, and Workshop on Security And Privacy Analytics, ser. IWSPA ’17.
accuracy. The Adaboost algorithm consumed more time in the New York, NY, USA: ACM, 2017, pp. 45–53. [Online]. Available:
training and testing stages, with 92% in the same evaluation http://doi.acm.org/10.1145/3041008.3041009
[11] M. Edkrantz and A. Said, “Predicting cyber vulnerability exploits with
metrics. On the other hand, K nearest neighbors, Logistic machine learning.” in SCAI, 2015, pp. 48–57.
Regression, and Gaussian Naive Bayes algorithms did not [12] C. Sabottke, O. Suciu, and T. Dumitra, “Vulnerability disclosure in the
perform well, having evaluation metrics of 77% or less. age of social media: exploiting twitter for predicting real-world exploits,”
in 24th {USENIX} Security Symposium ({USENIX} Security 15), 2015,
The developed predictive models will assist not only ven- pp. 1041–1056.
dors and information technology specialists, but also end- [13] L. Allodi and F. Massacci, “Comparing vulnerability severity and
users, in managing and understanding the impact of the exploits using case-control studies,” ACM Trans. Inf. Syst. Secur.,
vol. 17, no. 1, pp. 1:1–1:20, Aug. 2014. [Online]. Available:
unfixed and newly discovered vulnerabilities of their computer http://doi.acm.org/10.1145/2630069
operating systems. [14] K. Nayak, D. Marino, P. Efstathopoulos, and T. Dumitraş, “Some
vulnerabilities are different than others,” in International Workshop on
In future work, we aim to perform statistical modeling of Recent Advances in Intrusion Detection. Springer, 2014, pp. 426–446.
vulnerability scores for Microsoft, Apple, and Linux computer [15] S. Zhang, D. Caragea, and X. Ou, “An empirical study on using
operating systems combined and separately. We also plan to the national vulnerability database to predict software vulnerabilities,”
in Database and Expert Systems Applications, A. Hameurlain, S. W.
develop a ranking process of the risk factors used to drive Liddle, K.-D. Schewe, and X. Zhou, Eds. Berlin, Heidelberg: Springer
computer operating systems vulnerability scores. Identifying Berlin Heidelberg, 2011, pp. 217–231.
the ranks of the risk factors for these vulnerabilities provides [16] C. Wu, T. Wen, and Y. Zhang, “A revised cvss-based system to improve
the dispersion of vulnerability risk scores,” Science China Information
vendors and information technology specialists valuable guid- Sciences, vol. 62, no. 3, p. 39102, Sep 2018. [Online]. Available:
ance in prioritizing the remediation process. https://doi.org/10.1007/s11432-017-9445-4
[17] S. Özkan, “Cve details,” https://www.cvedetails.com.
[18] O. Security, “Exploit database,” https://www.exploit-db.com/.
ACKNOWLEDGMENT [19] S. Özkan, “Cve details,” Retrieved, vol. 16, p. 2017, 2017.
[20] L. Richardson, “Beautiful soup documentation,” April, 2007.
We thank Majmaah University and University of South [21] G. L. Hajba, Using Beautiful Soup. Berkeley, CA: Apress, 2018, pp.
Florida for funding and supporting the research. 41–96. [Online]. Available: https://doi.org/10.1007/978-1-4842-3925-4_
3
[22] N. M. Share, “Operating system market share,” https://www.
R EFERENCES netmarketshare.com/.
[23] C. M. Bishop, Pattern recognition and machine learning. springer,
[1] P. Mell, K. A. Kent, and S. Romanosky, The common vulnerability 2006.
scoring system (CVSS) and its applicability to federal agency systems. [24] A. Liaw, M. Wiener et al., “Classification and regression by randomfor-
Citeseer, 2007. est,” R news, vol. 2, no. 3, pp. 18–22, 2002.
[2] K. Scarfone and P. Mell, “An analysis of cvss version 2 vulnerability [25] S. L. Taylor and K. Kim, “A jackknife and voting classifier approach
scoring,” in Proceedings of the 2009 3rd International Symposium on to feature selection and classification,” Cancer informatics, vol. 10, pp.
Empirical Software Engineering and Measurement, ser. ESEM ’09. CIN–S7111, 2011.
Washington, DC, USA: IEEE Computer Society, 2009, pp. 516–525. [26] X. Wang, “Time dependent kernel density estimation: A new parame-
[Online]. Available: http://dx.doi.org/10.1109/ESEM.2009.5314220 ter estimation algorithm, applications in time series classification and
[3] A. Gorbenko, A. Romanovsky, O. Tarasyuk, and O. Biloborodov, “From clustering (doctoral dissertation,” Graduate Theses and Dissertations.
analysing operating system vulnerabilities to designing multiversion https:// scholarcommons.usf.edu/ etd/ 6425, 2016.
intrusion-tolerance architectures,” IEEE Transactions on Reliability, [27] Y. Freund, R. E. Schapire et al., “Experiments with a new boosting
2019. algorithm,” in icml, vol. 96. Citeseer, 1996, pp. 148–156.
[4] S. Furnell and D. Emm, “The abc of ransomware protection,” [28] N. Lachiche and P. A. Flach, “Improving accuracy and cost of two-class
Computer Fraud & Security, vol. 2017, no. 10, pp. 5 – 11, and multi-class probabilistic classifiers using roc curves,” in Proceedings
2017. [Online]. Available: http://www.sciencedirect.com/science/article/ of the 20th International Conference on Machine Learning (ICML-03),
pii/S1361372317300891 2003, pp. 416–423.
[5] P. K. Kaluarachchi, C. P. Tsokos, and S. M. Rajasooriya, “Cybersecurity: [29] M. Sokolova and G. Lapalme, “A systematic analysis of performance
a statistical predictive model for the expected path length,” Journal of measures for classification tasks,” Information Processing & Manage-
information Security, vol. 7, no. 03, p. 112, 2016. ment, vol. 45, no. 4, pp. 427–437, 2009.
[6] N. R. Pokhrel and C. P. Tsokos, “Cybersecurity: A stochastic predictive [30] T. Hastie, R. Tibshirani, and J. Friedman, “The elements of statistical
model to determine overall network security risk using markovian learning: data mining, inference, and prediction, springer series in
process,” Journal of Information Security, vol. 8, no. 02, p. 91, 2017. statistics,” 2009.
[7] N. R. Pokhrel, H. Rodrigo, and C. P. Tsokos, “Cybersecurity: Time series
predictive modeling of vulnerabilities of desktop operating system using
linear and non-linear approach,” Journal of Information Security, vol. 8,
no. 04, p. 362, 2017.
[8] M. Bozorgi, L. K. Saul, S. Savage, and G. M. Voelker, “Beyond
heuristics: Learning to classify vulnerabilities and predict exploits,”
in Proceedings of the 16th ACM SIGKDD International Conference
on Knowledge Discovery and Data Mining, ser. KDD ’10. New
York, NY, USA: ACM, 2010, pp. 105–114. [Online]. Available:
http://doi.acm.org/10.1145/1835804.1835821
[9] A. Feutrill, D. Ranathunga, Y. Yarom, and M. Roughan, “The effect of
common vulnerability scoring system metrics on vulnerability exploit
delay,” in 2018 Sixth International Symposium on Computing and
Networking (CANDAR), Nov 2018, pp. 1–10.

Authorized licensed use limited to: University College London. Downloaded on May 25,2020 at 13:36:47 UTC from IEEE Xplore. Restrictions apply.