A VISUAL EXPLORATION OF

EXPLOITATION
IN THE WILD
THE INAUGURAL STUDY OF EPSS DATA AND PERFORMANCE

J U LY
2024
A Visual Exploration of Exploitation in the W i l d Page | 3

INTRODUCTION TABLE OF CONTENTS

Have you ever heard claims about a security INTRODUCTION FROM EPSS CREATOR &
product or model that never made any attempt About EPSS
to validate whether actual performance lived FIRST.ORG SIG CHAIR 4
up to those claims? Nah, neither have we. That’s
obviously a joke—not backing our mouths with ACKNOWLEDGMENTS 5
measurement is the norm in our field. But we EPSS is a data-driven effort for estimating
don’t want to follow that script with the Exploit the likelihood (probability) that a published SPONSOR COMMENTS 6
Prediction Scoring System (EPSS). vulnerability will be exploited in the wild. Its
goal is to assist defenders to better prioritize
EXPLOITATION ACTIVITY 8
vulnerability remediation efforts. While other How many vulnerabilities have been published? Exploited? 10
For years now, we’ve been collecting evidence industry standards have been useful for capturing
of exploitation activity from data contributors. innate characteristics of a vulnerability and What proportion of vulnerabilities have been exploited? 12
This data was used to train the EPSS model that provide measures of severity, they are limited in
produces the daily scores that are freely available their ability to assess threat. EPSS fills that gap Does exploitation activity fluctuate over time? 14
to the security community. With the passage of by using current threat information targeting
time, we now have a rich history of predictions CVEs along with real-world exploit data. The What’s the typical pattern of exploitation activity? 16
that we can test with the benefit of hindsight. EPSS model produces a daily updated prediction
of the probability that a given vulnerability will What’s the ratio of new vs. old exploitation? 18
be exploited in the next 30 days.
This inaugural study seeks to evaluate EPSS How long since exploitation was last observed? 20
performance over the last few years. In addition, we
tackle a host of questions related to understanding A growing set of organizations contribute data How long until exploitation was first observed? 22
the ins and outs of vulnerability exploitation in the to EPSS (and you can join them!). The Cyentia
wild. We hope it offers measured KPIs for EPSS as How “old” is current exploitation activity? 24
Institute developed the EPSS model and
well as valuable insights for using it to manage
and prioritize vulnerabilities in your environment.
crunches the data to generate the daily scores.
First.org hosts the EPSS SIG and makes the data
How widespread is exploitation among organizations? 26
available to the community. Find out more at
https://www.first.org/epss/
EVALUATING EPSS PERFORMANCE 28
How do we evaluate exploit predictions? 30
How does CVSS perform? 32
How does the KEV perform? 34

The Cyentia Institute and FIRST.org have made all of the charts in Can metadata help predict exploitation? 36

this report avalible for download. These resources provide valuable Can exploit tools help predict exploitation? 38

insights into vulnerability exploitation patterns and EPSS performance. How does EPSS perform? 40
How do EPSS and CVSS compare? 42
Access and download the full set of charts here. What EPSS score warrants priority remediation? 44

CONCLUSION: WHAT’S NEXT FOR EPSS? 46

APPENDIX: EPSS OVERVIEW & HISTORY 48
A Visual Exploration of Exploitation in the W i l d Pag e | 5

OPENING THOUGHTS ACKNOWLEDGMENTS

From EPSS Creator Jay Jacobs ATIN
U M SPONSO
R G O LD SPONSO
R G O LD SPONSO
R
PL

I’ve been fortunate in my career to have worked We have two major goals in this research. First,
with some very interesting data sets. Data often we want to investigate all of the exploitation
surprises me and challenges many commonly activity we were able to collect and ask some
held beliefs across the security industry. But more seemingly simple questions. We want to
importantly, they can generate opportunities to understand everything we can about the timing,
learn if we are ready to do so. This is one of those volume and prevalence of exploitation activity.
opportunities. As you’ll see in the first half of this research,
“exploited in the wild” is a relatively meaningless
label. Exploitation today does not always
The opportunity to learn generally comes in only ER SPONS ER SPONS ER SPONS
mean exploitation tomorrow and me seeing LV O LV O LV O
one form: feedback. If we want to learn how to exploitation activity doesn’t also mean you’ll see SI SI SI

R
R

R
play better golf we hit a golf ball and get feedback exploitation. Exploitation activity is incredibly
by watching what happens. While “practice makes varied across time, targets and volume and we
perfect”, it’s actually the feedback we receive while need better language to talk about it.
practicing that creates improvement. How quickly
would someone improve if they couldn’t hear the
sound coming out of their instrument? How fast Second, we want to collect and analyze feedback
could someone improve their free throws if they on how the Exploit Prediction Scoring System
ER SPONS ER SPONS ER SPONS ER SPONS
couldn’t see what happened after the basketball (EPSS) is performing. EPSS generates a score LV O LV O LV O LV O
left their hands? The same is true in vulnerability every day for every published vulnerability SI SI SI SI

R
R

R
management. When is the last time anyone went (with a CVE ID) on how likely it is that we will
back to what was prioritized in the last cycle to observe exploitation activity in the following 30
collect feedback on their decisions? It generally days. Well, EPSS has been publishing scores for
doesn’t happen, but that’s exactly what we are over three years now, that’s a lot of predictions
doing here. over many 30 day windows. With the power of
hindsight, we can look back at each and every
daily prediction and compare against the actual
Now, I don’t want to spoil the surprise, but EPSS exploitation activity we (our data partners)
is not perfect. It will rate some vulnerabilities very observed in the 30 day windows following each
low that end up with exploitation activity, and
The Cyentia Institute and FIRST.org offer our thanks to the sponsors
prediction.
some very high that don’t. However, perfection
isn’t an option for anyone in reality, so EPSS (and
every other prioritization strategy) needs to be Speaking of data partners, I want to personally of this study. Your commitment to supporting a resource that’s
compared to real and practical alternatives. We thank each and every one of them for their increasingly used by organizations around the world is commendable.
explore some of those comparisons in this research contribution, so in no particular order, thank
with CISA’s Known Exploited Vulnerability (KEV) you to GreyNoise, Shadow Server Foundation,
list and the Common Vulnerability Scoring System Fortinet, AlienVault, Cisco, F5, Efflux and
(CVSS). Cyentia. EPSS would be nothing without their
contributions, so please join me in thanking them! As we explore the intricacies of exploits in the wild and assess the efficacy of the Exploit Prediction Scoring
System (EPSS), we recognize the invaluable role of community contributions. Your participation in sharing
exploitation activity data is crucial for refining our predictive models and enhancing the security landscape.
We invite you to join our efforts in advancing the EPSS initiative by becoming a data contributor. Together,
we can build a more robust and accurate system that benefits the entire security community. Visit the
Cyentia website to learn how you can get involved and contribute to our ongoing projects.
A Visual Exploration of Exploitation in the W i l d Page | 6 Pag e | 7

COMMENTS FROM PLATINUM SPONSOR TENABLE

EPSS is an effective input of efforts in a vulnerability management
practice. Environmental and organizational
future of vulnerability management by
demonstrating that this sea of risk can be
INAUGURAL EPSS
for risk-based vulnerability
management.
factors outside the scope of EPSS (e.g.
asset criticality, network exposure and
business impact) are crucial for assessing
drained down to a manageable pond.
PERFORMANCE
REPORT BY
overall risk. By design, EPSS does not Using EPSS as a supplemental input
account for the criticality of affected alongside Tenable’s proprietary scoring
The Exploit Prediction Scoring System assets, their role in business perations, system, the Vulnerability Priority Rating

FIRST & CYENTIA

(EPSS), plays a crucial role in the risk formula or their interconnectedness within the (VPR), sharpens that focus even further.
by providing a predictive measure of the network. This focus can lead to misaligned VPR helps organizations improve their
likelihood that a specific vulnerability with prioritization, where vulnerabilities deemed remediation efficiency and effectiveness by

INSTITUTE
a Common Vulnerabilities and Exposures likely to be exploited are addressed at rating vulnerabilities based on severity level
(CVE) identifier will be exploited. EPSS helps the expense of those that, while less determined by two components: technical
organizations prioritize and triage known likely, could have severe consequences if impact and threat. Technical impact

BRINGS HOPE TO
vulnerabilities based on the likelihood of exploited. Integrating EPSS with inputs like measures the impact on confidentiality,
exploitation. By assigning a probability threat intelligence, patch availability and integrity and availability following
score to each CVE, EPSS enables security compliance requirements offers a more exploitation of a vulnerability. The threat
teams to efficiently allocate resources to comprehensive risk management approach. component reflects both recent and potential
address the most pressing threats. This
targeted approach enhances the overall
risk management strategy and ensures the
EPSS must be used in conjunction with this
contextual information to provide a more
complete picture and ensure effectiveness in
future threat activity against a vulnerability.
Examples of such threat sources include
intelligence feeds, observations of Indicators
THE FUTURE OF
most critical vulnerabilities are addressed
promptly.
guiding holistic vulnerability management
strategies.
of Compromise (IoC), reports of exploitation
on social media or code repositories, and
more. VPR provides context that is otherwise
VULNERABILITY
MANAGEMENT
missing from EPSS. In other words, not only
does VPR tell you how bad a vulnerability
is, but it tells you why it’s bad. Using these
EPSS is just one input. Tenable brings it all scores in parallel provides a much more
Understanding context is together with VPR. holistic risk prioritization approach.

key.
As this report highlights, there will always
be more risk than you can address in your
Despite its strong performance in both environment. Focusing on the exposures that
coverage and efficiency as noted in matter - we call them the critical few - through
this report, EPSS should not be used in an effective vulnerability prioritization
isolation for the effective prioritization strategy is key. EPSS brings hope to the
A Visual Exploration of Exploitation in the W i l d Pag e | 9

EXPLOITATION REMARKS FROM

ACTIVITY JUPITERONE
Prioritize
Before measuring the predictive performance of EPSS, we first analyze our data sources for the
exploitation of vulnerabilities. We start with some historical trends and then examine activity patterns,

Effectively:
timelines, and prevalence of exploit activity in the wild.

IN THIS SECTION
How many vulnerabilities have been published?
The Power of
What proportion of vulns have been exploited?
Time-to-
Does exploitation activity fluctuate over time?
Exploitation
What’s the typical pattern of exploitation activity?
Metrics
What’s the ratio of new vs. old exploitation?
Traditional vulnerability management approaches often overwhelm security teams with numerous
How long since exploitation was last observed? alerts, many of which may not pose immediate threats. For security analysts this can lead to alert
fatigue and inefficient use of time and resources. Time-to-exploitation (TTE) metrics address this
problem by helping teams focus on vulnerabilities that are most likely to be exploited soon.
How long until exploitation was first observed?
How “old” is current exploitation activity? As organizations strive to reduce their risk and enhance vulnerability management, incorporating
EPSS and TTE metrics alongside traditional vulnerability scores offers a holistic view that integrates
How widespread is exploitation among organizations? severity with exploitation probability.

With JupiterOne and EPSS, eliminate guesswork and focus on what really matters. Many
vulnerabilities aren’t exploited immediately—don’t waste resources on non-urgent patches.

“EPSS is a positive step forward for the industry as organizations now have an
independent risk-focused scoring metric to augment the long-standing CVSS
Prioritize effectively, stay secure, and maintain control.
severity metrics that have been the underpinnings of many VM programs.
Coupled with the contextualization of vulnerability intelligence data as well as JupiterOne is the asset, attack surface and exposure
the impacted assets, organizations will have the ability to better make true risk- management platform for security and IT, that empowers
based prioritization decisions that are oriented towards their environments. organizations to prioritize and remediate what matters
most. Continuously monitor exposure with complete
visibility across assets and relationships. See out key
- Luke Tamagna-Darr | Senior Director, Engineering, Tenable takeaways at jupiterone.com/epss.
HOW MANY HISTORY OF PUBLISHED AND EXPLOITED CVEs

VULNERABILITIES HAVE
There were 237,687 published CVEs as of May 31, 2024, with 13,807 being observed with exploitation
activity as shown in the top plot. The bottom plots show that we just passed 30,000 CVEs published in
the last 12 months with the annual rate varying around the average of 16%.

BEEN PUBLISHED? We’re nearing a quarter million published CVEs.

EXPLOITED? 240k
220k
200k
180k
Let’s begin with the big picture. There’s been This rising tide of vulnerabilities inundates

Count of CVEs
160k
no shortage of charts created that depict VM teams with the challenge of assessing 140k
the number of published vulnerabilities over and remediating them all. Given the volume 120k
time. But it’s an appropriate starting point of vulnerabilities out there, tracking which
100k
for this study, so here’s one more. We’re ones have been exploited or attacked
nearing a quarter million published CVEs, becomes imperative to managing risk. Per 80k
and that’s been growing faster in recent the chart, the number of CVEs known to 60k
years. There are many contributing factors be exploited keeps rising… though not as 40k
behind this trend, which we can’t dig into quickly as the rate of publication. We’ll zoom 20k
in this report. Suffice it to say that more into that red “Exploited” line next. 0
vulnerabilities don’t necessarily mean the 2017 2018 2019 2020 2021 2022 2023 2024
world is less secure; much of this growth is
a reflection of changes in the CVE disclosure

CVEs Published Within

process.

Previous Year
We’ll easily add 30k+
CVEs to the public
record during 2024.

TAKEAWAY: The rising tide of vulnerabilities will overwhelm

Annual Growth Rate

VM teams if remediation can’t be prioritized.

That number has
grown at a rate of
16% annually over the
last 7 years.

A Visual Exploration of Exploitation in the W i l d Pag e | 1

WHAT PROPORTION VULNERABILITIES WITH EXPLOITATION ACTIVITY

OF VULNERABILITIES
Newly observed exploitation actvity has been rather steady over the last few years. The top left plot shows
the cumulation of 13,807 CVEs with exploitation activity over time, while the bottom right plot shows the
count as a percentage of published CVEs over time.

HAVE BEEN 14k

EXPLOITED?

CVEs with Exploitation Activity

12k

10k

8k
The number of
known-exploited
Let’s take a closer look at the red exploitation While the total number keeps rising, vulns is steadily
6k
trendline from the previous chart. The top the bottom right chart shows that the approaching 15k.
left chart in the figure below shows steady proportion of published CVEs known to be
growth in the number of vulnerabilities exploited remains fairly steady, fluctuating 4k
with known exploitation in the wild. Keep around the 6% mark. The apparent decline
in mind that this doesn’t mean that ~14,000 over the last few years isn’t so much a 2k
vulnerabilities are actively being exploited decline as it may be a delay. As we’ll soon
right now. It shows that we know of ~14,000 see, the majority of vulnerabilities aren’t 0
CVEs that have, at some point in their history, immediately exploited when initially 2017 2018 2019 2020 2021 2022 2023 2024
been reported as exploited by primary published. It can take time for attackers to
sources. We’ll examine the age, duration, discover them and develop exploits and for
and prevalence of exploitation over the next defenders to detect exploitation activity.
several pages of this report. Monitoring these precursors of exploitation
via its many data contributors is what drives
updates to EPSS scores on a daily basis.

Percent of Published CVEs with

Observed Exploitation Activity
6%

About 6% of
all published 4%
CVEs have been
TAKEAWAY: Tracking (and predicting) the subset of exploited exploited; that
CVEs is critical for efficient remediation.
rate is holding
2%
relatively steady.

0%
2017 2018 2019 2020 2021 2022 2023 2024

A Visual Exploration of Exploitation in the W i l d Pag e | 13

DOES EXPLOITATION
UNIQUE CVES WITH EXPLOITATION ACTIVITY

ACTIVITY FLUCTUATE
Counting the unique CVEs with exploitation activity within each month (blue) and within each calendar
year (red), there is evidence of sporadic exploitation activity and an indication that once a vulnerability
is exploited it may not always be exploited.

OVER TIME?

Count of Unique CVEs with

Exploitation Activity
On the prior page, we showed that nearly What should VM teams do in light of this
14,000 vulnerabilities have evidence of pattern of sporadic exploitation? The
exploitation and caveated that not all of them answer has a lot to do with risk tolerance.
are actively being exploited right now. That’s Risk-averse organizations may wish to take a
actually a really important point because “once exploited, always exploited” approach
many people have the misconception that to eradicate any vulns with a history of
exploitation is a static or persistent trait. So, exploitation, however brief. Risk-tolerant or
we’ve devoted the next several charts in this resource-challenged organizations may be
report to exploring the ebbs and flows of best served by prioritizing those exploited
exploitation activity. recently and/or those most likely to be
targeted in the near future. EPSS provides
data to support whatever strategy you
As we’ve already established, the number choose.
of vulnerabilities with exploit activity
detected within each year rises over time.
But the monthly tally fluctuates quite a bit
(sometimes because of data issues). Of the
~14,000 CVEs known to be exploited, about Vulns exploited in the past aren’t all being attacked right now.
10,000 had observed exploitation activity
in 2023. Thus, there’s definitely a temporal
element to track and consider when
prioritizing vulnerability remediation based
on exploitation activity. Don’t get too excited about the dip in 2024; it’s not over yet.

TAKEAWAY: The number of actively exploited vulns grows as

some drop off and others get attacked.

A Visual Exploration of Exploitation in the W i l d Pag e | 15

WHAT’S THE DISPARITY IN OBSERVED EXPLOITATION ACTIVITY

TYPICAL PATTERN
Five out of the 10,106 CVEs with observed exploitation activity are shown here to highlight the volume
and variety. Each data source measures “volume” on dif erent scales, so they are normalized here with
red representing the highest volume and blue is just a trickle of activity. Not shown is that most of the
exploitation activity looks a lot more like the top CVEs than the bottom shown here.

OF EXPLOITATION
ACTIVITY? Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

SMTWTFS
Exploitation of this CVE was
short-lived and very sparse.

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Five Different CVEs over 2023

What does this fluctuating pattern of This one saw fairly regular,

SMTWTFS
exploitation activity look like? Well, that albeit sporadic, weekday
depends on the vulnerability in question. activity.
Some vulns are continuously exploited for
long periods of time. Some are just a flash in Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Daily to weekly exploit

SMTWTFS
the pan. Exploits of others come in fits and
starts. Some real-world examples of these attempts with a spike in
patterns are demonstrated in the chart mid-Dec.
below, which depicts observed exploitation
activity for five CVEs over the course of 2023. Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

SMTWTFS
Sustained daily exploitation
at its highest in Q1–Q2.

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

SMTWTFS
Extremely high rate of
unrelenting exploitation
activity.

TAKEAWAY: Don’t treat “Exploited” as a binary variable;

intensity and duration matter for prioritization.

A Visual Exploration of Exploitation in the W i l d Pag e | 17

WHAT’S THE RATIO VULNERABILITIES WITH KNOWN EXPLOITATION ACTIVITY

OF NEW VS. OLD

We break up the monthly exploitation activity into three categories: exploitation activity has been
observed this month and observed previously (blue), exploitation activity has been observed before
but not in this month (sea green), no previous exploitation activity has been observed (red).

EXPLOITATION?
100%

90%

with Known Exploitation Activity

80%

Percent of Vulnerabilities
70%

60%
We’ve seen that exploitation activity You’ll also notice a splash of red flitting across
targeting vulnerabilities ebbs and flows the bottom of the chart. That represents net 50%
over time, but what proportion constitutes new exploitations that have never before
an ebb vs. a flow? The chart below plots been detected. It’s just a fraction of the 40%
that distinction over the last several years. overall activity, but those are the attacks 30%
In it we see that the majority of observed that keep many VM teams up at night (and
exploitations in a given month flows over sometimes working over the weekend). 20%
from the previous month (represented by
the blue area). Also apparent is the third 10%
or so of exploit activity that ebbs away— 0%
temporarily, at least (the teal area). 2018 2020 2022 2024

“We’ve seen these before, and “We’ve seen these before, “We are seeing these for
we are seeing them again” but not this month” the first time this month”
The majority of exploitation activity 30-40% of previous exploitation A small percent of
observed in any given month has activity is not observed in the exploitation activity has never
been previously reported current month been observed previously

TAKEAWAY: Newly exploited vulns get the most attention, but The vast majority of monthly exploitation activity has been seen before.
the older ones get the most action.

About a third of previously observed exploitations will periodically go

dormant.

A Visual Exploration of Exploitation in the W i l d Pag e | 19

HOW LONG SINCE
THE RECENCY OF EXPLOITATION ACTIVITY

EXPLOITATION WAS
Just because something has been reported as exploited in the wild does not mean it will always be
exploited in the wild. This chart looks at all of the exploitation activity and how recently vulnerabilities
have had observed exploitation activity.

LAST OBSERVED

Percent of Vulnerabilities with

Known Exploitation Activity
Imagine we could take the vulnerabilities Here’s a memorable one: 5% of CVEs had
represented in the blue and teal areas of the exploitation activity over five years ago and
prior chart and more precisely measure how haven’t been seen or heard from since.
long it’s been since they were last exploited.
Some may have a “So what?” reaction here,
Good news — no imagination needed. but look at it this way: how long do you need
We’ve visualized it for you! to keep previously exploited vulnerabilities
on your prioritization radar in case they
wake up again? This chart can help answer
For half of the nearly 14,000 known exploited that question and should help you rethink
vulnerabilities, the most recent detected your remediation efforts.
activity was within the last week. Another Time Since Last Known
quarter of CVEs have been attacked in the Exploitation Activity (Years)
last twelve months and the remaining
quarter have been dormant for longer than
a year. You can triangulate any point on
the horizontal and vertical axis to pick out
whatever stats you like. Most exploitation activity is a continuation of recent attacks.

It’s rare for exploits that haven’t been seen in years to flare up again.

TAKEAWAY: Just because a vulnerability is known to have

exploitation activity, doesn’t mean it always will.

A Visual Exploration of Exploitation in the W i l d Pag e | 21

HOW LONG UNTIL THE URGENCY OF EXPLOITATION ACTIVITY

EXPLOITATION WAS
A vulnerability being published is usually accompanied by a range of other possible events (patches,
disclosures, scanner and detection signatures, etc.), but how soon are we observing exploitation
activity? Roughly about 1 in every 9 CVEs with observable exploitation activity are observed before the
end of the first week after publication.

FIRST OBSERVED?

Percent of Exploited CVEs

What about the flip side of exploitation exploit kits. Within a month of publication,
duration? Once a vulnerability is published 40% of CVEs observed exploitation in the
with a newly minted CVE, how much time wild.
typically passes until detected exploitation
in the wild begins? Some people assume
that happens immediately, often urging VM A strong majority (70%) of vulns see initial
teams to drop everything else to remediate attack activity in a year or less. It levels out
“critical” vulns ASAP. Others take the quickly from there. Just 7% of published
opposite approach and presume there’s CVEs go three years before being exploited.
plenty of time before they’ll see actual
attacks. As with most things, the truth is
somewhere in the middle. These statistics are all fine and dandy. The
rub is in determining how long before THIS Time from Publication to First
PARTICULAR vulnerability is likely to be Known Exploitation Activity (Years)
Among the ~14,000 known exploited exploited. That’s where EPSS comes in by
vulnerabilities in our dataset, 8% were helping to remove the guesswork. It gives a
targeted BEFORE the CVE was published. daily assessment of the probability that any
Rather than zero days, most of these are given CVE will be exploited within the next
“reserved but public” CVEs that, while not month.
officially published, contain information The countdown to initial exploitation is often pretty quick.
sufficient for them to be incorporated into
vulnerability scanners, detection tools, and

That said, there are hundreds of CVEs that went several years before being
attacked.

TAKEAWAY: What if remediation SLAs were based on time-to-

exploitation instead of vulnerability severity?

A Visual Exploration of Exploitation in the W i l d Pag e | 23

A Visual Exploration of Exploitation in the W i l d

HOW “OLD” IS THE TIMELINESS OF EXPLOITATION ACTIVITY

CURRENT
The typical CVE with exploitation activity is observed a median of 284 days. This chart breaks down over
8.6 million unique observations of daily exploitation activity and the difference between the publication
of the target vulnerability and the date exploitation activity was observed.

EXPLOITATION

Percent of All Exploitation Activity

ACTIVITY?
Beyond time to/since exploitation, this Here are a few statistical highlights
question gets at another important temporal to help you interpret what the chart
aspect of exploitation. We analyzed all conveys:
observed exploitation and recorded the
age of all CVEs when they were targeted
with exploitation activity. The point is to About 1% of observed exploitations
understand whether attackers are targeting target unpublished CVEs (first bar). Time Since Publication of
older or newer vulnerabilities on the whole. Targeted CVE (Years)
The chart below will aid that understanding.
About 5% of exploitation activity
targets CVEs less than a year old
What do you do with this information?
1
(second bar).
10
Only 6% of the 8.6 million 38% of the exploitation
Well, for starters, we suggest that VM teams < daily observed exploitation > attempts targeted
maintain their long-term memory. The data year attempts targeted years vulnerabilities more than
clearly shows that the hackings will continue About 6% of current exploits target vulnerabilities before they 10 years after they were
until security improves. Don’t be fooled into CVEs released 12 years ago. were a year old published
thinking that attackers only look for the
cool new exploit. They are still probing for
decade-old vulnerabilities and are happy to About 39% of exploit observations
exploit them if found. target CVEs that are five or fewer
years old.

3X more exploits target CVEs 10+ years old than those published in last
2 years.

The rate of exploitation for unpublished CVEs equals those published 20

TAKEAWAY: Attackers are content to keep exploiting the years ago.

“oldies but goodies” as long as we let them.

Pag e | 25
HOW WIDESPREAD Q: How many organizations see exploit activity from a typical vulnerability?

IS EXPLOITATION THE PREVALENCE OF EXPLOITATION ACTIVITY

AMONG By identifything which data collection point reports the exploitaiton activity we can get a sense of how
far the activity spreads around the world. In the case of published vulnerabilities, it’s relatively rare to

ORGANIZATIONS?
have widespread exploitation: only 5% of exploited CVEs reach more than 10% of collection points.

100%

(With Activity)
Activity)
90%
This one was an eye-opener for us. Rather Less than 5% of exploited vulns hit more
than exploited CVEs or timelines, let’s than 1 in 10 organizations. The scope of 80%

ExceedingPrevalence
Prevalence
examine the prevalence of exploitation exploitation becomes important when 70%
observed across a large population of trying to discern whether your organization

CVEs(With
100,000+ organizations around the world. is in the crosshairs. 60%
Before looking at the figure below, ask 50%
Out of all of the CVEs with
yourself this question: what percent of observed exploitation activity,

ofofCVEs
Half of the CVEs with observed

Exceeding
organizations typically see exploitation There is another challenge here to 40% exploitation activity won't 9.1% managed to reach more
than 1 in every 100 (1%) of Only 4.5% of CVEs with
targeting a particular vulnerability? Perhaps conventional thinking. When vulnerabilities reach more than 1 in every
4.6k organizations (0.02%). organizations. exploitation activity
30%
are reported as being exploited in the

Percent
1% of them? Or 10%? Half? managed to reach more
wild, they are generally portrayed as being

Percent
20% than 1 in every 10 (10%)
of organizations.
exploited everywhere. This is clearly not
10%
It turns out that widespread exploitation in the case. When someone cries, “This is
the wild is a pretty rare feat. The chart (you being exploited!”, we should request more 0%
can look now) records this reality. Half of all information about the nature and scope 1 in
100k
1 in
10k
1 in
1k
1 in
100
1 in
10
1 in
1
known exploited CVEs are never observed by of that exploitation rather than treating all
more than 0.02% of organizations! such reports equally. Prevalence (Percent of Collection
Prevalence (Percent of Collection Points Reporting Activity)
Points Reporting Activity)

A: Not many. Exploits hitting more than 1 in 10 organizations are pretty rare.

TAKEAWAY: A small minority of CVEs “go big,” achieving

widespread exploitation across organizations.

A Visual Exploration of Exploitation in the W i l d Pag e | 27

A Visual Exploration of Exploitation in the W i l d Pag e | 29

EVALUATING EPSS REMARKS FROM

PERFORMANCE NUCLEUS SECURITY
This section leverages the clarity of hindsight to measure the reliability of EPSS’ predictions
EPSS Thresholds As risk–based vulnerability management
programs mature, they shift their focus from

Operationalized
‘What is being exploited now?’ to ‘What is likely
of exploitation. We start by describing the methodology used to measure performance and to be exploited next?’. EPSS uniquely addresses
then put several vulnerability rating and prioritization approaches to the test before turning the latter question. It provides an estimate of
to EPSS.
with Business the likelihood that a software vulnerability will
be exploited in the wild based on probability

IN THIS SECTION Context and machine learning.

Setting an EPSS threshold based on the organization’s risk tolerance is the first step to
How do we evaluate exploit predictions? operationalizing EPSS. However, this only provides a global prediction. Without organizational
context, the effectiveness of using EPSS as a measure of prediction is limited. To manage risk-
How does CVSS perform? based prioritization at enterprise scale, Nucleus combines your EPSS threshold with extensive
asset and business context including internet accessibility, data sensitivity, asset criticality, and
How does the KEV perform? compliance scopes. This unified approach enables teams to effectively operationalize EPSS scores
and shift from reactive to proactive prioritization.
Can meta-data help predict exploitation?
Can exploit tools help predict exploitation?
How does EPSS perform?
How do EPSS and CVSS Compare?
What EPSS score warrants priority remediation?

“Threat centric scoring systems like EPSS are the foundation of

data-driven vulnerability management programs. Organizations
must leverage the insights and context these scores provide, but
they can’t stop there. They must also determine their organization’s As the leader in unified vulnerability management, Nucleus
unique risk tolerance and contextualise assets based on the enables enterprises to prioritize and mitigate vulnerabilities
faster, at scale. Powered by the Nucleus Data Core, the platform
business impact caused by a critical vulnerability being exploited automatically unifies, organizes, and operationalizes finding,
on those systems, to prioritise remediation and mobilise response.” threat, and business, data from all your tools.

- Gavin Millard | VP, Product Management, Tenable

HOW DO WE
MEASURING PERFORMANCE OF PRIORITIZATION

EVALUATE EXPLOIT
No matter what strategy is used, there is a tradeoff between true and false positives and true and false
negatives. We highlight what each of those mean for vulnerabilities by measuring the performance of a
strategy to prioritize CVSS “critical” (9 and above) vulnerabilities.

PREDICTIONS?
At the outset of evaluating the performance of EPSS, it makes sense to discuss what
that entails and how we measure it. A perfect prediction model will correctly identify all
vulnerabilities that are exploited (true positives) with zero omissions (false negatives)
or false positives. No prioritization method is perfect, of course, which can be seen in
the diagram below that depicts the accuracy of using CVSS scores above 9 to predict
exploitation. This sets up the classic performance metrics of precision and recall that are
widely used to evaluate classification and prediction models. In the context of VM, we term
these efficiency (precision) and coverage (recall) to make the concepts more memorable
and practical.

Coverage (recall) Efficiency (precision) Effort

Measures the Measures the accuracy Measures the overall

completeness of of prioritizations. workload created by the Meauring the relative workload as
prioritizing the What percentage of prioritization strategy Effort the proportion of vulnerabilities
prioritized out of all the possible
exploitation activity. vulnerabilities prioritized and is simply the vulnerabilities: (TP+FP)/everything.

What percentage of (for remediation) were percentage of prioritized

all known exploited actually exploited? vulnerabilities out of all Meauring the accuracy of our
strategy as the vulnerabilities
vulnerabilities were If 100 vulnerabilities vulnerabilities. Typically, Efficiency
with exploitation activity out of
the all prioritized vulnerabilities:
correctly prioritized? If were predicted to be we can improve our TP/(TP+FP).
100 vulnerabilities get exploited but only coverage by increasing
exploited but only 40 of 60 had observed our effort, but this The perfect model would identify all
Meauring the completeness of
Coverage our strategy as the vulnerabilities
those were prioritized, exploitation activity, comes at the expense of exploited vulns (TPs) with zero omissions we prioritized out of the all
vulnerabilities with exploitation
the coverage is 40%. the efficiency is 60%. our efficiency. We can (FNs) or false positives (FPs). activity: TP/(TP+FN).
Technically, coverage Technically, efficiency only increase all three
is the true positives is the true positives metrics at the same
divided by the sum of divided by the sum of time by having a better No perfect approach exists, but we can
the true positives and the true positives and prioritization strategy. objectively compare their performance with
false negatives. false positives. well-established metrics.

A Visual Exploration of Exploitation in the W i l d Pag e | 31

HOW DOES CVSS
THE PERFORMANCE OF CVSS

PERFORM?
Even though CVSS was not designed specifically for exploitation prediction, most people will think of
CVSS as having some predictive power for exploitation activity. However, there is very little correlation
between a higher CVSS score and observed exploitation activity.

Since CVSS was used as the example of how It’s only fair to mention here that CVSS
to measure performance in the prior topic, wasn’t made to predict exploitation. That
we might as well see that through to actually said, people often use it that way, and there’s
measure its performance. CVSS has long a general belief that vulnerabilities with

Efficiency (Precision)
been a de facto input for many organizations higher scores are more likely to be attacked
in determining which vulnerabilities should and should therefore be remediated ASAP.
be prioritized for remediation. So, it makes Thus, measuring its performance for this
sense to establish a predictive performance purpose is fair game. Overall, CVSS achieves
baseline with CVSS. coverage by increasing effort with a rather
low and consistent efficiency.
The left chart below plots the coverage
(x-axis), efficiency (y-axis), and effort (dot
size) achieved by using various CVSS score
thresholds to predict exploitation. It’s not
a great look. A strategy of remediating
vulnerabilities with a score of 7 or above—a
common recommendation in security and
compliance standards—would address the
majority (63%) of known exploited CVEs. Coverage (Recall)
However, the efficiency is quite low at 10%,
indicating quite a bit of misplaced effort
spent prioritizing vulnerabilities that did not
have any observed exploitation activity. There’s little correlation: just
Many assume high CVSS scores
~37% of vulnerabilities with a
indicate a high likelihood of
CVSS score of 9+ have known
exploitation.
exploits.

TAKEAWAY: CVSS is a very inefficient predictor of exploitation;

it just wasn’t designed for that purpose.

A Visual Exploration of Exploitation in the W i l d Pag e | 3

HOW DOES THE KEV THE PERFORMANCE OF CISA’S KNOWN EXPLOITED
VULNERABILITY (KEV) LIST

PERFORM? As with many sources leveraging expertise or threat intel, the KEV list is quite efficient. Out of the 1,117
CVEs on the KEV, we have observed exploitation activity on 705 (63%) at some point, but that rather
high efficiency drops off by 10% (to 53% on average) as we measure month-to-month.

Another popular resource for prioritizing apparent that about a third of CVEs in the KEV
remediation is the Known Exploited are NOT among those observed by EPSS data
Vulnerabilities Catalog (KEV) maintained by sources. That alone makes the KEV useful for The KEV performs
the Cybersecurity and Infrastructure Security VM teams to help prioritize remediation. well for efficiency
Agency (CISA). Although created to guide U.S. and effort metrics..
government agencies, CISA recommends
that all organizations monitor the KEV to But the KEV’s real strength is its performance
reduce the likelihood of compromise by on the efficiency scale. It’s a great (and FREE!)
known threat actors (and we do too, for resource for vulnerability remediation that, A third of the
what it’s worth). We’ll briefly review how the unlike CVSS, will minimize wasted effort. It vulnerabilities it
KEV performs relative to that goal. shouldn’t be the totality of your prioritization marks as
strategy, but it’s a strong indicator for VM exploited aren’t
teams to build on. in our datasets.
The Venn diagram makes it clear that EPSS
data sources contain evidence of exploitation
for many vulnerabilities that are not on the
KEV. That’s not a knock; the KEV is relatively
new and has a particular focus. It is also

Efficiency (Precision)
TAKEAWAY: The KEV is a good starting point for prioritizing
remediation with little wasted effort.

A Visual Exploration of Exploitation in the W i l d Coverage (Recall) Pag e | 35

CAN METADATA
THE PERFORMANCE OF INDIVIDUAL VULNERABILITY
ATTRIBUTES

HELP PREDICT Nobody would base their prioritization on a single variable, but it’s informative to look at their
perfomance. It can align our expectations and build our intuition about how different vulnerability
features may help predict exploitation.

EXPLOITATION?
One thing security researchers do when It is indeed true that a large proportion
assessing vulnerabilities is parse the of exploited vulns enable remote code
descriptive details included with the execution (high coverage). But so do many
published CVE. This gives rise to inferences more that haven’t been exploited (leading
like “This enables remote code execution; to low efficiency and high effort). There are
it’s gonna be bad.” some decent indicators here, but on the
whole, these don’t perform very well as
individual predictors of exploitation.
Can such inferences form the basis of reliable
predictions? This series of charts plot the
performance of CVSS metrics, Common

Efficiency (Precision)
Weakness Enumeration (CWE) types, various
attributes derived from the description, and
the associated vendor(s). EPSS includes all of these info
sources (and more) as inputs
for its predictions.

TAKEAWAY: None of these attributes are reliable individual

predictors; they’re best modeled collectively.

Coverage (Recall)

A Visual Exploration of Exploitation in the W i l d Pag e | 37

CAN EXPLOIT TOOLS THE FULL VIEW OF THE PERFORMANCE OF INDIVIDUAL
VULNERABILITY ATTRIBUTES

HELP PREDICT Curated lists based on expertise (Metasploit and off sec scanners) increase in efficiency at the expense
of coverage with less effort (smaller circles). Meanwhile, static attributes of vulnerabilities can drive a

EXPLOITATION?
lot of effort to achieve coverage but at a much lower efficiency.

This next chart brings together the sources of

metadata from the previous page and adds
some popular exploit tools and databases
to the mix (in red). While Metasploit, Sn1per, The exploit tools and
ExploitDB and their ilk aren’t intended databases shown here
to score severity like CVSS or to predict are limited in scope (low
exploitation like EPSS, they do offer a window coverage).
into which vulnerabilities have seemed

Efficiency (Precision)
interesting enough to be “weaponized” to But a high proportion of the
some degree. Given that, it makes sense that
the vulnerabilities included in them would vulnerabilities they contain
correlate with those exploited in the wild. have known exploitation in
The results shown here bear that out. the wild (high efficiency).

They include vulnerabilitiess

There’s an important pattern here. Note that likely to be on attackers’
everything listed tends to perform better radar and thus worth
for coverage OR efficiency. None does both prioritizing for remediation.
very well. Perhaps a model that factors all of
this into making exploit predictions can do
better? We’ll find out on the next page.

TAKEAWAY: Exploit tools and databases generally offer

high signal-to-noise ratio (high efficiency) but with limited
individual coverage.
Coverage (Recall)

A Visual Exploration of Exploitation in the W i l d Pag e | 39

HOW DOES EPSS THE PERFORMANCE OF THE EXPLOIT PREDICTION SCORING
SYSTEM (EPSS)

PERFORM? The output of EPSS is a probability (0%–100%) of exploitation activity being oberved in the next 30 days.
Because it’s a continuous value, the “point” slides across the plot, creating a line from high eff iciency to
high coverage.

At 20+ pages into a study that promises to

evaluate the performance of EPSS, we are
now ready to make good on that promise. 100%
Recall that the perfect predictive model will
max out on the coverage and efficiency axes
in the upper right. Nothing we’ve shown thus
far comes close, but EPSS has moved closer
to that coveted upper-right corner with each Remediating vulnerabilities
successive version. with an EPSS score of 0.6+
achieves a coverage of ~60%
with 80% efficiency.

Efficiency (Precision)
This plot likely prompts the question “Why
lines vs. dots?” That stems from EPSS
producing scores ranging from 0 to 1, with
each achieving different coverage and At 0.1+, that changes to 80%
efficiency levels. Each line plots the daily coverage and 50% efficiency.
results for each version’s lifespan. The
number bubbles indicate the performance
of thresholds in that range. We discuss how
to choose the ideal EPSS threshold for your
team later.

TAKEAWAY: Versions of EPSS show increasingly strong

performance across the range of scores.
Coverage (Recall)

A Visual Exploration of Exploitation in the W i l d Pag e | 41

HOW DO EPSS AND
PERFORMANCE FROM CVSS TO EPSS

CVSS COMPARE?
It is difficult to map direction from CVSS scores to EPSS scores. But if we hold one of the performance
measures the same (such as effort or coverage) we can look at the changes in the other metrics as shown
here.

Since we’ve now measured the predictive If we compare based on

performance of EPSS and CVSS, we suspect equivalent level of effort
readers may have this question. There are
many ways to go about answering it, but (remediating about 21% of
we think the “apples to apples” comparison vulnerabilities), EPSS achieves
shown here is the most fair and useful. almost 3x more coverage
(93% vs. 37%) and over twice
the efficiency (16% vs. 7%) of
We feel compelled to assert that we’re not CVSS.
trying to pick on CVSS here. But it’s important
to understand which of these scoring systems
is better suited to prioritizing remediation
based on the probability of exploitation.
EPSS clearly wins in that regard.

Achieving equivalent coverage

(87%) requires 6x more
effort (63% vs. 10%) and is
6 times less efficient (5% vs.
30%) with CVSS than with
EPSS.

TAKEAWAY: EPSS performs demonstrably better than CVSS in

equivalent metric-based comparisons.

A Visual Exploration of Exploitation in the W i l d Pag e | 43

WHAT EPSS SCORE
WARRANTS PRIORITY PICKING THRESHOLDS FOR EPSS

REMEDIATION?
Select a threshold for EPSS along the horizontal axis and trace it upwards to each metric to determine
the coverage, efficiency, and level of eff ort. These represents the performance of EPSS from March 7,
2023 to to May 1, 2024.

We showed previously that EPSS produces a

range of scores that achieve different levels
of coverage and efficiency. Because of that,
many organizations attempting to use EPSS
wonder what score(s) should trigger priority Using EPSS to prioritize
remediation. EPSS doesn’t come with that remediation is a balancing act
guidance because the answer is ultimately of competing priorities.

Value of Metric
dependent upon your organization’s risk
tolerance and capabilities. The figure below
should offer some insight that helps guide
these decisions. There’s no “easy button” to
achieve high coverage.

Maximizing coverage comes with the cost

of lower efficiency and higher effort. Risk-
averse firms may be willing to make that Performance metrics can help
trade. Resource-strained or less mature dial in and maintain a balance
organizations may wish to maximize that works for your firm.
efficiency first and work to broaden coverage
over time.

EPSS Probability

The percent of
The percent of
Coverage vulnerabilities with
prioritized
observed exploitation The percent of Efficiency
Effort vulnerabilities with
activity in the vulnerabilities being

TAKEAWAY: EPSS supports a remediation strategy tailored to

observed exploitation
following 30 days prioritized activity in the
that had been
following 30 days.
prioritized.

your risk tolerance and capabilities.

A Visual Exploration of Exploitation in the W i l d Pag e | 45

CONCLUSION APPENDIX
What’s next for EPSS EPSS Overview & History

Thanks for sticking with us this far. If you are still hungry for May Cyentia released the first report in the Prioritization to
more, please visit First.org and consider joining the EPSS Special Prediction series with Kenna Security. This research
Interest Group (SIG). The SIG discusses all things EPSS and is 2018 launched discussions that would lead to EPSS.
working on the adoption of EPSS and discussing ways that EPSS
can and should be used in modern vulnerability management
practices. If you still want more, much of the details behind EPSS First EPSS model and performance results presented
at the Workshop on the Economics of Information
June
are covered in the handful of publications we have published
about EPSS (see the website). Security (WEIS) conference in Boston, MA. 2019

The Exploit Prediction Scoring System is and always will be data- Pre-publication paper “Exploit Prediction Scoring
driven. Because of that we are continually working to expand August System” was presented at Blackhat, Las Vegas, NV
the coverage of our data. Additionally, and with the help of 2019 and later published to Digital Threats: Research and
sponsorships we are upgrading our data collection infrastructure Practice in July 2021.
this summer and will be releasing the next version of EPSS “real
soon now” (watch the website!)
EPSS Special Interest Group formed at FIRST.org; first
meeting held April 17th, 2020.
February
Having gone through all of that, the future for EPSS is simple: 2020
more of the same but better. We want to keep EPSS as simple as
possible and to keep EPSS exactly what it is, a prediction scoring September First EPSS paper published in the Journal of
system that anyone can use. We hope that we can continue to Cybersecurity titled “Improving vulnerability
improve and evolve EPSS, so please, join in the discussion, share 2020 remediation through better exploit prediction.”
your thoughts or better yet, share your data!
Cyentia began producing daily EPSS scores published January
via FIRST.org.
2021
February EPSS version 2 published based on a more powerful
machine learning model and more data sources.
2022
“Enhancing Vulnerability Prioritization: Data-Driven February
Exploit Predictions with Community-Driven Insights”
posted to arxiv; presented at WEIS 2023 in July 2023. 2023

March EPSS version 3 published with further improvements

to the core ML model and even more data sources.
2023
A Visual Exploration of Exploitation in the W i l d Pag e | 47
A VISUAL EXPLORATION OF
EXPLOITATION
IN THE WILD