EBOOK MethodologyDeepDive 3.0 - v2 1
EBOOK MethodologyDeepDive 3.0 - v2 1
in Scoring
Methodology
By Bob Sohval, PhD
VP Data Science
1 ©2024
| A Deep Dive in Scoring
SecurityScorecard Methodology
Inc. All Rights Reserved.
Table of Contents
2 || AADeep
Deep Dive
Dive in Scoring
in Scoring Methodology
Methodology
Cybersecurity Ratings
SecurityScorecard evaluates The rise of the internet and its global role in e-commerce, business operations,
organizations’ security profiles communications, and social media, has created both opportunities and risks. While
non-intrusively, using an it can fuel economic growth and speed up the dissemination of news and ideas, the
‘outside-in’ methodology. existence of vulnerabilities in commonly used software products and services, and
poor adherence to recommended security practices can expose organizations to
This approach enables SecurityScorecard significant financial and reputational harm at the hands of malicious actors — including
to operate at scale, measuring and updating
both individuals and nation states.
cybersecurity ratings daily on more than one
million organizations globally.
Cybersecurity ratings provide a means for objectively monitoring the security hygiene of
organizations and gauging whether their security posture is improving or deteriorating
over time. The ratings are valuable for vendor risk management programs, determining
risk premiums for cyber insurance, credit underwriting and financial trading decisions,
M&A due diligence information, executive-level reporting, and for self-monitoring.
Cybersecurity ratings, and the extensive information on which they are based, are also
helpful for assessing compliance with cybersecurity risk standards.
3 DNS Health captured and presented to users for improved awareness, but do not contribute to score.
4 Endpoint Security The security issues measured by SecurityScorecard, along with the assigned factor, severity-
based weight, update cadence and age out window, are presented in the following table.
5 Hacker Chatter
Notes:
6 Informational Leak • Severity levels are subject to change as we continue to improve and refine our scoring
algorithm. These changes will occur as part of our quarterly scoring recalibrations, and the
version number will be updated accordingly.
7 IP Reputation
• Detailed descriptions, risks, and recommendations for each issue type can be found in the
SecurityScorecard platform.
8 Network Security
9 Patching Cadence
10 Social Engineering
5 || AADeep
Deep Dive
Dive in Scoring
in Scoring Methodology
Methodology
Issue Type Factor Severity Recommendation Frequency Age Out
Active CVE Exploitation
IP Reputation LOW Investigate the IP listed in the Findings table below. Then perform an incident response management process. Varies* 15
Attempted
Adware Installation IP Reputation LOW Investigate the devices associated with the IP addresses listed, checking for evidence of adware installations. Varies* 30
Adware Installation Trail IP Reputation LOW Investigate the devices associated with the IP addresses listed, checking for evidence of adware installations. Varies* 365
Reset the password for the compromised account. If the username is no longer active, ensure that no other
Age exposed Information Leak INFO Varies* 15
services link to the affected email address, such as cloud-based applications that your organization uses.
Alleged Breach Incident Hacker Chatter MEDIUM Investigate the alleged activity to determine if it can be substantiated and remediate as necessary. Varies* 30
Review the business necessity of hosting a public proxy server, and remove it from the Internet if possible.
If not possible, consider restricting the service by allowlisting the IP addresses that require access, or
Anonymous Open Proxy IP Reputation LOW Varies* 45
implementing authentication. If there is no known reason for a proxy service to be present, check for
evidence of malware infections or other types of compromise.
Exposing database services to the Internet is not recommended. Consider placing the service behind a VPN,
Apache Cassandra Service
Network Security MEDIUM preventing public access. If making the service private is not possible, restrict the service by allowlisting the IP Weekly 45
Observed
addresses that require access.
Exposing database services to the Internet is not recommended. Consider placing the service behind a VPN,
Apache CouchDB
Network Security MEDIUM preventing public access. If making the service private is not possible, restrict the service by allowlisting the IP Weekly 45
Service Observed
addresses that require access.
Reset the password for the compromised account. If the username is no longer active, ensure that no other
API key exposed Information Leak INFO Varies* 15
services link to the affected email address, such as cloud-based applications that your organization uses.
Apple AirPort Device Detected Network Security LOW Place the wireless administrative portal behind a firewall. Weekly 45
Attack Detected IP Reputation LOW Investigate the devices associated with the IP addresses listed, checking for evidence of malware infections. Varies* 30
Attempted Information Leak Information Leak LOW Investigate the IP listed in the Findings table below. Then perform an incident response management process. Varies* 15
Reset the password. Subscribe to an identity-monitoring service to ensure no unauthorized accounts were made in
Birthday exposed Information Leak INFO Varies* 15
the user's name.
Bitcoin Server Exposed Network Security INFO Assess the business need for exposing a Bitcoin server to the internet, and consider placing it behind a firewall. Weekly 45
Browser Average Age Update the web browsers in question to the latest major release versions. Enable automatic updates
Endpoint Security LOW Varies* None
Indicates Older Versions if available from your web browser vendor and permitted in your environment.
Browser logs contain debug
Application Security LOW Follow best practices to keep sensitive information out of browser logs. Weekly 15
messages
Identification of a CDN could be useful information to your customers and partners, and there is no
CDN Used Network Security LOW Weekly 45
recommended action.
Services presenting expired certificates should cause noticeable failures, so confirm the service is still in use. If the
Certificate Is Expired Network Security LOW Weekly 45
service is not in use, decommission it. Otherwise, contact the CA and arrange issuance of a new certificate.
Certificate Is Revoked Network Security HIGH If the service is not in use, decommission it. Otherwise, contact the CA and arrange issuance of a new certificate. Weekly 45
Certificate Is Self-Signed Network Security LOW If the service is not in use, decommission it. Otherwise, contact the CA and arrange issuance of a new certificate. Weekly 45
Vulnerable Log4j version Update Log4j to 2.17.1 or a later version immediately. This version only runs on Java 8, so make sure to
Application Security INFO Varies* 15
detected update Java if you are using an earlier version.
Apply VMWare's update to any unpatched servers as soon as possible. Otherwise, deactivate OpenSLP services or
Vulnerable VMWare limit access to a list of trusted IP addresses. Maintain up-to-date backups of data that threat actors may target for
Application Security HIGH Weekly 45
ESXi Server Detected encryption. Only expose services to the wider internet when necessary. Consistently monitor network traffic for any
unexpected behavior.
*There is no regular scanning frequency for this issue type. We collect data from multiple sources when it is available.
based on its digital footprint. the domain-IP pair is accepted if the overall confidence level is Based on an independent
satisfactory. The IP digital footprints are updated daily.
Attribution of IPs is a challenging process due to the
assessment by security firm, the
dynamic nature of the internet. Netblocks of IPs can be In addition to IP attribution, SecurityScorecard operates False Positive Rate for domain
a domain discovery process to find related domains and
assigned dynamically by Internet Service Providers (ISP), attribution was less than 1%.
Cloud Service Providers (CSP), and Content Delivery subdomains that are controlled by each scored organization.
protected and more servers mean more the following page, the company has this company has 3 findings, which is worse
Size normalization begins with scatter plots to
chances for an exposed port which should 3 instances of DNS Open Resolver, a than average.
capture how the number of occurrences of a
properly sit behind a firewall. Some issue given issue varies with organization size.
types scale with the number of IPs. Others
For each organization and each security issue,
might scale with the number of related
the number of occurrences of the issue type
domains or number of employees.
is captured. The example shown is open
As noted above, the digital footprint of port 3389, which corresponds to Microsoft’s
different organizations can vary from a single Remote Desktop Protocol. A scatter plot
IP to hundreds of millions of IPs. This range is generated in which every scored entity
spans more than eight orders of magnitude, represents a point on a log-log plot of the
or more than eight multiples of ten. The best logarithm of the number of issue counts (y-axis)
way to make meaningful measurements vs. the logarithm of the number of IPs (x-axis).
over such a large dynamic range is to use A typical scatter plot will contain millions of data
a logarithmic scale, where each increment points, providing a large statistical “mass” for
corresponds to a multiple of 10. better accuracy and stability.
In version 3.0 of our scoring methodology, we no longer use a factor After calculating the raw total score, we scale it based on the expected
score to calculate the total score. We calculate the raw total score (RTS) value of issue finding counts. We want to fairly score an organization by
by adding up all the z-scores associated with issue findings multiplied comparing it to others with similar Digital Footprint sizes.
by their weights, or severity levels (low, medium, high, critical).
Informational and positive issues do not contribute to the score.
We use machine learning to calculate weights based on their
correlation to likelihood of breach: the greater the correlation, the
greater the severity level.
intrusion has occurred, reflecting increased risk. To reflect this risk, its
score is reduced by 10% upon disclosure of a breach. The negative score
impact of the penalty gradually diminishes to zero over a 30-day period.
The score history at right illustrates the impact of a data breach that
occurred in early June. The breach penalty reduced the score by 10
percent from 90 to 81. The penalty’s impact on the score diminished over
the next 30 days and then no longer affected the score in early July.
Users are provided with a Score Planner tool on the platform which enables them to interactively
develop a remediation plan to improve their score. The tool proposes a path to better scores that
users may customize according to their preferences.
In addition, users may dispute findings on their scorecard, due, for example, to compensating
controls or attribution error, by submitting a refute online along with appropriate evidence.
SecurityScorecard reviews each submitted refute and associated supporting evidence and, if
warranted, corrects and updates the scorecard. A refute is accepted or denied within 48-hours
on average. If accepted, the scorecard is updated between 48-72 hours.
• SecurityScorecard employs an “outside-in” approach, which enables • The dynamic nature of the internet also imposes limitations. Dynamic
external assessment of the cybersecurity posture of organizations IPs can be reassigned daily or even hourly. Communication ports
non-intrusively, and at scale. However, it is generally not possible can be opened and closed at different times. Changes in domain
to detect the presence of compensating controls internal to an and IP ownership can occur at any point, but take time to propagate
organization’s network. In such cases, SecurityScorecard will likely across the internet. The dynamic nature of the internet imposes a
report a score that is too low. However, users may correct their fundamental limitation on the accuracy of any process seeking to
own scores to reflect the presence of compensating controls by characterize its current state. Results of such efforts are necessarily
submitting a refute together with supporting evidence. A refute is probabilistic rather than deterministic. For SecurityScorecard, this
accepted or denied within 48-hours on average. If accepted, the means that while scores and attribution are substantially correct, they
scorecard is updated between 48-72 hours. will always be subject to some errors in the form of false positives
and false negatives. SecurityScorecard has developed a suite of
algorithms powered by machine learning to minimize these errors and
is continuously enhancing our system architecture to improve update
cadences to keep attribution and scoring as current as possible.
Q: How often do scoring algorithm changes occur? an upcoming recalibration, you will see a banner on the platform four weeks issue types weights, since factors themselves will not have any weights.
prior to the recalibration date to see the impact on the score changes along
A: Our scoring algorithm changes every three to four years.
with a link to our knowledge base article for more detail.
Q: How are factor scores calculated?
Q: Why do scores fluctuate? A: Factor scores are calculated based on the issue types that are part of
Q: I see an IP on my digital footprint that is not mine. How can I trust
those factors. Each issue type has a weight, based on their severity, which
A: Scores fluctuate marginally from a regular scoring update cadence (once a your attribution?
contributes to the factor score.
month). This enables SecurityScorecard to preserve fair cybersecurity risk ratings
A: SecurityScorecard performs IP attribution using automated processes
in a dynamic threat environment and also to introduce new issue types reflecting
operating at scale, using public RIR, DNS, and SSL data as well as third party
new risk metrics, as needed, to keep users and their ecosystems better informed.
data sources. Owing to the dynamic nature of the internet, in which IPs can Q: How much is the weight of each factor and how are factor weights
Outside of scoring updates, scoring of an organization is a purely deterministic
be reassigned to different organizations by the day or even by the hour, IP determined?
process. It is a function of the digital footprint and the number of security issues
attribution has a fundamentally probabilistic character and cannot be error-
found. If these are unchanged, then the score will also be unchanged. A: There are no longer factor weights with the new scoring algorithm, overall
free. A team of independent pentest experts audited a random sample of scores are a direct representation of issue types. The factors will continue to
SecurityScorecard scorecards to objectively determine the accuracy of have factor scores, but will not have factor weights.
Q: Does SecurityScorecard normalize the score for organizational size? SecurityScorecard IP and domain attribution. They found the attribution
A: Larger enterprises typically have a larger attack surface than smaller process to have an accuracy of 95%. Accuracy was 94% for positively
companies. SecurityScorecard levels the playing field to deliver fair scores for attributing IP addresses, and 100% for DNS records.
Founded in 2013 by security and risk experts Dr. Aleksandr Yampolskiy and Sam Kassoumeh, SecurityScorecard’s
patented rating technology is used by over 30,000 organizations for enterprise risk management, third-party risk
management, board reporting, due diligence, cyber insurance underwriting, and regulatory oversight. SecurityScorecard
is the first cybersecurity ratings company to offer digital forensics and incident response services, providing a 360-degree
approach to security prevention and response for its worldwide customer and partner base.
SecurityScorecard continues to make the world a safer place by transforming the way companies understand, improve
and communicate cybersecurity risk to their boards, employees and vendors. Every organization has the universal right
to their trusted and transparent Instant SecurityScorecard rating. For more information, visit securityscorecard.com
or connect with us on LinkedIn.
Get Started