International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288

The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

Technical Security Metrics Model in Compliance with ISO/IEC 27001

Standard

M.P. Azuwa, Rabiah Ahmad, Shahrin Sahib and Solahuddin Shamsuddin

[email protected], {rabiah,shahrin}@utem.edu.my
[email protected]

ABSTRACT 1 INTRODUCTION

Technical security metrics provide The phenomena of instant grow and

measurements in ensuring the effectiveness increasing number of cyber attacks has
of technical security controls or technology urged the organizations to adopt security
devices/objects that are used in protecting standards and guidelines. International
the information systems. However, lack of Organization for Standardization and the
understanding and method to develop the
technical security metrics may lead to
International Electrotechnical
unachievable security control objectives and Commission (ISO/IEC) has developed
inefficient implementation. This paper the ISO/IEC 27000 series of standards
proposes a model of technical security that have been specifically reserved for
metrics to measure the effectiveness of information security matters. Through
network security management. The ISO/IEC 27001 Information Security
measurement is based on the security Management System (ISMS) –
performance for (1) network security Requirements [1], the organization may
controls such as firewall, Intrusion Detection comply and obtain the certification in
Prevention System (IDPS), switch, wireless increasing level of protection for their
access point and network architecture; and information and information systems.
(2) network services such as Hypertext
Transfer Protocol Secure (HTTPS) and
Information security metrics can be
virtual private network (VPN). The ineffective tools if organizations do not
methodology used is Plan-Do-Check-Act have data to measure, procedures or
process model. The proposed technical processes to follow, indicators to make
security metrics provide guidance for good protection decisions and people to
organizations in complying with develop and report to the management.
requirements of ISO/IEC 27001 Information To be useful, measurement of
Security Management System (ISMS) information security effectiveness
standard. The proposed model should also should be comparable. Comparisons are
be able to provide a comprehensive usually made on the basis of quantifiable
measurement and guide to use ISO/IEC measurement of a common
27004 ISMS Measurement standard.
characteristic. The main problems in the
information security metrics
KEYWORDS
development are identified; (i) lack of
Information security metrics, technical
clarity on defining quantitative effective
security metrics model, measurement, security metrics to the security standards
vulnerability assessment, ISO/IEC and guidelines; (ii) lack of method to
27001:2005, ISO/IEC 27004:2009, Critical guide the organizations in choosing
National Information Infrastructure. security objectives, metrics and

280
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

measurements for mitigating current The metrics are derived from

cyber attacks [2][3]. comparing two or more
measurements taken over time with
Hulitt and Vaughn [4] report, lack of a predetermined baseline.
clarity in a standard quantitative metric
to describe information system’s level of Brotby The metric is a term used to denote
compliance with the FISMA standard, [9] a measure based on a reference and
even though thorough and repeatable involves at least two points, the
measure and the reference. A
compliance assessment conducted using
security is the protection from or
Risk Management Framework (RMF). absence of danger.
Bellovin [5] remarks that defining The security metrics are categorized
metrics is hard. It is not infeasible, by what they measure. The
because an attacker’s effort is often measures include the process,
performance, outcomes, quality,
linear, even when the exponential
trends, conformance to standards
security work is needed. Those pursuing and probabilities.
the development of a security metrics Masera et “Security metrics are indicators,
program should think of themselves as al. [10] and not measurements of security.
pioneers and be prepared to adjust Security metrics highly depend on
strategies as experience dictate [6]. It is the point of reference taken for the
measurement, and shouldn’t be
also known that ISO/IEC 27001 considered as absolute values with
provides generic guidance in developing respect to an external scale.”
the security objectives and metrics and
still lack of method to guide the Hallberg “A security metric contains three
organizations [2][3]. et al. [11] main parts: a magnitude, a scale
and an interpretation.
The security values of systems are
1.1 Information Security Metrics measured according to a specified
magnitude and related to a scale.
In understanding the meaning of The interpretation prescribes the
information security metrics, the security meaning of obtained security
values.”
practitioners and researchers have
simplified their definitions of Lundholm The measurement quantifies only a
information security metrics and et al. [12] single dimension of the object of
measures (as described in Table 1). measurement that does not hold
value (facilitate decision making) in
Table 1: Definitions of Information Security itself.
Metrics and Measures The metric is derived from two or
more of the measurement to
Author Definition demonstrate an important
correlation that can aid a decision.
Stoddard A metric is a measurement that is
et al. [7] compared to a scale or benchmark
to produce a meaningful result. From these definitions, we propose the
Metrics are a key component of risk
management.
definition as information security
metrics is a measurement standard for
Savola [8] Security Metric is a quantitative and information security controls that can be
objective basis for security quantified and reviewed to meet the
assurance. It eases in making security objectives. It facilitates the
business and engineering decisions
relevant actions for improvement,
concerning information security.
provide decision making and guide

281
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

compliancy to security standards. 1.3 Effective Measurement

Information security measurement is a Requirement from ISO/IEC 27001
process of measuring/assessing the Standard
effectiveness of information security
controls that can be described by the Information security measurement is a
relevant measurement methods to mandatory requirement in ISO/IEC
quantify data and the measurement 27001 standard where it is indicated in a
results are comparable and reproducible. few clauses in: 4.2.2(d) “Define how to
Hence, information security measure the effectiveness of the selected
measurement is a subset of information controls or groups of controls and
security metric. specify how these measurements are to
be used to assess control effectiveness to
1.2 Technical Security Metrics and produce comparable and reproducible
Measurement results”, 4.2.3(c) “Measure the
effectiveness of controls to verify that
We found the research activities for security requirements have been met”,
technical security metrics are very 4.3.1(g) “documented procedures needed
limited. Also, there is lack of specific by the organization to ensure the
technical security metrics research to effective planning, operation and control
measure the technical security controls of its information security processes and
from a total 133 security controls from describe how to measure the
the ISO/IEC 27001 standard. effectiveness of controls”, 7.2(f) “results
Vaughn et al. [13] define Technical from effectiveness measurements” and
Target of Assessment (TTOA) as to 7.3(e) “Improvement to how the
measure how much a technical object, effectiveness of controls is being
system or product is capable of measured”. The importance of
providing assurance in terms of information security measurement is
protection, detection and response. well defined in these clauses.
According to Stoddard et al. [7],
technical security metrics are used to
assess technical objects, particularly 2 SECURITY METRICS
products or systems [8], against DEVELOPMENT APPROACH
standards; to compare such objects; or to
assess the risks inherent in such objects. The development of technical security
Additionally, the technical security metrics model (TSMM) is derived from
metrics should be able to evaluate the the following approach:
strength in resistance and response to
attacks and weaknesses (in terms of (1) The requirements of technical
threats, vulnerabilities, risks, anticipation security controls are based on
of losses in face of attack) [13]. At the ISO/IEC 27002 ISMS – Code of
same time, it indicates the security Practices standard [14].
readiness with respect to a possible set (2) Identify relevant security
of attack scenarios [10]. requirements
(3) Achieve security performance
objectives
(4) Align to risk assessment value

282
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

(5) The development of technical the Annex A of ISO/IEC 27001

security metrics should not be an standard.
extensive list, but more focus on We define technical security metrics as a
the critical security controls that measurement standard to address the
provide high impact to the performance of security
organizations. According to countermeasures within the technical
Lennon [15], “the metrics must be security controls and to fulfill the
prioritized to ensure that the final security requirements. The technical
set selected for initial security measures are based on
implementation facilitates information security performance
improvement of high priority objectives that can be accomplished by
security control implementation. quantifying the implementation,
Based on current priorities, no efficiency, and effectiveness of security
more than 10 to 20 metrics at a controls.
time should be used. This ensures ISO/IEC 27002 [14] provides the best
that an IT security metrics program practice guidance in initiating,
will be manageable.” implementing or maintaining the
(6) Align to risk assessment value security control in the ISMS. This
(7) Ease of measurement. standard regards that “not all of the
(8) Provide the process to obtain controls and guidance in this code of
data/evidence, method and formula practice may be applicable and
to assess the security measurement additional controls and guidelines not
(9) Resistance and response to known included in this standard may be
and unknown attacks required”.
(10) Provide the threshold values to Federal Information Processing
determine the level of protection Standards 200 (FIPS 200) [16] defines
(11) Provide actions to improve technical controls as “the security
(12) Comply to the ISO/IEC 27001 controls (i.e., safeguards or
standard countermeasures) for an information
system that are primarily implemented
and executed by the information system
3 TECHNICAL SECURITY through mechanisms contained in the
METRICS MODEL (TSMM) hardware, software, or firmware
components of the system”. These are
The development of TSMM is based on the basis of our definition for technical
Plan-Do-Check-Act (PDCA) model. The security controls.
development of TSMM is described in Based on NIST SP800-53 guidelines
Figure 1. [17], the technical security controls
comprise of:
3.1 PLAN Phase: (Selection of (1) Access Control (AC-19 controls)
Controls and Definition) (2) Audit and Accountability (AU-
14 controls)
The focus is on the technical security (3) Identification and Authentication
controls that will be extracted from the (IA-8 controls)
total 133 security controls as stated in (4) System and Communications
Protection (SC-34 controls)

283
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

• Operational controls:
The total of technical security controls Configuration Management
from NIST SP800-53 guidelines is (CM), Maintenance (MA),
seventy-five (75). In the Appendix H of Media Protection (MP),
[18], the technical security controls are Physical and Environmental
extracted from Table H-2. This table is Protection (PE), Personnel
mapping from the security controls in Security (PS), System and
ISO/IEC 27001 (Annex A) to NIST Information Integrity (SI).
Special Publication 800-53. We extract
and analyze these technical security Figure 1: Technical Security Metrics Model
controls. We discover that: (TSMM)
(1) Within three (3) main domains
from ISO/IEC 27001 (Annex A)

 A.10 Communications and

that include:

 A.11 Access Control

operations management

 A.12 Information systems

acquisition, development and
maintenance
(2) The initial total of technical
security controls is forty-five
(45).
(3) The identified technical security
controls only require a process or
policy implementation and not
related to technical
implementation, such as
A.11.1.1 Access control policy, The technical security controls should be
A.11.4.1 Policy on use of practical, customized and measured
network services, A.11.5.1 according to organization’s business
Secure log-on procedures, requirements and environments.
A.11.6.2 Sensitive system A risk management approach will be
isolation, A.11.7.2 Teleworking, used in identifying the relevant security
A.12.3.1 Policy on the use of controls. Threat and vulnerability
cryptographic control and assessment will be carried out.
A.12.6.1 Control of technical Threat and vulnerability assessment will
vulnerabilities. be carried out. Also, identifying both
(4) There are relationships with other impact and risk exposure to determine
security controls in NIST SP800- the prioritization of security controls.
53 document, including:
• Management controls: Cyber-Risk Index: A cyber-risk index is
Security Assessment and used to evaluate the vulnerability and
Authorization (CA), Planning threat probabilities related to the
(PL), System and Services successfulness of current and future
Acquisition (SA) attacks. Attack-Vulnerability-Damage

284
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

(AVD) model [19] and Common connectivity of hosts, and possible attack
Vulnerability Scoring System (CVSS) - paths. These factors are modeled into
Base Metric [20] are used to determine three network dimensions: Vulnerability,
this weighted-index. We will extent and Exploitability, and Attackability. The
include the criticality or impact of loss to overall VEA-bility score, a numeric
the organization. The CVSS base score value in the range [0,10], is a function of
is calculated using the information these three dimensions.
provided by the U.S. National
Vulnerability Database (NVD) Common At this phase, the data collection must be
Vulnerability Scoring System Support easily obtainable and the measurements
v2 [21] and other relevant Cyber are not complicated. The measurement
Emergency Response Team (CERT) should be able to cater for current
Advisories and Report. (through audit report and evidence of
events) and future attacks.
3.2 DO Phase: (Effective
Measurement) 3.3 CHECK Phase: (Security
Indicators and Corrective Action)
The security requirements describe the
actual security functional for technical In verifying the effectiveness of controls,
security controls in protecting the we measure how much the control
information systems. Security functional decreases the probability of realization
includes the identification and of the described risks. The attributes
authentication, access control, must be significant in determining the
configurations/algorithm, architecture increase or decrease of risk. The
and communication. expected measure function can be
A set of performance objectives is derived by the percentage of the
developed for each security requirement. successful or failure occurrences. For
Vulnerability Assessment (VA) Index: example, number of patches successfully
The VA index is that can be derived by installed on information systems (>
conducting the security or vulnerability 95%), number of security incidents
assessment to the information systems caused by attacks from the network (<
through a simulation assessment, 3%). The determination of the
vulnerability scanning or penetration percentage should consider that even
testing. This is based on the current though the security controls are
assessment of potential attacks and will implemented, the risk of attacks can still
be weighted-index using the numeric occur. Therefore, the percentage depicts
CVSS scores: "Low" severity (CVSS the strength of the existing security
base score of 0.0-3.9), "Medium" controls in mitigating the risks.
severity (CVSS score of 4.0-6.9) and
"High" severity ( CVSS base score of Security Indicator Index: If the measure
7.0-10.0). The VAI can also be derived is equal to or below the
from Vulnerability-Exploits-Attack recommendation, the risk is adequately
(VEAbility) metrics [22]. The VEAbility controlled, thus explain the effectiveness
measures the security of a network that of the security controls. The proposed
is influenced by the severity of existing indicators are the trends of the derived
vulnerabilities, distribution of services, measures and they must be within the

285
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

same measurement scale in order to

establish that the risk is adequately 4 CONCLUSIONS AND FUTURE
controlled [23]. This indicator index can WORK
also act as a compliance index to
ISO/IEC 27001 standard. Algorithm or Malaysia government has seen the
calculation combining one or more base importance of Critical National
and/or derived measures with associated Information Infrastructure (CNII)
decision criteria. For example: 0-60% - organizations to protect their critical
Red; 60-90% - Yellow; 90-100% Green. information systems. In the year of 2010,
the government has mandated for their
Decision Criteria: Thresholds, targets, or systems to be ISO/IEC 27001 ISMS
patterns used to determine the need for certified within 3 years [24].
action or further investigation, or to
describe the level of confidence in a The ISO 27001 certification is one of the
given result (for example, Red – most used corporate best practices for IT
intervention is required, causation security standards, addressing
analysis must be conducted to determine management requirements as well as
reasons for non-compliance and poor identifying specific control areas for
performance; Yellow – indicator should information security. It provides a
be watched closely for possible slippage comprehensive framework for designing
to Red; Green – no action is required). and implementing a risk-based
Information Security Management
Corrective actions provide the range of System. The requirements and guidance
Potential changes in improving the cover policies and actions that are
efficiency and effectiveness of the necessary across the whole range of
security controls. They can be prioritized information security vulnerabilities and
based on overall risk mitigation goals threats. By customizing the security
and select based on cost-benefit analysis. requirements from ISO/IEC 27002 and
other relevant security standards and
guidelines, the CNII organizations will
3.4 ACT Phase: implement the necessary security
controls in compliance with ISO/IEC
The developed technical security metric 27001 ISMS standard.
and measurement will be validated by
the respective organizations. The metric The proposed TSMM is to provide
is to comply to ISO/IEC 27001 standard guidance for CNII organizations to
requirements. The development of measure the effectiveness of the network
technical security metrics will be based security controls in compliance with
on Information security measurement ISO/IEC 27001 standard. The relevant
model in ISO/IEC 27004 standard. type of information security
measurement and metrics are interrelated
The measurement result should be and worth to use in aligning with
reported to the management in ensuring business risk management. We also want
the continuity and improvement of to explore the usability of the ISO/IEC
information security in the organization. 27004 standard and conduct a case study
at several CNII organizations.

286
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

Metrics – State of Practice,” Institute for

Information Infrastructure Protection (I3P),
vol. Research R, no. August, 2005.
ACKNOWLEDGMENT 8. R. Savola, “Towards a Security Metrics
Taxonomy for the Information and
The authors wish to acknowledge and thank Communication Technology Industry,” in
members of the research teams of the Long International Conference on Software
Term Fundamental Research Grant Scheme Engineering Advances, 2007.
(LRGS) number 9. W. K. Brotby, Information Security
LRGS/TD/2011/UKM/ICT/02/03 for this Management Metrics: A Definitive Guide to
work. The research scheme is supported by Effective Security Monitoring and
Measurement. Auerbach Publications, 2009.
the Ministry of Higher Education (MOHE)
10. M. Masera and I. N. Fovino, “Security
under the Malaysian R&D National Funding metrics for cyber security assessment and
Agency Programme. testing,” Joint Research Centre of the
European Commission,, vol. ESCORTS D4,
no. August, pp. 1–26, 2010.
5 REFERENCES 11. J. Hallberg, M. Eriksson, H. Granlund, S.
Kowalski, K. Lundholm, Y. Monfelt, S.
Pilemalm, T. Wätterstam, and L. Yngström,
1. International Organization for
“Controlled Information Security: Results
Standardization and International
and conclusions from the research project,”
Electrotechnical Commission, “Information
FOI Swedish Defence Research Agency, pp.
technology - Security techniques -
1–42, 2011.
Information security management systems-
12. H. Lundholm, K., Hallberg, J., Granlund,
Requirements,” ISO/IEC 27001:2005, 2005.
“Design and Use of Information Security
2. R. Barabanov, S. Kowalski, and L.
Metrics,” FOI, Swedish Defence Research
Yngström, “Information Security Metrics:
Agency, pp. ISSN 1650–1942, 2011.
Research Directions,” FOI Swedish Defence
13. J. Rayford B. Vaughn, R. Henning, and A.
Research Agency, 2011.
Siraj, “Information Assurance Measures and
3. C. Fruehwirth, S. Biffl, M. Tabatabai, and E.
Metrics - State of Practice and Proposed
Weippl, “Addressing misalignment between
Taxonomy,” in Proceedings of the 36th
information security metrics and business-
Hawaii International Conference on System
driven security objectives,” Proceedings of
Sciences, 2003, p. 10 pp.
the 6th International Workshop on Security
14. International Organization for
Measurements and Metrics - MetriSec ’10,
Standardization and International
p. 1, 2010.
Electrotechnical Commission, “Information
4. E. Hulitt and R. B. Vaughn, “Information
technology - security techniques - Code of
system security compliance to FISMA
practice for information security
standard: A quantitative measure,” 2008
management,” ISO/IEC 27002:2005, vol.
International Multiconference on Computer
2005, 2005.
Science and Information Technology, no. 4,
15. E. B. Lennon, M. Swanson, J. Sabato, J.
pp. 799–806, Oct. 2008.
Hash, L. Graffo, and N. Sp, “IT Security
5. S. M. Bellovin, “On the Brittleness of
Metrics,” ITL Bulletin, National Institute of
Software and the Infeasibility of Security
Standards and Technology, no. August,
Metrics,” IEEE Security & Privacy
2003.
Magazine, vol. 4, no. 4, pp. 96–96, Jul.
16. W. J. Carlos M. Gutierrez, “Federal
2006.
Information Processing Standards 200 -
6. K. Stouffer, J. Falco, and K. Scarfone,
Minimum Security Requirements for
“Guide to Industrial Control Systems ( ICS )
Federal Information and Information
Security,” National Institute of Standards
Systems,” National Institute of Standards
and Technology, NIST Special Publication
and Technology,, no. March, 2006.
800-82, no. June, 2011.
17. Computer Security Division and Information
7. J. Stoddard, M., Bodeau, D., Carlson, R.,
Technology Laboratory, “Recommended
Glantz, C., Haimes, Y., Lian, C., Santos, J.,
Security Controls for Federal Information
and Shaw, “Process Control System Security
Systems and Organizations,” National

287
 International Journal of Cyber-Security and Digital Forensics (IJCSDF) 1(4): 280-288
The Society of Digital Information and Wireless Communications (SDIWC), 2012 (ISSN: 2305-0012)

Institute of Standards and Technology, NIST

Special Publication 800-53 , Revision 3,
2010.
18. Computer Security Division and I. T.
Laboratory, “Security and Privacy Controls
for Federal Information Systems and
Organizations,” National Institute of
Standards and Technology, NIST Special
Publication 800-53 , Revision 4, no.
February, 2012.
19. T. Fleury, H. Khurana, and V. Welch,
“Towards A Taxonomy Of Attacks Against
Energy Control Systems,” in Proceedings of
the IFIP International Conference on
Critical Infrastructure Protection, 2008.
20. P. Mell, K. Scarfone, and S. Romanosky, “A
Complete Guide to the Common
Vulnerability Scoring System,” Forum of
Incident Response and Security Teams,
FIRST Organization, pp. 1–23, 2007.
21. “NVD Common Vulnerability Scoring
System Support v2,” NIST, National
Vulnerability Database (NVD),
http://nvd.nist.gov/cvss.cfm?version= 2.
22. M. Tupper and a. N. Zincir-Heywood,
“VEA-bility Security Metric: A Network
Security Analysis Tool,” 2008 Third
International Conference on Availability,
Reliability and Security, pp. 950–957, Mar.
2008.
23. M. H. S. Peláez, “Measuring effectiveness in
Information Security Controls,” SANS
Institute InfoSec Reading Room,
http://www.sans.org/reading_room/whitepa
pers/basics/measuring-effectiveness-
information-security-controls_33398, 2010.
24. J. P. M. Malaysia, “Pelaksanaan Pensijilan
MS ISO/IEC 27001:2007 Dalam Sektor
Awam,” Unit Pemodenan Tadbiran dan
Perancangan Pengurusan Malaysia
(MAMPU), vol. MAMPU.BPIC, p. 1, 2010.

288