Cybersecurity Log Management
Cybersecurity Log Management
Karen Scarfone
Murugiah Souppaya
Karen Scarfone
Scarfone Cybersecurity
Murugiah Souppaya
Computer Security Division
Information Technology Laboratory
October 2023
1 Certain commercial entities, equipment, or materials may be identified in this document in order to describe an
2 experimental procedure or concept adequately. Such identification is not intended to imply recommendation or
3 endorsement by the National Institute of Standards and Technology (NIST), nor is it intended to imply that the
4 entities, materials, or equipment are necessarily the best available for the purpose.
5 There may be references in this publication to other publications currently under development by NIST in
6 accordance with its assigned statutory responsibilities. The information in this publication, including concepts and
7 methodologies, may be used by federal agencies even before the completion of such companion publications. Thus,
8 until each publication is completed, current requirements, guidelines, and procedures, where they exist, remain
9 operative. For planning and transition purposes, federal agencies may wish to closely follow the development of
10 these new publications by NIST.
11 Organizations are encouraged to review all draft publications during public comment periods and provide feedback
12 to NIST. Many NIST cybersecurity publications, other than the ones noted above, are available at
13 https://csrc.nist.gov/publications.
14 Authority
15 This publication has been developed by NIST in accordance with its statutory responsibilities under the Federal
16 Information Security Modernization Act (FISMA) of 2014, 44 U.S.C. § 3551 et seq., Public Law (P.L.) 113-283.
17 NIST is responsible for developing information security standards and guidelines, including minimum requirements
18 for federal information systems, but such standards and guidelines shall not apply to national security systems
19 without the express approval of appropriate federal officials exercising policy authority over such systems. This
20 guideline is consistent with the requirements of the Office of Management and Budget (OMB) Circular A-130.
21
22 Nothing in this publication should be taken to contradict the standards and guidelines made mandatory and binding
23 on federal agencies by the Secretary of Commerce under statutory authority. Nor should these guidelines be
24 interpreted as altering or superseding the existing authorities of the Secretary of Commerce, Director of the OMB, or
25 any other federal official. This publication may be used by nongovernmental organizations on a voluntary basis and
26 is not subject to copyright in the United States. Attribution would, however, be appreciated by NIST.
39 Submit Comments
40 [email protected]
41
42 National Institute of Standards and Technology
43 Attn: Applied Cybersecurity Division, Information Technology Laboratory
44 100 Bureau Drive (Mail Stop 2000) Gaithersburg, MD 20899-2000
45 All comments are subject to release under the Freedom of Information Act (FOIA).
46
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
47 Abstract
48 A log is a record of events that occur within an organization’s computing assets, including
49 physical and virtual platforms, networks, services, and cloud environments. Log management is
50 the process for generating, transmitting, storing, accessing, and disposing of log data. It
51 facilitates log usage and analysis for many purposes, including identifying and investigating
52 cybersecurity incidents, finding operational issues, and ensuring that records are stored for the
53 required period of time. This document defines a playbook intended to help any organization
54 plan improvements to its cybersecurity log management.
55 Keywords
56 auditing; cybersecurity artifacts; incident response; log management; logging; threat detection.
i
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
68 Audience
69 This publication has been created for cybersecurity staff and program managers; system,
70 network, and application administrators; incident response teams; and others who perform duties
71 related to cybersecurity log management. Its content is intended to be relevant to any
72 organization. Certain portions of the document are specific to federal agencies.
73 Acknowledgments
74 The authors thank everyone who has contributed to this publication or the original NIST SP 800-
75 92.
76 Note to Reviewers
77 NIST welcomes public comments on any aspect of this publication, including the following
78 questions asked on behalf of the Office of Management and Budget (OMB) and the
79 Cybersecurity and Infrastructure Security Agency (CISA):
80 1. This revision is informed by NIST SP 800-207 and the NCCoE’s Zero Trust Architecture
81 project calling out data analytics for zero trust purposes. Should the scope of this
82 publication be expanded to directly support and map to zero trust?
83 2. Should this publication be expanded to include log management implementation
84 guidance?
85 3. Are there additional considerations for different types of log sources that should be
86 included in this publication (e.g., on-premises, cloud, managed services, or hybrid)?
87 4. Should the standardization of log management planning to facilitate the sharing of cyber
88 threats or incidents be included?
89 5. Should guidance on how to determine the purposefulness of logging categories and types
90 be included?
91 6. Should guidance for determining storage retention periods be included?
92 7. Should this publication address how new technologies may change log management
93 planning (e.g., blockchains, zero trust, generative AI, quantum computing)?
94 8. Should this publication address how and which major triggers may necessitate reviewing
95 or changing log management plans?
96 9. Should this publication address storage costs and offer guidance on prioritizations and
97 trade-offs for cost-effective log management planning?
98 Trademark Information
99 All names are registered trademarks or trademarks of their respective companies.
ii
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
iii
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
156
iv
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
1
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
180 Introduction
181 A log is a record of the events that occur within an organization’s computing assets, including
182 physical and virtual platforms, networks, services, and cloud environments. Logs are composed
183 of log entries, and each entry contains information related to a specific event, which is an
184 observable occurrence in a computing asset. Logs serve many functions within most
185 organizations, such as optimizing system and network performance, recording the actions of
186 users, and providing useful data for investigating malicious activity. Many logs contain records
187 that are relevant for cybersecurity, such as operating system logs that capture system events and
188 audit records, application logs that capture application operational and security events, and
189 cybersecurity software logs that record routine events, adverse events, and possible malicious
190 activity.
191 Log management is the process for generating, transmitting, storing, accessing, and disposing of
192 log data. It facilitates an organization’s log usage and analysis. Log management can benefit an
193 organization in many ways. For example, it helps ensure that records are stored in sufficient
194 detail for an appropriate period of time. The continuous monitoring and analysis of logs are
195 beneficial for identifying security incidents, policy violations, fraudulent activity, and
196 operational problems shortly after they have occurred, and for providing useful information for
197 resolving such problems. Logs can also be useful for performing auditing and forensic analysis,
198 supporting the organization’s internal investigations, establishing baselines, verifying that assets
199 operate as intended, and identifying operational trends and long-term problems.
200 Logging and log management practices are part of many cybersecurity and privacy-related laws,
201 regulations, standards, guidance, and recommendations for every sector.
2
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
3
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
4
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
270 There is no “correct” way to use the playbook. An organization may choose to use it as the start
271 of its own new playbook for log management planning, integrate it with an existing log
272 management playbook, or use its information as reference material when considering its own
273 plans, policies, and processes.
274 Each play includes the following components:
275 • Unique ID number (for example, INV-5)
276 • Title (for example, Update the Work Role Inventory)
277 • Summary, to include the play’s desired outcome
278 • Tasks to perform. Each task is briefly stated and has a unique identifier based on its
279 play’s ID. Tasks that need a more detailed explanation are either defined as separate
280 plays or have additional explanatory text following them under the label supporting
281 information for tasks. Each list of tasks is assumed to typically be performed
282 sequentially unless otherwise noted by the phrase “not necessarily in order.”
283 Some plays also include examples related to the play, such as possible roles and
284 responsibilities, use cases, or sources of requirements.
285 In addition to the plays themselves, Appendix B contains crosswalks that indicate how
286 performing each play and its tasks can most significantly help an organization achieve
287 recommended outcomes, controls, and other concepts from a variety of NIST resources.
288 The plays and tasks are intentionally focused on important actions to perform for planning and,
289 thus, avoid any recommendations on the details of log management. Log management needs are
290 incredibly variable from one organization to another and frequently change, so this playbook
291 avoids specifying who is responsible for planning or performing any of the plays or tasks.
292 Rather, this playbook helps organizations identify and prioritize their needs and determine how
293 to best meet those needs.
5
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
6
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
7
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
364 Summary: Update the organization’s inventory of logging use cases and the desired outcomes
365 of each use case. This information is valuable when developing or updating logging policies.
366 One desired outcome is that all logging is done for a purpose and not just for the sake of
367 collecting log data. Another desired outcome is that all logging use cases are taken into
368 consideration when defining the target state for cybersecurity log management.
369 Tasks to perform include the following:
370 1. INV-3.1: Document summaries of the known logging use cases and the desired outcome
371 of each.
372 2. INV-3.2: Share the use cases with stakeholders, solicit their feedback, and make
373 revisions if needed.
374 Examples of possible use cases include:
375 • Continuous monitoring
376 • Early detection of malicious behavior, potentially malicious behavior, and advanced
377 threats that involve all user and non-user accounts
378 • Evidence of compliance with a standard/compliance reporting
379 • Evidence of verification of the functional and security operations of the components
380 • Incident response
381 • Passive DNS request analysis
382 • Security operations
383 • SIEM tools
384 • Software development artifact capture, for continuous integration/continuous delivery
385 (CI/CD)
386 • Threat detection and investigation, including threat hunting
387 • Zero trust implementation
8
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
399 3. INV-4.3: Identify all requirements applicable to new log source types or new logging use
400 cases.
401 Examples of sources of requirements include:
402 • General laws and regulations (e.g., GDPR)
403 • Sector-specific laws and regulations (e.g., HIPAA, NERC)
404 • Federal agency-specific requirements (e.g., OMB memoranda; FISMA/NIST Risk
405 Management Framework; EO 14028, Section 8 [EO14028])
406 • Standards that the organization chooses to follow (e.g., ISO 27001)
407 • The organization’s cybersecurity, privacy, and data retention policies
408 • Requirements and policies of a parent organization/enterprise
9
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
10
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
11
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
12
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
545 10. TS-2.10: Determine how log generation should or must be protected.
546 11. TS-2.11: Define how the protection of log generation should or must be monitored and
547 validated to ensure that logging is enabled and functioning normally.
548 Supporting information for tasks: Recording more log data is not necessarily better. Generally,
549 organizations should only require logging the necessary data and also have recommendations for
550 which other types and sources of data should be logged if resources permit. Some organizations
551 choose to have all or nearly all log data generated and stored for at least a short period of time in
552 case it is needed; this approach favors security considerations over usability and resource usage.
553 When establishing requirements and recommendations, organizations should be flexible since
554 each host is different and will log different amounts of data than other hosts. The logging
555 behavior of a host may also change rapidly due to an upgrade, patch, or configuration change.
556 Organizations may permit administrators to temporarily reconfigure log sources during adverse
557 conditions, such as unsuccessful malware attacks that cause the same type of log entry to be
558 generated many times. These configuration changes should be performed as a last resort and be
559 as precise as possible. Log source administrators should inform logging infrastructure
560 administrators of such configuration changes to ensure that log management processes are
561 modified if needed.
562 In some cases, software licenses may need to be upgraded in order to generate the required or
563 desired information for logs.
564 For more information on log generation in the context of incident response, see NIST SP 800-61,
565 Rev. 2, Computer Security Incident Handling Guide [SP800-61r2].
566 TS-3, Define Target State for Log Storage and Transfer
567 [Tasks] [Previous Play] [Next Play]
568 Summary: Define your organization’s log storage and transfer-related requirements and goals.
569 This should take log source types, log event types, system locations, and any other pertinent
570 attributes into account. The desired outcome is a comprehensive set of prioritized requirements
571 and goals for cybersecurity log storage and transfer that help define your organization’s target
572 state.
573 Tasks to perform include the following (not necessarily in order):
574 1. TS-3.1: Determine how long each log event should or must be preserved at the log
575 source.
576 2. TS-3.2: Determine which events, if any, should or must be transferred to a log
577 infrastructure from log sources, which log infrastructure systems should receive the
578 transferred event data, and which data characteristics should or must be transferred for
579 each event. This may necessitate estimating network bandwidth needs for log transfers.
580 3. TS-3.3: Determine how event correlation across log sources should or must be performed
581 within the log infrastructure.
13
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
582 4. TS-3.4: Determine how log data should or must be transferred to the log infrastructure,
583 including out-of-band methods where appropriate, and how frequently log data should or
584 must be transferred.
585 5. TS-3.5: Determine how the confidentiality, integrity, and availability of each log event
586 should or must be protected while in storage at the log source, while in the log
587 infrastructure, while being transferred from the log source to the log infrastructure, and
588 while being transferred from one log infrastructure component to another.
589 6. TS-3.6: Determine how much log storage space should or must be available at the log
590 sources and the log infrastructure.
591 7. TS-3.7: Determine how to handle log storage and log transfer errors at the log sources
592 and the log infrastructure.
593 8. TS-3.8: Determine if and when each type of log event should or must be transferred from
594 active storage to cold data storage for data retention purposes.
595 9. TS-3.9: Determine which log format/type to use (if this is an option).
596 10. TS-3.10: Determine how the protection of log storage and transfers should or must be
597 monitored and validated to ensure their confidentiality, integrity, and availability.
598 Supporting information for tasks: Completing these tasks effectively designs the high-level
599 architecture of the log infrastructure, such as the logical and physical locations of centralized log
600 data storage and various log analysis services. When most organizations only had one or a few
601 closely related logging use cases, a centralized logging server or group of servers could often
602 handle all of an organization’s logging infrastructure needs. Now, organizations often have
603 several diverse logging use cases and magnitudes more data to process and store, so more
604 complex logging architectures have become commonplace. For example, some organizations use
605 a massive data lake to hold all of their security log data instead of a centralized log management
606 and analysis service. Each logging use case probably has one or more different tools that
607 consume log data, and these tools can retrieve data from the data lake instead of having to
608 interact with all of the log sources. This can be much more efficient in terms of time, network
609 bandwidth, and especially storage.
610 Another consideration when performing these tasks is the possibility of another entity, such as a
611 parent organization or an outsourcer, taking care of some or all of your centralized log storage
612 needs in the future. Log storage needs are highly variable over time, although they generally
613 continue to increase. Cloud-based log storage can provide rapid scalability that on-premises log
614 storage often cannot.
615 Organizations will also need to determine whether their original logs need to be preserved for a
616 certain period of time or if preserving log data copied to a centralized log server or storage is
617 acceptable. In many environments, original logs only need to be preserved if they may be needed
618 as evidence.
14
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
15
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
656 3. TS-5.3: Define requirements for monitoring, validating, and testing the safeguarding of
657 log disposal (to avoid unauthorized destruction of logs).
658
16
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
17
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
695 2. GRC-2.2: Summarize the findings, get stakeholder feedback, and make revisions as
696 needed.
18
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
19
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
20
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
767 4. PMG-3.4: Revise the draft plan and policies to address the feedback based on the
768 adjudications. This includes harmonizing changes throughout the documents.
769 o If stakeholder feedback results in significant changes to the draft plan or policies,
770 this play may need to be repeated so that affected stakeholders can provide
771 feedback on the changes.
772 o If stakeholder feedback is not needed, the revised plan and policies should be
773 finalized and communicated to all affected parties.
774 Supporting information for tasks: It may be appropriate to specify future effective dates for
775 certain policy changes, depending on the timeframes for the corresponding items in the draft
776 project plan.
21
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
777 References
778 [CSF11] National Institute of Standards and Technology (2018) Framework for
779 Improving Critical Infrastructure Cybersecurity, Version 1.1. (National
780 Institute of Standards and Technology, Gaithersburg, MD), NIST
781 Cybersecurity White Paper (CSWP) NIST CSWP 6.
782 https://doi.org/10.6028/NIST.CSWP.6
783 [EO14028] Executive Order 14028 (2021) Improving the Nation’s Cybersecurity.
784 (The White House, Washington, DC), DCPD-202100401, May 12, 2021.
785 Available at https://www.govinfo.gov/app/details/DCPD-202100401
786 [NIST-CRSW] National Institute of Standards and Technology (2021) Security Measures
787 for “EO-Critical Software” Use Under Executive Order (EO) 14028.
788 (National Institute of Standards and Technology, Gaithersburg, MD), July
789 9, 2021. Available at https://www.nist.gov/itl/executive-order-improving-
790 nations-cybersecurity/security-measures-eo-critical-software-use-2
791 [OMB21-31] Office of Management and Budget (2021) Improving the Federal
792 Government’s Investigative and Remediation Capabilities Related to
793 Cybersecurity Incidents. (The White House, Washington, DC), OMB
794 Memorandum M-21-31, August 27, 2021. Available at
795 https://www.whitehouse.gov/wp-content/uploads/2021/08/M-21-31-
796 Improving-the-Federal-Governments-Investigative-and-Remediation-
797 Capabilities-Related-to-Cybersecurity-Incidents.pdf
798 [OMB22-09] Office of Management and Budget (2022) Moving the U.S. Government
799 Toward Zero Trust Cybersecurity Principles. (The White House,
800 Washington, DC), OMB Memorandum M-22-09, January 26, 2022.
801 Available at https://www.whitehouse.gov/wp-content/uploads/2022/01/M-
802 22-09.pdf
803 [SP800-53r5] Joint Task Force (2020) Security and Privacy Controls for Information
804 Systems and Organizations. (National Institute of Standards and
805 Technology, Gaithersburg, MD), NIST Special Publication (SP) 800-53,
806 Rev. 5. Includes updates as of December 10, 2020.
807 https://doi.org/10.6028/NIST.SP.800-53r5
808 [SP800-61r2] Cichonski PR, Millar T, Grance T, Scarfone KA (2012) Computer
809 Security Incident Handling Guide. (National Institute of Standards and
810 Technology, Gaithersburg, MD), NIST Special Publication (SP) 800-61,
811 Rev. 2. https://doi.org/10.6028/NIST.SP.800-61r2
812 [SP800-92] Kent K, Souppaya MP (2006) Guide to Computer Security Log
813 Management. (National Institute of Standards and Technology,
814 Gaithersburg, MD), NIST Special Publication (SP) 800-92.
815 https://doi.org/10.6028/NIST.SP.800-92
816 [SP800-181r1] Petersen R, Santos D, Wetzel KA, Smith MC, Witte GA (2020)
817 Workforce Framework for Cybersecurity (NICE Framework). (National
818 Institute of Standards and Technology, Gaithersburg, MD), NIST Special
819 Publication (SP) 800-181, Rev. 1. https://doi.org/10.6028/NIST.SP.800-
820 181r1
22
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
821 [SP800-207] Rose SW, Borchert O, Mitchell S, Connelly S (2020) Zero Trust
822 Architecture. (National Institute of Standards and Technology,
823 Gaithersburg, MD), NIST Special Publication (SP) 800-207.
824 https://doi.org/10.6028/NIST.SP.800-207
825 [SP800-218] Souppaya MP, Scarfone KA, Dodson DF (2022) Secure Software
826 Development Framework (SSDF) Version 1.1: Recommendations for
827 Mitigating the Risk of Software Vulnerabilities. (National Institute of
828 Standards and Technology, Gaithersburg, MD), NIST Special Publication
829 (SP) 800-218. https://doi.org/10.6028/NIST.SP.800-218
23
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
845 log
846 A record of events occurring within an organization’s computing assets, including physical and virtual platforms,
847 networks, services, and cloud environments. ([SP800-92], adapted)
24
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
25
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
26
NIST SP 800-92r1 ipd Cybersecurity Log Management
October 2023 Planning Guide
27