Symantec™ Endpoint Detection and Response 4.0 Sizing and Scalability Guide
Symantec™ Endpoint Detection and Response 4.0 Sizing and Scalability Guide
Legal Notice
Copyright © 2018 Symantec Corporation. All rights reserved.
Symantec, the Symantec Logo, the Checkmark Logo and are trademarks or registered trademarks of
Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks
of their respective owners.
This Symantec product may contain third party software for which Symantec is required to provide attribution
to the third party (“Third Party Programs”). Some of the Third Party Programs are available under open
source or free software licenses. The License Agreement accompanying the Software does not alter any
rights or obligations you may have under those open source or free software licenses. Please see the
Third Party Legal Notice Appendix to this Documentation or TPIP ReadMe File accompanying this Symantec
product for more information on the Third Party Programs.
The product described in this document is distributed under licenses restricting its use, copying, distribution,
and decompilation/reverse engineering. No part of this document may be reproduced in any form by any
means without prior written authorization of Symantec Corporation and its licensors, if any.
THE DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS,
REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE
DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY
INVALID. SYMANTEC CORPORATION SHALL NOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL
DAMAGES IN CONNECTION WITH THE FURNISHING, PERFORMANCE, OR USE OF THIS
DOCUMENTATION. THE INFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO
CHANGE WITHOUT NOTICE.
The Licensed Software and Documentation are deemed to be commercial computer software as defined
in FAR 12.212 and subject to restricted rights as defined in FAR Section 52.227-19 "Commercial Computer
Software - Restricted Rights" and DFARS 227.7202, et seq. "Commercial Computer Software and
Commercial Computer Software Documentation," as applicable, and any successor regulations, whether
delivered by Symantec as on premises or hosted services. Any use, modification, reproduction release,
performance, display or disclosure of the Licensed Software and Documentation by the U.S. Government
shall be solely in accordance with the terms of this Agreement.
Symantec Corporation
350 Ellis Street
Mountain View, CA 94043
https://www.symantec.com
Symantec Support
All support services will be delivered in accordance with your support agreement and the
then-current Enterprise Technical Support policy.
Introduction ................................................................................... 7
■ Introduction
Introduction
This sizing guide provides guidance about which Symantec EDR appliance you should use in
your environment. It also offers information about what you need to know to set up an appliance.
The architecture, designs, and recommendations that are provided in this guide are based on
metrics from internal testing of the product. These tests are performed in an isolated
environment. Implementations in production environments may result in some performance
metrics that vary from the testing scenarios. These variations can alter the recommended
sizing and architecture. This guide references possible changes and modifications to Symantec
EDR capability, functions, metrics, and features. These changes are subject to continuous
evaluation and should not be considered as firm commitments.
For additional information about installing Symantec EDR (including system requirements,
deployment and operating modes, etc.) see the Symantec™ Endpoint Detection and Response
Installation Guide.
Chapter 2
Symantec EDR cloud
support
This chapter includes the following topics:
Fetching events from a Symantec EDR appliance 1.6 MB for every 1000 events.
using Investigation Playbooks:
Dissolvable agent server is typically capable of scanning 150 endpoints per hour for each
logical CPU core. For example, to scan 1200 endpoints per hour with a single vault, a CPU
with eight cores are the minimum to successfully meet that demand.
Chapter 3
Appliance architecture
This chapter includes the following topics:
Horizontal growth
Scanner/network:
■ Method: Add scanners.
■ Drivers: Network bandwidth.
Management Server:
■ Method 1: Add additional management servers.
■ Method 2: To overcome data retention limits of Symantec EDR management server, use
API to dump events to Splunk service or other bulk data analysis device or service.
■ Drivers: More SEPMs, link latency across regions. Another driver is if the sum of all
endpoints is greater than what one management appliance can handle.
Appliance architecture 13
Best practices for SEPM integration
Note: When a SEPM is added to an EDR appliance console, it's defined as SEPM/Domain.
So if there were ten domains running on a single SEPM, there would be ten entries that are
defined on Symantec EDR.
Basic architecture
Symantec EDR contains the following main architectural components:
■ EDR appliance console – The management server that is used for network appliance
management, policy management, investigation, and remediation.
■ Symantec EDR network scanner – Scans network traffic.
Other required components in the architecture, but not deployed as part of Symantec EDR:
■ SEPM
■ SEPM databases
We currently do not support connecting to multiple SEPMs that the same database backs.
■ SEP clients
External cloud services:
■ Email Security.Cloud
■ Web Security.Cloud
Appliance architecture 14
Best practices for SEPM integration
Deployment architecture
The site design begins with the choice of the basic site design architecture. This choice normally
follows the same site design as for SEPM sites. The reason for this parallel site design is so
that the EDR appliance console can communicate with the SEPM server, SEPM database,
and SEP clients. The following content describes how SEP recommends deploying sites and
how Symantec EDR can be deployed into that environment.
Single-site design | Multi-site design
Single-site design
An organization with one data center can generally use a single-site design with the following
attributes:
■ Two instances of SEPM for redundancy and load balancing
■ Database clustering to support high availability
Note: The following diagrams assume the total number of endpoints that a single Symantec
EDR instance can support. If this number is greater than 50K endpoints for ECC 2 and 100K
for ECC 1, then you still have options. See Figure 3-4 and Figure 3-6.
One Symantec EDR connects to one database and one SEPM per SEP site.
Each Symantec EDR appliance has its own embedded database. This deployment configuration
allows for full visibility into events, command status, and policy. This deployment configuration
does not support high availability or load balancing. Configure EDR appliance console to
connect with one SEPM per site and one connection to the database cluster.
Appliance architecture 15
Best practices for SEPM integration
One Symantec EDR connects to one database and one SEPM per SEP site.
Figure 3-4 More endpoints than one Symantec EDR can handle - scenario 2
If each SEP site is greater than what one Symantec EDR can handle, you must run Advanced
Threat Protection 3.0.5 or later or Symantec EDR 4.x or later. The reason is because these
versions support multiple Symantec EDR instances for a single SEP site. The only restriction
is how many endpoints are in a given SEP group.
Appliance architecture 16
Best practices for SEPM integration
Multi-site design
Figure 3-5 Multi-site SEP design
One Symantec EDR connects to one database and one SEPM per SEP site. One Symantec
EDR can connect to up to ten SEP sites. The only other limitation is the total number of
endpoints.
Assume that no SEP site is greater than what one Symantec EDR can handle but two or more
are. In this scenario, deploy multiple Symantec EDRs in the single SEP site deployment. Use
a SIEM to centralize events.
Appliance architecture 17
Considerations for selecting a network scanner
Figure 3-6 More endpoints than one Symantec EDR can handle
¹ A SEP site is based on one or more SEPMs that are connected to a single database cluster.
A SEP site can choose to replicate policy, commands, and logs between sites.
Use hardware or virtual If you have an extensive VMware investment, you might want
to use virtual appliances. If you have little or no VMware
investment, use the hardware.
Available bandwidth The hardware solutions have higher throughput than virtual
solutions.
Total endpoints in the organization 8840 has a capacity of ~10K simultaneous connections. 8880
can support twice that with 25K simultaneous connections.
These numbers are for inline mode. In TAP mode, hardware
can support approximately twice the number of connections as
inline. VMs can handle 2K simultaneous connections.
Symantec EDR features intended to be If the deployment is to use mostly network scanning, then a
used separate scanner and management platform deployment
provides room to increase scanning capacity. The number of
scanners would depend on the number of ingress and egress
points in the network and the amount of traffic at those points.
Chapter 4
Appliance network impact
This chapter includes the following topics:
■ Required bandwidth
Required bandwidth
Table 4-1 provides the estimated bandwidth usage.
SEPM and Symantec EDR ■ Symantec EDR queries each SEPM hourly to get Group and endpoint
information. The breakdown is roughly as follows:
■ 500 B/Group
If you have 20 Groups, then Symantec EDR generates 20 Groups
* 500 B/Group = 10KB each hour.
■ 4 KB / endpoint
If you have 50K endpoints, then Symantec EDR generates 50K
endpoints * 4 KB/endpoint = 200 MB each hour.
■ Summary: 4.8GB / day (for organizations with 50K endpoints and
20 SEPM Groups)
(200 MB + 10 KB) / hour * 24 hours per day = 4.8GB / day for
50K endpoints and 20 Groups.
Appliance network impact 20
Endpoint performance metrics
Symantec EDR and Symantec Updating of definitions requires 1 - 2 MB a day. When a new Symantec
Cloud EDR release is available, you must download 2 GB of data.
Network 0 1 MB/day/endpoint
Appliance network impact 21
Reputation request service (RRS) rate
Note: Expect spikes to double these numbers. Batching size affects the network load. Endpoints
can send batches at a minimum of 1 event a minimum of every minute and a maximum of 100
events every 24 hours. Expect that an average client sends about two events per minute. Less
than this amount (fewer than ten events per 5 minutes) can back up the clients. More than this
rate (greater than 15 events per 5 minutes) increases the load on your server during peak
performance. Ensure that your system isn't already fully loaded if you increase the batch size
significantly.
Note: Your organization might be able to accept Symantec EDR as a single point of failure in
your environment. However, the 8840 appliance typically does not have enough bandwidth to
service a network protected by dual firewall cluster.
Chapter 5
Appliances
This chapter includes the following topics:
■ Vertical scaling
■ Platforms overview
■ How to set up your network scanner appliance (all-in-one vs. management console)
Table 5-1 Supported appliances for ATP 3.x and Symantec EDR 4.0
Table 5-1 Supported appliances for ATP 3.x and Symantec EDR 4.0 (continued)
Vertical scaling
Table 5-2 Appliance upgrade requirements
8880 v2 - Dell 730 OEM XL ■ Upgrade parts available from Dell permit use of 8880 v2 as a
management server with the endpoint activity recorder feature.
■ For each appliance being upgraded, you need to order the following:
■ 4x: R730XL Drives 400-AJQP 1.8TB 10K RPM SAS 512e 2.5in
Hot-plug hard drive, Cus Kit 13G
■ 12x: R730XL Memory 370-ACNU or A8711887 Dell 16GB RDIMM,
2400MT/s, Dual Rank, x8 Data Width
■ The specific installation steps that are required to build RAID10 array,
replace memory, and get Symantec EDR to detect the newly-installed
physical components.
8880 v1 - Dell R720 OEM XL ■ Upgrade parts available from Dell permit use of Symantec EDR 8880
v1 as a management server with the endpoint activity recorder feature.
■ For each appliance being upgraded, you must order the following:
■ 4x: R720XL Drives 400-AJQM 1.8TB 10K RPM SAS 512e 2.5in
Hot-plug hard drive, Cus Kit 12G
■ Optionally, to maximize performance for 12x: R720XL Memory
A7187318 Dell 16GB Certified Memory Module - 2Rx4 DDR3
RDIMM 1866MHz SV
■ The specific installation steps that are required to build RAID10 array,
replace memory, and get Symantec EDR to detect the newly-installed
physical components.
VMware Symantec EDR 4.x does more than Advanced Threat Protection 3.x/2.x,
so it consumes a larger footprint of resources. ESXi resources must be
reserved. If you have outdated or insufficient hardware in your ESX
environment, you might experience performance issues. Symantec
recommends at least 10K RPM or SSD disk drives.
Table 5-4 reflects the number of endpoints that are supported with ECC 2.0 enabled.
Appliances 28
Platforms overview
Virtual + HD 2K 10K
¹ Symantec recommends that you enable the endpoint activity recorder but disable the collection
of Process Launch and Process Terminate events. In this scenario, the total supported endpoints
on 720 appliance and 730 appliance increases from 50K to 100K.
Enabling ECC on VM with less than 12 CPU cores or 48-GB RAM results in the following
warning message. If you ignore this message, expected performance is not guaranteed: "To
enable ECC on a virtual machine, it is recommended that you have 12 or more cores and 48
GB or more memory”.
The 8880-30 HDD is 4 disks of 1.8 TB, as directed by the Symantec EDR Hardware Guide.
You can select specific SEPM groups for which you enable ECC. This feature lets you limit
the number of endpoints that use the ECC feature. However, Symantec EDR still has a
maximum number of endpoints that it can support. If your SEPM has more than 200K endpoints
in a single instance, Symantec EDR begins to experience significant performance issues.
Platforms overview
Table 5-5
Parameter VMware 8840: R220 8880: R720 8880: R730 8840: R330 8880-30:
v2 R730 v3
Parameter VMware 8840: R220 8880: R720 8880: R730 8840: R330 8880-30:
v2 R730 v3
Not available
Appliances 30
Platforms overview
Not available
Inline monitor/block
Memory 32 GB 48 GB 48 GB
Cores 4 12 12
8880: a = 50K, c = 30: size = (50,000 * 1.7 MB * 30) + (10 * 7.5GB) ~2.4 TB
VM: a = 10K, c = 30: size = (10,000 * 1.7 MB *30) + (10 * 7.5GB) ~ 0.5 TB
a Number of endpoints
This number represents the size of events on Symantec EDR when all events are enabled.
c Number of days
e Dump size
This number is the amount of disk space that you allocated on the endpoint.
■ VM
400 EPS (approximately 35 million events per day)
Symantec EDR has numerous levers for controlling the event rate. You can exclude specific
file paths and specific hashes. They can also select the general categories of events that are
sent to Symantec EDR as follows:¹.
Table 5-10
Event types that are sent to Symantec EDR Percentage of ECC events
¹ These event types account for the majority of the traffic that is sent to Symantec EDR. They
are enabled by default, but are also considered an important part of the endpoint activity
recorder feature.
Aggregated overhead per day:
■ ~ 33 GB/day for 10K endpoints
■ ~165 GB/day for 50K endpoints
Table 5-11 Endpoint activity recorder load retention per form factor
Table 5-11 Endpoint activity recorder load retention per form factor (continued)
Event rate per second (EPS) is inclusive of ECC 2.0 live response data, commands execution
overhead, etc., along with endpoint traffic and network traffic. Connection rate per second
(CPS) is inclusive of all ECC 2.0 data connections. It is also inclusive of all control connections
along with endpoint traffic and network data connections and control connections.
Per event - average size on Symantec EDR is ~1K byte.
The number of events that are generated and the data related to those events affects Symantec
EDR performance, such as follows:
■ Data such as file information, endpoint IPs, user and host identity, URLs visited, external
domain data, etc.
■ Command execution rate.
■ Environments vary, so you should watch for the number of events that are generated rather
than the absolute number of endpoints managed.
■ Symantec EDR certifies an average of 150 non-endpoint activity recorder events per second
flowing into the management console from different control points.
Chapter 6
Database
This chapter includes the following topics:
GetEvents API
Using APIs to retrieve all events is the next least performant way to move events from Symantec
EDR to another data system. APIs transfer copies of ALL the events in batches of up to
1000/batch. The batch sequencing is the responsibility of the partner’s API connector. The
rule of thumb for the expected impact: Symantec EDR management server event per second
number drops by approximately one-half.
Syslog
Syslog is the least performant way to move events from Symantec EDR to another data system.
Syslog transfers copies of ALL the events. The rule of thumb for the expected impact: Symantec
EDR management server event per second number drops by almost half.
Chapter 7
Symantec EDR appliance
and the endpoint activity
recorder
This chapter includes the following topics:
■ Sizing considerations
3 Click the actions menu (three vertical dots) to the far right of the
SEPM connection that you want to update.
2 Check Enable Endpoint Activity Recorder to enable endpoint activity recorder on the
clients that this SEPM manages.
Checking this box enables functionality on the endpoint for recording activities for every
process on the endpoint. It also enables the logic to determine which of those events to
send back to Symantec EDR in real time.
3 If you enable the endpoint activity recorder, specify the maximum amount of disk space
(in MB or GB) on the endpoint to store recorded data.
The minimum size is 250 MB; the maximum is 20 GB. The default value is 1 GB.
This setting configures how much space to allocate to retain ECC events on the endpoint
before they are purged. The exact duration depends on the endpoint activity, but the
average is 1 GB every 7 days of events. The exact ratio depends on the activity of the
endpoint.
We recommend that you allocate enough space on the endpoint to handle the activity
that may happen while roaming.
4 Do one of the following:
To limit when to send Clients submit data to Symantec EDR based on a minimal time interval
endpoint events to and maximum batch size.
Symantec EDR
1 Configure the maximum frequency (in minutes or hours) that
batches of events are sent to Symantec EDR.
Expect that an average client sends about 2 events per minute. Fewer
than 10 events per 5 minutes can result in events accumulating on the
clients, which means you might not be getting the important event
information in a timely manner. More than that (greater than 15 events
per 5 minutes) increases the load on your SEPM server during peak
performance. Ensure that your system isn`t already fully loaded if you
increase the batch size significantly.
Symantec EDR appliance and the endpoint activity recorder 39
Endpoint activity recorder considerations
5 Check the boxes for the types of events that you want submitted to Symantec EDR.
Load point changes This event type consists of any events that are associated with the
ability to maintain persistence on an endpoint. This event type includes
but is not limited to: Startup registry keys, services, scheduled jobs,
etc.
Suspicious system activity This event consists of expert rules such as suspicious protocol-port
usage by system processes, the system files that are launched from
unexpected locations, etc.
Heuristic detections This event type consists of the rules that match a sequence of events
that are often seen in malicious activity.
Process launch activity Sends to Symantec EDR every process launch event with parent|child
relationship and command line. Very useful for identifying what ran
in your environment, what command line arguments were used, and
under what user context. While valuable, Process Launch events
account for 49% of the events being sent up to Symantec EDR.
Process terminate activity This event type is less useful than Process Launch events, but it does
indicate if a process is still running. This category accounts for 49%
of all events being sent to Symantec EDR. If you need to reduce the
load, start by disabling this category first.
Troubleshooting
ATP UI shows Sepm returned non 200 HTTP response
■ Consider the impact of forwarding events to Symantec EDR and purchase the appliance
that is needed to support the event load.
See “Endpoint activity recorder event rate load” on page 32.
■ Enable the endpoint activity recorder feature in a way that targets separate SEPM groups.
Sizing considerations
When considering how to size the deployment, consider the following:
■ Number of SEP endpoints that the Symantec EDR management console manages
■ Volume of the network traffic that the network control point inspects
■ Number of network control points managed by the management console
■ Number of SEPMs deployed
Symantec EDR management appliances if you want to enable the full ECC logging capabilities
for more than 50,000 clients.
Appendix A
Upgrading the 8880
appliance
This appendix includes the following topics:
Table A-1 8880 hardware specs for v1, v2, and 8880-30
CPU 2X12 Core Intel Xeon 2X18 Core Intel Xeon 2X18 Core Intel Xeon
Memory 96 GB 96 GB 192 GB
Upgrading the 8880 appliance 43
Upgrading the 8880 appliance
Table A-1 8880 hardware specs for v1, v2, and 8880-30 (continued)
Hard drive RAID 10 4X600GB 10K SAS RAID 10 4X300GB 15K SAS RAID 10 4X300GB 15K SAS
6Gbps 12Gbps 12Gbps
2 WAN/LAN pairs (10 Gb) 2 WAN/LAN pairs (10 Gb) 2 - 1 Gb Ethernet ports
1 Management port (1 Gb) 1 Management port (1 Gb) 2 WAN/LAN pairs (10 Gb)
Note: You should keep the purchase order details (number, dates, etc., for the upgrade
order). This information is required when you need hardware support or for warranty
purposes. Existing appliance support extends to the newly added components, and the
components take on the remaining support life of the appliance.
All HDDs are hot pluggable and can be installed from the front of the appliance.
However, Symantec EDR requires a restart after the HDDs are installed.
■ Open the chassis to swap the 12 X 8-Gb memory with the 12 X 16 memory.
You can find the steps for the component removal and installation in Dell’s manuals:
http://topics-cdn.dell.com/pdf/poweredge-r720_Owner's%20Manual_en-us.pdf
http://www.dell.com/support/home/us/en/04/product-support/
product/poweredge-r730/manuals
Important: Make sure that you follow Dell's instructions to ensure that the appliance
recognizes that the new components are properly installed.
5 Create a new RAID 10 configuration for the newly-added 4 X 1.8-TB drives.
The steps are the same for the R720XL and the R730XL. Detailed steps can be found at
the following URL. Use Option A by iDRAC.
Note: iDRAC must be upgraded as part of this process. You do not need to turn off the
appliance.