Zabbix
Performance
Tuning
Getting most out of your hardware
1/31 Zabbix Webinar
What is all about
Overview of Zabbix Performance
Step 1. Identify & fix common problems
Step 2. Tuning of Zabbix Parameters
Step 3. Do extra work
2/31 Zabbix Webinar
Overview
What's Zabbix performance?
3/31 Zabbix Webinar
Basic data flow
Alerter Poller DATA
GUI
Escalator
Poller
History
syncer
DATA
DATA
History Trapper
syncer
Database
Conf
Conf syncer OTHER Trapper DATA
syncer
Zabbix Server
No proxies, not a distributed setup
4/31 Zabbix Webinar
Metrics of Zabbix
performance
Number of values processed per second (NVPS)
A rough estimate of NVPS is visible in Zabbix
Dashboard
NVPS
....:..........................
5/31 Zabbix Webinar
Performance delivered by
Zabbix
Hardware: Quad Core CPU, 6GB, RAID10 BBWC
Budget: around 2K EUR
Zabbix is able to deliver 1 million of values per
minute or around 15.000 of values per second
In real life performance would be worse. Why?!
.....:.........................
6/31 Zabbix Webinar
Factors making
performance lower
Type of items, value types, SNMPv3, number of
triggers and what the triggers are
Housekeeper settings and thus size of the
database
Number of front-end users
Complexity of triggers
7/31 Zabbix Webinar
Performance VS number of
hosts
60 items per host, update frequency once per minute
Number of hosts Performance (values per second)
100 100
1000 1000
10000 10000
600 items per host, update frequency once per minute
Number of hosts Performance (values per second)
100 1000
1000 10000
10000 100000
8/31 Zabbix Webinar
Slow v.s. Fast
What Slow Fast
Database size Large Fits into memory
Trigger expressions min(),max(),avg() last(), nodata()
Data collection Polling (SNMP, Trapping (active
Agent-less, agents)
Passive agent)
Data types Text, string Numeric
9/31 Zabbix Webinar
Visible symptoms of bad
Zabbix performance
Queue has too many delayed items
Administration->Queue
Frequent gaps in graphs, no data for some of
the items
False positives for triggers having nodata()
function
Unresponsive front-end
10/31 Zabbix Webinar
Nice looking Queue
11/31 Zabbix Webinar
Identify & fix
common problems
STEP 1
12/31 Zabbix Webinar
Different views on
performance
I just added 5 hosts and Zabbix died :-(
Zabbix is so slooooow, I have only 48 hosts :-(
however:
Zabbix Milestone achieved - 1000 hosts and
growing :-)
Our status update: 8500 hosts, 950400 items,
670340 triggers, 9550 vps :-)
:-) - Happy! :-( - Unhappy
13/31 Zabbix Webinar
Common problems of
initial setup
Use of default templates
Make your own smarter templates
Default database settings
Tune database for the best performance
Not optimal configuration of Zabbix Server
Tune Zabbix Server configuration
Housekeeper
Use of older releases
Always use the latest one!
14/31 Zabbix Webinar
How do I know database
performance is bad?
Zabbix Server configuration file, zabbix_server.conf:
LogSlowQueries=1000
15/31 Zabbix Webinar
Tune Zabbix
Configuration
STEP 2
16/31 Zabbix Webinar
Get internal stats
Real number of VPS
zabbix[wcache, values, all]
zabbix[queue,1m] number of items delayed for more
than 1 minute
Zabbix Server components
Alerter, Configuration syncer, DB watchdog,
discoverer, escalator, history syncer, http poller,
housekeeper, icmp pinger, ipmi poller, poller, trapper
17/31 Zabbix Webinar
Get internal stats
Before Zabbix 1.8.5 no way to
see clearly how well Zabbix
components work!
18/31 Zabbix Webinar
Get internal stats
Now we have a very nice way of monitoring internal
performance
Percentage of time a component is in BUSY state
zabbix[process,<type>,<mode>,<state>]
<type> - trapper, discoverer, escalator, alerter, etc
<mode> - avg, count, min, max
<state> - busy, idle
19/31 Zabbix Webinar
How it looks like
A graph indicating a problem
20/31 Zabbix Webinar
Tune number of processes
Zabbix Server configuration file, zabbix_server.conf:
StartPollers=80
StartPingers=10
StartPollersUnreachable=80
StartIPMIPollers=10
StartTrappers=20
StartDBSyncers=8
21/31 Zabbix Webinar
Do extra work
STEP 3
22/31 Zabbix Webinar
Use Proxies
Alerter Trapper DATA
Escalator Proxy Pollers DATA
Trapper
History
syncer
DATA
History Trapper
syncer
Proxy Pollers DATA
Conf
Conf syncer OTHER Trapper DATA
syncer
Zabbix Server
Proxies do data collection
23/31 Zabbix Webinar
Table partitioning
It is a way to split large tables into smaller
partitions.
Make sense for historical tables:
history_*, trends*, events
Benefits
Easy to remove older data
Much better performance
24/31 Zabbix Webinar
No table partitioning
Zabbix
Server History
& GUI
25/31 Zabbix Webinar
With table partitioning
Partition 2013_09
Zabbix Partition 2013_08
Server
& GUI Partition 2013_07
Partition 2013_06
26/31 Zabbix Webinar
Hey, I tried everything!
Performance is still not
good.
Run all Zabbix components on separate hardware!
Database
Zabbix Server Zabbix GUI
24 core CPU
8 core CPU Fast CPU
64GB of RAM
4GB of RAM 4GB of RAM
Fast storage
27/31 Zabbix Webinar
Summary
Make sure you did everything
28/31 Zabbix Webinar
Check list
Zabbix internal statistics is monitored!
Otherwise you don't know anything about
Zabbix health
Zabbix configuration is tuned
Database performance is tuned
Housekeeper is not used, you use table partitions
DisableHousekeeper=1
29/31 Zabbix Webinar
Additional reading
MySQL & PostgreSQL Performance Tuning Guides
Table partitioning for Zabbix
MySQL: zabbixzone.com
PostgreSQL:
https://www.zabbix.org/wiki/Docs/howto/zabbix2_postgresql_partitio
ning
Zabbix Internal Checks
http://blog.zabbix.com/monitoring-how-busy-zabbix-processes-are
http://www.zabbix.com/documentation/1.8/manual/config/items#int
ernal_checks
30/31 Zabbix Webinar
WWW.ZABBIX.COM
Any questions?
31/31 Zabbix Webinar
