Optimizing Linux Performance
Optimizing Linux Performance
options
– Mpstat –p {cpu | all}
– Mpstat delay
• Sar (System Activity Reporter)
Stores information in a binary file to be replayed later
Performance Information: Memory
• Memory subsystem (Virtual memory)
• Swapd
– Total amount of memory currently swapped to disk.
• Buffers and Cache
– Too Much physical memory
• Active v/s Inactive memory
– Managing paging
• Processor and high v/s low memory
– 32 bit addressing more than 1 gb of memory
Performance Tools: Memory
• Vmstat
– Swap information, memory free and used
• Top
– Run time toggles sort via memory
• Procinfo
– Paging information
• Free
– How is the system using memory
• Slabtop
– Kernel allocated caches and how full they are
• Sar
• /proc/meminfo
Performance Information: Process
specific CPU
• Application runtime
– Kernel or user or library routines
– Application profiling
• Kernel time v/s user time
• Library time v/s application time
• Subdividing application time
• Ps
– Dynamic process information
– Other options
– -A all options
– -e all processes
– r only running processes etc…
Performance Tools: Process Specific
CPU
• Time
– Can be used with the application switch
• Strace
– Trace system calls of an application
• Ltrace
– Trace library calls of an application
• Gprof
– Profile applications (overhead issue, needs to recompile)
• Oprofile
– Profile applications – process level sampling
Performance Information: I/O
• Total amount and type (read/write) of disk
I/O on a system
• Which devices are servicing most of the
disk
• Effectiveness of a disk responding to I/O
requests
• Which processes are using a given set of
files
Performance Tools: I/O
• Vmstat
– Block/sector read/write information
• Iostat
– Per device, per partition breakdown
• Sar
– Lesser functionality than iostat but can simultaneously
record many different types of statistics
• Lsof
– Which processes use which file
Performance Information: Network
• Speed and duplex settings of interfaces
• Amount of network traffic over each interface
• Types of IP traffic flowing in and out of the
system
• Amount of each type of IP traffic
• Which applications are generating IP traffic
Performance Tools: Network
• Mii-tool and ethtool - Interface configuration
information
• Ifconfig - Configuration and Packet information
• Ip - Performance statistics
• Sar - Detailed network information
• Gkrellm - Graphical real time monitor
• Iptraf - Specific monitoring
• Netstat - Sockets and packet information
• Etherape - Network connection information
Optimizing an application
• Memory
• Is memory usage a problem?
– Top or ps
– What type of memory is it using?
– Cat /proc/<pid>/status
– Large VmStk? Use gdb and find out which function is using most
of the stack
– Large VmExe? Use nm and remove large symbol code
– Large VmLib? Use /proc/<pid>/map and streamline large library
usage
– Can also use memory profiler of the programming language
Contd…
• Start up time
• Is application start up time slow?
– Is loader the problem? Use ld and reduce the
number and size of libraries linked
– Large CPU usage? Use gprof or oprofile to
measure critical functions repeatedly
– Add switches to speed up after looking at
gprof information or edit code
Contd…
• I/O
• Is the disk usage a problem?
– Run iostat and look for overload
– Spread these files to multiple disks
– Which function is causing large amount of
disk I/O?
– Which files are accessed most? Use strace
and track system calls, spread this evenly
between disks or a faster disk
Contd…
• Network
• Is network usage a problem?
– Use ethtool and check the interface limit
– Use iptraf and look at the traffic
– Use ifconfig to check for packet errors
– Use netstat and find out which application is using
that port and generating traffic
– Use strace and find out which application socket is
responsible for the traffic