The Ultimate Guide to

Java Performance Tuning

Ender Aydin Orak
koders.co
1 INTRODUCTION

koders.co
WHAT YOU WILL LEARN ?
You WILL LEARN:

Application performance principles &

methods
You WILL LEARN:

JVM structure and internals regarding

application performance
You WILL LEARN:

Garbage Collection types and when to

use which
You WILL LEARN:

Monitoring, Profiling, Tuning,

Troubleshooting JVM applications
You WILL LEARN:

Using OS and JVM tools for better

application performance
You WILL LEARN:

Applying performance best practices

You WILL LEARN:

Java language level tips & tricks

YOU WILL PRACTICE ON:
• Dead locks • Collections

• Memory leaks • Locks

• Lock contention • Multithreading

• CPU utilization • Best practices

Performance Approaches

• Top-Down: Focus on top level application

• Application Developers (our approach)

Performance Approaches

• Bottom-Up: Focus on the lowest level: CPU.

• Performance Specialists
Performance Tuning Steps

Monitoring
Performance Tuning Steps

Profiling
Performance Tuning Steps

Tuning
 JVM Overview &
2 INTERNALS
koders.co
Objectives
• JVM Runtime & Architecture

• Command Line Options

• VM Life Cycle

• Class Loading
JAVA PROGRAMMING LANGUAGE
• Object oriented, Garbage collected*

• Class based

• .java files (source) compiled into .class files (bytecode)

• JVM executes platform independent bytecodes

“All problems in computer science can be
solved by another level of indirection”

–DAVID WHEELER
JVM Overvıew
• JVM: Java Virtual Machine

• A specification (JCP, JSR)

• Can have multiple implementations

• OpenJDK, Hotspot*, JRockit (Oracle), IBM J9, much

more

• Platform independent: “Write once, run everywhere”

“All non-trivial abstractions, to some
degree, are leaky.”

–JOEL SPOLSKY
HOTSPOT VM ARCHITECTURE
HOTSPOT VM ARCHITECTURE
COMMAND LINE OPTIONS
• Standard: Required by JVM specification, standard
on all implementations (-server, -classpath)

• Nonstandard: JVM implementation dependent. (Start

with -X)

• Developer Options: Non-stable, JVM implementation

dependent options for specific cases (Start with -XX in
HotSpot VM)
JVM LIFE CYCLE
1. Parse command line options

2. Establish heap sizes and JIT compiler (if not specified)

3. Establish environment variables (CLASSPATH, etc.)

4. Fetch Main-Class from Manifest (if not specified)

5. Create HotSpot VM (JNI_CreateJavaVM)

6. Load Main-Class and get main method attributes

7. Invoke main method passing provided command line arguments

 PERFORMANCE
3 Overview
koders.co
Objectives

• Key concepts regarding application performance

• Common performance problems and principles

• Methodology to follow in solving problems

QUESTIONS & Expectations
• Expected throughput ?

• Acceptable latency per request ?

• How many concurrent users/tasks ?

• Expected throughput and latency ?

• Acceptable garbage collection latency ?

Terminology

• CPU Utilization: Percentage of the CPU usage

(user+kernel)

• User CPU Utilization: the percent of time the application

spends in application code
TERMINOLOGY

• Memory Utilization: Memory usage percentage

(ram/swap)

• Swapping should be avoided all times.

TERMINOLOGY

• Lock Contention: The case where a thread or process

tries to acquire a lock held by another process or
thread.

• Prevents concurrency and utilization. Should be avoided as

much as possible.
TERMINOLOGY

• Network & Disk I/O Utilization: The amount of data

sent and received via network and disk.

• Should be traced and used carefully.

Performance
• Aspects of performance:

• Responsiveness

• Throughput

• Memory Footprint

• Startup Time

• Scalability
RESPONSIVENESS
• Ability of a system to complete assigned tasks within
a given time

• Critical on most of modern software applications

(Web, Desktop, CRUD apps, Web services)

• Long pause times are not acceptable

• The focus is on responding in short periods of time

THROUGHPUT
• The amount of work done in a specific period of time.

• Critical for some specific application types

(e.g. Data analysis, Batch operations, Report generation)

• High pause times are acceptable

• Focus is on how much work are getting done over a longer

period of time
Memory Footprint
• The amount of main memory used by the application

• How much memory ?

• How the usage changes ?

• Does application uses any swap space ?

• Dedicated or shared system ?

STARTUP TIME

• The time taken for an application to start

• Important for both the server and client applications

• “Time ‘till performance”

SCALABILITY
• How well an application performs as the load on it
increases

• Huge topic that shapes the modern software architectures

• Should be linear, not exponential

• Can be measured on different layers in a complex system

Scalability
Focus areas

• Java application performance

• Tuning JVM for throughput or responsiveness

• Discovery, troubleshooting and tuning JVM

Performance Methodology
• Our steps to follow

1.Monitoring

2.Profiling

3.Tuning
Performance Monitoring
• Non-intrusively collecting and observing performance
data

• Early detection of possible problems

• Essential for production environments

• Early stage for troubleshooting problems

• OS and JVM tools

PERFORMANCE PROFILING
• Collecting and observing performance data using
special tools

• More intrusive & has affect on performance

• Narrower focus to find problems

• Not suitable for production environments

PERFORMANCE TUNING

• Changing configuration, parameters or even source

code for optimizing performance

• Follows monitoring and profiling

• Targets responsiveness or throughput

Development PROCESS
PERFORMANCE PROCESS
 JVM AND GARBAGE
4 COLLECTION
koders.co
Objectives
• What garbage collection is and what it does

• Types of garbage collectors

• Differences and basic use cases of different garbage

collectors

• Garbage collection process

Garbage collectıon

• In computer science, garbage collection (GC) is a

form of automatic memory management.

• The garbage collector, attempts to reclaim memory

occupied by objects that are no longer in use by the
program.
Garbage Collectıon
• Main tasks of GC

• Allocating memory for new objects

• Keeping live (referenced) objects in memory

• Removing dead (unreferenced) objects and reclaiming

memory used by them
GC Steps: MARKING
GC Steps: DELETION [normal]
GC Steps: DELETION [COMPACTING]
GENERATIONAL GC
• Hotspot JVM is split into generational spaces
WHY GENERATIONAL GC ?

• Object life patterns in OO languages:

• Most objects “die young”

• Older objects rarely references to young ones

GENERATIONAL GC
GC STEPS: YOUNG GC
GC STEPS: YOUNG GC
GC STEPS: YOUNG GC
GC STEPS: YOUNG GC
GC STEPS: YOUNG GC
GC STEPS: YOUNG GC
GC STEPS: YOUNG GC
OLD & PERMANENT GENERATIONS
 GARBAGE
5 COLLECTORS
koders.co
Objectives
• Garbage collection performance metrics

• Garbage collection algorithms

• Types of garbage collectors

• JVM ergonomics
GC PERFORMANCE METRICS
• There are mainly 3 ways to measure GC
performance:

• Throughput

• Responsiveness

• Memory footprint
FOCUS: Throughput

• Mostly long-running, batch processes

• High pause times can be acceptable

• Responsiveness per process is not critical

FOCUS: RESPONSIVENESS

• Priority is on servicing all requests within a predefined

time interval

• High GC pause times are not acceptable

• Throughput is secondary
GC ALGORITHMS

• Serial vs Parallel

• Stop-the-world vs Concurrent

• Compacting vs Non-Compacting vs Copying

Serial vs Parallel
STOP-THE-WORLD vs CONCURRENT
• STW: Simpler, more pause time,
memory need is less, simpler to
tune

• CC: Complicated, harder to tune,

memory footprint is larger,
less pause time
CoMPACTING vs Non-Compactıng
TYPES OF GC
• Serial Collector

• Parallel Collector

• Young (Parallel Collector)

• Young & Old (Parallel Compacting Collector)

• Concurrent Mark-Sweep Collector

• G1 Collector
SERIAL / Parallel Collector
SERIAL COllector
• Serial collection for both young and old generations

• Default for client-style machines

• Suitable for:

• Applications that do not have low pause reqs

• Platforms that do not have much resources

• Can be explicitly enabled with: -XX:+UseSerialGC

PARALLEL COLLECTOR
• Two options with parallel collectors:

• Young (-XX+UseParallelGC)

• Young and Old (-XX+UseParallelOldGC - Compacting)

• Throughput is important

• Suitable for

• Machines with large memory, multiple processors & cores

CMS COLLECTOR

• Focus: Responsiveness

• Low pause times are required

• Concurrent collector
CMS COLLECTOR
g1 Collector
g1 Collector [REGIONS]
g1: YOUNG GC
g1: YOUNG GC
g1: YOUNG GC [end]
g1: PHASES
1. Initial Mark (stop-the world)

2. Root region scanning

3. Concurrent marking

4. Remark (stop-the-world)

5. Cleanup (stop-the-world & concurrent)

* Copying (stop-the-world)
g1: PHASES [INITIAL MARK]
g1: PHASES [Concurrent mark]
g1: PHASES [REMARK]
g1: PHASES [COPYING/CLEANUP]
g1: PHASES [AFTER COPYING]
 COMMAND LINE
6 Monitoring
koders.co
Objectıves
• Using JVM command line tools

• jps, jmd, stat

• Monitor JVMs

• Identify running JVMs

• Monitor GC & JIT activity

MONITORING

• First step to observe & identify (possible) problems

MONITORING
WHAT TO MONITOR
• Parts of interest

• Heap usage & Garbage collection

• JIT compilation

• Data of interest

• Frequency and duration of GCs

• Java heap usage

• Thread counts & states

JDK COMMAND LINE TOOLS

• jps

• jmcd

• jstat
JIT COMPILATION
• JIT compiler: optimizer, just in-time compiler

• Command line tools to monitor

• -XX:+PrintCompilation (~2% CPU)

• jstat

• Data of interest

• Frequency, duration, opt/de-opt cycles, failed compilations

INTERFERING JIT COMPILER
• .hotspot_compiler file

• Turns of jit compilation for specified methods/classes

• Very rarely used

• Opt/de-opt cycles, failure or possible bug in JVM

INTERFERING JIT COMPILER
• Via .hotspot_compiler file:

• exclude Package/to/Class method

• exclude java/lang/String toString

• Via command line:

• -XX:CompileCommand=exclude,java/lang/String,toString
 Monitoring OS
7 Performance
koders.co
Objectıves
• Monitor CPU usage

• Monitor processes

• Monitor network & disk & swap I/O

• On Linux (+Windows)
Terminology

• CPU Utilization: Percentage of the CPU usage

(user+kernel)

• User CPU Utilization: the percent of time the application

spends in application code
TERMINOLOGY

• Memory Utilization: Memory usage percentage and

whether all the memory used by process reside in
physical (ram) or virtual (swap) memory.

• Swapping (using disk space as virtual memory) is pretty

expensive and should be avoided all times.
TERMINOLOGY

• Lock Contention: The case where a thread or process

tries to acquire a lock held by another process or
thread.

• Prevents concurrency and utilization. Should be avoided as

much as possible.
TERMINOLOGY

• Network & Disk I/O Utilization: The amount of data

sent and received via network and disk.

• Should be traced and used carefully.

Monitoring CPU Usage
• Monitor general and process based CPU usage

• Key definitions & metrics

• User (usr) time

• System (sys) time

• Voluntary context switch (VCX)

• Involuntary context switch (ICX)

MONITORING CPU
• Key points

• CPU utilization

• High sys/usr time

• CPU scheduler run queue

Monitoring CPU Usage
• Tools to use (Linux)

• top • prstat

• htop • gnome-system-monitor

• vmstat
MONITORING MEMORY
• Key points

• Memory footprint

• Change in usage of memory

• Virtual memory usage

MONITORING MEMORY

• Tools to use (Linux)

• free

• vmstat
MONITORING DISK I/O
• Key points

• Number of disk accesses

• Disk access latencies

• Virtual memory usage

MONITORING DISK I/O
• Tools to use (Linux)

• iostat

• lsof

• iotop
MONITORING NETWORK I/O
• Key points

• Connection count

• Connection statistics & states

• Total network traffic

MONITORING NETWORK I/O
• Tools to use (Linux)

• netstat • iftop

• iptraf • monitorix

• tcpdump
 USING
8 Visual Tools
koders.co
Objectıves
• Monitor Java applications using visual tools:

• JConsole

• VisualVM

• Mission Control
JConsole
• Ships with JVM

• Enables to monitor and

control JVM

• CPU, Memory,
Classloading, Threads

• Demo
 VISUALVM
• Graphical monitoring,
profiling, troubleshooting
tool

• Has Profiling and

Sampling capabilities

• Has plugin support

(Visualgc, btrace and
more)

• Demo
MISSION CONTROL
• Comprehensive
application

• Better UI

• Lots of useful information

• Monitor,
operate,manage, profile
Java applications

• Demo
JMX - MANAGED BEANS
• JMX: Java Management Extensions

• Used to monitor & manage JVM

• Managed Beans (MBeans)

• Objects used to manage Java resources

• Managed by JMX agents

 PROFILING JAVA
9 APPLICATIONS
koders.co
Objectives
• Profiling Java applications using:

• jmap and jhat

• JVisual VM

• Java Flight Recorder

JMAP and JHAT
• JVM command line tools

• jmap: Creates heap profile data

• jhat: Primitively Presents data in browser

• Demo
 VISUALVM

• Sampling & profiling

abilites

• Sampling: less intrusive

• Demo
10 Profiling
Performance Issues
koders.co
Objectives
• Profiling Java applications to troubleshoot and
optimize

• Detecting memory leaks

• Detecting lock contentions

• Identifying anti-patterns in heap profiles

HEAP PROFILING
• Necessary when:

• Observing frequent garbage collections

• Need for a larger heap by application

• Tune application for better performance & hardware

utilization
HEAP PROFILING: TIPS
• What to look for ?
• Objects with
• a large amount of bytes being allocated
• a high number of object allocations
• Stack traces where
• large amounts of bytes are being allocated
• large number of objects are being allocated
HEAP PROFILING: TOOLS
• jmap and jhat

• Snapshot of the application

• Top consumers & Allocation stack traces

• Compare multiple snapshots

MEMORY LEAK
• Refers to the situation when an object unintentionally
resides in memory thus can not be collected by GC.

• Frequent garbage collection

• Poor application performance

• Application failure (Out of memory error) Frequent

garbage collection
MEMORY LEAK: TOOLS

• Visual VM

• Flight Recorder

• jmap and jhat

MEMORY LEAK: TIPS
• Monitor running application

• Look for memory changes, survivor generations

• Profile applications, compare snapshots

• Look for object count changes, top grovers

• Always use -XX:+HeapDumpOnOutOfMemoryError

parameter on production
LOCK CONTENTION

• Usage of synchronization utilities (synchronized,

locks, conc. collections, etc.) cause threads to wait or
perform worse.

• Should be kept as minimum as possible.

LOCK CONTENTION: MONITOR
• Things to observe:

• High number of voluntary context switches

• Thread states and state changes (Visual VM, Flight

Recorder)

• Possible deadlocks (jstack, Visual Tools)

PROFILING ANTI-PATTERNS
• Frequent garbage collections

• Overallocation of objects

• High number of threads

• High volume of lock contention

• Large number of exception objects

 GARBAGE COLLECTION
11 Tuning
koders.co
Objectives

• Learning to tune GC by setting generation sizes

• Comparing and selecting suitable GC for

performance requirements

• Monitor and understand GC outputs

Garbage Collectıon
• Main tasks of GC

• Allocating memory for new objects

• Keeping live (referenced) objects in memory

• Removing dead (unreferenced) objects and reclaiming

memory used by them
JVM Heap Size Options
JVM Heap Size Options
-Xmx<size> : Maximum size of the Java heap
-Xms<size> : Initial heap size
-Xmn<size> : Sets initial and max heap sizes as same
-XX:MaxPermSize=<size> : Max Perm size
-XX:PermSize=<size> : Initial Perm size
-XX:MaxNewSize=<size> : Max New size
-XX:NewSize=<size> : Initial New size
-XX:NewRatio=<size> : Ratio of Young to Tenured space
GARBAGE COLLECTORS
• Serial Collector

• Parallel (Throughput) Collector

• Concurrent Mark-Sweep (CMS) Collector

• Garbage First (G1) Collector

SERIAL COLLECTOR

• Single-threaded young generation collector

• Single-threaded old generation collector

• Parameter: -XX:+UseSerialGC
SERIAL COLLECTOR: TIPS
• Not suitable for applications with high performance
requirements

• Can be suitable for client applications with limited

hardware resources

• More suitable for platforms that has less than 256

MB of memory for JVM and do not have multicores
PARALLEL COLLECTOR
• Multi-threaded young generation collector

• Multi-threaded old generation collector

• Parameters:

• -XX+UseParallelGC (Parallel Young, Single-Threaded Old)

• -XX:+UseParallelOldGC (Young&Old BOTH MultiThreaded)

PARALLEL COLLECTOR: TIPS
• Suitable for applications that target throughput rather
than responsiveness

• Suitable for platforms that have multiple processors &

cores

• -XX:ParallelGCThreads=[N] can be used to specify GC

thread count

• default = Runtime.availableProcessors() (JDK 7+)

• Better reduced if multiple JVMs running on the same machine

CMS COLLECTOR

• Multi-threaded young generation collector

• Single-threaded concurrent old generation collector

• Parameter: -XX:+ConcMarkSweepGC
CMS COLLECTOR: GOOD TO KNOW
• CMS targets responsiveness and runs concurrently.
And it doesn’t come for free.

• More memory (~20%) and CPU resources needed

• Memory fragmentation

• It can lose the race. (Concurrent mode failure)

CMS COLLECTOR: GOOD TO KNOW

• CMS has to start earlier to collect not to lose the race

• -XX:CMSInitiatingOccupancyFraction=n (default 60%, J8)

• n: Percentage of tenured space size

CMS COLLECTOR: TIPS
• Size young generation as large as possible

• Small young generation puts pressure on old generation

• Consider heap profiling

• Choose tuning survivor spaces

• Enable class-unloading if needed (appservers, etc.)

-XX:+CMSClassUnloadingEnabled, -XX+PermGenSweepingEnabled
CMS: TIPS

• TODO : CMS important parameters

G1 Collector
• Parallel and concurrent young generation collector

• Single-threaded old generation collector

• Parameter: -XX:+UseG1GC

• Expected to replace CMS (J9)

G1 Collector: GOOD TO KNOW
• Concurrent & responsiveness collector like G1.
Suitable for multiprocessor platforms and heap sizes
of 6GB or more.

• Targets to stay within specified pause-time

requirements.

• Suitable for stable and predictable GC time 0.5 seconds or

below.
G1 COLLECTOR: TIPS
• G1 optimizes itself to meet pause-time requirements.

• Do not set the size of young generation space

• Use 90% goal instead of average response time (ART)

• A lower pause-time goal causes more effort of GC,

throughput decreases
 Language-Level
12 TIPS & TRICS
koders.co
Objectives
• Object allocation best practices

• Java reference types and differences between them

• Usage of finalizers

• Synchronization tips & tricks & best practices

OBJECTS: BEST PRACTICES

• The problem is not the object allocation, nor the

reclamation

• Not expensive: ~10 native instructions in common case

• Allocating small objects for intermediate results is fine

OBJECTS: BEST PRACTICES
• Use short-lived immutable objects instead of long-
lived mutable objects.

• Functional Programming is rising !

• Use clearer, simpler code with more allocations

instead of more obscure code with fewer allocations

• KISS: Keep It Simple Stupid

• “Premature optimization is root of all evil” - Donald Knuth

OBJECTS: BEST PRACTICES
• Large Objects are expensive !

• Allocation

• Initialization

• Different sized large objects can cause fragmentation

• Avoid creating large objects

JAVA REFERENCE TYPES
REFERENCES: SOFT REFERENCE
• “Clear this object if you don’t have enough memory, I
can handle that.”

• get() returns the object if it is not reclaimed by GC.

• -XX:SoftRefLRUPolicyMSPerMB=[n] can be used to

control lifetime of the reference (default 1000 ms)

• Use case: Caches

REFERENCES: WEAK REFERENCE

• “Consider this reference as if it doesn’t exist. Let me

access it if it is still available.”

• get() returns the object if it is not reclaimed by GC.

• Use case: Thread pools

REFERENCES: PHANTOM REFERENCE

• “I just want to know if you have deleted the object or

not”

• get() always returns null.

• Use Case: Finalize actions

FINALIZERS
• Finalizers are not equivalents of C++ destructors

• Finalize methods have almost no practical and

meaningful use case

• Finalize methods of objects are called by GC threads.

• Handled differently than other objects, create pressure on GC

• Time consuming operations lengthen GC cycle

• Not guaranteed to be called

LANGUAGE TIPS: STRINGS

• Strings are immutable

• String “literals” are cached in String Pool

• Avoid creating Strings with “new”

LANGUAGE TIPS: STRINGS

• Avoid String concatenation

• Use StringBuilder with appropriate initial size

• Not StringBuffer (avoid synchronization)

LANGUAGE TIPS: USE PRIMITIVES

• Use primitives whenever possible, not wrapper

objects.

• Auto Boxing and Unboxing are not free of cost.

LANGUAGE TIPS: AVOID EXCEPTIONS
• Exceptions are very expensive objects

• Avoid creating them for

• non-exceptional cases

• flow control
THREADS
• Avoid excessive use of synchronized

• Increases lock contention, leads to poor performance

• Can cause dead-locks

• Minimize the synchronization

• Only for the critical section

• As short as possible

• Use other locks, concurrent collections whenever suitable

Threads: TIPS
• Favor immutable objects

• No need for synchronization

• Embrace functional paradigm

• Do not use threads directly

• Hard to maintain and program correctly

• Use Executers, thread pools

• Use concurrent collections and tune them properly

CACHING
• Caching is a common source of memory leaks

• Avoid when possible

• Avoid creating large objects in the first place

• Mind when to remove any object added to cache

• Make sure it happens, in any condition

That’s all folks!
Congrats!
Ender Aydin Orak

koders.co