Java Concurrency and Performance

Training Content
4/15/2013 MOVAA Technologies Mohit Kumar

MOVAA Technologies 2 Java Concurrency and Performance

JAVA CONCURRENCY
(The most important reason for most applications(Enterprise and real-time) to fail or underperform is improper design for concurrency. This is especially true because of multi cores/processor becoming ubiquitous. This training course takes a wholistic view of concurrency including designing for multi-cores/processors(including NUMA).)

Pre-requisites: Basic knowledge of Java (introductory course or equivalent practical experience). Target Audience: The target group is programmers who want to know foundations of concurrent programming and existing concurrent programming environments, in order, now or in future, to develop multithreaded applications for multi-core processors and shared memory multiprocessors. Objective: Understand concurrency control issues in general. Know the instruments available in Java. Avoid common errors and pitfalls. Understand concurrency control idioms. What you will learn: To quickly identify the root cause of poor performance in your applications. Eliminate conditions that will prevent you from finding performance bottlenecks. Dealing with threads and collections on a multi-core/ multiprocessor. JDK 5, 6, 7 which have features to harness the power of the underlying hardware.

MOVAA Technologies | www.movaatechnologies.com

MOVAA Technologies 3 Java Concurrency and Performance

Brief Table of Contents

Producer Consumer(Basic Hand-Off) (Day:1) Common Issues with thread Java Memory Model(JMM) Applied Threading techniques Building Blocks for Highly Concurrent Design Highly Concurrent Data Structures-Part1 (Day 2) Designing For Concurrency Canned Synchronizers Highly Concurrent Data Structures-Part2 (Day 3) Crash course in Mordern hardware Concurrent Reasoning Concurrency Patterns Designing for multi-core/processor environment

MOVAA Technologies | www.movaatechnologies.com

MOVAA Technologies 4 Java Concurrency and Performance

Detailed Table of Contents

Producer Consumer(Basic Hand-Off) (Part-1)

Why wait-notify require Synchronization

locking handling done by OS Hidden queue Structural modification to hidden queue by wait-notify use cases for notify-notifyAll notifyAll used as work around design issues with synchronization

Common Issues with thread

problem with stop Dealing with InterruptedStatus Uncaught Exception Handler

Java Memory Model(JMM)

Sequential Consistency would disallow common optimizations Instruction Reordering

heavily pipelines processors super-scalar processors NUMA(Non uniform memory access)

Cache Coherency

Real Meaning and effect of synchronization Volatile Final The changes in JMM
4

Applied Threading techniques

Thread Local Storage Safe Construction techniques

MOVAA Technologies 5 Java Concurrency and Performance

UnSafe Construction techniques Thread safety levels

Building Blocks for Highly Concurrent Design

CAS Hardware based locking

Optimistic Design ABA problem

Markable reference Stamped reference weakCompareAndSet

Wait-free Stack implementation Wait-free Queue implementation Design issues with synchronization Multiple user conditions and wait queues Lock Polling techniques Reentrant Lock

Lock Implementation

ReentrantReadWriteLock ReentrantLock

Based on CAS Lock Striping on table Lock Striping on LinkNodes. segregating them based on Thread safety levels

Lock Striping

Indentifying scalability bottlenecks in java.util.Collection

MOVAA Technologies | www.movaatechnologies.com

MOVAA Technologies 6 Java Concurrency and Performance

Highly Concurrent Data StructuresPart1

Structure Almost immutability Using volatile to detect interference Read does not block in common code path remove/put/resize lock

Weakly Consistent Iterators vs Fail Fast Iterators LockFreeHashMap

For systems with more than 100 cpus/cores Constant Time key-value mapping no locks even during resize all CAS spin loop bounded faster than ConcurrentHashMap State based Reasoning

Designing For Concurrency

Confinement Immutability Almost Immutability Atomicity Visibility Restructuring and refactoring

Canned Synchronizers

Synchronous Queue Framework

MOVAA Technologies 7 Java Concurrency and Performance

Future Semaphore Mutex Barrier Latches SynchronousQueue Exchanger

Highly Concurrent Data Structures-Part2

CopyOnWriteArray(List/Set) Queue interfaces

Queue BlockingQueue Deque BlockingDeque

Queue Implementations

ConcurrentLinkedQueue LinkedBlockingQueue and LinkedBlockingDeque ArrayBlockingDeque ArrayDeque and ArrayBlockingDeque WorkStealing using Deques LinkedTransferQueue

Skiplists

ConcurrentSkipList(Map/Set)

Sequential Skiplist Lock based concurrent Skiplist Lock free concurrent Skiplist Concurrent Skiplist
7

MOVAA Technologies | www.movaatechnologies.com

MOVAA Technologies 8 Java Concurrency and Performance

Executor FrameWork

Configuration Hardware shapes programming idiom Exposing fine grained parallelism Divide and conquer Fork and Join Anatomy of Fork and Join Work Stealing Fork -join decomposition ParallelArray Limitations

Fork and Join Framework

Crash course in Mordern hardware

Amdahl's Law Cache

Direct mapped Address mapping in cache read write cache controller

Memory Architectures

UMA NUMA

Concurrent Reasoning

Sequential Consistency Linearizability Quiescent Consistency Compositionality

MOVAA Technologies 9 Java Concurrency and Performance

Concurrency Patterns
Fine grained Synchronization Optimistic Synchronization Lazy Synchronization Lock free Synchronization

Designing for multi-core/processor environment

Harsh Realities of parallelism Parallel Programming Concurrent Objects

Concurrency and Correctness Quiescent Consistency Sequential Consistency Linearizability Progress Conditions Lock suitable for NUMA systems Coarse Grained Synchronization Fine Grained Synchronization Optimistic Synchronization Lazy Synchronization Non Blocking Synchronization Bounded Partial Queue Unbounded Total Queue Unbounded lock-free Queue
9

Spinlocks

Lists

Concurrent Queues

Concurrent Stack
MOVAA Technologies | www.movaatechnologies.com

MOVAA Technologies 10 Java Concurrency and Performance

Concurrent Hashing Closed Address Hashing Open Address Hashing Lock Free Hashing Sequential Skiplist Lock based Concurrent Skiplist Lock free Skiplist Array Based bounded Priority Queue Tree based Bounded Priority Queue Heap Based Unbounded Priority Queue Skiplist based Unbounded priority Queue

Skiplist

Priority Queues

10

MOVAA Technologies | www.movaatechnologies.com