Reactive Programming Reactor
Reactive Programming Reactor
in Modern Java
using
Project Reactor
Dilip
About Me
• Dilip
• Reactive Streams
• Any developer who has the requirement to write fast performing code under
heavy load
App Blocking DB
Thread
• No downtime
Today’s Architecture (Backend RestFul API)
DB
1
App 2 {API}
Embedded Server 3
• More threads leads to more heap space which leads to shortage of JVM
memory for handling the request
Today’s Architecture (Backend RestFul API)
DB
1
App 2 {API}
Embedded Server 3
App 2 {API}
Embedded Server 3
{API}
What are the tools/api that’s
available in Java ?
Asynchronous/Concurrency APIs in Java
• CallBacks
• Future
Callbacks
Callbacks
• Asynchronous methods that accept a callback as a parameter and invokes it
when the blocking call completes.
• Writing code with Callbacks are hard to compose and difficult to read and
maintain
• Callbackhell
Future
Concurrency APIs in Java
Future CompletableFuture
• This holds the contract for reactive streams but no implementation is available
as part of JRE
Summary
requestForData( )
request(n)
onNext(1) DB
App onNext(2)
.
.
.
onNext(n)
onComplete( )
requestForData( )
request(n)
onNext(1)
DB
App onNext(2)
.
.
.
onNext(n)
onComplete( )
requestForData( )
request(2 )
DB
App onNext(1)
onNext(2)
cancel( )
Backpressure
Push-based data flow model
Push-Pull based data flow model
When to use Reactive Programming ?
App 2 {API}
Embedded Server 3
{API}
• Lightbend
• Netflix
• VmWare (Pivotal)
Reactive Streams Specification
• Reactive Streams Specification:
• Publisher
• Subscriber
• Subscription
• Processor
Publisher
public interface Publisher<T> {
requestForData( )
onNext(1)
DB
App onNext(2)
.
.
.
onNext(n)
onComplete( )
Subscription
public interface Subscription {
requestForData( )
request(2 )
DB
App onNext(1)
onNext(2)
cancel( )
Success Scenario
subscribe( this ) 1
DB
onSubscribe( ) 2
Subscription
request( n ) 3
onNext( 1 ) 4
Subscriber Publisher
onNext( 2 ) 4.1
.
.
.
4.n
onNext( n )
{API}
onComplete( ) 5
Reactive Streams - How it works together ?
Error/Exception Scenario
subscribe( this ) 1
DB
onSubscribe( ) 2
Subscription
request( n ) 3
Subscriber Publisher
onError( ) 4
{API}
map()
filter ()
Mono - 0 to 1 Element
onComplete( )
onError( )
Project Setup
Functional Programming
In
Modern Java
Why Functional Programming ?
Why Functional Programming ?
• Reactive programming uses Functional Programming style of code
requestForData( )
request(n) 1
onNext(1) 2.1
DB
App onNext(2) 2.2
.
.
.
2.3
onNext(n)
onComplete( ) 3
Testing Flux and Mono
using
StepVerifier & JUnit5
Transforming Data
Using Operators
in
Project Reactor
Why Transform Data ?
• It is pretty common for apps to transform data from its original form
UpperCase
UpperCase &
Filter String length
is greater then 3
Flux.just("alex", "ben", "chloe") Flux.just("ALEX", “CHLOE")
Why Transform Data ?
DB
App
{API}
map()
map() Operator
• Used to transform the element from one form to another in a Reactive Stream
• Returns a Flux<Type>
map() flatmap()
• One to One Transformation • One to N Transformations
• Does the simple transformation • Does more than just
from T to V transformation. Subscribes to
Flux or Mono that’s part of the
transformation and then flattens it
and sends it downstream
• Used for simple synchronous
transformations • Used for asynchronous
transformations
• Does not support transformations
that returns Publisher • Use it with transformations that
returns Publisher
concatMap()
concatMap()
• Works similar to flatMap()
• Use flatMap if the transformation involves making a REST API call or any kind
of functionality that can be done asynchronously
flatMapMany( )
in
Mono
flatMap in Mono
• Works very similar to flatMap( )
• Its not mandatory for a data source to emit data all the time
requestForData( )
request(n) 1
DB
App
onComplete( ) 3
{API}
App
Response from
Data Source
DB
{
"response": "output"
}
concat()
&
concatWith()
concat() & concatWith()
• Used to combine two reactive streams in to one
App
return Flux.merge(abcFlux,defFlux).log();
}
merge() & mergeWith()
• Both the publishers are subscribed at the same time
• Publishers are subscribed eagerly and the merge happens in an interleaved fashion
• Even though the publishers are subscribed eagerly the merge happens in a
sequence
zip()
&
zipWith()
zip()
• Zips two publishers together in this example
// AD, BE, CF
public Flux<String> explore_zip() {
} AD BE CF
zip() & zipWith()
• zip()
• Static method that’s part of the Flux
• Can be used to merge up-to 2 to 8 Publishers (Flux or Mono) in to one
• zipWith()
• This is an instance method that’s part of the Flux and Mono
• Used to merge two Publishers in to one
• Publishers are subscribed eagerly
• Waits for all the Publishers involved in the transformation to emit one element
• Continues until one publisher sends an OnComplete event
zip()
• Zips four publishers together in this example
// AD14, BE25, CF36
public Flux<String> explore_zip_1() {
MovieInfo
Service
getAllMovies
Movie Service
Client movieId
(Reactive)
getMovieById()
Review
Service
doOn CallBacks
doOn* CallBacks
• These operators allow you to peek in to the events that are emitted by the
Publisher(Flux or Mono)
• There are many different callback operators that are available in Project
Reactor
DoOn* CallBack operators
doOnSubscribe( )
Invoked for every new subscription from the Subscriber
External
App
Service
Any Exception will terminate the
Reactive Stream
Exception Handling
In
Project Reactor
Exception Handling in Project Reactor
• onErrorReturn() • onErrorMap()
• onErrorResume() • doOnError()
• onErrorContinue()
onErrorReturn()
onErrorReturn()
Flux A B C X
onErrorReturn() D
onErrorResume()
onErrorResume()
Flux A B C X
onErrorResume()
Flux D E F
Conditional Recovery
onErrorContinue()
onErrorContinue()
• Catches the exception
• This drops the element that caused the exception and continue emitting the
remaining elements
Flux A B C
map
onErrorContinue()
onErrorMap()
onErrorMap()
• Catches the exception
• Transforms the exception from one type to another
• Any RunTimeException to BusinessException
• Does not recover from the exception
Flux A B C
map
onErrorMap()
doOnError()
doOnError()
• onErrorReturn() • onErrorMap()
• Catches the exception and provides a recoverable
single default value
• Catches the exception and
• Stream that caused the error will be terminated transform to some Custom
• onErrorResume() Exception type
• Catches the exception and provides another
dynamic reactive stream as a fallback value • doOnError()
• Stream that caused the error is terminated
• Catch the exception and
• Conditional Recovery propagate it down stream
• onErrorContinue()
• Catches the exception and allows the reactive
stream to continue emitting elements
Exception Handling
Operators
In
Mono
Mono has the support for all the
exception handling operators
that we coded until now for Flux
Exception Handling in Mono
Recover From an Exception Take an Action and throw the Exception
(Category 1) (Category 2)
• onErrorReturn() • onErrorMap()
• Catches the exception and provides a recoverable
single default value
• Catches the exception and
• Stream that caused the error will be terminated transform to some Custom
• onErrorResume() Exception type
• Catches the exception and provides another
dynamic reactive stream as a fallback value • doOnError()
• Stream that caused the error is terminated
• Catch the exception and
• Conditional Recovery propagate it down stream
• onErrorContinue()
• Catches the exception and allows the reactive
stream to continue emitting elements
retry()
retry()
• When to use it ?
• retry()
• retry(N)
• N is a long value
• This operator gets invoked after the onCompletion( ) event from the existing
sequence
• repeat( )
• repeat(n)
• repeat( )
• This operator gets invoked when there is an OnComplete() event from the original
sequence
Reactor Execution Model
Reactor Execution Model
• Reactor Execution model is determined by Scheduler
return Flux.fromArray(charArray)
Switched the thread to “parallel”
.delayElements(Duration.ofMillis(delay));
}
• Schedulers.parallel()
• It has a fixed pool of workers. No of threads is equivalent to no of CPU cores
• The time based operators use this by default (delayElements(), interval())
• Schedulers.boundElastic()
• It has a bounded elastic thread pool of workers
• The no of threads can grow based on the need. It can increase up to 10 X no of CPU cores
• This is ideal for making Blocking IO calls
• Schedulers.single()
• A single reusable thread for executing the tasks
publishOn(Scheduler s)
publishOn(Scheduler s)
• This operator is used to hop the Thread of execution of the reactive pipeline from one to
another.
• Blocking operation in the reactive pipeline can be performed after publishOn operator.
MovieInfo
Service
getAllMovies
Revenue
Service
Push model
subscribe()
API
subscription()
request(unbounded)
onComplete() DB
2. Data might be pushed at a faster rate than the subscriber can handle
BackPressure
subscribe()
API
subscription()
request(2)
onNext(1)
Subscriber Publisher
onNext(2)
cancel()
DB
onCancel()
onBackPressureDrop( )
onBackPressureDrop( )
• Overrides the subscribers request and requests for unbounded data
• Drops the remaining elements that are not needed by the subscriber
• This operator helps to track the items that are not needed by the subscriber
onBackPressureBuffer( )
onBackPressureBuffer( )
• Overrides the subscribers request and requests for unbounded data
• Buffers the remaining elements that are not needed by the subscriber
• The advantage is that the following requests after the initial request for data
from the subscriber does not need to go all the way to the Publisher
onBackPressureError( )
onBackPressureError( )
• Overrides the subscribers request and requests for unbounded data
• Throws an OverflowException when the publisher sends more data than the
subscriber’s requested amount
Processing Data in
Parallel
in
Project Reactor
Reactive Flow is sequential by
default
ParallelFlux
ParallelFlux
• The idea behind ParallelFlux is to leverage the multi-core processors that we
have in today’s hardware
No of elements that can be processed in parallel is equal to the no of cores in your machine
Parallelism using flatMap( )
flatMapSequential()
flatMapSequential( )
@Test
public void coldPublisherTest() throws InterruptedException {
• Any new subscriber will only get the current state of the Reactive Stream
• Type 1: Waits for the first subscription from the subscriber and emits the
data continuously
• Type 2: Emits the data continuously without the need for subscription
Hot Streams
• Examples:
• Uber Driver Tracking -> Emits the of the current position of the Driver
Continuously
Building Non Blocking Client
Using
Spring WebClient
Movie Service
Static Data
MovieInfo
getAllMovies
Service
getAllMovies MovieInfo
• Spring WebFlux
• Uses H2 InMemory DB
Programmatically
Creating
Flux/Mono
Creating Flux/Mono Until Now
Flux Mono
Flux.just("A", "B", "C"); Mono.just("alex");
Flux.fromIterable(namesList) Mono.empty()
Flux.fromArray(charArray) Mono.fromCallable()
webClient.get().uri("/v1/movie_infos/{id}",
Flux.range(0, max) movieInfoId)
.retrieve()
.bodyToMono(MovieInfo.class)
webClient.get().uri(uri)
.retrieve()
.bodyToFlux(Review.class)
Any External System will have its own Reactive Adapters to build Flux and Mono
Creating Flux/Mono Programmatically
Flux Mono
Flux.generate(); Mono.create()
Flux.create()
Flux.push()
Flux.handle()
We need to explicitly emit the
OnNext, OnComplete and onError events
from our code.
Flux.generate( )
Flux.generate( )
• This operator takes a initial value and a generator function as an input and
continuously emit values
• We will be able to generate the OnNext, OnComplete and onError events using the
SynchronousSink class
• Use this operator, if you have a use case to emit values from a starting value until a
certain a certain condition is met. (Similar to for loop)
Generate a sequence from 1 to 10
and Multiply each element by 2
1,2,3,4,5,6,7,8,9,10
2,4,6,8,10,12,14,16,18,20
Flux.create( )
Flux.create( )
• Used to bridge an existing API in to the Reactive World
• We will be able to generate the OnNext, OnComplete and onError events using
the FluxSink class
• reactor-tools
Using ReactorDebugAgent in SpringBoot
ReactorDebugAgent.init();
SpringApplication.run(Application.class, args);
}
Next Steps
Next Steps:
• Build Reactive APIS using Spring Webflux and Project Reactor