Making master/slave systems work better with pgpool-II

SRA OSS, Inc. Japan ats!o Ishii

Abo!t me
G

R!nning SRA OSS, Inc. Japan &ostgreS'( )eveloper/committer

G G

I$*+ works Several contrib tools incl!)ing pgbench

,riting books/articles abo!t &ostgreS'(

"#$#/%/"

ats!o Ishii

"

Abo!t SRA OSS, Inc. Japan

G G

G G

G G

.stablishe) in "##/ &rovi)s commercial s!pport 0or &ostgreS'( an) other OSS +o MyS'( s!pport Sells commercial packages base) on &ostgreS'( rainings an) certi0ications S!pports &ostgreS'( comm!nity
G G

1!n)ing J&23 4oar) members

ats!o Ishii -

"#$#/%/"

&ostgreS'( is 9!ite pop!lar in Japan

&ost0i< vs0tp) OpenSS( OpenSS; 4in) Sen)mail Samba M yS'( &ostgreS'( Apache # $# "# -# 5# /# =# %#

&ostgreS'( is ne<t to Apache

1rom I&A6In0ormation technology promotion agency78s s!rvey in "##* "#$#/%/" ats!o Ishii 5

"#$#/%/"

ats!o Ishii

,hat is pgpool-II>
G

G G

?et another cl!ster mi))le ware )e)icate) 0or &ostgreS'( @omm!nity )evelope) open so!rce so0tware proAect Rich 0eat!res
G G G G G G

Synchrono!s replication (oa) balancing A!tomatic 0ailover @onnection pooling &arallel 9!ery @ollaborating with other replication tools
G

Slony-I, ,orm stan)by, Streaming replication

ats!o Ishii =

"#$#/%/"

4asic i)ea o0 pgpool-II

ransparent against both &ostgreS'( server an) client 9!ery &ostgreS'( server 9!ery &ostgreS'( client pgpool-II 9!ery &ostgreS'( server &ostgreS'( server

9!ery

"#$#/%/"

ats!o Ishii

he architect!re o0 pgpool-II
+ew in pgpool-II -.#
pcp process pgpool-II parent ;ealth @hecking &ostgreS'( @lient pgpool-II pgpool-II chil) chil) '!ery worker process

A)min

Replication Belay checking

'!ery

'!ery
"#$#/%/" ats!o Ishii

&ostgreS'(
*

Replication mo)e
ransparent against both &ostgreS'( server an) client 9!ery &ostgreS'( server

9!ery

B4 cl!sters are i)entical pgpool-II &ostgreS'( server

&ostgreS'( client

9!ery

"#$#/%/"

ats!o Ishii

&ros an) cons o0 pgpool-II replication

G

&ros
G G

G G

Synchrono!s replicationD no nee) to worry abo!t Eevent!ally consistentF problem. +o transaction loss A!tomatic 0ailoverD no phone calls 0rom angry c!stomers while yo! are in be) @onnection pooling an) loa) balancingD boost per0ormance Online recovery. ,itho!t stopping pgpool-II, yo! can repair or a)) a B4 no)e .asy to con0ig!re

@ons
G

,rite per0ormance is not goo)6abo!t -#: overhea)7 G Some 9!eries con0!se pgpool-IID ran)om67, se9!ences, 0!nctions those have a si)e e00ect6write to the )atabase7 "#$#/%/" ats!o Ishii

$#

Master slave mo)e with Slony-I

master

write 9!ery

9!ery

B4 cl!sters are event!ally i)entical pgpool-II rea) 9!ery slave

&ostgreS'( client

"#$#/%/"

ats!o Ishii

$$

&ros an) cons o0 master/slave mo)e

G

&ros
G G G

,rite per0ormance is goo) 6$#-"#: overhea)7 A!tomatic 0ailover o0 slaves @onnection pooling an) loa) balancingD boost per0ormance Asynchrono!s replication
G

@ons
G

Replication )elay is relatively large A)ministrator8s hea)ache

+o BB( replication
G

G G
"#$#/%/"

+o large obAects replication ;ar) to con0ig!re

ats!o Ishii $"

"#$#/%/"

ats!o Ishii

$-

,hat we nee) isD

G G G

(ow write overhea) (ow replication )elay ransparent replication

G G

BB( replication (arge obAect replication

.asy to manage

Sreaming ReplicationG;ot Stan)by

"#$#/%/" ats!o Ishii $5

Streaming replicationG;ot Stan)by

&rimary rea)/write 9!ery

,A( log ,A( sen)er

rea) 9!ery

,A( receiver ,A( log Stan)by

"#$#/%/"

ats!o Ishii

$/

(ow write overhea)

"#$#/%/"

ats!o Ishii

$=

(ow replication )elay

"#$#/%/"

ats!o Ishii

$%

;owever...
G G G

+o a!tomatic 0ailover +o connection pooling, loa) balancing ?o! nee) to know what the stan)by )islikeD

Bata Manip!lation (ang!age 6BM(7 - I+S.R , 2&BA ., B.(. ., @O&? 1ROM, R2+@A .. +ote that there are no allowe) actions that res!lt in a trigger being e<ec!te) )!ring recovery. Bata Be0inition (ang!age 6BB(7 - @R.A ., BRO&, A( .R, @OMM.+ . his also applies to temporary tables also beca!se c!rrently their )e0inition ca!ses writes to catalog tables. S.(.@ ... 1OR S;AR. H 2&BA . which ca!se row locks to be written R!les on S.(.@ statements that generate BM( comman)s. (O@I that e<plicitly re9!ests a mo)e higher than RO, .J@(2SIK. MOB.. (O@I in short )e0a!lt 0orm, since it re9!ests A@@.SS .J@(2SIK. MOB.. ransaction management comman)s that e<plicitly set non-rea)-only stateD 4.3I+ R.AB ,RI ., S AR S. RA+SA@ IO+ R.AB ,RI . RA+SA@ IO+ R.AB ,RI ., S. S.SSIO+ @;ARA@ .RIS I@S AS RA+SA@ IO+ R.AB ,RI .

S. transactionLrea)Lonly M o00 wo-phase commit comman)s - &R.&AR. RA+SA@ IO+, @OMMI &R.&AR.B, RO((4A@I &R.&AR.B beca!se even rea)-only transactions nee) to write ,A( in the prepare phase 6the 0irst phase o0 two phase commit7. Se9!ence !p)ates - ne<tval67, setval67 (IS .+, 2+(IS .+, +O I1? "#$#/%/" ats!o Ishii $*

Streaming replicationG;ot Stan)byG pgpool-II con0ig!ration

&rimary rea)/write 9!ery ,A( log ,A( sen)er

pgpool-II

rea)/write 9!ery

rea) 9!ery

,A( receiver ,A( log Stan)by

"#$#/%/"

ats!o Ishii

$C

&ros an) cons o0 Streaming replicationG ;ot Stan)byGpgpool-II

G

&ros
G G G G G

,rite per0ormance is goo) 6$#-"#: overhea)7 A!tomatic 0ailover o0 slaves @onnection pooling an) loa) balancingD boost per0ormance BB( replication (arge obAect replication Asynchrono!s replication
G

@ons
G

;owever replication )elay is relatively low

"#$#/%/"

ats!o Ishii

"#

+ew 0eat!res in pgpool-II -.#

G

+ew master/slave Es!bLmo)eF )e)icate) 0or Streaming replication an) ;ot stan)by A!tomatic 9!ery )ispatches
G

Sen) write 9!eries to the primary &rior releasesD sen) rea) 9!eries to the stan)by whenever possible -.#D @heck the replication )elay between the primary an) the stan)by. I0 it8s e<cee)s E)elayLthreshol)F, then sen) to the primary only.
ats!o Ishii "$

Intelligent loa) balancing

G G

A))ing stan)by servers witho!t stopping pgpool-II

"#$#/%/"

Replication )elay )etectionD how it works

S.(.@ pgLc!rrentL logLlocation

S.(.@ pgLlastL<logL receiveLlocation

call the 0!nctions every health check perio)s secon). calc!late the )i00erence. i0 it e<cee)s )elay threshol), sen) rea) 9!eries to the primary only. worker process
"#$#/%/" ats!o Ishii ""

(ogging replication )elay

G

logLstan)byL)elay
G G

8none8D no logging )elay 8i0LoverLthreshol)8D log only when )elay e<cee)s )elayLthreshol) 8always8D always log )elay
"#$#-#=-"* $/D/$D-" (O3D pi) $-""-D Replication 0rom the primary server 6no)eD#7 "#$#-#=-"* $/D/$D5" (O3D pi) $-""-D Replication 0rom the primary server 6no)eD#7 "#$#-#=-"* $/D/$D/" (O3D pi) $-""-D Replication 0rom the primary server 6no)eD#7 "#$#-#=-"* $/D/"D#" (O3D pi) $-""-D Replication 0rom the primary server 6no)eD#7 "#$#-#=-"* $/D/"D$" (O3D pi) $-""-D Replication 0rom the primary server 6no)eD#7 "#$#-#=-"* $/D/"D"" (O3D pi) $-""-D Replication 0rom the primary server 6no)eD#7 o0 no)eD$ is behin) $""**## bytes o0 no)eD$ is behin) --"/C/" bytes o0 no)eD$ is behin) C%5*5* bytes o0 no)eD$ is behin) "CC##*# bytes o0 no)eD$ is behin) C#$$"# bytes o0 no)eD$ is behin) "5--#"5 bytes
"-

"#$#/%/"

ats!o Ishii

Scaling !p with pgpool-IIGStreaming replicationG;ot stan)by

G

Streaming replication scales well

G

,e can a)) stan)by servers witho!t sacri0icing per0ormance More stan)by servers means better rea) per0ormance with loa) balancing

"#$#/%/"

ats!o Ishii

"5

S!mmary
G G

pgpool-II overview Kario!s con0ig!rations o0 pgpool-II an) their pros an) cons
G G G

Replication mo)e Master/Slave mo)e 6Slony-I7 Master/Slave mo)e 6Streaming replication G ;ot stan)by7

pgpool-II -.# G Streaming replication G ;ot stan)by will o00er one o0 the i)eal replication sol!tions
G G G

.asy to !se/manage (ow replication )elay (ow write overhea)Ggoo) scalability

ats!o Ishii "/

"#$#/%/"

2R(S
G

pgpool-II can be )ownloa)e) hereD

G

httpD//pg0o!n)ry.org/proAects/pgpool/

Kisit SRA OSS8s website i0 yo! nee) commercial s!pport 0or pgpool-II an)/or &ostgreS'(D
G

httpD//www.srasoss.co.Ap/in)e<Len.php

"#$#/%/"

ats!o Ishii

"=