1

A 1echnlcal Cvervlew of Couchbase

uslng !ava
8aghavan 8ags" Srlnlvas
ueveloper AdvocaLe
S|mp|e. Iast. L|asnc.
2
ArchlLecL and LvangellsL worklng wlLh developers
Speaker aL !avaCne, 8SA conferences, Sun 1ech uays,
!uCs and oLher developer conferences
1aughL undergrad and grad courses
1echnology LvangellsL aL Sun MlcrosysLems for 10+
years
Sull Lrylng Lo undersLand and work more eecuvely on
!ava and dlsLrlbuLed sysLems
Couchbase ueveloper AdvocaLe worklng wlLh !ava and
8uby developers
hllosophy: !"#"$ &' ()*" )+ ,+)+-."$"/ 0,"-1'+
&()+ ) ,+0,"-1'+"/ )+-."$"
Speaker lnLroducuon
3
ulsLrlbuLed SysLems and erformance
A blL of noSCL
A qulck Lour of Lhe Couchbase Server
Cemng SLarLed and uemonsLrauons
A More deLalled Case SLudy
Secondary lndexlng and vlews
8oadmap, 8esources and Summary
Agenda
4
1he noSCL 8oom
CarLoon CourLesy: geek and poke
3
DIS1kI8U1LD SS1LMS AND
LkICkMANCL
6
ulsLrlbuLed Appllcauon ueslgn
www.myslLe.com/parLy
Web Servers
8elauonal
uaLabase
Load 8alancer
- Lxpenslve and dlsrupuve shardlng
- uoesnL perform aL Web Scale
7
1he neLwork ls rellable
LaLency ls zero
8andwldLh ls lnnlLe
1he neLwork ls secure
1opology doesn'L change
1here ls one admlnlsLraLor
1ransporL cosL ls zero
1he neLwork ls homogenous
LlghL lallacles of ulsLrlbuLed Compuung
8
A 8I1 CI NCSL
9
CA 1heorem
1he CA Lheorem sLaLes LhaL 2& 2- 234'--256" 7'$ ) /2-&$25,&"/ 8'34,&"$ -9-&"3 &' -23,6&)+"',-69
4$'*2/" )66 &($"" '7 &(" 7'66'.2+: :,)$)+&""-;
- ConslsLency means LhaL each node/cllenL always has Lhe
same vlew of Lhe daLa,
- AvallablllLy means LhaL all cllenLs can always read and
wrlLe,
- aruuon Lolerance means LhaL Lhe sysLem works well
across physlcal neLwork paruuons.

10
Web Application Architecture and Performance
Application Scales Out
Just add more commodity web
servers
Database Scales Up
Get a bigger, more complex
server
www.facebook.com/anlmalparLy
Web Servers
8elauonal
uaLabase
Load 8alancer
- Lxpenslve and dlsrupuve shardlng
- uoesnL perform aL Web Scale
11
Couchbase data layer scales like application logic tier
Data layer now scales with linear cost and constant performance.
Application Scales Out
Just add more commodity web
servers
Database Scales Out
Just add more commodity data
servers
Scaling out attens the cost and performance curves.
Couchbase Servers
www.facebook.com/anlmalparLy
Web Servers
Load 8alancer
PorlzonLally scalable, schema-less, auLo-
shardlng, hlgh-performance aL Web Scale
12
A ulsLrlbuLed Pash 1able
Appllcauon
seL(key, value)
geL(key)
reLurns value
uuaLa
ClusLer
uaLabase
13
A ulsLrlbuLed Pash 1able uocumenL SLore
Appllcauon
seL(key, [son)
geL(key)
reLurns [son
uuaLa
ClusLer
uaLabase
{
id": brewery_Legacy,
type : brewery,
"name" : "Legacy Brewing,
address": "525 Canal Street Reading",
"updated": "2010-07-22 20:00:20",
}
14
!SCn ls processed speclally on Lhe server
A !SCn-based documenL sLore
13
Plgh Level ArchlLecLure of !ava cllenL llbrary
Client
NIO
Connections to
server nodes
View Connections on
port 8092 (1.1-dp)
Set Connection Paramters in
CouchbaseConnectionFactoryBuilder
Bucket type?
KetamaNodeLocator
using Ketama Hash
VBucketNodeLocator
using Default Hash
Cluster
Updates
Check Status of Op.
with OperationFuture
(blocking)
Enqueue operations
Shutdown Connections
16
CCUCn8ASL SLkVLk
17
Couchbase Server ls Slmple, lasL, Llasuc
llve mlnuLes or less Lo a worklng clusLer
uownloads for Wlndows, Llnux and CSx
SLarL wlLh a slngle node
Cne buuon press [olns nodes Lo a clusLer
Lasy Lo develop agalnsL
!usL SL1 and CL1 - no schema requlred
urop lL ln. 10,000+ exlsung appllcauons already
speak Couchbase (vla memcached)
racucally every language and appllcauon
framework ls supporLed, ouL of Lhe box
Lasy Lo manage
Cne-cllck fallover and clusLer rebalanclng
Craphlcal and programmauc lnLerfaces
Congurable alerung
18
Couchbase Server ls Slmple, lasL, Llasuc
redlcLable
never keep an appllcauon walung
Cuasl-deLermlnlsuc laLency and LhroughpuL
Low laLency
8ullL-ln Memcached Lechnology
AuLo-mlgrauon of hoL daLa Lo lowesL laLency
sLorage Lechnology (8AM, SSu, ulsk)
SelecLable wrlLe behavlor - asynchronous,
synchronous (on repllcauon, perslsLence)
Plgh LhroughpuL
Mulu-Lhreaded
Low lock conLenuon
Asynchronous wherever posslble
AuLomauc wrlLe de-dupllcauon
19
Couchbase Server ls Slmple, lasL, Llasuc
Zero-downume elasuclLy
Spread l/C and daLa across commodlLy
servers (or vMs)
ConslsLenL performance wlLh llnear cosL
uynamlc rebalanclng of a llve clusLer
All nodes are creaLed equal
no speclal case nodes
Clone Lo grow
LxLenslble
Change feeds
8eal-ume map-reduce
8LS1ful lnLerface for managemenL
Couchbase Web Console
20
Couchbase Server 1.8 ArchlLecLure
P
e
a
r
L
b
e
a
L

r
o
c
e
s
s

m
o
n
l
L
o
r

C
l
o
b
a
l

s
l
n
g
l
e
L
o
n

s
u
p
e
r
v
l
s
o
r

C
o
n

g
u
r
a
u
o
n

m
a
n
a
g
e
r

on each node
8
e
b
a
l
a
n
c
e

o
r
c
h
e
s
L
r
a
L
o
r

n
o
d
e

h
e
a
l
L
h

m
o
n
l
L
o
r

one per clusLer
v
8
u
c
k
e
L

s
L
a
L
e

a
n
d

r
e
p
l
l
c
a
u
o
n

m
a
n
a
g
e
r

hup
8
L
S
1

m
a
n
a
g
e
m
e
n
L

A

l
/
W
e
b

u
l

P11
8091
Lrlang porL mapper
4369
ulsLrlbuLed Lrlang
21100 - 21199
Lrlang/C1
C|uster Manager
ers|stence Layer
sLorage lnLerface
Memcached
Couchbase L Lng|ne
11210
Memcapable 2.0
Mox|
11211
Memcapable 1.0
Data Manager
21
Couchbase Server 1.8 ArchlLecLure
ers|stence Layer
sLorage lnLerface
P
e
a
r
L
b
e
a
L

r
o
c
e
s
s

m
o
n
l
L
o
r

C
l
o
b
a
l

s
l
n
g
l
e
L
o
n

s
u
p
e
r
v
l
s
o
r

C
o
n

g
u
r
a
u
o
n

m
a
n
a
g
e
r

on each node
8
e
b
a
l
a
n
c
e

o
r
c
h
e
s
L
r
a
L
o
r

n
o
d
e

h
e
a
l
L
h

m
o
n
l
L
o
r

one per clusLer
v
8
u
c
k
e
L

s
L
a
L
e

a
n
d

r
e
p
l
l
c
a
u
o
n

m
a
n
a
g
e
r

hup
8
L
S
1

m
a
n
a
g
e
m
e
n
L

A

l
/
W
e
b

u
l

Memcached
Couchbase L Lng|ne
11210
Memcapable 2.0
Mox|
11211
Memcapable 1.0
P11
8091
Lrlang porL mapper
4369
ulsLrlbuLed Lrlang
21100 - 21199
Lrlang/C1
22
Couchbase Server 1.8 ArchlLecLure
P
e
a
r
L
b
e
a
L

r
o
c
e
s
s

m
o
n
l
L
o
r

C
o
n

g
u
r
a
u
o
n

M
a
n
a
g
e
r

C
l
o
b
a
l

s
l
n
g
l
e
L
o
n

s
u
p
e
r
v
l
s
o
r

on each node
8
e
b
a
l
a
n
c
e

o
r
c
h
e
s
L
r
a
L
o
r

n
o
d
e

h
e
a
l
L
h

m
o
n
l
L
o
r

one per clusLer
v
8
u
c
k
e
L

s
L
a
L
e

a
n
d

r
e
p
l
l
c
a
u
o
n

m
a
n
a
g
e
r

hup
k
L
S
1

M
a
n
a
g
e
m
e
n
t

]
W
e
b
U
I

P11
8091
Lrlang porL mapper
4369
ulsLrlbuLed Lrlang
21100 - 21199
Lr|ang]C1
ers|stence Layer
sLorage lnLerface
Memcached
Couchbase L Lng|ne
11210
Memcapable 2.0
Mox|
11211
Memcapable 1.0
23
uaLa buckeLs are secure Couchbase sllces
Couchbase daLa servers
ln Lhe daLa cenLer
Web appllcauon server
Appllcauon user
Cn Lhe admlnlsLraLor console
8uckeL 1
8uckeL 2
AggregaLe ClusLer Memory and ulsk CapaclLy
24

































CCuCP8ASL CLlLn1 Ll88A8?


8aslc Cperauon
! uocs dlsLrlbuLed evenly across
servers ln Lhe clusLer
! Lach server sLores boLh )81*"
& $"4628) docs
! Cnly one server acuve aL a ume
! CllenL llbrary provldes app wlLh
slmple lnLerface Lo daLabase
! ClusLer map provldes map Lo
whlch server doc ls on
! App never needs Lo know
! App reads, wrlLes, updaLes
docs
! Muluple App Servers can
access same documenL aL
same ume















uoc 4
uoc 2
uoc 3
SLkVLk 1
uoc 6
uoc 4
SLkVLk 2
uoc 7
uoc 1
SLkVLk 3
uoc 3
user Congured 8epllca CounL = 1
8ead/WrlLe/updaLe





CCuCP8ASL CLlLn1 Ll88A8?


8ead/WrlLe/updaLe
uoc 9
uoc 7
uoc 8 uoc 6
uoc 3
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uoc 9
uoc 3
uCC
uCC
uCC
uoc 1
uoc 8 uoc 2
8epllca uocs 8epllca uocs 8epllca uocs
Acuve uocs Acuve uocs Acuve uocs
CLuS1L8 MA


CLuS1L8 MA


A SLkVLk 1 A SLkVLk 2
CCUCn8ASL SLkVLk CLUS1Lk
23




























Add nodes
! 1wo servers added Lo
clusLer
! Cne-cllck operauon
! uocs auLomaucally
rebalanced across
clusLer
! Lven dlsLrlbuuon of
docs
! Mlnlmum doc
movemenL
! ClusLer map updaLed
! App daLabase calls now
dlsLrlbuLed over larger #
of servers
user Congured 8epllca CounL = 1
8ead/WrlLe/updaLe 8ead/WrlLe/updaLe
uoc 7
uoc 9
uoc 3
Acuve uocs
8epllca uocs
uoc 6





CCuCP8ASL CLlLn1 Ll88A8?


CLuS1L8 MA


A SLkVLk 1





CCuCP8ASL CLlLn1 Ll88A8?


CLuS1L8 MA


A SLkVLk 2










































uoc 4
uoc 2
uoc 3
SLkVLk 1
uoc 6
uoc 4
SLkVLk 2
uoc 7
uoc 1
SLkVLk 3
uoc 3
uoc 9
uoc 7
uoc 8 uoc 6
uoc 3
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uoc 9
uoc 3
uCC
uCC
uCC
uoc 1
uoc 8 uoc 2
8epllca uocs
8epllca uocs 8epllca uocs
Acuve uocs Acuve uocs Acuve uocs
SLkVLk 4 SLkVLk S
Acuve uocs Acuve uocs
8epllca uocs 8epllca uocs
CCUCn8ASL SLkVLk CLUS1Lk
26




























lall Cver node
! App servers happlly accesslng docs
on Server 3
! Server falls
! App server requesLs Lo server 3 fall
! ClusLer deLecLs server has falled
! romoLes $"4628)- of docs Lo )81*"
! <4/)&"- 86,-&"$ 3)4
! App server requesLs for docs now
go Lo approprlaLe server
! 1yplcally rebalance would follow

user Congured 8epllca CounL = 1
uoc 7
uoc 9
uoc 3
Acuve uocs
8epllca uocs
uoc 6





CCuCP8ASL CLlLn1 Ll88A8?


CLuS1L8 MA


A SLkVLk 1





CCuCP8ASL CLlLn1 Ll88A8?


CLuS1L8 MA


A SLkVLk 2










































uoc 4
uoc 2
uoc 3
SLkVLk 1
uoc 6
uoc 4
SLkVLk 2
uoc 7
uoc 1
SLkVLk 3
uoc 3
uoc 9
uoc 7 uoc 8
uoc 6
uoc 3
uCC
uCC
uCC
uCC
uCC
uCC
uCC uCC
uCC
uCC
uCC uCC
uCC
uCC
uCC
uoc 9
uoc 3 uCC
uCC
uCC
uoc 1
uoc 8
uoc 2
8epllca uocs
8epllca uocs 8epllca uocs
Acuve uocs Acuve uocs Acuve uocs
SLkVLk 4 SLkVLk S
Acuve uocs Acuve uocs
8epllca uocs 8epllca uocs
CCUCn8ASL SLkVLk CLUS1Lk
27
Couchbase wrlLe uaLa llow - appllcauon vlew
user acuon resulLs ln Lhe need Lo
change Lhe vALuL of kL?
Appllcauon updaLes keys vALuL,
performs SL1 operauon
Couchbase cllenL hashes kL?, ldenues
kL?s masLer server
SL1 requesL senL over
neLwork Lo masLer server
Couchbase repllcaLes kL?-vALuL palr, caches lL
ln memory and sLores lL Lo dlsk
1
2
3
4
3
28
Couchbase uaLa llow - under Lhe hood
SL1 requesL arrlves aL kL?s
masLer server
LlsLener-Sender
MasLer server for kL? 8epllca Server 2 for kL? 8epllca Server 1 for kL?
2 2
1
SL1 acknowledgemenL
reLurned Lo appllcauon
3
ulsk ulsk ulsk
8AM*
C
o
u
c
h
b
a
s
e

s
L
o
r
a
g
e

e
n
g
l
n
e

ulsk ulsk ulsk
2
4
29
roven aL Small, and LxLra Large Scale
Leading cloud service (PAAS)
provider
Over 150,000 hosted applications
Couchbase Server serving over
6,200 Heroku customers
Social game leader FarmVille,
Maa Wars, Empires and Allies,
Caf World, FishVille
Over 230 million monthly users
Couchbase Server is the
primary database behind key
Zynga properties
30
CusLomers and arLners
CusLomers (parual llsung) arLners
31
GL11ING S1Ak1LD
32
uevelopmenL LnvlronmenL: CbLalnlng a CllenL
Plgh performance, omclal cllenL llbrarles for !ava, .nL1,
P, 8uby, C
Pead Lo couchbase.com/develop for Suks where you
wlll nd
CllenL llbrarles
ScreencasLs on Cemng SLarLed
Cemng sLarLed guldes
1uLorlal wlLh a sample appllcauon
A compleLe Al reference
33
uevelopmenL LnvlronmenL: uevelopmenL u8
uownloads aL hup://couchbase.com/download_all

3
3
rovlslon vla wlzard ln Web Console
Cr provlslon vla 8LS1 lnLerface: operauons folks Lo auLomaLe provlslonlng
(l.e Chef, puppeL, 8lghLscale rlghLscrlpL)
Llnux: ubunLu and 8ed PaL/CenLCS
ackages for mosL common
dlsLrlbuuons.
dpkg -l , rpm -l, eLc.
Mac
uownload a .zlp, open, drag Lo
Appllcauons,
Wlndows
uownload a seLup.exe, double cllck
34
CllenL SeLup: Cemng ClusLer Congurauon
3
4
Couchbase
Server
node
Couchbase CllenL
hup:]]myserver:8091]poo|s
{

"bucketCapabilities": [
"touch",
"sync",
"couchapi"
],
"bucketCapabilitiesVer": "sync-1.0",
"bucketType": couchbase",
"name": "default",
"nodeLocator": "vbucket",
"nodes": [
.
ClusLer Congurauon
over 8LS1
Couchbase
Server
node
Couchbase
Server
node
Couchbase
Server
node
33
CllenL aL 8unume: Addlng a node
3
3
Couchbase CllenL
Couchbase
Server
node
Couchbase
Server
node
Couchbase
Server
node
Couchbase
Server
node
Couchbase
Server
node
Cluster
Topology
Update
=". +'/"
8'32+: '+62+"
36
CllenL SeL up aL a Code Level
// Set up at least two URIs in case one server fails
URI oneserver = new URI("http://10.1.6.171:8091/pools/");
URI twoserver = new URI("http://10.1.6.172:8091/pools/");
List<URI> servers = new ArrayList<URI>();
servers.add(oneserver);
servers.add(twoserver);

// Now create a client talking to the default bucket
CouchbaseClient cbc =
new CouchbaseClient(servers, "default", "");

System.err.println(cbc.get(Rags) +
" is off developing with Couchbase!");
<-" &(" >)*) 862"+& .2&( 9',$ 7)*'$2&" >?@= 625$)$9; >"A-'+B C'':6" C?@=B "&8D
37
AuLhenucauon
8uckeLs auLhenucaLed uslng SASL
CreaLe a ConnecuonlacLory, Lhen cllenL:
List<URI> baseURIs = new ArrayList<URI>();
baseURIs.add(base);
CouchbaseConnectionFactory cf = new
CouchbaseConnectionFactory(baseURIs,
"userbucket", "password");

client = new CouchbaseClient(cf);
38
Cperauons Avallable Lo a CllenL of Couchbase Server
SLore Cperauons
Add: SLore Lhe documenL lf lL does noL yeL exlsL
SeL: SLore Lhe documenL, overwrlung exlsung lf necessary
8eLrleve Cperauons
CeL: leLch Lhe documenL
CeL and Louch: leLch Lhe documenL and updaLe Lhe 11L
MulugeL: leLch muluple documenLs aL Lhe same ume
updaLe operauons
Append/prepend: Add daLa ln fronL of or on Lhe end of a documenL
ueleLe: 8emove Lhe documenL from Lhe sLore
Compare and Swap (CAS): 8eplace Lhe currenL documenL, lf CAS maLches
8eplace: 8eplace Lhe documenL lf lL exlsLs, oLherwlse do noL
1ouch: updaLe Lhe 11L for a documenL
39
Common cllenL leaLures
All operauons are aLomlc
All daLa operauons requlre key
no lmpllclL locklng (no row lock, eLc.)
ulerenL cllenLs lmplemenL core proLocol
Some cllenLs oer language/envlronmenL unlque
funcuonallLy
40
SLore Cperauons
Cperanon Descr|pnon
add() Adds new value lf key does noL exlsL or reLurns error.
replace() 8eplaces exlsung value lf specled key already exlsLs.
seL() SeLs new value for specled key.
All seLs allow for 'explry' ume specled ln seconds:
Lxp|ry Va|ue Descr|pnon
values < 30*24*60*60 1lme ln seconds Lo explry
values > 30*24*60*60 AbsoluLe ume from epoch for explry
41
8eLrleve Cperauons
Cperanon Descr|pnon
geL() CeL a value.
geLAnd1ouch() CeL a value and updaLe Lhe explry ume.
geL8ulk() CeL muluple values slmulLaneously, more emclenL.
geLs() CeL a value and Lhe CAS value.
42
updaLe Cperauons
Cperanon Descr|pnon
append() Appends daLa Lo an exlsung key.
prepend() repends daLa Lo an exlsung key.
lncr() lncremenLs an lnLeger value by specled amounL, defaulL 1.
decr() uecremenLs an lnLeger value by specled amounL, defaulL 1.
replace() 8eplaces an exlsung key/value palr.
Louch() updaLes Lhe explry ume for a glven lLem.
43
Asynchronous Cperauons
SeL Cperauon

// Do an asynchronous set
OperationFuture<Boolean> setOp = client.set(KEY, EXP_TIME, VALUE);

// Check to see if our set succeeded
try {
if (setOp.get().booleanValue()) {
System.out.println("Set Succeeded");
} else {
System.err.println("Set failed: " +
setOp.getStatus().getMessage());
}
} catch (Exception e) {
System.err.println("Exception while doing set: " +
e.getMessage());
}
44
MeLadaLa
Server sLores meLadaLa wlLh each key/value palr
Lxplry (11L)
CAS (checksum) value
llags
Lxplry ls used by server Lo deleLe values aer Lhe
specled ume
llags can be used Lo ldenufy Lhe daLa Lype of sLored
lnformauon
ln !ava, Lhls lnformauon ls used Lo serlallze Lhe daLa Lype of
Lhe ob[ecL sLored
43
11L
roperLy Lo seL explrauon on Lhe documenL
Lhe acLual value senL may elLher be
unlx ume (number of seconds slnce !anuary 1, 1970, as a 32-
blL value
C8 number of seconds sLarung from currenL ume. number of
seconds may noL exceed 60*60*24*30 (number of seconds ln
30 days)
lf Lhe number senL by a cllenL ls larger Lhan
LhaL, Lhe server wlll conslder lL Lo be real unlx ume value
raLher Lhan an oseL from currenL ume.
1lme 1o Llve (11L)
46
ulsLrlbuLed SysLem ueslgn: Concurrency ConLrols
Compare and Swap Cperauons
Cen referred Lo as CAS"
Cpumlsuc concurrency conLrol
Avallable wlLh many muLauon
operauons, dependlng on cllenL.
CeL wlLh Lock
Cen referred Lo as CL1L"
esslmlsuc concurrency conLrol
Locks have a shorL 11L
Locks released wlLh CAS
operauons
useful when worklng wlLh ob[ecL
graphs
4
6
AcLor 1 AcLor 2
Couchbase Server
CAS mlsmaLch Success
A
8
l
C u
L
47
Check and SeL/Compare and Swap (CAS)
uses checksum Lo valldaLe a change Lo a value:
CllenL geLs key and checksum (cas_Loken)
CllenL updaLes uslng key and checksum
lf checksum doesn'L maLch, updaLe falls
CllenL can only updaLe lf Lhe key + checksum maLch
used when muluple cllenLs access Lhe same daLa
llrsL cllenL updaLes wlLh checksum
SubsequenL cllenL updaLes fall wlLhouL Lhe rlghL
checksum
use CASMutation and CASMutator for hlgher
level of absLracuon
48
!SCn (!avaScrlpL Cb[ecL noLauon)
LlghLwelghL uaLa-lnLerchange formaL
Lasy for humans Lo read and manlpulaLe
Lasy for machlnes Lo parse and generaLe (many parsers are
avallable)
!SCn ls language lndependenL (alLhough lL uses slmllar
consLrucLs)

Why use !SCn?
49
!SCn (!avaScrlpL Cb[ecL noLauon) daLa Lype supporL
number (elLher lnLeger or oaung polnL)
SLrlng - supporung unlcode characLers and backslash escape
A SLrlng"
8oolean - Lrue or false
[ value": Lrue
Array - a llsL of values enclosed ln square brackeLs
one", Lwo", Lhree"
Cb[ecL - a seL of key/value palrs l.e. an assoclauve array or
hash. key musL be sLrlng, value can be any supporLed !SCn
values
[
servlngs" : 4, dlmculLy" : Lasy", cookume" : 60, uLle" : 8asam"

!SCn 8aslcs
30
uslng !SCn ln Couchbase
resldenLs.[son

[
{"presidency":"1",
"name":"George Washington",
"wikipedia_entry":"http://en.wikipedia.org/wiki/George_Washington",
"took_office":1789,
"left_office":1797,
"party":"Independent",
"portrait":"GeorgeWashington.jpg",
"thumbnail":"thmb_GeorgeWashington.jpg",
"home_state":"Virginia",
"type":"president"},
{"presidency":"2",
"name":"John Adams",
"wikipedia_entry":"http://en.wikipedia.org/wiki/John_Adams",
"took_office":1797,
"left_office":1801,
"party":"Federalist ",
"portrait":"JohnAdams.jpg",
"thumbnail":"thmb_JohnAdams.jpg",
"home_state":"Massachusetts",
"type":"president"},

]
31
uslng !SCn ln Couchbase
resldenL Class

class President
{
String presidency;
String name;
String wikipedia_entry;
String took_office;
String left_office;
String party;
String portrait;
String thumbnail;
String notable;
String home_state;
String type;
}
32
uslng !SCn ln Couchbase
?ou could use any !ava !SCn llbrarles (Lhls example uses Cson)
use Lhls class and Lhe !SCn le

Gson gson = new Gson();
President[] Presidents = gson.fromJson(new
FileReader("Presidents.json"),
President[].class);

for (President entry : Presidents) {
String JSONentry = gson.toJson(entry);
c.set(entry.presidency, 1200, JSONentry);
}
33
DLMC
34
3 Cb[ecLs (documenLs) wlLhln game:
layers
MonsLers
lLems

Cameplay:
|ayers E:(& monsters
Monsters /$'4 |tems
|ayers '.+ |tems
uemo: 1he nexL blg soclal game
33
[
_ld: kelLh4340,
_rev: 1-ab334009ce09f198c333b693e037adce,
[son1ype: player,
uuld: 33767d02-a938-4b83-8179-616816692de1,
name: kelLh4340,
hlLpolnLs: 73,
experlence: 663,
level: 4,
loggedln: false


layer uocumenL
36
[
_ld: kaLana_e3890c94-11c6-48-63746ce6c360,
_rev: 1-d6bbd3e814c32c66e22db2918a2efcd9,
[son1ype: lLem,
name: kaLana_e3890c94-11c6-63746ce6c360,
uuld: e3890c94-11c6-4836-a7a6-63746ce6c360,
ownerld: uale9887

lLem uocumenL
layer _ld"
37
MonsLer uocumenL
layer _ld"
[
_ld: 8auchan9932,
_rev: 1-3c90be38be38134a0fc3e7db77dab3f2,
[son1ype: monsLer,
name: 8auchan9932,
uuld: d10dfc1b-0412-4140-b4ec-adbf2aa3ec,
hlLpolnLs: 370,
experlenceWhenkllled: 32,

38
3
8
38
Lhanks for Lhe phoLo hup://lc.kr/p/88fCcP
G
A
M
E


O
N
!

39
CASL S1UD: 1kI8AL CkCSSING
39
60
1rlbal Crosslng: Challenges
Common sLeps on scallng up daLabase:
# 1une querles (lndexlng, explaln query)
# uenormallzauon
# Cache daLa (Memcache)
# 1une MySCL congurauon
# 8epllcauon (read slaves)

Where do we go from here Lo prepare for Lhe scale of a
successful soclal game?
61
1rlbal Crosslng: Challenges
# WrlLe-heavy requesLs
Cachlng does noL help
MySCL / lnnou8 llmlLauon (ercona)
# need Lo scale drasucally over nlghL
My olls - 100 Lo 1m users over a weekend
# Small Leam, no dedlcaLed sysadmln
locus on whaL we do besL - maklng games
# keeplng cosL down
62
1rlbal Crosslng: Cld ArchlLecLure and Cpuons
# MySCL wlLh masLer-Lo-masLer repllcauon and
shardlng
Complex Lo seLup, hlgh admlnlsLrauon cosL
8equlres appllcauon level changes
# Cassandra
Plgh wrlLe, buL low read LhroughpuL
Llve clusLer recongurauon and rebalance ls qulLe compllcaLed
LvenLual conslsLency glves Loo much burden Lo appllcauon
developers
# Mongou8
Plgh read/wrlLe, buL unpredlcLable laLency
Llve clusLer rebalance for exlsung nodes only
LvenLual conslsLency wlLh slave nodes
63
1rlbal Crosslng: Why Couchbase Server?
# SLLu, SLLu, SLLu
# lmmedlaLe conslsLency
# lnLerface ls dead slmple Lo use
We are already uslng Memcache
# Low sysadmln overhead
# Schema-less daLa sLore
# used and roven by blg guys llke Zynga
# and lasLly, because 1rlbal CAn
8lgger rms wlLh legacy code base = hard Lo adapL
Small Leam = ablllLy Lo geL on Lhe cumng edge
64
1rlbal Crosslng: new Challenges WlLh Couchbase
# 8uL, Lhere are some dlerences ln
uslng Couchbase (currenLly 1.8) Lo handle Lhe game
daLa:
no easy way Lo query daLa
no Lransacuon / rollback
! Couchbase Server 2.0 resolves Lhese by uslng
Couchu8 as Lhe underlylng daLabase englne
# Can Lhls work for an onllne game?
8reak ouL of Lhe old C8M / relauonal paradlgm
1rlbal: we are noL handllng crlucal Lransacuons
63
1rlbal Crosslng: ueploylng Couchbase ln LC2
Couchbase Cluster
Web Server
# 8aslc producuon
envlronmenL seLup
# uev/SLage envlronmenL -
feel free Lo lnsLall
Couchbase on your web
server
Apache
Couchbase Couchbase
DNS Entry
Client-side Moxi
Cluster Mgmt. Requests

66
1rlbal Crosslng: ueploylng Couchbase ln LC2
# Amazon Llnux AMl,
64-blL, L8S backed lnsLance
# SeLup swap space
# lnsLall Couchbases
Server 1.8
# Access web console
http://<hostname>:8091
# SLarL Lhe new clusLer wlLh a
slngle node
# Add Lhe oLher nodes Lo Lhe
clusLer and rebalance Couchbase Cluster
Web Server
Apache
Couchbase
DNS Entry
Client-side Moxi
Cluster Mgmt. Requests

Couchbase
67
use case - slmple farmlng game:

A player can have a varleLy of planLs on Lhelr farm.
A player can add or remove planLs from Lhelr farm.
A layer can see whaL planLs are on anoLher players
farm.
1rlbal Crosslng: 8epresenung Came uaLa ln Couchbase
68
1rlbal Crosslng: 8epresenung Came uaLa ln Couchbase
8epresenung Cb[ecLs
# Slmply LreaL an ob[ecL as a Map/dlcuonary/!SCn ob[ecL
# ueLermlne Lhe key for an ob[ecL uslng Lhe class name (or
Lype) of Lhe ob[ecL and an unlque lu
8epresenung Cb[ecL LlsLs
# uenormallzauon
# Save a comma separaLed llsL or an array of ob[ecL lus
69
1rlbal Crosslng: 8epresenung Came uaLa ln Couchbase
layer Cb[ecL
Key: 'Player1'

JSON
{
_id : Player1,
nid : 1,
name : Shawn
}
lanL Cb[ecL
Key: 'Plant201'

JSON
{
_id : Plant201,
nid : 201,
player_id : 1
name : Starflower
}
layerlanL LlsL
Key: 'Player1_PlantList'
JSON
{
_id : Player1_Plantlist,
plants : [201,
202,
204]
}


70
1rlbal Crosslng: Schema-less Came uaLa
# no need Lo AL1L8 1A8LL
# Add new elds all ob[ecLs aL any ume
Speclfy defaulL value for mlsslng elds
lncreased developmenL speed
# uslng !SCn for daLa ob[ecLs Lhough,
# Cers Lhe ablllLy Lo query and analyze arblLrary elds ln
Couchbase 2.0
71
1rlbal Crosslng: Accesslng Came uaLa ln Couchbase
CeL all planLs belong Lo a glven player
Request: GET /player/1/farm

// Create a new PlantList from players
PlantList playersPlants =
new PlantList(cbclient.get("Player1_PlantList");

for (Plant plant : playersPlants) {
aPlayer.addPlant(plant);

72
1rlbal Crosslng: Modlfylng Came uaLa ln Couchbase
Clve a player a new planL
// Create the new plant
Plant givenPlant = new Plant(100, "Mushroom");

cbclient.set("Plant100", givenPlant);

// Update the player plant list
Player thePlayer = player.fetch(cbclient.get("Player1");

// Add the plant to the player
thePlayer.receivePlant(givenPlant);

// Store the player's new plant list
cbclient.set("Player1_PlantList",
thePlayer.getPlantsArray());
73
1rlbal Crosslng: Concurrency
Concurrency lssue can occur when
muluple requesLs are worklng wlLh Lhe
same plece of daLa.
Soluuons:
# CAS (compare-and-swap)
CllenL can know lf someone else
has modled Lhe daLa whlle you
are Lrylng Lo updaLe
rovldes opumlsuc concurrency
conLrol
# CL1L (geL wlLh lock)
Locking (try/wait cycle)
rovldes pesslmlsuc concurrency
conLrol
74
1rlbal Crosslng: uaLa 8elauonshlp
8ecord ob[ecL relauonshlps boLh ways
- Lxample: loLs and lanLs
loL ob[ecL sLores ld of Lhe planL LhaL lL hosLs
lanL ob[ecL sLores ld of Lhe ploL LhaL lL grows on
- 8esoluuon ln case of mlsmaLch
uonL sweaL Lhe exLra calls Lo load daLa ln a one-Lo-
many relauonshlp
- use muluCeL
- 1rlbal: We can sull cache aggregaLed resulLs ln a
Couchbase buckeL lf needed
73
1rlbal Crosslng: Mlgraung Lo Couchbase Servers
llrsL mlgraLed large or slow performlng Lables and
frequenLly updaLed elds from MySCL Lo Couchbase
Web Server
memcached
proLocol llsLener/sender
englne lnLerface
Couchbase SLorage Lnglne
1A
1A CllenL
Apache + P
CllenL-slde Moxl
8eporung
Appllcauons
MySCL
76
1rlbal Crosslng: ueploymenL
77
1rlbal Crosslng: ueploymenL
78
SlgnlcanLly reduced Lhe cosL lncurred by scallng up
daLabase servers and managlng Lhem.
Achleved slgnlcanL lmprovemenLs ln varlous
performance meLrlcs (e.g., read, wrlLe, laLency, eLc.)
Allowed focus more on game developmenL and
opumlzlng key meLrlcs
lan Lo use real-ume Map-8educe, querylng, and
lndexlng ablllues provlded by Couchbase Server 2.0
1rlbal Crosslng: Concluslon
79
SLCCNDAk INDLkING AND VILWS
79
80
Couchbase Server 2.0: vlews
vlews can cover a few dlerenL use cases
Slmple secondary lndexes (Lhe mosL common)
Aggregauon funcuons
Lxample: counL Lhe number of norLh Amerlcan Ales
Crganlzlng relaLed daLa
81
WhaL are vlews?
LxLracL elds from !SCn documenLs and produce an
lndex of Lhe selecLed lnformauon
82
Secondary lndexlng and vlews
UserId Message 1ype date
<user-1> Pello London Message Mon Mar-19 09:02:02
<user-2> Pello 8erlln Message Mon Mar-19 10:03:03
<user-3> Pello 8ome Message Mon Mar-19 10:03:06
<user-3> Pello Zurlch Message Mon Mar-19 10:06:08
<user-3> Pello arls Message Mon Mar-19 10:06:09
<user-4> Pello AmsLerdam Message Mon Mar-19 10:07:03
<user-3> Pello Madrld Message Mon Mar-19 10:11:01
All !SCn documenLs of 1ype
Message slnce Mon Mar-19
10:00:00
83
uenlng vlews
84
vlew ueLalls
map funcuon creaLes a mapplng beLween lnpuL daLa
and ouLpuL
reduce funcuon provldes a summary (such as counL,
sum, eLc.)
83
Crlglnal !SCn documenL
{ email : [email protected], name : Rags Srinivas }
vlew
function(doc) {
emit([doc.name, doc.email], null);
}
new !SCn documenL
{ email : [email protected], first : Rags, last :
Srinivas }
new vlew
function(doc) {
if (doc.name != null) {
emit([doc.name, doc.email], null);
} else {
emit([doc.last + , + doc.first, doc.email], null);
}


vlews - adapung Lo changlng schemas
86
8uckeLs conLaln ueslgn uocumenLs LhaL conLaln vlews
vlews
87
vlews
All Lhe vlews wlLhln a deslgn documenL ls lncremenLally
updaLed when Lhe vlew ls accessed (lazlly by defaulL)
1he enure vlew ls recreaLed lf Lhe vlew denluon has changed
vlews can be condluonally updaLed by speclfylng Lhe sLale
argumenL Lo Lhe vlew query
1he lndex lnformauon sLored on dlsk conslsLs of Lhe
comblnauon of Lhe comblnauon of boLh Lhe key and value
lnformauon dened wlLhln your vlew.
uurlng a vlew query, Lhe lndex lnformauon for Lhe glven vlew
query on )66 Lhe v8uckeLs wlLhln Lhe clusLer ls collaLed and
reLurned Lo Lhe cllenL.
vlews - ueLalls
88
Worklng wlLh Cuerles and vlews
Date today = new Date(System.currentTimeMillis());
System.out.println("Date: " + today);
today.setHours(10);
today.setMinutes(0);

Query query = new Query();


query.setRange(String.valueOf(today.getTime()),
String.valueOf(currentTime));
query.setReduce(false);
query.setIncludeDocs(true);
query.setStale(Stale.FALSE);

View view = client.getView("chatrags", "messages");
89
lndexlng and Cuerylng










































A SL8vL8 1





CCuCP8ASL CLlLn1 Ll88A8?


! lndexlng work ls dlsLrlbuLed
amongsL nodes
! Large daLa seL posslble
! arallellze Lhe eorL
! Lach node has lndex for daLa
sLored on lL
! Cuerles comblne Lhe resulLs
from requlred nodes

CLuS1L8 MA
uoc 4
uoc 2
uoc 3
SLkVLk 1
uoc 6
uoc 4
SLkVLk 2
uoc 7
uoc 1
SLkVLk 3
uoc 3
A SL8vL8 2





CCuCP8ASL CLlLn1 Ll88A8?


CLuS1L8 MA
uoc 9
uoc 7
uoc 8 uoc 6
uoc 3
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uCC
uoc 9
uoc 3
uCC
uCC
uCC
uoc 1
uoc 8 uoc 2
8epllca uocs 8epllca uocs 8epllca uocs
Acuve uocs Acuve uocs Acuve uocs





CCuCP8ASL CLlLn1 Ll88A8?







CCuCP8ASL CLlLn1 Ll88A8?


CLuS1L8 MA


CLuS1L8 MA


A SLkVLk 1 A SLkVLk 2
Cuery 8esponse
90
kCADMA
91
Couchbase Server 2.0
nexL ma[or release of Couchbase Server
CurrenLly ln ueveloper revlew
WhaL's new:
lndexlng and Cuerylng
lncremenLal Map 8educe
Cross uaLa CenLer 8epllcauon
lully backwards compauble wlLh exlsung
Couchbase Server
92
Couchbase Server 2.0 ArchlLecLure
P
e
a
r
L
b
e
a
L

r
o
c
e
s
s

m
o
n
l
L
o
r

C
l
o
b
a
l

s
l
n
g
l
e
L
o
n

s
u
p
e
r
v
l
s
o
r

C
o
n

g
u
r
a
u
o
n

m
a
n
a
g
e
r

on each node
8
e
b
a
l
a
n
c
e

o
r
c
h
e
s
L
r
a
L
o
r

n
o
d
e

h
e
a
l
L
h

m
o
n
l
L
o
r

one per clusLer
v
8
u
c
k
e
L

s
L
a
L
e

a
n
d

r
e
p
l
l
c
a
u
o
n

m
a
n
a
g
e
r

hup
k
L
S
1

m
a
n
a
g
e
m
e
n
t

A

I
]
W
e
b

U
I

P11
8091
Lrlang porL mapper
4369
ulsLrlbuLed Lrlang
21100 - 21199
Lr|ang]C1
sLorage lnLerface
Couchbase L Lng|ne
11210
Memcapable 2.0
Mox|
11211
Memcapable 1.0
Memcached
New ers|stence Layer
8092
Cuery Al

u
e
r
y

L
n
g
|
n
e

93
kLSCUkCLS AND SUMMAk
94
Summary: noSCL ros / Cons
I|ve advantages of NoSL
Llasuc scallng
8lg daLa
Cpumlzed for dlsLrlbuLed neLwork
Lconomlcs
llexlble daLa models
I|ve cha||enges of NoSL
MaLurlLy / SLandard Al
SupporL
Analyucs and buslness lnLelllgence
AdmlnlsLrauon
Lxperuse
lrom 10 Lhlngs you should know abouL noSCL daLabases
93
Couchbase Server uownloads
hup://www.couchbase.com/downloads-all
hup://www.couchbase.com/couchbase-server/overvlew
ueveloplng wlLh !ava CllenL llbrarles
hup://www.couchbase.com/develop/[ava/currenL
Couchbase forums
hup://www.couchbase.com/forums/
Sqoop-based Padoop ConnecLor for Couchbase
hup://www.couchbase.com/develop/connecLors/hadoop
Couch Conference (comlng Lo a clLy near you)
hup://www.couchbase.com/couchconf-world-Lour
1rlbal Crosslng blog
hup://www.fablelabs.com/press/real-world-nosql-membase-aL-Lrlbal-crosslng.hLml


8esources, Summary and Call lor Acuon
96
1nANkS - &A
96
Raghavan Rags Srinivas, Couchbase Inc.
([email protected], @ragss)