100% found this document useful (1 vote)
941 views33 pages

Horizontal Scaling With HiveDB Presentation

This document discusses scaling a product catalog database at Cafepress.com using HiveDB. Key points: - Cafepress had hundreds of millions of products with millions added weekly, requiring a database that could scale with accelerating growth. - HiveDB uses key-based partitioning to distribute data across nodes without broadcasting or repartitioning. This allows easy addition of capacity and relocation of records. - Testing showed HiveDB met Cafepress' performance goals, with read throughput of 2250/s (goal 1500/s) and response time of 8ms (goal 100ms). - HiveDB provided the highest database uptime at Cafepress, handling billions of

Uploaded by

yejr
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (1 vote)
941 views33 pages

Horizontal Scaling With HiveDB Presentation

This document discusses scaling a product catalog database at Cafepress.com using HiveDB. Key points: - Cafepress had hundreds of millions of products with millions added weekly, requiring a database that could scale with accelerating growth. - HiveDB uses key-based partitioning to distribute data across nodes without broadcasting or repartitioning. This allows easy addition of capacity and relocation of records. - Testing showed HiveDB met Cafepress' performance goals, with read throughput of 2250/s (goal 1500/s) and response time of 8ms (goal 100ms). - HiveDB provided the highest database uptime at Cafepress, handling billions of

Uploaded by

yejr
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 33

Scaling with HiveDB

Project Genesis
• Cafepress.com Product Catalog
• Hundreds of Millions of Products
• Millions
week
of new products every

• Accelerating growth
Enter Jeremy and
HiveDB
Our Requirements
• OLTP Optimized
• Constant response time is more
important than low latency
• Related sets vary wildly in size
• Growth hotspots
• Usage hotspots
Partition by key
Directory
• No broadcasting
• No re-partitioning
• Easy to relocate records
• Easy to add capacity
Disadvantages
• Intelligence moved from the
database to the application
(Queries have to be planned and
indexed)
• Can’t join across partitions
• NO OLAP! (We consider this a
benefit.)
• Directory is a bottleneck
Original Design
• Smallest possible implementation
• HiveDB was just a JDBC Gatekeeper
for applications.
Development
Complexity Problem
• You have to maintain
synchronization between the
directory and the data nodes.
• Lots of code for simple operations
• Data access objects have to be re-
implemented
Enter Hibernate
Shards
• Partitioned Hibernate from Google
• Why did we write this thing
again?
• Oh wait, you have to tell it how
to look things up...we’re good at
that.
Benefits of Shards
• Unified data access layer
• Result set aggregation across
partitions
• Everyone in the JAVA-world knows
Hibernate.
Show don’t tell!
Competitive Landscape

• Clustered Relational Databases


• Non-relational
Databases
Structured

• Non-relational,
Storage
Unstructured
Competitive Landscape
Clustered Non-relational Unstructured

Oracle RAC
MS SQL Server Hypertable
Hadoop
MySQL Cluster HBase
MogileFS
DB2 CouchDB
S3
Teradata (OLAP) SimpleDB
HiveDB
Competitive Landscape
Storage Interface Partitioning Expansion Node Types Maturity

Oracle RAC Shared SQL Transparent No downtime Identical 7 years

Memory / Requires
MySQL Cluster SQL Transparent Mixed 3 years
Local Restart

?
Hypertable Local HQL Transparent No downtime Mixed (released
2/08)
3 years
Degraded
DB2 Local SQL Fixed Hash Identical (25 years
Performance
total)

Key-based 18 months
HiveDB Local SQL No downtime Mixed
Directory (+13 years!)
Case Study:
CafePress
• Leader in User-generated Commerce
• Same number of products as eBay
(>150,000,000)
• 24/7/365
Case Study:
Performance Requirements
• Thousands of queries/sec
• 10 : 1 read/write ratio
• Geographically distributed
Case Study:
Test Environment
• Real schema
• Production-class hardware
• Partial data (~40M records)
CafePress HiveDB 2007
Performance Test Environment

JMeter (1 thread) JMeter (no threads)


command &

client.jar client.jar
control

Measurement Test Controller


Workstation Workstation
100MBit switch
load generators

JMeter (100s of threads) JMeter (100s of threads)


client.jar client.jar

48GB backplane non-


Dell 2950 / 2x2 Xeon blocking gigabit switch Dell 2950 / 2x2 Xeon
16GB, 6x72GB 15k 16GB, 6x72GB 15k
web service

Hardware LB
(hivedb)

Dell 1950 / 2x2 Xeon Dell 1950 / 2x2 Xeon Dell 1950 / 2x2 Xeon
Tomcat 5.5 Tomcat 5.5 Tomcat 5.5

Directory Partition 0 Partition 1


databases
(mysql)

Dell 2950 / 2x2 Xeon Dell 2950 / 2x2 Xeon Dell 2950 / 2x2 Xeon
16GB, 6x72GB 15k 16GB, 6x72GB 15k 16GB, 6x72GB 15k

[email protected] Modified on April 09 2007


Case Study:
Performance Goals
• Large object reads: 1500/s
• Large object writes: 200/s
• Response time: 100ms
Case Study:
Performance Results
• Large object reads: 1500/s
Actual result: 2250/s
• Large object writes: 200/s
Actual result: 300/s
• Response time: 100ms
Actual result: 8ms
Case Study:
Performance Results
• Max read throughput
Actual result: 4100/s
(CPU limited in Java layer;
MySQL <25% utilized)
Case Study:
Organizational Results
• Billions of queries served
• Highest DB uptime at CafePress
• Hundreds
performed
of millions of updates
High Availability &
Replication
• We don’t specify a fail over
strategy
• We delegate to MySQL replication
Non-JAVA Deployment
Options
• Web service
• JVM Dynamic Languages
HiveDB Accessories
Class Generation
• Automatically generate Data
Transfer Objects from interfaces
(and soon web services).
Blobject
• Gets around the problem of ALTER
statements
• Compression
• The hive can contain multiple
versions of a serialized record.
• No data set of this size can be
transformed synchronously.
Features Teaser
• We’re taking over HA...you’re still
on your own for replication.
• Generated Web Services
• Monitoring
graphs!)
& RRD stats (with

• Query/transform tool
• Record migration & balancing tools
Contributing
• Post to the mailing list
http://groups.google.com/group/hivedb-dev

• Comment on our site


http://www.hivedb.org

• File a bug
http://hivedb.lighthouseapp.com

• Submit a patch / pull request


git clone git://github.com/britt/hivedb.git
Photo Credits
• http://www.flickr.com/photos/7362313@N07/1240245941/sizes/o

• http://www.flickr.com/photos/99287245@N00/2229322675/sizes/o

You might also like