Scribd
Upload
3K views19 pages

Scribd Architecture Overview

These are slides from my first X1 Tech Talk about Scribd architecture overview. Blog post about this event: http://kpumuk.info/presentations/x1-tech-talks-1-scribd-architecture-overview/ Video from this talk is available here (in Russian): Part 1: http://www.vimeo.com/4087769 Part 2: http://www.vimeo.com/4088251

Uploaded by

Dmytro Shteflyuk
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3K views19 pages

Scribd Architecture Overview

These are slides from my first X1 Tech Talk about Scribd architecture overview. Blog post about this event: http://kpumuk.info/presentations/x1-tech-talks-1-scribd-architecture-overview/ Video from this talk is available here (in Russian): Part 1: http://www.vimeo.com/4087769 Part 2: http://www.vimeo.com/4088251

Uploaded by

Dmytro Shteflyuk
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 19

Scribd.

com
Architecture
Overview
X1 Tech Talks
Dmytro Shteflyuk
04/09/2009
Who Am I?
Ruby developer at Scribd.com
Lazy blogger at kpumuk.info
Experienced in Ruby & Rails, ASP.NET,
MySQL, Sphinx, etc.
Author of several projects
Sphinx Ruby API maintainer
What Is Scribd.com
Social document sharing
The largest Rails site over the Net
65th place on Quantcast (before Digg)
53.5M visitors, 178M page views
10.5M users, 14M document, over 1PB
15 app, 17 db, 7 search, 3 web, 4 proxy
boxes
Online Viewer
Groups
Partners
Desktop Uploader
The Big Picture
Nginx
Delivers static content
Handles file uploads
Selects app cluster (main, api, etc)
Forwards doc page requests to Squid
Forwards all requests to HAProxy
HAProxy
Performs load balancing among application
servers
That’s all - as easy as pie :-)
Squid
Caches all document pages for bots and
anonymous users
Forwards requests to HAProxy
Allows gracefully clear whole cache
Clears cached pages by request (HTCP)
Handles 90% of Scribd traffic!
MySQL
All writes to master
Almost all reads from slaves.
Texts are in separate DB (sharded)
All tables are in InnoDB
Mysql 5.0 / 5.1 with Percona patches
Application Boxes

Apache + Passenger (Ruby on Rails)


Memcached
Monit (we are looking for another
monitoring tool)
Sphinx Search Boxes
Used for browsing, private documents
search, extended site search, API search
We are using Google Custom Search Engine,
but users are able to switch to internal
search engine
Index consists of many small date-based
chunks for fast indexing
Amazon Services

Amazon S3 for images and documents


Each document in several formats
Background task for doc backups
Amazon EC2 for documents converting
What Are You Talking
About?
Ask Your Questions
It was just an introduction
More information is coming
Contacts
Email: [email protected]
Blog: hp://kpumuk.info/
Twier: hp://twier.com/kpumuk
Github: hp://github.com/kpumuk
Scribd: hp://www.scribd.com/kpumuk
We Are Hiring
Advanced knowledge of Ruby on Rails
Java, .NET, Python Gurus are welcome
Experience with heavily loaded apps
Deep understanding of MySQL and RDBMS
Ability to use Google and... Google
etc.

You might also like