An introduction to Apache

Traffic Server

Leif  Hedstrom  
December  2011  
 Who am I?
•  One of the drivers to open source Traffic Server
(at Yahoo)
•  Committer for Apache Traffic Server
•  VP of Apache Traffic Server PMC
•  ASF member
•  Go Daddy’s Principal Architect for Hosting

[email protected]
@zwoop
+zwoop
 History of Traffic Server

Inktomi Yahoo Apache

Traffic Server Traffic Server Traffic Server

1995 2000 2005 2010

Plenty of FOSS Proxy Servers
 Features
ATS HAproxy nginx Squid Varnish mod_proxy
Worker Threads Y N N N Y Y
Multi-Process N Y Y N Y Y
Event-driven Y Y Y Y sometimes Y
Plugin APIs Y N Y part Y Y
Forward Proxy Y N N Y N Y
Reverse Proxy Y Y Y Y Y Y
Transp. Proxy Y Y N Y N N
Load Balancer weak Y Y Y Y Y
Persistent Cache Y N Y Y sorta Y
ESI Y N N Y kinda N
ICP soon N N Y N N
Keep-Alive Y N Y Y Y Y
SSL Y N Y Y N Y
Pipeline Y N Y Y N Y
Mandatory useless benchmark …

120,000"

100,000"

80,000"
Throughput)

60,000"

40,000"

20,000"

0"
ATS"2.1.9" Nginx"0.8.53" Varnish"2.1.5"

Req"/"sec"
Less useless benchmark …
120,000# 4.0#
3.5#
100,000#

Time%to%first%response%
3.0#
80,000#
Throughput%

2.5#
60,000# 2.0#
1.5#
40,000#
1.0#
20,000#
0.5#
0# 0.0#
ATS#2.1.9# Nginx#0.8.53# Varnish#2.1.5#

Req#/#sec# Latency#(ms)#
Intermediaries - Forward Proxy

The browser (UA) is

explicitly configured to use
the FP server as a proxy

Web site A
GET http://c.com/ HTTP/1.1

Forward Web site B

Proxy
Browser

GET / HTTP/1.1
Host: c.com
Web site C
Intermediaries - Reverse Proxy

The reverse proxy has explicit

rules for a.com, b.com and
c.com. Other requests are
(typically) denied

Web site A
GET / HTTP/1.1
Host: c.com

Reverse Web site B

Proxy
Browser

GET / HTTP/1.1
Host: c.com
Web site C
Intermediaries - Intercepting Proxy

There are no explicit configurations

in browser of proxy. The transparent
proxy intercepts HTTP requests, and
proxies them to the destination.

GET / HTTP/1.1
Web site A
Host: c.com

Intercepting Web site B

Browser
Router Proxy
or
Firewall
GET / HTTP/1.1
Host: c.com
Web site C
Web Cash?

h4p://www.flickr.com/photos/amagill/3366720659/  
No, Web Cache!

h4p://www.flickr.com/photos/dglazkov/287092560/  
 Common enemies of performance

3-Way handshake

Congestion control

DNS lookups
 Common enemies of performance

3-Way handshake

Congestion control

DNS lookups
TCP 3-way Handshake

SY
N

C K
N –A
SY

AC
K

Client

Established Server
 Common enemies of performance

3-Way handshake

Congestion control

DNS lookups
Congestion control
#2 #1 #0

ACK 0-2

#8 ... #3

ACK 3-8

#21 ... #9
Client
ACK 9-21 Server
 Common enemies of performance

3-Way handshake

Congestion control

DNS lookups
http://news.example.com/
http://finance.example.com/
http://groups.example.com/
http://news.example.com/
http://finance.example.com/
http://groups.example.com/
vs
http://www.example.com/news
http://www.example.com/finance
http://www.example.com/groups
The concurrency problem
Solution 1: Multithreading

Single CPU Dual CPU

Thread 1 Thread 1 Thread 2

Thread 3
Thread 2
Thread 1

Thread 3
Thread 3
Thread 1

Thread 3

Time Time
Problems with multi-threading

"When two trains approach each other at a

crossing, both shall come to a full stop
and neither shall start up again until
the other has gone."

From Wikipedia, Illogical statute passed by Kansas legislation.

Solution 2: Event Processing
Scheduled Network Disk I/O
events events events

Queue

Event
Loop

Disk HTTP state Accept

handler machine handler

Can generate new events

Problems with event processors
 Asynchronous event processor

Asynchronous event processor

RAM cache
...
n threads per core

Asynchronous event processor

Disk cache
Disk I/O thread

Disk I/O thread

...
m threads per disk

Disk I/O thread

Shared Resources

Configs
Reloadable

Accept threads (per port)

Logging threads
Traffic Server threads model

etc.
Stats, logs
~10 threads

Various "admin" threads

http://www.flickr.com/photos/stuartpilbrow/3345896050/

plugin.config ip_allow.config
records.config storage.config
partition.config logs_xml.config
hosting.config parent.config
cache.config remap.config
update.config icp.config
 records.config for reverse proxy
CONFIG proxy.config.http.server_port INT 80
CONFIG proxy.config.cache.ram_cache.size INT 1G
CONFIG proxy.config.cache.ram_cache_cutoff INT 1M
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 0
CONFIG proxy.config.http.negative_caching_enabled INT 1
CONFIG proxy.config.http.negative_caching_lifetime INT 120
CONFIG proxy.config.http.cache.ignore_client_cc_max_age INT 1
CONFIG proxy.config.http.normalize_ae_gzip INT 1
 records.config for reverse proxy
CONFIG proxy.config.http.server_port INT 80
CONFIG proxy.config.cache.ram_cache.size INT 1G
CONFIG proxy.config.cache.ram_cache_cutoff INT 1M
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 0
CONFIG proxy.config.http.negative_caching_enabled INT 1
CONFIG proxy.config.http.negative_caching_lifetime INT 120
CONFIG proxy.config.http.cache.ignore_client_cc_max_age INT 1
CONFIG proxy.config.http.normalize_ae_gzip INT 1
 records.config for forward proxy
CONFIG proxy.config.http.server_port INT 8080
CONFIG proxy.config.cache.ram_cache.size INT 1G
CONFIG proxy.config.cache.ram_cache_cutoff INT 1M
CONFIG proxy.config.reverse_proxy.enabled INT 0
CONFIG proxy.config.url_remap.remap_required INT 0
CONFIG proxy.config.url_remap.pristine_host_hdr INT 0
CONFIG proxy.config.http.negative_caching_enabled INT 1
CONFIG proxy.config.http.negative_caching_lifetime INT 120
CONFIG proxy.config.http.cache.ignore_client_cc_max_age INT 1
CONFIG proxy.config.http.normalize_ae_gzip INT 1
 remap.config examples
map http://www.example.com/css http://css.example.com
map http://www.example.com http://real.example.com
reverse_map http://real.example.com http://www.example.com

redirect http://example.com http://www.example.com

regex_map http://(.*)\.example.com http://o.example.com/$1

map / http://kitchensink.example.com
 remap.config examples
map http://www.example.com/css http://css.example.com
map http://www.example.com http://real.example.com
reverse_map http://real.example.com http://www.example.com

redirect http://example.com http://www.example.com

regex_map http://(.*)\.example.com http://o.example.com/$1

map / http://kitchensink.example.com
 remap.config examples
map http://www.example.com/css http://css.example.com
map http://www.example.com http://real.example.com
reverse_map http://real.example.com http://www.example.com

redirect http://example.com http://www.example.com

regex_map http://(.*)\.example.com http://o.example.com/$1

map / http://kitchensink.example.com
 remap.config examples
map http://www.example.com/css http://css.example.com
map http://www.example.com http://real.example.com
reverse_map http://r.example.com http://www.example.com

redirect http://example.com http://www.example.com

regex_map http://(.*)\.example.com http://o.example.com/$1

map / http://kitchensink.example.com
 remap.config examples
map http://www.example.com/css http://css.example.com
map http://www.example.com http://real.example.com
reverse_map http://r.example.com http://www.example.com

redirect http://example.com http://www.example.com

regex_map http://(.*)\.example.com http://o.example.com/$1

map / http://kitchensink.example.com
 Some interesting features
§ RAM: Optional compression
– Various compression levels supported
– Very fast (virtually no overhead using the faster algo)
§ RAM: Several cache eviction algorithms
– CLFUS algorithm avoids the classic problem on an LRU
during “scans”
§ Disk: Supports various block sizes (SSD)
§ Disk: Can be tuned for various object sizes
§ Disk: No file system overhead (RAW disks)
 Future directions
§ Release early, and often
§ Full IPv6 support
§ NPN, SNI and other SSL improvements
§ Improved memory management, support debug
features in e.g. jemalloc and tcmalloc
§ Optimizations for SSD (“stacked” caches)
§ More focus on plugins, and plugin APIs
– Hopefully a real load balancer plugin!
Was this a huge waste of time, or?

h4p://www.flickr.com/photos/simpologist/16734948/  
A versatile tool

h4p://www.flickr.com/photos/airosan/2232394342/  
41
Yes, it’s fast, ridiculously fast

42
A vibrant community!

The
Apache
Software Foundation
http://www.apache.org/

43
Artur Bergman tells us if we’re
not using SSD drives, we’re
wasting our lives.
Artur Bergman tells us if we’re
not using SSD drives, we’re
wasting our lives.

I’d say, if you are not using a

caching proxy, you’re wasting
your SSDs!
46
Contact me

47
 records.config for reverse proxy
CONFIG proxy.config.http.server_port INT 80
CONFIG proxy.config.cache.ram_cache.size INT 1G
CONFIG proxy.config.cache.ram_cache_cutoff INT 1M
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 0
CONFIG proxy.config.http.negative_caching_enabled INT 1
CONFIG proxy.config.http.negative_caching_lifetime INT 120
CONFIG proxy.config.http.cache.ignore_client_cc_max_age INT 1
CONFIG proxy.config.http.normalize_ae_gzip INT 1
 Four Horseman of Bad Performance
– Data copies
– Context Switches
– Memory allocation
– Lock contention

– Avoid these!

http://pl.atyp.us/content/tech/servers.html
Disk cache: “Cyclone”