Take a Close Look at

MapReduce
Xuanhua Shi
Acknowledgement
Most of the slides are from Dr. Bing Chen,
http://grid.hust.edu.cn/chengbin/
Some slides are from SHADI IBRAHIM,
http://grid.hust.edu.cn/shadi/

What is MapReduce
Origin from Google, [OSDI04]
A simple programming model
Functional model
For large-scale data processing
Exploits large set of commodity computers
Executes process in distributed manner
Offers high availability

Motivation
Lots of demands for very large scale data
processing
A certain common themes for these
demands
Lots of machines needed (scaling)
Two basic operations on the input
Map
Reduce

Distributed Grep
Very
big
data
Split data
Split data
Split data
Split data
grep
grep
grep
grep
matches
matches
matches
matches
cat
All
matches
Distributed Word Count
Very
big
data
Split data
Split data
Split data
Split data
count
count
count
count
count
count
count
count
merge
merged
count
Map+Reduce





Map:
Accepts input
key/value pair
Emits intermediate
key/value pair






Reduce :
Accepts intermediate
key/value* pair
Emits output key/value
pair
Very
big
data
Result
M
A
P
R
E
D
U
C
E
Partitioning
Function
The design and how it works
Architecture overview
Job tracker
Task tracker Task tracker Task tracker
Master node
Slave node 1
Slave node 2 Slave node N
Workers
user
Workers Workers
GFS: underlying storage system
Goal
global view
make huge files available in the face of node failures
Master Node (meta server)
Centralized, index all chunks on data servers
Chunk server (data server)
File is split into contiguous chunks, typically 16-64MB.
Each chunk replicated (usually 2x or 3x).
Try to keep replicas in different racks.

GFS architecture
GFS Master
C
0
C
1
C
2
C
5
Chunkserver 1
C
0
C
5
Chunkserver N
C
1
C
3
C
5
Chunkserver 2

C
2
Client
Functions in the Model
Map
Process a key/value pair to generate
intermediate key/value pairs
Reduce
Merge all intermediate values associated with
the same key
Partition
By default : hash(key) mod R
Well balanced
Diagram (1)
Diagram (2)
A Simple Example
Counting words in a large set of documents

map(string value)
//key: document name
//value: document contents
for each word w in value
EmitIntermediate(w, 1);

reduce(string key, iterator values)
//key: word
//values: list of counts
int results = 0;
for each v in values
result += ParseInt(v);
Emit(AsString(result));


How does it work?
Locality issue
Master scheduling policy
Asks GFS for locations of replicas of input file blocks
Map tasks typically split into 64MB (== GFS block
size)
Map tasks scheduled so GFS input block replica are
on same machine or same rack
Effect
Thousands of machines read input at local disk speed
Without this, rack switches limit read rate
Fault Tolerance
Reactive way
Worker failure
Heartbeat, Workers are periodically pinged by master
NO response = failed worker
If the processor of a worker fails, the tasks of that worker are
reassigned to another worker.

Master failure
Master writes periodic checkpoints
Another master can be started from the last checkpointed
state
If eventually the master dies, the job will be aborted
Fault Tolerance
Proactive way (Redundant Execution)
The problem of stragglers (slow workers)
Other jobs consuming resources on machine
Bad disks with soft errors transfer data very slowly
Weird things: processor caches disabled (!!)

When computation almost done, reschedule
in-progress tasks
Whenever either the primary or the backup
executions finishes, mark it as completed

Fault Tolerance
Input error: bad records
Map/Reduce functions sometimes fail for particular
inputs
Best solution is to debug & fix, but not always
possible
On segment fault
Send UDP packet to master from signal handler
Include sequence number of record being processed
Skip bad records
If master sees two failures for same record, next worker is
told to skip the record


Status monitor
Refinements
Task Granularity
Minimizes time for fault recovery
load balancing
Local execution for debugging/testing
Compression of intermediate data



Points need to be emphasized
No reduce can begin until map is complete
Master must communicate locations of
intermediate files
Tasks scheduled based on location of data
If map worker fails any time before reduce
finishes, task must be completely rerun
MapReduce library does most of the hard
work for us!

Model is Widely Applicable
MapReduce Programs In Google Source Tree
distributed grep distributed sort web link-graph reversal
term-vector / host web access log stats inverted index construction
document clustering machine learning statistical machine translation
... ... ...
Examples as follows
How to use it
User to do list:
indicate:
Input/output files
M: number of map tasks
R: number of reduce tasks
W: number of machines
Write map and reduce functions
Submit the job
Detailed Example: Word Count(1)
Map
Detailed Example: Word Count(2)
Reduce
Detailed Example: Word Count(3)
Main
Applications
String Match, such as Grep
Reverse index
Count URL access frequency
Lots of examples in data mining

MapReduce Implementations
MapReduce
Cluster,
1, Google
2, Apache Hadoop
Multicore CPU,
Phoenix @ stanford
GPU,
Mars@HKUST
Hadoop
Open source
Java-based implementation of MapReduce
Use HDFS as underlying file system
Hadoop
Google Yahoo
MapReduce Hadoop
GFS HDFS
Bigtable HBase
Chubby (nothing yet but
planned)
Recent news about Hadoop
Apache Hadoop Wins Terabyte Sort
Benchmark

The sort used 1800 maps and 1800
reduces and allocated enough memory to
buffers to hold the intermediate data in
memory.

Phoenix
The best paper at HPCA07
MapReduce for multiprocessor systems
Shared-memory implementation of MapReduce
SMP, Multi-core
Features
Uses thread instead of cluster nodes for parallelism
Communicate through shared memory instead of network
messages
Dynamic scheduling, locality management, fault recovery

Workflow
The Phoenix API
System-defined functions


User-defined functions
Mars: MapReduce on GPU
PACT08

GeForce 8800 GTX, PS3, Xbox360
Implementation of Mars
NVIDIA GPU (GeForce 8800
GTX)
CPU (Intel P4 four cores,
2.4GHz)
Operating System (Windows or Linux)
CUDA
System calls
MapReduce
User applications.
Implementation of Mars

Discussion
We have MPI and PVM,Why do we need MapReduce?
MPI, PVM MapReduce
Objective
General distributed
programming model
Large-scale data
processing
Availability
Weaker, harder better
Data
Locality
MPI-IO GFS
Usability
Difficult to learn easier
Conclusions
Provide a general-purpose model to
simplify large-scale computation

Allow users to focus on the problem
without worrying about details

References
Original paper
(http://labs.google.com/papers/mapreduce
.html)
On wikipedia
(http://en.wikipedia.org/wiki/MapReduce)
Hadoop MapReduce in Java
(http://lucene.apache.org/hadoop/)
http://code.google.com/edu/parallel/mapre
duce-tutorial.html