Map Reduce Machine Learning
Map Reduce Machine Learning
Abebe Zerihun
Introduction
• Multicore Era: Increasing number of processing cores
per chip, but lack of effective programming
frameworks for multicore machine learning.
• Goal: Develop a general, parallel programming
method to leverage multicore architectures for a
wide range of machine learning algorithms.
• Challenge: traditional approaches focus on specific
optimizations for individual algorithms, lacking
scalability and generalizability.
Map-Reduce
Contribution
• General Framework:
– Algorithms fitting the Statistical Query Model are
expressed in a summation form, enabling easy
parallelization.
– This method ensures exact, non-approximated
implementations of machine learning algorithms.
• Example instead of directly accessing a dataset {(xi, yi)}ni=1,
an algorithm may query.
• Aggregate gradients
• Core 1: 𝐷1={(𝑥1,𝑦1),…,(𝑥100,𝑦100)},
where each subset contains 100 samples:
• Core 2: 𝐷2={(𝑥101,𝑦101),…,(𝑥200,𝑦200)},
• Core 3: 𝐷3={(𝑥201,𝑦201),…,(𝑥300,𝑦300)},
• Core 4: 𝐷4={(𝑥301,𝑦301),…,(𝑥400,𝑦400)}
• Core 5: 𝐷5={(𝑥401,𝑦401),…,(𝑥500,𝑦500)}.
• Each core computes the partial
gradients for its assigned data • Core 4:
subset.
• Core1:
• Core 5:
• Core 2:
• Core 3
Aggregating Gradients
• After all cores compute their partial gradients, the gradients are
aggregated to form the total gradient.
• Update
Time complexity
• Logistic Regression
• Single machine/ core
– O(mn2 + n3)
• Multi-core
– O(+ n2log(p))
Q and A ?