InternationalUniversity - VNUHCM
school of computer science and engineering
MidtermExamination
Date: 23/04/2024; Duration: 60 minutes
Open book; Offline at Il-J.Laptops and mobileare not allowed.
SUBJECT: Scalable and Distributed Computing (ID: IT1391U)
Approval by the School of Computer Science and Lecturer
Engineering Signature
Signature
Full name: Full name: Mai Hoang Bao An
Proctor 1 Proctor 2
Signature Signature
Full name: Full name:
STUDENT INFO
Student name:
Student ID:
INSTRUCTIONS: the total of point is 100 (equivalent to 30% of the course)
1. Purpose:
• Test your knowledge in the concepts of Hadoop, MapReduce (CLO: 2, 3).
• Examine your skill of MapReduce in Python (CLO: 2, 3).
• Test your knowledge, skill in using Hadoop in colab environment (CLO: 3, 4).
2. Requirement:
• Read carefully each question and answer it following the requirements
• Write the answers and draw models CLEAN and TIDY directly in the exam notebook.
• Submit your exam including this paper inside.
• The sample file of question 3 in the last page.
1
Exercise I (30 points)
Can you providea short description of the Hadoop eco-system and MapReduce Algorithm. Please give an
example of map and reduce functions in Python.
Exercise Il (30 points)
2
Suppose that we have a list of numbers and we want to alculate the sum f squares of these numbers.
Please use map/reduce mechanism in Pythonto do this task.
Specifically, we'll first implement a map
function to square each number in a list, then develop reduce
a function to sum up those squares. Finally,
please parallelize the process using multithreadingfor
potentiallyfaster execution.
Exercise Ill (40 points)
Write your own mapper. py and reducer. py in the below case, then run
map/reduce in Hadoop system as
in the leture you're given.
The data purchases. txt is about the purchasing informationof a store.
2ø12-ø1-ø1 09:øø San Jose Men's Clothing 214.05 Amex
2012-01-01 09:øø Fort Worth Women'sClothing 153.57 Visa
2012-øl-øl 09:øø San Diego Music 66.08 Cash
2012-øl-øl 09:øø Pittsburgh Pet Supplies 493.51 Discover
You are required to write mapper. py and reducer. py to outputthe summary of the item and the
maximum value by store region as follow:
San Jose Men's Clothing 214.05
Fort Worth Men's Clothing 493.86
Fort Worth Women'sClothing 498.92
San Jose Women'sClothing 497.7
Note: Run below code to get the data to your session:
!wget —qhttp://content.udacity—data.com/courses/ud617/purchases.txt.gz
!gunzip purchases.txt.gz
2
2012-01-01 09:00 San Jose Men's Clothing 214.05 Amex
2012-01-01 09:00 Fort Worth Women's Clothing 153.57 Visa
2012-01-01 09:00 San Diego Music 66.08 Cash
2012-01-01 09:00 Pittsburgh Pet Supplies 493.51 Discover
2012-01-01 09:00 Omaha Children's Clothing 235.63 MasterCard
2012-01-01 09:00 Stockton Men's Clothing 247.18 MasterCard
2012-01-01 09:00 Austin Cameras 379.6 Visa
2012-01-01 09:00 New York Consumer Electronics 296.8 Cash
2012-01-01 09:00 Corpus Christi Toys 25.38 Discover
2012-01-01 09:00 Fort Worth Toys 213.88 Visa
2012-01-01 09:00 Las Vegas Video Games 53.26 Visa
2012-01-01 09:00 Newark Video Games 39.75 Cash
2012-01-01 09:00 Austin Cameras 469.63 MasterCard
2012-01-01 09:00 Greensboro DVDs 290.82 MasterCard
2012-01-01 09:00 San Francisco Music 260.65 Discover
2012-01-01 09:00 Lincoln Garden 136.9 Visa
2012-01-01 09:00 Buffalo Women's Clothing 483.82 Visa
2012-01-01 09:00 San Jose Women's Clothing 215.82 Cash
2012-01-01 09:00 Boston Cameras 418.94 Amex
2012-01-01 09:00 Houston Baby 309.16 Visa
2012-01-01 09:00 Las Vegas Books 93.39 Visa
2012-01-01 09:00 Virginia Beach Children's Clothing 376.11 Amex
2012-01-01 09:01 Riverside Consumer Electronics 252.88 Cash
2012-01-01 09:01 Tulsa Baby 205.06Visa
2012-01-01 09:01 Reno Crafts 88.25 Visa
2012-01-01 09:01 Chicago Books 31.08 Cash
2012-01-01 09:01 Fort Wayne Men's Clothing 370.55 Amex
2012-01-01 09:01 San Bernardino Consumer Electronics 170.2 Cash
2012-01-01 09:01 Madison Men's Clothing 16.78 Visa
2012-01-01 09:01 Austin Sporting Goods 327.75 Discover
2012-01-01 09:01 Portland CDs 108.69 Amex
2012-01-01 09:01 Riverside Sporting Goods 15.41 Discover
2012-01-01 09:01 Reno Toys 80.46 Visa
2012-01-01 09:01 Anchorage Music 298.86 MasterCard
2012-01-01 09:01 Pittsburgh Sporting Goods 475.26 Amex
2012-01-01 09:01 Spokane Garden 3.85 Amex
2012-01-01 09:01 Spokane Computers 287.65 MasterCard
2012-01-01 09:01 Fresno CDs 466.64 MasterCard
2012-01-01 09:01 Omaha Baby 255.68 MasterCard
2012-01-01 09:01 Anchorage DVDs 6.38 Amex
2012-01-01 09:01 Aurora Consumer Electronics 117.81 MasterCard
2012-01-01 09:01 Philadelphia DVDs 351.31 Cash
2012-01-01 09:01 Fremont Baby 222.61 Cash
2012-01-01 09:01 Anchorage Crafts 22.36 Amex
