National Computing Education Accreditation

Council
NCEAC

NCEAC.FORM.001.D

COURSE DESCRIPTION FORM: CS-3006: Parallel and Distributed Computing

INSTITUTION FAST School of Computing, National University of Computer

and Emerging Sciences, Islamabad Campus
BS-CS/BS-AI/DS: Spring-2023

PROGRAM(s) TO BE EVALUATED

Course Description
Course Code CS-3006
Course Title Parallel and Distributed Computing
Credit Hours 3
Prerequisites by CS 2006 Operating Systems
Course(s) and Topics
Grading Policy Absolute grading
Policy about missed Retake of missed assessment items (other than midterm/ final exam) will not be held.
assessment items in For a missed midterm/ final exam, an exam retake/ pretake application along with
the course necessary evidence are required to be submitted to the department secretary. The
examination assessment and retake committee decides the exam retake/ pretake cases.
Course Plagiarism Plagiarism in project or midterm/ final exam may result in F grade in the course.
Policy Plagiarism in an assignment will result in zero marks in the whole assignments category.
Assessment
Instruments with Assignments 4 8%
Weights (homework,
quizzes, midterms, Project 1 8%
final, programming Quizzes 5-7 12%
assignments, lab work, Sessional-I 1 12%
etc.)
Sessional-II 1 15%
Final Term Exam 1 45%
Course Coordinator Dr. Muhammad Aleem
URL (if any) -
Current Catalog -
Description
Textbook (or -
Laboratory Manual
for Laboratory
Courses)
Reference Material 1. Anshul Gupta, George Karpis, Vipin Kumar, Ananth Grama, Introduction to
Parallel Computing, Second Edition, ISBN-10 : 0201648652, Pearson

1
NCEAC.FORM.001.D
 National Computing Education Accreditation
Council
NCEAC

NCEAC.FORM.001.D

2. Michael J. Quinn, Parallel Programming in C with Mpi and Openmp, 1st Edition,
ISBN-10 : 0072822562, McGraw-Hill Science/Engineering/Math;
3. Aaftab Munshi, Benedict Gaster, Timothy G. Mattson, James Fung,Dan
Ginsburg, OpenCL Programming Guide, 1st Edition, Publisher: Addison-Wesley
Professional; 1 edition (July 23, 2011)
4. Tom White, Hadoop: The Definitive Guide: Storage and Analysis at Internet
Scale, 4th Edition, ISBN: 1491901632, O'Reilly Media

Course Goals
Course Learning Outcomes (CLOs)
1. Demonstrate understanding of various concepts involved in parallel and distributed
computer architectures.
2. Implement different parallel and distributed programming paradigms and algorithms
using Message-Passing Interface (MPI) and OpenMP
3. Perform analytical modelling, dependence, and performance analysis of parallel
algorithms and programs.

Program Learning Outcomes (PLOs)

PLO 1 Computing Apply knowledge of mathematics, natural sciences,

Knowledge computing fundamentals, and a computing
specialization to the solution of complex computing
problems
PLO 2 Problem Analysis Identify, formulate, research literature, and analyse
complex computing problems, reaching substantiated
conclusions using first principles of mathematics,
natural sciences, and computing sciences.
PLO 3 Design/Develop Design solutions for complex computing problems
Solutions and design systems, components, and processes
that meet specified needs with appropriate
consideration for public health and safety, cultural,
societal, and environmental considerations.
PLO 4 Investigation & Conduct investigation of complex computing
Experimentation problems using research-based knowledge and
research-based methods.
PLO 5 Modern Tool Create, select, and apply appropriate techniques,
Usage resources and modern computing tools, including
prediction and modelling for complex computing
problems.
PLO 6 Society Apply reasoning informed by contextual knowledge to
Responsibility assess societal, health, safety, legal, and cultural
issues relevant to context of complex computing
problems.
PLO 7 Environment and Apply ethical principles and commit to professional
Sustainability ethics and responsibilities and norms of computing
practice.
PLO 8 Ethics Apply ethical principles and commit to professional
ethics and responsibilities and norms of computing
2
NCEAC.FORM.001.D
 National Computing Education Accreditation
Council
NCEAC

NCEAC.FORM.001.D

practice.
PLO 9 Individual and Function effectively as an individual, and as a
Teamwork member or leader in diverse teams and in multi-
disciplinary settings.
PLO 10 Communication Communicate effectively on complex computing
activities with the computing community and with
society at large
PLO 11 Project Demonstrate knowledge and understanding of
Management and management principles and economic decision
Finance making and apply these to one’s own work as a
member of a team.
PLO 12 Lifelong Learning Recognize the need for, and have the preparation
and ability to engage in independent and life-long
learning in the broadest context of technological
changes.

C. Mapping of CLOs to PLOs

(CLO: Course Learning Outcome, PLOs: Program Learning Outcomes)
PLOs

1 2 3 4 5 6 7 8 9 10 11 12

1 ✔

CLOs 2 ✔

3 ✔

Topics Covered in
the Course, with Module-1 & 2 [7 Weeks]
Number of Lectures 1. Fundamentals of Parallel and Distributed Computing
on Each Topic 2. Need for Parallelism, Parallelizable vs Un-Parallelizable Tasks, Domain and
(assume 15-week Functional Decomposition
instruction and one- 3. Flynns Taxonomy, Parallel Architectures (Shared Nothing, Shared Disk, Shared
hour lectures) Memory),
4. Building a Cluster
5. Distributed Memory Programming using MPI – Basics
6. Advanced Distributed Memory System: MPI
7. Amdahl’s Law vs Gustafson's law, Speedup, Efficiency, Measuring Parallelism:
Work Done and Speedup
8. GPU Architecture and Difference between CPU and GPU, Data-Parallel
Programming (CPU & GPU) using OpenCL.

Module-3 [6.5 Weeks]

9. Data Parallelism Vs Task Parallelism,
10. Parallelism Granularity, Load balancing, Concurrency and Synchronization

3
NCEAC.FORM.001.D
 National Computing Education Accreditation
Council
NCEAC

NCEAC.FORM.001.D

11. Task graph, Task Dependency graph,

12. Data Dependency Analysis
13. Introduction to Parallel Programing Environments
14. Shared Memory Systems: Parallel Algorithms using Open MP - (Mutual
Exclusion, Reduction, locks),

Module-4 [1.5 Weeks]

15. Hadoop/Map Reduce: Distributed File System, Map Reduce Programming
model

Laboratory 1 semester project

Projects/Experiments
Done in the Course
Programming Yes (at least 4)
Assignments Done in
the Course
Class Time Spent on Theory Problem Analysis Solution Design Social and Ethical Issues
(in credit hours) 30% 10% 55% 5%
Oral and Written Every student is required to submit at least 1 written reports/implementation of the
Communications semester project and to make oral presentation/demo of typically 20 minute’s duration.

Tentative course outline and lecture plan:

Theoretical Component of the course
Assessment Items
(Case Study/ Exercise
Weeks Contents/Topics
Assignment/
Quiz etc.)
Fundamentals of Parallel and Distributed Computing
Need for Parallelism, Parallelizable vs Un-Parallelizable Tasks, Domain
Week-01 and Functional Decomposition, Data Parallelism Vs Task Parallelism
Parallelism Granularity, Load balancing,
Slides 1 & 2
Flynns Taxonomy, Parallel Architectures (Shared Nothing, Shared Disk,
Assignment 1
Week-02 Shared Memory), Intrinsic Programming
(Pthreads)
Slides 3 & 3_1

Cluster Computing
Building a Cluster
Week-03 Assignment 1 Due
Distributed Memory Programming using MPI – Basics
Slides 3_2, 4, 5

4
NCEAC.FORM.001.D
 National Computing Education Accreditation
Council
NCEAC

NCEAC.FORM.001.D

Advanced Distributed Memory System: MPI

Week-04
Slides 5, 6

GPU Architecture and Difference between CPU and GPU, Data-Parallel

Programming (CPU & GPU) using OpenCL.
Assignment 2 MPI
Week-05 Slides 15

Week-06 Mid I

Paper showup
Hadoop/Map Reduce: Distributed File System, Map Reduce Programming
Week-07 Assignment 2 Due
model

Introduction to Parallel Programing Environments

Hadoop/Map Reduce: Distributed File System, Map Reduce Programming
Week-08 Assignment 3 Hadoop
model
Slides 10, 11,12
Shared Memory Systems: Parallel Algorithms using Open MP - (Mutual
Week-09 Exclusion, Reduction, locks), Assignment 3 Due
Slide 7

Amdahl’s Law vs Gustafson's law, Speedup, Efficiency, Measuring

Week-10 Parallelism: Work Done and Speedup Assignment 4 OpenMP
Slide 8

Data Dependency Analysis

Week-11 Assignment 4 Due

Mid II
Project (MPI/Hadoop)
Week -12 Assignment 4 Due

Paper Showup
Week 13 Data Dependency Analysis
Slide 9

Data Dependency Analysis

Week-14 Project Due

Week-15 Revision

5
NCEAC.FORM.001.D
 National Computing Education Accreditation
Council
NCEAC

NCEAC.FORM.001.D

6
NCEAC.FORM.001.D