Course Syllabus: Parallel and Distributed Computing
Program: Bachelor of Science in Computer Science
Course Code: PD101
Credit Hours: 3 Units
Prerequisites: Data Structures, Operating Systems, Computer Architecture
Course Description
This course introduces the fundamental concepts, principles, and techniques of parallel and
distributed computing. Students will explore the design, implementation, and evaluation of
parallel and distributed systems, focusing on concurrency, communication, synchronization, and
scalability.
Course Objectives
By the end of this course, students will be able to:
1. Understand the fundamentals of parallel and distributed systems.
2. Design and implement parallel algorithms using appropriate tools and programming
models.
3. Analyze and optimize performance for parallel and distributed applications.
4. Develop applications that leverage distributed computing frameworks.
Weekly Topics and Course Schedule
Week Topic Learning Outcomes
Introduction to Parallel and Understand basic concepts, differences between
1
Distributed Computing parallel and distributed systems.
Parallel Programming Models and Explore shared memory, distributed memory, and
2
Architectures hybrid models; architectural overview.
Learn about threads, processes, locks, semaphores,
3 Concurrency and Synchronization
and condition variables.
Implement distributed systems using MPI;
4 Message Passing Interface (MPI)
communication and synchronization in MPI.
Thread-based Parallelism: Use OpenMP for multi-threaded programming;
5
OpenMP explore tasks and loops parallelization.
Performance Metrics and Analyze speedup, efficiency, scalability, and load
6
Benchmarking balancing in parallel programs.
Week Topic Learning Outcomes
Explore client-server, peer-to-peer, and cloud
7 Distributed System Architectures
architectures.
Distributed File Systems and Understand distributed storage systems like HDFS;
8
Databases study CAP theorem and NoSQL.
Fault Tolerance in Distributed Learn replication, checkpointing, and recovery
9
Systems mechanisms.
Distributed Coordination and
10 Study leader election, Paxos, and RAFT algorithms.
Consensus Algorithms
MapReduce and Big Data Develop applications using MapReduce and explore
11
Frameworks Hadoop and Spark frameworks.
Cloud Computing and Understand cloud models, virtualization, and
12
Virtualization containerization (e.g., Docker, Kubernetes).
Design and analyze algorithms for sorting,
13 Parallel Algorithms
searching, and graph processing in parallel.
GPU and Heterogeneous Use CUDA or OpenCL for programming GPUs;
14
Computing explore hybrid systems.
Security in Parallel and Study issues like data privacy, secure
15
Distributed Systems communication, and attack mitigation.
Future Trends and Emerging Explore quantum computing, edge computing, and
16
Technologies new paradigms in distributed systems.
Project Presentations and Course Present and evaluate final projects; reflect on course
17
Wrap-Up learnings.
Assessment Methods
1. Class Participation and Quizzes (20%)
2. Programming Assignments (30%)
3. Midterm Exam (20%)
4. Final Project and Presentation (30%)
Course Materials
Textbooks:
1. Grama, A., Gupta, A., Karypis, G., & Kumar, V. (2003). Introduction to Parallel
Computing. Addison Wesley.
2. Coulouris, G., Dollimore, J., & Kindberg, T. (2011). Distributed Systems: Concepts and
Design. Pearson.
Supplementary Resources:
Online tutorials on MPI, OpenMP, and CUDA.
Research papers on distributed systems and emerging trends.
