Scribd
Upload
60 views17 pages

Parallel Programming With: MPI For Python

This document discusses parallel programming with MPI for Python. It provides an overview of MPI including common implementations like Open MPI and MPICH. It then discusses how to assign a manager process, examples of point-to-point communication, broadcasting, scattering, synchronization, and gathering. It concludes with an example of calculating pi in parallel by splitting the calculation among processes and collecting the results.

Uploaded by

Trâm Thùy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
60 views17 pages

Parallel Programming With: MPI For Python

This document discusses parallel programming with MPI for Python. It provides an overview of MPI including common implementations like Open MPI and MPICH. It then discusses how to assign a manager process, examples of point-to-point communication, broadcasting, scattering, synchronization, and gathering. It concludes with an example of calculating pi in parallel by splitting the calculation among processes and collecting the results.

Uploaded by

Trâm Thùy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

Parallel Programming with

MPI for Python


Mehdi Rezaie
Postdoctoral Researcher
Kansas State U

Pre DESI meeting, December 2021


Outline

1. Point-to-Point Communication
2. Broadcasting
3. Scattering
4. Synchronization
5. Gathering
6. Calculating π
Message Passing Interface
● Message passing standard which
allows parallel computing on high
performance machines.
See mpi-forum.org

● Implementations
○ Open source:
■ Open-MPI
■ MPICH
○ Closed source:
■ Cray MPICH
■ Intel MPICH

MPI for Python: MPI C++ bindings for


python. See mpi4py.readthedocs.io
Assign the Manager
Take a single process as “root” to deal with
reading and writing; we do not want multiple
processes attempt opening the same file. Core 1
Core 2

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size() Core 0 Core N

It is common to take rank == 0 as the root


process.
Example 1
Hello World
Example 2
Point-to-point Communication
from source to destination

Process 1
Send �� Process 2
Receive
Example 3
Broadcasting

🍐 🍐

🍐
Example 4
Scattering

🍐🍎🍋 🍐

🍋
Synchronization

When placed before a call,


comm.Barrier() blocks the call until all
processes are synchronized.
Example 5
Gathering

🍐 🍐🍎🍋

🍋
Calculating π
π = S1 + S2 + S3 + S4 + S5

S1 S2 S3 S4 S5
Ex6. Simple Version
Ex7. MPI Version

This is where we split the


integral among MPI processes.
Ex7. MPI Version

This is where we split the


integral among MPI processes.

This is where we collect the


integrals from MPI processes.
Ex7. MPI Version

This is where we split the


integral among MPI processes.

S1 + S2 + S3 + S4 + S5 This is where we collect the


integrals from MPI processes.
Ex7. MPI Version
Quiz: Merge comm.gather and
sum into comm.reduce

This is where we split the


integral among MPI processes.

S1 + S2 + S3 + S4 + S5 This is where we collect the


integrals from MPI processes.
Further Reading: Parallelism in Python (NERSC)
docs.nersc.gov/development/languages/python/parallel-python/#mpi4py

You might also like