By Kelvin Chou
Programming Parallelism
Data Level Parallelism
Single
Multiple
Instruction Instruction
Single Data SISD
MISD
Multiple
Data
SIMD
MIMD
Single Instruction Multiple
Data
One Clock Cycle
Vectorized
Question from Fall 2012
What would we
want START set
to be?
Question from Fall 2012
We can see that
we want to handle
the fringe first!
What part does
the fringe take car
of for us?
Question from Fall 2012
Since we are
32 bit integers
Into a 128 bit
vector, can fit
4 integers into
one vector.
Question from Fall 2012
We would want
Start set to be:
(n % 4)
#define START (n % 4)
Question from Fall 2012
What would we
want J_INIT set
to be?
#define START (n % 4)
Question from Fall 2012
Notice what
we are loading
into j_on_steroids?
#define START (n % 4)
Question from Fall 2012
The j_on_steroids
is being added to
a vector of what
START is.
#define START (n % 4)
Question from Fall 2012
We see that
j += START. Where
else do we notice
that START is bein
used to calculate?
#define START (n % 4)
Question from Fall 2012
There is a
START + 1 in
the fringe case!
That is where we
will start.
Therefore,
J_INIT = 1
#define START (n % 4)
#define J_INIT 1
Question from Fall 2012
Next will be
STEROIDS_INIT.
We see that this
Is being added
to the set1 of
START.
#define START (n % 4)
#define J_INIT 1
Question from Fall 2012
Given that we wan
to start at the
START, we know
that for
Differentiation
Multiply by one
Greater than
Current.
#define START (n % 4)
#define J_INIT 1
Question from Fall 2012
Since initial is
Added to set1.
Then, we can
see that we want
each field
Initialized
Differently.
#define START (n % 4)
#define J_INIT 1
Question from Fall 2012
Example:
If START = 2.
Want the vector
Going into main
Loop to contain
{3, 4, 5, 6}
#define START (n % 4)
#define J_INIT 1
Question from Fall 2012
{3, 4, 5, 6}
Therefore
{2, 2, 2, 2} +
{a, b, c, d}?
{1, 2, 3, 4}
#define START (n % 4)
#define J_INIT 1
#define STEROIDS_INIT \
{1, 2, 3, 4}
Question from Fall 2012
Last thing we
Want to find
Is where to
END the loop?
#define START (n % 4)
#define J_INIT 1
#define STEROIDS_INIT \
{1, 2, 3, 4}
Question from Fall 2012
Keep in mind
The constraints
Of the problem.
What is the
Last index that we
Need to keep track
#define START (n % 4)
#define J_INIT 1
#define STEROIDS_INIT \
{1, 2, 3, 4}
Question from Fall 2012
In differentiation
the nth term falls
off!
So this means
END = (n-1)
Thread Level Parallelism
Each thread executes on
different data
Allows things to be run in
parallel
OpenMP is an example
Thread Level Parallelism
Question from Spring 2013
Suppose we have int *A that points to the head of an
array of length len. Assume we have n > 1 threads.
#pragma omp parallel for
for (int x = 0; x < len; x++){
*A = x;
A++;
}
Is this always incorrect, sometimes incorrect, always
correct?
Question from Spring 2013
#pragma omp parallel for
for (int x = 0; x < len; x++){
*A = x;
A++;
}
Is this always incorrect, sometimes incorrect,
always correct?
This is due to the fact that we will have data races to
see who can increment A correctly. It is possibly
correct if the stars align.
Question from Spring 2013
#pragma omp parallel
{
for (int x = 0; x < len; x++) {
*(A+x) = x;
}
}
Is this always incorrect, sometimes incorrect, always
correct?
Question from Spring 2013
#pragma omp parallel
{
for (int x = 0; x < len; x++) {
*(A+x) = x;
}
}
Is this always incorrect, sometimes incorrect, always
correct?
Now, is this faster or slower than serial?
Question from Spring 2013
#pragma omp parallel
{
for (int x = 0; x < len; x++) {
*(A+x) = x;
}
}
Is this always incorrect, sometimes incorrect, always correct?
Now, is this faster or slower than serial?
Slower, due to duplication of work. And false sharing.
