Lecture 3.1.4 (Amdahl's Law)
Lecture 3.1.4 (Amdahl's Law)
ASSISTANT PROFESSOR
BE-CSE
AMDAHL'S LAW
• In parallel computing, Amdahl's law is mainly used to predict the theoretical
maximum speed up for program processing using multiple processors.
• It is named after computer scientist Gene Amdahl (a computer architect from
IBM and Amdahl corporation), and was presented at the AFIPS Spring
Joint Computer Conference in 1967.
• It is also known as Amdahl’s argument.
• It is a formula which gives the theoretical speedup in latency of the execution of
a task at a fixed workload that can be expected of a system whose resources are
improved.
• In other words, it is a formula used to find the maximum improvement possible
by just improving a particular part of a system. It is often used in parallel
computing to predict the theoretical speedup when using multiple processors.
Speedup
Speedup is defined as the ratio of performance for the entire task using the
enhancement and performance for the entire task without using the enhancement
or speedup can be defined as the ratio of execution time for the entire task without
using the enhancement and execution time for the entire task using the
enhancement.
If Pe is the performance for entire task using the enhancement, when
possible, Pw is the performance for entire task without using the enhancement,
Ew is the execution time for entire task without using the enhancement and Ee
is the execution time for entire task using the enhancement when possible
then,
Speedup = Pe/Pw
or
Speedup = Ew/Ee
Amdahl’s law uses two factors to find speedup from some enhancement.
Fraction enhanced – The fraction of the computation time in the original
computer that can be converted to take advantage of the enhancement.
For example- if 10 seconds of the execution time of a program that takes 40
seconds in total can use an enhancement, the fraction is 10/40. This obtained value
is Fraction Enhanced.
Fraction enhanced is always less than 1.
Speedup enhanced – The improvement gained by the enhanced execution mode;
that is, how much faster the task would run if the enhanced mode were used for the
entire program.
For example – If the enhanced mode takes, say 3 seconds for a portion of the
program, while it is 6 seconds in the original mode, the improvement is 6/3. This
value is Speedup enhanced. Speedup Enhanced is always greater than 1.
The overall Speedup is the ratio of the execution time.
Proof:
Consider that the execution time required to complete the given task on a
computer with single processor is T. When the same task is executed on parallel
computer with p processors, then the time required will depend upon sequential
computing time and parallel computing time that is,
T(p) = Sequential computing Time + Parallel Computing Time
From equation (2), we can say that S(p) will always be ≤ 1, that is
Example: Suppose, Moni have to attend an invitation. Moni’s another two
friend Diya and Hena are also invited. There are conditions that all three friends
have to go there separately and all of them have to be present at door to get into
the hall. Now Moni is coming by car, Diya by bus and Hena is coming by foot.
Now, how fast Moni and Diya can reach there it doesn’t matter, they have to
wait for Hena. So, to speed up the overall process, we need to concentrate on
the performance of Hena other than Moni or Diya.
where,
SpeedupMAX = maximum performance gain
s = performance gain factor of p after implement the enhancements. p =
the part which performance needs to be improved.
So, we can see, if 1-p can’t be improved, the overall performance of the system
cannot be improved so much. So, if 1-p is 1/2, then speed cannot go beyond
that, no matter how many processors are used.
Reference Books:
●J.P. Hayes, “Computer Architecture and
Organization”, Third Edition.
●Mano, M., “Computer System Architecture”, Third
Edition, Prentice Hall.
●Stallings, W., “Computer Organization and Architecture”, Eighth
Edition, Pearson Education.
Text Books:
●Carpinelli J.D,” Computer systems organization &Architecture”, Fourth
Edition, Addison Wesley.
●Patterson and Hennessy, “Computer Architecture”, Fifth Edition Morgaon
Kauffman.
Other References:
• https://www.tutorialspoint.com/what-is-amdahl-s-law
• https://www.techopedia.com/definition/17035/amdahls-law
• https://www.geeksforgeeks.org/computer-organization-amdahls-law-and-its-
proof/
Video Links:
https://youtu.be/2dYV_lADfFI?si=DNu3JcYHY23ZDJK7
https://youtu.be/DpMVYR8FauE?si=14MZ4ffnCh4kG6VO