Scribd
Upload
58 views13 pages

Analytical Modeling of Parallel Programs S. Oliveira

This document discusses analytical modeling of parallel programs. It introduces concepts like efficiency, speedup, scalability, isoefficiency functions, and cost-optimality. The key points are: - Efficiency and speedup of a parallel program decrease with increasing number of processors due to overhead. - A scalable parallel system can maintain constant efficiency by increasing problem size and processors simultaneously. - The isoefficiency function determines how fast problem size must grow relative to processors to maintain efficiency. Smaller functions indicate better scalability.

Uploaded by

anupam20099
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
58 views13 pages

Analytical Modeling of Parallel Programs S. Oliveira

This document discusses analytical modeling of parallel programs. It introduces concepts like efficiency, speedup, scalability, isoefficiency functions, and cost-optimality. The key points are: - Efficiency and speedup of a parallel program decrease with increasing number of processors due to overhead. - A scalable parallel system can maintain constant efficiency by increasing problem size and processors simultaneously. - The isoefficiency function determines how fast problem size must grow relative to processors to maintain efficiency. Smaller functions indicate better scalability.

Uploaded by

anupam20099
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

AnalyticalModelingofParallel

Programs
S.Oliveira

08/25/05

S.Oliveira,22C177/22M178
Fall2005

ScalabilityofParallelSystems

Efficiencyofaparallelprogram
E=S/P=Ts/PTp

Usingtheparalleloverheadexpression
E=1/(1+To/Ts)

ThetotaloverheadfunctionToisanincreasing
functionofP.
Foragivenproblemsize,T remainsconstant,
s
asweincreasethenumberofP,Toincreases.
Efficiencydecreaseswithincreasingnumberof
processingelements~characteristiccommonto
S.Oliveira,22C177/22M178
allparallelprograms.
08/25/05
Fall2005
2

ScalabilityofParallelSystemsContd

SpeedupandefficiencyasfunctionstoP:

08/25/05

Tp=n/p+2logp

S=n/[(n/p)+2logp]
E=1/1+(2plogp/n)

BothEfficiencyandSpeedupincreasewithn,forthe
samenumberofP.
BothEfficiencyandSpeedupdropwithincreasingP.i
S.Oliveira,22C177/22M178
Fall2005

ScalabilityofParallelSystemsContd

Itispossibletokeeptheefficiencyinmany
parallelsystemsfixedbyincreasingboththe
sizeoftheproblemandthenumberof
processingelementssimultaneouslyscalable
parallelsystem.
Scalability:measureofaparallelsystems
capacitytoincreasespeedupinproportiontothe
numberofprocessingelements.
Recallthatacostoptimalparallelsystemhas
efficiencyof(1).So,scalabilityandcostoptimalityof

parallelsystemsarerelated.Wecanhaveacostoptimal
parallelsystemifthe#ofPandsizeofcomputationare
S.Oliveira,22C177/22M178
08/25/05
4
chosenappropriately. Fall2005

IsoefficiencyMetricofScalability

Wesaidthatforagivenproblemsize,asweincrease#of
P,theoverallefficiencyofparallelsystemdecreases.
ScalableParallelSystemisoneinwhichefficiencycanbe
keptconstantasthenumberofprocessingelementsis
increased,providedproblemsizeisalsoincreased.
TherateofchangeofbothPandproblemsizedetermine
thedegreeofscalabilityofaparallelsystem.Alowerrateis
moredesirablethanahighergrowthrateinproblemsize.
ProblemSizeisdefinedasthenumberofbasic
computationstepsinthebestsequentialalgorithmtosolve
theproblemonasingleP.
Weassumethatittakesunittimetoperformonebasic
computationofthealgorithm.ThusProblemsize=W=Ts
(timeoffastestknownsequentialalgorithm).
08/25/05

S.Oliveira,22C177/22M178
Fall2005

IsoefficiencyFunction

ParallelExecutionTimecanbeexpressedasafunctionof:

ProblemSize
Overheadfunction
NumberofProcessingelements

Wecanrewriteparallelruntime(Tp)as:

08/25/05

S.Oliveira,22C177/22M178
Fall2005

IsoefficiencyFunctioncontd

Conclusions:

Iftheproblemsize(W)iskeptconstantandpisincreased,
efficiency(E)decreasesbecausethetotaloverhead(To)
increaseswithp.
Fordifferentparallelsystems,Wmustbeincreasedatdifferent
rateswithrespecttopinordertomaintainafixedefficiency.
ForscalableparallelsystemsEcanbemaintainedatafixed
value(0<E<1)iftheratioTo/Wismaintainedconstant.

*
08/25/05

K=E/(1E),weget:
S.Oliveira,22C177/22M178
Fall2005

IsoefficiencyFunctioncontd
IsoefficiencyFunction:W=KTo(W,P).Thisfunction
determinestheeasewithwhichaparallelsystemcan
maintainaconstantefficiencyandachievespeedups
increasinginproportiontonumberofprocessingelements.
Asmallisoefficiencyfunctionmeansthatsmallincrementsin
problemsizearesufficientforefficientutilizationofincreasing
#ofP,indicatingthatthesystemishighlyscalable.
Howeveralargeisoefficiencyfunctionindicatesapoorly
scalableparallelsystem.
Theisoefficiencyfunctiondoesnotexistforunscalable
parallelsystemsbecausetheEcantbekeptatanyconstant
valueaspincreases,nomatterhowfasttheproblemsizeis
increased.
Finally,theisoefficiencyfunctioncapturesthecharacteristics
S.Oliveira,22C177/22M178
ofaparallelalgorithmaswellastheparallelarchitectureon
08/25/05
Fall2005
8
whichitisimplemented.

CostOptimality&Isoefficiency
Function

Aparallelsystemiscostoptimaliftheproductofthe
numberofprocessingelementsandtheparallel
executiontimeisproportionaltotheexecutiontimeof
thefastestknownsequentialalgorithmonasingle
processingelement.
AparallelsystemiscostoptimalIFF:

Theabove2equationssuggestthataparallelsystemis
costoptimalIFFitsoverheadfunctiondoesnot
asymptoticallyexceedtheproblemsize.
~mustbesatisfiedtoensurethecost
optimalityofaparallelsystemasitisscaledup.
S.Oliveira,22C177/22M178

08/25/05

Fall2005

LowerBoundonIsoefficiency
Function

Anideallyscalableparallelsystemmusthave
thelowestpossibleisoefficiencyfunction.
ForaproblemconsistingofWunitsofwork,no
morethanWprocessingelementscanbeused
costoptimally,asadditionalpwillbeidle.
Theproblemsizemustincreaseatleastasfast
as(p)tomaintainfixedefficiency;hence(p)
istheasymptoticlowerboundonthe
isoefficiencyfunction.

08/25/05

S.Oliveira,22C177/22M178
Fall2005

10

DegreeofConcurrency&
IsoefficiencyFunction

Themaximumnumberoftasksthatcanbeexecuted
simultaneouslyatanytimeinaparallelalgorithmis
calleditsdegreeofconcurrency.
DegreeofConcurrency:isameasureofthenumberof
operationsthatanalgorithmcanperforminparallelfora
problemofsizeW.
IfC(W)isthedegreeofconcurrencyofaparallel
algorithm,thenforaproblemofsizeW,nomorethanC
(W)processingelementscanbeemployedeffectively.
Theisoefficiencyfunctionduetoconcurrencyisoptimal,
i.e.(p),onlyifthedegreeofconcurrencyofthe
parallelalgorithmis
(W)
S.Oliveira,22C177/22M178

08/25/05

Fall2005

11

MinimumExecutionTime&Minimum
CostOptimalExecutionTime

Ifweincreasethenumberofprocessingelementsfora
givenproblemsize,eithertheparallelruntimecontinues
todecreaseandasymptoticallyapproachesaminimum
valueoritstartsrisingafterattainingaminimumvalue.
AproblemofsizeW,acostoptimalsolutionrequires
thatp=O(f1(W)).Sincetheparallelruntimeis(W/P)
foracostoptimalparallelsystem.

08/25/05

S.Oliveira,22C177/22M178
Fall2005

12

AsymptoticAnalysisofParallelPrograms.
Comparisonoffourdifferentalgorithmsforsorting
agivenlistofnumbers.Thetableshowsnumber
ofprocessingelements,parallelruntime,speedup,
efficiencyandthepTpproduct.

08/25/05

S.Oliveira,22C177/22M178
Fall2005

13

You might also like