Motivation for Research

Software performance problems become more difficult as systems grow more complex.
Improvements in hardware do not present an automatic solution, since performance is not solely
dependent on available CPU power, but other factors, such as communication delays, play an
increasingly significant role. In addition, hardware improvements increase problem complexity
growth.

Traditional software performance improvement methods, such as benchmarks and execution

profiling, are not enough to guarantee good software performance. More refined methods are
necessary to find difficult-to-handle situations (inputs), program execution bottlenecks and
reasons for these.

Software Performance
'Good performance' sounds like an easy concept, but it isn't so.

Many different kinds of performance:

• computing time requirements (response time, throughput)

• computing space requirements (size, job count)
• flexibility (updatability)

Performance is a compromise.

• too good a performance is no good

• there are other objectives besides performance

Differing Solutions
Different solutions are needed when the desired result is:

• good response time for one job

o fast reaction in exceptional circumstances
o with many jobs, response time increases dramatically
• good response time for one job
o good in time sharing
o even with few jobs, response time doesn't decrease
• large throughput
o good in batch jobs
o may have an abysmal response time
• ready for continued development
o hardly tuned for the best possible performance

Problems in Performance
Explosive growth in CPU power hasn't solved problems in performance

• increased memory
• program growth
• program functionality harder to understand
• memory access delays
• data communication delays
• hierarchical structure

Especially

• large software
• parallel software
• layered software
• embedded software
• realtime software

Domains of Performance Research

The majority of performance research does not deal with software.

• research of computing systems

• land traffic, factories, storerooms etc.

Researching software performance is a relatively new subject

• traditional methods have been used

• the growth in software complexity has made them difficult to use

Cousins of Software Performance Research

Profiling

• automatic analysis
• use of ready methods
• SPR designs profiling methods

Program tuning

• bit fiddling
• local improvements at the cost of clarity

Algorithmic research

• SPR does not aim to invent new algorithms

• algorithm research benefits SPR
Optimizing compilers

• automatic software improvement

Goals of Software Design

Software performance is rarely the only objective in software development, but
often it is an important one, if not the most important.

• functionality
• correctness
• usability
• performance
• maintainability

Performance requirements are often focused on a limited part of the program's functionality.

Software Performance Design

Not only to come up with the best possible implementation

• flexibility design
• evaluation of external requitements
• investigating hardware requirements

Two approaches

• fix-it-later, improving what has been done

o typical
o development without performance estimation
o analysis after the disgrace
o attempt at remedying the problem
• SPE, design of new
o a better way
o estimate of performance at design phase
o performance problems are often related to the software structure
o avoiding mistakes costs less than fixing them

Regions of SPE
• execution modelling
o general models
o software models
• input modelling
o input characterization
o common/difficult input classes
 • model solving
o analytical methods
o simulation
• gathering of performance data
o monitoring
o derivation of parameters
• tools
o stand-alone tools
o tool environments
• design principles
o real time systems
o embedded systems
• software engineering
o inclusion of performance into development

Performance Models
A performance model describes the program's resource usage during computing.

• total performance
o a function from input to measures
• typically structured
o contributing factors

Several kinds of performance models

• execution structure
o typically control flow
• resource usage
o typically time

Flow Graph
A simple model of performance; a timed flow graph
Problems

• choices, repetitions
• parallelism
• locks

Petri Nets
A Petri net consists of

• nodes
o places P
o transitions T
• directed edges P-T or T-P

System state

• depicted with tokens

• changed by a firing event
• triggering an event

Usage

• describing parallelism
• describing locking

Example
Several kinds of Petri nets

• amount of edges
• amount of tokens

Usage of a critical resource

Describing performance via Petri nets

• timed events
• stochastic events
• static values

Queuing networks
Queuing networks consist of

• nodes
o queuing servers
o queueless servers
• directed graphs

System state

• jobs
• queuing discipline (FCFS, LCFS-PR, RR, PS, SJF, SRT, etc.)

Usage

• Servers as system components

• jobs moving from one component to another

Example
Below a simple system:

• servers with queues are system's shared resources

• no competition for resources in queueless servers
o infinite servers, delay servers

Performance

• service times
• branching probabilities
• static values
• several job types

The queuing discipline is essential for solvability of the model.

Extended Queuing Networks

Several locking and competition situations can't be described by using an ordinary
queueing network.

• extensions neccessary
• new types of nodes
• more difficult to solve

Above: a queued resource (REQ-REL)

 • dynamic service time
• more difficult to solve

and a changing node (CHG)

• may change job type

• may change job count

Other Models
There are many methods to model performance. Their basis is usually

• a partial order (network)

o queuing networks
o Petri nets
o task graphs
• state machine
o SDL (CCITT)
o state diagrams
o flow diagrams
• process algebra
o CCS
o CCP
o LOTOS

Network-based models are usually represented in a graphical form; several other models have a
source-code-like representation.

Software Modelling
Software complexity presents a problem

• immense number of operations

• mostly insignificant
• important operations (perhaps implicitly)
o locking
o queuing
o I/O

Important locations hidden under other structure

• abstractions
• modularity
• parallelism

How to reveal the significant parts and clear away the rest?
Example: Execution Networks
One way to deal with the problem are heterogenous and hierarchical models

• execution model (below: execution network)

• system model (below: queueing network)
• connection between the two (transformation analysis)

Input Models
Besides execution, input must be modelled

• performance figures are always for some input

• execution models are often related to a certain input class

Input model characteristics depend on usage

• declarative
• procedural

Several inputs to execute on a model

• real
• syntethic
 • artificial

Often the same program has several input classes

• clustering the input classes

• classing and search algorithms
o minimal spanning tree
o AQ, ID3, CART, PAC
o neural nets
o genetical algorithms

Statistical Input Models

• analytical models
• simulation models

Statistical models describe the input sample space

• possible inputs
• probability of each

The model may be used in several ways in a simulation

• continuous
• one-shot inputs

Example: G(n, p)
G(n, p) is a commonly used input model in the analysis of network algorithms

• statistical
• efficiently executable
• one-shot

G(n, p) chooses a single net from the set of all graphs

• n nodes in the graph

• probability of each edge is p
• expected value of edges p*n^2 (directed)

Properties

• suitable for local algorithms

• unsuitable for structural inspection
o a giant component in the graph
Benchmarks
Benchmarks are typically single workloads

• designed to correspond to typical system workload

• programs, transactions, job mixes, etc.

Established benchmarks

• Whetstone
• Dhrystone
• Linpack
• Perfect Club
• SPEC
• TCP (A,B,C)

Example: Dhrystone
Dhrystone is a typical system program execution benchmark

• image of a typical program

• integer computation
• if, while, etc.
• modern models are more complex

A brief sample of Dhrystone (in C)

One_Fifty *Int_Par_Ref;
{
One_Fifty Int_Loc;
Enumeration Enum_Loc;

Int_Loc = *Int_Par_Ref + 10;

do /* executed once */
if (Ch_1_Glob == 'A')
/* then, executed */
{
Int_Loc -= 1;
*Int_Par_Ref = Int_Loc - Int_Glob;
Enum_Loc = Ident_1;
} /* if */
while (Enum_Loc != Ident_1); /* true */
} /* Proc_2 */

Model Interpretation
The measurement of performance as a function of input parameters is what should
be solved from a performance model

• performance goal
 • differnet views on the same model

For instance, a demand on response time may limit throughput.

Analytical Solution
Analytical solution of net models is often based on statistical analysis.

• static analyysi
• model states
• state transformations

Markov state graph

A probability for each state (steady-state analysis).

• solve the equations related to the state graph

• difficult to solve

In practice more efficient methods are used

• also approximate methods

Simulation
In simulation, the model is executed

• dynamical analysis
• simulation program
• can be solved well, but requires computation

Simulation can be

• continuous (steady-state)
• repetition of experiments with parameter variations

Tools needed for

• statistical processing
• input generation
• system warm-up

Result consistent with reality?

• verification
• validation

Simulation Example I
Job-based simulation

A sample from a program

• queuing network model

• steady-state

job(char *name) {
create(name);
while (completions < warm_up + measure) {
reserve(cpu);
hold(10.0);
release(cpu);
if (prob() < 0.5) {
reserve(disk1);
hold(20.0);
release(disk1);
} else {
reserve(disk2);
hold(30.0);
release(disk2);
}
completion_count++;
 if (completions == warm_up)
reset();
}
if (completions == warm_up + measure)
set(done);
}

Simulation Example II
Server-based

The above in a different form

cpu() {
int job;
int ndisk;

create("CPU");
while(job_count) {
job = wait_any(enter_cpu);
use(fcpu, 10.0);
set(cpu_free);
ndisk = prob() < 0.5 ? 0 : 1;
if (num_busy(fdisk[ndisk]))
queue(disk_free[ndisk]);
if (ndisk)
set(enter_disk1[job]);
else
set(enter_disk0[job]);
}
}

Computation with variables

Typical variables

• time T
• throughput X
• completion count C
• service demand D
• job time W
• job count L
• utilization U
• response time R
• arrival rate lambda
• service rate mu
Software Performance Engineering
Covers the whole lifespan of the software

• requirement analysis
o realism of the goals?
o equipment suitability?
o usage of the software?
• designing the overall structure
o alternatives fit for implementation?
o unfeasible alternatives?
• detailed software design
o performance prediction
o comparison of implementation alternatives
• implementation, testing
o monitoring the development of the actual performance
o solving unexpected problems
• maintenance
o the impact of the development of the external factors?
o impact of software updates?

The Making of an Analysis

Analysis is mostly collecting information

• neccessary for large software

o participants: managers, designers, programmers
o an analyst is usually unfamiliar with the subject
o others know nothing of the analysis

Analysis execution is task-dependent

A possible analysis plan

• overview
• modeling
• experimentation
• interpretation

Overview
• Defining the system under study
o consists of what, does what
o prerequisite for systematic inspection
o often includes the hardware
• Performance goals
o important!
 bad: find out if the new system is a good one
o
good: response time upper limit of C for input B, under the
o
circumstances A.
o goal cannot be biased
• design of analysis
o how to do analysis
o neccessary tools
o reports produced

Modeling
Modeling aims to divide execution to a structure explaining performance.

• I/O
program input
o
program output
o
difficult inputs
o
typical inputs
o
• measures, parameters, factors
o how to measure performance
o accurate methods!
o factors involved
o which factors to model
• models
o input model
o execution structure
o execution resources
o from hardware viewpoint

Measurements
Practical measurements are involved in many analyses

• existing software
o real software in real environment
o embedded products
o non-interference of measurements
• software being designed/developed
o few packages are built from scratch
o performance prototypes
o connection to the real software
o change management
• measurement planning
o beforehand!
o statistical confidence
o equipment usability, measurement organization
o measurement errors
 • monitors
o hardware, software, hybrid
o tracing
o sampling

Interpretation
model solving

• validation
o does the model correspond to the reality?
• tools
• simulation
o verification
• error bounds

results

• raw data not usable

• clear results neccessary

Example: a Database Server

Definition of the system

• DB server
• interface: queries through the network
• hardware
o CPU
o three disk drives
• an old method of optimizing queries
o could be replaced by a new one
o halves the needed CPU time
o 50% less traffic on main disk

Example: Overview
Performance goal

• improvement of response times by using the new method

• on a typical load

Analysis design

• find out the typical load

• model the program
 • model the execution environment
• measure load
• predict improvement by using the model

Example: Typical Load

System monitored by the built-in DB statistics.

Statistics could be grouped as follows:

UC - Updates for customer data

UO - Updates for order data
UI - Updates for inventory data
QC - Queries for customer data
QO - Queries for order data
QI - Queries for inventory data
OT - All other transactions

Choice of measures

• response time through the network from a request to a reply

• usage of hardware (for validation)

Example: software model

The software was modeled by

• reading the source

• discussing with the designers
• a large amount of the work!

The model enabled the solving of the DB table usage per service type

UC UO UI QC QO QI OT
Table access count
Tbl_Cust 2 1 1
Tbl_Ord 2 1
Tbl_Ord_Row 12 30 6 15
Tbl_Inv 2 1
System data access cnt 3

Example: System model

Environment was modeled by using an open queuing network
Example: Measurements
The system was measured for an hour

• typical load

Load figures from the DB monitor

UC UO UI QC QO QI OT
Number of transactions 5 3010 2460 300 4000 6000 2000
CPU time (sec) 0.7 630 470 12 350 670 650
Page fault count 16 9510 6000 840 9800 13050 4400
Total I/O counts 15 41000 86000 350 34000 94000 7100

Hardware usage as given by the DB monitor

CPU 85%
DISK A 80%
DISK B 10%
DISK C 40%

Example: Model Parameters

From the measurements, the system model parameters were derived

• could not directly measure the parameters

• analytical derivation

Service requests

UC UO UI QC QO QI OT
CPU 0.025 0.013 0.006 0.009 0.008 0.006 0.058
diskA 0.018 0.011 0.013 0.014 0.013 0.012 0.000
diskB 0.000 0.000 0.021 0.000 0.000 0.019 0.023
diskC 0.033 0.033 0.033 0.033 0.033 0.033 0.033
Branching probabilities

UC UO UI QC QO QI OT
diskA 0.323 0.771 0.847 0.208 0.699 0.782 0.0
diskB 0.323 0.771 0.903 0.208 0.699 0.834 0.484
diskC 0.839 0.945 0.972 0.791 0.913 0.947 0.839

Example: Baseline Model

The model was solved by a steady-state simulation

• warmup 5000sek
• batch 500sek

Response times (90% confidence)

UC 0.184 .. 0.544 sek
UO 2.370 .. 3.019 sek
UI 4.491 .. 5.788 sek
QC 0.558 .. 0.716 sek
QO 1.479 .. 1.823 sek
QI 2.354 .. 2.962 sek
QT 1.091 .. 1.288 sek

Hardware usage (90% confidence)

CPU 0.833 .. 0.861

Disk A 0.799 .. 0.823
Disk B 0.098 .. 0.103
Disk C 0.393 .. 0.405

Consistent enough with reality

Example: Comparison
Change of parameters in accordance with the improvements from the new
optimization method

• comparison
• paired-T

Improvement of response times

UC 0.025 .. 0.794 sek
UO 1.811 .. 2.716 sek
UI 3.612 .. 5.417 sek
QC 0.339 .. 0.566 sek
QO 1.134 .. 1.620 sek
QI 1.867 .. 2.718 sek
QT 0.692 .. 0.984 sek
Response times

UC 0.022 .. 0.151 sek

UO 0.431 .. 0.448 sek
UI 0.661 .. 0.689 sek
QC 0.163 .. 0.193 sek
QO 0.273 .. 0.289 sek
QI 0.366 .. 0.382 sek
QT 0.361 .. 0.381 sek