Software Quality,Software

Testing, and Risk Management

Dr. Huma Hayat Khan

What is Quality?
Quality – developed product meets it’s specification

Problems:

• Development organization has requirements exceeding customer's

specifications (added cost of product development)

• Certain quality characteristics can not be specified in unambiguous terms

(i.e. maintainability)

• Even if the product conforms to it’s specifications, users may not consider
it to be a quality product (because users may not be involved in the
development of the requirements)
What is Quality Management?

Quality Management – ensuring that required level of

product quality is achieved

• Defining procedures and standards

• Applying procedures and standards to the product and
process
• Checking that procedures are followed
• Collecting and analyzing various quality data

Problems:

• Intangible aspects of software quality can’t be standardized (i.e elegance

and readability)
What are SQA, SQP, SQC, and SQM?
Software Quality includes all 4 elements…
• Software Quality Assurance – establishment of network of
organizational procedures and standards leading to high-
quality software
2. Software Quality Planning – selection of appropriate
procedures and standards from this framework and adaptation
of these to specific software project
3. Software Quality Control – definition and enactment of
processes that ensure that project quality procedures and
standards are being followed by the software development
team
4. Software Quality Metrics – collecting and analyzing quality
data to predict and control quality of the software product
being developed
Software Quality Assurance
Software Quality Assurance Techniques
• Auditing- Investigation of work products and its
related info to determine if the standard processes
are followed or not.
• Reviewing- A meeting in which the SW product is
examined by both the internal and external
stakeholders
• Walkthroughs-It is used to review documents with
peers, managers, and fellow team members who are
guided by the author of the document to gather
feedback and reach a consensus. A walkthrough can
be pre-planned or organised based on the needs.
• Inspections- A formal review that evaluates and
perform the static testing to find bugs and avoid
defect growth.
CMMI-Software Quality Standard
Capability Maturity Model Integration

Capability Maturity Model Integration (CMMI) is a successor of CMM and is

a more evolved model that incorporates best components of individual
disciplines of CMM like Software CMM, Systems Engineering CMM, People
CMM, etc.

Since CMM is a reference model of matured practices in a specific discipline,

so it becomes difficult to integrate these disciplines as per the requirements.

This is why CMMI is used as it allows the integration of multiple disciplines as

and when needed.
CMMI Model- Maturity Levels
Maturity level 1 : Initial
• processes are poorly managed or controlled.
• unpredictable outcomes of processes involved.
• ad hoc and chaotic approach used.
• No KPAs (Key Process Areas) defined.
• Lowest quality and highest risk.

Maturity level 2 : Managed

• requirements are managed.
• processes are planned and controlled.
• projects are managed and implemented according to their documented plans.
• This risk involved is lower than Initial level, but still exists.
• Quality is better than Initial level.

Maturity level 3 : Defined

• processes are well characterized and described using standards, proper
procedures, and methods, tools, etc.
• Medium quality and medium risk involved.
• Focus is process standardization.
Maturity level 4 : Quantitatively managed
• quantitative objectives for process performance and quality are set.
• quantitative objectives are based on customer requirements, organization
needs, etc.
• process performance measures are analyzed quantitatively.
• higher quality of processes is achieved.
• lower risk

Maturity level 5 : Optimizing

• continuous improvement in processes and their performance.
• improvement has to be both incremental and innovative.
• highest quality of processes.
• lowest risk in processes and their performance.
 Software Testing

■ Many IT professionals
think of testing as a
stage that comes near
the end of IT product
development
Software Testing
• It is process to evaluate the functionality of the SW
application with an intent to find whether the developed SW
met the specified requirements or not and to identify the
defects to ensure that the product is defect free in order to
produce a quality software.

• Software Testing
• Manual Testing
• Automation Testing (via tools; Selenium, Catalan

• Software Testing Methods

• Static Testing- it is based on examination of a number of
documents, namely requirement documents, software
models, requirements documents and source code. It
includes code review, inspection, walkthroughs.
• Dynamic Testing- It involves actual program execution in
order to expose possible program failures. The
behavioural and performance properties of the program
are also observed.
Software Testing Types
• White box testing- it is also called Glass box, Clear
box, structural testing. It is based on internal code
structure of the application. The test cases are
designed and executed. This testing is usually
done at unit level.

• Black box testing- also called

behavioural/specification/input-output based
testing. It is testing without looking at code.

• Grey box testing- combination of black box and

white box testing. Tester for this type of testing
should have access to design documents. It helps
to generate better test cases.
Testing Levels

• Unit testing
• Integration testing
• System testing
• Acceptance testing
Cont..
Unit Testing

• It is done to check whether the individual modules

of the source code are working properly.

• It is module or component testing

Cont…
Integration Testing

• It is the process of testing the connectivity or data

transfer between unit tested modules.

• It is also called string testing.

• It is sub-divided into Top-down, Bottom-up, and
sandwich approach.

• This process is carried out using dummy programs

called stubs and drivers.

• Stub and drivers do not implement the entire

programming logic of the software module but just
simulate data communication with the calling module.
Cont…
System Testing- End to End Testing

• Testing a fully integrated application is also called

end to end scenario testing.

• To ensure that the software works in all intended

target systems

• Verify thorough testing every input in the

application to check for desired output.

• Testing of the user experience with the application

Cont…
Acceptance Testing

• To obtain customer sign-off so that software can be delivered

and payments received.

• Types of acceptance testing are

• Alpha-It is a testing done by in-house development or QA
team. Its main purpose is to find SW bugs not found
before.
• Beta-It can be called as pre-release testing.It can be
conducted by limited number of end-users before the
final delivery.
• Gamma-It is the final stage of testing before SW release.
It makes sure that the product is ready for the market
release according to all the specified requirements.
Risk Components and Drivers
■ The U.S. Air force has published an excellent
guideline for software risk identification and
abatement.
■ Their approach requires project manager to
identify risks drivers that affect software risk
components
■ Following are the Risk Components
○ Performance
○ Cost
○ Support
○ Schedule
Risk Components and Drivers
■ These risk components are defined as follows in
context of these guidelines:
○ Performance risk
■ The degree of uncertainty that the product will meet its
requirements and be fit for its intended use
○ Cost risk
■ The degree of uncertainty that the project budget will be
maintained
○ Support risk
■ The degree of uncertainty that the resultant software will
be easy to correct, adapt and enhance
○ Schedule risk
■ The degree of uncertainty that the project schedule will be
maintained and that the product will be delivered on time
Risk Components and Drivers
■ The impact of each risk driver on the risk
component is determined. This impact can
be any of these four impact categories
○ Negligible
○ Marginal
○ Critical
○ Catastrophic
Risk Projection
■ Risk Projection is also known as Risk Estimation

■ Risk projection rates each risk in two ways:

○ the probability that the risk is real and
○ the consequences of the problems associated with the
risk, should it occur

■ 4 Risk projection steps performed by project planner

along with other managers & tech staff
1. Establish a scale that reflects the perceived likelihood of a
risk
2. Delineate the consequences of the risk
3. Estimate the impact of the risk on the project and the
product
4. Note overall accuracy of the risk projection so that there
will be no misunderstandings
Risk Projection
■ The aim of these steps is to eventually
determine prioritization of the risk

■ By prioritizing risks, the software team can

allocate resources where they will have the
most impact.
 Developing a Risk Table
■ A risk table provides a project manager with a simple
technique for risk projection (estimation)

■ The risk table can be implemented as a spread sheet

model (enabling easy manipulation & sorting of
entries)

■ The table has the following content:

■ 1st column: list of all risks
■ 2nd column: categorization of risks, e.g., PS
(Project Size risk), BU (Business risk)
■ 3rd column: probability of risk occurrence
■ 4th column: risk impact category
○ determined by averaging the impact categories of the 4
risk components – performance, support, cost, schedule)
Developing a Risk Table
■ After the first four columns are completed, the table is sorted by
probability and by impact.

■ High probability and high impact risks move to the top of the
table and low ones drop to bottom

■ Project Manager studies the resultant (1 st order risk prioritization)

table and defines a cutoff line.

■ Risks above the cutoff line must be managed

■ Risks below cutoff line are reevaluated for 2nd order prioritization.

■ 5th column: contains a pointer into a Risk Mitigation, Monitoring

and Management Plan (RMMM) or alternatively, a collection of
risk information sheets developed for all risks above the cutoff
What is RMMM?
■ RMMM = Risk Mitigation, Monitoring and
Management

■ An effective strategy to deal with risks must

consider issues such as:
○ Risk avoidance (risk mitigation)
○ Risk monitoring
○ Risk management and contingency planning

■ RMMM steps incur additional project cost

Example Scenario of a RISK…
■ Scenario
○ Assume that high staff turnover is a project risk r1.
○ Based on past history, the likelihood l1, of high turnover is
estimated to be 70%
○ The impact of r1, is projected as critical
○ So, high turnover will have a critical impact on project cost
and schedule

■ Steps to mitigate r1:

○ Meet the staff to find the causes of turnover (poor working
conditions, low pay, etc.)
○ Try to reduce the causes of turnover (if possible) before
project starts
○ Once project starts, assume turnover will occur and
develop techniques to ensure continuity when people leave
Cont…
○ Organize project teams so that information about each
development activity is widely dispersed.
○ Define documentation standards & establish mechanisms
to ensure that documents are timely developed.
○ Conduct peer reviews of all work.
○ Assign backup staff member for every critical technologist.

■ Risk Monitoring for r1:

As project proceeds, risk monitoring activities commence
to find indications whether the risk is becoming more or less likely.
Following factors are considered for r1:
○ General attitude of team members based on project
pressures.
○ The degree to which the team is jelled.
Cont…
○ Interpersonal relationships among team members.
○ Potential problems with compensation and benefits.
○ The availability of jobs within or outside the company.

■ Risk Management & Contingency Planning:

This stage comes into play when mitigation efforts have
failed and risk has become a reality.
○ Considering again scenario for r1.
○ The project is underway and a number of people announce
that they will be leaving.
○ If mitigation strategy has been followed, backup is
available, information is documented and knowledge is
dispersed across the team.
Cont…
○ Project manager may refocus resources to those functions
that are fully staffed and re-adjust schedule accordingly.
○ The newcomers can “get up to speed” in the mean time.
○ Individuals who are leaving are asked to stop all work and
spend their last weeks in “knowledge transfer mode”.
○ This may include
■ video-based knowledge capture,
■ development of “commentary documents”,
■ And / or meeting with other team members who will stay on the
project team
Cont…
■ Cost/benefit analysis:
○ Since RMMM steps incur additional cost, the
project managers & planners must consider if the
benefit of these steps are outweighed by costs
associated in implementing them.

○ If risk management steps are projected to increase

costs by 5% and duration by only 3%, then there is
no harm in putting them in place.

○ If these steps increase both project cost and

duration by 15%, then they may not be undertaken.
Risk Management
■ For a large project, 30 to 40 risks can be identified. If
between 3 & 7 risk management steps are identified for
each risk, risk management may become a project itself!

■ For this reason, we adapt the Pareto 80-20 rule to software

risks
○ Experience indicates that 80% of overall project risk (potential
for failure) can be accounted for by 20% of the identified risks
(the critical 20 risks with highest project priority).
The RMMM Plan
■ The RMMM plan documents all work performed as
part of risk analysis.
■ It is used by project manager as part of overall
project plan.
■ Some software teams do not develop formal RMMM
document. Rather, each risk is documented
individually using a risk information sheet (RIS).
■ In most cases, RIS is maintained using a database
system so that creation & information entry, priority
ordering, searches, and other analysis may be
accomplishes easily.
The RMMM Plan
■ RIS contains the following information
○ Risk ID, Date, Probability & Impact
○ Description
○ Mitigation/monitoring
○ Management/Contingency Plan/trigger
○ Current Status
○ Originator & Assigned (to whom) information
■ Once RMMM has been documented & project has
begun, risk mitigation and monitoring steps
commence.