IME 671A: Software Project Management

2020-21 II Sem.

Chapter 11: SOFTWARE RELIABILITY AND QUALITY MANAGEMENT

Submitted by: Group 7

Submitted to:
Dr. Subhas C. Misra, Akash Singhal 201250007
PhD (Carleton), PDF (Harvard)
ADVISOR, Indo Canada Education Council Neelanjan Ghosh 20125042
PROFESSOR, IME Department, IIT Kanpur Swarn Kumar 20125069
NSERC Fellow 2008, Harvard University, USA Upneet Gandhi 20125073
Former Visiting Scientist, Harvard University, USA
Vaibhav Goyal 20125075
 Q. Define the terms software reliability and software quality.

Software Reliability:
It means Operational reliability. It is described as the ability of a system or component to perform its
required functions under static conditions for a specific period.
Software reliability is also defined as the probability that a software system fulfils its assigned task in a
given environment for a predefined number of input cases, assuming that the hardware and the input
are free of error.

Software quality:
Here the product is defined in term of its fitness of purpose. That is, a quality product does precisely
what the users want it to do. For software products, the fitness of use is generally explained in terms
of satisfaction of the requirements laid down in the SRS document. Although "fitness of purpose" is a
satisfactory interpretation of quality for many devices such as a car, a table fan, a grinding machine,
etc. for software products, "fitness of purpose" is not a wholly satisfactory definition of quality.
Q. Identify the factors which make the measurement of software
reliability a much harder problem than the measurement of hardware
reliability.
Software Reliability Hardware Reliability
Reflects the design perfection. Reflects the manufacturing perfection.
Concerned with design faults which are harder to Concerned mostly with physical faults.
visualize, classify, detect, and correct.
With the increase of systems complexity and the Relies on the analysis of stationary processes,
introduction of design faults in software, it becomes because only physical faults are considered.
unsuitable to address non-stationary phenomena
such as reliability growth or reliability decrease
experienced in software.

Software fail due to bugs. Hardware fails mostly due to wear and tear.
Error is tracked down and either the design or the Either replace or repair the failed part
code is changed to fix the bug.
Reliability may either increase or decrease after After repairing, reliability would be maintained at the
repairing. level that existed before the failure occurred.
Q. Through a simple plot explain how the reliability of a software product
changes over its lifetime. Draw the reliability change for a hardware product
over its life time and explain why the two plots look so different.

The software product show the highest failure ra

just after purchase and installation. As the system
is used, more and more errors are identified and
removed resulting in reduced failure rate.
This error removal continues at a slower pace
during the useful life of the product. As the
software becomes obsolete no more error
correction occurs and the failure rate remains
unchanged.
Q. Through a simple plot explain how the reliability of a software
product changes over its lifetime. Draw the reliability change for a
hardware product over its life time and explain why the two plots
look so different.

• For a hardware component the failure rate is

initially high, but decreases as the faulty
components identified are either repaired or
replaced. The system then enters its useful
life, where the rate of failure is almost
constant.

• After sometime (called product life time ) the

major components wear out, and the failure
rate increases. The initial failures are usually
covered through manufacturer’s warranty.
 Q. What do you understand by a reliability growth model?

A reliability growth model is a model of how the system reliability changes over time during the
testing process. As system failures are discovered, the underlying faults causing these failures are
repaired so that the reliability of the system should improve during system testing and debugging. To
predict reliability, the conceptual reliability growth model must then be translated into a
mathematical model.
Q. Explain using one simple sentence each what you
understand by the following reliability measures:
1. POFOD( probability of failure on demand) of 0.001 –
• 1 out of every 1000 service requests would result in a failure

2. A ROCOF of 0.002 of 0.002 –

• There are 2 occurrences of failure for every 1000 unit of run time of the software

3. MTBF of 200 units –

• The average time between two occurrences of failure is 200 units of time. Once a
failure occurs the next one is expected to happen after 200 units of time

4. Availability of 0.998 –
• The system is likely to be available for use 0.998 units of time per 1 unit of time of
observation.
 Q. What is statistical testing? How is it useful during software
development? Explain the different steps of statistical testing.

Statistical testing is a testing process with its objective being to determine the reliability of
the product instead of discovering errors.
Statistical testing allows one to concentrate on testing parts of the system that are most likely
to be used. Therefore, it results in a system that the users can find to be more reliable (than
actually it is!). Also, the reliability estimation arrived by using statistical testing is more
accurate compared to some other methods.
 
The steps involved in statistical testing are -
The first step is to determine the operation profile of the software. The next step is to
generate a set of test data corresponding to the determined operation profile. The third step
is to apply the test cases to the software and record the time between each failure. After a
statistically significant number of failures have been observed, the reliability can be
computed.
 Q. Define three metrics to measure software reliability. Do you
consider these metrics entirely satisfactory to provide measure of the
reliability of a system? Justify your answer.
1. Rate of occurrence of failure (ROCOF): ROCOF measures the • All the above reliability metrics suffer from several
frequency of occurrence of failures. ROCOF measure of a shortcomings as far as their use in software reliability
software product can be obtained by observing the behaviour measurement is concerned.
of a software product in operation over a specified time • One of the reasons is that these metrics are centred on
interval and then calculating the ROCOF value as the ratio of the probability of occurrence of system failures but take
the total number of failures observed and the duration of no account of the consequences of failures. That is, these
observation. reliability models do not distinguish the relative severity
2. Mean time to failure (MTTF): MTTF is the time between two of different failures.
successive failures, averaged over a large number of failures. • In order to estimate the reliability of a software product
3. Probability of failure on demand (POFOD): POFOD measures more accurately, it is necessary to classify various types of
the likelihood of the system failing when a service request is failures, these are generally categorized as – Transient,
made. For example, a POFOD of 0.001 would mean that 1 out Permanent, Recoverable, Unrecoverable (restart needed),
of every 1000 service requests would result in a failure. The and Cosmetic.
reliability of a software product should be determined through
specific service invocations, rather than making the software
run continuously. Thus, POFOD metric is very appropriate for
software products that are not required to run continuously.
 Q. State TRUE or FALSE of the following. Support your answer with
proper reasoning:
(a) The reliability of a software product increases almost linearly, each time a defect gets detected and fixed.
False : Correction of different defects contributes differently to the reliability growth of the software.

(b) As testing continues, the rate of growth of reliability slows down representing a diminishing return of reliability growth
with testing effort.
True : As errors are repaired, the average improvement to the product reliability per repair decreases.
 
(c) Modern quality assurance paradigms are centred on carrying out thorough product testing.
True
 
(d) An important use of receiving an ISO 9001 certification by a software organisation is that it can improve its sales efforts
by advertising its products as conforming to ISO 9001 certification.
False : ISO mandates that a certified organisation can use the certificate for corporate advertisements but cannot use the
certificate for advertising any of its products.
 
(e) If an organisation assessed at SEI CMM level 1 has developed one software product successfully, then it is expected to
repeat its success on similar products.
False : At level 1 maturity success is not generally repeatable.
 Q. With the help of suitable examples discuss the types of software
organizations to which ISO 9001, 9002, and 9003 standards
respectively are applicable.
● ISO 9001. This standard applies to the organizations engaged in design,
development, production, and servicing of goods. This is the standard that is
applicable to most software development organizations.
● ISO 9002. This standard applies to those organizations which do not design
products but are only involved in production. Examples of this category of
industries include steel and car manufacturing industries who buy the product
and plant designs from external sources and are involved in only manufacturing
those products. Therefore, ISO 9002 is not applicable to software development
organizations.
● ISO 9003. This standard applies to organizations involved only in installation
and testing of the products. Examples of this could be Gas companies.
 Q. During software testing process, why is the reliability growth
initially high, but slows down later on?

Because it is extremely inefficient to make substantial design changes in later stages of

development or after deployment, it is important to identify any problems in design or, more
generally, the likely inability for a system to meet its reliability requirements, during the
design and early development stages. Therefore, careful reliability testing of systems,
subsystems, and components while the design remains in flux is crucial to achieving desired
reliability levels in a cost-efficient way prior to fielding. During development, the program of
test, analyse, fix, and test can be used to identify and eliminate design weaknesses inherent
to intermediate prototypes of complex systems
 Q. If an organization does not document its quality system, what
problems would it face?

● Lack of a document control and management system will prevent a business from getting better at
what it does, because it has no formal starting point. Total quality management sets out
requirements for the development, approval, and periodic review of procedures that govern a
business’ activities.
● Without such documents, the workforce is left to operate as it sees fit, with no accountability and
no recognized method of manufacture or product control.
● Long-term employees who leave the company take irreplaceable information with them, leaving
others to figure it out and the business to suffer in terms of quality.
● It is also virtually impossible to pinpoint the cause of a defect in the process develop corrective
actions without formal procedures, leaving a company to founder with excessive costs for scrap.
● Without a quality management system, a business will not be asked to bid on or be granted a
contract, limiting the ability of the business to expand its outreach and position.
 Q. What according to you is a quality software product?

● Portability: A software product is said to be portable, if it can be easily made to work in

different hardware and operating system environments, and easily interface with external
hardware devices and software products.
● Usability: A software product has good usability, if different categories of users (i.e., both
expert and novice users) can easily invoke the functions of the product.
● Reusability: A software product has good reusability, if different modules of the product can
easily be reused to develop new products.
● Correctness: A software product is correct, if different requirements as specified in the SRS
document have been correctly implemented.
● Maintainability: A software product is maintainable, if errors can be easily corrected as and
when they show up, new functions can be easily added to the product, and the functionalities
of the product can be easily modified, etc.
Q. Discuss the stages through which the quality system paradigm and
the quality assurance methods have evolved over the years?
 Q. Which standard is applicable to software industry, ISO 9001, ISO
9002, or ISO 9003?

ISO 9001
Sets out the requirements for an organization whose business processes range all the
way from design and development, to production, installation and servicing.
Q. In a software development organization, identify the persons
responsible for carrying out the quality assurance activities. Explain
the principal tasks they perform to meet this responsibility?

Responsibilities for Quality Assurance Engineers

● Create quality measurements to track improvement in products

● Execute quality improvement testing and activities
● Develop quality assurance standards and company processes
● Adhere to industry quality and safety standards
● Ensure products meet customer expectations and demand
● Create reports documenting errors and issues for fixing
● Work closely with the development team to improve existing products
● Maintain standards for reliability and performance of production
 Q. Suppose an organization mentions in its job advertisement that it has been
assessed at level 3 of SEI CMM, what can you infer about the current quality practices
at the organization? What does this organization have to do to reach SEI CMM level
4?

● At this level, the processes for both management and development activities are defined and
documented. There is a common organization-wide understanding of activities, roles, and
responsibilities. The processes though defined, the process and product qualities are not
measured. At this level, the organization builds up the capabilities of its employees through
periodic training programs. Also, review techniques are emphasized and documented to
achieve phase containment of errors.
Key areas on which to focus to take an organization from one level of maturity to the next-
● Focus on - Product and process quality
● Key Process areas - Quantitative process metrics Software quality management
Q. Suppose as the president of a company, you have the choice to either
go for ISO 9000 based quality model or SEI CMM based model, which
one would you prefer? Give the reasoning behind your choice.

It depends on where the company is on its maturity journey. This can be done by a
diagnostic study to really find out how the firm is doing on the following 3 factors
★ Product or Service Quality
★ Process Performance
★ People Performance.
A survey with data on Client returns/complaints and In-process rejections would
decide what is to be done first.
If the product quality is not comparable to the competitors, then ISO 9000 is the
preferred choice but if the company is on a benchmark level with relatively excellent
quality but have problems with sustaining the process performance indicators SEI
CMM is more feasible option to go for.
 Q. What do you understand by total quality management (TQM)? What
are the advantages of TQM? Does ISO 9000 standard aim for TQM?

Total quality management (TQM) consists of organization-wide efforts to "install and make
permanent climate where employees continuously improve their ability to provide on
demand products and services that customers will find of particular value.
The Advantages of TQM are as follows.
TQM leads to better products at lower cost. The focus on using high quality information to
improve processes reduces waste and saves time, leading to reduced expenses that can be
passed along to clients in the form of lower prices. Companies that successfully implement
TQM are able to reduce variability, providing the consistency that customers value. This
creates customer loyalty and earns their continued business.The emphasis on engagement
at all levels leads to employee engagement, which reduces turnover and saves money on
training and mistakes due to inexperience.
ISO 9000 is a quality assurance management system that is rapidly becoming the world
standard for quality.
Q. What are the principal activities of a modern quality system?

The principal activities of a modern quality system are as follows -

★ Auditing of projects to check if the processes are being followed.
★ Collect process and product metrics and analyze them to check if
quality goals are being met.
★ Review of the quality system to make it more effective.
★ Development of standards, procedures, and guidelines.
★ Produce reports for the top management summarizing the
effectiveness of the quality system in the organization.
Q. In a software development organization whose responsibility is it to
ensure that the products are of high quality? Explain the principal tasks
they perform to meet this responsibility.

A quality system is the responsibility of the organization as a whole. However,

every organization has a separate quality department to perform several
quality system activities. The major people responsible for ensuring high
quality are -
Project Sponsor, Subject Matter Experts, Product Owner, Project Manager,
Technical Lead, Software Developers, Software Testers and User Acceptance
Testers
As already discussed the principal tasks performed by them are
Auditing of Projects, Review of Quality System, Development of Standards &
Procedures, Produce Reports for Top Management and collect product &
process metrics.
 Q. What do you understand by repeatable software development?
Organizations assessed at which level SEI CMM maturity achieve
repeatable software development?

Repeatable Software Development is the Second Level in SEI Capability

Maturity Model
At this level, the basic project management practices such as tracking
cost and schedule are established. Configuration management tools are
used on items identified for configuration control. Size and cost
estimation techniques such as function point analysis, COCOMO, etc.
are used. The necessary process discipline is in place to repeat earlier
success on projects with similar applications. Though there is a rough
understanding among the developers about the process being followed
the process is not documented.
 Q. What do you understand by key process area (KPA), in the context of
SEI CMM? Would there be any problem if an organisation tries to
implement higher level SEI CMM KPAs before achieving lower level KPAs?
Justify your answer using suitable examples.

The Key Process Areas (KPAs) capture the focus areas of a level in SEI CMM
Except for level 1, each maturity level is characterized by several KPAs that
indicate the areas an organization should focus to improve its software
process to this level from the previous level. Each of the focus areas
identifies a number of key practices or activities that need to be
implemented.
Implementing a higher level KPA is not feasible. For example: Trying to
implement a defined process (level 3) before a repeatable process (level 2)
would be counterproductive as it becomes difficult to follow the defined
process due to schedule and budget pressures.
Q. What is the Six Sigma quality initiative? To which category of industries
is it applicable? Explain the Six Sigma technique adopted by software
organization with respect to the goal, the procedure, and the outcome.

Six Sigma is a quality management methodology used to help businesses improve current processes,
products or services by discovering and eliminating defects. The goal is to streamline quality control
in manufacturing or business processes so there is little to no variance throughout.
Six Sigma can be used for any activity that is concerned with cost, timeliness, and quality of results.
Therefore, it is applicable to every industry.
GOALS
•Find and eliminate causes of defects and errors
•Reduce cycle times and cost of operations
•Improve productivity
•Better meet customer expectations
•Achieve higher asset utilization
•Provided an improved return on investment
Q. What is the Six Sigma quality initiative? To which category of industries
is it applicable? Explain the Six Sigma technique adopted by software
organization with respect to the goal, the procedure, and the outcome.

METHODOLOGY
This is accomplished through the use of two Six Sigma sub-methodologies—DMAIC and
DMADV.
❏ The Six Sigma DMAIC process (Define, Measure, Analyze, Improve, Control) is an
improvement system for existing processes falling below specification and looking for
incremental improvement.
❏ The Six Sigma DMADV process (Define, Measure, Analyze, Design, Verify) is an
improvement system used to develop new processes or products at Six Sigma quality
levels.
Q. What is the difference between process metrics and product metrics?
Give four examples of each.
Q. A software system is composed of 50 modules. Each module is guaranteed to have a
reliability R not less than 0.999. What would be the best case and reliability of the entire
system? What should be the reliability of the modules if we require that the system exhibits
reliability equal to 0.99999?

In series:
Reliability of system = Product of Reliability of each component

In parallel:
Reliability of system = 1 – [Product of Reliability of each
component] Reliability of system given = 0.99999
N^50 = 0.99999
N= 0.99999^0.02 = 0.99999979999
Since there are 50 components:
Reliability of the system in series = 0.999^50 = 0.95120562819

Reliability of the system in parallel = 1 – [0.999^50] = 0.0487943718

Hence the best Reliability is in series = 0.999^50 = 0.95120562819

Q. Suppose you want to buy a certain software product and you have kept a purchase precondition that the vendor
must install the software, train your manpower on that, and maintain the product for at least a year, only then would
you release the payment. Also, you do not foresee any maintenance requirement for the product once it works
satisfactorily. Now, you receive bids from three vendors. Two of the vendors quote Rs. 3 lakhs and Rs. 4 lakhs,
whereas the third vendor quotes Rs. 10 lakhs saying that the prices would be high because they would be following a
good development process as they have been assessed at the Level 5 of SEI CMM. Discuss how you would decide
whom to award the contract.

The assessment on the cost effectiveness would be required to be carried out for awarding the contract.
The vendors providing service for Rs. 3 lakhs and Rs. 4 lakhs respectively would not differentiate much in terms of product
functionality and value proposition but the vendor quoting Rs. 10 lakhs (having been assessed at the Level 5 of SEI CMM) would
make a difference in the right organization.

Highly systematic and measured approach to software development suits large organizations dealing with negotiated software,
safety-critical software, etc. For those large organizations, SEI CMM model is perfectly applicable. But small organizations typically
handle applications such as Internet, e-commerce, and are without an established product range, revenue base, and experience on
past projects, etc. For such organizations, a CMM-based appraisal is probably excessive. These organizations need to operate
more efficiently at the lower levels of maturity. For example, they need to practice effective project management, reviews,
configuration management, etc.

SEI CMM provides a list of key areas on which to focus to take an

organisation from one level of maturity to the next. Thus, it provides a way
for gradual quality improvement over several stages. Each stage has been
carefully designed such that one stage enhances the capability already built
up. For example, trying to implement a defined process (level 3) before a
repeatable process (level 2) would be counterproductive as it becomes
difficult to follow the defined process due to schedule and budget pressures.
Q. Explain the importance of software configuration management in modern quality paradigms such
as SEI CMM and ISO 9001. An organisation not using any configuration management tool can qualify
for which SEI CMM level(s)?

Both the CMM and the ISO 9001 standards were designed to improve organisational processes. Paulk (1995a,b) and Rozman et al.
(1997) have performed comparison of the models clearly where there is a strong correlation between ISO 9001 and CMM, although
some issues in ISO 9001 are not covered in CMM and vice versa. The two documents are the explicit emphasis of the CMM on
continuous process improvement. ISO 9001 addresses only the minimum criteria for an acceptable quality system. Another
difference is that the CMM focuses strictly on software, whereas ISO 9001 has a much broader scope that encompasses system,
software, processed materials and services. The biggest similarity between the two documents is their bottom line: ‘Say what you
do: do what you say’. ISO 9001 requires documentation that contains instructions or guidance on what should be done or how it
should be done. CMM shares emphasis on processes that are documented and practised as documented. From the above
statements, comparison of ISO 9001 and CMM is evident. Mapping can also be oriented towards users who are already using CMM
and are additionally interested in ISO 9001:2000 compliance or towards users concurrently addressing CMM process improvement
and ISO 9001 compliance.
Q. SEI CMM and ISO 9001 Mapping
Q. An organisation not using any configuration management tool can qualify for
which SEI CMM level(s)?

An organisation not using any configuration management tool can be limited to either Level 2 or Level
Q. Discuss the salient features of the organisational reporting structure of the SQA
group as recommended by ISO 9001. What is the rationale behind having such a
reporting structure?
The salient features recommended by ISO 9001 cover all the steps related to software development,
these recommendations are as follows:

•Document control: All documents concerned with the development of a software product should
be properly managed, authorised, and controlled. This requires a configuration management system
to be in place.

•Planning: Proper plans should be prepared and then progress against these plans should be
monitored. Review: Important documents across all phases should be independently checked and
reviewed for effectiveness and correctness.

•Testing: The product should be tested against specification.

•Organisational aspects: Several organisational aspects should be addressed e.g., management

reporting of the quality team.
Q. Suggest two development organisations for whom SEI capability
model is not likely to be appropriate?

• SEI CMM model because it is highly systematic and uses measured approach to software
development generally only suits large organisations dealing with negotiated software,
safety-critical software, etc. But for small organisations – which typically handle
applications such as small Internet, e-commerce applications, and often are without an
established product range, revenue base, and experience on past projects, etc. – a CMM-
based appraisal is probably excessive. These organisations tend to operate more
efficiently at the lower levels of maturity.
 
• Another issue with SEI capability model is that thicker documents, more detailed
information, and longer meetings are considered to be better. This is in contrast to the
principles of software economics—reducing complexity and keeping the documentation
to a minimum without sacrificing the relevant details.
 Q. What do you understand by Key Process Area (KPA), in the context of
SEI CMM?

• Key Process Area identifies a cluster of related

activities that, when performed collectively,
achieve a set of goals considered important for
enhancing process capability.
• The key process areas have been defined to
reside at a single maturity level.
• The key process areas are building blocks that
indicate the areas an organization should focus
on to improve its software process.
• Key process areas identify the issues that must
be addressed to achieve a maturity level.

The Key Process Areas of CMM by Maturity Level

Q. What does the quality parameter “fitness of purpose” mean in the context of
software products? Why is this not a satisfactory criterion for determining the
quality of software products?

Traditionally, the quality of a product is defined in terms of its fitness of

purpose. That is, a good quality product does exactly what the users
want it to do, since for almost every product, fitness of purpose is
interpreted in terms of satisfaction of the requirements laid down in the
SRS document.

Although “fitness of purpose” is a satisfactory definition of quality for many products such as a car, a table
fan, a grinding machine, etc.—“fitness of purpose” is not a wholly satisfactory definition of quality for
software products. To give an example of why this is so,
consider a software product that is functionally correct. That is, it correctly performs all the functions that
have been specified in its SRS document. Even though it may be functionally correct, we cannot consider it to
be a quality product, if it has an almost unusable user
interface. Another example is that of a product which does everything that the users wanted but has an
almost incomprehensible and unmaintainable code. Therefore, the traditional concept of quality as “fitness of
purpose” for software products is not wholly satisfactory.
Q. Can reliability of a software product be determined by estimating the number of latent defects
in the software? If your answer is “yes”, explain how reliability can be determined from an
estimation of the number of latent defects in a software product. If your answer is “no”, explain
why
can’t reliability of a software product be determined from an estimate of the number of latent
defects.

Unfortunately, it is very difficult to characterise the observed reliability of a system in terms of the number of latent
defects in the system using a simple mathematical expression. consider the following. Removing errors from those
parts of a software product that are very infrequently executed, makes little difference to the perceived reliability of
the product. It has been experimentally observed by analysing the behaviour of a large number of programs that 90
per cent of the execution time of a typical . Based on this discussion we can say that reliability of a product depends
not only on the number of latent errors but also on the exact location of the errors. Apart from this, reliability also
depends upon how the product is used, or on its execution profile.

The reasons why software reliability is difficult to measure can be summarized as

follows:
• The reliability improvement due to fixing a single bug depends on where
the bug is located in the code.
• The perceived reliability of a software product is highly observer dependent.
• The reliability of a product keeps changing as errors are detected and
fixed.
 Q. Why is it important for a software development organisation to obtain ISO 9001
certification?

• Confidence of customers in an organisation increases when the organisation qualifies for ISO 9001
certification. This is especially true in the international market. In fact, many organisations awarding
• international software development contracts insist that the development organisation have ISO 9000
certification. For this reason, it is vital for software organisations involved in software export to
• obtain ISO 9001 certification.

• ISO 9001 requires a well-documented software production process to be in place. A well- documented
software production process contributes to repeatable and higher quality of the developed software.

• ISO 9001 makes the development process focused, efficient, and cost-effective.

• ISO 9001 certification points out the weak points of an organisations and recommends remedial action.

• ISO 9001 sets the basic framework for the development of an optimal process and TQM.
Q. Discuss the relative merits of ISO 9001 certification and the SEI CMM based qualit
assessment.
• ISO 9000 is awarded by an international standards body. Therefore,
• ISO 9000 certification can be quoted by an organisation in official
• documents, communication with external parties, and in tender
• quotations. However, SEI CMM assessment is purely for internal use.
• SEI CMM was developed specifically for software industry and therefore
• addresses many issues which are specific to software industry alone.
• SEI CMM goes beyond quality assurance and prepares an organisation
• to ultimately achieve TQM. In fact, ISO 9001 aims at level 3 of SEI
• We identified ISO 9000 and SEI CMM as two
CMM model.
sets of guidelines for setting up a quality 9000
• SEI CMM model provides a list of key process areas (KPAs) on which an series is a standard applicable to a broad
• organisation at any maturity level needs to concentrate to take it from spectrum of industries, whereas SEI CMM model
• one maturity level to the next. Thus, it provides a way for achieving is a set of guideline system. ISO s for setting up
• gradual quality improvement. In contrast, an organisation adopting ISO a quality system specifically addressing the
• 9000 either qualifies for it or does not qualify. needs of the software development
organisations. Therefore, SEI CMM model
addresses various issues pertaining to software
industry in a more focussed manner. For
example, SEI CMM model suggests a 5-tier
structure. On the other hand, ISO 9000 has been
formulated by a standards body and therefore
the certificate can be used as a contract between
externally independent parties, whereas SEI
CMM addresses step by step improvements of
an organisation’s quality practices.
Q. List five salient requirements that a software development organisation must
comply with before it can be awarded the ISO 9001 certificate. What are some of
the shortcomings of the ISO certification process?

We can summarise the salient features all the requirements as follows:

• Document control: All documents concerned with the development of a software
product should be properly managed, authorised, and controlled. This requires a
configuration management system to be in place.
• Planning: Proper plans should be prepared and then progress against these plans
should be monitored.
• Review: Important documents across all phases should be independently checked and
reviewed for effectiveness and correctness.
• Testing: The product should be tested against specification.
• Organisational aspects: Several organisational aspects should be addressed e.g.,
management reporting of the quality team.
Q. List five salient requirements that a software development organisation must
comply with before it can be awarded the ISO 9001 certificate. What are some of
the shortcomings of the ISO certification process?

Some of the shortcoming of the ISO certification process are the following:
• ISO 9000 requires a software production process to be adhered to, but does not guarantee the process to be of high
quality. It also does not give any guideline for defining an appropriate process.
• ISO 9000 certification process is not fool-proof and no international accreditation agency exists. Therefore it is likely that
variations in the norms of awarding certificates can exist among the different accreditation agencies and also among the
registrars.
• Organisations getting ISO 9000 certification often tend to downplay domain expertise and the ingenuity of the developers.
These organisations start to believe that since a good process is in place, the development results are truly person-
independent. That is, any developer is as effective as any other developer in performing any particular software
development activity. In manufacturing industry there is a clear link between process quality and product quality. Once a
process is calibrated, it can be run again and again producing quality goods. Many areas of software development are so
specialised that special expertise and experience in these areas (domain expertise) is required. Also, unlike in case of
general product manufacturing, ingenuity and effectiveness of personal practices play an important part in determining
the results produced by a developer. In other words, software development is a creative process and individual skills and
experience are important.
• ISO 9000 does not automatically lead to continuous process improvement. In other words, it does not automatically lead
to TQM.
Q. Can a program be correct and still not exhibit good quality? Explain.

It is not necessary that a correct program would exhibit good quality.

For example, a program may be correct but the logic may be such that it does not efficiently
utilize the system resources. As a result, the quality of the product may not be too good. A
good quality program should utilize the system resources efficiently and the output should
be produced in a minimum possible time. A good quality software must be delivered on
time and within budget, and should be maintainable.
Q. What do you understand by defect prevention? Explain how defect
prevention can be achieved.
Defect Prevention (DP) is a strategy applied to the software
development life cycle that identifies root causes of defects and
prevents them from recurring. DP has been identified by the
Software Engineering Institute as a level 5 Key Process Area (KPA)
in the Capability Maturity Model (CMM). It involves analyzing
defects encountered in the past and specifying checkpoints and
actions to prevent the occurrence of similar defects in the future.

Software Defect Rate Of Discovery Versus Time

Defect Prevention Strategy for Software Development Process

Thank You