Software Testing & Quality Assurance

Prof. Dr. Tariq Javid

Graduate School of Engineering Sciences & Information Technology
Faculty of Engineering Sciences & Technology
Hamdard University
Spring 2024
Review and Audit Processes

2
 Reviews

• Reviews are a form of static testing where people, rather than tools, analyze
the project or one of the project’s work products, such as a requirements
specification, design specification, database schema, or code
• Primary goal is typically to find defects in that work product before it serves
as a basis for further project activity

3
 Preparation for a Review

• Read work product being reviewed completely and carefully; and reflect on
your understanding of it ~ 1-2 hours
• This process takes longer than reading a blog post or a newspaper article
• You need to take notes as you go about questions you have, problems you have
found, and potential problems you need to check further prior to the review
• You need to check internal and external cross-references for any contradictions

4
 After a Review

• There are three possible outcomes

• Ideal case is that the document is okay as is or with minor changes
• Another possibility is that document requires some changes but not a re-
review
• The most costly outcome—in terms of both effort and schedule time—is
that the document requires extensive changes and a re-review

5
 Extensive Changes and Re-Review

• While this is a costly outcome, it’s less costly than simply ignoring the
serious problems and then dealing with them during component,
integration, system, or—worse yet—acceptance testing

6
 Reviews: Formal vs. Informal

• In an informal review, there are no defined rules, no defined roles, and no

defined responsibilities
• Informal reviews typically find only around 20 percent of defects
• Very formal reviews like inspections can find up to 85 percent of defects
• If something is important, you probably want to have a formal review

7
 Objectives of Review, 1/2

• One common objective is finding defects

• Other objectives are: Building confidence that we can proceed with the item
under review, reducing risks associated with the item under review, and
generating information for the team and for management

8
 Objectives of Review, 2/2

• Another common objective is to ensure uniform understanding of the

document—and its implications for the project—and building consensus
around the statements in the document
• Reviews of test plans, test processes, and tests themselves can also reveal
gaps in coverage

9
 Reviews Usually Precede Dynamic Tests

• Reviews should complement dynamic tests

• Because the cost of a defect increases the longer that defect remains in the
system, reviews should happen as soon as possible
• However, because not all defects are easy to find in reviews, dynamic tests
should still occur

10
 Checklist

• Regardless of the individual author and their skills, there is nothing personal
about locating potential problems and improvements for their work from a
checklist
• The checklists serve as a valuable leveling and depersonalizing tool, both
factually and psychologically

11
 Checklist Formulation

• If there are common templates, style guides, variable naming rules, or

word-usage standards in the company for certain types of work products,
the checklists should reflect those guidelines

12
 Static Analysis as Pre-Requisite for Review

• Verification of guidelines, mentioned in the last slide, can be implemented

in static analysis tools
• In that case, the checklist or the review entry criteria can simply mention
that successful completion of the static analysis is a pre-requisite to review

13
 Checklist Sections

• Typical organizations or systems, specific quality characteristics such as

security and performance are important and should be verified
• Checklists can include specific sections for each such characteristic, along
with good and bad things to watch for
• If these sections are applicable only to certain projects or products, the
checklist should give guidance on when the items in a given section are
important to consider
14
 Important Factors for Checklist

• Product • History
• People • Quality Characteristics
• Tools • Traceability
• Priorities

15
 There is More to Checklist

• One risk is whether it will focus attention on verification (are we building the
product right?) rather than validation (are we building the right product?)
• Checklists will tend to bring your thinking into a verification mindset
• Remember to think outside of the checklist to look at whether the product
will actually solve the end users’ problems.

16
 Software: Architecture vs. Design

• Software architecture refer to higher levels of component relationships,

where the components can be databases, application servers, and other
such coarse-grained elements
• Software design can refer to components at a lower level, such as individual
units or groups of units within a single application

17
 Software Architecture List Elements

• Performance • Security • Conceptual Integrity

• Reliability • Modifiability • Usability
• Availability • Extensibility • Traceability
• Efficiency • Reusability • Ease of Administration
• Scalability • Functionality • Debug-ability and Monitoring
• Development Productivity
18

https://www.codeproject.com/Articles/20467/Software-Architecture-Review-Guidelines
 Example Checklist

• The network is reliable • Topology does not change

• Latency is zero • There is one administrator
• Bandwidth is infinite • Transport cost is zero
• The network is secure • The network is homogenous

19
 General Checklist Items for Code Reviews

• Checklist starts with the structure of the code (evaluated against any
architecture and design specifications to ensure consistency and
completeness)
• Checklist looks at documentation of the code (any applicable style
guidelines—for example, for class declaration and function headers)
• The next category on the checklist is variables
• Computation, programming logic, data structures, loops, conditions
• Programming practices 20
 Brian Marick’s Code Review Checklist

• Question catalog with categories

• • Computation
• Variable declaration • Pointer operation
• Data item and operation on data item • Assignment
• Memory allocation, deallocation, reallocation • Function calls
• File operations • Miscellaneous
21
 Computation Category

• Are parentheses correct? Do they mean what we want them to mean?

• When using synchronization, are we updating variables in the critical
sections together?
• Do we allow division by zero to occur?
• Are floating-point numbers compared for exact equality?

22
 Open Laszlo Code Review Checklist

• Questions are related to the code changes

• Categories: All changes, coding standards, design changes, maintainability,
documentation

23
Triage (5)
Investigation (10)
Hunting (4)
Incident Management (5)
Automation (4)
24
25
26
 Audits

• An independent examination of a work product or set of work products to

assess compliance with specifications, standards, contractual agreements,
or other criteria
• Certification, or third-party assessment (referred to as registration in some
countries), is carried out by an independent organization against a particular
standard

27
 Software Process Assessment

“Software Process Assessment is a disciplined examination of the software processes

used by an organization, based on a process model. The objective is to determine the
maturity level of those processes, as measured against a process improvement road
map. The result should identify and characterize current practices, identifying areas
of strengths and weaknesses, and the ability of current practices to control or avoid
significant causes of poor software quality, cost, and schedule. The assessment
findings can also be used as indicators of the capability of those processes to achieve
the quality, cost, and schedule goals of software development with a high degree of
predictability”
28
 Audit vs. Assessment

• Outcome of an audit is in compliance or not in compliance, or pass or fail

• Software process assessment is not an audit but a review of a software
organization to advise its management and professionals on how they can
improve their operation
• Organization level and project level

29
 Methods

• Internal auditing is performed on a regular basis by the in-house team and is

generally more frequent
• External audits are often performed by a third party with the goal of
obtaining an unbiased report, particularly if the software must comply with
specific policies, licenses, and legislative regulations

30
 Software Audit

• The most versatile purpose of software audits is to look for tools that no
longer contribute to overall software performance or even slow it down
• This is why software audits should be performed on all types of software
• As a result of the examination, software owners gain a better understanding
of the flaws that must be addressed, whether it means replacing a few
features or updating the entire platform

31
https://savvycomsoftware.com/blog/what-is-software-audit-why-do-you-need-it
 More Purposes

• Software project audit’s goal is to detect inactive licenses and subscriptions,

as well as tools that software owners no longer require — and thus help
avoid unnecessary expenses
• Advice on business decisions, for example, upgrade or replace software
• Regular audits assist software owners in avoiding legal issues related to
missing licenses, compliance with essential legal requirements and industry
standards, and risks associated with data breaches
32
 Example

• Managing software for the healthcare industry entails working with a large
amount of sensitive data, such as electronic health records. That is why it is
critical for the service or platform to comply with certain certifications, data
protection laws, and regulations, such as HIPAA in the United States or DPA
in the United Kingdom
• HIPAA = Health Insurance Portability and Accountability Act
• DPA = Data Protection Act
33
 Maintain Software Quality

• With technology constantly evolving, there are always ways to improve

software products, whether in terms of cybersecurity, new features, cloud
computing solutions, or product maintainability
• Software audits assist product owners in making informed decisions about
which problems and updates should be prioritized

34
 Conducting Software Audit, 1/2

• A software audit can be conducted at any stage of the SDLC

• However, it is best to conduct an audit when the software is in its
development or testing phase
• This is because it is easier to identify and fix issues early in the SDLC, rather
than after the software has been released

35
 Conducting Software Audit, 2/2

• Additionally, it can also be conducted when there is a change in the business

requirements, regulatory environment, or technology landscape
• These changes can affect the software product, and a software audit can
help identify any gaps or weaknesses in the product

36
 Types of Software Audit

• Code review
• Infrastructure inspection
• Architecture inspection
• Security audit
• Maintainability audit
• Usability and accessibility audit
37
 Software Audit Checklist, 1/3

• Review the software development process to ensure it meets industry

standards and best practices
• Verify that the software product meets the business requirements and
specifications
• Check for compliance with licensing agreements and intellectual property rights
• Review the software testing process to ensure that it is rigorous and thorough

38
 Software Audit Checklist , 2/3

• Test the software for performance, security, and reliability

• Verify that the documentation is complete and accurate
• Conduct a risk analysis to identify potential risks and vulnerabilities
• Review the change management process to ensure that changes are
controlled and documented

39
 Software Audit Checklist , 3/3

• Verify that the software product is maintainable and scalable

• Prepare a report of findings and recommendations
• What should be included in the results of the software audit?

40
 Audit Report

• Results of the software audit should include a report of findings and recommendations
• Report should provide a summary of the audit scope, objectives, and methodology
• Include a detailed analysis of the findings and recommendations for improvement
• Report should be easy to understand and provide actionable recommendations
• Include a prioritized list of recommendations, based on their importance and impact on
the software product

41
Metrics and Models in Software Quality
42
Engineering, Second Edition By Stephen H. Kan
43
 Reading

• Section 5.5 Code review exercise from textbook Advanced Software Testing
• https://savvycomsoftware.com/blog/what-is-software-audit-why-do-you-need-it

44