2023/2

Alex Mwotil
§ What is software?
§ ………………………………………………
§ Our lives and work are increasingly reliant on software
§ Hospital
§ School
§ Finance
§ Transport
§ Power grids
§ …..

§ Good vs Bad/Flawed Software

§ “Bad (poor quality) software cost the US economy $2.41 trillion in 2022”
- Synopsys, Inc. (Nasdaq: SNPS)

§ “Each year, 150 Billion Euros is lost in Europe due to bad software quality – hidden bugs”
- Verifysoft Technology (https://www.verifysoft.com/)

§ “Finding and fixing a software problem after delivery is often 100 times more expensive than
finding and fixing it before.”
- Barry Boehm (TRW Emeritus Professor, USC)
§ Failure of touch screens
due to a memory limit
issue
§ Model S sedan and Model
X SUVs from 2021 and
2022
§ Model 3 cars
§ Model Y SUVs from 2022
§ Flight ET 302 from Addis Ababa to
Nairobi
§ Boeing 737 Max 8
§ “Faulty Boeing software played a
role in the crash”
§ Ethiopian Officials

§ Aircrafts grounded for 20 months

§ Estimated cost (fines, compensation, legal fees &
other indirect costs)
§ $60 billion
§ Microsoft Vaccination
Management Platform
§ Registration, scheduling and
tracking of COVID-19 vaccination

§ Software bug in the platform

§ Duplicate appointments
§ Blocked users
§ Crash
§ 3 Days
§ Massive credit-card attack
with over 41M accounts
affected
§ All Target stores (1800)
affected in the US and
Canada (124)
§ Started on Black Friday
2013 – Dec. 15
§ What is good|quality software?
§ Define the metrics and determine the quality of the
software product
§ Dependability
§ Performance
§ Scalability
§ Availability
§ Usability
§ …..

§ Key to good software

§ Verification and Validation (V&V)

§ Course explores testing and analysis activities of the V&V

process
§ Knowledge and understanding
§ Software Verification vs Software Validation
§ Basic concepts of software testing, techniques and approaches
§ Software development phases and kinds of testing
§ Description of software testing methods
§ Application of software testing in practical situations
§ Skills and abilities
§ Define metrics for monitoring the quality of software and
development processes
§ Write models in at least one formal specification language
§ Write appropriate and meaningful test cases, and interpret and
explain (to stakeholders) the results of the application of such test
cases (using appropriate tools) to practical examples
§ Plan and produce appropriate documentation for testing
§ Apply different testing techniques on realistic examples
§ Judgement and approach
§ Exemplify and describe tools used for testing software, and be able
to use them and interpret their output
§ Exemplify and describe the area of formal verification in general,
including model checking and runtime verification, and its
relationship to software quality
§ Identify and hypothesize about sources of program failures, and
reflect on how to better verify the correctness of such programs
§ Instructor
§ Alex Mwotil ([email protected])
§ Office 306, Level 3, Block A
§ Lab: Software Center, Level 3, Block B
§ Office Hours: Monday, Wednesday & Friday – 12PM to 1:30PM

§ Course Page
§ MUELE - https://muele.mak.ac.ug/course/view.php?id=17296
§ Please enrol by Valentines Day (14th February 2024)
§ Course information, material, announcements, homework and results
§ Check the course page frequently
§ Physical
§ Tuesdays
§ 8am EAT – 11am EAT (Day)
§ LLT6B
§ 5pm EAT – 8pm EAT (Evening)
§ LLT2B

§ Online
§ Tuesdays
§ 9am EAT – 12pm EAT
§ Webinar https://zoom.us/j/91918745809?pwd=bmVlTWtKQ1MvMXl5V1RpdmhHNjJFdz09
§ Use real names and mute
§ Use the chat and Q&A for live interaction
§ Recordings
§ To be posted on MUELE end of Tuesdays
§ Course work (30%)
§ Tests (2)
§ Group assignment/project
§ Case study
§ Attendance
§ Aim for 80%+
§ Class participation
§ Reading/Research

§ Final examination (70%)

§ Written
1. Paul Felten, 2017, Software Testing Basics: Software Verification Fundamentals for
Dedicated Testers in the Medical Device Industry 1st Edition, ISBN-10 :
1543269540
2. Mukesh Sharma 2016, Software Testing 2020: Preparing for New Roles 1st
Edition, Auerbach Publications; 1st edition, ISBN-10 : 9781498788878
3. Beizer, Boris. 2011, Software testing techniques. Dreamtech Press, 2003.
4. Khannur, Arunkumar. Software Testing: Techniques and Applications. Pearson
Education India.
5. Dasso, Aristides, 2006, Verification, validation and testing in software
engineering. IGI Global.
6. Others - MUELE
1. Software Quality
1. Properties & Scenarios
2. Software Tests
1. Unit Testing
2. System/Integration Testing
3. Exploratory Testing
4. Automated Testing
3. Adequacy of Testing
1. Structural Testing
2. Fault-based Testing
4. Formal Verification
1. Model-based Testing and Verification
§ Integrity
§ Coursework & Exams
§ Your role to always prevent unauthorized access to your work
§ All work must have been done by you or your group as per instructions

§ Time
§ Ensure you are in the classroom as per the timetable
§ Online classes start <= 5 minutes from scheduled time

§ Discrimination
§ NO form of discrimination is tolerated in this class
§ Please report on email for next steps

§ Other special needs

§ Please talk to me, classmate or anyone that can provide required support in
this course or life in general!
§ Your Full Name

§ Course Expectation (1)

§ Mention one unique thing about you that no one in this
class knows
§ Won’t be a secret going forward 🙂
§ High-quality software is important for ALL domains
§ Quality?
§ What the system does & how it does it
§ How fast is the system?
§ How secure is the system?
§ How available is the system?
§ How easy is it to use the system?
§ How easy is it to scale to more users?
§ It is difficult to measure and assess quality objectively 😭
§ Quality attributes describe the
desired properties of the system
§ Prioritize the attributes
§ Design system that meets defined
thresholds
§ Dependability
§ Consistently offer correct functionality
even under unsafe conditions
§ Performance
§ Meet timing requirements and respond
quickly to events
§ Security
§ Provide service to intended users,
prevent unauthorized access
§ Scalability
§ Support more concurrent requests
when required
§ Availability
§ Recover from failure, carry out tasks
where needed to minimize downtime
§ Modifiability
§ Enhance the software by adding
features, fix issues and adapt to new
environments
§ Testability
§ Ability to easily identify faults in a
system.
§ Probability that a fault will result in a
visible failure.
§ Interoperability
§ Ability to exchange information with
and provide functionality to other
systems.
§ Usability
§ Ability to enable users to perform tasks
and provide support to users.
§ Ease of:
§ Use
§ Learn
§ User needs adaptation
§ Increase confidence and satisfaction in
usage.
§ Software quality attributes often conflict [Duals].
§ Fewer subsystems improves performance, but hurts modifiability.
§ Redundant data helps availability, but lessens security.
§ Localizing safety-critical features ensures safety, but degrades
performance.
§ Important to decide what is important, and set a threshold on when it
is “good enough”.
§ A software product is ready for release when considered
dependable
§ Correct, reliable, usable, secure, …. [properties] -> Done
§ Shown through Software Verification and Validation process
§ Verification
§ Prove that software conforms to its specified functional and
non-functional requirements
§ Validation
§ Prove that the software meets the customer’s true
requirements, needs and expectations
§ Verification
§ Are we building the product
right?
§ Based on requirements and
design specifications
§ Inspections
§ Reviews
§ Walkthroughs
§ Validation
§ Are we building the right
product
§ Testing of the software
application
§ Much harder
§ Both V&V are important – Double edge
§ A well-verified system may not meet the user’s needs
§ A system can’t meet the users needs if not well constructed
§ The required level of V&V depends on:
§ Software purpose, user expectations and marketing
environment
§ Verification
§ Main focus of the class
§ Software testing is the primary activity of verification
§ A process to identify correctness, completeness and
generally the quality of software
§ Activities intended to find errors in software before release
§ Avoid ramifications of buggy software in the wild
§ A check on the actual vs expected product
§ Ensure the product is defect free
§ An execution of software or system components to evaluate 1+
properties of interest
§ Software quality attributes
§ For the System Under Test
(SUT)
§ Define a test case
§ Test Input
§ Expected Output
§ Execute with test input
§ To get the actual output
§ Compare the actual output and
the expected output
§ Verdict
§ Pass – No revealed faults
§ Fail – Revealed faults
§ Test Oracle
§ A mechanism to determine the
verdict of a test
§ Does the software meet requirements that guided its design and development?
§ Requirements and design specifications

§ Does the software product respond correctly to all kinds of inputs?

§ Does the software perform its functions within acceptable time bounds?
§ Is the product sufficiently usable?
§ Can the software product be installed and run in its intended environments?
§ Does the software product achieve the general result as desirable by/to the
stakeholders?
§ ..
§ ..
§ ..

§ ..
§ ..

§ ..
§ ..
§ Discover defects
§ Avoid detecting problems
§ Prove that the software has no faults

§ Learn about the reliability of the software

§ Avoid being sued by customers

§ Ensure that product works as user expected

§ Stay in business
§ Detect defects early, which helps in reducing the cost of defect fixing?
§ Static (Formal) Verification
§ Analysis of system artifacts to discover problems.
§ Proofs: Posing hypotheses and making arguments using
specifications, models, etc.
§ Inspections: Manual “sanity check” on artifacts (e.g., source code),
searching for issues.

§ Dynamic Verification
§ Exercising and observing the system to argue that it meets the
requirements.
§ Testing: Formulating sets of input to demonstrate requirement
satisfaction or find faults.
§ Fuzzing: Generating semi-random input to locate crashes and other
anomalies.
§ Taint Analysis: Monitoring how faults spread by corrupting system
variables.
§ Static Verification § Dynamic Verification
§ One error can hide other errors. § Discovers problems from runtime
Inspections not impacted by interaction, timing problems, or
program interactions. performance issues.
§ Incomplete systems can be § Often cheaper than static
inspected without special code to verification.
run partial system. § Easier to automate.
§ Inspection can assess quality § However, cannot prove that
attributes such as maintainability, properties are met
portability, code style, program § Cannot try all possible
inefficiencies, etc. executions.
§ Software Quality & Quality Attributes
§ Desirable properties of a system under development
§ Software quality is subjective
§ Define objective thresholds for measurement

§ Software should be dependable and usable before release

§ Verification demonstrates an implementation meets specifications
§ A measurement of software dependability
§ Software testing is the most common form of verification

§ Validation demonstrates the usability of the software