SOTWARE TESTING

INSTRUCTOR: Mehwish
 Specification
“if you can’t say it, you can’t do it”

• You have to know what your product is before you

can say if it has a bug.
• A specification defines the product being created
and includes:
– Functional requirements that describes the features
the product will support. E.g., on a word processor
• Save, print, check spelling, change font, …
– Non-functional requirements are constraints on the
product. E.g,
• Security, reliability, user friendliness, platform, …
 A software bug occurs when at least one
of these rules is true
• The software does not do something that the specification
says it should do.
• The software does something that the specification says it
should not do.
• The software does something that the specification does not
mention.
• The software does not do something that the product
specification does not mention but should.
• The software is difficult to understand, hard to use, slow …
Most bugs are not because of mistakes in
the code …

• Specification (~= 55%)

• Design (~= 25%)
• Code (~= 15%)
• Other (~= 5%)
 Relative cost of bugs
“bugs found later cost more to fix”

• Cost to fix a bug increases exponentially (10x)

– i.e., it increases tenfold as time increases
• E.g., a bug found during specification costs $1 to fix.
• … if found in design cost is $10
• … if found in code cost is $100
• … if found in released software cost is $1000
 Bug Free Software
• Software is in the news for the wrong reason
– Security breach, hackers getting credit card information,
etc.
• Why can’t software engineers develop software that
just works?
– As software gets more features and supports more
platforms it becomes increasingly difficult to make it create
bug-free.
 Discussion …
• Do you think bug free software is unattainable?
– Are their technical barriers that make this impossible?
– Are we missing technology or processes?
 Goal of a software tester
• … to find bugs
• … as early in the software development processes as
possible
• … and make sure they get fixed.

• Advice: Be careful not to get get caught in the

dangerous spiral of unattainable perfection.
 What to look for when interviewing someone
for the position of software tester
• Are they explorers?
• Are they troubleshooters?
• Are they relentless?
• Are they creative?
• Are they perfectionists (within reason)?
• Do they exercise good judgment?
• Are they tactful and diplomatic?
• Are they persuasive?
 You now know …
• … what is a bug
• … the relationship between specification and bugs
• … the cost of a bug relative to when it is found
• … the unattainable goal of perfect software
• … the goal of the software tester
• … valuable attributes of a software tester
 The Software
Development Process
 Software is …
• requirements specification documents
• design documents
• source code
• test suites and test plans
• interfaces to hardware and software operating
environment
• internal and external documentation
• executable programs and their persistent data
 Software effort
• Specification
• Product reviews
• Design
• Scheduling
• Feedback
• Competitive information acquisition
• Test planning
• Customer surveying
• Usability data gathering
• Look and feel specification
• Software architecture
• Programming
• …
 Discussion …
• What is software engineering?
• Where does testing occur in the software
development process?
 Customer requirements
• The software development team must determine
what the customer wants.
• How can you do this?
– Guess?
– Collect detailed information from surveys?
– Get feedback from a previous version of the software?
– Read reviews in magazines?
– Get information about the competition?
– Other ways?
• The collected data is used to guide the specification
effort.
 Specification
“If you don't know where you're going any road will take you there”

• The specification takes the data from the

customer requirements and other sources and
defines:
– The features of the software (functional requirements).
– The constraints on these features (non-functional
requirements).
• Specifications can be:
– formal (e.g., aerospace industry), rigid
– informal (e.g., a .com start up), on a cocktail napkin or
a whiteboard.
 Schedules
• The goals of scheduling are to know:
– What work needs to be completed?
– How much work is left to do?
– When will the work be finished?
– Who will finish each task?
– Other measurable queries.
• A Gantt chart is a popular type of bar chart that
illustrates a project schedule.
Gantt Chart
 Design
• Before coding begins on non-trivial software
projects, a set of design documents are created to
serve as blueprints.
– Software Architecture
– Data flow diagram
– State transition diagram
– Flowchart
– Commented code
 Source code … of course
• The ultimate specification of the software!
• ‘Code is king’ philosophy is still dominant.
• Many programming languages and tools to choose
from.
 Test documents
• Test plan
– Quality objectives, resource needs, schedules,
assignments, methods, etc.
• Test cases
– Inputs and expected outputs.
• Bug reports
– E.g., the Bugzilla web-based bug tracker.
• Test tools and automation
• Metrics, statistics, and summaries
– Number of unresolved bugs, mean time to repair a bug,
etc.
 Software Project Staff
• Project managers
– Write product specification, manage the schedule, critical decision
tradeoffs
• Software architects, system engineers
– Design the software, work closely with programmers
• Programmers, developers, coders
– Write code, fix bugs
• Testers, quality assurance staff
– Find bugs, document bugs, track progress of open bugs
• Technical writers
– Write manuals, on line documentation
• Configuration managers, builders
– Packaging and code, documents, and specifications
 Software Development Lifecycles
• Code and Fix
• Waterfall
• Spiral
 You now know …
• … what is software
• … what is software engineering
• … what is the composition of a software
development organization
• … what are the major phases of a software
development project
• … how major phased are organized