Object-Oriented Testing

Special Considerations on Testing

Object-Oriented Systems
Object-Oriented Testing

What is “special” about

testing object-oriented software?

● Object-oriented development provides an abstract way of

thinking a problem
● Absent in traditional structured-oriented development
● e.g. What are the parts that make up a help guide vs. what
functionality does it need to provide

● Techniques and tools for testing a traditional structured-oriented

software are not sufficient to completely test object-oriented
systems
● OO poses some new challenges on testing systems
• Conventional techniques can be used but they probably will not be
enough
Object-Oriented Testing

● OO Testing is Still “Testing”

● We still do
● Unit testing, but we expand the definition of unit
● Component testing to make sure individual subsystems work
correctly
● Integration testing to make sure subsystems work correctly
together
● System testing to verify that requirements are met
● Regression testing to make sure previous functionality still
works after new functionality is added
● But…
Object-Oriented Testing

● Testing strategy changes…

● The concept of the unit broadens due to
encapsulation
● Integration focuses on classes and their execution
across a thread or in the context of a use case
● e.g. Do your classes work together to allow a user to
complete a task from your scenarios?
● Validation uses conventional black box methods

● Test case design draws on conventional methods,

but also encompasses special features
Object-Oriented Development

Object-Orientation is not a “silver bullet”!!!

● Although…
● Object-oriented analysis and design can lead to a better system
architecture, and
● Object-oriented programming enforces a disciplined coding style

● Object-Orientation does not…

● Shield against programmers’ mistakes or a lack of understanding of
the specification
● Ensure the production of correct programs by itself

● Object-oriented systems still need TESTING

Object-Oriented Development

Object-oriented development involves different types and

proportions of errors that require a different approach to testing.

● Common features of OO development are complex and,

therefore, prone to errors
● Encapsulation
• Information hiding
● Inheritance
• Consistency in abstraction hierarchy
• Multiple inheritance
● Polymorphism
• Dynamic binding
● Structured Exception Handling

These impact on testing OO software!

Object-Oriented Development

● In other words…

To what extent should inherited features be retested?

When can we trust the reports of an untested object about its state?

How can we verify state-dependent behavior when state control is

distributed over an entire object-oriented application?

What is an effective test and integration strategy given the iterative

approach preferred for object-oriented development?
Impact of OO on Testing

● Encapsulation

A control access mechanism that determines

the visibility of methods and attributes in a class.
● Enforces information hiding
● The interface and implementation of a class may be syntatically separated
● Avoids undesirable dependencies between client and a server class
• Only the class interface is visible to the client
• Design decisions and implementation details are hidden
● Encourages program modularity
● Restricts the implication of changes
Impact of OO on Testing

● Impact of Encapsulation on Testing

● Testing requires...
● A complete report on the states of an object
● The possibility to change states easily

● Why does encapsulation make OO testing harder?

● Lack of visibility of states
• Besides the behavior encapsulated by an object, it also encapsulates states
● Difficulty in initializing data items
● Difficulty in calling methods

Although encapsulation does not directly contribute

to the introduction of faults, it may be an obstacle in testing by reducing
controllability and observability.
Impact of OO on Testing

● Inheritance
Relationship between classes that allows the definition
of a new (derived) class based on the definition of an existing (base) class.
The derived class can then be refined by modifying or removing the inherited
operations, or by adding new properties.

● Single inheritance
● Properties are inherited from a single class
• Strict: the derived class can only be refined by adding new properties
• Subtyping: besides the properties of strict refinement, it allows the redefinition
(overriding) of the inherited properties

● Multiple inheritance
● Properties are inherited from multiple (many) classes
Impact of OO on Testing

● Inheritance
● Allows reuse of both specification and implementation
● Shares features present in classes previously defined
● The preexisting classes do not have to be modified or made
aware of the new class
Impact of OO on Testing

● Inheritance can contribute to errors in several ways:

● Weakens encapsulation
● Introduces hazards similar to those associated with global data in conventional
languages
● Used inappropriately as
● A powerful macro substitution for convenience
● A model of hierarchy
● Can lead to homograph operations
● Two or more classes with similar name and profile
• When using multiple inheritance

● Leads to side-effects, inconsistencies or incorrect behavior

● Deep and wide inheritance hierarchies can defy comprehension

● Leads to errors and reduction of testability
Impact of OO on Testing

● Impact of Inheritance on Testing

● Opens the issue of “retesting”

When do we need to retest the superclass?

When can we assume that the superclass has already been tested?

Should operations inherited from ancestor classes be retested

in the context of the descendant class?
Impact of OO on Testing

● Impact of Inheritance on Testing

● Inheritance can lead to a false conclusion that...

● Aspects already tested in the superclasses do not need to be retested in the
subclasses

● But, in fact... Inheritance defines new context for methods

● Behavior of inherited methods can be changed because of other methods
that are called within the inherited class

● So...
● A method which has been tested in a superclass needs a retesting when it
is reused by a subclass
● Even if a method is inherited from a superclass without any modification
(strict inheritance), it has to be retested in the subclass context
Impact of OO on Testing

● Polymorphism

Ability to refer more than one type of object

using the same name or variable.

● Replace explicit compile-time binding and static

type checking by implicit binding and run-time type checking
● Facilitate compact, elegant and extensible code
● Sometimes constrained by inheritance
Impact of OO on Testing

● Polymorphism contributes to errors in several ways:

● As classes grow deeper and systems grow wider, the likelihood of misusing a
dynamically bound method increases
● The combination of polymorphism and method refinement make it difficult to
understand the behavior of lower level classes
● Corrected by class flattening
● Object polymorphism with late binding can result in messages being sent to the
wrong class
● Dynamic binding can cause a client class to
● Assume/require a method not provided by the server class
● Misuse an available method
● Incorrectly construct an interface signature
Impact of OO on Testing

● Impact of Polymorphism on Testing

● Brings undecidability to testing
● Dynamic binding: unanticipated bindings

It is impossible, when invoking an operation with a polymorphic name,

to predict until run-time whether the original or a redefined
implementation will be selected.
● Each possible binding of a polymorphic component requires a separate
test
● It may be difficult to find/exercise all such bindings
● Erroneous casting (type conversions)
● Can lead to non-easily detectable errors
Impact of OO on Testing

● Other issues…
● Concurrency
● Correct behavior of concurrently executing objects

● Sequences of messages
● The packaging of methods in a class is fundamental to OO
● Messages have to be executed in some sequence

Which message sequences are valid?

● Activation by message-passing vs. Conventional activation
● Dataflow involves objects, not primitive types
● Structured Exception Handling
OO Testing Phases: Unit

● Unit Testing
What is a unit?
A method? A class? A group of classes?

● We will consider a “Class” as the proper subject of unit testing

● Smallest stand-alone unit for testing
• Instantiation of a single class
● Operations within the class should be tested
● The state behavior of the class should be tested
● Some minimal integration usually required (or stubs)
• Instance variables typically are instances of other classes
● Inheritance and message passing influence how much testing will be
needed
● Intra-method, inter-method and intra-class testing approaches can be
used
OO Testing Phases: Unit

● Unit Testing
● Intra-Method Testing
● Tests a specific method of a class

● Inter-Method Testing
● Methods of the same class may interact to implement a given functionality
• Kind of integration that should be tested

● Intra-Class Testing
● Tests public method interactions through different sequences of calls to
such methods
● Identifies possible sequences that lead the object to an invalid state
• Since the user may invoke the public method in many different orders, the
intra-class testing gives one the confidence that different sequences of calls act
correctly
OO Testing Phases: Integration

● Integration Testing
● Integration of classes to create the application system
● Closely tied to the overall development approach
● Inter-Class Testing
● Test public methods: not only those in a single class, but also those in
different classes
• For each client class, use the list of class operators to generate a series of
random test sequences. The operators will send messages to other server
classes.
• For each message that is generated, determine the collaborator class and the
corresponding operator in the server object.
• For each operator in the server object (that has been invoked by messages sent
from the client object) determine the messages that it transmits.
• For each of the messages, determine the next level of operators that are invoked
and incorporate these into the test sequence.
OO Testing Phases: System

● System Testing
● Considers the OO software as a “whole”
● Looks for the errors in the whole system, including operating
system call errors, performance errors, partition errors...
● Alpha and beta testing can also be used as a complement
● Generally based on black box criteria
● No fundamental difference from procedural and OO software
Procedural vs. Object-Oriented Testing

Testing Phases

Procedural Testing Unit Testing Object-Oriented Testing

Procedure or Class
Subroutine (Methods)

Integration Testing

Two or More Two or More

Procedures Classes

System Testing

Entire Entire
System System
Procedural vs. Object-Oriented Testing

● Smallest Unit: Class

Phase Procedural Testing Object-Oriented Testing

Intra-method,
Unit Intra-procedural Inter-method and
Intra-class

Integration Inter-procedural Inter-class

System Entire system Entire system

Testing OO Systems

● Base Class
● Testing base classes is similar to unit testing of modules in
traditional systems
● Goal: ensure the base classes work correctly in isolation, before
their integration
● Manually prepare a “thorough” test suite
● Apply it to each base class
● Keep information on the designed test cases
● SUnit Framework
● Test Plan
Testing OO Systems

● A complete object test suite requires:

● Every operation is executed
● All message parameters and exported attributes are checked
● Every out-going exception is raised and every in-coming exception is
handled
● Every variable attribute is updated
● Every state is achieved
● Every operation is executed in each state
● Every transition is exercised to test assertions
● Appropriate stress, performance and suspicion tests are performed
Testing OO Systems

● Testing a subclass
● Test the subclass class in conjunction with its parent
● Determine test cases required for the subclass
• Incrementally update the parent class history (methods tested, test
suite applied, test status) to reflect differences from the parent class
• Test new attributes or those inherited, affected attributes
– Also test their interactions

● Also… Keep information on the designed test cases

● SUnit Framework
● Test Plan
Testing OO Systems

● Testing a subclass
● New or untested feature in the subclass
● Develop new complete test suite for it
● Prepare intra- and inter-class tests
• Test a new data member by testing the class methods with which it
interacts

● Inherited Features
● Very limited retesting of classes is required
● Integration testing is required if class either:
• Interacts with new or redefined variable; or
• Accesses the same instances in the class’s representation as other
methods
Testing OO Systems

● Testing a subclass
● Redefined Features
● May be able to reuse black box test cases
• Functionality unchanged
● New white box tests will be required
• Implementation sure to change
General Test Case Design Issues
● What design models are useful for reasoning about probable sources of error?

● What techniques may be used to construct test cases from models, specifications and
implementations?

● How much testing is sufficient for methods, classes, application systems and components of
reusable libraries?

● To what extent should inherited features be retested? Under what conditions must features
inherited from a superclass be retested in a subclass?

● How should abstract and generic classes be tested?

● What models for understanding collaboration patterns are useful for testing?

● How can models of the state of an object, class or system under test be used?

● How can state-dependent behavior be verified when state control is often distributed over an
entire object-oriented application?

● How can clients and servers linked by dynamic binding (polymorphism) be tested?

● How can regression tests be formulated and conducted to ensure the integrity of modified
classes?
Specific OO Test Techniques
● Combinatorial Methods
● Decision Tables
● Decision Trees
● State Machines
● Events, Transitions, Guards
● Activities, Actions
● UML-Based Model
● Association Tests
● Use Case Tests, Interaction Tests
● Responsibility-Based
● Traditional
● Boundary, Control Flow, Partitions
● Controlled Exception Testing (exceptions/handlers)
● Frequency Allocation
Comments on Testing Interfaces

● Can test for many things

● Top 3:
● Functionality
● Robustness
● Usability
Functionality of an Interface

● Does your interface provide everything in the

specification / everything users need to
accomplish the supported tasks?

● Testing methods:
● Work through your scenarios
● Check functionality explicitly mentioned in the spec
Robustness of an Interface

● Does your interface prevent the user from crashing the

application, putting it into a bad state, or trashing data?

● Testing method:
● Developers tend to avoid doing things they know will break the
system, so it’s harder to test this yourself
● Give your interface to 2-3 of your friends and ask them to play
with it
● CS majors are really good at breaking things!
• Profs are even better
● Some actions do “void the warranty”
Usability of an Interface

● In this context, can the user figure out how to

accomplish a particular task with your interface?

● Test methods:
● Give your system to 2-3 people who haven’t seen it
before and ask them to accomplish a particular task
● Can’t reuse people for this!
● Aim for people in your user group
● Watch what they do and listen to them
Next on the Menu

● Midterm!
● Questions?