Software Quality Engineering

(SE3613)
Testing (initial processes)

SQE-SE3613 1
 Test Automation
• Basic understanding:
• Aims to automate some manual tasks
• Automation needed for large systems
• Fully automated: impossible
• Some level of automation for individual activities is possible
• Issues
• Specific needs and potential for automation
• Selection of existing testing tools, if available
• Possibility and cost of constructing specific test automation tools
• Availability of user training for these tools and time/effort needed
• Overall cost, including costs for tool acquisition, support, training, and usage
• Impact on resource, schedule and project management
2
Benefits
• Simplifies Test Case Execution: Automation testing can be left virtually unattended and
thus it allows monitoring of the results at the end of the process. Thus, simplifying the
overall test execution and increasing the efficiency of the application.
• Improves Reliability of Tests: Automation testing ensures that there is equal focus on all
the areas of the testing, thus ensuring the best quality end product.
• Increases amount of test coverage: Using automation testing, more test cases can be
created and executed for the application under test. Thus, resulting in higher test
coverage and the detection of more bugs. This allows for the testing of more complex
applications and more features can be tested.
• Minimizing Human Interaction: In automation testing, everything is automated from
test case creation to execution thus there are no changes for human error due to
neglect. This reduces the necessity for fixing glitches in the post-release phase.
• Saves Time and Money: The initial investment for automation testing is on the higher
side but it is cost-efficient and time-efficient in the long run. This is due to the reduction
in the amount of time required for test case creation and execution which contributes to
the high quality of work.

SQE-SE3613 3
Automation for Test Execution
• Execution – a prime candidate for automation among the
preparation, execution, and follow-up
• Earliest test automation tools found some successes
• Example: various semi-automatic debugging tools or
debuggers allow tester
• To set and reset variable values
• To set and reset execution states during execution
• To observe the dynamic execution behavior at different observation
points

4
 Automation for Test Execution
• The most effective technique to improve the efficiency, test coverage, and
execution speed is through automation testing in software testing.
• Various automated task sequencing tools for job transfer from one test run
to another work in much the same way as job dispatcher/scheduler (related
to Batch processing) in various operating systems.
• An additional functionality for many of the test automation tools is to allow
information recording and collection
• IBM’s test automation tool called T3 was used to generate workload,
monitor the execution, and record various execution details

5
Automation for Test Planning and Preparation
• Potential for automation is different for different sub-
activities
• Human intensive
• Not much automation can be achieved
• Nor is there a high demand for automation
• Test case generation: Area of focus
• Semi-automated test case generation
• Legacy products: automated selection from existing test suites
• New test case generation: use scripts to generate workload, test
cases, and test runs
6
Automation for Test Planning and Preparation
• Automation of test model construction:
• requires high levels of human intelligence and expertise, and is
therefore not easily automated
• Individual activities can be automated under human
supervision/control e.g., Data gathering
• Generate certain models and test cases directly, like using
compilers to generate object code from source code (in most of the
models, various decisions need to be made, which can only be carried out by
people with proper expertise)
• Once model constructed, Automatic generation of test cases from
automated models
7
 Automation for Test Measurement, Analysis, and
Follow-up
• Situation is similar to test planning and preparation
• Very little can be automated because
• Most of the follow-up actions involve problem fixing and various other
remedial and improvement initiatives
• Various coverage analysis tools exist to collect coverage information
during test execution. For example:
• McCabe Test from McCabe & Associates
• S-TCAT (System Test Coverage Analysis Tool)
• ATAC (Automatic Test Analysis for C)

8
Test automation Frameworks
What is a Test Framework?
• A testing framework is a set of guidelines or rules used for
creating and designing test cases. A framework is comprised
of a combination of practices and tools that are designed to
help QA professionals test more efficiently.
• These guidelines could include coding standards, test-data
handling methods, object repositories, processes for storing
test results, or information on how to access external
resources.

SQE-SE3613 9
Cont..
Types of Automated Testing Framework
• Linear Automation Framework
• Modular Based Testing Framework
• Data-Driven Framework
• Keyword-Driven Framework
• Hybrid Testing Framework
• Behavior driven development Framework

SQE-SE3613 10
Linear Automation Framework
With a linear test automation framework, also referred to as a
record-and-playback framework, testers don’t need to write
code to create functions and the steps are written in a
sequential order. In this process, the tester records each step
such as navigation, user input, or checkpoints, and then plays
the script back automatically to conduct the test.

SQE-SE3613 11
Modular Based Testing Framework
• Implementing a modular framework will require testers to
divide the application under test into separate units,
functions, or sections, each of which will be tested in
isolation. After breaking down the application into individual
modules, a test script is created for each part and then
combined to build larger tests in a hierarchical fashion. These
larger sets of tests will begin to represent various test cases.

SQE-SE3613 12
Data-Driven Framework
Using a data-driven framework separates the test data from
script logic, meaning testers can store data externally. Very
frequently, testers find themselves in a situation where they
need to test the same feature or function of an application
multiple times with different sets of data. In these instances,
it’s critical that the test data not be hard-coded in the script
itself, which is what happens with a Linear or Modular-based
testing framework.

SQE-SE3613 13
Keyword-Driven Framework
In a keyword-driven framework, each function of the
application under test is laid out in a table with a series of
instructions in consecutive order for each test that needs to
be run. In a similar fashion to the data-driven framework, the
test data and script logic are separated in a keyword-driven
framework, but this approach takes it a step further.

SQE-SE3613 14
Hybrid Test Automation Framework
As with most testing processes today, automated testing
frameworks have started to become integrated and overlap
with one another. As the name suggests, a hybrid framework
is a combination of any of the previously mentioned
frameworks set up to leverage the advantages of some and
mitigate the weaknesses of others.

SQE-SE3613 15