Htu Chapter 1 Test Automation Framework
Htu Chapter 1 Test Automation Framework
Htu Chapter 1 Test Automation Framework
HTU CHAPTER 1
TEST AUTOMATION FRAMEWORK
This section gives the introduction about test automation framework, various types of
the framework and the analysis of the best suitable framework for the application under test
(the application under test is referred as AUT). This also includes the detailed description of
the format of the input (the input to the framework is referred as the test tables) that is given
to our test automation framework.
1.1 RECORD/PLAYBACK MYTH
The test automation tool vendors market their product as the main feature of the tool
is the ability to capture the user actions and later to playback them. Here is the basic
paradigm for GUI-based automated regression testing the so called Record/Playback
method (also called as Capture/Replay approach)
1. Design a test case in the test management tool.
2. Using the capture feature of the automation testing tool record the user actions. The result
is a macro-like script where each user action is presented.
3. Enhance the recorded script with verification points, where some property or data is
verified against an existing baseline. Add delay and wait states points where the different
actions are synchronized.
4. Playback the scripts and observe the results in the log of the test management tool.
The basic drawback in this method is the scripts resulting from this method contain
hard-coded values which must change if anything at all changes in our AUT. The costs
associated with maintaining such scripts are astronomical, and unacceptable. These scripts
are not reliable, even if the application has not changed, and often fail on replay (pop-up
windows, messages, and other things can happen that did not happen when the test was
recorded).
Chapter 1
If the tester makes an error entering data, etc., the test must be re-recorded. If the
application changes the test must be re-recorded. All that is being tested are things that
already work. Areas that have errors are encountered in the recording process (which is
manual testing, after all). These bugs are reported, but a script cannot be recorded until the
software is corrected. So logically nothing is tested by this approach.
Data-Driven Testing
Chapter 1
under-test. Then these small scripts are taken and combined them in a hierarchical fashion
to construct larger tests. The use of this framework will yield a higher degree of
modularization and add to the overall maintainability of the test scripts.
1.2.2 Test Library Architecture
The test library architecture framework is very similar to the test script modularity
framework and offers the same advantages, but it divides the application-under-test into
procedures and functions (or objects and methods depending on the implementation
language) instead of scripts. This framework requires the creation of library files (SQABasic
libraries, APIs, DLLs, and such) that represent modules, sections, and functions of the
application-under-test. These library files are then called directly from the test case script.
Much like script modularization this framework also yields a high degree of modularization
and adds to the overall maintainability of the tests.
1.2.3 Data-Driven Testing
A data-driven framework is where test input and output values are read from data
files (ODBC sources, CVS files, Excel files, DAO objects, ADO objects, and such) and are
loaded into variables in captured or manually coded scripts. In this framework, variables are
used for both input values and output verification values. Navigation through the program,
reading of the data files, and logging of test status and information are all coded in the test
script. This is similar to table-driven testing (which is discussed shortly) in that the test case
is contained in the data file and not in the script; the script is just a "driver," or delivery
mechanism, for the data. In data-driven testing, only test data is contained in the data files.
1.2.3.1 Merits of data-driven testing
The merits of the Data-Driven test automation framework are as follows,
Scripts may be developed while application development is still in progress
Utilizing a modular design, and using files or records to both input and verify
data, reduces redundancy and duplication of effort in creating automated test
scripts
If functionality changes, only the specific "Business Function" script needs to
be updated
Chapter 1
Chapter 1
Window
Control
Action
Arguments
Window Name
Menu
Click
File, Open
Window Name
Menu
Click
Close
Window Name
Pushbutton
Click
Folder Name
Verify
Results
Window Name
Once creating the test tables, a driver script or a set of scripts is written that reads in
each step executes the step based on the keyword contained the Action field, performs error
checking, and logs any relevant information.
1.2.4.2 Merits of keyword driven testing
The merits of the Keyword Driven Testing are as follows,
The Detail Test Plan can be written in Spreadsheet format containing all input
and verification data.
If "utility" scripts can be created by someone proficient in the automated tools
Scripting language prior to the Detail Test Plan being written, then the tester
can use the Automated Test Tool immediately via the "spreadsheet-input"
method, without needing to learn the Scripting language.
The tester need only learn the "Key Words" required, and the specific format
to use within the Test Plan. This allows the tester to be productive with the
test tool very quickly, and allows more extensive training in the test tool to be
scheduled at a more convenient time.
1.2.4.3 Demerits of keyword driven testing
The demerits of the Keyword Driven Testing are as follows,
Development of "customized" (Application-Specific) Functions and Utilities
requires proficiency in the tools Scripting language. (Note that this is also
true for any method)
If application requires more than a few "customized" Utilities, this will require
the tester to learn a number of "Key Words" and special formats. This can be
time-consuming, and may have an initial impact on Test Plan Development.
Chapter 1
Chapter 1
The Application Map is referred to App Map. This App Map in WRAFS
is the Application Map file created from GUI Map of WinRunner
All
of
these elements of the framework rely on the information provided in the App Map to
interface or bridge the automation framework with the application being tested. The App Map
is the only means by which the framework could identify the objects in the application under
test. Each of these elements is described in more detail in the following sections. The
following figure shows the diagrammatic representation of the Hybrid Test Automation
Framework.
Chapter 1
APPLICATION MAP
The Application Map is one of the most critical components, which is used for
mapping the objects from names humans can recognize to a data format useful for the
automation tool. For a given project it is needed to define a naming convention or specific
names for each component in each window as well as a name for the window itself. Then
use the Application Map to associate that name to the identification method needed by the
automation tool to locate and properly manipulate the correct object in the window.
Application Map not only gives the ability to provide useful names for the objects, it
also enables the scripts and keyword driven tests to have a single point of maintenance on
the object identification strings. Thus, if a new version of an application changes the title of
the window or label of the components or the index of an image element within it, they
should not affect the test tables. The changes will require only a quick modification in one
place--inside the Application Map.
COMPONENT FUNCTIONS
Chapter 1
Component Function keywords and their arguments define the low-level vocabulary and
individual record formats will be used to develop the test tables.
TEST TABLES
The input to the framework apart from the application map are the test tables, which holds
the arguments needed for the Component Functions and other information. There are three
levels in which the test tables are organized, they are as follows,
Low-Level Test Tables (or) Step Tables
Intermediate-Level Test Tables (or) Suite Tables
High-Level Test Tables (or) Cycle Tables.
LOW-LEVEL TEST TABLES
Low-level Test Tables or Step Tables contain the detailed step-by-step instructions of
the tests. Using the object names found in the Application Map, and the vocabulary defined
by the Component Functions; these tables specify what window, what component, and what
action to take on the component. The columns in the Step Tables are as follows,
Action Command
Window Name
Component Name
Values Need to Perform the Specified Action
The StepDriver module is the one that initially parses and routes all low-level
instructions that ultimately drive our application.
INTERMEDIATE-LEVEL TEST TABLES
Intermediate-level Test Tables or Suite Tables do not normally contain such low-level
instructions. Instead, these tables typically combine Step Tables into Suites in order to
perform more useful tasks. The same Step Tables may be used in many Suites. In this way
the minimum numbers of Step Tables necessary are developed. Then they are organized in
Suites according to the purpose and design of the tests, for maximum reusability. The
columns in the Suite Tables are as follows,
Step Table Name
Specific Arguments to be Passed to the Step Tables
The Suite Tables are handled by the SuiteDriver module which parses each record in
the Suite Table and passes each Step Table to the StepDriver module for processing.
Chapter 1
High-level Test Tables or Cycle Tables combine intermediate-level Suites into Cycles.
The Suites can be combined in different ways depending upon the testing Cycle which is
efficient to execute. Each Cycle will likely specify a different type or number of tests. The
columns in the Cycle Tables are as follows,
Suite Table Name
Specific Arguments to be Passed to the Suite Table
These Cycles are handled by the CycleDriver module which passes each Suite to
SuiteDriver for processing.
CORE DATA DRIVEN ENGINE
The Core Data Driven Engine is the primary part of the framework and it has three
main modules, they are as follows
StepDriver
SuiteDriver
CycleDriver
CycleDriver processes Cycles, which are high-level tables listing Suites of tests to
execute. CycleDriver reads each record from the Cycle Table, passing SuiteDriver each
Suite Table it finds during this process. SuiteDriver processes these Suites, which are
intermediate-level tables listing Step Tables to execute. SuiteDriver reads each record from
the Suite Table, passing StepDriver each Step Table it finds during this process. The
following figure represents the Core Data Driven Engine,
Chapter 1
StepDriver then routes the complete instruction record to the appropriate Component
Function for final execution.
SUPPORT LIBRARIES
The Support Libraries are the general-purpose routines and utilities that let the
overall automation framework do what it needs to do. They are the modules that provide
services like,
File Handling
String Handling
Buffer Handling
Variable Handling
Database Access
Logging Utilities
System\Environment Handling
Application Mapping Functions
System Messaging or System API Enhancements and Wrappers
They also provide traditional automation tool scripts access to the features of our
automation framework including the Application Map functions and the keyword driven
engine itself. Both of these items can vastly improve the reliability and robustness of these
scripts until such time that they can be converted over to keyword driven test tables.