Frameworks Introduction

Automated testing is most effective when implemented within a framework. Although, in automated testing, the term framework is often used to describe the collective structures that compose unit testing tools, this section will largely focus on a different kind of framework. Here we will discuss the types of frameworks that may be defined as a set of abstract concepts, processes, procedures and environment in which automated tests will be designed, created and implemented. In addition, this framework definition includes the physical structures used for test creation and implementation, as well as the logical interactions among those components. Over the years, automated testing (and thus automated test frameworks) has evolved, becoming increasingly defined and sophisticated with each new evolutionary phase. These phases can be described in terms of three generations, each generation wielding a set of advantages and challenges, thus each generation remains relevant despite new developments. These generations generally apply to Functional System Test Automation, but in some instances may expand to address Unit Test Automation. For more information scroll to the lower half of the Frameworks menu page.

Frameworks
1st Generation Frameworks Linear The Linear "framework" typically yields a one-dimensional set of automated tests Read more 2nd Generation Frameworks Data-driven Frameworks built on Data-driven scripting are similar to Linear scripts, in that Read more Functional Decomposition

Linear The Linear "framework" typically yields a one-dimensional set of automated tests in which each automated test is treated simply as an extension of its manual counterpart. Driven mostly by the use of the Record & Playback, all components that are executed by a Linear script largely exist within the body of that script. There is little to no modularity, reusability, or any other Quality Attribute considered in the creation of Linear scripts. Linear Scripts, may be useful in environments with a very small scope. An example of a small linear script is shown below.
1 2 3 4 5 6 7 8 Input John into Username textbox Input JPass into Password textbox Click Login button If Welcome Screen exists then Pass the test Else Fail the test End If

Notice that there are not calls to external modules, or external data. Linear scripts are often driven by Record & Playback should, but in most cases, Record & Playback should not be treated as a framework or serious approach for an automation effort meant to span any real length of time. Some will agree, however, that while Record and Playback is not good as a framework, it may be useful as a technique. Linear Script Pros  Quick development There is little planning or consideration of the various Quality Attributes when creating Linear scripts. Couple this information with the fact that Record and Playback is often the driving force behind Linear script development, then its easy to see that cost of development in time, resources and money could be relatively low.  Smaller tool learning curve Test automation typically requires knowledge of the automation tool features and language, as well as the knowledge of how the tool interacts with the application under test. The use of Record and Playback in the creation of Linear scripts dynamically creates code that corresponds with actions performed in the application. This code may then be studied by the automator for answers to questions about the tools language syntax as well as how the automated test tools interaction with the AUT.  Quickly gather information about application dynamics This advantage is closely related to the smaller tool learning curve advantage. The use of Record and Playback provides information about the Aut dynamics, such as how object properties change as activity in the application is altered.  Script Independence Given the fact that all the components executed by a script exist within that script, there is little apprehension about making script modifications that may unknowingly affect some other script.  Simplifies error location The more advanced an automation framework becomes the more complex it becomes. With complexity comes challenges related to locating the source of an error, particularly for those that are unfamiliar with the makeup of the automation

framework. Given the fact that all components of a Linear script exist within that script, there is little question about where the error occurred. Linear Script Challenges (Cons)  Improper playback When relying heavily on Record and Playback, recorded scripts will often not playback properly. This is due to the fact that the Record and Playback feature is not analytical. It doesnt evaluate the application objects and behavior nor does it make decisions on how best to deal with those objects and behaviors. In an effort to fix the playback issues, a series of patchwork solutions are often introduced that wont stand the test of time.  Redundancy Linear scripts dont take advantage of reuse, therefore if more than one script performs the same or similar functionality, the functionality will be duplicated in each respective script. When the application changes in subsequent releases of the application, excessive maintenance will be the result.  One dimensional scripts Little flexibility is offered in altering the way Linear scripts may be executed. Scripts are only readily available to be run at one level, in one location and in one way. If a test manager wants the automator to execute a subset of the total test bed based on defined priorities and risks, or execute the tests in a different environment, or in a different sequence, the automator would have to perform time consuming, run-time analysis of the scripts to make it happen. This will decrease the amount of time available for script execution and analysis.  Scripts are difficult to read With the absence of reusable components, Linear scripts are replete with code or automation statements. This will make the script difficult to read and therefore tedious to analyze and maintain. If a statement in the script fails, it is often difficult to determine what application functionality was affected by failure. Likewise, when it becomes necessary to make changes to the script, it is a little more tedious to determine where the changes need to be made.  Requires higher degree of application knowledge In order to maintain Linear scripts, a higher degree of application knowledge is required. This is directly related to the fact that there is little modularity. Modularity helps to make it clear what a block of application statements is responsible for doing. The loose statements in Linear Scripts make it a little less clear. So when it comes to analyzing and/or debugging a script the automator must first burn a substantial amount of time gaining a mental understanding of what a given block of statements does in order to effectively maintain the script.

Worksoft Certify
Worksoft Certify is an automated functional testing solution for SAP lifecycle management and cross-platform business process validation. Worksoft Certify was first to market with a solution designed for business analysts that is powerful enough to validate cross-platform business processes without requiring coding. Worksoft Certify eliminates custom coding and programming, a requirement of most legacy test automation products, making it fast and easy to implement and maintain. Using an object-driven approach rather than generating scripts or code, Worksoft Certify validates business process workflows using a data model of fields, screens, and transactions making it easy to keep pace with dynamic changes. Overview Worksoft Certify Connect is a family of web-services based applications that extend and enhance Worksoft Certify to provide a comprehensive lifecycle management solution for SAP by providing web-based access and programming interfaces to connect with extended audiences and external applications. Worksoft Certify Connect provides advanced functionality that extends Worksoft Certify in order to solve the following set of enterprise challenges:

Enterprise Reporting
Connect your data and information to an extended audience, including executive management, via a web-based dashboard that includes rich graphical displays of important metrics and key performance indicators.
y y

Certify Executive Dashboard Certify Business Process Procedure (BPP)

Compliance
Connect your business validation maps, tests, and results to IT auditors and other compliance and security stakeholders to illustrate the relevant artifacts of lifecycle management.
y y y

Certify Executive Dashboard Certify Business Process Procedure (BPP) Certify Electronic Signature

Enterprise Integration
Connect Worksoft Certify with your other enterprise test management solutions to preserve and enhance your existing investments. Certify Connect provides bi-directional integration which allows

you to increase your automation capabilities while extending your existing test and validation processes. Worksoft supports the following enterprise solutions:
y y y

SAP Solution Manager IBM Rational Quality Manager HP Quality Center

Worksoft Application Lifecycle Management Solution can consist of a number of additional packages to suit the client's needs: Certify Learn Certify Data Certify Impact Certify Executive Dashboard Certify Executive Dashboard provides graphical key performance indicators (KPIs) and drill-down reporting capabilities to ensure that users at all levels have the information and visibility needed for improved decision making and optimal risk management. By identifying and quantifying the overall effectiveness of business process validation tasks, this solution provides key decision-making tools with up-to-the-minute project status and test results through customizable, web-based charts and graphs. Key Features
y y y y y

Track and report on test requirements including parent-child dependencies Detail reports and summary reports on test execution and test results Historical trending that compares execution results over time Email communication for easy team collaboration and task delegation Integration with Certify BPP and Certify eSignature for a complete compliance solution

Certify Executive Dashboard is an integral solution that reduces risk and simplifies compliance initiatives for your complex lifecycle and test automation projects. Certify Business Process Procedure (BBP) Certify Business Process Procedure (BPP) is a web-based reporting product that automatically generates comprehensive reports for development, quality assurance, training and compliance. Reports capture and catalog a sequential set of test steps, grouped by test process and application screen.

Key Features
y y y y y

Dynamic reports: on test process execution and test results Screen captures included for a detailed audit and compliance trail Complements any Worksoft Certify application and project Browser-based user interface for multi-user collaboration Supports one or multiple Certify instances and databases

Certify Business Process Procedure provides. dynamic reporting against your Worksoft Certify test. executions and test results thus providing a complete audit trail of the business process validation activities. Certify eSignature Certify Electronic Signature provides web-based workflow and approval for Worksoft Certify assets including test requirements, test processes and test results. Certify Electronic Signature records and preserves test assets and generates documentation to aid in compliance including regulations such as FDAs 21 CFR Part 11. Key Features
y y y y y

Role-based access controls and permissions Binding electronic signatures ensures nonrepudiation Customizable, automatic notification and reminder features Pre-built set of project templates for custom tailoring Web-based interface for team collaboration and workflow

Certify Electronic Signature secures your valuable testing assets against unauthorized change and accidental removal, providing a permanent system of record for reporting, auditing and compliance.

Products Worksoft Certify Certify Performance Test

Code-free, automated functional testing Code-free load and stress solution for SAP lifecycle management and testing solution specifically designed for cross-platform business process validation. SAP applications and built within the native SAP user interface. Certify Impact Analyze SAP transports to identify type of change, what is affected and what test should be run. Certify Electronic Signature Workflow and approval for Worksoft Certify assets including test requirements, test processes and test results. Certify Data Automatically extracts and moves SAP production data for test, development and training needs. Certify Executive Dashboard Provides management visibility into lifecycle and test management reports, metrics and KPIs. Certify Business Process Procedure Automatically generates comprehensive reports for development, quality assurance, training and compliance.

Certify Learn Automatically extracts SAP application objects to create a fully populated object repository in Worksoft Certify.