Assurance Services -Unit 01

Testing Definitions

Glossary: TESTING DEFINITIONS

Assurance Service Unit 01

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

This glossary of software testing terms and conditions is a compilation of knowledge, gathered from many different sources for reference only. If you have any questions or queries about the contents of this glossary, please contact TCS Assurance Service Practices, ISU 01.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

GLOSSARY: TESTING DEFINITIONS Acceptance Testing: Formal testing conducted to enable a user, customer, or other authorized entity to determine whether to accept a system or component. Normally performed to validate the software meets a set of agreed acceptance criteria. Accessibility Testing: Verifying a product is accessible to the people having disabilities (visually impaired, hard of hearing etc.) Actual Outcome: The actions that are produced when the object is tested under specific conditions. Ad hoc Testing: Testing carried out in an unstructured and improvised fashion. Performed without clear expected results, ad hoc testing is most often used as a compliment to other types of testing. See also Monkey Testing.

Alpha testing: It is simulated or actual operational testing by potential users/customers or an independent test team at the developers' site, but outside the development organization. Alpha-testing is often employed for off-the-shelf software as a form of acceptance testing.
Agile Testing: Testing practice for projects using Agile methodologies, treating development as the customer of testing and emphasizing a testfirst design philosophy. In agile development testing is integrated throughout the lifecycle, testing the software throughout its development. See also Test Driven Development. Application Binary Interface (ABI): Describes the low level interface between an application program and the operating system, between an application and its libraries, or between component parts of the application. An ABI differs from an application programming interface (API) in that an API defines the interface between source code and libraries, so that the same source code will compile on any system supporting that API, whereas an ABI allows compiled object code to function without changes on any system using a compatible ABI. Application Development Lifecycle The process flow during the various phases of the application development life cycle The Design Phase depicts the design phase up to the point of starting development. Once all of the requirements have been gathered, analyzed,

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

verified, and a design has been produced, we are ready to pass on the programming requirements to the application programmers. Application Programming Interface (API): Provided by operating systems or libraries in response to support requests for services to be made of it by computer programs Automated Software Quality (ASQ): The use of software tools, such as automated testing tools, to improve software quality Automated Software Testing: The use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions, without manual intervention Automated Testing Tools: Software tools used by development teams to automate and streamline their testing and quality assurance process. Backus-Naur Form (BNF): A metasyntax used to express context-free grammars: that is, a formal way to describe formal languages. BNF is widely used as a notation for the grammars of computer programming languages, instruction sets and communication protocols, as well as a notation for representing parts of natural language grammars. Many textbooks for programming language theory and/or semantics document the programming language in BNF.

Backlog: Work waiting to be done; for IT this includes new systems to

be developed and enhancements to existing systems. To be included in the development backlog, the work must have been cost-justified and approved for development.

Baseline: A specification or software product that has been formally

reviewed or agreed upon, that thereafter serves as a basis for further development, and that can be changed only through a formal change control process. Basic Block: A sequence of one or more consecutive, executable statements containing no branches. Basis Path Testing: A white box test case design technique that fulfills the requirements of branch testing & also tests all of the independent paths that could be used to construct any arbitrary path through the computer program. Basis Test Set: A set of test cases derived from Basis Path Testing.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Baseline: The point at which some deliverable produced during the software engineering process is put under formal change control. Bebugging: A popular software engineering technique used to measure test coverage. Known bugs are randomly added to a program source code and the programmer is tasked to find them. The percentage of the known bugs not found gives an indication of the real bugs that remain processes against best practices, or competitive practices, to help define superior performance of a product, service, or support process. Benchmark Testing: Benchmark testing is a normal part of the application development life cycle. It is a team effort that involves both application developers and database administrators (DBAs), and should be performed against your application in order to determine current performance and improve it. An application benchmark tests the same throughput capabilities under conditions that is closer production conditions. Benchmarking is helpful in understanding how the database manager responds under varying conditions. You can create scenarios that test deadlock handling, utility performance, different methods of loading data, transaction rate characteristics as more users are added, and even the effect on the application of using a new release of the product. Benchmark Testing Methods: Benchmark tests are based on a repeatable environment so that the same test run under the same conditions will yield results that you can legitimately compare. Characteristics of good benchmarks or measurements include: Tests are repeatable. Each iteration of a test starts in the same system state. No other functions or applications are active in the system unless the scenario includes some amount of other activity going on in the system. The hardware and software used for benchmarking match your production environment. For benchmarking, you create a scenario and then applications in this scenario several times, capturing key information during each run. Capturing key information after each run is of primary importance in determining the changes that might improve performance of both the application and the database.

Benchmarking: Comparing your companys products, services, or

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Behavior: The combination of input values and preconditions along with the required response for a function of a system. The full specification of a function would normally comprise one or more behaviors.

Beta testing: Operational testing by potential and/or existing

users/customers at an external site not otherwise involved with the developers, to determine whether or not a component or system satisfies user/customer needs and fits within the business process. Beta testing is often employed as a form of external acceptance testing for off-theshelf software to acquire feedback from the market. It comes after alpha testing. Big-Bang Testing: An inappropriate approach to integration testing in which you take the entire integrated system and test it as a unit. Can work well on small systems but is not favorable for larger systems because it may be difficult to pinpoint the exact location of the defect when a failure occurs. Binary Portability Testing: Testing an executable application for portability across system platforms and environments, usually for conformation to an ABI specification

Black Box Testing: Testing either functional or non-functional,

without reference to the internal structure of the component or system testing without knowledge of the internal workings of the item being tested. Block Matching: Automated matching logic applied to data and transaction driven websites to automatically detect block s of related data. This enables repeating elements to be treated correctly in relation to other elements in the block without the need for special coding Bottom Up Testing: An approach to integration testing where the lowest level components are tested first, then used to facilitate the testing of higher level components. Boundary Testing: Tests focusing on the boundary or limits of the software are tested. Boundary Value: An input value or output value which is on the boundary between equivalence classes, or an incremental distance either side of the boundary. Boundary Value Analysis: In boundary value analysis, test cases are generated using the extremes of the input domain, e.g. maximum,

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

minimum, just inside/outside boundaries, typical values, and error values. Boundary Value Coverage: The percentage of boundary values which have been exercised by a test case suite. Branch: A conditional transfer of control from any statement to any other statement in a component, or an unconditional transfer of control from any statement to any other statement in the component except the next statement, or when a component has more than one entry point, a transfer of control to an entry point of the component. Branch Condition Coverage: The percentage of branch condition outcomes in every decision that has been tested. Branch Condition Combination Coverage: The percentage of combinations of all branch condition outcomes in every decision that has been tested. Branch Condition Combination Testing: A test case design technique in which test cases are designed to execute combinations of branch condition outcomes. Branch Condition Testing: A technique in which test cases are designed to execute branch condition outcomes. Branch Testing: A test case design technique for a component in which test cases are designed to execute branch outcomes. Breadth Testing: A test suite that exercises the full functionality of a product but does not test features in detail. Bug: A fault in a program which causes the program to perform in an unintended. See fault. Build: Application delivered to test. It can be called build/drop. Capture/Playback Tool: A test tool that records test input as it is sent to the software under test. The input cases stored can then be used to reproduce the test at a later time. CAST: Acronym for computer-aided software testing. Automated software testing in one or more phases of the software life-cycle

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Cause-Effect Graph: A graphical representation of inputs or stimuli (causes) with their associated outputs (effects), which can be used to design test cases. Capability Maturity Model for Software (CMM): The CMM is a process model based on software best-practices effective in large-scale, multiperson projects. The CMM has been used to assess the maturity levels of organization areas as diverse as software engineering, system engineering, project management, risk management, system acquisition, information technology (IT) or personnel management, against a scale of five key processes, namely: Initial, Repeatable, Defined, Managed and Optimized. Capability Maturity Model Integration (CMMI) Capability Maturity Model Integration (CMMI) is a process improvement approach that provides organizations with the essential elements of effective processes. It can be used to guide process improvement across a project, a division, or an entire organization. CMMI helps integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes. Seen by many as the successor to the CMM, the goal of the CMMI project is to improve the usability of maturity models by integrating many different models into one framework. Certification: The process of confirming that a system or component complies with its specified requirements and is acceptable for operational use. Code Complete: A phase of development where functionality is implemented in its entirety; bug fixes are all that are left all functions found in the Functional Specifications have been implemented attributes of an application system. Well-constructed checklists cause evaluation of areas, which are prone to problems. It both limits the scope of the test and directs the tester to the areas in which there is a high probability of a problem. Code Coverage: A measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that looks at the code directly and as such comes under the heading of white box testing. To measure how well a program has been tested, there are a number of coverage criteria the main ones being:

Checklists: A series of probing questions about the completeness and

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Functional Coverage has each function in the program been tested? Statement Coverage has each line of the source code been tested? Condition Coverage has each evaluation point (i.e. a true/false decision) been tested? Path coverage has every possible route through a given part of the code been executed? Entry/exit Coverage has every possible call and return of the function been tested?

Code-Based Testing: The principle of structural code based testing is to have each and every statement in the program executed at least once during the test. Based on the premise that one cannot have confidence in a section of code unless it has been exercised by tests, structural code based testing attempts to test all reachable elements in the software under the cost and time constraints. The testing process begins by first identifying areas in the program not being exercised by the current set of test cases, follow by creating additional test cases to increase the coverage. Code-Free Testing: Next generation software testing technique from Original Software which does not require complicated scripting language to learn Instead, a simple point and click interface is used to significantly simplify the process of test creation. Code Inspection: A formal testing technique where the programmer reviews source code with a group who ask questions analyzing the program logic, analyzing the code with respect to a checklist of historically common programming errors, and analyzing its compliance with coding standards Code Walkthrough: A formal testing technique where source code is traced by a group with a small set of test cases, while the state of program variables is manually monitored, to analyze the programmer's logic and assumptions Coding: The generation of source code. Compatibility Testing: The process of testing to understand if software is compatible with other elements of a system with which it should operate, e.g. browsers, Operating Systems, or hardware. Complete Path Testing: See exhaustive testing.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Component: A minimal software item for which a separate specification is available. Component Testing: The testing of individual software components. Component Specification: A description of a component's function in terms of its output values for specified input values under specified preconditions. Computation Data Use: A data use not in a condition. Also called C-use. Concurrent Testing: Multi-user testing geared towards determining the effects of accessing the same application code, module or database records. See Load Testing Condition: A Boolean expression containing no Boolean operators. For instance, A<B is a condition but A and B is not. Condition Coverage: See branch condition coverage. Condition Outcome: The evaluation of a condition to TRUE or FALSE. Conformance Criterion: Some method of judging whether or not the component's action on a particular specified input value conforms to the specification. Conformance Testing: The process of testing to determine whether a system meets some specified standard. To aid in this, many test procedures and test setups have been developed, either by the standard's maintainers or external organizations, specifically for testing conformance to standards. Conformance testing is often performed by external organizations; sometimes the standards body itself, to give greater guarantees of compliance. Products tested in such a manner are then advertised as being certified by that external organization as complying with the standard. Context Driven Testing: The context-driven school of software testing is similar to Agile Testing that advocates continuous and creative evaluation of testing opportunities in light of the potential information revealed and the value of that information to the organization right now. Control Flow: An abstract representation of all possible sequences of events in a program's execution. Control Flow Graph: The diagrammatic representation of the possible alternative control flow paths through a component. TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Control Flow Path: See path. Conversion Testing: Testing of programs or procedures used to convert data from existing systems for use in replacement systems. Correctness: The degree to which software conforms to its specification. Coverage: The degree, expressed as a percentage, to which a specified coverage item has been tested. Coverage Item: An entity or property used as a basis for testing. Cyclomatic Complexity: A software metric (measurement). It was developed by Thomas McCabe and is used to measure the complexity of a program. It directly measures the number of linearly independent paths through a program's source code Data Case: Data relationship model simplified for data extraction and reduction purposes in order to create test data. Data Definition: An executable statement where a variable is assigned a value. Data Definition C-use Coverage: The percentage of data definition Cuse pairs in a component that is exercised by a test case suite. Data Definition C-use Pair: A data definition and computation data use, where the data use uses the value defined in the data definition. Data Definition P-use Coverage: The percentage of data definition P-use pairs in a component that is tested. Data Definition P-use Pair: A data definition and predicate data use, where the data use uses the value defined in the data definition. Data Definition-use Coverage: The percentage of data definition-use pairs in a component that is exercised by a test case suite. Data Definition-use Pair: A data definition and data use, where the data use uses the value defined in the data definition. Data Definition-use Testing: A test case design technique for a component in which test cases are designed to execute data definitionuse pairs.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Data Dictionary: A database that contains definitions of all data items defined during analysis. Data Driven Testing: A framework where test input and output values are read from data files and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status and information is all coded in the test script. This is similar to keyworddriven testing in that the test case is contained in the data file and not in the script; the script is just a "driver," or delivery mechanism, for the data. Unlike in table-driven testing, though, the navigation data isn't contained in the table structure. In data-driven testing, only test data is contained in the data files. Data Flow Diagram: A modeling notation that represents a functional decomposition of a system. Data Flow Coverage: Test coverage measure based on variable usage within the code. Examples are data definition- use coverage, data definition P-use coverage, data definition C-use coverage, etc. Data Flow Testing: Data-flow testing looks at the lifecycle of a particular piece of data (i.e. a variable) in an application. By looking for patterns of data usage, risky areas of code can be found and more test cases can be applied. Data Protection: Technique in which the condition of the underlying database is synchronized with the test scenario so that differences can be attributed to logical changes. This technique also automatically resets the database after tests - allowing for a constant data set if a test is re-run. Data Protection Act: UK Legislation surrounding the security, use and access of an individuals information. May impact the use of live data used for testing purposes. Data Use: An executable statement where the value of a variable is accessed. Database Testing: The process of testing the functionality, security, and integrity of the database and the data held within. Functionality of the database is one of the most critical aspects of an application's quality; problems with the database could lead to data loss or security breaches, and may put a company at legal risk depending on the type of data being stored. TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Debugging: A methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece of electronic hardware thus making it behaves as expected. Debugging tends to be harder when various subsystems are tightly coupled, as changes in one may cause bugs to emerge in another. Decision: A program point at which the control flow has two or more alternative routes. Decision Condition: A condition held within a decision. Decision Coverage: The percentage of decision outcomes that have been exercised by a test case suite. Decision Outcome: The result of a decision. Defect: Definition differs from different view point: From the producer's viewpoint a defect is a product requirement that has not been met or a product attribute possessed by a product or a function performed by a product that is not in the statement of requirements that define the product. From the customer's viewpoint a defect is anything that causes customer dissatisfaction, whether in the statement of requirements or not. A flaw in the component or system that can cause the component or system to fail to perform its required function (e.g., an incorrect statement or data definition A defect, if encountered during execution, may cause the failure of the component or system Delta Release: A delta, or partial, release is one that includes only those areas within the release unit that have actually changed or are new since the last full or delta release. For example, if the release unit is the program, a delta release contains only those modules that have changed, or are new, since the last full release of the program or the last delta release of certain modules. Dependency Testing: Examines an application's requirements for preexisting software, initial states and configuration in order to maintain proper functionality. Depth Testing: A test that exercises a feature of a product in full detail. Desk Checking: The testing of software by the manual simulation of its execution.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Design-Based Testing: Designing tests based on objectives derived from the architectural or detail design of the software (e.g., tests that execute specific invocation paths or probe the worst case behavior of algorithms). Disaster Recovery Testing: Ensures that the system's controls and manual and automated back up and recovery mechanism work as expected Dirty Testing: Testing which demonstrates that the system under test does not work. (Also known as negative testing) Documentation Testing: Testing concerned with the accuracy of documentation. Domain: The set from which values are selected. Domain Expert: A person who has significant knowledge in a specific domain. Domain Testing: Domain testing is the most frequently described test technique. The basic notion is that you take the huge space of possible tests of an individual variable and subdivide it into subsets that are (in some way) equivalent. Then you test a representative from each subset. Downtime: Total period that a service or component is not operational. Dynamic Analysis: the examination of the physical response from the system to variables that are not constant and change with time. Dynamic Testing: Testing that involves the execution of the software of a component or system. Emulator: A device that duplicates (provides an emulation of) the functions of one system using a different system, so that the second system behaves like (and appears to be) the first system. This focus on exact reproduction of external behavior is in contrast to simulation, which can concern an abstract model of the system being simulated, often considering internal state. Endurance Testing: Checks for memory leaks or other problems that may occur with prolonged execution. End-to-End Testing: End-to-End testing which is similar to Systems testing involves testing of a complete application environment in a situation that mimics real world use, such as interacting with a

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

database, using network communications, or interacting with other hardware, applications, or systems if appropriate Environmental Test: A test to verify that a system works the same, or at least in an appropriately similar manner, across different platforms, operating systems, DBMS's, mixes of concurrent applications, or client/server network configurations. Entry Point: The first executable statement within a component. Equivalence Class: A mathematical concept, an equivalence class is a subset of given set induced by an equivalence relation on that given set. (If the given set is empty, then the equivalence relation is empty, and there are no equivalence classes; otherwise, the equivalence relation and its concomitant equivalence classes are all non-empty.) Elements of an equivalence class are said to be equivalent, under the equivalence relation, to all the other elements of the same equivalence class. Equivalence Partitioning: Leverages the concept of "classes" of input conditions. A "class" of input could be "City Name" where testing one or several city names could be deemed equivalent to testing all city names. In other words each instance of a class in a test covers a large set of other possible tests. Equivalence Partition Coverage: The percentage of equivalence classes generated for the component, which have been tested. Equivalence Partition Testing: A test case design technique for a component in which test cases are designed to execute representatives from equivalence classes Error: A Human action that produces the incorrect result Error Guessing: Error Guessing involves making an itemized list of the errors expected to occur in a particular area of the system and then designing a set of test cases to check for these expected errors. Error guessing is more testing art than testing science but can be very effective given a tester familiar with the history of the system. Error Seeding: The process of injecting a known number of "dummy" defects into the program and then check how many of them are found by various inspections and testing. If, for example, 60% of them are found, the presumption is that 60% of other defects have been found as well. See Bebugging.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Evaluation Report: A document produced at the end of the test process summarizing all testing activities and results. It also contains an evaluation of the test process and lessons learned. Executable statement: A statement which, when compiled, is translated into object code, which will be executed procedurally when the program is running and may perform an action on program data. Exercised: A program element is exercised by a test case when the input value causes the execution of that element, such as a statement, branch, or other structural element. Exhaustive Testing: Testing which covers all combinations of input values and preconditions for an element of the software under test. Exit Point: The last executable statement within a component. Expected Outcome: See predicted outcome. Expert System: A domain specific knowledge base combined with an inference engine that processes knowledge encoded in the knowledge base to respond to a user's request for advice. Expertise: Specialized domain knowledge, skills, tricks, shortcuts and rules-of-thumb that provide an ability to rapidly and effectively solve problems in the problem domain. Failure: Non performance or deviation of the software from its expected delivery or service. Failover Testing: Failover Testing in the midrange environments involves the ability to move hardware and software resources from one host to another, which provides for high availability. Feasible Path: A path for which there exists a set of input values and execution conditions which causes it to be executed. Feature Testing: A method of testing which concentrates on testing one feature at a time. Firing a Rule: A rule fires when the if part (premise) is proven to be true. If the rule incorporates an else component, the rule also fires when the if part is proven to be false.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Fit for Purpose Testing: Validation carried out to demonstrate that the delivered system can be used to carry out the tasks for which it was designed and acquired. Forward Chaining: Applying a set of previously determined facts to the rules in a knowledge base to see if any of them will fire. Full Release: All components of the release unit that are built, tested, distributed and implemented together. See also delta release. Functional Specification: The document that describes in detail the characteristics of the product with regard to its intended capability. Functional Decomposition: A technique used during planning, analysis and design; creates a functional hierarchy for the software. Functional Decomposition broadly relates to the process of resolving a functional relationship into its constituent parts in such a way that the original function can be reconstructed (i.e., recomposed) from those parts by function composition. In general, this process of decomposition is undertaken either for the purpose of gaining insight into the identity of the constituent components (which may reflect individual physical processes of interest, for example), or for the purpose of obtaining a compressed representation of the global function, a task which is feasible only when the constituent processes possess a certain level of modularity (i.e. independence or non-interaction). Functional Requirements: Define the internal workings of the software: that is, the calculations, technical details, data manipulation and processing and other specific functionality that show how the use cases are to be satisfied. They are supported by non-functional requirements, which impose constraints on the design or implementation (such as performance requirements, security, quality standards, or design constraints). Functional Specification: A document that describes in detail the characteristics of the product with regard to its intended features. Functionality Testing: The process of testing to determine the functionality of a software product Genetic Algorithms: Search procedures that use the mechanics of natural selection and natural genetics. It uses evolutionary techniques, based on function optimization and artificial intelligence, to develop a solution.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Glass Box Testing: A form of testing in which the tester can examine the design documents and the code as well as analyze and possibly manipulate the internal state of the entity being tested. Glass box testing involves examining the design documents and the code, as well as observing at run time the steps taken by algorithms and their internal data. Goal: The solution that the program or project is trying to reach. Gorilla Testing: An intense round of testing, quite often redirecting all available resources to the activity. The idea here is to test as much of the application in as short a period of time as possible. Graphical User Interface (GUI): A type of display format that enables the user to choose commands, starts programs, and see lists of files and other options by pointing to pictorial representations (icons) and lists of menu items on the screen. Gray (Grey) Box Testing: A testing technique that uses a combination of black box testing and white box testing. Gray box testing is not black box testing because the tester does know some of the internal workings of the software under test. In gray box testing, the tester applies a limited number of test cases to the internal workings of the software under test. In the remaining part of the gray box testing, one takes a black box approach in applying inputs to the software under test and observing the outputs Harness: A test environment comprised of stubs and drivers needed to conduct a test. Heuristics: The informal, judgmental knowledge of an application area that constitutes the "rules of good judgment" in the field. Heuristics also encompass the knowledge of how to solve problems efficiently and effectively, how to plan steps in solving a complex problem, how to improve performance, etc. High Order Tests: High-order testing checks that the software meets customer requirements and that the software, along with other system elements, meets the functional, behavioral, and performance requirements It uses black-box techniques and requires an outsider perspective. ITIL (IT Infrastructure Library): A consistent and comprehensive documentation of best practice for IT Service Management. ITIL consists of a series of books giving guidance on the provision of quality IT

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

services, and on the accommodation and environmental facilities needed to support IT. Implementation Testing: See Installation Testing. Incremental Testing: Partial testing of an incomplete product. The goal of incremental testing is to provide an early feedback to software developers. Independence: Separation of responsibilities which ensures the accomplishment of objective evaluation Independent Test: Test conducted by someone independent of the person or group responsible for the design and implementation of the software product to avoid introduction of bias. Independent test indicates functional testing, based on the user perspective, and to verify that the software product meets requirements. Independent Test Group (ITG): A group of people whose primary responsibility is to conduct software testing for other companies. Infeasible path: A path which cannot be exercised by any set of possible input values. Inference: Forming a conclusion from existing facts. Inference Engine: Software that provides the reasoning mechanism in an expert system. In a rule based expert system, typically implements forward chaining and backward chaining strategies. Infrastructure: The organizational artifacts needed to perform testing, consisting of test environments, automated test tools, office environment and procedures. Inheritance: The ability of a class to pass on characteristics and data to its descendants. Input: A variable (whether stored within a component or outside it) that is read by the component. Input Domain: The set of all possible inputs. Inspection: A group review quality improvement process for written material. It consists of two aspects; product (document itself) improvement and process improvement.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Installability: The ability of a software component or system to be installed on a defined target platform allowing it to be run as required. Installation includes both a new installation and an upgrade. Installability Testing: Testing whether the software or system installation being tested meets predefined installation requirements. Installation Guide: Supplied instructions on any suitable media, which guides the installer through the installation process. This may be a manual guide, step-by-step procedure, installation wizard, or any other similar process description. Installation Testing: Confirms that the application under test recovers from expected or unexpected events without loss of data or functionality. Events can include shortage of disk space, unexpected loss of communication, or power out conditions. Such testing focuses on what customers will need to do to install and set up the new software successfully and is typically done by the software testing engineer in conjunction with the configuration manager. Implementation testing is usually defined as testing which places a compiled version of code into the testing or pre-production environment, from which it may or may not progress into production. This generally takes place outside of the software development environment to limit code corruption from other future releases which may reside on the development network. Installation Wizard: Supplied software on any suitable media, which leads the installer through the installation process. It shall normally run the installation process, provide feedback on installation outcomes and prompt for options. Instrumentation: The insertion of additional code into the program in order to collect information about program behavior during program execution. Integration: The process of combining components into larger groups or assemblies Integration Testing: Testing performed to expose defects in the interfaces and in the interactions between integrated components or systems. Interface Testing: Integration testing where the interfaces between system components are tested. Isolation Testing: Component testing of individual components in isolation from surrounding components TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

KBS (Knowledge Based System): A domain specific knowledge base combined with an inference engine that processes knowledge encoded in the knowledge base to respond to a user's request for advice. Key Performance Indicator: Quantifiable measurements against which specific performance criteria can be set. Keyword Driven Testing: An approach to test script writing aimed at code based automation tools that separates much of the programming work from the actual test steps. The result is the test steps can be designed earlier and the code base is often easier to read and maintain. Knowledge Engineering: The process of codifying an expert's knowledge in a form that can be accessed through an expert system. Known Error: An incident or problem for which the root cause is known and for which a temporary Work-around or a permanent alternative has been identified LCSAJ: A Linear Code Sequence And Jump, consisting of the following three items (conventionally identified by line numbers in a source code listing): the start of the linear sequence of executable statements, the end of the linear sequence, and the target line to which control flow is transferred at the end of the linear sequence. LCSAJ Coverage: The percentage of LCSAJs of a component which are exercised by a test case suite. LCSAJ Testing: A test case design technique for a component in which test cases are designed to execute LCSAJs. Logic-Coverage Testing: Sometimes referred to as Path Testing, logiccoverage testing g attempts to expose software defects by exercising a unique combination of the program's statements known as a path. Load Testing: The process of creating demand on a system or device and measuring its response. Load testing generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the program's services concurrently. As such, this testing is most relevant for multi-user systems; often one built using a client/server model, such as web servers. Localization Testing: This term refers to making software specifically designed for a specific locality. This test is based on the results of TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

globalization testing, which verifies the functional support for that particular culture/locale. Localization testing can be executed only on the localized version of a product. The test effort during localization testing focuses on: Areas affected by localization, such as UI and content Culture/locale-specific, language-specific, and region-specific areas In addition, localization testing should include: Basic functionality tests Setup and upgrade tests run in localized environment Plan application and hardware compatibility tests according to the product's target region. Log: A chronological record of relevant details about the execution of tests. Loop Testing: Loop testing is the testing of a resource or resources multiple times under program control can be modified to correct faults, modified to meet new requirements, modified to make future maintenance easier, or adapted to a changed environment. Maintenance Requirements: A specification of the required maintenance needed for the system/software. The released software often needs to be revised and/or upgraded throughout its lifecycle. Therefore it is essential that the software can be easily maintained, and any errors found during re-work and upgrading. Within traditional software testing techniques, script maintenance is often a problem as it can be very complicated and time consuming to ensure correct maintenance of the software as the scripts these tools use need updating every time the application under test changes. See Code-Free Testing and Self Healing Scripts. Manual Testing: The oldest type of software testing. Manual testing requires a tester to perform manual test operations on the test software without the help of test automation. Metric: A standard of measurement. Software metrics are the statistics describing the structure or content of a program. A metric should be a real objective measurement of something such as number of bugs per lines of code. Modified Condition/Decision Coverage: The percentage of all branch condition outcomes that independently affect a decision outcome that have been exercised by a test case suite

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Modified Condition/Decision Testing: A test case design technique in which test cases are designed to execute branch condition outcomes that independently affect a decision outcome Monkey Testing: Testing a system or an application on the fly, i.e. a unit test with no specific end result in mind. Multiple Condition Coverage: See Branch Condition Combination Coverage. Mutation Analysis: A method to determine test case suite thoroughness by measuring the extent to which a test case suite can discriminate the program from slight variants (mutants) of the program. See also Error Seeding. Mutation Testing: Testing done on the application where bugs are purposely added to it. See Bebugging. N-switch Coverage: The percentage of sequences of N-transitions that have been tested. N-switch Testing: A form of state transition testing in which test cases are designed to execute all valid sequences of N-transitions. N-transitions: A sequence of N+ transitions. N+1 Testing: A variation of regression testing. Testing conducted with multiple cycles in which errors found in test cycle N are resolved and the solution is retested in test cycle N+1. The cycles are typically repeated until the solution reaches a steady state and there are no errors. See also Regression Testing. Natural Language Processing (NLP): A computer system to analyze, understand and generate natural human languages. Negative Testing: Testing a system or application using negative data. (For example testing a password field that requires a minimum of 9 characters by entering a password of 6) Neural Network: A system modeled after the neurons (nerve cells) in a biological nervous system. A neural network is designed as an interconnected system of processing elements, each with a limited number of inputs and outputs. Rather than being programmed, these systems learn to recognize patterns.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Non-functional Requirements Testing: Testing of those requirements that do not relate to functionality. i.e. performance, usability, etc. Normalization: A technique for designing relational database tables to minimize duplication of information and, in so doing, to safeguard the database against certain types of logical or structural problems, namely data anomalies. Object: A software structure which represents an identifiable item that has a well-defined role in a problem domain. Object Orientated: An adjective applied to any system or language that supports the use of objects. Objective: The purpose of the specific test being undertaken. Operational Testing: Testing performed by the end-user inn software in its normal operating environment. Oracle: A mechanism to produce the predicted outcomes to compare with the actual outcomes of the software under test. Outcome: The result or visible effect of a test. Output: A variable (whether stored within a component or outside it) that is written to by the component. Output Domain: The set of all possible outputs. Output Value: An instance of an output. Page Fault: A program interruption that occurs when a page that is marked not in real memory is referred to by an active page. Pair Programming: A software development technique that requires two programmers to participate in a combined development effort at one workstation. Each member performs the action the other is not currently doing: for example, while one types in unit tests, the other thinks about the class that will satisfy the test. The person who is doing the typing is known as the driver while the person who is guiding is known as the navigator. It is often suggested for the two partners to switch roles at least every half-hour or after a unit test is made. It is also suggested to switch partners at least once a day.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Pair Testing: In much the same way as Pair Programming, two testers work together to find defects. Typically, they share one computer and trade control of it while testing. Pair wise Testing: A combinatorial software testing method that, for each pair of input parameters to a system (typically, a software algorithm) tests all possible discrete combinations of those parameters. Using carefully chosen test vectors, this can be done much faster than an exhaustive search of all combinations of all parameters, by "parallelizing" the tests of parameter pairs. Parallel Testing: A test where the existing and new systems are operated in parallel and the before-and-after results are compared. Partial Test Automation: The process of automating parts but not all of the software testing process. If, for example, an oracle cannot reasonably be created, or if fully automated tests would be too difficult to maintain, then a software tools engineer can instead create testing tools to help human testers perform their jobs more efficiently. Testing tools can help automate tasks such as product installation, test data creation, GUI interaction, problem detection (consider parsing or polling agents equipped with oracles), defect logging, etc., without necessarily automating tests in an end-to-end fashion. Pass: Software has deemed to have passed a test if the actual results of the test matched the expected results. Pass/Fail Criteria: Decision rules used to determine whether an item under test has passed or failed a test. Path: A sequence of executable statements of a component, from an entry point to an exit point. Path Coverage: The percentage of paths in a component exercised by a test case suite. Path Sensitizing: Choosing a set of input values to force the execution of a component to take a given path. Path Testing: Used as either black box or white box testing, the procedure itself is similar to a walk-through. First, a certain path through the program is chosen. Possible inputs and the correct result are written down. Then the program is executed by hand, and its result is compared to the predefined. Possible faults have to be written down at once.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Performance: The degree to which a system or component accomplishes its designated functions within given constraints regarding processing time and throughput rate Performance Testing: The process of testing to determine the performance of a particular product Portability: The ease with which the system/software can be transferred from one hardware or software environment to another. Portability Requirements: A specification of the required portability for the system/software. Portability Testing: The process of testing the ease with which a software component can be moved from one environment to another. This is typically measured in terms of the maximum amount of effort permitted. Results are expressed in terms of the time required to move the software and complete data conversion and documentation updates. Post condition: Environmental and state conditions that must be fulfilled after the execution of a test or test procedure. Positive Testing: Testing aimed at showing whether the software works in the way intended. See also Negative Testing. Precondition: Environmental and state conditions which must be fulfilled before the component can be executed with a particular input value. Predicate: A logical expression which evaluates to TRUE or FALSE, normally to direct the execution path in code. Predication: The choice to execute or not to execute a given instruction. Predicted Outcome: The behavior expected by the specification of an object under specified conditions Priority: The level of business importance assigned to an individual item or test. Predication: The choice to execute or not to execute a given instruction. Predicted Outcome: The behavior expected by the specification of an object under specified conditions TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Priority: The level of business importance assigned to an individual item or test. Process: A course of action which turns inputs into outputs or results. Process Cycle Test: A black box test design technique in which test cases are designed to execute business procedures and processes. Progressive Testing: Testing of new features after regression testing of previous features. Project: A planned undertaking for presentation of results at a specified time in the future. Prototyping: A strategy in system development in which a scaled down system or portion of a system is constructed in a short time, then tested and improved upon over several iterations. Pseudo-Random: A series which appears to be random but is in fact generated according to some prearranged sequence. Quality Assurance: The activity of providing evidence needed to establish confidence among all concerned, that quality-related activities are being performed effectively. All those planned or systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality For software development organizations, TMM (Testing Maturity Model) standards are widely used to measure the Quality Assurance. These standards can be divided in to 5 steps, which a software development company can achieve by performing different quality improvement activities within the organization. Quality Attribute: A feature or characteristic that affects an items quality. Quality Audit: A systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives. Quality Circle: A group of individuals with related interests that meet at regular intervals to consider problems or other matters related to the quality of outputs of a process and to the correction of problems or to the improvement of quality.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Quality Control: The operational techniques and the activities used to fulfill and verify requirements of quality. Quality Management: That aspect of the overall management function that determines and implements the quality policy. Direction and control with regard to quality generally includes the establishment of the quality policy and quality objectives, quality planning, quality control, quality assurance and quality improvement. Quality Conundrum: Resource, risk and application time to- market are often in conflict as IS teams strive to deliver quality applications within their budgetary constraints. This is the quality conundrum. Quality Policy: The overall intentions and direction of an organization as regards quality as formally expressed by top management. Quality System: The organizational structure, responsibilities, procedures, processes, and resources for implementing quality management. Query: A question. Often associated with an SQL query of values in a database. Queuing Time: Incurred when the device, which a program wishes to use, is already busy. The program therefore has to wait in a queue to obtain service from that device. ROI: Return on Investment. A performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments. To calculate ROI, the benefit (return) of an investment is divided by the cost of the investment; the result is expressed as a percentage or a ratio. Ramp Testing: Continuously raising an input signal until the system breaks down. Random Testing: A black-box testing approach in which software is tested by choosing an arbitrary subset of all possible input values. Random testing helps to avoid the problem of only testing what you know will work. Re-testing: Testing that runs test cases that failed the last time they were run, in order to verify the success of corrective actions.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Recoverability: The capability of the software product to re-establish a specified level of performance and recover the data directly affected in case of failure Recovery Testing: Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems. Recovery testing evaluates the contingency features built into the application for handling interruptions, and for returning to specific points in the application processing. Any backup, restoration, and restart capabilities are also tested here. This test may be conducted by the test team during system test. Recreation Materials: A script or set of results containing the steps required to reproduce a desired outcome. Regression Testing: A test to ensure that all unmodified functions still operate as expected after a change has been introduced. Regression testing is used to validate that everything will work together after all changes and fixes have been made. Relational Operator: Conditions such as is equal to or is less than that link an attribute name with an attribute value in a rules premise to form logical expressions that can be evaluated true or false. Release Candidate: A pre-release version, which contains the desired functionality of the final version, but which needs to be tested for bugs (which ideally should be removed before the final version is released). Release Note: A document identifying test items, their configuration, current status and other delivery information delivered by development to testing, and possibly other stakeholders, at the start of a test execution phase. Reliability: The ability of the system/software to perform its required functions under stated conditions for a specified period of time, or for a specified number of operations. Reliability Requirements: A specification of the required reliability for the system/software. Reliability Testing: Testing to determine whether the system/software meets the specified reliability requirements. Requirement: A capability that must be met or possessed by the system/software (requirements may be functional or non-functional). TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Requirements-based Testing: An approach to testing in which test cases are designed based on test objectives and test conditions derived from requirements. For example: tests that exercise specific functions or probe nonfunctional attributes such as reliability or usability. Result: The consequence or outcome of a test. Review: A process or meeting during which a work product, or set of work products, is presented to project personnel, managers, users or other interested parties for comment or approval. Risk: A chance of negative consequences. Risk Management: Systematic application of procedures and practices to the tasks of identifying, analyzing, prioritizing, and controlling risk. Robustness: The degree to which a component or system can function correctly in the presence of invalid inputs or stressful environmental conditions. Root Cause: An underlying factor that caused a non-conformance and possibly should be permanently eliminated through process improvement. Rule: A statement of the form: if X then Y else Z. The if part is the rule premise, and the then part is the consequent. The else component of the consequent is optional. The rule fires when the if part is determined to be true or false. Rule Base: The encoded knowledge for an expert system. In a rule-based expert system, a knowledge base typically incorporates definitions of attributes and rules along with control information. Safety Testing: The process of testing to determine the safety of a software product. Sanity Testing: A test to ensure that the system has been installed correctly and is ready for testing or live operation Scalability Testing: Performance testing focused on ensuring the application under test gracefully handles increases in work load. Schedule: A scheme for the execution of test procedures. The test procedures are included in the test execution schedule in the order in which they are to be executed. TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Scrambling: Data obfuscation routine to de-identify sensitive data in test data environments to meet the requirements of the Data Protection Act and other legislation. See TestBench. Scribe: The person who has to record each defect mentioned and any suggestions for improvement during a review meeting, on a logging form. The scribe has to make ensure that the logging form is understandable. Script: See Test Script. Security: Preservation of availability, integrity and confidentiality of information. Security Requirements: A specification of the required security for the system or software. Security Testing: Verify the system meets the requirements for detecting, protecting from, and attempting security breaches. The six basic concepts that need to be covered by security testing are: confidentiality, integrity, authentication, authorization, availability and non-repudiation. Self-Healing Scripts: A next generation technique pioneered by Original Software which enables an existing test to be run over an updated or changed application, and intelligently modernize itself to reflect the changes in the application all through a point-and-click interface.

Simple Subpath: A subpath of the control flow graph in which no program part is executed more than necessary. Simulation: The representation of selected behavioral characteristics of one physical or abstract system by another system. Simulator: A device, computer program or system used during software verification, which behaves or operates like a given system when provided with a set of controlled inputs.
Smoke Testing: Tests that makes sure that system is stable enough to be tested more thoroughly. It is very good for daily builds Software Requirements Specification: A deliverable that describes all data, functional and behavioral requirements, all constraints, and all validation requirements for software. TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Specification: A description, in any suitable form, of requirements. Specification testing: An approach to testing wherein the testing is restricted to verifying the system/software meets an agreed specification. Specified input: An input for which the specification predicts an outcome. State Transition: A transition between two allowable states of a system or component. State Transition Testing: A test case design technique in which test cases are designed to execute state transitions Software Quality Assurance: The set of support activities including facilitation, training, measurement and analysis needed to provide adequate confidence that processes are established and continuously improved to produce products that meet specifications and are fit for use. Quality Assurance is a management responsibility, frequently performed by a staff function. QA personnel should not ever perform Quality control unless it is to validate quality control. QA activities in an IT environment would determine the need for acquire or help install system development methodologies, estimation processes, system maintenance processes, requirements definition processes, and testing processes and standards. Software Quality Control: Quality Control is the process by which the product quality is compared with applicable standards, and the action taken when non-conformance is detected. QC is a line function, and the work is done within a process to ensure that the work product conforms to standard and/or requirements. Its focus is defect detection and removal. The activities begin at the start of the software development process with reviews or requirements, and continue until all application testing is complete. Software Testing: Testing is a process of technical investigation, performed on behalf of stakeholders, that is intended to reveal qualityrelated information about the product with respect to the context in which it is intended to operate. This includes, but is not limited to, the process of executing a program or application with the intent of finding errors.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Static Testing: Testing of a component or system at specification or implementation level without execution of that software, e.g., reviews or static analysis Statement: An entity in a programming language which is typically the smallest indivisible unit of execution. Statement Coverage: The percentage of executable statements in a component that have been exercised by a test case suite. Statement Testing: A test case design technique for a component in which test cases are designed to execute statements. Statement Testing is a structural or white box technique, because it is conducted with reference to the code. Statement testing comes under Dynamic Analysis. In an ideal world every statement of every component would be fully tested. However, in the real world this hardly ever happens. In statement testing every possible statement is tested. Compare this to Branch Testing, where each branch is tested, to check that it can be traversed, whether it encounters a statement or not. Static Analysis: Analysis of a program carried out without executing the program. Static Analyzer: A tool that carries out static analysis. Static Code Analysis: The analysis of computer software that is performed without actually executing programs built from that software. In most cases the analysis is performed on some version of the source code and in the other cases some form of the object code. The term is usually applied to the analysis performed by an automated tool, with human analysis being called program understanding or program comprehension. Static Testing: A form of software testing where the software isn't actually used. This is in contrast to dynamic testing. It is generally not detailed testing, but checks mainly for the sanity of the code, algorithm, or document. It is primarily syntax checking of the code or and manually reading of the code or document to find errors. This type of testing can be used by the developer who wrote the code, in isolation. Code reviews, inspections and walkthroughs are also used. From the black box testing point of view, static testing involves review of requirements or specifications. This is done with an eye toward completeness or appropriateness for the task at hand. This is the verification portion of Verification and Validation.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Statistical Testing: A test case design technique in which a model is used of the statistical distribution of the input to construct representative test cases. Storage Testing: Testing that verifies the program under test stores data files in the correct directories and that it reserves sufficient space to prevent unexpected termination resulting from lack of space. This is external storage as opposed to internal storage. See TestBench Stress Testing: Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements to determine the load under which it fails and how. Often this is performance testing using a very high level of simulated load. Structural Coverage: Coverage measures based on the internal structure of the component. Structural Test Case Design: Test case selection that is based on an analysis of the internal structure of the component. Structural Testing: See structural test case design. Structured Basis Testing: A test case design technique in which test cases are derived from the code logic to achieve % branch coverage. Structured Walkthrough: See walkthrough. Stub: A skeletal or special-purpose implementation of a software module, used to develop or test a component that calls or is otherwise dependent on it. Subgoal: An attribute which becomes a temporary intermediate goal for the inference engine. Subgoal values need to be determined because they are used in the premise of rules that can determine higher level goals. Subpath: A sequence of executable statements within a component. Suitability: The capability of the software product to provide an appropriate set of functions for specified tasks and user objectives. Suspension Criteria: The criteria used to (temporarily) stop all or a portion of the testing activities on the test items. Symbolic Execution: A static analysis technique used to analyze if and when errors in the code may occur. It can be used to predict what code statements do to specified inputs and outputs. It is also important for TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

considering path traversal. It struggles when dealing with statements which are not purely mathematical. Symbolic Processing: Use of symbols, rather than numbers, combined with rules-of-thumb (or heuristics), in order to process information and solve problems. Syntax Testing: A test case design technique for a component or system in which test case design is based upon the syntax of the input. System Testing: Testing that attempts to discover defects that are properties of the entire system rather than of its individual components. TMM (Testing Maturity Model): A model developed by Dr Ilene Bernstein of the Illinois Institute of Technology, for judging the maturity of the software testing processes of an organization and for identifying the key practices that are required to increase the maturity of these processes. Such a maturity model provides: organization. A maturity model can be used as a benchmark for assessing different organizations for equivalent comparison. The model describes the maturity of the company based upon the project the company is handling and the related clients. Technical Review: A peer group discussion activity that focuses on achieving consensus on the technical approach to be taken. A technical review is also known as a peer review. Test Approach: The implementation of the test strategy for a specific project. It typically includes the decisions made that follow based on the (test) projects goal and the risk assessment carried out, starting points regarding the test process and the test design techniques to be applied. Test Automation: The use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process. Test Bed: Test Bed is the environment in which you are going to test the application and all the associated data regarding the environment essentially is the test bed data.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

TestBench: A suite of test automation solutions from Original Software that facilitate the management and manipulation of the database and visual layer components. TestBench addresses test verification, disk space, and data confidentiality issues. In addition, control of test data ensures that every test starts with a consistent data state, essential if the data is to be predictable at the end of testing. Test Case Specification: A document specifying a set of test cases (objective, inputs, test actions, expected results, and execution preconditions) for a test item Test Cases: A Set of Input Values, Execution Preconditions, Expected Result and Execution Post Conditions, developed for a particular test Conditions, such as to exercise a particular Program Path or to verify compliance with a specific requirement Test Conditions: An Item or event of a component or System that could be verified by one or more Test cases Examples: A Function, a Transaction, a Feature, Quality Attributes or Structural Events etc Test Cycle: Execution of the test process against a single identifiable release of the test object Test Case Design Technique: A method used to derive or select test cases. Test Case Suite: A collection of one or more test cases for the software under test. Test Charter: A statement of test objectives, and possibly test ideas. Test charters are amongst other used in exploratory testing. Test Comparison: The process of identifying differences between the actual results produced by the component or system under test and the expected results for a test. Test comparison can be performed during test execution (dynamic comparison) or after test execution. Test Completion Criterion: A criterion for determining when planned testing is complete, defined in terms of a test measurement technique. Test Comparator: A test tool that compares the actual outputs produced by the software under test with the expected outputs for that test case. Test Data: Set of input elements used in the testing process. The data that exists (for example, in a database) before a test is executed and that

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

affects or is affected by the component or System/ Application under Test Test Design Specification: A document specifying the test conditions (coverage items) for a test item, the detailed test approach and the associated high-level test cases. Test Data Management: The management of test data during tests to ensure complete data integrity and legitimacy from the beginning to the end of test. See TestBench Test-Drive Assist: A new concept in testing from Original Software that delivers active support for manual testing by compiling history on recent testing, making it easy to recreate and isolate software defects. With powerful tracking functions that operate in a non intrusive and natural fashion, testers can detect and highlight defects more quickly and effectively resulting in developers correcting these defects fast and efficiently. This means defects can simply be re-created at a touch of a button, whilst the results can eventually be used to build fully automated tests. Test-Drive: Next Generation automated testing solution from Original Software that allows technicians to define and execute sophisticated tests, without being hindered by complex programming languages. State of the art self updating technology automatically adapts tests to new software releases and upgrades. Test Driven Development: Testing methodology associated with Agile Programming in which every chunk of code is covered by unit tests which must all pass all the time, in an effort to eliminate unit-level and regression bugs during development. Practitioners of TDD write a lot of tests, i.e. an equal number of lines of test code to the size of the production code. Test Driver: A program or test tool used to execute software against a test case suite. Test Design Specification: A document that specifies the details of the test approach for a software feature or a combination of features and identifies the associated tests Test Environment: An environment containing hardware, instrumentation, simulators, software tools, and other support elements needed to conduct a test.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Test Execution: The process of carrying out a test, whether it is manually or using automated test software. Test Execution Phase: The period of time in the application development life cycle during which the components of a software product are executed, and the software product is evaluated to determine whether or not requirements have been satisfied. Test Execution Schedule: A scheme for the execution of test procedures. The test procedures are included in the test execution schedule in their context and in the order in which they are to be executed. Test Execution Technique: The method used to perform the actual test execution, e.g. manual, capture/playback tool, etc. Test First Design: Test-first design is one of the mandatory practices of Extreme Programming (XP).It requires that programmers do not write any production code until they have first written a unit test. Test Generator: A program that generates test cases in accordance to a specified strategy. Test Harness: A program or test tool used to execute a test. Also known as a Test Driver. Test Infrastructure: The organizational artifacts needed to perform testing, consisting of test environments, test tools, office environment and procedures. Test Level: A group of test activities that are organized and managed together. A test level is linked to the responsibilities in a project. Examples of test levels are component test, integration test, system test and acceptance test. Test Log: A chronological record of relevant details about the execution of tests Test Measurement Technique: A method used to measure test coverage items. Test Object: The component/system/application to be tested. Test Plan: A document describing the intended scope, approach, resources, and schedule of testing activities, It identifies test items, the

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

features to be tested, the testing tasks, the personnel performing each task, and any risks requiring contingency planning. Test Plan: A planning solution from Original Software that enables effective planning, team communications, and accurate tracking of every testing activity. Test Point Analysis: A formula based test estimation method based on function point analysis. Test Procedure: A document providing detailed instructions for the execution of one or more test cases. Test Run: Execution of a test on a specific version of the test object. Test Scenario: Definition of a set of test cases or test scripts and the sequence in which they are to be executed. Test Report: Test report is document which contains detailed test results. Test Scripts: A document specifying a sequence of actions for the execution of a test, also known as Test Scripts or Manual Test Scripts Commonly used to refer to a test procedure specification, especially an automated one Test Specification: A document specifying the test approach for a software feature or combination or features and the inputs, predicted results and execution conditions for the tests. Test Strategy: A high-level description of the test-levels to be performed and the testing within those levels for an organization program (one or more projects). Test Summary Report: A document summarising testing activities and results, it also contains an evaluation of the corresponding test items against exit criteria. Test Suite: A collection of tests used to validate the behavior of a product. The scope of a Test Suite varies from organization to organization. There may be several Test Suites for a particular product for example. In most cases however a Test Suite is a high level concept, grouping together hundreds or thousands of tests related by what they are intended to test. Test Target: A set of test completion criteria for the test. TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Test Type: A group of test activities aimed at testing a component or system regarding one or more interrelated quality attributes. A test type is focused on a specific test objective, i.e. reliability test, usability test, regression test etc., and may take place on one or more test levels or test phases. Tester: A person (either a professional tester or a user) who is involved in the testing of a component or system. Testability: The degree to which a system or component or requirement is stated in terms that facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met. Testing: The process of exercising software to verify that it satisfies specified requirements and to detect errors. The process of analyzing a software item to detect the differences between existing and required conditions (that is, bugs), and to evaluate the features of the software item. The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component. Test Tools: Computer programs used in the testing of a system, a component of the system, or its documentation. Thread Testing: A version of component integration testing where the progressive integration of components follows the implementation of subsets of the requirements, as opposed to the integration of components by levels of a hierarchy. Top Down Testing: An approach to integration testing where the component at the top of the component hierarchy is tested first, with lower level components being simulated by stubs. Tested components are then used to test lower level components. The process is repeated until the lowest level components have been tested. Test Driven Development: A software development technique consisting of short iterations where new test cases covering the desired improvement or new functionality are written first, then the production code necessary to pass the tests is implemented, and finally the software is re factored to accommodate the changes. The availability of tests before actual development ensures rapid feedback after any change. Practitioners emphasize that test-driven development is a method of designing software, not merely a method of testing. (i.e. avoid designing software that is difficult to test). TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Total Quality Management: A company commitment to develop a process that achieves high quality product and customer satisfaction. Traceability: The ability to identify related items in documentation and software, such as requirements with associated tests. Traceability Matrix: A document showing the relationship between Test Requirements and Test Cases. Tracked Field: A value captured in one part of an automated test process and retained for use at a later stage. Timeline: A chronological representation of components in a result set, combining different areas of test analysis, such as visual, database, messages and linked programs. See TestBench. Total Testing: A complete approach to testing from Original Software describing the desire to optimize all the steps in the testing process (e.g. from unit testing to UAT), as well as the different layers of the applications architecture. (e.g. From the user interface to the database Understandability: The capability of the software product to enable the user to understand whether the software is suitable, and how it can be used for particular tasks and conditions of use. Unit Testing (UT): The lowest level of testing involves testing at a component, module or subprogram level. Developers are involved in Unit Testing. User Acceptance Testing (UAT): Formal testing with respect tom user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the users, customers or other authorized entity to determine whether or not to accept the system. Unreachable Code: Code that cannot be reached and therefore is impossible to execute. Unstructured Decisions: This type of decision situation is complex and no standard solutions exist for resolving the situation. Some or all of the structural elements of the decision situation are undefined, ill-defined or unknown. Usability: The capability of the software to be understood learned, used and attractive to the user when used under specified conditions. TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Usability requirements: A specification of the required usability for the system/software. Usability Testing: Testing to determine whether the system/software meets the specified usability requirements. Use Case: The specification of tests that are conducted from the enduser perspective. Use cases tend to focus on operating software as an end-user would conduct their day-to-day activities. Use Case Testing: A black box test design technique in which test cases are designed to execute user scenarios. Unit Testing: Testing of individual software components V Model: Describes how inspection and testing activities can occur in parallel with other activities. Validation: Doing the right thing!! Validation is usually accomplished by verifying each stage of SDLC. Confirming by examination and thorough provision of objective evidence that the requirements for a specific intended use or application have been fulfilled Validation Testing: Determination of the correctness of the products of software development with respect to the user needs and requirements. Variable Data: A repository for multiple scenario values which can be used to drive repeatable automated processes through a number of iterations when used in conjunction with an automation solution such as Test Drive. . Version Identifier: A version number; version date, or version date and time stamp. Volume Testing: Testing which confirms that any values that may become large over time (such as accumulated counts, logs, and data files), can be accommodated by the program and will not cause the program to stop working or degrade its operation in any manner. Verification: Conforming by examination and thorough provision of objective evidence that specified requirements have been fulfilled Walkthrough: A review of requirements, designs or code characterized by the author of the material under review guiding the progression of the review. TATA Consultancy Services Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Warping: The capability to manipulate dates in data to simulate data of different ages in support of testing date driven systems. Found in TestBench Waterline: The lowest level of detail relevant to the Customer. What If Analysis: The capability of "asking" the software package what the effect will be of changing some of the input data or independent variables. White Box Testing: (a.k.a. clear box testing, glass box testing or structural testing). Uses an internal perspective of the system to design test cases based on internal structure. It requires programming skills to identify all paths through the software. Wide Band Delphi: A consensus-based estimation technique for estimating effort. It was developed in the 1940s at the RAND Corporation as a forecasting tool. It has since been adapted across many industries to estimate many kinds of tasks, ranging from statistical data collection results to sales and marketing forecasts. It has proven to be a very effective estimation tool, and it lends itself well to software projects. However, many see great problems with the technique, such as unknown manipulation of a group and silencing of minorities in order to see a preset outcome of a meeting. Workaround: Method of avoiding an incident or problem, either from a temporary fix or from a technique that means the Customer is not reliant on a particular aspect of a service that is known to have a problem. Workflow Testing: Scripted end-to-end testing which duplicates specific workflows which are expected to be utilized by the end-user. XML: Extensible Markup Language. XML is a set of rules for designing text formats that let you structure your data. XML makes it easy for a computer to generate data, read data, and ensure that the data structure is unambiguous. XML avoids common pitfalls in language design: it is extensible, platform-independent, and it supports internationalization and localization.

TATA Consultancy Services

Date: 05:12:2008

Assurance Services -Unit 01

Testing Definitions

Bibliography:
The Software Test Management Guide (www.ruleworks.com) The Software Testing Glossary (www.aptest.com) The International Software Testing Qualifications Board (www.istqb.org) The Testing Standards Working Party (www.testingstandards.co.uk) Wikipedia (wikipedia.org) IEEE (www.ieee.org)

TATA Consultancy Services

Date: 05:12:2008