QF-Test - The manual

Version 3.5.3

Gregor Schmid Quality First Software GmbH1

Copyright c 1999-2013 Quality First Software GmbH2

September 17, 2013

1 http://www.qfs.de 2 http://www.qfs.de

Preface
As the name indicates, Quality First Software GmbH is dedicated to quality assurance for the software development process. Our contribution to this area is the product QFTest the manual of which you are currently reading. QF-Test is a professional tool for automating functional tests for Java or web applications with a graphical user interface. Depending on size and structure of a company the ungrateful task of testing sometimes falls to a QA department or team, sometimes to the developers and sometimes to the unlucky customer. Users of QF-Test are therefore usually developers or testers with varying knowledge about software development and testing in general and Java or web GUIs in particular. This manual is the primary source of information for QF-Test. We have tried to explain things in a way that is understandable for all users, independent of their technical knowledge, yet provide a complete and concise reference. In case of Java specic questions, testers may fare best by contacting their developers who will surely be able to assist. Initially QF-Test did only support Java Swing GUIs. With version 2.0 support for Eclipse/SWT was added and web support with version 3.0. Parts of QF-Test and this manual owe to this history and most things are explained from the perspective of testing a Swing GUI. In most cases the concepts are universal apply similarly to all GUIs. Where things differ, specic notes explain the particularities of a web or SWT GUI.

How to use this manual

This manual is available in HTML and PDF versions. The HTML version is split across multiple les for better navigation and to avoid excessive memory consumption of the web browser. Due to extensive cross-linking, the HTML document is the preferred version for online viewing, while the PDF version is better suited for printing. Searching should also be done in the PDF version since the splitting of the HTML les prevents searches through the whole manual. The PDF version of the manual is located at

ii qftest-3.5.3/doc/manual_en.pdf, the entry page of the HTML version is at qftest-3.5.3/doc/manual/en/manual.html. A web-browser for the HTML manual can be started directly from QF-Test. The HelpManual... menu item will take you to the entry page of the manual and HelpNews... will bring up the section documenting the latest changes. Context-sensitive help is also available for all kinds of tree-nodes, attributes and congurable options by clicking with the right mouse button and selecting Whats this? from the popup menu. This may not work if the systems browser is not accessible from QF-Test. The manual consists of three parts which are kept in one document for technical reasons (it simplies cross-linking and index generation). They may be split in a later version. These parts are User manual
(2)

This part explains how to install and run QF-Test and how to work with its user interface. It shows how to create and organize tests, then continues with more advanced material. To avoid duplication of text, the user manual often refers to the reference manual for detailed explanation. We recommend that you follow these links. Reference manual
(217)

This is a complete reference that covers all congurable options, all parts of a test-suite , etc. When looking for specic information, this is the place to go. The reference manual also serves as the source for context-sensitive help. Technical reference
(535)

The part about technical details contains in-depth and background information about miscellaneous topics as well as a comprehensive API reference for the scripting interface. Beginners will rarely need to take a look at this part, but for the advanced user and the technically interested it is a valuable resource. A learning-by-doing tutorial is also available in HTML and PDF versions. The HTML version, which is directly accessible from the HelpTutorial (HTML)... menu item, is located at qftest-3.5.3/doc/tutorial/en/tutorial.html. The PDF version is to be found at qftest-3.5.3/doc/tutorial/tutorial_en.pdf and is also directly accessible through HelpTutorial (PDF)... . The following notations are used throughout the manual: MenuSubmenu represents a menu or menu item.

iii Modier-Key stands for a keystroke, where the modier is one (or a combination) of Shift , Control or Alt . Monospaced font is used for names of directories and les, user input and program output. In order to transfer at least part of the convenience of cross-linking to the paper (i) version, references in the PDF version are underlined and show the target page number in small braces.

iv

Contents
I User manual 1
2 2 3 4 5 5 6 7 7 9 9

1 Installation and startup 1.1 1.2 1.3 1.4 1.5 1.6 Unix Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Windows Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The license le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The conguration les . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Starting QF-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Firewall Security Warning . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 The user interface 2.1 2.2 The test-suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Basic editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 2.2.2 2.2.3 2.2.4 2.2.5 2.2.6 2.2.7 2.3 2.3.1 2.3.2 2.3.3 2.3.4 Navigating the tree . . . . . . . . . . . . . . . . . . . . . . . . . .

Insertion marker . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Moving nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Transforming nodes . . . . . . . . . . . . . . . . . . . . . . . . . 11 Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Packing and Unpacking . . . . . . . . . . . . . . . . . . . . . . . 13 Sorting Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Replacing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Complex searches and replace operations . . . . . . . . . . . . . 20 Multiple views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Advanced editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

CONTENTS 3 Quickstart your application 3.1 3.1.1 3.1.2 3.1.3 3.2 3.3

v 22

Quickstart Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Setup sequence creation . . . . . . . . . . . . . . . . . . . . . . 23 Executing the setup sequence . . . . . . . . . . . . . . . . . . . 24 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Program output and the Clients menu . . . . . . . . . . . . . . . . . . . . 26 An application started indirectly from an already connected SUT . . . . . 26 28

4 Capture and replay 4.1 4.2 4.3 4.4 4.5

Recording sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Running tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Recording checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Recording components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Recording of HTTP Requests (GET/POST) . . . . . . . . . . . . . . . . . 33 35

5 Components 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9

GUI component hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Components in QF-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Component Ids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Component class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Component inspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 The importance of naming components . . . . . . . . . . . . . . . . . . . 40 Considerations for setting names . . . . . . . . . . . . . . . . . . . . . . 42 Updating Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Troubleshooting component recognition problems . . . . . . . . . . . . . 45

5.10 Accessing hidden elds on a web page . . . . . . . . . . . . . . . . . . . 47 6 Sub-items of complex GUI components 6.1 6.2 6.3 6.4 48

Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 The Id of an Item . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Special Id-syntax for direct element access . . . . . . . . . . . . . . . . . 50 Activating item recording for web tables . . . . . . . . . . . . . . . . . . . 51

CONTENTS 7 Variables 7.1 7.2 7.3 7.4 7.5 7.6 7.7

vi 52

Variable lookup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Dening variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Variable example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Fetching data from the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . 56 External data and special groups . . . . . . . . . . . . . . . . . . . . . . 57 Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Immediate and lazy binding . . . . . . . . . . . . . . . . . . . . . . . . . 60 63

8 Problem analysis and debugging 8.1 8.1.1 8.1.2 8.1.3 8.1.4 8.1.5 8.1.6 8.2 8.2.1 8.2.2 8.2.3 8.2.4

The run-log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Error states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Navigating the run-log tree . . . . . . . . . . . . . . . . . . . . . 66 Accepting values of failed checks as good . . . . . . . . . . . . . 66 Split run-logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Run-log options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Creating a test-suite from the run-log . . . . . . . . . . . . . . . . 68 Entering the debugger . . . . . . . . . . . . . . . . . . . . . . . . 69 The debugger window . . . . . . . . . . . . . . . . . . . . . . . . 70 Debugger commands . . . . . . . . . . . . . . . . . . . . . . . . 71 Manipulating breakpoints . . . . . . . . . . . . . . . . . . . . . . 72 73

The debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

9 Organizing the test-suite 9.1 9.2

Sequence and Test nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Test management with Test-set and Test-case nodes . . . . . . . . . . . . 75 9.2.1 9.2.2 9.2.3 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Variables and special attributes . . . . . . . . . . . . . . . . . . . 76 Migration and backwards compatibility . . . . . . . . . . . . . . . 76 Local Procedures and Packages . . . . . . . . . . . . . . . . . . . . 78 Relative Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 78

9.3

Procedures and Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 9.3.1 9.3.2

CONTENTS 9.3.3 9.3.4 9.3.5 9.4

vii Inserting Procedure call nodes . . . . . . . . . . . . . . . . . . . . 79 Parameterizing nodes . . . . . . . . . . . . . . . . . . . . . . . . 79 Transforming a Sequence into a Procedure . . . . . . . . . . . . . . 79

Documenting test-suites . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 81 83 85

10 Projects 11 The standard library 12 Dependencies

12.1 Dependency basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 12.2 The dependency stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 12.3 Error escalation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 12.4 Special variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 12.5 Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 13 Scripting 94

13.1 Fundamentals of the Jython integration . . . . . . . . . . . . . . . . . . . 94 13.2 The run-context rc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 13.3 Scripting examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 13.3.1 Logging messages . . . . . . . . . . . . . . . . . . . . . . . . . . 97 13.3.2 Performing checks . . . . . . . . . . . . . . . . . . . . . . . . . . 98 13.3.3 Accessing variables . . . . . . . . . . . . . . . . . . . . . . . . . 98 13.3.4 Setting variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 13.3.5 Global Jython variables . . . . . . . . . . . . . . . . . . . . . . . 100 13.3.6 Exchanging Jython variables between interpreters . . . . . . . . 101 13.3.7 Accessing the SUTs GUI components . . . . . . . . . . . . . . . 101 13.3.8 Calling Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 13.3.9 Setting options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 13.3.10 A complex example . . . . . . . . . . . . . . . . . . . . . . . . . 104 13.3.11 Override components . . . . . . . . . . . . . . . . . . . . . . . . 105 13.4 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 13.5 Post-mortem debugging of Jython scripts . . . . . . . . . . . . . . . . . . 108

CONTENTS

viii

13.6 Differences between Jython version 2.2.1 and 2.5 . . . . . . . . . . . . . 108 13.6.1 Boolean type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 13.6.2 Character encodings . . . . . . . . . . . . . . . . . . . . . . . . . 109 13.6.3 Getting the name of a Java class . . . . . . . . . . . . . . . . . . 110 13.7 Scripting with Groovy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 13.7.1 Accessing variables in Groovy . . . . . . . . . . . . . . . . . . . 113 13.7.2 Global Groovy variables . . . . . . . . . . . . . . . . . . . . . . . 114 13.7.3 Groovy packages . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 14 Data-driven testing 116

14.1 Data-driver examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 14.2 General use of Data drivers . . . . . . . . . . . . . . . . . . . . . . . . . . 120 14.3 Examples for Data drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 14.4 Advanced use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 15 Reports and test documentation 124

15.1 Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 15.1.1 Report concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 15.1.2 Report contents . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 15.1.3 Creating reports . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 15.2 Testdoc documentation for Test-sets and Test-cases . . . . . . . . . . . . . 129 15.3 Pkgdoc documentation for Packages, Procedures and Dependencies . . . . . 130 16 Test execution 133

16.1 Test execution in batch mode . . . . . . . . . . . . . . . . . . . . . . . . . 133 16.1.1 Command line usage . . . . . . . . . . . . . . . . . . . . . . . . 134 16.1.2 Windows batch script . . . . . . . . . . . . . . . . . . . . . . . . 136 16.1.3 Groovy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 . . . . . . . . . . . . . . . . . . . . . . 139 16.2 Executing tests in daemon mode

16.2.1 Launching the daemon . . . . . . . . . . . . . . . . . . . . . . . . 139 16.2.2 Controlling a daemon from QF-Tests command line . . . . . . . 140 16.2.3 Controlling a daemon with the daemon API . . . . . . . . . . . . 141

CONTENTS 17 Distributed test development

ix 145

17.1 Referencing nodes in another test-suite . . . . . . . . . . . . . . . . . . . 145 17.2 Managing Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 17.3 Merging test-suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 17.3.1 Importing Components . . . . . . . . . . . . . . . . . . . . . . . . 148 17.3.2 Importing Procedures and Testcases . . . . . . . . . . . . . . . 148 17.4 Strategies for distributed development . . . . . . . . . . . . . . . . . . . . 148 17.5 Static validation of test-suites . . . . . . . . . . . . . . . . . . . . . . . . 150 153

18 Automated Creation of Basic Procedures

18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 18.2 How to use the Procedure Builder . . . . . . . . . . . . . . . . . . . . . . 153 18.3 Conguration of the Procedure Builder . . . . . . . . . . . . . . . . . . . 154 18.3.1 The Procedure Builder denition le . . . . . . . . . . . . . . . . 154 19 Interaction with Test Management Tools 156

19.1 HP TestDirector for Quality Center . . . . . . . . . . . . . . . . . . . . . . 156 19.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 19.1.2 Step-by-step integration guide . . . . . . . . . . . . . . . . . . . . 158 19.2 Imbus TestBench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 19.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 19.2.2 Creating QF-Test template from interactions . . . . . . . . . . . . 159 19.2.3 Importing test-execution results . . . . . . . . . . . . . . . . . . . 160 19.3 QMetry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 19.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 19.3.2 Sample Conguration . . . . . . . . . . . . . . . . . . . . . . . . 162 19.4 Klaros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 19.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 19.4.2 Importing QF-Test results into Klaros . . . . . . . . . . . . . . . . 163 19.5 TestLink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 19.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 19.5.2 Generating template test-suites for QF-Test from test-cases . . . 164

CONTENTS

19.5.3 Execution of test-cases . . . . . . . . . . . . . . . . . . . . . . . 166 19.5.4 Importing QF-Test results into TestLink . . . . . . . . . . . . . . . 166 20 Integration with Build Tools 168

20.1 Ant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 20.2 Maven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 20.3 Jenkins/Hudson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 20.3.1 Install and start Hudson . . . . . . . . . . . . . . . . . . . . . . . 171 20.3.2 Create a new job for QF-Test . . . . . . . . . . . . . . . . . . . . 172 20.3.3 Install QF-Test Demo Project . . . . . . . . . . . . . . . . . . . . 173 20.3.4 Requirements for GUI tests . . . . . . . . . . . . . . . . . . . . . 175 21 Performing GUI-based load tests 176

21.1 Background and comparison with other techniques . . . . . . . . . . . . 176 21.2 Load tests with QF-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 21.2.1 Running tests with parallel threads . . . . . . . . . . . . . . . . . 177 21.2.2 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 21.2.3 Coordinating tests on multiple machines . . . . . . . . . . . . . . 178 22 Executing Manual Tests in QF-Test 180

22.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 22.2 Step-by-step Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 22.3 Structure of the Excel le . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 22.4 The ManualTestRunner test-suite . . . . . . . . . . . . . . . . . . . . . . 183 22.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

II

Best Practices

185
186 187 . . . . . . . . . . . . . . . . . . . 187

23 Introduction 24 How to start a testing project 24.1 Infrastructure and testing environment

24.2 Location of les . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

CONTENTS

xi

24.2.1 Network installation . . . . . . . . . . . . . . . . . . . . . . . . . 190 24.3 Component Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 25 Organizing test-suites 193

25.1 Organizing tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 25.2 Modularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 25.3 Parameterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 25.4 Working in multiple test-suites . . . . . . . . . . . . . . . . . . . . . . . . 195 25.5 Roles and responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 25.6 Managing components at different levels . . . . . . . . . . . . . . . . . . 199 25.7 Reverse includes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 26 Efcient working techniques 200

26.1 Using QF-Test projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 26.2 Creating test-suites from scratch . . . . . . . . . . . . . . . . . . . . . . . 200 26.3 The standard library qfs.qft . . . . . . . . . . . . . . . . . . . . . . . . . . 201 26.4 Component storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 26.5 Extending test-suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 26.6 Working in the script editor . . . . . . . . . . . . . . . . . . . . . . . . . . 203 27 How to achieve robust component recognition 204

27.1 Using the default conguration . . . . . . . . . . . . . . . . . . . . . . . . 204 27.2 Using the Name overrides everything mode . . . . . . . . . . . . . . . . 206 27.3 Using regular expressions for working with dynamic window titles . . . . 208 27.4 Inuencing the names by implementing a NameResolver . . . . . . . . . 209 27.5 Handling class changes due to re-factoring by using Record systemclasses only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 27.6 Avoiding recording every component or using generic components 27.7 . . . 212 Switching component recognition settings dynamically . . . . . . . . . . 213 214

28 Test execution

28.1 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 28.2 Timeout vs. delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

CONTENTS

xii

28.3 What to do if the run-log contains an error . . . . . . . . . . . . . . . . . 215

III

Reference manual

216
217

29 Options

29.1 General options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 29.1.1 Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 29.1.2 Editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 29.1.3 Bookmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 29.1.4 External tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 29.1.5 Backup les . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 29.1.6 Library path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 29.1.7 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 29.2 Recording options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 29.2.1 Events to record . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 29.2.2 Events to pack . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 29.2.3 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 29.2.4 Recording sub-items . . . . . . . . . . . . . . . . . . . . . . . . . 246 29.2.5 Recording procedures . . . . . . . . . . . . . . . . . . . . . . . . 247 29.3 Replay options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 29.3.1 Client options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 29.3.2 Event handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 29.3.3 Component recognition . . . . . . . . . . . . . . . . . . . . . . . 259 29.3.4 Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 29.3.5 Timeouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 29.4 Web options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 29.5 Debugger options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 29.6 Run-log options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 29.6.1 General run-log options . . . . . . . . . . . . . . . . . . . . . . . 275 29.6.2 Options determining run-log content . . . . . . . . . . . . . . . . 280 29.7 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

CONTENTS 30 Elements of a test-suite

xiii 285

30.1 The test-suite and its structure . . . . . . . . . . . . . . . . . . . . . . . . 285 30.1.1 Test-suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 30.2 Test and Sequence nodes . . . . . . . . . . . . . . . . . . . . . . . . . . 288 30.2.1 Test-case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 30.2.2 Test-set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 30.2.3 Test call . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 30.2.4 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 30.2.5 Sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 30.2.6 Test-step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 30.2.7 Sequence with time limit . . . . . . . . . . . . . . . . . . . . . . . . 312 30.2.8 Extras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 30.3 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 30.3.1 Dependency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 30.3.2 Dependency reference . . . . . . . . . . . . . . . . . . . . . . . . . 320 30.3.3 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 30.3.4 Cleanup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 30.3.5 Error handler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 30.4 Data driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 30.4.1 Data driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 30.4.2 Data table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 30.4.3 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 30.4.4 Excel data le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 30.4.5 CSV data le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 30.4.6 Data loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 30.5 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 30.5.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 30.5.2 Procedure call . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 30.5.3 Return . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 30.5.4 Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 30.5.5 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361

CONTENTS

xiv

30.6 Control structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 30.6.1 Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 30.6.2 While . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 30.6.3 Break . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 30.6.4 If . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 30.6.5 Elseif . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 30.6.6 Else . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 30.6.7 Try . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 30.6.8 Catch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 30.6.9 Finally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 30.6.10 Throw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 30.6.11 Rethrow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 30.6.12 Server script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 30.6.13 SUT script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 30.7 Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 30.7.1 Start Java SUT client . . . . . . . . . . . . . . . . . . . . . . . . . . 395 30.7.2 Start SUT client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 30.7.3 Start browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 30.7.4 Start process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406 30.7.5 Execute shell command . . . . . . . . . . . . . . . . . . . . . . . . 409 30.7.6 Wait for client to connect . . . . . . . . . . . . . . . . . . . . . . . . 411 30.7.7 Stop client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 30.7.8 Wait for process to terminate . . . . . . . . . . . . . . . . . . . . . . 416 30.8 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 30.8.1 Mouse event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 30.8.2 Key event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 30.8.3 Text input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 30.8.4 Window event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430 30.8.5 Component event . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 30.8.6 Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 30.8.7 File selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

CONTENTS

xv

30.9 Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 30.9.1 Check text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 30.9.2 Boolean check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 30.9.3 Check items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 30.9.4 Check selectable items . . . . . . . . . . . . . . . . . . . . . . . . . 456 30.9.5 Check image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460 30.9.6 Check geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466 30.10 Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470 30.10.1 Fetch text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 30.10.2 Fetch index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474 30.10.3 Fetch geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477 30.11 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 30.11.1 Set variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 30.11.2 Wait for component to appear . . . . . . . . . . . . . . . . . . . . . 484 30.11.3 Wait for document to load . . . . . . . . . . . . . . . . . . . . . . . 488 30.11.4 Wait for download to nish . . . . . . . . . . . . . . . . . . . . . . . 492 30.11.5 Load resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495 30.11.6 Load properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 30.12 HTTP Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500 30.12.1 Server HTTP request . . . . . . . . . . . . . . . . . . . . . . . . . . 500 30.12.2 Browser HTTP request . . . . . . . . . . . . . . . . . . . . . . . . . 504 30.13 Windows, Components and Items . . . . . . . . . . . . . . . . . . . . . . 507 30.13.1 Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 30.13.2 Web page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510 30.13.3 Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 30.13.4 Item . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519 30.13.5 Window group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522 30.13.6 Component group . . . . . . . . . . . . . . . . . . . . . . . . . . . 523 30.13.7 Windows and components . . . . . . . . . . . . . . . . . . . . . . . 524 31 Exceptions 526

CONTENTS

xvi

IV

Technical reference

534
535

32 Command line arguments and exit codes

32.1 Call syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 32.2 Command line arguments . . . . . . . . . . . . . . . . . . . . . . . . . . 539 32.2.1 Arguments for the starter script . . . . . . . . . . . . . . . . . . . 539 32.2.2 Arguments for the Java VM . . . . . . . . . . . . . . . . . . . . . 540 32.2.3 Arguments for QF-Test . . . . . . . . . . . . . . . . . . . . . . . . 540 32.2.4 Placeholders in the lename parameter for run-log and report . . 554 32.3 Exit codes for QF-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 33 GUI engines 34 Running an application from QF-Test 557 559

34.1 Various methods to start the SUT . . . . . . . . . . . . . . . . . . . . . . 559 34.1.1 A standalone script or executable le . . . . . . . . . . . . . . . . 560 34.1.2 An application launched through Java WebStart . . . . . . . . . . 561 34.1.3 An applet in a web browser . . . . . . . . . . . . . . . . . . . . . 562 34.1.4 An application started with java -jar <archive> . . . . . . . 565 34.1.5 An application started with java -classpath <classpath> <class> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 34.1.6 A web application in a browser . . . . . . . . . . . . . . . . . . . 568 35 JDK and SWT instrumentation 35.2 JDK instrumentation background 571 . . . . . . . . . . . . . . . . . . . . . . 574

35.1 JDK instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571 35.2.1 Modications to the JDK . . . . . . . . . . . . . . . . . . . . . . . 574 35.2.2 Effects of the instrumentation . . . . . . . . . . . . . . . . . . . . 575 35.3 SWT instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 35.3.1 Preparation for manual SWT instrumentation . . . . . . . . . . . 576 35.3.2 Manual SWT instrumentation for eclipse based applications . . . 576 35.3.3 Manual instrumentation for standalone SWT applications . . . . 577 36 Technical details about miscellaneous issues 578

CONTENTS

xvii

36.1 Drag&Drop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578 36.2 Component recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579 36.3 Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581 36.4 Regular expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581 36.5 Line breaks under Unix and Windows . . . . . . . . . . . . . . . . . . . . 583 36.6 Quoting and escaping special characters . . . . . . . . . . . . . . . . . . 583 36.7 Include le resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584 37 Jython and Groovy scripting 586

37.1 Module load-path (Jython) . . . . . . . . . . . . . . . . . . . . . . . . . . 586 37.2 The plugin directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587 37.3 The package cache (Jython) . . . . . . . . . . . . . . . . . . . . . . . . . 587 37.4 Initialization (Jython) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588 37.5 Namespace environment for script execution (Jython) . . . . . . . . . . . 588 37.6 Run-context API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589 37.7 The qf module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607 37.8 Image API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610 37.8.1 The ImageWrapper class . . . . . . . . . . . . . . . . . . . . . 610 37.9 Exception handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613 37.10 Debugging scripts (Jython) . . . . . . . . . . . . . . . . . . . . . . . . . . 613 38 Special features for web applications 615

38.1 Pseudo DOM API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615 38.1.1 The abstract Node class . . . . . . . . . . . . . . . . . . . . . . . 615 38.1.2 The DocumentNode class . . . . . . . . . . . . . . . . . . . . . 620 38.1.3 The FrameNode class . . . . . . . . . . . . . . . . . . . . . . . . 622 38.1.4 The DomNode class . . . . . . . . . . . . . . . . . . . . . . . . . 623 38.1.5 The DialogNode class . . . . . . . . . . . . . . . . . . . . . . . 624 38.2 Special support for various AJAX toolkits . . . . . . . . . . . . . . . . . . 625 38.2.1 AJAX resolver concepts . . . . . . . . . . . . . . . . . . . . . . . 626 38.2.2 Supported AJAX toolkits . . . . . . . . . . . . . . . . . . . . . . . 627 38.2.3 Setting unique Ids . . . . . . . . . . . . . . . . . . . . . . . . . . 631

CONTENTS 39 Extension APIs

xviii 634

39.1 Resolver hooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634 39.1.1 Overview over the supported kinds of resolvers . . . . . . . . . . 635 39.1.2 The ClassNameResolver interface . . . . . . . . . . . . . . . 636 39.1.3 The NameResolver2 interface . . . . . . . . . . . . . . . . . . 637 39.1.4 The IdResolver interface . . . . . . . . . . . . . . . . . . . . . 637 39.1.5 The FeatureResolver2 interface . . . . . . . . . . . . . . . . 638 39.1.6 The ExtraFeatureResolver interface . . . . . . . . . . . . . 639 39.1.7 The ItemNameResolver2 interface . . . . . . . . . . . . . . . 642 39.1.8 The ItemValueResolver2 interface . . . . . . . . . . . . . . . 643 39.1.9 The TreeTableResolver interface . . . . . . . . . . . . . . . . 643 39.1.10 Simplied resolver creation with the resolvers module . . . . . 644 39.1.11 The ResolverRegistry . . . . . . . . . . . . . . . . . . . . . . 652 39.1.12 Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658 39.2 Implementing custom item types with the ItemResolver interface . . 658 39.2.1 ItemResolver concepts . . . . . . . . . . . . . . . . . . . . . . 659 39.2.2 The ItemResolver interface . . . . . . . . . . . . . . . . . . . . 659 39.2.3 The class SubItemIndex . . . . . . . . . . . . . . . . . . . . . 664 39.2.4 The ItemRegistry . . . . . . . . . . . . . . . . . . . . . . . . . 665 39.2.5 Default item representations . . . . . . . . . . . . . . . . . . . . . 668 39.3 Implementing custom checks with the Checker interface . . . . . . . . 669 39.3.1 The Checker interface . . . . . . . . . . . . . . . . . . . . . . . 670 39.3.2 The class Pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671 39.3.3 The CheckType interface and its implementation DefaultCheckType . . . . . . . . . . . . . . . . . . . . . . . . 672

39.3.4 The class CheckDataType . . . . . . . . . . . . . . . . . . . . . 673 39.3.5 The class CheckData and its subclasses . . . . . . . . . . . . . 674 39.3.6 The CheckerRegistry . . . . . . . . . . . . . . . . . . . . . . 676 39.3.7 Custom checker example . . . . . . . . . . . . . . . . . . . . . . 677 39.4 Working with the Eclipse Graphical Editing Framework (GEF) . . . . . . 679 39.4.1 Recording GEF items . . . . . . . . . . . . . . . . . . . . . . . . 679 39.4.2 Implementing a GEF ItemNameResolver2 . . . . . . . . . . . . . 681

CONTENTS

xix

39.4.3 Implementing a GEF ItemValueResolver2 . . . . . . . . . . . . . 683 39.5 Test-run listeners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683 39.5.1 The TestRunListener interface . . . . . . . . . . . . . . . . . 684 39.5.2 The class TestRunEvent . . . . . . . . . . . . . . . . . . . . . 685 39.5.3 The class TestSuiteNode . . . . . . . . . . . . . . . . . . . . . 686 39.6 DOM processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687 39.6.1 The DOMProcessor interface . . . . . . . . . . . . . . . . . . . . 688 39.6.2 The DOMProcessorRegistry . . . . . . . . . . . . . . . . . . . 689 39.6.3 Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 690 39.7 Image API extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 690 39.7.1 The ImageRep class . . . . . . . . . . . . . . . . . . . . . . . . . 690 39.7.2 The ImageComparator interface . . . . . . . . . . . . . . . . . 693 40 Daemon mode 695

40.1 Daemon concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695 40.2 Daemon API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696 40.2.1 The DaemonLocator . . . . . . . . . . . . . . . . . . . . . . . . 697 40.2.2 The Daemon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697 40.2.3 The TestRunDaemon . . . . . . . . . . . . . . . . . . . . . . . . 699 40.2.4 The DaemonRunContext . . . . . . . . . . . . . . . . . . . . . . 702 40.2.5 The DaemonTestRunListener . . . . . . . . . . . . . . . . . . 709 40.3 Daemon security considerations . . . . . . . . . . . . . . . . . . . . . . . 709 40.3.1 Creating your own keystore . . . . . . . . . . . . . . . . . . . . . 710 40.3.2 Specifying the keystore . . . . . . . . . . . . . . . . . . . . . . . 711 40.3.3 Specifying the keystore on the client side . . . . . . . . . . . . . 711 41 The Procedure Builder denition le 712

41.1 Placeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712 41.2 Conditions for Package and Procedure Denition . . . . . . . . . . . . . 714 41.3 Interpretation of the Component Hierarchy . . . . . . . . . . . . . . . . . 714 41.4 Details about the @CONDITION tag . . . . . . . . . . . . . . . . . . . . 715 42 The ManualStepDialog 717

CONTENTS

xx

42.1 The ManualStepDialog API . . . . . . . . . . . . . . . . . . . . . . . . . . 717 43 Details about transforming nodes 719

43.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719 43.2 Transformation with type changes . . . . . . . . . . . . . . . . . . . . . . 719 43.3 Additional transformations below the Extras node . . . . . . . . . . . . . . 720 43.3.1 Transformations without side-effects . . . . . . . . . . . . . . . . 720 43.3.2 Transformations with side-effects . . . . . . . . . . . . . . . . . . 721 44 Details about the algorithm for image comparison 722

44.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722 44.2 Description of algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 723 44.2.1 Classic image check . . . . . . . . . . . . . . . . . . . . . . . . . 723 44.2.2 Pixel-based identity check . . . . . . . . . . . . . . . . . . . . . . 724 44.2.3 Pixel-based similarity check . . . . . . . . . . . . . . . . . . . . . 725 44.2.4 Block-based identity check . . . . . . . . . . . . . . . . . . . . . 727 44.2.5 Block-based similarity check . . . . . . . . . . . . . . . . . . . . 728 44.2.6 Histogram check . . . . . . . . . . . . . . . . . . . . . . . . . . . 730 44.2.7 Analysis with Discrete Cosine Transformation . . . . . . . . . . . 731 44.2.8 Block-based analysis with Discrete Cosine Transformation . . . . 733 44.2.9 Bilinear Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735 44.3 Description of special functions . . . . . . . . . . . . . . . . . . . . . . . 736 44.3.1 Image-in-image search . . . . . . . . . . . . . . . . . . . . . . . 736 45 Result lists 739

45.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739 45.2 Specic list actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741 45.2.1 All types of lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741 45.2.2 Replacing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741 45.2.3 Error list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742 45.3 Exporting and loading results A FAQ - Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . . 742 743

CONTENTS B Release notes B.1 B.2 B.3 B.4 B.5 B.6 B.7 B.8 B.9

xxi 751

Version 3.5.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751 Version 3.5.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752 Version 3.5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753 Main new features in version 3.5 . . . . . . . . . . . . . . . . . . . . . . . 754 Version 3.5.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755 Version 3.5M3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 758 Version 3.5M2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 759 Version 3.5M1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 760 Version 3.4.11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764

B.10 Version 3.4.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765 B.11 Version 3.4.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765 B.12 Version 3.4.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766 B.13 Version 3.4.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766 B.14 Version 3.4.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767 B.15 Version 3.4.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 768 B.16 Version 3.4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769 B.17 Version 3.4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 770 B.18 Version 3.4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 771 B.19 Main new features in version 3.4 . . . . . . . . . . . . . . . . . . . . . . . 773 B.20 Version 3.4.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773 B.21 Version 3.4M2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775 B.22 Version 3.4M1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 777 B.23 Version 3.3.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 779 B.24 Version 3.2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 782 B.25 Version 3.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 782 B.26 New features in QF-Test version 3.2 . . . . . . . . . . . . . . . . . . . . . 783 B.27 Version 3.2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784 B.28 Version 3.1.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785 B.29 Version 3.1.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787 B.30 Version 3.1.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 788

CONTENTS

xxii

B.31 Version 3.1.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 790 B.32 Version 3.1.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 792 B.33 Version 3.0.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794 B.34 Version 3.0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795 B.35 New features in QF-Test version 3 . . . . . . . . . . . . . . . . . . . . . . 796 C Keyboard shortcuts C.1 C.2 C.3 798

Navigation and editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 798 Record and replay functions . . . . . . . . . . . . . . . . . . . . . . . . . 800 Keyboard helper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 801 803 804

D Glossary E Third party software

xxiii

List of Figures
2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.1 3.2 4.1 5.1 5.2 5.3 5.4 6.1 7.1 7.2 7.3 7.4 Structure of a test-suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Insertion marker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Example table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 The simple search dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 The advanced search dialog . . . . . . . . . . . . . . . . . . . . . . . . . 15 Result list for Locate references . . . . . . . . . . . . . . . . . . . . . . . 16 Incremental search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 The replace dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 The replace query dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Quickstart Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Startup sequence created by the Quickstart Wizard . . . . . . . . . . . . 24 Disabled and enabled Record button . . . . . . . . . . . . . . . . . . . . 28 Components of a GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Component hierarchy of a Swing SUT . . . . . . . . . . . . . . . . . . . . 37 Pseudo class hierarchy for web elements . . . . . . . . . . . . . . . . . . 39 Update components dialog . . . . . . . . . . . . . . . . . . . . . . . . . . 44 An Item for a table cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Direct and fallback bindings . . . . . . . . . . . . . . . . . . . . . . . . . 53 System variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Variable example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Variable bindings example . . . . . . . . . . . . . . . . . . . . . . . . . . 56

LIST OF FIGURES 8.1 8.2 9.1 9.2

xxiv

A simple test and its run-log . . . . . . . . . . . . . . . . . . . . . . . . . 64 Error states in a run-log . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Sequences and Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Packages and Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

10.1 The project view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 11.1 Standard library qfs.qft . . . . . . . . . . . . . . . . . . . . . . . . . . 83 12.1 Dependency stack A-B-C-D . . . . . . . . . . . . . . . . . . . . . . . . . 87 12.2 Dependency C has forced cleanup . . . . . . . . . . . . . . . . . . . . . 88 12.3 Rollback and build up stack to A-E . . . . . . . . . . . . . . . . . . . . . . 89 12.4 Exception in forced cleanup sequence of C causes B to clean up . . . . 90 12.5 Change in characteristic variable causes cleanup of A . . . . . . . . . . . 91 12.6 Execution of Catch and Error handler nodes . . . . . . . . . . . . . . . . . 93 13.1 Detail view of a Server script with help window for rc methods . . . . . . 95

14.1 A simple data-driven test . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 14.2 Data table example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 14.3 Run-log of a data-driven test . . . . . . . . . . . . . . . . . . . . . . . . . 118 14.4 Data-driven test with nested loops . . . . . . . . . . . . . . . . . . . . . . 118 14.5 Second data table example . . . . . . . . . . . . . . . . . . . . . . . . . . 119 14.6 Run-log of a data-driven test with nested loops . . . . . . . . . . . . . . . 120 15.1 Example report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 17.1 Result of analyzing references . . . . . . . . . . . . . . . . . . . . . . . . 152 18.1 The Procedure Builder denition le . . . . . . . . . . . . . . . . . . . . . 155 19.1 QF-Test VAPI-XP test-case in HP QualityCenter . . . . . . . . . . . . . . 158 19.2 QF-Test run-log in QMetry . . . . . . . . . . . . . . . . . . . . . . . . . . 162 20.1 Hudson after start-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 20.2 QF-Test Demo Job in Hudson . . . . . . . . . . . . . . . . . . . . . . . . 174

LIST OF FIGURES

xxv

22.1 Example for a ManualStepDialog . . . . . . . . . . . . . . . . . . . . . . 181 25.1 Structure of multiple test-suites . . . . . . . . . . . . . . . . . . . . . . . 196 25.2 Including test-suites of level 1 . . . . . . . . . . . . . . . . . . . . . . . . 197 25.3 Structure of different test-suites with roles . . . . . . . . . . . . . . . . . 198

27.1 Default conguration for component recording . . . . . . . . . . . . . . . 205 27.2 Default conguration for component recognition . . . . . . . . . . . . . . 206 27.3 Name overrides everything conguration for component recording . . . 207 27.4 Name overrides everything conguration for component recognition . . 208 27.5 Using a regular expression in the Feature attribute . . . . . . . . . . . . . 209 27.6 Option to congure recording of system classes only . . . . . . . . . . . 211 29.1 Options tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 29.2 General options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 29.3 Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 29.4 Editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 29.5 Bookmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 29.6 External tools options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 29.7 Backup le options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 29.8 Library path option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 29.9 License options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 29.10 Recording options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 29.11 Options for events to record . . . . . . . . . . . . . . . . . . . . . . . . . 235 29.12 Options for events to pack . . . . . . . . . . . . . . . . . . . . . . . . . . 236 29.13 Dragging to a sub-menu . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 29.14 Options for recording components . . . . . . . . . . . . . . . . . . . . . . 240 29.15 Popup menu for recording components . . . . . . . . . . . . . . . . . . . 241 29.16 Options for recording sub-items . . . . . . . . . . . . . . . . . . . . . . . 246 29.17 Procedure Builder options . . . . . . . . . . . . . . . . . . . . . . . . . . 247 29.18 Replay options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 29.19 Client options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 29.20 Event handling options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

LIST OF FIGURES

xxvi

29.21 Component recognition options . . . . . . . . . . . . . . . . . . . . . . . 260 29.22 Delay options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 29.23 Timeout options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 29.24 Web options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 29.25 Debugger options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 29.26 General run-log options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 29.27 Options determining run-log content . . . . . . . . . . . . . . . . . . . . . 280 29.28 Variable options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 30.1 Test-suite attributes 30.3 Test-set attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

30.2 Test-case attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 30.4 Test call Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 30.5 Test attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304 30.6 Sequence attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 30.7 Test-step attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310 30.8 Sequence with time limit attributes . . . . . . . . . . . . . . . . . . . . . . . 313 30.9 Extras attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 30.10 Dependency attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 30.11 Dependency reference attributes . . . . . . . . . . . . . . . . . . . . . . . . 321 30.12 Setup attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 30.13 Cleanup attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 30.14 Error handler attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 30.15 Data driver attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 30.16 Data table attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 30.17 Database attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 30.18 Excel data le attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 30.19 CSV data le attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 30.20 Data loop attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 30.21 Procedure Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 30.22 Procedure call Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 30.23 Return Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358

LIST OF FIGURES

xxvii

30.24 Package Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 30.25 Procedures Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 30.26 Loop attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 30.27 While attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 30.28 Break attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 30.29 If attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 30.30 Elseif attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 30.31 Else attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 30.32 Try attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 30.33 Catch attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 30.34 Finally attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 30.35 Throw attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 30.36 Rethrow attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 30.37 Server script attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 30.38 SUT script attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 30.39 Start Java SUT client attributes . . . . . . . . . . . . . . . . . . . . . . . . . 396 30.40 Start SUT client attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 30.41 Start browser attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 30.42 Start process attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 30.43 Execute shell command attributes . . . . . . . . . . . . . . . . . . . . . . . 410 30.44 Wait for client to connect attributes . . . . . . . . . . . . . . . . . . . . . . 412 30.45 Stop client attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 30.46 Wait for process to terminate attributes . . . . . . . . . . . . . . . . . . . . . 417 30.47 Mouse event attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 30.48 Key event attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424 30.49 Text input attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 30.50 Window event attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431 30.51 Component event attributes . . . . . . . . . . . . . . . . . . . . . . . . . . 433 30.52 Selection attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438 30.53 File selection attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 30.54 Check text attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

LIST OF FIGURES

xxviii

30.55 Boolean check attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449 30.56 Check items attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 30.57 Check selectable items attributes . . . . . . . . . . . . . . . . . . . . . . . . 457 30.58 Check image attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 30.59 Check geometry attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 30.60 Fetch text attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472 30.61 Fetch index attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 30.62 Fetch geometry attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . 478 30.63 Set variable attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482 30.64 Wait for component to appear attributes . . . . . . . . . . . . . . . . . . . . 485 30.65 Wait for document to load attributes . . . . . . . . . . . . . . . . . . . . . . 489 30.66 Wait for download to nish attributes . . . . . . . . . . . . . . . . . . . . . 493 30.67 Load resources attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 30.68 Load properties attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . 499 30.69 Server HTTP request Attribute . . . . . . . . . . . . . . . . . . . . . . . . . 502 30.70 Browser HTTP request Attribute . . . . . . . . . . . . . . . . . . . . . . . . 505 30.71 Window attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508 30.72 Web page attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 30.73 Component attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 30.74 Item attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520 30.75 Window group attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522 30.76 Component group attributes . . . . . . . . . . . . . . . . . . . . . . . . . . 523 30.77 Windows and components attributes . . . . . . . . . . . . . . . . . . . . . . 524 34.1 Starting the SUT from a script or executable . . . . . . . . . . . . . . . . 560 34.2 Starting the SUT through Java WebStart . . . . . . . . . . . . . . . . . . 561 34.3 Starting the SUT as an applet in a web browser . . . . . . . . . . . . . . 563 34.4 Starting the SUT from a jar archive . . . . . . . . . . . . . . . . . . . . . 566 34.5 Starting the SUT via the main class . . . . . . . . . . . . . . . . . . . . . 567 34.6 Starting a web-based SUT in a browser . . . . . . . . . . . . . . . . . . . 569 35.1 Dialog for JDK instrumentation . . . . . . . . . . . . . . . . . . . . . . . . 572

LIST OF FIGURES

xxix

44.1 Original image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723 44.2 Classic image check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724 44.3 Pixel-based identity check . . . . . . . . . . . . . . . . . . . . . . . . . . 725 44.4 Pixel-based similarity check . . . . . . . . . . . . . . . . . . . . . . . . . 726 44.5 Block-based identity check . . . . . . . . . . . . . . . . . . . . . . . . . . 727 44.6 Block-based similarity check . . . . . . . . . . . . . . . . . . . . . . . . . 729 44.7 Histogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 730 44.8 Analysis with Discrete Cosine Transformation . . . . . . . . . . . . . . . 732 44.9 Block-based analysis with Discrete Cosine Transformation . . . . . . . . 734 44.10 Bilinear Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735 44.11 Image-in-image search: Expected image . . . . . . . . . . . . . . . . . . 737 44.12 Image-in-image search: Got image . . . . . . . . . . . . . . . . . . . . . 737 45.1 Sample result list for Locate references . . . . . . . . . . . . . . . . . . 740 A.1 C.1 Set browser maximum memory . . . . . . . . . . . . . . . . . . . . . . . 749 Keyboard helper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 802

xxx

List of Tables
4.1 6.1 7.1 9.1 Test result counter in the status line . . . . . . . . . . . . . . . . . . . . . 31 Separators and index formats for element access . . . . . . . . . . . . . 50 Denitions in the special group qftest . . . . . . . . . . . . . . . . . . 60

Relative procedure calls . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

22.1 Description of the Excel le for the denition of manual tests . . . . . . . 182 22.2 Description of the Excel le with the results of manual tests . . . . . . . . 183 22.3 Description of the global variables in the ManualTestRunner test-suite . . 183 22.4 States of manual test-execution . . . . . . . . . . . . . . . . . . . . . . . 184 26.1 List of variables with auto-completion. . . . . . . . . . . . . . . . . . . . . 203 30.1 Placeholders for the Name for separate run-log attribute . . . . . . . . . . . 291 30.2 Placeholders for the Name for separate run-log attribute . . . . . . . . . . . 297 30.3 Placeholders for the Name for separate run-log attribute . . . . . . . . . . . 302 30.4 Placeholders for the Name for separate run-log attribute . . . . . . . . . . . 305 30.5 Placeholders for the Name for separate run-log attribute . . . . . . . . . . . 333 30.6 Iteration range examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 30.7 Iteration range examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 30.8 Database drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 30.9 Database connection strings . . . . . . . . . . . . . . . . . . . . . . . . . 341 30.10 Iteration range examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 30.11 Iteration range examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 30.12 Iteration range examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

LIST OF TABLES

xxxi

30.13 Condition examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 30.14 Condition examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 30.15 Condition examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 30.16 Modier values 30.17 Modier values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

30.18 Supported SWT widgets for a Selection event . . . . . . . . . . . . . . . . 436 30.19 Supported DOM nodes for a Selection event . . . . . . . . . . . . . . . . 437 30.20 Provided Check types of Boolean check . . . . . . . . . . . . . . . . . . . 443 30.21 Components supported by Fetch text . . . . . . . . . . . . . . . . . . . . . 471 30.22 Components supported by Fetch geometry . . . . . . . . . . . . . . . . . . 477 30.23 Extra features assigned by QF-Test . . . . . . . . . . . . . . . . . . . . . 513 30.24 Extra features assigned by QF-Test . . . . . . . . . . . . . . . . . . . . . 517 30.25 Sub-items of complex Swing components . . . . . . . . . . . . . . . . . . 519 32.1 Placeholders in lename parameters . . . . . . . . . . . . . . . . . . . . 554 32.2 Exit codes for QF-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 32.3 calldaemon exit codes for QF-Test . . . . . . . . . . . . . . . . . . . . . 556 38.1 Supported AJAX toolkits . . . . . . . . . . . . . . . . . . . . . . . . . . . 626 38.2 GWT resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628 38.3 Ext JS resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . . . 628 38.4 ICEfaces resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . . 629 38.5 PrimeFaces resolver versions . . . . . . . . . . . . . . . . . . . . . . . . 629 38.6 Qooxdoo resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . . 629 38.7 RAP resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 630 38.8 RichFaces resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . 630 38.9 Vaadin resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 38.10 ZK resolver versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 39.1 Internal item representations for Swing GUI elements . . . . . . . . . . . 668 39.2 Internal item representations for SWT GUI elements . . . . . . . . . . . . 669 39.3 Internal item representations for DOM nodes . . . . . . . . . . . . . . . . 669

LIST OF TABLES

xxxii

40.1 The run state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703 41.1 Placeholders for component procedures . . . . . . . . . . . . . . . . . . 713 41.2 Additional placeholders for container procedures . . . . . . . . . . . . . . 713 41.3 Comment attributes for procedure creation . . . . . . . . . . . . . . . . . 714 41.4 Hierarchy placeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715 41.5 Samples for the @CONDITION tag . . . . . . . . . . . . . . . . . . . . . 716 B.1 B.2 B.3 B.4 C.1 C.2 New features in QF-Test version 3.5 . . . . . . . . . . . . . . . . . . . . . 755 New features in QF-Test version 3.4 . . . . . . . . . . . . . . . . . . . . . 773 New features in QF-Test version 3.2 . . . . . . . . . . . . . . . . . . . . . 784 New features in QF-Test version 3 . . . . . . . . . . . . . . . . . . . . . . 797 Shortcuts for navigation and editing . . . . . . . . . . . . . . . . . . . . . 800 Shortcuts for special record and replay functions . . . . . . . . . . . . . . 801

Part I User manual

Chapter 1 Installation and startup

QF-Test is available in three installation options: The qftest-3.5.3.tar.gz archive for Unix only, the executable setup le qftest-3.5.3.exe for Windows only and the qftest-3.5.3.zip archive for both Unix and Windows. Basically it is possible to have different versions of QF-Test installed in parallel. Existing conguration les will not be overwritten in this process.
Note

Both QF-Test and the SUT require Java version 1.5 or higher. In section 24.2
(189)

you can nd best practices about the QF-Test installation.

1.1

Unix Installation

First select a convenient directory that will contain this release of QF-Test as well as future updates. Common choices are /opt or /usr/local. Make sure you have write access to this directory and change to it. When upgrading to a new QF-Test version, use the same directory again. Unpack either the .zip archive with unzip qftest-3.5.3.zip or the .tar.gz archive with tar xfzv qftest-3.5.3.tar.gz. This will create a directory named qftest, which we will refer to as the main or root directory of QF-Test. When the qftest product family grows, this directory will become the home for other products as well. After unpacking a QF-Test archive for the rst time, QF-Tests root directory will hold only the version-specic subdirectory qftest-3.5.3. When upgrading, a new subdirectory for the current version will be added. To nish the installation, change to the specic directory for the current QF-Test version with cd qftest/qftest-3.5.3 and run one of the two setup scripts provided. setup.sh is a plain Bourne shell script while setup.ksh is written for the Korn shell. On Solaris you must run ./setup.ksh. On other Unix systems both scripts should

1.2.

Windows Installation

work equally well but the Bourne shell version ./setup.sh is preferred. Note that if you need to use the Korn shell scripts and want to create you own link or start QF-Test directly from its bin directory, you need to use the qftest.ksh start script instead of the plain qftest script. These setup scripts will create the directories log and jython under QF-Tests root directory if they dont already exist. Additionally they will offer to create a symbolic link from the /usr/local/bin directory (or /usr/bin if there is no /usr/local/bin) to the respective Bourne shell or Korn shell run script for the qftest command. You need to have write permission to the /usr/local/bin directory for the link to be created. Subsequently the default java program for QF-Test is determined. This can be over(540) argument. The setup script searches ridden later with the -java <executable> PATH and proposes to use the rst java program it detects. If you want to use a different program or if none was found, you can enter one. The script determines the JDK version automatically. Next setting to perform is the maximum amount of memory to be used by QF-Test. As default 256MB are taken. Alternatively QF-Test can be started with the -J-XmxZZZm command line argument, where ZZZ denes the memory in MB. Finally the language for QF-Test can be congured. By default the language depends on the system settings, but you can also choose to always use the English or the German version. Note that this setting will affect all QF-Test users. Alternatively you can run QFTest with the -J-Duser.language=XX option using en for English or de for German. Those of the above settings that differ from the default are written to the le launcher.cfg in QF-Tests root directory. This le is read by the qftest launch-script and also evaluated during an update of QF-Test.

1.2

Windows Installation
the Windows specic

On Windows you can install QF-Test using either qftest-3.5.3.exe or the archive qftest-3.5.3.zip.

Installing QF-Test from qftest-3.5.3.exe follows the standard setup procedures for Windows. You will be asked to accept the license agreement and select the root directory for QF-Test. When upgrading to a new version, make sure you select the same directory again (this will be the default). To install QF-Test from the qftest-3.5.3.zip archive, rst extract the archive in a convenient place. The default location is C:\Program Files\qfs. This will create a directory called qftest which we will refer to as the root directory of QF-Test. When the qftest product family grows, this directory will become the home for other products as well. After extracting the archive, run the executable le called setup.exe in the

1.3.

The license le

version-specic directory qftest-3.5.3 under QF-Tests root directory. This will take you through the same setup steps as qftest-3.5.3.exe, except that no les will be extracted. If you later want to uninstall QF-Test youll have to remove its les by hand after running the uninstaller.
Note

Make sure that you select QF-Tests root directory as the target directory for the setup. In both cases, setup will create a menu shortcut and an optional desktop icon for QFTest and create a few registry entries to associate *.qft, *.qrz and *.qrl les with QF-Test. It will also copy the qftest.exe executable to the Windows system directory and store the location of QF-Tests root directory and some version information in the registry, so qftest.exe can be run from the console or batch scripts without further ado. Information about the QF-Test version will also be fetched from the Windows registry. When nished, the setup program will offer to congure the Java program for QF-Test. This can be done with the help of a small dialog in which you can select the Java program for QF-Test. Furthermore, the dialog let you adjust the maximum amount of memory to be used by QF-Test. 256MB are taken by default. The third value to be congured is the language for QF-Test. Normally the language is determined by the system settings, but you can also choose to always use the English or the German version. The values above are stored in the Windows registry from where they are read by the qftest start program. You can run the conguration program any time from the system menu to change these settings. To remove them, clear the respective eld or set is to system standard respectively and select OK.

1.3

The license le

QF-Test requires a license le to run, which you should have received from Quality First Software GmbH. Place this le in QF-Tests root directory. Make sure it is named license with no extension. Some mail clients try to guess the le type and add an extension on their own. When upgrading to a new version you can simply keep the license le provided that it is valid for the new version. If you need to upgrade your license, for example to increase the number of concurrent QF-Test instances or when upgrading to a new version, you will receive a le called license.new from Quality First Software GmbH which is typically not a valid license in itself but must be combined with your current license. To do so, proceed as follows: Place the le license.new in the same directory as the current license. Make sure that this directory and the le license are writable by you.

1.4.

The conguration les

Start QF-Test in interactive mode. QF-Test will detect the license update, verify its validity and offer to upgrade your license le. If you agree, the current license will be renamed to license.old and the new, combined license will be written to license. When you are satised that everything is OK, you can remove the les license.old and license.new. If QF-Test doesnt seem to recognize the license upgrade, make sure that the timestamp of the le license.new is newer than that of the le license. Also make sure that no other instance of QF-Test is running on your computer. In case you need to specify a special name or location for the license le or work with (545) more than one license, this can be achieved with help of the -license <file> (535) argument as described in chapter 32 .

1.4

The conguration les

QF-Test saves all of its window conguration and those global options that represent personal preferences together in one le which is usually called config and placed in the directory .qftest under your home directory (typically called Personal Files or %WINDIR%\profiles\<username> under Windows). This le is not read when (5) QF-Test is run in batch mode (see section 1.5 ). System specic options that need to be shared between users are saved in a le called qftest.cfg in the root directory of QF-Test. The locations of the congura(553) tion les can be changed with the command line arguments -usercfg <file> and (551) -systemcfg <file> and you will have to make use of the latter if you dont have write permission for QF-Tests root directory.

1.5

Starting QF-Test

QF-Test can be run in two modes. In normal mode QF-Test is the editor for test-suites and run-logs and the control center for running programs, capturing events and execut(539) ing tests. When run with the -batch argument, QF-Test goes into batch mode. Instead of opening an editor window, the test-suites given on the command line are loaded and executed automatically without the need for supervision. The result of the (555) (63) test is reected in QF-Tests exit code , optional run-logs (see section 8.1 ) and re(124) ports (see chapter 15 ). The setup script for Unix offers to create a symbolic link from /usr/local/bin to the qftest start script in the qftest-3.5.3/bin directory under QF-Tests root directory. That way you can simply enter qftest at the shell prompt to launch the application.

1.6.

Firewall Security Warning

On Windows a menu shortcut is created as well as an optional desktop icon. You can either launch QF-Test from one of these or by double-clicking a test-suite or a run-log, since these les are associated with the QF-Test application. To run QF-Test from the console type qftest. When run from the command line, QF-Test offers a wide range of arguments for customization, like selecting the Java VM to use. These are explained in detail in chapter (535) 32 . In case different versions of QF-Test are installed at the same time, a specic version can be started by calling the qftest executable directly from the respective qftest-X.Y.Z/bin directory.

1.6

Firewall Security Warning

On startup of QF-Test and/or the System Under Test (SUT) via QF-Test you might get a security warning from the Windows rewall asking whether to block Java or not. As QF-Test communicates with the SUT by means of network protocols, this must not be blocked by the local rewall in order to allow automated testing.

Chapter 2 The user interface

This chapter explains the structure of QF-Tests main window. When you are done reading, it might be a good idea to run QF-Test and try things out. In the Help menu there is an entry labeled Tutorial that should bring up your web browser with a hands-on, learning-by-doing tutorial. Should this fail because QF-Test cannot determine your systems standard browser, the tutorial can be found in the directory qftest-3.5.3/doc/tutorial, where a PDF version is also available.

2.1

The test-suite

Automating a GUI test basically requires two things: control structure and data. The control structure denes what to do and when to do it. The data for a test consists of information about the SUTs GUI components, the events that will be triggered and the expected results. QF-Test combines all of these into one data structure, a tree hierarchy that we call a testsuite. The elements of the tree are called nodes. Nodes can contain child nodes (often just called children) and are themselves contained in a parent node (or just parent ). The root node of the tree represents the test-suite as a whole. There are more than 60 different kinds of nodes all of which are explained in detail in the (285) reference manual . Some nodes are used as containers for data while others control the execution of a test. All of them have their own unique set of attributes. The attributes of the currently selected node are displayed to the right of the tree in a detail view which can be toggled on and off via the ViewDetails menu item.

2.1.

The test-suite

Figure 2.1: Structure of a test-suite

The image above shows an example for a simple test-suite. The attributes of the node named Simple test can be edited in the detail view to the right. The basic structure of a test-suite and thus the child nodes of the Test-suite root node (294) (288) (303) is xed. An arbitrary number of Test-set , Test-case or Test nodes are followed by (361) (315) (524) the Procedures , Extras and Windows and components nodes. The Procedures node (359) (352) (77) and Procedures which are explained further in section 9.3 . The holds Packages Extras node is a kind of playground or clipboard where all kinds of nodes can be added for experimentation or temporary storage. The windows and components of the SUTs (507) (514) user interface are represented as Window and Component nodes which are located below the Windows and components node. To get detailed information about a node or one of its attributes, click on it with the right mouse button and select Whats this? from the context menu. This will bring up a browser displaying the corresponding section of the reference manual.
(285)

2.2.

Basic editing

2.2

Basic editing

Editing a test-suite falls into two categories: operations like Cut/Copy/Paste on the trees nodes and changing the attributes of a node. The latter done either by editing the can be elds in the detail view and selecting OK or pressing Return , or by bringing up a dialog for the selected node with EditProperties or Alt-Return and changing the values there. If you change some values in the detail view and forget to press OK before moving the selection to another node, QF-Test will pop up a dialog with the changed values, asking you to either conrm your changes or discard them. This feature can be (223) turned off with the option Ask before implicitly accepting detail modications . Some non-obvious may come in handy when editing multi-line text key-bindings Ctrl-TAB and Shift-Ctrl-TAB attribute: move the focus out of the text eld, while Ctrl-Return is a shortcut to select the OK button. ) which will take back An extremely useful feature is the EditUndo function (Ctrl-Z any kind of change made to the test-suite, including recordings or use of the replace dialog. Changes are undone step by step. If you nd you went too far and undid more than you wanted, you can use EditRedo (Ctrl-R ) to undo the undone. The number of steps that can taken back are limited only by available memory and can be congured (224) with the option Number of undo levels per suite (default 30).

2.2.1

Navigating the tree

Though the key-bindings for tree navigation are similar to those of most tree components, it wont hurt to mention them here. Besides, QF-Test comes with a few nonstandard bindings that may come in handy.

The cursor keys are used for basic navigation. Up and Down are obvious. Right either expands a closed node or moves down one row while Left closes an open node or moves to its parent. QF-Tests trees support a special variant of multi-selection. Multiple discontinuous regions can be selected, but only among siblings, i.e. children of the same node. If multi-selection across the whole tree were allowed, cutting and pasting nodes would Shift-Up and Shift-Down to extend the become a real brain-teaser. Keys to try are selection, Ctrl-Up and Ctrl-Down to move without affecting the selection and Space to toggle the selection of the current node. Similarly, mouse-clicks with Shift extend the selection while clicks with Ctrl toggle the selection of the node being clicked on. Special bindings include Alt-Right and Alt-Left which recursively expand or collapse a node and all of its children. Alt-Down and Alt-Up can be used to move to the next or previous sibling of a node, skipping the intermediate child nodes.

2.2.

Basic editing

10

QF-Test keeps a history of recently visited nodes. Ctrl-Backspace will take you back to the previously selected node. Also worthy of note are Ctrl-Right and Ctrl-Left which will scroll the tree to the right or left if it doesnt t its frame.

2.2.2

Insertion marker

When inserting a new node or pasting in a copy of some other nodes, the insertion marker shows the place where the nodes will end up.

Figure 2.2: Insertion marker

Nodes are always inserted after the selected node. If the selected node is expanded, the new node is inserted as the rst child of the selected node, otherwise it becomes a sibling of same. This behavior will take a little to get used to, especially for long-time users of the Windows explorer. However, there is no other way to insert a node at a (10) denite position. In the example shown in gure 2.2 above, a new node would be (420) inserted as the rst child of the sequence called Events, just before the Mouse event .

2.2.3

Moving nodes

Nodes can be copied and pasted or moved around within a or to suite. another test-suite The standard keyboard shortcuts for cut, copy and paste, Ctrl-X , Ctrl-C and Ctrl-V are available as well as entries in the context menu.
2.0+

Alternatively, nodes can be moved using standard Drag&Drop operations. The default operation will move the selected node(s). If the CTRL key is held down during the drop, the nodes are copied instead. While dragging the mouse over the tree of a test-suite, the insertion marker shows where the nodes will be dropped when the mouse button is released and whether the operation is allowed. A green marker signals a legal operation, a red marker an illegal one. Nothing will happen if the nodes are dropped on an illegal target position.

2.2.

Basic editing

11

During the drag you can expand or collapse nodes by dragging the mouse cursor over the expansion toggle and keeping it there for a moment. That way you can easily navigate to the desired target location without interrupting and restarting the drag. The Drag&Drop operation can be aborted at any time by pressing Esc .

2.2.4

Transforming nodes

Some nodes can be transformed into different node types, which is a lot more convenient than rst creating the desired target node and then copying over the required at(307) (303) (390) tributes. Examples of interchangable nodes are Sequence and Test or Server script (392) and SUT script . The transformation of a node is possible only if its childnodes and its current position in the tree are also valid for the desired target node. The potential transformation targets can be selected from the entry Transform node into in the context menu. If the entry is not available there are no valid target nodes. In that case, (315) moving the node to the Extras node rst may help. You can nd more details about the conversion mechanism under Details about (719) transforming nodes .

2.2.5

Tables

In various places QF-Test employs tables to view and edit a set of values, e.g. when (52) (442) dening variables or for checks of multiple elements.

Figure 2.3: Example table

The buttons above the tables have the following keyboard shortcuts and effects:

2.2.

Basic editing

12

Shift-Insert

Insert a new row.

, Alt-Return Shift-Return

Edit a row. Opens a dialog with elds for every cell of the selected row.

Shift-Delete

Delete the selected row.

Shift-Ctrl-Up Move the selected row up by one.

Shift-Ctrl-Down

Move the selected row down by one.

Some tables also offer the ability to add and remove columns and edit the column title. For these, the following additional buttons are available: Insert a new column. Delete the selected column. Edit the title of the selected column. To enter a value directly into the selected cell just start typing. This way you overwrite the current value of the cell. To edit the current value, either double click the cell or press . To nish editing press Return , to cancel and restore the old value press Escape . F2 If you try to enter an invalid value the cells border will turn red and you cant accept the value. Multi-selection of table rows is supported via mouse-clicks with Shift/Ctrl and Shift/Ctrl-Up/Down . Cut copy and paste of the selected rows is done with Ctrl-X/C/V . Pasting is restricted to tables with a similar column structure. In the tables context menu additional actions might be available, e.g. show line numbers, locate component, etc. A mouse click in a column header will activate sorting of table rows. A double-click in a column header will resize the column to t the largest value in the column or opens the editor for the header text (data table).

2.3.

Advanced editing

13

2.2.6

Packing and Unpacking

During test development it is often necessary to move several nodes into a new parent node. A typical situation could be the re-factoring of procedures to re-organize them in packages or to wrap a workow into a Try/Catch block. For such requirements QF-Test allows the user to pack nodes into others. This can be achieved by selecting the nodes to pack, right-clicking and selecting Pack nodes and the desired parent node. QF-Test also allows the user to unpack such nodes and remove their parent. This can be used to remove unnecessary packages or test-sets from the structure or to dispense with sequences or Try/Catch blocks that are no longer required. For unpacking right-click the node to unpack and select Unpack nodes .
Note

The packing and unpacking actions are only shown in the menu if the desired target structure is legal.

2.2.7

Sorting Nodes

QF-Test allows sorting nodes. This can be achieved by clicking at a node with the right mouse button and selecting Sort child nodes . Alternatively you can also select multiple nodes, perform a right mouse click and then choose Sort nodes , which will sort the current selected nodes. To guarantee a better overview the sorting algorithm puts ciphers prior to capital letter and those prior to small letters. Sorting doesnt modify the base structure of QF-Test nodes. It also follows the rule to keep Package nodes always prior to Dependency nodes and those always prior to Procedure nodes.
Note

The base structure of a test-suite will not be altered during sorting. You can sort testcases or procedures but the Procedures node will always stay prior to the Windows and components node.

2.3

Advanced editing

This section explains how to use the more advanced editing techniques such as search/replace and multiple views on the same test-suite.

2.3.

Advanced editing

14

2.3.1

Searching

QF-Test provides two kinds of search operations, a general search through all nodes and attributes of a test-suite or run-log and in incremental search through the contents of a text area, including script terminals or program output. General search Though search and replace operations in QF-Test have much in common, there are signicant differences, especially in the scope of the operation. Searching normally starts at the selected node and traverses the whole tree depth-rst to the end. After asking for conrmation the search continues from the root of the tree to the original start of the search so each node is traversed exactly once. This is not unlike search operations in common text processors and should be intuitive to use. By default QF-Test shows the search dialog in simple mode, which allows searching for any appearance of a given text.

Figure 2.4: The simple search dialog

For a more specic search QF-Test allows limiting the search to specic attributes, node types or specic states of nodes. Therefore you have to switch to the advanced mode by clicking the Switch mode button in the toolbar of the search dialog.

2.3.

Advanced editing

15

Figure 2.5: The advanced search dialog

By default QF-Test will search all attributes in all kinds of nodes for the requested string. Use the In attribute option to limit the search to a specic attribute and/or the Node type option to limit it to nodes of a specic kind. The other options should be self(581) explanatory. Regular expressions are explained in section 36.4 . If Match whole attribute is selected, a search for the word tree, for example, will not match an attribute value of treeNode. Activating options in Only nodes with the following states limits the search to nodes that have all of the ectivated states, e.g. a green mark and a breakpoint. The option Scope of operation tells QF-Test where to search for the given expression, below the selected node(s), in the current test-suite or in all currently opened suites.
Note

To search for values of boolean attributes like Implicitly catch exceptions , use true or false (no quotes). If you want to search for an empty value you have to check Match

(306)

2.3.

Advanced editing

16

whole attribute. If the search is successful, the resultant node is selected and a message in the status line displays the name of the attribute that contains the value.
3.4+

As already mentioned the searching process usually starts from the currently selected node. In case you want to select other nodes during your search process you can continue the previous search by using the Search continue button. . Once you have closed the search dialog you can still continue the search pressing F3 You can even trigger the same search from a new node pressing Ctrl-G . A very useful feature is the ability to quickly locate all Procedure call nodes that call a given Procedure or all event nodes that refer to a given Component node, etc. Simply select the entry Locate references... from the context menu of a node that can be called or referred to. This will show a new frame showing all available references of it. You can reach the node in the test-suite via a double click at the row in the list.

Figure 2.6: Result list for Locate references

3.1+

It is also possible to get a list of all found nodes via pressing the Show result list button

2.3.

Advanced editing

17

in the search dialog. From this dialog you can then reach any single node in your testsuite. Incremental text search
3.1+

In addition to searching the tree, components containing text like terminal areas or respective attributes in the details view can be searched independently by use of QF-Tests incremental search feature. This feature can be invoked either by selecting Search... from the components context menu or by pressing Ctrl-F when the component is selected and owns the keyboard focus. Then the incremental search popup dialog appears at the upper right corner of the respective component. The gure below shows an incremental search for the terminal with highlighted search hits.

Figure 2.7: Incremental search

The search can be limited to a certain part of the contained text by selecting the region to be searched and invoking Search in selection... from the components context menu

2.3.

Advanced editing

18

or pressing Ctrl-Shift-F . Beside this, the way the incremental search works as well as the available options should be self-explanatory.

2.3.2

Replacing

Once you understand how the scope of the replace operation differs from searching, the replace dialog should be just as intuitive to use as the search dialog. When the replace operation is in progress, you have a choice of replacing one match at a time or all matches at once. To avoid unexpected results when selecting the latter, there needs to be a way to limit the nodes that will possibly be affected. To that end, replace operations can be limited to the currently selected nodes and their direct or indirect child nodes. For a replace operation that covers the whole tree, either select the root node or choose the respective Scope of the operation option in the dialog.

2.3.

Advanced editing

19

Figure 2.8: The replace dialog

The options are identical to the ones for searching. When Match whole attribute is turned off, multiple replacements within one attribute are possible. When replacing a with b for example, banana would change to bbnbnb. Be sure to read section (581) 36.4 about how to use regular expressions for replacing. If the search is successful, the resultant node is selected and a conrmation dialog is brought up that shows the target attribute and its value before and after the change.

2.3.

Advanced editing

20

Figure 2.9: The replace query dialog

It offers the following choices: When Replace is selected, the attributes value is changed and the search carries on, showing the query again for the next match. Replace all means change this value and all the rest of the matches in one go without asking again. Skip leaves this attribute unchanged. The search continues and the query dialog is shown again in case of another match. Obviously Cancel ends the replace operation. If you know what to expect you can skip the query entirely by selecting Replace all in the replace dialog. After the attributes have been changed, the number of affected nodes is shown in a message dialog.
3.1+

After performing the actual replacement QF-Test will show a list of all touched nodes. You can also open a list of all nodes, which will be touched before the actual replacement pressing the Show result list button in the replace dialog. Whether values are replaced one by one or all at once also affects the way the undo function will take these changes back. All changes of a Replace all operation are taken back in one step, while single changes are undone one at a time.

Note

2.3.3
3.3+

Complex searches and replace operations

Sometimes a simple search is not enough. Imagine, for example, that you want to set a Timeout of 3000 milliseconds for all text checks on a certain component. You know the

2.3.

Advanced editing

21

components Id, but if you search for that Id you will also nd events and other kinds of checks referencing it. And if you search for the node text Check text you will nd all Check text nodes, not just those for the given component. Instead of providing several combinable levels of search criteria QF-Test offers complete exibility through its marks. First perform a search for your rst criterion, e.g. the node text and select Show result list. In the resulting dialog select all entries in the table by pressing Ctrl-A , press Set mark to assign the blue mark to all result nodes and close the dialog. You can now perform a second search or a replacement with the scope limited to nodes with a given mark. In our example you would perform a replacement of the empty string with 3000 on all Timeout attributes with the search scope set the all nodes with the blue mark in the whole tree.

2.3.4

Multiple views

It is possible to open multiple views that show different parts of the same tree structure simultaneously. This can be useful when managing large test-suites or to compare the attributes of different nodes. Additional views are opened via the ViewNew window... menu item. The current node will be the root node for the new view. Additional views are similar to the primary views, but with a limited range of menus.

Chapter 3 Quickstart your application

This chapter provides instructions on how to quickly set up your application as the SUT (System Under Test). In order to make you application recognized by QF-Test as SUT it basically needs to be started out of QF-Test. There are a number of special process nodes available within the InsertProcess nodes to perform this task but the straight forward way is to use the Quickstart Wizard as described below. For those with an aversion to wizard dialogs, (559) the manual way is explained at section 34.1 . A precondition for testing Java based SUTs is that QF-Test can hook into the GUI toolkit:
Swing

For Swing or combined Swing and SWT applications QF-Test hooks into the Javas accessibility interface. Normally QF-Test can do this directly. Only for some non-standard (571) JDKs it may be necessary to instrument those rst. See JDK instrumentation for details if necessary. For Eclipse/SWT based applications, an instrumentation of the SWT library is necessary. The Quickstart Wizard, which is described below, will support you in the task to add the necessary step to the setup sequence. Background information can be found (575) in section 35.3 . Web application testing does not require instrumentation but there are some restrictions to notice. QF-Test only supports only 32bit browsers and needs to run on a 32bit JDK for this task. For Internet Explorer 32bit is ensured automatically, for Firefox please check your installed version. (32bit Firefox releases can be downloaded from http://releases.mozilla.org/pub/mozilla.org/refox/releases/). 32bit JDKs can be downloaded from http://www.java.com. Chapter Installation and (2) startup describes how to congure the JDK to be used by QF-Test.

SWT

Web

3.1.

Quickstart Wizard

23

3.1
2.0+

Quickstart Wizard
Setup sequence creation

3.1.1

With the Quickstart Wizard QF-Test offers a convenient utility for creating a startup sequence for your application. You can open the Quickstart Wizard via the ExtrasQuickstart Wizard... menu item. Please follow the steps which should be self explanatory.

Figure 3.1: Quickstart Wizard

As result the Wizard delivers a startup sequence under the Extras, as shown in the following gure:

3.1.

Quickstart Wizard

24

Figure 3.2: Startup sequence created by the Quickstart Wizard

The setup sequence contains a start node that may vary in type and attributes dependent on your application. The start node is followed by a Wait for client node that ensures QF-Test connects to your application during the startup procedure. (Details (559) about the different start node types and attributes can be found in section 34.1 .)
Swing SWT Web

For Swing based applications the setup sequence typically is straight forward except for (83) Applet testing in a browser which uses a special procedure from the standard library . For SWT based applications an additional procedure call node for SWT (575) instrumentation is added. The standard startup sequence for web includes some additional nodes for setting variables, initializing browser cache and cookie settings and possibly install an AJAX toolkit resolver.

3.1.2

Executing the setup sequence

The setup sequence is either directly started by the Quickstart Wizard after creation or can be executed by selecting the green setup sequence node in the tree and pressing Replay toolbar button .

When executing the setup sequence your application should start up and the Start recording button in the QF-Test toolbar should become activated which indicates that QF-Test is properly connected to the SUT. Now you are able to record and replay your rst test sequences as described in chapter (28) 4 . There is also a learning by doing tutorial available from the QF-Test help menu which guides you through all features of QF-Test. In case you are facing an error message or the red Start recording button stays inactive, please proceed with the following paragraph.

3.1.3

Troubleshooting

If your application (or the browser window in case of web testing) doesnt come up at all:

3.1.

Quickstart Wizard

25

The error dialog QF-Test typically displays should provide a rst indication. Please look for error messages in the terminal window. If there is no terminal window visible in the bottom area of QF-Test, it can be activated through the menu item ViewTerminalShow . Additional information about program output can (26) be found in section 3.2 . Be sure to double-check the attribute values in the setup sequence nodes are correct. Possibly a typo has crept in somewhere. Details about the different start (559) node types and attributes can be found in section 34.1 .
Swing

If you are testing an Applet (possibly on Windows 8) please see section 34.1.3 for additional information.

(562)

Web

As browser development cycles i.e. those of Firefox tend to shorten, be sure the installed browser is supported by the QF-Test version you are using. The terminal (751) output should show a respective error message. See release notes in appendix B for latest supported browser versions. Possibly you need to update QF-Test to a later version or temporarily use another browser. If the error dialog indicates that no 32bit browser and/or 32bit JDK is used, please refer to the beginning of this chapter. If the SUT gets visible but QF-Test is not able to connect to the client (530) (ClientNotConnectedException ): Please double-check the terminal output content (see also above) for possible error messages. In case the case the red record button in the toolbar gets activated after the error (411) message occured, the timeout value in the Wait for client to connect node needs to be increased.

Web

Swing

Please ensure the option Connect without JDK instrumentation (Swing) is acti(571) vated. If you are using a non-standard JDK please check out the section 35.1 chapter. If you are testing an Applet, please see section 34.1.3 There are a number of special cases to consider.
(562)

(253)

Swing

for additional information.

SWT

For an Eclipse/SWT application rst make sure that you specied the correct ap(63) plication directory. You may want to take a look at the run-log (see section 8.1 ) to see if any warnings or errors were logged during execution of the Procedure qfs.swt.instrument.setup.

3.2.

Program output and the Clients menu

26

Check the run-log in general for possible additional error indications (see section (63) 8.1 ). After possibly having adapted your test-suite or settings retry executing your setup sequence. If you are not getting any further you might want to consider trying a sample test-suite from the tutorial or you contact our support.

3.2

Program output and the Clients menu

The standard output and error streams of all processes started by QF-Test are captured and stored in the run-log under the node that represents the respective starter node. In this QF-Test does not distinguish between SUT clients and arbitrary processes or shell (406) (409) scripts started with a Start process or Execute shell command node. The main window contains a shared terminal view that shows the output of all processes started by a test that was run from this window. The ViewTerminal sub-menu holds items to congure whether this terminal is visible, whether long lines are wrapped and whether it is automatically scrolled to the end when new output arrives. Other items let you clear the terminal or save its contents to a le. The maximum amount of text that (252) the terminal holds is congurable in the option Maximum size of shared terminal (kB) . In addition to the shared terminal, for each active or recently terminated process there is an individual terminal window that shows its output. These individual terminal windows can be opened from the Clients menu. The shared terminals intention is to provide visual feedback whenever new output arrives, while the individual terminals are better suited for actually studying that output. Active processes can also be stopped with the help of the Clients menu, either individually in the respective sub-menu or all at once with ClientsStop all . The number of terminated clients that are kept in the Clients menu is set with the option (252) Number of terminated clients in menu . If your processes generate lots of output and you are low on memory you may want to reduce that number.

3.3

An application started indirectly from an already connected SUT

If a second Java VM is started from an already connected SUT, QF-Test will recognize this as an indirect connection attempt from a child process of the rst SUT and automatically assign an articial client name to this new SUT. The name is created by appending :2 to the client name of the parent SUT, signifying that this is the second process for

3.3.

An application started indirectly from an already connected SUT

27

this client. Yet another Java VM started by either of these SUTs would get :3 appended to the original client name unless the second process was already terminated so the :2 was again free for use. To summarize, the sequence for connecting to an indirectly started SUT typically consists of an event node that triggers something like a button click or menu selection, (411) causing the SUT to launch the second SUT, followed by a Wait for client to connect node for the :2 extended client name.

Chapter 4 Capture and replay

Once the SUT is up and running under QF-Test, the next step is to record sequences of events and play them back.

4.1

Recording sequences

In order to record a sequence of events in the SUT, the SUT must have been run from (22) QF-Test (see chapter 3 ) and the connection between QF-Test and the SUT must be established. A visual indicator of this is the color of the record button which turns red when it is enabled.

Figure 4.1: Disabled and enabled Record button

or seTo record a sequence, simply start recording be pressing the record button lecting RecordStart . Then switch to the SUT, execute a few commands, switch back to QF-Test and stop the recording with the stop button or RecordStop . The recorded events will be added to the test-suite, either directly at the position indicated (10) (307) (315) by the insertion marker or as a new Sequence under the Extras node, depending (233) on the setting of the Insert recording at current selection option. Pause the recording if you need to execute some steps in the with RecordPause or the pause button SUT that should not be recorded and you dont want to stop and restart the recording.

4.2.

Running tests

29

Recording mode can be started and stopped direktly in the SUT by use of the Hotkey (233) for recording . Default key is F11 . Any Components referred to by the newly recorded events are added automatically to (524) the Windows and components node if they are not there already. There are many options that inuence the way QF-Test records events and how it treats (233) the components of the GUI. All of these are explained in detail in section 29.2 of the reference manual. Once you are familiar with QF-Test you should take the time to skim through it. Heres some general advice for recording: Record short sequences at a time. After recording, take a look at the sequence, try to understand what you got and whether it represents the actions you took. Edit the sequence to remove unnecessary events, especially those at the beginning and end caused by switching windows. QF-Test has excellent lters that should catch nearly all of these, but some might remain and have to be removed manually. Finally, try out the new sequence to see whether it replays OK. Then you can cut/copy/paste as needed to integrate it into larger parts.
Mac
(514)

For SUTs running on Mac OS X QF-Test disables use of the screen menu bar and activates normal menu bar behavior like on other platforms. This is due to the fact that QF-Test cannot fully access the screen menu bar which prevents proper capture/replay of menu actions. In case the typical Mac screen menu bar behavior is necessary for any reason, this can be forced by adding the line qfs.apple.noScreenMenuBar=false to the le qfconnect.properties that is located in QF-Tests root directory. After restarting the SUT the screen menu bar is supposed to work as normal on Mac.

4.2

Running tests

To run some tests, select the node or nodes to execute and press Return or the play or select RunStart . QF-Test will mark each node with a small arrow as button it is executed and also show progress messages in the status bar. This can slow down (249) execution a little and can be turned off with the options Mark nodes during replay and (249) Show replay messages in status line . When the test is nished, the result is shown in the status bar. If things are ne you should see No errors, otherwise the number of warnings, errors and exceptions is

4.2.

Running tests

30

shown. Additionally, a message dialog is shown in case of errors or exceptions to make sure you dont miss these. As for recording there are many options that inuence the replay of tests. Some of these are only for convenience while others have a major impact on the outcome of the tests. (248) Be sure to read section 29.3 some time to familiarize yourself with these. To abort execution before the test is nished, press the stop button or select or
(63)

RunStop . You can also suspend execution temporarily via the pause button

by selecting RunPause . This will also enable the debugger (see chapter 8 ). To continue, press pause again. While a test is run at full speed it can be tricky to stop or interrupt it, especially when the mouse cursor is actually moved across the screen or the SUTs windows are raised on (248) (the default is Alt-F12 ). every event. To regain control, press the (Dont) Panic key This will pause all running tests immediately. To continue, press the same combination again. While building a test-suite you will often want to execute some sequences to get the SUT to a point where you can continue recording. Sometimes you may want to skip certain nodes at this stage because they dont get you where you want, but you dont want to delete them or move them to some other place. In that case use the EditToggle disabled state menu item to disable the node(s). When you want to use them again later you can re-enable them. The current error state during replay as well as the nal result is shown in the status line at the bottom of the QF-Test main window. The visibility of this status line can be controlled via ViewShow status line . In case Test-set or Test-case nodes (section 9.2 describes their usage) are executed the status line also contains relevant result counters from the following list.
(294) (288) (75)

4.3.

Recording checks

31

Counter Icon

Description Total number of test-cases. This counter value starts with a > symbol in case there are skipped test-sets. Number of test-cases with exceptions. Number of test-cases with errors. (292) Number of test-cases with expected errors. Expected to fail if... marks a test-case expected to fail. Number of successful test-cases. (292) Number of skipped test-cases. A test-case is skipped when its (optional) Condition fails. This counter value starts with a > symbol in case there are skipped test-sets. (298) Number of skipped test-sets. A test-set is skipped when its (optional) Condition fails. Number of not implemented test-cases. A test-case is not implemented when it doesnt contain nodes that were executed during the test-run. Number of executed test-cases. Percent test-cases passed.

Table 4.1: Test result counter in the status line

The nal test result counts also appear in the report which can be created for any test (124) run. Reports are discussed in chapter 15 .
Note

The counter values above can also be accessed as variables during the test-run. (683) A TestRunListener can help to keep track of counter values and trigger dependent actions.

(57)

4.3

Recording checks

Though it can be quite entertaining to record sequences and watch the magic dance of the SUT as they are played back, the task at hand is to nd out whether the SUT (442) actually works as expected. This is where checks come into play. The most common (443) check, the Check text node, reads the text displayed by a component, e.g. a text eld, and compares it to a given value. If the values differ, an error is signaled. How checks work is explained in detail in the Reference manual . There is a range (460) available from simple text check to advance Check image and even custom check (669) types can be implemented. Here we are going to concentrate on the most convenient way to create checks, which is to record them. While recording, the SUT is in record mode, which means that all events are collected and sent to QF-Test. With the help of the check button or by selecting RecordCheck you can bring it into check mode, recognizable through the different
(442)

4.4.

Recording components

32

mouse cursor. In this mode, recording events is suspended. Instead, the mouse cursor is tracked and the component under it is highlighted. When you click on the component, a check for the component is recorded using the value that is currently displayed. To get back to record mode, select the check button or menu item again. There are different kinds of checks that can be performed. Which kinds of checks are applicable depends on the selected component. Some components dont display (443) any text, so a Check text node doesnt make sense for, say, a scroll bar. Clicking on a component with the right mouse button while in check mode brings up a menu of applicable checks for this component. Select one of the items to create the respective check node. Clicking with the left mouse button always records the default check, which is the topmost one in the popup menu. If you hold down the Shift or Ctrl key while clicking with the right mouse button, the check menu will stay open after making a selection. That way, you can easily record multiple kinds of checks for the same component. Checks integrate well with events and youll soon develop a recording style ala click, click, type, click, check, click, click, check... Having to switch back and forth between QF-Test and the SUT every time you want to create a check can be a real pain. That (234) is where the Hotkey for checks option comes into play. It denes a key which toggles the SUT between record mode and check mode. The default value is F12 , but if this key has some dened meaning for your application you can change it to whatever you like. To record a sequence of interspersed events and checks, simply start recording, switch to the and record the sequence. Whenever you want to add some checks, SUT you dened), record the checks, then switch back to record just press F12 (orwhatever mode by pressing F12 again and continue. This way you can work with the SUT for the whole sequence and need to switch back to QF-Test only to stop the recording. One word of warning should be repeated: Dont let this convenience tempt you into recording extremely long sequences. When something changes that causes such a sequence to fail you will be hard put to nd out what went wrong and how to cope.

(442)

4.4

Recording components

As already described component information is automatically stored when recording events or checks. Nevertheless there are situations where capturing of just components proves useful. To activate component recording mode you simply need to press the record components or select RecordRecord components from the menu. Then switch to the button SUT window where you will notice a special behavior in which the component below the mouse cursor is highlighted.

4.5.

Recording of HTTP Requests (GET/POST)

33

Clicking with the left mouse button on a component will record the single component while pushing the right button instead will pop up a menu with choices to record the nested components as well or all components in the window. Multiple components can be captured in this way. Then switch back to QF-Test and release the record components button or deactivate RecordRecord components . Now the recorded (514) component information is stored in shape of respective Component nodes under the (524) Windows and components node. Component recording hotkey. mode can be alternatively controlled by a congurable (240) , the option that congures it is Hotkey for components . Default binding is F11 Pressing F11 (default setting) in the SUT window starts component recording. Further (240) details can be found in the documentation for the option Hotkey for components .
Note

Only one test-suite at a time can receive the recorded components. If more than one test-suite is open and each is shown in an individual window, i.e. workbench mode is deactivated, either the test-suite in which the recording is stopped (toolbar button or menu) or - when using F11 , the test-suite that can be selected via the menu item RecordSuite is receiver for recording will receive the components. The component recording feature can also be used to quickly locate a component independent of whether it has been recorded before. When creating event or checks by (421) hand or changing the target component, the Component id attribute needs to be specied. When a component is recorded, its Id issaved in the clipboard and can be pasted Ctrl-V . You can also jump directly to the Compodirectly into the Component id eld with Shift-Ctrl-Backspace or by choosing the EditSelect next node menu nent node with item or clicking the respective toolbar button. The popup menu which appears when clicking with the right mouse button on a component also contains an entry Show methods which allows of some kind of component (40) inspection (see section 5.5 ). Components play a central role in the structure of a test-suite which is explained further (35) in chapter 5 .

4.5
Web

Recording of HTTP Requests (GET/POST)

In order to record a (GET/POST) request sent by the SUT, the SUT must have been (22) launched from QF-Test (see chapter 3 ) and the connection between QF-Test and the SUT must be established. While recording, the SUT is in record mode, which means that all events are collected and sent to QF-Test. With the help of the HTTP-request button or by selecting

4.5.

Recording of HTTP Requests (GET/POST)

34

RecordRecord HTTP Requests you can bring it into request recording mode. In (28) contrast to Recording sequences all GET/POST-request send by the web browser are saved as http-request nodes in this special recording mode. To get back to record mode, select the HTTP-request button or menu item again. In section Web options the ability to change the type of the recorded request is de(504) scribed. By default Browser HTTP request is recorded. This Request type is likely used to automate large web form inputs, the use of separate input nodes will be avoided. The form data will be submitted within the browser, so that the response will be shown as well. At this point the test execution could be continued. In opposition to this the (500) Server HTTP request will be directly submitted through QF-Test whitout the need of a running browser. The response is also only available in QF-Test and a eventually running browser will not be affected. All attributes of an accordingly recorded request node as well as the parametrization of (500) requests are explained in detail in the HTTP Requests section of the reference part of this manual.
(268)

Chapter 5 Components
Though they often go unnoticed, at least until the rst (526) (514) ComponentNotFoundException occurs, the Component nodes are the heart of a test-suite. Everything else revolves around them. Explaining this requires a little side-tracking:

5.1

GUI component hierarchy

The GUI of an application consists of one or more windows which hold a number of components. The components are nested in a hierarchical structure. Components that hold other components are called containers. As QF-Test itself is a complex application, its main window should serve well as an example:

5.1.

GUI component hierarchy

36

Figure 5.1: Components of a GUI

The window contains a menu bar which holds the menus for QF-Test. Below that is the toolbar with its toolbar buttons. The main area employs a split pane to separate the tree view from the details. The tree view consists of a label (Test-suite) and the tree itself. The detail view contains a complex hierarchy of various components like text elds, buttons, a table, etc. Actually there are many more components that are not obvious. The tree, for example, is nested in a scroll pane which will show scroll bars if the tree grows beyond the visible area. Also, various kinds of panes mainly serve as containers and background for other components, like the region that contains the OK and Cancel buttons in the detail view.
SWT

In SWT the main GUI components are called Control, Widget or Item. Unless explicitely stated otherwise the term component, as used in this manual, also applies to these and not only to AWT/Swing Components. The internal representation of an HTML page is based on the Document Object Model (DOM) as dened by the W3C, a tree structure consisting of nodes. The root node, a Document can contain Frame nodes with further Document nodes and/or a root

Web

5.2. Components in QF-Test

37

Element with a tree structure of further Element nodes. Though an HTML page with its DOM is quite different from a Swing or SWT interface, the abstractions QF-Test uses work just as well and the general term component also applies to DOM nodes. Actions by the end-user of an application are transformed into events by the Java VM. Every event has a target component. For a mouse click this is the component under the mouse cursor, for a key press it is the component that has the keyboard focus. When an event is recorded by QF-Test, the component information is recorded as well, so that the event can later be replayed for the same component. This may sound trivial and obvious, but component recognition is actually the most complex part of QF-Test. The reason for this is the necessity to allow for change. QFTest is a tool designed for regression testing, so when a new version of the SUT is released, tests should continue to run, ideally unchanged. So when the GUI of the SUT changes, QF-Test needs to adapt. If, for example, the OK and Cancel buttons were moved from the bottom of the detail view to its top, QF-Test would still be able to replay events for these buttons correctly. The extent to which QF-Test is able to adapt varies and depends on the willingness of developers to plan ahead and assist a little bit in (40) making the SUT well-suited to automated testing. But more on that later (section 5.6 (42) and section 5.7 ).

5.2

Components in QF-Test
(507) (514)

The recorded components are transformed into Window and Component nodes which form a hierarchy that represents the actual structure of the GUI. These nodes (524) are located under the Windows and components node. The following image shows part of the Components representing QF-Tests main window.

Figure 5.2: Component hierarchy of a Swing SUT

5.3. Component Ids

Web

38
(510)

Instead of a Window, the root Document of a web page is represented as a Web page node. Nested Documents inside Frames are represented as Component nodes.

Every time a sequence is recorded, nodes are generated for components that are not yet represented. When the sequence is discarded later on, the Components remain, hence Component nodes have a tendency to proliferate. The popup menu (right button click) for Window and Component nodes has two items, Mark unused components... and Remove unused components , which will mark or remove those Component nodes that are no longer being referred to. Be careful though if you are referencing Components across test-suite boundaries or use variable values in Component id attributes as these are not taken into account unless the test-suites belong to the same project or the (286) Dependencies (reverse includes) attribute of the Test-suite root node is set correctly . The attributes of Components and the algorithm for component recognition are explained (579) in detail in section 36.2 . Here we will concentrate on the association between Component nodes and the rest of the test-suite.

5.3

Component Ids

Every node of the test suite has an Id attribute which is secondary for most kinds of (515) has an important function. It is the nodes. For Component nodes however, the Id unique identier for the Component node by which events, checks and other nodes that have a target component refer to it. Such nodes have a Component id attribute which is set to the Components Id. This level of indirection is important. If the GUI of the SUT changes in a way that QF-Test cannot adapt to automatically, only the Component nodes for the unrecognized components need to be updated to reect the change and the test will run again. It is essential to understand that the Components Id is an articial concept for QF-Tests internal use and should not be confused with the Name attribute, which serves for identifying components in the SUT and is explained in detail in the following section. The actual value of the Id is completely irrelevant, except for the requirement to be unique, and it bears no relation whatever to the actual component in the GUI of the SUT. However, the Id of the Component is shown in the tree of the test-suite, for Component nodes as well as for events and other nodes that refer to a Component. For this reason, Components should have expressive Ids that allude to the actual GUI component. When creating a Component node, QF-Test has to assign an Id automatically. It does its best to create an expressive value from the information available. The option Prepend (245) parent id to component id controls part of this process. If the generated Id doesnt suit you, you can change it. QF-Test will warn you if you try to assign an Id that is not unique and if you have already recorded events that refer to the Component, it will change their Component id attribute to reect the change. Note that this will not cover references with

5.4.

Component class

39

a variable Component id attribute.

Note

A common mistake is changing the Component id attribute of an event instead of the Id itself. This will break the association between the event and the Component, leading to (531) an UnresolvedComponentIdException . Therefore you should not do this unless you want to change the actual target component of the event. Experienced testers with a well-structured concept for automated testing will nd the (32) component recording feature described in section 4.4 useful. It can be used to record the component hierarchy rst in order to get an overview over the structure of the GUI and to assign Ids that suit you. Then you can continue to record the sequences and build the test-suite around the components.

5.4

Component class

The class of a component is a very important attribute. Simply spoken, when QFTest records a button, it will only look for a button on replay, not for a table or a tree. Thus the component class conveniently serves to partition the components of a GUI. This improves performance and reliability of component recognition, but also helps you associate the component information recorded by QF-Test with the actual component in the GUI.
Web

For Swing and SWT QF-Test works with the actual Java GUI classes whereas a pseudo class hierarchy is used for web applications as follows:

Figure 5.3: Pseudo class hierarchy for web elements

5.5.

Component inspection

40

As shown, NODE is at the root of the pseudo class hierarchy. It matches any kind of element in the DOM. Derived from NODE are DOCUMENT, FRAME, DOM_NODE and DIALOG, the types of nodes implementing the pseudo DOM API explained in (615) section 38.1 . DOM_NODE is further sub-classed according to the tag name of the node, e.g. H1, A or INPUT where some tags have an additional subclass like INPUT:TEXT. Generic classes are used for unication of typical elements like Button or Table that are implemented differently in each AJAX toolkit. The option Record generic (269) class names for components determines whether generic classes are recorded. Finally, a custom class can be assigned to DOM nodes via a ClassNameResolver (see (636) section 39.1.2 ) which is a basic principle for the AJAX resolvers that come with QF(625) Test (see section 38.2 ). Besides its role in component identication, the class of a component is also important for registering various kinds of resolvers that can have great inuence on the way QF(634) Test handles components. Resolvers are explained in detail in section 39.1 .

5.5
3.1+

Component inspection

Depending on its class a component has a set of (public) methods and elds which (392) can be used in an SUT script once you have a reference to the object (see section (101) 13.3.7 ). Select the entry Show the components methods... from the context menu (524) of a node under the Windows and components branch to display the methods and elds of the corresponding class or right click on a component in the SUT while you are in (32) component recording mode (see section 4.4 ).
Web

The methods and elds displayed for (HTML) elements in a browser cannot be used directly with an object returned by rc.getComponent(). These are at Javascript level (615) and reqiure a wrapping of the method calls into evalJS (cf. section 38.1 ).

5.6

The importance of naming components

Test automation can be improved tremendously if the developers of the SUT have either planned ahead or are willing to help by dening names for at least some of the components of the SUT. Such names have two effects: They make it easier for QFTest to locate components even after signicant changes were made to the SUT and they are highly visible in the test-suite because they serve as the basis for the Ids QFTest assigns to components. The latter should not be underestimated, especially for components without inherent features like text elds. Nodes that insert text into components called textName, textAddress or textAccount are far more readable and maintainable than similar nodes for text, text2 or text3. Indeed, coordinated naming

5.6.

The importance of naming components

41

of components is one of the most deciding factors for the efciency of test automation and the return of investment on QF-Test. If development or management is reluctant to spend the little effort required to set names, please try to have them read this chapter of the manual.
Note

Please note that recorded names are stored in the Name attribute of Component nodes. Because they also serve as the basis for the Id for the same node, Name and Id are often identical. But always keep in mind that the Id is used solely within QF-Test and that the Name is playing the critical part in identifying the component in the SUT. If the name of a component changes, it is the Name attribute that must be updated, there is no need to touch the Id. The technique to use for setting names during development depends on the kind of SUT:

(516)

Swing

All AWT and Swing components are derived from the AWT class Component, so its method setName is the natural standard for Swing SUTs and some developers make good use of it even without test automation in mind, which is a great help. Unfortunately SWT has no inherent concept for naming components. An accepted standard convention is to use the method setData(String key, Object value) with the String name as the key and the designated name as the value. If present, QF-Test will retrieve that data and use it as the name for the component. Obviously, with no default naming standard, very few SWT applications today have names in place, including Eclipse itself. Fortunately QF-Test can derive names for the major components of Eclipse/RCP based applications from the underlying models with good results - provided that IDs were specied for those models. See the Automatic component names for Eclipse/RCP (243) applications option for more details. The natural candidate for naming the DOM nodes of a web application is the id attribute of a DOM node - not to be confused with the Id attribute of QF-Tests Component nodes. Unfortunately the HTML standard does not enforce IDs to be unique. Besides, id attributes are a double-edged sword because they can play a major role in the internal Javascript operations of a web application. Thus there is a good chance that id attributes are dened, but they cannot be dened as freely as the names in a Swing or SWT application. Worse, many DHTML and Ajax frameworks need to generate id attributes automatically, which can make them unsuited for naming. The option Turn (269) id attribute into name where unique enough determines whether QF-Test uses id attributes as names. In case you want to test a web application using a supported AJAX toolkit, please take (631) a look at section 38.2.3 for details about assigning ids. If developers have implemented some other consistent naming scheme not based on the above methods, those names can still be made accessible to QF-Test by implement(634) ing a NameResolver as described in section 39.1 .

SWT

Web

Web

5.7.

Considerations for setting names

42

The reason for the tremendous impact of names is the fact that they make component recognition reliable over time. Obviously, locating a component that has a unique name assigned is trivial. Without the help of a name, QF-Test uses lots of different kinds of information to locate a component. The algorithm is fault-tolerant and congurable and has been ne-tuned with excellent results. However, every other kind of information besides the name is subject to change as the SUT evolves. At some time, when the changes are signicant or small changes have accumulated, component recognition will fail and manual intervention will be required to update the test-suite. Another aspect of names is that they make testing of multi-lingual applications independent of the current language because the name is internal to the application and does not need to be translated.

5.7

Considerations for setting names

There is one critical requirement for names: They must not change over time, not from one version of the SUT to another, not from one invocation of the SUT to the next and not while the SUT executes, for example when a component is destroyed and later created anew. Once a name is set it must be persistent. Unfortunately there is no scheme for setting names automatically that fullls this requirement. Such schemes typically create names based on the class of a component and an incrementing counter and invariably fail because the result depends on the order of creation of the components. Because names play such a central role in component identication, non-persistent names, specically automatically generated ones, can cause a lot of trouble. If development cannot be convinced to replace them with a consistent scheme or at least drop them, such names can be suppressed with the help of a NameResolver as described (634) in section 39.1 . QF-Test does not require ubiquitous use of names. In fact, over-generous use can even be counter-productive because QF-Test also has a concept for components being interesting or not. Components that are not considered interesting are abstracted away so they can cause no problem if they change. Typical examples for such components are panels used solely for layout. If a component has a non-trivial name QF-Test will always consider it interesting, so naming trivial components can cause failures if they are removed from the component hierarchy in a later version. Global uniqueness of names is also not required. Each class of components has its own namespace, so there is no conict if a button and a text eld have the same name. Besides, only the names of components contained within the same window should be unique because this gives the highest tolerance to change. If your component names (260) are unique on a per-window basis, set the options Name override mode (replay) and (242) Name override mode (record) to Override everything. If names are not unique per window but identically named components are at least located inside differently named

5.8.

Updating Components

43

ancestors, Hierarchical resolution is the next best choice for those options. Two questions remain: Which components should have names assigned and which names to use? As a rule of thumb, all components that a user directly interacts with should have a name, for example buttons, menus, text elds, etc. Components that are not created directly, but are automatically generated as children of complex components dont need a name, for example the scroll bars of a JScrollPane, or the list of a JComboBox. The component itself should have a name, however. If components were not named in the rst place and development is only willing to spend as little effort as possible to assign names to help with test automation, a good strategy is to assign names to windows, complex components like trees and tables, and to panels that comprise a number of components representing a kind of form. As long as the structure and geometry of the components within such forms is relatively consistent, this will result in a good compromise for component recognition and useful Id attributes. Individual components causing trouble due to changing attributes can either be named by development when identied or taken care of with a NameResolver. Since QF-Test knows the components for which setName is most useful, it comes with a feature to locate and report these components. QF-Test even suggests names to assign, though these arent necessarily useful. This feature is similar to component record(240) ing and is explained in the documentation for the option Hotkey for components .
Web

The suggested names for DOM nodes are currently not very useful.

5.8

Updating Components

Unavoidably the components of the SUT are going to change over time. If names are used consistently this is not really a problem, since in that case QF-Test can cope with just about any kind of change. Without names however, changes tend to accumulate and may reach a point where component recognition fails. To avoid that kind of problem, QF-Tests representation of the SUTs components should be updated every now and then to reect the current state of affairs. This can be done with the help of the Update component(s) menu-item in the context menu that you get by right-clicking on any node under the (524) Windows and components node.
Note

This function can change a lot of information in your test-suite at once and it may be difcult to tell whether everything went ne or whether some components have been misidentied. To avoid problems, always create a backup le before updating multiple components. Dont update too many components at once, take things Window by Window. Make sure that the components you are trying to update are visible except for the menu-items. After each step, make sure that your tests still run ne.

5.8.

Updating Components

44

Provided that you are connected to the SUT, this function will bring up the following dialog:

Figure 5.4: Update components dialog

If you are connected to multiple SUT clients, you must choose one to update the components for. Select whether you only want to update the selected Component nodes as well.
(514)

node or all its child

You can choose to include components that are not currently visible in the SUT. This is mostly useful for menu-items. The Id for an updated node is left unchanged if Use Id of original node is selected. Otherwise, updated nodes will receive an Id generated by QF-Test. If the Id of a node is changed, all nodes referring to that node via their Component id attribute will be updated accordingly. QF-Test also checks for references to the component in all suites of the (286) same project and in those suites that are listed in the Dependencies (reverse includes) (285) attribute of the Test-suite node. Those suites are loaded automatically and indirect dependencies are resolved as well.
Note
(515)

In this case, QF-Test will open modied test-suites automatically, so you can save the changes or undo them. After pressing OK, QF-Test will try to locate the selected components in the SUT and fetch current information for them. Components that are not found are skipped. The Component nodes are then updated according to the current structure of the SUTs GUI, which may include moving nodes to different parents.

Note

For large component hierarchies this very complex operation can take a while, in extreme cases even a few minutes. This function is especially useful when names have been set for the rst time in the SUT. If you have already generated substantial test-suites before convincing the developers

5.9.

Troubleshooting component recognition problems

45

to add names, you can use this function to update your Components to include the new names and update their Ids accordingly. This will work best if you can get hold of an SUT version that is identical to the previous one except for the added names.
Note

Very important note: When updating whole windows or component hierarchies of significant size you may try to update components that are not currently visible or available. In that case it is very important to avoid false-positive matches for those components. You may want to temporarily adjust the bonus and penalty options for component recognition (259) to prevent this. Specically, set the Feature penalty to described in section 29.3.3 a value below the Minimum probability, i.e. to 49 if you have not changed the default settings. Dont forget to restore the original value afterwards. If you need to change the setting of the options Name override mode (replay) and (242) Name override mode (record) because, for example, component names turned out not to be unique after all, change only the setting for the recording options before updating the components. When nished, change the replay option accordingly.
(260)

5.9

Troubleshooting component recognition problems

If your SUT has changed in a way that makes it impossible for QF-Test to locate a (526) component, your test will fail with a ComponentNotFoundException . This should (531) not be confused with an UnresolvedComponentIdException which is caused by removing a Component node from the test-suite or changing the Component id attribute of an Event node to a non-existing Id. When you get a ComponentNotFoundException, rerun the test with QF-Tests debugger activated so that the test gets suspended and you can look at the node that caused the problem. Here it pays if your Id attributes are expressive because you need to understand which component the test tried to access. If you cannot gure out what this node is supposed to do, try to deactivate it and rerun the test to see if it runs through now. It could be a stray event that was not ltered during recording. In general your tests should only contain the minimum of nodes required to achieve the desired effect. If the node needs to be retained, take a look at the SUT to see if the target component is currently visible. If not, you need to modify your test to take that situation into account. If the component is visible, ensure that it was already showing at the time of replay by checking the screenshot in the run-log and try to re-execute the failed node by singlestepping. If execution now works you have a timing problem that you need to handle by (263) either modifying the options for default delays (section 29.3.4 ) or with the help of a (484) Wait for component to appear node or a Check node with a Timeout. As a last resort you can work with a xed delay. If the component is visible and replay fails consistently, the cause is indeed a change in the component or one of its parent components. The next step is identifying what

5.9.

Troubleshooting component recognition problems

46

changed and where. To do so, re-record a click on the component, then look at the old (524) and new Component node in the hierarchy under Windows and components .
Note

You can jump directly from the Event node to the corresponding Component node by pressing Ctrl-W or right-clicking and selecting Locate component . You can jump back via Ctrl-Backspace or EditSelect previous node . A clever trick is to mark the Component nodes to compare by setting breakpoints on them to make them easier to spot. The crucial point is where the hierarchy for those two components branches. If they are located in different Window nodes, the difference is in the Window itself. Otherwise the old and new Component have a common ancestor just above the branching point and the crucial difference is in the respective nodes directly below that branch. When you have located those nodes, examine their attributes top-to-bottom and look for differences.

Note

You can open a second QF-Test window via ViewNew window... so as to place the detail views of the nodes to compare side to side. The only differences that will always cause recognition failures are Class name and Name. Differences in Feature, structure or geometry attributes can usually be compensated unless they accumulate. A change in the Class name attribute can be caused by refactoring done by development, in which case you need to update your Class name attribute(s) to reect the change(s). Another possible cause is obfuscation, a technique for making the names of the application classes illegible for protection against prying eyes. This poses a problem because the class names can then change with each version. You can prevent both refactoring (242) and obfuscation problems by activating the option Record system class only . If the Name has changed things get more difcult. If the change is apparently intentional, e.g. a typo was xed, you can update the Name attribute accordingly. More likely the cause is some automatically generated name that may change again anytime. As explained in the previous section, your options in this case are discussing things with development or suppressing such names with the help of a NameResolver as described (634) in section 39.1 . Changes to the Feature attribute are common for Window nodes, where the Feature represents the window title. When combined with a signicant change in geometry such a change can cause recognition to break. This can be xed by updating the Feature to match the new title or, preferably, by turning it into a regular expression that matches all variants. Depending on the kind and amount of changes to accommodate there are two ways to deal with the situation: Update the attributes of the old node and remove the newly recorded ones. Keep the new node and remove the old one, rst taking care that events referring to the old node are updated to the new Id. Theres a clever trick to achieve that:

5.10.

Accessing hidden elds on a web page

47

Edit the Id of the old Component node to match the new one. QF-Test will complain about the duplicate Id, which you can ignore, and offer to update all references, which you should accept. Then you can safely remove the old node.
Note

Automatic updates for references from other test-suites require that the suites belong to (286) the same project or the correct setting the Dependencies (reverse includes) attribute of the Test-suite root node.

5.10
Web

Accessing hidden elds on a web page

Hidden elds are not captured by default and therfore not stored under the (524) Windows and components node. In case you frequently need to access hidden elds you can deactivate the Take visibility (271) of DOM nodes into account option. Another way to get hidden elds recorded is the following: Activate the record components mode, navigate the mouse cursor to the parent item containing the hidden eld (most likely a FORM element). Press right mouse button and select Component and children from the popup menu. Deactivate the record components
(14) (32) (32)

mode.
(524)

A search within Windows and components your destination component quickly.

for e.g. HIDDEN should get you to

To access a hidden elds attributes (e.g. the value attribute) you can create a simple (392) SUT script as shown below. Details on scripting in general, the used methods and (94) (589) (615) and Pseudo DOM API parameters can be found in Scripting , Run-context API respectively.
node = rc.getComponent(id of the hidden component, hidden=1) node.getAttribute(value)

Example 5.1: Accessing the value attribute of a hidden eld

Chapter 6 Sub-items of complex GUI components

Some components of a Java GUI, for example trees or tables, are rather complex and can contain an arbitrary number of elements like tree-nodes or table-cells. These elements are not themselves GUI components in the Java sense, but just a graphical representation of some data. This is a purely technical distinction and because it makes sense, from the testers point of view, to treat these elements as separate entities and possible targets for events, QF-Test has special support for them. The representation (519) for such an element is called an Item .

6.1

Items

An Item is dened by two parts, the component it belongs to and an index into this component. The component is identied by the parent node of the Item node. The index can be either numeric or textual. Numeric indexes start with 0. In a JList, for example, the element with index 1 is the second element from the top. Plain numeric indexes are not very useful for tree-nodes, because only the visible nodes are counted. Expanding or collapsing a tree node changes the numeric indexes of all nodes below it. A textual index refers to the element by the text that is shown in the GUI. A list-item shown as item1 in a JList component would be recorded with at textual index of item1. The textual representation is more exible but can cause problems if the names of the elements in a component are not unique. A textual index can also be given as (581) a regular expression (see section 36.4 ). In this case, the rst element found that matches the regular expression will be used. The option Sub-item format
(246)

controls the format for recording indexes.

Almost all kinds of Items have only one index. The exception is the table-cell in a JTable

6.1. Items

49

component. Since tables are two-dimensional structures, two indexes are required to (520) refer to a table-cell. The rst one, the Primary index , denes the column of the table (520) and the Secondary index the row.

Figure 6.1: An Item for a table cell

Tree-nodes are also special. The hierarchic structure of a tree doesnt map well to a linear one. One reason is the dependence of the numeric index on the current expansion state of the nodes. Another reason is that many kinds of trees have nodes with non-unique names. However, when the direct and indirect parent-nodes are taken into account to work with paths instead of names, uniqueness is often restored. QF-Test uses a special syntax to represent tree nodes. An index that starts with a / character is considered to be a path index. Imagine a tree that shows a le-system hierarchy. The le /tmp/le1 could be represented at as just le1, possibly colliding with another le1 in some other directory, or as the full, unique path /tmp/le1. QF-Test also supports numeric indexes with this syntax. As a numeric index /2/3 designates the fourth child node of the third child of the root. A mixed-format index like /tmp/2 to get the third child of the node named tmp is currently not possible.
Note

This special syntax makes / a special character for elements of tree components, so (583) it needs to be escaped when a literal / character is required. See section 36.6 for details about escaping and quoting special characters.

6.2.

The Id of an Item

50

6.2

The Id of an Item
(515) (38)

Everything that was said about the Id attribute of Components in section 5.3 also (520) applies to an Items Id . It has to be unique and can be referred to by events and similar kinds of nodes that require a component target. When assigning an Items Id automatically, QF-Test creates it by concatenating the containing Components Id and the index (or indexes) of the element. These kinds of Ids are expressive and typically easy to recognize. However, they are also the source of a common misunderstanding: If you want to change the index of a recorded Item to point to some other element, you must not change the Component id attribute of the node that refers to it (unless you use the special syntax described in the following section). Change the Primary index of the Item node instead.

6.3

Special Id-syntax for direct element access

There is an alternative way to target an event on an element of a complex component that does not require creation of an Item node. By using a special syntax you can specify the index of the element directly in the Component id attribute of the respective node. Instead of the Id of an Item node specify the Id of the containing Component and follow it with a special separator character and the index of the element. The character that separates the Id and the index designates the format for the index:
Separator @ & % Index format Textual index Numeric index Regular expression

Table 6.1: Separators and index formats for element access

To address a table-cell with Primary index and Secondary index simply append another separator character and the second index. That way, the Primary index and Secondary index can be of different format. Example: If your test-suite contains a Component node for a JTable component with the Id table and you want to simulate a click on table-cell in the 6th row of the column (420) (421) with the title Name, create a Mouse event node and set its Component id attribute to table@Name&5. Almost all internal ItemResolver implementations (cf. section 39.2 ) allow of negative indices to start counting from the end. For example, table&-1&-1 addresses the last cell in the rightmost table column.
(658)

3.3+

6.4.

Activating item recording for web tables

(519)

51

Whether it is preferable to use an Item node or the special syntax depends on the situation. You can use either method as you see t and mix them as you like. As a rule of thumb, Item nodes are better suited to components with a few more or less constant elements, the columns of a JTable for example, or the tabs of a JTabbedPane. The special syntax is generally preferable for indexes with variable values and for elements (246) with varying or editable values. The option Sub-item type determines whether Item nodes are created or the special syntax is used when recording. With the default value Intelligent QF-Test will follow the rules outlined above.
Note

The special meaning of the separator characters implies that literal @, & or % char(583) for details about acters in element indexes need to be escaped. See section 36.6 escaping and quoting special characters.

6.4
Web

Activating item recording for web tables

Within web pages tables are used both for real visible tables and for invisible layout purpose. Also nested tables are possible. QF-Test typically can not know where a real table starts and therefore item based recording is not possible without further information. Hence this information can be provided by the user. By use of the procedure qfs.web.table.recording.activateItemRecording from the standard (83) library item recording can be activate for a given web table. In case you dont know which component id to provide to the procedure, simply record a click to an arbitrary table cell. Jump to the respective component node by pressing Ctrl-W or right-clicking and selecting Locate component and look for the next node with the class name TABLE upwards in the component hierarchy. That should be the one to use.
Note

When using one of the AJAX toolkit resolvers you typically dont have to do any separate activation. AJAX tables are handled by the resolver.

(625)

Chapter 7 Variables
Variables are the primary means to add exibility to a test-suite. Though they are used (352) mainly as parameters for Procedures , they are also useful in many other cases. Variable syntax can be used in almost every attribute of the nodes of a suite, except for boolean values (check boxes). There are three versions of variable references: $(varname) expands to the value of some previously dened variable. ${group:name} accesses external data from a resource bundle or property le. The groups system and qftest are always dened an have a special mean(57) ing (see section 7.5 ). $[expression] evaluates some mathematical expressions .
(57) (57)

7.1

Variable lookup

To understand the reasons for why and how variables are dened in multiple places, you rst have to learn about how the values of variables are determined. Each set of variable denitions, called bindings, is placed on one of two stacks of bindings. One stack is used for actual - or direct - denitions and one for fallback bindings or default values. When a variables value is requested via $(...), QF-Test rst searches the stack of direct bindings from top to bottom, then the stack of fallbacks, also top down. The rst value found is used. If there is no binding at all for a (528) name, an UnboundVariableException is thrown unless you use the special syntax ${default:varname:defaultvalue} to provide a default value for this case as (57) described in section 7.5 .

7.2.

Dening variables Topmost bindings (highest precedence) ... Bottommost bindings (lowest precedence) Direct bindings stack Topmost bindings (highest precedence) ... Bottommost bindings (lowest precedence) Fallback bindings stack

53

Figure 7.1: Direct and fallback bindings

This mechanism supports recursive or self-referencing variable denitions. For example, setting a variable named classpath to the value some/path/archive.jar:$(classpath) will extend a binding for classpath with lower precedence. If no such binding exists, a (528) RecursiveVariableException is thrown.

7.2

Dening variables

Variables are dened in various places with different orders of precedence. You dene a variable by adding a row to the appropriate table with the name and value of the variable in the respective columns.

Figure 7.2: System variables

7.2.

Dening variables
(283)

54 :

Four sets of variable denitions are accessible from the global options System variables

The most basic set of bindings, commonly used for system-specic denitions like path names, JDK version, operating system stuff etc. These are placed at the bottom of the fallback stack so they have the least possible precedence. The system bindings are saved in the system conguration le together with the other QF-Test system options. Suite variables These are general denitions for a test-suite and saved as part of the suite. The bindings of the suite from which a test run is started, are placed at the bottom of the direct bindings stack. When a procedure from a different suite is called, or a different suites window or component is accessed, the bindings of that suite are temporarily placed on top of the fallback stack. Variables from the command line You can dene variables on the command line when starting QF-Test with the (553) -variable <name>=<value> argument which can be used multiple times. These bindings are placed above the suite bindings on the direct bindings stack, so they override the bindings of the suite. For example, a variable count=1 can (364) be dened for the suite and used as $(count) for the Number of iterations of a (362) Loop node for quick trial runs. Then you can use qftest -batch -variable count=100 ... for the actual testing. The command line bindings are accessible mainly for your information, but you can also edit them for experimentation. Global variables Placed above the command line variables on the direct call stack, these bindings are a means to pass values between unrelated parts of a test run. Values read (471) (474) from the SUT by a Fetch text or a Fetch index node are bound here, as well as (481) denitions from a Set variable node. The globals are not cleared in between test runs as a convenience and you can edit them for the same reason. To simulate a real test run you should clear them via the RunClear global variables menu (5) rst. When run in batch mode (see section 1.5 ) QF-Test clears the global (551) variables before the execution of each test given with the -test <n>|<id> command line argument. The rest of the variable denitions are part of the test-suite itself:

7.3.

Variable example

55

Each sequence type node denes a set of bindings that are placed on top of the direct bindings stack when the test run enters the node and removed when the node is exited. A Procedure calls bindings are placed on top of the direct bindings stack before (352) executing the Procedure . The bindings dened by a Procedure are placed on top of the fallback stack during execution of the Procedure. Think of them as default values for the expected parameters. Once the procedure returns, both sets of bindings are removed.
(355)

7.3

Variable example

Consider the following example:

Figure 7.3: Variable example

The Sequence Login contains a Procedure call of the Procedure login which expects two parameters: user and password. The Procedures default bindings are user=username and password=pwd. The Procedure call overrides these with user=myname and password=mypassword. The login Procedure itself holds Procedure calls of Procedures in another test-suite called lib.qft to write the user and password to some GUI components. We assume that the Procedures in this library have most parameters in common, so they dont dene default values themselves but in the suite bindings of lib.qft where they are set to user=libuser and password=libpwd. This is useful for creating and editing the procedures in the library since they can all be executed and tried out separately without having to dene Procedure calls. The following diagram shows the State of the binding stacks during the execution of the Procedure lib.qft#setUser:

7.4.

Fetching data from the GUI Procedure call lib.qft#setUser Procedure call login user=myname password=mypassword Sequence Login Global bindings Command line bindings Suite bindings Direct bindings stack Procedure lib.qft#setUser Test-suite lib.qft user=libuser password=libpwd Procedure Login user=user password=pwd System bindings Fallback bindings stack

56

Figure 7.4: Variable bindings example

The important thing to note here is that the Procedure call to lib.qft#setUser inside the Procedure login does not need to dene the user parameter again, the parameter is passed through. As a rule of thumb, when calling one Procedure from another, dene a parameter value in the Procedure call if and only if the value has not been explicitly dened yet or you want to pass a different value.

7.4

Fetching data from the GUI

Often it is necessary to fetch some kind of data from the SUTs GUI to use it as test input. QF-Test offers a special set of query nodes InsertMiscellaneous : Fetch text
(471) (470)

for this kind of task, available via

to fetch a component or item text, to fetch the index of an components item,

(477)

Fetch index

(474)

Fetch geometry

to fetch a component or items geometry.

The retrieved values are aligned to local or global variables that can be declared in the fetch node. Instead of inserting fetch nodes by hand they can be easily created by rst recording a (11) mouse click node the the respective component and then use the transform operation to convert the same to the fetch node you need.

7.5.

External data and special groups

57

7.5

External data and special groups

(495) (498)

External data is made available by the Load resources and Load properties nodes. These assign a group name to a set of denitions from a resource bundle or properties le. To access the value of the denition for name, use the syntax ${group:name}. When run in batch mode (see section 1.5 ) QF-Test clears the resources and properties (551) before the execution of each test given with the -test <n>|<id> command line argument. In normal mode QF-Test keeps them around to ease building a suite, but for a true trial run you should clear them via the RunClear resources and properties menu rst. Some special group names are predened and always available: The group system gives access to the system properties of the Java VM (for programmers: java.lang.System.getProperties()), e.g. ${system:user.home} for the users home directory or ${system:java.class.path} for the CLASSPATH with which QF-Test was started. Which names are dened in the group system depends on the utilized JDK. The group always refers to the VM QF-Test was started with, because variable expansion takes place there. On operating systems which support environment variables like PATH or CLASSPATH (practically all systems QF-Test runs on), these environment variables can be accessed with the help of the group env.
3.4+
(5)

You can specify a default value for a variable with the group default. The syntax is ${default:varname:defaultvalue}. This is extremely useful for things like generic components or in almost every place where there is a reasonable default for a variable because the default value is then tightly coupled with the use of the variable and doesnt have to be specied at Sequence or test-suite level. Of course you should only use this syntax if the variable lookup in question is more or less unique. If you are using the same variable with the same default in different places it is preferable to use normal syntax and explicitly set the default, so that the default for all values can be changed in a single place. The group id can be used to reference component IDs. Values in this group simply expand to themselves, i.e. ${id:whatever} expands to whatever. Though component IDs can be referenced without the help of this group, its use increases the readability of tests. Most notably however, ID references in this group will be updated automatically in case the referenced target component gets moved or its ID changed.

3.1+

7.5.

External data and special groups

58

The special group named qftest provides miscellaneous values that may be useful during a test run. The following tables list the values currently dened.
Name batch client.exitcode.<name> Meaning true if QF-Test is running in batch mode, false for interactive mode. (396) The exit-code of the last process started with the Client attribute set to <name>. In case the process is still alive the result is the empty string. (396) The output of the last process started with the Client attribute set to <name>. The maximum size for buffered output is dened by the option Maximum size of client (252) terminal (kB) . A list of the names of all active process clients, separated by a newline. A list of the names of all process clients, separated by a newline. This includes live clients as well as the recent dead clients similar to those listed in the Clients menu. Number of exceptions in the current test-run. Number of errors in the current test-run. Number of warnings in the current test-run. Total number of total test-cases (run and skipped) in the current test-run. Number of test-cases with exceptions in the current testrun. Number of test-cases with errors in the current test-run. Number of test-cases expected to fail in the current testrun. Number of successful test-cases in the current test-run. Percentage of successful test-cases in the current testrun. Number of skipped test-cases in the current test-run. Number of not implemented test-cases in the current testrun. Number of run test-cases in the current test-run. Number of skipped test-sets in the current test-run. QF-Tests root directory Version-specic directory of QF-Test The qftest executable matching the currently running QF-Test version, including the full path to its bin directory and with .exe appended on Windows. Useful if you need to run QF-Test from QF-Test for example to call a daemon or create reports.

client.output.<name>

clients clients.all

count.exceptions count.errors count.warnings count.testcases count.testcases.exception count.testcases.error count.testcases.expectedtofail count.testcases.ok count.testcases.ok.percentage count.testcases.skipped count.testcases.notimplemented count.testcases.run count.testsets.skipped dir.root dir.version executable

7.5.

External data and special groups

59

java

return runid Name screen.height screen.width skipnode

suite.dir suite.file suite.path testcase.name testcase.qname

testcase.reportname testcase.splitlogname

testset.name testset.qname

testset.reportname

Standard Java program (javaw under Windows, java under Unix) or the explicit Java argument if QF-Test is (540) started with -java <executable> (352) The most recent value returned from a Procedure (358) through a Return node. (125) The runid of the current test-run. See section 15.1 for further information about the runid. Meaning Screen height in pixels Screen width in pixels This magic value is not for the casual user. It causes QF-Test to skip execution of the current node. Its primary use is as the value for a variable dened in the (428) (427) Text attribute of a Text input node which also has (429) attribute set. An empty its Clear target component rst value would clear the eld whereas $_{qftest:skipnode} leaves the eld unchanged. But skipnode is also applicable for ne-grained execution control by placing a variable in the comment of a node and selectively passing $_{qftest:skipnode} to that variable. Please note that you almost always want to use lazy syntax $_ with this variable. Otherwise its expansion as the parameter in a Procedure call node would cause skipping the whole call. Directory of the current suite File name of the current suite without directory File name of the current suite including directory The name of the current Test-case, empty if no Test-case is currently being executed. The qualied name of the current Test-case, including the names of its parent Test-sets. Empty if no Test-case is currently being executed. The expanded report name of the current Test-case, empty if no Test-case is currently being executed. The qualied name of the current Test-case converted to a lename, including the names of its parent Test-sets as directories. Empty if no Test-case is currently being executed. The name of the current Test-set, empty if no Test-set is currently being executed. The qualied name of the current Test-set, including the names of its parent Test-sets. Empty if no Test-set is currently being executed. The expanded report name of the current Test-set, empty if no Test-set is currently being executed.

7.6.

Expressions

60

testset.splitlogname

teststep.name teststep.qname

teststep.reportname thread

threads

version windows

The qualied name of the current Test-set converted to a lename, including the names of its parent Test-sets as directories. Empty if no Test-set is currently being executed. The name of the current Test-step, empty if no Test-step is currently being executed. The qualied name of the current Test-step, including the names of its parent Test-steps, but not including Test-cases or Test-sets. Empty if no Test-step is currently being executed. The expanded report name of the current Test-step, empty if no Test-step is currently being executed. The index of the current thread. Always 0 except if QF-Test is started with the argument (553) -threads <number> . The number of parallel threads. Always 1 except if QF-Test is started with the argument (553) -threads <number> . QF-Test version true under Windows, false under Unix

Table 7.1: Denitions in the special group qftest

7.6

Expressions

The $[...] syntax for mathematical expressions supports the operations +, -, *, / and % (modulo) as well as braces. It comes in handy when calculating coordinates, e.g. in a loop. In fact these expressions are far more powerful, since they are evaluated by the Jython interpreter. You can use arbitrary constructs that are legal syntax for the Jython method (94) eval. See chapter 13 for details about Jython scripting.
Note

Accessing QF-Test variables in an expression follows the same rules as in Jython (98) scripts (see section 13.3.3 ). You can use the standard QF-Test syntax $(...) and ${...:...} for numeric or Boolean values. String values should be accessed with rc.lookup(...).

7.7
3.0+

Immediate and lazy binding

There is a very subtle issue in using QF-Test variables that requires further explanation:

7.7.

Immediate and lazy binding

61

When a new set of variable bindings is pushed on one of the variable stacks, there are two possibilities for handling variable references in the value of a binding, for example when the variable named x is bound to the value $(y). The value $(y) can be stored literally, in which case it will be expanded some time in the future when $(x) is referenced somewhere, or it can be expanded immediately, so that the value of the variable y is bound instead. The rst approach is called lazy or late binding, the second approach immediate binding. The difference, of course, is the time and thus the context in which a variable is expanded. In most cases there is no difference at all, but there are situations where it is essential to use either lazy or immediate binding. Consider the following two examples: A utility test-suite contains a procedure for starting the SUT with different JDK versions. The variable jdk is passed as a parameter to this procedure. For ease of use, the author of the test-suite denes some additional useful variables at test-suite level, for example a variable for the java executable named javabin with the value /opt/java/$(jdk)/bin/java. At the time javabin is bound in the test-suite variables, jdk may be undened, so immediate binding would cause an exception. But even if jdk were bound to some value, immediate binding would not have the desired effect, because the java executable is supposed to be the one from the JDK dened later by passing the parameter jdk to a procedure. Thus lazy binding is the method of choice here. Imagine another utility test-suite with a procedure to copy a le. Two parameters called source and dest specify the source le and destination directory. The caller of the procedure wants to copy a le called data.csv from the same directory as the calling test-suite to some other place. The natural idea is to bind the variable source to the value ${qftest:suite.dir}/data.csv in the procedure call. With immediate binding, ${qftest:suite.dir} will indeed expand to the directory in which the calling suite resides. However, if lazy binding were used, the actual expansion would take place inside the procedure. In that case, ${qftest:suite.dir} would expand to the directory of the utility suite, which most likely is not what the caller intended. In versions of QF-Test up to and including 2.2 all variable expansion was lazy. As the examples above show, both variants are sometimes necessary. Since immediate binding is more intuitive it is now the default. This can be changed with the option When (283) binding variables, expand values immediately . The option Fall back to lazy binding (284) if immediate binding fails complements this and helps to ease migration of old testsuites to the use of Immediate Binding. The warnings issued in this context help locating the few spots where you should use explicit lazy binding as described below. Except for very rare cases where lazy binding is required but immediate binding also works so that the fallback is not triggered, all tests should work out of the box. In the few cases where it makes a difference whether a variable is expanded immediately or lazily, the expansion of choice can be selected individually, independent of the setting of the above option, by using an alternative variable syntax. For immediate binding use $! instead of just $. Lazy binding is selected with $_. For example, to dene a

7.7.

Immediate and lazy binding

62

variable at test-suite level that species a le located in this test-suites directory, use $!{qftest:suite.dir}/somele. If immediate binding is the default and you require lazy binding as in the jdk example above, use $_(jdk).
Note

With lazy binding the order of variable or parameter denitions in a node or a data driver did not matter because nothing was expanded during the binding stage. With immediate bindings, variables are expanded top-to-bottom or, in a data driver, left-to-right. This means that if you dene x=1 and y=$(x) it will work, with y being set to 1, if x is dened rst. If y is dened rst the denition will either fail or trigger the lazy denition fallback described above.

Chapter 8 Problem analysis and debugging

The whole point of creating automated tests is to uncover problems in the SUT. Therefore we can justiably expect the tests to fail occasionally. After the execution of a test has nished, a message will appear in the status line that will hopefully say No errors. If something went wrong, the numbers of warnings, errors and exceptions that occurred is shown. Additionally an error dialog may pop up. In that case you will need to nd out what went wrong. For some problems the cause may be obvious, but very often it is not. First and foremost in this case is the need to determine whether the test failed due to a bug in the SUT or whether the SUT behaved correctly but the logic of the tests was wrong. The dilemma here is that any potential problem in the SUT must be duly reported as soon as possible, yet every false bug report is a waste of time and will cause resentment among the developers. Therefore, each problem needs to be thoroughly analyzed and every alleged bug in the SUT should ideally be reproducible before a report is submitted. QF-Test supports testers in this crucial task in two ways. A detailed log is created for each test-run that holds all the relevant information for post mortem error analysis, including screenshots taken at the time that an error occurred. The integrated test debugger helps with analyzing and understanding the ow of control and information during a test-run.

8.1

The run-log

During test replay QF-Test creates a run-log that records everything that is going on. The run-logs for recent tests are accessible from the Run menu, the current or most (68) recent run-log can also be opened by typing Ctrl-L . See section 8.1.5 for information about options inuencing run-log creation.

8.1.

The run-log

64

The structure of a run-log is similar to that of a test-suite, but there are subtle differences. (323) (325) Nodes are added to the run-log when they are executed. Setup and Cleanup nodes, for example, are typically executed more than once, in which case multiple copies will be recored in the run-log as shown below:

Test-suite

Run-log

Figure 8.1: A simple test and its run-log

A run-log is the essential resource for determining what went wrong in a test, where it went wrong and maybe even get an idea about why it went wrong. Therefore the emphasis is on completeness of information rather than readability and a run-log should not be confused with a report or summary. Report generation is covered in chapter (124) 15 . In addition to the nodes copied from the test-suite, a run-log contains failure information, optional annotations, information about variable expansion, various kinds of messages and, not to forget, timestamps and durations.
Note

The difference between the values of Duration and Real time spent are explicit delays introduced in nodes via the Delay before/after attribute or user interrupts. The information gathered from a long test-run accumulates and can eat up enormous amounts of memory, so QF-Test normally creates a special form of compact run-log in which only the information relevant for report generation and for error diagnosis is re(282) tained. This functionality is congurable through the option Create compact run-log (542) and the command line argument -compact . This should not be mixed up with the option to save compressed run-logs as *.qrz les which simply uses plain gzip compression to reduce the size of the log le, but doesnt have any impact on memory usage. Another alternative to save memory is to create split run-logs as described in (67) section 8.1.4 . Compact and split run-logs can be combined for maximum efciency.

8.1.

The run-log

65

8.1.1

Error states

There are three kinds of failures differing in the level of severity: Warnings Warnings indicate problems that are typically not serious, but might lead to trouble in the future and may need looking at. For example, QF-Test issues a warning, if the best match for a component barely meets the requirements and differs in some signicant way. Errors Errors are considered to be serious defects that require closer inspection. They indicate that the SUT does not fulll some requirement. A typical cause for an (443) error is a mismatch in a Check text node. Exceptions Exceptions are the most serious kinds of errors. An exception is thrown when a situation occurs in which QF-Test cannot simply continue with the execution of the test. Most exceptions indicate problems with the logic of the test, though they (526) can just as well be caused by the SUT. A ComponentNotFoundException , for example, is thrown when no component in the SUT matches the intended (526) target for an event. A list of all possible exceptions is available in chapter 31 . Each node of a run-log has an associated state which can be one of normal, warning, error or exception. This state is visually represented by a frame around the nodes icon which is orange for warning, red for error and bold red for exception.

Figure 8.2: Error states in a run-log

8.1.

The run-log

66

As shown in the (somewhat reduced) screenshot above, error states propagate from bottom to top. The exception state takes precedence over the error state, which in turn overrides warning. The most severe kind of error that propagates to the top of the tree determines the nal result of a test and QF-Tests exit code when run in batch mode (555) (see section 32.3 ). If necessary, the propagation of errors can be restricted for all kinds of Sequence (308) nodes with the help of the Maximum error level attribute. This can be useful for sequences which are known to contain errors that should not be taken into account just (379) (382) and Catch nodes. The yet. Exceptions can be handled with the help of the Try (383) Maximum error level attribute of the Catch node determines the state to propagate for a caught exception.
(307)

8.1.2

Navigating the run-log tree

All of the basic editing methods for a run-log are similar to those for a test-suite. One signicant difference is that can neither add or remove any nodes nor edit the attributes of the nodes copied from the test-suite. You can add annotations though, for example to document the reason for an error if it is known. The rst question to answer when looking at a run-log is What happened? The function EditFind next error , Ctrl-N for short, moves the selection to the next place at which an error actually occurred and EditFind previous error (Ctrl-P ) moves backwards. (276) (277) The options Find next error function locates and Skip suppressed errors determine the kinds of errors to look for and whether to ignore those that didnt propagate up to the root node. Theres a menu item shortcut EditSkip suppressed errors to quickly toggle the latter option. The next question might be Where did this happen? Though a run-log is similar in many ways to a test-suite, the connection isnt always obvious. The function EditFind node in test-suite (Ctrl-T ) will take you to the exact node in the test-suite that is represented by the selected node in the run-log, always provided that the test-suite can be located and hasnt changed in a way that prevents this. If the run-log is loaded from a le, the corresponding test-suite may not be located at the same place as when the test was executed. If the test-suite cannot be found, a dialog will pop up that lets you select a different le. In case you select a wrong le or some other test-suite is found instead of the one the run-log was created from, you may end up at some totally different node, or none at all. In that case you can use the menu item EditLocate corresponding test-suite to explicitely change the test-suite.

8.1.

The run-log

67

8.1.3

Accepting values of failed checks as good

A noteworthy feature of QF-Tests run-log is the ability to quickly accept the actual values of a failed Check node as good. When QF-Test logs a failed Check it includes the complete current state of the Check nodes target component in the SUT. This is much more useful than the failure message alone, which, for example, might just tell you that a table column has 10 rows instead of the expected 9, but not what its contents are. If you are analyzing a failed Check and see that the value in the SUT was actually correct and the expected value stored in the test-suite wrong, you can press Ctrl-U or select Update check node with current data from the context menu to accept the data from the failed Check as the new correct value for the Check node. Warning: QF-Test currently doesnt preserve regular expressions in Check text (452) Check items nodes, they will simply get overwritten.
(443)

or

8.1.4
3.0+

Split run-logs

Run-logs for long-running tests can get very large and consume an enormous amount of memory, even more so in case many screenshots are kept. Compact run-logs help, but not enough to make tests that run for days on end possible without turning off the run-log entirely. The best way to overcome the memory problem are split run-logs. For split run-logs, whenever a certain part of a test has nished, QF-Test takes the runlog for that part, removes it from the main run-log, saves it to a separate le and replaces it with a single node that references that le. The partial logs are complete run-logs in themselves and can be viewed and archived independently, though normally they are accessed through the main run-log. When navigating the main run-log or when creating reports, QF-Test transparently loads the partial run-logs from the separate les as required and removes them from memory when no longer needed. This makes it possible to navigate huge run-logs while still retaining a relatively small memory footprint. Of course operations like searching or report creation that need to traverse the whole run-log become slower, but jumping from error to error remains quite fast and loading the main run-log is sped up drastically. There are two ways for storing a main run-log and its partial logs: All combined together in a single ZIP le with the extension .qzp or with the partial logs in a separate directory. The latter is named after the main run-log with the extension .qrl or .qrz removed and the sufx _logs appended. Inside a .qzp ZIP le the same layout is used so that it is possible to zip or unzip les manually without breaking the internal references in the run-log. This compatibility is also the reason why by default partial logs inside the ZIP are are stored compressed with the extension .qrz. This is slightly less efcient than storing uncompressed .qrl les, but that way a .qzp run-log can be unzipped without its overall size exploding.

8.1.

The run-log

68

To make use of split run-logs you can explicitely dene points at which a run-log is (332) broken and split into parts. This is done via the Name for separate run-log attribute of (330) (288) (294) (299) (303) Data driver , Test-case , Test-set , Test call or Test nodes. When used with a Data driver, the logs for each iteration are saved separately, in the other cases the node with the Name for separate run-log attribute is split off. Otherwise partial run-logs are split off automatically when they reach a certain size. This functionality can be congured (278) via the option Minimum size for automatic splitting (kB) . Split run-logs are also very handy for tracking the progress of a test in batch mode. In that context it is extremely useful that the le names for the partial logs can be created using the same placeholders as when specifying the name of the main run-log on the command line. In particular the error state of the partial log can be made part of its (332) lename. Please see the documentation for the attribute Name for separate run-log for details.

8.1.5

Run-log options

There are several options that inuence the creation of run-logs and their content. These (275) are explained in detailed in section 29.6 . Following is a short summary of the most important options: is turned off, recent run-logs are saved in Unless Automatically save run-logs the directory .qftest in the users home directory to save memory and to make them persistent between sessions. sets the number of run-logs to keep around after Number of run-logs in menu a test is run. Unless you turn off automatically saved run-logs there is no need to reduce this number. Log level for the SUT determines which kinds of messages are logged. Logging all messages uses up lots of memory. If Create compact run-log is set, only nodes containing relevant error or report information are retained. This is a very useful setting in most cases. off, split run-logs can be disabled. Normally By turning Create split run-logs the Name for separate run-log attributes of various nodes determine if and where a run-log is split. With Dont create run-log creation of run-logs can be suppressed entirely. This is dangerous, since no information will be available in case of errors. Since split run-logs have become available this option is no longer required to make longrunning demos or stress tests possible, but it is retained for backwards compatibility.
(282) (277) (282) (280) (276) (276)

8.2.

The debugger

69

8.1.6
3.3+

Creating a test-suite from the run-log

If several people are involved in the test development process, it might be useful to generate a test-suite from the run-log directly. The generated test-suite could be used to reproduce a test-run on-the-y without having the entire structure of test-suites. You can create a test-suite from the run-log via performing a right mouse click at any node in the run-log and selecting Create test-suite from the context menu. QF-Test creates a new le containing all executed steps of the respective tests under (315) Extras as well as the used components.
Note

QF-Test only adds the executed steps to the new test-suite. Variables will be expanded immediately, so you can only see their value in the new le. Organizational nodes like procedures or control structures will not become created. You have to set a couple of options in order to get this feature properly working (Under Run-log -> Content): Create compact run-log Log variable expansion
(282)

needs to be switched off. needs to be switched on.

(280)

(280)

Log parent nodes of components

needs to be switched on.

If you have access to all test-suites, you can use also use information from them for creating the new one. Therefore select Create test-suite from existing structure from the context menu. In contrast to the approach described above, it is not required to (280) switch on the option Log parent nodes of components .

8.2

The debugger

As in any development environment, at some point the need will arise to debug problems introduced into a test-suite which cannot readily be solved by a straight-forward analysis of the elements and structure of the suite. To this end, QF-Test includes an intuitive debugger. Those of you familiar with debugging programs in Java or other programming languages will nd this debugger similar in function and usefulness.

8.2.1

Entering the debugger

The QF-Test debugger can be started directly by selecting a node (or some nodes) to execute and pressing the step-in or step-over buttons, or by using the menu

8.2.

The debugger

70

operations DebuggerStep in and DebuggerStep over or the keyboard shortcuts

F7 and . F8

See section 8.2.3

(71)

for a detailed explanation of these operations.

If you are running tests on your test-suite and use the play button to start execution (see (29) section 4.2 ), the debugger will normally not be entered. However, the debugger will be activated automatically when any one of the following occur: A user-dened breakpoint is reached (see section 8.2.4 points).
(72)

on turning on/off break

Execution is interrupted manually by pressing the pause button or F9 or selecting the RunPause menu item. A caught or uncaught exception is thrown, an error happens or a warning is logged and the respective option to break under that condition is set (see option Automatic (274) breaks ). When the debugger suspends execution of the test, the node about to be executed will be shown with a colored frame around its icon that reects the cause for the break. If the debugger is stopped due to manual intervention, a user breakpoint or when stepping, the frame will be black. When stopping due to a warning, error or exception the frame will be orange, red or thick red respectively, exactly like the error indicators in the run-log.
Note

When the debugger is entered due to a warning, error or exception it will move execution back to the beginning of the node that caused it, giving you a chance to x the cause of the problem and re-execute that node again. If this is undesirable or impossible you can (71) simply skip the node (see section 8.2.3 ).

8.2.2

The debugger window

The debugger can be run either from within the normal test-suite view, or by opening a dedicated debugger window by selecting DebuggerShow debugger window once the debugger has been entered. You can also cause the debugger window to open automatically whenever the debugger (274) is entered by setting the option Always open debugger window in the global options dialog or under the DebuggerOptions menu. If you open or close the debugger window explicitely, this is considered a manual override and this option will be ignored for the rest of the test-run. The debugger window is similar to a normal test-suite window. You can select nodes and edit their attributes, but you cannot delete or insert nodes, there are no le operations and no recorder. For the more complex operations you can quickly jump from the debugger window to the same node in the respective test-suite window by pressing

8.2.

The debugger

71

Ctrl-T or

selecting Find node in test-suite from the Edit menu or the context popup

menu.

The lower half of the debugger window shows the nodes that are binding variables on (52) the primary and fallback variable stacks (see chapter 7 ). For the primary stack all nodes are shown, even if they are not binding any variables. This is useful because it serves as a kind of stack-trace for the current point of execution of the test-run. You can double-click on any node to quickly navigate to the node in its associated test-suite. A single click on a node brings up its variable bindings in the right half of the variable display. There the variable values can be edited, new variables can be added or existing ones removed. These changes immediately affect the current test-run, but are of a temporary nature. They are not propagated to the nodes in which the variables were bound in the rst place.

8.2.3

Debugger commands

Most of the debugger commands are similar to those of any other debugger. However, some additional commands are included that deal with special situations. Step-wise debugging of a test-suite is available through three operations: (F7 , DebuggerStep in ) executes the currently selected The step-in button node and will set the execution mark to the next deepest node, regardless of how deep that node may lie in the node structure. This operation is useful, for example, to step into and debug a Procedure or Sequence. The step-over button (F8 , DebuggerStep over ) executes the currently selected node as well as any child nodes that lie under it and then sets the execution mark to the next node at the same level. This is helpful for being able to execute an entire Procedure or Sequence without stepping through each step individually. The step-out button (Ctrl-F7 , DebuggerStep out ) executes the currently selected node as well as any other nodes at the same level (including any child nodes of these nodes) and then sets the execution mark to the next node at the next higher level. This type of operation is useful when, for example, you are debugging a Procedure or Sequence and dont want to step through the rest of the nodes in it. By simply using step-out, you can execute the rest of the nodes and return. The skip functions expand the QF-Test debugger in a powerful way which is not typically possible for a debugger in a standard programming environment. In short, they allow you to jump over one or more nodes without having to execute those nodes at all.

8.2.

The debugger

72

The skip-over button (Shift-F9 , DebuggerSkip over ) jumps over the current node without executing it, moving the execution mark to the next node. The skip-out button (Ctrl-F9 , DebuggerSkip out ) ends the execution of the current Procedure or Sequence and jumps to the next node at the next higher level. Even more powerful is the ability to continue the test-run at any arbitrary node, even in a completely different test-suite. QF-Test will keep as much of the current execution context as possible, including variable bindings. The closer the new target location is to the current point of execution, the more information can be salvaged. You can switch execution to a different node by pressing Ctrl-, or by selecting the menu item RunContinue execution from here or the respective item in the context menu. When you do so, execution will not continue immediately, only the next node to be executed will change. You can continue the test as usual by single-stepping or resuming the test-run. The following additional commands are available: ( DebuggerRethrow exception ) is only active The rethrow-exception button when the debugger was entered due to an exception. It lets you rethrow the exception to be handled by the test-suite just as if the debugger had never caught it in the rst place. This is especially useful when the Break on caught exception option is set. ( DebuggerLocate current node ) quickly The locate-current-node button moves the selection in the tree-view to the node that is about to be executed. It is a useful shortcut in case you get lost while moving around the test-suite.

8.2.4

Manipulating breakpoints

Setting a breakpoint on a node will tell the debugger to suspend execution just before it enters that node. Breakpoints are displayed in the tree-view by prepending (B) to the name of a node. Breakpoints can be set or removed individually with Ctrl-F8 or with the DebuggerBreakpoint on/off menu item. After nishing a debugging session you can use DebuggerClear all breakpoints to remove any breakpoints that might have been left hanging around. This command will remove all breakpoints from all test-suites.
Note

Breakpoints are transient and will not be saved with the test-suite.

Chapter 9 Organizing the test-suite

Creating useful, reliable tests requires more than just recording sequences and playing them back. You can ll a test-suite with lots of sequences in a short time, but you are bound to lose track of what youve got sooner or later. You need to give structure to your tests to make sure that: You have a good idea of what you are testing. You are testing the right thing. Your tests are reliable, repeatable and easy to maintain in case the SUT changes in a way that QF-Test cant adapt to automatically. The results of your tests are conclusive. The essential prerequisite of getting the components right has been discussed in (35) chapter 5 . Here we are going to concentrate on structuring the actual sequences, events, checks, etc.

9.1

Sequence and Test nodes

(307)

The primary building block of a test is the Sequence node which executes its child (303) nodes one by one in the order they appear. The Test node is very similar but adds semantics for setup and cleanup. Before QF-Test version 2 it was the prime repre(288) sentative of a test-case. In that role it has been replaced by the new Test-case and (294) (75) Test-set nodes which are described in the following section 9.2 . However, Sequence and Test nodes are still very useful low-level building blocks. Their difference lies in their semantics.

9.1. Sequence and Test nodes

74

Figure 9.1: Sequences and Tests

The concept behind a Sequence node is that its child nodes must be run in exactly that order and that each node depends on the outcome of the previous node to work correctly. A sequence of recorded events is a very good example for what constitutes a Sequence node. Imagine what would happen if the events were rearranged in a random order. The semantic contract of the Test node on the other hand is that each of its child nodes is independent of all of the other children, so the order in which they are executed shouldnt really matter. Achieving such a state of independence requires that each child of a Test node can rely on the same set of preconditions. It is the Test nodes job to ensure that these conditions are met. To that end, a Test node can have two special additional child (323) (325) nodes, the Setup and the Cleanup nodes. When a Test node is executed, its child nodes are executed one by one in the order they appear. We said this before, but it is a half truth only. Every time it executes one of its normal children, the Test node rst runs the Setup node which is responsible for providing the settings that the child node requires. Whenever a child node is done, the Cleanup node is executed to clean up any mess in the SUT the child node may have caused. This may sound needlessly complex, but implementing all the preconditions of a testcase and isolating test-cases so that they are completely independent are so important that in QF-Test version 2 Setup and Cleanup nodes have essentially been replaced by (317) nodes which are an even more powerful mechanism based on the new Dependency (85) described in chapter 12 . But as Tests still have their use at lower levels, so do Setup and Cleanup nodes. A Test node with well-matched Setup and Cleanup nodes has the following important qualities: You can add child nodes to the Test at any time and any position without disturbing the other child nodes. When you want to extend a test or just try out some sequence, you dont need to execute all the previous steps rst to get the SUT into the proper state. You can run any sub-set of a Test node if you dont want to run the full test.

9.2.

Test management with Test-set and Test-case nodes

75

If one child of a Test node fails for whatever reason, the rest of its children can still be executed, provided that the Setup and Cleanup node can be relied on to provide the proper settings under these circumstances. In a Sequence, where each node depends on the outcome of its predecessor, the failure of one node will invalidate the whole sequence. The Implicitly catch exceptions of the kind described above.
(306)

attribute of a Test node helps to facilitate error handling

9.2
2.0+

Test management with Test-set and Test-case nodes

(294) (288)

The Test-set and Test-case nodes provide a small-scale, pragmatic form of test management right inside QF-Test. Their main feature is the smart dependency management (85) described in the following chapter 12 that allows Test-cases to be implemented completely independent from each other. With properly written Dependencies, cleanup of the SUT from previously executed tests is handled automatically along with the setup for the next test and all error handling.

9.2.1

Concepts

Conceptually a Test-case node represents a single elementary test case. As such it is the main link between test planning, execution and result analysis. With the help of Dependencies, Test-cases can be isolated from each other so that they can be run in any arbitrary order. QF-Test automatically takes care of the necessary test setup. Cleanup is also automatic and will be performed only when necessary in order to minimize overhead in the transition from one test to the next. This enables things like running subsets of functional test-suites as build tests or retesting only failed Test-cases. Test-sets basically are bundles of Test-cases that belong together and typically have similar requirements for setup and cleanup. Test-sets can be nested. The whole structure of Test-sets and Test-cases is very similar to Package and Procedure nodes. The Test-suite root node can be considered a special kind of Test-set. Test-suite, Test-set and Test-case nodes can be called from other places using a Test call node. That way, tests that run only a subset of other tests can easily be created and managed. Test call nodes are allowed everywhere, but should not be executed from inside a Test-case node because that would break the atomicity of a Test-case from the reports point of view. A warning is issued if Test-case execution is nested.

9.2.

Test management with Test-set and Test-case nodes

76

9.2.2

Variables and special attributes

As both Test-sets and Test-cases can be called from a Test call node they each have a (352) set of default parameters similar to those of a Procedure . These will be bound on the fallback stack and can be overridden in the Test call node. A Test-case has an additional (303) set of variable bindings similar to those of a Test node. These are hard bindings for the primary variable stack that will be dened during the execution of the Test-case and cannot be overridden in a Test call node. The list of Characteristic variables is a set of names of variables that are part of the characteristics of the test for data-driven testing. Each execution of the Test-case with a different set of values for these variables is considered a separate test case. The expanded values of these variables are shown in the run-log and report for improved error analysis. of an If Another useful attribute is the Condition which is similar to the Condition node. If the Condition is not empty, the test will only be executed if the expression evaluates to true. Otherwise the test will be reported as skipped. Sometimes a Test-case is expected to fail for a certain period of time e.g. when it is created prior to the implementation of the respective feature or before a bug-x is available in the SUT. The Expected to fail if... attribute allows marking such Test-cases so they are counted separately and dont inuence the percentage error statistics.
(371) (370)

9.2.3

Migration and backwards compatibility

For backwards compatibility and to ease transition from old-style Test nodes to Test-set and Test-case nodes QF-Test treats nodes as a Test-set or Test-case for documentation and report if their place in the hierarchy allows it. Old test-suites with a structure based on Test nodes can by migrated to make use of the new features of Test-sets and Test-cases. To this end, right-click on a Test node to bring up the context menu. If a transformation is allowed, QF-Test will offer to transform the Test node into a Test-set or Test-case node.
3.0+ Note

It is possible to convert a whole hierarchy of Test nodes to a hierarchy of Test-set and Test-case nodes by selecting the recursive conversion option in the popup menu. Both Test-set and Test-case nodes may contain Setup or Cleanup nodes for backwards compatibility. In a Test-set, these work just as in a Test: Setup and Cleanup are executed for each test contained in the Test-set. In a Test-case however, Setup and Cleanup are only run once at the beginning and end of its execution. If a Test-set or Test-case has both a Dependency and Setup/Cleanup nodes, the Dependency will be executed rst. Setup and (86) Cleanup will have no impact on the dependency stack described in section 12.2 .

9.3. Procedures and Packages

77

9.3

Procedures and Packages

In a way, writing good tests is a little like programming. After mastering the initial steps, tests and source code alike tend to proliferate. Things work ne until some building block that was taken for granted changes. Without a proper structure, programs as well as tests tend to collapse back upon themselves at this point as the effort of adapting them to the new situation is greater than the one needed for recreating them from scratch. The key to avoiding this kind of problem is reuse or avoidance of redundancy. Generating redundancy is one of the main dangers of relying too much on recording alone. To give an example, imagine you are recording various sequences to interact with the components in a dialog. To keep these sequences independent of each other, you start each one by opening the dialog and nish it by closing the dialog again. This is good thinking, but it creates redundancy because multiple copies of the events needed to open and close the dialog are contained in these sequences. Imagine what happens if the SUT changes in a way that invalidates these sequences. Lets say a little conrmation window is suddenly shown before the dialog is actually closed. Now you need to go through the whole suite, locate all of the sequences that close the dialog and change them accommodate the conrmation window. Pure horror. To stress the analogy again, this kind of programming style is called Spaghetti Programming and it leads to the same kind of maintenance problems. These can be avoided by collecting the identical pieces in one place and referring to them wherever they are needed. Then the modications required to adapt to a change like the one described above are restricted to this place only.

Figure 9.2: Packages and Procedures

QF-Test comes with a set of nodes that help to achieve this kind of modularization, (352) (355) (359) namely the Procedure , Procedure call and Package nodes. A Procedure is similar to (307) (353) a Sequence except that its Name attribute is a handle by which a Procedure call node can refer to it. When a Procedure call is executed, the Procedure it refers to is looked up and execution continues there. Once the last child node of the Procedure has nished, the Procedure call has completed as well. Packages are just a way to give even more structure to Procedures. A hierarchy of Pack(361) ages and Procedures, rooted at the special Procedures node, is used to group sets of Procedures with a common context together and to separate them from other Procedures

9.3. Procedures and Packages used in different areas.

78

A Procedure that always does exactly the same, no matter where it is called from, is only marginally useful. To expand on the above example, lets say we want to extend the Procedure that opens the dialog to also set some initial values in some of its elds. Of course we dont want to have these initial values hard-coded in the Procedure node, but want to specify them when we call the Procedure to get different values in different contexts. To that end, parameters can be dened for the Procedure. When the Procedure call is executed, it species the actual values for these parameters during this run. How (52) all of this works is explained in Variables . Also please take a look at the detailed (352) (355) explanation for the Procedure and Procedure call nodes for a better understanding of how these complement each other. A test-suite library with a set of commonly useful Procedures is provided with QF-Test under the name qfs.qft. An entire chapter of the Tutorial is devoted to this library and (145) section 17.1 explains how to include it in your test-suites.

9.3.1
3.1+

Local Procedures and Packages

If you work with several test-suite libraries you might face a situation, where you dene reusable test-steps or sequences, which you only want to use within a dedicated testsuite. If you want to create such local Procedures, you can put a _ as rst sign of the procedures name. This marks a Procedure as test-suite local. A call of a local Procedure can only be inserted within the test-suite, where it is dened. You can use the same concept for local Packages.

3.1+

9.3.2

Relative Procedures

If you call Procedures from other Procedures, it could be convenient not to specify the full procedure name all the time. So called relative procedure calls can only be added to a Package, which has the Border (360) for relative calls (see Border for relative calls ) attribute specied. The structure of that call follows the concept below:
Level Procedures of the same level Procedures one level higher Procedures one level deeper Call .Name of Procedure ..Name of Procedure .Name of Package.Name of Procedure

Table 9.1: Relative procedure calls

9.3. Procedures and Packages

79

As you can see each dot stands for one level. So calling a Procedure two levels higher requires three dots (Current level also requires a dot.)

9.3.3

Inserting Procedure call nodes

As you should organize your tests in separate test steps, which are ideally the same like QF-Tests procedures, QF-Test offers several ways to insert those Procedure call nodes: 1. Via the menu InsertProceduresProcedureCall 2. Via right mouse click and selecting Insert nodeProceduresProcedureCall 3. Copy a Procedure node and insert it at the location of the Procedure call using the normal Copy/Paste actions 4. Via Drag&Drop operation, i.e. dragging the Procedure node to its target node 5. Via the keyboad shortcut Ctrl-A This approach is also valid for inserting Dependency reference nodes, except the keyboard shortcut.

9.3.4
3.1+

Parameterizing nodes

You can create parameters for a Procedure, Dependency or Test-case automatically via the menu OperationsParameterize node . The parameter details dialog allows you to dene for which actions you want to create parameters, e.g. only text-inputs or check nodes.

9.3.5
3.0+

Transforming a Sequence into a Procedure

This transformation is very useful for developing procedures immediately after recording! Under Extras you can convert a recorded Sequence node into a Procedure and move that to the Procedures node.
3.1+

If you transform a Sequence under Test-cases QF-Test automatically creates a Procedure node and inserts a Procedure call to the previous location of the transformed node.

9.4.

Documenting test-suites

80

9.4

Documenting test-suites

Like with any programming-related task it is important for successful test-automation to properly document your efforts. Otherwise there is a good chance (some might say a certainty) that you will lose the overview over what you have done so far and start re-implementing things or miss out tests that should have been automated. Proper documentation will be invaluable when working through a run-log, trying to understand the cause of a failed test. It will also greatly improve the readability of test reports. attributes of Test-set , Test-case , Package and Based on the Comment (352) Procedure nodes, QF-Test can create a set of comprehensive HTML documents that will make all required information readily available. The various kinds of documents (124) and the methods to create them are explained in detail in chapter 15 .
(299) (294) (288) (359)

Chapter 10 Projects
3.5+

Projects provide a better overview, improve navigation between test-suites and expand the scope for search and replace operations. Also, QF-Test automatically manages dependencies resulting from includes or absolute references between test-suites that (145) belong to the same project (see section 17.1 ). Many other features have already been implemented or are under development. Technically a QF-Test project is a set of test-suites located in one or more directories with a common root. There is a 1:1 relation between the project and its directory and the name of the directory automatically becomes the name of the project. To create a new project, select the menu item FileNew project... and choose the directory. QF-Test then creates a le called qftest.qpj in that directory which identies it as a project. All test-suites located below that directory, except those specied in (222) the option Project les and directories to exclude automatically belong to this project. (221) Please see section 29.1.1 for options affecting projects, including the exclusion list. A sub-project is sub-directory of a project which is itself a project. Test-suites within a sub-project also belong to all outer projects containing the sub-project. The project of a test-suite is the innermost sub-project it belongs to. Automatic dependency resolution always covers the whole outermost project of a suite including all sub-projects.

Projects

82

Figure 10.1: The project view

The project view with one or more projects can be turned on or off via the menu item ViewShow projects . The project tree shows the hierarchy of directories and testsuites starting from the project root. Double clicking a test-suite opens it, as does press ing the Return key. You can select several les or directories to be opened in one go, including all test-suites located below the selected directories. The hierarchy is refreshed automatically at intervals dened in the option Project refresh (222) interval (s) . You can refresh a directory including its complete hierarchy at any time by selecting it and pressing F5 . For a more thorough rescan that does not rely on modication times but may take signicantly longer for large projects, press Shift-F5 instead. To switch focus back and forth between the test-suite and the project view, keyboard press F6 . Via Shift-F6 you can navigate to the node representing the current testsuite in the project tree. If necessary, project view and project are automatically shown.

Chapter 11 The standard library

The standard library qfs.qft, a test-suite that is part of the QF-Test distribution, contains many useful procedures for a diverse set of tasks.

Figure 11.1: Standard library qfs.qft

Among others there are procedures for accessing and checking components (AWT,

The standard library

84

Swing, SWT, Web) in a generic manner, le system and database access, logging messages or screenshots to the run-log and report and for performing cleanup. A complete description of all packages and procedures including parameters and return values is provided in the librarys HTML documentation, also accessible from the QFTest Help menu. The latest version is also available online. qfs.qft is included by default in every newly created test-suite. As its directory is on (231) (286) the library path , specifying just qfs.qft in the Include les of the Test-suite node is sufcient.
Note

All procedures referring to an SUT use the generic variable $(client) as an implicit parameter. You must make sure that this variable is set correctly either globally or locally or specied as an explicit parameter in the procedure call.

Chapter 12 Dependencies
Dependencies are a very advanced feature, albeit a little complex. You should have a reasonably good grasp of QF-Test, especially for things like control ow, variable binding and error handling, before you start using them. However, when properly implemented, Dependencies will feel almost like magic when you run several non-related Test-cases and all setup and cleanup is handled automatically. Dependencies are also crucial for running (695) tests in a QF-Test Daemon as described in chapter 40 . To illustrate the following sections, two test-suites doc/tutorial/dependencies.qft and demo/carconfig/carconfig_en.qft are provided. The former illustrates the various concepts for Dependencies and the latter is a simple real-world example. The SWT demo test-suite swt_addressbook.qft and the data driver demo datadriver.qft also make use of Dependencies. Single-stepping through these suites in the debugger, looking at the variable bindings and examining the run-logs should help you to familiarize yourself with this feature.
(317)

12.1 Dependency basics

Conceptually a Dependency describes a set of preconditions that have to be met before a test can be executed. These preconditions are often dependent on each other, for example parameter settings may have to be read which are required for starting the SUT. The target Window for a specic test-case cannot be opened before the SUT is running, and so on. Dependencies can be dened in two places: Generic Dependencies that will often be reused or serve as the basis for other Dependencies can be implemented just like a Pro(361) (359) node, for example inside a Package node cedure and placed below the Procedures called Dependencies. Their fully qualied name is built analogous to a Procedure name and Dependencies can be referenced using a Dependency reference almost exactly like a

12.2.

The dependency stack

(355)

86

Procedure call

Alternatively, Dependencies can be implemented at the beginning of a Test-suite, Test-set or Test-case node. In addition to having their own Dependency, Test-cases and Test-sets can inherit the Dependency from their parent node. Each Dependency should always take care of only one specic precondition and refer to other Dependencies to ensure that the more basic preconditions on which it depends are taken care of. This is done implicitely by inheriting the Dependency of a parent node or explicitely with a Dependency reference. The actual implementation is done in Setup and Cleanup nodes inside the Dependency. The Setup node should always be implemented in a way that ensures the minimum of effort to fulll the requirement. For example, the Setup to start the SUT should rst check whether the SUT is already connected or not and execute the SUT startup sequence only in the latter case. The reason for this will be explained below. Setup and Cleanup nodes should always be prepared to handle errors that do not affect the outcome. For example, the Cleanup node that terminates the SUT should terminate cleanly if the SUT is already disconnected.

12.2

The dependency stack

When a Dependency is executed, QF-Test linearizes the Dependency and its ancestors to create a dependency stack. For example, if Dependency D depends on Dependencies B and C which in turn depend on Dependency A, the stack will look like [A,B,C,D]. If this is the rst Dependency to be executed, there will be no previous dependency stack and QF-Test will simply execute the Setup nodes of the Dependencies from most basic to most specic, i.e. A-B-C-D.

12.2.

The dependency stack

87

Figure 12.1: Dependency stack A-B-C-D

After the Test-case that required the Dependency has been executed, part of the dependency stack may be rolled back. If any of the Dependencies on the stack has the Forced cleanup attribute set, it and all of the following Dependencies will have their Cleanup sequences executed, this time in the opposite order. If, in the example above, C has Forced cleanup set, rst Ds Cleanup node will be run, then Cs Cleanup node. After that, the remaining dependency stack will look like [A,B].

12.2.

The dependency stack

88

Figure 12.2: Dependency C has forced cleanup

Now lets say another Test-case is executed that depends on a Dependency E which in turn depends on A, so the new target dependency stack is [A,E]. Now QF-Test compares the current dependency stack [A,B] to the new target stack [A,E] and nds the rst point of difference, which is B vs. E in this case. Next the old dependency stack will be unrolled to this point from right to left, so Bs Cleanup node is executed. The current stack now looks like [A]. Then dependency E is added which results in [A,E].

12.2.

The dependency stack

89

Figure 12.3: Rollback and build up stack to A-E

The next point is crucial: To get from the cleaned up old dependency stack [A] to the new target stack [A,E], QF-Test doesnt simply execute the Setups of the rest of the new dependency stack - E in this case. Instead, the Setup nodes of the whole new target stack are executed, here rst As Setup node and then Es. This repeated execution is the main reason why Setup nodes should always be implemented to do as little as possible. Why such a weird asymmetry? Lets assume that A is the Dependency responsible for bringing up the SUT. In the normal case for the previous example, the SUT will already be running and if As Setup is properly implemented it will notice that and there will be very little overhead. However, QF-Test has no control over what has happened since the last time As setup was run. You might have closed the SUT manually or it might have been closed as a side effect of the previous test or even crashed. In this case, if As setup were not run again, Es precondition could never be satised and the whole Test-case would fail along with all subsequent tests that also rely directly or indirectly on Dependency A. Thus, always executing all Setup nodes on the target dependency stack is the only way to ensure proper setup in all circumstances. If get stuck or lost you can clear the dependency stack manually in two ways: By selecting the RunRollback dependencies or the RunReset dependencies menu item. The former will completely roll back the dependency stack, executing the Cleanup nodes of all Dependencies that were left on the stack, while the latter will simply clear the stack without executing anything.

12.3.

Error escalation

90

12.3

Error escalation

Another thing that is just grand about Dependencies is the convenient way that errors can be escalated without any additional effort. Lets again consider the example from the previous section after the rst dependency stack has been initialized to [A,B,C,D] and the Setups have been run. Now what happens if the SUT has a really bad fault, like going into a deadlock and not reacting to user input any longer? When a Cleanup node fails during rollback of the dependency stack, QF-Test will roll back an additional Dependency and another one if that fails again and so on until the stack has been cleared. Similarly, if one of the Setups fails, an additional Dependency is rolled back and the execution of the Setups started from scratch. In the example above, the deadlocked client would cause failures until the rollback of the SUT Dependency terminates the SUT and causes a new one to be started in the next round.

Figure 12.4: Exception in forced cleanup sequence of C causes B to clean up

For this to work it is very important to write Cleanup sequences in a way that ensures that either the desired state is reached or that an exception is thrown and that there is a more basic dependency with a more encompassing Cleanup. For example, if the Cleanup node for the SUT Dependency just tries to cleanly shut down the SUT through its File->Exit menu without exception handling and further safeguards, an exception in that sequence will prevent the SUT from being terminated and possibly interfere with all subsequent tests. Instead, the shutdown should be wrapped in a Try/Catch with a Finally node that checks that the SUT is really dead and if not, kills the process as a last resort. With good error handling in place, Test-cases will rarely interfere with each other even in case of really bad errors. This helps avoid losing a whole nights worth of test-runs just because of a single error.

12.4.

Special variables

91

12.4

Special variables

A Dependency can in turn depend on the values of certain variables. For example, you may want to test your SUT with different JDK versions. Thus, the JDK would be a characteristic variable for a Dependency that brings up the SUT. If you have a current dependency stack [A] and the target stack is also [A], but last time A was executed with one JDK value and now some other JDK is requested, QF-Test must roll back the Dependency to terminate the current SUT and ensure that a new one with the proper JDK is started.

Figure 12.5: Change in characteristic variable causes cleanup of A

12.5.

Error handling

92

All this is handled fully automatically if you add the JDK variable to the list of Characteristic variables of the Dependency. The values of the Characteristic variables are always taken into account when comparing dependency stacks and two Dependencies on the stack are only considered identical if the values of all Characteristic variables from the previous and the current run are equivalent. Consequently it is also possible for a Dependency to directly or indirectly refer to the same base Dependency with different values for its Characteristic variables. In that case the base Dependency will appear multiple times in the linearized dependency stack. Furthermore, QF-Test stores the values of the Characteristic variables during execution of the Setup of a Dependency. When the Dependency is rolled back, i.e. its Cleanup node is executed, QF-Test will ensure that these variables are bound to the same value as during execution of the Setup. This ensures that a completely unrelated Test-case with conicting variable denitions can be executed without interfering with the execution of the Cleanup nodes during Dependency rollback. Consider for example the commonly used client variable for the name of an SUT client. If a set of tests for one SUT has been run and the next test will need a different SUT with a different name, the client variable will be changed. However, the Cleanup node for the previous SUT must still refer to the old value of client, otherwise it wouldnt be able to terminate the SUT client. This is taken care of automatically as long as client was added to the list of Characteristic variables.

12.5

Error handling

Besides supporting automatic escalation of errors a Dependency can also act as an error or exception handler for the tests that depend on it. Catch nodes, which can be placed at the end of a Dependency, are used to catch and handle exceptions thrown during a test. Exceptions thus caught will still be reported as exceptions in the run-log and the report, but they will not interfere with subsequent tests or even abort the whole test-run. (306) (303) This is very similar to the Implicitly catch exceptions attribute of Test nodes, but more specic. An Error handler node is another special node that may be added to a Dependency after the Cleanup and before the Catch nodes. It will be executed whenever the result of a Test-case is error. In case of an exception, the Error handler node is not executed automatically because that might only cause more problems and even interfere with the exception handling, depending on the kind of exception. To do similar things for errors and exception, implement the actual handler as a Procedure and call it from the Error handler and the Catch node. Error handlers are useful for capturing and saving miscellaneous states that are not automatically provided by QF-Test. For example, you may want to create copies of temporary les created during execution of your SUT that may hold information pertaining to the error.

12.5.

Error handling

93

Figure 12.6: Execution of Catch and Error handler nodes

Only the topmost Error handler that is found on the dependency stack is executed, i.e. if in a dependency stack of [A,B,C,D] both A and C have Error handlers, only Cs Error handler is run. Otherwise it would be difcult to modify the error handling of the more basic Dependency A in the more specialized Dependency C. To reuse As error handling code in C, implement it as a Procedure.

Chapter 13 Scripting
One of QF-Tests benets is that complex tests can be created without writing a single line of code. However, there are limits to what can be achieved with a GUI alone. When testing a program which writes to a database, for example, one might want to verify that the actual values written to the database are correct; or one might want to read values from a database or a le and use these to drive a test. All this and more is possible with the help of powerful scripting languages like Jython and Groovy.
3.0+

While Jython is supported since the beginning of QF-Test, Groovy has found its way into QF-Test a bit later (QF-Test version 3). This language might be more convenient than Jython for those who are familiar with Java. Its mainly a matter of individual preference whether to utilize Jython or Groovy scripting inside QF-Test. In this chapter the basics of the scripting integration are explained in detail with Jython. Most of that also holds true for Groovy, especially QF-Tests run-context methods are the same for both languages. The peculiarities of Groovy will be explained in the section (110) Scripting with Groovy .

3.0+

The scripting language to use for a given Server script or SUT script node is deter(391) mined by its Script language attribute, so you can mix both languages inside a testsuite. The default language to use for newly created script nodes can be set via the (220) option Default script language for script nodes .

(390)

(392)

13.1

Fundamentals of the Jython integration

Python is an excellent, object oriented scripting language written in C by Guido van Rossum. A wealth of information including an excellent Python tutorial is available at http://www.python.org. Python is a standard language that has been around for years with extensive freely accessible documentation. Therefore this manual only explains how Jython is integrated into QF-Test, not the language itself. Python is a very natural

13.1.

Fundamentals of the Jython integration

95

language. Its greatest strength is the readability of Python scripts, so you should have no problems following the examples. Jython (formerly called JPython) is a Java implementation of the language Python. It has the same syntax as Python and almost the same set of features. The object systems of Java and Jython are very similar and Jython can be integrated seamlessly into applications like QF-Test. This makes it an invaluable tool for Java scripting. Jython has its own web page at http://www.jython.org. There is also an extensive tutorial available which may help you getting started with this scripting language. QF-Test uses Jython version 2.5 which supports a large majority of the standard Python library. The approach to scripting in QF-Test is inverse from that of other GUI test tools. Instead of driving the whole test from a script, QF-Test embeds scripts into its own test-suite. (390) (392) This is achieved with the two nodes Server script and SUT script . Both nodes have a Script
(390)

attribute for the actual code.

Figure 13.1: Detail view of a Server script with help window for rc methods
3.0+

The internal script editor has some useful features to ease the typing of code. Reserved key words, built-in functions, standard types, literals and comments are highlighted. Indentation is handled automatically inside of code blocks. With TAB and Shift-TAB respectively several selected lines can be indented manually.

13.1.

Fundamentals of the Jython integration

96

However, the probably most useful feature - at least for the QF-Test newbie - might be the input assistance for many built-in methods. Type, rc. and maybe for example, Ctrl-Space to make a popup winsome initial letters of a method name. Then press dow displaying the appropriate methods and descriptions of QF-Tests run-context (cf. (586) chapter 37 ). Select one of the methods and conrm with Return to insert it into the script code. To get a list of all objects equipped with help, just press Ctrl-Space with the mouse cursor positioned after whitespace. Server scripts are useful for tasks like calculating the values of variables or reading and parsing data from a le and using that to drive a test. SUT scripts on the other hand give full access to the components of the SUT and to every Java API that the SUT exposes. An SUT script might be used to retrieve or check values in the SUT to which QF-Test (393) doesnt have access. The SUT script node additionally requires a Client attribute for the name of the SUT client to run in. Server scripts are run in a Jython interpreter embedded in QF-Test itself, while SUT scripts are run in a Jython interpreter embedded in the SUT. These interpreters are independent of each other and do not share any state. However, QF-Test uses the RMI connection between itself and the SUT for seamless integration of SUT scripts into the execution of a test. Through the menu ExtrasJython terminal... you can open a window with an interactive command prompt for the Jython interpreter embedded into QF-Test. You can use this terminal to experiment with Jython scripts, get a feel for the language, but also to try stuff like setting up database connections. The keystrokes sophisticated out some Ctrl-Up and Ctrl-Down let you cycle through previous input and you can also edit any other line or mark a region in the terminal and simply press Return to send it to the Jython interpreter. In that case QF-Test will lter the > and ... prompts from previous interpreter output. Similar Jython terminals are available for each SUT client. The respective menu items are located below the Clients menu.
Note

When working in a Jython terminal, theres one thing you need to be aware of: The commands issued to the interpreter are not executed on the event dispatch thread. This may not mean anything to you and most of the time it doesnt cause any problems, but it may deadlock your application if you access any Swing or SWT components or invoke their methods. To avoid that, QF-Test provides the global method runAWT (and runSWT respectively) that executes arbitrary code on the event dispatch thread. For example, to get the number of visible nodes in a JTree component named tree, use runAWT(tree.getRowCount()) (or runAWT { tree.getRowCount() } in Groovy) to be on the safe side.

13.2.

The run-context rc

97

13.2

The run-context rc

When executing Server scripts and SUT scripts, QF-Test provides a special environment in which a local variable named rc is bound. This variable represents the run-context which encapsulates the current state of the execution of the test. It provides an interface (589) (fully documented in section 37.6 ) for accessing QF-Tests variables, for calling QFTest Procedures and can be used to add messages to the run-log. To SUT scripts it also provides access to the actual Java components of the SUTs GUI. For those cases where no run-context is available, i.e. Resolvers, TestRunListeners, code executing in a background thread etc. QF-Test also provides a module called qf (607) with useful generic methods for logging and other things. Please see section 37.7 for details.

13.3

Scripting examples

Probably the best way to learn about Jython and QF-Test is through examples, so were going to provide a few here. Full technical background and a comprehensive API refer(586) ence are available in chapter 37 . Working examples are also doc/tutorial/demo-script.qft. provided in the test-suite

13.3.1

Logging messages

One thing the run-context can be used for is to add arbitrary messages to the run-log that QF-Test generates for each test-run. These messages may also be agged as warnings or errors.
rc.logMessage("This is a plain message") rc.logWarning("This is a warning") rc.logError("This is an error")

Example 13.1: Logging messages from scripts

When working with compact run-logs (which is strongly encouraged, see the option (282) Create compact run-log ), plain messages may be removed from the run-log to preserve memory. When an error happens, the most recent 100 or so nodes in the run-log are kept even in a compact run-log, so in general this is not a problem. If you really need to make sure that a message will denitely be kept in the run-log you can enforce this by specifying the optional second parameter dontcompactify, e.g.

13.3.

Scripting examples

98

rc.logMessage("This message will not be removed", dontcompactify=true) # or simply rc.logMessage("This message will not be removed", 1)

Example 13.2: Logging messages that will not get removed in compact run-logs

Note

Only the logMessage method has that extra parameter. Warnings and errors are never removed from the run-log, so it doesnt apply to logWarning and logError.

13.3.2

Performing checks

Most of the time logging messages is tied to evaluating some condition. In that case, it is often desirable to get a result in the HTML or XML report equivalent to that of a Check node. The methods rc.check and rc.checkEqual will do just that:
var = 0 rc.check(var == 0, "!Value of var is 0") rc.checkEqual(${system:user.language}, en, "English locale required", rc.EXCEPTION)

Example 13.3: Performing checks

For the old-style report the message is treated like a Check if it starts with an !. The optional last argument changes the error level in case of failure.

13.3.3

Accessing variables

Using QF-Test variables in Jython scripts is not difcult, but there are two viable ways to do so and it is important to understand the difference and which method to apply in which case. First, standard QF-Test variable expansion takes place before the script is parsed and executed, so you can use $(var) or ${group:name} style variables. This is very handy if you know that the variable values are either numbers or Boolean values, because Jython will recognize these without quoting:
if ${qftest:batch}: rc.logMessage("We are running in batch mode") else: rc.logMessage("We are running in interactive mode")

Example 13.4: QF-Test variable expansion

13.3.

Scripting examples

99

The example above will work ne, because ${qftest:batch} will expand to either true or false. Though standard Jython does not recognize these as Boolean values, the special environment QF-Test provides for the scripts makes this work. The following (362) example also works well, provided $(i) is a numeric value, for example a Loop index.
# log some value rc.logMessage("data[$(i)]:" + data[$(i)])

Example 13.5: More variable expansion

It gets a bit more complicated when using QF-Test variables with arbitrary string values. Strings need to be quoted for Jython, using either single quotes or double quotes .
rc.logMessage("$(someText)")

Example 13.6: Simple text expansion

The code above will work very well unless $(someText) expands to a value that contains line-breaks or double quote characters. In that case, the script is not valid Jython (532) code and a ScriptException is thrown. To avoid that kind of problem you should make it a habit to use the run-contexts lookup (589) for API reference) instead of $(...) or ${...:...} method (see section 37.6 whenever you want to access a QF-Test value as a string. That way youll never have to worry about quoting.
# access a simple variable rc.logMessage(rc.lookup("someText")) # access a property or resource rc.logMessage(rc.lookup("qftest", "version"))

Example 13.7: Using rc.lookup to access string variables

If you want to combine multiple variables in one string, it is easier to use rc.expand instead of rc.lookup. Note that you must escape the $ characters by doubling them (583) to prevent QF-Test from expanding the values itself (see section 36.6 ).
rc.logMessage("The resource is" + rc.expand("$${$$(group):$$(name)}"))

Example 13.8: Using rc.expand for complex variable access

Note

Let us again stress the difference between the $ and the rc.lookup methods for

13.3.

Scripting examples

100

accessing variables: $ expressions are expanded before the script is passed to the Jython interpreter. That means the text $(var) in the script is replaced by a verbatim copy of the value of the variable var. The method rc.lookup however returns the value of var during the processing of the script and, as explained above, is recommended for accessing string values.

13.3.4

Setting variables

To make the results of a Jython script available during further test execution, values can be stored in global or local variables. The effect is identical to that of a (481) Set variable node. The corresponding methods in the run-context are rc.setGlobal and rc.setLocal.
# Test if the file /tmp/somefile exists from java.io import File rc.setGlobal("fileExists", File("/tmp/somefile").exists())

Example 13.9: Using rc.setGlobal

After executing the above example $(fileExists) will expand to 1 if the le /tmp/somefile exists and to 0 if it doesnt. To clear a variable, set it to None, to clear all global variables use rc.clearGlobals() from a Server script.

13.3.5

Global Jython variables

Sometimes it is helpful to have a Jython variable available in several scripting nodes. If the value of the variable is not a simple string or integer, it is normally not sufcient to use rc.setGlobal(...) to store it in a global QF-Test variable because the value will be converted to a string in the process. Instead, such a variable should be declared global as shown in the following example.
global globalVar globalVar = 10000

Example 13.10: Global Jython variable

The globalVar is now accessible within all further scripting nodes of the same type (Server scripts or SUT scripts of the same client). For changing the value of globalVar within another script, the global declaration is necessary again. Otherwise a new local

13.3.

Scripting examples

101

variable is created instead of accessing the existing global. Use the del statement to remove a global Jython variable:
global globalVar del globalVar

Example 13.11: Delete a global Jython variable

13.3.6

Exchanging Jython variables between interpreters

Sometimes one would like to use variable values that have been dened in one Jython interpreter in a different interpreter. For example, an SUT script might have been used to create a list of items displayed in a table. Later we want to iterate over that list in a Server script. To simplify such tasks, the run-context provides a symmetrical set of methods to access or set global variables in a different interpreter. For SUT scripts these methods are named toServer and fromServer. The corresponding Server script methods are toSUT and fromSUT. The following example illustrates how an SUT script can set a global variable in QF-Tests interpreter:
cellValues = [] table = rc.lookup("idOfTable") for i in range(table.getRowCount()): cellValues.append(table.getValueAt(i, 0)) rc.toServer(tableCells=cellValues)

Example 13.12: Setting a server variable from an SUT script

After the above script is run, the global variable named tableCells in QF-Tests interpreter will hold the array of cell values.
Note

The cell values in the above example are not necessarily strings. They could be numbers, date values, anything. Unfortunately Jythons pickle mechanism isnt smart enough to transport instances of Java classes (not even serializable ones), so the whole exchange mechanism is limited to primitive types like strings and numbers, along with Jython objects and structures like arrays and dictionaries.

13.3.7

Accessing the SUTs GUI components

For SUT scripts the run-context provides an additional method that is extremely useful. Calling rc.getComponent(componentId) will retrieve the information of the

13.3.

Scripting examples
(514) (515)

102

Component node in the test-suite with the Id componentId and pass that to QFTests component recognition mechanism. The whole process is basically the same as when simulating an event, including the possible exceptions if the component cannot be found. If the component is located, it will be passed to Jython, not as some abstract data but as the actual Java object. All methods exposed by the Java API for the components class can now be invoked to retrieve information or achieve effects which are not possible (40) through the GUI alone. To get a list of a components method see section 5.5 .
# get the custom password field field = rc.getComponent("tfPassword") # read its crypted value passwd = field.getCryptedText() rc.setGlobal("passwd", passwd) # get the table component table = rc.getComponent("tabAddresses") # get the number of rows rows = table.getRowCount() rc.setGlobal("tableRows", rows)

Example 13.13: Accessing components with rc.getComponent

You can also access sub-items this way. If the componentId parameter references an item, the result of the getComponent call is a pair, the component and the items index. The index can be used to retrieve the actual value. The following example shows how to get the value of a table cell. Note the convenient way Jython supports sequence unpacking during assignment.
# first get the table and index table, (row,column) = rc.getComponent("tableAddresses@Name@Greg") # then get the value of the table cell cell = table.getValueAt(row, column)

Example 13.14: Accessing sub-items with rc.getComponent

13.3.8

Calling Procedures
(352)

The run-context can also be used to call back into QF-Test and execute a Procedure node. Jython is perfect for reading and parsing data from a database or from a le, so this feature can be used to run data-driven tests.

Parameters are passed from a Jython script to a QF-Test Procedure in a Jython dictionary. The keys and values of the dictionary can be any kind of Jython object. They are converted to strings before they are passed to QF-Test.

13.3.

Scripting examples

103

rc.callProcedure("text.clearField", {"component": "nameField"})

Example 13.15: Simple procedure call

In the example above the Procedure named clearField in the Package named text will be called. The single parameter for the call named component is set to the value nameField. The value returned by the Procedure through a Return of the rc.callProcedure call.
Note
(358)

(359)

node is returned as the result

Great care must be taken when using rc.callProcedure(...) in SUT script nodes. Only short-running Procedures should be called that wont trigger overly complex ac(527) tions in the SUT. Otherwise a DeadlockTimeoutException might be caused. For data-driven tests where for some reason the data must be determined in the SUT, use rc.toServer(...) to transfer the values to QF-Tests interpreter, then drive the test from a Server script node where these restrictions do not apply.

13.3.9
3.1+

Setting options
(217)

can also be set at runtime via Many of the options described in chapter 29 rc.setOption. Constants for option names are predened in the class Options which is automatically available for Jython and Groovy scripts. A real-life example where this might be useful is if you want to replay an event on a disabled component, so you need to temporarily disable QF-Tests check for the enabled/disabled state:
rc.setOption(Options.OPT_PLAY_THROW_DISABLED_EXCEPTION, false)

Example 13.16: Jython/Groovy example for setOption

After replaying this special event, the original value read from the conguration le or set in the option dialog can be restored by unsetting the option as the following example shows:
rc.unsetOption(Options.OPT_PLAY_THROW_DISABLED_EXCEPTION)

Example 13.17: Jython/Groovy example for unsetOption

Note

Be sure to set QF-Test options in a Server script node and SUT options in an SUT script node, otherwise the setting will have no effect. The option documentation in chapter (217) 29 shows which one to use.

13.3.

Scripting examples

104

13.3.10

A complex example

We are going to close this section with a complex example, combining features from Jython and QF-Test to execute a data-driven test. For the example we assume that a simple table with the three columns Name, Age and Address should be lled with values read from a le. The le is assumed to be in comma-separated-values format with | as the separator character, one line per table-row, e.g.: John Smith|45|Some street, some town Julia Black|35|Another street, same town To verify the SUTs functionality in creating new table rows, a QF-Test Procedure should be created that takes three parameters name, age, and address, creates a new table-row and lls it with these values. Then we can use Jython to read and parse the data from the le, iterate over the data-sets and call back to QF-Test for each table-row to be created. The name of the le to read is passed in a QF-Test variable named lename. When we have nished lling the table, we compare the state of the actual table component with the data read from the le to make sure everything is OK.

13.3.

Scripting examples

105

import string data = [] # read the data from the file fd = open(rc.lookup("filename"), "r") line = fd.readline() while line: # remove whitespace line = string.strip(line) # split the line into separate fields # and add them to the data array if len(line) > 0: data.append(string.split(line, "|")) line = fd.readline() # now iterate over the rows for row in data: # call a qftest procedure to create # one new table row rc.callProcedure("table.createRow", {name: row[0], age: row[1], address: row[2]}) # verify that the table-rows have been filled correctly table = rc.getComponent("tabAddresses") # check the number of rows rc.check(table.getRowCount() == len(data), "Row count") if table.getRowCount() == len(data): # check each row for i in range(len(data)): rc.check(table.getValueAt(i, 0)) == data[i][0], "Name in row " + i) rc.check(table.getValueAt(i, 1)) == data[i][1], "Age in row " + i) rc.check(table.getValueAt(i, 2)) == data[i][2], "Address in row " + i)

Example 13.18: Executing a data-driven test

Of course, the example above serves only as illustration. It is too complex to be edited comfortably in QF-Test and too much is hard-coded, so it is not easily reusable. For real use, the code to read and parse the le should be parameterized and moved to a module, as should the code that veries the table. This topic is covered in the following section.

13.3.11

Override components

You might face a situation where you want to work with a component, which you have to search before working with it. Sometimes recording all required components can be exhaustive or might be to complicated. For such cases you can use the method

13.4.

Modules

106

rc.overrideElement to set the found component (either by generic components or via scripting) to a QF-Test component. Now you can work with the assigned component and use all available QF-Test nodes. Lets imagine that we have a panel and we want to work with the rst texteld, but because of changing textelds we cannot rely on the standard way of the recognition. Now we can implement a script, which looks for the rst texteld and assigns that texteld to the PriorityAwtSwingComponent from the standard library qfs.qft. Once we have executed that script we can work with any QF-Test nodes using the PriorityAwtSwingComponent, which actually performs all actions on the found texteld.
from de.qfs.apps.qftest.extensions import ResolverRegistry panel = rc.getComponent("myPanel") for component in panel.getComponents(): if ResolverRegistry.instance().isInstance(component, \ "javax.swing.JTextField"): rc.overrideElement("PriorityAwtSwingComponent", component) break

Example 13.19: Using rc.overrideElement

This concept is very useful if you know an algorithm to determine the target component of your test-steps. You can nd such priority-components for all engines in the standard library qfs.qft. You can also nd an illustrative example in the provided demo test-suite carconfig_en.qft, located in the directory demo/carconfig in your QF-Test installation.

13.4

Modules

Modules for Jython in QF-Test are just like standard Python modules. You can import these modules into QF-Test scripts and call their methods, which simplies the development of complex scripts and increases maintainability since modules are available across test-suites. Modules intended to be shared between test-suites should be placed in the directory jython under QF-Tests root directory. Modules written specically for one testsuite can also be placed in the test-suites directory. The version-specic directory qftest-3.5.3/jython/Lib is reserved for modules provided by Quality First Software GmbH. Jython modules must have the le extension .py.

13.4.

Modules
(105)

107

To improve example 13.18 you could write a module csvtable.py with methods loadTable to read the data from the le and verifyTable to verify the results. An example module is provided in qftest-3.5.3/doc/tutorial/csvtable.py. Following is a simplied version:
import string def loadTable(file, separator=|): data = [] fd = open(file, "r") line = fd.readline() while line: line = string.strip(line) if len(line) > 0: data.append(string.split(line,separator)) line = fd.readline() return data def verifyTable(rc, table, data): ret = 1 # check the number of rows if table.getRowCount() != len(data): if rc: rc.logError("Row count mismatch") return 0 # check each row for i in range(len(data)): row = data[i] # check the number of columns if table.getModel().getColumnCount() != len(row): if rc: rc.logError("Column count mismatch " + "in row " + i) ret = 0 else: # check each cell for j in range(len(row)): val = table.getModel().getValueAt(i, j) if str(val) != row[j]: if rc: rc.logError("Mismatch in row " + i + " column " + j) ret = 0 return ret

Example 13.20: Writing a module

The code above should look familiar. It is an improved version of parts of example (105) 13.18 . With that module in place, the code that has to be written in QF-Test is reduced to:

13.5.

Post-mortem debugging of Jython scripts

108

import csvtable # load the data data = csvtable.loadTable(rc.lookup("filename")) # now iterate over the rows for row in data: # call a qftest procedure to create # one new table row rc.callProcedure("table.createRow", {name: row[0], age: row[1], address: row[2]}) # verify that the table-rows have been filled correctly table = rc.getComponent("tabAddresses") csvtable.verifyTable(rc, table, data)

Example 13.21: Calling methods in a module

For more complex import of data QF-Test can be extended with existing Python modules. For example, at http://python-dsv.sourceforge.net/ an excellent module for very exible CSV import is freely available.

13.5

Post-mortem debugging of Jython scripts

Python comes with a simple line-oriented debugger called pdb. Among its useful features is the ability for post-mortem debugging, i.e. analyzing why a script failed with an exception. In Python you can simply import the pdb package and run pdb.pm() after an exception. This will put you in a debugger environment where you can examine the variable bindings in effect at the time of failure and also navigate up to the call stack to examine the variables there. It is somewhat similar to analyzing a core dump of a C application. Though Jython comes with pdb, the debugger doesnt work very well inside QF-Test for various reasons. But at least post-mortem debugging of Jython scripts is supported from (94) (390) node fails, open QFthe Jython terminals (see section 13.1 ). After a Server script (392) Tests Jython terminal, for a failed SUT script node open the respective SUT Jython terminal, then just execute debug(). This should have a similar effect as pdb.pm() described above. For further information about the Python debugger please see the documentation for pdb in Python version 2.5 at http://www.python.org/doc/2.5/lib/modulepdb.html.

13.6.

Differences between Jython version 2.2.1 and 2.5

109

13.6

Differences between Jython version 2.2.1 and 2.5

Jython version 2.5 was a major rewrite of the Java version of Python. Though most of the changes are backwards compatible, there are some subtle differences resulting from changes in the Java integration as well as the Python language itself.

13.6.1

Boolean type

Jython now has a real boolean type with values True and False whereas in older versions integer values 0 and 1 served as boolean values. This can cause problems if boolean results from calls like file.exists() are assigned to a QF-Test variable, e.g. (371) leExists and later checked in a Condition attribute in the form $(fileExists) == 1. Such conditions generally be written as just $(fileExists) which works well with all Jython versions.

13.6.2

Character encodings

All Java strings are sequences of 16-bit characters. Pythons original strings are made of 8-bit characters. Later, unicode strings with 16-bit characters were added. Jython literal strings like abc are 8-bit, prepending u for uabc turns them into unicode strings. In Jython 2.2, Java strings were converted to 8-bit Python strings based on the default encoding of the Java VM, typically ISO-8859-1 (also known as latin-1) in western countries. In Jython 2.5, every Java string is now interpreted as a unicode Jython string. This results in a lot more implicit conversion between 8-bit and unicode strings, for example when concatenating a Java string - now converted to unicode - and a literal string like rc.lookup(path) + /file. Most of the time this works well, but if the literal string contains characters outside the 7-bit ASCII character-set, things start to get messy. The default encoding for 8-bit Jython characters can be specied in the option (220) Default character encoding for Jython with a default of latin-1 for maximum backwards compatibility. On the upside it is now possible to have default encodings other than latin-1 and to specify literal strings of characters in international character sets. One thing to watch out for is existing code of the form import types if type(somevar) == types.StringType: ... The type types.StringType is the 8-bit string. It does not match unicode strings. To

13.6.

Differences between Jython version 2.2.1 and 2.5

110

test whether some variable is a Jython string, regardless of whether its 8-bit or unicode, change that to import types if type(somevar) in types.StringTypes: ... One new requirement - coming from newer Python versions - is that Python module les containing characters outside the 7-bit ASCII character must specify the character encoding to be used in a comment line close to the top of the le, e.g. # coding: latin-1

Please see http://www.python.org/peps/pep-0263.html for details.

13.6.3

Getting the name of a Java class

This simple operation is surprisingly difcult in Jython. Given a Java object you would expect to simply write obj.getClass().getName(). For some objects this works ne, for others it fails with a cryptic message. This can be very frustrating. Things go wrong whenever there is another getName method dened by the class, which is the case for AWT Component, so getting the class name this way fails for all AWT/Swing component classes. In Jython 2.2.1 the accepted workaround was to use the Python idiom obj.__class__.__name__. This no longer works in Jython 2.5 because it no longer returns the fully qualied class name, only the last part. Instead of java.lang.String you now get just String. The only solution that reliably works for version 2.5 is: from java.lang import Class Class.getName(obj.getClass()) This also works for 2.2, but it is not nice, so we initiated a new convenience module with utility methods called qf that gets imported automatically. As a result you can now simply write qf.getClassName(obj).

13.7.

Scripting with Groovy

111

13.7

Scripting with Groovy

Groovy is another established scripting language for the Java Platform. It was invented by James Strachan and Bob McWhirter in 2003. All you need for doing Groovy is a Java Runtime Environment (JRE) and the groovy-all.jar le. This library contains a compiler to create Java class les and provides the runtime when using that classes in the Java Virtual Machine (JVM). You may think of Groovy as being Java with an additional .jar le. In contrast to Java, Groovy is a dynamic language, meaning that the behaviour of an object is determined at runtime. Groovy also allows to load classes from sources without creating class les. Finally, it is easy to embed Groovy scripts into Java applications like QF-Test. The Groovy syntax is similar to Java, maybe more expressive and easier to read. When coming from Java you can embrace the Groovy style step by step. Of course we cannot explain all aspects of the Groovy language here. For in-depth information, please take a look at the Groovy home page at http://groovy.codehaus.org or read the excellent book Groovy in Action by Dierk Koenig and others. Perhaps the following tips may help a Java programmer getting started with Groovy. The semicolon is optional as long as a line contains only one statement. Parentheses are sometimes optional, e. g. println hello qfs means the same as println(hello qfs). Use for (i in 0..<len) { ... len; i++) { ... }. } instead of for (int i = 0; i <

The following imports are made by default: java.lang.*, java.util.*, java.io.*, java.net.*, groovy.lang.*, groovy.util.*, java.math.BigInteger, java.math.BigDecimal. Everything is an object, even integers like 1 or booleans like true. Instead of using getter and setter methods like obj.getXxx(), you can simply write obj.xxx to access a property. The operator == checks for equality, not identity, so you can write if (somevar == somestring) instead of if (somevar.equals(somestring)). The method is() checks for identity. Variables have a dynamic type when being dened with the def keyword. Using def x = 1 allows for example to assign a String value to the variable x later in the script.

13.7.

Scripting with Groovy

112

Arrays are dened differently from Java, e. g. int[] a = [1, 2, 3] or def a = [1, 2, 3] as int[]. With def a = [1, 2, 3] you dene a List in Groovy. Groovy extends the Java library by dening a set of extra methods for many classes. Thus you can for example apply an isInteger() method to any String object in a Groovy script. Thats what is called GDK (according to the JDK in Java). To get a list of those methods for an arbitrary object obj, you can simply invoke obj.class.metaClass.metaMethods.name or use the following example:
import groovy.inspect.Inspector def s = abc def inspector = new Inspector(s) def mm = inspector.getMetaMethods().toList().sort() { it[Inspector.MEMBER_NAME_IDX] } for (m in mm) { println(m[Inspector.MEMBER_TYPE_IDX] + + m[Inspector.MEMBER_NAME_IDX] + ( + m[Inspector.MEMBER_PARAMS_IDX] + )) }

Example 13.22: GDK methods for a String object

Inner classes are not supported, in most cases you can use Closures instead. A Closure is an object which represents a piece of code. It can take parameters and return a value. Like a block, a Closure is dened with curly braces { ... }. Blocks only exists in context with a class, an interface, static or object initializers, method bodies, if, else, synchronized, for, while, switch, try, catch, and finally. Every other occurrence of {...} is a Closure. As an example lets take a look at the eachFileMatch GDK method of the File class. It takes two parameters, a lter (e. g. a Pattern) and a Closure. That Closure takes itself a parameter, a File object for the current le.
def def def new dir = rc.lookup(qftest, suite.dir) pattern = ~/.*\.qft/ files = [] File(dir).eachFileMatch(pattern) { file -> files.add(file.name)

} files.each { // A single Closure argument can also be refered with "it" rc.logMessage(it) }

Example 13.23: Closures

13.7.

Scripting with Groovy

113

Working with Lists and Maps is simpler than in Java.

def myList = [1, 2, 3] assert myList.size() == 3 assert myList[0] == 1 myList.add(4) def myMap = [a:1, b:2, c:3] assert myMap[a] == 1 myMap.each { this.println it.value }

Example 13.24: Working with lists and maps

13.7.1

Accessing variables in Groovy

In QF-Test Groovy scripts we decided not to support the $-Expansion for QF-Test vari(98) ables. It only takes place in Jython scripts (see section 13.3.3 ). The reason is that Groovy already uses the dollar sign to dereference script variables and evaluate expressions within a GString.
def x = 3 assert "$x" == 3 assert "${2 * x}" == 6

Example 13.25: GString expansion

Values of QF-Test variables can be obtained at runtime by means of several rc methods: String lookup(varname) or String lookup(group, varname) String getStr(varname) or String getStr(group, varname) Integer getNum(varname) or Integer getNum(group, varname) Boolean getBool(varname) or Boolean getBool(group, varname)
rc.setGlobal(fileExists, new File(c:/tmp/somefile.foo).exists()) assert rc.lookup(fileExists) == false assert rc.getStr(fileExists) == false assert ! rc.getBool(fileExists) rc.setGlobal(myvar, 3) assert rc.getNum(myvar) == 3

Example 13.26: Lookup for QF-Test variables in a Groovy script

13.7.

Scripting with Groovy

114

13.7.2

Global Groovy variables

Exchanging variables between several script nodes of the same type (Server scripts or SUT scripts of the same client) is even easier than in Jython. The rule is that undeclared variables are assumed to be dened in the binding of the script. If they are not, they will be added automatically to the list of binding variables.
myGlobal = global

Example 13.27: Dene a global Groovy variable

assert myGlobal == global def globals = binding.variables assert globals[myGlobal] == global globals.remove(myGlobal) assert globals.find { it == myGlobal } == null

Example 13.28: Use and delete a global Groovy variable

Predened global variables are the QF-Test run-context rc and the PrintWriter out, which is used for the scripts println method.

13.7.3

Groovy packages

Just like Java classes, Groovy source les (.groovy) can be organized in packages. Those intended to be shared between test-suites should be placed in the directory groovy under QF-Tests root directory. Others that are written specically for one test-suite can also be placed in the test-suites directory. The version-specic directory qftest-3.5.3/groovy is reserved for Groovy les provided by Quality First Software GmbH.
package my class MyModule { public static int add(int a, int b) { return a + b } }

Example 13.29: MyModule.groovy

13.7.

Scripting with Groovy

115

The le MyModule.groovy could be saved in a subdirectory my below the suite directory. Then you can use the add method from MyModule as follows:
import my.MyModule as MyLib assert MyLib.add(2, 3) == 5

Example 13.30: Using MyModule

This code also shows another groovy feature: Type aliasing. By using import and as in combination you can reference a class by a name of your choice.

Chapter 14 Data-driven testing

Data-driven testing is a very important aspect of test automation. In short, the goal is to run a given test or set of tests multiple times with different sets of input data and expected results. QF-Test has various means to store data or load external data for use in data-driven tests. The most convenient is based on a Data driver node which sets up an environment for iterating over the sets of data combined with one or more Data binder nodes to bind the variables for test execution. Note that there is no Data binder node as such. The name serves as a generic term for the specic nodes like a Data table or a CSV data le. This is best explained through some examples. A demo test-suite with simple and advanced examples named datadriver.qft is provided in the directory doc/tutorial below QF-Tests root directory.

14.1

Data-driver examples

Figure 14.1: A simple data-driven test

14.1.

Data-driver examples

117

The image above shows a Test-set with a Data driver node that contains a single Data binder in the form of a Data table node. The contents of the Data table are as follows:

Figure 14.2: Data table example

When the Test-set is executed, it will iterate over the rows of the Data table shown above. For each of the three iterations the values of the respective row in the table are bound to the variable named in the matching column header. Thus, during the rst iteration the variable named Model is bound to Rolo, Variant to None and Price to 19000. During the second iteration, Model is set to I5 and to Minigolf during the third and last iteration. For each iteration, all Test-case child nodes of the Test-set are run. The following image shows a run-log for the above Test-set.

14.1.

Data-driver examples

118

Figure 14.3: Run-log of a data-driven test

The next example shows that data-driven testing is not limited to a single loop:

Figure 14.4: Data-driven test with nested loops

The Data driver now contains a second Data table node with the following contents:

14.1.

Data-driver examples

119

Figure 14.5: Second data table example

The Test-set will now go through a total of six iterations because for each of the three iterations of the outer loop cars, both iterations of the inner loop accessories will be run as shown in the following run-log.

14.2.

General use of Data drivers

120

Figure 14.6: Run-log of a data-driven test with nested loops

Note

The extremely useful dynamic names of the loop nodes in the run-logs are obtained by setting the attribute Name for loop pass in the run-log to the value car Model: $(Model)in the rst and to car Model: $(Model), accessory Name: $(Accessory) in the second example. As you can see, that name is expanded individually for each iteration, so you can make use of the variables bound for that iteration.

14.2

General use of Data drivers

As seen in the example above the Data driver node must be placed in a Test-set node, between the optional Dependency and Setup nodes. When the Test-set is executed it will check for Data driver and run it. The contents of the Data driver node are not limited to Data binders. Like a normal Sequence the Data driver node can hold any executable node to be able to perform any setup that may be required to retrieve the data. Thus it is also possible to share Data binders by putting them inside a Procedure and calling the Procedure from inside the Data driver. Conceptually, a Data binder represents a loop where a different set of variables is bound for each iteration. A Data binder must be registered with a name in the Data driver context (368) node with of a Test-set. This ensures that the loop can be interrupted by a Break the same name. Once the Test-set has run the Data driver node, it will iterate over the registered data loops and perform the tests. In case of nested loops the Data binder that was registered rst represents the outermost loop. Its variables are bound rst and have lesser precedence than the variables from the inner loop(s).

14.3.

Examples for Data drivers

121

14.3

Examples for Data drivers

We provide a couple of examples for reading CSV or Excel les in the test-suite doc/tutorial/datadriver.qft.

14.4

Advanced use
(334)

Besides the Data table node there are various other means for binding data in a data (342) (346) (337) (350) driver. The Excel data le , CSV data le , Database and Data loop nodes are all (330) explained in detail in section 30.4 . It is also possible to bind data by calling the Procedures qfs.databinder.bindList or qfs.databinder.bindSets in the standard library qfs.qft. These take as parameters strings with lists or sets of values to split and iterate over. Please see tutorial chapter 8 for information about the standard library. And nally, data can be bound directly from Jython (and analogous from Groovy) with the help of the databinder module, which offers the following methods: bindList(Object rc, String loopname, String varname, Object values, String separator=None, String counter=None, String intervals=None) Create and register a databinder that binds a list of values to a variable. Parameters rc The current run-context. loopname The name under which to bind the data, equivalent to the Name attribute of a Data binder node. varname The name of the variable to bind to. values The values to bind. Either a sequence type or a string to split. separator Optional separator character to split the values at in case theyre a string. Default is whitespace. counter An optional variable name for the iteration counter. intervals Optional ranges of indices, separated by comma, e.g. 0,2-3.

14.4.

Advanced use

122

bindSets(Object rc, String loopname, Object varnames, Object values, String separator=None, String counter=None, String intervals=None) Create and register a databinder that binds a list of value-set to a set of variables. Parameters rc The current run-context. loopname The name under which to bind the data, equivalent to the Name attribute of a Data binder node. varnames The names of the variables to bind to. Either a sequence type or a string to split. values The value-sets to bind. Either a sequence of sequences each inner sequence being one set of data to bind - or a string to split. separator Optional separator character to split the varnames and the values of a value-set at in case theyre a string. Default is whitespace. Value-sets are separated by line-breaks. counter An optional variable name for the iteration counter. intervals Optional ranges of indices, separated by comma, e.g. 0,2-3. bindDict(Object rc, String loopname, dictionary dict, String counter=None, String intervals=None) Create and register a databinder that binds data from a dictionary. The keys of the dictionary are the names of the variables and the values are sequences of values to be bound. Parameters rc The current run-context. loopname The name under which to bind the data, equivalent to the Name attribute of a Data binder node. dict The dictionary to bind. counter An optional variable name for the iteration counter. intervals Optional ranges of indices, separated by comma, e.g. 0,2-3. Some examples:

14.4.

Advanced use

123

import databinder # Three iterations with the values "spam", "bacon" and "eggs" # bound to the variable named "ingredient" databinder.bindList(rc, "meal", "ingredient", ["spam", "bacon", "eggs"]) # Same with string values databinder.bindList(rc, "meal", "ingredient", "spam bacon eggs") # Same with string values and special separator databinder.bindList(rc, "meal", "ingredient", "spam|bacon|eggs", "|") # Two iterations, the first with item="apple" and number="5", # the second with item="orange" and number="3" databinder.bindSets(rc, "fruit", ["item", "number"], [["apple",5], ["orange",3]]) # Same with string values, note the linebreak databinder.bindSets(rc, "fruit", "item number", """apple 5 orange 3""") # Same as before with the data stored in a dict databinder.bindDict(rc, "fruit", {"item": ["apple", "orange"], "number": [5,3]})

Example 14.1: Examples for use of the databinder module

Chapter 15 Reports and test documentation

Besides test-suites and run-logs QF-Test can create a number of additional documents. Most important of these is the report, which summarizes the overall results of a testrun along with an overview over the test-suites executed and their individual results. The report is easy to read and understand without further knowledge about QF-Test and thus complements the run-log which is geared towards error analysis and requires some QF-Test experience to fully understand. Following is an example of a report summary:

15.1.

Reports

125

Figure 15.1: Example report

The other documents are more static in nature, describing the content of test-suites instead of test-run results. The testdoc document gives an overview over the structure (294) (288) of Test-set and Test-case nodes in a given set of test-suites. It is intended for the test architect or QA project manager and documents the current state of test devel(359) (352) opment. The pkgdoc documentation is similar, but focuses on Package , Procedure (317) and Dependency nodes instead. The result is a library reference comparable to Javas JavaDoc. The target audience for pgkdoc are test implementers requiring information about already existing procedures and their parameters.

15.1

Reports

Before we can start explaining how to create reports, some basic terms and concepts need to be dened.

15.1.

Reports

126

15.1.1

Report concepts

A report represents the results of one or more test-runs. A test-run comprises the execution of either a single test-suite or multiple test-suites, typically executed together in one batch-run. A test-run is identied by a runid. It is possible to execute a test-run in several steps by assigning the same runid to the resulting run-logs. A report is identied by a report name. For a report that covers a single test-run, the report name is usually the same as the runid. For reports summarizing the results of several test-runs a distinct report name can be specied. Reports can be created in multiple variants: XML, HTML and JUnit. Most users will probably use the HTML variant which can be viewed in a browser, printed and archived. The XML variant can serve as the basis for collecting the results of a test-run for further processing, for example to collect test-results in a database or to create customized HTML reports. We suggest that you always create both HTML and XML reports unless you have a good reason to do otherwise. JUnit reports base on the JUnit XML format as created by Apache Ant by use of its JUnitReport task. This format is not as pretty and detailed as the rst two report variants QF-Test offers but it is directly understood by many continuous integration tools and may prove useful for a quick integration with those. A report consists of one summary document, plus one document per run-log. These les are collected together with complementary les like icons, stylesheets and screenshot images in a directory. At the le level, this directory represents the report. The layout of the les inside the report directory depends on some command line options explained below. Basically there are two ways to lay out the les: Based on the le structure of the original test-suites or based on the le structure of the run-logs.

15.1.2

Report contents

In advance to the overall test result, a report as shown above starts with a summary containing informational system data and a legend describing the meaning of counter (29) icons used in the report (see Running tests ).
Note

The difference between Time spent in tests and Elapsed time are explicit delays introduced in nodes via the Delay before/after attribute or user interrupts. The contents of a report are based on the original structure of the executed test-suites. (294) (288) (299) The main structure is created from Test-set and Test-case nodes. The Comment attributes of the root node as well as the Test-set and Test-case nodes share the doctags (129) with testdoc documents as explained in section 15.2 . In addition to those doctags the @title doctag can be specied in the comment of the root node to set a title for the report document created for the respective test-suite.

15.1.

Reports
(548)

127

If -report.teststeps is specied in batch mode (true by default) or the respective option is active in the interactive dialog, Test-cases can be further broken down into steps (309) nodes. In addition to explicitly wrapping steps into a Testwith the help of Test-step step, any node can be turned into a test-step by specifying the doctag @teststep in its Comment, followed by an optional name for the step. For Test-step nodes the @author, @version and @since doctags are also applicable. The names, comments and tag values of the various nodes can contain variables that will be expanded at execution time so that the expanded value is shown in the report. This is especially useful for test-steps within a procedure. If listing of test-steps is active, Setup, Cleanup and Dependency nodes are also listed and checks, screenshots and messages, including warnings, errors and exceptions are properly integrated into the nested steps. If the test-suites are set up properly the resulting report can serve as a very readable summary of what was going on during the execution of a test. Whether warnings and checks are listed is determined by the command line arguments (548) (547) -report.warnings and -report.checks or the respective interactive options. Warnings from component recognition are never listed because they are too technical and could easily ood the report. For checks one must distinguish between checks that represent an actual verication step and those that are used solely for control ow, for example to check whether a checkbox is already selected and click it only in case it is not. By default QF-Test lists those Check nodes in the report that have the default result settings, i.e. the Error level of message is Error, no exception is thrown and no result variable bound. All others are treated as helpers for control ow and not listed in the report. For cases where this default treatment is not appropriate, you can force a Check into the report via the doctag @report in its Comment attribute or prevent its listing via @noreport. Of course failed checks are treated as warnings, errors or exceptions (depending on their Error level of message) and cannot be excluded from the report if messages at the respective level are shown. Additional messages, checks and screenshots can be added to the report by scripts via the methods rc.logMessage, rc.logScreenshot and rc.check and its variants, which have an optional report parameter. For details, please see the run-context API (589) documentation in section 37.6 .

15.1.3

Creating reports

There are three ways to create reports: Interactively from a run-log FileCreate HTML/XML report... . In batch mode as the result of a test-run. through the menu item

15.1.

Reports

128

In batch mode by transforming already existing run-logs. The interactive variant is easy to use. Just select the target directory for the report and whether you want the XML and/or the HTML variant. For report creation in batch mode there are a number of command line options which (539) are listed and explained in section 32.2 . Lets look at the variant of creating reports as the result of a test-run rst: The command line syntax for plain test-execution in batch mode is qftest -batch <test-suite> [<test-suite>...] To create a combined XML and HTML report, use -report <directory> . To create only one version or to separate the XML, HTML variants, use (548) (547) -report.xml <directory> and/or -report.html <directory> . For (547) JUnit reports -report.junit <directory> works respectively. The runid of a test-run is specied with -runid <id> , the name of the report with (548) -report.name <name> . If the report name is unspecied it will default to the runid. To lay out the les in the report directory according to the le structure of the test-suites, (550) use -sourcedir <directory> . To use the le structure of the run-log as the (549) basis, use -runlogdir <directory> . The following is a typical example of a command line for a batch run making use of the (554) placeholders explained in section 32.2.4 :
qftest -batch -runid +M+d -runlog logs/+i -report report_+i -sourcedir . suite1.qft subdir/suite2.qft
(549) (547)

Example 15.1: Creating a report as the result of a test-run

Creating a report as a separate step by transforming a set of run-logs is similar in many respects. The run-logs to transform have to be specied instead of the test-suites to execute and the -runid <id> and -sourcedir <directory> command line options have no effect. The following is an example for how to create a weekly summary report based on the assumption that you have collected all run-logs below the directory named logdir, possibly in subdirectorys thereof:
qftest -batch -genreport -report report_+M+d -report.name week_of_+y+M+d logdir

Example 15.2: Creating a weekly summary report

15.2.

Testdoc documentation for Test-sets and Test-cases

129

15.2

Testdoc documentation for Test-sets and Test-cases

The type of test documents called testdoc provide overview and detailed information (294) (288) over the Test-set and Test-case nodes of one or more test-suites. They are a valuable tool for QA project managers to keep track of the current state of test development. Similar to reports, testdoc documents are laid out as directories with one summary le and one detailed le per test-suite. A testdoc document for a single suite can be created interactively from a test-suite by selecting Create HMTL/XML testdoc... from the File menu. This is very useful during test development to quickly check whether all tests are properly documented. For actual use as a reference it is preferable to create complete sets of documents spanning multiple test-suites for a whole project. This can be done by running QF-Test (543) in batch mode with the -gendoc command line argument. In its simplest form, a call to create testdoc documentation for a whole directory tree would look as follows:
qftest -batch -gendoc -testdoc test_documentation directory/with/test-suites

Example 15.3: Creating testdoc documentation

Please see chapter 32 arguments.

(535)

for detailed information about the available command line

(299)

To get optimal results you can use HTML markup in the Comment attributes of Test-set and Test-case nodes and also make use of doctags. A doctag is a keyword beginning with @, sometimes followed by a name and always by a description. This is a proven concept in JavaDoc, the standard documentation format for Java programs (see http://www.oracle.com/technetwork/java/javase/documentation/index137868.html#tag). The following doctags are supported for Test-set and Test-case nodes: @deprecated If a Test-set or Test-case is no longer to be used, this description should explain when and why the node was deprecated and especially which replacement should be used. @condition Non-formal explanation of the condition under which the node is executed or skipped.

15.3.

Pkgdoc documentation for Packages, Procedures and Dependencies

130

@param Description for a parameter. Following are the name of the parameter and its description. @charvar Description for a characteristic variable. Following are the name of the variable and its description. @author Author of the Test-set or Test-case. @version Version of the Test-case or Test-case. @since The version since which this Test-set or Test-case has been available. In addition to the doctags described above, the doctag @title in the comment of the root node can be used to specify a title for the testdoc document created for the respective test-suite.

15.3

Pkgdoc documentation for Packages, Procedures and Dependencies

The concepts of and methods for creation of pkgdoc documents are nearly identical to testdoc, so this section is brief. Instead of Test-set and Test-case nodes, pkgdoc docu(359) (352) (317) and Dependency nodes. They are intended for ments cover Package , Procedure the test developer to keep track of the procedures available for use in implementing tests. Please refer to the standard library qfs.qft as a good example how a pkgdoc may look like. A pkgdoc document can also either be created interactively using (535) FileCreate HMTL/XML pkgdoc... or in batch mode. Again, please see chapter 32 for detailed information about the available command line arguments. As the following example shows, testdoc and pkgdoc can even be created together in a single batch run:

15.3.

Pkgdoc documentation for Packages, Procedures and Dependencies

131

qftest -batch -gendoc -testdoc tests -pkgdoc procedures directory/with/test-suites

Example 15.4: Creating testdoc and pkgdoc documentation in a single run

Of course pkgdoc also supports HTML markup and doctags. The following doctags are supported for Package, Procedure and Dependency nodes: @deprecated If a Procedure, Dependency or Package is no longer to be used, this description should explain when and why the node was deprecated and especially which replacement should be used. @param (Procedure and Dependency only) A parameter of a Procedure or Dependency. parameter and its description. @charvar (Dependency only) Description for a characteristic variable of a Dependency. Following are the name of the variable and its description. @return (Procedure only) The return value of the procedure. @result (Procedure and Dependency only) Can be used to document side-effects of the Procedure or Dependency like setting a global variable. @throws (Procedure only) Expected exception. Following are the name of the exception and a description of its cause. @catches (Dependency only) An exception being caught by the Dependency. Following are the name of the exception and a description of the handler. @author Author of the Package, Procedure or Dependency. Following are the name of the

15.3.

Pkgdoc documentation for Packages, Procedures and Dependencies

132

@version Version of the Package, Procedure or Dependency. @since The version since which this Package, Procedure or Dependency is available. In addition to the doctags described above, the doctag @title in the comment of the root node can be used to specify a title for the pkgdoc document created for the respective test-suite.

Chapter 16 Test execution

When talking about test execution, there are two aspects to be considered. On one hand you need to run tests while they are developed to check them for proper operation. (29) This situation has already been described in section 4.2 . Basically all you have to do to run a test interactively is invoking RunStart from the main menu. On the other hand you want to run your tests periodically to ensure the stability of the system under test, for example in nightly regression tests. Instead of launching QF-Test, loading the test-suite and running it from the graphical user interface, it is much more convenient here to execute tests from the command line in batch mode. This kind of running tests is explained in the rst section of this chapter (Test execution in batch (133) mode ). Sometimes, for instance when you want to run the test on a remote computer, a second variant comes into play: the daemon mode. This type of test execution, which uses a running QF-Test instance to execute tests, is the topic of the second section (Executing (139) tests in daemon mode ). For integration of QF-Test with build tools like ant, maven or Hudson/Jenkins, please (168) refer to chapter 20 .

16.1

Test execution in batch mode

There are a lot of command line arguments when running QF-Test in batch mode; an (535) overview can be found in chapter 32 . Here we will present examples showing the most important of them. The examples are written for the Windows operating system, but you may easily adapt them for the Linux platform. What is different is the path specication and also the (554) syntax for placeholders (section 32.2.4 ): On Linux you can use +X as well as %X. On Windows theres a separate console application qftestc.exe. In contrast to its

16.1.

Test execution in batch mode

134

GUI variant qftest.exe, it waits until the execution of QF-Test has terminated and (390) also displays print output from a Server script . You can use qftestc.exe in place of qftest.exe wherever youll nd it convenient.

16.1.1

Command line usage

Lets start with the most simple QF-Test command to execute a test:
qftest -batch -run c:\mysuites\suiteA.qft

Example 16.1: Test execution from the command line

The argument -batch makes QF-Test start without a graphical user interface. The second argument, -run, is the specier for test execution. Finally, at the end of the command line, you nd the test-suite to be executed.
Note

The argument -run is optional, i. e. the test execution is dened as default for the batch mode. When running the above command, all top-level Test, Test-case and Test-set nodes of suiteA.qft will be executed one after another. After the test-run you will nd a runlog le in the current directory; it has the same name as the test-suite (except from the extension, which can be .qrl, .qrz or .qzp). The run-log le shows the result of the test-run. By specifying -nolog you can suppress the creation of a run-log. Probably this only makes sense, if you have extended your test by your own log output (written to a le). Otherwise youd have to check the result code of QF-Test, whereas 0 means that everything is alright. A positive value in contrast indicates that warnings, errors or exceptions (555) occurred during the test-run (see section 32.3 ). Thats why in most situations youll probably prefer to create a run-log and save it at a xed place in the le system. This can be achieved with the parameter -runlog:
qftest -batch -compact -runlog c:\mylogs\+b c:\mysuites\suiteA.qft

Example 16.2: Test execution with run-log creation

A run-log le suiteA.qrz will now be created in the specied directory c:\mylogs. The placeholder +b is responsible for its name being identical with that of the test-suite. The additional switch -compact prevents the run-log from growing too large: Only the nodes needed for a report and those immediately before an error or an exception are kept in the run-log. Especially in case of very long test-runs this may help to reduce the

16.1.

Test execution in batch mode

135

amount of required memory. In the meantime however, QF-Test offers split run-logs for (63) the same reason. For more information about this topic see section 8.1 .
Note

Whether the le is indeed created as compressed run-log (to be distinguished from the above compact) with extension .qrz, depends on the system settings. To force the creation of a particular format you can set the le extension explicitly. With -runlog c:\mylogs\+b.qrl, for example, an uncompressed XML le will be produced. Sometimes you may want to execute not the whole test-suite but only parts of it. By using the parameter -test you can run a specic node of the test-suite:
qftest -batch -runlog c:\mylogs\+b -test "My test-case" c:\mysuites\suiteA.qft

Example 16.3: Executing a specied node

The parameter -test expects the Id attribute of the node to follow or the qualied name of a Test-case or Test-set. If you want to execute several nodes, you can dene -test <Id> multiple times. Apart from the nodes Id, -test accepts also the numerical index of a top-level node. For example, -test 0 will run the rst child of the Test-suite node. The run-log provides a rather technical view of the test-run; it is helpful mainly when (63) analyzing errors (cf. section 8.1 ). The report in contrast contains a summary of the (124) executed test-cases and errors (cf. chapter 15 ) in XML or HTML format. It is created from the run-log either in a separate step after running the test or automatically with the test-run:
qftest -batch -runlog c:\mylogs\+b -report c:\mylogs\rep_+b_+y+M+d+h+m c:\mysuites\suiteA.qft

Example 16.4: Creating a report

In this example the XML and HTML les are saved in a directory which name consists of the test-suite and a timestamp like c:\mylogs\rep_suiteA_0806042152. When replacing the argument -report with -report.xml or -report.html respectively, only an XML or HTML report will be created. Test-cases often uses variables to control the execution of the test. For example, you (285) may have dened the variable myvar in the Test-suite node of the suite. You can overwrite its default value when running the test-suite from the command line:
qftest -batch -variable myvar="Value from command line" -runlog c:\mylogs\+b c:\mysuites\suiteA.qft

Example 16.5: Test execution with variables

16.1.

Test execution in batch mode

136

If needed, you can specify -variable <name>=<wert> multiple times to set values for different variables.

16.1.2

Windows batch script

Running tests from the command line is fundamental for integrating QF-Test in test (156) management systems (see Interaction with Test Management Tools ). Otherwise, living without such a tool, you may nd it convenient to embed the command for the test execution into a script. A simple Windows batch script (qfbatch.bat) looks like this:
@echo off setlocal if "%1" == "" ( echo Usage: qfbatch Testsuite goto end ) else ( set suite=%~f1 ) set logdir=c:\mylogs pushd c:\programs\qftest\qftest-3.5.3\bin @echo on .\qftest -batch -compact -runlog %logdir%\+b %suite% @echo off if %errorlevel% equ 0 ( echo Test terminated successfully goto end ) if %errorlevel% equ 1 ( echo Test terminated with warnings goto end ) if %errorlevel% equ 2 ( echo Test terminated with errors goto end ) if %errorlevel% equ 3 ( echo Test terminated with exceptions goto end ) if %errorlevel% leq -1 ( echo Error %errorlevel% goto end ) :end popd

Example 16.6: Batch script qfbatch.bat to execute a test-suite

16.1.

Test execution in batch mode

137

Now you can simply run that script with only the le name of the test-suite as parameter. Everything else is done automatically: The test-suite will be executed, the run-log le stored in logdir and nally the script will print out the state of the test-run (depending on the QF-Test result code).

16.1.3
3.0+

Groovy
(94)

Since version QF-Test 3 the language Groovy is part of the release (cf. chapter 13 ). It is meant mainly for scripting inside QF-Test (Server and SUT scripts), but it can, like Jython, also be used outside of QF-Test. Groovy is probably well suited to create a little test execution management system by yourself. By the way, Groovy simplies working with Ant, too: Instead of dealing with bulky XML les, which makes it hard to dene conditions, you can work with the Groovy AntBuilder. However, thats out of scope here, the following example doesnt rely on Ant but only on the basic Groovy features:

16.1.

Test execution in batch mode

138

def suite = if (args.size() == 0) { println Usage: groovy QfExec Testsuite return } else { suite = args[0] } def qftestdir = c:\\programs\\qfs\\qftest\\qftest-3.5.3 def qftest = qftestdir + \\bin\\qftest.exe def command = "$qftest -batch -compact -runlog c:\\mylogs\\+b \"$suite\"" def printStream = { stream -> while (true) { try { stream.eachLine { println it } } catch (IOException) { break } } } println "Running command: $command" def proc = command.execute() new Thread().start() { printStream(proc.in) } new Thread().start() { printStream(proc.err) } proc.waitFor() switch (proc.exitValue()) { case 0: println Test terminated successfully; break case 1: println Test terminated with warnings; break case 2: println Test terminated with errors; break case 3: println Test terminated with exceptions; break default: println "Error ${proc.exitValue()}" }

Example 16.7: Groovy script QfExec.groovy to execute a test-suite

If you have Groovy installed on your computer independently of QF-Test, you can run the example test-suite simply via groovy QfExec c:\mysuites\suiteA.qft. Otherwise you can use the Groovy jar le from the QF-Test installation, preferably again with help of a batch script:

16.2.

Executing tests in daemon mode

139

@echo off setlocal if "%1" == "" ( echo Usage: qfexec Testsuite goto end ) set qftestdir=c:\programs\qftest\qftest-3.5.3 set scriptfile=QfExec.groovy java -cp %qftestdir%/lib/groovy-all.jar groovy.ui.GroovyMain %scriptfile% %* :end

Example 16.8: Batch script qfexec.bat to run a Groovy script (here: QfExec.groovy)

Now execute the test-suite with qfexec c:\mysuites\suiteA.qft.

16.2

Executing tests in daemon mode

In daemon mode QF-Test listens to RMI connections and provides an interface for remote test execution. This is useful for simplifying test execution in a distributed load(176) testing scenario (chapter 21 ), but also for integration with existing test-management (156) or test-execution tools (chapter 19 ).

16.2.1

Launching the daemon

!!! Warning !!! Anybody with access to the QF-Test daemon can start any program on the machine running the daemon with the rights of the user account that the daemon is running under, so access should be granted only to trusted users. If you are not running the daemon in a secure environment where every user is trusted or if you are creating your own library to connect to the QF-Test daemon, you denitely (709) should read section 40.3 about how to secure daemon communication with SSL. To work with a daemon, you must rst launch it on any computer in your network (of course, this host can also be localhost):
qftest -batch -daemon -daemonport 12345

Example 16.9: Launching a QF-Test daemon

Note

Important compatibility note:

16.2.
3.5+

Executing tests in daemon mode

140

Starting with QF-Test version 3.5, SSL is used for daemon communication by default. To interact with a QF-Test version older than 3.5 you must start the daemon with an (544) empty -keystore <keystore file> argument in the form:
qftest -batch -keystore= -daemon -daemonport 12345

Example 16.10: Launching a QF-Test daemon without SSL

If you omit the argument -daemonport, the daemon will listen on QF-Tests standard port 3543. You may check whether the daemon is running by means of the netstat utility:
Windows Linux

netstat -a -p tcp -n | findstr 12345 netstat -a --tcp --numeric-ports | grep 12345 If you want to launch a daemon on a remote host, you may use for instance ssh or VNC. Your network administrator knows whether and how this works. To follow the examples below, a local daemon will be sufcient.

16.2.2
3.0+

Controlling a daemon from QF-Tests command line

The easiest way to get in touch with a daemon is running QF-Test from the command line in the calldaemon mode. The following example checks if a daemon is listening at the specied host and port:
qftestc -batch -calldaemon -daemonhost localhost -daemonport 12345 -ping

Example 16.11: Pinging a QF-Test daemon

In contrast to the netstat command from above -ping also works between different computers (if you check the daemon on your local computer, you can omit the argument -daemonhost). What you actually want from a daemon is executing your test-case(s) and getting back a run-log le. It sounds and indeed looks quite similar to what you have seen before when running a test in batch mode:
qftest -batch -calldaemon -daemonhost somehost -daemonport 12345 -runlog c:\mylogs\+b -suitedir c:\mysuites suiteA.qft#"My test-case"

Example 16.12: Running a test-case with the QF-Test daemon

16.2.
Note

Executing tests in daemon mode

141

In contrast to the batch mode, a Test-case or a Test-set node is always referenced here by its qualied name, for instance My Test-set.My Test-case (just to remember: -test <Id> may be used in batch mode). To execute the complete suite suiteA.qft, you can simply omit the test-case or write suiteA.qft#.. If the daemon is running on a remote host, you have to specify it explicitly via -daemonhost (default is -daemonhost localhost). Note that the parameter -suitedir refers to the remote host (where the daemon is running) while -runlog denes a local le.

3.4+

In case you cannot easily observe the test running on a remote host, you may nd it convenient to add the argument -verbose to get status output in the console (on Windows, use qftestc to see the output). A running daemon, no matter whether local or remote, can be terminated with the calldaemon command -terminate:
qftest -batch -calldaemon -daemonport 12345 -daemonhost localhost -terminate

Example 16.13: Terminating a QF-Test daemon

A complete list of the calldaemon parameters can be found in the chapter Command (535) line arguments and exit codes .

16.2.3

Controlling a daemon with the daemon API

Using the QF-Test command line to control a daemon was quite easy. On the other hand, to get all capabilities of a daemon, you have to deal with the daemon API. In this section we will concentrate on some basic examples, the whole interface is described in (695) chapter 40 . To get started with the daemon API, insert a Server script node with the following code:

16.2.

Executing tests in daemon mode

142

from de.qfs.apps.qftest.daemon import DaemonRunContext from de.qfs.apps.qftest.daemon import DaemonLocator host = "localhost" port = 12345 # Leading r means raw string to allow normal backslashes in the path string. testcase = r"c:\mysuites\suiteA.qft#My test-case" timeout = 60 * 1000 def calldaemon(host, port, testcase, timeout=0): daemon = DaemonLocator.instance().locateDaemon(host, port) trd = daemon.createTestRunDaemon() context = trd.createContext() context.runTest(testcase) if not context.waitForRunState(DaemonRunContext.STATE_FINISHED, timeout): # Run did not finish, terminate it context.stopRun() if not context.waitForRunState(DaemonRunContext.STATE_FINISHED, 5000): # Context is deadlocked raise UserException("No reply from daemon RunContext.") rc.logError("Daemon call did not terminate and had to be stopped.") result = context.getResult() log = context.getRunLog() rc.addDaemonLog(log) context.release() return result result = calldaemon(host, port, testcase, timeout) rc.logMessage("Result from daemon: %d" %result)

Example 16.14: Daemon API in a Server script

The script shows the basic mechanisms to control a daemon: First nd a running daemon with locateDaemon. Provide an environment for test-runs by calling createTestRunDaemon. To run a test, you need a context object (createContext). The creation of that object requires a QF-Test run-time license. Now the context enables you to start a test-run (runTest) and to query about its current state. waitForRunState waits during the dened timeout (in milliseconds) until the specied state has occurred. In the example above, we wait for the test to terminate within one minute. Finally, when the test-run has terminated, the context can query the test result with (555) the method getResult (cf. Exit codes for QF-Test ). Moreover, you can use the context to get the run-log of the daemon test-run. It can be included in the local run-log by means of the rc method addDaemonLog.

16.2.
Note

Executing tests in daemon mode

143

To keep it small and simple, the example script does not contain any error handling. However, particularly when working with a daemon, you should check every method call. Driving a daemon from a Server script has the disadvantage of consuming an additional QF-Test license to run the script node interactively or in batch mode. However, this doesnt apply nor for the above-mentioned calldaemon mode neither for the case when controlling a daemon outside QF-Test (see below). The usage of the daemon API is not restricted to Server scripts. Outside QF-Test a daemon can be contacted by means of a Java program or, more easily, a Groovy script. The following Groovy script works with several running daemons and may serve as a starting point for load tests. Suppose we have started some daemons in our network, each on a separate machine. We want to execute a test-case simultaneously by all of the daemons and we want to save a run-log for every single test-run (daemon1.qrl, ..., daemonN.qrl). The test-suite containing the test-case to be executed may be available for all daemon instances via the network drive z:).
import de.qfs.apps.qftest.daemon.DaemonLocator import de.qfs.apps.qftest.daemon.DaemonRunContext def testcase = "z:\\mysuites\\suiteA.qft#My test-case" def logfile = "c:\\mylogs\\daemon" def timeout = 120 * 1000 def daemons = DaemonLocator.instance().locateDaemons(10000) def contexts = [] // Start tests for (daemon in daemons) { def trd = daemon.createTestRunDaemon() trd.setGlobal(machines, daemons.size().toString()) def context = trd.createContext() contexts << context context.runTest(testcase) } // Wait for tests to terminate for (i in 0..<contexts.size()) { def context = contexts[i] context.waitForRunState(DaemonRunContext.STATE_FINISHED, timeout) byte[] runlog = context.getRunLog() def fos = new FileOutputStream("$logfile${i + 1}.qrl") fos.write(runlog) fos.close() context.release() }

Note

Example 16.15: Groovy daemon script CallDaemon.groovy

To run that Groovy script, you need the QF-Test libraries qftest.jar, qfshared.jar, and qflib.jar as well as the Groovy library, which is also part of

16.2.

Executing tests in daemon mode

144

the QF-Test installation. The following batch script shows how it works:
@echo off setlocal set qftestdir=c:\programs\qftest\qftest-3.5.3 set qflibdir=%qftestdir%\qflib set classpath=%qftestdir%\lib\groovy-all.jar set classpath=%classpath%;%qflibdir%\qftest.jar;%qflibdir%\qfshared.jar; %qflibdir%\qflib.jar java -cp %classpath% groovy.ui.GroovyMain CallDaemon

Example 16.16: Batch script calldaemon.bat to run Calldaemon.groovy

To make the daemon example a load test (cf. chapter 21 ), you have to synchronize the test-runs inside of My test-case (e. g. after starting the SUT). This can be done by means of the rc method syncThreads:
def machines = rc.getNum(machines) rc.syncThreads(startup, 60000, -1, machines)

(176)

Example 16.17: Groovy Server script node to synchronize the test-runs

The variable machines denotes the number of hosts with a daemon running on them. Best dene it in the Test-suite node of the test-suite with a default value of 1. When running the Groovy script, it will be overwritten with the correct value.

Chapter 17 Distributed test development

The previous chapters all focused on creating and organizing sets of reliable tests in a single test-suite. However, when testing a large application, a single suite may not be enough. There are at least two scenarios where splitting tests into multiple test-suites becomes essential: Multiple developers are creating and editing tests. To avoid redundancy and duplication of code, separately developed tests should use common Procedures and Components where possible, but only one person can edit a test-suite at a time. Tests are simply getting too large. Run-logs for extensive test-runs may cause the system to run out of memory and organizing a large number of tests in a single suite is difcult. Things may become unwieldy. It may also be desirable to be able to run some of the tests as part of the whole test as well as standalone. QF-Test provides a set of advanced features that make it possible to split and arrange tests across a set of test-suites. Multiple developers can work on separate parts of a test, then coordinate their efforts, merge the Components of their suites and create libraries of shared Procedures. This chapter rst explains the various mechanisms for distributed test development and how they interact. The nal section then summarizes these in concise step-by-step instructions on how to approach large testing efforts with QF-Test.

17.1

Referencing nodes in another test-suite

(352) (514)

It is possible to reference Procedures and Components in a test-suite other than the current one. These references can be explicit or implicit through included les:

17.1.

Referencing nodes in another test-suite

146

Explicit references use a syntax similar to the one used in URLs to specify an item inside a web page. The referenced suite must be prepended to the (356) (355) Procedure name attribute of a Procedure call of the Component id attribute of a Component dependent node, separated by a # character. The usual packagepath.procedure becomes suite#packagepath.procedure. Implicit references make use of the Include les attribute of the Test-suite node. Whenever a node is not found in the current suite, QF-Test will search for a matching Procedure or Component within all the suites directly or indirectly included les (a le is considered indirectly included by a suite if it is found as an included le within one of the suites own included les; for example, if suite A includes B, and suite B includes C, then C is indirectly included by A). A test-suite that references a node in another test-suite becomes dependent on that suite. If the Name of a Procedure or the Id of a Component in the referenced suite changes, the suite with the reference must get updated, otherwise the link is broken and the suite will no longer work correctly. In such cases QF-Test will automatically update references if it knows about them. The best way to ensure that is to have both testsuites in a common project because QF-Test automatically tracks all includes and all explicit references within a project. Alternatively you can list the calling suite in the (286) Dependencies (reverse includes) attribute of Test-suite root node of the referenced suite. While implicit references are more convenient in most cases, they can make tests harder to understand because it is not immediately obvious where the Procedure or Component referenced by some node is actually located. One possibility to nd out is to select ) or Locate component (Ctrl-W ) from the context menu. Locate procedure (Ctrl-P Additionally, QF-Test provides the menu items OperationsMake references explicit and OperationsMake references implicit which let you toggle quickly between the two modes without changing the actually referenced nodes. In both cases, the referenced suite can either be given a relative or absolute lename. Relative lenames will be resolved relatively to the directory of current suite, or - if that fails - relatively to the directories on the library path (see option Directories holding (231) test-suite libraries ). Always use the forward / as the directory separator, even under Windows. QF-Test will map it to the correct character for the system it runs on. This keeps your test-suites independent from the operating system.
Note
(286) (285)

Your Package and Procedure names as well as Component Ids should not contain any \ or # characters. If they do, you need to include an escape character in the Procedure (583) call or the Component id id attribute. See section 36.6 for details about escaping and quoting special characters. When choosing the Procedure for a Procedure call or the Component for some event in the dialog, QF-Test offers a selection of all currently opened test-suites. If a Procedure or Component from another test-suite is selected, QF-Test automatically creates the correct

17.2.

Managing Components

147

reference, taking included suites into account. When the test is run at a later time, the referenced test-suite is loaded automatically if necessary. During execution QF-Test keeps a stack of currently executing suites. Whenever a Procedure is called in another suite, the called suite is pushed on to the top of this stack and removed when execution returns to the calling suite. Whenever during the execution of a Procedure a Window or Component is referenced by its id, QF-Test searches through this stack of suites from the top to the bottom, i.e. rst in the test-suite of the called Procedure and then in the calling suite, always checking any included les along the way. This process is quite complicated and you should take care to keep your include hierarchies simple. In case you encounter problems anyway, a detailed explanation is given (584) in section 36.7 .

17.2

Managing Components
(35)

As we have emphasized in chapter 5 , the Components are the essential part of a test-suite. If the SUT changes between releases, these will likely be affected most. If changes are so massive that QF-Test cannot adapt automatically, the Components will have to be updated manually. This is why you should try to avoid redundancy in the Component hierarchy of your tests more than in any other part. Therefore, when splitting your tests across multiple test-suites you should try to keep the Components together in one central test-suite and include this suite from the other suites. For very large applications you may want to split the Component hierarchy into parts, each related to a separate part of the SUTs GUI. Maintaining this central Component library is not trivial. The problems that will arise can be resolved with QF-Test as follows: When multiple test developers are recording new Components simultaneously, they cannot be integrated immediately into the central suite, because only one user can edit the central suite at a time. Instead, Components must be integrated later by importing them into the central suite when the new tests have stabilized. This is explained in the following section. When the SUT changes, Components in the central suite may need to be updated. If this involves changing any Component Ids, this will break any references to these Components from other suites. To avoid that, QF-Test must update those references and it will do so, provided that the suites that depend on the central suite are currently loaded, belong to the same project or are listed in the (286) (285) Dependencies (reverse includes) attribute of the Test-suite node of the central suite.

17.3.

Merging test-suites

148

17.3

Merging test-suites

Test-suites can be merged by importing one test-suite into another with the FileImport... menu item. You can select the areas of the test-suite, which should be imported. You have to take care about a correct Include/Reverse-Include of your test-suites to (193) for ensure, that all calls and component references are still valid. See chapter 25 details.

17.3.1

Importing Components

During import, all Windows and Components of the imported test-suite are integrated into the component hierarchy of the importing suite. Components that already exist are not copied. An Id conict (identical components with different Ids or differing components with identical Ids) is resolved automatically by changing the Id of the imported component. Afterwards, all Windows and Components are removed from the imported suite. All nodes in the imported suite that referred to these Components are updated accordingly. Ideally, the imported suite should include the importing suite so no explicit suite references will have to be created.

17.3.2
3.3+

Importing Procedures and Testcases

As you can import Components QF-Test also allows to import Procedures, Packages, Dependencies and Test-cases as well as Test-sets by choosing Procedures or Tests in the import dialog. You should take care about keeping all calls consistent, e.g. in most cases it does not make sense to import Procedures without their required Components. In case you only want to import one dedicated Procedure or Test-case you can use the button Detailimport on the importdialog. Here you can choose any node you want to import separately.

17.4

Strategies for distributed development

There is no single best way of test development or organization, but one approach that works well is the following: Start with a central test-suite that has the functionality needed to start and stop the

17.4.

Strategies for distributed development

149

SUT and a basic set of Tests and Procedures. This will become your master suite which will contain all Components. Make sure that your developers have understood the importance of assigning names with setName() and that unique names are assigned consistently where needed. Where setName() is not an option, try to implement (634) ComponentNameResolvers to achieve this (see section 39.1 ). You should be able to record and replay sequences without much ado and without polluting the Component hierarchy after trivial changes in the user interface. Move as much functionality as possible into Procedures, especially commonly-used stuff and the setup and cleanup routines for the SUT. To create new tests, start with an empty test-suite. Include the master test-suite by (286) (285) editing the Include les attribute of the Test-suite node of the new suite. Create the Setup and Cleanup nodes to start and stop the SUT by calling the respective Procedures in the master suite. Create your tests as required. When recording sequences, the Components of the master-suite will be used if possible. New Components are added to the new suite, so the master suite will not be modied at this stage. Where possible, call Procedures in the master suite for common operations. When your new set of tests is complete and you are satised that they work well, import any required nodes of your new test-suite into the master suite. You have to ensure that all new Component nodes that you recorded are imported into the master suites Component hierarchy in any case. The master suites existing Components will not be affected by this, so other suites that depend on the master suite will not need to be modied. After importing Components you can import all or only the required Procedures into the master suite. You now have various options of how to arrange the actual sequences of events and checks that form your tests. In any case it is a good idea to move everything to Procedures and Packages structured after your test-plan. Then the top-level Tests nodes of the master suite and your new suite will only contain the required hierarchy of Test and Sequence nodes lled with Procedure calls to the actual test-cases. Such an arrangement has several advantages: All your tests are structured cleanly. You can easily create different sets of tests with varying complexity and runtime.

17.5.

Static validation of test-suites

150

You have the option to keep the test-cases in separate test-suites and have the master suite call them. These test-case-libraries must include the master-suite, so they need not contain any Components themselves. You can organize your tests so that the master-suite will run the whole set of tests, while each separate suite can also be run standalone. The tests can be maintained by several developers as long as modications to the master suite are coordinated. If you decide to keep your new tests in the newly created test-suite instead of moving them to the master suite, modify the master suite to tell QF-Test that there is a new test-suite that depends on it. To do so, either ensure that both test-suites (286) belong to the same project or add the new test-suite to the Dependencies attribute of the master suites Test-suite node. If you need to modify or extend the new test-suite later, proceed as before. You can record new sequences as needed. When you are done, merge any newly created Components back into the master suite. If your SUT changes in a way that requires updates or adaptions to the mastersuites Component hierarchy, you must coordinate your test developers. Before you start updating the Components, make sure that all suites that directly or indirectly include the master suite belong to the same project as the master suite or are listed in the Dependencies attribute of the master suites Test-suite node. If modifying the Components of the master suite involves any Component id changes, QF-Test will update the depending test-suites accordingly, so they should not be edited simultaneously by others. The le format for QF-Test test-suites is XML and thus plain text. As a result, testsuites can be managed very well by version control systems like CVS. Changes to some Component id attributes of the depending suites can typically be merged with other changes without conicts, alleviating the need for coordination. Of course, the above scheme can be extended to have several master suites for testing different parts or aspects of an application. It may be a good idea to ensure that the component hierarchies in these suites dont overlap too much though. This will save you the effort of maintaining all these hierarchies in case the user interface of the SUT changes signicantly.

17.5
3.1+

Static validation of test-suites

Working in a project over time will cause modications, refactoring or deletion of steps in your test-suite structure, e.g. you may consider renaming Procedures or simply removing them once they are not required anymore.

17.5.

Static validation of test-suites

151

In such cases it is quite important that you adapt all references of the according Procedure in order to guarantee that the remaining tests keep running. For this purpose QF-Test automatically updates all references during the process of renaming or moving elements on demand. If you want to ensure that your created test structure doesnt contain any call of nonexisting Procedures anymore, you can also use the Analyze references command of QF-Test in order to perform a static validation of your test-suite. This command will open a dialog showing all references and whether they are still okay or something is missing. You can trigger the analysis via a right mouse-click and selecting Additional node operationsAnalyze references... or selecting the according entry from the main menu under Operations . This method is also available in batch mode.

17.5.

Static validation of test-suites

152

Figure 17.1: Result of analyzing references

3.5+

QF-Test also provides features to search through your test-suites for duplicate nodes, empty Packages or Procedures or to analyze for nodes having invalid characters in their names. This kind of static validation is available for Procedures, Dependencies, Test-cases, Test-sets and Components and their references.

Chapter 18 Automated Creation of Basic Procedures

3.0+

18.1

Introduction

At the beginning of a typical QF-Test project the tester records the rst tests and starts them. After a couple of such recordings and rst success stories he notices that only recording or performing copy/paste bares some hidden pitfalls in maintaining the tests. Just think about a possible workow change in a main panel, then the tester might have to adapt all test-cases. Thats why we recommend to make use of the modularization concept using procedures and variables as early as possible in a project. For more (77) information about the modularization concept, please see section 9.3 . In projects containing a lot of dialogs and graphical components it might be sufcient to split those procedures into component-specic ones, e.g press button ok and separate workow procedures, e.g. create a vehicle combining the component-specic steps together. This approach enables the tester to create new test-cases very fast. However, he has to put a lot of efforts into creating those basic procedures rst. QF-Test comes with a Procedure Builder, which will create those basic procedures for you. Using the Procedure Builder will drastically decrease the efforts of recording and creating procedures for graphical components. So the tester can solely concentrate on his main focus, i.e. desigining the workow of the test itself and the according test-data.

18.2

How to use the Procedure Builder

For creating the basic procedure automatically, you have to perform following steps: Start the SUT from QF-Test.

18.3.

Conguration of the Procedure Builder

154

Navigate to the window or frame you want to create procedures for in the SUT. Press the Record Procedures button, select the according menu-item in QF-Test (247) or use the Hotkey for procedure recording . Perform a right mouse-click on the respective component in the SUT. Select the according recording-mode. Stop procedure recording by releasing the Record Procedures button, by deselecting the according menu-item in QF-Test or using the Hotkey for procedure (247) recording . Now you should be able to nd a newly created package containing the created (361) procedures for the components in the Procedures of the test-suite, where you have stopped the recording.

18.3

Conguration of the Procedure Builder

The act of building procedures is controlled by a template suite, which is located at qftest-3.5.3/include/procbuilderdef.qft. This le should be copied to any project-specic location, if you want to adapt it to your project. You can dene its location (247) in the options at Conguration le for recorded procedures . The template suite contains procedures for the most common GUI elements and actions. If you require other test-steps, you can add the according procedure to this testsuite. The le itself is a test-suite with a dedicated structure. You can nd a detailed expla(154) nation of this structure in the subsection section 18.3.1 . The denition le allows the tester to dene procedures for components of dedicated classes or to dene procedures for working with all components of one certain window.

18.3.1

The Procedure Builder denition le

The automated creation of basic procedures delivers different procedures depending on the components. A text-eld requires a setter procedure for setting its text, a button requires a press procedure for pressing it or a window could require a setter which calls the setter procedures of all text-elds or combo-boxes on that window to call just one procedure for using the window etc..

18.3.

Conguration of the Procedure Builder

155

Figure 18.1: The Procedure Builder denition le

The topmost package in the Procedures node is the name of the target package for the newly created packages. By default this is called procbuilder. This package will be (361) inserted below the Procedures , when you have nished recording procedures. If the package procbuilder already exists, a package procbuilder1 will be created and so on. The next level is the class level. Here you can dene a package per class. The package name represents the full class name, but with _ as separators instead of .. Thats because . is not allowed in package names. The Procedure Builder creates the procedures also for descendants of specied classes. In case the names of your classes contain a _, you have to mark this via \_. The following levels can be chosen freely because those levels are intended to structure the procedures. At the last level you have to dene the steps of the procedure itself. Of course there are a lot of variable data in that denition, e.g. like <COMPID>. Using those you can specify variables for the procedure names, like the current component-id or component-name. You can also record the current value of the text-eld or the current selected status of a checkbox. Its even possible to keep the package structure variable. For an overview of all possible variables, please see (712) chapter 41 .

(361)

Chapter 19 Interaction with Test Management Tools

3.0+

QF-Test contains some pragmatic test management approaches, like creating a testcase overview or documenting test-cases within QF-Test. In bigger projects it might be neccessary to make use of an own dedicated test management system to track the development status of test-cases or to link test-cases and their results to defects, usecases or features. Besides support for planning of test-cases and tracking their results a test management system could also contain a test execution engine, which supervises the occupation of test-systems during different test-runs. As QF-Test doesnt come with all of those features though continously improving in that area, it is very easy to integrate QF-Test with such a test management or test execution system using the QF-Test Batch mode or the QF-Test Daemon mode. For (133) more information about the Batch mode or the Daemon mode, please see chapter 16 . The following chapters describe some exemplary solutions which we provide for established test management systems. If you cannot nd your test management system in that list, please contact us or our mailing-list to get hints about a possible integration approach.

19.1
19.1.1

HP TestDirector for Quality Center

Introduction

The current integration of QF-Test and HP TestDirector for Quality Center utilizes the built-in VAPI-XP test type of Quality Center. It is intended to be an automated test-case for any test-tool. QF-Test comes with a template le for the VAPI-XP script, which is qcVapiXPTemplate.txt, see qftest-3.5.3/ext/qualitycenter. This script

19.1.

HP TestDirector for Quality Center

157

can be used as template for all QF-Test tests in Quality Center. Please see section (158) 19.1.2 for a detailed step-by-step description. The QF-Test template VAPI-XP script employs an external worker VBScript script, called qcTemplate.vbs. This script is also part of the QF-Test distribution (see qftest-3.5.3/ext/qualitycenter) and has to be adapted to your specic needs. So we encourage you to copy that le to a project specic location and adapt it according to your needs. The worker script launches QF-Test in batch mode on each test system locally, i.e. it has to be accessible for each test system. As the test suite les and the conguration les have to be available on the test system too, we recommend to put all those les on a shared network drive or into the version management system. After the execution of the test the run-log of QF-Test will be appended to the test instance as well as the status of the test will be set to the result. You can also change the worker script to make use of a daemon call (for details about (695) the daemon mode, please see chapter 40 ). In this case QF-Test will establish the network connection to the test system and launch the test by itself. In case of the normal batch call Quality Center establishes the connection to the test-system and triggers the local QF-Test installation to perform the test. If you make use of the daemon call, the worker script has to be located on the Quality Center system, but the test suite still needs to be accessible on each test system. If you do not make use of VBscript in your project, feel free to port the QF-Test demo scripts to JScript or any other supported language. The following gure shows the VAPI-XP test-case in Quality Center:

19.1.

HP TestDirector for Quality Center

158

Figure 19.1: QF-Test VAPI-XP test-case in HP QualityCenter

19.1.2

Step-by-step integration guide

Please perform the following steps on the test systems: Copy the template worker script from qftest-3.5.3/ext/qualitycenter/qcTemplate.vbs to your project location and rename it to a proper name. We recommend to use the same path on all test-systems. Perhaps you should use a shared network drive. Please perform following steps in Quality Center: Start Quality Center. Create a new VAPI-XP test-case. Copy the content of the template qftest-3.5.3/ext/qualitycenter/qcVapiXPTemplate.txt into Script Viewers text area. le the

Change the pathToModule variable in the script to the location you have copied the worker script qcTemplate.vbs to.

19.2.

Imbus TestBench

159

Change the testSuiteFile variable to your desired test-suite le. If you want to execute one specic test, you can also change the test variable to the desired test-case name. Change to the Test lab section in Quality Center and add the test-case to the execution grid. Then launch the test-case. Now Quality Center should start the test run on the test system. After the test has terminated, the run-log of the test will be uploaded as attachment to the test instance and the status of the test will be set to the according result. Please read the comments in the script carefully, because you can also use test-case specic settings optionally.

19.2
19.2.1

Imbus TestBench
Introduction

The current integration of QF-Test and the TestBench consists of two parts: Creating a QF-Test template le using the TestBench interactions. Importing QF-Test results into TestBench. You can nd a wrapper test-suite for launching qftest-3.5.3/ext/testbench/testbench.qft. TestBench scenarios in

The following section provides a short overview about the integration concept.

19.2.2

Creating QF-Test template from interactions

After planning your tests and designing the interactions in the TestBench, you can create a template QF-Test le using the QF-Test export plug-in for interactions. Imbus will provide all required information, how to install this plugin. After exporting the interactions you will nd all interactions as procedures and their structure as packages in the QF-Test le. Now you can start recording the respective steps in QF-Test and ll the empty procedures. The completed le has to be saved in your project-specic test-suite folder, because this le should be used as input le for the test execution later. We recommend to use a project-specic location, perhaps a shared network drive or the version management system.

19.3.

QMetry

160

19.2.3

Importing test-execution results

The tests will be executed by a wrapper test-suite which is capable of reading the iTEP export les of the TestBench. This specic wrapper test-suite has to include the template test-suite which has been created during the export of the interactions (for details see (159) the previous chapter section 19.2.2 ). QF-Test provides a sample qftest-3.5.3/ext/testbench/testbench.qft. wrapper at

Before you start, you have to export the required test-sets from the TestBench using the iTEP export plug-in. After exporting you can launch QF-Test using the wrapper test-suite. The wrapper contains variables specifying the iTEP export directory, the QF-Test run-log directory and the iTEP results During the test-run QF-Test will create one run-log per test-case and a summary run-log containing all test-runs at the end. When the test-run has been completed, you can import all those results using the iTEP or iTORX import plug-in into the TestBench. The single QF-Test run-logs will then be attached to the test-instances.

19.3
19.3.1

QMetry
Introduction

The current integration between QF-Test and QMetry relies on planning the tests and its steps within QMetry and forwarding the actual test-execution to QF-Test. Once the the test-run terminates the QF-Test run-log and its HTML report will be automatically uploaded to QMetry to the respective result area as well as the state of the test-case in QMetry will be set to the according result. You need to prepare your test-system in order to run QF-Test tests. Please perform the following steps: In the Admin area of the QMetry Testmanagement view install a test-execution agent at the Agent view. Download the required agent and conguration les to install the QMetry execution agent on your test-system. Install the respective QF-Test QMetry Launcher at your test-agent. Install and set-up a platform at the Platform view, which is also located in the Admin area of QMetry.

19.3.

QMetry

161

Congure QMetryAgent.properties correctly to use the required environment variables of QMetrys QF-Test wrapper. Congure QMetryConfig.properties correctly to show to the right QF-Test executable. Congure additional parameters for the QF-Test QMetryConfig.properties, see next section for details. call in

Launch the QMetry agent. Please do not launch the agent as Windows-Service to avoid running GUI-Tests within the service-session. If you launch the agent as service you should run the QF-Test tests via the QF-Test daemon, which shouldnt run in a service session then. After setting up the agent and launcher, you need to plan the test-execution. QMetry supports several ways of integrating QF-Test test-cases. You can nd all supported integrations in QMetrys integration guide document. Please perform following steps for a simple integration: In the Testmanagement view change to Test Cases and plan the test-cases there. At the individual test-case you have to set the value Test Script Name to the path of the required QF-Test test-suite holding the actual implementation of the test-case. The name of the test-case must be exactly the same as the specied value for the Id attribute in QF-Test. Add the test-case to an executable test-suite in the Test Suites view. Now you are ready to run the test-cases: Open the Test Suites view and select the required test-suite for execution. Select the Execute TestSuite tab. Run or schedule a test-run via assigning an agent to the Automation column. The next time when the local QMetry agent is polling the QMetry server it will get the necessary information to run the test-case. Once the test-run terminates you will nd the run-log of QF-Test and its HTML report attached to the Execution History of the executed test-suite. The state of the test-case will also be updated accordingly.

19.3.

QMetry

162

The following gure shows the Execution History tab in Test Suites holding the run-log:

Figure 19.2: QF-Test run-log in QMetry

You will nd a more detailed description of how to setup QMetry in the manual of QMetry and in QMetrys integration guide document.

19.3.2

Sample Conguration
following values in the conguration le

Its recommended to set QMetryConfig.properties:

Set the value of generic.adapter.success.code to 0,1. Set qftest.additional.arguments to -test ${QMTestCaseName} in case of local test execution. In case of using QF-Tests daemon set qftest.additional.arguments to -test ${QMTestCaseName} -calldaemon -daemonhost <testsystem> -daemonport <daemonport> . As already mentioned in the previous section, you need to use the same name for the test-case within QMetry and for the value of the Id attribute within QF-Test.

19.4.

Klaros

163

Further ways for integrating QMetry and QF-Test can be found in the integration guide document provided by QMetry.

19.4
19.4.1

Klaros
Introduction

Klaros is a test management tool developed and supported by verit Informationssysteme GmbH, Kaiserslautern, Germany. Klaros is available in two kinds of editions, a free community edition and an enterprise edition with an extended set of functionality, individual conguration options and full customer support. The current integration of QF-Test with Klaros comprises: Import of QF-Test results into Klaros.

19.4.2

Importing QF-Test results into Klaros

(124)

After creating the XML report le as discribed in chapter 15 , you can upload the results to Klaros. An example for a QF-Test import URL may look like this, where the result le is contained in the HTTP request body.
http://localhost:18080/klaros-web/seam/resource/rest/importer? config=P00001&env=ENV00001&sut=SUT00001&type=qftest& time=05.02.2013_12:00&username=me&password=secret

Example 19.1: Importing test results into Klaros

The curl command line tool can be used on Linux or Windows/Cygwin to trigger an import in a single command line.
curl -v -H "Content-Type: text/xml" -T "my_qftest_report.xml" \ "http://localhost:18080/klaros-web/seam/resource/rest/importer\ ?config=P00001&env=ENV00001&sut=SUT00001&type=qftest\ &time=05.02.2013_12:00&user=me&password=secret"

Example 19.2: Using curl command to import test results into Klaros

Further information can be found within the Klaros online manual at http://www.klarostestmanagement.com/les/doc/html/User-Manual.Import-Export.TestResultImport.html.

19.5.

TestLink

164

19.5
19.5.1

TestLink
Introduction

The current integration of QF-Test with the open-source tool TestLink consists of two parts: Generating template test-suites for QF-Test from the planned test-cases of TestLink. Importing QF-Test results into TestLink.
3.5.1+

If you use TestLink 1.9.4 or newer you can use the TestLink API for interacting with TestLink. The TestLink API requires a valid development key. Therefore open TestLink and go to My Settings. In the settings you can generate a development key by pressing Generate key under the API interface section. For TestLink 1.9.3 or older versions the integration mechanism accesses the database of TestLink directly. This approach requires a JDBC database driver to use the provided scripts. You can download those drivers from the web page of their providers. Exporting the planned test-cases including its test-steps from TestLink to QF-Test supports the test-creator to implement the test-cases exactly as planned. Importing the test results into TestLink provides a better overview over all executed manual and automated tests-cases in one tool.

Note

Test results can also be uploaded to TestLink without exporting them before. Therefore you have to take care, that the ID of the test-case from TestLink is part of the test-cases name in QF-Test. The name has to be called like this: <TestLink-ID>: Name of the test-case.

19.5.2

Generating template test-suites for QF-Test from test-cases

QF-Test offers the capability to generate template test-suites following the same structure as the planned tests in TestLink to guarantee a synchronized structure of automated tests and test planning. In the QF-Test le you can nd one Test-case node per test-case and one Test-set node per suite from TestLink. If you have specied the elds Steps and Expected Results of a test-case, the generating-script will also create an empty Test-step for each test-step in the according test-case. The expected result will be shown in the Comment attribute of the Test-step node. Now the template test-suite has to be lled by the test automation engineer with the according steps by adding QF-Test steps to the generated Test-step nodes.

19.5.
3.5.1+

TestLink

165

In case you use TestLink 1.9.4 or newer you need to perform following steps: 1. Take care that test automation is enabled in TestLink. Therefore set the respective enable_test_automation key to ENABLED in the conguration le config.inc.php. 2. Copy the folder qftest-3.5.3/ext/testlink/api to a project-specic location. 3. Open the launcher script you want to use with a text editor. The launcher scripts are exportTests.bat for Windows and exportTests.sh for Unix. 4. Adapt the paths of the variables JAVA, QFTDIR and TESTLINKINTEGRATOR. 5. Open the le TestLinkUserSpecifics.py with a text editor. 6. Adjust the variables serverurl and devkey. 7. If you want to export custom elds from TestLink, also adjust the variable custom_fields. 8. Run the adapted export script, like shown below.
exportTests.bat --testproject projectname --targetsuite /path/to/testsuite.qft

Example 19.3: Sample call of exporting test-cases from 1.9.4

If you use TestLink 1.9.3 or older, please perform those steps: 1. Copy the folder qftest-3.5.3/ext/testlink/export to a project-specic location. 2. Open the launcher script you want to use with a text editor. The launcher scripts are exportTestLinkToQFT.bat for Windows and exportTestLinkToQFT.sh for Unix. 3. Adapt the paths of the variables JAVA, QFTDIR and TESTLINKINTEGRATOR. 4. Open the le TestLinkDBIntegrator.py with a text editor. 5. Adjust the variables dbdriver, conncetionstr, dbuser and dbpass according to your database connection. 6. If you want to export custom elds from TestLink, also adjust the variable custom_fields.

19.5.

TestLink

166

7. Run the adapted export script, like shown below.

exportTestLinkToQFT.bat --testproject projectname --targetsuite /path/to/testsuite.qft

Example 19.4: Sample call of exporting test-cases till 1.9.3

19.5.3

Execution of test-cases

Executing the QF-Test tests can be performed as usual. But you should create a XMLreport at the end of the test-run, because the import mechanism is using this report. Therefore you have to use the -report.xml parameter during test execution. If you create the reports via the GUI, you have to check the checkbox Create XML report.
Note

In case you did not export test-cases from TestLink the ID of the test-case from TestLink has to be part of the test-cases name in QF-Test. The name has to be called like this: <TestLink-ID>: Name of the test-case.
qftest -batch -report.xml reportFolder testsuite.qft

Example 19.5: Sample execution to create a XML report

19.5.4

Importing QF-Test results into TestLink

After creating the XML report le, you can upload the results to TestLink. The import mechanism creates a new build for every test-run. The build number of TestLink will be created by the run-id of the QF-Test report. You can change the run-id, by setting the parameter -runid during executing the tests.
3.5.1+

In case you use TestLink 1.9.4 or newer you need to perform following steps: 1. Take care that test automation is enabled in TestLink. Therefore set the respective enable_test_automation key to ENABLED in the conguration le config.inc.php. 2. Copy the folder qftest-3.5.3/ext/testlink/api to a project-specic location. (If you have copied them already for exporting you can use the same les.) 3. Open the launcher script you want to use with a text editor. The launcher scripts are importResults.bat for Windows and importResults.sh for Unix. 4. Adapt the paths of the variables JAVA, QFTDIR and TESTLINKINTEGRATOR.

19.5.

TestLink

167

5. Open the le TestLinkUserSpecifics.py with a text editor. 6. Adjust the variables serverurl and devkey. (If you have adapted them already for exporting you can use the same values.) 7. Run the adapted import script, like shown below.
importResults.bat --testproject projectname --resultfile qftestReport.xml --testplan testplanname

Example 19.6: Importing test results into TestLink from 1.9.4

If you use TestLink 1.9.3 or an older version, please perform following steps: 1. Copy the folder qftest-3.5.3/ext/testlink/import to a project-specic location. 2. Open the launcher script you want to use with a text editor. The launcher scripts are importToTestLink.bat for Windows and importToTestLink.sh for Unix. 3. Adapt the paths of the variables JAVA, QFTDIR and TESTLINKINTEGRATOR. 4. Open the le ReportParser.py with a text editor. 5. Adjust the variables dbdriver, conncetionstr, dbuser and dbpass according to your database connection. 6. If you want to export custom elds from TestLink, also adjust the variable custom_fields. 7. Run the adapted import script, like shown below.
importToTestLink.bat --testproject projectname --resultfile qftestReport.xml --testplan testplanname --tester tester

Example 19.7: Importing test results into TestLink till 1.9.3

Chapter 20 Integration with Build Tools

Automated GUI testing is just one step in the build process of a software development project. The basic need for coordinating all those steps of compiling, testing, documenting and deploying of software has lead to a variety of build tools (e.g. make, ant, maven) and so called continuous integration tools (e.g. Jenkins/Hudson, Cruise Control, Continuum). In general, by use of QF-Tests command line interface as documented in chapter 16 (535) and chapter 32 a straight forward integration with those tools should be possible.
(133)

The following sections contain examples for integrations with a some of the tools mentioned above.

20.1

Ant

People who are using Apache Ant (http://ant.apache.org) as build system may easily integrate QF-Test in their build le:

20.2.

Maven

169

<project name="QF-Test" default="runtest"> <property name="qftest" location="c:\Program Files\qfs\qftest\qftest-3.5.3\bin\qftes <property name="logdir" value="c:\mylogs" /> <target name="runtest" description="Run a test in batchmode"> <echo message="Running ${suite} ..." /> <exec executable="${qftest}" failonerror="false" resultproperty="returncode"> <arg value="-batch" /> <arg value="-compact" /> <arg value="-runlog" /> <arg value="${logdir}\+b" /> <arg value="${suite}" /> </exec> <condition property="result" value="Test terminated successfully."> <equals arg1="${returncode}" arg2="0" /> </condition> <condition property="result" value="Test terminated with warnings."> <equals arg1="${returncode}" arg2="1" /> </condition> <condition property="result" value="Test terminated with errors."> <equals arg1="${returncode}" arg2="2" /> </condition> <condition property="result" value="Test terminated with exceptions."> <equals arg1="${returncode}" arg2="3" /> </condition> <echo message="${result}" /> </target> </project>

Example 20.1: Ant build le build.xml to execute a test-suite

The above example assumes the test-suite to be dened as property when running ant: ant -Dsuite=c:\Program Files\qfs\qftest\qftest-3.5.3\demo\carconfig\carconfig_en.qft.

20.2

Maven

People who are using Apache Maven (http://maven.apache.org) as build system may easily integrate QF-Test in their build. This can be achieved by using the antrun plugin of Maven. A demo pom.xml le, where QF-Tests tests are executed in the test phase could look like this:

20.2.

Maven

170

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <artifactId>testant</artifactId> <packaging>jar</packaging> <name>testant</name> <groupId>de.qfs</groupId> <version>1</version> <properties> <qf.exe>"C:\Program Files\qfs\qftest\qftest-3.5.3\bin\qftest.exe</qf.exe> <qf.reportfolder>qftest</qf.reportfolder> <qf.log>logFile.qrz</qf.log> <qf.suite>"c:\path\to\testsuite.qft"</qf.suite> </properties> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-antrun-plugin</artifactId> <executions> <execution> <phase>test</phase> <configuration> <tasks> <exec executable="${qf.exe}"> <arg value="-batch"/> <arg value="-report"/> <arg value="${qf.reportfolder}"/> <arg value="-runlog"/> <arg value="${qf.log}"/> <arg value="${qf.suite}"/> </exec> </tasks> </configuration> <goals> <goal>run</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </project>

Example 20.2: Maven build le pom.xml to execute a test-suite

In your project it might become required to run the tests during another build phase,

20.3.

Jenkins/Hudson

171

than the congured test phase in the example. In this case you have to congure the plugin accordingly, like described in the Maven documentation.

20.3
3.3+

Jenkins/Hudson

Jenkins/Hudson (http://jenkins-ci.org or http://hudson-ci.org) are continuous integration build tools. Therefore they are meant to control and monitor the build process within a software project. One important step in this build process is automated testing. There are number of benets to be gained when integrating QF-Test with Jenkins/Hudson: In case Jenkins/Hudson is already used for the continuous integration process, integration of automated GUI tests can be easily achieved. Easy-to-use administration of scheduled test runs and notication of results via email or RSS. Jenkins/Hudsons web based GUI provides good overview and control of test results. By use of the HTML Publisher Plugin it is possible to embed QF-Tests HTML reports directly into the Jenkins/Hudson GUI. Results generated during the test run such as run-logs and reports can be archived automatically. Therefore maintaining an own directory structure is not needed anymore. As Jenkins and Hudson share a common history, following chapters just use Hudson as representative.

20.3.1

Install and start Hudson

In general, current information on download and installation of Hudson can be found at http://hudson-ci.org. Installing Hudson just means to download the war Archive and start it e.g. via java -jar hudson.war. Another option without installation is to run Hudson by use of Java Webstart directly via https://hudson.dev.java.net/hudson.jnlp. As soon as Hudson is started its web interface can be accessed via http://localhost:8080. It should look like the following:

20.3.

Jenkins/Hudson

172

Figure 20.1: Hudson after start-up

20.3.2

Create a new job for QF-Test

To create a new job you just need to follow the respective link in Hudson (see above gure) and to select Build a free-style software project which leads you to a form containing all the build and report option. All options are documented (? buttons right beside each option), so lling the form is very easy. Please note the following specics related to QF-Test: QF-Test test execution can be dened as Hudson build step, e.g. as Windows (133) batch le or shell script. Please see chapter 16 how to execute QF-Test from (173) the command line. The QF-Test demo project, as described in section 20.3.3 , (555) contains respective example scripts also handling Exit codes for QF-Test to show test-runs with the correct passed/failed state in Hudson. (543) The usage of command line argument -exitcodeignoreexception now directly ignores exeptions, errors and warnings for the exit code. In case you want to use your own custom workspace, this can be congured in the

3.4.2+

20.3.

Jenkins/Hudson Advanced Project Options.

173

The option Archive the artefacts allows to keep given les, helpful e.g. for generated run-logs to allow a later error analysis. Another important option is Publish JUnit test result report. JUnit reports, as QF-Test is able to create when being invoked with the (547) -report.junit <directory> parameter, allow Hudson to generate a graphical trend chart as well as a short result summary for a test-run. Note that (288) only results of Test-cases are taken into account for JUnit reports. To be able to embed the comprehensive HTML reports QF-Test generates by use (547) parameter, the HTML Publisher of the -report.html <directory> plugin in Hudson needs to be installed. This can be easily achieved via HudsonManage HudsonManage Plugins . After reloading Hudson there should be a respective option available in the conguration form of your job.

20.3.3
Note

Install QF-Test Demo Project

Having the HTML Publisher Plugin installed, as described above, is recommended before starting with the demo. When Hudson is successfully installed and run once, there should be a directory .hudson (note the leading dot) available in you user directory (e.g. C:/Documents and Settings/Your User Name) in which all Hudson related les are stored. Among others it contains a folder called jobs holding all data related to a certain project i.e. conguration les and build results. To use activate the demo project in Hudson you simply need to extract the Zip archive QF-Test_Demo_Job.zip from qftest-3.5.3/ext/hudson of your QF-Test installation into your .hudson/jobs directory. To make the project now visible in Hudson you might need to restart it or you simply use HudsonManage HudsonReload Conguration from Disk . After choosing the now available QF-Test demo job you should see a page like the following:

20.3.

Jenkins/Hudson

174

Figure 20.2: QF-Test Demo Job in Hudson

The test result trend chart as well as the latest test result summary are generated from the QF-Test JUnit reports. Thanks to the HTML Publisher Plugin also HTML reports are directly available. You may now have a look at the various builds or start a new one yourself via Build Now . The Conguration link allows you to inspect and possibly adapt the build options. They might prove useful as basis for your own projects.
Note

3.4.2+

The batch le runtest.bat and the shell script runtest.sh, located in the (555) qftest_demo directory, contain handling of the Exit codes for QF-Test to show test-runs with the correct passed/failed state in Hudson. (543) The usage of command line argument -exitcodeignoreexception now directly ignores warnings for the exit code. For troubleshooting the consol output within the build details is a good point to start.

20.3.
Note

Jenkins/Hudson

175

On Linux you can replace the Execute windows batch command by a Execute shell build step with following content:
cd "$WORKSPACE/qftest_demo" ./runtest.sh

Example 20.3: Hudson Execute shell build step for QF-Test demo job

20.3.4

Requirements for GUI tests

At least when it comes to distributed GUI testing there are requirements to consider but they are valid in general. Hudson allows execution of tasks on remote machines. This is of course also relevant for GUI testing. Due to its nature GUI tests are typically not intended to run on the central build server. In addition, tests might need to be executed for different environments, operating systems and SUT versions. On a remote machine, a Hudson slave agent needs to be launched in order to connect to the Hudson server and wait for jobs to be processed. As described in the Hudson documentation, there are several options to launch this agent, e.g. as windows service or via Java Webstart. For GUI tests it is vital to have an active, unlocked user session. Therefore it is not possible to start the agent via a windows service but a real (test) user must be logged in e.g. via auto login using Windows Autostart to launch the Hudson agent. Also screen locking needs to be disabled.
Note

Please see also FAQ 14 for more technical background details.

Chapter 21 Performing GUI-based load tests

21.1 Background techniques and comparison with other

In addition to functional and system tests, QF-Test can also be used to perform smallto-medium sized load tests, stress tests or performance tests. The idea is to test the performance of some server by running a number of clients concurrently. There are many different ways for setting up and performing load tests, most of which do not work with real clients. Instead they directly make use of the protocol between the client and server, e.g. by sending HTTP request or performing RMI or other kinds of remote procedure calls. There are a number of pros and cons for protocol-based or GUI-based load testing: Protocol-based testing uses very little resources at the client side, so it can easily scale up to the breaking point of the server without requiring too much hardware. GUI-based tests incur the full memory and performance overhead for each client, which can be quite signicant, especially in case of Swing-based rich clients. Besides QF-Test itself, a typical PC can sustain between 5 and 20 parallel SUT clients without signicant loss of performance on the client side, depending on available RAM and the size and activity of the clients. Rich clients typically represent a complex user interface which correlates with a certain complexity of the client/server API. Creating protocol-based tests that cover most of that API can be quite an effort. On the other hand, GUI-based tests that have already been implemented for functional testing may be available for reuse. If not, it is still much easier to automate complete use-cases with QF-Test than at the protocol level. With GUI-based testing, actual end-to-end response times (the time span from

21.2.

Load tests with QF-Test

177

user action until the result shows up) are measured, while protocol-based tests measure only the times for the server call. Either can be useful, depending on the situation. In summary, GUI-based load testing can be very useful and efcient - especially if functional tests can be reused - provided that either the number of clients that need to be simulated is not too high, or that sufcient hardware is available for the client side.

21.2

Load tests with QF-Test

QF-Test is probably unique with its ability to drive several SUT clients in parallel on a single desktop without these clients interfering with each other. To that end, QF-Test lters hard events coming from the operating system and prevents them from reaching the SUT at the wrong time, most notably when popup menus are shown or when a component must not lose keyboard focus. Without that functionality, interference between SUT clients on the same desktop would quickly break any test.

21.2.1

Running tests with parallel threads

To avoid the additional overhead of having to run one QF-Test instance per SUT client, (553) can be used to run QF-Test in the command line argument -threads <number> batch mode with multiple threads for testing. The same test-suite will be executed in each thread. One runtime license is required per thread. If insufcient runtime licenses are available, full development licenses will be used unless the command line argu(549) ment -runtime is given which forces use of runtime licenses only. Please see the (539) documentation for the -runtime command line argument in section 32.2 for further information about runtime licenses. All threads are independent and each thread will create and control its own set of processes and SUT clients. To that end, QF-Test appends the thread identier to the (400) Client attribute of any node that needs to refer to a SUT client. Thus it is impossible to access one threads SUT client from another thread. Two variables have special values when running with multiple threads: ${qftest:threads} expands to the number of threads given with -threads <number> (default is 1 when running single-threaded) and ${qftest:thread} is the index of the current thread (default is 0). Running multiple clients on one desktop can sometimes lead to delays longer than usual, especially during startup of the SUTs. Therefore, timeouts should be increased (411) (484) generously, especially in Wait for client to connect and Wait for component to appear

21.2.

Load tests with QF-Test

(265)

178

nodes. Also, the options under Replay->Timeouts (see section 29.3.5 ) should be increased to at least twice their default values. It is a good idea to use a separate set of system options for load testing by specifying the conguration le with the command (551) line argument -systemcfg <file> .

21.2.2

Synchronization

To get consistent results, it may sometimes be necessary to coordinate the tests in the parallel threads, either to make sure that all clients access the server simultaneously, or to prevent just that. Threads can be synchronized with the help of a Server script script: rc.syncThreads("identifier", timeout) The identier is a name for the synchronization point and timeout is the maximum time in milliseconds to wait for all threads to reach the given synchronization point. If not all threads are expected to reach that point because they follow different execution paths, the additional parameter count can be given to identify the number of threads to expect, e.g. rc.syncThreads("case1", 120000, count=3) If the timeout is exceeded without the expected number of threads reaching the syn(526) chronization point, a TestException is thrown. To log an error instead of raising an exception, set the optional parameter throw to 0 (default value 1), e.g.: rc.syncThreads("case1", 120000, count=3, throw=0)
(390)

node with the following

21.2.3

Coordinating tests on multiple machines

If more than one machine is required to sustain the desired number of clients, tests can be run in parallel on multiple machines. Each QF-Test instance on each machine can use multiple threads as long as sufcient licenses for the total number of threads are available. Currently QF-Test has no special support for coordinating the start of QF-Test instances on different machines. This has to be done manually, through a script or through some third party tool. However, QF-Test does support synchronization of threads across multiple machines. The syncThreads method has one additional parameter called remote which defaults to 0 (no remote synchronization) and represents the number of QF-Test

21.2.

Load tests with QF-Test

179

instances to synchronize. Each instance will rst synchronize its running threads internally, then use network calls to synchronize with other running instances. Only when all threads in all requested machines have reached the synchronization point the test will continue. For example, to synchronize all threads in tests on 4 machines after SUT startup, add a Server script node at the end of the main setup sequence with something like: rc.syncThreads("startup", 300000, remote=4) It is a good idea not to hard-code the number of remote instances to expect. Instead, use a variable dened at test-suite or system level that defaults to 0 so the tests will run ne on a single machine. When running on multiple machines, override that variable on (52) the command line as explained in chapter 7 .

Chapter 22 Executing Manual Tests in QF-Test

3.0+

22.1

Introduction

QF-Test is primarily a tool for the creation and execution of automated tests. However, it is rarely possible - or economical - to automate 100% of the required tests for a project. In most projects some manual tests need to be performed as well. One of the biggest challenges in testing a project is consolidating the different results and reports of automated and manual testing to get an overview about the execution status of all tests. To facilitate reporting the results of manual test execution along with those of automated testing, QF-Test now offers the capability of tracking manual tests from within itself. The steps to be performed during a manual test have to be dened in an Excel le which is read by a test-suite called ManualTestRunner.qft. This test-suite is provided along with a sample specication le in the directory demo/manualtester below the QF-Test installation directory. The test designer has to specify each step in that Excel le including the expected result. After stepping through the manual tests QF-Test provides the usual results - a run-log, HTML and XML reports. Additionally, a newly created Excel le with the results of the respective test-run is created. For a detailed description (181) please see section 22.2 . The dialog used for the test execution is called ManualStepDialog and looks like this:

22.2.

Step-by-step Guide

181

Figure 22.1: Example for a ManualStepDialog

The title of the dialog shows the name of the test-case. The detailed step description and the expected result are shown in the rst two text-boxes. After performing the test the tester has to specify whether the test succeeded or not. In case the test failed the tester also has to enter the received result which is intended to show the differences between the actual and the expected result. This dialog can also be used for your own (717) purposes, see section 42.1 .

22.2

Step-by-step Guide

Please perform the following steps on your system to launch a manual test from QFTest. Copy the denition Excel le from qftest-3.5.3/demo/manualtester/SampleTestDescription.xls to your project location and rename it to a suitable name. We recommend to use the same path on all test-systems. Perhaps you can make use of a shared network drive.

22.3.

Structure of the Excel le

182 from your

Also copy the execution test-suite qftest-3.5.3/demo/manualtester/ManualTestRunner.qft project location. You may want to rename it as well. Open the Excel le and dene the test-steps.

to

After saving the changes to the Excel le, open the execution test-suite and adapt the global variable testFile variable to target your specic Excel le. Turn off the QF-Test debugger. It would only interfere with the steps of the manual tester. Start the test-suite via selecting the test-suite node and pressing Start test-run. QF-Test will now read the data from the Excel le and open a dialog containing the rst test-step. Enter the result of the test-step and proceed with executing each test-step. At the end of the test execution QF-Test will write a new Excel le containing the test description and the according results. You can also store the run-log of that execution or create an HTML report. Please read the comments in the test-suite and Excel le carefully, because you can adapt this concept according to your needs. It is even possible to start only specic tests.

22.3

Structure of the Excel le

The Excel le has a specic structure which allows you to describe the manual teststeps quite exibly. The meaning of the columns is explained in the following table:
Column TestCase Type of Test Comment Short Description Step Description Expected Result Description A unique identier for each test-case. If the step belongs to the same test-case as the previous step, just leave this column empty. Optional denition of the kind or function of the test or step, e.g. a functional test or a usability test, startup, etc. An individual comment for the test-case. This comment will be shown in the runlog of QF-Test. A short description about the content of the test. The detailed description of the manual step. The description of the expected result of that test-step.

Table 22.1: Description of the Excel le for the denition of manual tests

22.4.

The ManualTestRunner test-suite

183

The Excel le with the results of the manual test execution will contain two additional columns as follows:
Column Received Result State Description The result the tester received during test-execution. If a test-step fails, the tester must specify a received result. The state of the test, i.e. PASSED, FAILED, CANCELED or SKIPPED.

Table 22.2: Description of the Excel le with the results of manual tests

22.4

The ManualTestRunner test-suite

The ManualTestRunner.qft test-suite contains some global variables at suite-level which provide ne-grained control over test-run. These are explained in the following table. All variables not listed here are used internally by the test-suite and should not be changed.
Global Variable testFile testSheet resultSheet tests Description The path to the test-step denition Excel le. The worksheet of the Excel le containing the test-steps. The name of the worksheet for the results. A list of tests to be intended to execute. If this variable is empty, all tests will be executed. If you want to execute only test 5 and 6, you can specify 5,6 or 5-6. It is even possible to specify things like: 1,3-5,7 to execute the tests 1, 3, 4, 5 and 7. The default selection of the state. You can set it either to PASSED or FAILED. All other states will be converted to FAILED. The heading of the column containing the test-case number. The heading of the column containing the comment. The heading of the column containing the short step description. The heading of the column containing the full step description. The heading of the column containing the expected result. The heading of the column containing the received result. The heading of the column containing the state of the test.

defaultState testCaseColumn commentColumn shortDescColumn stepDescColumn expResultColumn recResultColumn stateColumn

Table 22.3: Description of the global variables in the ManualTestRunner test-suite

22.5.

Results

184

22.5

Results

An executed test-step can be set to one of the following states:

Result PASSED FAILED CANCELED SKIPPED Description The test-step was successful. The test-step failed. The test-step was canceled. The test-step was skipped.

Table 22.4: States of manual test-execution

Part II Best Practices

Chapter 23 Introduction
This part of the manual describes best practices based on lessons learned from several customer projects and user feedback. The concepts described should assist you in nding the best strategy for using QF-Test in your projects.
Note

QF-Test is a very generic tool. The hints and experiences described here are just suggestions from our point of view, which we hope will support you in working efciently and successfully with QF-Test in your project. But they are just one way of doing things and you will have to nd your own solution that works best for your specic project. In case you are interested in how other customers use QF-Test you can post a question to our mailing list which you can reach via http://www.qfs.de/mailman/listinfo/qftest-list.

Chapter 24 How to start a testing project

This chapter talks about the most important aspects that should be considered before you start to use QF-Test widely in your testing project. It mostly raises questions and gives general answers with references to more detailed information. The aim of this chapter is to provide hints about issues which you should take care of in order to make your GUI tests reliable, stable, repeatable and especially maintainable.

24.1

Infrastructure and testing environment

Before you start creating and running automated tests you should think about some general matters pertaining to the environment where the tests have to run. In order to make tests reliable and repeatable you have to take into account that you must be able to bring your SUT into a well-dened state, which includes the state of its backend, e.g. a server and/or a database. If you do not think about such aspects it might become very difcult and sometimes quite tricky to re-run a test or simply to analyze test-results and maintenance of tests can become a nightmare. Please consider the following topics: 1. What is the initial state of your SUT? Which user is the actual user running the tests in your SUT? Most projects work with dedicated test-users for running tests. Another approach could be to have one test-user per test engineer. Which language setting of your SUT is the primary one? Is it really required to reach a full coverage of all supported languages or is it sufcient to run the bulk of the tests in one primary language and create only a few tests to specically test localization? In most cases repeating tests in several languages just

24.1.

Infrastructure and testing environment

188

covers the same functionality, so you gain no real new information after running them. However, unless you take precautions, the language setting will (204) inuence the component recognition of QF-Test, please see chapter 27 for details. 2. What is the initial state of your database? Can you work with an extra test database or do you have to use a production database? Test databases contain test-data with designed and planned content whereas production databases contain real-life data. Is the latter predictable and reliable? What about the danger that tests can mess with and possibly destroy production data? If at all possible you should avoid running automated tests in a production environment. Can you clean up or reset the environment after one test-run for re-running the test? Is it possible to undo changes in the database or is it required to use new test-data for the next regression phase? How can you read or write test-data? Do you want to use standard SQL scripts or can you reuse libraries from developent? Some projects even reinstall the whole database before every test-run because they cannot reuse any test-data or clean the database correctly. 3. Do you want to integrate QF-Test with other tools, e.g. build tools or test management tools? How to integrate QF-Test with a test management tool? If you can reuse already planned test-steps you can avoid redundant work in planning tests and creating them. For the standard integration for such tools, please see (156) chapter 19 . Should tests be launched by a build-tool? If you have created tests you can run them unattended and trigger the run by a build-system like Ant or (133) CruiseControl. Please see chapter 16 for details about test-execution. Should test-results be uploaded to a reporting system or into a test-management system or is it more sufcient to put the HTML reports and run-logs on a centralized HTTP-server? 4. Who will work with QF-Test? Do only one or two engineers work with QF-Test or do all developers and business testers participate in test development? You can nd some hints (197) about working in a team with different roles in section 25.5 . What are the skills of the engineers? It is recommended to have at least one dedicated person with a good QF-Test knowledge in the team, who is also capable of implementing scripts and understanding software development principles.

24.2.

Location of les

189

Of course there will be more issues to take care about which are specic for your project. Try to gure them out.

24.2

Location of les

You should also think about following aspects of saving or installing les: 1. Where to install QF-Test to? QF-Test can be installed locally on every system but this forces you to update every system manually whenever you need to upgrade to a new version. You can also install QF-Test on a shared network drive, if your (190) network is reliable, see section 24.2.1 for details. 2. Where to store the conguration le qftest.cfg? Among other things that le contains information about how QF-Test should recognize components or what should go into the run-log. These options have to be the same for every QFTest user, otherwise you cannot share tests in your team. To ensure that you can either use a shared network installation for QF-Test or specify the cong le via command-line parameters when launching QF-Test. Make sure the shared cong le is write-protected unless you explicitly want to change it. For details, (5) see section 1.4 . 3. Where to store the license le license? You should put the license le to a central place in order to update the license only once when you receive an update for it. Again, you can either have a shared network installation for QF-Test or can use command-line parameters to specify the location of that le when launching (4) QF-Test. For details, see section 1.3 . 4. Where to store the test-suites? The best place to store test-suites is a version management system where you can track the changes and access any version of the les. If this is not possible you should store them on a shared network drive. 5. Where to store the test data les? Test data les are associated with test-suites so you should store them closely to the suites, i.e. either in the same version management system or on a shared network drive. 6. Where to store the HTML reports and run-logs? You should put those les in a centralized place where any engineer can take a look at them to evaluate the tests-results. Most people tend to use an HTTP server or a shared network drive for that.

24.3.

Component Recognition

190

24.2.1

Network installation

If you plan to install QF-Test on a shared network drive you have to take care about some specic things. The main source of conict is the system settings le qftest.cfg. It is actually a good (and necessary) thing to have all users use the same system settings, especially for the recognition options. Sharing the system settings le facilitates this. However, this le should be made read-only so that one user will not inadvertently change the system settings for everyone. If the le is read-only, QF-Test will not save the system settings upon exit. Any change to these settings will have to be made by explicitly making that le writable, then exiting QF-Test and then making it read-only again. Alternatively each user could specify a different system settings le via -systemcfg but thats not advisable. If you test a Java/SWING application and you uncheck the option Connect without JDK instrumentation, then the list of instrumented JDKs could become another source of troubles. That le is stored centrally. But as each user may have different JDK versions or directories that le could get confused. However, you can simply just let that list become a superset of all JDK directories on all systems. Whether or not a JDK is instrumented on a given machine is always determined at runtime when the JDK instrumentation dialog is opened. The running QF-Test instances will also share the log directory (internal logging, not a problem) and the Jython package cache which can occasionally cause problems so that QF-Test cannot initialize its Jython interpreter. This doesnt happen often and can be xed by clearing (not removing) the Jython cachedir. For Windows, each user should also execute the setup.exe for the primary QF-Test version, located in the installed qftest-x.y.z directory, to get proper registry settings and documentation links on his machine. In the rare case when a QF-Test patch overwrites existing jar les of QF-Test, running instances based on those jars may crash on Windows.

24.3

Component Recognition

The most important aspect of a GUI testing tool is a stable and reliable recognition of the graphical components. In that area QF-Test is very exible and can be congured in several ways. In most cases the default conguration for the component recognition works well, but sometimes you may have to change it. If you change the component recognition options after creating lots of test-cases, those test-cases may break. Therefore you should try to nd the most appropriate settings for your project as early as possible. It is worth spending time in that area before starting

24.3.

Component Recognition

191

to implement a huge amount of tests because in the worst case you might have to rerecord or at least update all or most of the existing test-cases after a critical change of the recognition options. Best start by recording some demo test-cases and gure out how QF-Test recognizes (35) your SUTs components. The recognition mechanism is described in chapter 5 and (48) chapter 6 . If you re-run those demo test-cases - ideally on different versions of your SUT - and run into recognition problems, you have to ask yourself following questions about those tests: 1. Are there enough synchronization points, like Wait for component to appear or Check nodes with timeouts to execute test-steps only if the SUT is ready for them? (a) Sample 1: After opening a window you can only work in that window, if it is really there -> Use a Wait for component to appear node. (b) Sample 2: After pressing on a search button, you can only continue with the test when the search is really over -> Use a Check node with a timeout. Another important aspect besides synchronization points is the correct approach of recognizing components. You have to ask yourself the following questions to determine, which recognition approach might be the most appropriate one: 1. Do the developers use unique and stable names for their components? Please (507) take a closer look at section 30.13 . 2. Perhaps it is sufcient to use a regular-expression for the Feature attribute of the component of the main window under the Windows and components node. Please (208) see section 27.3 for details. 3. If development did not set useful or even dynamic names it may be required to (204) implement a NameResolver. Please take a closer look at chapter 27 . 4. Perhaps problems come from recording changing component classes, typically (210) resulting from obfuscation. Please see section 27.5 for further information. 5. Do any of the QF-Test recognition options need to be changed? These are de(204) scribed in chapter 27 . 6. Is it possible to use generic components? See section 27.6
(212)

for details.

In some cases it is enough to change the default conguration. Let us assume the developers have set unique and stable names for the target components, i.e. buttons, textelds, checkboxes etc. In such cases it may be sufcient to just change the Name override mode setting of QF-Test to Name overrides everything. This setting tells QFTest to ignore any changes in the component hierarchy and just work with the target components and the window directly.

24.3.
Note

Component Recognition

192

You have to change this option in two places: Once at Record -> Components -> Name override mode and at Replay -> Recognition -> Name override mode. See (204) chapter 27 for more details.

Chapter 25 Organizing test-suites

One of the most challenging tasks in a project is keeping the test-suites maintainable over a long period of time. Especially if some window or workow changes signicantly the maintenance effort should be kept to a minimum. It is also worth thinking about how to minimize the creation efforts of tests that contain a lot of similar or even the same steps. A typical use-case is launching the SUT or the login process or a very important and basic workow like navigating to a certain part of the SUT. Another aspect to think about is how to efciently organize test-suites if different people work in your testing project.
3.5+

In any case you should create your test-suites within a QF-Test project as described in (81) chapter 10 . This feature provides a better overview over your test-suites and directories. The following sections show some best practices how to keep your tests maintenable, extensible and well-organized.

25.1

Organizing tests
(75)

In chapter section 9.2 we describe the concepts of Test-set and Test-case nodes. A Test-case node stands for one dedicated test-case and its test-data. A typical Test-case could be derived from a use-case, a requirement or a defect description in your environment, e.g Calculate the price for vehicle xyz at 10% discount for the JCarCongurator application. Test-set nodes are collections of Test-sets and Test-cases which can be used for organizing test-cases, e.g. Tests for calculating prices. Test-step nodes represent the individual test-steps of a Test-case like Open window or

25.2.

Modularization

194

Check calculation. If you have an external description of the Test-case or any other associated information which might be important for it, it is recommended to add an HTML link to it in the Comment attribute of the Test-case. You will see that link in the report later. It is even possible to create a separate test documentation using the menu action FileCreate HTML/XML testdoc . More details about documentation can be found in (124) chapter 15 .
3.1+

The report and test documentation also contain the Test-step nodes which are used in a Test-case. If a Test-case consists of lots of Procedure calls or Sequences, you should organize the single test-steps in Test-step nodes. Those Test-step nodes have the advantage that you can really see every signicant step in the QF-Test window and also in the report later. If you want to put several nodes into a Test-step you can pack the respective nodes into a Test-step via selecting them, perform a right-mouse click and selecting Pack nodesTestStep .

25.2

Modularization

One of the most important concepts for effective test automation is modularization. Modularization here means placing reusable sequences in a dedicated location and calling these whenever possible. This concept enables you to create a sequence only once and reuse it as often as you require it in your tests without re-recording the same steps all the time. Changes in the SUT that require an update of the tests that rely on such a sequence, e.g. a change to some basic workow, can then be handled by updating just the procedure in a single location instead of many identical sequences spread all over the test-suites. The modularization concept in QF-Test is implemented via Procedure nodes. Procedures (77) are well described in section 9.3 . If you have lots of test cases, it is best to have almost every test-step as a Procedure and create those procedures up front, if possible. With those procedures in place you can ll your test-cases very fast by just adding the respective Procedure call nodes. In larger projects it is useful to have Procedures at different level, e.g. component specic procedures like Click OK and workow oriented procedures like Create a vehicle.

25.3.

Parameterization

195

25.3

Parameterization

The concept of modularization enables you to maintain test-steps at a single location in your test-suites. But how to use different test data for different tests? If you have a Procedure that can be called with different test data, e.g. a typical Login process with name and password or the Select accessory procedure of the JCarCongurator, you can use variables within the QF-Test nodes. Those variables should be used at places where test data is usually being accessed. In most cases variables will be used for Text input nodes or for selections of items in a list or a table or tree nodes. If a procedure requires variables you should dene the required variables in its list of (354) Variable denitions . This is to ensure that you get a list of all required parameters whenever you add a respective Procedure call node for that Procedure to your tests. Some customers even set dummy default values for parameters so they can recognize immediately when a parameter has not been initialized by the calling test. The next step is to move those variables from the Procedure call either into the (293) Variable denitions section of the Test-case node or to put the test data into a Data driver node with a Data table or using an external data source. The usage of variables and parameters is well described in section 9.3 . Parameters (79) can also be created automatically, please see section 9.3.4 . You can nd more details (116) about the data driver concept for loading test data from a data source in chapter 14 .
(77)

25.4

Working in multiple test-suites

Up to now you have read about using the modularization and parameterization concept to avoid unnecessary and redundant work in the creation process of your tests. You should have recognized that those concepts will reduce the maintenance efforts of your test-suites by changing or updating only one single sequence instead of several ones. But we still do not know how to organize our work for different test engineers or for a very large project with a lot of GUI elements. The answer for an effective working organization comes again from the software development area and it is to use several libraries for different areas and different responsibilities. Importing other test-suites into a test-suite enables you to follow that encapsulation approach. A typical organization of test-suites in your project could look like this:

25.4.

Working in multiple test-suites

196

Figure 25.1: Structure of multiple test-suites

Level 0 is the level that contains test-steps (i.e. Procedures) which are required for nearly all test-cases in your projects. Such test-steps could be Launch SUT or Perform the login. Level 1 contains test-steps for a specic part of the SUT. For the JCarCongurator you can think about a test-suite Vehicles containing Procedures like Create a vehicle, Remove a vehicle and another test-suite Accessories, which contains Procedures like Create an accessory or Remove an accessory. Level 2 is the test-case level. It contains Test-cases and Test-sets for the respective area of your software, e.g. Tests for vehicle creation or Tests for accessory creation. Of course you could also have a test-suite like Integration tests which refers to test-steps from different test-suites at level 1 and level 0. Level 3 is the so called scenario level. Those test-suites usually just contain Test calls to level 2 and stand for different scenarios within your test project, e.g. Nightly test scenario, Defect verication scenario or Quick-test build verication.
Note

The structure described in this document is of course just one possible solution for handling test-suites in a project and is not a strict rule you have to follow. You could also think about splitting level 1 into a GUI-element level and a workow level or merge level

25.5.

Roles and responsibilities

197

2 and level 3 to one level. Which structure you nally implement also depends on the experience and knowledge of the test-engineers working in your project. The including area of level 1 test-suites looks like this:

Figure 25.2: Including test-suites of level 1

You can nd a detailed description of hot to include test-suites in section 17.1 (584) section 36.7 .
(202)

(145)

and

In section 26.5 you can nd a step-by-step description how to extend an already (201) existing test-suite and in section 26.4 you can nd strategies of handling components in such a scenario.

25.5

Roles and responsibilities

(195)

If you take a closer look at the organization shown in the previous section section 25.4 you may recognize that it is also possible to organize your test-suites based on different knowledge levels of test engineers.

25.5.

Roles and responsibilities

198

Figure 25.3: Structure of different test-suites with roles

Level 0 and level 1 require a good knowledge in working with QF-Test but not a deep knowledge of the SUT. On the other hand level 2 and level 3 require a very good knowledge of the SUT and the planned test-cases but those engineers usually do not require a very deep knowledge of QF-Test as long as they just use procedures from level 0 and level 1. Test engineers working in level 0 and level 1 should be capable of implementing scripts or control structures like the Try/Catch concept which enables them to create strong and powerful test libraries. At least the engineers working on level 0, but also recommended for engineers working on level 1, should have a good knowledge about compo(35) (48) (204) nent recognition in QF-Test. Please see chapter 5 , chapter 6 and chapter 27 .
Note

Even if you are working alone on a project it is strongly recommended to split the tests and procedures into different levels because maintenance will become easier than with everything kept in one huge test-suite.

25.6.

Managing components at different levels

199

25.6

Managing components at different levels

(195)

If you follow the approach suggested in the previous section (section 25.4 to dene where the components belong. There are two possibilities: 1. Put all components into level 0. 2. Split the components like the procedures into several levels.

) you have

Storing all components in level 0 is the most simple solution but this could cause you to update level 0 very often, just because one single component in your project changes. You have to assign responsible persons to keep that structure cleanly. In big projects you may consider storing the common components like the login dialog or the main frame menus, that are important for everyone, in level 0. Components specic to a certain area, e.g. a dedicated vehicle dialog, appear only in the test-suite that holds the procedures operating on those components. The workow for moving components between test-suites is described in section 26.4 (202) and the workow for extending existing test-suites is described in section 26.5 .
(201)

25.7
3.5+

Reverse includes
(286)

of test-suites belonging to a QF-Test You dont need to care about Dependencies (81) project as described in chapter 10 , because QF-Test automatically resolves them. If you work in different test-suites in your project you might sometimes want to rename a Procedure or a Test-case. If you do that you may encounter some troubles in updating the references to that Procedure or Test-case in other test-suites. If you want to keep the other les also being updated after renaming such an element you have to maintain the (286) Dependencies attribute of the root node of the library test-suite. If you follow the approach described in section section 25.4 , you should ensure that level 0 contains a reverse include to level 1, level 1 should contain one to level 2 and level 2 should contain another one to level 3. A sample from the provided demo test-suites is (197) shown in gure 25.2 .
(195)

Chapter 26 Efcient working techniques

This chapter will help you in optimizing your working techniques and avoid unnecessary steps when working with QF-Test.

26.1
3.5+

Using QF-Test projects

The previous chapter describes the creation of several test-suites. As you can image the amount of test-suites will increase over time. You can get a better overview over them using a QF-Test project. QF-Test projects show all involved test-suites in a very nice way. Furthermore projects automatically take care of propagating modications to referring test-suites. You can (81) nd more information about projects at chapter 10 .

26.2

Creating test-suites from scratch

In the previous chapter we described the concept of creating maintainable test-suites by utilizing procedures and variables within QF-Test. Normally people start recording very long sequences and splitting those into smaller parts or even procedures later. It is very hard to split up long sequences as you have really to walk through the whole one to nd proper boundaries. Another disadvantage is that you cannot see parts which have already been implemented in existing test-cases or procedures. Instead of that described workow we recommend to plan the tests and their test-steps including procedures rst. Then you can start recording procedure by procedure. We came to the conclusion that anticipatory recording and creation is very helpful especially for working in bigger teams. A typical workow for creating those procedures looks like this:

26.3.

The standard library qfs.qft

201

1. Plan the required procedures rst. 2. Also plan the required package structure rst. 3. Record every procedure as a separate sequence. 4. Rename the recorded sequence like the procedure should be called. 5. Transform the recorded sequence into a Procedure, Transform node into... action from the context-menu of QF-Test. 6. Move the Procedure to the correct location. 7. Replace test data with variables, either manually or by using the parameterizer (79) (see section 9.3.4 ) 8. Add the variables to the Variable denitions of the Procedure node, possibly specifying default values. 9. Describe the procedure in its Comment attribute, see section 9.4 . An alternative approach of creating procedures is the automated creation provided by (153) QF-Test. This concept is described in chapter 18 .
(80)

using

the

26.3

The standard library qfs.qft

QF-Test provides the standard library qfs.qft which is included per default in every test-suite. This suite contains many useful procedures for accessing components, the le system or a database. Please take a look at that library before you begin implementing something that has already been solved by us.

26.4

Component storage

QF-Test records new components in the test-suite where you press the Stop recording button. Therefore it could happen that you record components in the wrong test-suite. If you want to move those components into another test-suite you must always use FileImport from the target suite. Take care to ensure that both suites belong to the same project or to specify correct Include les/Dependencies relations in those test(147) suites. This workow is described in detail in section 17.2 .
3.1+

For cleaning your component structure in a test-suite you can rst import the test-suite

26.5.

Extending test-suites

202

into itself. Then you can select the Windows and components step, open the context menu via a right mouse-click and click at Mark unused components... . You will get a list of all components which are not used in the project. If you are sure that those components can be removed, then select the Remove unused components in the context menu of the Windows and components step.

26.5

Extending test-suites

Several workows can be followed to extend existing test-suites: 1. Simply record and extend the target test-suite directly. 2. Work with a scratch suite as described in chapter 17
(145)

If you extend testsuites directly via clicking at Stop recording in that suite, then you have to care about the recorded components. In case you have changed the recorded component hierarchy under Windows and components the newly recorded components will be recorded in the normal hierarchy again and you have to move the new components to the optimized hierarchy. Another aspect is that it could become very difcult for moving single components into another test-suite under the right location there. If you work in a scratch suite you can create the new test-steps temporarily in a completely new test-suite and import the recorded components and the created procedures and test-cases into the target suites together. A detailed workow for extending a test-suite from level 1 (from section 25.4 look as follows: 1. Create a new test-suite. 2. Add the test-suite to be extended to the Include les area of the new one. 3. Save the new test-suite. 4. Ensure that both test-suites belong to the same project or add the new test-suite to the Dependencies area of the test-suite to extend. 5. Record the new test-steps in the scratch suite. Also create Procedures, if required. 6. Then import the components, procedures and tests into the target test-suite as (201) (148) described in section 26.4 and section 17.3 . A more detailed description of working in multiple test-suite can be found in chapter (145) 17 .
(195)

) could

26.6.

Working in the script editor

203

26.6

Working in the script editor

The script editor of QF-Test contains some fancy features to avoid too much typing actions. If to call methods of the run-context rc, you can simply type rc. and press you want Ctrl-Space , then you will get a list of all available methods. The auto-completion is also working for several variable names, which are:
Variables doc frame iw node Options rc resolvers Just press Ctrl-Space without typing anything Methods Methods of a DocumentNode. Methods of a FrameNode. Methods of the ImageWrapper. Methods of a DOMNode. The keys and values of QF-Test options to set. Methods of the Run-context. Methods of the Resolvers module. A list of all variables with auto-completion.

Table 26.1: List of variables with auto-completion.

Chapter 27 How to achieve robust component recognition

The most important feature of a GUI test-tool is the recognition of the graphical components. QF-Test offers a lot of conguration capabilities in that area. This chapter gives an overview over the most common strategies and settings to make the component recognition as stable as possible in a project. We also recommend to take a closer look at section 30.13
Note
(507)

and section 36.2

(579)

You have to gure out the component recognition strategy before you start utilizing QFTest for a large testing area in your project. Otherwise you can get into serious troubles (187) in maintaining your tests. Some hints for doing that are described in chapter 24 .

27.1

Using the default conguration

In most of the projects the default conguration works very well. The default conguration takes the name of a component, its class (type) and also its hierarchy (structure of window) into account for recognizing a component. You will get troubles using this conguration, if The developers do not specify unique names. The component hierarchy changes a lot. The names change (see section 27.4
(209)

). ).

The classes change (see section 27.5

(210)

If the developers do not use any names, QF-Test uses other properties, like the label, the text of a button or the geometry for the recognition of a component. In this case

27.1.

Using the default conguration

205

the default conguration can also work if those labels, the component structure and the geometry stay stable. In this case your tests should make sure that the windows of the SUT are always opened with or resized to the same size. The default conguration for recording components looks like this:

Figure 27.1: Default conguration for component recording

The default conguration for recognizing a component during playback looks like this:

27.2.

Using the Name overrides everything mode

206

Figure 27.2: Default conguration for component recognition

27.2

Using the Name overrides everything mode

If your developers use stable and unique names for all important components which can be accessed by a test, e.g. buttons, tables or text-elds, you can consider changing the conguration to the Name overrides everything mode. This has the advantage that only names of those components and the window they belong to will be used for recognizing the component. Changes to the SUTs component hierarchy will the have no effect on the recognition of those components based on their unique name. This setting will cause trouble if:

27.2.

Using the Name overrides everything mode

207

The developers do not use unique names. The names change (see section 27.4
(209)

). ).

The classes change (see section 27.5

(210)

The Name overrides everything conguration for component recording looks like this:

Figure 27.3: Name overrides everything conguration for component recording

The Name overrides everything conguration for recognizing a component during playback looks like this:

27.3.

Using regular expressions for working with dynamic window titles

208

Figure 27.4: Name overrides everything conguration for component recognition

27.3

Using regular expressions for working with dynamic window titles

In a lot of applications you will face the situation that the developers do not use unique names and QF-Test keeps recording the same components again and again in different places. Playback with previously recorded components may still works unless the window geometry changes signicantly. In this case it is very likely that the title of the main window changes frequently, e.g. to display a version string, a user name, a le name or some other variable information.

27.4.

Inuencing the names by implementing a NameResolver

209

If you want to keep your tests working and prevent recording multiple variants of this window and all its components, you have to select the respective Window node in the and edit its Feature attribute to replace the dynamic parts of that title with a regular expression. Be sure to check the Use regexp box. Now your tests should work again. Here you see the use of a regular expression for a component of the JCarCongurator. Its Feature attribute has to start with Edit followed by an optional dynamic part:

Figure 27.5: Using a regular expression in the Feature attribute

QF-Test uses regular expressions at many places. You can nd detailed information at (581) section 36.4 to learn more about how to use them.
Note

QF-Test assigns the Id of a component automatically and may use the Feature attribute (38) to generate it like described in section 5.3 . The Id attribute is an articial concept for QF-Tests internal use to map actions onto recorded components. Thus it can be pleasing to change it for better readability afterwards. When changing the Id QF-Test provides the opportunity to adapt all references automatically.

27.4

Inuencing the NameResolver

names

by

implementing

In GUI testing projects you can face a lot of interesting naming concepts. Sometimes the components in an application have no names, but the testers know an algorithm how to name them reliably. Sometimes existing names change from time to time or

27.5. Handling class changes due to re-factoring by using Record system-classes only

210

are completely dynamic, e.g. you can get a name button1 after the rst recording and after the second recording you get button2. Another situation could be that the current version of the application is part of the name of a dialog window. In such cases you should take a closer look at the NameResolver interface of QF-Test. A NameResolver can be used to change or remove names set by developers for the QF-Test perspective. They are only removed for QF-Test not from the real source code. You can think about utilizing NameResolvers in following cases: The SUT has dynamically changing names. You know a method to set the names uniquely. You want to map names to other names (e.g. due to new versions or for testing other languages.) You want to tune the names of components, e.g. to remove some parts and get nicer component IDs in QF-Test. If you can achieve per-window uniqueness of names with the help of a NameResolver you can also think about switching to the Name overrides everything setting described (206) in section section 27.2 .
Note

Whenever possible it is preferable that developers set the names directly in their source code as they best know the context of that component. Implementing a NameResolver can become an excruciating task if the developers change the content of the GUI a lot. NameResolvers are well described in section 39.1
(634)

27.5

Handling class changes due to re-factoring by using Record system-classes only

You can congure QF-Test to record the system base classes of components instead of custom class names.System classes are javax.swing... and java.awt... classes for Java/Swing and org.eclipse.swt.widgets... classes for Java/SWT applications.

27.5. Handling class changes due to re-factoring by using Record system-classes only

211

Figure 27.6: Option to congure recording of system classes only

If you set this option, you will be safe against re-factoring steps of the developers who may change the class structure or rename classes. This option will also protect you from obfuscators which change the name of classes. In most cases you can work with this option as it could always happen that a class name changes and it is denitely better to be on the safe side instead of having problems, which could be solved easily. However, you should not work with this option if the classes are important for the recognition of components, i.e. if development did not assign useful names but implemented different custom classes for specic containers that QF-Test can use to partition the set of components of the SUT.
3.1+

A mixed approach is also possible by implementing a ClassNameResolver. This can be useful you only want to get system classes for some components, but not all, e.g. when (636) using a third-party library. Please see section 39.1.2 for more details.

27.6.

Avoiding recording every component or using generic components

212

27.6

Avoiding recording every component or using generic components

By now you know how to optimize the component recognition for your needs, but you still have to record every component before you can use it in your tests. As you can imagine, it can be tiring to record each and every component required for a test. Sometimes this may even become impossible, e.g. when you have changing identiers or changing labels. A typical situation could be localization testing of an application. Another situation could be the usage of a GUI framework of your developers. You could have a lot of very similar dialogs, where only some components are different. But you still have to record all parts, which are actually the same, like navigation buttons. QF-Test offers a feature to reduce the recording of components to a minimum. This concept is called Generalizing components. It consists of making use of variables in the components details or simply to remove dynamic parts of it. The general approach for generalizing components is as follows: 1. Record some components that you want to generalize and compare them. 2. Create one generic component with an ID, which contains generic so that you can recognize it later. 3. Remove all attributes that you do not want to use in the recognition in that generic component. 4. Determine the main properties that you want to use for recognition, e.g. Name, Feature or Class index. 5. Set a variable for the respective attribute, e.g. $(name). 6. To prevent false positive matches, disable recognition based on geometry by specifying - for the X and Y attributes. 7. Specify @generic in the Comment attribute for the generic component to prevent it from inadvertently being removed by the Remove unused components action. 8. Create a procedure accessing the generic component and use the variable(s) from the previous step as parameter(s).
Note

Generic components are very handy for replaying tests, but QF-Test will not use them for recording. You will always get the real component recorded in the test-suite, but can then change the created sequence to make use of the respective generic component.

27.7.

Switching component recognition settings dynamically

213

27.7
3.1+

Switching dynamically

component

recognition

settings

Since QF-Test version 3.1 it is even possible to change component recognition options dynamically. Thus you are very exible and you can combine different settings for your SUT. Setting options at runtime can be done via a Server script or SUT script - depending on the option - and calling rc.setOption(OPTION_NAME, OPTION_VALUE). All possible (233) (248) options can be found in section 29.2 for recording and in section 29.3 for replaying tests.

Chapter 28 Test execution

This chapter gives some hints about how to implement your tests to get stable and reliable test execution.

28.1

Dependencies

The Dependencies concept of QF-Test provides functionality to guarantee that all prerequisites for a test-case are fullled before running it. It is also capable of reacting to unexpected behavior, e.g. closing an error dialog, which pops up and blocks your tests. The concept is described in section 30.3 in the chapter Dependencies.
(316)

and a use-case can be found in the tutorial

You should at least implement a Dependency which is responsible for launching the SUT, containing a Setup for launching, a Cleanup for a normal exit and a Catch to react on any unexpected behavior.
Note

If you implement a Cleanup, try to close the SUT normally rst and only if the SUT does not terminate correctly, kill it via Stop client. For SWING and SWT applications please use the procedures qfs.cleanup.swing.closeAllModalDialogs and qfs.cleanup.swt.closeAllModalDialogsAndShells from the standard library qfs.qft for closing unexpected error dialogs.

28.2

Timeout vs. delay

Instead of using the Delay before and Delay after attributes you should try to use QF-Tests synchronization nodes to optimize test execution time.

28.3.

What to do if the run-log contains an error

215

The rst kind of synchronization nodes are the waiter nodes like Wait for component to appear, Wait for client to connect, Wait for document to load and Wait for process to terminate. You can specify the Timeout attribute to wait for a component, process or document. The Wait for component to appear node even provides the functionality to wait for the absence of a component. The second kind are the check nodes which allow you to specify the Timeout attribute as well. Those nodes can be used to continue the test when a GUI element of your SUT has reached a dened state.

28.3

What to do if the run-log contains an error

If the test report contains an error message or exceptions, the following steps should be performed to nd the source of that failure very fast: 1. Analyze the run-log, especially the screenshots and any other messages. 2. If you cannot nd the cause immediately, jump to the failing location in your testsuite by typing Ctrl-T in the run-log. 3. Set a breakpoint before or at the failing step. 4. Ensure that the debugger of QF-Test is enabled. 5. Run the failing test. 6. When QF-Test reaches the breakpoint and stops, open the debugger window and check the active variable bindings to wee whether they contain any wrong values. 7. Perhaps at that time you can also see the error immediately in your SUT. 8. If you cannot see any source of that error, run the failing step. 9. If you still encounter errors you might have to re-debug some steps executed before the failing step. Use the Continue execution from here menu entry to jump to previous steps instead of re-running the whole test again.
3.1+

Since QF-Test version 3.1 it is possible to mark nodes via the context menu-item Set mark or setting bookmarks for specic nodes via the menu item Add bookmark . These features enable you to nd important nodes very fast again. If you encounter problems with component recognition, please see section 5.9 (204) chapter 27 .
(45)

and

Part III Reference manual

Chapter 29 Options
Since QF-Test is a tool that is intended for a wide range of applications, the one size ts all approach doesnt quite work. Thats why QF-Test has a great number of options that control its functionality. There are two kinds of options for QF-Test: user options and system options. User options adjust the behavior of QF-Tests own GUI while system options inuence how tests are recorded and replayed. Each user has its own set of user options whereas (5) system options are saved in a common system le. See section 1.4 for details about conguration les.
3.1+

Many options can have their value changed at run time from a script via rc.setOption (589) as described in section 37.6 . Depending on whether the option takes effect in QFTest itself or in the SUT, the documentation for those options shows a Server script name or SUT script name matching the constant from the Options class. Obviously (390) (392) the option has to be set in a matching Server script or SUT script node. Where the options value can be selected from a drop-down list, the documentation also lists the constants that can be specied as the options value. Though the number of options may look daunting, dont let yourself be deterred by it. All options have reasonable default values, so QF-Test works well out of the box for most cases. However, if you nd you need to change something or simply want to explore the range of QF-Tests abilities, this chapter is for you. The options can be set in the dialog available through the menu item EditOptions... . The settings are saved in two conguration les, one for personal settings and one for (5) system-wide settings (see section 1.4 ).

Options

218

Figure 29.1: Options tree

29.1.

General options

219

To get at an option, rst select the appropriate node of the tree. The options for that topic are then displayed in the right part of the view. When switching from one group to the other, the current values are veried but not adopted yet. This happens only after conrmation with the OK button.

29.1

General options

This is the node for general QF-Test settings.

Figure 29.2: General options

Ask before closing (User) When a test-suite or a test run-log has been modied, QF-Test asks whether it should be saved before it closes its main window. That query can be suppressed by turning off this option. Be warned that auto saving is not implemented yet, so you may lose data if you forget to save before closing. Ask before overwriting (User)

29.1.

General options

220

When you try to save a test-suite or a run-log or generate a report, pgkdoc or (460) testdoc or save the image of a Check image over an existing le or directory, QF-Test asks for conrmation unless you turn off the option for the respective type of le. Restore last session on startup (User) If this option is set and QF-Test is opened in the workbench view, the previous session is restored by loading previously opened test-suites and selecting the previously selected node in each suite. If one or more test-suites are specied on the command line, this are loaded in addition to the previous session and receive the initial focus on startup. Discard empty suite after loading (User) A common situation in daily work is that a test-suite is loaded right after starting QF-Test. In that case the initial empty test-suite is typically unwanted and even a burden. Setting this option will cause the initial suite to be closed automatically. Number of recent les in menu (User) The File menu offers quick access to recently used test-suites or run-logs. This option determines the maximum number of recent le entries in the menu. Default script language for script nodes (User) This option can be set to either Jython or Groovy and determines the default (391) (390) attribute of newly created Server script or setting for the Script language (392) SUT script nodes. Default character encoding for Jython (System) This option sets the default encoding for Jython scripts in Server script and (392) SUT script nodes as well as the interactive Jython terminals for QF-Test and the SUT. The default encoding for standard Jython is ascii. To ease migration and improve two-way-compatibility with older QF-Test versions, the default setting for QF-Test is latin-1. You can also experiment with other settings that better suit your locale. For background information please see http://docs.python.org/library/codecs.html.
Note
(390)

This setting does not affect the encoding for Python modules in separate les. If you get an error that a module cannot be loaded because no encoding is declared, you probably need to add a line of the form

29.1.

General options

221

# coding:

latin-1

close to the top of that le. Please see http://www.python.org/peps/pep-0263.html for details. Use old-style GNU regexps (from before QF-Test version 3.1) (System) Server or SUT script name: OPT_USE_GNU_REGEXP Since version 3.1 QF-Test uses the standard Java regular expression syntax by default. In case this causes problems with regular expressions in older tests, you can switch back to using the GNU regexp package via this option. For further (581) information about regular expressions please see section 36.4 . Use native le chooser on Windows systems (User)
3.5+

Server script name: OPT_USE_NATIVE_FILECHOOSER On Windows systems the native le chooser is more advanced and more convenient to use than the Swing le chooser so QF-Test uses the native one by default. In case you prefer the Swing le chooser you can get it back by deactivating this option.

29.1.1
3.5+

Projects

There are several options that inuence the way QF-Test manages and displays projects.

Figure 29.3: Projects

29.1.

General options

222

Always open a test-suites project (User) If active, the project to which a test-suite belongs is automatically opened along with the suite. Project refresh interval (s) (User) The interval at which a project automatically gets completely refreshed. You can . To refresh the refresh a directory at any time by selecting it and pressing F5 complete hierarchy below the selected directory, press Shift-F5 instead. Number of test-suite to open in one go without warning (User) From the project tree you can open all test-suites contained in one directory hierarchy in one go. If you accidently select too many test-suites, QF-Test will rst issue a warning with the number of test-suites, allowing you to cancel that action. This option determines the threshold for that warning. Project les and directories to exclude (System) In many cases a directory hierarchy holds les and directories that dont really belong to a project, most notably sub-directories created by version control systems like subversion or cvs. In this option you can specify patterns for les and directories to generally exclude from projects. The patterns used here are not regular expressions but a simpler form often used by development tools: An * stands for any number of characters up to the next le separator - for compatibility reasons only forward / is used - while ** means 0 or more characters of any kind, including /. Every pattern is relative to the root directory of the project. Some examples: **/.svn All directories named .svn at any depth. **/.* All directories starting with a . at any depth. deprecated A directory named deprecated directly below the project root.

29.1.

General options

223

29.1.2

Editing

These options are used to congure various settings regarding editing in the tree or detail view.

Figure 29.4: Editing

Warn when modifying a test-suite that cannot be saved (User) If saving test-suites is prohibited, for example when working without a license, QF-Test will warn you that you will not be able to save your changes when a test-suite is modied for the rst time. Deactivating this option suppresses that warning. Ask before implicitly accepting detail modications (User) A very common mistake made while editing a test-suite is to forget pressing OK after making changes in the detail view of a node before switching to some other node, running a test, etc. If that happens QF-Test can either accept the modied values automatically or ask for conrmation by popping up a dialog with the detail view. The following options are available:

29.1.

General options Always Dont accept values implicitly, always ask for conrmation. Only if values are suspect or invalid

224

Try to accept values implicitly as long as they are valid and not suspect. Currently being suspect is dened as having leading or trailing whitespace which can lead to subtle problems which are very hard to locate. Never Accept all valid values implicitly without asking for conrmation. This option doesnt change the effect discarding your modications of explicitly with the Cancel button or by pressing Escape . Number of undo levels per suite (User) This option lets you set the number of edits that can be undone in a test-suite or run-log. Intelligent scrolling in trees (User) The default methods for interacting with Swing trees are not ideal. Moving the selection around causes a log of unnecessary horizontal scrolling and Swing has the tendency to scroll trees to a position where little context is visible around the selected node. Because tree navigation is essential for QF-Test, some of these methods are implemented differently to provide a more natural interface and to make sure that there is always enough context visible around the selected node. However, your mileage may vary, so if you dont like the alternative methods you can switch back to the default Swing way of things by deactivating this option. Minimum font size (pt) (User) This option lets you set the minimum font size (as point value) used within QF-Test. A change in this value becomes operative after restarting QF-Test. Show line numbers in script nodes (User) If this option is set, line numbers are shown in SUT script nodes.
(392)

and Server script

(390)

29.1.

General options

225

Show symbols for tabulator and linebreaks (User)

3.5+

If this option is set, QF-Test shows symbols for tabulator and linebreaks in tables and relevant textareas. Check references before deletion (User)
3.4+

If this option is set, QF-Test searches for references of nodes before nodes will be deleted. If references can be found, they will be shown in a dialog. Ask after changing component-ids or use default (User)
3.5.3+

If this option is set, QF-Test asks whether the user wants to update the component-ids of any referring node after the id of a component has been changed. If this option isnt set QF-Test updates all references in case of unique component-ids. Ask after changing callable names or use default (User)
3.5.3+

If this option is set, QF-Test asks whether the user wants to update the callable names (i.e. procedures, packages, tests and dependencies) of any referring node after the name of a callable node has been changed. If this option isnt set QF-Test updates all references in case of unique names.

29.1.3

Bookmarks

Here you can edit your bookmarks, a list of les and nodes that can be accessed quickly via the menu FileBookmarks .

Figure 29.5: Bookmarks

Though you can also create new bookmarks manually, it is preferable to use the menu item FileAdd to bookmarks to add a bookmark for a whole test-suite or run-log or

29.1.

General options

226

to select Add to bookmarks in the context menu of a node in a test-suite to add a bookmark for this sepecic node.

29.1.4

External tools

The following options determine which external programs are called by QF-Test.

Figure 29.6: External tools options

External editor command (User) Scripts can be edited in an external editor by pressing Alt-Return or by clicking the button above the text area. The contents of the text area are then saved to a temporary le and the external editor is run to edit that le. It is recommended to dene a name for the script before opening it in the external (227) editor (see also Warn when running external editor without le name ). Otherwise a random number is chosen as le name, which makes it difcult to distinguish several scripts opened in the external editor. Changes made to an external le are picked up automatically by QF-Test. Depending on your settings, you may get a warning message when this happens (see (227) Warn when test-suite is changed by an external editor ). In case you are tempted to edit your script code parallel in the internal QF-Test editor: These changes are also saved in the temporary le. Editors like jEdit on their part are smart enough to detect the change and reload the le automatically.

29.1.

General options

227

This option determines the external editor command to use. There are two variants, the plain name of an executable le or a complex command including options. The latter is distinguished by the string $(le) which is the placeholder for the name of the temporary le. Additionally, $(line) may be used to pass the current line number to the editor as well.
Note

The $(le)/$(line) syntax is used simply to avoid yet another different convention for variable attributes. No standard QF-Test $(...) variable expansion is taking place. Plain commands need never be quoted. Examples are: emacsclient notepad C:\Program Files\Crimson Editor\cedt.exe Complex commands on the other hand may need to use quotes, especially on windows. QF-Test takes care of quoting the $(le) argument itself: C:\Program Files\eclipse-3.6\eclipse.exe launcher.openFile $(le) javaw.exe -jar C:\Programs\jEdit4.2\jedit.jar -reuseview $(le) C:\Program Files\Crimson Editor\cedt.exe $(le) xterm -e vi +$(line) $(le) If this option is left empty, the value of the environment variable EDITOR is used, if it is dened when QF-Test is started. Warn when test-suite is changed by an external editor (User) Display a warning message when changes to a script made by an external editor (226) are picked up by QF-Test (see also External editor command ). Warn when running external editor without le name (User) Display a warning message when a script without name is opened in an external (226) editor (see also External editor command ). External imaging program (User) The Image of a Check image node can be edited in an external imaging program. The image is saved to a temporary PNG le and the external imaging program is run to edit that le. When nished editing, the le must be saved and the program exited. QF-Test will read the image back from the temporary le.
(463) (460)

29.1.

General options

228

This option determines the program to use for the operation. There are two variants, the plain name of an executable le or a complex command including options. The latter is distinguished by the string $(le) which is the placeholder for the name of the temporary le.
Note

The $(le)/$(line) syntax is used simply to avoid yet another different convention for variable attributes. No standard QF-Test $(...) variable expansion is taking place. Plain commands need never be quoted. Examples are: gimp mspaint C:\Windows\System32\mspaint.exe Complex commands on the other hand may need to use quotes, especially on windows. QF-Test takes care of quoting the $(le) argument itself: gimp no-splash $(le) C:\Windows\System32\mspaint.exe $(le) HTML browser (Unix only) (User) On Unix systems this option allows you to set the HTML browser for the context sensitive help system. You can specify a complex command using $url as placeholder for the URL to show, e.g. netscape -remote openURL($url) or just a simple command like firefox in which case the URL is passed as the last argument. On Windows the system browser is used. PDF reader (Unix only) (User) On Unix systems this option allows you to set the program used to display PDF les. This is necessary to open the PDF manual from the Help menu. On Windows the program associated with the .pdf extension is used.

29.1.5

Backup les

Unless told to do otherwise, QF-Test creates backups of existing les when saving testsuites or run-logs. These backup les are useful only in protecting against failures when

29.1.

General options

229

saving a le. They are by no means a replacement for proper system backups. The following options determine if, when and how backup les are created.

Figure 29.7: Backup le options

Create backup les for test-suites (User) Backup les for test-suites are created only if this option is activated. Please be careful and dont turn it off without a good reason, such as using a version control system for test-suites which obviates the need to create backups. Just think about the amount of work that goes into creating a useful test-suite and imagine the frustration if it gets destroyed accidentally. Create backup les for test run-logs (User) Usually a run-log is far less valuable than a test-suite, so there is a separate option that determines whether backups are created for run-logs. Backup frequency (User) There are two possibilities for the frequency with which backup les are created. With the rst option, One backup per session, QF-Test creates a backup le only the rst time a le is saved. If you continue editing the suite or run-log and save

29.1.

General options

230

it again, the backup le is left unchanged. Only when you edit a different le or restart QF-Test, a new backup is created. This setting is useful is you keep only one backup per test-suite. If, on the other hand, you keep multiple backups per suite, Backup on every save may be the preferred choice. Name of the backup le (User) Like many other things, the conventions for the names of backup les differ between Unix and Windows. While the common extension for a backup le under Windows is .bak, there are many variants under Unix. One of the most common is appending a tilde character . Number of backup les to keep (User) You can keep more than one backup le for each test-suite or run-log. If you do so, backup les are named after the scheme .bak1, .bak2... for the .bak naming style and 1, 2... for the other. The most recent backup is always numbered 1. When a new backup is created, the old number 1 is renamed to 2, 2 renamed to 3 and so on. When the maximum is reached, the oldest les are deleted. Auto-save interval (s) (User) Interval after which a modied test-suite is saved automatically. Setting this value to 0 will disable auto-saving. Otherwise values less than about 20 seconds are not useful. Run-logs are never saved automatically. Auto-save les are created in the same directory as the test-suite or - in the case of new suites that have never been saved - in the directory .qftest under the users home directory.

29.1.

General options

231

29.1.6

Library path

Figure 29.8: Library path option

Directories holding test-suite libraries (System) This is a list of directories that are searched for test-suites whenever a suite reference is given as a relative path that cannot be resolved relative to the current (356) attribute of a suite. This includes direct suite references in the Procedure name (355) (514) (515) Procedure call or a Component Id reference as well as suites included (286) (285) through the Include les attribute of the Test-suite node. The include directory belonging to the current version of QF-Test is automatically and invisibly placed at the end of the library path. This ensures that the common library qfs.qft can always be included without knowing its actual location and that its version is matching the version of QF-Test at all times.
Note

is given it will override the If the command line argument -libpath <path> settings of this option. In interactive mode, the value of the command line argument is displayed here, but it will not be saved with the system conguration unless it is modied.

(545)

29.1.

General options

232

29.1.7

License

Figure 29.9: License options

Normally QF-Test license bundles contain a homogenous mix of GUI engines. For example, a bundle of QF-Test/swing licenses only supports the AWT/Swing engine, QFTest/suite licenses support both AWT/Swing and SWT for all included licenses. For these kinds of simple licenses these license settings can be ignored. A small problem arises in case of mixed engine licenses where some GUI engine is included only for a part of the licenses. An example for this is a license bundle that was formerly purchased for qftestJUI, upgraded to QF-Test/suite with QF-Test 2.0 and then extended with further QF-Test/swing licenses, say two licenses for QF-Test/suite and another two for QF-Test/swing. Such a license allows running four concurrent instances of QF-Test, but only two of these can make use of the SWT engine. If more than two instances are started with SWT support there will be a license conict. When QF-Test detects such a mixed license bundle for the rst time it asks the user which engine licenses to use. The choice made then can be changed here at any

29.2.

Recording options

233

time. Besides, QF-Test can be started with the command line argument (543) -engine <engine> to override the supported GUI engines for this execution.

29.2

Recording options

The following options determine which kinds of events are recorded and which lters are applied, how components are recorded and arranged, and how sub-items are handled.

Figure 29.10: Recording options

Hotkey for recording (User) SUT script name: OPT_RECORD_HOTKEY Event recording can be directly started/stopped by pressing a key in the SUT. The . key to be used for this function is set through this option. The default key is F11 Keep variables in recorded client names (System) This option is very useful if the client name assigned to your SUT contains variables (e.g. $(client)), which generally makes sense when creating procedures. If this option is set, the client attribute of all recorded nodes is set to (400) (399) the unexpanded Client value of the Start SUT client node through which the SUT was started. Insert recording at current selection (User) Depending on what you are currently working on, it may or may not make sense to add newly recorded sequences directly at the current insertion mark. If you

29.2.

Recording options deactivate this option, new recordings are placed in a new Sequence (315) Extras .
(307)

234 under

Hotkey for checks (User) SUT script name: OPT_RECORD_CHECK_HOTKEY To simplify recording a sequence of events with interspersed checks, you can switch between plain recording and recording checks by pressing a key in the SUT. The key that triggers this switch is set through this option. The default key is . F12 Highlight components when checking (User) SUT script name: OPT_RECORD_CHECK_HIGHLIGHT When QF-Test is recording checks, it can give visual feedback on the component the mouse is currently over by inverting its foreground and background colors. Rarely this may have unwanted visual side effects, so you can turn that feature off with this option. Old style image grab (from before version 2) (System)
Swing

SUT script name: OPT_RECORD_CHECK_IMAGE_OLD_STYLE (460) When recording images for Check image nodes, non-opaque components used to be recorded with a black background. Thus, the image could deviate from what the user was seeing. This error has been corrected and normally the background is now recorded correctly. If this option is activated, the old, broken mechanism is used instead which may be useful if many checks for non-opaque components have already been recorded using that method.

29.2.1

Events to record

These options specify which kinds of events are recorded and which arent. You should not tinker with these unless you know what you are doing.

29.2.

Recording options

235

Figure 29.11: Options for events to record

Abstract Mouse click events (System) Activating this option causes a sequence of MOUSE_MOVED, MOUSE_PRESSED, MOUSE_RELEASED and MOUSE_CLICKED events to be recorded as a single (420) Mouse click pseudo event (see section 30.8.1 ). Abstract Keystroke events (System) This option lets you record a sequence of KEY_PRESSED, KEY_TYPED and KEY_RELEASED events (or just KEY_PRESSED and KEY_RELEASED for function (423) keys) as a single Keystroke pseudo event (see section 30.8.2 ). Record MouseEvents without coordinates where possible (System) SUT script name: OPT_RECORD_REPOSITION_MOUSE_EVENTS For many types of components and sub-items it doesnt matter where exactly a MouseEvent occurs. However, if large values are recorded for the X or Y (420) coordinate of a Mouse event , theres the danger that the event might miss its target upon replay if the component has shrunk a little due to font changes or because the window has been resized. This is also a possible source for (352) problems when converting a recorded sequence to a Procedure with variable target components. If this option is activated, QF-Test ignores the coordinates of recorded MouseEvents if it thinks that the coordinates dont matter for the target component, e.g. for all kinds of buttons, for menu items, table cells, list items and tree nodes. For the latter QF-Test distinguishes between clicks on the node itself and on the expand/collapse toggle. When MouseEvents without coordinates are played back, QF-Test targets the center of the respective component or item except that the X coordinate for items is limited to 5 because item bounds cannot always be calculated correctly.

29.2.

Recording options
(484)

236 (System)

Convert opening of a window into Wait for component to appear

When replaying a sequence during which a new window is opened, it may be useful to allow for a longer than usual delay until the window is opened. By activating this option, a recorded WINDOW_OPENED event will be turned into a (484) Wait for component to appear node automatically.
Web

For web clients this option causes a Wait for document to load node to be inserted whenever loading of a document completes. This is important for proper synchronization when navigating to another page.

(488)

29.2.2

Events to pack

In order to keep the amount of raw event data generated during normal use of a Java GUI manageable, QF-Test employs a set of recording lters and packers. These do their best to keep everything needed for successful replay and throw away the rest. To get an impression of the actual data behind a recorded sequence, try recording a (235) (235) short sequence with all of these turned off, and with Mouse click and Keystroke pseudo events disabled.

Figure 29.12: Options for events to pack

MOUSE_MOVED events (System)

29.2.

Recording options

237

SUT script name: OPT_RECORD_PACK_MOUSE_MOVED MOUSE_MOVED events are especially frequent. Every mouse cursor motion generates a handful of these. Under most circumstances, only the last of a consecutive series of MOUSE_MOVED events is actually useful, so all events except the last one are dropped, if this option is activated. An example where this is not advisable is recording some freehand drawing in a graphics application.
Note

One might think that MOUSE_MOVED events are completely useless, since MOUSE_PRESSED or MOUSE_RELEASED events have their own set of coordinates, but this is not the case. Some Java components require a MOUSE_MOVED event before a MOUSE_PRESSED event is recognized. MOUSE_DRAGGED events (System) SUT script name: OPT_RECORD_PACK_MOUSE_DRAGGED MOUSE_DRAGGED events are like MOUSE_MOVED events, but with one mouse button held down. Similarly, only the last of a series of MOUSE_DRAGGED events is recorded unless you turn off this option, except for special cases (see next option). Mouse drag hover delay (System) SUT script name: OPT_RECORD_MOUSE_DRAGGED_HOVER There are situations where not only the nal target of a mouse drag is of interest, but intermediate points as well. The most common is invoking a menu item in a sub-menu.

Note

As of QF-Test 1.05.2, the following no longer applies because MOUSE_MOVED or MOUSE_DRAGGED events that are required for opening sub-menus are not optimized away anymore. However, there may be other situations where intermediate stops are useful when recording drags.

29.2.

Recording options

238

Figure 29.13: Dragging to a sub-menu

As illustrated above, creating a new Test node for a suite could be done by clicking on the Insert menu button, dragging the mouse to the Test and Sequence nodes item for the sub-menu, so the sub-menu pops up, then dragging on to the Test menu item and releasing the mouse button. Normally such a sequence would be reduced to a press, a drag to the nal Test item and a release. It would fail to replay, since the sub-menu would never be popped up. To work correctly, an additional drag to the Test and Sequence nodes item must be recorded. For that reason QF-Test recognizes a MOUSE_DRAGGED event as important if you hover over an intermediate component for a while during the drag. The delay (in milliseconds) needed to recognize such an event is set through this option. To record the above example correctly with this option set to 1000, youd have to click on the Insert menu button, drag to the Sequences item and keep the mouse pointer stationary for one second, then move on to the Test sub-menu item and release the mouse button. Maximum drag distance for Mouse click event (System) It sometimes happens unintentionally that the mouse cursor is moved between pressing the mouse button and releasing it. This movement may be registered as a MOUSE_DRAGGED event, depending on the JDK version and the distance of the move. QF-Test is able to compensate for small movements and still convert the click into an abstract Mouse click event. This option denes the maximum distance between pressing and releasing the mouse button that QF-Test will ignore. Every MOUSE_DRAGGED event above that distance will be left unchanged.

29.2.

Recording options

239

Collect key events into a Text input node (System) Another example where a lot of events are generated is entering a short string of text into a text eld. Each character typed leads to at least one KEY_PRESSED, one KEY_TYPED and one KEY_RELEASED event. For additional fun, the KEY_RELEASED events may arrive out of order or not at all, depending on operating system and JDK version. If this option is activated, sequences of KeyEvents on a text component (to be exact: a component whose class is derived from java.awt.TextField or (427) node. javax.swing.text.JTextField) are converted into a Text input Onlytrue character input is packed, function or control keys or key combinations with Control or Alt are left unchanged. When the packed sequence is replayed, only KEY_TYPED events are generated. KEY_PRESSED and KEY_RELEASED events cannot be generated, since the required key code is system dependent and cannot be determined from the character alone. This is not a problem however, since text components usually handle only KEY_TYPED events and some special keys. Automatically set Clear... attribute of recorded Text input nodes (System) This option determines the value of the Clear target component rst attribute of a (427) node. If the option is not set, the attribute will not be set recorded Text input either. Otherwise, the Clear target component rst attribute is set if and only if the text eld or text area was empty before the input started. Always set Replay single events attribute of recorded Text input nodes (System) attribute of a This option determines the value of the Replay single events (427) recorded Text input node. If the option is set, the attribute will be set and vice versa. The conservative way is to keep the option set, but for a typical application that does not add its own KeyListeners to text elds it should be safe to turn it off so as to speed up replay of Text input nodes.
(429) (429)

29.2.

Recording options

240

29.2.3

Components

Figure 29.14: Options for recording components

Hotkey for components (User) SUT script name: OPT_RECORD_COMPONENT_HOTKEY This option denes the key for a very useful functionality: recording components directly from the SUT. Pressing this key in the SUT will switch it to component recording mode, regardless of whether event recording is currently activated or not. The default key is Shift-F11 . In this mode, clicking on a component with the mouse will cause the component to be recorded and added to the test-suite if it was unknown before. If more than one test-suite is currently opened, the menu item RecordReceive components de(515) (514) termines the suite that will receive the components. The Id of the Component is put on the clipboard so it can be pasted into any text eld with Control-V . This latter feature is very handy when creating an event or a check from scratch.

29.2.

Recording options

241

Figure 29.15: Popup menu for recording components

You can also record a whole component hierarchy at once by clicking with the right mouse button instead. This will bring up a popup menu with the following four options: Component only This is similar to clicking with the left mouse button. component is recorded. Component and children The selected component and all components contained therein are recorded. Whole window Records every component in the whole window. Suggest names This is a special feature to improve the collaboration between testers and developers in deciding which components should have names set with setName(). All components in the whole window are recorded and put into a test-suite of their own. Each unnamed component for which a name will improve testability is marked with a name of the form SUGGESTED NAME (n): suggestion. The running count in braces is just used to avoid duplicates. The suggested name is built from the components class and other available information. It should be taken with a grain of salt. Show methods This is another special feature that brings up a component inspector window showing the attributes and methods of the selected components class. See (40) section 5.5 for further information. Only the selected

29.2.

Recording options

242

Normally component recording mode is turned off by either pressing the hotkey again or by selecting a component. If you want to record multiple single components, hold down an additional modier key when switching to component recording mode. That way selecting a component will not turn off the mode, only pressing the hotkey again will. Record system class only (System) SUT script name: OPT_RECORD_COMPONENT_SYSTEM_CLASS_ONLY If this option is set, QF-Test does not record any custom classes for (514) Components . Instead it moves up the class hierarchy until it encounters a system class and records that. Set this option if the class names of your custom GUI classes tend to differ between releases.
Note

You must activate this option if you intend to obfuscate the jar les of your application or if you are using a custom class loader to load your applications GUI classes. This option does not apply to web SUTs. Validate component recognition during recording (System)

Web

3.5+

SUT script name: OPT_VALIDATE_RECORDED_COMPONENTS In case non-unique names are assigned to components QF-Test can still (516) distinguish between these components with the help of the Extra feature qfs:matchindex that species the index of the component with the given name. If this option is set, QF-Test will check the name of the component during recording and try to assign qfs:matchindex correctly.
Note

You should only deactivate this option if you are sure that component names are reasonably unique and the component validation signicantly impacts performance during recording. Convert HTML components to plain text (System)

Swing

SUT script name: OPT_RECORD_COMPONENT_CONVERT_HTML Swing support HTML markup in various kinds of labels, buttons and sub-elements of complex components. For component identication and validation, the HTML markup is often not useful and will clutter up things. If this option is set, QF-Test converts HTML to normal text by removing all HTML markup so only the actual text content is left. Name override mode (record) (System) Server or SUT script name: OPT_RECORD_COMPONENT_NAME_OVERRIDE

29.2.

Recording options

243

Note

Possible Values: VAL_NAME_OVERRIDE_EVERYTHING, VAL_NAME_OVERRIDE_HIERARCHY, VAL_NAME_OVERRIDE_PLAIN There are two versions of this option which are closely related. This one is (260) effective during recording, the other one during replay. Obviously, both options should always have the same value. Theres one exception though: When migrating from one setting to another, QF-Tests components have to be updated. During that process, keep the replay option at the old setting and change this option to the new one. Be sure to update the replay setting after updating the components. This option determines the weight given to the names of components for recording. Possible choices are: Override everything This is the most effective and adaptable way of searching components, but it requires that the names of the components are unique, at least within the same window. If that uniqueness is given, use this choice. Dont use this value for a web page with frames. Use Hierarchical resolution instead. Hierarchical resolution This choice should be used if component names are not unique on a per-window basis, but naming is still used consistently so that two components with identical names have at least parent components or ancestors with distinct names. That way, component recognition is still tolerant to a lot of change, but if a named component is moved to a different named parent in the SUT, the test-suite will have to be updated to reect the change. Plain attribute If there are components with identical names in the SUT within the same parent component you must use this setting. The name will still play an important role in (516) component recognition, but not much more than the Feature attribute. Automatic component names for Eclipse/RCP applications (System)

Web

SWT

SUT script name: OPT_RECORD_COMPONENT_AUTOMATIC_RCP_NAMES Eclipse and applications based on the Rich Client Platform (RCP) have a complex GUI with support for changing perspectives. Such a change causes components to be rearranged which can make it hard for QF-Test to recognize them unless names are set at least on the major components. This is further complicated by the fact that the structure of the components is not what it appears to be - the main components are all arranged in a relatively at hierarchy within the workbench. On the upside, RCP based applications have a uniform inner structure based on Views and Editors, many of which are named.

29.2.

Recording options

244

If this option is turned on, QF-Test will do its best to automatically associate GUI elements with their RCP counterparts and assign names based on that association. This can drastically improve component recognition for such applications. However, if some names thus assigned turn out not to be reliable over time, they can also interfere. In such a case, names can be assigned to the affected com(35) ponents either using setData as described in chapter 5 or with the help of a (634) NameResolver as described in section 39.1 . Both will override automatically generated names. Component hierarchy (System) Server script name: OPT_RECORD_COMPONENT_HIERARCHY Possible Values: VAL_RECORD_HIERARCHY_INTELLIGENT, VAL_RECORD_HIERARCHY_FULL, VAL_RECORD_HIERARCHY_FLAT QF-Test supports three different kinds of views for the components of the SUT. For more information about their effect on component recognition, see section (579) 36.2 . The at view collects all components of a window as direct child nodes of the (507) respective Window node. The advantage of this view is that structural changes of the component hierarchy have little effect on component recognition. This is also its greatest disadvantage: since structural information is not available, this view gives reasonable recognition quality only if setName() is used ubiquitously. Another drawback is the lack of clearness. The complement to the at view is the full hierarchy. It includes every single component of the SUTs GUI, emulating all parent/child relationships. This view can be a useful tool for developers or testers that want to gain insights into the SUTs structure, but is not very applicable for testing, since structural changes affect it too much. As long as you dont change the GUI however, it will give you excellent recognition without the help of setName(). A compromise between at and full hierarchy is available through the choice Intelligent. For this view only the interesting components of the SUT are recorded. Interesting in this case means that either the user can interact with the component, or it is located at some important point in the hierarchy, like the children of a split pane or a tabbed pane. In some later version of QF-Test this decision may be made congurable as well. Prepend id of window parent to component id (System) Server script name: OPT_RECORD_COMPONENT_PREPEND_WINDOW_ID (507) (514) If selected, QF-Test prepends the Id of the Window parent of a Component to

29.2.

Recording options
(515)

245

its Id during recording. This is useful to disambiguate ids of components with identical names in different windows. Prepend parent id to component id (System) Server script name: OPT_RECORD_COMPONENT_PREPEND_PARENT_ID Possible Values: VAL_RECORD_COMPONENT_PREPEND_PARENT_ALWAYS, VAL_RECORD_COMPONENT_PREPEND_PARENT_NAMED, VAL_RECORD_COMPONENT_PREPEND_PARENT_FEATURE, VAL_RECORD_COMPONENT_PREPEND_PARENT_NEVER (514) is recorded for the rst time, QF-Test assigns an When a Component (515) automatically generated Id . The id of a direct or indirect parent node may be prepended to this id. This is useful to distinguish between similar components that dont have a name of their own. Example: Imagine two JScrollPanes, one named TreeScrollPane and the other named DetailScrollPane. Without this functionality, their vertical scrollbars would get the ids scrollbarVertical and scrollbarVertical2. With this function turned on, the ids would be TreeScrollPane.scrollbarVertical and DetailScrollPane.scrollbarVertical. That way it is immediately obvious which component is the target of an event. There are four possible settings: Never turns this option off. Nearest named ancestor is a useful setting, if your developers have assigned names to all major components with the Java method setName. A component that doesnt have a name of its own, gets the id of its nearest named ancestor node prepended. If setName is used sparingly or not at all, it is better to set this option to Nearest ancestor with name or feature. That way either the name or a distinctive feature of an ancestor node will be applicable. Always is only useful if the option Component hierarchy is set to Flat. With this setting, every component gets the id of its parent node prepended, which can lead to unusably long ids when components are nested deeply. Maximum id length (System) If set, this option limits the length of automatically generated ids. This can improve clearness since ids can get very long, e.g. when generated from a components feature.
(244)

29.2.

Recording options

246

29.2.4

Recording sub-items

Events on complex components like tables or trees can be recorded relative to a sub(519) item of the component. For further information about sub-items see the Item node or (50) section 6.3 .

Figure 29.16: Options for recording sub-items

Record sub-items as event targets (System) SUT script name: OPT_RECORD_SUBITEM This option activates recording sub-items. When turned off, events on complex components are no different from events on simple components. Sub-item format (System) SUT script name: OPT_RECORD_SUBITEM_FORMAT Possible Values: VAL_RECORD_SUBITEM_FORMAT_INTELLIGENT, VAL_RECORD_SUBITEM_FORMAT_TEXT, VAL_RECORD_SUBITEM_FORMAT_NUMBER (519) index can be dened When recording an event for a sub-item, the Items (521) (521) As string or As number . The third choice, Intelligent, causes QF-Test to record the index in the format most appropriate for the item. If the name of the item is unique within the complex component, a string index is recorded, a numeric index otherwise. Sub-item type (System) Server script name: OPT_RECORD_SUBITEM_TYPE

29.2.

Recording options

247

Possible Values: VAL_RECORD_SUBITEM_TYPE_INTELLIGENT, VAL_RECORD_SUBITEM_TYPE_NODE, VAL_RECORD_SUBITEM_TYPE_SYNTAX (50) As explained in section 6.3 , there are two ways to address a sub-item: either (519) node. This option determines through a special syntax or by creating an Item the method to use in recorded events. Choosing Intelligent will cause an Item node to be created if the index of the sub-item is a string and if the sub-item itself is not editable. Otherwise the special syntax is used. Represent tree node as path (System) SUT script name: OPT_RECORD_SUBITEM_TREE_PATH It is not uncommon that trees have identically named nodes under different parent nodes, e.g. a le system with the directories /tmp and /usr/tmp. By (519) using a path format in the Items for tree nodes QF-Test can make full use of the hierarchical structure to distinguish between these nodes. The slash character / is used as separator. If this option is deactivated, trees will be treated as at lists.
(519)

29.2.5

Recording procedures

The following options determine the conguration of the Procedure Builder which is (153) described in detail in Automated Creation of Basic Procedures .

Figure 29.17: Procedure Builder options

Hotkey for procedure recording (User) SUT script name: OPT_RECORD_PROCEDURE_HOTKEY This option denes a key for automatic procedure recording directly in turning on the SUT. The default key is Shift-F12 .

29.3.

Replay options

248

Conguration le for recorded procedures (System) Here you can specify your own template le for the Procedure Builder. If a relative path is given, QF-Test looks for the denition le in the directory that QF-Test was started from and in the default include directory.

29.3

Replay options

The following settings change the way test-suites are executed.

Figure 29.18: Replay options

(Dont) Panic key (User) Server or SUT script name: OPT_PLAY_DONT_PANIC_HOTKEY When running a test at full speed it can be rather difcult to get the focus to QF-Tests window and interrupt the test so you can do something different without having all these windows ashing around the screen. This is all the more true (256) when the options Actually move mouse cursor or Raise SUT windows (255) automatically are activated or when running in batch mode.

29.3.

Replay options

249

This option lets you dene a key combination (the default being Alt-F12 ) that will instantly pause all running tests if it is pressed in any SUT or QF-Test window (unless multiple QF-Test instances are run simultaneously, of course). Pressing the same key combination again will resume all tests, unless you manually resume or stop any of them. In that case its effect is automatically reset to suspend tests. Call stack size (System) Server script name: OPT_PLAY_CALLSTACK_SIZE (307) or The call stack size is a limit for the nesting depth of Sequences (355) Procedure calls during replay. This limit is needed to detect and handle endless recursion. When the nesting depth exceeds the call stack size, a (532) StackOverflowException is thrown. The default value of 200 should be sufcient but can be increased for very complex tests. Log warning for nested test-cases (System) Server script name: OPT_PLAY_WARN_NESTED_TEST_CASE (288) nodes should not be nested because such Test-cases Execution of Test-case cannot be listed properly in the report. If this option is active, a warning is logged in case a Test-case is executed within another Test-case. Mark nodes during replay (User) If set, tree nodes that are currently executed are marked with an arrow. Show replay messages in status line (User) Determines whether the name of the currently executing node is shown in the status line. Raise test-suite window after replay (User) This option is mainly used together with the option Raise SUT windows (255) automatically . It causes the window of a test-suite to be raised after a test run. See also option Force window to the top when raising Minimize test-suite window during replay (User) If this option is set, QF-Test will minimize the window of a test-suite while its tests are being executed. The window will pop back up automatically when the test is stopped or suspended. This feature is especially useful on Windows 2000/XP
(255)

29.3.

Replay options

250

systems where programs are prohibited from bringing their windows to the top so QF-Test cannot raise the windows of the SUT. Show message dialog after (User) After replay is nished, the status line shows the number of errors and warnings that occurred. If an uncaught exception was thrown, an error dialog is displayed. Additionally, a message dialog can be displayed in case of warnings or errors. This options sets the minimum error level that triggers such a message dialog. Always locate the source of an error (User) When an exception is thrown during replay, the node that caused the exception will be made visible and selected. If you dont like this, you can turn this feature off and locate the node via the RunFind last error source... menu item instead. Salt for crypting passwords (System)
3.0+

QF-Test can store encrypted passwords in the Text attribute of a Text input (438) (435) node for a password eld or the Detail attribute of a Selection used for a login dialog in a web SUT. When such passwords are en- or decrypted, QF-Test combines the key with the salt specied in this option. Without this salt, anybody with sufcient knowledge is able to decrypt your passwords to get the plain-text version.
Note

(428)

(427)

Dont let this option give you a false sense of security. Anybody that gains access to this salt and anybody that can execute your tests can also gain access to the plain-text version of the password. However, encrypting passwords is still useful to prevent obvious plain-text passwords getting stored in test-suites and run-logs, and crypted passwords are reasonably safe from someone who only gets hold of a test-suite or run-log without access to this salt.

29.3.1

Client options

Various settings for process and SUT clients can be adjusted with the following options:

29.3.

Replay options

251

Figure 29.19: Client options

Ask whether to stop clients before exiting (User) If there are still active clients upon exit of QF-Test, these are terminated after asking for conrmation. If this option is turned off, the clients are terminated unconditionally. When terminating a process, kill its whole process tree (System) Server script name: OPT_PLAY_KILL_PROCESS_TREE The process of an SUT or a helper program started during a test can be (415) terminated via a Stop client node or manually via the Client menu. In case of an SUT, QF-Test rst tries to communicate with it and initiate a clean System.exit call. Non-Java programs have to be killed. If the program has started further child processes these may or may not get terminated by a normal shutdown or kill, depending on circumstances. It is normally undesirably to keep such processes around as they might interfere with other tests or lock les that need to be removed or overwritten. Unless this option is disabled, QF-Test will try to determine the whole process hierarchy for any

29.3.

Replay options

252

program it started and make sure that the main process and all child processes get killed explicitly. Number of terminated clients in menu (User) Server script name: OPT_PLAY_MAX_CLIENTS This option limits the number of menu items for terminated clients that are kept in the Clients menu. Maximum size of client terminal (kB) (User)
3.0+

Server script name: OPT_PLAY_TERMINAL_SIZE The maximum amount of text (in kilobyte) that the individual client terminal will hold. If the limit is exceeded, old text will be removed when new text arrives. A value of 0 means no limit.
Note

This option also determines the amount of output available for the special variable ${qftest:client.output.<name>}. Maximum size of shared terminal (kB) (User) Server script name: OPT_PLAY_SHARED_TERMINAL_SIZE The maximum amount of text (in kilobyte) that the shared terminal will hold. If the limit is exceeded, old text will be removed when new text arrives. A value of 0 means no limit. Highlight selected component in the SUT (User) If this option is set, QF-Test will highlight the associated component in the SUT (514) node or a node that references a Component is whenever a Component selected. How to handle exceptions in the SUT (System) SUT script name: OPT_PLAY_SUT_EXCEPTION_LEVEL Possible Values: VAL_PLAY_EXCEPTION_LEVEL_WARNING, VAL_PLAY_EXCEPTION_LEVEL_ERROR, VAL_PLAY_EXCEPTION_LEVEL_EXCEPTION Exceptions that are thrown during event handling in the SUT are typically a sure sign for a bug in the SUT. This option determines what to do if such an exception is caught. You can Log a warning message

29.3.

Replay options Log an error message Throw an UnexpectedClientException

(530)

253

Connect without JDK instrumentation (Swing) (System)

3.1.4+

Server script name: OPT_PLAY_IMPLICIT_INSTRUMENTATION With this option activated there is no need to instrument standard JDKs/JREs in order to connect to an AWT/Swing SUT. In cases where automatic connections dont work - or if you deactivate this option - you need to fall back to explicit JDK (571) (574) instrumentation as described in section 35.1 and section 35.2 .
Note

If this option is activated, QF-Test sets the environment variable _JAVA_OPTIONS (and also IBM_JAVA_OPTIONS for IBM JDKs) so that any JVM started directly or indirectly by QF-Test will execute QF-Tests accessibility hook when initializing the AWT toolkit. In contrast to other tools QF-Test does not set this environment variable globally and thus does not interfere with Java applications not started from QF-Test. However, if the variable is already set globally and your Java application relies on it, you must deactivate this option and use JDK instrumentation instead. Pure Eclipse/SWT applications only require SWT instrumentation (see section (575) 35.3 ). Some Eclipse/RCP applications show an AWT based splash screen on startup and if this option is active that splash screen will trigger initialization of QFTests AWT/Swing engine, which will then become the primary engine for that SUT. (392) (411) and Wait for client to connect If you rely on the default engine for SUT script nodes to be swt, you need to deactivate this option. Reuse IDs for SUT clients in nested sub-processes (System) Server script name: OPT_PLAY_REUSE_SUT_IDS This is a complex option which you should hopefully never care about. When an SUT client launches another process that itself connects to QF-Test, the new SUT client is identied by the name of the original SUT client with a : and a numeric ID appended. The rst ID will always be 2, with increasing numbers for additional sub-processes. When a sub-process terminates and another sub-process connects, QF-Test can either reuse the ID of the terminated process or continue incrementing to create a new ID. In most cases it is preferable to reuse the sub-process ID. The most common case is a single sub-process that is started, terminated, then started again. By activating this option you can always address the single sub-process with the same client name.

SWT

29.3.

Replay options

254

In a more complex situation, multiple sub-processes may be launched and terminated more or less at random, depending on progression of the test-run. In such a case, always incrementing the ID for a new process is more deterministic. In either case the ID counter will be reset when the original SUT client is started anew. Automatically perform garbage collection in the SUT (System) SUT script name: OPT_PLAY_SUT_GARBAGE_COLLECTION By default QF-Test performs a full garbage collection in the SUT once every few (392) executions. This is necessary due to a limitation in Javas hundred SUT script default garbage collection mechanism that allows an OutOfMemoryError to happen for the so called PermGen space, even though the required memory could easily be reclaimed by a garbage collection. When you are trying to analyze the memory performance of your application, these explicit garbage collections might inuence the results. For that case you can disable QF-Tests garbage collection via this option.

29.3.2

Event handling

These options inuence some details of how events are simulated in the SUT during replay.

29.3.

Replay options

255

Figure 29.20: Event handling options

Raise SUT windows automatically (System) SUT script name: OPT_PLAY_RAISE_SUT_WINDOWS If this option is set, windows of the SUT for which a MouseEvent or KeyEvent is replayed will be raised automatically when they get activated. This eases switching between QF-Test and the SUT to visually verify that a sequence is replaying correctly. See also options Raise test-suite window after replay (255) top when raising . Force window to the top when raising (System)
3.4.1+ Note
(249)

and Force window to the

SUT script name: OPT_PLAY_RAISE_SUT_WINDOWS_FORCED This option only has an effect on Windows systems.

29.3.

Replay options

256

Windows only allows an application to bring one of its own windows to the front if that application currently has the focus. This can make it difcult for QF-Test to raise SUT windows and to automatically switch between the SUT and QF-Test. If this option is activated, QF-Test temporarily sets the always on top attribute to force windows to the top. See also options Raise test-suite window after replay (255) automatically . Check for modal dialogs (System) SUT script name: OPT_PLAY_CHECK_MODAL A modal dialog is a window that blocks all input to other windows until it is closed. It is often used to display an error message or request user input. Because the events simulated by QF-Test are articial, they are not blocked by a modal dialog and can reach any window. This is normally not desirable, since the existence of a modal dialog may signal an unexpected error. Activating this option causes QF-Test to check for modal dialogs itself before replaying MouseEvents or KeyEvents. If such an event is targeted to a window that is blocked by a modal (526) dialog, a ModalDialogException is thrown. Actually move mouse cursor (System) SUT script name: OPT_PLAY_MOVE_MOUSE_CURSOR If this option is set, the mouse cursor is actually moved across the screen as MouseEvents are simulated. This feature requires a working AWT robot. While this option is mainly intended to give visual feedback, it can have a positive impact on test reliability because it reduces side-effects through events from the underlying system that might interfere with the automated test. However, for tests where precise mouse movement is essential, for example a drawing tool, this option should be turned off. Delay for hard events during replay (ms) (System) SUT script name: OPT_PLAY_DELAY_HARD_EVENTS During event replay QF-Test blocks or delays some hard events, i.e. events that come directly from the system, for example when the user is moving the mouse cursor. This is done to prevent them from interfering with the SUT in an unlucky moment. Popup windows, which are used for menus and combo boxes among other things, are especially sensitive to such events which can cause them to pop down accidently. Therefore, these lters improve testing stability considerably.
(249)

and Raise SUT windows

29.3.

Replay options

257

This option sets the maximum time that such events may be delayed. In the unlikely case that the lters have unwanted side effects with your application, you can turn them off by setting the value to 0. Scroll automatically to display sub-items (System) SUT script name: OPT_PLAY_SCROLL_ITEM If this option is set, accessing a sub-item of a complex component inside a scroll pane will automatically cause the sub-item to be scrolled into view. In that case you can remove most recorded events on scroll bars or scroll buttons, which are not required for correct replay. Expand tree nodes as needed (System) SUT script name: OPT_PLAY_EXPAND_TREE When accessing nodes of a tree component as hierarchic sub-items it is possible to select a node that is not visible because one of its ancestral nodes is not expanded. If this option is set, all ancestors of the node will be expanded as needed. Otherwise this situation leads to a (526) ComponentNotFoundException . Throw DisabledComponentException (System) SUT script name: OPT_PLAY_THROW_DISABLED_EXCEPTION If QF-Test replays events on a component that is not enabled, these events are ignored silently. In most cases this indicates an error which is signaled by (527) throwing a DisabledComponentException . Old test-suites may not be prepared to deal with this exception. These test-suites should be xed, but as a quick workaround DisabledComponentExceptions can be suppressed by deactivating this option. Tolerance for checking images (System)
SWT Note

SUT script name: OPT_PLAY_IMAGE_TOLERANCE Initially this option was intended for SWT/Gtk only but it turned out to be universally applicable and useful. Image rendering in Java applications and web browsers is not always fully deterministic. Even within the same session on a display with limited color depth the RGB values of an icon image can vary slightly and it becomes worse when running tests on different machines. Graphics driver, JDK version and operating system settings also play a role. This makes strict image checks almost unusable in some cases.

29.3.

Replay options

258

To work around the problem, this option denes a tolerance setting for default image checks. For a pixels red, green and blue color values a deviation from the expected value by the given amount is tolerated. Thus exact image checking for the default algorithm can be enforced by setting this value to 0 but it is preferable to use the identity check algorithm instead (see Details about the algorithm for (722) image comparison ). The default setting of 5 is a good compromise, allowing checks with differences that are normally not visually perceivable to succeed. How to handle events coming from the wrong thread (System)
Swing

SUT script name: OPT_PLAY_WRONG_THREAD_ERROR_LEVEL Possible Values: VAL_PLAY_THREAD_LEVEL_WARNING, VAL_PLAY_THREAD_LEVEL_ERROR, VAL_PLAY_THREAD_LEVEL_EXCEPTION It is a rather common mistake in Swing based Java applications to access GUI components from the wrong thread. Since Swing is not thread-safe, such calls may only be made from the AWT event dispatch thread. Otherwise the potential consequences are race conditions, leading to very subtle and hard-to-debug errors, or deadlocks, freezing the application and making it unusable. Background information about this topic is available from http://download.oracle.com/javase/tutorial/uiswing/concurrency/index.html , specically the sections on Initial Threads and The Event Dispatch Thread. When QF-Test registers an event on a thread other than the AWT event dispatch thread it issues an error message including a stack trace which can be useful in xing the problem. This set of options denes the severity of the message, whether to perform strict checking and a maximum for the number of messages to log. The possible choices for the option Error level are Error and Warning. We strongly suggest that you keep the default setting of Error and make sure that such problems are xed sooner rather than later because they represent a serious risk. Strict checking (System)
Swing

SUT script name: OPT_PLAY_WRONG_THREAD_STRICT If strict checking for wrong thread errors is activated, error or warning messages will be issued for all kinds of events triggered from the wrong thread. Otherwise the problem will be ignored for less relevant events. Which events are considered more or less relevant is arbitrary, based on the fact that theres a lot of Java literature (including early Java documentation from Sun) claiming that it is safe to create components on any thread and that thread-safety only needs to be enforced once a component is made visible. A lot of code follows this pattern and the risk for causing problems in this case is indeed minimal. In the presence of

29.3.

Replay options

259

such code disabling Strict checking will cause error messages to be logged only for the more relevant problems. If you want to get rid of all thread violations - as we recommend - you should turn strict checking on. Maximum number of errors to log per SUT client (System)
Swing

SUT script name: OPT_PLAY_WRONG_THREAD_MAX_ERRORS In case an SUT contains code that violates thread safety it is possible that a very large number of events are triggered from the wrong thread. Logging all those errors can signicantly impact test performance, yet logging more than the rst few errors does not really contribute much. The option Maximum number of errors to log per SUT client limits the possible number of error messages for this case.

29.3.3

Component recognition

How component recognition works - and the impact of these options on it - is explained (579) in section 36.2 . The pre-dened values should give good results. If you experience problems with component recognition, you can try to improve it by adjusting the probabilities.

29.3.

Replay options

260

Figure 29.21: Component recognition options

The name of a component plays a special role. The following two options affect the impact of names: Name override mode (replay) (System) SUT script name: OPT_PLAY_RECOGNITION_NAME_OVERRIDE Possible Values: VAL_NAME_OVERRIDE_EVERYTHING, VAL_NAME_OVERRIDE_HIERARCHY, VAL_NAME_OVERRIDE_PLAIN There are two versions of this option which are closely related. This one is (242) effective during replay, the other one during recording. Obviously, both options should always have the same value. Theres one exception though: When migrating from one setting to another, QF-Tests components have to be updated. During that process, keep this option at the old setting and change the record option to the new one. Be sure to update the replay setting after updating the components.

Note

29.3.

Replay options

261

This option determines the weight given to the names of components for component recognition. Possible choices are: Override everything This is the most effective and adaptable way of searching components, but it requires that the names of the components are unique, at least within the same window. If that uniqueness is given, use this choice. Dont use this value for a web page with frames. Use Hierarchical resolution instead. Hierarchical resolution This choice should be used if component names are not unique on a per-window basis, but naming is still used consistently so that two components with identical names have at least parent components or ancestors with distinct names. That way, component recognition is still tolerant to a lot of change, but if a named component is moved to a different named parent in the SUT, the test-suite will have to be updated to reect the change. Plain attribute If there are components with identical names in the SUT within the same parent component you must use this setting. The name will still play an important role in (516) component recognition, but not much more than the Feature attribute. Log warning for missing name (System) SUT script name: OPT_PLAY_WARN_MISSING_NAME If this option is set, a warning will be logged whenever a component is targeted that does not have a name, but QF-Test thinks it should have one. A plausible name is suggested where possible. Log warning for ambiguous name (System) SUT script name: OPT_PLAY_WARN_AMBIGUOUS_NAME (260) is set to Override everything or If the option Name override mode (replay) Hierarchical resolution, a warning will be logged whenever QF-Test encounters more than one potential target components with the same name. That warning can be suppressed with the help of this option. Log warning for feature mismatch (System) SUT script name: OPT_PLAY_WARN_FEATURE_MISMATCH A component is considered to have a feature mismatch if it is determined by QF-Test as the target best suited for an event or check even though at one or (516) more levels of the hierarchy the recorded Feature attribute did not match the current state of the component. If this option is activated, feature mismatches are

Web

29.3.

Replay options

262

logged as warnings, notifying you that it may be a good idea to update the affected components. Log warning for extra feature mismatch (System) SUT script name: OPT_PLAY_WARN_EXTRA_FEATURE_MISMATCH An extra feature mismatch is similar to a feature mismatch as explained above, except that it applies to extra features with status should match. If this option is activated, extra feature mismatches are logged as warnings, notifying you that it may be a good idea to update the affected components. Log warning for structure mismatch (System) SUT script name: OPT_PLAY_WARN_STRUCTURE_MISMATCH A structure mismatch is similar to a feature mismatch as explained above, except that instead of the feature it is the structure information represented by the (518) (518) and Class count where the mismatch occurred. If this attributes Class index option is activated, structure mismatches are logged as warnings, notifying you that it may be a good idea to update the affected components. Log warning for intermediate named ancestor (System) SUT script name: OPT_PLAY_WARN_NAMED_ANCESTOR An intermediate named ancestor is a direct or indirect parent component of the target component in the SUT which is not part of the component hierarchy in (260) QF-Test even though it has a name. If the option Name override mode (replay) is set to Hierarchical resolution, this is considered as a mismatch, comparable to a feature or structure mismatch. If this option is activated, interferences through intermediate named ancestors are logged as warnings, notifying you that it may be a good idea to update the affected components. For an explanation of the remaining options for component recognition please refer to (579) section 36.2 . The respcetive SUT script names for these options are: OPT_PLAY_RECOGNITION_BONUS_NAME OPT_PLAY_RECOGNITION_PENALTY_NAME OPT_PLAY_RECOGNITION_BONUS_FEATURE OPT_PLAY_RECOGNITION_PENALTY_FEATURE OPT_PLAY_RECOGNITION_BONUS_EXTRAFEATURE OPT_PLAY_RECOGNITION_PENALTY_EXTRAFEATURE OPT_PLAY_RECOGNITION_BONUS_STRUCTURE OPT_PLAY_RECOGNITION_PENALTY_STRUCTURE OPT_PLAY_RECOGNITION_PENALTY_MODAL OPT_PLAY_RECOGNITION_MINIMUM_PROBABILITY

29.3.

Replay options

263

29.3.4

Delays

Here values can be set for various delays.

Figure 29.22: Delay options

Default delays (ms) (System) Server script name: OPT_PLAY_DELAY_BEFORE, OPT_PLAY_DELAY_AFTER These two options set the delay in milliseconds before and after the execution of (309) a node. If a node denes its own Delay before/after , its value overrides this default. These options are useful to slow a test down so youll be able to follow it. Drag&Drop and interpolation of mouse movement Simulating Drag&Drop is non-trivial. It is made possible only by generating hard mouse events that actually move the mouse cursor. On Windows systems, some mouse drivers (578) can interfere with these hard events. See section 36.1 for further details about Drag&Drop. To make Drag&Drop as reliable as possible, movement of the mouse cursor is highly congurable. Since the requirements for Drag&Drop and hard mouse events differ from

29.3.

Replay options

264

those for general mouse moves which only provide visual feedback, two sets of options are provided. The settings for demo mouse moves are ignored unless the respective (256) option Actually move mouse cursor is activated. Typically movements for Drag&Drop and hard events should be slower and involve more interpolation steps than those for demo moves, which could slow down a test considerably. All of the following values inuence the overall speed of mouse moves. A little experimentation may be required to get the desired effect. Delay (ms) (System) SUT script name: OPT_PLAY_DND_DELAY, OPT_PLAY_MOVEMOUSE_DELAY After each single movement of the mouse cursor QF-Test will wait for the specied number of milliseconds. This value should be between 2 and 20 if interpolation is enabled and between 20 and 200 if interpolation is turned off. With interpolation, 10 is a good value for Drag&Drop and 5 for demo mouse moves. Interpolation step size (System) SUT script name: OPT_PLAY_DND_STEP, OPT_PLAY_MOVEMOUSE_STEP The size of the steps for interpolation of mouse movement. Set this to 0 to turn interpolation off. Good values are between 1 and 3 for Drag&Drop and between 2 and 10 for demo mouse moves. Acceleration (System) SUT script name: OPT_PLAY_DND_ACCELERATION, OPT_PLAY_MOVEMOUSE_ACCELERATION To avoid needless slowdown of tests, long distance mouse movement can be accelerated. A value of 0 turns off acceleration. Useful values range from 1 for very little acceleration to 10 or more for high acceleration. Good values are between 3 and 5 for Drag&Drop and between 6 and 20 for demo mouse moves. Acceleration threshold (System) SUT script name: OPT_PLAY_DND_THRESHOLD, OPT_PLAY_MOVEMOUSE_THRESHOLD To ensure that small movements as well as the start end end of each movement remain precise, acceleration is turned off for movements that require less than this thresholds number of steps. Good values are between 4 and 8 for Drag&Drop and between 0 and 6 for demo mouse moves.

29.3.

Replay options

265

29.3.5

Timeouts

These timeouts are essential for reliable replay of tests under varying conditions. They dene how long QF-Test waits for a component to be in the proper state for an event before throwing an exception. Dont make these values too small, so a little hiccup due to high load wont interrupt a test needlessly. QF-Test continues as soon as the conditions for replaying an event are met, so higher values for the timeouts wont slow down execution (except for focus, see below). On the other hand, dont set any values higher than a few seconds or youll have to wait too long until you nally get an error message when a component is truly not found.

Figure 29.23: Timeout options

29.3.

Replay options

266

Deadlock detection (s) (System) Server script name: OPT_PLAY_TIMEOUT_DEADLOCK (527) If the SUT does not react for the given time, a DeadlockTimeoutException is thrown. Setting this value to 0 will suppress deadlock detection. Wait for GUI engine (ms) (System) SUT script name: OPT_PLAY_TIMEOUT_ENGINE This option is useful only for multi-engine SUTs, like Eclipse with embedded (411) node nishes as soon as AWT/Swing components. A Wait for client to connect (413) the rst GUI engine connects to QF-Test, unless its GUI engine attribute species to wait for a specic engine. To prevent a subsequent (484) Wait for component to appear node for a component of the wrong engine from failing immediately, QF-Test rst waits for the time specied with this option to give the second GUI engine a chance to connect also. Wait for non-existent component (ms) (System) SUT script name: OPT_PLAY_TIMEOUT_COMPONENT The maximum time in milliseconds that QF-Test waits for the target component of an event to become visible. Wait for non-existent item (ms) (System) SUT script name: OPT_PLAY_TIMEOUT_ITEM If an event is targeted on a sub-item of a complex component, QF-Test rst waits for the component to become visible. Then it gives the SUT the chance to make the intended sub-item available before this timeout is exceeded. Wait for modal dialog (ms) (System) SUT script name: OPT_PLAY_TIMEOUT_MODAL If an event is targeted at a window that is blocked by a modal dialog, a (526) ModalDialogException will be thrown. However, modal dialogs are often temporary, informing the user about some ongoing processing. If this option is set to a non-zero value, QF-Test will wait for the given time before it throws the exception. If the modal dialog disappears before the time limit is exceeded, the test will continue immediately. This greatly simplies handling of temporary modal dialogs.
Note

If the option Convert opening of a window into Wait for component to appear is activated, recording a sequence of events during which a temporary dialog is

(484) (235)

29.3.

Replay options
(484)

267

displayed may result in a Wait for component to appear node for that dialog. If the dialog is displayed for a short time only, it is best to remove such nodes to avoid timing issues. If the SUT employs temporary modal dialogs often it may be best to (484) disable the option Convert opening of a window into Wait for component to appear (235) . Wait for busy GlassPane (ms) (System)
Swing

SUT script name: OPT_PLAY_TIMEOUT_GLASSPANE As an alternative to temporary modal dialogs some applications employ a so called GlassPane together with a busy mouse cursor (typically in the form of an hourglass) to inform the user that the application is busy. The GlassPane is an invisible component that covers an entire window. If an event is delivered to such a window, the GlassPane will typically intercept it, preventing normal event processing, which can throw a test-run severely off course. QF-Test handles this case automatically by waiting until the GlassPane disappears before delivering an event, performing a check, etc. If the timeout given in this (527) option is exceeded and the GlassPane is still active, a BusyPaneException is thrown. If the value of this option is set to 0, GlassPane checking is disabled and the event is delivered regardless. A BusyPaneException is never thrown in this case. The Wait for component to appear node is a special case. When waiting for a com(486) ponent (not its absence) covered by a busy GlassPane, the Timeout attribute of the node overrides this option and is used to wait for both, the appearance of the component and the removal of the busy GlassPane. Thus it is possible to handle cases where the SUT is expected to be busy for an exceptionally long time on an individual basis without changing the default timeout of this option. Wait for button/menu enable (ms) (System) SUT script name: OPT_PLAY_TIMEOUT_ENABLED A MouseEvent on a button or menu item is simply ignored, if the component is not enabled. This could lead to unwanted side effects during a test, so QF-Test waits until the component is enabled or the specied timeout is exceeded. If the component does not become activated within the given time, a (527) DisabledComponentException is thrown unless the option Throw (257) DisabledComponentException is deactivated. Wait for focus (ms) (System) SUT script name: OPT_PLAY_TIMEOUT_FOCUS If set, this timeout causes QF-Test to wait for a component to get the keyboard
(484)

29.4.

Web options

268

focus before it simulates any KeyEvents on it. This option actually can slow down a test noticeably if a component fails to get the focus, so dont set it higher than about 100. A good value is 20. Poll interval for component wait (ms) (System) SUT script name: OPT_PLAY_POLL_COMPONENT When waiting for a component in the SUT to appear, QF-Test cant always rely on Javas event mechanism. Instead it has to repeatedly scan the SUT for the component. This option determines the interval between searches. Sub-item poll interval (ms) (System) SUT script name: OPT_PLAY_POLL_ITEM When waiting for a non-existent sub-item in the SUT, QF-Test cant rely on the event mechanism. Instead it has to repeatedly search the complex component for the sub-item. This option determines the interval between searches. Retry check interval (ms) (System) SUT script name: OPT_PLAY_POLL_CHECK (442) (447) fails for which a Timeout is dened, QF-Test repeatedly queries If a Check the components state until either it matches the given values or the time is up. The interval to wait between queries is set with this option.

29.4
Web

Web options

The following options are used specically for web testing.

29.4.

Web options

269

Figure 29.24: Web options

Record generic class names for components (System) SUT script name: OPT_WEB_RECORD_GENERIC_CLASS Where possible QF-Test assigns generic class names to web components like Button, Table or Link in addition to the AJAX specic class names like X-BUTTON or rf-dt. These generic class names are more descriptive and can improve compatibility when an AJAX toolkit evolves. Both class names can be used for component recognition or registering resolvers. If this option is active, generic class names are recorded where available. Turn id attribute into name where unique enough (System) SUT script name: OPT_WEB_ID_AS_NAME

29.4.

Web options

270

Use the id attribute of a DOM node as the component name, provided the id is sufciently unique. Uniqueness is determined per node type and depending on (260) and Name override the setting of the options Name override mode (replay) (242) mode (record) . Eliminate all numerals from id attributes (System) SUT script name: OPT_WEB_SUPPRESS_NUMERALS Of course this option only changes the way QF-Test treats id attributes. The attributes themselves are left unchanged or the application would most likely no longer work. With this option activated QF-Test removes all numerals from id attributes to prevent problems caused by automatically generated ids often found in Ajax frameworks like GWT. Such dynamic ids can change after the slightest modication to a web application which causes tests to break, especially if names are based on ids. By removing the dynamic part from such ids they become less useful, because they are no longer unique, but also less harmful. Uniqueness of names (516) and is taken care of by QF-Test. Since ids also serve as a basis for Feature (516) Extra features attributes, this option is helpful even if ids are not used as names. Replace no-break space with normal space (System) SUT script name: OPT_WEB_REPLACE_NBSP A no-break space character, &nbsp; in HTML, is used to tell the browser that it should not use this place for an articial line break. No-break space is also non-collapsible so several &nbsp; characters are often used for indentation or spacing. The problem with no-break space is that it looks exactly like normal space but is treated as non equal when comparing text. A check that contains normal space performed against text with no-break space would fail, but you would not be able to spot the difference in the error message. As these kinds of errors are very difcult to analyze, unless you deactivate this option QF-Test treats no-break space like normal space when determining Name, Feature and Extra feature attributes, (471) executing check nodes and for the result of a Fetch text node. Limit URL feature of Web page node to host or le (System) SUT script name: OPT_WEB_LIMIT_URL If this option is active, all pages coming from the same host are recorded as the same page by reducing the URL feature to the host part of the URL. This is often useful when pages share a common look and navigation structure.

Note

29.4.

Web options

271

For le URLs, the URL is reduced to the lename, with intermediate directories removed. Retarget mouse event on trivial node to parent (System) SUT script name: OPT_WEB_RETARGET_MOUSE_EVENT When recording mouse events on DOM nodes in a web page it is often useful to ignore trivial nodes and concentrate on the important ones. For example, when clicking on a text hyperlink it is typically not of interest whether part of the link is formatted with a bold font. It is the link that is important. If this option is active QF-Test does not simply record the event for the deepest DOM node under the mouse cursor. Instead it moves up the hierarchy until it nds an interesting node. In the example above, QF-Test would record the event on the A node with the option active and on the contained B node otherwise. Tolerate intermediate parent components (System) SUT script name: OPT_WEB_TOLERATE_INTERMEDIATE_PARENT Normally QF-Tests component recognition is tolerant to changes in the component hierarchy. For web pages with deeply nested tables this can lead to performance problems because the potential variants of determining the target component grow exponentially with the nesting depth. If you experience such problems, try to deactivate this option. It will reduce adaptability but should help with performance.
Note

The by far preferable solution is to set unique id attributes for the different tables and other components so that QF-Tests name override mechanism can apply. This not only speeds up recognition drastically, it is also much more reliable and tolerant to change. Take visibility of DOM nodes into account (System) SUT script name: OPT_WEB_TEST_VISIBILITY Similar to AWT/Swing or SWT, QF-Test normally only recognizes visible DOM nodes as target components. However, visibility of DOM nodes is not as well dened as that of components in a Java GUI. For example it is possible that an invisible DOM node has visible child nodes. Also, if a web page contains illegal HTML constructs it is possible that a DOM node is considered invisible, even though it is displayed in the browser window. If you come across such a problem you can turn off this option. Let the browser determine the target element for check recording (System)

29.4.

Web options

272

SUT script name: OPT_WEB_CHECK_VIA_BROWSER When recording checks, components or procedures QF-Test needs to determine the target element under the mouse cursor. In case of overlapping nodes there are two different ways for calculating which one should be used. By default QF-Test lets the browser decide, which is usually the best choice. Since the different browsers dont always behave in the same reliable way, this option can be turned off in case of problems to use the older mechanism based on the z-order of elements instead. This option has no effect on check replay. Record HTTP Request as browser request (System) SUT script name: OPT_WEB_RECORD_CLIENT_REQUEST_STEP (504) is created. This When recording HTTP Requests a Browser HTTP request request will be submitted directly from Brpwser so that the response is shown afterwards. The test execution could be continued directly in the browser. By (500) deactivating this option a Server HTTP request will be recorded. This request will be submitted by QF-Test and dont have any effect to the browser. The response is only accessible in QF-Test. How to handle Javascript errors in a Web application (System) SUT script name: OPT_WEB_JAVASCRIPT_ERROR_LEVEL Possible Values: VAL_WEB_JAVASCRIPT_LEVEL_WARNING, VAL_WEB_JAVASCRIPT_LEVEL_ERROR Dynamic HTML and AJAX are implemented via a lot of Javascript code that is executed in the Browser. If an error occurs in such a script, Internet Explorer shows an error dialog with some details about the error. Some of these errors are relatively harmless, others quite severe. Either way the error dialog would prevent proper test execution, so instead of showing the dialog, QF-Test intercepts the error message and logs an error or warning in the run-log instead. This set of options denes the severity of the message and a maximum for the number of such kinds of messages to log. The possible choices for the option Error level are Error and Warning. We advise that you keep the default setting of Error and make sure that such problems are reported to development and xed sooner rather than later because they represent a bug in the application you are testing. Maximum number of errors to log per SUT client (System) SUT script name: OPT_WEB_JAVASCRIPT_MAX_ERRORS In case a web page contains erroneous code it is possible that a lot of errors are triggered. Logging all those errors can signicantly impact test performance, yet

29.5.

Debugger options

273

logging more than the rst few errors does not really contribute much. The option Maximum number of errors to log per SUT client limits the possible number of error messages for this case. Errors that should be ignored (System)
3.5+

SUT script name: OPT_WEB_JAVASCRIPT_ERROR_FILTERS It is possible that some Javascript errors cannot or will not be xed, for example when they are coming from third-party code. In such a case it is preferable to ignore the known errors while still having QF-Test report unexpected ones. When the browsers reports a Javascript error, QF-Test searches its error message for the occurrence of any of the regular expressions specied in this option. In case of a match, the error is ignored. If no exceptions are dened or none match, the error is reported in accordance with the previous options.

29.5

Debugger options

These options modify the behavior of the debugger.

Figure 29.25: Debugger options

Enable debugger (User) By default the debugger is disabled unless this option, which can also be modied through the menu item DebuggerEnable debugger , is activated.

29.5.

Debugger options

274

If a test is interrupted by a breakpoint or be pressing the Pause button, the debugger is activated automatically. Similarly, starting a test-run with Step in or Step over will activate the debugger for the duration of that test-run. Always open debugger window (User) When test execution is halted and the debugger entered QF-Test can optionally open a separate window for the debugger. This option determines whether debugging should happen in a separate window or the normal test-suite window. Show variable bindings automatically (User) When test execution is halted and the debugger entered QF-Test can display the current variable bindings in the workbench window. If this option is active the variables are shown automatically each time a test-run rst enters the debugger. Alternatively the variables can be viewed in the debugger window or shown in the workbench window via the menu DebuggerShow variables . Automatic breaks (User) These options describe the situations in which execution of a test will be suspended and the debugger entered: Break on uncaught exception The debugger will break if an exception is thrown that will not be handled by (382) a Catch node. Break on error The debugger will break if an error occurs. Break on warning The debugger will break if a warning occurs. Break on caught exception You practically never want this! The debugger will break if an exception is (382) thrown, even if that exception will be handled by a Catch node.

29.6.

Run-log options

275

29.6

Run-log options

These options let you control which information is collected in a run-log, if and when a run-log is shown and how to locate errors.

29.6.1

General run-log options

Figure 29.26: General run-log options

When to show run-logs (User)

29.6.

Run-log options

276

A run-log is created for every execution of a test. A number of recent run-logs are available from the Run menu, the most recent run-log can also be opened by pressing Control-L . Additionally the run-log can be shown during execution or after an error as follows: At start This choice causes QF-Test to open the run-log when it begins executing a test sequence. The nodes of the log will be added as execution proceeds. After nish With this choice the run-log is shown after replay is nished. On exception The run-log is shown only if an uncaught exception is thrown. Dont show The run-log is not displayed automatically. You have to open it via the Run menu or by pressing Control-L . Number of run-logs in menu (User) A limit for the number of menu items for recent run-logs kept in the Run menu. Automatically save run-logs (User)
3.0+

To prevent excessive memory use through run-logs and also to make the most recent run-logs persistent between QF-Test sessions, the recent run-logs kept in the Run menu are automatically saved in the directory .qftest in the current users home directory. The lename for the run-log is based on a timestamp. QF-Test uses le locks to prevent collisions and accidental removal in case of parallel sessions and automatically keeps the .qftest directory clean by removing unreferenced logs, so there should be no reason to disable this feature. Still, you can do so by disabling this option. Show expanded variable values in tree nodes (User)
3.1+

The nodes in the tree view of a run-log can either be displayed with variables expanded to the value they had at run-time or with the original variables. Both views have their use, so you can toggle between them via this option or more quickly via the menu item ViewShow nodes expanded . Find next error function locates (User) The search for an error in a run-log, accessible through the EditFind next error menu item or by pressing Control-N , is a very useful utility. This option

29.6.

Run-log options

277

determines whether the search stops at exceptions only, errors and exceptions or even warnings. Skip suppressed errors (User) Like the previous one, this option controls the search for errors in a run-log. If activated, warnings, errors or exceptions that have not propagated to the top, are (379) (382) not found. Thus exceptions caught by a Try /Catch clause or messages (308) suppressed through the Maximum error level attribute are skipped. This option is also accessible through the EditSkip suppressed errors menu item. Cleanup tree when jumping to next or previous error (User)
3.0+

When repeatedly jumping to errors in a run-log the tree can easily get cluttered with many expanded nodes. If this option is activated, QF-Test will automatically clean up the tree each time you navigate to an error so that only the parents of the error node are expanded.
Note

When viewing split run-logs, partial run-logs containing an error will remain in memory as long as their nodes are expanded. Keeping this option activated will ensure that partial run-logs will be released as soon as possible, keeping memory use manageable even viewing the errors of a very large run-logs. Save compressed run-logs (*.qrz) (System) Server script name: OPT_LOG_SAVE_COMPRESSED Run-logs can either be saved as plain or as compressed XML les. For large run-logs without screenshots the compression factor can be as high as 10, so its advisable to use compressed logs where possible. The only reason not to use compression is if you want to transform the XML run-log afterwards. But even then compressed run-logs are an option because the compression method used is standard gzip format, so converting to and from compressed run-logs can easily be done using gzip. When saving a log-le interactively you can always switch between compressed or non-compressed format by choosing the appropriate lter or by giving the le the extension .qrz or .qrl. In batch mode, the default run-log format is compressed. To create an uncompressed run-log, simply specify the extension .qrl in the parameter for (549) the -runlog [<file>] command line argument. Create split run-logs (User)

3.0+

29.6.

Run-log options

278

Server script name: OPT_LOG_SAVE_SPLIT (332) A run-log can be split into several parts by setting the Name for separate run-log (330) attribute of a Data driver or any of the various test nodes. By turning this option off you can temporarily disable support for split run-logs in order to get a normal, single run-log without having to modify any Name for separate run-log attributes. See section 8.1.4
3.0+
(67)

for further information about split run-logs.

Save split run-logs as ZIP les (*.qzp) (User) Server script name: OPT_LOG_SAVE_SPLIT_ZIP Split run-logs can either be saved as a single ZIP le with the extension .qzp, containing the main run-log and all partial logs together, or as a normal .qrl or .qrz run-log that is accompanied by a directory with the same base name and the sufx _logs, e.g. the le runlog.qrz plus the directory runlog_logs. This option determines the format in which split run-logs are created in interactive (276) mode. It has no effect if the option Automatically save run-logs is turned off. See section 8.1.4
3.4+
(67)

for further information about split run-logs.

Minimum size for automatic splitting (kB) (System) Server script name: OPT_LOG_AUTO_SPLIT_SIZE Split run-logs are the only reliable way to prevent running out of memory during very long running tests or when the run-log grows quickly due to screenshots or output from the SUT. They are also more efcient when transforming run-logs into reports. However, explicit setting of Name for separate run-log attributes requires an understanding of the issues involved and either making decisions about where best to split a run-log or tedious typing when trying to split into small pieces. As a compromise, QF-Test makes a very rough calculation about the size of a runlog during executing, taking screenshots and program output into account. When execution of a Test-case or Test-set has nished and the approximate size of the run-log pertaining to that node is larger than the threshold specied in this option, the run-log is split off and saved automatically. A value of 0 prevents automatic splitting. See section 8.1.4
3.4+
(67)

for further information about split run-logs.

Name for automatically split Test-case run-log (System) Server script name: OPT_LOG_AUTO_SPLIT_TESTCASE_NAME This option species the name to use for an external log when it is split off automatically after execution of a Test-case has nished as described in the previous option. Variables can be used as well as the %... placeholders (291) documented for the attribute Name for separate run-log .

29.6.

Run-log options

279

The special variable ${qftest:testcase.splitlogname} is a good base. It expands to a path name created from the name of the Test-case with possible parent Test-set nodes as directories. See section 8.1.4
3.4+
(67)

for further information about split run-logs.

Name for automatically split Test-set run-log (System) Server script name: OPT_LOG_AUTO_SPLIT_TESTSET_NAME This option species the name to use for an external log when it is split off automatically after execution of a Test-set has nished as described in the option (278) Minimum size for automatic splitting (kB) . Variables can be used as well as the (297) %... placeholders documented for the attribute Name for separate run-log . The special variable ${qftest:testset.splitlogname} is a good base. It expands to a path name created from the name of the Test-set with possible parent Test-set nodes as directories. See section 8.1.4
(67)

for further information about split run-logs.

29.6.

Run-log options

280

29.6.2

Options determining run-log content

Figure 29.27: Options determining run-log content

Log variable expansion (System) Server script name: OPT_LOG_CONTENT_VARIABLE_EXPANSION (52) If this option is activated, every variable expansion is logged. Log parent nodes of components (System) Server script name: OPT_LOG_CONTENT_PARENT_COMPONENTS Setting this option will cause all direct and indirect parent nodes to be logged in addition to the target component node for every event, check, etc. Log level for the SUT (System) Server script name: OPT_LOG_CONTENT_SUT_LEVEL

29.6.

Run-log options

281

Possible Values: VAL_LOG_SUT_LEVEL_MESSAGE, VAL_LOG_SUT_LEVEL_WARNING, VAL_LOG_SUT_LEVEL_ERROR The level for automatically generated messages in the SUT during replay, e.g. details for component recognition. Only messages with the respective level, i.e. plain messages, warnings or errors will be logged. This option has no effect on messages created explicitly via rc.logMessage or qf.logMessage. Number of events to log for error diagnosis (System) SUT script name: OPT_LOG_CONTENT_DIAGNOSIS During replay of a test QF-Test logs various events and other things going on behind the scenes. This information is quickly discarded except when an error happens. In that case the most recent events are written to a special run-log node. The information may also be useful to developers but is mostly required for error diagnosis when requesting support from Quality First Software GmbH. This option determines the number of recent internal events to keep. Setting it to 0 disables the feature altogether. You should not set this value to less than about 400 without a good reason. Because the information is logged only for errors, the cost for gathering it is minimal. Maximum number of errors with screenshots per run-log (System) Server script name: OPT_LOG_CONTENT_SCREENSHOT_MAX The maximum number of screenshots that QF-Test takes and stores in the run-log during a test-run on situations of exception or errors. Setting this value to 0 disables taking screenshots entirely, a negative value means unlimited screenshots. Count screenshots individually for each split log (System) Server script name: OPT_LOG_CONTENT_SCREENSHOT_PER_SPLIT_LOG If this option is set, each partial log of a split run-log may contain the maximum number of screenshots dened above without affecting the count for the main run-log. Otherwise the limit applies for the sum of all parts belonging to the same main run-log. See section 8.1.4
(67)

for further information about split run-logs.

Create screenshots of the whole screen (System) Server script name: OPT_LOG_CONTENT_SCREENSHOT_FULLSCREEN

29.6.

Run-log options

282

Activating this option causes QF-Test to take an image of the whole screen and save it in the run-log when a screenshot is triggered by an exception or error. Create screenshots of individual windows (System) Server script name: OPT_LOG_CONTENT_SCREENSHOT_WINDOW Activating this option causes QF-Test to record images of all windows and dialogs of the SUT and store them in the run-log when screenshots are triggered due to exceptions or errors. Most of the time this will work even for windows that are covered by other windows or in cases where a full screenshot is not possible, for example when a screen is locked. Log successful advanced image checks (System)
3.4+

Server script name: OPT_LOG_CONTENT_SCREENSHOT_ADVANCED_ALWAYS If this option is activated, QF-Test will store the expected and actual images as well as the transformed images for successful advanced image checks in the run-log. Otherwise these details are kept for failed image checks only. Activating this option can raise the size of the run-log drastically so be sure to use it in combination with compact run-logs and split run-logs. Create compact run-log (System) Server script name: OPT_LOG_CONTENT_COMPACT Activating this option causes QF-Test to discard every node from a run-log that is neither relevant for error diagnosis, nor for the XML/HTML report. After an error or exception, as well as at the end of a test-run, the 100 most recent nodes are not discarded, so all the relevant information should remain available. Large tests should almost always be run with this option enabled to reduce the risk of running out of memory due to excessively large run-logs. This option is used only when QF-Test is run in interactive mode. It is ignored in (5) batch mode (see section 1.5 ) to avoid accidental loss of information. To create a (542) compact run-log in batch mode, use the -compact command line argument. Dont create run-log (System) Server script name: OPT_LOG_CONTENT_SUPPRESS For very-long-running tests or demos that are run in an endless loop, memory consumption of the run-log is an issue, but split run-logs are an ideal solution. Before split run-logs were available, turning run-logs off completely via this option

29.7.

Variables

283

was sometimes the only way to get long-running tests to work. Now this option is only retained for backwards compatibility.
Note

In batch mode this option is ignored. To suppress the run-log, use the argument (545) -nolog . Log SUT output individually (System)

3.0+

SUT script name: OPT_LOG_CONTENT_SPLIT_IO If set, any text that an SUT client prints to its stdout or stderr stream is also logged in the run-log. For each interaction with the SUT QF-Test collects text printed before the event and after the event during synchronisation. This makes it possible to associate output like an excetption stacktrace that is triggered by an event with the event itself, something that is impossible if all output is kept in a single piece. Compactify individually logged SUT output (System)
3.0+

Server script name: OPT_LOG_CONTENT_COMPACTIFY_SPLIT_IO Output from an SUT client tends to accumulate and can consume a lot of memory. If this option is activated, individually logged SUT output for events that are no longer of interest can be removed along with the events in compact (282) run-logs. Please see the option Create compact run-log for further information about compact run-logs.

29.7

Variables

The following options pertain to variable binding.

Figure 29.28: Variable options

When binding variables, expand values immediately (System)

3.0+

Server script name: OPT_VARIABLE_IMMEDIATE_BINDING

29.7.

Variables

284

When a set of variable bindings is pushed onto a variable stack, any additional variable references in the values of these variables can either be expanded immediately, or the values can be left unchanged to be expanded lazily as (60) needed. This is explained in detail in section 7.7 . For immediate expansion turn this option on, for lazy expansion turn it off. Fall back to lazy binding if immediate binding fails (System)
3.0+

Server script name: OPT_VARIABLE_LAZY_FALLBACK Old tests that rely on lazy variable binding may fail with the new default of immediate binding. If this option is activated, variables that cannot be bound immediately due to references to not-yet-dened variables are bound lazily instead and a warning is issued. To get rid of the warning simply change the value of the respective variable to use explicit lazy binding with $_. Please see (60) section 7.7 for further information. Furhtermore, various kinds of variables can be dened here. These are explained in (52) chapter 7 .

Chapter 30 Elements of a test-suite

30.1 The test-suite and its structure

There are more than 60 different kinds of nodes for a test-suite which are all listed in this chapter. Each node type has a unique set of features. The attributes of a node are displayed and edited in the detail view of the editor. The restrictions that apply to each (52) attribute are listed as well as whether it supports variable expansion (see chapter 7 ). Additional features of a node include the behavior of the node during execution of a test and the kinds of parent and child nodes allowed.

30.1.1 Test-suite
The root node of the tree represents the test-suite itself. Its basic structure (303) is xed. The root node contains an arbitrary number of Tests , followed (361) (315) (524) by the Procedures the Extras and the Windows and components . When executed, the top-level Tests are executed one by one. Contained in: None Children: An arbitrary number of Test nodes, followed by the Procedures, Extras and Windows and components nodes. Execution: The top-level Test nodes are executed one by one. Attributes:

30.1.

The test-suite and its structure

286

Figure 30.1: Test-suite attributes

Include les This is a list of test-suites that are included by the suite. If a Component or (352) Procedure reference cannot be resolved in the current suite, the included suites are searched for it. When recording new components, QF-Test will search the included suites for a matching Component node before creating a new one. Relative pathnames are treated relative to the directory of the suite, or to a direc(231) tory on the library path (see option Directories holding test-suite libraries ). When you change anything in this attribute QF-Test will offer to update all affected nodes to compensate for the change. For example, if you add or remove a suite from the includes, QF-Test will make all references to that suites Procedures or Components implicit or absolute so that the actual referenced nodes remain unchanged. In such a case, choose Yes. If, on the other hand, you renamed a suite or moved it to some other directory and are simply updating the includes to reect that, chose No so all former implicit references into the old suite will now point to the new one. Variable: No Restrictions: None Dependencies (reverse includes) This list of test-suites is the reverse of the Include les attribute. It has no impact
(514)

30.1.

The test-suite and its structure

287

on test execution. Instead it serves a hint to tell QF-Test which test-suites depend on Components in this suite, either because they (directly or indirectly) include this suite or because they explicitely reference Components in it. This information is used when Ids of Components are manipulated (for example after updating (43) components, see (section 5.8 )) and the Component id attributes of nodes depending on these components have to be updated. QF-Test always checks all currently loaded suites for dependencies. Additionally, it will automatically load and check all suites listed here. Relative pathnames are treated relative to the directory of the suite, or to a direc(231) tory on the library path (see option Directories holding test-suite libraries ).
Note

Like the Include les, the Dependencies are also resolved indirectly. For example if suite A has the dependency B which has the dependency C, both suites B and C will be loaded and checked for references automatically when components in suite A are manipulated. Variable: No Restrictions: None Variable denitions These variables denitions are identical to the suite variable bindings accessible (283) pane of the global options. A detailed explanation of variable in the Variables (52) (11) denition and lookup is given in chapter 7 . See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.

Test and Sequence nodes

288

30.2

Test and Sequence nodes

(288)

Tests and sequences are the main structural elements of a test-suite. Test-case nodes (303) and their Test node predecessors represent logical test-cases and are implemented as specialized sequences. A sequence is a container that executes its child nodes one (52) by one. It can dene variables (see chapter 7 ) that remain bound during the execution of the children. Other kinds of sequences differ either in the way their child nodes are executed, or in the restrictions they impose on their child or parent nodes.

30.2.1 Test-case
A Test-case node represents one or more logical test cases. In a sense it is the most important of all QF-Test nodes and everything else only serves to lend structure to Test-cases or to implement their logic. Functionally it is a highly (307) specialized Sequence with a number of important extensions and special attributes. A Test-case should focus on the actual test it supposed to perform. Setup and cleanup tasks required to ensure that the Test-case executes in the required environment and does not interfere with subsequent tests should be implemented in the form of (317) (85) Dependencies as described in chapter 12 . Alternatively - or in addition to (323) (325) Dependencies - a Test-case can have Setup and Cleanup nodes to be executed before and after the Test-case. Because a Test-case can be called from a Test call node it is also somewhat simi(352) (290) lar to a Procedure in that its Name attribute is mandatory and that it has a list of (293) Parameter default values that can be overridden in the calling node. Test-cases also play a central role in run-logs and test reports. During a test-run a Testcase node can be executed several times in different contexts and with different parameters. Logically these executions may represent the same or different test-cases. (292) By dening a set of Characteristic variables you can specify which variable values are used to differentiate between executions, thus characterizing the run-time environment of the test. The values of these variables at the time of entry to the Test-case are stored in the run-log. To emphasize the fact that each execution of a Test-case node may represent a seperate logical test-case there is an alternative name attribute called (290) Name for run-log and report . Its value may contain references to the Characteristic variables of the test. In the run-log or report the test will then be listed with this name, including the expanded run-time variable values. Finally, there are situations in which a test cannot or should not be executed for specic (292) variable settings. If the Condition attribute of the Test-case is dened, the Test-case will only be executed if that expression expands to a true value. If the Test-case is not
(299)

30.2.

Test and Sequence nodes

289

executed due to the Condition it will be listed as skipped in the report. Contained in: Test-suite
(285)

, Test-set
(317)

(294)

.
(320)

Children: Optional Dependency or Dependency reference as the rst element. A (323) (325) Setup may be the next and a Cleanup the last node with an arbitrary number of normal child nodes in between. A Test-case that does not contain normal child nodes will be listed as not implemented in the report. of the Test-case are bound on the primary Execution: First the Variable denitions (293) (52) and its Parameter default values on the fallback variable stack (see chapter 7 ). With (292) these in place the Condition is evaluated and the node will be skipped if a non-empty Condition evaluates to false. Next the dependency of the Test-case - possibly inherited (85) from its parent node - is determined and resolved as described in chapter 12 . Then the optional Setup is executed once, followed by the normal child nodes and a single execution of the optional Cleanup. An exception raised during the course of the Test-case (382) will be caught and passed to the Dependency for handling in a Catch node. Even if the exception is not handled by a Dependency it is not propagated beyond the Test-case to prevent aborting the whole test-run. The error state is duly noted in run-log and report however. Attributes:
(293)

30.2.

Test and Sequence nodes

290

Figure 30.2: Test-case attributes

Name A Test-case is identied by its name and the names of its Test-set ancestors, so you should assign a name with a meaning that is easy to recognize and remember. Variable: No Restrictions: Must not be empty or contain the characters . or #.
(294)

30.2.

Test and Sequence nodes

291

Name for run-log and report A seperate name to be used for run-log and report. This is useful to differentiate between multiple executions with potentially different values for the Characteristic variables. Variable: Yes Restrictions: None Name for separate run-log If this attribute is set it marks the node as a breaking point for split run-logs and denes the lename for the partial log. When the node nishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains (67) references to the partial logs and is fully controllable. Please see section 8.1.4 for further information about split run-logs. This attribute has no effect if the option Create split run-logs is disabled or split (550) run-logs are explicitely turned off for batch mode via the -splitlog command line argument. There is no need to keep the lename unique. If necessary, QF-Test appends a number to the lename to avoid collisions. The lename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a % or a + character:
Character % + i r w e x t y Y M d h m s Replacement Literal % character. Literal + character. (549) The current runid as specied with -runid <id> . The error level of the partial log. The number of warnings in the partial log. The number of errors in the partial log. The number of exceptions in the partial log. The thread index to which the partial log belongs (for tests run with parallel threads). The current year (2 digits). The current year (4 digits). The current month (2 digits). The current day (2 digits). The current hour (2 digits). The current minute (2 digits). The current second (2 digits).
(277)

Table 30.1: Placeholders for the Name for separate run-log attribute

30.2.

Test and Sequence nodes Variable: Yes

292

Restrictions: None, characters that are illegal for a lename will be replaced with _. Inherit dependency from parent node This option allows inheriting the Dependency from the parent node as a replacement for or in addition to specifying a Dependency for the node itself. Variable: No Restrictions: None Condition If the condition is non-empty it will be evaluated and if the result is false the execution of the current node is aborted. In this case the node will be reported as skipped. of an If node, the Condition is evaluated by the Jython Like the Condition interpreter, so the same rules apply. Variable: Yes Restrictions: Valid syntax Expected to fail if... This attribute allows specifying a condition under which the Test-case is expected to fail. This is helpful to distinguish new problems from already known ones. Of course the latter should be xed, but if and when that happens may be outside the testers sphere of inuence. Most of the time it is sufcient to simply set this attribute to true to mark the Testcase as an expected failure. But if the Test-case is executed multiple times, e.g. (330) or on multiple systems and fails only in specic cases, the within a Data driver condition should be written so that it evaluates to true for exactly those cases, e.g. ${qftest:windows} for a test that fails on Windows but runs ne on other systems. If this attribute evaluates to true and the Test-case fails with an error, it will be listed separately in the run-log, report and on the status line. It still means that there is an error in the application, so the overall percentage of successful tests is not changed. It is treated as an error if this attribute evaluates to true and the Test-case does not fail because this means that either the test is not able to reproduce the problem reliably or that the problem has been xed and the Test-case must be updated accordingly. Variable: Yes Restrictions: Valid syntax
(371) (370) (317)

30.2.

Test and Sequence nodes

293

Characteristic variables These variables are part of the characteristics of a Test-set or Test-case. Two executions of a Test-case are considered to represent the same logical test-case if the run-time values of all Characteristic variables are identical. The run-time values of the Characteristic variables are stored in the run-log. Variable: No Restrictions: None Parameter default values Here you can dene default or fallback values for the Test-cases parameters (see (52) chapter 7 ). Dening these values also serves as documentation and is a valuable time-saver when using the dialog to select the Test-case for the (301) (299) (11) Test name attribute of a Test call . See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Variable denitions These variables are bound on the direct bindings stack (see chapter 7 ). They remain valid during the execution of the Test-cases child nodes and cannot be (299) (11) overridden by a Test call node. See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note
(52)

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None

30.2. Id

Test and Sequence nodes

294

When using the command line argument -test <n>|<id> for execution in batch mode you can specify the Id of the node as an alternative to its qualied name. Variable: Nein Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

(551)

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.2 Test-set
The main purpose of a Test-set is to give structure to the Test-cases of a (296) test-suite. Test-sets can be nested. The Name of a Test-set is part of the (299) Test-cases fully qualied name to which a Test call node refers. Test-sets are (298) that can be also callable themselves and thus have a set of Parameter default values overridden in the Test call. One way of structuring Test-cases is to collect those with similar setup and cleanup (317) requirements in the same Test-set which can provide a Dependency or (320) Dependency reference node to be inherited by the Test-case or nested Test-set child (298) nodes. If the Inherit dependency from parent node attribute of the Test-set is set, the (85) Dependency nodes of the Test-sets parent(s) are also inherited. See chapter 12 for
(288)

30.2.

Test and Sequence nodes

295

information about QF-Tests dependency mechanism. Alternative or additional setup (323) (325) and cleanup is available in the form of Setup and Cleanup nodes which will be executed before and after each of the Test-set and Test-case child nodes. Another important feature of a Test-set is data-driven testing. This can be achieved by (330) (116) adding a Data driver node with one or more Data binders as described in chapter 14 . Like a Test-case, a Test-set plays an important role for reports. As it can also be executed (298) several times with different parameter settings, it has a set of Characteristic variables (296) that work just like for a Test-case. The and an alternative Name for run-log and report (298) same is true for the Condition which can be used to skip an entire Test-set depending on current variable values. Contained in: Test-suite
(285)

, Test-set
(317)

(294)

.
(320)

Children: Optional Dependency or Dependency reference as the rst child, followed (330) (323) (325) by an optional Data driver . A Setup may come next and a Cleanup at the end with (294) (288) (299) an arbitrary number of Test-set , Test-case and Test call nodes in between. Execution: First the Parameter default values of the Test-set are bound on the fallback (52) (298) variable stack (see chapter 7 ). With these in place the Condition is evaluated and (317) the node will be skipped if a non-empty Condition evaluates to false. The Dependency (320) or Dependency reference node will not be resolved at this time unless its (318) Always execute, even in test-suite and test-set nodes attribute is set. If there is a (330) Data driver node, it is executed to create a data driving context, bind one or more Data binders and then repeatedly execute the other child nodes as described in chapter (116) 14 . For each loop iteration - or once in case no Data driver is present - the Test-set (323) executes each of its Test-set or Test-case child nodes. If an optional Setup or (325) Cleanup node are present, they are executed before and after each of these child (303) nodes respectively, just like in a Test node. Attributes:
(298)

30.2.

Test and Sequence nodes

296

Figure 30.3: Test-set attributes

Name The name of a Test-set is part of its own identication and of that of the Test-case and Test-set nodes it contains, so you should assign a name with a meaning that is easy to recognize and remember. Variable: No Restrictions: Must not be empty or contain the characters . or #. Name for run-log and report
(288)

30.2.

Test and Sequence nodes

297

A seperate name to be used for run-log and report. This is useful to differentiate between multiple executions with potentially different values for the Characteristic variables. Variable: Yes Restrictions: None Name for separate run-log If this attribute is set it marks the node as a breaking point for split run-logs and denes the lename for the partial log. When the node nishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains (67) references to the partial logs and is fully controllable. Please see section 8.1.4 for further information about split run-logs. This attribute has no effect if the option Create split run-logs is disabled or split (550) run-logs are explicitely turned off for batch mode via the -splitlog command line argument. There is no need to keep the lename unique. If necessary, QF-Test appends a number to the lename to avoid collisions. The lename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a % or a + character:
Character % + i r w e x t y Y M d h m s Replacement Literal % character. Literal + character. (549) The current runid as specied with -runid <id> . The error level of the partial log. The number of warnings in the partial log. The number of errors in the partial log. The number of exceptions in the partial log. The thread index to which the partial log belongs (for tests run with parallel threads). The current year (2 digits). The current year (4 digits). The current month (2 digits). The current day (2 digits). The current hour (2 digits). The current minute (2 digits). The current second (2 digits).
(277)

Table 30.2: Placeholders for the Name for separate run-log attribute

Variable: Yes

30.2.

Test and Sequence nodes

298

Restrictions: None, characters that are illegal for a lename will be replaced with _. Inherit dependency from parent node This option allows inheriting the Dependency from the parent node as a replacement for or in addition to specifying a Dependency for the node itself. Variable: No Restrictions: None Condition If the condition is non-empty it will be evaluated and if the result is false the execution of the current node is aborted. In this case the node will be reported as skipped. of an If node, the Condition is evaluated by the Jython Like the Condition interpreter, so the same rules apply. Variable: Yes Restrictions: Valid syntax Characteristic variables These variables are part of the characteristics of a Test-set or Test-case. Two executions of a Test-case are considered to represent the same logical test-case if the run-time values of all Characteristic variables are identical. The run-time values of the Characteristic variables are stored in the run-log. Variable: No Restrictions: None Parameter default values Here you can dene default or fallback values for the Test-sets parameters (see (52) chapter 7 ). Dening these values also serves as documentation and is a (301) valuable time-saver when using the dialog to select the Test-set for the Test name (299) (11) attribute of a Test call . See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute,
(371) (370) (317)

30.2.

Test and Sequence nodes

299

the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id When using the command line argument -test <n>|<id> for execution in batch mode you can specify the Id of the node as an alternative to its qualied name. Variable: Nein Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
(551)

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.3 Test call

30.2.

Test and Sequence nodes

300

With this node a call to some other executable test node can be made. Pos(285) (294) (288) and Test-case nodes in the same sible targets are Test-suite , Test-set or a different test-suite. Execution will continue in the called node and when nished return to the Test call and thus to its parent node. The name of a Test-case or Test-set to call is determined by its Name and the Names of its Test-set ancestors. These are concatenated with a dot (.) as separator, starting with the outermost Test-set and ending in the Test-cases name. Thus to call a Test-case named nodeTest in a Test-set named Tree that is itself a child of a Test-set named (301) Main, the Test name attribute would be set to Main.Tree.nodeTest. A node in a different test-suite is addressed by prepending the lename of the test-suite followed by a # to the Test name. A Test-suite node is addressed by a single ., so calling a whole test-suite is done with a Test name attribute of the form suitename.qft#.. It is generally easiest to pick the target node interactively from a dialog by clicking on the button above the Test name attribute. See also section 17.1 Children: None of the Test call are bound, the target node is deExecution: The Variable denitions termined and execution is passed to it. After the Test call returns, the Test calls variables are unbound again. Attributes:
(302) (145) (290) (296)

for further information about cross-suite calls.

(288)

Contained in: All kinds of sequences

30.2.

Test and Sequence nodes

301

Figure 30.4: Test call Attributes

Name In case of a Test-case or Test-set it is the full name, created from the Name of its (290) Test-set parents and its own Name , joined by a dot. A Test-suite node is addressed by a single . character. The button above the attribute brings up a dialog in which you can select the target You can also get to node interactively. this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld. By selecting the Copy parameters checkbox you can adopt the Test-cases or Test-sets Parameter default values as parameters for the Test call node to save typing. Variable: Yes Restrictions: Must not be empty. Name for separate run-log If this attribute is set it marks the node as a breaking point for split run-logs and denes the lename for the partial log. When the node nishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains (67) references to the partial logs and is fully controllable. Please see section 8.1.4 for further information about split run-logs.
(296)

30.2.

Test and Sequence nodes

(277)

302

This attribute has no effect if the option Create split run-logs is disabled or split (550) run-logs are explicitely turned off for batch mode via the -splitlog command line argument. There is no need to keep the lename unique. If necessary, QF-Test appends a number to the lename to avoid collisions. The lename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a % or a + character:
Character % + i r w e x t y Y M d h m s Replacement Literal % character. Literal + character. (549) The current runid as specied with -runid <id> . The error level of the partial log. The number of warnings in the partial log. The number of errors in the partial log. The number of exceptions in the partial log. The thread index to which the partial log belongs (for tests run with parallel threads). The current year (2 digits). The current year (4 digits). The current month (2 digits). The current day (2 digits). The current hour (2 digits). The current minute (2 digits). The current second (2 digits).

Table 30.3: Placeholders for the Name for separate run-log attribute

Variable: Yes Restrictions: None, characters that are illegal for a lename will be replaced with _. Variable denitions This is where you dene the parameter values for the target node. These (52) variables are bound on the primary variable stack (see chapter 7 ) so they (11) override any Parameter default values. See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Id

30.2.

Test and Sequence nodes At the moment the Id attribute has no meaning for this type of node. Variable: No

303

Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.4 Test
Note

Before QF-Test version 2 the Test node was one of the main building blocks of a test-suite. It had a few shortcomings however: Its setup/cleanup structure was too linear and inexible for complex scenarios and it was never clear whether a Test node represented a logical test-case or was just used to implement some (294) (288) sequence. Thus Test nodes have been replaced with Test-set and Test-case nodes. Tests are still available for backwards compatibility and because they can be useful at lower levels or in situations where Test-set and Test-case nodes are not applicable. A Test is a special Sequence that executes extra setup and cleanup code before and after the execution of its child nodes to ensure that each of the children runs under similar conditions and to prevent unwanted side effects between the execution of one child and the next. To do so, a Test has two special, optional child nodes besides its (323) (325) normal children, a Setup as the rst and a Cleanup as the last node.
(307)

30.2.

Test and Sequence nodes

(306)

304

With its Implicitly catch exceptions attribute a Test also offers special exception handling to prevent exceptions in one Test from aborting a whole test-run. For special cases of data driven testing a Test may also contain a Data driver , whereas (294) (116) such is typically done in combination with Test-sets as described in chapter 14 . Contained in: All kinds of sequences
(330) (288) (330)

.
(323)

Children: Optional Data driver followed by an optional Setup at the beginning, then (325) any kind of executable nodes and an optional Cleanup as last node. Execution: The Variable denitions of the Test are bound. If there is a Data driver node, it is executed to create a data driving context and bind one or more Data binders (116) for iteration over the determined data sets as described in chapter 14 . For each of (323) (325) its normal child nodes, the Setup is executed, then the child and then the Cleanup . After the last execution of the Cleanup is complete, the variables are unbound again. Attributes:
(306) (330)

Figure 30.5: Test attributes

Name

30.2.

Test and Sequence nodes

305

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Name for separate run-log If this attribute is set it marks the node as a breaking point for split run-logs and denes the lename for the partial log. When the node nishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains (67) references to the partial logs and is fully controllable. Please see section 8.1.4 for further information about split run-logs. This attribute has no effect if the option Create split run-logs is disabled or split (550) run-logs are explicitely turned off for batch mode via the -splitlog command line argument. There is no need to keep the lename unique. If necessary, QF-Test appends a number to the lename to avoid collisions. The lename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a % or a + character:
Character % + i r w e x t y Y M d h m s Replacement Literal % character. Literal + character. (549) The current runid as specied with -runid <id> . The error level of the partial log. The number of warnings in the partial log. The number of errors in the partial log. The number of exceptions in the partial log. The thread index to which the partial log belongs (for tests run with parallel threads). The current year (2 digits). The current year (4 digits). The current month (2 digits). The current day (2 digits). The current hour (2 digits). The current minute (2 digits). The current second (2 digits).
(277)

Table 30.4: Placeholders for the Name for separate run-log attribute

Variable: Yes

30.2.

Test and Sequence nodes

306

Restrictions: None, characters that are illegal for a lename will be replaced with _. Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Implicitly catch exceptions When an exception is thrown during the execution of one of the Tests normal child nodes, the Test is usually terminated prematurely. This may not be what you want, since no information is gained from the execution of the rest of the child nodes. If the Setup and Cleanup of the test are set up so you can guarantee the same initial conditions for each child node even in the case of an exception, you can set this attribute to make the Test catch the exception implicitly. That way, if an exception is caught from a normal child node, the exception is logged and the execution of that child node is stopped. Then the Test continues with the Cleanup as if nothing had happened. Exceptions thrown during the execution of either the Setup or the Cleanup cannot be caught that way and will always terminate the Test Variable: No Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None

30.2. Id

Test and Sequence nodes

307

When using the command line argument -test <n>|<id> for execution in batch mode you can specify the Id of the node as an alternative to its qualied name. Variable: Nein Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

(551)

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.5 Sequence
This is the most general kind of sequence. Its children are executed one by one and their number or type is not limited in any way. Contained in: All kinds of sequences Children: Any Execution: The Variable denitions of the Sequence are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(308) (288)

30.2.

Test and Sequence nodes

308

Figure 30.6: Sequence attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a

30.2.

Test and Sequence nodes

309

run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.6 Test-step

30.2.

Test and Sequence nodes

310

A Test-step is a special Sequence that serves to divide a Test-case into steps that can be documented individually and will show up in the report or testdoc documentation. In contrast to Test-cases, which should not be nested, Teststeps can be nested to any depth. Contained in: All kinds of sequences Children: Any Execution: The Variable denitions of the Test-step are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(311) (288)

Figure 30.7: Test-step attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No

30.2.

Test and Sequence nodes Restrictions: None

311

Name for run-log and report A seperate name to be used for run-log and report. This is useful to differentiate between multiple executions of the surrounding Test-case with potentially different values for its Characteristic variables. Variable: Yes Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after

30.2.

Test and Sequence nodes

312

These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.7 Sequence with time limit

This node extends the basic Sequence node with time-constraint checking. Child nodes are executed as usual, but upon completion of the sequence the elapsed time is compared to the time-limit. Exceeding the time limit will result in a warning, error or exception, depending on the value of the attribute (314) (315) Error level if time limit exceeded . Explicit delays like Delay before/after or user interrupts are deducted from the duration before the constraints are checked, unless (313) Check realtime is activated. For report generation, time-constraints are treated like checks. If the Comment tribute starts with an ! character, the result will be logged in the report.
Note
(315) (307)

at-

The function of this node is to check time constraints in the user-acceptance range, i.e. between a few hundred milliseconds and a few seconds. Real-time constraints of a few milliseconds or less are beyond its limits. Contained in: All kinds of sequences Children: Any Execution: The Variable denitions of the Sequence with time limit are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. The elapsed time is compared to the given time limit. Attributes:
(314) (288)

30.2.

Test and Sequence nodes

313

Figure 30.8: Sequence with time limit attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Time limit The time (in milliseconds) allowed for the execution of the sequence. Variable: Yes Restrictions: Must not be negative. Check realtime

30.2.

Test and Sequence nodes

(315)

314

Normally explicit delays like Delay before/after or user interrupts are deducted from the duration, before the time constraints are checked. To prevent this deduction and therefore check the real-time, this attribute can be activated. Variable: No Restrictions: None Error level if time limit exceeded This attribute determines what happens in case the time limit is exceeded. If set (529) to exception, a CheckFailedException will be thrown. Otherwise a message with the respective error-level will be logged in the run-log. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No

30.2.

Test and Sequence nodes

315

Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.2.8 Extras
This node is a kind of clipboard or playground where the usual restrictions on the parent of a node dont apply. You can add any kind of node here to assemble and try out some test sequences. Contained in: Root node Children: Any Execution: Cannot be executed Attributes:

30.3.

Dependencies

316

Figure 30.9: Extras attributes

Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.3

Dependencies

Dependencies are an advanced feature for handling setup and cleanup requirements for test-cases automatically. The goal is to isolate test-cases so that each one can be run independently without interfering with others. This is a very important requirement for things like testing arbitrary sub-sets of test-cases, for example to re-test only failed tests, or during test-development where it must be possible to quickly run and debug any given test-case.

30.3.

Dependencies

317

30.3.1 Dependency
Dependency nodes are used to implement advanced, automatic handling of (294) (288) and Test-cases . A detailed setup and cleanup requirements for Test-sets (85) description of the Dependency mechanism is given in chapter 12 . This section focuses on formal requirements for the Dependency node and its attributes. As Dependencies are complex, they should be reused as much as possible. This can be done by grouping Test-cases with identical dependencies in a Test-set and have them inherit the Dependency of the Test-set. However, this mechanism alone is not exible (352) enough, so a Dependency can also be implemented just like a Procedure and placed (361) among the Procedures of a test-suite to be referenced from a Dependency reference (290) attribute is mandatory and it also has a list of node. For this to work, the Name (319) Parameter default values that can be overridden in the referencing node. The Characteristic variables of a Dependency are part of its identity and play an important role in the dependency resolution mechanism. Contained in: Test-suite
(285) (319)

, Test-set

(294)

, Test-case

(288)

, Procedures

(361)

, Package

(359)

on which the Dependency is based, Children: Zero or more Dependency references (323) (325) (328) optional Setup and Cleanup nodes and an optional Error handler followed by zero (382) or more Catch nodes. Execution: Normally Dependencies are executed only indirectly in the setup phase of a Test-set or Test-case. If a Dependency node is executed interactively, the dependency (85) stack is resolved as described in chapter 12 . Attributes:

(320)

30.3.

Dependencies

318

Figure 30.10: Dependency attributes

Name A Dependency is identied by its name, so you should assign a name with a meaning that is easy to recognize and remember. Variable: No Restrictions: Must not be empty or contain the characters . or #. Name for run-log and report A seperate name to be used for run-log and report. This is useful to differentiate between multiple executions with potentially different values for the Characteristic variables. Variable: Yes Restrictions: None

30.3.

Dependencies

319

Always execute, even in test-suite and test-set nodes Normally a Dependency is only executed if it belongs to a Test-case node. Dependencies in Test-suite or Test-set nodes are simply inherited by the Test-case descendants of these nodes. However, in some cases it is useful to resolve a Dependency early, for example when the Dependency provides parameters for a (292) of a subsequent Test-case. test-run that are required to evaluate a Condition This can be achieved by activating this option. Variable: No Restrictions: None Forced cleanup Normally Dependencies are only rolled back and their cleanup code executed as (85) required by the dependency resolution mechanism described in chapter 12 . In some cases it makes sense to force partial cleanup of the dependency stack immediately after a Test-case nishes. This is what the Forced cleanup attribute is for. If this option is activated, the dependency stack will be rolled back at least up to and including this Dependency . Variable: No Restrictions: None Characteristic variables These variables are part of a Dependencys identity. During Dependency resolution (85) as described in chapter 12 two Dependencies are considered equal only if they are one and the same node and the run-time values of all their Characteristic variables are identical. Additionally, the values of the Characteristic variables are stored during the setup phase of the Dependency. Later, when the Dependency is rolled back, these settings will be temporarily restored for the cleanup phase. Variable: No Restrictions: None Parameter default values Here you can dene default or fallback values for the Dependencys parameters (52) (see chapter 7 ). Dening these values also serves as documentation and is a valuable time-saver when using the dialog to select the Dependency for the (321) (320) Referenced dependency attribute of a Dependency reference . See section (11) 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None

30.3. Id

Dependencies

320

At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.3.2 Dependency reference

A Dependency reference is simply a stand-in for a Dependency dened in (359) some other place, typically a Package . The name of the referenced De(318) (360) pendency is determined by its Name and the Names of its Package parents. These are concatenated with a dot (.) as separator, starting with the outermost Package and ending in the Dependencys name. Thus to reference a Dependency named demoStarted in a Package named Demo that is itself a child of a Package named Main, (321) set the Referenced dependency attribute to Main.Demo.demoStarted. See section 17.1 Children: None
(145) (317)

about how to reference a Dependency in a different test-suite.

(285)

Contained in: Test-suite

, Test-set

(294)

, Test-case

(288)

and Dependency

(317)

30.3.

Dependencies

321

Execution: Normally Dependency references are executed only indirectly in the setup phase of a Test-set or Test-case. If a Dependency reference node is executed interactively, the referenced Dependency is determined and the dependency stack resolved accord(85) ingly as described in chapter 12 . Attributes:

Figure 30.11: Dependency reference attributes

Referenced dependency of its Package The full name of the Dependency , created from the Names (318) parents and its own Name , joined by a dot. The button above the attribute brings up a dialog in which you can select the Dependency interactively. By selecting the Copy parameters checkbox you can adopt the Dependencys (319) Parameter default values as parameters for the Dependency reference node to save typing. Variable: Yes Restrictions: Must not be empty. Dependency namespace
(317) (360) (359)

30.3.

Dependencies

322

Normally there is only a single stack of dependencies, but in some cases, e.g. when mixing tests for independent SUT clients, it can be useful to have independent sets of dependencies for different parts of the test, so that resolving the dependencies for one part doesnt necessarily tear down everything required for a different part. By setting the Dependency namespace attribute of a Dependency reference node you can tell QF-Test to resolve the target dependency in that namespace. The default dependency stack will be completely ignored in that case. If there is a dependency stack remaining from a previous dependency resolution in the same namespace it will be used for comparison instead, otherwise a new stack will be created. Variable: Yes Restrictions: None Always execute, even in test-suite and test-set nodes Normally a Dependency is only executed if it belongs to a Test-case node. Dependencies in Test-suite or Test-set nodes are simply inherited by the Test-case descendants of these nodes. However, in some cases it is useful to resolve a Dependency early, for example when the Dependency provides parameters for a (292) of a subsequent Test-case. test-run that are required to evaluate a Condition This can be achieved by activating this option. Variable: No Restrictions: None Variable denitions These variables override the Parameter default values of the Dependency (11) referenced by this Dependency reference. See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used.
(319)

30.3.

Dependencies Variable: Yes Restrictions: Valid number >= 0

323

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.3.3 Setup
This node is just like a Sequence except for its special place (317) (294) (288) (303) in a Dependency , Test-set , Test-case or Test node. Contained in: Dependency Children: Any of the Setup are bound and its child nodes exeExecution: The Variable denitions cuted one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(324) (317) (307)

, Test-set

(294)

, Test-case

(288)

or Test

(303)

30.3.

Dependencies

324

Figure 30.12: Setup attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a

30.3.

Dependencies

325

run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.3.4 Cleanup

30.3.

Dependencies
(307)

326

This node is just like a Sequence except for its special place (317) (294) (288) (303) in a Dependency , Test-set , Test-case or Test node. Contained in: Dependency Children: Any Execution: The Variable denitions of the Cleanup are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(326) (317)

, Test-set

(294)

, Test-case

(288)

or Test

(303)

Figure 30.13: Cleanup attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the

30.3.

Dependencies
(52)

327
(11)

execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets

30.3.

Dependencies

328

you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.3.5 Error handler

This node is just like a Sequence (317) (85) in a Dependency (see chapter 12 ). Contained in: Dependency Children: Any Execution: The Variable denitions of the Error handler are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(329) (317) (307)

except for its special place

Figure 30.14: Error handler attributes

30.3. Name

Dependencies

329

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used.

30.4.

Data driver Variable: Yes Restrictions: Valid number >= 0

330

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.4

Data driver

The term Data-driven Testing refers to a common method in automated testing where test-cases are executed several times with different sets of data dened. With QF-Tests highly exible variables there is no limit on how this data can be used but the most common case is for event input values and expected check values. node used to provide QF-Tests data-driving mechanism consists of the Data driver a data-driving context and several kinds of Data binders. Currently available are the (334) (346) Data table node that stores data internally within QF-Test the CSV data le node that (337) reads data from a CSV le, the Database node that reads data from a database and (342) the Excel data le that reads data from an Excel le. An extension API for plugging in arbitrary external data is also available. Further information about how the various parts of the data driver mechanism are work(116) ing together is provided in chapter 14 .
(330)

30.4.1 Data driver

Except for its special place in a Test-set , a Data driver is just like a normal (307) Sequence . It provides a context for one or more Data binders to register themselves during the execution of the Data driver. The Test-set then iterates over the sets of data provided by the registered Data binders and executes its child nodes
(294)

30.4.

Data driver
(116)

331

as described in chapter 14 . For this purpose a Data driver node needs to be placed in (294) (317) (323) a Test-set node, between the optional Dependency and Setup nodes. Contained in: Test-set Children: Any Execution: When a Test-set or Test is executed it checks for a Data driver and runs it. The contents of the Data driver node are not limited to Data binders, but can hold any executable node so that they can perform any setup that may be required to retrieve the (352) data. Thus it is also possible to share Data binders by placing them inside a Procedure and calling the Procedure from inside the Data driver. Any Data binders registered within this Data drivers context will then be queried for data by the Test-set. Attributes:
(294)

, Test

(303)

Figure 30.15: Data driver attributes

Name The name of a Data driver is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the kind of data provided. Variable: No

30.4.

Data driver Restrictions: None

332

Name for loop pass in the run-log A separate name for each iteration to be used in the run-log. It can make use of the variables bound as a result of the data-driving which makes it easier to locate a specic step of the iteration. Variable: Yes Restrictions: None Name for separate run-log If this attribute is set it marks the node as a breaking point for split run-logs and denes the lename for the partial log. Every time an iteration of the Data driver nishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains references to the partial logs and is fully controllable. Please see (67) section 8.1.4 for further information about split run-logs. This attribute has no effect if the option Create split run-logs is disabled or split (550) run-logs are explicitely turned off for batch mode via the -splitlog command line argument. There is no need to keep the lename unique. If necessary, QF-Test appends a number to the lename to avoid collisions. The lename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a % or a + character:
(277)

30.4.

Data driver
Replacement Literal % character. Literal + character. (549) The current runid as specied with -runid <id> . The error level of the partial log. The number of warnings in the partial log. The number of errors in the partial log. The number of exceptions in the partial log. The thread index to which the partial log belongs (for tests run with parallel threads). The current year (2 digits). The current year (4 digits). The current month (2 digits). The current day (2 digits). The current hour (2 digits). The current minute (2 digits). The current second (2 digits).

333

Character % + i r w e x t y Y M d h m s

Table 30.5: Placeholders for the Name for separate run-log attribute

Variable: Yes Restrictions: None, characters that are illegal for a lename will be replaced with _. Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used.

30.4.

Data driver Variable: Yes Restrictions: Valid number >= 0

334

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.4.2 Data table

A Data table provides a convenient interface for storing test-data in tabular form directly inside QF-Test. For details about the data-driving mechanism (116) please see chapter 14 . Contained in: Any Children: None Execution: The Data table expands variable values in the table according to the option (283) When binding variables, expand values immediately . Each row is expanded individually left to right, meaning that - within the same row - a cell may refer to a variable bound in a column further to the left. Then the Data table registers itself with the Data driver context. If no such context is available a warning message will be issued. When (294) will query the Data table in order to all Data binders have been registered the Test-set iterate over the available sets of data. Attributes:

30.4.

Data driver

335

Figure 30.16: Data table attributes

Name The Name of a Data binder is mandatory. It is used to distinguish Data binders in the (368) same Data driver context. A Break node executed during data-driven testing can be used to break out of a specic loop by referring to the Data binders Name. Variable: Yes Restrictions: None Iteration counter The name of the variable that the iteration counter will be bound to. Variable: Yes Restrictions: None Iteration ranges An optional set of indexes or ranges to use from the bound data. This is especially useful during test development in order to run sample tests with just a single index or a subset of the given data. Ranges are separated by ,. Each range is either a single index or an inclusive range of the form from-to or from:to where to is optional. Indexes or ranges

30.4.

Data driver

336

may be specied multiple times, overlap or be given in descending order. Indexes are 0-based, negative indexes are counted from the end, -1 being the last item. An invalid syntax or an index outside the valid data range will cause (529) a BadRangeException . The following table shows some example range specications and the resulting indexes, based on a set of 20 entries.
Iteration ranges 0 -2, -1 1-2,4:5 18:,-33-2,16:15 Resulting indexes [0] [18, 19] [1, 2, 4, 5] [18, 19, 17, 18, 19] [3, 2, 16, 15]

Table 30.6: Iteration range examples

Note

The value bound for the Iteration counter reects the index in the current interval, not the counter of actual iterations, e.g. if you specify 2 there will be a single iteration with the Iteration counter bound to 2, not 0. Variable: Yes Restrictions: Valid syntax and index values Data bindings This is where the actual test data is dened. Each column of the table represents one variable with its name specied in the column header. Each row is a set of data, one value per variable. Thus the number of rows determines the number of iterations of the data-driven loop. To start entering the data you rst need to add columns to the table to dene the variables, then add rows to ll in the values. (11) See also section 2.2.5 about how to work with the table. Variable: Yes, even the column headers Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after

30.4.

Data driver

337

These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.4.3 Database
A Database node is used to load external data from a database. To access the database it is required to have the jar le with the database driver in QF-Tests (587) plugin-directory (section 37.2 ) or in the CLASSPATH environment variable at QF-Tests start-up time. For more information about the connection mechanism to your database ask the developers or see www.connectionstrings.com. For further details about the data-driving mechanism please see chapter 14 Contained in: Any Children: None Execution: The Database loads the data from the database and expands variable values (283) according to the option When binding variables, expand values immediately . Each row is expanded individually left to right, meaning that - within the same row - a cell may refer to a variable bound in a column further to the left. Then the Database node registers itself with the Data driver context. If no such context is available a warning message will be issued. The result columns of the used SQL statement will be used for the variable names. The capitalization depends on the database driver, e.g. completely capitalized letters for Oracle. When the SQL statement has been executed all Data binders are (294) (294) registered for the Test-set . The Test-set will query the Database node in order to iterate over the available sets of data. Attributes:
(116)

30.4.

Data driver

338

Figure 30.17: Database attributes

Name The Name of a Data binder is mandatory. It is used to distinguish Data binders in the (368) same Data driver context. A Break node executed during data-driven testing can be used to break out of a specic loop by referring to the Data binders Name. Variable: Yes Restrictions: None Iteration counter The name of the variable that the iteration counter will be bound to. Variable: Yes Restrictions: None Iteration ranges

30.4.

Data driver

339

An optional set of indexes or ranges to use from the bound data. This is especially useful during test development in order to run sample tests with just a single index or a subset of the given data. Ranges are separated by ,. Each range is either a single index or an inclusive range of the form from-to or from:to where to is optional. Indexes or ranges may be specied multiple times, overlap or be given in descending order. Indexes are 0-based, negative indexes are counted from the end, -1 being the last item. An invalid syntax or an index outside the valid data range will cause (529) a BadRangeException . The following table shows some example range specications and the resulting indexes, based on a set of 20 entries.
Iteration ranges 0 -2, -1 1-2,4:5 18:,-33-2,16:15 Resulting indexes [0] [18, 19] [1, 2, 4, 5] [18, 19, 17, 18, 19] [3, 2, 16, 15]

Table 30.7: Iteration range examples

Note

The value bound for the Iteration counter reects the index in the current interval, not the counter of actual iterations, e.g. if you specify 2 there will be a single iteration with the Iteration counter bound to 2, not 0. Variable: Yes Restrictions: Valid syntax and index values SQL statement The SQL query that should be executed to get the desired test-data. This statement is supposed to be a select statement. Each column will stand for a variable. The capitalization of the columns depends on the kind of the used database driver, e.g. most of the Oracle drivers return completely capitalized variables. Variable: Yes Restrictions: Must not be empty Driver class The class name of the database driver.

Note

The jar le with the database driver has to be placed in the CLASSPATH or in

30.4.

Data driver QF-Tests plugin directory before launching QF-Test. Here is a list of the most common database drivers:

340

Database Borland Interbase DB2 Informix IDS Server MS SQL Server 2000 MS SQL Server 2005 mSQL MySQL Oracle Pointbase PostgreSQL Standard Driver Sybase

Classname of JDBC-driver interbase.interclient.Driver com.ibm.db2.jcc.DB2Driver com.informix.jdbc.IfxDriver ids.sql.IDSDriver com.microsoft.jdbc.sqlserver.SQLServerDriver com.microsoft.sqlserver.jdbc.SQLServerDriver COM.imaginary.sql.msql.MsqlDriver com.mysql.jdbc.Driver oracle.jdbc.driver.OracleDriver com.pointbase.jdbc.jdbcUniversalDriver org.postgresql.Driver sun.jdbc.odbc.JdbcOdbcDriver com.sybase.jdbc2.jdbc.SybDriver

Table 30.8: Database drivers

Variable: Yes Restrictions: Must not be empty Connection string The connection string for database connection, typically something like: jdbc:databasetype://databasehost/databasename. Here is a list of the most common database connection strings:

30.4.

Data driver
Example jdbc:derby:net://databaseserver :port/ jdbc:db2://database jdbc:hsqldb:file:database jdbc:interbase://databaseserver /database.gdb jdbc:microsoft:sqlserver://databaseserver : port;DatabaseName=database; jdbc:sqlserver://databaseserver : port;DatabaseName=database; jdbc:mysql://databaseserver /database jdbc:postgresql://databaseserver /database jdbc:odbc:database jdbc:oracle:thin:@databaseserver :port: database jdbc:sybase:tds:databaseserver :port/database

341

Database Derby IBM DB2 HSQLB Interbase MS SQL Server 2000 MS SQL Server 2005 MySQL PostgreSQL ODBC Data Sources Oracle Thin Sybase

Table 30.9: Database connection strings

Variable: Yes Restrictions: Must not be empty User The name of the user to use when connecting to the database. If your database connection doesnt require a user you can leave this eld empty. Variable: Yes Restrictions: None Password The password to use when connecting to the database. If your database connection doesnt require a password you can leave this eld empty. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_).

30.4.

Data driver

342

Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.4.4 Excel data le

An Excel data le is used to load external data from an Excel-le to provide test-data for data-driven testing. The rst row must contain the names of the variables to bind. The rest of the rows should contain the values to be used for the iteration steps.
Note

It is possible that the contents of some cells are not read correctly. This can happen especially for cells of the type date, time or currency. The reason is that Excel stores the value and the format of a cell separately and the Java package used to parse Excel les doesnt support all possibilities. Still, the Excel data le should read most cells correctly, but in case of problems please change the type of the problematic cells in the Excel le to text. Once read by QF-Test, all values are treated as strings anyway. For further details about the data-driving mechanism please see chapter 14 Contained in: Any Children: None Execution: The Excel data le node loads the data from the Excel le and expands variable values in the table according to the option When binding variables, expand (283) values immediately . Each row is expanded individually left to right, meaning that within the same row - a cell may refer to a variable bound in a column further to the left.
(116)

30.4.

Data driver

343

Then the Excel data le registers itself with the Data driver context. If no such context is available a warning message will be issued. When all Data binders have been registered (294) the Test-set will query the Excel data le node in order to iterate over the available sets of data. Attributes:

Figure 30.18: Excel data le attributes

Name The Name of a Data binder is mandatory. It is used to distinguish Data binders in the (368) same Data driver context. A Break node executed during data-driven testing can be used to break out of a specic loop by referring to the Data binders Name. Variable: Yes Restrictions: None Iteration counter

30.4.

Data driver The name of the variable that the iteration counter will be bound to. Variable: Yes Restrictions: None

344

Iteration ranges An optional set of indexes or ranges to use from the bound data. This is especially useful during test development in order to run sample tests with just a single index or a subset of the given data. Ranges are separated by ,. Each range is either a single index or an inclusive range of the form from-to or from:to where to is optional. Indexes or ranges may be specied multiple times, overlap or be given in descending order. Indexes are 0-based, negative indexes are counted from the end, -1 being the last item. An invalid syntax or an index outside the valid data range will cause (529) a BadRangeException . The following table shows some example range specications and the resulting indexes, based on a set of 20 entries.
Iteration ranges 0 -2, -1 1-2,4:5 18:,-33-2,16:15 Resulting indexes [0] [18, 19] [1, 2, 4, 5] [18, 19, 17, 18, 19] [3, 2, 16, 15]

Table 30.10: Iteration range examples

Note

The value bound for the Iteration counter reects the index in the current interval, not the counter of actual iterations, e.g. if you specify 2 there will be a single iteration with the Iteration counter bound to 2, not 0. Variable: Yes Restrictions: Valid syntax and index values Excel le name The name of the Excel le to read the test data. resolved relative to the directory of the test-suite. Relative path names are

The button above the attribute brings up a dialog in which you can select the Excel le interactively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld. Variable: Yes

30.4.

Data driver Restrictions: Must not be empty

345

Worksheet name The name of the worksheet that contains the test-data. If empty, the rst sheet will be used. Variable: Yes Restrictions: None Override date format This value species a date format which will be used for all date values specied in the Excel le. The format must be specied as for the Java class SimpleDateFormat, i.e. d for day, M for month, y for year. Thus dd stands for two digits of a day and similar for MM, yy or yyyy. Variable: Yes Restrictions: None Variables in rows If this attribute is checked, the names of the variables will be taken from the rst row. If its not checked the names will come from the rst column. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that

30.4.

Data driver
(226)

346

are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.4.5 CSV data le

A CSV data le node is used to load external data from a le and make it available for data-driven testing. CSV stands for Comma-separated Values, a more or less standard plain text le format. Each line in the le contains one set of data with the values separated by a separator character, often but not always a comma (,). For use with a CSV data le node the rst line of the CSV le must contain the names of the variables to bind. The rest of the lines should contain the values to be used for the iteration steps. Unfortunately, with CSV les there are non-uniform denitions for things like quoting, white-space, multi-line values or embedded separator characters. Two de-facto standards exist, one used by Microsoft Excel and one by the rest of the world. QF-Test supports both of these. For further details about the data-driving mechanism please (116) see chapter 14 . Contained in: Any Children: None Execution: The CSV data le node loads the data from the CSV le and expands variable values in the table according to the option When binding variables, expand values (283) immediately . Each row is expanded individually left to right, meaning that - within the same row - a cell may refer to a variable bound in a column further to the left. Then the CSV data le registers itself with the Data driver context. If no such context is available a warning message will be issued. When all Data binders have been registered the (294) Test-set will query the CSV data le node in order to iterate over the available sets of data. Attributes:

30.4.

Data driver

347

Figure 30.19: CSV data le attributes

Name The Name of a Data binder is mandatory. It is used to distinguish Data binders in the (368) same Data driver context. A Break node executed during data-driven testing can be used to break out of a specic loop by referring to the Data binders Name. Variable: Yes Restrictions: None Iteration counter The name of the variable that the iteration counter will be bound to. Variable: Yes Restrictions: None Iteration ranges An optional set of indexes or ranges to use from the bound data. This is

30.4.

Data driver

348

especially useful during test development in order to run sample tests with just a single index or a subset of the given data. Ranges are separated by ,. Each range is either a single index or an inclusive range of the form from-to or from:to where to is optional. Indexes or ranges may be specied multiple times, overlap or be given in descending order. Indexes are 0-based, negative indexes are counted from the end, -1 being the last item. An invalid syntax or an index outside the valid data range will cause (529) a BadRangeException . The following table shows some example range specications and the resulting indexes, based on a set of 20 entries.
Iteration ranges 0 -2, -1 1-2,4:5 18:,-33-2,16:15 Resulting indexes [0] [18, 19] [1, 2, 4, 5] [18, 19, 17, 18, 19] [3, 2, 16, 15]

Table 30.11: Iteration range examples

Note

The value bound for the Iteration counter reects the index in the current interval, not the counter of actual iterations, e.g. if you specify 2 there will be a single iteration with the Iteration counter bound to 2, not 0. Variable: Yes Restrictions: Valid syntax and index values CSV le name The name of the CSV le to get the test data from. Relative path names are resolved relative to the directory of the test-suite. The button above the attribute brings up a dialog in which you can select the CSV le interactively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld. Variable: Yes Restrictions: Must not be empty File encoding An optional encoding for the CSV le, UTF-8 for example. If no encoding is specied, the le will be read with the default encoding of the Java VM. Variable: Yes

30.4.

Data driver Restrictions: The encoding must be supported by the Java VM.

349

Read Microsoft Excel CSV format If this option is active QF-Test will try to parse the CSV le using the format used by Microsoft Excel. Variable: Yes Restrictions: None Separator character In this attribute you can specify the character to be used as separator for data values within the CSV le. If no separator is dened a comma (,) is used as the (349) default value. If Read Microsoft Excel CSV format is activated this attribute is ignored. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.4.

Data driver

350

30.4.6 Data loop

A Data loop node is a simple loop with a single variable bound as the iteration (288) counter. It is useful for executing Test-cases multiple times by placing them (294) (330) inside a Test-set with a Data driver holding a Data loop. Contained in: Any Children: None Execution: During execution all the Data loop node does is register itself with the Data driver context. If no such context is available a warning message will be issued. When (294) all Data binders have been registered the Test-set will query the Data loop node in order to iterate through the loop and bind the iteration counter. Attributes:

Figure 30.20: Data loop attributes

Name The Name of a Data binder is mandatory. It is used to distinguish Data binders in the (368) same Data driver context. A Break node executed during data-driven testing can be used to break out of a specic loop by referring to the Data binders Name. Variable: Yes Restrictions: None

30.4.

Data driver

351

Iteration counter The name of the variable that the iteration counter will be bound to. Variable: Yes Restrictions: None Iteration ranges An optional set of indexes or ranges to use from the bound data. This is especially useful during test development in order to run sample tests with just a single index or a subset of the given data. Ranges are separated by ,. Each range is either a single index or an inclusive range of the form from-to or from:to where to is optional. Indexes or ranges may be specied multiple times, overlap or be given in descending order. Indexes are 0-based, negative indexes are counted from the end, -1 being the last item. An invalid syntax or an index outside the valid data range will cause (529) a BadRangeException . The following table shows some example range specications and the resulting indexes, based on a set of 20 entries.
Iteration ranges 0 -2, -1 1-2,4:5 18:,-33-2,16:15 Resulting indexes [0] [18, 19] [1, 2, 4, 5] [18, 19, 17, 18, 19] [3, 2, 16, 15]

Table 30.12: Iteration range examples

Note

The value bound for the Iteration counter reects the index in the current interval, not the counter of actual iterations, e.g. if you specify 2 there will be a single iteration with the Iteration counter bound to 2, not 0. Variable: Yes Restrictions: Valid syntax and index values Number of iterations The number of iterations of the loop. Variable: Yes Restrictions: > 0

30.5. Id

Procedures

352

At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.5

Procedures

Procedures are a means to collect some of the basic building blocks of a test-suite like (419) (442) events and checks into a larger, reusable structure. Procedures can be called from any other part of the test-suite and even from different (145) suites (see section 17.1 ). You can pass parameters to a procedure in the form of (52) variable denitions (see chapter 7 ). A typical example would be a procedure that selects a menu item in a menu. Its parameters could be the client name of the SUT, the name of the menu and the name of the menu item.

30.5.1 Procedure

30.5.

Procedures A Procedure is a Sequence (355) Procedure call .

(307)

353 that is executed from some other place by a

The parameters of the procedure are not dened explicitely. Instead they are a consequence of the variable references in the children of the Procedure. You may want (354) to dene fallback values for some or all of the parameters in the Variable denitions . (355) In any case it is a good idea to document the required parameters in the Comment attribute. A Procedure can return a value to the calling node with the help of a Return Without such a node a Procedure implicitly returns the empty string. Contained in: Package Children: Any Execution: The Procedures variables are bound as fallback values. The child nodes are executed one by one, then the fallback values are unbound again. Attributes:
(359) (358)

node.

, Procedures

(361)

Figure 30.21: Procedure Attributes

Name

30.5.

Procedures
(359)

354

A Procedure is identied by its name and the names of its Package ancestors, so you should assign a name with a meaning that is easy to recognize and remember. Variable: No Restrictions: Must not be empty or contain the characters . or #. Variable denitions Here you can dene default or fallback values for the Procedures parameters (see (52) chapter 7 ). Dening these values also serves as documentation and is a valuable time-saver when using the dialog to select the Procedure for the (356) (355) (11) Procedure name attribute of a Procedure call . See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used.

30.5.

Procedures Variable: Yes Restrictions: Valid number >= 0

355

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.5.2 Procedure call

This node calls a Procedure in the same or a different test-suite, meaning that execution continues in the Procedure. When the Procedure is nished, the value returned by the Procedure is bound to the variable dened in the (356) Variable for return value attribute and execution returns to the Procedure call and thus to its parent node. to call is determined by its Name and the Names of The name of the Procedure (359) its Package parents. These are concatenated with a dot (.) as separator, starting with the outermost Package and ending in the Procedures name. Thus to call a Procedure named expandNode in a Package named tree that is itself a child of a Package named (356) main, set the Procedure name attribute to main.tree.expandNode. See section 17.1 Children: None Execution: The Variable denitions of the Procedure call are bound, the target (352) Procedure is determined and execution passed to it. After the Procedure returns, the Procedure calls variables are unbound again. Attributes:
(357) (145) (352) (353) (360) (352)

about how to call a Procedure in a different test-suite.

(288)

Contained in: All kinds of sequences

30.5.

Procedures

356

Figure 30.22: Procedure call Attributes

Name The full name of the Procedure

(353) (352)

, created from the Names

(360)

of its Package

(359)

parents and its own Name , joined by a dot. The button brings up a dialog in which you can select the Procedure interactively. By selecting the Copy parameters checkbox you can adopt the Procedures default values as parameters for the Procedure call node to save typing. Variable: Yes Restrictions: Must not be empty. Variable for return value The value returned by the Procedure, either through a Return node or the empty string, is bound to the variable dened in this optional attribute. Additionally, the most recent return value is always available as the special variable ${qftest:return}. Variable: Yes Restrictions: None
(358)

30.5.

Procedures

357

Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Variable denitions This is where you dene the parameter values for the Procedure (52) (11) 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note
(352)

(see chapter

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the

button.

30.5.

Procedures Variable: Yes Restrictions: None

358

30.5.3 Return
This node can be used to return from a Procedure pass a return value to the calling node. From a script, ReturnException. Children: None Execution: A ReturnException is thrown. If it is caught by a matching Procedure, the Procedure is terminated and the return value passed to the caller. If the node is executed outside a Procedure it will lead to an error. Attributes:
(352)

prematurely and also to

the same effect can be achieved by raising a

(288)

Contained in: All kinds of sequences

Figure 30.23: Return Attributes

Return value The value to return from the Procedure. May be empty in which case the empty string is returned. Variable: Yes Restrictions: None

30.5. Id

Procedures

359

At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.5.4 Package
The only use of Packages is to structure the Procedures of a test-suite. The (360) Name of a Package is part of the Procedures fully qualied name, as required (355) by a Procedure call . Contained in: Package Children: Package Attributes:
(359) (359) (352)

, Procedures
(352)

(361)

, Procedure

Execution: Cannot be executed.

30.5.

Procedures

360

Figure 30.24: Package Attributes

Name The name of a Package is part of the identication of the Procedures it contains, so you should assign a name with a meaning that is easy to recognize and remember. Variable: No Restrictions: Must not be empty or contain the characters . or #. Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Border for relative calls This ag determines whether relative procedure calls are allowed within that certain node. Relative calls passing that border are not allowed. If that attribute is not specied in the hierarchy, no relative procedure calls are allowed. Variable: No Restrictions: None Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note
(352)

For detailed documentation, especially for Test-set, Test-case or Procedure nodes,

30.5.

Procedures

361

this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.5.5 Procedures
This is the root of all Packages Contained in: Root node Children: Package Attributes:
(359) (359)

and Procedures

(352)

, Procedure

(352)

Execution: Cannot be executed.

Figure 30.25: Procedures Attributes

Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Border for relative calls This ag determines whether relative procedure calls are allowed within that

30.6.

Control structures

362

certain node. Relative calls passing that border are not allowed. If that attribute is not specied in the hierarchy, no relative procedure calls are allowed. Variable: No Restrictions: None Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6

Control structures
(288)

Besides the standard sequence types QF-Test has a set of special control structures. (370) (373) (377) (362) and Else nodes. Loops and Conditional processing is handled via If , Elseif (365) (368) (379) While nodes can be aborted with a Break node. Exceptions are handled by Try , (382) (384) Catch and Finally nodes. Beyond that, full scripting is available for the Jython language (formerly called JPython) (94) and Groovy as documented in chapter 13 .

30.6.1 Loop
This node is basically the same as a Sequence except that its children can be executed more than once. This is useful in two ways. For one thing, a test sequence that executes OK a hundred times is more trustworthy than a sequence that only runs once. The other use is to run a number of similar jobs with slight variations. To that end, the count of the current iteration is bound as a variable during execution. Special Loops with varying increments can be achieved by changing the value of the (364) Iteration counter during execution.
(307)

30.6.

Control structures
(368)

363

Execution of a Loop can be terminated prematurely with the help of a Break node. An (377) optional Else node may be placed at the end of the Loop. It is executed if all iterations of the Loop are run through completely without hitting a Break. Contained in: All kinds of sequences Children: Any Execution: The Variable denitions of the Loop are bound. The Iteration counter is initialized to 0 and the child nodes are executed one by one. For each iteration the Iteration counter is increased by one and the children are executed again. After the nal execution of the last child is complete, the Iteration counter and the Variable denitions are unbound again. Attributes:
(364) (364) (288)

Figure 30.26: Loop attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence.

30.6.

Control structures Variable: No Restrictions: None

364

Number of iterations The number of iterations of the loop. Variable: Yes Restrictions: > 0 Iteration counter The name of the variable that will hold the iteration count during the execution. Make sure to use different Iteration counter names for nested loops. Variable: Yes Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id

30.6.

Control structures The Id of the Loop node can be used in a Break loop explicitely when loops are nested. Variable: No
(368)

365 node to terminate an outer

Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.2 While
This is a sequence that is executed repeatedly as long as a condition is fullled. The loop can be terminated prematurely with the help of a Break Contained in: All kinds of sequences Children: Any Execution: The Variable denitions of the While node are bound. If the condition evaluates to true, the child nodes are executed one by one. This is repeated until the (368) condition evaluates to false or the loop is terminated through a Break node or an Exception. Finally the Variable denitions are unbound again. Attributes:
(367) (288) (368)

node.

30.6.

Control structures

366

Figure 30.27: While attributes

Condition A condition is an expression that evaluates to either true or false. QF-Test discriminates between simple expression that it evaluates itself and complex expressions that are passed to the Jython script language to evaluate. An empty string or the string false (regardless of case) is interpreted as false, the string true as true. Whole numbers are true if and only if they are non-zero. Evaluating expressions in Jython opens the way for powerful expression handling. Jython supports the standard operators ==, !=, >, >=, < and <=. You can combine expressions with and and or and dene their priority with braces.
Note

Accessing QF-Test variables in a condition follows the same rules as in Jython (98) scripts (see section 13.3.3 ). You can use the standard QF-Test syntax $(...) and ${...:...} for numeric or boolean values. String values should be accessed with rc.lookup(...). Some examples:

30.6.

Control structures
Expression Empty String 0 21 False True bla bla 25 > 0 ${qftest:batch} not ${qftest:batch} rc.lookup(system, java.version) == 1.3.1 rc.lookup(system, java.version)[0] == 1 (1 > 0 and 0 == 0) or 2 < 1 Value False False True False True Syntax error True True if QF-Test is run in batch mode True if QF-Test is run in interactive mode True if JDK Version is 1.3.1 True is JDK Version starts with 1 True

367

Table 30.13: Condition examples

Variable: Yes Restrictions: Valid syntax Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.

30.6.
Note

Control structures

368

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id The Id of the While node can be used in a Break loop explicitely when loops are nested. Variable: No Restrictions: Must not be empty, contain any of the characters #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
(368)

node to terminate an outer

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.3 Break

30.6.

Control structures This node is used to terminate a Loops Contained in: All kinds of sequences Children: None
(362)

369 or a While
(365)

node prematurely.

(288)

Execution: A BreakException is thrown. If it is caught by a matching loop, the loop is terminated, otherwise it will lead to an error. Attributes:

Figure 30.28: Break attributes

Loop id For nested loops you can specify the loop to terminate by specifying the Id of a (362) (368) (365) Loop node and refer to it here. It works with Id for a While node respectively. In case this eld is empty the innermost loop is terminated. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used.
(364)

30.6.

Control structures Variable: Yes Restrictions: Valid number >= 0

370

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.4 If
Like in Java the child nodes of this node are executed only if a condition evaluates to true. However QF-Test differs from common programming languages in the way alternative branches are arranged. Contained in: All kinds of sequences
(288)

.
(373)

Children: Any executable node, followed by an arbitrary number of Elseif (377) an optional Else at the end.
(372)

nodes with

Execution: The Variable denitions of the If node are bound. If the condition evaluates to true, the normal child nodes are executed one by one. Otherwise the conditions of (373) the Elseif nodes are evaluated and the rst Elseif node whose condition evaluates to (377) true is executed. If none of the conditions are true of no Elseif nodes exist, the Else node is executed, if one exists. Finally the Variable denitions are unbound again. Attributes:

30.6.

Control structures

371

Figure 30.29: If attributes

Condition A condition is an expression that evaluates to either true or false. QF-Test discriminates between simple expression that it evaluates itself and complex expressions that are passed to the Jython script language to evaluate. An empty string or the string false (regardless of case) is interpreted as false, the string true as true. Whole numbers are true if and only if they are non-zero. Evaluating expressions in Jython opens the way for powerful expression handling. Jython supports the standard operators ==, !=, >, >=, < and <=. You can combine expressions with and and or and dene their priority with braces.
Note

Accessing QF-Test variables in a condition follows the same rules as in Jython (98) scripts (see section 13.3.3 ). You can use the standard QF-Test syntax $(...) and ${...:...} for numeric or boolean values. String values should be accessed with rc.lookup(...). Some examples:

30.6.

Control structures
Expression Empty String 0 21 False True bla bla 25 > 0 ${qftest:batch} not ${qftest:batch} rc.lookup(system, java.version) == 1.3.1 rc.lookup(system, java.version)[0] == 1 (1 > 0 and 0 == 0) or 2 < 1 Value False False True False True Syntax error True True if QF-Test is run in batch mode True if QF-Test is run in interactive mode True if JDK Version is 1.3.1 True is JDK Version starts with 1 True

372

Table 30.14: Condition examples

Variable: Yes Restrictions: Valid syntax Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.

30.6.
Note

Control structures

373

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.5 Elseif

30.6.

Control structures
(370)

374

This node is an alternative branch in an If node. If the condition of the If node evaluates to false, the rst Elseif node whose condition is true is executed. Contained in: If Children: Any of the Elseif are bound and its child nodes exeExecution: The Variable denitions cuted one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(375) (370)

Figure 30.30: Elseif attributes

Condition A condition is an expression that evaluates to either true or false. QF-Test discriminates between simple expression that it evaluates itself and complex expressions that are passed to the Jython script language to evaluate. An empty string or the string false (regardless of case) is interpreted as false, the string true as true. Whole numbers are true if and only if they are non-zero.

30.6.

Control structures

375

Evaluating expressions in Jython opens the way for powerful expression handling. Jython supports the standard operators ==, !=, >, >=, < and <=. You can combine expressions with and and or and dene their priority with braces.
Note

Accessing QF-Test variables in a condition follows the same rules as in Jython (98) scripts (see section 13.3.3 ). You can use the standard QF-Test syntax $(...) and ${...:...} for numeric or boolean values. String values should be accessed with rc.lookup(...). Some examples:
Expression Empty String 0 21 False True bla bla 25 > 0 ${qftest:batch} not ${qftest:batch} rc.lookup(system, java.version) == 1.3.1 rc.lookup(system, java.version)[0] == 1 (1 > 0 and 0 == 0) or 2 < 1 Value False False True False True Syntax error True True if QF-Test is run in batch mode True if QF-Test is run in interactive mode True if JDK Version is 1.3.1 True is JDK Version starts with 1 True

Table 30.15: Condition examples

Variable: Yes Restrictions: Valid syntax Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes

30.6.

Control structures Restrictions: None

376

Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the

button.

30.6.

Control structures Variable: Yes Restrictions: None

377

30.6.6 Else
An Else node is executed if neither the condition of its If (373) condition of its Elseif siblings evaluate to true. Contained in: If Children: Any Execution: The Variable denitions of the Else are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(378) (370) (370)

parent, nor the

Figure 30.31: Else attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree

30.6.

Control structures

378

view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0

30.6.

Control structures

379

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.7 Try
A Try is a special Sequence with exception handling support. Like a Sequence it has a set of normal child nodes that it executes one by one. After these may (382) (384) come an arbitrary number of Catch nodes with an optional Finally node at the end. If an exception is thrown during the execution of one of the normal child nodes, the Catch nodes are tested for whether they are able to catch that exception. The rst one found is executed and the Try will be exited normally afterwards without continuing with the execution of the normal child nodes and without passing the exception on. If no matching Catch is found, the exception will terminate the Try immediately (almost, see below) and be passed onto the Trys parent. child node, this node will be executed just before the Try If the Try has a Finally nishes, no matter whether an exception is thrown and whether it is handled or not.
Note
(384) (307)

Exceptions can also be caught implicitly by a Test convenient. Contained in: All kinds of sequences
(288)

(303)

, which is less exible but often more

.
(382)

Children: Any executable node, followed by an arbitrary number of Catch (384) an optional Finally node at the end.
(380)

nodes with

Execution: The Variable denitions of the Try are bound and its normal child nodes executed one by one. If an exception is thrown, execution of the normal children is (383) is found it is executed. terminated. If a Catch node with a matching Exception class Before exiting the Try its Finally node is executed unconditionally. After unbinding the Variable denitions, the Try is either exited cleanly if no exception was thrown or the exception was caught, or it passes on the uncaught exception.

30.6.

Control structures

380

Attributes:

Figure 30.32: Try attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level When a warning, error or exception occurs during a test run, the state of the

30.6.

Control structures

381

corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. For a Try node, the error state of the run-log is additionally affected by the (383) (382) Maximum error level of the Catch node that handles an exception. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.

Control structures

382

30.6.8 Catch
A Catch is a special Sequence that can only be placed inside a Try node (317) or a Dependency . Its job is to catch exceptions that may arise during the (288) execution of a Trys or a Test-case with a Dependency. A Catch can handle an exception if the class of the exception is the same as the Catch (383) nodes Exception class attribute or a derived class thereof, just as in Java. Contained in: Try Children: Any of the Catch are bound and its child nodes exeExecution: The Variable denitions cuted one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:
(383) (379) (307) (379)

, Dependency

(317)

Figure 30.33: Catch attributes

30.6.

Control structures

383

Exception class This ComboBox lets you select the class of the exception that is to be caught. All (526) QF-Test exceptions are derived from the class TestException . For details (526) about the possible execptions see chapter 31 . Variable: No Restrictions: None Expected message You can further qualify the exception to catch by specifying a message to look for. If this attribute is empty, all exceptions of the specied class are caught. Otherwise it is compared to exception.getMessage() and the exception is caught only in case of a match. Variable: Yes Restrictions: Valid regexp if required. As regexp If this attribute is set, the exception message is matched against a regexp (see (581) section 36.4 ) instead of comparing plain strings. Variable: Yes Restrictions: None Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table. Variable: Variable names no, values yes Restrictions: None Maximum error level In contrast to the Maximum error level of other sequences , this attribute does not determine the error state propagated by the run-log node for the Catch node
(308) (288)

30.6.

Control structures itself, but for the log of its parent Try in order to handle an exception.
(379)

384 node, provided that the Catch is executed

The error state for any warnings, errors or exceptions that happen during the execution of the Catch node are not limited by the setting of this attribute. Otherwise problems occurring during exception handling might accidentally go unnoticed. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.9 Finally
A Finally node, which can only be placed at the end of a Try node, will always be executed as the last thing just before exiting the Try, no matter what happened there. This is used primarily to ensure that cleanup code like removing a temporary le or terminating a process is executed under any conditions.

30.6.

Control structures
(379)

385

Contained in: Try Children: Any

Execution: The Variable denitions of the Finally are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. Attributes:

(385)

Figure 30.34: Finally attributes

Name The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence. Variable: No Restrictions: None Variable denitions This is where you dene the values of the variables that remain bound during the (52) (11) execution of the sequences child nodes (see chapter 7 ). See section 2.2.5 about how to work with the table.

30.6.

Control structures Variable: Variable names no, values yes Restrictions: None

386

Maximum error level When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.
Note

This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in (5) batch mode (see. section 1.5 ). It also has no effect on the creation of compact (542) run-logs (see command line argument -compact ). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the

button.

30.6.

Control structures Variable: Yes Restrictions: None

387

30.6.10 Throw
If you need to handle an exceptional situation, you can use this node to throw an explicit Exception. Contained in: All kinds of sequences Children: None Execution: A UserException (387) Exception message attribute. Attributes:
(532) (288)

. its message taken from the

is thrown,

Figure 30.35: Throw attributes

Exception message An arbitrary message for the UserException Variable: Yes Restrictions: Must not be empty. Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_).
(532)

to throw.

30.6.

Control structures

388

Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.11 Rethrow
An exception that was caught by a Catch node can be thrown again with the help of a Rethrow node. This is especially useful if you need to catch all (379) kinds of exceptions except one. To handle that case, create a Try node with (526) a Catch for the special exception followed by a Catch for a TestException . Then place a Rethrow in the rst Catch node. Contained in: All kinds of sequences directly below a Catch node. Children: None node is thrown again. If no such Execution: The last exception caught by a Catch (532) exception exists, a CannotRethrowException is thrown. Attributes:
(382) (288) (382)

. The Rethrow node doesnt have to be placed

30.6.

Control structures

389

Figure 30.36: Rethrow attributes

Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.

Control structures

390

30.6.12 Server script

Server side scripts are executed by an interpreter (Jython or Groovy) embed(94) (586) ded into QF-Test. Scripting is explained in chapter 13 and chapter 37 . Contained in: All kinds of sequences Children: None Execution: The script is executed by an embedded interpreter. Attributes:
(288)

Figure 30.37: Server script attributes

Script The script to execute.

Note

QF-Test variables are expanded before the script is passed to the Jython interpreter. This is convenient for numeric values. Strings values on the other hand require quoting for the Jython interpreter and if the value of the variable contains

30.6.

Control structures

391

quotes itself this can cause problems. rc.lookup(...) is the preferred method (586) in this case (see chapter 37 for details).
Note

In spite of syntax highlighting and automatical indentation this attribute might not be the right place to write complex scripts. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which scripts can be edited conveniently by pressing by clicking the button. Complex scripts can also be written as separate modules which can then be imported for use in this attribute. See (586) chapter 37 for details. Variable: Yes Restrictions: Valid syntax Script language This attribute determines the interpreter in which to run the script, or in other words, the scripting language to use. Possible values are Jython and Groovy. Variable: No Restrictions: None Name The name of a script is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the script. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0
Alt-Return or

30.6.

Control structures

392

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.6.13 SUT script

Client side scripts are executed by an interpreter (Jython or Groovy) that QF(94) Test embeds into the SUT. Scripting is explained in chapter 13 and chapter (586) 37 . Contained in: All kinds of sequences Children: None Execution: The script is executed inside the SUT by an embedded interpreter. Attributes:
(288)

30.6.

Control structures

393

Figure 30.38: SUT script attributes

Client The name of the SUT client process in which to execute the script. Variable: Yes Restrictions: Must not be empty. Script The script to execute.
Note

QF-Test variables are expanded before the script is passed to the Jython interpreter. This is convenient for numeric values. Strings values on the other hand require quoting for the Jython interpreter and if the value of the variable contains quotes itself this can cause problems. rc.lookup(...) is the preferred method (586) in this case (see chapter 37 for details).

30.6.
Note

Control structures

394

In spite of syntax highlighting and automatical indentation this attribute might not be the right place to write complex scripts. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which scripts can be edited conveniently by pressing by clicking the button. Complex scripts can also be written as separate modules which can then be imported for use in this attribute. See (586) chapter 37 for details. Variable: Yes Restrictions: Valid syntax Script language This attribute determines the interpreter in which to run the script, or in other words, the scripting language to use. Possible values are Jython and Groovy. Variable: No Restrictions: None GUI engine The GUI engine in which to execute the script. Only relevant for SUTs with more (557) than one GUI engine as described in chapter 33 . Variable: Yes Restrictions: See chapter 33 Name The name of a script is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the script. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used.
(557)

Alt-Return or

30.7.

Processes Variable: Yes Restrictions: Valid number >= 0

395

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7

Processes

In order to enable QF-Test to communicate with the SUT, the SUTs process must be started by QF-Test. For details about how the application is started by QF-Test and (559) which kind of node is most applicable in your case, see chapter 34 . The following nodes are used to start or stop processes, to wait for a connection with the SUT or to wait for the termination of a process and check its exit value. The number of concurrent processes managed by QF-Test is limited only by the underlying system, but you have to assign a unique name to each process by which other nodes will identify it. We will refer to processes run by QF-Test as clients. QF-Test distinguishes between two types of clients: arbitrary processes that are simply started and stopped and SUT clients, the actual Java applications that QF-Test interacts with. Standard input and output of a client is redirected to a terminal that you can open from the Clients menu. The clients output is also stored in the log of a test run.

30.7.1 Start Java SUT client

This node provides the standard and most exible way of starting an SUT client. To use it, you must know the Java command that starts your applica(399) tion. If the SUT is run through a script, use a Start SUT client node instead (559) (see chapter 34 ).

30.7.

Processes
(288)

396 .

Contained in: All kinds of sequences Children: None

Execution: The command line for the program is built from the attributes and the process is started. Its input and output are redirected to QF-Test. Attributes:

Figure 30.39: Start Java SUT client attributes

Client This is the identier for the SUT client. It must remain unique as long as the process is alive. Other nodes will refer to the client by this name.

30.7.

Processes Variable: Yes Restrictions: Must not be empty

397

Executable The Java program to run, typically java or javaw on Windows. If you want to run a specic Java virtual machine you should give the full path name of the executable. brings up a dialog in which you can le inThe le button select the program teractively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. Variable: Yes Restrictions: Must not be empty Directory Here you can set the working directory for the program. If you leave this value empty, the program will inherit QF-Tests working directory. brings up a dialog in which you inThe directory button directory can select the teractively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. Variable: Yes Restrictions: Must be empty or an existing directory Class name The fully qualied name of the Java class of the SUT whose main(String[]) method will be called. If your application is run from an executable jar archive with the -jar parameter, you must leave this attribute empty.
Note

QF-Test can only run the main method of the class if both the class itself and the main method are declared public. Please verify this if you get an IllegalAccessException when you try to start your application. Variable: Yes Restrictions: Valid class name Parameters There is one tab for each of the following parameter lists: Executable The command line arguments for the Java executable. Put each parameter

30.7.

Processes

398

on a line of its own. Special quoting of whitespace or symbols is not required. For example to set the classpath, set one line to -classpath and the following line to the desired value of the classpath. You dont need to concern yourself with QF-Tests jar les. Class These are the arguments that are passed to the main(String[]) method of the class to be started. Again, each argument requires a row of its own and special quoting is not required. Extra Extra parameters for the class de.qfs.apps.qftest.start.StartClient which is the wrapper class started by QF-Test in order to gain access to the EventQueue, establish an RMI connection between the SUT and QF-Test and then start the main class of the SUT. These parameters are intended for future extensions, scripting and internal logging. See section 2.2.5 Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note
(11)

about how to work with the tables.

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that

30.7.

Processes
(226)

399

are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7.2 Start SUT client

If your application is normally started through a shell script or a special executable starter, the most convenient way to run it from QF-Test is through this kind of node. Depending on how the application is actually started by this script, some modications may be required. A typical symptom is that the SUT starts up (559) ne, but the connection to QF-Test is not established. In that case, read chapter 34 . Contained in: All kinds of sequences Children: None Execution: The command line for the program is built from the attributes and the process is started. Its input and output are redirected to QF-Test. Attributes:
(288)

30.7.

Processes

400

Figure 30.40: Start SUT client attributes

Client This is the identier for the SUT client. It must remain unique as long as the process is alive. Other nodes will refer to the client by this name. Variable: Yes Restrictions: Must not be empty Executable The program to run. If the executable le is not located in a directory on the PATH you must give the full path name. The le button brings up a dialog in which you can le in select the program teractively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld.

30.7.

Processes Variable: Yes Restrictions: Must not be empty

401

Directory Here you can set the working directory for the program. If you leave this value empty, the program will inherit QF-Tests working directory. The directory button brings up a dialog in which you in can select the directory , teractively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld.
Note

This directory will become the current working directory after executing the program. As a result, a script named, say, ./copy_data will be looked up relative to QF-Tests working directory and not the directory given. Only the path names referred to in the script itself will be resolved relative to the new directory. Variable: Yes Restrictions: Must be empty or an existing directory Parameters There is one tab for each of the following parameter lists: Executable The command line arguments for the executable. Put each parameter on a line of its own. Special quoting of whitespace or symbols is not required. Extra Extra parameters for the class de.qfs.apps.qftest.start.StartClient which is used to gain access to the EventQueue, establish an RMI connection between the SUT and QF-Test and then start the main class of the SUT. These parameters are intended for future extensions, scripting and internal logging. See section 2.2.5 Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_).
(11)

about how to work with the tables.

30.7.

Processes

402

Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7.3 Start browser

This node is used to start a web browser specically for web testing. To (399) test a Java applet in a browser, use a Start SUT client node instead. If the process for the SUT is already running, this node can also be used to open an additional browser window in the same process. Contained in: All kinds of sequences Children: None Execution: The command line for the browser program is built from the attributes and the process is started. Its input and output are redirected to QF-Test. Attributes:
(288)

30.7.

Processes

403

Figure 30.41: Start browser attributes

Client

30.7.

Processes

404

This is the identier for the SUT client. It must remain unique as long as the process is alive. Other nodes will refer to the client by this name. Variable: Yes Restrictions: Must not be empty URL The URL of the web page to be shown in the browser. As a special case the URL about:nowindow can be used to start the SUT process without opening an initial browser window. In that state, cache or preference settings for the browser can be modied which may be a prerequisite for opening a given URL. The browser window with the actual target URL can then be opened via another Start browser node. Variable: Yes Restrictions: Must not be empty Open new browser window in existing client If this attribute is set, QF-Test will try to open another browser window in an existing SUT client instead of starting a new process. Of course in that case the SUT client must already be running. Variable: Yes Restrictions: None Browser type The kind of browser to open a window for. Currently supported values are ie for Internet Explorer and mozilla for Mozilla derivatives like Firefox, SeaMonkey or Mozilla itself. It is possible to open browser windows for different browser types in the same SUT client. Variable: Yes Restrictions: Currently only ie or mozilla are supported. Name of the browser window This attribute can be ignored unless you need to test a web application with multiple open browser windows holding similar documents. In that case the Name of the browser window attribute can be used to identify the browser window. Here you can specify the name for the browser window to be opened. Variable: Yes Restrictions: None

30.7.

Processes

405

Directory of Mozilla installation For Mozilla and Mozilla-based browsers the installation directory of the browser must be set here. Variable: Yes Restrictions: Must be empty (for ie) or an existing directory Geometry of browser window These optional attributes for the X/Y coordinate, width and height can be used to specify the geometry of the browser window to open. Variable: Yes Restrictions: Width and height must not be negative. Parameters There is one tab for each of the following parameter lists: Java VM Special command line arguments for the java executable in which the browser is embedded. Can be used to specify the available memory (default is 200MB), dene system properties, enable debugging, etc. Put each parameter on a line of its own. Special quoting of whitespace or symbols is not required. Extra Extra parameters for the QF-Test wrapper that embeds the browser and establishes an RMI connection between the SUT and QF-Test. These parameters are intended for future extensions, scripting and internal logging. See section 2.2.5 Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after
(11)

about how to work with the tables.

30.7.

Processes

406

These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7.4 Start process

To run an arbitrary program during a test, you can either use this node or an (409) Execute shell command node. This node is preferable if you need to pass possibly complex arguments to the executable. Contained in: All kinds of sequences Children: None Execution: The command line for the program is built from the attributes and the process is started. Its input and output are redirected to QF-Test. Attributes:
(288)

30.7.

Processes

407

Figure 30.42: Start process attributes

Client This is the identier for the SUT client. It must remain unique as long as the process is alive. Other nodes will refer to the client by this name. Variable: Yes Restrictions: Must not be empty Executable The program to run. If the executable le is not located in a directory on the PATH you must give the full path name. The le button brings up a dialog in which you can le in select the program teractively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. Variable: Yes Restrictions: Must not be empty

30.7.

Processes

408

Directory Here you can set the working directory for the program here. If you leave this value empty, the program will inherit QF-Tests working directory. The directory button brings up a dialog in which you in directory can select the , teractively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld.
Note

This directory will become the working directory of the newly started process. It does not affect the working directory of QF-Test. As a result, a program named, say, ./startserver will be looked up relative to QF-Tests working directory and not the directory given. Only the path names referred to in the program itself will be resolved relative to the given directory. Variable: Yes Restrictions: Must be empty or an existing directory Parameters The command line arguments for the executable. Put each parameter on a line of its own. Special quoting of whitespace or symbols is not required. See section 2.2.5 Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
(11)

about how to work with the tables.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that

30.7.

Processes
(226)

409

are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7.5 Execute shell command

This node is a convenient way to execute shell commands during a test. The shell that will execute the command can be specied with the command line (550) (550) and -shellarg <argument> arguments -shell <executable> for QF-Test. The default for Unix is /bin/sh, on Windows either command.com or cmd.exe is used, the value of the COMSPEC environment variable. When the shell is started it is treated like every other process started by QF-Test, so you can kill it or wait for it to terminate and check its exit code. Contained in: All kinds of sequences Children: None Execution: A shell is started to execute the command. Its input and output are redirected to QF-Test. Attributes:
(288)

30.7.

Processes

410

Figure 30.43: Execute shell command attributes

Client This is the identier for the SUT client. It must remain unique as long as the process is alive. Other nodes will refer to the client by this name. Variable: Yes Restrictions: Must not be empty Shell command The command to execute in the shell. command prompt.
Windows

Enter this just as you would at the

On Windows systems, quoting of arguments with blanks can be a little tricky. If youre using the standard Windows shell, simply use double quotes as always, for example dir C:\Program Files. If youre using a Unix shell on Windows (550) by specifying the -shell <executable> command line argument, use single quotes instead, i.e. ls C:/Program Files. Variable: Yes Restrictions: Must not be empty Directory

30.7.

Processes

411

Here you can set the working directory for the shell. If you leave this value empty, the shell will inherit QF-Tests working directory. The directory button brings up a dialog in which you in can select the directory teractively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. Variable: Yes Restrictions: Must be empty or an existing directory Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7.6 Wait for client to connect

30.7.

Processes

412

This node is used to make sure that the connection to a client is established. If after a congurable timeout the client is still not connected, a (530) ClientNotConnectedException is thrown. You can also use the Variable for result attribute to store the result into a variable and the Throw exception on failure attribute to suppress the exception. Contained in: All kinds of sequences Children: None Execution: QF-Test waits until either the connection to the Java client is established or the timeout is up. Attributes:
(288)

Figure 30.44: Wait for client to connect attributes

Client

30.7.

Processes The name of the Java client to wait for. Variable: Yes Restrictions: Must not be empty

413

Timeout Time limit in milliseconds. Variable: Yes Restrictions: >= 0 GUI engine The GUI engine to wait for. Only relevant for SUTs with more than one GUI (557) engine as described in chapter 33 . Variable: Yes Restrictions: See chapter 33 Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note
(557)

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message

30.7.

Processes

414

This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.
Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7.

Processes

415

30.7.7 Stop client

This node forcibly terminates a client process started by QF-Test. If the process has already terminated, nothing is changed. Otherwise, if the client is an SUT client, QF-Test rst tries to call the Java method System.exit(-1) in the SUT to achieve a clean shutdown. If the client is not an SUT client or the process is still alive after the exit call, the process is killed and its exit code is set to -1.
Windows

Note: On Windows child processes started by a process are not terminated when the (559) parent process is killed. As explained in chapter 34 , the java program for the SUT is not started directly by QF-Test, but through an intermediate program. This means that when the System.exit(-1) call fails to terminate the SUT for some reason, the process for the SUTs Java VM will be left hanging around. Contained in: All kinds of sequences Children: None Execution: Terminates the last process that was started under the given name. Attributes:
(288)

Figure 30.45: Stop client attributes

Client The client name of the process that is to be killed. Variable: Yes Restrictions: Must not be empty Id

30.7.

Processes At the moment the Id attribute has no meaning for this type of node. Variable: No

416

Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.7.8 Wait for process to terminate

This node waits until a process that was started by QF-Test terminates. If a given time limit is exceeded and the process is still alive. Per default a (531) ClientNotTerminatedException is thrown. Otherwise the clients exit code is read and tested against a given value. You can also use the Variable for result attribute to store the result into a variable and the Throw exception on failure attribute to suppress the exception. Contained in: All kinds of sequences Children: None Execution: Waits for the end of a process and checks its exit value unless the timeout is exceeded. Attributes:
(288)

30.7.

Processes

417

Figure 30.46: Wait for process to terminate attributes

Client The client name of the process to wait for. Variable: Yes Restrictions: Must not be empty Timeout Time limit in milliseconds. Variable: Yes Restrictions: >= 0 Expected exit code

30.7.

Processes

418

If this attribute is set it is used to validate the exit code of the process. The type of the comparison is dened by prepending one of the four operators ==, !=, < and >. Just a number without preceding operation test for equality. If the test fails, an (531) UnexpectedExitCodeException is thrown. The exception wont be thrown, if the attribute Throw exception on failure is not set. Examples: If this attribute is set to 0, every exit code that is not 0 causes an exception. This is the same as for ==0. A value of >0 causes an exception for every exit code equal to or less than 0. This exception can be suppressed by un-checking the attribute Throw exception on failure. Variable: Yes Restrictions: See above Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored.

30.8.

Events Variable: No Restrictions: None

419

Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.8

Events

This section lists all kinds of nodes that trigger actions in the SUT. Besides true Java events these include pseudo events with special behavior.

30.8.

Events

420

30.8.1 Mouse event

Mouse events simulate mouse movement and clicks as well as drag and drop operations. Contained in: All kinds of sequences Children: None Execution: The coordinates and other attributes of the event are sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT, adjusts the coordinates and triggers the resulting event. Attributes:
(288)

Figure 30.47: Mouse event attributes

Client

30.8.

Events The name of the SUT client process to which the event is sent. Variable: Yes Restrictions: Must not be empty.

421

Component id The Id event.

(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

button brings up a dialog in which you can select the component interThe actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Event This ComboBox lets you choose the type of the event. MOUSE_MOVED, MOUSE_PRESSED, MOUSE_RELEASED, MOUSE_CLICKED and MOUSE_DRAGGED are the standard event ids of the Java class MouseEvent. The abstract Mouse click event is a compound of the events MOUSE_MOVED, MOUSE_PRESSED, MOUSE_RELEASED and MOUSE_CLICKED. During replay the pseudo event is simulated as four separate events. This adds to the clarity of a test-suite and simplies editing. The events MOUSE_DRAG_FROM, MOUSE_DRAG_OVER and MOUSE_DROP_TO are (578) used to simulate Drag&Drop in the SUT. See section 36.1 for details. Variable: No Restrictions: None X/Y These are the coordinates of the MouseEvent. They are relative to the upper left (507) (514) (519) or Item that is the target of the event. corner of the Window , Component They can be negative, e.g. to simulate a click on the expansion toggle of a node in a JTree. Most of the time the exact coordinates for a mouse dont really matter, any place within the target will do. In this case you should leave the X and Y values empty to tell QF-Test to aim at the center of the target. Where possible QF-Test will leave the
(519)

30.8.

Events

422

values empty when recording, provided the option Record MouseEvents without (235) coordinates where possible is active. Variable: Yes Restrictions: Valid number or empty Modiers

, This value of the mouse buttons and the modier keys Shift reects the state Control , Alt and Meta during a mouse or key event. States are combined by adding up their values.
Value 1 2 4 8 16 Key/Button Shift Control Meta or right mouse button Alt or middle mouse button Left mouse button

Table 30.16: Modier values

Variable: Yes Restrictions: Valid number Click count This value lets a Java program distinguish between a single and a double (or even multiple) click. Variable: Yes Restrictions: Valid number Popup trigger If this attribute is set, the event can trigger a PopupMenu. This is Javas somewhat peculiar way of supporting different conventions for triggering PopupMenus on different platforms. Variable: Yes Restrictions: None Replay as hard event If this attribute is set the event is replayed as a hard event, meaning it is triggered as a real system event that moves the mouse around and not just inserted as soft event into the event queue. Soft events are are typically better because they

30.8.

Events

423

avoid impact of concurrent user mouse actions and are less likely to break due to interference from an overlapping window. Nevertheless there are certain special situations where hard events are helpful. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.8.2 Key event

Key events simulate keyboard input in the SUT. These are used mainly for (427) control and function keys. Input of text is better represented as a Text input . The special keyboard event InputMethodEvent supports international character codes. QF-Test doesnt support InputMethodEvents directly. Instead, it converts events of type INPUT_METHOD_TEXT_CHANGED into Key events of type KEY_TYPED.

30.8.

Events
(288)

424 .

Contained in: All kinds of sequences Children: None

Execution: The key codes of the event are sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT and triggers the resulting event. Attributes:

Figure 30.48: Key event attributes

Client The name of the SUT client process to which the event is sent. Variable: Yes Restrictions: Must not be empty. Component id The Id event.
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

30.8.

Events

425

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node . with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V directly or access components By using a special syntax you can target Items (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Event This ComboBox lets you choose the type of the event. KEY_PRESSED, KEY_TYPED and KEY_RELEASED are the standard event ids of the Java class KeyEvent. The Keystroke pseudo event is a compound of the events KEY_PRESSED, KEY_TYPED and KEY_RELEASED. During replay the pseudo event is simulated as two or three separate events, depending on whether it is a printable character key, or a control or function key. In the latter case, no KEY_TYPED event is generated. Variable: No Restrictions: None Key and Modiers This is a convenience method to set Key code , Key char directly by pressing the corresponding key while this component has the (425) keyboard focus. For KEY_TYPED events Key char is set to 0.

(425) (425) (426) (519)

Unfortunately you cant select the Tab key this way since it is used for keyboard (425) (425) traversal. Key code and Key char for the Tab key are both 9. Variable: No Restrictions: None Key code This is a Java specic code for the key, the keyCode member of the Java class KeyEvent. Variable: Yes Restrictions: Valid number Key char This is the keyChar member of the Java class KeyEvent. Its value is the

30.8.

Events

426

character generated created by the last key press, taking the state of the Shift key into account. Control and function keys always have a Key char value of 65535. Variable: Yes Restrictions: Valid number Modiers

This value of the mouse buttons and the modier keys Shift , reects the state , Alt and Meta during a mouse or key event. States are combined by Control adding up their values.
Value 1 2 4 8 16 Key/Button Shift Control Meta or right mouse button Alt or middle mouse button Left mouse button

Table 30.17: Modier values

Variable: Yes Restrictions: Valid number Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes,

30.8.

Events

427

this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.8.3 Text input

This is a pseudo event for simulating text input from the keyboard. A single (423) Text input node replaces a whole sequence of Key event nodes. To achieve this, QF-Test takes advantage of the fact that AWT and Swing text elds listen only to KEY_TYPED events, so input of text can be simulated by triggering one KEY_TYPED event per text character. If your SUT uses custom text components that actually require the KEY_PRESSED and KEY_RELEASED events, you cannot use this node (423) type but have to resort to plain Key events . Contained in: All kinds of sequences Children: None Execution: The text is sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT and triggers the resulting events. Attributes:
(288)

30.8.

Events

428

Figure 30.49: Text input attributes

Client The name of the SUT client process to which the event is sent. Variable: Yes Restrictions: Must not be empty. Component id The Id event.
(508) (507) (514) (519)

of the Window

, Component

or Item

node that is the target of the

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . directly or access components By using a special syntax you can target Items (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty.
(519)

30.8. Text

Events

429

The text that is to be sent to the SUT.

Note

When inserting text into a password eld it may be desirable to avoid having the password show up as plain text in the test-suite or a run-log. To that end the password can be crypted by inserting the plain-text password, right-clicking and selecting Crypt password from the popup menu. Please be sure to specify a (250) password salt before crypting via the option Salt for crypting passwords . Variable: Yes Restrictions: No line breaks are possible. Clear target component rst If this attribute is set, and the target component is a text eld or text area, the contents of the target component are removed before the new text is inserted. Variable: Yes Restrictions: None Replay single events If the target component is a text eld or a text area, the text can optionally be inserted by manipulating the component directly through its API. This is much faster, but if KeyListeners are registered on the component they will not be notied of the change. If this attribute is set, key events are simulated for every single character. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0

30.8.

Events

430

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.8.4 Window event

WindowEvents are of limited use for a test-suite since most of them are not generated as a direct consequence of some user interaction. The only exception is the WINDOW_CLOSING event that is triggered when the user closes a window. It is also possible to simulate WINDOW_ICONIFIED and WINDOW_DEICONIFIED events. Contained in: All kinds of sequences Children: None Execution: The event is sent to the SUT together with the data about the target window. The TestEventQueue determines the corresponding window in the SUT and triggers the resulting event. Attributes:
(288)

30.8.

Events

431

Figure 30.50: Window event attributes

Client The name of the SUT client process to which the event is sent. Variable: Yes Restrictions: Must not be empty. Component id The Id event.
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty.
(519)

30.8. Event

Events

432

This ComboBox lets you choose the type of the event. Possible values are WINDOW_OPENED, WINDOW_CLOSING, WINDOW_CLOSED, WINDOW_ACTIVATED, WINDOW_DEACTIVATED, WINDOW_ICONIFIED and WINDOW_DEICONIFIED, the standard event ids of the Java class WindowEvent. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.8.5 Component event

The name Component event for this node may be misleading, since QF-Test lters all of these events except for windows, but since they represent the Java class ComponentEvent, it is better to stick to that name. Except for

30.8.

Events

433

COMPONENT_MOVED and COMPONENT_SIZED events on a window, which are the result of the user moving or resizing the window interactively, all ComponentEvents are articial and thus ignored. Contained in: All kinds of sequences Children: None Execution: The event is sent to the SUT together with the data about the target window. The TestEventQueue determines the corresponding window in the SUT and triggers the resulting event. Attributes:
(288)

Figure 30.51: Component event attributes

Client The name of the SUT client process to which the event is sent. Variable: Yes Restrictions: Must not be empty. Component id

30.8.

Events The Id event.

(508)

434 of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter , actively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . directly or access components By using a special syntax you can target Items (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Event This ComboBox lets you choose the type of the event. Possible values are COMPONENT_SIZED and COMPONENT_MOVED, the standard event ids of the Java class ComponentEvent. Variable: No Restrictions: None Y/Height For a COMPONENT_MOVED event set this to the new Y-coordinate of the window, for a COMPONENT_SIZED event to its new height. Variable: Yes Restrictions: Valid number, height > 0 X/Width For a COMPONENT_MOVED event set this to the new X-coordinate of the window, for a COMPONENT_SIZED event to its new width. Variable: Yes Restrictions: Valid number, width > 0 Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_).
(519)

30.8.

Events

435

Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.8.6 Selection
A Selection node represents an abstract event like selecting a menu item, choosing an entry in a combo box or selecting something from or closing a system dialog. Currently this event node is used only for SWT and web SUT clients, where some events cannot be triggered by soft mouse events, . The alternative of using hard events has some disadvantages as described for the (422) (420) Replay as hard event attribute of a Mouse event . The Detail attribute determines the kind of operation to perform, or the value to select, depending on the target component.
SWT
(438)

The following combinations of widgets and Detail values are currently supported for SWT:

30.8.

Events
Detail attribute Empty Color value in hexadezimal #rrggbb format CANCEL Empty Empty close Directory name CANCEL File name, including directory CANCEL Font description, system specic CANCEL close Empty Any of OK YES NO CANCEL ABORT RETRY IGNORE OPEN SAVE ProgressBar value Scale value Slider value Spinner value Empty

436
Action Choose item as current value Select given color Abort color selection Choose item as current value Select tab Close tab Select given directory Abort directory selection Select given le Abort le selection Select given font Abort le selection Cancel the menu (close without selection) Select item Close with the given value as the users choice Set the given value Set the given value Set the given value Set the given value Select tab

Class CCombo sub-item ColorDialog ColorDialog Combo sub-item CTabFolder sub-item CTabFolder sub-item DirectoryDialog DirectoryDialog FileDialog FileDialog FontDialog FontDialog Menu MenuItem MessageBox

ProgressBar Scale Slider Spinner TabFolder sub-item

Table 30.18: Supported SWT widgets for a Selection event

Note

Eclipse/RCP makes heavy use of dynamic CTabFolders where the items can be moved between folders. The items represent the actual business objects whereas the folders are just scaffolding to hold them. Besides, the layout of the folders and items can change drastically when switching perspectives. Thus it is often desirable to be able to select or close an item independent of the CTabFolder it currently resides in. This can be done by (352) using the Procedures qfs.qft#qfs.swt.ctabfolder.selectTab and qfs.qft#qfs.swt.ctabfolder.closeTab, provided in the qfs.qft standard library. Besides the ubiquitous client parameter, the only other parameter tabname must be set to the name of the item to select or close. The following combinations of DOM nodes and Detail values are currently supported for web SUT clients:

Web

30.8.

Events
Detail attribute Any of OK YES NO CANCEL RETRY The le to save to or CANCEL

437
Action Close the dialog with the given value as the users choice Close the dialog and save to the specied le or abort the download Close the dialog and login with the specied data or abort. The password can by crypted by right-clicking the Detail eld and selecting Crypt password . Please be sure to specify a password salt before crypting via the option Salt for crypting (250) passwords . Close the dialog and return the specied text or abort Navigate back to the previous page Navigate forward to the next page Navigate to the specied URL Reload the current page Stop loading the current page Choose the OPTION as current value Add the OPTION to the selection Remove the OPTION from the selection

Node type Conrmation dialog File dialog for download

Login dialog

Username|Password CANCEL

or

Prompt dialog Top-level DOCUMENT Top-level DOCUMENT Top-level DOCUMENT Top-level DOCUMENT Top-level DOCUMENT OPTION or SELECT sub-item OPTION or SELECT sub-item OPTION or SELECT sub-item

The text to enter or CANCEL back forward goto:URL refresh stop 0 1 -1

Table 30.19: Supported DOM nodes for a Selection event

Contained in: All kinds of sequences Children: None

(288)

Execution: The event is sent to the SUT together with the data about the target component. The component is resolved and an action performed which depends on the type of component as listed in the table above. Attributes:

30.8.

Events

438

Figure 30.52: Selection attributes

Client The name of the SUT client process to which the event is sent. Variable: Yes Restrictions: Must not be empty. Component id The Id event.
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . directly or access components By using a special syntax you can target Items (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty.
(519)

30.8. Detail

Events

439

The Detail attribute species the kind of operation to perform or the value to select, depending on the target component. The possible combinations are listed in detail above. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.8.7 File selection

A File selection is a pseudo event that you only need in a special case. If your SUT uses the standard AWT le selection dialog, implemented by the class java.awt.FileDialog, QF-Test has no chance to record the events the user generates while selecting a le, since they are all handled by the underlying

30.8.

Events

440

system and never passed on to Java. For the same reason, the selection cannot be simulated as a sequence of mouse and key events. This is not the case with Swings javax.swing.JFileChooser which is implemented as a normal dialog. Therefore QF-Test just records the result of the le selection in the form of this node and stores that data in the FileDialog upon replay before closing it. For the SUT there is no difference to an actual selection by the user. Contained in: All kinds of sequences Children: None Execution: File and directory are stored in an open java.awt.FileDialog and (526) the dialog is closed.If no FileDialog is open, a ComponentNotFoundException is thrown. Attributes:
(288)

Figure 30.53: File selection attributes

Client The name of the SUT client process to which the event is sent. Variable: Yes Restrictions: Must not be empty.

30.8. File

Events

441

The name of the le (without the directory part) that is to be selected. Variable: Yes Restrictions: Must not be empty Directory The directory of the le that is to be selected. Variable: Yes Restrictions: Must not be empty GUI engine The GUI engine in which to look for a le selection dialog. Only relevant for SUTs (557) with more than one GUI engine as described in chapter 33 . Variable: Yes Restrictions: See chapter 33 Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note
(557)

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.9.

Checks

442

30.9

Checks

Checks are the means by which QF-Test validates correctness of the SUT. If the current state of a component doesnt match the expected value of the check node a message with a congurable error level is written to the run-log. Additionally or alternatively a (529) CheckFailedException can be thrown and the result of the check can be assigned to a variable. By setting a Timeout checks can also be used to wait for a component to attain a certain state, e.g. to wait for a MenuItem to become enable or for a CheckBox to become selected. Like events each check must have a target window, component or sub-item. Depending on that target, different kinds of checks are supported which can be recorded (31) in check-mode as described in section 4.3 via reight-clicking the target component and selecting the check from the resulting menu. In case you need a check for a special component that is not offered by default you can implement custom checks via the (669) Checker extension API described in section 39.3 . There are six different data types available for checks and each of these corresponds to a specic check node. Because it is possible to have different checks of the same data type for the same target component, e.g. for the enabled state and the editable state of a text eld, both boolean, checks are further qualied by their Check type identier attribute. In most cases, when data type and target component are sufcient to uniquely identify the kind of check, the value of this attribute is default. In case the specied check is (529) not supported for the given target component a CheckNotSupportedException is thrown. By default a Boolean check provides the following Check types (if the respective component supports it):
(419) (447)

30.9.

Checks
Example 1 Example 2 Details

443

Check

visible

Example 1 shows a invisible Texteld and CheckBox.. Both are visible at Example 2.

editable

Example 1 shows a Texteld which is not editable. The Texteld at Example 2 is editable. The CheckBox does not support this kind of check.

enabled

Neither the Texteld nor the CheckBox is enabled at Example 1, which means you cannot interact with them. Example 2 shows enabled components.

selected

Example 1 shows a CheckBox which is not selected. It is selected at Example 2. The Texteld does not support this kind of check.

selected (Table)

Example 1 shows a Table where no cell is selected. The lower left cell is selected at Example 2, indicated by a frame. The check concerns the selection, not the CheckBox.

focused

Example 1 shows a focused Texteld (indicated by the cursor). The CheckBox is focused at Example 2 (indicated by a frame).

Table 30.20: Provided Check types of Boolean check

30.9.1 Check text

30.9.

Checks Validates the text displayed in a component or sub-item. Contained in: All kinds of sequences Children: None
(288)

444

Execution: The text is sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT, reads its displayed text and compares it to the required value. Attributes:

30.9.

Checks

445

Figure 30.54: Check text attributes

Client The name of the SUT client process to which the check is sent. Variable: Yes

30.9.

Checks Restrictions: Must not be empty.

446

Component id The Id check.

(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node . with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Text The value to which the text displayed by the component or sub-item is compared. Variable: Yes Restrictions: Valid regexp if required. As regexp If this attribute is set, the components text is matched against a regexp (see (581) section 36.4 ) instead of comparing plain strings. Variable: Yes Restrictions: None Check type identier This attribute species the kind of check to perform. This makes it possible to support different kinds of checks of the same data type for a given target component without any ambiguity. With the help of a Checker additional check (669) types can be implemented as shown in section 39.3 . Variable: Yes Restrictions: Must not be empty Negate This ag determines whether to execute a positive or a negative check. If it is set, a negative check will be performed, i.e. the checked property must not match the expected value.
(519)

30.9.

Checks Variable: Yes Restrictions: None

447

Timeout Time limit in milliseconds until the check must succeed. To disable waiting, leave this value empty or set it to 0. Variable: Yes Restrictions: Must not be negative. Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None

30.9.

Checks

448

Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Name An optional name for the Check, mostly useful for the report. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.9.2 Boolean check

30.9.

Checks

449

Compares the current state of a component or sub-item with an expected value. Contained in: All kinds of sequences Children: None Execution: The expected state is sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT, determines its state and compares it to the required value. Attributes:
(288)

Figure 30.55: Boolean check attributes

30.9. Client

Checks

450

The name of the SUT client process to which the check is sent. Variable: Yes Restrictions: Must not be empty. Component id The Id check.
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . directly or access components By using a special syntax you can target Items (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Expected state The value to which the current state of the component or sub-item is compared. Variable: Yes Restrictions: None Check type identier This attribute species the kind of check to perform. This makes it possible to support different kinds of checks of the same data type for a given target component without any ambiguity. Boolean check nodes in particular are often used to check different kinds of states like enabled, editable or selected (like (442) described in section 30.9 ). With the help of a Checker additional check types (669) can be implemented as shown in section 39.3 . Variable: Yes Restrictions: Must not be empty Timeout Time limit in milliseconds until the check must succeed. To disable waiting, leave this value empty or set it to 0. Variable: Yes Restrictions: Must not be negative.
(519)

30.9.

Checks

451

Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Name An optional name for the Check, mostly useful for the report.

30.9.

Checks Variable: Yes Restrictions: None

452

Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.9.3 Check items

Validates multiple displayed text strings in a component or sub-item. Contained in: All kinds of sequences Children: None Execution: The text strings are sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT, reads its displayed values and compares them to the required values. Attributes:
(288)

30.9.

Checks

453

Figure 30.56: Check items attributes

Client The name of the SUT client process to which the check is sent. Variable: Yes

30.9.

Checks Restrictions: Must not be empty.

454

Component id The Id check.

(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node . with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Items This table holds the values to which the text strings displayed by the component or sub-item are compared. Each row represents one sub-item of the target component. The Text column holds the actual value while the Regexp column determines if this value is treated as a plain string or as a regexp (see section (581) 36.4 ). See section 2.2.5
(11) (519)

about how to work with tables in QF-Test.

Variable: Yes for the Text column, otherwise no. Restrictions: Valid regexp if required. Check type identier This attribute species the kind of check to perform. This makes it possible to support different kinds of checks of the same data type for a given target component without any ambiguity. With the help of a Checker additional check (669) types can be implemented as shown in section 39.3 . Variable: Yes Restrictions: Must not be empty Timeout Time limit in milliseconds until the check must succeed. To disable waiting, leave this value empty or set it to 0. Variable: Yes Restrictions: Must not be negative.

30.9.

Checks

455

Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Name An optional name for the Check, mostly useful for the report.

30.9.

Checks Variable: Yes Restrictions: None

456

Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.9.4 Check selectable items

Validates multiple displayed text strings in a component or sub-item and additionally checks their selection state. Contained in: All kinds of sequences Children: None Execution: The text strings and selection state data are sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT, reads its displayed values, compares them to the required values and checks the required selection state.
(288)

30.9.

Checks

457

Attributes:

Figure 30.57: Check selectable items attributes

Client

30.9.

Checks The name of the SUT client process to which the check is sent. Variable: Yes Restrictions: Must not be empty.

458

Component id The Id check.

(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter , actively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld. As an alternative you can copy the target node . with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Items This table holds the values to which the text strings displayed by the component or sub-item are compared. Each row represents one sub-item of the target component. The Text column holds the actual value while the Regexp column determines if this value is treated as a plain string or as a regexp (see section (581) 36.4 ). The Selected column holds the required selection state. See section 2.2.5
(11) (519)

about how to work with tables in QF-Test.

Variable: Yes for the Text column, otherwise no. Restrictions: Valid regexp if required. Check type identier This attribute species the kind of check to perform. This makes it possible to support different kinds of checks of the same data type for a given target component without any ambiguity. With the help of a Checker additional check (669) types can be implemented as shown in section 39.3 . Variable: Yes Restrictions: Must not be empty Timeout Time limit in milliseconds until the check must succeed. To disable waiting, leave this value empty or set it to 0.

30.9.

Checks Variable: Yes Restrictions: Must not be negative.

459

Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None

30.9. Name

Checks

460

An optional name for the Check, mostly useful for the report. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.9.5 Check image

Checks the displayed image of a component. This check is supported for all kinds of components and for sub-items. It is possible to check only parts of an image. To that end, a region within the image can be dened, either by dragging a rectangle or by editing the region attributes. Also, if the check image is smaller than the component, an offset into the component

30.9.

Checks

461

can be given. When recording only the visible part of a component or when cropping the check image to size after dening a region, the offset is determined automatically. Contained in: All kinds of sequences Children: None Execution: The image data is sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT and compares its current image to the required value. Attributes:
(288)

30.9.

Checks

462

Figure 30.58: Check image attributes

30.9. Client

Checks

463

The name of the SUT client process to which the check is sent. Variable: Yes Restrictions: Must not be empty. Component id The Id check.
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

button brings up a dialog in which you can select the component interThe actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node . with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Position of image relative to component If the image is smaller than the component, these offset coordinates together with the size of the image determine the sub-region of the component that is checked. Variable: Yes Restrictions: Must not be negative. Image The recorded image of the component. It can be displayed at different zoom levels, saved to or loaded from disk in the PNG format, or edited in an external imaging tool. The tool to use must be dened with the option External imaging (227) program . Variable: Yes Restrictions: Must not be negative. Actual check region inside image If only some part of the displayed image of a component needs to be checked, a region within the image can be dened that limits the comparison to that area. Variable: Yes Restrictions: Must not be negative.
(519)

30.9.

Checks

464

Algorithm for image comparison This attribute denes a special algorithm for image comparison. See Details (722) about the algorithm for image comparison for further information. Variable: Yes Restrictions: Must match a special syntax. Check type identier This attribute species the kind of check to perform. This makes it possible to support different kinds of checks of the same data type for a given target component without any ambiguity. With the help of a Checker additional check (669) types can be implemented as shown in section 39.3 . Variable: Yes Restrictions: Must not be empty Negate This ag determines whether to execute a positive or a negative check. If it is set, a negative check will be performed, i.e. the checked property must not match the expected value. Variable: Yes Restrictions: None Timeout Time limit in milliseconds until the check must succeed. To disable waiting, leave this value empty or set it to 0. Variable: Yes Restrictions: Must not be negative. Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable

30.9.

Checks

465

This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.
Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Name An optional name for the Check, mostly useful for the report. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after

30.9.

Checks

466

These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.9.6 Check geometry

Checks the location and size of a component. This check is supported for all kinds of components but not for sub-items. Contained in: All kinds of sequences Children: None Execution: The geometry data is sent to the SUT together with the data about the target component. The TestEventQueue determines the corresponding component in the SUT and compares location and size to the required values. Attributes:
(288)

30.9.

Checks

467

Figure 30.59: Check geometry attributes

Client

30.9.

Checks The name of the SUT client process to which the check is sent. Variable: Yes Restrictions: Must not be empty.

468

Component id The Id check.

(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is the target of the

The button brings up a dialog in which you can select the component inter , actively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld. As an alternative you can copy the target node . with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Geometry The X/Y coordinates, width and height to which the location and size of the component are compared. To check only some of these, e.g. just the location or just the size, leave the others empty. Variable: Yes Restrictions: Valid number, width and height > 0 Check type identier This attribute species the kind of check to perform. This makes it possible to support different kinds of checks of the same data type for a given target component without any ambiguity. With the help of a Checker additional check (669) types can be implemented as shown in section 39.3 . Variable: Yes Restrictions: Must not be empty Negate This ag determines whether to execute a positive or a negative check. If it is set, a negative check will be performed, i.e. the checked property must not match the expected value. Variable: Yes Restrictions: None
(519)

30.9.

Checks

469

Timeout Time limit in milliseconds until the check must succeed. To disable waiting, leave this value empty or set it to 0. Variable: Yes Restrictions: Must not be negative. Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a

30.10.

Queries
(529)

470 is thrown, for Wait for... nodes the respective

CheckFailedException specic exception. Variable: No Restrictions: None Name

An optional name for the Check, mostly useful for the report. Variable: Yes Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.10

Queries

Automating a GUI test gets difcult whenever the SUT shows dynamic or unpredictable behavior, i.e. when it displays values that change with every run of the program. This is

30.10.

Queries

471

typically the case for elds that are lled automatically with things like the current time, an automatic ID from a database, etc. QF-Test addresses this issue with means to read values from the SUTs components or to determine the numerical index of a sub-item when given its name. These values are stored in variables to be used again later as the test proceeds.

30.10.1 Fetch text

This node lets you read a value from the SUT during the execution of a test (52) run. The text is assigned to the local or global variable (see chapter 7 ) (473) named by the Variable name attribute. Not all components display text and some complex components contain multiple textual items, so this operation is only supported for certain components or sub-items. If you try to fetch the text from the wrong component, an (529) OperationNotSupportedException is thrown, while an unsupported sub-item (530) leads to an UnexpectedIndexException . The following table lists the supported component and sub-item targets for this operation. (P/S) means primary/secondary index.
Web

In web applications every node could contain some text, so QF-Test returns either the text or an empty value, but never throws an OperationNotSupportedException.
Class AbstractButton Dialog Frame JComboBox JComboBox JEditorPane JLabel JList JTabbedPane JTable JTableHeader JTextArea JTextComponent JTree Label TextField Index (P/S) -/-/-/-/List item/Character index/-/Item/Tab/Column/Row Column/Line number/-/Node/-/-/Result getText() getTitle() getTitle() Current value (use renderer) List item (use renderer) Structural element at index (experimental) getText() Item (use renderer) Title of tab Cell contents (use renderer) Column title (use renderer) Line of text getText() Node (use renderer) getText() getText()

Table 30.21: Components supported by Fetch text

30.10.

Queries
(288)

472 .

Contained in: All kinds of sequences Children: None

Execution: The data of the target component is sent to the SUT. The TestEventQueue determines the corresponding component, retrieves the requested value and sends it back to QF-Test, where it is stored in a global variable. Attributes:

Figure 30.60: Fetch text attributes

Client The name of the SUT client process from which to query the data. Variable: Yes Restrictions: Must not be empty. Component id The Id
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is to be queried.

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return ,

30.10.

Queries

473

when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Variable name The name of the global variable to which the result of the query is assigned (see (52) chapter 7 ). Variable: Yes Restrictions: Must not be empty. Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment
(519)

30.10.

Queries

474

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.10.2 Fetch index

With the help of this node you can determine the index of a sub-item during the execution of a test, provided its displayed text is known. Obviously only (519) is supported as the target component. The result is assigned to an Item (52) (475) the local or global variable (see chapter 7 ) named by the Variable name attribute. Contained in: All kinds of sequences Children: None Execution: The data of the target component is sent to the SUT. The TestEventQueue determines the corresponding component, searches for the requested sub-item and sends its index back to QF-Test, where it is stored in a global variable. Attributes:
(288)

30.10.

Queries

475

Figure 30.61: Fetch index attributes

Client The name of the SUT client process from which to query the data. Variable: Yes Restrictions: Must not be empty. Component id The Id
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is to be queried.

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty.
(519)

30.10.

Queries

476

Variable name The name of the global variable to which the result of the query is assigned (see (52) chapter 7 ). Variable: Yes Restrictions: Must not be empty. Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the

button.

30.10.

Queries

477

Variable: Yes Restrictions: None

30.10.3 Fetch geometry

Use this node to nd out the geometry of a window, component or sub-item in the SUT. The result is stored in up to four local or global variables, one each for the X and Y coordinates, width and height. This node is useful if you want to set the X and Y coordinates of a Mouse event relative to the right or bottom border of the target component. Simply fetch the components width and height and dene the coordinates using the extended variable syntax (60) for expressions (see section 7.6 ). The following table lists the supported sub-item targets for this operation. (P/S) means primary/secondary index.
Class JList JTabbedPane JTable JTable JTableHeader JTree Index (P/S) Item/Tab/Column/Column/Row Column/Node/Result Item Tab Column Cell Column title Node
(421) (421) (420)

Table 30.22: Components supported by Fetch geometry

Contained in: All kinds of sequences Children: None

(288)

Execution: The data of the target component is sent to the SUT. The TestEventQueue determines the corresponding component, retrieves its geometry and sends it back to QF-Test, where the values are stored in global variables. Attributes:

30.10.

Queries

478

Figure 30.62: Fetch geometry attributes

Client The name of the SUT client process from which to query the data. Variable: Yes Restrictions: Must not be empty. Component id The Id
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is to be queried.

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details).
(519)

30.10.

Queries

479

Variable: Yes Restrictions: Must not be empty. Location relative to window This attribute determines whether the X and Y coordinate of a component or sub-item is calculated relative to its parent component or relative to its parent window. For windows there is no difference. Variable: No Restrictions: None Variable for x The name of the global variable to which the X coordinate of the window, (52) component or sub-item is assigned (see chapter 7 ). If you are not interested in the X coordinate, leave this value empty. Variable: Yes Restrictions: None Variable for y The name of the global variable to which the Y coordinate of the window, (52) component or sub-item is assigned (see chapter 7 ). If you are not interested in the Y coordinate, leave this value empty. Variable: Yes Restrictions: None Variable for width The name of the global variable to which the width of the window, component or (52) sub-item is assigned (see chapter 7 ). If you are not interested in the width, leave this value empty. Variable: Yes Restrictions: None Variable for height The name of the global variable to which the height of the window, component or (52) sub-item is assigned (see chapter 7 ). If you are not interested in the height, leave this value empty. Variable: Yes Restrictions: None

30.10.

Queries

480

Local variable This ag determines whether to create local or global variable bindings. If unset, the variables are bound in the global variables. If set, the topmost current binding for a variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.11.

Miscellaneous

481

30.11

Miscellaneous

This section lists the remaining nodes that dont t in well with any of the other sections.

30.11.1 Set variable

This node lets you set the value of a global variable. If the test is run in(5) teractively from QF-Test and not in batch mode (see section 1.5 ) you can optionally set the value interactively. Contained in: All kinds of sequences Children: None Execution: If the test is run interactively and the Interactive attribute is set, a dialog is (483) shown in which the value for the variable can be entered. If the Timeout is exceeded or the value is conrmed with the OK button, the variable is bound accordingly in the global variables. If the dialog is canceled, the test run is stopped. In the non-interactive (483) case the variable is bound directly to the Default value . Attributes:
(483) (288)

30.11.

Miscellaneous

482

Figure 30.63: Set variable attributes

Variable name The name of the global variable to which the value is assigned (see chapter 7 ). Variable: Yes Restrictions: Must not be empty. Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No
(52)

30.11.

Miscellaneous

483

Restrictions: None Default value The default value for the variable if the test is run non-interactively, the (483) (483) Interactive attribute is not set or the Timeout is exceeded. Variable: Yes Restrictions: None Interactive Whether a dialog should be shown in which the value for the global variable can be entered. Ignored if the test is run non-interactively. Variable: Yes Restrictions: None Description A short description to display in the dialog. If you leave this value empty, the description Value for <Variable name> will be used. Variable: Yes Restrictions: None Timeout An optional timeout value for the dialog. If the dialog is shown and the value is left unchanged for the specied period of time, the dialog is closed automatically and the default value is used. This avoids blocking a test that is started interactively from QF-Test and then left to run unattended. Variable: Yes Restrictions: Empty or > 0. Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0

30.11.

Miscellaneous

484

Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.11.2 Wait for component to appear

This node is very important for the timing of a test run. The reaction time of the SUT varies depending on system and memory load, so it may take a while until, say, a complex dialog is opened. This node will delay further execution of the test until the desired component or sub-item is available. If the time (526) is thrown. limit is exceeded without success, a ComponentNotFoundException You can also use the Variable for result attribute to store the result into a variable and the Throw exception on failure attribute to suppress the exception. This node is intended only for relatively long delays. Short delays are handled automat(265) ically by the Timeout options . By setting the Wait for absence absence of a component. Children: None Execution: The data of the target component are sent to the SUT. The TestEventQueue waits until either the corresponding component becomes available or the time limit is exceeded. Attributes:
(486)

attribute this node can also be used to ensure the

(288)

Contained in: All kinds of sequences

30.11.

Miscellaneous

485

Figure 30.64: Wait for component to appear attributes

Client The name of the Java process in which to wait. Variable: Yes Restrictions: Must not be empty. Component id The Id
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

to wait for.

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return ,

30.11.

Miscellaneous

486

when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . By using a special syntax you can target Items directly or access components (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Timeout Time limit in milliseconds. Variable: Yes Restrictions: >= 0 Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Wait for absence If this attribute is set, QF-Test waits for the absence of a component. This is useful e.g. to ensure that a dialog has closed or was never opened in the rst (526) place. If the component does exist, a ComponentFoundException is thrown. It is possible to check for the absence of a sub-item. In this case, the complex component that holds the item must be present, otherwise a (526) ComponentNotFoundException is thrown. Variable: Yes Restrictions: None Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note
(519)

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None

30.11.

Miscellaneous

487

Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.
Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets

30.11.

Miscellaneous

488

you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.11.3 Wait for document to load

Web

This node is a variant of the Wait for component to appear node specically for web pages. It not only checks the existence of a given document. If the target document was already known to exist the last time an event was replayed, this node waits for the document to get reloaded. When working with web pages it is often the case that the same or very similar documents are loaded many times. Without this nodes functionality QF-Test could not discern the case where the old document is still around from the one where the document is already reloaded. In the former case, replaying an event could cause it to have no effect at all because at the same time reloading of the document begins. The Name of the browser window attribute can be used to limit the search to a given browser window or to dene a name for a new window. If Stop loading if timeout exceeded is set, QF-Test will abort loading the document if it doesnt nish in time. You can also use the Variable for result attribute to store the result in a variable and the Throw exception (526) on failure attribute to suppress the DocumentNotLoadedException . Contained in: All kinds of sequences Children: None Execution: The data of the target document are sent to the SUT. The TestEventQueue waits until corresponding document gets loaded or the time limit is exceeded in which case a DocumentNotLoadedException is thrown. Attributes:
(288)

(484)

30.11.

Miscellaneous

489

Figure 30.65: Wait for document to load attributes

Client The name of the SUT client process from which to query the data. Variable: Yes Restrictions: Must not be empty. Component id The Id
(508)

of the Window

(507)

, Component

(514)

or Item

(519)

node that is to be queried.

30.11.

Miscellaneous

490

The button brings up a dialog in which you can select the component inter actively. You can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. As an alternative you can copy the target node . with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V directly or access components By using a special syntax you can target Items (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. Name of the browser window This attribute has a dual use. If set to an existing name for a browser window, QF-Test waits for the document to load in that window. If the name is set but no browser window by that name exists, the search is limited to documents in new or not-yet-named windows. If the wait succeeds and a new document is loaded, the window name is assigned to the documents browser window. This is the only way to dene a name for a popup window. Explicitly launched browsers can have their (404) (402) name set via the Name of the browser window attribute of a Start browser node. Variable: Yes Restrictions: None Timeout Time limit in milliseconds. Variable: Yes Restrictions: >= 0 Stop loading if timeout exceeded If this attribute is set and the timeout is exceeded without a matching document nishing to load, QF-Test will cause loading to stop, either in all browsers or, if Name of the browser window is set, the browser window specied therein. Result and exception handling are not affected by this attribute. If the timeout is exceeded the operation is considered a failure regardless of whether loading is stopped or not. Variable: Yes Restrictions: None Variable for result This optional attribute determines the name for the result variable of the action. If
(519)

30.11.

Miscellaneous

491

set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No

30.11.

Miscellaneous

492

Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.11.4 Wait for download to nish

Web

This specialized node is applicable only for web Clients. It can be used to wait for the completion of a download that was earlier started via QF-Test. This is important in case you need to verify the contents of the le or to measure the time it took to download it. If the timeout is exceeded without the download nishing, a DownloadNotCompleteException is thrown unless suppressed via the Throw exception on failure attribute. Either way the download can be canceled by activating the Cancel download if timeout exceeded attribute, which may be necessary to enable another download to the same le. The result can also be stored in a variable by dening its name in the Variable for result attribute. Contained in: All kinds of sequences Children: None Execution: The target le identifying the download is sent to the SUT where QFTest waits for the download to nish or the time limit is exceeded in which case a DownloadNotCompleteException is thrown.
(288)

30.11.

Miscellaneous

493

Attributes:

Figure 30.66: Wait for download to nish attributes

Client The name of the SUT client process from which to query the data. Variable: Yes Restrictions: Must not be empty. File The target le for the downloaded. Variable: Yes Restrictions: Valid le name

30.11. Timeout

Miscellaneous

494

Time limit in milliseconds. Variable: Yes Restrictions: >= 0 Cancel download if timeout exceeded If this attribute is set and the timeout is exceeded without the download nishing the download is canceled. Variable: Yes Restrictions: None Variable for result This optional attribute determines the name for the result variable of the action. If set, the respective variable will be set to true for a successful check or wait and to false in case of failure.
Note

If this attribute is set, the attribute Error level of message is ignored and no error is reported. The attribute Throw exception on failure always remains effective, so it is possible to set a result variable and still throw an exception. Variable: Yes Restrictions: None Local variable This ag determines whether to create a local or global variable binding. If unset, the variable is bound in the global variables. If set, the topmost current binding for the variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Error level of message This attribute determines the error level of the message that is logged in case of failure. Possible choices are message, warning and error.

Note

If the attribute Throw exception on failure is set, this attribute is irrelevant and if Variable for result is set this attribute is ignored. Variable: No

30.11.

Miscellaneous

495

Restrictions: None Throw exception on failure Throw an exception in case of failure. For Check... nodes a (529) CheckFailedException is thrown, for Wait for... nodes the respective specic exception. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.11.5 Load resources

30.11.

Miscellaneous

496

This node is used to load a ResourceBundle and make its values available for the extended variable syntax ${group:name} (see section (57) 7.5 ). To learn more about ResourceBundles see the description of the (496) ResourceBundle attribute. Contained in: All kinds of sequences Children: None Execution: The ResourceBundle is loaded and its values are made available under (496) the Group name. Attributes:
(288)

Figure 30.67: Load resources attributes

Group The name of the group by which values of the ResourceBundle are referred to. The value of a denition of the form name=value in the ResouceBundle can be (57) retrieved with ${group:name} (see section 7.5 ). Variable: Yes Restrictions: Must not be empty and should not contain special characters like : or $. ResourceBundle

30.11.

Miscellaneous

497

The name of the ResourceBundle to load. A little Java background is needed to understand this attribute. The resources are read with the help of the Java method ResourceBundle.getBundle(). For this to work, a matching le with the extension .class or .properties must be located somewhere on the CLASSPATH. Use the fully qualied name for the le, including packages, with a dot (.) as separator, but without extension or locale identier. Example: QF-Test comes with a German ResourceBundle in the le de/qfs/apps/qftest/resources/properties/qftest_de.properties, which is contained in the archive qfshared.jar. To load that ResourceBundle, set this attribute to (497) de.qfs.apps.qftest.resources.properties.qftest and the Locale to de. Variable: Yes Restrictions: Must name a ResourceBundle on the CLASSPATH. Locale The main use of ResourceBundles is to provide data in different languages. This attribute determines, which version of a ResourceBundle is retrieved. The value must follow the ISO standard language_country_variant. Language is a two letter lowercase code like en for English, country a two letter uppercase code like US for American or UK for British English. The variant discriminates further but is rarely used. As mentioned, QF-Test relies on the Java method ResourceBundle.getBundle() to load the ResourceBundle, which is described in detail in the Java documentation and works as follows: To load a ResourceBundle named res for the locale en_US, Java rst searches the CLASSPATH for a le named res_en_US.class or res_en_US.properties, then for res_en.class or res_en.properties and nally for res.class and res.properties. The less specic les are loaded even if more specic les are found, but only values not dened in the more specic les are used. That way you can dene all English resources in res_en.properties and place only those that differ in res_en_UK.properties and res_en_US.properties. Unfortunately Java has a feature that can lead to surprising results. If no specic le but only the base le res.properties is found, Java tries the whole process a second time, this time for the current default locale of the VM. As a result, if the current locale for QF-Test is German and you want to load English resources that are dened in res.properties and no res_en.properties exists, Java will load the German version from res_de.properties, even if you request the

30.11.

Miscellaneous

498

locale en. You can work around this be setting this attribute to the underscore _. In that case, only the base le res.properties is loaded. To use the current locale of the VM, leave this value empty. Variable: Yes Restrictions: Empty or valid locale identier. Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.11.6 Load properties

This node is used to load data from a Properties le and make its values available for the extended variable syntax ${group:name} (see section (57) 7.5 ). Properties les are easier to handle than ResourceBundles since you request the le directly, but they are less powerful. The format of a Properties le is simple: lines

30.11.

Miscellaneous

499

of the form name=value with arbitrary whitespace around the = character. Complex denitions spanning multiple lines are possible. Please see the Java documentation for details or ask a developer. Contained in: All kinds of sequences Children: None Execution: The Properties le is loaded and its values are made available under the (499) Group name. Attributes:
(288)

Figure 30.68: Load properties attributes

Group The name of the group by which values of the Properties le are referred to. The value of a denition of the form name=value in the Properties le can be (57) retrieved with ${group:name} (see section 7.5 ). Variable: Yes Restrictions: Must not be empty and should not contain special characters like : or $. Properties le The le to load the Properties from. This can either be an absolute path name or a path relative to the directory of the current suite. In either case you should always use / as the separator for directories, even under Windows. QF-Test will translate this to the correct value for the current operating system.

30.12.

HTTP Requests

500

The ... button brings up a dialog in which you can select You the le interactively. can also get to this dialog by pressing Shift-Return or Alt-Return , when the focus is in the text eld. Variable: Yes Restrictions: Must be an existing Properties le. Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.12

HTTP Requests

This section describes how to send HTTP Request using QFTest.

30.12.1 Server HTTP request

30.12.
Web

HTTP Requests

501

This highly specialized node sends a GET or POST request via HTTP/HTTPS directly to a web server. Such a request can be very helpful for load tests or mass data computing scenarios (e.g. lling out a form) since the simulation of user interactions and the respective loading time of the SUT are omitted during replay. The use of requests is an enhancement of the functionalities for load tests and data(176) (330) driven testing described in chapter 21 and section 30.4 . If the status code returned from the server is 400 or higher, a corresponding error message is created in the run-log. A detailed description of the different status codes can be found at http://www.w3.org/Protocols/HTTP/HTRESP.html. Additionally you can store the response from the server in a variable and if the attribute Add server response to runlog is active the response is also written to the run-log. Contained in: All kinds of sequences Children: None Execution: The GET/POST request is sent directly by QF-Test via HTTP/HTTPS to the specied URL. If the status code returned from the server is not equal to 200 (status=OK) a corresponding error message is created in the run-log. Attributes:
(288)

30.12.

HTTP Requests

502

Figure 30.69: Server HTTP request Attribute

URL The URL to which to send the request, not including parameters. HTTP and HTTPS are acceptable values for the protocol. Variable: Yes Restrictions: Must not be empty. Method This attribute denes the method of the request, GET or POST. Variable: No Restrictions: None

30.12.

HTTP Requests

503

Parameters Here you can specify the parameters for the request. The parameters will be URL (11) encoded by QFTest at execution. See section 2.2.5 for further information how to work with the table. Variable: Yes Restrictions: None Variable for server response The name of the variable to which the server response is assigned (see chapter (52) 7 ). Variable: Yes Restrictions: None Variable for HTTP status code The name of the variable to which the HTTP status code is assigned (see chapter (52) 7 ). Variable: Yes Restrictions: None Local variable This ag determines whether to create local or global variable bindings. If unset, the variables are bound in the global variables. If set, the topmost current binding for a variable is replaced with the new value, provided this binding is within the (352) (317) (288) context of the currently executing Procedure , Dependency or Test-case node. If no such binding exists, a new binding is created in the currently executing Procedure, Dependency or Test-case node or, if there is no such node, in the global (52) bindings. See chapter 7 for a detailed explanation of variable binding and lookup. Variable: No Restrictions: None Add server response to run-log If activated the server response is written to the run-log in addition to the status code. Variable: Yes Restrictions: None Timeout

30.12.

HTTP Requests

504

Time limit in milliseconds until the HTTP Request must succeed. To disable the limit, leave this value empty. Variable: Yes Restrictions: Must not be negative. Error level if time limit exceeded This attribute determines what happens in case the time limit is exceeded. If set (529) to exception, a CheckFailedException will be thrown. Otherwise a message with the respective error-level will be logged in the run-log. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.12.2 Browser HTTP request

30.12.
Web

HTTP Requests

505

This highly specialized node sends a GET or POST request via HTTP/HTTPS directly to a web server. Such a request can be very helpful for load tests or mass data computing scenarios (e.g. lling out a form) since the simulation of user interactions and the respective loading time of the SUT are omitted during replay. The use of requests is an enhancement of the functionalities for load tests and data(176) (330) driven testing described in chapter 21 and section 30.4 . Contained in: All kinds of sequences Children: None Execution: The GET/POST request is sent within the browser via HTTP/HTTPS to the specied URL. The response is shown directly in the Brwoser. Attributes:
(288)

Figure 30.70: Browser HTTP request Attribute

Client

30.12.

HTTP Requests

506

The name of the SUT client process in which to execute the request. Variable: Yes Restrictions: Must not be empty. Component id The Web page
(510)

in which the request should be submitted.

The button brings up a dialog in which you can select the component inter , actively. You can also get to this dialog by pressing Shift-Return or Alt-Return when the focus is in the text eld. As an alternative you can copy the target node with Ctrl-C or EditCopy and insert its Id into the text eld by pressing Ctrl-V . directly or access components By using a special syntax you can target Items (50) (145) from another suite (see section 6.3 and section 17.1 for details). Variable: Yes Restrictions: Must not be empty. URL The URL to which to send the request, not including parameters. HTTP and HTTPS are acceptable values for the protocol. Internationalized domain names (IDN) are not supported in the URL attribute as well as links to local le system starting with le:///. Variable: Yes Restrictions: Must not be empty. Method This attribute denes the method of the request, GET or POST. Variable: No Restrictions: None Parameters Here you can specify the parameters for the request. The parameters will be URL (11) encoded by QFTest at execution. See section 2.2.5 for further information how to work with the table. Variable: Yes Restrictions: None
(519)

30.13. Id

Windows, Components and Items

507

At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Delay before/after These attributes cause a delay before or after the execution of the node. If a (263) value is empty, the Default delay from the global options is used. Variable: Yes Restrictions: Valid number >= 0 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.13
(507)

Windows, Components and Items

(514) (519)

Windows , Components and Items are the foundation on which a test-suite is built. They represent the windows and components of the SUT as well as the sub-items of complex components. Many of the other node types need a window or component as a target, e.g. events or (442) (508) checks . To that end their Component id attribute must be set to the Id of an existing (507) (514) (519) Window , Component or Item node. All of the windows and components of a test-suite are collected under the (524) Windows and components node, which is always located at the bottom of the suite.
(419)

30.13.1 Window

30.13.

Windows, Components and Items This node is a surrogate for a window in the SUT. Events (508) other nodes refer to it by its Id . Contained in: Window group
(522) (419)

508 , checks
(442)

and

, Windows and components .

(524)

Children: Component group Attributes:

(523)

, Component

(514)

Execution: Cannot be executed.

Figure 30.71: Window attributes

Id This id is the means by which other nodes refer to this window. Therefore it may appear in many places and you should take care to assign an id with a meaning, i.e. one that is easy to remember and recognize. An id must be unique within the test-suite.

30.13.

Windows, Components and Items

509

Variable: No Restrictions: Must not be empty, contain any of the characters #, $, @, &, or % or start with an underscore (_). Class name The fully qualied name of the Java class of the window or one of its superclasses. Variable: Yes Restrictions: Must not be empty. Name The name of the window that was set by the developers of the SUT with the (579) setName(String) method. See section 36.2 for why and how names should be set on all interesting components. Variable: Yes Restrictions: None Feature If there is no Name available for the window, QF-Test tries to recognize it by a characteristic feature. In the case of a Frame or a Dialog this is the windows title. See section 36.2 Variable: Yes Restrictions: None As regexp If this attribute is set, the Feature Variable: Yes Restrictions: None Modal In case of a Dialog this attribute determines whether the dialog is modal. Variable: Yes Restrictions: None Geometry The X/Y coordinate, width and height of the window form the basis for the (579) recognition of the window. However, they play a minor roll as long as either a (509) (509) Name was provided or a Feature is available.
(509) (579) (509)

for more information about component recognition.

is a regular expression (see section 36.4

(581)

).

30.13.

Windows, Components and Items

510

For windows whose location and size vary widely you should clear these attributes.
Note

If now values are specied, the recognition algorithm starts with a perfect geometry match for all candidates. To prevent false positive hits you can disable geometry matching by setting the values to a single - character. Variable: Yes Restrictions: Width and height must not be negative. GUI engine The GUI engine to which the Window and all its Component children belong. QF-Test records awt for AWT/Swing and swt for SWT. Only really relevant for (557) SUTs with more than one GUI engine as described in chapter 33 . Variable: Yes Restrictions: See chapter 33 Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
(557)

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.13.2 Web page

Web

A Web page is a variant of a Window node specically used for testing web applications. It represents the top-level document in a Browser. Nested doc(514) uments in FRAME nodes are represented as Components . In contrast to a Window a Web page has no Class name, Modal, geometry or GUI engine attributes as these are either implicitly dened or redundant. Contained in: Window group Children: Component group
(522)

(507)

, Windows and components

(514)

(524)

(523)

, Component

Execution: Cannot be executed.

30.13.

Windows, Components and Items

511

Attributes:

Figure 30.72: Web page attributes

Id This id is the means by which other nodes refer to this page. Therefore it may appear in many places and you should take care to assign an id with a meaning, i.e. one that is easy to remember and recognize. An id must be unique within the test-suite. Variable: No Restrictions: Must not be empty, contain any of the characters #, $, @, &, or % or start with an underscore (_). Name of the browser window This attribute can be ignored unless you need to test a web application with multiple open browser windows holding similar documents. In that case the Name of the browser window attribute can be used to identify the browser window. The (404) name of a browser window can be dened via the Name of the browser window (402) attribute of a Start browser node.

30.13.

Windows, Components and Items

512

Variable: Yes Restrictions: None Name A web page has no name unless one is implemented via a NameResolver. See (634) section 39.1 about the extension API for NameResolvers. Variable: Yes Restrictions: None Feature The main Feature of a web page is its URL with the parameters removed. If the (270) is set, the URL is option Limit URL feature of Web page node to host or le further reduced to the host or le name. See section 36.2 Variable: Yes Restrictions: None As regexp If this attribute is set, the Feature Variable: Yes Restrictions: None Extra features Besides the main Feature a Component can have additional features represented as name/value pairs. The kind of component determines which extra features are recorded. Each extra feature can have one of three states: Ignore (Hint for searching: currently represented as 0) This extra feature is just for information. It has no inuence on component recognition. Should match (Hint for searching: currently represented as 1) Target components matching this extra feature have a higher probability for component recognition than those that dont match it. Must match (Hint for searching: currently represented as 2) The target component must match this extra feature. Any component not matching it is no candidate for component recognition.
(512) (579)

for more information about component recognition.

is a regular expression (see section 36.4

(581)

).

30.13.

Windows, Components and Items

513

Additional columns allow for matching against a regular expression or to negate the expression, e.g. to dene that the class attribute of a DOM node in a web page should not be dummy. The absence of an extra feature can be enforced by adding one with an empty value. QF-Test automatically assigns some extra features to recorded components.
Name columns imagehash qfs:item Web Swing, SWT Web Engine Description Column count in TABLE components. Shows the hash value of icons of a button or menuitem. Shows the item index of a DomNode, if its an item of a complex GUI component. This could get recorded, if some child nodes are additionally recorded as those component might be interesting as well. This could affect content of lists, tables, tabfolders or trees. Shows a matching label for the component, e.g. the text of a button or a label close to the respective component. If no own text or label could be found, it also tries to use tooltips or icon descriptions. Index of components with the same name. Possibly assigned automatically when the Validate component recognition during (242) recording option is active. Shows if a component of the class Window is modal.

qfs:label

All

qfs:matchindex

All

qfs:modal

Web

Table 30.23: Extra features assigned by QF-Test

Variable: Yes Restrictions: Names must not be empty Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes,

30.13.

Windows, Components and Items

514

this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.13.3 Component
Component nodes represent the components of the SUT. Other nodes refer to (515) (507) them by their Id , similar to the way Windows are referenced. The Window node that is the direct or indirect parent of the component must be the equivalent of the components window parent in the SUT. Contained in: Component group Children: Item Attributes:
(519) (523) (507)

, Window

(507)

Execution: Cannot be executed.

30.13.

Windows, Components and Items

515

Figure 30.73: Component attributes

Id This id is the means by which other nodes refer to this component. Therefore it may appear in many places and you should take care to assign an id with a meaning, i.e. one that is easy to remember and recognize. An id must be unique within the test-suite. Variable: No Restrictions: Must not be empty, contain any of the characters #, $, @, &, or % or start with an underscore (_).

30.13.

Windows, Components and Items

516

Class name For SWT and Swing application this is the fully qualied name of the Java class of the component or one of its super-classes whereas for web applications there is a (39) pseudo class hierarchy described in section 5.4 . The actual class recorded by QF-Test depends on the setting of the option Record (242) system class only and on potentially registered ClassNameResolvers (see (636) section 39.1.2 ). Upon replay, class matching is based on the actual or pseudo class hierarchy, so you can manually change this attribute to any of the elements base classes. Variable: Yes Restrictions: Must not be empty. Name The name of the component that was set by the developers of the SUT with the (579) setName(String) method. See section 36.2 for why and how names should be set on all interesting components. Variable: Yes Restrictions: None Feature If there is no Name available for the component, QF-Test tries to recognize it by a characteristic feature. See section 36.2 Variable: Yes Restrictions: None As regexp If this attribute is set, the Feature Variable: Yes Restrictions: None Extra features Besides the main Feature a Component can have additional features represented as name/value pairs. The kind of component determines which extra features are recorded. Each extra feature can have one of three states: Ignore (Hint for searching: currently represented as 0) This extra feature is just for information. It has no inuence on component recognition.
(516) (581) (579) (516)

for more information about component recognition.

is a regular expression (see section 36.4

).

30.13.

Windows, Components and Items

517

Should match (Hint for searching: currently represented as 1) Target components matching this extra feature have a higher probability for component recognition than those that dont match it. Must match (Hint for searching: currently represented as 2) The target component must match this extra feature. Any component not matching it is no candidate for component recognition. Additional columns allow for matching against a regular expression or to negate the expression, e.g. to dene that the class attribute of a DOM node in a web page should not be dummy. The absence of an extra feature can be enforced by adding one with an empty value. QF-Test automatically assigns some extra features to recorded components.
Name columns imagehash qfs:item Web Swing, SWT Web Engine Description Column count in TABLE components. Shows the hash value of icons of a button or menuitem. Shows the item index of a DomNode, if its an item of a complex GUI component. This could get recorded, if some child nodes are additionally recorded as those component might be interesting as well. This could affect content of lists, tables, tabfolders or trees. Shows a matching label for the component, e.g. the text of a button or a label close to the respective component. If no own text or label could be found, it also tries to use tooltips or icon descriptions. Index of components with the same name. Possibly assigned automatically when the Validate component recognition during (242) recording option is active. Shows if a component of the class Window is modal.

qfs:label

All

qfs:matchindex

All

qfs:modal

Web

Table 30.24: Extra features assigned by QF-Test

30.13.

Windows, Components and Items

518

Variable: Yes Restrictions: Names must not be empty Structure These two elds hold additional structural information needed for component (579) recognition during a test run. The Class count is the number of components within the same parent container and with the same class (or a class derived thereof). The Class index is the index that this component has in the list of these components with matching class. As usual the rst component has index 0. When counting the components of matching class, invisible components are considered as well. This is more robust but means that the values may be higher than expected.
Note

It is possible to speciy just the Class index or the Class count attribute. Variable: Yes Restrictions: None Geometry The X/Y coordinate, width and height of the component form the basis for the (579) recognition of the component. However, they play a minor roll as long as either (516) (516) (518) a Name was provided or a Feature or structural information is available. For components whose location or size typically vary widely at runtime, these values are not recorded.

Note

If no values are specied, the recognition algorithm starts with a perfect geometry match for all candidates. To prevent false positive hits you can disable geometry matching by setting the values to a single - character. Variable: Yes Restrictions: Width and height must not be negative. Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.13.

Windows, Components and Items

519

30.13.4 Item
For complex Swing components like JTable or JTree it is possible to dene (420) (442) (470) Mouse events , checks or queries relative to a sub-item of the component instead of the component itself. Such a sub-item is identied with the (520) help of an index, the Primary index . This index can be given in one of three ways: as (581) a string, a number or a regexp (see section 36.4 ). A string or regexp designate a sub-item with a corresponding representation while a number refers to a sub-item by its index. Like in Java the rst sub-items index is 0. The JTable supports an additional index to refer directly to a table cell. The (520) (520) Primary index determines the column and the Secondary index the row of the cell. There are two representations for the nodes of a JTree component, at like a list or as a hierarchy using paths. A node named tmp under a node named usr is represeneted as just tmp in the rst case, as /usr/tmp in the latter. The option Represent tree node (247) as path determines the representation used when recording sub-items. Currently sub-items are supported for the following Swing components:
Class JComboBox JEditorPane JList JTabbedPane JTable JTableHeader JTextArea JTree Primary List element Structural element (experimental) List element Tab Table column Table column Line Node/Row Secondary Row -

Table 30.25: Sub-items of complex Swing components

Contained in: Component Children: None.

(514)

Execution: Cannot be executed. Attributes:

30.13.

Windows, Components and Items

520

Figure 30.74: Item attributes

Id This id is the means by which other nodes refer to this sub-item. Therefore it may appear in many places and you should take care to assign an id with a meaning, i.e. one that is easy to remember and recognize. An id must be unique within the test-suite. Variable: No Restrictions: Must not be empty, contain any of the characters #, $, @, &, or % or start with an underscore (_). Primary index Designates the sub-item. Depending on whether As string , As number or (521) As regexp is selected, the sub-item is determined by its index or by a string or regexp match. It is OK to have an empty index, e.g. to designate a table column with an empty heading. Variable: Yes Restrictions: Must be a valid number or regexp if required. Secondary index For the JTable class two kinds of sub-items are supported. If only the (520) Primary index is given, a whole column is referenced. An additional Secondary index designates a cell in this column.
(521) (521)

30.13.

Windows, Components and Items

521

To dene a secondary index its checkbox must be selected rst. This is necessary to tell an empty index from a non-existent one. An empty secondary index designates a table cell with empty content. Variable: Yes Restrictions: Must be a valid number or regexp if required. As string or Secondary index is interpreted as a plain string. The The Primary index sub-item is determined by matching its representation against that string. Variable: No Restrictions: None As number or Secondary index The Primary index sub-item is determined by its index. Variable: No Restrictions: None As regexp The Primary index or Secondary index is interpreted as a regexp (see section (581) 36.4 ). The sub-item is determined by matching its representation against that regexp. Variable: No Restrictions: None Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note
(520) (520) (520) (520) (520) (520)

is interpreted as a number.

The

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.13.

Windows, Components and Items

522

30.13.5 Window group

A Window groups only purpose is to provide structure to the Windows of a (507) test-suite. If you have to manage a large number of Windows you can even nest Window groups. Contained in: Windows and components Children: Window group Attributes:
(522) (524) (507)

, Window group

(522)

, Window

(507)

Execution: Cannot be executed.

Figure 30.75: Window group attributes

Name You can choose an arbitrary name. It is displayed in the tree view of the suite. Variable: No Restrictions: None Id At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes,

30.13.

Windows, Components and Items

523

this text area might not be the right place. There are many excellent editors that (226) are much better suited to this task. The option External editor command lets you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.13.6 Component group

A Component groups only purpose is to provide structure to the Components (507) (514) of a Window . If you have to manage a large number of Components inside (507) a Window you can even nest Component groups. Contained in: Window
(507) (514)

, Component group
(523)

(523)

Children: Component group Attributes:

, Component

(514)

Execution: Cannot be executed.

Figure 30.76: Component group attributes

Name You can choose an arbitrary name. It is displayed in the tree view of the suite. Variable: No Restrictions: None

30.13. Id

Windows, Components and Items

524

At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

30.13.7 Windows and components

The Windows and components node is always located at the end of the suite. (507) (514) (519) It is the place where all of the Windows , Components and Items of the test-suite are collected. Contained in: Root node Children: Window group Attributes:
(522)

, Window

(507)

Execution: Cannot be executed.

Figure 30.77: Windows and components attributes

30.13. Id

Windows, Components and Items

525

At the moment the Id attribute has no meaning for this type of node. Variable: No Restrictions: Must not contain any of the characters \, #, $, @, &, or % or start with an underscore (_). Comment Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.
Note

For detailed documentation, especially for Test-set, Test-case or Procedure nodes, this text area might not be the right place. There are many excellent editors that (226) lets are much better suited to this task. The option External editor command you dene an external editor in which comments can be edited conveniently by pressing Alt-Return or by clicking the Variable: Yes Restrictions: None

button.

Chapter 31 Exceptions
There are quite a lot of exceptions that can be thrown during the execution of a test. This chapter lists the exceptions in hierarchical order, shows the typical error messages and gives a short explanation. If you want to work with those exceptions in scripts, please take a look into section (613) 37.9 . TestException This is the base class of all exceptions that can be thrown during a test run. The (382) actual exception thrown should almost always be of a derived class. A Catch (383) with the Exception class set to TestException will catch all possible (382) exceptions. Just like in Java you should normally not use such a Catch since it may hide unexpected Exceptions. ComponentNotFoundException This exception is thrown whenever the target component for an event or a (442) (484) check cannot be determined. Failure of a Wait for component to appear will also cause a ComponentNotFoundException unless the nodes (486) Wait for absence attribute is set. DocumentNotLoadedException This exception is a variant of ComponentNotFoundException and thrown (488) specically if a Wait for document to load node fails. ComponentFoundException
(419)

Exceptions

527

This is the opposite of a ComponentNotFoundException, thrown by a (484) (486) Wait for component to appear with the Wait for absence attribute set. ModalDialogException This exception is thrown when an event is blocked by a modal dialog. See the (256) option Check for modal dialogs for details. ComponentCannotGetFocusException
Note
(419)

This exception is obsolete and should not occur anymore. or This exception is thrown when the target component for a Key event (427) Text input is a text component that cannot get the keyboard focus for some reason. With JDK 1.4 the event cannot be delivered to the component in that case. DisabledComponentException This exception is thrown when the target component for a Mouse event , (423) (427) Key event or Text input is not enabled. In that case the event would be silently ignored, very likely leading to unexpected results during furhter execution of the test. For backwards compatibility this kind of exception can be suppressed by deacti(257) vating the option Throw DisabledComponentException . BusyPaneException This exception is thrown when the target component for a Mouse event , (423) (427) Key event or Text input is covered by a GlassPane with a busy mouse (267) cursor. The option Wait for busy GlassPane (ms) determines how long QF-Test will wait for the GlassPane to disappear before the exception is actually triggered. InvisibleDnDTargetException This exception is thrown when the location in the target component for a (420) Mouse event of type DRAG_FROM, DRAG_OVER or DROP_TO for a Drag&Drop operation is invisible and cannot be made visible by scrolling the target component. DeadlockTimeoutException
(420) (420) (423)

Exceptions

528

This exception is thrown when the SUT does not react for a given amount of time (265) which is dened in the option Deadlock detection (s) . VariableException This exception is never thrown itself but is the base class for exceptions thrown in the context of variable expansion. BadVariableSyntaxException This exception is thrown when a value that is to be expanded doesnt follow a proper variable syntax (e.g. no closing brace). MissingPropertiesException This exception is thrown when no properties or ResourceBundle are available for (495) the id of a property or resource looked up with ${id:prop} (see Load resources (498) and Load properties ). MissingPropertyException This exception is thrown when a property looked up with ${id:property} is not (495) (498) available (see Load resources and Load properties ). RecursiveVariableException This exception is thrown when the expansion of a variable expression leads to recursive variable lookup, e.g. if you set the variable named x to $(y) and the variable named y to $(x) and then try to expand the value $(x). UnboundVariableException This exception is thrown when a variable for a value that is to be expanded doesnt exist. VariableNumberException This exception is thrown when a variable expansion for a numeric attributes results in something other than a number. BadExpressionException This exception is thrown when evaluating a $[...] expression fails (see section (60) 7.6 ).

Exceptions BadTestException This exception is thrown when evaluating the Condition node fails. BadRegexpException
(371)

529

of an If

(370)

or Elseif

(373)

This exception is thrown whenever converting a String to a regular expression (581) (519) (443) (see section 36.4 ) fails, e.g. for Item or Check text nodes. BadRangeException This exception is thrown when the syntax of the Iteration ranges for a Data binder node has invalid syntax or species an index outside the valid data range. CannotExecuteException This exception is thrown when execution of a process from a Start SUT client (395) (406) Start Java SUT client or Start process node fails. InvalidDirectoryException This exception is thrown when the Directory node refers to a non-existent directory. CheckFailedException This exception is thrown when a (447) Throw exception on failure attribute fails. CheckNotSupportedException As explained in the section about checks , each check can handle only a limited set of target components. This exception is thrown when the target component is not suitable for a check. OperationNotSupportedException This exception is thrown when an operation like Fetch text the designated target component.
(471) (442) (401) (399)

attribute of a Start SUT client

(399)

check

(442)

node

with

activated

is not supported for

Exceptions BadComponentException This exception is thrown, when a component for an event (430) non-window for a Window event . IndexFormatException
(419)

530

is not suitable, i.e. a

This exception is thrown when an invalid index format for a sub-item is (50) encountered (see section 6.3 ). IndexFoundException This exception is thrown when a sub-item is found during execution of a (484) Wait for component to appear node that looks for the absence of the item. IndexNotFoundException This exception is thrown when no sub-item can be located for a given index. IndexRequiredException This exception is thrown when no sub-item index is provided for an operation that (443) requires one, e.g. a Check text on a JTree. UnexpectedIndexException This exception is thrown when a sub-item index is provided for an operation that (452) does not require one, e.g. a Check items on a JTree. ClientNotConnectedException This exception is thrown when the target client for an operation is not connected. (530) It differs from a NoSuchClientException in that there is an active process for that name but no RMI connection. NoSuchClientException This exception is thrown when the target client for an operation does not exist. DuplicateClientException This exception is thrown if an attempt is made to run more than one client simultaneously under the same name.

Exceptions UnexpectedClientException

531

This exception is thrown when an unexpected exception is thrown in the SUT (419) during the replay of an event . Unless it is due to a bug in QF-Test, it indicates a problem in the SUT. ClientNotTerminatedException This exception is thrown when a Wait for process to terminate and the process doesnt terminate. UnexpectedExitCodeException This exception is thrown when the exit code of a terminated client doesnt match (416) (417) the expected value in a Wait for process to terminate nodes Expected exit code attribute. BadExitCodeException attribute of a This exception is thrown when the Expected exit code (416) Wait for process to terminate node doesnt match the specication and cannot be parsed. ComponentIdMismatchException This exception is thrown when the Component id attribute of a node points to a (507) (514) (519) node that is not a Window , Component or Item . UnresolvedComponentIdException This exception is thrown when the target of the Component id node cannot be determined. TestNotFoundException This exception is thrown when the Test DependencyNotFoundException This exception is thrown when the Dependency cannot be determined.
(317) (303) (421) (421) (417) (416)

node is executed

attribute of a

for a Test call

(299)

cannot be determined.

for a Dependency reference

(320)

Exceptions ProcedureNotFoundException This exception is thrown when the Procedure determined. StackOverflowException
(352)

532

for a Procedure call

(355)

cannot be

gets too deep, This exception is thrown when the nesting of Procedure calls hinting to a problem with endless recursion. See also the option Call stack (249) size . UserException This exception is thrown explicitly by a Throw CannotRethrowException This exception is thrown when an attempt is made to rethrow an exception with a (388) (382) Rethrow node but no exception was caught by a Catch node. ScriptException This exception is thrown when the execution of a script from a Server script (392) SUT script fails. BreakException This is not a standard TestException and cannot be caught by a Catch node. It (368) is thrown by a Break node in order to break out of a loop. From a script, raising a BreakException will have the same effect. If thrown outside of a loop, a BreakException will cause the error below. ReturnException This is not a standard TestException and cannot be caught by a Catch node. It (358) (352) is thrown by a Return node in order to return from a Procedure . From a script, raising a ReturnException will have the same effect. If thrown outside of a Procedure, a ReturnException will cause the error below. TestOutOfMemoryException This is a special exception that is thrown when QF-Test determines that it is
(382) (382) (390) (387)

(355)

node.

or

Exceptions

533

running out of memory during test execution. The exception causes the test to abort immediately and cannot be caught because once QF-Test has run out of memory there is little it can do to handle it. QF-Test tries to keep a little reserve memory so it will at least try to save the run-log.

Part IV Technical reference

Chapter 32 Command line arguments and exit codes

32.1 Call syntax

The call syntax for interactive and batch mode varies widely since some command line arguments are specic to interactive mode or batch mode or even sub-modes of batch mode. Note that all of the arguments have sensible default values which you only need to override for special cases. In most cases youll only need to execute either qftest [<suite> | <run-log>]* to run QF-Test in interactive mode, or qftest -batch [-runlog [<file>]] [-report <directory>] <suite> to execute a test in batch mode.
Windows

The program qftest.exe is a Windows GUI application. When started from a command shell, it will not wait for QF-Test to terminate but return immediately. Thus, when executing a test in batch mode, you cannot see whether QF-Test has nished or not (you may put the command into a .bat le to deal with this behaviour). Furthermore you wont see any output from QF-Test in the console window when using qftest.exe. For both reasons you may prefer to utilize the qftestc.exe Console application when launching QF-Test from a command shell: It waits for QF-Test to terminate and print (390) will be displayed in the console window. Apart from that, output from Server scripts everything said about qftest.exe in this chapter holds true for qftestc.exe too. Interactive mode The full call syntax for interactive mode is: qftest [-dbg(539)] [-java <executable>(540)] [-noconsole(540)] [-plugindir <directory>(540)] [-J<java-argument>]* [-daemon(542)] [-daemonport <port>(542)] [-dontkillprocesses(542)] [-engine <engine>(543)] [-help(543)]

32.1.

Call syntax

536

[-jythonport <number>(544)] [-keystore <keystore file>(544)] [-keypass <keystore password>(544)] [-libpath <path>(545)] [-license <file>(545)] [-options <file>(545)] [-port <number>(546)] [-reuse(549)] [-runtime(549)] [-serverhost <host>(550)] [-shell <executable>(550)] [-shellarg <argument>(550)] [-systemcfg <file>(551)] [-tempdir <directory>(551)] [-usercfg <file>(553)] [-variable <name>=<value>(553)]* [-version(553)] [<suite> | <run-log>]* There are several sub-modes for running QF-Test in batch mode. The default is to execute one or more test-suites. Alternatively QF-Test can be invoked to create test documentation from test-suites or reports from run-logs or to instrument one or more JDKs or JREs. QF-Test can also be run in daemon mode where it sits in the background waiting (695) for calls from the outside telling it what to do (see chapter 40 for further information about the daemon mode). Finally, showing help or version information can also be seen as separate sub-modes. Test execution To execute one or more test-suites and create a run-log and/or report as a result, use: qftest -batch [-run(549)] [-dbg(539)] [-java <executable>(540)] [-noconsole(540)] [-plugindir <directory>(540)] [-J<java-argument>]* [-clearglobals(542)] [-compact(542)] [-engine <engine>(543)] [-exitcodeignoreexception(543)] [-exitcodeignoreerror(543)] [-exitcodeignorewarning(543)] [-jythonport <number>(544)] [-keystore <keystore file>(544)] [-keypass <keystore password>(544)] [-libpath <path>(545)] [-license <file>(545)] [-nolog(545)] [-nomessagewindow(545)] [-options <file>(545)] [-port <number>(546)] [-report <directory>(547)] [-report.checks(547)] [-report.doctags(547)] [-report.errors(547)] [-report.exceptions(547)] [-report.html <directory>(547)] [-report.ignorenotimplemented(547)] [-report.ignoreskipped(547)] [-report.junit <directory>(547)] [-report.name <name>(548)] [-report.nodeicons(548)] [-report.passhtml(548)] [-report.piechart(548)] [-report.scale-thumbnails <percent>(548)] [-report.teststeps(548)] [-report.thumbnails(548)] [-report.warnings(548)] [-report.xml <directory>(548)] [-runid <id>(549)] [-runlogdir <directory>(549)] [-runlog [<file>](549)] [-runtime(549)] [-serverhost <host>(550)] [-shell <executable>(550)] [-shellarg <argument>(550)]

32.1.

Call syntax [-sourcedir <directory>(550)] [-splitlog(550)] [-systemcfg <file>(551)] [-test <n>|<id>(551)]* [-threads <number>(553)]* [-variable <name>=<value>(553)]* [-verbose [<level>](553)] <suite>+

537

Test execution via QF-Test daemon The following parameters can be specied when executing a test-case by calling a daemon: qftest -batch -calldaemon(542) [-cleanup(542)] [-dbg(539)] [-java <executable>(540)] [-noconsole(540)] [-J<java-argument>]* [-daemonhost <host>(542)] [-daemonport <port>(542)] [-keystore <keystore file>(544)] [-keypass <keystore password>(544)] [-nomessagewindow(545)] [-ping(545)] [-options <file>(545)] [-runid <id>(549)] [-runlogdir <directory>(549)] [-runlog [<file>](549)] [-startclean(550)] [-startsut(551)] [-stopclean(551)] [-stoprun(551)] [-suitedir <dir>(551)] [-terminate(551)] [-timeout <milliseconds>(553)] [-variable <name>=<value>(553)]* [-verbose [<level>](553)] <suite#test-case> Create test documentation Package or test-case documentation can be create for one or more test-suites or (124) whole directories. This is described further in chapter 15 . The command line syntax is: qftest -batch -gendoc(543) [-dbg(539)] [-java <executable>(540)] [-noconsole(540)] [-J<java-argument>]* [-license <file>(545)] [-nomessagewindow(545)] [-options <file>(545)] [-pkgdoc <directory>(545)] [-pkgdoc.dependencies(545)] [-pkgdoc.doctags(546)] [-pkgdoc.html <directory>(546)] [-pkgdoc.includelocal(546)] [-pkgdoc.nodeicons(546)] [-pkgdoc.passhtml(546)] [-pkgdoc.sortpackages(546)] [-pkgdoc.sortprocedures(546)] [-pkgdoc.xml <directory>(546)] [-sourcedir <directory>(550)] [-testdoc <directory>(552)] [-testdoc.doctags(552)] [-testdoc.followcalls(552)] [-testdoc.html <directory>(552)] [-testdoc.nodeicons(552)] [-testdoc.passhtml(552)] [-testdoc.sorttestcases(552)] [-testdoc.sorttestsets(552)] [-testdoc.teststeps(553)] [-testdoc.xml <directory>(553)] (<suite> | <directory>)+ Create a report from run-logs To create a report from one or more run-logs or whole directories use:

32.1.

Call syntax

538

qftest -batch -genreport(543) [-dbg(539)] [-java <executable>(540)] [-noconsole(540)] [-J<java-argument>]* [-license <file>(545)] [-nomessagewindow(545)] [-options <file>(545)] [-report <directory>(547)] [-report.checks(547)] [-report.doctags(547)] [-report.errors(547)] [-report.exceptions(547)] [-report.html <directory>(547)] [-report.ignorenotimplemented(547)] [-report.ignoreskipped(547)] [-report.junit <directory>(547)] [-report.name <name>(548)] [-report.nodeicons(548)] [-report.passhtml(548)] [-report.piechart(548)] [-report.scale-thumbnails <percent>(548)] [-report.teststeps(548)] [-report.thumbnails(548)] [-report.warnings(548)] [-report.xml <directory>(548)] [-runlogdir <directory>(549)] (<run-log> | <directory>)+ Instrument one or more JDKs or JREs This is useful for automating tests that include clean installation of an SUT that comes with its own JRE, so it needs to be instrumented before the test on the SUT can be run. This can be done as follows: qftest -batch -instrument(544) [-dbg(539)] [-java <executable>(540)] [-noconsole(540)] [-J<java-argument>]* [-license <file>(545)] [-nomessagewindow(545)] [-options <file>(545)] <path to JDK or JRE>+ Daemon mode To run QF-Test in daemon mode as described chapter 40 use: qftest -batch -daemon(542) [-dbg(539)] [-java <executable>(540)] [-noconsole(540)] [-plugindir <directory>(540)] [-J<java-argument>]* [-daemonport <port>(542)] [-engine <engine>(543)] [-jythonport <number>(544)] [-keystore <keystore file>(544)] [-keypass <keystore password>(544)] [-libpath <path>(545)] [-license <file>(545)] [-nolog(545)] [-nomessagewindow(545)] [-options <file>(545)] [-port <number>(546)] [-runtime(549)] [-serverhost <host>(550)] [-shell <executable>(550)] [-shellarg <argument>(550)] [-splitlog(550)] [-systemcfg <file>(551)] [-variable <name>=<value>(553)]* Import one test-suite into another qftest -batch -import(544) [-import.from <test-suite>(544)]
(695)

32.2.

Command line arguments [-import.into <test-suite>(544)] [-import.components(544)] [-import.procedures(544)] [-import.tests(544)]

539

Analyze references of a test-suite qftest -batch -analyze(541) [-analyze.target <directory>(541)] [-suitedir <dir>(551)] [-analyze.references(541)] [-analyze.duplicates(541)] [-analyze.invalidchar(541)] [-analyze.emptynodes(541)] [-analyze.components(541)] [-analyze.procedures(541)] [-analyze.dependencies(541)] [-analyze.tests(541)] [-analyze.packages(541)] [-analyze.transitive(542)] [-analyze.followincludes(542)] (<suite> | <directory>)+ Get version information qftest -batch -version(553) Get help qftest -batch -help(543)

32.2

Command line arguments

Command line arguments for QF-Test fall in three categories. They can be mixed freely.

32.2.1

Arguments for the starter script

These arguments are evaluated directly by the qftest shell script or executable and override settings determined during installation. Under Unix these settings are stored in the le launcher.cfg in QF-Tests root directory, under Windows they are stored in the registry. -batch Run QF-Test in batch mode. This causes QF-Test to load and execute a test-suite directly and nish with an exit code that represents the result of the test run. -dbg Turn on debugging output for the starter script. The same effect is achieved by

32.2.

Command line arguments

540

setting the environment variable QFTEST_DEBUG to a non-empty value. On Windows this causes QF-Test to open a console window to display the output that (540) is also given. would otherwise be invisible unless the argument -noconsole This also turns on debugging output for the qfclient and java helper (559) programs when using the old connection mechanism (see chapter 34 ). -java <executable> The Java executable used to run QF-Test. The default is java on Unix and javaw.exe on Windows, unless a different value was set during installation. -noconsole (Windows only) On Windows this argument suppresses the console window that would otherwise (539) be opened in case -dbg is specied. -plugindir <directory> This argument overrides the default location of the directory for plugins, jar les that should be made accessible to scripts. The default directory called plugin is (587) for more located under QF-Tests root directory qftest. See section 37.2 information about plugins.

32.2.2

Arguments for the Java VM

You can pass arguments to the Java VM through the starter script by prepending them with -J, e.g. -J-Duser.language=en to set a system property. To set the classpath, prepend -J only to the -cp or -classpath argument, not to the actual value, e.g. -J-classpath myclasses.jar. When setting the classpath this way, QF-Tests own jar archives need not to be taken into account.

32.2.3

Arguments for QF-Test

The rest of the arguments are handled by QF-Test itself when it is executed by the Java virtual machine. These arguments can also be placed in a le using the syntax <name>=<value> for arguments with parameters or <name>=true or <name>=false to turn a simple argument on or off. By default this le is called qftest.options, located in the bin directory of QF-Test and used only for internal logging purposes. If you change anything in this le, you can move it to QF-Tests root directory so your changes (545) will still apply after upgrading. Alternatively you can use the -options <file>

32.2.

Command line arguments

541

argument to specify a different le. Arguments given on the command line override arguments from an option le except for those which can be given multiple times. In the latter case the arguments are merged. -analyze (batch mode only) Analyze a given test-suite or all test-suite from a given directory. The result is stored in a given le. -analyze.target <directory> (batch mode only) The target folder, where the result le should be created. -analyze.references (batch mode only) Switch for analyzing references the given test-suite. -analyze.duplicates (batch mode only) Switch for analyzing duplicates of the given test-suite. -analyze.invalidchar (batch mode only) Checks the given test-suite for invalid characters in node names. -analyze.emptynodes (batch mode only) Checks the given test-suite for empty nodes. -analyze.components (batch mode only) Switch for analyzing components of the given test-suite. -analyze.dependencies (batch mode only) Switch for analyzing dependency references of the given test-suite. -analyze.procedures (batch mode only) Switch for analyzing procedure calls of the given test-suite. -analyze.tests (batch mode only) Switch for analyzing test calls of the given test-suite.

32.2.

Command line arguments

542

-analyze.packages (batch mode only) Switch for analyzing packages of the given test-suite. -analyze.transitive (batch mode only) Switch for analyzing references and calls transitively, i.e. follow the calls and analyze its content also. -analyze.followincludes (batch mode only) Switch for analyzing all included test-suites of the given test-suite. -calldaemon (batch mode only) Connect to a running QF-Test daemon to execute a test-case. -cleanup (calldaemon mode only) With this argument all TestRunDaemons belonging to a daemon are cleaned up and all clients killed before running the test. -clearglobals (batch mode only) If more than one test-suite is specied for batch execution, clear global variables and resources before the execution of each test-suite. -compact (batch mode only) Create a compact run-log which retains only those branches in which a warning, error or exception occurred. Ignored if log le creation is suppressed with (545) -nolog . -daemon Run QF-Test in daemon mode. Further information is provided in chapter 40 -daemonhost <host> (calldaemon mode only) Specify the host where to locate the QF-Test daemon. The default is localhost. -daemonport <port> Specify the registry port for the QF-Test daemon to listen on and to connect to (546) respectively. The default is 3543 or the port dened with -port <number> .
(695)

32.2.

Command line arguments

543

-dontkillprocesses (batch mode only) When nishing batch execution, dont explicitly kill processes started by QF-Test as part of the tests. However, whether or not a sub-process of QF-Test survives QF-Tests exit is system-dependent. -engine <engine> Specify which engine license(s) to use. This option is only useful in case the QF-Test license contains a mix of GUI engine(s) with different numbers of engine licenses. In that case it may be necessary to specify the engine license(s) to use in order to prevent license conicts with colleagues using the same license. Possible values are all to use all supported licenses, ask to bring up a dialog for engine selection or any combination of awt for AWT/Swing, swt for SWT or web for Web, e.g. awt,web. This is explained in more detail in section (232) 29.1.7 . -exitcodeignoreexception (batch mode only) For exit code calculation exceptions, errors and warnings are ignored. This means if exceptions, errors or warnings occurred during a test-run the exit code will be 0. This option is helpful when integrating QF-Test with build tools that rate a build as failed dependent on the the exit code. -exitcodeignoreerror (batch mode only) For exit code calculation errors and warnings are ignored. This means if just errors and warnings occurred during a test-run the exit code will be 0. This option is helpful when integrating QF-Test with build tools that rate a build as failed dependent on the the exit code. -exitcodeignorewarning (batch mode only) For exit code calculation warnings are ignored. This means if just warnings occurred during a test-run the exit code will be 0. This option is helpful when integrating QF-Test with build tools that rate a build as failed dependent on the the exit code. -gendoc (batch mode only) Tell QF-Test that this batch run serves to create test documentation from test-suites. -genreport (batch mode only)

32.2.

Command line arguments Tell QF-Test that this batch run serves to create a report from run-logs.

544

-help Show help about available command line arguments. -import (batch mode only) Import a given test-suite into another one. This mode can be used to merge two test-suites. -import.from <test-suite> (batch mode only) The source test-suite, which should be imported into another one. -import.into <test-suite> (batch mode only) The target tests-suite for importing. -import.components (batch mode only) Switch for merging components of two given test-suites. -import.procedures (batch mode only) Switch for merging packages and procedures of two given test-suites. -import.tests (batch mode only) Switch for merging test-cases and test-sets of two given test-suites. -instrument (batch mode only) Instrument one or more JDKs or JREs automatically. Instrumentation and running of a subsequent test cannot be combined in a single call. They have to be executed as separate commands. -jythonport <number> Tell the embedded Jython interpreter to listen for TCP connections at the specied port. You can then use telnet to connect to that port and get an interactive Jython command line. -keystore <keystore file> An alternative keystore le to use for securing daemon communication with SSL.

32.2.

Command line arguments

(709)

545

See section 40.3 for details. To disable SSL by specifying no keystore, use this argument in the form -keystore=. -keypass <keystore password> The password for the keystore le used for securing daemon communication with (709) SSL. See section 40.3 for details. -libpath <path> Override the library path option (Directories holding test-suite libraries ). The directories of the library path should be separated by the standard path separator character fot the system, i.e. ; for Windows and : for Unix. QF-Tests include directory will automatically be appended to the path. -license <file> Set the location of the license le (see section 1.3 ). -nolog (batch mode only) Suppress the automatic creation of a run-log. If any of -runlog [<file>] , (547) (547) -report <directory> , -report.html <directory> , (548) (547) -report.xml <directory> or -report.junit <directory> is given, this argument is ignored. This option is reatined for backwards compatibility only. To keep memory use manageable, split run-logs should be (550) used instead (see -splitlog ). -nomessagewindow (batch mode only) In case of a fatal error in batch mode QF-Test prints an error message to the console and for improved visibility also brings up an error dialog for about 30 seconds. That dialog can be suppressed with the help of this argument. -options <file> Override the location of the le used to specify additional command line arguments. This argument can be given more than once to use several sources of command line arguments. -ping (calldaemon mode only) Use this option, if you want to check whether a daemon is up and running. -pkgdoc <directory> (batch mode only)
(549) (4) (231)

32.2.

Command line arguments

546

With this argument QF-Test creates both HTML and XML pkgdoc documentation. If no directory is given, it is created from the basename of the suite. -pkgdoc.dependencies (batch mode only) Whether to list dependencies when creating the pkgdoc documentation. Default is true, use -pkgdoc.dependencies=false to disable. -pkgdoc.doctags (batch mode only) Whether to use the QFS doctag extensions when creating the pkgdoc documentation. Default is true, use -pkgdoc.doctags=false to disable. -pkgdoc.html <directory> (batch mode only) With this argument QF-Test creates HTML pkgdoc documentation. If no directory is given, it is created from the basename of the suite. -pkgdoc.includelocal (batch mode only) Whether to include local packages and procedures (those whose names begin with an _). Default is false. -pkgdoc.nodeicons (batch mode only) Whether to show icons for nodes in the pkgdoc documentation. Default is true, use -pkgdoc.nodeicons=false to disable. -pkgdoc.passhtml (batch mode only) Whether to pass HTML tags in comments through to the HTML pkgdoc. Default is true, use -pkgdoc.passhtml=false to disable. -pkgdoc.sortpackages (batch mode only) Whether to sort packages alphabetically. -pkgdoc.sortpackages=false to disable. -pkgdoc.sortprocedures (batch mode only) Whether to sort procedures alphabetically. -pkgdoc.sortprocedures=false to disable. -pkgdoc.xml <directory> (batch mode only) Default is true, use Default is true, use

32.2.

Command line arguments

547

With this argument QF-Test creates XML pkgdoc documentation. If no directory is given, it is created from the basename of the suite. -port <number> The TCP port on which QF-Test communicates with the SUT. By default QF-Test uses an arbitrary dynamic port where it creates its own RMI registry. A specic port should only be requested if it must be hard-coded when starting the SUT. -report <directory> (batch mode only) Create a combined XML/HTML report. The directory name may contain (554) placeholders as explained in section 32.2.4 . -report.checks (batch mode only) Whether to list checks in the report. Default is false. -report.doctags (batch mode only) Whether to use the QFS doctag extensions when creating the report. Default is true, use -report.doctags=false to disable. -report.errors (batch mode only) Whether to list errors in the report. Default is true, use -report.errors=false to disable. -report.exceptions (batch mode only) Whether to list exceptions in the report. -report.exceptions=false to disable. -report.html <directory> (batch mode only) Create an HTML report. The directory name may contain placeholders as (554) explained in section 32.2.4 . -report.ignorenotimplemented (batch mode only) Whether to ignore nodes that are not implemented in the report in which case the legend and respective columns for not implemented tests are also not shown. Default is false. -report.ignoreskipped (batch mode only) Default is true, use

32.2.

Command line arguments

548

Whether to ignore skipped nodes in the report in which case the legend and respective columns for skipped tests are also not shown. Default is false. -report.junit <directory> (batch mode only) Create a report in JUnit XML format as understood by many Continuous Integration Tools. The directory name may contain placeholders as explained in (554) section 32.2.4 . -report.name <name> (batch mode only) Specify the name for the report, meaning its identier, not a le name. Default is (554) the runid. The name may contain placeholders as explained in section 32.2.4 . -report.nodeicons (batch mode only) Whether to show icons for nodes in the report. -report.nodeicons=false to disable. -report.passhtml (batch mode only) Whether to pass HTML tags in comments through to the HTML report. Default is true, -report.passhtml=false to disable. -report.piechart (batch mode only) Whether to create a pie chart in the top part of the HTML report. Default is true, -report.piechart=false to disable. -report.scale-thumbnails <percent> (batch mode only) How to scale thumbnail images for screenshots in the error listings of the report, an integer value interpreted as percent of the original image size. Default is 20. -report.teststeps (batch mode only) Whether to list test-steps in the report. -report.teststeps=false to disable. -report.thumbnails (batch mode only) Whether to display thumbnail images for screenshots in the error listings of the report. Default is false. Default is true, use Default is true, use

32.2.

Command line arguments

549

-report.warnings (batch mode only) Whether to show warning information in the report. Default is true. -report.xml <directory> (batch mode only) Create an XML report. The directory name may contain placeholders as (554) explained in section 32.2.4 . -reuse (interactive mode only) This argument is used mainly when launching QF-Test from a desktop icon or the Windows explorer through a le association. It tells the newly started QF-Test instance to search for an already running version of QF-Test and ask that to open the given le(s). If another instance can be reused in that way, the newly started program will terminate immediately and new windows for the le(s) will be opened by the old instance. -run (batch mode only) Explicitely tell QF-Test that this batch run is for actual test execution as opposed to generating documentation or a report. As this is the default operation for batch mode this argument can be omitted. -runid <id> (batch and calldaemon mode) Specify an ID for the test-run. The ID may contain placeholders as explained in (554) section 32.2.4 and will itself serve as a replacement for the placeholder %i/+i. -runlog [<file>] (batch and calldaemon mode) Save the run-log in the given le. The optional lename may contain placeholders (554) as explained in section 32.2.4 . If no lename is given, it is composed of the basename of the suite and a timestamp. If missing, the extension .qrz is added automatically and the run-log is saved compressed. Otherwise the extension .qrl or .qrz determines compression. Even without this argument a run-log is (545) or when a report is generated. The created unless suppressed with -nolog default value is %p%b.qrz. In calldaemon mode, a run-log will be stored only if a (local) lename is specied. -runlogdir <directory> (batch mode only) This argument serves as the target directory for saving run-logs unless the (549) lename of the run-logs specied with -runlog [<file>] is an absolute path. If this argument is given and a report is generated, the les in the report will

32.2.

Command line arguments

550

be laid out according to the structure of the run-logs relative to this directory. The (554) directory name may contain placeholders as explained in section 32.2.4 . -runtime Use a runtime license only. In batch mode, QF-Test will normally use a runtime (553) license (or multiple runtime licenses if -threads <number> is given). If not enough free runtime licenses are available, full development licenses will be used instead unless -runtime is given in which case no development license is used and QF-Test will fail with an error message. In interactive mode, if -runtime is given, QF-Test will use a runtime license instead of a full development license. In that mode, any test-suite can be loaded and tests can be run interactively as usual, including debugging support. Saving of test-suites will be disabled, however, though test-suites can be modied for temporary experiments. -shell <executable> The shell to use when executing a Execute shell command is /bin/sh, for Windows COMMAND.COM or cmd.exe. -shellarg <argument> The argument that causes the shell specied with -shell <executable> to execute the following argument and then exit. For Unix shells this is typically -c, COMMAND.COM and cmd.exe expect /c. If you have Unix tools installed on Windows and specify sh or bash as the shell to use, dont forget to change -shellarg <argument> to -c. -serverhost <host> Set the hostname for the machine QF-Test is running on. This is passed to the SUT and usually determined automatically. You may need to set it when running QF-Test and the SUT on different machines and you experience troubles with reverse name lookup. -sourcedir <directory> (batch mode only) If this argument is given and a report is generated, the les in the report will be laid out according to the structure of the test-suites relative to this directory (549) is also specied. In any case, the unless -runlogdir <directory> directory of a test-suite listed in the report will only be listed if this argument is specied and the test-suite is located below this directory. -splitlog (batch mode only)
(550) (409)

node. Default for Unix

32.2.

Command line arguments

(67)

551

In batch mode split run-logs (see section 8.1.4 ) are enabled by default and can be turned off via -splitlog=false. If -splitlog is explicitely specied without parameter, the default extension for run-logs is changed from .qrz to .qzp so as to create split run-logs in ZIP format. The same can be achieved by specifying the name of the run-log explicitely with the desired extension. -startclean (calldaemon mode only) With this argument all contexts of the shared TestRunDaemon are cleaned up and released before running the test. -startsut (only for internal use) This argument is used to start a client application on a remote host. You should not work with this argument directly. The standard library qfs.qft contains a procedure qfs.daemon.startRemoteSUT which can be use for this purpose. -stopclean (calldaemon mode only) With this argument all contexts of the shared TestRunDaemon are cleaned up and released after running the test. -stoprun (calldaemon mode only) Stop a running test executed by the Daemon on the given host and port. This (542) (551) argument can be combined with -cleanup or -stopclean . -suitedir <dir> (calldaemon mode only) Specify the (remote) directory where the QF-Test Daemon looks for test-suites. Use an absolute path when specifying the test to execute, if this argument is not given. -systemcfg <file> Set the location of the system conguration le (see section 1.4 ). -tempdir <directory> (interactive mode only) Can be used to specify a temporary directory which is needed on Windows only for the context sensitive help system. By default the values of the environment variables TEMP and TMP are tried. -terminate (calldaemon mode only) Use this option, if you want to terminate a running QF-Test daemon.
(5)

32.2.

Command line arguments

552

-test <n>|<id> (batch mode only) Without this argument all the top-level tests of the suite are executed one after the other. Using -test <n>|<id>, you can select specic tests. An arbitrary (306) (288) node located anywhere in a test-suite can be accessed by its Id . Test-case or (294) Test-set nodes can be referenced also by its qualied name. Tests on the top level of the suite can also be selected by their index, the index for the rst Test being 0. You can use this argument multiple times, even for the same test. -testdoc <directory> (batch mode only) With this argument QF-Test creates both HTML and XML testdoc documentation. If no directory is given, it is created from the basename of the suite. -testdoc.doctags (batch mode only) Whether to use the QFS doctag extensions when creating the testdoc documentation. Default is true, use -testdoc.doctags=false to disable. -testdoc.followcalls (batch mode only) nodes during testdoc creation. With this By default QF-Test ignores Test call argument the target Test-case, Test-set or whole test-suite are processed as if they were part of the original test-suite. Thus it is possible to create a subset testdoc documentation by creating a dedicated test-suite with Test calls to the required parts. -testdoc.html <directory> (batch mode only) With this argument QF-Test creates HTML testdoc documentation. If no directory is given, it is created from the basename of the suite. -testdoc.nodeicons (batch mode only) Whether to show icons for nodes in the testdoc documentation. Default is true, use -testdoc.nodeicons=false to disable. -testdoc.passhtml (batch mode only) Whether to pass HTML tags in comments through to the HTML testdoc. Default is true, use -testdoc.passhtml=false to disable. -testdoc.sorttestsets (batch mode only)
(299)

32.2.

Command line arguments Whether to sort test-sets alphabetically. -testdoc.sorttestsets=false to disable. Default is true,

553 use

-testdoc.sorttestcases (batch mode only) Whether to sort test-cases alphabetically. -testdoc.sorttestcases=false to disable. -testdoc.teststeps (batch mode only) Whether to list test-steps in the testdoc documentation. -testdoc.teststeps=false to disable. -testdoc.xml <directory> (batch mode only) With this argument QF-Test creates XML testdoc documentation. If no directory is given, it is created from the basename of the suite. -threads <number> (batch mode only) Run the same test-suite in a number of parallel threads. Typically used for the purpose of load testing. One license is required per thread, so normally (549) (176) -runtime should be specied as well. See chapter 21 for more information about load tests. -timeout <milliseconds> (calldaemon mode only) Give a timeout value in milliseconds for the test run when executing a test through the QF-Test daemon. Default is innite. -usercfg <file> (interactive mode only) Set the location of the user conguration le (see section 1.4 ). -variable <name>=<value> To override a system or suite variable denition (see chapter 7 ) use this argument to set the variable named <name> to the value <value>. Using this argument multiple times you can dene more than one variable. -verbose [<level>] Print progress and status information during a test-run to the console. This is in particular useful when driving a test via -calldaemon, because the actual test usually runs on a different host where you may not be able to observe it. Note
(52) (5)

Default

is

true,

use

Default is true, use

32.2.

Command line arguments

554

that on Windows you need to use qftestc.exe (instead of qftest.exe) to see the output. Specifying a level of verbosity is optional, possible values are all (print all nodes) and tests (default, only Test-set, Test-case and Test nodes are printed out). Each level can be combined with errors (print error and exception messages) like tests,errors. -version Print version information and exit.

32.2.4

Placeholders in the lename parameter for run-log and report

(549)

The lename given in any of the command line arguments -runid <id> , (549) (549) -runlog [<file>] , -runlogdir <directory> , (547) (547) -report <directory> , -report.html <directory> , (548) (548) -report.name <name> , -report.xml <directory> or (547) -report.junit <directory> , may contain placeholders of the form %X or +X (the latter must be used on Windows where % is a special character) where X may be any of the characters listed in the table below. QF-Test will ll in the respective value when creating the run-log or report. All time values refer to the time the test was started.
Note

When executing multiple test-suites, be sure to include the base name of the suite in the lename by specifying %b. Otherwise only a single run-log or report may be written that represents only the test-run of the last test-suite.

32.3.

Exit codes for QF-Test

Replacement

555

Character % + i p

P b r w e x y Y M d h m s

Literal % character. Literal + character. (549) The current runid as specied with -runid <id> . (550) The directory of the test-suite relative to -sourcedir <directory> . Expands to the absolute directory in case -sourcedir <directory> is unspecied and is empty if -sourcedir <directory> is specied but the test-suite is not located below it. The absolute directory of the test-suite. May only be given at the beginning. The basename of the test-suite, exclusive directory and .qft extension. The return value (or exit code) of the test-run (-runlog only). The number of warnings in the test-run (-runlog only). The number of errors in the test-run (-runlog only). The number of exceptions in the test-run (-runlog only). The current year (2 digits). The current year (4 digits). The current month (2 digits). The current day (2 digits). The current hour (2 digits). The current minute (2 digits). The current second (2 digits).

Table 32.1: Placeholders in lename parameters

So if, for example, you want to save the run-log in a subdirectory of your test-suite directory called logs and want to include a timestamp and the exit code, use -runlog %p/logs/%b-%y%M%d-%h%m%s-%r.qrl
Note

It is possible to use %b, %p and %P for collective parameters like runid or report. This makes sense only when processing a single test-suite. When processing multiple testsuites, the name of the rst test-suite is used.

32.3

Exit codes for QF-Test

When run in interactive mode, the exit code of QF-Test is not very useful. It is either negative if QF-Test fails to start or 0. In batch mode however the exit code expresses the result of the test run. Negative values represent fatal errors that prevent the test from being executed, while positive values stand for errors during the test run. Note that many systems only support exit codes between 0 and 255, so every exit code may have to be calculated modulo 256, i.e. -1=255, -2=254 and so on.

32.3.

Exit codes for QF-Test

556

The following exit codes are currently dened:

Value 0 1 2 3 -1 -2 -3 -4 -5 -6 Meaning Everything OK Warnings occurred during the test run Errors occurred during the test run Exceptions occurred during the test run Unexpected Exception Bad command line arguments Missing or invalid license Errors while setting up the RMI connection Errors while loading the test-suite The test-suite doesnt contain any tests

Table 32.2: Exit codes for QF-Test

Besides, there are special exit codes when running QF-Test with the -calldaemon argument:
Value -7 -8 -9 -10 -11 Meaning Daemon could not be found Failed to get or create a TestRunDaemon Failed to get or create a run context The test could not be started The test has not ended within the given timeout

Table 32.3: calldaemon exit codes for QF-Test

Chapter 33 GUI engines

SWT and Swing can be combined together in a single application, both by creating different kinds of toplevel windows for SWT and Swing, but also by embedding components of one kind into the other. QF-Test supports testing such kinds of applications. To that end, the new concept of a GUI engine was introduced. One GUI engine is responsible for handling recording and replay for one GUI toolkit thread. Normal applications have only one such thread. As explained above, combinations of Swing and SWT are possible that have one thread each and will thus require two GUI engines to operate in parallel. In theory it is also possible to have multiple GUI engines of the same kind, e.g. by creating multiple instances of the SWT Display class. Each QF-Test GUI engine is identied by a token for the GUI toolkit and a number. awt0 is the primary GUI engine for AWT/Swing and swt0 the primary engine for SWT. Unless you have a very special application you will never need to concern yourself with the number of the engine, as there will never be an engine called awt1 or swt1 and the alias awt or swt is sufcient. When recording, QF-Test always uses the latter. If your application is either AWT/Swing only or SWT only, engine names can be left empty. In a test-suite, engine identiers are now stored in the following places: nodes. Only required if your application combines Wait for client to connect AWT/Swing and SWT. By specifying the engine attribute you can wait for the respective GUI engine to become initialized. Window nodes. The engine of a Window node marks the window and all its child nodes as being either AWT/Swing or SWT. Embedded components of the other kind will be moved to a node for a pseudo window. SUT script nodes. An SUT script is executed on the event dispatch thread of the SUT, so for combined AWT/Swing and SWT applications the engine is required to specify whether the script should be run on the AWT/Swing thread or the SWT
(392) (507) (411)

GUI engines

558

thread. Thus, an SUT script node can only retrieve and interact with components of one kind. File selection nodes. For Swing applications, the File selection node is rarely used because the Swing JFileChooser is implemented in Java and can be fully controlled by QF-Test. The SWT FileDialog on the other hand is similar to the AWT FileChooser. Both are implemented natively and QF-Test has no control over the individual controls. Thus le selection must be replayed using the File selection node. Because this node is not explicitely associated with a Component or Window node, the engine has to be specied within the node.
(439)

Chapter 34 Running an application from QF-Test

Note

The Quickstart Wizard is the recommended tool to set up your SUT for testing. It results in an advanced setup sequence already prepared for later requirements. This chapter contains some details in case you want to create a setup sequence by hand.

(23)

34.1

Various methods to start the SUT

With the Quickstart Wizard QF-Test offers a uitility to guide you throught the steps of (22) creating a start sequence for your SUT. Please refer to chapter 3 for more information about the Quickstart Wizard. Nevertheless we also want to describe how to create a start sequence for your application manually. There are basically two ways to start a Java application as an SUT from QF-Test. The rst one is based on the standard java ... command line with its two variants for either starting a class or a jar le. The alternative is running a script or executable le which will then start the Java program. Indirect methods like launching the SUT through ant also fall into this category, as do Java WebStart and applets run in the Java plugin of a web browser. The following examples show some typical setups. To get more detailed information about the required attributes, please follow the respective links to the reference manual. The tutorial also includes a number of examples. Independent of how the SUT is started, the respective node should typically be fol(411) lowed immediately by a Wait for client to connect node with an identical Client attribute. Again, see the reference manual for further details.

34.1.

Various methods to start the SUT

560

34.1.1

A standalone script or executable le

If your application is started through a script or a binary executable, create a (399) Start SUT client as follows:

Figure 34.1: Starting the SUT from a script or executable

Create a Start SUT client

(399)

node.
(400)

Assign a name to the client in the Client

(400)

attribute.

Set the Executable to the script or executable that starts your application. If the program is not located in a directory on the PATH, the full path is required. Set the Directory
(401)

attribute to the working directory for your application.

One thing to watch out for in scripts is redirection of the standard output and error streams (e.g. >myapp.log) which you may want to remove so that the output

34.1.

Various methods to start the SUT

561

of the SUT reaches QF-Test and is captured in the run-log. Similarly, the start command in Windows batch les causes the SUT to detach and keeps the output away from QF-Test.

34.1.2

An application launched through Java WebStart

Using the new connection mechanism, an application launched through Java WebStart can be started directly from QF-Test without the need to modify any JNLP les (so do Instead create a not use ExtrasCreate WebStart SUT client starter... ). (399) Start SUT client node as follows:

Figure 34.2: Starting the SUT through Java WebStart

Create a Start SUT client

(399)

node.
(400)

Assign a name to the client in the Client

attribute.

34.1.

Various methods to start the SUT

(400)

562

Set the Executable attribute to the Java WebStart executable which is typically called javaws and located somewhere inside the JDK or JRE. Youll probably have to specify the full path. For Java WebStart the Directory attribute typically is of no consequence except that Java WebStart is looking in that directory for a le named .javaws which can contain settings like debug levels. Create an entry in the Parameters applications JNLP descriptor.
(401) (401)

for the executable to specify the URL for the

34.1.3

An applet in a web browser

Applet testing can be done directly in a browser.

Note

In case you own a license that allows web testing with QF-Test and the supported browsers sufce you can also use the normal web testing feature (Quickstart your (22) application ).

34.1.

Various methods to start the SUT

563

Figure 34.3: Starting the SUT as an applet in a web browser

Create a Start SUT client

(399)

node.
(400)

Assign a name to the client in the Client

(400)

attribute.

Set the Executable attribute to the browser that you want to start, i.e. firefox, iexplore, chrome, safari to name just a few. You may have to specify the full path. For starting of an applet, the Directory Create an entry in the Parameters applets web page.
Note
(401) (401)

attribute is of no consequence.

for the executable to specify the URL for the

Important notes on applet testing Applet testing will only work in a reliable way when

34.1.

Various methods to start the SUT

564

a new browser process is started, no dialogs (like questions on session restore or default browser setting) will block the browser start, on Windows 8 the browser is started on the desktop, not as a Modern UI (former Metro) application. The procedure qfs.web.browser.general.startAsNewProcess that is also (23) used when creating the start sequence with the Quickstart Wizard should solve the rst two points. It uses browser specic parameters to ensure a new process when possible and avoids blocking dialogs e.g. by using new empty proles. Please see the procedure documentation for further details.
Note

Browser or the Java plugin itself might warn when detecting a new or an outdated JRE. In case you cannot or dont want to always test with the latest plugin, you might want to avoid those blocking dialogs. The JRE should provide respective option in its settings. For Google chrome you can use the additional command line parameters -always-authorize-plugins and -allow-outdated-plugins. For Firefox some hints are given at http://mzl.la/MN0qiT. Concerning Windows 8 there are following means to ensure Internet Explorer is started as desktop application: Either set another browser as default browser. This should force IE to desktop mode. or within the IE settings set the option Internet optionsProgramsChoose how to open links to Always in Internet Explorer on the Desktop or directly change set the registry key HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\ Main\AssociationActivationMode to value 2. 0 = Let Internet Explorer decide 1 = Always in Internet Explorer (this is Metro IE) 2 = Always in Internet Explorer on the desktop

Windows

Further background information

34.1.

Various methods to start the SUT

565

As described above, the qfs.web.browser.general.startAsNewProcess helps to ensure a new process. Hence some hints should be given on some of the techniques used. You can have a look the procedures code as well, of course.
Windows

There are different command line parameters for different Internet Explorer versions to force a new process is being started: IE6: -new IE7: -new (default) IE8 and later: -noframemerging (See also http://msdn.microsoft.com/en-us/library/hh826025%28v=vs.85%29.aspx). There is also the option to ensure this behavior be respective settings in the registry but using the commandline parameters is the less invasive way.

Note

Firefox has a built-in session restore feature that causes conicts with QF-Test. When QF-Test terminates a Java applet and thus the browser, Firefox treats this as a crash and offers to restore the session on the next start through a dialog that cannot be controlled by QF-Test so that the test cannot continue unobserved. To work around this problem, either always start with an empty new prole or create a separate user prole as described at http://kb.mozillazine.org/Prole_Manager and always start Firefox from QF-Test with this prole using the command firefox -P <profile>. Then disable session restore for this user by editing the preferences as described at http://kb.mozillazine.org/Editing_conguration. You need to add an option called browser.sessionstore.resume_from_crash and set it to false.

34.1.4

An application started with java -jar <archive>

If your application is normally launched through a command of the form java -jar (395) <archive>, create a Start Java SUT client node as follows:

34.1.

Various methods to start the SUT

566

Figure 34.4: Starting the SUT from a jar archive

Create a Start Java SUT client

(395)

node.
(396)

Assign a name to the client in the Client

attribute.
(397)

attribute. Its default If necessary, change the Executable ${qftest:java} is the java executable that QF-Test was started with. Set the Directory
(397)

value

attribute to the working directory for your application.

(397)

Create two entries in the Parameters table for the executable. Set the rst to -jar and the second to the name of the archive. Unless the archive is located in (397) the Directory selected above, its full path is required.

34.1.

Various methods to start the SUT

567

34.1.5

An application started <classpath> <class>

with

java -classpath

If your application is normally launched through a command of the form java (395) -classpath <classpath> <class>, create a Start Java SUT client node as follows:

Figure 34.5: Starting the SUT via the main class

Create a Start Java SUT client

(395)

node.
(396)

Assign a name to the client in the Client

attribute.

34.1.

Various methods to start the SUT

(397)

568 value

If necessary, change the Executable attribute. Its default ${qftest:java} is the java executable that QF-Test was started with. Set the Directory
(397)

attribute to the working directory for your application.

Set the Class name attribute to the fully qualied name of the applications starter class (the class with the main() method), just like for java. table for the executable. Set the rst to Create two entries in the Parameters -classpath and the second to the list of jar les and directories that constitute the classpath. The full path is required for jar archives not located in the (397) Directory selected above. This argument can get very long and hard to edit (11) directly in the table. See section 2.2.5 about how to pop up a dialog for more convenient editing.
(397)

(397)

34.1.6
Web Note

A web application in a browser

Currently web testing with QF-Test requires a 32bit Browser. Since there is little need for more than 4GB of memory in a Browser, 64bit Browsers are still the exception but on 64bit Linux systems the default browser may be 64bit in which case you need to install an additional 32bit Firefox. Also, the default JDK/JRE on 64bit Systems is often a 64bit one. In that case you must install an additional 32bit JDK or JRE and run QF-Test with that. To do so, either make the 32bit JDK the default when installing QF-Test or via the Java conguration dialog, or run QF-Test from the command line (540) with -java <executable> . If you have a 32bit browser and a 32bit JDK and the browser still wont start from QF-Test, please check if the JDK is an original from Sun or an openJDK. Like Swing or SWT clients, a web-based SUT - i.e. a browser - is started as a separate process from within QF-Test. In order to gain access to the internals of the browser and the web page shown with its Document Object Model (DOM), QF-Test embeds a standard browser like Internet Explorer or Mozilla in a special wrapper application. The technology for embedding and accessing those standard browsers enables efcient access to the DOM beyond the browsers standard interfaces and a unied interface that hides browser differences and enables QF-Test - and thus you - to focus on test automation with a single set of tests for all supported browsers on multiple platforms.

34.1.

Various methods to start the SUT

569

Figure 34.6: Starting a web-based SUT in a browser

A Start browser node can be used to launch a browser. One SUT web client process can display multiple browser windows, which can even be a mix of Internet Explorer and Mozilla windows. Additional browser windows in an already

(402)

34.1.

Various methods to start the SUT

570

running process can be opened via a Start browser node with the attribute (404) Open new browser window in existing client set or as the result of clicking a link in a web page that opens a popup window.
Note Note

QF-Test works with 32bit Firefox from version 3.6. On a 64bit machine you must install a 32bit variant of the browser you want to test with. When setting up the startup sequence with the Quickstart Wizard or dening your (404) own Directory of Mozilla installation attribute, try pointing QF-Test to a current Firefox installation. If you get a message saying that the Gecko Runtime Environment (GRE) cannot be determined, please make sure that you have got a 32bit browser installed. On Linux, the standard browser for your distribution may be installed in various places.
(23)

Chapter 35 JDK and SWT instrumentation

35.1
Swing Note

JDK instrumentation

QF-Test connects to AWT/Swing applications automatically unless you explicitly turn off (253) the option Connect without JDK instrumentation (Swing) or use a non-standard JDK. In such a special case you must instrument the JDK rst. To instrument a JDK you need write permission for some of its directories, so you may have to run QF-Test with administrator privileges to perform this step. The accessibility interface and all the technical details of the modications to the JDK are (571) described in detail in chapter 35 . This section concentrates on how to instrument or de-instrument JDKs and how QF-Test maintains information about which JDKs have been instrumented. Having started QF-Test, please select the menu ExtrasManage JDK instrumentation... to bring up the following dialog: item

35.1.

JDK instrumentation

572

Figure 35.1: Dialog for JDK instrumentation

Note

If you get a warning that the le used for storing information about JDK instrumentation is currently locked, this typically means that some other user working with the same QF-Test installation is currently instrumenting JDKs. In that case, try again a little bit later. If the lock doesnt go away even though nobody seems to be working on the instrumentation, you can ignore it. The instrumentation dialog shows a list of JDKs and JREs which QF-Test knows about. Initially the list will only contain the JDK with which QF-Test itself is running. If your SUT will be started with a different JDK or JRE you need to tell QF-Test about it. To do so, use the Search JDK/JRE button to bring up the standard le selection dialog and choose a directory. QF-Test will search that directory and its direct or indirect sub-directories and add all JDKs it nds to the list. If you dont know which JDK your SUT is going to run with, rst instrument the one JDK will thats always shown, then try if it works. If it doesnt work, the record button remain deactivated when you start the SUT from QF-Test. In that case you can either ask your developers or search for the JDK or JRE yourself. On Windows, C:\Program Files\Java is the most likely candidate. On Unix systems locations vary. Possible candidates are /usr/lib/java, /usr/share/java, /usr/local/java, etc. It is also possible that your application comes with its own JRE so letting QF-Test search the directory where the SUT is installed may also turn up something. The Type displayed in the rst column of the list is usually JDK or JRE, denotes an invalid path. This type information is only given for completeness and you dont really need to concern yourself with it.

35.1.

JDK instrumentation

573

The value in the State column is important. The values in this column can be any of the following: Not instrumented The JDK is in its initial, unmodied state. Instrumentation current The JDK has been instrumented and the installed jar le corresponds the current QF-Test version. Instrumentation outdated The JDK has been instrumented but the current QF-Test version has a newer version of the installed jar le, so the JDK should be instrumented again. Instrumentation inconsistent The JDK has been partly instrumented, either because something has gone wrong during (de)instrumentation or because somebody (or some other tool) has modied the JDK. You should instrument or de-instrument such a JDK as required. Path invalid Either the JDK no longer exists, in which case you can remove it safely from the list, or it belongs to some other machine as explained below, in which case you should ignore it. To instrument, de-instrument or remove one or more JDKs from the list, select the entry or entries in the list and press the respective button. When done, save the list and close the dialog. The list of JDKs QF-Test knows about is stored in the le qfconnect.properties in QF-Tests root directory. If QF-Test is installed on a network le system which is shared by multiple computers, possibly with different operating systems, this single le will be shared by all systems. As you can see in the image above, this is not a problem. When you bring up the dialog, QF-Test will check all the saved JDKs to see whether they exist on the current system. If not, they will be marked with the state Path invalid and can be safely ignored. In short, you can run the instrumentation rst from a Windows system and instrument your Windows JDKs, then from a Unix system to instrument those JDKs and that information can safely coexist.

35.2.

JDK instrumentation background

574

In case you dont have write permissions for this le, QF-Test will not be able to save the list of instrumented JDKs, but that shouldnt cause any trouble with the instrumentation itself.

35.2

JDK instrumentation background

Introduced in QF-Test version 1.07, instrumentation of the JDK or JRE on which the SUT will run is now the primary method of providing a way for QF-Test to interact with the SUT. This chapter explains the technical details of whats going on behind the scenes when the JDK is instrumented and when the SUT is started in an instrumented JDK.
3.1.4+

Normally you dont need to instrument standard JDKs because QF-Test passes the same information to the SUTs JDK via environment variables. However, for special cases instrumentation may be required anyway and the information in this chapter is still valid.

35.2.1

Modications to the JDK

When Instrumenting the JDK, QF-Test makes use of the ofcial accessibility interface which is provided by Java for just this purpose. It can be used by accessibility and capture replay tools to interact with Java applications without those applications knowing about it and without requiring any changes to those applications. To activate this interface, QF-Test creates or modies the le .../lib/accessibility.properties in the JDK installation and adds the class de.qfs.apps.qftest.start.Connector to the property assistive_technologies. This has the effect that this class will be instantiated whenever the AWT toolkit is initialized in any Java application or applet that is run in this Java VM. To make sure that this class can always be found without modications to the CLASSPATH, the le qfconnect.jar, which contains the Connector class, is placed in the Java extensions directory .../lib/ext. When de-instrumenting the JDK, QF-Test rst removes its entry from the assistive_technologies property in accessibility.properties so the Connector class will no longer be used. Next QF-Test tries to remove qfconnect.jar from the .../lib/ext directory. On Windows systems this will not work if QF-Test is running inside the same Java VM, so the le should be removed by hand. However, this is not essential because it wont affect the JDK in any way after accessibility.properties has been updated.

35.3.

SWT instrumentation

575

35.2.2

Effects of the instrumentation

When the class de.qfs.apps.qftest.start.Connector is instantiated during AWT toolkit initialization of a Java application, the rst thing it does is check whether the application was launched by QF-Test, i.e. whether it is a System Under Test or a plain application. In the latter case, the connector terminates immediately and doesnt affect the application in any way. In case of an SUT, the connector sets up a special class loader that will load the classes required for QF-Tests SUT wrapper, then set up and initialize that wrapper which will nally connect to QF-Test over RMI, just as the old wrapper used to. This technique differs from the old technique of modifying the SUTs command line in a number of subtle ways: The SUT process is launched directly by QF-Test without interim qfclient and java envelopes. The classes of the QF-Test wrapper around the SUT are all loaded through the special class loader, so they are separated completely from the SUTs classes. So even if the SUT uses some software that QF-Test also uses and the required versions differ, this shouldnt cause any problems. The class loader automatically ensures that the QF-Test wrapper has all the necessary permissions. No explicit permission grants or changes to the policy le are required. This technique makes it possible to test applications launched directly from an executable le. Java WebStart based applications can be tested without the need to modify jnlp les and even applets that run in the Java plugin inside a web browser.

35.3
SWT

SWT instrumentation

Some special setup is required for testing SWT based applications with QF-Test/swt. Because SWT was not written with testability in mind, applications need to be run with a slightly modied SWT version to enable testing. In this version we have added the necessary hooks for event ltering and component tracking. The changes are transparent so that the behavior of an application is not changed, regardless of whether it is run inside or outside of QF-Test. If you use QF-Tests Quickstart Wizard to create the start sequence for your SUT (see (22) chapter 3 ), it will take care of SWT instrumentation as well. For those with an aversion to wizard dialogs, the manual way is described next.

35.3.

SWT instrumentation

576

The standard library qfs.qft, which is part of the QF-Test distribution and described in (352) detail in the tutorial, contains a Procedure with which to perform the SWT instrumen(359) tation. It is named setup and located in the Package qfs.swt.instrument. Insert (355) a Procedure call node before the start node for your SUT in your setup sequence. (356) Set its Procedure name attribute to qfs.qft#qfs.swt.instrument.setup and in (357) the Variable denitions set the parameter sutdir to the installation directory of your application. The plugin parameter can be left empty except when you are testing an Eclipse/RCP application that does not follow the standard plugin directory layout. In that case you can specify the plugin le to instrument directly via the plugin parameter. Thats all. In case you want to know what goes on behind the scenes, all manual steps are described further on in this section.

35.3.1

Preparation for manual SWT instrumentation

Supported architectures for SWT testing are 32 and 64 bit Windows and 32 and 64 bit Linux with Gtk. The instrumentation les are provided in directories called .../qftest-3.5.3/swt/$ARCH/$VERSION where $ARCH is either win32, win32-64, linux-gtk or linux-gtk-64 and $VERSION is one of the supported SWT versions. First you need to determine whether your application is a standalone SWT application or is based on eclipse. To do so, simply take a look at the directory structure of your application. If you nd a directory called plugins containing a le called org.eclipse.swt.win32.win32.x86_X.Y.Z.jar (on Windows) or org.eclipse.swt.gtk.linux.x86_X.Y.Z.jar (on Linux), with X.Y.Z representing a version number like 3.2.0, your application is based on eclipse. For a standalone SWT application you should nd a le called swt.jar, typically inside a directory called lib.

35.3.2

Manual SWT instrumentation for eclipse based applications

Simply replace the SWT plugin jar with one instrumented by QF-Test. To create the instrumented plugin you must run the Procedure qfs.qft#qfs.swt.instrument.setup described above once with your original plugin (or a copy thereof) specied in the plugin parameter. QF-Test will create a backup copy of the original jar named _org.eclipse.swt....jar.orig. Next copy the instrumented plugin to the plugin directory of your application. Finally, start your application once from the command line with the -clean command line argument to have it rebuild its plugin cache, e.g. eclipse -clean

35.3.

SWT instrumentation

577

Your applications binary name may be different from eclipse, but all eclipse based applications should support the -clean argument.

35.3.3

Manual instrumentation for standalone SWT applications

For standalone SWT applications, replace the swt.jar le with the one provided with QF-Test. You may want to create a backup of the original rst.
Note

If you are launching the client application by means of a Start Java SUT client node, you can set the classpath to point to the corresponding .../qftest-3.5.3/swt/$ARCH/$VERSION/swt.jar archive and leave your original le untouched.

(395)

Chapter 36 Technical details about miscellaneous issues

36.1 Drag&Drop

From the beginning Drag&Drop has been hard to implement in Java. JDK 1.1 had no Drag&Drop support at all and the rst steps taken with JDK 1.2 were far from satisfying. In JDK 1.3 Drag&Drop has matured, but there are still problems with stability. That Drag&Drop is so hard to get right is partly due to the fundamental difference in the way Drag&Drop is implemented by the underlying systems. Another reason is the goal to support Drag&Drop between Java and the underlying system and not just between Java applications. As a result Drag&Drop for Java is implemented native, meaning at the level of the operating system, so the events involved can be neither interpreted, nor generated by QF-Test. Hence Drag&Drop cannot be replayed directly. Even so QF-Test has support for Drag&Drop, at least for JDK 1.3 and above. Direct recording of Drag&Drop is supported with JDK 1.4 and above. Replay of Drag&Drop (420) DRAG_FROM and DROP_TO as well as is achieved through the special Mouse events the optional DRAG_OVER. To replay these events, QF-Test makes use of the AWT Robot that was added to Java with version 1.3. The AWT Robot makes it possible to generate hard events at system level. These hard events actually move the mouse cursor across the screen and can trigger a Drag&Drop operation. Using Drag&Drop together with To the Shift and or Control key is also supported. (422) simulate a Drag&Drop with the Control key pressed, change the Modiers attributes of the DRAG_FROM, DRAG_OVER and DROP_TO events to include the Control modier by adding 2 to the current value. It is even possible to simulate the Shift or Control key being pressed or released during the drag operation by changing the Modiers of only some of the events.

36.2.

Component recognition

579

As mentioned above, Drag&Drop used to be a bit unstable on some systems. In some cases with older JDKs the use of the AWT Robot to simulate Drag&Drop could crash the SUT or even the whole system. Nowadays the situation is much better. The introduction of mouse movement interpolation helped improve the reliability of Drag&Drop replay a (263) lot. Please see section 29.3.4 for details.
Note

On Windows simulating Drag&Drop may conict with some mouse drivers. In case of problems reduce the speed of the mouse cursor to 50% in the mouse settings of the control panel.

36.2

Component recognition

The process of identifying components in the SUT during a test run is rather complicated. The problem is that QF-Test has to be extremely exible in order to accommodate a wide range of applications and to allow for a certain amount of change in the GUI of the SUT, due to dynamic behavior or changes between different versions of the SUT, while still being able to nd the component intended by the developer of the test-suite. This process can be simplied - and its reliability improved considerably - by dening unique names for most of the interesting components of the SUT. This has to be done by the developer of the application with the Java method setName(String). If you can get your developers to set unique names for your components or windows, you wont have any problems with component recognition. Even drastic changes in the GUI can be handled if names are being used. Another advantage is that the name is language-independent, so you can use one and the same test-suite to test applications that support different languages. To ease the process of assigning names, QF-Test has a function to suggest names for those components where a name will improve testability. (240) See the option Hotkey for components for further information. To determine the target component for an event, QF-Test calculates match probabilities for the components of the SUT. The component with the highest probability is used, provided that probability is higher than a congurable bound. First the probabilities for the known windows are determined. Then the search continues among the components of the windows with sufcient probability. This process is repeated for every hierarchy level of the component, meaning each di(514) rect or indirect parent node of the Component node, but working from top to bottom. (515) At every level all components matching the Class name attribute are ordered by decreasing probability and then searched for the component on the next hierarchy level. Components that arent visible are not considered. The probability of a component is calculated as follows: Each component starts with a probability of 99%. If geometry information is avail-

36.2.

Component recognition able, deviations in position or size lead to a probability reduction.

580

Then follow three tests that may either be skipped if no info is given, succeed or fail. A skipped test doesnt change the probability, a failed test reduces it below a given value, while a successful test raises it above another given value. The rst test is structural and applies only to Component , not to Windows . All components within the currently searched parent container, whose class is the (515) same as or derived of the given Class name , are collected in a list (including (518) invisible components). Then the Class count is compared to the size of the list (518) and the Class index must match the position of the component in the list. The next test is for a Feature match, possibly including tests for Extra features If an Extra feature check of type must match fails, the component is discarded.
(516) (516) (516) (514) (507)

Finally the Name is tested. The facts that this is the nal test and that the bonus and penalty values for this test default to the highest and lowest values, makes this the deciding test, but only if a name is given. For dialogs theres an additional step to test its modality. Typically a dialog is either modal or non-modal, so by default a modal mismatch prevents recognition. However, sometimes the same dialog may be modal or non-modal depending on the context. If your SUT contains such a dialog, adjust the modal penalty to some value above the minimum value for component recognition. If the nal probability doesnt reach the minimum value, the component is discarded. Finally the component with the highest probability is used. If the component used had a structure, feature or name mismatch, a warning is issued, since this indicates that it may not be the right component after all. Most of the time this is just a sign that the SUT has changed slightly and the components should be updated before too many changes accumulate and start confusing the identication process. Though this process is already dominated by the Name attribute, you can attach even (260) more importance to it by setting the options Name override mode (replay) and Name (242) override mode (record) to Override everything. In that case QF-Test simplies the search for a component by skipping the intermediate containers and going straight from the window to the component with the given name and class. The big advantage of this approach is, that the component will still be recognized, even if you introduce a new container in between the window and the component. The prerequisite for using this method is that you can guarantee that if a name is set on a component, it is going to be unique among the simultaneously visible components of the same class in one window. If this kind of uniqueness is not possible, the next best setting for these options is Hierarchical resolution. It requires that components with identical names at least have differently named ancestors. This setting still retains most of the advantages and exibility of names but in this case recognition will break if a named component is moved from its parent to a differently named parent.
(516)

36.3.

Timing

581

36.3

Timing

Besides component recognition, timing is an inherently difcult problem in automatic testing. No two runs of a Java program are identical when it comes to timing. Too much depends on things like system load or memory usage. This can lead to a situation where a target component for an event is not available, because the VM is still busy popping up the dialog window that contains it. To avoid needless failures of tests, QF-Test combines several tactics: Events are synchronized with the AWT event queue, meaning that after each event sent to the SUT, QF-Test waits until the events generated as side effects have been processed, before it sends another event. In some cases, especially with asynchronous updates, this doesnt sufce, so whenever a component is not available, QF-Test waits for a certain amount of time to give it a chance to appear. This delay can be customized through the option (266) Wait for non-existent component (ms) . An additional timeout dened by the option Wait for non-existent item (ms) plies when looking for sub-items of a component, e.g. a tree node.
(266)

ap-

These default timeouts should be kept rather short, a few seconds at most, so in addition you can set individual timeouts and delays wherever applicable.

36.4

Regular expressions

The regular expressions that you can use in the search and replace dialogs and in (516) (520) (519) places like the Feature attributes, the Primary index of an Item node or in (442) checks all use standard Java regexp syntax. Before QF-Test version 3.1 the default (804) was the GNU regexp package (see also appendix E ). It can still be activated via the (221) option Use old-style GNU regexps (from before QF-Test version 3.1) . One major difference between the two is that { and } are special characters that need to be escaped with \ for Java regexps, but normal characters for GNU: Detailed regexp documentation with links to further information and even a whole book about regular expressions are provided in the Java documentation for the class java.util.regex.Pattern at http://download.oracle.com/javase/1.4.2/docs/api/java/util/regex/Pattern.html. Its also worth to have a look at the Wikipedia article about regular expressions. Following is a short summary of the basics: A . stands for any one character except line breaks. With the new Java regexps you can start your regexp with the embedded ag (?s) to treat multi-line text like

36.4.

Regular expressions

582

a single line so . will match everything. The old GNU regexps have no such ag, so you need to use (.|\n) to match everything. Unfortunately this expression causes a StackOverflowException with Java regexps, so if QF-Test nds this expression in any regular expression it will treat it as a GNU regexp regardless of the option setting. Character in between [ and ] match any one of these characters. A ? says the preceding element is optional, i.e. it may appear 0 or 1 times. + means at least one of the preceding element. * means 0 or more of the preceding element. A group is created with ( and ). A ?, + or * after the closing brace refers to the whole group. All groups in a regexp are numbered in the order of their opening brace. The rst group has the number 1, 0 stands for the whole regexp. For search and replace, $n in the replacement string expands to the part of the original value matched by the nth group. Example: To change the extension of le names starting with /tmp/ from .foo to .bar, search for (/tmp/.*)\.foo and replace with $1.bar. A | separates alternatives in a group. \ quotes to suppress the special meaning of the following character or introduces special characters, e.g. \n for LineFeed (a line break), \r for CarriageReturn (not (583) needed for QF-Test, see section 36.5 ) or \t for Tab. Examples: To match any text that contains the word tree use .*tree.*. To match arbitrary text possibly including line breaks: (?s).* for Java regexps and (.|\n)* for GNU. To replace tree in arbitrary text with node use (.*)tree(.*) to search and $1node$2 to replace. In the replace dialog simply replace tree with node and disable the Match whole string check box to achieve the same effect. To search for name or names: names? To search for tree or node: (tree|node) An arbitrary word consisting of letters and numbers: [0-9a-zA-Z]+ ...

36.5.

Line breaks under Unix and Windows

583

36.5

Line breaks under Unix and Windows

The difference in the treatment of line breaks between Unix and Windows is a wellknown problem. While Unix uses a single LineFeed character (\n, hex 0x0A) as line separator, Windows uses the combination CarriageReturn/LineFeed (\r\n, hex 0x0D0A). Java automatically converts text as needed which generally works well. However, the XML standard species that an XML parser has to convert line breaks of any type into LineFeed only, regardless of the system under which it is running. This could lead to trouble, for example when checking a multi line text eld. QF-Test works around the problem by converting all text strings read from the SUT to the Unix version with LineFeed only. This has the added benet that tests created on one system will run unchanged on the other.

36.6

Quoting and escaping special characters

A common problem for most complex systems is the treatment of characters with a special meaning. A typical example are blanks in lenames. To specify such lenames on the command line, they need to be protected either by using double quotes or by escaping the blanks with a backslash character (\). Since QF-Test makes use of special characters in various contexts while reading arbitrary strings from the SUT that may contain any character, some kind of quoting mechanism is unavoidable. That QF-Test runs on various operating systems and makes use (581) of regular expressions which have their own set of special characters doesnt make things any easier. However, QF-Test attempts to keep things as simple as possible by restricting quoting to the places where it cant be avoided and by quoting all strings read from the SUT during recording correctly. The most prominent special character for QF-Test is the $ sign used for variable expansion. Variable syntax is applicable in almost all attributes. If you need a literal $ character in an attribute value you have to double it. Example: To verify with a Check text (446) set the Text attribute to 4 US$$.
(443)

node that a text eld contains the string 4 US$,

Other special characters are used only in a few places and must be quoted only there. (352) (514) These are the # character used for Procedure and Component access across suites (50) and the characters @, & and % for the special syntax for sub-item access . Since these are used as separators they cannot be escaped by doubling them, so QF-Test follows the convention to use the backslash \ as escape character which turns the backslash itself into another special character. To avoid quoting-hell with Windows lenames, QF-Test only uses quoting where the above characters are used and even there a single backslash that is not followed by a special character is interpreted literally.

36.7.

Include le resolution

584
(356)

To be precise, you have to escape the characters # and \ in the Procedure name (355) attribute of a Procedure call and the characters #, \, @, & and % in the attributes (421) (520) Component id of events and checks as well as the Primary index and (520) (519) Secondary index of an Item . Remember that the backslash is also used as the (581) escape character for regular expressions , so to get a literal \ into a regexp for a sub-item, you rst need to escape it for the regexp itself, i.e. \\, then escape these for QF-Test leading to \\\\. There is one more special case that requires a special character and corresponding quoting. This is the / character used as separator for tree path sub-items of a JTree component. Thus the / must be quoted if and only if you need a literal / in a sub-item of a JTree component. Sub-items of other components dont require special handling.

36.7

Include le resolution

This is a section you should hopefully never need to read. It explains in detail how implicit Procedure and Component references are resolved during test replay. If you need to read it, your test-suite include hierarchy probably is too complicated and you should consider simplifying your includes. There are basically two scenarios in which QF-Test must implicitly resolve a Procedure or Component reference to another suite when the requested Procedure or Component cannot be found in the current (or explicitly referenced) suite: The current suite includes other suites (by dening them in the Include les at(285) tribute of the Test-suite root node). In this case, QF-Test searches all included suites in the given order. The current suite (or rather one of its Procedures) was called by another suite. Here QF-Test searches the calling suite for the requested node. The whole thing gets complicated, when (possibly indirect) Procedure calls across testsuite boundaries and (possibly indirect, maybe even recursive) includes are combined. Following are detailed explanations of the search algorithm that will hopefully enable you to debug and resolve any include-le related problems. Whenever execution leaves the current suite to continue with some Procedure or to retrieve a Component, the other suite becomes the current suite. This process is complemented by two things: the old suite is pushed onto the so-called callstack and the variable bindings of the new current suite are pushed on top of the (52) fallback bindings stack (see chapter 7 ), so they override the bindings of the old suite. In the run-log this process is documented by adding a Suite-change node
(286)

36.7.

Include le resolution

585

which holds all of the run-log nodes for the execution that takes place outside the old suite. Any search through test-suites starts with the current suite, then continues topdown through the call-stack. So if, for example, A calls B which calls C, then C is searched rst, followed by B and nally A. Includes are considered stronger bindings than the call-stack. This means that during the search through the current suite and the suites on the call-stack, at each step the included test-suites are searched before moving to the next suite on the call-stack. For example, if A calls B which includes C, A is on the call-stack and B is the current Suite, then B will be searched rst, then C, and lastly A. In case of multiple, possibly indirect includes, the search is always conducted depth-rst in the oder in which the include les are listed. This means that if A includes B and C, and B includes D, rst A is searched, followed by B, then D and then C. If a Procedure is found (possibly indirectly) in an included test-suite (as opposed to the current suite, an explicitly referenced suite or a suite on the call-stack), the change from the old current suite to the new current suite doesnt take place in one step. This has to be illustrated with an example right from the start or well get totally lost: Lets say A calls B and that A includes C. B calls a Procedure which is found in C by way of A. Instead of changing suites directly from B to C, A will rst become the current suite and then C. As a consequence, A gets pushed onto the call-stack again on top of B and its variable bindings are also pushed again on top of Bs bindings on the fallback bindings stack. The reasoning behind this is that C, which is now the current suite, is closer to A, which includes C, than it is to B, which only happened to be called by A. One could also say that inclusion creates a kind of union, so that to B, A and C will always appear as a single test-suite as long as B doesnt call C expicitly. Thats it, except for one thing: During a search QF-Test never searches a suite twice. This would be useless in any case, but it is more than an optimization, since it prevents trouble with recursive includes if A includes B and B includes A. If you really have a problem determining how, why or why not a certain Procedure or Component was retrieved, rst take a look at the run-log. It shows exactly which suites were used and which variable expansions took place.

Chapter 37 Jython and Groovy scripting

This section explains technical details about the Jython integration in QF-Test and serves as a reference for the whole API exposed by QF-Test for use in Jython and Groovy scripts. For a more gentle introduction including examples please take a look at (94) chapter 13 .

37.1

Module load-path (Jython)

The load-path for Python/Jython modules, sys.path, is assembled from various sources in the following order: the directory qftest/jython the directory qftest/qftest-3.5.3/jython/Lib directories dened in the system property python.path In addition, during Server script or SUT script node execution, the directory of the containing test-suite is prepended to the path. The directory qftest/qftest-3.5.3/jython/Lib contains Jython-internal stuff, the complete Python library and the following modules provided by Quality First Software GmbH: qfcommon.py, qftest.py and qfclient.py. You should not modify these les, since they may change in later versions of QF-Test. The directory qftest/jython is the place to put your own shared modules. These will be left untouched during an update of QF-Test. Modules that are specic to a test-suite can also be placed in the same directory as the test-suite. The le extension for all modules must be .py. To add additional directories to the load-path, dene the python.path system property.

37.2.

The plugin directory

587

37.2

The plugin directory

Jython and Groovy can also be used to access Java classes and methods beyond the scope of QF-Test by simply importing such classes, e.g.
from java.util import Date from java.text import SimpleDateFormat print SimpleDateFormat("yyyy-MM-dd").format(Date())

Example 37.1: Accessing Java classes from Jython

The classes available for import are those on the CLASSPATH during startup of QFTest or the SUT respectively, all classes of the standard Java API and QF-Tests own classes. For the SUT things also depend on the ClassLoader concept in use. WebStart and Eclipse/RCP in particular make it difcult to import classes directly from the SUT. Additionally, there are plugin directories into which you can simply drop a jar le to make it available to scripts. By default, the main plugin directory is called plugin (big surprise here) and is located under QF-Tests root directory qftest. This location can (540) be overriden with the command line argument -plugindir <directory> . Jar les in the main plugin directory are available to both Server script and SUT script nodes. To make a jar available solely to Server scripts or solely to SUT scripts, drop it in the respective sub-directory called qftest or sut instead.

37.3

The package cache (Jython)

In order to be able to import Java packages, Jython maintains a cache of package information. By default this cache is located under qftest/jython/cachedir. If this directory is not writable, .qftest/jython-cachedir under the users home directory is used instead. The location of the directory can be overridden with the python.cachedir system property. When running QF-Test for the rst time after installing it, you may see a number of messages of the form *sys-package-mgr*: processing... Later you should see these messages only when jar-les on the CLASSPATH have been modied or new ones added. Sometimes Jython goes into hiccup mode and regenerates the cache for some jar-les every time QF-Test or the SUT is started. In that case, simply remove the whole qftest/jython/cachedir or .qftest/jython-cachedir and the problem should go away.

37.4.

Initialization (Jython)

588

37.4

Initialization (Jython)

During QF-Test and SUT startup an embedded Jython interpreter is created. For QF-Test, the module named qftest is imported, for the SUT the module named qfclient. Both are based on qfcommon which contains shared code. These modules are required to provide the run-context interface and to set up the global namespace. Next the load-path sys.path is searched for your personal initialization les. For QFTest initialization, the le called qfserver.py is loaded, the le called qfsut.py is used for the SUT. In both cases execfile is used to execute the contents of these les directly in the global namespace instead of loading them as modules. This is much more convenient for an initialization le because everything dened and all modules imported will be directly available to Server scripts and SUT scripts. Note that at initialization time no run-context is available and no test-suite-specic directory is added to sys.path.

37.5

Namespace environment for script execution (Jython)

The environments in which Server scripts or SUT scripts are executed are dened by the global and local namespaces in effect during execution. Namespaces in Jython are dictionaries which serve as containers for global and local variable bindings. The global namespace is shared between all scripts run in the same Jython interpreter. Initially it will contain the classes TestException and UserException, the module qftest or qfclient for QF-Test or the SUT respectively, and everything dened in or imported by qfserver.py or qfsut.py. When assigning a value to a variable declared to be global with the global statement, that variable is added to the global namespace and available to scripts run consecutively. Additionally, QF-Test ensures that all modules imported during script execution are globally available. The local namespace is unique for each script and its lifetime is limited to the scripts execution. Upon invocation the local namespace contains rc, the interface to QF-Tests run-context, and true and false bound to 1 and 0 respectively for better integration with QF-Test. Accessing or setting global variables in a different Jython interpreter is enabled through the methods fromServer, fromSUT, toServer and toSUT.

37.6.

Run-context API

589

37.6

Run-context API

The run-context object rc is an interface to the execution state of the currently running test in QF-Test. Providing this wrapper instead of directly exposing QF-Tests Java API leaves us free to change the implementation of QF-Test without affecting the interface for scripts. Following is a list of the methods of the run-context object rc in alphabetical order. The syntax used is a bit of a mixture of Java and Python. Python doesnt support static typing, but the parameters are passed on to Java, so they must be of the correct type to avoid triggering exceptions. If a parameter is followed by an = character and a value, that value is the default and the parameter is optional.
Note

Please note that the Groovy syntax for keyword parameters is different from Jython and requires a : instead of =. The tricky bit is that, for example, rc.logMessage(bla, report=true) is perfectly legal Groovy code yet doesnt have the desired effect. The = here is an assignment resulting in the value true, which is simply passed as the second parameter, thus the above is equal to rc.logMessage(bla, true) and the true is passed to dontcompactify instead of report. The correct Groovy version is rc.logMessage(bla, report:true). void addDaemonLog(byte[] data, String name=None, String comment=None, String externalizename=None) Add a run-log retrieved from a DaemonRunContext to the current run-log. Parameters data The byte array retrieved via DaemonRunContext.getRunLog(). name An optional name for the daemon log node. If unspecied the ID of the Daemon is used. comment An optional comment for the daemon log node. externalizename An optional name to externalize the daemon log and save it as a partial log of a split run-log. void addTestRunListener(TestRunListener listener) Register a TestRunListener with the current run-context. In interactive mode and batch mode there is a single, shared run-context, so the listener will remain in effect until it gets removed via removeTestRunListener or clearTestRunListeners. In daemon mode, each DaemonRunContext has its own set of listeners. See section (683) 39.5 for details about the TestRunListener API. Parameters listener The listener to register.

37.6.

Run-context API

590

String callProcedure(String name, dictionary parameters=None) (352) Call a Procedure in a test-suite. As a convenience, this method can also be called from an SUT script. Care should be taken however, because the script is executed inside the AWT event dispatch thread, so weird side-effects are possible, though QF-Test does its best to avoid these. If possible, call Procedures from a Server script instead. Parameters name The fully qualied name of the Procedure. parameters The parameters for the Procedure. This should be a dictionary. Its keys and values can be arbitrary values. They are converted to strings for the call. Returns The value returned from the Procedure through an optional (358) Return node. int callTest(String name, dictionary parameters=None) Server only. Call a Test-case or Test-set in a test-suite. Parameters name The fully qualied name of the Test-case or Test-set. parameters The parameters for the Test-case or Test-set. This should be a dictionary. Its keys and values can be arbitrary values. They are converted to strings for the call. Returns The nal state of the execution. Either rc.OK, rc.WARNING, rc.ERROR, rc.EXCEPTION, rc.SKIPPED or rc.NOT_IMPLEMENTED.
(288) (294)

37.6.

Run-context API

591

Boolean check(boolean condition, String message, int level=rc.ERROR, boolean report=true, boolean nowrap=false) Check or assert that a condition is true and log a message according to the result. Parameters condition The condition to evaluate. message The message to log. It will be preceded by Check OK: or Check failed: depending on the result. For the oldstyle XML or HTML report the message will be treated like a Check node if it starts with an ! character. level The error level in case of failure. The following constants are dened in the run-context: rc.OK rc.WARNING rc.ERROR rc.EXCEPTION If the level is rc.EXCEPTION, a UserException will be thrown if the check fails. If true, the check will appear in the report. Only applicable if level <= rc.WARNING. If true, lines of the message will not be wrapped in the report. Use for potentially long messages. The result of the check.
(532)

report nowrap Returns

37.6.

Run-context API

592

Boolean checkEqual(Object actual, Object expected, String message, int level=rc.ERROR, boolean report=true, boolean nowrap=false) Check or assert that an object matches a given value and log a message according to the result. Comparison is done using the == operator. Parameters actual The actual value. expected The expected value. message The message to log. It will be preceded by Check OK: or Check failed: depending on the result. In case of failure, the expected and actual values will also be logged. level The error level in case of failure. The following constants are dened in the run-context: rc.OK rc.WARNING rc.ERROR rc.EXCEPTION If the level is rc.EXCEPTION, a UserException will be thrown if the check fails. If true, the check will appear in the report. Only applicable if level <= rc.WARNING. If true, lines of the message will not be wrapped in the report. Use for potentially long messages. The result of the check.
(532)

report nowrap Returns

37.6.

Run-context API

593

Boolean checkImage(ImageRep actual, ImageRep expected, String message, int level=rc.ERROR, boolean report=true, boolean nowrap=false) (690) Check or assert two given ImageRep (see section 39.7.1 ) objects for equality and log a message according to the result. Comparison is done using the equals method (693) of the ImageComparator (see section 39.7.2 ) of the expected object. Parameters actual The actual value ImageRep object. expected The expected ImageRep object. message The message to log. It will be preceded by Check OK: or Check failed: depending on the result. In case of failure, the expected and actual values will also be logged. For the old-style XML or HTML report the message will be treated like a Check node if it starts with an ! character. level The error level in case of failure. The following constants are dened in the run-context: rc.OK rc.WARNING rc.ERROR rc.EXCEPTION If the level is rc.EXCEPTION, a UserException will be thrown if the check fails. If true, the check will appear in the report. Only applicable if level <= rc.WARNING. If true, lines of the message will not be wrapped in the report. Use for potentially long messages. The result of the check.
(532)

report nowrap Returns

37.6.

Run-context API

594

Object[] checkImageAdvanced(ImageRep actual, ImageRep expected, String message, String algorithm, int level=rc.ERROR, boolean report=true, boolean nowrap=false) (690) Check or assert two given ImageRep (see section 39.7.1 ) objects for equality and log a message according to the result. Comparison is done using the specied algorithm. Parameters actual The actual value ImageRep object. expected The expected ImageRep object. message The message to log. It will be preceded by Check OK: or Check failed: depending on the result. In case of failure, the expected and actual values will also be logged. For the old-style XML or HTML report the message will be treated like a Check node if it starts with an ! character. algorithm The algorithm to use for the comparison as described in (722) chapter 44 . level The error level in case of failure. The following constants are dened in the run-context: rc.OK rc.WARNING rc.ERROR rc.EXCEPTION If the level is rc.EXCEPTION, a UserException will be thrown if the check fails. If true, the check will appear in the report. Only applicable if level <= rc.WARNING. If true, lines of the message will not be wrapped in the report. Use for potentially long messages. An array with following content: The result of the check as Boolean. The result of the check as probability. The transformed image of the expected image as ImageRep, depending on the algorithm. The transformed image of the actual image as ImageRep, depending on the algorithm. Further information where appropriate.
(532)

report nowrap Returns

37.6.

Run-context API

595

void clearGlobals() Server only. Undene all global variables. void clearProperties(String group) Server only. Delete a given set of loaded properties or resources. Parameters group The group name of the properties or resources. void clearTestRunListeners() Remove all TestRunListeners from the current run-context. String expand(String text) Expand a string using standard QF-Test variable expansion for $(...) or ${...:...} syntax. Remember to double the $ signs to avoid expansion before the script is executed (see (583) section 36.6 ). Parameters text The string to expand. Returns The expanded string. Object fromServer(String name) SUT only. Retrieve the value of a global variable in the Jython or Groovy interpreter of QF-Test. If the variable is undened, a KeyError is raised. Parameters name The name of the variable. Returns The value of the variable. Object fromSUT(String client, String name) Server only. Retrieve the value of a global variable in the Jython or Groovy interpreter of the SUT. If the variable is undened, a KeyError is raised. Parameters client The name of the SUT client. name The name of the variable. Returns The value of the variable.

37.6.

Run-context API

596

Boolean getBool(String varname, boolean expand=true) Look up the value of a QF-Test variable, similar to lookup(), but treat it as a boolean. Parameters varname The name of the variable. expand Whether to expand the value of the variable recursively. If, for example, the value of $(varname) is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Note that if you want to set this parameter, you must use Python keyword syntax to avoid conicts with getBool(String group, String name), i.e. rc.getBool(var, expand=0) instead of rc.getBool(var, 0). Returns The value of the variable. Boolean getBool(String group, String name, boolean expand=true) Look up the value of a QF-Test resource or property, similar to lookup(), but treat it as a boolean. Parameters group The name of the group. name The name of the resource or property. expand Whether to expand the value of recursively. If, for example, the value of ${group:name} is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Returns The value of the resource or property. Exception getCaughtException() (382) Server only. If the script is run inside a Catch node, the exception that was caught (303) (306) is returned. Similarly, if a Test node has the Implicitly catch exceptions attribute set, (325) the caught exception is available during execution of the Test nodes Cleanup node. In all other cases, None is returned. Returns The caught exception.

37.6.

Run-context API

597

Component getComponent(String id, int timeout=-1, boolean hidden=false) SUT only. Find a component or a cmponents sub-item using QF-Tests component recognition mechanism. Parameters (515) (514) of the Component node that represents the id The Id component in the test-suite. timeout The maximum time to wait for the component if it is not found instantly. A negative value, the default, means use the value from the option Wait for non-existent component (266) (ms) . hidden If true, nd invisible components as well. Useful for menuitems. Returns The actual Java component. For sub-items, a pair of the form (component, index) is returned, where the type of index depends on the type of the item. For tree nodes it is a javax.swing.tree.TreePath object, for tablecells a pair of the form (row, column) and an integer for all other kinds of items. Column indexes returned are always given in table coordinates, not in model coordinates. List getConnectedClients() Get the names of the currently connected SUT clients. Returns A list with the names of the currently connected SUT clients, an empty list in case there are none. Properties getGlobals() Get the global variables bound in the current context. Returns The global variables of the current context.

37.6.

Run-context API

598

Integer getInt(String varname, boolean expand=true) Look up the value of a QF-Test variable, similar to lookup(), but treat it as an integer. Parameters varname The name of the variable. expand Whether to expand the value of the variable recursively. If, for example, the value of $(varname) is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Note that if you want to set this parameter, you must use Python keyword syntax to avoid conicts with getInt(String group, String name), i.e. rc.getInt(var, expand=0) instead of rc.getInt(var, 0). Returns The value of the variable. Integer getInt(String group, String name, boolean expand=true) Look up the value of a QF-Test resource or property, similar to lookup(), but treat it as an integer. Parameters group The name of the group. name The name of the resource or property. expand Whether to expand the value of recursively. If, for example, the value of ${group:name} is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Returns The value of the resource or property. Object getLastComponent() SUT only. Get the last item that was addressed by QF-Test for replaying some event, check or miscellaneous operation. Calls to rc.getComponent() have no impact. Returns The last component addressed by QF-Test. Exception getLastException() Server only. Get the last exception (caught or uncaught) that was thrown during the test-run. In most cases getCaughtException is probably more useful. Returns The most recent exception that was thrown. Object getLastItem() SUT only. Get the last item that was addressed by QF-Test for replaying some event, check or miscellaneous operation. Calls to rc.getComponent() have no impact. Returns The last item addressed by QF-Test.

37.6.

Run-context API

599

Properties getLocals(nonEmpty=false) Get the innermost local bindings of the context. Mostly useful within a procedure to get the parameters of the procedure call and implement something similar to keyword arguments in Jython or Groovy. Parameters nonEmpty True to get the rst non-empty set of bindings, false to get the innermost bindings even when empty. Returns The innermost local variable bindings of the current context. Number getNum(String varname, boolean expand=true) Look up the value of a QF-Test variable, similar to lookup(), but treat it as a number, i.e. as int or oat for Jython and as Integer or BigDecimal for Groovy. Parameters varname The name of the variable. expand Whether to expand the value of the variable recursively. If, for example, the value of $(varname) is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Note that if you want to set this parameter, you must use Python keyword syntax to avoid conicts with getNum(String group, String name), i.e. rc.getNum(var, expand=0) instead of rc.getNum(var, 0). Returns The value of the variable. Number getNum(String group, String name, boolean expand=true) Look up the value of a QF-Test resource or property, similar to lookup(), but treat it as a number, i.e. as int or oat for Jython and as Integer or BigDecimal for Groovy. Parameters group The name of the group. name The name of the resource or property. expand Whether to expand the value of recursively. If, for example, the value of ${group:name} is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Returns The value of the resource or property.

37.6.

Run-context API

600

Object getOption(String name) Get an option value at run time. This method is provided more for the sake of completeness, you will probably not need it. For the obvious use case of restoring the value of an option to its previous value after a change with setOption you should use unsetOption instead because values set at script level hide values set interactively in the options dialog. Parameters name The name of the option, a constant from the Options class which is automatically imported in Jython and Groovy scripts. The names of the options that can be read (217) in this way are documented in chapter 29 . Returns The current value of the option. Properties getProperties(String group) Get a set of loaded properties or resources. Parameters group The group name of the properties or resources. Returns The variables bound for the given group or null if no such group exists. String getStr(String varname, boolean expand=true) Look up the value of a QF-Test variable, similar to lookup(). Parameters varname The name of the variable. expand Whether to expand the value of the variable recursively. If, for example, the value of $(varname) is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Note that if you want to set this parameter, you must use Python keyword syntax to avoid conicts with getStr(String group, String name), i.e. rc.getStr(var, expand=0) instead of rc.getStr(var, 0). Returns The value of the variable.

37.6.

Run-context API

601

String getStr(String group, String name, boolean expand=true) Look up the value of a QF-Test resource or property, similar to lookup(). Parameters group The name of the group. name The name of the resource or property. expand Whether to expand the value of recursively. If, for example, the value of ${group:name} is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Returns The value of the resource or property. String id(String id) Return the ID of a specied component. This method should be used to take care, that the component ID becomes updated after moving or changing the component ID. Parameters id The component ID. Returns The component ID. void logDiagnostics(String client) Server only. Adds event information stored in the SUT for possible error diagnosis to the run-log. Parameters client The name of the SUT client from which to get the information. void logError(String msg, boolean nowrap=false) Add an error message to the run log. Parameters msg The message to log. nowrap If true, lines of the message will not be wrapped in the report. Use for potentially long messages.

37.6.

Run-context API

602

void logMessage(String msg, boolean dontcompactify=false, boolean report=false, boolean nowrap=false) Add a plain message to the run log. Parameters msg The message to log. dontcompactify If true, the message will never be removed from a compact run-log. report If true, the message will appear in the report. nowrap If true, lines of the message will not be wrapped in the report. Use for potentially long messages. void logWarning(String msg, boolean report=true, boolean nowrap=false) Add a warning message to the run log. Parameters msg The message to log. report If true (the default), the warning will be listed in the report. Set this to false to exclude this specic warning from the report. nowrap If true, lines of the message will not be wrapped in the report. Use for potentially long messages. void logImage(ImageRep image, String title=None, boolean dontcompactify=false, report=false) Add an ImageRep (see section 39.7.1 ) object to the run log. Parameters image The ImageRep object to log. title An optional title for the image. dontcompactify If true, the message will never be removed from a compact run-log. report True to log the image in the report (implies dontcompactify).
(690)

37.6.

Run-context API

603

String lookup(String varname, boolean expand=true) Look up the value of a QF-Test variable, similar to $(varname). Parameters varname The name of the variable. expand Whether to expand the value of the variable recursively. If, for example, the value of $(varname) is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Note that if you want to set this parameter, you must use Python keyword syntax to avoid conicts with lookup(String group, String name), i.e. rc.lookup(var, expand=0) instead of rc.lookup(var, 0). Returns The value of the variable. String lookup(String group, String name, boolean expand=true) Look up the value of a QF-Test resource or property, similar to ${group:name}. Parameters group The name of the group. name The name of the resource or property. expand Whether to expand the value of recursively. If, for example, the value of ${group:name} is the literal string $(othervar), this method will return the expanded value of $(othervar) if expand is true and $(othervar) itself if expand is false. Returns The value of the resource or property. void overrideElement(String id, Component com) SUT only. Override the target GUI element for component recognition for an element with the given ID. When that ID is referenced QF-Test ignores all associated information and directly returns the given element. Parameters id com

The id of the GUI element to override. The GUI element to return as the resolved target. None/null to revert to the default mechanism.

void removeTestRunListener(TestRunListener listener) Remove a TestRunListener from the current run-context. Parameters listener The listener to remove.

37.6.

Run-context API

604

void resetDependencies(String namespace=None) Completely reset the dependency stack without executing any cleanup. Parameters namespace An optional namespace to reset the dependencies for. void resolveDependency(String dependency, String namespace=None, dictionary parameters=None) Resolve a Dependency Parameters dependency namespace parameters
(317)

. The fully qualied name of the Dependency to resolve. An optional namespace to resolve the Dependency in. The parameters for the Dependency. This should be a dictionary. Its keys and values can be arbitrary values. They are converted to strings for the call.

void rollbackDependencies(String namespace=None) Unroll the dependency stack. Parameters namespace An optional namespace to unroll the dependencies in. void setGlobal(String name, object value) Dene a global QF-Test variable. Parameters name The name of the variable. value An arbitrary value for the variable. It is automatically converted to a string. A value of None unsets the variable. void setLocal(String name, object value) Dene a local QF-Test variable. Parameters name The name of the variable. value An arbitrary value for the variable. It is automatically converted to a string. A value of None unsets the variable.

37.6.

Run-context API

605

void setOption(String name, object value) Set an option value at run time. Any value thus set overrides the value read from the system conguration le or set via the option dialog, but is never shown in the option dialog or saved to a conguration le. The default value can be restored via unsetOption. Parameters name The name of the option, a constant from the Options class which is automatically imported in Jython and Groovy scripts. The names of the options that can be set (217) in this way are documented in chapter 29 . value The value to set, typically a boolean, a number or a constant from the Options class for options edited via a drop-down list. For hotkey options like the Dont panic hotkey this value should be a string like F12 or ShiftF6. Supported modiers are Shift, Control or Ctrl, Alt and Meta and combinations thereof. Key speciers are prepended with VK_ and then looked up in the class java.awt.event.KeyEvent. Case is irrelevant for both, so shift-alt-enter will work as well. void setProperty(String group, String name, object value) Set the value of a resource or property in a group. Parameters group The name of the group. A new group is created automatically if necessary. name The name of the resource or property. value An arbitrary value for the property. It is automatically converted to a string. A value of None unsets the property. This method also works for the special groups system and env and can be used as a means to set environment variables and system properties. Values in the special group qftest cannot be set or changed that way. void skipTestCase() Stop the execution of the current test-case and mark it as skipped. void skipTestSet() Stop the execution of the current test-set and mark it as skipped. void stopTest() Terminate the current test-run.

37.6.

Run-context API

606

void stopTestCase(boolean expectedFail=false) Stop the execution of the current test-case. Parameters expectedFail If true, mark possible errors in this test-case as expected failures. void stopTestSet() Stop the execution of the current test-set. void syncThreads(String name, int timeout, int count=-1, boolean throw=true, int remote=0) Server only. Synchronize a number of parallel threads for load testing. The current thread is blocked until all threads have reached this synchronization point or the timeout (526) is exceeded. In the latter case, a TestException is thrown or an error logged. Parameters name An identier for the synchronization point. timeout The maximum time to wait in milliseconds. count The number of threads to wait for. Default value -1 means all threads in the current QF-Test instance. throw Whether to throw an exception (default) or just log an error if the timeout is exceeded without all threads reaching the synchronization point. remote The number of QF-Test instances - potentially running on different machines - to synchronize. Default 0 means dont do remote synchronization. void toServer(...) SUT only. Set some global variables in the Jython or Groovy interpreter of QF-Test. Each argument can be any of: A string This is treated as the name of a global variable in the local interpreter. The variable by the same name in QF-Tests interpreter is set to its value. A dictionary with string keys For each key in the dictionary, a global variable by that name is set to the corresponding value from the dictionary. A keyword argument in the form name=value The global variable named name is set to value.

37.7.

The qf module

607

void toSUT(String client, ...) Server only. Set some global variables in the Jython or Groovy interpreter of the SUT. Except for client, each argument can be any of: A string This is treated as the name of a global variable in the local interpreter. The variable by the same name in SUTs interpreter is set to its value. A dictionary with string keys For each key in the dictionary, a global variable by that name is set to the corresponding value from the dictionary. A keyword argument in the form name=value The global variable named name is set to value. Parameters client

The name of the SUT client.

void unsetOption(String name) Restore an option value by removing a possible override from a previous call to setOption. Parameters name The name of the option to unset, a constant from the Options class which is automatically imported in Jython and Groovy scripts. The constants for options that can be (217) set in this way are documented in chapter 29 .

37.7

The qf module

In some cases there is no run-context available, especially when implementing some of the extension interfaces described in the following sections. The module qf enables logging in those cases and also provides some generally useful methods that can be used without depending on a run-context. Following is a list of the methods of the qf module in alphabetical order. Unless metioned otherwise, methods are available in Groovy and Jython and for both Server script and SUT script nodes.
Note

Please note that the Groovy syntax for keyword parameters is different from Jython and requires a : instead of =. The tricky bit is that, for example, qf.logMessage(bla, report=true) is perfectly legal Groovy code yet doesnt have the desired effect. The = here is an assignment resulting in the value true, which is simply passed as the

37.7.

The qf module

608

second parameter, thus the above is equal to qf.logMessage(bla, true) and the true is passed to dontcompactify instead of report. The correct Groovy version is qf.logMessage(bla, report:true). String boolean getClassName(Object objectOrClass) Get the fully qualied name of the Class of a Java object, or of a Java class itself. Mostly useful for Jython where getting the name of a class can become a real hassle. Parameters objectOrClass The Java object or class to get the class name for. Returns The class name or None in case something non-Java is passed in. boolean isInstance(Object object, String className) This is a simple alternative to instanceof in Groovy and isinstance() in Jython that deliberately compares class and instance names only so conicts with differing class loaders are avoided. Parameters object The object to check. className The name of the class or interface to test for. Returns True if the object is an instance of the given class or implements the given interface. String toString(Object object, String nullValue) Get the string representation of an object. Mostly useful for Jython but sometimes also useful for Groovy thanks to the default conversion of null to the empty string. Parameters object The object to get the string representation for. nullValue The value to return if object is None, the empty string by default. Returns Jython 8-bit or unicode strings are returned unchanged, Java objects are turned into a string via toString. In Jython, everything else is converted into an 8-bit Jython string. void setProperty(Object object, String name, Object value) Set an arbitrary property for an object. For Swing, SWT or web components the value is stored in the respective user data via putClientProperty, setData or setProperty respectively. For everything else a WeakHashMap is used. Either way the property will not prevent garbage collection of the object. Parameters object The object to set the property for. name The name of the property. value The value to set. Null to remove the property.

37.7.

The qf module

609

Object getProperty(Object object, String name) Get a property for an object that was previously set via setProperty. Parameters object The object to get the property for. name The name of the property. Returns The property value. void logError(String msg, boolean nowrap=false) Add an error message to the run-log. If a run-context is available it is used and logging takes effect immediately. Otherwise the message is buffered and logged at the next opportunity. Parameters msg The message to log. nowrap If true, lines of the message will not be wrapped in the report. Use for potentially long messages. This parameter has no effect if the message needs to be buffered. void logMessage(String msg, boolean dontcompactify=false, boolean report=false, boolean nowrap=false) Add a plain message to the run log. If a run-context is available it is used and logging takes effect immediately. Otherwise the message is buffered and logged at the next opportunity. Parameters msg The message to log. dontcompactify If true, the message will never be removed from a compact run-log. report If true, the message will appear in the report. nowrap If true, lines of the message will not be wrapped in the report. Use for potentially long messages. This parameter has no effect if the message needs to be buffered.

37.8.

Image API

610

void logWarning(String msg, boolean report=true, boolean nowrap=false) Add a warning message to the run log. If a run-context is available it is used and logging takes effect immediately. Otherwise the message is buffered and logged at the next opportunity. Parameters msg The message to log. report If true (the default), the warning will be listed in the report. Set this to false to exclude this specic warning from the report. nowrap If true, lines of the message will not be wrapped in the report. Use for potentially long messages. This parameter has no effect if the message needs to be buffered.

37.8
3.0+

Image API

The Image API provides classes and interfaces to take screenshots, to save or load images or for own image comparisons.

37.8.1

The ImageWrapper class

For taking screenshots you can use the Jython class ImageWrapper, located in the module imagewrapper.py, which comes with the QF-Test installation. Here is a short sample Jython script demonstrating the usage of the Image API:
from imagewrapper import ImageWrapper #create ImageWrapper instance iw = ImageWrapper(rc) #take screenshot of the whole screen currentScreenshot = iw.grabScreenshot() #save screenshot to a file iw.savePng("/tmp/screenshot.png", currentScreenshot)

Example 37.2: Image API in Jython

And the same in Groovy:

37.8.

Image API

611

import de.qfs.ImageWrapper def iw = new ImageWrapper() def currentScreenshot = iw.grabScreenshot() iw.savePng("/tmp/screenshot.png", currentScreenshot)

Example 37.3: Image API in Groovy

Following is a list of the methods of the ImageWrapper class in alphabetical order. The syntax used is a bit of a mixture of Java and Python. Python doesnt support static typing, but the parameters are passed on to Java, so they must be of the correct type to avoid triggering exceptions. If a parameter is followed by an = character and a value, that value is the default and the parameter is optional. ImageWrapper ImageWrapper(RunContext rc) Constructor method of the ImageWrapper class. Parameters rc The current run context of QF-Test. ImageRep grabScreenshot(int x=None, int y=None, int width=None, int height=None) Take screenshot of the whole screen. If you use the parameters x, y, width and height, you can take a screenshot of a specic region of the screen. Parameters x The X co-ordinate of the left upper corner of the region to take the screenshot. y The Y co-ordinate of the left upper corner of the region to take the screenshot. width The width of the region to take the screenshot. height The height of the region to take the screenshot. Returns An ImageRep object containing the actual screenshot.

37.8.

Image API

612

ImageRep grabImage(Object com, int x=None, int y=None, int width=None, int height=None) Take screenshot of a given component. If you use the parameters x, y, width and height, you can take a screenshot of a specic region of the component. Parameters com The component-ID of the component to take a screenshot from. x The X co-ordinate of the left upper corner of the region to take the screenshot. y The Y co-ordinate of the left upper corner of the region to take the screenshot. width The width of the region to take the screenshot. height The height of the region to take the screenshot. Returns An ImageRep object containing the actual screenshot. ImageRep[] grabScreenshots(int monitor=None) Take screenshots of all available screens. This procedure might be useful, if you work with more than one screen. If you want to take a screenshot of one specic screen, you can also use this procedure. Parameters monitor Index of the monitor to take the screenshot from. The rst monitor has 0, the second 1 and so on. Returns An array of ImageRep objects of all screenshots or the specic ImageRep object, if the monitor parameter has been used. int getMonitorCount() Return the number of monitors. Returns The total number of monitors. void savePng(String filename, ImageRep image) Save the given ImageRep object to a le. The le will be in PNG format. Parameters filename The path to the le, where the image should be stored to. image The ImageRep object to store. ImageRep loadPng(String filename) Load an image from a given le return an ImageRep object containing this image. The le has to contain the image in PNG format. Parameters filename The path to the le, where the image is stored. Returns An ImageRep object containing loaded image.

37.9.

Exception handling

613

37.9

Exception handling
(526)

All QF-Test exceptions listed in chapter 31 are automatically imported in Jython scripts and can be used for try/except clauses like try: com = rc.getComponent("someId") except ComponentNotFoundException: ... When working with Groovy you must rst import the exception: import de.qfs.apps.qftest.shared.exceptions.ComponentNotFoundException try { com = rc.getComponent("someId") } catch (ComponentNotFoundException) { ... } Only the following exceptions should be raised explicitely from script code (with raise or throw new respectively): UserException(Some message here...) should be used to signal exceptional error conditions. BreakException() or raise BreakException(loopId) can be used to (362) (365) break out of a Loop or While node, either without parameters to break out of the innermost loop or with the id parameter to break out of a specic loop. ReturnException() or raise ReturnException(value) can be used (352) to return - with or without a value - from a Procedure node, similar to executing a (358) Return node.

37.10

Debugging scripts (Jython)

When working with Jython modules you dont have to restart QF-Test or the SUT after you made changes. You can simply use reload(<modulename>) to load the module anew. Debugging scripts in an embedded Jython interpreter can be tedious. To simplify this task, QF-Test offers an active terminal window for communicating with each

37.10.

Debugging scripts (Jython)

614

interpreter. These terminals are accessible through the Clients menu or through ExtrasJython Terminal... . Alternatively, a network connection can be established to talk remotely to the Jython interpreter - in QF-Test as well as within the SUT - and get an interactive command line. To enable this feature you must use the command line argument (544) -jythonport <number> to set the port number that the Jython interpreter should listen on. For the SUT -jythonport=<port> can be dened in the Extra (397) (395) (399) Parameters of the Start Java SUT client or Start SUT client node. You can then connect to the Jython interpreter, for example with telnet localhost <port> Combined with Jythons ability to access the full Java API, this is not only useful for debugging scripts but can also be used to debug the SUT itself.

Chapter 38 Special features for web applications

Web

This chapter addresses topics that are only relevant when testing web applications in a browser.

38.1

Pseudo DOM API

(392)

To a certain extent QF-Test exposes the DOM of a web based SUT to SUT script nodes. This API is not equivalent to working directly at the Javascript level which can be done via the methods toJS and evalJS described in this chapter. With this API is possible to traverse the DOM and retrieve and set attributes of the respective nodes, but not to manipulate the structure of the DOM. Thus this API is useful for implementing (634) Name- or FeatureResolvers as described in section 39.1 .
Note

The DOM can differ depending on the browser, so you should try not to rely too much on child indexes in your resolvers or scripts in case of cross-browser testing if viable. QF-Tests DOM API comprises a hierarchy of the following ve classes:

38.1.1

The abstract Node class

are derived from this class Its is located in the package

All classes of QF-Tests pseudo DOM and thus implement the following interface. de.qfs.apps.qftest.client.web.dom.

38.1.

Pseudo DOM API

616

String getNodeType() Get an identier for the type of the node. Though this method is not fully in line with pure OO doctrine, while traversing the DOM it is often necessary to nd out the type of a given node and this is quite convenient. Returns A string specifying the type of node. The respective constants are dened in the concrete sub-classes: DocumentNode.DOCUMENT_NODE, FrameNode.FRAME_NODE, DomNode.DOM_NODE and DialogNode.DIALOG_NODE. String getName() Get the tag name of the node. The tag name is the type of node in upper case like HTML for an <HTML> node. Plain text nodes are represented as a DomNode with tag name #text. Pseudo tag names are dened for DocumentNodes (DOCUMENT) and DialogNodes (DIALOG). Returns The tag name of the node. String getAttribute(String name) Get the value of an attribute of the node. For convenience this method is dened at Node level. If this node is not a DomNode the result will always be null. In some cases this method also returns attributes that have not been explicitly specied, like the width and height of an IMG node. Which attributes can be retrieved in that way is browser dependent. Parameters name The name of the attribute Returns The value of the attribute or null if no such attribute is available for the node. String getAttributeIfSpecified(String name) Get the value of an attribute of the node if it is explicitly specied in the HTML code. For convenience this method is dened at Node level. If this node is not a DomNode the result will always be null. Parameters name The name of the attribute Returns The value of the attribute or null if no such attribute is explicitly specied for the node. boolean isAttributeSpecified(String name) Test whether an attribute of the node is explicitly specied in the HTML code. For convenience this method is dened at Node level. If this node is not a DomNode the result will always be false. Parameters name The name of the attribute Returns True if an attribute is explicitly specied for the node.

38.1.

Pseudo DOM API

617

String getText() Get the text content of a Node. In case of a plain text node, return its value. Otherwise traverse the DOM and collect all nested text nodes values. Care is taken to collapse whitespace between nodes to get as close as possible to what is displayed in the browser. Returns The whole direct and indirect text content of a node. String getVisibleText() Get the text content of a Node. In case of a plain text node, return its value. Otherwise traverse the DOM and collect all nested text nodes values, ignoring invisible nodes. Care is taken to collapse whitespace between nodes to get as close as possible to what is displayed in the browser. Returns The whole direct and indirect visible text content of a node. String getFlatText() Get the at text content of a Node. In case of a plain text node, return its value. Otherwise collect the values of all text nodes directly contained within this node, excluding those nested at a deeper level. Returns The whole direct text content of a node. String getVisibleFlatText() Get the at text content of a Node. In case of a plain text node, return its value. Otherwise collect the values of all text nodes directly contained within this node, excluding invisible ones and those nested at a deeper level. Returns The whole direct visible text content of a node. String getSimpleText() Get the simple text content of a Node. In case of a plain text node, return its value. Otherwise traverse the DOM and collect all text nodes contained directly within this node or in simple nodes like <B>, but do not descend into structurally more complex nodes like <TABLE>. Returns The simple text content of a node. String getVisibleSimpleText() Get the simple text content of a Node. In case of a plain text node, return its value. Otherwise traverse the DOM and collect all visible text nodes contained directly within this node or in simple nodes like <B>, but do not descend into structurally more complex nodes like <TABLE>. Returns The visible simple text content of a node. DocumentNode getDocument() Get the document to which this node belongs. Returns The document to which this node belongs. A DocumentNode will return itself, a DialogNode will return null.

38.1.

Pseudo DOM API

618

Browser isBrowserIE() Test whether the browser to which the node belongs is an Internet Explorer. Returns True for Internet Explorer, false for Mozilla derivatives. Browser isBrowserMozilla() Test whether the browser to which the node belongs is a Mozilla derivative. Returns True for Mozilla derivatives, false for Internet Explorer. Node getParent() Get the parent of a node. A top-level DocumentNode or a DialogNode will return null. Everything else should have a parent unless removed from the DOM via Javascript. Returns The parent of the node. int getChildCount() Get the number of child nodes. Returns The number of child nodes. Node getChild(int index) Get the child node at the given index. Parameters index The index of the child node, starting with 0. Returns The child node at the index. Throws IllegalArgumentException If index is negative or exceeds the number of child nodes. Node[] getChildren() Get the children of this node as an array. Returns The child nodes. int getIndexOfChild(Node child) Get the index of a given child node. Parameters child The child node to get the index for. Returns The index of the child node, starting with 0, or -1 in case it is not a child of the this node.

38.1.

Pseudo DOM API

619

Node[] getElementsByTagName(String tagName) Get all direct or indirect child nodes of this node with a given tag name. Parameters tagName The tag name to look for. Returns An array of child nodes with the given tag name. If none are found, an empty array is returned. A DialogNode will always return an empty array, a FrameNode forwards the call to its DocumentNode child node and a DocumentNode to its <HTML> root element. Node getElementById(String id) Get a direct or indirect child node of this node with a given ID. The given ID is not compared to the nodes original id attribute, but to its ID, which may have been modied (637) by an IdResolver (see section 39.1.4 ). Parameters id The ID to look for. Returns The child node with the given ID, an arbitrary one in case of multiple matches or null if none is found. A DialogNode will always return null, a FrameNode forwards the call to its DocumentNode child node and a DocumentNode to its <HTML> root element. Node[] getElementsById(String id) Get all direct or indirect child nodes of this node with a given ID. The given ID is not compared to the nodes original id attribute, but to its ID, which may have been modied (637) by an IdResolver (see section 39.1.4 ). Parameters id The ID to look for. Returns An array of child nodes with the given ID. If none are found, an empty array is returned. A DialogNode will always return an empty array, a FrameNode forwards the call to its DocumentNode child node and a DocumentNode to its <HTML> root element.

38.1.

Pseudo DOM API

620

Node[] getElementsByIdAndTagname(String id, String tagName) Get all direct or indirect child nodes of this node with a given ID attribute and a given tag name. Parameters id The ID attribute to look for. tagName The tag name to look for. Returns An array of child nodes with the given ID attribute and the given tag name. If none are found, an empty array is returned. A DialogNode will always return an empty array, a FrameNode forwards the call to its DocumentNode child node and a DocumentNode to its <HTML> root element. Object getProperty(String name) Retrieve a user-dened property. Parameters name The name of the property to retrieve. Returns The value of the specied property or null. setProperty(String name, Object value) Set a user-dened property. Parameters name The name of the property. value The property value or null to remove an existing property.

38.1.2

The DocumentNode class

The DocumentNode class is derived from Node and also resides in the package de.qfs.apps.qftest.client.web.dom. In addition to the methods dened in the Node class and explained above, DocumentNode provides the following: boolean hasParent() To test whether a document is a top-level document this method should be used instead of testing whether the result of getParent() is null. The reason is that loading of nested child documents may be completed before loading of the main document. During this time it is known that the nested document will have a parent, but the parent is not available yet. Returns True if the document has a parent, false if it is a top-level document.

38.1.

Pseudo DOM API

621

DomNode getRootElement() Get the <HTML> root element of the document. Returns The root element of the document. FrameNode[] getFrames() Get the child frames of the document. Returns The child frames of the document, an empty array in case there are none. String getUrl() Get the URL of the document. Returns The URL of the document. String getTitle() Get the title of the document as dened in the <TITLE> of the <HEAD> of its root element. Returns The title of the document. String getSourcecode() Get the HTML source code of the document in its current state, which is not necessarily the same as what was loaded when the document was opened because attributes or the DOMs structure may have been changes since, e.g. via Javascript. Returns The current HTML code of the document. Object evalJS(String script) Evaluate some Javascript code in the context of this document. Parameters script The script to execute. Returns Whatever the script returns, converted to the proper object type. Even returning a DOM node, frame or document works. In addition to the instance methods describe above, the DocumentNode class provides some static convenience methods for manipulating URLs. static String getUrlBase(String url) Get the base part of an URL, including host and directory but excluding possible parameters separated by ? and excluding a possible Tomcat session ID. Parameters url The URL to get the base part of. Returns The base part of the URL.

38.1.

Pseudo DOM API

622

static String getUrlHost(String url) Get the host part of an URL, i.e. the part between http(s):// and the next /. Parameters url The URL to get the host part of. Returns The host part of the URL or null if the protocol is neither http nor https. static String normalizeUrl(String url) Normalize an URL by replacing \ with /, le:/(/(/)) with le:///, up-casing windows drive letters and decoding %.. sequences. Parameters url The URL to normalize. Returns The normalized URL. static ExtraFeatureSet getUrlParameters(String url) Get the parameters of an URL, including those separated by ? and a possible Tomcat session ID, in the form of an ExtraFeatureSet that can be used for an (639) ExtraFeatureResolver as described in section 39.1.6 . Parameters url The URL to get the parameters for. Returns The parameters of the URL.

38.1.3

The FrameNode class

The FrameNode class is derived from Node and also resides in the package de.qfs.apps.qftest.client.web.dom. In addition to the methods dened in the Node class and explained above, FrameNode provides the following: DocumentNode getChildDocument() Get the child document of the frame. Returns The child document of the frame. String getFrameName() Get the name of the frame as dened by its name attribute. Returns The name of the frame. String getFrameUrl() Get the URL of the frame which should normally be the same as the URL of its child document. Returns The URL of the frame.

38.1.

Pseudo DOM API

623

int[] getGeometry() Get the location and size of the frame relative to its parent frame or the browser windows display area. Returns The frame geometry in the form [x, y, width, height].

38.1.4

The DomNode class

The DomNode class is derived from Node and also resides in the package de.qfs.apps.qftest.client.web.dom. In addition to the methods dened in the Node class and explained above, DomNode provides the following: getId() Get the cached id attribute of the node that may have been processed by an (637) IdResolver as described in section 39.1.4 . The original, unmodied id attribute is available via getAttribute(id). Returns The cached ID of the node. setAttribute(String name, String value) Set an attribute value on the node. Parameters name The name of the attribute. value The value to set. Node[] getChildrenByTagName(String tagName) Get all direct child nodes of this node with a given tag name. Parameters tagName The tag name to look for. Returns An array of child nodes with the given tag name. If none are found, an empty array is returned. boolean hasFocus() Test whether the node has the keyboard focus. Returns True if the node has the keyboard focus. requestFocus() Tell the node to request the keyboard focus. This is a best-effort implementation. Whether the node actually receives the keyboard focus depends on the browser and operating system settings. boolean isShowing() Test whether the node is visible or can be made visible by scrolling. Returns True if the node is visible.

38.1.

Pseudo DOM API

624

void scrollVisible() Try to scroll the page or a frame so as to make the node fully visible. This is a best effort implementation. Whether the node can actually be made visible depends on the browser and operating system settings. int[] getLocationOnScreen() Get the location and size of the node relative to desktop. Returns The nodes geometry in the form [x, y, width, height]. void toJS(String name) Set a Javascript variable in the context of this nodes document to this node. Parameters name The name of the variable to set. Object evalJS(String script) Evaluate some Javascript code in the context of this nodes document. Parameters script The script to execute. Returns Whatever the script returns, converted to the proper object type. Even returning a DOM node, frame or document works.

38.1.5

The DialogNode class

The DialogNode class, also derived from Node is not a standard DOM class, but created solely for convenient access to dialogs within QF-Test. It also resides in the package de.qfs.apps.qftest.client.web.dom but has very little to do with the other Node classes. A DialogNode represents a message or error dialog which can be triggered via Javascript. It provides the following methods: long getStyle() Get the style of the dialog. Returns

The style of the dialog. Any of the constants STYLE_ALERT, STYLE_CONFIRM or STYLE_AUTHENTICATE dened in the DialogNode class.

String getTitle() Get the title of the dialog. Returns

The title of the dialog.

38.2.

Special support for various AJAX toolkits

625

String getText() Get the message text of the dialog. Returns The message text of the dialog.

38.2

Special support for various AJAX toolkits

The acronym AJAX for Asynchronous Javascript and XML is generally associated with modern web applications that provide a more interactive look and feel than classical FORM based applications, almost comparable to desktop applications. Behind this common acronym there is a whole zoo of toolkits that drive those applications, each with different key aspects and unique widget sets. Such toolkits pose a problem for QF-Test and in fact any automated testing tool for several reasons: The actual component hierarchy is created automatically from abstract widgets like Buttons or Lists. Often each widget is implemented as a number of DIV nodes. This leads to very deeply nested hierarchies with very little structure. Ids are either not assigned at all or automatically created and thus worse than useless. The asynchronous communication with the web server and dynamic creation of DOM nodes may cause timing-related problems. There is no panacea to address these problems in a generic way. In most cases QFTest can interact with AJAX applications out-of-the-box, but component recognition and performance are not ideal. Optimal testability can only be achieved with special case handling that exactly ts a given toolkit and takes advantage of its peculiarities. For the following list of toolkits QF-Test provides resolvers for optimized handling:

38.2.

Special support for various AJAX toolkits

Tookit homepage http://www.extjs.com http://www.extjs.com http://www.gwtproject.org http://icesoft.org http://primefaces.org http://qooxdoo.org http://www.eclipse.org/rap/ http://www.jboss.org/richfaces http://vaadin.com http://zkoss.org

626

Toolkit name Ext GWT Ext JS Google Web Toolkit (GWT) ICEfaces Prime Faces Qooxdoo Rich Ajax Platform (RAP) RichFaces Vaadin ZK

Table 38.1: Supported AJAX toolkits

QF-Test is even able to automatically detect whether one of those toolkits is used in your web application and to install the respective resolver as described below.

38.2.1

AJAX resolver concepts

An AJAX resolver is a set of resolvers and other methods implemented specically for a given AJAX toolkit. Most notably QF-Test tries to assign individual classes matching the high-level widgets to DOM nodes and remove intermediate nodes that are just an implementation detail. Name, Feature and Extra feature attributes are determined in a way suitable for the toolkit and events are simulated on the correct DOM node in a way that most closely matches user interaction. These measures drastically reduce the component hierarchy and increase the reliability and performance of component recognition and replay. Timing and synchronization are also addressed. As a necessary consequence the components and events recorded for a given web application vary drastically with and without an active AJAX resolver and are not compatible with each other. Thus the decision whether or not to use an AJAX resolver should be made as early as possible, otherwise tests will either need to be reimplemented after activating the resolver or tests with and without resolver must be cleanly separated. If a resolver is available for your application you should practically always use it. The only exception is if the existing test-base is already too large, mostly complete and stable. Implementing AJAX resolvers is an ongoing process. As changes to a resolver can have consequences in terms of how components are recorded and recognized we need to make sure that you can update your QF-Test version without losing backwards compatibility with existing tests. To that end, AJAX resolvers have their own three-digit versioning scheme, starting with version 1.0.0, with the following meaning:

38.2.

Special support for various AJAX toolkits

627

A change in the minor version, e.g. 1.0.1, means that compatibility is not affected. Medium version changes, e.g. 1.1.0, imply that tests recorded with a lower medium version should still replay, but recording additional sequences may record components in a different way. This is not critical, but you may want to avoid it in case the benets of the newer version dont outweigh it. A major version change, e.g. 2.0.0, can be incompatible even for replay so that manual updates or new recording of the affected components may be required. AJAX resolvers are activated via the Procedure qfs.web.ajax.installToolkitResolver in the standard library qfs.qft where you have to provide the version to use. You can choose to specify only the major version, in which case QF-Test uses the latest medium.minor version available for this major version. This is normally the best option and used in the SUT startup sequences (22) created with QF-Tests quickstart wizard (see chapter 3 ). Alternatively you can specify major.medium version or even major.medium.minor to use an exact version and thus run your tests with the resolver version with which they were created.

38.2.2

Supported AJAX toolkits

Unfortunately the number of available AJAX toolkits keeps growing rapidly and popularity changes frequently so we have to weigh carefully which toolkit to support and to which degree. Cooperation with QF-Test users also plays a major role. Resolver development may also be frozen - except for critical bug xes - if a toolkit loses relevance. This section lists the supported AJAX toolkits and versions with the available resolver versions. Newer toolkit versions dont necessarily require an update of the resolver. If your application uses a newer version of the toolkit than the supported versions listed below, please use the latest AJAX resolver available for this toolkit and let us know about your results. Google Widget Toolkit - GWT The Google Widget Toolkit has been supported starting with GWT version 1.5. A GWT demo test-suite, tested against GWT version 2.3, is provided in the le .../demo/web/gwt.qft. This demo makes use of generic components and provides several useful procedures for working with some common GWT widgets.

38.2.

Special support for various AJAX toolkits

QF-Test version 3.4.2 3.4.1 3.4.0 3.3.0 3.1.4 3.1.1

628

Resolver version 1.4.0 1.3.0 1.2.0 1.1.0 1.0.1 1.0.0

Remarks Fixed ltering of DIV nodes Added support for GWT-TABLAYOUTPANEL and GWTTABLAYOUTPANELTAB Filtering of DIV nodes, improved extra features, limited handling for obfuscated GWT class names Improved support for GWT-TREEITEM, GWTSTACKPANELITEM and GWT-TABBARITEM Replay improvements for GWT 2.0 Initial version

Table 38.2: GWT resolver versions

Ext JS The resolver was initially created for Ext JS version 2 but does support Ext JS version 4 in the meantime.
Resolver version 2.0.0 1.2.0 1.1.0 1.0.0 QF-Test version 3.5M1 3.4.1 3.3.0 3.1.1 Remarks Support for Ext JS version 4.1, generic classes and sub-items Support for Ext JS version 4 Improved item support for trees, grids, tab panels and combo lists Initial version

Table 38.3: Ext JS resolver versions

Ext GWT There is no separate Ext GWT resolver. Ext GWT is in fact a combination of Ext JS and GWT and QF-Test supports it by installing both the Ext JS and the GWT resolver. ICEfaces The resolver was developed against ICEfaces version 3.3.

38.2.

Special support for various AJAX toolkits

QF-Test version 3.5.3

629

Resolver version 1.0.0

Remarks Initial version with support for generic classes and sub-items based on the showcase demo, including support for ACE components

Table 38.4: ICEfaces resolver versions

jQuery UI jQuery UI appears to be a rising star among current AJAX toolkits. A resolver for jQuery UI is under development. PrimeFaces The resolver was developed against PrimeFaces version 3.5.
Resolver version 1.0.0 QF-Test version 3.5.1 Remarks Initial version with support for generic classes and sub-items based on the showcase demo

Table 38.5: PrimeFaces resolver versions

Qooxdoo The resolver was developed against qooxdoo version 0.7.2. Currently not under active development.
Resolver version 1.1.0 1.0.0 QF-Test version 3.3.0 3.1.1 Remarks Improved support for list and menu items Initial version

Table 38.6: Qooxdoo resolver versions

Remote Application Platform - RAP The resolver was developed against RAP version 1. Support for RAP version 2 and higher is complete.

38.2.

Special support for various AJAX toolkits

QF-Test version 3.5.1 3.5M1 3.3.0 3.3.0 3.1.1

630

Resolver version 3.0.0 2.0.0 1.1.0 1.0.1 1.0.0

Remarks Updated for RAP Version > 2.0 (Remote Application Platform) Support for generic classes and sub-items. RAP Version 1.5 (Rich Ajax Platform) Improved item support for lists, tab folders, tables and trees Improved widget recognition Initial version

Table 38.7: RAP resolver versions

RichFaces The resolver was developed against RichFaces versions 3 and 4.

Resolver version 2.0.2 2.0.1 2.0.0 1.4.0 1.3.0 QF-Test version 3.5.2 3.5M3 3.5M1 3.5.0 3.4.8 Remarks Fixes for replaying clicks on Windows 7 systems. Recording and replaying of le:upload elements works correctly now. Minor xes Support for generic classes and sub-items Supports items for trees, lists, tabs and tables for RichFaces 3 and RichFaces 4. Replaying menu selections on ddm-itms as hard events. Fixed wrong event ltering for rich-tree components. Several minor xes for abstract coordinates and replay. Tested with RichFaces v3.3.3 and v4.2.2. Supports RichFaces 4 Major rewrite Initial version

1.2.0 1.1.0 1.0.0

3.4.1 3.3.0 3.1.4

Table 38.8: RichFaces resolver versions

Vaadin Vaadin is another toolkit that is based on GWT but comes with additional features and its own widget set.

38.2.

Special support for various AJAX toolkits

QF-Test version 3.5.0 3.5M1

631

Resolver version 1.1.0 1.0.0

Remarks Updated for Vaadin 7 based on the dashboard Vaadin demo Initial version with support for generic classes and sub-items

Table 38.9: Vaadin resolver versions

ZK The resolver was developed for version 6.5.

Resolver version 1.0.1 1.0.0 QF-Test version 3.5.2 3.5.1 Remarks Minor improvements on component recognition and better ltering of auto generated ZK ids. Initial version with support for generic classes, sub-items and checks.

Table 38.10: ZK resolver versions

38.2.3

Setting unique Ids

Any AJAX toolkit has its custom way of setting unique ids. Please nd details about the supported ones in the following chapter: Ext JS You can set ids like var container = Ext.create(Ext.container.Container, { id: MyContainerId, ... });. As alternative you can also call container.getEl().set({ qfs-id: myId });. In this case you will need to implement a NameResolver for reading qfs-id as name for QF-Test. GWT The simplest way is calling the method widget.getElement().setId(myId); for the required widgets.

38.2.

Special support for various AJAX toolkits

632

As an alternative you can also call widget.ensureDebugId(myId). But if you want to use that method you need to modify your xxx.gwt.xml le to enable debug ids. Add <inherits name=com.google.gwt.user.Debug/> to the le. Its also possible to set a custom id, which you need to read via a NameResolver later. Therefore call setAttribute(qfs-id, myId). ICEfaces The simplest solution is to set the id attribute <p:inputText id=myId/> for any required component in the xhtml denition. PrimeFaces The simplest solution is to set the id attribute <p:inputText id=myId/> for any required component in the xhtml denition. RAP Call the method widget.setData(WidgetUtil.CUSTOM_WIDGET_ID, myId); for the required widgets. After applying ids to components you need to modify your webserver environment and specify following parameter -Dorg.eclipse.rwt.enableUITests=true before launching the webserver. RichFaces You need to set the id attribute in your source code or graphical editor. Vaadin The simplest solution is to call widget.setId(id) (widget.setDebugId(id) for Vaadin version < 7) for any required component. You can also set a custom stylesheet class, which you could read with a NameResolver. Therefore call widget.setStyleName(qfs-id=myId).

38.2. ZK

Special support for various AJAX toolkits

633

The simplest solution is setting the id attribute <button id=myId/> for any required component in the xml denition or calling button.setId(myId); in the Java code.

Chapter 39 Extension APIs

QF-Test provides some extension APIs that let you extend its functionality. The interfaces can conveniently be implemented in Jython, Groovy or or Java. In the latter case, the compiled classes should be put in a jar le and placed in the plugin directory (see (587) section 37.2 ) so Jython or Groovy can be used to glue things together.

39.1
Note

Resolver hooks

Resolvers have quite a history in QF-Test. In version 2.0 SWT support was added to QF-Test. The ComponentNameResolver interface of previous QF-Test versions had been built for AWT/Swing Components and was not portable. It was replaced with the more generic NameResolver interface and associated ResolverRegistry methods. With QF-Test version 3.1, both NameResolver and FeatureResolver were replaced with more elegant versions named - for lack of alternatives - NameResolver2 and FeatureResolver2. These modern versions pass the default name or feature determined by QF-Test directly to the resolvers method which greatly simplies implementation of the interface. Similarly, the old ItemNameResolver that was only available for Swing was replaced with the new ItemNameResolver2 with a matching ItemValueResolver2, which work for all engines. For backwards compatibility all the old interfaces and methods are still available and will continue to work, but they should no longer be used in new tests and are no longer documented here.

39.1.

Resolver hooks

635

39.1.1

Overview over the supported kinds of resolvers

This extension API lets you install hooks that can modify the way QF-Test recognizes and records components and items. This is a very powerful feature that gives you negrained control over QF-Tests component management. It is far less complicated than (644) it looks. Using the resolvers module described in section 39.1.10 , a full featured resolver can be implemented and installed with a minimum of code. The following kinds of hooks are currently provided: A ClassNameResolver can change the class that QF-Test records for a component. For Swing and SWT its use is rather limited, because it is only allowed to return an actual base class of the components class. However, for web applications this is perhaps the most important of all resolvers, because it can assign arbitrary custom class names to DOM nodes, thus turning deeply nested hierar(39) chies of DIV nodes into something useful. See also section 5.4 about the pseudo class hierarchy for web applications. A NameResolver2 can change (or provide) the name of a GUI element as set with setName() for AWT/Swing, setData(name, ...) for SWT or via the id attribute of a DOM node for web applications. This can be very useful when setting names in the source code is not an option, like for third-party code or when child components of complex components are not readily accessible. For example, QFTest provides a name resolver for Swings JFileChooser dialog which you can read more about in the Tutorials chapter about the standard library.
Web

An IdResolver can change, suppress or provide the id attribute of a DOM node at an early stage which makes it very efcient. A FeatureResolver2 can provide a feature for a GUI element. An ExtraFeatureResolver can provide extra features for a GUI element. An ItemNameResolver2 can change (or provide) the textual representation of the index for addressing a sub-item of a complex component. An ItemValueResolver2 can change (or provide) the textual representation of the value of a sub-item of a complex component as used for a Check text or retrieved via a Fetch text node.

Swing

A TreeTableResolver helps QF-Test recognize TreeTable components. A TreeTable is a mixture between a table and a tree. It is not a standard Swing component, but most TreeTables are implemented similarly by using a tree as the renderer component for one column of the table. Once QF-Test recognizes a TreeTable as such, it treats the row indexes of all table cells as tree indexes which is a lot more useful in that context than standard table row indexes. In addition,

39.1.

Resolver hooks

636

geometry information for cells in the tree column is based on tree nodes instead of table cells. Multiple resolvers with different tasks can be created and registered at run-time. The respective interfaces can be implemented directly by Java classes. However, this would make the code of your application depend on QF-Test classes. The preferred alternative is to implement the resolver interfaces in Jython or Groovy, ideally via the Jython resolvers module or the Groovy Resolvers class. That way the whole mechanism can be strictly separated from the SUT and will not interfere with the build process. Before we explain how to (un-)register resolvers, well take a look at their interfaces.

3.1+

39.1.2

The ClassNameResolver interface

The de.qfs.apps.qftest.extensions.ClassNameResolver interface consists of a single method: String getClassName(Object element, String name) Determine the name of the class of a GUI element. Parameters element The element to determine the class name for. name The original class name that QF-Test would use without a resolver. Returns The class name to use or null if the element is not handled by this resolver. After QF-Test determines the class name of a GUI element the registered ClassNameResolvers get a chance to override this class name. The rst resolver that returns a non-null value determines the outcome. If no resolvers are registered or all of them return null the original class name is used. For Swing and SWT applications classes are matched based on actual class hierarchy, so the ClassNameResolver is restricted to the actual class of the element or any of its base classes. It can still be useful in case you dont want to activate the option Record (242) system class only but still record a system class or some other general base class for certain components. Swing and SWT classes are not cached so the resolver is called every time QF-Test needs the class of an element. For web applications QF-Test uses a pseudo class hierarchy which is described in (39) section 5.4 . The resolver is free to return any arbitrary class name, which QF-Test then treats like a sub-class of the elements pseudo class. For performance reasons classes are cached for web elements so the resolver will only be called at most once for each element.

39.1.

Resolver hooks

637

3.1+

39.1.3

The NameResolver2 interface

The de.qfs.apps.qftest.extensions.NameResolver2 interface consists of a single method: String getName(Object element, String name) Determine the name of a GUI element. Parameters element The element to determine the name for. name The original name that QF-Test would use without a resolver. Returns The name to use or null if the element is not handled by this resolver. Returning an empty string suppresses the original name. After QF-Test determines the name of a GUI element the registered NameResolvers get a chance to override or suppress this name. The rst resolver that returns a non-null value determines the outcome. If no resolvers are registered or all of them return null the original name is used. In some cases it may be desirable to suppress an elements name, for example for names which are not unique or which - even worse - vary depending on the situation. To do so, getName should return the empty string.

Web

39.1.4

The IdResolver interface

The de.qfs.apps.qftest.extensions.IdResolver interface consists of a single method:

39.1.

Resolver hooks

638

String getId(DomNode node, String id) Determine the ID of a DomNode. The resolved ID will be cached and can later be retrieved via node.getId(), whereas node.getAttribute(id) always returns the original, unmodied ID. Parameters node The DomNode to determine the ID for. id The id attribute that QF-Test has determined, possibly with suppressed numerals, depending on the setting of (270) the option Eliminate all numerals from id attributes . To implement the resolver based on the original id attribute, simply fetch this with node.getAttribute(id). Returns The ID or null if the element is not handled by this resolver. Returning an empty string will suppress or hide the nodes actual ID. When QF-Test registers the nodes of a web pages DOM it also caches the id attribute of those nodes. Since many web applications are based on automatically generated IDs, some processing is often required, similar to the suppression of numerals that QF-Test does automatically, depending on the option Eliminate all numerals from id (270) attributes . The id attribute of a node can show up in multiple places, most notably as the nodes (516) Name (depending on the option Turn id attribute into name where unique (269) (516) (516) enough ), its Feature and an Extra feature . Thus it is much more efcient to implement an IdResolver to modify the ID once, than to implement separate Name-, Feature- and ExtraFeatureResolvers. More importantly, changing a nodes id attribute can have a major impact on whether the attribute is unique and QF-Tests mechanism for using an ID as a Name takes uniqueness into account, so an IdResolver is allowed to return non-unique IDs whereas a NameResolver2 is not.

3.1+

39.1.5

The FeatureResolver2 interface

The de.qfs.apps.qftest.extensions.FeatureResolver2 interface consists of a single method:

39.1.

Resolver hooks

639

String getFeature(Object element, String feature) Determine the feature of a GUI element. Parameters element The element to determine the feature for. feature The original feature that QF-Test would use without a resolver. Returns The feature to use or null if the element is not handled by this resolver. Returning an empty string suppresses the original feature. After QF-Test determines the feature of a GUI element the registered FeatureResolvers get a chance to override or suppress this feature. The rst resolver that returns a non-null value determines the outcome. If no resolvers are registered or all of them return null the original feature is used. To suppress an elements feature, getFeature should return the empty string.

39.1.6

The ExtraFeatureResolver interface

The de.qfs.apps.qftest.extensions.ExtraFeatureResolver interface consists of a single method: ExtraFeatureSet getExtraFeatures(Object element, ExtraFeatureSet features) Determine extra features for a GUI element. Parameters element The element to determine the extra features for. features The extra features determined by QF-Test itself, an empty set in case there are none. These can be modied in place or ignored and a different ExtraFeatureSet returned. Returns The original features modied in place or a different set, null if the element is not handled by this resolver. To suppress all the elements original extra features return an empty ExtraFeatureSet. After QF-Test has determined the extra features for a GUI element, the registered ExtraFeatureResolvers will get a chance to override these features. In contrast to other resolvers, QF-Test does not stop when the rst resolver returns a non-null value. Instead it passes its result as input to the next resolver which makes it possible to register several ExtraFeatureResolvers that handle different extra features.

39.1.

Resolver hooks

640

If no resolvers are registered or all of them return null, QF-Test will proceed to use the original set. Of course, in order to be able to implement the getExtraFeatures method properly, you need to know the details for the API of the classes involved, namely ExtraFeature and ExtraFeatureSet. These are explained below. For an example implementation (647) please see example 39.6 . Instances of the class de.qfs.apps.qftest.shared.data.ExtraFeature represent one extra feature for a GUI element, comprising its name and value along with information about whether the feature is expected to match, whether it is a regular expression and whether the match should be negated. For possible states the class denes the constants STATE_IGNORE, STATE_SHOULD_MATCH and STATE_MUST_MATCH. ExtraFeature (String name, String value) Create a new ExtraFeature with the default state STATE_IGNORE. Parameters name The name of the ExtraFeature. value The value of the ExtraFeature. ExtraFeature (int state, String name, String value) Create a new ExtraFeature with a given state. Parameters state The state of the ExtraFeature. Possible values are STATE_IGNORE, STATE_SHOULD_MATCH, STATE_MUST_MATCH. name The name of the ExtraFeature. value The value of the ExtraFeature. int getState() Get the state of the ExtraFeature. Returns The state of the ExtraFeature. void setState(int state) Set the state of the ExtraFeature. Parameters state The state to set. String getName() Get the name of the ExtraFeature. Returns The name of the ExtraFeature.

39.1.

Resolver hooks

641

void setName(String name) Set the name of the ExtraFeature. Parameters name The name to set. String getValue() Get the value of the ExtraFeature. Returns The value of the ExtraFeature. void setValue(String value) Set the value of the ExtraFeature. Parameters value The value to set. boolean getRegexp() Get the regexp state of the ExtraFeature. Returns Whether the ExtraFeatures value is a regular expression. void setRegexp(boolean regexp) Set the regexp state of the ExtraFeature. Parameters regexp The regexp state to set. boolean getNegate() Get the negate state of the ExtraFeature. Returns The negate state of the ExtraFeature. void setNegate(boolean negate) Set the negate state of the ExtraFeature. Parameters negate The negate state to set. The class de.qfs.apps.qftest.shared.data.ExtraFeatureSet ExtraFeatures into set: ExtraFeatureSet () Create a new, empty ExtraFeatureSet. void add(ExtraFeature extraFeature) Add an extra feature to the set, potentially replacing a feature with the same name. Parameters extraFeature The extra feature to add. collects

39.1.

Resolver hooks

642

ExtraFeature get(String name) Get an extra feature from the set. Parameters name The name of the extra feature to get. Returns The extra feature or null if no feature by that name is stored in the set. ExtraFeature remove(String name) Remove an extra feature from the set. Parameters name The name of the extra feature to remove. Returns The extra feature that was removed or null if no feature by that name was stored in the set. ExtraFeature[] toArray() Get all extra features in the set. Returns An array of the contained extra features, sorted by name.

3.1+ Note

39.1.7

The ItemNameResolver2 interface

(644)

Unless you use the resolvers module described in section 39.1.10 , an ItemNameResolver2 must be registered with the ItemRegistry described in (665) section 39.2.4 and not the ResolverRegistry. The de.qfs.apps.qftest.extensions.items.ItemNameResolver2 interface consists of a single method: String getItemName(Object element, Object item, String name) Determine the name of an item of a complex GUI element that will be used for the textual representation of the items index. Parameters element The GUI element to which the item belongs. item The item to get the name for. Its type depends on the GUI element and the registered ItemResolvers as de(668) scribed in section 39.2.5 . name The original name that QF-Test would use without a resolver. Returns The name to use or null if the resolver does not handle this element or item. After QF-Test determines the name for an items index the registered

39.1.

Resolver hooks

643

ItemNameResolvers get a chance to override. The rst resolver that returns a non-null value determines the outcome. If no resolvers are registered or all of them return null the original name is used.

3.1+ Note

39.1.8

The ItemValueResolver2 interface

(644)

Unless you use the resolvers module described in section 39.1.10 , an ItemValueResolver2 must be registered with the ItemRegistry described in (665) section 39.2.4 and not the ResolverRegistry. The de.qfs.apps.qftest.extensions.items.ItemValueResolver2 interface consists of a single method: String getItemValue(Object element, Object item, String value) Determine the value of an item of a complex GUI element that will be used for its textual (443) (471) representation in a Check text or a Fetch text node. Parameters element The GUI element to which the item belongs. item The item to get the value for. Its type depends on the GUI element and the registered ItemResolvers as de(668) scribed in section 39.2.5 . value The original value that QF-Test would use without a resolver. Returns The value to use or null if the resolver does not handle this element or item. After QF-Test determines the value for an items index the registered ItemValueResolvers get a chance to override. The rst resolver that returns a non-null value determines the outcome. If no resolvers are registered or all of them return null the original value is used.

39.1.9
Note

The TreeTableResolver interface

This interface is irrelevant for SWT. Multi-column SWT trees are automatically supported by QF-Test. A de.qfs.apps.qftest.extensions.TreeTableResolver must implement the following two methods:

39.1.

Resolver hooks

644

JTree getTree(JTable table) Determine the tree component used to implement a TreeTable. Parameters table The JTable component to determine the tree for. Returns The tree or null if the JTable is a plain table and not a TreeTable. int getTreeColumn(JTable table) Determine the column index of the tree component in a TreeTable. Most implementations place the tree in the rst column, in which case the index is 0. Parameters table The JTable component to determine the trees column index for. Returns The column index or -1 if the JTable is a plain table and not a TreeTable. The column index must always be given in the tables model coordinates, not in view coordinates. Most TreeTableResolvers are trivial to implement. The following Jython example works well for the org.openide.explorer.view.TreeTable component used in the popular netBeans IDE, provided that the resolver is registered for the TreeTable class:
class TTResolver(TreeTableResolver): def getTree(self, table): return table.getCellRenderer(0,0) def getTreeColumn(self, table): return 0

Example 39.1: TreeTableResolver for the netBeans IDE.

39.1.10

Simplied resolver creation with the resolvers module

Resolvers of all kinds can be implemented in Java, Jython or Groovy and then registered with the ResolverRegistry. Before describing this relatively complex process in detail in the following section, well show some resolver examples implemented with the help of the resolvers module provided with QF-Test. Unless you want to understand how the resolvers module itself works or want to implement a resolver in Java, these examples should be sufcient to get you going and skip the following sections. Except for the more complex ItemResolver and Checker interfaces described in the following sections, which are not supported by the resolvers module, a resolver consists of a single method that needs to be implemented (one and a half in case of the TreeTableResolver). What makes resolvers difcult at Java level is the need for

39.1.

Resolver hooks

645

describing such a method as an interface that has to implemented by a class. Then an instance of that class has to be created and registered for use by QF-Test. If you dont get it right on rst try, that instance has to be deregistered before a new instance from a new class can be created and instance thereof registered, otherwise there may be interference between the two versions of the resolver. Add some code for error handling and youve got many times more glue code than actual esh. Fortunately, methods are rst level objects in Jython, as are Groovy closures. This and the fact that the glue code is practically the same for all resolvers make it possible to move the tedious parts out of the way into a module and concentrate on the actual task at hand. Following is an example of a NameResolver2 for JMenuItems that promotes the label of a menu item to a component name for menus that have no name of their own:
def getName(menuItem, name): if not name: return menuItem.getLabel() resolvers.addNameResolver2("menuItems", getName, "javax.swing.JMenuItem")

Example 39.2: Simple Jython NameResolver2 for JMenuItems

As you see it took only four lines of code to do that. The rst three lines dene a getName method for the resolver. The fourth line calls the addNameResolver2 function of the resolvers module which is always automatically available in Jython SUT script nodes. The three parameters are an identier for the resolver, the actual method and one or more optional targets, in this case the class name of the components for which we want to register the resolver. Give it a try. Copy the example above into an SUT script node and execute it. If your application is based on SWT instead of Swing, replace getLabel() with getText() and javax.swing.JMenuItem with org.eclipse.swt.MenuItem. Then record some menu actions into a new, empty test-suite. Youll nd that all recorded menu item components without their own name will now have names set according to their labels. If setName is not used in your application and the labels of menu items are more or less static while the structure of the items changes often, this could even be a very useful thing. To install the resolver permanently, just move the SUT script node right after the (411) Wait for client to connect node of your SUT startup sequence. The same can be done in Groovy via the resolvers object that is also always available and has the same API. It expects a closure to create a resolver from as follows:
(392)

39.1.

Resolver hooks

646

def getName = {menuItem, name -> if (name == null) { return menuItem.getLabel() } } resolvers.addNameResolver2("menuItems", getName, "javax.swing.JMenuItem")

Example 39.3: Simple Groovy NameResolver2 for JMenuItems

It is possible to register a resolver for multiple component classes at once:

def getName(com, name): return com.getLabel() resolvers.addNameResolver2("labels", getName, "javax.swing.JLabel", "javax.swing.AbstractButton")

Example 39.4: Registering a NameResolver2 for multiple target classes

The other resolvers work similarly. Following is an example for an ItemNameResolver2 that makes the ID of a JTable column available as an item index:
def getColumnId(table, item, name): if item[1] < 0: # whole column addressed id = table.getColumnModel().getColumn(item[1]).getIdentifier() if id: return str(id) resolvers.addItemNameResolver2("tableColumnId", getColumnId, "javax.swing.table.TableColumn")

Example 39.5: ItemNameResolver2 for JTable columns

FeatureResolvers and ExtraFeatureResolvers are handled just like NameResolvers, but ExtraFeatureResolvers are a bit more complex, though not overly so, as the following example shows:

39.1.

Resolver hooks

647

from de.qfs.apps.qftest.shared.data import ExtraFeature, ExtraFeatureSet def getExtraFeatures(node, features): features.add(ExtraFeature(ExtraFeature.STATE_SHOULD_MATCH, "myname", "myvalue")) return features resolvers.addExtraFeatureResolver("silly example", getExtraFeatures, "INPUT")

Example 39.6: ExtraFeatureResolver that adds an extra feature to all INPUT nodes

TreeTableResolvers for Swing TreeTables are created and registered in a slightly different way:
def getTree(table): return table.getTree() def getColumn(table): return 0 resolvers.addTreeTableResolver("treeTable", getTree, getColumn, "my.package.TreeTable")

Example 39.7: TreeTableResolver for Swing TreeTable with optional getColumn method

The above example shows a typical TreeTableResolver. Since almost all TreeTables display the tree in the rst column of the table, the getColumn method is optional. If none is provided a default implementation for the rst column is used:
def getTree(table): return table.getTree() resolvers.addTreeTableResolver("treeTable", getTree, None, "my.package.TreeTable")

Example 39.8: Simplied TreeTableResolver

If no dedicated getTree method is available, the cell renderer of the column containing the tree (typically 0) might work, as it is typically derived from JTree.
def getTree(table): return table.getCellRenderer(0,0) resolvers.addTreeTableResolver("treeTable", getTree, "my.package.TreeTable")

Example 39.9: Simple TreeTableResolver using getCellRenderer method

39.1.

Resolver hooks

648

Following is the complete API of the resolvers module: addNameResolver2(String name, Method method, Object target=None, ...) Register a NameResolver2 for the given target(s), based on the given method. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. method The method that implements the resolvers getName() method. target One or more optional targets to register the resolver for. Each can be any of the following: An individual component The fully qualied name of a class If no target is given a global resolver for all components is registered. addIdResolver(String name, Method method, Object target=None, ...) Register an IdResolver for the given target(s), based on the given method. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. method The method that implements the resolvers getId() method. target One or more optional targets to register the resolver for. Each can be any of the following: An individual component The fully qualied name of a class If no target is given a global resolver for all components is registered.

39.1.

Resolver hooks

649

addFeatureResolver2(String name, Method method, Object target=None, ...) Register a FeatureResolver2 for the given target(s), based on the given method. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. method The method that implements the resolvers getFeature() method. target One or more optional targets to register the resolver for. Each can be any of the following: An individual component The fully qualied name of a class If no target is given a global resolver for all components is registered. addExtraFeatureResolver(String name, Method method, Object target=None, ...) Register an ExtraFeatureResolver for the given target(s), based on the given method. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. method The method that implements the resolvers getExtraFeature() method. target One or more optional targets to register the resolver for. Each can be any of the following: An individual component The fully qualied name of a class If no target is given a global resolver for all components is registered.

39.1.

Resolver hooks

650

addItemNameResolver2(String name, Method method, Object target) Register an ItemNameResolver2 for the given target(s), based on the given method. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. method The method that implements the resolvers getItemName() method. target The target to register the resolver for. It can be any of the following: An individual component The fully qualied name of a class addItemValueResolver2(String name, Method method, Object target=None, ...) Register an ItemValueResolver2 for the given target(s), based on the given method. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. method The method that implements the resolvers getItemValue() method. target The target to register the resolver for. It can be any of the following: An individual component The fully qualied name of a class

39.1.

Resolver hooks

651

addTreeTableResolver(String name, Method getTable, Method getColumn=None, Object target=None) Register a TreeTableResolver for the given target(s), based on the given methods. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. getTable The method that implements the resolvers getTable() method. getColumn The method that implements the resolvers getColumn() method. target One optional target to register the resolver for. It can be any of the following: An individual component The fully qualied name of a class If no target is given a global resolver for all components is registered. addClassNameResolver(String name, Method method, Object target=None, ...) Register a ClassNameResolver for the given target(s), based on the given method. If another resolver was previously registered under the given name, unregister that rst. Parameters name The name under which to register the resolver. method The method that implements the resolvers getClassName() method. target One or more optional targets to register the resolver for. Each can be any of the following: An individual component The fully qualied name of a class If no target is given a global resolver for all components is registered. removeResolver(String name) Unregister a resolver from all the targets it was registered to. Parameters name The name under which the resolver was registered.

39.1.

Resolver hooks

652

39.1.11

The ResolverRegistry

Now it is time to take a look at the Java API for registering resolvers on which the resolvers module is built. The singleton class de.qfs.apps.qftest.extensions.ResolverRegistry is the central agent for registering and removing name resolvers. All kinds of resolvers can be registered on individual components or on classes. Resolvers registered on a component will only be called if their service is required for that specic component. Resolvers registered on a class will be called for every object of that class or a class derived from it. NameResolvers, FeatureResolvers and TreeTableResolvers can also be registered globally so they will be called for each and every name or feature. This is similar to but more efcient than registering them on the java.lang.Object class. If multiple resolvers are registered globally or registered on the same object or class, the resolver added last will be called rst. The ResolverRegistry API is pretty straightforward: static ResolverRegistry instance() There can only ever be one ResolverRegistry object and this is the method to get hold of this singleton instance. Returns The singleton ResolverRegistry instance. static String getElementName(Object element) This static method should be used instead of com.getName() or widget.getData() by resolvers that need to operate based on existing names of elements, except for a NameResolver2 that gets just the original name passed to its getName method. This method suppresses trivial or AWT/Swing specic component names that QF-Test handles specially. Parameters element The element to get the name for. Returns The name of the element or null if a trivial or special name is suppressed. static boolean isInstance(Object object, String className) This static method should be used instead of instanceof (or isinstance() in Jython). It checks whether an object is an instance of a given class. The check is performed by name instead of through reection, so conicts with differing class loaders are prevented and there is no need to import the class to check against. Parameters object The object to check. className The name of the class to test for. Returns True if the object is an instance of the given class.

39.1.

Resolver hooks

653

void registerNameResolver2(Object element, NameResolver2 resolver) Register a name resolver for an individual GUI element. The resolver does not prevent garbage collection and will be removed automatically when the element becomes unreachable. Parameters element The GUI element to register for. resolver The resolver to register. void unregisterNameResolver2(Object element, NameResolver2 resolver) Unregister a name resolver for an individual GUI element. Parameters element The GUI element to unregister for. resolver The resolver to unregister. void registerNameResolver2(String clazz, NameResolver2 resolver) Register a name resolver for a GUI element class. Parameters clazz The name of the class to register for. resolver The resolver to register. void unregisterNameResolver2(String clazz, NameResolver2 resolver) Unregister a name resolver for a GUI element class. Parameters clazz The name of the class to unregister for. resolver The resolver to unregister. void registerNameResolver2(NameResolver2 resolver) Register a generic or global name resolver. Parameters resolver The resolver to register. void unregisterNameResolver2(NameResolver2 resolver) Unregister a generic or global name resolver. Parameters resolver The resolver to unregister.

39.1.

Resolver hooks

654

void registerIdResolver(Object element, IdResolver resolver) Register an id resolver for an individual GUI element. The resolver does not prevent garbage collection and will be removed automatically when the element becomes unreachable. Parameters element The GUI element to register for. resolver The resolver to register. void unregisterIdResolver(Object element, IdResolver resolver) Unregister an id resolver for an individual GUI element. Parameters element The GUI element to unregister for. resolver The resolver to unregister. void registerIdResolver(String clazz, IdResolver resolver) Register an id resolver for a GUI element class. Parameters clazz The name of the class to register for. resolver The resolver to register. void unregisterIdResolver(String clazz, IdResolver resolver) Unregister an id resolver for a GUI element class. Parameters clazz The name of the class to unregister for. resolver The resolver to unregister. void registerIdResolver(IdResolver resolver) Register a generic or global id resolver. Parameters resolver The resolver to register. void unregisterIdResolver(IdResolver resolver) Unregister a generic or global id resolver. Parameters resolver The resolver to unregister.

39.1.

Resolver hooks

655

void registerFeatureResolver2(Object element, FeatureResolver2 resolver) Register a feature resolver for an individual GUI element. The resolver does not prevent garbage collection and will be removed automatically when the element becomes unreachable. Parameters element The GUI element to register for. resolver The resolver to register. void unregisterFeatureResolver2(Object element, FeatureResolver2 resolver) Unregister a feature resolver for an individual GUI element. Parameters element The GUI element to unregister for. resolver The resolver to unregister. void registerFeatureResolver2(String clazz, FeatureResolver2 resolver) Register a feature resolver for a GUI element class. Parameters clazz The name of the class to register for. resolver The resolver to register. void unregisterFeatureResolver2(String clazz, FeatureResolver2 resolver) Unregister a feature resolver for a GUI element class. Parameters clazz The name of the class to unregister for. resolver The resolver to unregister. void registerFeatureResolver2(FeatureResolver2 resolver) Register a generic or global feature resolver. Parameters resolver The resolver to register. void unregisterFeatureResolver2(FeatureResolver2 resolver) Unregister a generic or global feature resolver. Parameters resolver The resolver to unregister.

39.1.

Resolver hooks

656

void registerExtraFeatureResolver(Object element, ExtraFeatureResolver resolver) Register an extra feature resolver for an individual GUI element. The resolver does not prevent garbage collection and will be removed automatically when the element becomes unreachable. Parameters element The GUI element to register for. resolver The resolver to register. void unregisterExtraFeatureResolver(Object element, ExtraFeatureResolver resolver) Unregister an extra feature resolver for an individual GUI element. Parameters element The GUI element to unregister for. resolver The resolver to unregister. void registerExtraFeatureResolver(String clazz, ExtraFeatureResolver resolver) Register an extra feature resolver for a GUI element class. Parameters clazz The name of the class to register for. resolver The resolver to register. void unregisterExtraFeatureResolver(String clazz, ExtraFeatureResolver resolver) Unregister an extra feature resolver for a GUI element class. Parameters clazz The name of the class to unregister for. resolver The resolver to unregister. void registerExtraFeatureResolver(ExtraFeatureResolver resolver) Register a generic or global extra feature resolver. Parameters resolver The resolver to register. void unregisterExtraFeatureResolver(ExtraFeatureResolver resolver) Unregister a generic or global extra feature resolver. Parameters resolver The resolver to unregister.

39.1.

Resolver hooks

657

void registerTreeTableResolver(Component com, TreeTableResolver resolver) Register a TreeTable resolver for a specic component. The resolver will be removed automatically if the component becomes invisible. Parameters com The component to register for. resolver The resolver to register. void unregisterTreeTableResolver(Component com, TreeTableResolver resolver) Unregister a TreeTable resolver for a specic component. Parameters com The component to unregister for. resolver The resolver to unregister. void registerTreeTableResolver(String clazz, TreeTableResolver resolver) Register a TreeTable resolver for a component class. Parameters clazz The name of the class to register for. resolver The resolver to register. void unregisterTreeTableResolver(String clazz, TreeTableResolver resolver) Unregister a TreeTable resolver for a component class. Parameters clazz The name of the class to unregister for. resolver The resolver to unregister. void registerTreeTableResolver(TreeTableResolver resolver) Register a generic or global TreeTable resolver. Parameters resolver The resolver to register. void unregisterTreeTableResolver(TreeTableResolver resolver) Unregister a generic or global TreeTable resolver. Parameters resolver The resolver to unregister.

39.2.

Implementing custom item types with the ItemResolver interface

658

void unregisterResolvers(Object element) Unregister all resolvers for a specic GUI element. Parameters element The element to unregister for. void unregisterResolvers(String clazz) Unregister all resolvers for the class of a GUI element, value or renderer. Parameters clazz The name of the class to unregister for.

39.1.12

Error handling

All exceptions thrown inside a name resolver will be caught and handled by the ResolverRegistry. However, instead of dumping a stack trace, the registry will only print a short message like Exception inside NameResolver2 because some resolvers may be called very often, so for a buggy resolver printing a stack trace for every error could ood the net and the client terminal. Therefore name resolvers should include their own error handling. This can still generate a lot of output in some cases, but especially for scripts the output will be more useful than a Java stack trace.

39.2
3.1+

Implementing custom item types with the ItemResolver interface

(48)

As described in chapter 6 , QF-Test is able to dig below the given structure of GUI elements and work with sub-items that are not GUI components themselves, like the cells of a table, nodes in a tree or drawings on a canvas. Such items are implemented via the ItemResolver mechanism which can be used to implement your own custom items. The ItemResolver interface is more complex than the simple NameResolver2 or FeatureResolver2 described in the previous section and cannot be implemented without solid programming skills and a thorough understanding of the underlying concepts. Also, ItemResolvers and Checkers, described in the next section, are closely related and need to be implemented together if you want to be able to perform checks for your items. On the upside the whole mechanism is very powerful and once implemented and registered, your ItemResolvers will integrate smoothly with QF-Test so that there is no distinction between standard and custom items, so dont let yourself be deterred.

39.2.

Implementing custom item types with the ItemResolver interface

659

39.2.1 ItemResolver concepts

Before you start implementing an ItemResolver you need to determine the kinds of items that your GUI element might hold. There might be more than one kind, though the decision is arbitrary. For example, we implemented items for a Swing or SWT table so that an item can either be a table cell or a complete column. The latter is useful because it makes it possible to implement a check that checks a whole table column at once. Next you need to decide how to represent your items internally. You can use any kind of Object because QF-Test doesnt ever examine your internal representation itself, it just passes it to the methods of your ItemResolver. What works best depends on the API of the GUI element. If it already comes with its own concept for sub-items it may be best to reuse those classes. The most important decision to make is how to represent the item to the user of QF-Test. (48) As described in chapter 6 , the user can address an item via a numerical index or a textual one which can also be matched by a regular expression. You need to be able to provide a two-way mapping between an item and its index(es), i.e. you need to be able to answer the following two questions: Given an item, what is its numerical and its textual index? Given a numerical or textual index, which item matches that index best? The issues involved in naming sub-items are the same as those for setting component (40) (42) names. Please take a thorough look at section 5.6 and section 5.7 before continuing. An single numerical or textual index is represented by a SubItemIndex object. The current item concept supports addressing an item like a table cell via a primary and a secondary index, but in the future we hope to support indexes to any depth, so instead of using a path for a tree node it could be addressed with a mixed-type index in a form like tree@Root&1%Name:.*. Therefore the complete index is represented as an array of SubItemIndex objects, though currently limited to single or two-element arrays. Most items have a geometry, i.e. a location and a size. The coordinates for an item are always calculated relative to the true upper left origin of the element, regardless of whether it is scrolled, so they are independent of the current scroll position of the element. For items where geometry is not applicable or cannot be determined, coordinates can be ignored and the methods getItemLocation and getItemSize should simply return [0,0].

39.2.

Implementing custom item types with the ItemResolver interface

660

39.2.2

The ItemResolver interface

The methods of the interface de.qfs.apps.qftest.extensions.items.ItemResolver fall into three categories: Retrieving an item, mapping between an item and its index and retrieving miscellaneous information for - or performing actions on - an item. Object getItem(Object element, int x, int y) Get a sub-item for a GUI element at a given location. For items where geometry is not applicable or cannot be determined, the coordinates can be ignored and the item determined based on some other means like selection. Parameters element The GUI element to get the item for. x The X coordinate relative to element. y The Y coordinate relative to element. Returns An arbitrary object representing an item or null if there is no item at the given location. SubItemIndex[] getItemIndex(Object element, Object item, int type) Get the SubItemIndex(es) for a sub-item of a GUI element. Parameters element The GUI element to which the item belongs. item The item to get the index for. type The type of index to get. Possible values are INTELLIGENT, AS_STRING and AS_NUMBER, all dened in the SubItemIndex class. Unless only one kind of index is supported, a textual index should be returned for AS_STRING and a numerical index for AS_NUMBER. If the type is INTELLIGENT you are free to return whatever best represents the given item, even a mixed index like a column title and a row index for a table cell. Returns An array of SutItemIndex objects. Currently only single or two-element arrays are allowed.

39.2.

Implementing custom item types with the ItemResolver interface

661

Object getItemForIndex(Object element, SubItemIndex[] idx) Get an item for a given sub-item index. Parameters element The GUI element to get the item for. idx The sub-item index(es) for the item. Returns The item that best matches the given index. Throws IndexNotFoundExceptionIf no item matches the given index. Use the constructor de.qfs.apps.qftest.shared.exceptions.IndexNotFoundEx for this case. String getItemValue(Object element, Object item) Get the value of a sub-item to be used for a text check. Parameters element The GUI element to which the item belongs. item The item whose value to get. Returns A string representing the value of the item, i.e. its contents, label, whatever. int[] getItemLocation(Object element, Object item) Get the location of a sub-item relative to its parent element. Parameters element The GUI element to which the item belongs. item The item whose location to get. Returns The items location as a two-element int array [x,y]. The location returned must always be relative to the upper left corner of the whole element, even if that corner is not currently visible, for example because it has been scrolled outside the visible area. For items without geometry simply return [0,0]. int[] getItemSize(Object element, Object item) Get the size of a sub-item relative to its parent element. Parameters element The GUI element to which the item belongs. item The item whose size to get. Returns The items size as a two-element int array [width,height]. For items without geometry simply return [0,0].

39.2.

Implementing custom item types with the ItemResolver interface

662

Boolean repositionMouseEvent(Object element, Object item, int[] pos) Change the coordinates for a mouse event on a sub-item of a GUI element to a default location - typically the center - if the coordinates may be safely ignored during replay. This method is called only if the option Record MouseEvents without coordinates where (235) is active. Whether or not it is safe to override the event coordinates may possible depend on the original coordinates. For example, QF-Test repositions events on a tree node with positive coordinates pointing inside the node to its center whereas negative coordinates indicate a click on the expansion toggle and are left unchanged. Parameters element The GUI element to which the item belongs. item The target item for the event. pos A two-element int array of the form [x,y] with the coordinates of the event relative to the item. Its values can be modied in place. You can either set them to a specic coordinate or to [Integer.MAX_VALUE,Integer.MAX_VALUE] to ignore coordinates for this event so that it will later be replayed on the center of the target. Returns Boolean.TRUE if the position was modied, Boolean.FALSE if it was left unchanged. Null to signal that this resolver does not handle the element. Throws BadItemException If element and item types dont match up (should never happen).

39.2.

Implementing custom item types with the ItemResolver interface

663

Boolean scrollItemVisible(Object element, Object item, int x, int y) Scroll a GUI element so that an item becomes visible. If possible, the item should be made fully visible. In case the item doesnt t into the visible region of the element, at least the given position should be shown. In most cases you can simple return null to let QF-Test handle scrolling. Sometimes however, scrolling cannot be implemented generically based on the items geometry, because some items need special compensation, for example an SWT Table with a visible header. For GUI elements that cannot be scrolled, this method should simply return Boolean.FALSE. If Boolean.TRUE is returned, QF-Test will call the method again after a short interval because SWT sometimes interferes with scrolling. In the ideal case the second call should return FALSE because the position is already OK. After three attempts that return TRUE QF-Test gives up and signals an error. Parameters element The GUI element to which the item belongs. item The item that must be made visible. x The X-coordinate of the position relative to the item that must always become visible. y The Y-coordinate of the position relative to the item that must always become visible. Returns Boolean.TRUE if the scroll position of the element change, Boolean.FALSE if the position is unchanged scrolled or if the element cannot be scrolled. Null to signal that QF-Test should use its default mechanism and try to scroll the item itself. int getItemCount(Object element, Object item) Get the number of items in a GUI element or at the next item level. Though this method is currently unused, partly because the multi-level item concept for trees is not implemented yet, it should be implemented if possible. In the future it may be used for a (474) (471) Fetch count node similar to Fetch index or Fetch text . Parameters element The GUI element for which to get the item count. item Null to get the number items at the top level of the element. An item to get the number of its sub-items. Returns The number of items or -1 if there is no further sub-item level.

39.2.

Implementing custom item types with the ItemResolver interface

664

39.2.3

The class SubItemIndex

As explained in the previous section, a de.qfs.apps.qftest.shared.data.SubItemIndex represents a (partial) index for a sub-item of a complex GUI element. This class denes some constants with the following meanings: STRING This is a textual index NUMBER This is a numerical index REGEXP This is a regular expression to match a textual index INTELLIGENT When retrieving an index, use whichever type best suits the item AS_STRING Retrieve a textual index AS_NUMBER Retrieve a numerical index It also provides the following methods: SubItemIndex(String index) Create a new SubItemIndex of type STRING. Parameters index The textual index. SubItemIndex(int index) Create a new SubItemIndex of type NUMBER. Parameters index The numerical index.

39.2.

Implementing custom item types with the ItemResolver interface

665

String getIndex() Get the index as a String. Returns String getType() Get the type of the index. Returns int asNumber() Get the index as a number. Returns Throws IndexFormatException

The index converted to a String.

The type of the index, one of STRING, NUMBER or REGEXP.

The numerical index. If the index is not of type NUMBER or cannot be parsed as an integer.

boolean matches(String name) Test whether the index matches a given item name. Parameters name The name to match. Returns True if the index is not numerical and matches the given name. Throws IndexFormatException If the index contains a malformed regular expression.

39.2.4

The ItemRegistry

Once implemented and instantiated, your ItemResolver must be registered with the ItemRegistry. The class de.qfs.apps.qftest.extensions.items.ItemRegistry has the following interface: static ItemRegistry instance() There can only ever be one ItemRegistry object and this is the method to get hold of this singleton instance. Returns The singleton ItemRegistry instance.

39.2.

Implementing custom item types with the ItemResolver interface

666

void registerItemResolver(Object element, ItemResolver resolver) Register an ItemResolver for an individual GUI element. Parameters element The GUI element to register for. The resolver does not prevent garbage collection and will be removed automatically when the element becomes unreachable. resolver The ItemResolver to register. void unregisterItemResolver(Object element, ItemResolver resolver) Unregister an ItemResolver for an individual GUI element. Parameters element The GUI element to unregister for. resolver The ItemResolver to unregister. void registerItemResolver(String clazz, ItemResolver resolver) Register an ItemResolver for a class of GUI elements. Parameters clazz The class of GUI element to register for. resolver The ItemResolver to register. void unregisterItemResolver(String clazz, ItemResolver resolver) Unregister an ItemResolver for a class of GUI elements. Parameters clazz The class of GUI element to unregister for. resolver The ItemResolver to unregister. void registerItemNameResolver2(Object element, ItemNameResolver2 resolver) Register an ItemNameResolver2 for an individual GUI element. Parameters element The GUI element to register for. The resolver does not prevent garbage collection and will be removed automatically when the element becomes unreachable. resolver The ItemNameResolver2 to register.

39.2.

Implementing custom item types with the ItemResolver interface

667

void unregisterItemNameResolver2(Object element, ItemNameResolver2 resolver) Unregister an ItemNameResolver2 for an individual GUI element. Parameters element The GUI element to unregister for. resolver The ItemNameResolver2 to unregister. void registerItemNameResolver2(String clazz, ItemNameResolver2 resolver) Register an ItemNameResolver2 for a class of GUI elements. Parameters clazz The class of GUI element to register for. resolver The ItemNameResolver2 to register. void unregisterItemNameResolver2(String clazz, ItemNameResolver2 resolver) Unregister an ItemNameResolver2 for a class of GUI elements. Parameters clazz The class of GUI element to unregister for. resolver The ItemNameResolver2 to unregister. void registerItemValueResolver2(Object element, ItemValueResolver2 resolver) Register an ItemValueResolver2 for an individual GUI element. Parameters element The GUI element to register for. The resolver does not prevent garbage collection and will be removed automatically when the element becomes unreachable. resolver The ItemValueResolver2 to register. void unregisterItemValueResolver2(Object element, ItemValueResolver2 resolver) Unregister an ItemValueResolver2 for an individual GUI element. Parameters element The GUI element to unregister for. resolver The ItemValueResolver2 to unregister.

39.2.

Implementing custom item types with the ItemResolver interface

668

void registerItemValueResolver2(String clazz, ItemValueResolver2 resolver) Register an ItemValueResolver2 for a class of GUI elements. Parameters clazz The class of GUI element to register for. resolver The ItemValueResolver2 to register. void unregisterItemValueResolver2(String clazz, ItemValueResolver2 resolver) Unregister an ItemValueResolver2 for a class of GUI elements. Parameters clazz The class of GUI element to unregister for. resolver The ItemValueResolver2 to unregister.

39.2.5

Default item representations

For the implementation of the ItemNameResolver2, ItemValueResolver2 and Checker interfaces it is important to know which kind of Object is used for the internal representation of an item. This internal representation will be passed to the methods getItemName, getItemValue, getCheckData and getCheckDataAndItem.
Swing

The following table lists the complex GUI elements and the default internal item representation used by QF-Test standard ItemResolvers for Swing.
GUI element class JComboBox JList JTabbedPane JTable JTableHeader JTextArea JTree Item type Integer index Integer index Integer index int array [column,row] with row < 0 to represent a whole column Integer column index Integer line TreePath path

Table 39.1: Internal item representations for Swing GUI elements

SWT

The following table lists the complex GUI elements and the default internal item representation used by QF-Test standard ItemResolvers for SWT.

39.3.

Implementing custom checks with the Checker interface

GUI element class Item type

669

CCombo Combo CTabFolder List StyledText TabFolder Table Text Tree

Integer index Integer index Integer index Integer index Integer line Integer index int array [column,row] or just Integer column to represent a whole column Integer line Object array [Integer column,TreeItem row] or just Integer column to represent a whole column

Table 39.2: Internal item representations for SWT GUI elements

Web

The following table lists the complex GUI elements and the default internal item representation used by QF-Test standard ItemResolvers for Web.
GUI element class SELECT node TEXTAREA node OPTION node Integer line Item type

Table 39.3: Internal item representations for DOM nodes

39.3
3.1+

Implementing custom checks with the Checker interface

Checks are one of QF-Tests most useful features. Test automation would be mostly useless without the ability to verify the results of simulated actions. However, the default set of Checks available in QF-Test is naturally limited to checking the most common attributes of standard components. For special attributes or custom components you (392) can resort to read the value in an SUT script and use the method rc.checkEqual() to compare it against the expected value. Such an SUT script is perfectly ne, it performs (352) and integrates well, is exible and can be modularized by placing it inside a Procedure . It has two major disadvantages however: It cannot be recorded and it is daunting for non-programmers. With the help of the API described in this section the default set of Checks in QF-Test can be extended. In fact, QF-Tests own new-style checks are implemented exactly this

39.3.

Implementing custom checks with the Checker interface

670

way. By implementing and registering a Checker for a given type of GUI element and possibly item you can create your own checks that can be recorded and replayed just like the standard ones. To make this as simple as possible, QF-Test handles everything from showing the check in the check popup menu, fetching the check data, recording the respective Check node to store that data, sending the data back to the SUT upon replay, fetching the then current check data, comparing it to the expected value and reporting success or mismatch. All that is left for you to do is tell QF-Test which checks your Checker implements and for each of these provide the check data on request. Illustrative examples are provided at the end of the chapter and in the test-suite carconfig_en.qft, located in the directory demo/carconfig in your QF-Test installation.

39.3.1

The Checker interface

The interface de.qfs.apps.qftest.extensions.checks.Checker must be implemented in order to add custom checks for your application. The associated helper classes and interfaces are documented in the subsequent sections. CheckType[] getSupportedCheckTypes(Object element, Object item) Get the types of checks supported for the given GUI element and optional item. Parameters element The GUI element for which to get the available checks. item An optional item within the element to check. Its type depends on the GUI element and the registered (668) ItemResolvers as described in section 39.2.5 . Returns An array of CheckTypes supported by your checker. The rst element is the default for recording a check with a leftclick. If the item is null, return only those kinds of checks that can be applied to the whole GUI element. Otherwise it is best to provide all available checks because even though the user may have right-clicked on an item, he may still want to record a check on the whole GUI element.

39.3.

Implementing custom checks with the Checker interface

671

CheckData getCheckData(Object element, Object item, CheckType type) Get the check data for the current state of the GUI element or item. This method is used during replay. Parameters element The GUI element for which to get the check data. item An optional item within the element to check. Its type depends on the GUI element and the registered (668) ItemResolvers as described in section 39.2.5 . type The type of check to perform. Returns The check data for the current state of the GUI element itself, in case item is null, or the given item within the element. The kind of check to perform is specied via the type parameter, which should normally be one of those formerly returned by getSupportedCheckTypes. If you cannot perform the requested check for the given type and target, return null. Pair getCheckDataAndItem(Object element, Object item, CheckType type) Get the check data for the current state of the GUI element or item and also the item that the check actually applies to. This method is used during recording, where QF-Test not only needs to know, which data to record, but also whether to record the check for the GUI element as a whole or for an item. To implement this method without duplicating any code, call your own getCheckData method to retrieve the check data. Parameters element The GUI element for which to get the check data. item An optional item within the element to check. Its type depends on the GUI element and the registered (668) ItemResolvers as described in section 39.2.5 . type The type of check to perform. Returns A Pair with the check data for the current state of the GUI element, and the item this check should be performed on, which may be null.

39.3.2

The class Pair

The class de.qfs.lib.util.Pair for the return value of getCheckDataAndItem is a simple utility class that often comes in handy for grouping two values. Youll only need its constructor, but of course you can also read its values:

39.3.

Implementing custom checks with the Checker interface

672

Pair(Object first, Object second) Create a new Pair. Parameters first second

The rst object. May be null. The second object. May be null.

Object getFirst() Get the rst object of the Pair. Returns The rst object. Object getSecond() Get the second object of the Pair. Returns The second object.

39.3.3

The CheckType interface DefaultCheckType

and

its

implementation

A de.qfs.apps.qftest.extensions.checks.CheckType encapsulates information for a specic kind of check. It combines a CheckDataType with an identier and provides a user-friendly representation of the check for the check popup menu. Unless you need to provide multi-lingual representations of the check you should never implement this interface yourself, but simply instantiate a de.qfs.apps.qftest.extensions.checks.DefaultCheckType instead: DefaultCheckType(String identifier, CheckDataType dataType, String description) Create a new DefaultCheckType. Parameters identifier The identier for the check. The QF-Test standard checks all use lower case identiers. To prevent conicts, simply start your custom check identiers with a capital letter. dataType The CheckDataType required for your check data. description The description to show in the check popup menu for this check. For completeness sake, following are the methods of the CheckType interface: String getIdentifier() Get the identier for the check type. Returns The identier for the check type.

39.3.

Implementing custom checks with the Checker interface

673

CheckDataType getDataType() Get the CheckDataType for the check type. Returns The data type for the check type. String getDescription() Get the localized description to show for this check type in the check popup menu. Returns The description for the check type.

39.3.4

The class CheckDataType

The class de.qfs.apps.qftest.extensions.checks.CheckDataType is similar to an Enum. It denes a number of constant CheckDataType instances that simply serve to identify the kind of data that a check operates on. Each constant corresponds to one or more of the available Check nodes of QF-Test. Besides serving as a constant identier, a CheckDataType has no public attributes or methods and you cannot add any new CheckDataTypes. If you want to implement a check of a kind that does not t the existing data types youll need to convert your data so that it does, for example by using a string representation. The following CheckDataType constants are dened: STRING A single string. Used by the Check text STRING_LIST A list of string items, like the cells in a table column. Used by the Check items node. SELECTABLE_STRING_LIST A list of selectable string items, like the elements of a list. (456) Check selectable items node. BOOLEAN A boolean state, either true of false. Used by the Boolean check GEOMETRY A set of four integer values for X and Y coordinates, width and height. Not all (466) have to be dened. Used by the Check geometry node.
(448) (452) (443)

node.

Used by the

node.

39.3. IMAGE

Implementing custom checks with the Checker interface

674

An image of a whole component or item or a sub-region thereof. Used by the (460) Check image node.

39.3.5

The class CheckData and its subclasses

The class de.qfs.apps.qftest.shared.data.check.CheckData and its subclasses, all from the same package, complete the Checker API. A CheckData encapsulates the actual data for a check, must be returned from Checker.getCheckData() and is used to exchange this check data between the SUT and QF-Test. There is one concrete CheckData subclass corresponding to each CheckDataType. Youll only ever need to use their constructors, so thats what well list here. Only two of these classes are publicly available so far: StringCheckData(String identifier, String value) Create a new StringCheckData. Parameters identifier The identier for the check type. Should normally match the identier of the type argument passed to Checker.getCheckData. value The actual value for the check, a String. BooleanCheckData(String identifier, boolean value) Create a new BooleanCheckData. Parameters identifier The identier for the check type. Should normally match the identier of the type argument passed to Checker.getCheckData. value The actual value for the check, a boolean state. StringItemsCheckData(String identifier, String[] values) Create a new StringItemsCheckData. Parameters identifier The identier for the check type. Should normally match the identier of the type argument passed to Checker.getCheckData. values The actual value for the check, an array of Strings.

39.3.

Implementing custom checks with the Checker interface

675

SelectableItemsCheckData(String identifier, Object[][] values) Create a new SelectableItemsCheckData. Parameters identifier The identier for the check type. Should normally match the identier of the type argument passed to Checker.getCheckData. values The actual value for the check, an array of arrays with a String, a Boolean for the regexp ag and a Boolean for the selected ag. GeometryCheckData(String identifier, int x, int y, int width, int height) Create a new GeometryCheckData. Parameters identifier The identier for the check type. Should normally match the identier of the type argument passed to Checker.getCheckData. x The x-coordinate for the check. y The y-coordinate for the check. width The width for the check. height The height for the check. ImageCheckData(String identifier, ImageRep image, int xOffset, int yOffset, int subX, int subY, int subWidth, int subHeight) Create a new ImageCheckData. Parameters identifier The identier for the check type. Should normally match the identier of the type argument passed to Checker.getCheckData. (690) image The image for the check. See section 39.7 . xOffset An optional x-offset. yOffset An optional y-offset. subX The X-coordinate of an optional check region. subY The Y-coordinate of an optional check region. subWidth The Width of an optional check region. subHeight The Height of an optional check region. Furthermore you can dene an optional algorithm for an ImageCheckData.

39.3.

Implementing custom checks with the Checker interface

676

void setAlgorithm(String algorithm) Sets an algorithm. A detailed description can be found in Details about the algorithm for (722) image comparison .

39.3.6

The CheckerRegistry
must class the

Once implemented and instantiated, your Checker be registered with the CheckerRegistry. The de.qfs.apps.qftest.extensions.checks.CheckerRegistry has following interface:

static CheckerRegistry instance() There can only ever be one CheckerRegistry object and this is the method to get hold of this singleton instance. Returns The singleton CheckerRegistry instance. void registerChecker(Object element, Checker resolver) Register a Checker for an individual GUI element. Parameters element The GUI element to register for. The checker does not prevent garbage collection and will be removed automatically when the element becomes unreachable. resolver The Checker to register. void unregisterChecker(Object element, Checker resolver) Unregister a Checker for an individual GUI element. Parameters element The GUI element to unregister for. resolver The Checker to unregister. void registerChecker(String clazz, Checker resolver) Register a Checker for a class of GUI elements. Parameters clazz The class of GUI element to register for. resolver The Checker to register. void unregisterChecker(String clazz, Checker resolver) Unregister a Checker for a class of GUI elements. Parameters clazz The class of GUI element to unregister for. resolver The Checker to unregister.

39.3.

Implementing custom checks with the Checker interface

677

39.3.7

Custom checker example

The following Jython SUT script illustrates how to put everything together. Lets say you have a Java Swing application and want to check all labels which reside in a panel at once. To this end, your custom checker needs to iterate over all components contained in the panel and its children respectively, identify the labels and generate a list of all their text contents. In QF-Test notation, this means you need to create a CheckDataType.STRING_LIST check type and return the data in an StringItemsCheckData object:

39.3.

Implementing custom checks with the Checker interface

678

from de.qfs.apps.qftest.extensions import ResolverRegistry from de.qfs.apps.qftest.extensions.checks import CheckerRegistry, \ Checker, DefaultCheckType, CheckDataType from de.qfs.apps.qftest.extensions.items import ItemRegistry from de.qfs.apps.qftest.shared.data.check import StringItemsCheckData from de.qfs.lib.util import Pair from java.lang import String import jarray componentClass = "javax.swing.JPanel" allLabelsCheckType = DefaultCheckType("AllLabels", CheckDataType.STRING_LIST, "All labels in the panel") class AllLabelsChecker(Checker): def __init__(self): pass def getSupportedCheckTypes(self, com, item): return jarray.array([allLabelsCheckType], DefaultCheckType) def getCheckData(self, com, item, checkType): if allLabelsCheckType.getIdentifier() == checkType.getIdentifier(): labels = self._findLabels(com) labels = map(lambda l: l.getText(), labels) values = jarray.array(labels, String) return StringItemsCheckData(checkType.getIdentifier(), values) return None def getCheckDataAndItem(self, com, item, checkType): data = self.getCheckData(com, item, checkType) if data is None: return None return Pair(data, None) def _findLabels(self, com, labels=None): if labels is None: labels = [] if ResolverRegistry.instance().isInstance(com, "javax.swing.JLabel"): labels.append(com) for c in com.getComponents(): self._findLabels(c, labels) return labels def unregister(): try: CheckerRegistry.instance().unregisterChecker( componentClass, allLabelsChecker) except: pass def register(): unregister() global allLabelsChecker allLabelsChecker = AllLabelsChecker() CheckerRegistry.instance().registerChecker( componentClass, allLabelsChecker) register()

Example 39.10: Check all labels in a panel

39.4.

Working with the Eclipse Graphical Editing Framework (GEF)

679

After running that script once, youll nd a new entry All labels in the panel among the entries in the check type menu as soon as you right click on a JPanel component while (31) being in recording mode (cf. section 4.3 ). If you want to use the allLabelsChecker all over your client application, you can put the above SUT script behind your Wait for client to connect node in the Setup sequence. Otherwise, you may register the checker only when it is actually needed as shown above and remove it afterwards by means of another SUT script:
from de.qfs.apps.qftest.extensions.checks import CheckerRegistry global allLabelsChecker def unregister(): try: CheckerRegistry.instance().unregisterChecker( "javax.swing.JPanel", allLabelsChecker) except: pass unregister()

Example 39.11: Remove the label checker

39.4
3.2+

Working with the Eclipse Graphical Editing Framework (GEF)

The Graphical Editing Framework (GEF) is a set of Eclipse plugins for creating editors that support visual editing of arbitrary models. This framework is very popular and QFTest has supported recording and playback of GEF items for a long time (since about version 2.2). It is also a good example for the power of the ItemResolver concept (658) (see section 39.2 ), because the gef Jython module contains an implementation of just that interface. The gef module can deal with GEF editors at a generic level and even support several editors at once. Though reasonable item names are provided out of the box also for GMF applications, there are limits to what can be determined automatically. Depending on the underlying model classes, there might still remain some work for you: Implementing custom resolvers to provide useful names and values for your items.

39.4.1

Recording GEF items

The actual GEF component is the FigureCanvas. This control displays Figures which represent EditParts. When recording a mouse click on such an element, QFTest does not register a pure Mouse event node for the canvas component with the cor-

39.4.

Working with the Eclipse Graphical Editing Framework (GEF)

680

responding (x,y) position but tries to recognize the object under the mouse cursor. For example, the recorded Component id may look like canvas@/Diagram/My ship/ShipLargeCargo (wine) canvas@Connection-2 canvas@/Diagram/Rectangle 16329001 where canvas is the Id of the FigureCanvas component, followed by the item index (48) of the recognized EditPart (see chapter 6 ). EditParts reside in a tree like hierarchy which is reected in the index by a path separator /. The names of the individual items are generated as follows: The item name is getModel().toString() unless it contains a hash value (e.g. NodeImpl@2d862). QF-Test tries to extract a name for the item from the model (My ship in the above examples). The class name along with a description gets recorded, e.g. ShipLargeCargo (wine). If theres no description, an index is appended to the class name when theres more than one item of that class, e.g. Connection-2 for the third connection. The root EditPart always reads Diagram. As one can imagine, those generated item names may not always be useful. For example, items might be deleted so that the recorded index is not longer valid. Or the generated item name is unstable as Rectangle 16329001 in the GEF Shapes example: The number is random and when restarting the application a different one will be created. Three options exist to overcome the problem: Instead of working with a textual index, you can try to go with a numerical one. To this end, open the recording options and set the Sub-Item format to Number (246) (see section 29.2.4 ). This is probably not satisfying because a numerical index like /0/1 tells nothing about an item. Get in touch with your developers and convince them to provide a useful implementation of the toString() method of the items model. It would make live easy for you, but only if the developers are cooperative. Write an ItemNameResolver2. This is the tough course but unfortunately the most likely scenario. It is covered in the next section.

39.4.

Working with the Eclipse Graphical Editing Framework (GEF)

681

39.4.2

Implementing a GEF ItemNameResolver2

(634)

As stated in section 39.1 , an ItemNameResolver2 is the hook to change or provide (392) (315) names for items. To get started, insert a new Jython SUT script in the Extras node with the following code:
def getItemName(canvas, item, name): print "name: %s" %name print "item: %s" %(item.__class__) model = item.getModel() print "model: %s" %(model.__class__) resolvers.addItemNameResolver2("myGefItemNames", getItemName, "org.eclipse.draw2d.FigureCanvas")

Example 39.12: Get started with a GEF ItemNameResolver2

To ease the installation of the resolver we use the resolvers module described in (644) section 39.1.10 . The resolver gets registered for the FigureCanvas class where the items reside. The default item name provided by QF-Test is supplied as the last argument to our function getItemName(). Now run the script, press the record button and then simply move the mouse over your gures on the canvas - supposing you have created some of them previously. Note that this rst resolver implementation does nothing but print out out some information into the terminal, something like name: Rectangle 16329001 item: org.eclipse.gef.examples.shapes.parts.ShapeEditPart model: org.eclipse.gef.examples.shapes.model.RectangularShape The question is now: Does the model of the GEF EditPart provide any property that might be used as name for the item? The answer in the case of the GEF Shapes example is No, and hopefully you are in a better situation with your application. To nd out insert a line print dir(model) in the getItemName() function and run the script again. Now you will also see the methods of the model when moving the mouse over the items in record mode. With a bit of luck you will nd methods like getId() or getLabel() and can create a resolver like this:
def getItemName(canvas, item, name): model = item.getModel() return model.getId() resolvers.addItemNameResolver2("myGefItemNames", getItemName, "org.eclipse.draw2d.FigureCanvas")

Example 39.13: A simple ItemNameResolver2

39.4.

Working with the Eclipse Graphical Editing Framework (GEF)

682

Lets go back to the GEF Shapes example where we dont have such useful methods. Only geometry information is available for the shapes and that is not really helpful. At least we can distinguish between rectangles and ellipses. To make the item names unique we simply add a child index as shown in the following resolver:
def getItemName(canvas, item, name): name = None shapes = "org.eclipse.gef.examples.shapes" diagrammEditPart = shapes + ".parts.DiagramEditPart" shapeEditPart = shapes + ".parts.ShapeEditPart" connectionEditPart = shapes + ".parts.ConnectionEditPart" ellipticalShape = shapes + ".model.EllipticalShape" rectangularShape = shapes + ".model.RectangularShape" if qf.isInstance(item, shapeEditPart): siblings = item.getParent().getChildren() for i in range(len(siblings)): if (item == siblings[i]): if qf.isInstance(item.getModel(), ellipticalShape): name = "Ellipse " + str(i) elif qf.isInstance(item.getModel(), rectangularShape): name = "Rectangle " + str(i) elif qf.isInstance(item, connectionEditPart): source = item.getSource() target = item.getTarget() sourceName = getItemName(canvas, source, str(source.getModel())) targetName = getItemName(canvas, target, str(target.getModel())) name = "Connection " + sourceName + " " + targetName elif qf.isInstance(item, diagrammEditPart): name = "Diagram" return name resolvers.addItemNameResolver2("shapesItemNames", getItemName, "org.eclipse.draw2d.FigureCanvas")

Example 39.14: An ItemNameResolver2 for GEF Shapes

With this resolver in place, the item index for a rectangle becomes /Diagram/Rectangle 1 where the trailing number is the child index of the item. The above implementation also provides names for the connections by calling getItemName() recursively for the source and the target item of the connection. Checking the types with (607) qf.isInstance() (see section 37.7 ) will save you the need to import the GEF classes, something that is not trivial. Once your resolver is working ne you should move the script into your Setup se(411) quence right behind the Wait for client to connect node. This way the resolver will be registered automatically when the SUT starts.
(323)

39.5.

Test-run listeners

683

39.4.3

Implementing a GEF ItemValueResolver2

Usually a GEF editor consists of two parts. Having focused so far on the canvas where you draw the gures, we now take a look at the palette where you select the kind of gure to draw (e.g. Rectangle, Ellipse or Connection). Its entries look like tool buttons but actually the palette is a FigureCanvas too. You will be glad to know that this one works out of the box, that is without implementing an ItemNameResolver2. When you click for example on the Rectangle button, QF-Test recognizes a /Palette Root/Palette Container (Shapes)/Palette Entry (Rectangle) item. What will happen when you record a check (cf. section 4.3 ) for the Object value for this button? You may expect to get the button text Rectangle but in fact the value of this item is Palette Entry (Rectangle) The reason is that by default name and value of an item are the same. To alter this behavior and provide customized values you need to implement an ItemValueResolver2. This interface is very similar to the ItemNameResolver2 above. For the palette we can code the following one:
def getItemValue(canvas, item, value): value = None paletteEditPart = \ "org.eclipse.gef.ui.palette.editparts.PaletteEditPart" if qf.isInstance(item, paletteEditPart): value = item.getModel().getLabel() return value resolvers.addItemValueResolver2("shapesItemValues", getItemValue, "org.eclipse.draw2d.FigureCanvas")
(31)

Example 39.15: An ItemValueResolver2 for the GEF Shapes palette

The method getLabel() returns the text as displayed in the palette.

39.5
3.1+

Test-run listeners

Once registered with the current run-context via rc.addTestRunListener, an implementation of the TestRunListener interface will get notied whenever test execution enters or exits a node and when a problem occurs. An illustrative example is provided in the test-suite TestRunListener.qft, located in the directory demo/runlistener in your QF-Test installation. Best deactivate the debugger before running the whole test-suite.

39.5.
Note

Test-run listeners

684

A variant of the TestRunListener interface called DaemonTestRunLister can be used to monitor a test-run remotely via the daemon API. It is described in section (709) 40.2.5 . The run listener API consists of the following classes:

39.5.1

The TestRunListener interface

The interface de.qfs.apps.qftest.extensions.qftest.TestRunListener has to be implemented and registered with a run-context via rc.addTestRunListener().
Note

To implement the interface you can also derive from the class de.qfs.apps.qftest.extensions.qftest.AbstractTestRunListener which provides empty implementations of all methods so you only need to implement those methods you are interested in. void runStarted(TestRunEvent event) Notify the listener that a test-run was started. Parameters event The event containing the details, irrelevant in this case. void runStopped(TestRunEvent event) Notify the listener that a test-run was stopped. Parameters event The event containing the details, irrelevant in this case. void nodeEntered(TestRunEvent event) Notify the listener that a node is being entered. Parameters event The event containing the details. void nodeExited(TestRunEvent event) Notify the listener that a node was just exited. Parameters event The event containing the details. void problemOccurred(TestRunEvent event) Notify the listener that a problem occurred. Parameters event The event containing the details.

39.5.

Test-run listeners

685

39.5.2

The class TestRunEvent

The class de.qfs.apps.qftest.extensions.qftest.TestRunEvent holds information about the currently executed nodes and the current error state. It denes the following constants for execution states and error levels: STATE_NOT_IMPLEMENTED, STATE_SKIPPED, STATE_OK, STATE_WARNING, (294) STATE_ERROR and STATE_EXCEPTION. The rst two states apply only to Test-set (288) and Test-case nodes. In the runStopped method you can also check whether the test run has been interrupted or completed normally. Therefore the constants STATE_RUN_INTERRUPTED and STATE_RUN_TERMINATED are dened. TestSuiteNode getNode() Get the current node Returns The current node runStopped. or null for runStarted and

TestSuiteNode[] getPath() Get the whole tree path for the current node. Returns The path for the current node as seen from the run-log, equivalent to the execution stack, or null for runStarted and runStopped. The last node in the array is the current node. int getState() Get the overall execution state for the current node. Returns The overall propagating state for the current node, the highest error level of this node and its children, possibly (308) limited by Maximum error level attributes. Available only for nodeExited and problemOccurred. int getLocalState() Get the execution state for the current node. Returns The local execution state for the current node, the highest error level of this node and its children regardless of whether this state propagates to the top. Available only for nodeExited and problemOccurred. For Test-set and Test-case nodes the local state can also be one of STATE_SKIPPED or STATE_NOT_IMPLEMENTED.

39.5.

Test-run listeners

686

String getMessage() Get the current error message. Returns The message for the current warning, error or exception. Only available for problemOccurred. String getExceptions() Get the exception count for the exited node. Returns The total exception count for the node just exited. Only available for nodeExited. String getErrors() Get the error count for the exited node. Returns The total error count for the node just exited. Only available for nodeExited. String getWarnings() Get the warning count for the exited node. Returns The total warning count for the node just exited. Only available for nodeExited.

39.5.3

The class TestSuiteNode

A de.qfs.apps.qftest.extensions.qftest.TestSuiteNode is a representation of a QF-Test node that is currently being executed, including information about the type of node, its name, comment, etc. String getSuite() Get the test-suite to which the node belongs. Returns The full path of the test-suite to which the node belongs. String getType() Get the type of the node. Returns String getName() Get the name of the node. Returns The name of the node or null if that type of node does not have a Name attribute.

The type of node, the last part of the name of the class implementing the nodes behavior.

39.6.

DOM processors

687

String getReportName() Get the report name of the node. (294) (288) Returns The expanded report name for Test-set and Test-case nodes. The normal name for other nodes or null if undened. String getTreeName() Get the tree name of the node. Returns The name of the node as displayed in the test-suite tree. String getId() Get the id of the node. Returns String getComment() Get the comment of the node. Returns Get the expanded comment of the node. The id of the node.

39.6

DOM processors

When creating reports from a run-log, or package documentation from a test-suite, QFTest operates in a two-step process. The rst step creates an XML document which is transformed to an HTML document in the second step. Both transformations are done using XSLT stylesheets.
Note

The term DOM (for Document Object Model ) also applies to XML documents, not only to HTML web pages. This section is all about XML and XSLT and not about the DOM of a web SUT. However, XSLT stylesheets are not very useful when it comes to parsing plain text. The Comment elds of Test, Sequence, Package or Procedure nodes often contain some internal structure that XSLT cannot make use of. Additionally, the internal structures employed by users may vary, depending on the conventions used. A typical example is the use of JavaDoc tags to describe parameters of Procedure nodes. Heres an example Comment for the Procedure qfs.swing.menu.select from our standard library after the rst step of the transformation:

39.6.

DOM processors

688

<comment>Select an item from a menu. For example: for the File -> Open action, the component ID of "File" is the menu, and the ID for "Open" is the item. @param client The name of the SUT client. @param menu The id of the menu. @param item The id of menu item.</comment>

Example 39.16: Example Comment after rst step transformation

It is very difcult to make use of the @param tags with XSLT alone. This is where DOM processors enter the scene. Between the rst and second transformation, QFTest can optionally run an additional transformation directly on the DOM tree of the XML document generated by the rst step. During that extra transformation, QF-Test traverses the DOM tree, calling the registered DOM processors for each node to give them a chance to manipulate the DOM.
Note

For JDK 1.4 the XML Document Object Model (DOM) is part of the standard API. For earlier JDK versions it is provided by XML parser xerces (from the Apache project) which QF-Test includes. The API documentation for the DOM is available at http://download.oracle.com/javase/1.4.2/docs/api/org/w3c/dom/packagesummary.html.

39.6.1

The DOMProcessor interface

is

The interface that must be implemented de.qfs.apps.qftest.extensions.DOMProcessor. It is quite trivial: Element process(Element node)

Process one element node. Returns An element node or null. If null is returned, the child nodes of the node are processed normally. Otherwise, the child nodes are not processed. If a node other than the original node is returned, the original node is replaced with the return value. In the process method, the processor is free to do whatever it likes, as long as it constrains itself to the node passed in and its sub-nodes. The node can be replaced simply by returning some different element node.
Note

To remove an element node from the DOM, the DOMProcessor must be registered on an ancestor of the node, its parent node, for example. The current node may not be removed from the DOM in the process method.

39.6.

DOM processors

689

QF-Test provides two example implementations of DOM processors. The ParagraphProcessor is available in the misc directory for illustration. It is used internally to break comments which contain empty lines into paragraphs. Also to be found in the misc directory is the DocTagProcessor which is used to transform JavaDoc tags like @param or @author to an XML DOM sub-tree. After processing, the above example would look as follows:
<comment>Select an item from a menu. For example: for the File -> Open action, the component ID of "File" is the menu, and the ID for "Open" is the item.</comment> <param name="client">The name of the SUT client.</param> <param name="menu">The id of the menu.</param> <param name="item">The id of menu item.</param>

Example 39.17: Example comment after DOM processing

Transforming the above into useful HTML during the second stage transformation is now straightforward.

39.6.2

The DOMProcessorRegistry

Before a DOM processor can be used, it must be registered for the kind of node(s) it applies to. This is done through the DOMProcessorRegistry. There is one DOMProcessorRegistry instance object per kind of transformation, each identied by a string. Currently these identiers are report for report generation and testdoc and pkgdoc for test-set and package documentation. To get hold of a registry instance, use the static instance method: DOMProcessorRegistry instance(String identifier) Get hold of a registry instance. Parameters identifier The identier for the kind of transformation. The rest of the methods consist of the typical set of register/unregister variants: void registerDOMProcessor(DOMProcessor processor) Register a generic DOM processor that will be called for all kinds of nodes. Parameters processor The processor to register.

39.7.

Image API extensions

690

void registerDOMProcessor(String node, DOMProcessor processor) Register a DOM processor for a specic kind of node. Parameters name The type of the node. processor The processor to register. void unregisterDOMProcessor(DOMProcessor processor) Unregister a generic DOM processor. Parameters processor The processor to unregister. void unregisterDOMProcessor(String node, DOMProcessor processor) Unregister a DOM processor for a specic kind of node. Parameters name The type of the node. processor The processor to unregister. void unregisterDOMProcessors() Unregister all DOM processors.

39.6.3

Error handling

Exceptions raised during DOM processing by the process method of a DOMProcessor are caught and duly reported, the transformation is stopped in that case.

39.7
3.0+

Image API extensions

The Image API of QF-Test makes use of a technology independent class for image representation ImageRep and it offers an interface to implement an own image compare algorithm.

39.7.1

The ImageRep class

The class de.qfs.apps.qftest.shared.extensions.image.ImageRep is the wrapper class for technology independent images. The class holds the image representation either as ARGB data, which is an int array

39.7.

Image API extensions

691

or as RGB data, which is a byte array. Besides the image data, you can also specify a name, the width and the height for the ImageRep object. The ImageRep class also contains an equals method for comparisons. If you want to implement your own image comparison algorithm, you have to implement your own ImageComparator. This implementation has to be registered at the ImageRep object. (693) See section 39.7.2 for further information. ImageRep ImageRep() Constructor for the ImageRep class. ImageRep ImageRep(String name, byte[] rgb, boolean png, int width, int height) Constructor method for the ImageRep class. Parameters name The name of the ImageRep object. rgb A byte array containing RGB data of the image. png Whether the image is already png encoded. width The width of the image. height The height of the image. ImageRep ImageRep(String name, int[] argb, boolean png, int width, int height) Constructor method for the ImageRep class. Parameters name The name of the ImageRep object. argb An int array containing the ARGB data of the image. png Whether the image is already png encoded. width The width of the image. height The height of the image. void setName(String name) Set the name. Parameters name String getName() Get the name. Returns The current name.

The new name.

39.7.

Image API extensions

692

void setWidth(int width) Set the width. Parameters width int getWidth() Get the width. Returns The current width.

The new width.

void setHeight(int height) Set the height. Parameters height The new height. int getHeight() Get the height. Returns Set the RGB data. Parameters png

The current height.

void setPng(byte[] png)

The new RGB data.

byte[] getPng() Get the RGB data. If no RGB data is set, the ARGB data will be translated into the RGB data. Returns The current RGB data. void setARGB(int[] argb) Set the ARGB data. Parameters argb

The new ARGB data.

int[] getARGB() Get the ARGB data. If no ARGB data is set, the RGB data will be translated into the ARGB data. Returns The current ARGB data. void setComparator(ImageComparator comparator) Set ImageComparator implementation, which will be used for comparing images. Parameters comparator The new ImageComparator implementation.

39.7.

Image API extensions

693

ImageComparator getComparator() Get the current ImageComparator implementation. Returns The current ImageComparator implementation. boolean equals(ImageRep compare) Return, whether the current object is equal to a given object. It uses the equals method of the current ImageComparator implementation. Parameters compare The ImageRep object to compare with. Returns True, if images are equal, otherwise false. int getPixel(int x, int y) Get an ARGB pixel value. Parameters x The X co-ordinate of the pixel. y The Y co-ordinate of the pixel. Returns The pixel value. void crop(int x, int y, int width, int height) Crop the image to a specied sub-region. Parameters x The X co-ordinate of the left upper corner of the subregion. y The Y co-ordinate of the left upper corner of the subregion. width The width of the sub-region. height The height of the sub-region.

39.7.2

The ImageComparator interface

The interface de.qfs.apps.qftest.shared.extensions.image.ImageComparator has to be implemented, if you want to implement your own image comparison algorithm. The implementation has then to be registered at the used ImageRep objects using the setComparator method.

39.7.

Image API extensions

694

boolean equals(ImageRep actual, ImageRep expected) Return, whether the current object is equal to a given object. It uses the equals method of the current ImageComparator implementation. Parameters actual The actual ImageRep object. expected The expected ImageRep object. Returns True, if images are equal, otherwise false.

Chapter 40 Daemon mode

!!! Warning !!! Anybody with access to the QF-Test daemon can start any program on the machine running the daemon with the rights of the user account that the daemon is running under, so access should be granted only to trusted users. If you are not running the daemon in a secure environment where every user is trusted or if you are creating your own library to connect to the QF-Test daemon, you denitely (709) should read section 40.3 below.

40.1

Daemon concepts

In daemon mode (comparable, but not equivalent to a service on Windows) QF-Test listens to RMI connections and providing an interface for remote test execution. This is useful for simplifying test execution in a distributed load-testing scenario, but also for integration with existing test-management or test-execution tools. There are two command line arguments both of which are available in batch and interactive mode: -daemon
(542)

- Run QF-Test in daemon mode.

(542)

-daemonport <port> default is 3543.

- Specify the port the daemon should listen at, the

When run in batch and daemon mode, QF-Test will not use a license. Licenses will be acquired during use as described below. In interactive daemon mode, QF-Test will be fully functional and thus use a license. In addition it will accept connections from the outside and use additional licenses during use, similar to batch mode. The latter scenario is useful during development of distributed tests.

40.2.

Daemon API

696

Tests can be executed on a running daemon in one of two ways: Use of QF-Tests batch commandline option -calldaemon
(542)

.
(696)

Direct implementation against the daemon API, documented in section 40.2

Both options are explained in more detail and accompanied by examples in section (139) 16.2 .

40.2

Daemon API

The daemon API provides all methods necessary to directly control test execution via a QF-Test daemon. Typically the following relativley simple steps need to be implemented: Get hold of a Daemon using the DaemonLocator service. Either get hold of the shared TestRunDaemon or have the Daemon create a TestRunDaemon. You can use the TestRunDaemon to dene global variables and the root directory of your test-suites for the upcoming test-runs. Either get hold of the TestRunDaemons shared DaemonRunContext or have it create one or more DaemonRunContext instances. A DaemonRunContext represents one test execution context for functional or load testing. Each DaemonRunContext is independent of the others, except for a group of contexts created via TestRunDaemon.createContexts(int threads) which creates a group of related contexts for load-testing similar to running QF-Test (553) command line argument. Each with the -threads <number> DaemonRunContext requires one QF-Test development or runtime license during its lifetime. Now you can tell the DaemonRunContext to execute tests on your behalf, either (294) (288) or Test-case nodes. With properly implemented whole test-suites or Test-set (317) Dependencies you can have a Daemon execute Test-cases on your behalf in any arbitrary order without explicitely having to take care of any setup or cleanup tasks. The DaemonRunContext also has methods to check the current state of a testrun or wait for it to nish. Finally you can get the run-log of the test-run from the Daemon. Right now, all you can do is save it, but we will open the run-log API so that you will be able to integrate run-logs from various daemons and test-runs into a single run-log.

40.2.

Daemon API

697

For proper dependency management, including rollback of no longer required dependencies between Test-case invocations, it is important to use the same DaemonRunContext for execution of each related Test-case. The easiest way to achieve this is to use the shared TestRunDaemon and its shared DaemonRunContext each time you talk to a given Daemon. The following sections provide a complete reference for the daemon API. Further expla(139) nations and examples are provided in section 16.2 .

40.2.1

The DaemonLocator

The singleton class de.qfs.apps.qftest.daemon.DaemonLocator can be used to get hold of Daemon instances. public static DaemonLocator instance() Get the singleton instance. Returns The singleton instance. public Daemon locateDaemon(String host, int port) Get a Daemon from a specic host and port. Parameters host The target host, name or IP string. port The target port. Returns The daemon or null if none can be found. public Daemon[] locateDaemons(long timeout) Get all known daemons. Parameters timeout Returns

The time in milliseconds to wait for daemons to react. The known daemons.

40.2.2

The Daemon

The de.qfs.apps.qftest.daemon.Daemon interface is an envelope for various kinds of QF-Test daemons. Currently only the TestRunDaemon is available, but something like an SUTClientStarterDaemon is already planned and others may follow.

40.2.

Daemon API

698

public void ping() Test whether the Daemon is still alive. Throws RemoteException If something RMI specic goes wrong. public String getHost() Get the host of the Daemon. Returns The host of the Daemon. Throws RemoteException If something RMI specic goes wrong. public String getIp() Get the ip of the Daemon. Returns The ip of the Daemon. Throws RemoteException If something RMI specic goes wrong. public int getPort() Get the port of the Daemon. Returns The port of the Daemon. Throws RemoteException If something RMI specic goes wrong. public TestRunDaemon getSharedTestRunDaemon() Get the shared TestRunDaemon. Returns The shared TestRunDaemon. Throws RemoteException If something RMI specic goes wrong. public TestRunDaemon createTestRunDaemon() Get access to a TestRunDaemon. Returns A TestRunDaemon. Throws RemoteException If something RMI specic goes wrong. public TestRunDaemon[] getTestRunDaemons() Get all TestRunDaemons created by the Daemon that are still live. Returns The set of live TestRunDaemons created by the Daemon. Does not include the shared TestRunDaemon. Throws RemoteException If something RMI specic goes wrong.

40.2.

Daemon API

699

public void terminate(int exitCode) Terminate the daemon process by calling System.exit. Parameters exitCode The exit code for the daemon. Throws RemoteException If something RMI specic goes wrong. public void killClients() Kill all clients that belong to the VM of the daemon. Throws RemoteException If something RMI specic goes wrong. void cleanup() Clean up all TestRunDaemons belonging to this Daemon and then kill all clients. The default timeout of 30 seconds is used to wait for possible dependency rollback. Throws RemoteException If something RMI specic goes wrong. void cleanup(long timeout) Clean up all TestRunDaemons belonging to this Daemon and then kill all clients. Parameters timeout The maximum time in milliseconds to wait for possible dependency rollback. Throws RemoteException If something RMI specic goes wrong.

40.2.3

The TestRunDaemon

The de.qfs.apps.qftest.daemon.TestRunDaemon is the outer interface for test execution. It is used to dene the environment for associated test-runs and create the DaemonRunContext instances that handle the actual test execution. Miscellaneous public Daemon getDaemon() Get the Daemon to which the TestRunDaemon belongs. Returns The Daemon of the TestRunDaemon. Throws RemoteException If something RMI specic goes wrong.

40.2.

Daemon API

700

Global variable handling public void setGlobal(String name, String value) Set a global variable value. Parameters name The name of the global variable. value The value of the global variable. Throws RemoteException If something RMI specic goes wrong. public String getGlobal(String name) Get a global variable value. Parameters name The name of the global variable. Returns The value of the global variable or null if undened. Throws RemoteException If something RMI specic goes wrong. public Properties getGlobals() Get all global variable values. Returns All global variable values. Throws RemoteException If something RMI specic goes wrong. public void clearGlobals() Clear all global variables. Throws RemoteException If something RMI specic goes wrong. Test execution public void setRootDirectory(String directory) Set the test-suite root directory for new created daemon run-contexts. Parameters directory The new root directory. Throws RemoteException If something RMI specic goes wrong.

40.2.

Daemon API

701

public DaemonRunContext getSharedContext() Get the shared daemon run-context. Needs to acquire a license if the shared context must be created rst. Returns The shared context or null if no license can be acquired. Throws RemoteException If something RMI specic goes wrong. public DaemonRunContext createContext() Create a single daemon run-context. Needs to acquire a license. Returns The context or null if no license can be acquired. Throws RemoteException If something RMI specic goes wrong. public DaemonRunContext[] createContexts(int threads) Create daemon run-contexts for multiple threads. Needs to acquire one license per thread. Parameters threads The number of threads for the contexts. Returns The contexts or null if not enough licenses can be acquired. Throws RemoteException If something RMI specic goes wrong. public DaemonRunContext[] getContexts() Get all DaemonRunContexts created by the TestRunDaemon that are still live and have not been released. Returns The set of live DaemonRunContexts created by the TestRunDaemon. Does not include the shared DaemonRunContext. Throws RemoteException If something RMI specic goes wrong. void cleanup() Clean up and release all contexts belonging to this TestRunDaemon. The default timeout of 30 seconds is used to wait for possible dependency rollback. Throws RemoteException If something RMI specic goes wrong.

40.2.

Daemon API

702

void cleanup(long timeout) Clean up and release all contexts belonging to this TestRunDaemon. Parameters timeout The maximum time in milliseconds to wait for possible dependency rollback. Throws RemoteException If something RMI specic goes wrong. Identication public void setIdentifier(String identifier) Set an identier for the TestRunDaemon. This can be useful in identifying a TestRunDaemon retrieved via Daemon.getTestRunDaemons(). Parameters identifier The identier to set. Throws RemoteException If something RMI specic goes wrong. public String getIdentifier() Get the identier for the TestRunDaemon. If no identier was previously set with setIdentier(), a default identier is created from the name of the Daemon, to which the TestRunDaemon belongs, and a counter. Returns The identier for the TestRunDaemon. Throws RemoteException If something RMI specic goes wrong.

40.2.4

The DaemonRunContext

The de.qfs.apps.qftest.daemon.DaemonRunContext interface is in charge of the actual test execution. Following run states are dened:

40.2.

Daemon API
State STATE_INVALID STATE_IDLE STATE_SCHEDULED STATE_RUNNING STATE_PAUSED STATE_FINISHED Value -1 0 1 2 3 4 Description Invalid after release - cannot be reactivated. No run scheduled. Run scheduled but not started. Running. Running but paused. Run nished, result and run-log available.

703

Table 40.1: The run state

public TestRunDaemon getTestRunDaemon() Get the TestRunDaemon to which the DaemonRunContext belongs. Returns The TestRunDaemon of the DaemonRunContext. Throws RemoteException If something RMI specic goes wrong. public int getNumThreads() Get the number of threads in the group to which the DaemonRunContext belongs. Returns The number of threads of the DaemonRunContexts group. Throws RemoteException If something RMI specic goes wrong. public int getThreadNum() Get the thread index of the DaemonRunContext. Parameters Returns Throws RemoteException The thread index of the DaemonRunContext. If something RMI specic goes wrong.

public int getRunState() Get the current run state of the context. One of STATE_IDLE, STATE_SCHEDULED, STATE_RUNNING, STATE_PAUSED, STATE_FINISHED. Returns The current run state. Throws RemoteException If something RMI specic goes wrong.

40.2.

Daemon API

704

public boolean waitForRunState(int state, long timeout) Wait for the context to reach a given state. Parameters state The state to wait for. timeout Maximum time in milliseconds to wait. Returns True if the state was reached, false if timed out. Throws RemoteException If something RMI specic goes wrong. public void setRootDirectory(String directory) Set the test-suite root directory for the next test-run. Parameters directory The new root directory. Throws RemoteException If something RMI specic goes wrong.

40.2.

Daemon API

705

public boolean runTest(String test, Properties bindings=None) Run a test in the run-context. Parameters test The test to run, of the form Suite#Test where #Test is optional and Test is the fully qualied name of a Test-set or Test-case or just .. The latter is equivalent to specifying just Suite and causes the whole test-suite to be executed. Examples: MySuite Runs whole testsuite MySuite. MySuite#. Runs whole testsuite MySuite. MySuite#MyTestSet Runs test-set MyTestSet in test-suite MySuite. MySuite#MyTestSet.MyTestCase Runs test-case MyTestCase located in testset MyTestSet in test-suite MySuite. bindings An optional set of variable bindings. These variables have higher precedence than the globals or any bindings on the fallback stack. Returns True if the test was started, false if suite or test could not be found. Throws RemoteException If something RMI specic goes wrong. IllegalStateException If no run can be started in the current state, i.e. if the state is neither STATE_IDLE nor STATE_FINISHED.

40.2.

Daemon API

706

public boolean callProcedure(String procedure, Properties bindings=None) Call a procedure in the run-context. Parameters procedure The procedure to run, of the form Suite#Procedure where Procedure is the fully qualied name of the Procedure. bindings An optional set of variable bindings. These variables have higher precedence than the globals or any bindings on the fallback stack. Returns True if the procedure call was started, false if suite or procedure could not be found. Throws RemoteException If something RMI specic goes wrong. IllegalStateException If no run can be started in the current state, i.e. if the state is neither STATE_IDLE nor STATE_FINISHED. public String getLastTest() Get the name of the test that is currently running or was last run on this DaemonRunContext. Returns The name of the currently running or most recently executed test. Throws RemoteException If something RMI specic goes wrong. public void stopRun() Stop the test-run. Throws RemoteException If something RMI specic goes wrong. IllegalStateException If no run was scheduled. public int getResult() Get the result of the test run. Returns 0, 1, 2, 3 for OK, warning, error, exception. Throws RemoteException If something RMI specic goes wrong. IllegalStateException If the state isnt STATE_FINISHED.

40.2.

Daemon API

707

public byte[] getRunLog() Get the run-log of the test run. Returns The run-log dumped into a byte array. Throws RemoteException If something RMI specic goes wrong. IllegalStateException If the state isnt STATE_FINISHED. void rollbackDependencies() Roll back the dependencies for this DaemonRunContext. Throws RemoteException If something RMI specic goes wrong. IllegalStateException If no run can be started in the current state, i.e. if the state is neither STATE_IDLE nor STATE_FINISHED. public void release() Release the DaemonRunContext and return its license. If a test is running, stop it. Throws RemoteException If something RMI specic goes wrong. IllegalStateException If no DaemonRunContext was allocated. public void setIdentifier(String identifier) Set an identier for the DaemonRunContext. This can be useful in identifying a DaemonRunContext retrieved via TestRunDaemon.getContexts(). Parameters identifier The identier to set. Throws RemoteException If something RMI specic goes wrong. public String getIdentifier() Get the identier for the DaemonRunContext. If no identier was previously set with setIdentier(), a default identier is created from the name of the TestRunDaemon, to which the DaemonRunContext belongs, and a counter. Returns The identier for the DaemonRunContext. Throws RemoteException If something RMI specic goes wrong.

40.2.

Daemon API

708

public String getGlobal(String name) Retrieve the value of a global variable from the DaemonRunContext. Parameters name The name of the variable. Returns The value of the variable or null if undened. Throws RemoteException If something RMI specic goes wrong. public Properties getGlobals() Retrieve all global variables from the DaemonRunContext. Returns The global variables. Throws RemoteException If something RMI specic goes wrong. public String getProperty(String group, String name) Retrieve the value of a property or resource from the DaemonRunContext. Parameters name The name of the property or resource group. name The name of the property. Returns The value of the property or null if undened. Throws RemoteException If something RMI specic goes wrong. public Properties getProperties(String group) Retrieve all properties from a property or resource group from the DaemonRunContext. Parameters name The name of the property or resource group. Returns The properties or null if no such group exists. Throws RemoteException If something RMI specic goes wrong.

40.3.

Daemon security considerations

709

void addTestRunListener(DaemonTestRunListener listener, boolean synchronous, long timeout) Add a DaemonTestRunListener to the DaemonRunContext. Parameters listener The listener to add. synchronous Whether the listener should get notied synchronously, in which case the test-run will be blocked while the listener is processing the event. timeout Timeout in milliseconds for callbacks to the listener. If the listener does not reply within that time the listener is automatically unregistered to prevent further problems. In case of a synchronous listener the test-run will then continue. A value of 0 means no timeout which is dangerous but may be useful. void removeTestRunListener(DaemonTestRunListener listener) Remove a DaemonTestRunListener from the DaemonRunContext. Parameters listener The listener to remove. void clearTestRunListeners() Remove all DaemonTestRunListeners from the DaemonRunContext.

3.1+

40.2.5

The DaemonTestRunListener

The de.qfs.apps.qftest.daemon.DaemonTestRunListener interface is identical to the interface de.qfs.apps.qftest.extensions.qftest.TestRunListener described in (683) section 39.5 , except that its methods can throw a RemoteException on RMI failure. When implementing this interface you must derive your class from java.rmi.server.UnicastRemoteObject. You can register the listener with a DaemonRunContext via its addTestRunListener method described in the previous section.

40.3
3.5+

Daemon security considerations

Anybody with access to the QF-Test daemon can start any program on the daemon machine with the rights of the user account that the daemon is running under, so care should be taken to only allow trusted users to connect to the daemon.

40.3.

Daemon security considerations

710

Of course the QF-Test daemon should always be run on a machine that is protected from outside access by a rewall. If all users that can access the machine behind the rewall are trusted, that is sufcient. If the set of trusted users needs to be limited further, please read on. By default the QF-Test daemon uses SSL to secure the RMI connection. However, unless you take additional precautions, this only means that the network trafc between the daemon and its client is encrypted. To restrict access to certain users, one further step is required. Setting up SSL communication can be very complex. One usually has to learn about keys, certicates, certicate authorities, chains of trust etc. Fortunately, this is a very special case, and the fact that once a user has access to the QF-Test daemon he also has control over the daemons machine means that there is no distinction between the daemon administrator and a daemon user. Without going into details, QF-Test normally uses a single keystore le with a single self-signed certicate on both daemon and client side. More complex scenarios are possible but beyond the scope of this manual. The default keystore le is named daemon.keystore and provided in QF-Tests versionspecic directory. By creating your own keystore as described below you can ensure that only users that have read access to that keystore le can interact with the daemon.

40.3.1

Creating your own keystore

To create the keystore le you need a current JDK version 1.5 or higher, a JRE is not sufcient. In a shell or console window, execute the following command (you may need to prepend the path to the keytool program which resides in the JDKs bin directory):

keytool -keystore daemon.keystore -genkey -alias "qftest daemon" -keyalg DSA -validity 9

Example 40.1: Creating a keystore for secure daemon communication

For further information about the keytool command please http://download.oracle.com/javase/6/docs/technotes/tools/solaris/keytool.htm.

see

When asked for the password for the keystore enter 123456. When asked for your name or organization, feel free to provide answers or not. For QF-Test these dont make any difference. Of course you can provide a secure password instead of 123456, but that will only complicate starting the daemon and its client and not contribute much to security. You could also use a shorter validity, but in case the keystore le gets into the wrong hands, all you need to do is set up your daemon and users with a new one, so the old one becomes useless.

40.3.

Daemon security considerations

711

40.3.2

Specifying the keystore

You have several options to tell QF-Test to use your keystore instead of the default one: Save the le as daemon.keystore in QF-Tests root directory Save the le as daemon.keystore in your QF-Test user directory, i.e. /.qftest on Unix, C:\Users\<username>\.qftest on Windows 7 and Vista or C:\Documents and settings\<username>\.qftest on Windows XP. Save the le wherever you like and provide its location to QF-Test via the command (544) line argument -keystore <keystore file> . In case you specied your own password for the keystore you also need to specify that (544) via the command line argument -keypass <keystore password> . In case you would like to start the daemon without any SSL support, for example to interact with a QF-Test version older than 3.5, either remove the le called daemon.keystore from QF-Tests version specic directory or use the command line argument in the form -keystore= to specify no keystore.

40.3.3

Specifying the keystore on the client side

If you use qftest -batch -calldaemon to access the daemon or script nodes from within QF-Test, the options for the client are the same as for the daemon. To access the daemon over SSL from your own code via the daemon API you must set the System properties javax.net.ssl.keyStore and javax.net.ssl.trustStore to the keystore le and javax.net.ssl.keyStorePassword to the password for the keystore le. See (696) (139) section 40.2 for details about the daemon API and section 16.2 for examples.

Chapter 41 The Procedure Builder denition le

For general information about the Procedure Builder, please see chapter chapter 18
(153)

41.1

Placeholders

In the denition of procedures, packages and even in the nodes of the procedures, you can use placeholders. The following table shows all placeholders and their meaning:

41.1.

Placeholders
Placeholder

713
Description The ID of the component The name of the component The feature of the component The class of the component The value of the given extra feature The current value of the component, e.g. the text of a text-eld or the current selection of a combo-box etc. The current enabled state of the component The current selectable state of the component The current editable state of the component The component-id of the parent component The name of the parent component The feature of the parent component The class of the parent component The value of the given extra feature of the parent component The component-id of the grandparent component The name of the grandparent component The feature of the grandparent component The class of the grandparent component The value of the given extra feature of the grandparent component

<COMPID> <COMPNAME> <COMPFEATURE> <COMPCLASS> <COMPEF-name-of-extra-feature> <CURRENTVALUE> <CURRENTENABLEDSTATE> <CURRENTSELECTEDSTATE> <CURRENTEDITABLESTATE> <PCOMPID> <PCOMPNAME> <PCOMPFEATURE> <PCOMPCLASS> <PCOMPEF-name-of-extrafeature> <GPCOMPID> <GPCOMPNAME> <GPCOMPFEATURE> <GPCOMPCLASS> <GPCOMPEF-name-of-extrafeature>

Table 41.1: Placeholders for component procedures

For procedures of container or composite components you can also use placeholders for the child-components. Please see following table for those additional variables:
Variable <CCOMPID> <CCOMPNAME> <CCOMPFEATURE> <CCOMPCLASS> <CCOMPEF-name-of-extrafeature> <CCURRENTVALUE> <CCURRENTENABLEDSTATE> <CCURRENTSELECTEDSTATE> <CCURRENTEDITABLESTATE> Description The ID of the child-component The name of the child-component The feature of the child-component The class of the child-component The value of the given extra feature The current value of the child-component, e.g. the text of a texteld or the current selection of a combo-box etc. The current enabled state of the child-component The current selectable state of the child-component The current editable state of the child-component

Table 41.2: Additional placeholders for container procedures

41.2.

Conditions for Package and Procedure Denition

714

41.2

Conditions for Package and Procedure Denition

You can inuence the creation of packages and procedures via using the comment attribute of the Package or Procedure node.
Comment-Attribute @NOTINHERIT @FORCHILDREN Description If this value is set, then this package or procedure is only dened for the exact class and not for its descendants. If you dene procedures for container procedures, you can set this value to tell QF-Test to create those procedures for calling the child-components procedures. Species, whether the package or the procedure should be dened for a certain class or not. This construct might be useful, if you dene a package or procedure for an abstract class and the procedures should not be created for all of its descendants. Species a dedicated condition for creating packages or procedures. You can dene conditions as Jython, Groovy or regular expression. Please see below for further details. Evaluate the style attribute of a given SWT-button. This SWT specic attribute got required, because SWT distinguishes between check-boxes, buttons or radio-buttons only through the style attribute. For SWT-buttons you can dene something like @SWTSTYLE=PUSH or @SWTSTYLE=RADIO for radio buttons. This value is only valid for menu actions up-to-now. If this ag is set, then the according package or procedure will be created for sub-items on the second level of a menu only.

@EXCEPT

@CONDITION

@SWTSTYLE

@SUBITEM

Table 41.3: Comment attributes for procedure creation

41.3

Interpretation of the Component Hierarchy

It might be interesting to make use of the component-hierarchy in the package structure. This approach allows the tester to locate the component-specic procedures quite easy. If you want to create component-hierarchy packages, you can use two placeholders in the package-names:

41.4.

Details about the @CONDITION tag

715

Hierarchy-Placeholder <HIERARCHY> <HIERARCHY_NAME>

<HIERARCHY_FEATURE>

<IHIERARCHY>

<IHIERARCHY_NAME>

<IHIERARCHY_FEATURE>

<MHIERARCHY>

<MHIERARCHY_NAME>

<MHIERARCHY_FEATURE>

Description Create packages for the full component-hierarchy by using the component-id. Create packages for the full component-hierarchy by using the name of the component. If a component in the hierarchy has no name, the component wont get taken into account. Create packages for the full component-hierarchy by using the recorded feature of the component. If a component in the hierarchy has no feature, the component wont get taken into account. Create packages only for interesting components in the componenthierarchy by using the component-id. An interesting component is a component with a feature. Create packages only for interesting components in the componenthierarchy ny using the name of the component. An interesting component is a component with a feature. Create packages only for interesting components in the componenthierarchy by using the feature of the component. An interesting component is a component with a feature. Create packages only for menu components in the componenthierarchy by using the component-id. It only takes components into account, which act as menu or menu-item. Create packages only for menu components in the componenthierarchy by using the component-name. It only takes components into account, which act as menu or menu-item. If no name is set, the component will be ignored. Create packages only for menu components in the componenthierarchy by using the component-feature. It only takes components into account, which act as menu or menu-item. If no feature is set, the component will be ignored.

Table 41.4: Hierarchy placeholders

41.4
3.4+

Details about the @CONDITION tag

Using the @CONDITION tag allows you to congure, whether a dedicated node should be created or not during creation time. Such conditions might be used to check a certain name or for appearance of a dedicated letter in the feature. If this condition is not fullled, the node wont be created. You can use all known placeholders, e.g. <COMPID> or <CCOMPNAME>.

41.4.

Details about the @CONDITION tag

716
Meaning Here we dene a Jython condition, which will create the according node, if the feature of the current component starts with abc. It is possible to use any string or comparing method of Jython. Here we dene a Groovy condition, which will create the according node, if the feature of the current component starts with abc. It is possible to use any string or comparing method of Groovy. Here we dene a regular expression, which will create the according node, if the feature of the current component starts with abc. It is possible to use all capabilities of Java regular expressions. Here we dene a regular expression condition, which will create the according node, if the feature of the current component does not start with abc. It is possible to use all capabilities of Java regular expressions.

Value of Condition @CONDITION jython <COMPFEATURE>.startswith(abc)

@CONDITION groovy <COMPFEATURE>.startsWith(abc)

@CONDITION regexp <COMPFEATURE> = abc.*

@CONDITION regexp <COMPFEATURE> ! abc.*

Table 41.5: Samples for the @CONDITION tag

If you need more than one row, you have to use a \ at the end of the rst row.

Chapter 42 The ManualStepDialog

The ManualStepDialog is a Java class delivered by QF-Test. If you want to make use of this dialog for your own tests, please see following sample script and API specication.
from de.qfs.apps.qftest import ManualStepDialog #create the dialog and show it immediately manualDialog = ManualStepDialog(None, "New Test Case Title", \ "Step Description", "Expected Test Result") #did the test fail or succeed? failOrSuccess = manualDialog.getResult() #get the content of the received result receivedResult = manualDialog.getReceivedResult() #get the execution information, whether skipped or canceled execInfo = manualDialog.getExecInfo()

Example 42.1: Example use of ManualStepDialog

42.1

The ManualStepDialog API

ManualStepDialog(Component parent, String title, String stepText, String expResult) Create a new ManualStepDialog. Parameters parent The parent component for the dialog. title The title of the dialog. stepText The text for the step description text-eld. expResult The text for the expected result text-eld.

42.1.

The ManualStepDialog API

718

String getResult() Get the result of the test-step. Please, see chapter section 22.5 Returns A string containing the result. void setResult(String newResult) Set the result of the test-step. Please, see chapter section 22.5 Parameters newResult The result of the test-step.
(184) (184)

for all possible results.

for all possible results.

String getExecInfo() Get the execution information of the test-step. Returns A string containing the execution information. void setExecInfo(String newExecInfo) Set the execution information of the test-step. Parameters newExecInfo The execution information of the test-step. String getReceivedResult() Get the received result of the test-step. Returns A string containing the received result. void setReceivedResult(String newRecResult) Set the received result of the test-step. Parameters newReceived Result The received result of the test-step. boolean isStatusFailed() Test whether the status is FAILED. Returns True if the status is failed, false otherwise. boolean isStatusPassed() Test whether the status is PASSED. Returns True if the status is passed, false otherwise. boolean isStatusSkipped() Test whether the status is SKIPPED. Returns True if the status is skipped, false otherwise. boolean isStatusCanceled() Test whether the status is CANCELED. Returns True if the status is canceled, false otherwise.

Chapter 43 Details about transforming nodes

3.1+

43.1

Introduction

The transformation mechanism allows you to turn a node into another type, e.g. a Sequence into a Procedure or a Test into a Test-case. Such actions could be required for re-factoring purposes or simply to make test development for efcient. You can transform nodes via a right Transform node into and the desired type.
Note

mouse

click

and

then

selecting

QF-Test only shows transformations that are possible in the current context of the testsuite, so sometimes you may not see all possible transformations.

43.2

Transformation with type changes

The following transformations also change the type of child nodes of the converted nodes: 1. Test-set into Test-case (a) Data driver into disabled Sequence (b) Test-case into Test 2. Test-set into Test (a) Dependency into disabled Sequence (b) Dependency reference into disabled Procedure call (c) Test-case into Test

43.3.

Additional transformations below the Extras node

720

3. Test-case into Test-set (a) All children are packed into a new sub Test-case node. 4. Test-case into Test (a) Dependency into disabled Sequence (b) Dependency reference into disabled Procedure call 5. Test into Test-set (a) If there are only Data driver and Test children, the Data driver is turned into a disabled Test. (b) Otherwise all children are packed into a new sub Test node. 6. Test into Test-set recursive (a) If there are only Data driver and Test children, the Data driver is turned into a disabled Test. (b) Otherwise all children are packed into a new sub Test-case node. 7. Test into Test-case (a) Data driver into disabled Sequence

43.3

Additional transformations below the Extras node

The following transformations are allowed below the Extras node only:

43.3.1

Transformations without side-effects

1. Sequence into Cleanup 2. Sequence into Setup 3. Sequence into Test-case 4. Cleanup into Procedure 5. Setup into Procedure 6. Procedure into Sequence 7. Procedure into Test-case 8. Procedure into Test

43.3.

Additional transformations below the Extras node

721

43.3.2

Transformations with side-effects

The following transformations also change the type of some child nodes: 1. Test-set into Package (a) Cleanup into disabled Procedure (b) Data driver into disabled Procedure (c) Dependency reference into disabled Dependency containing the previous Dependency reference (d) Setup into disabled Procedure (e) Test into Procedure (f) Test-case into Procedure (g) Test call into disabled Procedure containing the Test call 2. Test-case into Procedure (a) Cleanup into disabled Sequence (b) Dependency into disabled Sequence (c) Dependency reference into disabled Sequence (d) Setup into disabled Sequence 3. Test into Procedure (a) Data driver into disabled Sequence 4. Test into Package (a) All child nodes are packed into a Procedure node. 5. Test into Sequence (a) Data driver into disabled Sequence 6. Package into Test-set (a) Package into Test-set (b) Procedure into Test-case 7. Package into Test (a) Dependency into disabled Test (b) Package into Test (c) Procedure into Test

Chapter 44 Details about the algorithm for image comparison

3.3+

44.1

Introduction
(460)

The classic Check image node is only minimally tolerant towards deviations. Using the default algorithm of comparing pixel by pixel it is not possible to check images that are generated in a not-quite-deterministic way or differ in size. By using the attribute Algorithm for image comparison it is possible to dene a special algorithm which is tolerant towards certain image deviations. The attribute must start with the algorithm denition in the form algorithm=<algorithm> followed by all required parameters, separated by semicolons. Parameters may be dened in any order, and use of variables is possible, e.g.:
3.5.1+
(463)

algorithm=<algo>;parameter1=value1;parameter2=value2;expected=$(expected) Since QF-Test 3.5.1 the denition does not need so start with algorithm=<algorithm> but can simply begin with <algorithm>. It is also no longer necessary to dene the parameter expected. QF-Test uses a default value if it is not set. Please see below for more information. A detailed description of the available algorithms and their parameters is provided in the following section. For illustration, explanations are based on their effects on the following image:

44.2.

Description of algorithms

723

Figure 44.1: Original image

In the related run-log (see section 8.1 ) of a failed check you have the opportunity to analyze the results of the algorithm as well as the result of probability calculation. (282) If you activate the option Log successful advanced image checks all tolerant image checks will be logged for further analysis.

(63)

44.2
44.2.1

Description of algorithms
Classic image check

Description The classic - or default - image check compares the color value of every single expected and actual pixel. If at least one expected pixel differs from the actual (257) denes the pixel the check fails. The option Tolerance for checking images tolerance for comparing pixel values. Purpose This pixel based check is suitable if you expect an exact image match with minimal tolerances or any deviations. Whenever your application renders the component not fully deterministically, this algorithm is not suitable. Example The classic image check doesnt transform the image, thus the result looks identical to the original image.

44.2.

Description of algorithms

724

Figure 44.2: Classic image check

The classic image check is used when the Algorithm for image comparison empty.

(463)

attribute is

44.2.2

Pixel-based identity check

Description This algorithm is similar to the classic algorithm, but accepts an amount of unexpected pixels. It splits every pixel in its three sub-pixels red, green and blue. Afterwards it checks every actual color value against the expected color value. The nal result is the amount of identical pixels divided by the total amount of pixels. The calculated result is checked against an expected value. Purpose If your images are not rendered fully deterministic but you accept a certain percentage of unexpected pixels, this algorithm may be useful. But it is not suitable, if the actual images are used to have shifts or distortions. Example The result image of the exemplary algorithm algorithm=identity;expected=0.95 looks identical to the original image because this algorithm does not manipulate the image.

44.2.

Description of algorithms

725

Figure 44.3: Pixel-based identity check

Parameters

algorithm=identity The Pixel based identity check should be used. expected (optional, but reommended) Denes which probability you expect. Valid values are between 0.0 and 1.0. If not dened, use 0.98. resize (optional) Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false. nd (optional) Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.2.3

Pixel-based similarity check

Description This algorithm splits every pixel in its three sub-pixels red, green and blue. Afterwards it checks every actual color value against the expected color value to calculate a percental similarity. All percental deviations are added up and used for calculation. The nal result is the average deviation over all color values and all pixels. The calculated result is checked against an expected value.

44.2.

Description of algorithms

726

Purpose If your images are not rendered fully deterministic but you accept a certain deviation, this algorithm is a possible candidate for your purpose. If you accept deviations for some pixels, but the average deviation all over the image is small, this algorithm is also suitable. But it is not suitable, if the actual images are used to have shifts or distortions. Example The result image of the exemplary algorithm algorithm=similarity;expected=0.95 looks identical to the original image because this algorithm does not manipulate the image.

Figure 44.4: Pixel-based similarity check

Parameters

algorithm=similarity The Pixel based similarity check should be used. expected (optional, but reommended) Denes which probability you expect. Valid values are between 0.0 and 1.0. If not dened, use 0.98. resize (optional) Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false.

44.2.

Description of algorithms nd (optional)

727

Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.2.4

Block-based identity check

Description This algorithm partitions the image into quadratic blocks with a selectable size. The color value of each of these blocks is calculated as the average of the color values of the pixels the block contains. If the width or height of the image is not a multiple of the block size, the blocks at the right and bottom edge are cropped and weighted accordingly. The actual blocks are checked against the expected blocks. The nal result is the amount of identical blocks divided by the total amount of blocks. Purpose This algorithms purpose is to check an image which only differs at some parts but is identical at the remaining parts. Example The exemplary algorithm algorithm=block;size=10;expected=0.95 results in the following image:

Figure 44.5: Block-based identity check

Parameters

44.2.

Description of algorithms algorithm=block The algorithm Block-based identity check should be used. size Denes the size of each block. Valid values are between 1 and the image size. expected (optional, but reommended) Denes the minimal match probability for the check to succeed. Valid values are between 0.0 and 1.0. If not dened, use 0.98. resize (optional)

728

Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false. nd (optional) Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.2.5

Block-based similarity check

Description This algorithm also partitions the image in quadratic blocks with a selectable size. The color value of each of these blocks is calculated as the average of the color values of the pixels the block contains. If the width or height of the image is not a multiple of the block size, the blocks at the right and bottom edge are cropped and weighted accordingly. The color value of each expected block is checked against the actual block. Their color values are analyzed for percental similarity. The nal result is the average similarity of all blocks with their weight taken into account. Purpose This algorithm is suitable for checking images with similar color variances. Example The exemplary algorithm

44.2.

Description of algorithms algorithm=blocksimilarity;size=5;expected=0.95 results in the following image:

729

Figure 44.6: Block-based similarity check

Parameters

algorithm=blocksimilarity The algorithm Block-based similarity check should be used. size Denes the size of each block. Valid values are between 1 and the image size. expected (optional, but reommended) Denes the minimal match probability for the check to succeed. Valid values are between 0.0 and 1.0. If not dened, use 0.98. resize (optional) Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false. nd (optional) Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.2.

Description of algorithms

730

44.2.6

Histogram check

Description To create a histogram, an image is rst broken into its three base colors red, green and blue. Then the color values for each pixel are analyzed to partition them into a denable amount of categories (known as buckets when talking about histograms). The actual ll level of each bucket is compared to the expected level. The result of the algorithm is a comparison of the relative frequencies of color categories. Purpose Histograms are used for many scenarios. For example it is possible to check for color tendencies or do brightness analyses. However, histograms are not suitable for checking rather plain-colored images. Example The exemplary algorithm algorithm=histogram;buckets=64;expected=0.95 results in the following image:

Figure 44.7: Histogram

Parameters

algorithm=histogram A Histogram should be used for this image check.

44.2.

Description of algorithms buckets Denes how many buckets to use. Valid values are a power of 2 between 2 and 256. expected (optional, but reommended) Denes the minimal match probability for the check to succeed. Valid values are between 0.0 and 1.0. If not dened, use 0.98. resize (optional)

731

Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false. nd (optional) Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.2.7

Analysis with Discrete Cosine Transformation

Description The Discrete Cosine Transformation is a real-valued, discrete, linear, orthogonal transformation which transforms the discrete signal from local range to frequency range. After transforming an image, you can eliminate low-order (fast oscillating) frequencies. The remaining high-order (slow oscillating) frequencies with the steady component (zeroth frequency = 0*cos(x) + y) can now be analyzed. You can dene how many frequencies per basic color should be used for this image check. You can also specify a tolerance to accept cosine-oscillations as identical which actually differ. Low-order frequencies get weighted less than high-order frequencies when calculating the result. Purpose The Discrete Cosine Transformation is suitable for many kinds of image checks, which require certain tolerances. The more frequencies are used for analysis the sharper the image check is. Example

44.2.

Description of algorithms The exemplary algorithm algorithm=dct;frequencies=20;tolerance=0.1;expected=0.95 results in the following image:

732

Figure 44.8: Analysis with Discrete Cosine Transformation

Parameters

algorithm=dct Analysis with Discrete Cosine Transformation should be used for this image check. frequencies Denes how many frequencies to analyze. Valid values are between 0 (steady component only) and the area of the image. The less frequencies are analyzed the more tolerant the check is. The tolerance is also dependent on the size of the image. tolerance Denes the (non-linear) tolerance for accepting different cosine-oscillations as identical. Valid values are between 0.0 and 1.0. The value 1.0 means every image matches every other image, because the maximum difference of each frequency is tolerated. A value of 0.0 means frequencies only match if they are exactly the same. A value of 0.1 is a good starting point because only quite similar frequencies are accepted as identical. expected (optional, but reommended)

44.2.

Description of algorithms Denes the minimal match probability for the check to succeed. Valid values are between 0.0 and 1.0. If not dened, use 0.98. resize (optional)

733

Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false. nd (optional) Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.2.8

Block-based analysis with Discrete Cosine Transformation

Description When using this algorithm the image is rst partitioned into quadratic blocks with (727) a selectable size (see section 44.2.4 ). Afterwards every partition is analyzed (731) using a Discrete Cosine Transformation (see section 44.2.7 ). The nal result is the average of all results of these Discrete Cosine Transformations with consideration of the blocks and their weight. Purpose The Discrete Cosine Transformation used on the whole image deviates strongly in case of signicant brightness differences occurring in the middle of the image because then the steady component (zeroth frequency), which is the highest weighted part, varies strongly. The partitioning circumvents this behavior because now only the affected partitions result in intense deviations while the other partitions stay untouched. Example The exemplary algorithm algorithm=dctblock;size=32;frequencies=4;tolerance=0.1;expected=0.95 results in the following image:

44.2.

Description of algorithms

734

Figure 44.9: Block-based analysis with Discrete Cosine Transformation

Parameters

algorithm=dctblock Blocks for analysis with Discrete Cosine Transformation should be used for this image check. size Denes the size of each block. Valid values are between 1 and the image size. frequencies Denes how many frequencies to analyze. Valid values are between 0 (steady component only) and the area of a block. The less frequencies are analyzed the more tolerant the check is. The tolerance is also dependent on the size of the image. tolerance Denes the (non-linear) tolerance for accepting different cosine-oscillations as identical. Valid values are between 0.0 and 1.0. The value 1.0 means every image matches every other image, because the maximum difference of each frequency is tolerated. A value of 0.0 means frequencies only match if they are exactly the same. A value of 0.1 is a good starting point because only quite similar frequencies are accepted as identical. expected (optional, but reommended) Denes the minimal match probability for the check to succeed. Valid values are between 0.0 and 1.0. If not dened, use 0.98.

44.2.

Description of algorithms resize (optional)

735

Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false. nd (optional) Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.2.9

Bilinear Filter

Description This algorithm shrinks the image to a choosable percental size. Afterwards the image gets resized to its original size by use of a bilinear lter. This lter effects a blurring, because every color value is calculated by use of neighbor pixels. The nal result is the average deviation over all color values and all pixels of this transformed images. Purpose Depending on the chosen sharpness the images loose any desired image information. Thus this algorithm is valuable for nearly any scenario. Example The exemplary algorithm algorithm=bilinear;sharpness=0.2;expected=0.95 results in the following image:

Figure 44.10: Bilinear Filter

44.3.

Description of special functions

736

Parameters

algorithm=bilinear A Bilinear Filter should be used for this image check. sharpness Denes the sharpness of this bilinear lter. Valid values are between 0.0 (complete loss of information) and 1.0 (no loss of information). The sharpness is a linear parameter. This means a value of 0.5 eliminates exactly half (plusmins rounding to entire pixels) of information. expected (optional, but reommended) Denes the minimal match probability for the check to succeed. Valid values are between 0.0 and 1.0. If not dened, use 0.98. resize (optional) Denes, if the actual image should be resized before calculation to match the size of the expected image. Valid values are true and false. nd (optional) Denes an image-in-image search. (736) A detailled description of this parameter can be found in section 44.3.1 .

44.3
44.3.1

Description of special functions

Image-in-image search

Description The image-in-image search allows to nd an expected image within a (larger) image. The check is successful when the expected image can be found anywhere using the dened algorithm. Furthermore, you can determine the position of the match. The following combination of parameters are valid: find=best or find=anywhere

44.3.

Description of special functions find=best(resultX, resultY) resultY) or

737 find=anywhere(resultX,

Purpose The image-in-image search allows to compare images if you dont know the exact position and thus cannot dene an offset. The search can be combined with any algorithm and is thus valuable for any purpose. Example The exemplary algorithm algorithm=similarity;expected=0.95;find=best(resultX,resultY) (725) uses pixel-based similarity check (see section 44.2.3 ) to nd an image of Q as part of the full image. The got image with highlighted region can be found within the run-log. Besides, the variables resultX and resultY are set to the location of the found image.

Figure 44.11: Image-in-image search: Expected image

Figure 44.12: Image-in-image search: Got image

Parameters

nd=best Denes that the best match should be used.

44.3.

Description of special functions nd=anyhwere

738

Denes that the rst match which exceeds the expected match probability should be used. The image-in-image search uses multiple threads and thus nding anywhere is non-deterministic. resultX resultX is the name of a QF-Test variable which holds the x-position of the found image. If a variable for the x-position is dened, a variable for the y-position has to be dened as well (see syntax above). resultY resultY is the name of a QF-Test variable which holds the y-position of the found image. If a variable for the y-position is dened, a variable for the x-position has to be dened as well (see syntax above).

Chapter 45 Result lists

3.2+

45.1

Introduction

Search operations like locating a reference or searching for a particular value can produce large result sets, as can operations like renaming components or procedures. In order to provide a better overview of all affected nodes QF-Test shows a mass result list at the end of such operations. This list holds all nodes that have been touched by the respective operation. Besides providing an overview that list allows you to apply mass operations on all nodes quite easily. Such mass operations could be toggling a mark at all found nodes or deleting all of them from the test-suites. Those operations can be triggered by the Edit menu or a the tables context menu.

45.1.

Introduction

740

Figure 45.1: Sample result list for Locate references

The table comes with a context menu as well. This context menu allows you to perform several actions on the shown nodes. If you select entries in the result list and perform a right mouse-click you can raise some very useful actions from that menu like jumping to the node or disabling it. Depending on the type of the list there might be specic actions which only make sense in that particular context. A list of possible actions is provided in (741) section 45.2 . Such result lists are available for the following actions: Once you press Show result list on the Search or Replace dialog. As result of the Locate reference action for callable nodes or components. As result of any command which updates several nodes. As result of any replacement action from the Replace dialog. As result of several analyzing actions under the Additional node operations menu.

45.2.

Specic list actions

741

Once you open a list of breakpoints under DebuggerList of all breakpoints... . Once a duplicate component id gets inserted or a suite with duplicate components get opened. Once a node transformation changes the type of some child nodes. Once you press EditOpen error list in the run-log. If you update components via Update components all errors will be shown in such a list. If you have lots of entries in the table you can also apply a lter at the top for limiting the nodes to the given lter values. Once you reset the lter you will see all nodes again.

45.2
45.2.1

Specic list actions

All types of lists

Following actions are available on all types of lists: Toggle mark Toggle disabled state Jump to node in test-suite Set a breakpoint Delete a node from the list only Delete a node from test-suite Store the result in a .qcv le, for details see section 45.3
(742)

If the according action affects only one node you can also jump back to this.

45.2.2

Replacing

The replacement list provides the capability of replacing only the selected values.

45.3.

Exporting and loading results

742

45.2.3

Error list

The error list of the run-log provides capabilities to update failing checks in a bunch. It doesnt allow to delete a node from the run-log of course.

45.3
3.5+

Exporting and loading results

In case you have a very large result list that you need to work on in more than one session, you can store its results in an external le using the Export button. The export process creates a .qcv le, which is more or less a CSV le holding the information in a QF-Test internal format. To proceed with your work at a later time, simply load the previously exported .qcv le via OperationsLoad result list... into QF-Test.

743

Appendix A FAQ - Frequently Asked Questions

Evaluation and licensing
1. Is an evaluation version available for download? Yes. Please visit www.qfs.de/en/qftest/download.html. 2. Do I need anything else? Normally a license le is required to run QF-Test. It will run without a license, but it will not let you save any les or load any les that were not provided by Quality First Software GmbH. This is sufcient for getting a rst impression, working through the tutorial and making a rst attempt at running your application under QF-Test. To go beyond that, youll need a license le. 3. So how do I get a license? You can obtain a free evaluation license valid for 4 weeks by lling in the request form at www.qfs.de/cgi-bin/eval. 4. How much does QF-Test cost? License types and prices for QF-Test are listed at www.qfs.de/en/qftest/license.html. 5. Does QF-Test need an additional license server? No. QF-Test handles multi user license management itself.

Support, training and feedback

6. Where do I get help troubleshooting? Before asking for help, please read through this FAQ to see if your question has already been answered. For beginners the learning-by-doing Tutorial proves useful, further questions are answered by the Manual.

FAQ - Frequently Asked Questions

744

Webinars for beginners every Monday 16:00 CEST. Registration via [email protected]. There is a mailing list for QF-Test questions and discussions that you can subscribe to (see www.qfs.de/mailman/listinfo/qftest-list). At www.qfs.de/archive/qftest-list the list archive is available which contains lots of helpful postings (full text search possible). During evaluation period of QF-Test we grant free support via email [email protected] or phone +49 (0)8171 3864820. For customers we offer support with a guaranteed response time in the form of our Software Maintenance Agreement (see www.qfs.de/en/qftest/support.html). 7. What about training for QF-Test? QF-Test trainings for beginners and advanced users in German and English language take place regularly here at QFS. There is also the option for webinar-based or on-site consulting and training. Details can be found at www.qfs.de/en/qftest/training.html. 8. How can I request an enhancement to QF-Test? Enhancement requests should preferably be sent to the mailing list (see above), but they are also welcome anytime at [email protected]. 9. Where do I report a QF-Test bug? Simply send an email to [email protected]. and we will have a look. Please be sure to provide as much information as possible, especially test-suites and run-logs.

Test execution
10. Why do I get warnings of the Form Missing name, I suggest...? This is a feature. Please read chapter 5 about how names are set with setName() and why they are important. The warning helps to nd components that dont have a name. It is not for general use, but the default setting makes people aware of the fact that the reliability of tests can be improved tremendously with little effort. You can turn the warning off with the option Log warning for missing name
(261) (35)

11. How do I run a test automatically from the command line, an ant task or some other kind of script? You can run QF-Test in batch mode through the command line argument -batch . Many other command line arguments can be used to congure the test-run. The exit (133) (539) code of QF-Test reects the outcome of the test. See chapter 16 , section 32.2 and (555) section 32.3 for details. 12. Is it possible to test two applications running at the same time in two different JVMs? Yes, just start two SUT clients with different names. You can then control both of them.
(539)

FAQ - Frequently Asked Questions

745

13. Ive got a long-running test and QF-Test runs out of memory. How can I prevent that? To increase the available memory, start QF-Test with the argument -J-Xmx512m (or any other value, QF-Test uses up to 256 MB by default). On Windows you can alternatively use the QF-Test Java Conguration tool, avaialble from the Windows system menu. On Unix re-running the QF-Test setup script (setup.sh, setup.ksh) also lets you adapt memory usage. Of course the amount of memory you can use depends on your computer. (2) Please refer also to chapter 1 for further details.
Note

If the memory chosen is too high, Java will not start and therefore also QF-Test. There are a number of ways to reduce the memory use of QF-Test: Make sure that the option Create compact run-log vant nodes are removed from run-logs.
(282)

is checked so that all irrele-

QF-Test keeps 4 run-logs accessible from the Run menu by default. Keep the (276) option Automatically save run-logs active so that QF-Test can save these runlogs to les and release their memory. Close run-log windows that you no longer need so that the memory for these runlogs can be reclaimed. For long-running tests the best option is to create split run-logs (which QF-Test uses as default) so QF-Test can save partial run-logs to les instead of holding the (67) entire run-log in memory. See section 8.1.4 for details. If the option Dont create run-log is checked, no run-log will be generated at all. This should also be used with caution since it can be extremely difcult to interpret what happened without the help of the run-log. Use split run-logs instead. If your SUT prints lots of output you can reduce the number of old clients that are (252) kept around by changing the option Number of terminated clients in menu . 14. Hard mouse events und dragndrop operations do not work awlessly, components cannot be found, the run-log contains black or corrupt screenshots. What do I have to take care about for test execution? GUI testing requires an unlocked, active desktop. That is the only way to ensure that the SUT behaves the same as if a normal user interacts with it. To make sure that your test environment complies with this requirement, youll probably need to tweak its setup. This holds true notably for continuous integration and build (168) tools like Jenkins/Hudson (c.f. chapter 20 ). Otherwise you can run into serious trou(133) ble during Test execution , for example black screenshots in the run-log (c.f. section (63) 8.1 ), failing dragndrop operations, non-working hard mouse events or even problems
(282)

FAQ - Frequently Asked Questions

(35)

746

during component recognition (c.f. chapter 5 ). Java WebStart applications fail to start up. Before running a GUI test, check whether the following conditions are met: QF-Test and the SUT have to run within an active, unlocked user session. The Test must not be executed within the Windows service session. It also must not run without an user session. During the Test execution with Jenkins/Hudson or similar tools you have to ensure that the Jenkins/Hudson Windows slave does not get started as service. It has to be started either via Windows Autostart or Windows Task Scheduler within a real user session. Please ensure that you select a valid user account at the Security options and that you have Run with highest privileges disabled. The selected (133) user must be logged in during the Test execution , e.g. by automatically logging in, and the desktop must not be locked in any way. RDP connections must not be minimized or closed, that would result in a locked session. Instead of RDP you should use VNC, Teamviewer or similar tools to observe the running tests. RDP must not be used for the initial user login and start of the related session. Technical background: The keywords are session 0 isolation. This means that every user has its own session ID, starting with 1. The session with ID 0 is reserved for services and applications without user context. It is restricted in its functionality and applications which run in this session are isolated from other sessions. Applications with GUI cannot be displayed within this session. When running a GUI application in the service session, it will not be rendered correctly and thus may not behave as expected. If you search for session 0 isolation with your preferred search engine you will get extensive information, especially for Windows Vista and newer. Due to security reasons, the well-known workaround for Windows XP and Windows 2000 via tscon.exe and redirection of session 0 is not working anymore since Windows Vista. To get round all that problems you should consider to work with virtual machines, especially from a security point of view. If you execute the tests on a virtual machine, the above mentioned requirements apply for this virtual machine only, not for the host. You can lock the host and dont care about session management on the host.
(133)

Scripting
15. How can I access objects in my application that are not components? You cannot get an object out of the blue, some kind of registry must exist that returns the object from a class static method. Typical examples in the

FAQ - Frequently Asked Questions standard Java API are java.lang.Runtime.getRuntime() java.awt.Toolkit.getDefaultToolkit(). 16. Fine, but how do I use these from Jython or Groovy respectively?

747 or

This is standard Jython stuff: Simply import the class and call its methods, e.g. from java.lang import Runtime runtime = Runtime.getRuntime() In Groovy the package java.lang gets imported even automatically: def runtime = Runtime.getRuntime() You can access any class of your application the same way, provided the class is declared public. Note that you must use an SUT script node, not a Server script node. 17. How can I access additional Java classes from a script? To make additional Java classes available to Jython and Groovy, put them in a jar le (587) and place that in QF-Tests plugin directory (see section 37.2 ). 18. How can I throw an exception from a script? There are two ways to do that: Jython: raise UserException(Some arbitrary message) Groovy: import de.qfs.apps.qftest.shared.exceptions.UserException throw new UserException(Some arbitrary message) rc.check(condition, Message, rc.EXCEPTION) will raise an exception only if the condition is false. 19. Which external editor should I use? Thats a matter of taste, to some even religion. A comprehensive list of editors for all kinds of operating systems that support Python syntax highlighting and other goodies is available at http://www.python.org/cgi-bin/moinmoin/PythonEditors. There are probably dozens of suitable editors with syntax highlighting for both Jython and Groovy - jEdit (http://www.jedit.org) is only one of them.

Web
20. Why is this small le-upload/le-download dialog showing up before the real le selection dialog gets displayed? Before a le selection dialog is displayed the QF-Test browser is showing up a small dialog with an OK-/Cancel-button. This helper dialog is needed by QF-Test to get the data out of the le selection dialog since this dialog is created natively by the operating system. After clicking the OK-button in the helper-dialog the native le selection dialog is displayed and you can enter the lename or select the le directly.

FAQ - Frequently Asked Questions

748

21. Why does the replay of an already recorded le-upload-/le-download-sequence fail if I use another browser and how can I bypass that issue? Depending on the implementation of the le-upload-/le-download on a specic page the replay might get a bit complicated and even vary between different browsers. The QF-Test standard library qfs.qft contains a special procedure qfs.web.input.fileUpload to handle this. Please use this procedure instead of your recorded sequence if you encounter problems during replay. 22. I get an error page notifying me about untrusted certicates. Unfortunately the standard dialog to add an exception is not working properly. How can I solve this problem? This problem wont occur in QF-Test version 3.5.1 and higher, because SSL certicates are now accepted automatically. Please use the following workaround if you are using an older version of QF-Test. There are different approaches to add a certicate using refox. Solution 1: Open the URL chrome://pippki/content/certManager.xul in a QF-Test browser window. This will show the certicate manager where you can add the certicate. After pressing OK the whole browser window will close. When loading the URL which needs a certicate again the page will load without a certicate error. Solution 2: You can run a normal refox from the command line with the following parameters firefox -profile [path to your userprofile]/.qftest/mozprofile (e.g.: firefox -profile c:/Users/user1/.qftest/mozprofile) (Please close all other running refox instances before executing this command.) Load the URL in the browser and add the certicate to the trusted certicates. When loading the URL which needs a certicate again the page will load without a certicate error. 23. Im getting an OutOfMemoryError for the browser. How to increase memory for the QF-Test browser?

FAQ - Frequently Asked Questions

749

In general it is recommended to create the setup sequence by use of the quickstart wizard. In the resulting sequence within the step Start browser without window there is a Start browser node in which you can specify the maximum memory as part of the Java VM parameters via e.g. -Xmx256m, which means 256 MB maximum memory. Current default is 128 MB.

Figure A.1: Set browser maximum memory

24. The web-application opens a popup-window. If I try to close this window using a Window event WINDOW_CLOSING the main-window is closed instead of the popup-window at times. How can I ensure that the correct browser-window is closed ? You need to set the Name of the browser window-attribute in the Wait for document to load-node which is corresponding to your popup-window as for example popup. The corresponding Wait for document to load-node can normally be found directly below the mouse-click (or other equivalent events) which opened the popup.

FAQ - Frequently Asked Questions

750

Additionally set the Name of the browser window-attribute of your Web page-node (located under windows and components) to ${default:windowname:}. Thereby the name of a browser window is considered for object recognition. Finally insert a Set variable-node directly before the Window event-node and dene windowname=popup so that the following Window event is replayed on the popup-window and closes the window correctly. It is recommended to delete the variable right after the Window event because QF-Test would execute the subsequent steps on the (already closed!) popup-window as well. To work around this you just need to add a second Set variable-node for windowname with an empty default value. A more convenient way to achieve that would be to pack the Window event in a sequence and dene the variable in the variable denitions table of the sequence. The windowname is then dened only in the scope of the sequence and you wont need to delete the variable.

751

Appendix B Release notes

B.1 Version 3.5.3

New features:
Web

Support was added for the AJAX toolkit ICEfaces. See section 38.2 for further information and a list of the supported AJAX toolkits and their versions. QF-Test can now automatically detect whether a web page is based on one of the supported AJAX toolkits and installs the respective resolver. Support was added for Firefox versions 24 and 25. The QF-Test demo video on the welcome dialog now links to the updated video on the QFS home page which is fully redesigned, including spoken explanations. It is now possible to copy and paste table cells back and forth between QF-Test tables and Excel or OpenOfce spreadsheets. When changing the name of a parameter in callable nodes like a Procedure QF-Test now offers to automatically update the variable name in all calling nodes. The standard library qfs.qft contains several new procedures for working with standard Swing dialogs based on the JOptionPane class in the package qfs.swing.optionpane. Bugs xed: Stability of startup of WebStart based clients and compatibility with older WebStart versions was further improved.

(625)

Web

Web

B.2.

Version 3.5.2

752

Canceling a long search did not work reliably and could even lead to a deadlock in some cases. Replace operations for the regular expression .* could lead to double replacements in some cases. The associated label for components was sometimes not determined correctly if the label was placed to the right of the component.

B.2

Version 3.5.2

Important changes: Handling of the security dialogs for Java WebStart and applets had to be changed again. Security warnings are now closed implicitly before connecting to QF-Test and it is possible to permanently accept a certicate during this step. The details for this operation can be congured via the procedure qfs.swing.startup.setSecurityDialogSpec. The default delays for WebStart and applets also changed as documented in the procedure qfs.swing.startup.setDelays. New features:
SWT Web

Eclipse/SWT support was updated for the nal Eclipse version 4.3 Kepler. Support was added for Firefox version 23 and updated for version 22. Testlink support was updated for Testlink version 1.9.7. Wait for component to appear nodes now automatically handle waiting for the respective GUI engine in addition to implicitly waiting for the SUT to connect. and Packages Nodes for Procedures crossed out in the tree view.
(352) (359)

marked as @deprecated are now

Web

The new procedure qfs.web.general.integrateFirebug in the standard library qfs.qft loads Firebug Lite into the running browser which can be very helpful when writing resolvers. The keyboard shortcut Shift-Control-I can now be used to quickly pack one or (370) more steps into an If sequence. Bugs xed:

B.3.

Version 3.5.1

753

The Windows executables qftest.exe and qftestc.exe failed to read the congured Java version, memory and language from the Windows registry. Saving test-suites and run-logs and other IO operations are now signicantly faster on slow network shares. When changing the currently executed node via Continue execution from here in the test debugger, QF-Test is now smarter in keeping as much of the current state of execution as possible, including the previously broken case when continuing (352) from a Procedure node.
Web

On Linux systems Firefox would crash when terminated while a dialog was still showing. The recently added support for ZK and PrimeFaces was ne-tuned and xes subtle problems with specic Trees, TreeTables and other components. Associated labels of Swing components, recorded as qfs:label extra feature, are (515) attribute. For web and SWT components this now used for generating the Id was already the case. In the QF-Test daemon DaemonTestRunListener instances were not unregistered correctly which could slow down daemon execution after a call like qftest -batch -calldaemon -verbose ... The SUT can now implement an RMIClassLoaderSpi without interference from QF-Tests RMI communication.

Web

Swing

B.3

Version 3.5.1

Important changes: The Jython version provided with QF-Test was rolled back to Jython 2.5.1 because Jython 2.5.2 caused strange but reproducible problems with some SUTs. Jython is mainly used as glue code, so stability and non-interference with the SUT are far more important than compatibility with newer Python language features and modules, so the Jython version for QF-Test will remain frozen at 2.5.1 for the foreseeable future. New features:
Web

New support was added for the AJAX toolkits PrimeFaces and ZK and support (625) for Eclipse RAP was updated to RAP version 2.0. See section 38.2 for further information and a list of the supported AJAX toolkits and their versions.

B.4.
SWT Web

Main new features in version 3.5

754

Eclipse/SWT support was added for Eclipse version 4.3 Kepler. Support was added for Firefox versions 21 and 22. The new procedure qfs.swing.filechooser.selectFiles can be used to select several les in a multi-select enabled Swing JFileChooser. The new Browser HTTP request node can be used to replay HTTP GET or POST request directly in the running browser. In contrast to the existing (500) Server HTTP request node, which interacts directly with the server, the Browser HTTP request can be combined with event and check nodes. HTML reports now contain pie charts to visualize the number of successful, failed and skipped tests. A click on the chart brings up a more detailed graph. The directory structure of reports has been cleaned up by moving all images and screenshots to sub-directories.
(504)

Web

When started from QF-Test Firefox now automatically accepts SSL certicates for HTTPS connections, limited to the current session. Testlink support was updated for Testlink version 1.9.4. Bugs xed: The initial connection sequence for applets and WebStart based applications had to be updated after security restraints were again tightened in recent JRE updates from Oracle. The stability of applet and WebStart startup should now be less dependent on timing and the procedure qfs.swing.dialogs.handleSecurityWarning is now able to handle all current security warnings for applets and WebStart. Exceptions caused by CSV or Excel les with inconsistent data are now handled cleanly. Updating components with Item
(519)

nodes could cause an exception.

B.4

Main new features in version 3.5

Following is an overview over the main new features that have been implemented for QF-Test since the last production version 3.4.

B.5.

Version 3.5.0
Further info chapter 10 (625) section 38.2 section 5.4
(39) (81)

755

Description Grouping and managing test-suites in projects Support for AJAX toolkits ICEfaces, PrimeFaces, Vaadin and ZK Better abstraction with generic classes for web pages Full sub-item support for several AJAX toolkits Secure daemon communication via SSL Improved component recognition without names thanks to associated label Integration with test management tool QMetry Several extensions to the standard library qfs.qft Groovy updated to version 2.1.0

section 38.2 (139) (695) section 16.2 , chapter 40 (516) Attribute Extra feature section 19.3 Standard library
(160)

(625)

Table B.1: New features in QF-Test version 3.5

B.5

Version 3.5.0

New features:
Web SWT

Support was added for Firefox version 20. Eclipse/SWT support was updated for Eclipse versions 4.2.2 and 3.8.2. Groovy was updated to version 2.1.0. Explicit references between test-suites belonging to the same project are now resolved automatically provided that those test-suites have been saved with QF-Test version 3.5.0 or higher. Multi-line values can now be edited directly in QF-Tests table cells. Extra mouse buttons can now be used to move backwards/forwards between recently selected nodes. Double-clicking on nodes that reference other nodes now navigates to the target, (355) (352) e.g. from Procedure call to Procedure . Test-suite and run-log les can now be dragged onto QF-Tests main window to open them.

B.5.

Version 3.5.0

756

The new procedure qfs.swing.dialogs.handleSecurityWarning in the standard library qfs.qft closes the security dialogs displayed by current JREs during startup of applets or WebStart based applications. The procedure qfs.web.browser.general.startAsNewProcess was updated to include security dialog handling. Another new procedure named qfs.swing.startup.startWebstartSUT can be used to run WebStart and connect to the SUT, also handling the security dialog automatically. The new procedure qfs.utils.files.readExcelFile can be used to read data from an excel sheet into Jython or QF-Test variables.
Web

The Server HTTP request node now offers an optional timeout and can return the HTTP status code. Parameters are now encoded during replay instead of recording, making it easier to specify values manually. Older test-suites are converted automatically during loading to reect this change. The default names recorded for GEF items are now more meaningful and result in improved recognition. QF-Test now integrates with the test management tool Klaros. See section 19.4 for further information.
(163)

(500)

SWT

The error list for a run-log now includes a timestamp column and shows information about expected errors. , input to the SUT is no longer blocked. When pausing a batch test via Alt-F12 If the new command line argument -dontkillprocesses is specied in batch mode, QF-Test will not explicitly terminate processes started as part of the tests. Bugs xed: A memory leak in case of large projects with many test-suites and complex includes has been plugged. A workaround has been implemented for a bug in recent JRE security updates that could cause batch tests to simply stop in the middle of execution. Variables could accidently be removed from component IDs when xing references automatically. In special cases externalized run-logs could be missing from run-logs saved as .qzp.
(542)

B.5.

Version 3.5.0

757

When opening an externalized log on its own, split logs contained therein could not be accessed correctly. A race condition during automatic project refresh could mess up the project tree view. The native le chooser for Windows is now implemented based on SWT instead of XFileDialog. This xes issues when opening directories and when changing the le lter during Save as... operations. In batch execution of multiple test-suites, global variables and resources are no longer cleared before the execution of each test-suite unless the new command (542) line argument -clearglobals is specied.
Web Web Web

Firefox on Linux no longer crashes on shutdown. For some MIME types downloading les did not work correctly with Firefox. Checks for the enabled state could fail in Internet Explorer if the state was set via Javascript. Waiting for the absence of an element could fail if the element just became invisible instead of being removed completely. Variables in Data binder nodes are now expanded immediately during execution of (330) the containing Data driver node so that local variables bound in the Data driver are evaluated correctly. in a Finally A Return (379) Try node.
(358) (384)

Web

node could hide an exception thrown in the containing

Using Java 7 on Mac systems could cause an exception during startup of QF-Test. In case a process simply refuses to be killed, the Stop client node did not remove the process from the Clients menu, thus preventing starting a new process under the same name. Image checks based on algorithms were not always shown correctly in the report. In case dependency resolution fails due to an exception during setup even after un(90) winding the complete dependency stack for error escalation (see section 12.3 ), QF-Test now cleans up after the nal attempt so that the next test-case starts with a clean slate. Changes to the location of the search dialog and other dialogs were often ignored instead of automatically saved to the user preferences.
(415)

B.6.

Version 3.5M3

758

B.6

Version 3.5M3

New features:
Web

Support was updated for Firefox versions 18 and 19. The project concept now includes support for sub-projects. See chapter 10 further information.
(81)

for

Web

are Javascript error lters dened in the option Errors that should be ignored now applied to the complete message including the URL and line number of the source of the error. Several checks were added for Eclipse JFace components like FormText, Link or Hyperlink. Bugs xed: The responsiveness of QF-Test severely suffered when opening many test-suites. Refresh of large projects on slow lesystems like Windows network shares is now signicantly faster. If several projects were opened, automatic refresh of the project hierarchy could mess up the project tree.

(273)

SWT

SWT

Geometry of events relative to an SWT Shell now includes the window title and borders. Highlighting widgets inside CTabFolders in Eclipse 4.2 now uses the correct offset. Errors happening outside a test-case were not listed correctly in the report if the containing test-set contained several test-cases.

SWT

Web

Downloading les with Firefox could cause a Javascript error if the le mimeTypes.rdf was missing from QF-Tests Mozilla prole. This le is now created on the y if necessary. Closing the browser window of Internet Explorer 10 could cause the browser to deadlock instead of terminate. QF-Tests embedding of Internet Explorer 10 caused the browser to run in IE9 mode. Now it starts up in IE10 mode by default with the usual settings for compatibility mode enabled.

Web

Web

B.7.
Web

Version 3.5M2

759

Replaying [Tab] keystrokes on Firefox did not always trigger the correct focus traversal. Conicts caused by an SUT using a Groovy 2 version other than the version provided with QF-Test are now resolved.

B.7

Version 3.5M2

New features:
SWT

Eclipse/SWT support was updated for Eclipse versions 4.2.1 and 3.8.1. Daemon communication is now secured via SSL. See chapter 40 about how to increase security by creating an individual keystore or how to disable this feature in case you need daemon interaction with older QF-Test versions. QF-Test generated JUnit reports now contain <skipped> tags, enabling build tools like Hudson/Jenkins to correctly report skipped tests. Bugs xed: License usage for idle QF-Test daemons was not determined correctly.
(695)

Swing

Replaying certain events in batch mode could switch QF-Test into recording mode, causing weird problems and signicant loss of performance. Timing for startup of Java WebStart based SUTs has been further improved to signicantly increase the stability of this setup. Some focus transitions were not triggered correctly when replaying events in a browser. The lename shown in a download dialog was not always correct. Downloads in a Vaadin based SUT could cause an exception. Internet Explorer 10 now reports the correct version information. Recording of drag and drop operations on CTabFolders in Eclipse 4.2 was broken. In some cases the height of SWT List items was not calculated correctly, leading to the wrong item being addressed in case of large lists with many items. The new Extra feature attribute qfs:label was sometimes not determined correctly in the presence of Swing JSlitPane or SWT SashForm components.

Swing

Web

Web Web Web SWT SWT

B.8.

Version 3.5M1

760

Handling of an SUT with combined SWT/AWT GUI was broken. Due to an error introduced in version 3.4.11 QF-Test was treating this as two SUT clients instead of one client with two GUI engines.

B.8

Version 3.5M1

New features: With the introduction of projects (see chapter 10 ), large sets of test-suites become much more manageable and updating reverse includes is now mostly a thing of the past.
Web Web
(81)

The increasingly popular AJAX toolkit Vaadin is now fully supported. The integration of QF-Test with various AJAX toolkits has been brought to a new level. Generic classes provide a unied view that abstracts from the technical details, focusing on typical UI elements like Tree or Table that users are familiar with. This even includes the concept of sub-items so that addressing table cells or tree nodes is now just as easy as with Swing or SWT. This level of support is currently available for Ext JS 4, RAP, RichFaces and Vaadin. Support was added for Firefox version 18. For SUTs where only few components are named - if any - component recognition (516) has been improved by adding a new Extra feature called qfs:label. It represents a best match for a label, based on various algorithms including a search for the nearest label for components that dont have one of their own. Classical forms consisting of text elds with labels next to them benet most from it. Handling the opposite case, where the same component name is assigned to more than one component, is now also handled via via the new option Validate (242) component recognition during recording . QF-Test now integrates with the test management tool QMetry. (160) 19.3 for further information. See section

Web

Components in QF-Tests detail view are now sized dynamically depending on the contained data. Combined with improved scrolling this vastly improves creating and maintaining test-suites. The menu item ReplayReset all can be used to stop the test, reset dependencies and global variables and kill all clients with a single action.

B.8.

Version 3.5M1

761

On Windows QF-Test now uses the native le chooser by default, congurable via (221) the option Use native le chooser on Windows systems . There are new operations to search for various kinds of duplicates, empty tests or illegal characters. The result dialogs for these and similar operations as well as searches have more options and exported results can now be read back. Search and replace operations now cover whole projects and can be interrupted in case they take too long. Tables holding variable denitions or parameter lists can now be sorted. It is now possible to navigate from a component id specied in a variable table directly to the component by right-clicking the respective table cell. Parameters of procedures can now be added to or removed from the variable denitions of all callers via the context menu of the parameter table. Recording procedures for windows with many components now works signicantly faster. Whitespace at the beginning or end of variable names is now generally removed to avoid confusion. ) to There is a new hotkey (congurable via Hotkey for recording , default F11 start and stop recording directly in the SUT. The previous default binding for F11 , . component recording, has been moved to Shift-F11 Theres a new kind of check for tree nodes that can handle whole (sub-)hierarchies at once, including indentation. Highlighting components in the SUT for event nodes now works even if the Component node for the event is not located in the same test-suite. Killing the SUT and other processes now works much faster. Leading and trailing whitespace in Feature attributes is now generally ignored. This implementation is fully backwards compatible, so it doesnt hurt if features were recorded with whitespace before. Line numbers are now displayed for tables where this is useful, for example in (334) (452) Data table or Check items nodes. In places where accidentally inserted TAB or line-break characters can do harm, most notably in variable denitions or checks, they are now represented as graphical symbols. This can be turned off via the option Show symbols for tabulator and (224) linebreaks .
(516) (233)

B.8.

Version 3.5M1

762

New packages and procedures in the standard library qfs.qft The new qfs.autoscreen package holds procedures for locating and interacting with components on the screen based on some previously captured image. Several new utility procedures: qfs.utils.files.compareFiles qfs.utils.variables.deleteGlobalVariablesExcept qfs.utils.variables.deletePropertyGroup qfs.utils.variables.deleteOnePropertyVariable qfs.utils.variables.listAllMembersOfPropertyGroup qfs.utils.xml.loadPropertiesFromXML qfs.utils.sleep
Swing

Various new procedures for interacting with Swing components: qfs.swing.checkbox.getValue qfs.swing.combobox.getIndexOfItem qfs.swing.combobox.getValue qfs.swing.combobox.getValueOfItem qfs.swing.general.doSelect qfs.swing.menu.selectMenuItemInHierarchy qfs.swing.menu.selectCheckMenuItemInHierarchy qfs.swing.radiobutton qfs.swing.radiobutton.getValue qfs.swing.radiobutton.select qfs.swing.tabbedpane.getIndexOfTab qfs.swing.tabbedpane.getValueOfTab qfs.swing.text.getText Various new procedures for interacting with SWT components: qfs.swt.checkbox.getValue qfs.swt.combo.getIndexOfItem qfs.swt.combo.getValue qfs.swt.combo.getValueOfItem qfs.swt.general.doselect qfs.swt.list.getIndexOfItem qfs.swt.menu.selectMenuItemInHierarchy qfs.swt.menu.selectCheckMenuItemInHierarchy qfs.swt.radiobutton qfs.swt.radiobutton.getValue qfs.swt.radiobutton.select qfs.swt.table.getIndexOfColumn qfs.swt.table.getRowIndexOfCell qfs.swt.tabfolder.getIndexOfTab

SWT

B.8.

Version 3.5M1 qfs.swt.tabfolder.getValueOfTab qfs.swt.text.getText qfs.swt.tree.multicolumn qfs.swt.tree.multicolumn.getRowIndexByCellText qfs.swt.tree.multicolumn.getIndexOfNode

763

Web

Various new procedures for interacting with web components: qfs.web.checkbox.getValue qfs.web.radiobutton qfs.web.radiobutton.getValue qfs.web.radiobutton.select qfs.web.select qfs.web.select.getItemCount qfs.web.select.getIndexOfItem qfs.web.select.getValueOfItem qfs.web.select.getValue qfs.web.select.selectValue qfs.web.table qfs.web.table.checks qfs.web.table.checks.checkNumberOfRows qfs.web.table.checks.checkNumberOfColums qfs.web.table.recording qfs.web.table.recording.activateItemRecording qfs.web.table.recording.deactivateItemRecording qfs.web.table.getColumnCount qfs.web.table.getIndexOfColumn qfs.web.table.getNameOfColumn qfs.web.table.getRowCount qfs.web.table.getRowIndexOfCell qfs.web.table.getValueOfCell qfs.web.table.selectCell qfs.web.text.getText More web utilities, specically qfs.web.ajax.uninstallToolkitResolver, the new package qfs.web.checks.style with several procedures for checking CSS style attributes of DOM nodes and a complete rewrite of qfs.web.browser.general.startAsNewProcess. Bugs xed: The modied connection mechanism still crashed Java WebStart on some occasions and has been further ne-tuned to prevent that. The option settings for Errors that should be ignored system conguration le.
(273)

Web

were not saved to the

B.9.

Version 3.4.11

764

B.9

Version 3.4.11

New features: QF-Test is now compatible with Java applets running on JRE version 1.6.0_35 or higher or JRE version 1.7.0_7 or higher. A security x from Oracle broke the existing applet connection mechanism. The new mechanism should also improve stability of Java WebStart based SUTs.
Web Web

Support was added for Firefox version 17. Via the new option Errors that should be ignored it is now possible to ignore specic Javascript error messages while still treating others as an error. When copying multiple lines of a table in the detail view of QF-Test these are now also copied to the system clipboard as a list of TAB-separated values, thus enabling pasting into a spreadsheet. The new procedure qfs.utils.logNativeMemory in the standard library qfs.qft can be used to get information about the native memory use of an SUT client. Windows executables and installers created by QFS are now signed with an ofcial certicate for smoother integration with Windows 7 and Windows 8. Bugs xed:
(273)

Web Web

Downloads of large les were sometimes truncated. The new INPUT types in HTML 5 are now mapped to INPUT:TEXT so that tests remain independent of whether the browser supports them. The command line argument -runlogdir <directory> daemon calls via qftest -batch -calldaemon.
(549)

now also works for

Empty formatted cells in an Excel 2010 sheet are now handled correctly by the (342) Excel data le node.
Web

Toolbar icons for QF-Tests browser wrapper are now transparent and no longer look horrible on Windows 7.

B.10.

Version 3.4.10

765

B.10

Version 3.4.10

New features:
Web

Support was added for Firefox versions 15 and 16. The performance of complex QF-Test operations like nding references has been signicantly improved for large projects on slow lesystems like network shares. Bugs xed:

Web

Crashes of IE and Firefox were further reduced.

B.11

Version 3.4.9

New features:
SWT Web

Eclipse/SWT support was added for Eclipse Juno versions 4.2 and 3.8. Support was added for Firefox version 14. Several new transformation options for nodes in the test-suite were introduced. Bugs xed:

Web Web

A signicant memory leak in QF-Tests browser wrapper was plugged. Several causes for browser crashes were xed, both for Internet Explorer and Firefox. Component recognition now also works correctly with components recorded in a at hierarchy, even if the web page contains FRAME nodes. The progress bar in Internet Explorer always remained visible even when a page was completely loaded. If the path for the Firefox installation directory contained special characters, Firefox could not be started correctly from QF-Test. On Windows, popup menus in an Eclipse/SWT application were sometimes mistakenly considered to be invisible. Files are no longer overwritten without asking when an image is saved from a (460) Check image node.

Web

Web

Web

SWT

B.12.

Version 3.4.8
(523) (522)

766

Component group and Window group nodes are now handled and integrated correctly when importing components from a test-suite. Another special case for escaping $ characters in component names has been xed.

B.12

Version 3.4.8

New features:
Web SWT

Support was added for the current Firefox version 13. Eclipse/SWT support was added for Eclipse versions 3.8M7 through 3.8RC2 and experimental support for Eclipse version 4.2M7 through 4.2RC2. Support for the Richfaces toolkit was improved. The Excel data le Bugs xed: Escaping of $ characters in component names was broken in the component and procedure recorder.
(342)

Web

node now also handles Excel les with the .xlsm extension.

Web

Enabled checks did not work correctly with Internet Explorer 9 running in IE 7 compatibility mode. Firefox could crash when closing a browser window. When recording procedures the calls for SELECT elements were created incorrectly.

Web Web

B.13

Version 3.4.7

New features:
Web

Support was added for the current Firefox version 12. The new Procedure qfs.swt.ctabfolder.openViewMenu can be used to generically open the ViewMenu of a CTabFolder based on the tab name, both for Eclipse 3.x and Eclipse 4.

B.14.

Version 3.4.6

767

Bugs xed: Script errors were not formatted correctly when shown in the error dialog. In the run-log those errors were shown correctly. Excel 2010 les used in a data driver with an Excel data le correctly.
Web
(342)

node were not closed

Theres a new workaround for the workaround for the missing MSVC manifest in a DLL from Firefox 10 and higher. Problems with another missing MSVC DLL on Windows XP and deadlocks when loading a Flash component in Firefox on Windows have been xed.

B.14

Version 3.4.6

New features:
SWT

Eclipse/SWT support was added for Eclipse versions 3.7.2 and 3.8M6 and experimental support for Eclipse version 4.2M6. The Procedures qfs.swt.ctabfolder.closeTab qfs.swt.ctabfolder.checkTabIsDirty now also support Eclipse 4. Bugs xed: The x for SUTs that use RMI with a custom implementation of the Service Provider Interface (SPI) could cause a subtle class loading problem that has been xed. Deleting the last row in a variable table in QF-Test could lead to an exception when subsequently trying to move a table row. and

Web Web

QF-Tests handling of Javascript errors could cause Internet Explorer to crash. If QF-Test was accidently started with the bin directory of an older QF-Test version as its working directory, starting a browser on Windows could fail due to conicts between DLL versions. Use of a special API in Internet Explorer can lead to very subtle deviations in Javascript execution. QF-Test no longer uses this interface. On some systems the workaround for the missing MSVC manifest in a DLL from Firefox 10 and higher could cause a problem with another DLL and prevent QFTest from starting Firefox.

Web

Web

B.15.

Version 3.4.4

768

B.15

Version 3.4.4

New features:
Web SWT

Support was added for current versions of Firefox 9 and 10. Eclipse/SWT support was added for Eclipse version 3.8M5 and experimental support for Eclipse version 4.2M5. New checks were introduced for SWT Decorations, small icons typically shown next to a text eld. You can now check the visibility, icon and tool tip of a Decoration. Iteration ranges can now also be used when setting up data at script level via the databinder module or by calling one of the Procedures qfs.databinder.bindList or qfs.databinder.bindSet in the standard (121) library qfs.qft. See section 14.4 for further information. The thumbnail images shown for screenshots in the HTML report are now of higher quality. Bugs xed: A small change in QF-Test version 3.4.3 regarding the RMI communication between QF-Test and the SUT was undone because of possible side-effects when the SUT was using RMI itself. When using Check image nodes with an advanced algorithm and a timeout QFTest always used up the full timeout even when the image matched immediately. When a Check node with a timeout fails, QF-Test used to show the initial state of the target component in the run-log. Now it shows the state at the end of the timeout period instead. When using Drag&Drop to move nodes within QF-Test the tree could jump erratically in some cases. Trying to remove the same node twice from a search result dialog could lead to an exception. In the standard test-suite library qfs.qft the Procedure qfs.shellutils.touch now correctly updates the timestamp for existing les and the Procedure qfs.autowin.getURLFromIE now also works with Internet Explorer version 6.
(460)

SWT

B.16.
SWT

Version 3.4.3

769

When recording a mouse click on an SWT ToolItem that opens a drop down menu QF-Test now records the coordinates so the event can replay correctly. If an alert dialog was shown after closing a modal browser window, the main browser window could remain blocked after closing the alert. Javascript errors in Internet Explorer were not logged correctly in QF-Tests runlog. In some cases the initial loading of a web page did not succeed. Highlighting the selected component in a browser on Linux has been improved. Web pages with a Flash element were sometimes not displayed correctly on Linux. The name recorded for le upload on Linux contained the Windows le separator character \ There were problems with browser startup on special Windows versions if QFTest was installed below a directory with blanks in its name, e.g. C:\Program Files (x86)

Web

Web

Web Web Web Web

Web

B.16

Version 3.4.3

New features:
Web

Support was added for current versions of Firefox 8 and 9. The Excel data le
(342)

node now supports reading .xslx les in Excel 2010 format.

The performance of some advanced image check algorithms was signicantly improved. It is now possible to fetch option values in a script via the new method (589) rc.getOption. See section 37.6 for details. Bugs xed: When switching test-suites by clicking a suite tab the keyboard focus was not correctly restored. The keyboard shortcut Ctrl-D for setting a bookmark did not work. Jumping from a node in the debugger window to the same node in the test-suite now works again. In some special cases QF-Test could kill the wrong process on Windows when trying to terminate the SUT.

B.17.

Version 3.4.2

770

B.17

Version 3.4.2

New features:
Web

Full support was added for Firefox 6 and experimental support for Firefox 7. For easier integration with continuous integration tools like Hudson/Jenkins or TeamCity the exit code of a batch run can be forced to 0 even in case of warnings, errors or exceptions with the help of the command line arguments (543) (543) -exitcodeignorewarning , -exitcodeignoreerror and (543) -exitcodeignoreexception .

Web

If the option Replace no-break space with normal space is activated, QF-Test now represents no-break space characters in a web page as normal space char(514) acters in checks and Component node attributes to avoid confusion. QF-Test now supports capture and replay of expand and collapse events for SWT ExpandItems. Bugs xed:

(270)

SWT

SWT

The integrated help viewer of Eclipse uses a separate SWT Display on Windows which caused problems during replay. It is now consistently represented with (510) GUI engine swt1. Direct execution of Javascript code now also works in Internet Explorer 9 despite stronger security restrictions. The AJAX resolver for GWT has been updated to version 1.4.0 to x ltering of (625) DIV nodes from the component hierarchy. See section 38.2 for details about AJAX resolvers and their versioning scheme. Opening a modal popup window could crash Firefox 6 on Windows. Double clicks are now recorded correctly in Internet Explorer. A Fetch text node for a SELECT node now retrieves the text of the selected OPTION node and Fetch text nodes for OPTION elements represented as items are working correctly. Printing unicode strings in Jython could cause problems when running in batch mode. The content of Data table
(334) (471)

Web

Web

Web Web Web

nodes was missing in the run-log.

B.18.

Version 3.4.1
(411)

771

A Wait for client to connect node with a very short timeout did not throw a (530) ClientNotConnectedException when single-stepping in the debugger. The QF-Test daemon was leaking threads when starting a remote SUT process for use with the procedure qfs.daemon.startRemoteSUT. When updating failed image checks from the run-log the size of the image was not always updated correctly. When recording procedures, QF-Test failed to create a procedure in case the respective component had no value for a placeholder like <COMPFEATURE>.

B.18

Version 3.4.1

New features:
Web SWT

Full support was added for Firefox 5 and experimental support for Firefox 6. Eclipse/SWT support was updated for Eclipse version 3.7 Indigo. Groovy was updated to version 1.7.10. Error messages for errors in Groovy scripts now show proper context and line numbers.

SWT

QF-Test can now handle custom SWT components that hide their child Widgets by overriding getChildren() to return an empty array. Support was added for ExtJS version 4 and for RichFaces version 4. See section (625) 38.2 for details about the support AJAX toolkits and versions. attribute of Window The As regexp ned as a variable value.
(516) (507)

Web

and Component

(514)

nodes can now be de-

Web

The startup sequence created by the quickstart wizard for web applications now has a smarter layout and is easier to adjust. On Windows systems QF-Test now makes use of the always-on-top attribute to force SUT windows to the front even if normally prevented by Windows. To disable (255) this feature, deactivate the option Force window to the top when raising . A new check was added to directly check the icon of a component instead of having to capture the whole component and dene a check region. Bugs xed:

B.18.

Version 3.4.1

772

When loading the test-suites to restore the last session upon startup, QF-Test did not check for auto-save les. It was possible to start parallel tests in the same context by hitting Return at the right time. This could lead to confusing results in the run-log and even to exceptions.
Swing

Swing applications started via Java WebStart have two independent application (394) contexts, a user context and a security context. Unless the GUI engine attribute (392) of a SUT script node explicitely denes the engine, the script is now always correctly executed in the user context and not in the security context. Special support for JSlider components was missing. The special variable ${qftest:runid} did not expand to the correct value if the runid (549) was specied explicitly via -runid <id> in batch mode.

Swing

Web

A forced download triggered via a special PHP construct could cause Internet Explorer to crash. Clearing the cache for Firefox did not always work correctly. Trying to record a check for an XML document displayed in Internet Explorer could crash the browser. It is now possible to record checks for applets even when they are embedded in QF-Tests browser wrapper. The Groovy implementation of qf.logError was broken. In a workbench view with many open test-suites and some test-suite tabs scrolled out to the left, coordinates for dragging test-suite or node to different test-suites were off, leading to surprising results.

Web Web

Web

SWT SWT

QF-Test was leaking resources when taking Screenshots in an Eclipse/SWT SUT. Calling Display.asyncExec from a timer could crash Eclipse on Linux Gtk with 64bit. The order of SWT.Activation events could show very subtle differences between manual replay and automatic replay with soft events. A very long search string could mess up the size of the search and replace dialog. Markers in run-logs did not propagate beyond External log nodes. Fallback to lazy binding did not work for not-yet-bound properties.

SWT

B.19.

Main new features in version 3.4

773

B.19
Note

Main new features in version 3.4

Support for Java 1.4.2 was dropped by both Jython and Groovy so QF-Test now requires at least Java version 1.5 for itself and the SUT. Following is an overview over the main new features that have been implemented for QF-Test since the last production version 3.2.
Description Jython updated to version 2.5.1 Groovy updated to version 1.7.6 Support for Eclipse/SWT 3.6.2 and 3.7 Web testing support for Firefox 4 and Internet Explorer 9 Integration with Continuous Integration tool Hudson/Jenkins Fault-tolerant image checks Advanced search and replace Iteration ranges for data drivers Direct interaction with web servers Extended import functionality Debugger integrated into the workbench Ability to create a test-suite from a run-log Automatic splitting of run-logs and graceful handling of OutOfMemoryErrors Improved backwards/forwards navigation Improved pkgdoc and testdoc documentation Further info section 13.6
(108)

section 20.3

(171)

chapter 44 (13) section 2.3 (335) Iteration ranges (500) Server HTTP request node (148) section 17.3 section 8.1.6 Option Minimum size for automatic splitting (278) (kB) . Command line arguments (553) -testdoc.teststeps , (552) -testdoc.nodeicons and (546) -pkgdoc.nodeicons Option Record MouseEvents without coordinates (235) (420) where possible , Mouse event (57) section 7.5
(68)

(722)

Improved coordinate abstraction for mouse events Variables with default values and many additional special variables

Table B.2: New features in QF-Test version 3.4

B.20

Version 3.4.0

New features:
SWT

Support for testing Eclipse/SWT version 3.7 was updated.

B.20.

Version 3.4.0
(335)

774

All Data binder nodes now have an attribute named Iteration ranges that can be used to limit the iteration to a single element or one or more intervals, which is very convenient when implementing tests, continuing a test that had to be stopped or performing a retest with a limited subset. In workbench mode the current session with the currently open test-suites and the positions therein is now saved when QF-Test is closed and restored upon startup, (220) provided the option Restore last session on startup is active. Also added to the workbench view were the ability to save all modied test-suites via the File menu and switching between recently active test-suites via Control-F6 . Moving backwards/forwards between selected nodes has been unied and should now work consistently even for special cases. After recording components you can jump directly to the recorded node by moving forwards, even if no node was actually recorded and the existing node is in a different test-suite. Filter capabilites were added to the search/replace result and other multi result dialogs. Marks can now also be set for nodes in a run-log, so multi-step search operations are now also available in run-logs. The quickstart wizard has improved support for launching applets in a browser and for setting the compatibility mode in internet explorer, backed by the procedures qfs.web.browser.general.startAsNewProcess and qfs.web.browser.settings.enableCompatibilityMode in the standard library qfs.qft. When started in batch mode with the command line argument (553) -verbose [<level>] , QF-Test now displays exception, error and warning counts for each Test-case. The new package qfs.autowin.acrobat in the standard library qfs.qft contains methods to save a PDF document showing in an Acrobat window or to copy its text content to a le via the clipboard (Windows only). These are complemented by the qfs.utils.files package with procedures to verify the content of a text le. The new methods setProperty and getProperty in the generic scripting mod(607) ule qf (see section 37.7 ) can be used to associate arbitrary data with a GUI element, e.g. to cache some resolved feature or name values or other information that is too expensive to compute every time a resolver method is called. The TestRunListener API (see section 39.5 ) has been extended to provide the number of exceptions, errors and warnings that occurred below a given node in the TestRunEvent of a call to nodeExited.
(683)

B.21.
Web

Version 3.4M2

775

The AJAX resolver for GWT has been updated to version 1.2.0. It now lters irrelevant DIV nodes from the component hierarchy and has improved support (625) for details about AJAX for obfuscated GWT class names. See section 38.2 resolvers and their versioning scheme. Bugs xed: Early checking to handle OutOfMemoryErrors more gracefully could impact test execution speed signicantly with JDK 1.5 on Windows. Tests should now run at full speed again with all JDK versions. Some of the advanced image check algorithms did not work correctly with JDK 1.5. All algorithms are now fully supported for JDK 1.5 and higher and return consistent results, independent of the JDK version, except for resizing, which is implemented differently in the various JDK versions and can lead to minor deviations in the resulting probability. The procedure qfs.shutil.copy in the standard library qfs.qft was broken. It now copies les and directories correctly, recursively if necessary and handles special cases like copying a directory into itself.

Web Web

Check image

(460)

nodes now work correctly in Firefox 4.0.

Scrolling an element visible was not always possible in IE9 and some special cases with a deeply nested hierarchy. In IE9 nodes were not highlighted properly and highlighting could interfere with recording image checks. Browser settings for IE9 did not always take effect. A download operation triggered from a popup window could crash the browser. A running test in the QF-Test daemon can now be interrupted and stopped cleanly (551) via -stoprun even if the test is currently waiting for something. The structure of the JUnit XML report is now compatible with Atlassian Conuence. In Groovy the method rc.getProperties() was not implemented correctly.

Web

Web Web

B.21

Version 3.4M2

New features:
SWT

Support was added for testing Eclipse/SWT versions 3.6.2 and 3.7 Milestone 5.

B.21.

Version 3.4M2

776

The search and replace dialogs are now integrated so it is possible to start with a search and continue replacing, based on the same criteria. Also, the replace dialog is no longer modal so it can be kept open while editing. Integration of the debugger functionality into the workbench view is now complete. The current variable bindings are displayed automatically whenever the debugger (274) is entered unless the option Show variable bindings automatically is turned off. The keyboard shortcuts Alt-1 through Alt-9 navigate directly to the respective test-suite tabs in the workbench view. Pressing F2 in the test-suite tree now moves focus directly to the details eld that is most relevant for the display name of the node. The new Procedure qfs.client.renameClient in the standard library qfs.qft can be used to rename an SUT client after it connects. This is especially useful for an SUT that launches multiple sub-processes that connect as client:2, client:3 etc. The new special variable $_{qftest:skipnode} can be used to skip execution of a (57) node. Please see section 7.5 for details and applicability.
SWT

Hyperlinks in the Eclipse FormText widget are now recognized as sub-items. Advanced image checks are now also available to scripts via the new run-context (589) method rc.checkImageAdvanced. Please see section 37.6 for details. The various rc.check* methods in the run-context now return the result of the (589) evaluated condition. Please see section 37.6 for details. The image of a Check image node in the test-suite and the run-log as well as screenshot images can now be opened in a separate window by double-clicking the image (run-log only) or clicking the respective button above the image. There is a new utility module for scripting called qf, documented in section 37.7 . It contains variants of commonly useful methods like isInstance and the ability to add messages to the run-log even when no run-context is available, e.g. in TestRunListener, Checker or Resolver implementations. Templates for procedure recording have become more powerful. Procedures can now be created conditionally by placing a @CONDITION tag with an arbitrary (712) Jython or Groovy script in the comment. Please see chapter 41 for details.
(607)

Web

The Procedure qfs.web.browser.settings.setProxy in the standard library qfs.qft now also supports passing a URL for automatic proxy conguration. Bugs xed:

B.22.

Version 3.4M1

777

Switching execution to a different node in the debugger could unbalance the parent/child relationships of the associated nodes in the run-log. Setting and unsetting options at script level could mess up default option values.
SWT SWT SWT

Improved double click replay for SWT on Windows. Some selection events for SWT now receive special treatment for focus handling. Empty lines at beginning of an SWT StyledText widget are now handled correctly on Linux. Errors in nested test-cases are now listed correctly in the report.

Web Web Web

Proxy server settings for Internet Explorer 9 are now working correctly. Image checks are now working correctly with Firefox 4. Javascript and authentication dialogs are now working correctly in Firefox 4. QF-Test now issues a warning each time you rst try to modify a write-protected test-suite. Doctags for Dependency nodes are now listed correctly in pkgdoc documentation.

B.22

Version 3.4M1

New features: Groovy was updated to version 1.7.6. QF-Test now actively monitors the memory available in the Java VM for running tests. When available memory runs low, the test is aborted cleanly and the run-log is saved. This also prevents extreme slow-down when memory gets tight. To prevent running out of memory in the rst place, QF-Test now splits run-logs automatically by default, based on the option Minimum size for automatic splitting (278) (kB) . Managing test-suites in the workbench has been improved. The tab for the active suite is now shown with a bold font, you can drag tabs to re-order test-suites and double-clicking to the right of existing tabs creates a new suite. The search dialog now has additional options for limiting the search and it is also possible to search all loaded test-suites at once. To prevent visual overload for the common case, the search dialog can now be toggled between simple and advanced mode.

B.22.

Version 3.4M1

778

The debugger has been mostly integrated into the workbench, except for the variables view which still requires the separate debugger window.
Web

There is a new recording mode for web applications that tells QF-Test to ignore individual events and record entire GET or POST requests instead, which are (500) then represented by the new Server HTTP request node. Upon replay QF-Test interacts directly with the HTTP/HTTPS server without requiring a browser. This can be useful for load testing or mass data entry. For Swing and SWT components with an icon QF-Test now records a hash value derived from the icon image as an Extra feature. As long as the icon remains stable this should lead to improved recognition, especially for toolbar buttons without a label. The default state of the image hash extra feature is Should match. You can change it to Ignore for components where the icon is not reliable and to Must match for components where the image is the best way to identify the component. There is a new special variable syntax that allows specifying a default value for a variable. This is extremely useful for things like generic components or really every place where a variable has a reasonable default, because that default value is then tightly coupled with the variable lookup itself and doesnt have to be specied at Sequence or test-suite level. The syntax is ${default:varname:defaultvalue}. Additional special variables are now available to retrieve the name of the currently executing Test-set or Test-step in addition to the already available Test-case name. Also, counters were added for the number of test-cases executed so far, failures (57) etc. Please see section 7.5 for details. Where available, the dialog showing the methods available for a component now also displays the results of calling those methods that represent Java Bean values. The option Log successful advanced image checks can be activated in order to store the results of a successful advanced image check in the run-log. This can be useful while tuning the parameters for an advanced image check algorithm. The new command line option -verbose [<level>] for batch execution with or without accessing a daemon causes QF-Test to print progress information during a test-run. The new Procedure qfs.utils.dumpHeap in the standard library qfs.qft can be used to trigger a heap dump for the Java VM of the SUT when it uses too much memory. This can be detected with the help of the Procedure qfs.utils.logMemory which now returns the amount of memory used. Bugs xed:
(553) (282)

B.23.

Version 3.3.0

779

When replaying mouse events without coordinates on items, QF-Test no longer targets the center of the item, but the position 5 pixels from the left, vertically centered. Apparently some events are sensitive to whether the click hits the text of an item and not just its bounding box, so short items in a broad column could cause problems. Calling stored procedures in a database could fail using the default Jython bindings. If the debugger was turned off for normal use, debugger-stepping after hitting a breakpoint could occasionally fail so the test simply continued. A positive search result for the type of an extra feature could cause an exception. Blank characters in names for saving external run-logs are now turned into an underscore because Windows doesnt handle trailing blanks in directory names gracefully. The input le for a CSV data le node is now closed immediately after reading the data. Recursive procedure calls could cause problems when importing procedures or analyzing references.

B.23

Version 3.3.0

Incompatible changes: Support for Java 1.4.2 had to be dropped in order to update Jython and Groovy. QF-Test now requires at least Java version 1.5. Interaction between Jython and Java has changed slightly between Jython 2.1 and 2.5 so occasionally scripts that worked in QF-Test may now fail or behave differently. We try to maintain compatibility where possible and provide a list of (108) things to look out for in section 13.6 . New features: Jython was updated to version 2.5.1. Most scripts should continue to run without problems. Incompatibilities may arise due to the changes in character encoding and because Java strings are now uniformly treated as unicode strings in Jython. (108) Please see section 13.6 for further information. Groovy was updated to version 1.7.2.

B.23.
SWT Web

Version 3.3.0

780

Support was added for Eclipse/SWT version 3.6.1. Experimental support was added for Firefox 4 and Internet Explorer 9. QF-Test now integrates well with the increasingly popular Continuous Integration system Hudson/Jenkins. This includes JUnit as an additional target format for (171) QF-Test reports. Please see section 20.3 for further information. Image checks are no longer restricted to testing for exact pixel-by-pixel equality. Several exible algorithms now allow image checks with a congurable degree of (722) deviation from the expected image. Please see chapter 44 for details. Besides components you can now also import test-cases and procedures from (148) one test-suite into another as described in section 17.3 . When coordinates for mouse events are irrelevant, QF-Test now removes them altogether instead of setting them to (1,1). Upon replay the event is then aimed at the center of the target component or item. See also option Record MouseEvents (235) without coordinates where possible . By combining search and replace operations with marks it is now possible to narrow searches in several steps, giving very ne-grained control over where to per(20) form replacements. Instructions and examples are provided in section 2.3.3 . Moving backwards or forwards between selected nodes now works across testsuites and even run-logs. Thus a single mouse click or keystroke is now sufcient to return when following a procedure call to its target in a different test-suite or when jumping from a run-log to the respective node in the suite. Several new toolbar buttons were added and all the buttons now look much better. The menu structure also was cleaned up with operations on tree nodes now collected under the new Operations menu, among them many new node transformations. Also, several new keyboard shortcuts were introduced. A summary of (798) all keyboard shortcuts is provided in appendix C . The layout of pkgdoc and testdoc documentation now equals that of reports, including node icons and support for test-steps. These can be congured via the (553) (552) command line arguments -testdoc.teststeps , -testdoc.nodeicons (546) and -pkgdoc.nodeicons or the respective options in the dialog for interactive creation of those documents. Skipped and Not implemented tests can now be ignored in the report via the (547) command line argument -report.ignoreskipped or the respective option in the interactive dialog. Daemon tests can now be stopped at the command line via the new argument (551) -stoprun .

B.23.

Version 3.3.0

781

The comment of a Procedure, Test-case or Dependency node can now be lled automatically with doctags for parameters, characteristic variables and author by rightclicking the node and selecting Additional node transactionsFill comment from the context menu. The TestRunListener API now also provides information about whether a testrun was terminated normally (with or without exceptions) or aborted. See section (683) 39.5 . The new method rc.callTest in the run context can be used to call a Test-case or Test-set. In contrast to callProcedure, callTest can only be called from Server script nodes. It returns the exit status of the called test. The content of QF-Test conguration les is now sorted which signicantly improves handling of those les in a version control system. Several procedures were added to or rened in the standard library qfs.qft. Its documentation is now also available online and indexed by the full text search engine on the QFS website. A link to the online text search covering all technical material is provided via the menu HelpOnline search... . It is now possible to create a test-suite from a run-log. It is now possible to pass GUI elements from an SUT script as individual targets to QF-Tests component recognition mechanism via the new run context method (105) overrideElement. See also section 13.3.11 . Bugs xed:
Web

When started from QF-Test, Internet Explorer 8 always ran in IE7 compatibility mode. This is no longer the case. Compatibility mode can now be actively selected in the quickstart wizard or via the procedure qfs.web.browser.settings.doStartupSettings in the standard library qfs.qft. On Linux QF-Test could sometimes crash SWT/Eclipse applications with disposed Combo popups. Test replay for Eclipse/SWT on Linux is now much more robust, especially when handling menus. Trailing empty newlines in StyledText widgets are now handled correctly on Linux. Creating reports with many screenshots now requires much less memory.

SWT

SWT

SWT

B.24.
Web Web

Version 3.2.2

782

Downloads in Internet Explorer should now work consistently. In case of overlapping nodes with transparent nodes on top QF-Test could pick the wrong node for check recording. It now lets the browser make the decision instead. Switching back to the old algorithm can be done via the option Let the (271) browser determine the target element for check recording . A memory leak in QF-Test resulting from SUT clients printing a lot of output to the shared terminal in workbench view was plugged.

B.24

Version 3.2.2

New features: Support was added for testing Eclipse/SWT version 3.6 Helios.
SWT

Multiple le selection in an SWT FileDialog can now be recorded and replayed. Bugs xed: Replay performance could suffer severely if the users home directory was located on a network drive.

Swing

A workaround for an AWT weakness introduced in QF-Test 3.2.1 could impact replay performance for AWT/Swing. Yet another performance problem resulting from very large hierarchies of Swing menus has also been xed. Several special cases for uploading les with Internet Explorer are now handled correctly. Downloading les with Internet Explorer now works correctly in case of HTTP redirection.

Swing

Web

Web

B.25

Version 3.2.1

Bugs xed: With a 1.4 JDK in interactive mode QF-Test would only start if given a le to open.
Web

Web-testing now works again on Windows 2000.

B.26.

New features in QF-Test version 3.2

783

The QF-Test debugger used to show duplicate entries on the variable stack for (355) (299) Procedure call and Test call nodes. There was a memory leak in connection with interactive editing of test-suites with (460) Check image nodes for large images. In very special cases an import of components into their own suite could result in incorrectly updated references. Saving entries for zipped run-logs could occasionally fail on Windows due to a problem in the underlying library. QF-Test now works around that. Two problems were xed for SUTs with more than one GUI engine, e.g. a WebStart application or combined SWT/Swing. In such a case QF-Test used to log more than one screenshot of the main screen after an error and sometimes connections from the second engine could fail.
Web

The upload dialog for web pages was not handled correctly for all browser versions and the procedure qfs.web.input.fileUpload in the standard library qfs.qft did not handle all cases correctly. In recent QF-Test versions there was an inofcial global Jython run-context rc available in addition to the local one. In special cases this could cause subtle namespace problems, so the (still inofcial) global version was renamed to __rc. Several minor problems and occasional exceptions in connection with the result list dialogs were xed.

B.26

New features in QF-Test version 3.2

Following is an overview over the main new features that have been implemented for QF-Test version 3.2. For detailed information about the differences between QF-Test 3.0 and 3.2, please see the following sections with the release notes for versions 3.2.0 and 3.1.x.

B.27.

Version 3.2.0
Further info Requires JDK 1.6.0_10 or higher section 35.1 section 15.1
(571)

784

Description Support for Web testing in combination with Java applets JDK instrumentation is no longer required Workbench view for a modern user interface (309) Test-step nodes for detailed reports Refactoring support with automatic update of references Integration with the open source test management tool TestLink New Best Practices part in the manual Eclipse/SWT support updated for the latest versions including GEF Incremental search in all of QF-Tests text elds Result list dialogs for complex operations Component method spy Options can be set at script level Relative and test-suite-local procedures New-style Check nodes and support for custom Checkers Static validation of references Automatic parameterization of procedures Persistent bookmarks and transient marks New APIs: ItemResolver, Checker, ClassNameResolver and TestRunListener

(125)

section 19.5
(186)

(164)

part II (679) section 39.4 section 2.3.1

(17)

section 5.5 (103) (589) section 13.3.9 and section 37.6 (78) (78) section 9.3.2 and section 9.3.1 (669) section 39.3

(40)

section 9.3.4 (225) section 29.1.3 (658) (669) section 39.2 , section 39.3 , (636) (683) 39.1.2 and section 39.5

(79)

section

Table B.3: New features in QF-Test version 3.2

B.27

Version 3.2.0

New features:
SWT

Support for Eclipse/SWT has been brought up-to-date with the current versions 3.5.2 and 3.6M6. Swing components are no longer highlighted by exchanging their foreground and background color but by painting a rectangle, similar to SWT and Web. Target sub-items are now highlighted as well. The new Boolean check node with Check type identier attribute visible is similar (484) to Wait for component to appear node, but in contrast to the latter is is listed in the report as a check and it has different default settings for the result handling.
(448)

Swing

B.28.

Version 3.1.4

785

The most recently accessed component and possibly item in the SUT can now be retrieved in Jython and Groovy SUT script nodes via rc.getLastComponent() and rc.getLastItem().
Web

Selecting a le for upload in a web form can now be recorded and replayed directly on Windows. For a cross-platform implementation it is still necessary to call the procedure qfs.web.input.fileUpload from the standard library qfs.qft. Additional features were added to the result lists for complex operations. For example, you can now perform actual replacements in the result list for the replace dialog or you can update multiple failed checks at once from the error list in a run-log. Besides, the result list for an operation that takes time is now lled continuously and the operation can be aborted if necessary. Recent search expressions are now saved in the user conguration. Dependency nodes inside Packages are now listed in the pkgdoc documentation (545) unless disabled via the command line argument -pkgdoc.dependencies or the respective interactive option. Bugs xed:
(317)

Web

Right-clicks for Javascript-based popup menus no longer need to be replayed as hard events in order to trigger the popup reliably. When recording checks the z-order of DOM nodes is now properly taken into account so that QF-Test no longer records nodes that are actually in the background, covered by other nodes.

Web

B.28

Version 3.1.4

New features:
Swing

It is no longer necessary to instrument standard JDKs or JREs, though it doesnt hurt to keep existing instrumentation. Please see the option Connect without JDK (253) instrumentation (Swing) for further information. Support was added for the latest Eclipse/SWT version 3.6. QF-Test now supports Firefox 3.6. The new part II Best Practices in the manual describes recommended procedures based on lessons learned from several customer projects and user feedback that will hopefully help you to nd the best strategy for using QF-Test in your own projects.
(186)

SWT Web

B.28.

Version 3.1.4

786

Reports have been restructured and further extended. Check nodes now have an optional Name attribute for their representation in the report. Besides, the performance of report creation has been drastically improved and its memory impact reduced. The tree display in run-logs now shows expanded values instead of variables. This can be toggled via the View menu. Compactication for run-logs is now more efcient and special placeholder log entries show where nodes were removed during compactication.
SWT

Support for the Eclipse Graphical Editing Framework (GEF) is now included by default and works correctly with multiple concurrent GEF editors. See section (679) 39.4 for further information. The new procedure qfs.web.input.fileUpload in the standard library qfs.qft handles INPUT:FILE nodes for the various browser variants. TestLink integration has been simplied (section 19.5
(164)

Web

).

For Swing and SWT tables and multi-column trees it is now possible to check a whole row at once. ExtraFeatureResolvers can now also be used for SWT and Swing. These are (639) described in section 39.1.6 . The standard library qfs.qft now includes wrapper procedures for the autowin module in the package qfs.autowin. These can be used to check for the existence of a window and to close it on Windows systems. The runid of the current test-run now is now available via ${qftest:runid}. Special variables or properties of type group:name can now be used for Characteristic variables. For nodes like Test-step or Test-set with several tabs for variable bindings the number of bound variables is now displayed in the tab which makes it much easier to locate those bindings. Bugs xed:
Web Web

Several causes for IE crashes were xed. Wait for absence now works correctly for invisible nodes. QF-Test doesnt get blocked as easily when the SUT generates a lot of output and the JTextArea for the shared terminal cannot keep up.

B.29.

Version 3.1.3

787

The internal variable stack of QF-Test could get messed up in some special cases. The procedure builder now correctly handles _ characters in package or class names.

B.29

Version 3.1.3

New features: The new Test-step node can be used to provide further structure and document (288) tests below Test-cases . Test-steps are listed individually in the report unless deac(548) tivated with -report.teststeps . Similar effects can be achieved by marking arbitrary nodes with the doctag @teststep, optionally followed by a name for the step. The report now also can list checks (off by default) and warnings (excluding component recognition warnings, on by default) integrated into the test-step listing and also shows icons for nodes. The respective settings can be changed via (547) (548) (548) -report.checks , -report.warnings and -report.nodeicons in batch mode or the respective options in the dialog for interactive report creation. In the workbench view there is now a context menu for the test-suite tabs. All Comment attributes can now be edited in an external editor similar to scripts. There are new toolbar buttons in the run-log window for navigating to the next/previous error or warning. The incremental search feature now has more intelligent for hits and also scrolling Up , Down , Page up and allows scrolling the component being searched via the Page down keys. Dependencies can now optionally be resolved in a namespace which allows having different sets of dependencies for different parts of a test so that the depen(321) dencies dont interfere with each other. See the Dependency namespace attribute for details. Bugs xed: Due to the JIDE libraries used, the new workbench view does not work with JDK 1.4. It now gets deactivated automatically when starting QF-Test with a 1.4 JDK. Several exceptions caused by the new workbench view and the multi result dialogs were xed.
(317) (309)

B.30.

Version 3.1.2

788

In the new workbench view QF-Test could lose track about which suites are currently loaded, causing a suite to get loaded more than once, leading to confusing behavior and sometimes an exception. The new mechanism for on-the-y SWT instrumentation only handled one SWT plugin, so that in case of a multi-platform Eclipse based application with several SWT plugins the correct plugin for the current platform could remain uninstrumented. Calling stopRun could cause the shared run-context in the QF-Test daemon to become unusable.
SWT Web

Text checks for SWT Group widgets were missing. Fixed automatic scrolling of web pages for image checks and replay of hard events. The popup sub-menu for setting and removing marks was sometimes empty.

B.30

Version 3.1.2

New features:
SWT Web

Support was added for the latest Eclipse/SWT version 3.5.1. We have started work on special support for several AJAX web toolkits, including Ext GWT, Ext JS, GWT, qooxdoo and RAP. Instead of deeply nested hierarchies of DIV nodes QF-Test now operates in terms of the actual rich interface of buttons, labels, trees etc. For information on how to activate this feature in its current state, please get in touch with QFS. QF-Test now comes with support for interacting with the open source test man(164) agement tool TestLink. Please see section 19.5 for details. By default QF-Test now starts with a Workbench View where all open test-suites are arranged as tabs in a single window. This feature is work in progress. In case it causes problems or if you prefer the old style, you can switch back to multiple windows via the View menu. The new function Analyze references in the context menu of a node can be used to statically validate the correctness of a test-suite by verifying the existence of the (355) (299) (320) target nodes of Procedure calls , Test calls , Dependency references and refer(514) ences to Components .

B.30.

Version 3.1.2

789

Operations that search for or result in changes to multiple nodes now bring up a mass result dialog listing the affected nodes. You can either navigate to specic nodes directly from that dialog or use it to set marks on some or all of the resulting nodes. Nodes in a test-suite can now be marked with markers in four different colors for easier reference and quick navigation. Marks can be set either individually via the menu EditMarks or the context menu, or from a mass result dialog. All text elds, terminals etc. in QF-Tests user interface can now be searched locally by pressing Ctrl-F when the eld owns the keyboard focus. Details are (17) described in section 2.3.1 . The new Component Method Spy retrieves and displays elds and methods of Swing or SWT components as well as the Javascript properties of a DOM node in (40) the web client (see section 5.5 ). After recording a new sequence and transforming it into a Procedure you can select Parameterize node from its context menu to tell QF-Test to automatically extract parameters for it from the contained nodes.
SWT
(352)

The Eclipse/SWT instrumentation performed by the Procedure qfs.qft#qfs.swt.instrument.setup now operates on-the-y, building a new plugin from an existing plugin and the QF-Test specic add-ons. Thus it is now possible to instrument custom SWT plugins or plugins for unofcial Eclipse versions for which QF-Test doesnt provide an exactly matching instrumented plugin. Event nodes can now be transformed into a Wait for component to appear
(484)

node.

The keyboard shortcut for the Redo operation was changed from Ctrl-R to Ctrl-Y . Over the last years the latter has become a de facto standard. The warning about nested Test-case execution can now be suppressed via the (249) option Log warning for nested test-cases .
Web
(288)

Where scrolling a web page is required to perform a hard event or an image check, QF-Test now tries to scroll a web page itself via Javascript calls instead of relying on the browsers internal mechanism which often completely messes up the display. Component recognition based on geometry can now be suppressed by setting the (514) coordinate and size attributes of a Component to -. It is now possible to specify just the Class count node and ignore the other.
(518)

or Class index

(518)

of a Component

B.31.

Version 3.1.1

790

Bugs xed: Using the replace dialog to replace a regular expression with an empty string caused an exception. The function to make Procedure calls or Dependency references relative did not work reliably. When moving several Procedures, the automatic correction of Procedure calls could fail in some special cases. Procedure recording now works correctly for custom classes with a special ClassLoader. When creating reports, warnings can be suppressed entirely by specifying -report.warnings=false for batch mode or deactivating the respective option in the dialog for interactive report generation.
SWT

Recording a click on an editable table cell could cause the cell editor to get activated in the wrong table row. Fetch index
(474)

SWT

nodes were not implemented for SWT.

Removing or marking unused components, importing and other complex compo(334) nent operations could fail if the test-suite contained a Data table node with no columns or empty table cells.

B.31

Version 3.1.1

Incompatible changes: The changes to the check system described below are fully backwards compatible. However, once a test-suite has been saved with QF-Test version 3.1.1 it cannot be run correctly with an older QF-Test version except for QF-Test version 3.0.3 which has explicit bidirectional compatibility support for this feature. Once you introduce QF-Test version 3.1.1 you can safely continue to work with QF-Test 3.0.3, but you should not use older versions like 3.1.0 or 3.0.2. New features: The transition to the new check mechanism is now complete. Custom Checkers (669) are now fully documented (section 39.3 ) and can be created for all check data types. All Check nodes now have a Check type identier attribute and the old Check editable, Check enabled and Check selected nodes were replaced with (448) (450) Boolean check nodes with a matching Check type identier .

B.31.

Version 3.1.1

791

It is now possible to bookmark test-suites, run-logs and individual nodes in a testsuite so they are directly accessible via the new sub-menu FileBookmarks . (225) Please see section 29.1.3 about how to create and manage bookmarks.
Web

The new ClassNameResolver (see section 39.1.2 ) - though also useful for SWT and Swing applications - is an important milestone for AJAX web testing. With its help and the new pseudo class hierarchy for DOM nodes (see section (39) 5.4 ) a deeply nested hierarchy of DIV nodes can be turned into a concise component hierarchy with expressive class names that vastly improve component recognition. Procedures can now automatically be parameterized via the sub-menu Parameterize node in the popup menu for the Procedure node. This feature is work in progress and will need more ne-tuning so we hope for your feedback. When creating testdoc documentation it is now possible to follow Test call nodes (288) (294) and include the target Test-case , Test-set or test-suite into the generated documentation by activating the respective option in the dialog for interactive use or via (552) the command line argument -testdoc.followcalls in batch mode. Several more variants for converting and (un)packing nodes were added. Bugs xed: Several problems with split run-logs were xed. Log entries could get lost in compact logs, when the split run-log was not saved as a zip le or when split run-logs were disabled but the attribute Name for separate run-log was dened. DaemonTestRunListeners did not work correctly in asynchronous mode. When creating a report, screenshots for failed Check image nodes were always saved regardless of whether the failed check actually represented an error or was just a part of a conditional construct. Use of the new Java regular expressions in QF-Tests replace dialog was not correct so that some replacements based on regexps were not possible.
(460) (299) (352)

(636)

Web

When determining uniqueness of id attributes for use as names, QF-Test was too strict so that in some cases potential names were not assigned. Instrumenting an OpenJDK directory now works.

Web Web SWT

Waiting for a new document did not handle the window name attribute correctly. Waiting for a download to nish always waited for the full timeout. In very rare cases replaying text input could crash an SWT application on Linux.

B.32.

Version 3.1.0

792

Option settings from rc.setOption() in the SUT were reset whenever options were changed in the option dialog.
SWT

Event replay for SWT MenuItems with Gtk 2.4 on RedHat Enterprise Linux 4 was not reliable. The ktable and gef Jython modules for the respective SWT components were not correctly updated for the new ItemResolver and Checker mechanisms. QF-Test now properly discerns the most modal SWT Shell in case multiple modal Shells are opened simultaneously.

SWT

SWT

B.32

Version 3.1.0

Incompatible changes: QF-Test now uses standard Java regular expressions by default which are much more powerful than the old GNU regexp package. One major difference is that { and } are special characters for Java regexps and may need to be escaped with a \. If necessary, you can switch back to the old GNU regexps via the option (221) Use old-style GNU regexps (from before QF-Test version 3.1) . See also section (581) 36.4 .
Web

In contrast to Mozilla, Internet Explorer reports a local navigation to an anchor on the current page as a normal navigation event, so up to now a (488) Wait for document to load node was recorded which failed when replaying the same test on Mozilla. QF-Test now treats this situation uniformly by ignoring the extra navigation from IE so no Wait for document to load node is recorded with either browser. Consequently, old Wait for document to load nodes for local navigation will now fail and need to be removed from your tests. New features:

Web

QF-Tests embedded browser for web testing now supports Java applets with JDK 1.6.0_10 and up, so QF-Test can now test the interaction of a web application with its embedded Java applet. Many of QF-Tests options, specically those with an impact on test execution, can now be changed at run-time via the new run-context method rc.setOption (103) (589) described in section 13.3.9 and section 37.6 .

B.32.

Version 3.1.0
(352) (317) (288)

793

When moving Procedure , Dependency or Test-case nodes, all references to these nodes can now be updated automatically. Similar updates are now per(514) IDs via the reformed when changing the name of such nodes or Component place dialog. As an additional feature the new special Variable group ${id:...} (see (57) (589) section 7.5 ) and the run-context method rc.id (see section 37.6 ) can be used to mark arbitrary text as the denition of a component id that will be updated whenever the referenced component changes. Procedure calls from one Procedure to another Procedure in the same Package hierarchy can now use a relative procedure name. Also, Procedures and Packages can now be made local to their test-suite by prexing the name with an _. See section (78) (78) 9.3.2 and section 9.3.1 for details. The selection dialog for a Procedure, Dependency or Test-case node now also shows the comment of the selected node. Also, it is now possible to select multiple nodes in order to insert multiple calls at once. A Procedure call to a specic Procedure can now be created by copying the Procedure node and pasting it to a place where insertion of the Procedure is not allowed, which (315) node. is basically everywhere a Procedure call is allowed except for the Extras Dragging and dropping a Procedure with the same effect is also supported.
Web
(355)

It is now possible to interact directly with the Javascript engine of the browser and execute arbitrary scripts via the methods DomNode.toJS, DomNode.evalJS and (615) DocumentNode.evalJS, described in section 38.1 . When an error or exception occurs, the run-log now contains a stacktrace node, showing all nodes on the execution stack including their variable bindings, similar to the variables view in the debugger window. There is a new API for various kinds of resolvers and checkers, including an ItemResolver interface for implementing your own sub-items of custom components and a Checker interface for custom checks that can be captured and (658) and section replayed just like QF-Tests standard checks. See section 39.2 (669) 39.3 for further information. The new TestRunListener API, described in section 39.5 , can be used to monitor a test-run from a script. Its variant DaemonTestRunListener supports the same remotely via QF-Tests daemon API. It is documented in section (709) 40.2.5 .
(683)

SWT Web

Support for Eclipse/SWT now includes the latest Eclipse milestone 3.5M7. The run-context method rc.callProcedure now also works from an (392) SUT script in a web application.

B.33.

Version 3.0.2

794

Execution of a Test-case or Test-set can now be aborted or skipped via the runcontext methods rc.skipTestCase, rc.skipTestSet, rc.stopTestCase and rc.stopTestSet, e.g. to prevent wasted time and irrelevant error messages (589) in case a dependency cannot be resolved. See section 37.6 for details. The name of the currently executing Test-case is now available via the special variables ${qftest:testcase.name}, ${qftest:testcase.qname} and (57) ${qftest:testcase.reportname} (see section 7.5 ).
Web

Via the new methods setProperty and getProperty in the class Node from QF-Tests pseudo DOM API for web applications, arbitrary data can be associated with DOM nodes without affecting the web application. The new command line argument -ping for qftest -batch -calldaemon can be used to check whether a QF-Test daemon is up and running. Bugs xed: In QF-Test version 3.0.2 hard TAB characters in Jython or Groovy scripts were replaced with blanks when showing the script, immediately causing the respective node to become modied. This change has been taken back.
(545)

Mac

For SUTs running on Mac OS X QF-Test disables use of the screen menu bar to allow proper capture/replay of menu actions. With JDK 1.6 some of QF-Tests dialogs were not displayed with the correct size. Activating an engine license in the option dialog now works as expected. When recording procedures, a new root Package is always created instead of merging the procedures into a possibly existing Package named procbuilder.

B.33

Version 3.0.2

New features:
SWT

Support was added for Eclipse/SWT versions 3.5M6 and 3.4.2. The method callProcedure was added to the class DaemonRunContext of QF-Tests daemon API. This method and the existing method runTest now also (695) support direct passing of parameters to this call. See chapter 40 for details. Bugs xed:

B.34.
Web

Version 3.0.1
(488)

795

Wait for document to load nodes for web clients did not work reliably in case of multiple browser windows. Checking the enabled/disabled state of components is now also implemented for web clients. Capture and replay of MOVED and SIZED events is now also supported for browser windows of web clients. QF-Test now performs regular explicit garbage collections to prevent running out of memory in the PermGen Space, a special Java memory region that causes trouble for a lot of Java applications. For the SUT these extra garbage collections can be turned off via the option Automatically perform garbage collection in the (254) SUT .

Web

Web

SWT SWT

Disposal and recreation of an SWT Display is now handled correctly. Custom SWT Widgets implemented as an anonymous local class were not treated correctly, possibly leading to a ComponentNotFoundException. For checks with a very long timeout deadlock detection is now adjusted accordingly. The demonstration test-suite for integration with the Imbus TestBench was updated to the latest TestBench version and now handles split run-logs correctly.

B.34

Version 3.0.1

New features: The Jython and Groovy run context rc now provides the convenience methods getBool, getInt, getNum and getString for simplifying lookup of typed variable values in scripts. It is now possible to specify proxy settings for a web SUT directly in the quickstart wizard or via the procedure qfs.web.browser.settings.setProxy in the standard library test-suite qfs.qft. An entry for Swing JInternalFrame components has been added to the template le for automatic procedure creation. Bugs xed: Web-testing with Mozilla on Linux no longer requires using a 1.4 JDK.

B.35.

New features in QF-Test version 3

796

Using the Update component feature for web pages with deeply nested tables could lead to duplicate component IDs. Left-to-right binding in Data table
(337) (334)

and other data nodes has been xed.

SQL statements in Database nodes did not work with Oracle databases, due to an automatically appended ; character. This has been xed. If your database engine requires that SQL statements are terminated with a ; you have to specify it yourself. The procedure qfs.swt.instrument.setup now always picks the SWT plugin with an exact version match for instrumentation, if available. This is important for SWT 3.4, where versions 3.4.0 and 3.4.1 are mutually incompatible. When recording procedures, a $ character displayed in one of the elds of the SUT could lead to an exception. Groovy class loading and script parsing in the SUT should now be faster, in some cases - specically applets - drastically. A caught exception in the Finally node of a Try (382) of an exception caught earlier in a Catch node.
(384) (379)

node could mess up the state

The demo test-suite swt_addressbook.qft was still based on SWT 3.1, support for which is no longer distributed with QF-Test. The suite is now based on SWT 3.4.

B.35

New features in QF-Test version 3

The following new features have been implemented for QF-Test version 3:

B.35.

New features in QF-Test version 3

Further info Pseudo DOM API and many other places (156) Interaction with Test Management Tools
(615)

797

Description Support for Web testing Interoperation with HP/Mercury Quality Center and Imbus TestBench Groovy as a second scripting language New Jython version 2.2.1 Improved Jython and Groovy script editor with syntax-highlighting Automated creation of basic procedures Extended and unied functionality for Check... and Wait... nodes Direct support for negated checks Split run-logs and IO redirection - no more OutOfMemoryErrors Extensions and an external driver for the daemon API New concept: Expected failure for Test-case nodes Directing and documenting manual tests Extended undo support Immediate vs. lazy variable bindings Image API for creating and saving screenshots Crypting of text input for password elds Recursive sorting and transformation of nodes Revised search with highlighting of results Extensions to the standard library test-suite qfs.qft New tutorial chapters for advanced features

Scripting , Scripting with Groovy (94) Scripting (94) Fundamentals of the Jython integration Automated Creation of Basic Procedures (442) (484) Checks , Wait for component to appear , (411) Wait for client to connect etc. (442) Checks (67) (283) Split run-logs , Log SUT output individually , (252) Maximum size of client terminal (kB) (133) (695) Test execution , Daemon mode Test-case
(288) (153)

(94)

(110)

Executing Manual Tests in QF-Test Self explanatory (60) Immediate and lazy binding (610) (690) Image API , Image API extensions (250) (428) Salt for crypting passwords , Text Self explanatory Self explanatory pkgdoc Advanced features

(180)

Table B.4: New features in QF-Test version 3

798

Appendix C Keyboard shortcuts

C.1 Navigation and editing

This table gives a useful overview of QF-Tests basic and advanced navigation and editing keyboard shortcuts:
Control-N Control-O Control-S Control-Z Control-Y Control-F F3 Control-G Control-H Control-PageDown Control-PageUp F5 Shift-F5 F6 Shift-F6 Control-F6 Up / Down / Right / Left File Navigation New Open Save Basic Editing Undo last change Redo last change Search and Replace Search Continue previous search Search again Replace Workbench View Next test-suite Previous test-suite

Refresh project directory Rescan project directory Switch focus back and forth between suite and project view Select the current suite in the project tree, showing the project if necessary Switch to previously active suite.Keep Control pressed and type F6 again Tree View Basic navigation

C.1.

Navigation and editing

799

Alt-Up Alt-Down Alt-Right Alt-Left Shift-Up Shift-Down Control-Up Control-Down Control-Right Control-Left Space Control-Backspace Shift-Control-Backspace Control-. Alt-Return Shift-F10 / Windows context menu key F2 Shift-Insert Shift-Return Shift-Delete Shift-Control-Up Shift-Control-Down F2 Return Escape Shift/Control-Up/Down Control-X / C / V Control-Space Control-P Control-T Control-W Control-TAB Shift-Control-TAB Control-Return Control-P

Jump to previous sibling of node Jump to next sibling of node Expand node recursively Collapse node recursively Extend selection upwards Extend selection downwards Move upwards without affecting selection Move downwards without affecting selection Scroll tree right Scroll tree left Toggle selection of current node Jump to last visited node Jump to next selected node Clean tree Bring up node properties window Bring up node popup menu Mark name or id of node in order to rename it Tables Insert new row Edit selected row Delete selected row Move selected row up Move selected row down Edit selected value Conrm changes Discard changes Multi selection Cut / Copy / Paste

Code Editor Open available QF-Test variables for scripts or for dedicated nodes open a list of available methods. Find procedure denition (for lines which call a procedure) Find test denition (for lines which call a test) Find component (for lines which refer to a component) Multi-line Text Elements Move focus to next attribute Move focus to previous attribute Conrm changes For Procedure call nodes Find procedure denition For nodes with a Component id attribute

C.2.

Record and replay functions

800

Control-W Control-P Control-P Control-I Control-L Control-N Sift-Control-N Control-T Control-W Shift-Control-W Control-A Control-D Shift-Control-D Control-I Shift-Control-I Shift-Control-P Shift-Control-S Shift-Control-T Shift-Control-Y

Find component For Test call nodes Find called test denition For Dependency reference nodes Find dependency denition Run-log Open error list dialog Open latest run-log Find next error Find previous error Find node in test-suite Find next warning Find previous warning Advanced Editing Inserting a procedure call Add selected node to bookmarks Toggle disabled state of selected nodes Find references of node Available for components, tests, dependencies and procedures Pack selected nodes into if-sequence Transform selected nodes into procedure Available only valid for seqeuence and similar nodes Pack selected nodes into sequence Pack selected nodes into test-step Pack selected nodes into try/catch

Table C.1: Shortcuts for navigation and ed

C.2

Record and replay functions

The following table contains important shortcuts for record and replay functions, which partly work also outside of QF-Test, i.e. directly in the SUT:

C.3.

Keyboard helper
Return F9 Alt-F12 (congurable) F11 (congurable) F12 (congurable) Left Mouse Button Right Mouse Button Replay Play (execute current node) Pause Pause testrun (Dont Panic) Record Toggle Record mode Record Checks Toggle Check mode Record default check Show list with available checks

801

Record Components Toggle Record single component mode Paste recorded component Id from clipboard Jump to last recorded component Toggle Record multiple components mode Record Procedures Shift-F12 (congurable) Toggle Record procedures mode Control-F12 (congurable) Toggle Record multiple procedures mode Debugger F7 Step in F8 Step over Control-F7 Step out Shift-F9 Skip over Skip out Control-F9 Control-F8 Breakpoint on/off Control-J Jump to run-log Find node in test-suite Control-T Continue execution from currently selected node Control-, Shift-F11 (congurable) Control-V Control-Backspace Control-F11 (congurable)

Table C.2: Shortcuts for special record and replay functions

C.3

Keyboard helper

The following graphic may help to easily remember the assignment of functional keys used by QF-Test. It is intended to be printed, cut out and xed above the area of functional keys F5 to F12 of your keyboard.

C.3.

Keyboard helper

802

Figure C.1: Keyboard helper

803

Appendix D Glossary
API Application Programming Interface, a set of package, class and method denitions that the programmer of an application can use. The Java API refers to the interface of the standardized Java class library that is shipped with each JDK. AWT Abstract Windowing Toolkit, the part of the Java library responsible for the display of windows and components as well as for the dispatch of events. GUI Graphical User Interface. An interface between a program and the user, usually consisting of windows built from components for displaying information and receiving input. RMI Remote Method Invocation, communication protocol/programming interface in Java for calling a method of a remote object. SUT System Under Test, the application being tested with QF-Test. VM The Java Virtual Machine executes Java programs that have been compiled to Java bytecode. It is responsible for the platform independence and compatibility of Java programs across various kinds of machines and operating systems.

804

Appendix E Third party software

QF-Test makes use of the following software: Apache POI Copyright c 2009 The Apache Software Foundation Distributed under the Apache Licence, Version 2.0 doc/licenses/apache-2.0 and doc/licenses/poi-notice). URL: http://poi.apache.org Bean Scripting Framework (BSF) Copyright c 1999 International Business Machines Corporation. See the license doc/licenses/bsf URL: http://www.alphaworks.ibm.com/tech/bsf Cryptix library Copyright c 1995, 1996, 1997, 1998, 1999, 2000 The Cryptix Foundation Limited. All rights reserved. See the license doc/licenses/cryptix URL: http://www.cryptix.org dom4j Copyright c 2001-2005 MetaStuff, Ltd. All Rights Reserved See the license doc/licenses/dom4j URL: dom4j.sourceforge.net GNU regex

(see

Third party software

805

Copyright c 1998-2001 Wes Biggs. Distributed under the GNU Lesser General Public License (see doc/licenses/LGPL). Source code included in misc/gnu_src.jar URL: http://nlp.stanford.edu/nlp/javadoc/gnu-regexp-docs/index.html Groovy scripting language Distributed under the Apache doc/licenses/apache-2.0). URL: http://groovy.codehaus.org JCommon Copyright c 2007-2012 Object Renery Limited and Contributors. Distributed under the GNU Lesser General Public License doc/licenses/LGPL). URL: http://www.jfree.org/jcommon/index.html jEdit Syntax Highlighting Public domain The jEdit 2.2.1 syntax highlighting package contains code that is Copyright c 1998-1999 Slava Pestov, Artur Biesiadowski, Clancy Malcolm, Jonathan Revusky, Juha Lindfors and Mike Dillon. URL: http://syntax.jedit.org JExcel Copyright c 2002 Andrew Khan. All rights reserved. Distributed under the GNU Lesser General Public License doc/licenses/LGPL). Source code included in misc/jexcelapi.tar.gz URL: http://www.andykhan.com/jexcelapi/index.html JFreeChart Copyright c 2005-2012 Object Renery Limited and Contributors. Distributed under the GNU Lesser General Public License doc/licenses/LGPL). URL: http://www.jfree.org/jfreechart/index.html JGoodies Looks Licence, Version 2.0 (see

(see

(see

(see

Third party software Copyright c 2001-2005 JGoodies Karsten Lentzsch. All rights reserved. See the license doc/licenses/jgoodies-looks URL: http://www.jgoodies.com JIDE Common Layer

806

Copyright c 2002 - 2009 JIDE Software, Inc, all rights reserved. Distributed under the GNU General Public License (see doc/licenses/GPL) with Classpath Exception (see doc/licenses/Classpath). URL: http://www.jidesoft.com/products/oss.htm JNIWrapper Copyright c 2000-2004 MIIK Ltd. All rights reserved. URL: http://www.jniwrapper.com Jython scripting language Copyright c 2007 Python Software Foundation; All Rights Reserved. See the license doc/licenses/jython URL: http://www.jython.org Mozilla Copyright c 1998-2006 Contributors to the Mozilla codebase. Distributed under the Mozilla Public License doc/licenses/MPL-1.1.html). URL: http://www.mozilla.org/projects/embedding/ Netty Copyright c 2009 Red Hat, Inc. Distributed under the Apache doc/licenses/apache-2.0). URL: http://netty.io PngEncoder Copyright c 1999-2000 J. David Eisenberg. All rights reserved. Distributed under the GNU Lesser General Public doc/licenses/LGPL). Source code included in misc/PngEncoder.tar.gz URL: http://catcode.com/pngencoder/

(see

Licence,

Version

2.0

(see

License

(see

Third party software Python library (distributed with Jython) Copyright c 2001-2008 Python Software Foundation; All rights reserved. See the license doc/licenses/python URL: http://www.python.org SendSignal.exe Permission to use granted by the author Louis K. Thomas. URL: http://www.latenighthacking.com/projects/2003/sendSignal/ Sixlegs PNG library Copyright c 1998, 1999, 2001 Chris Nokleberg Distributed under the GNU Lesser General doc/licenses/LGPL). URL: http://www.sixlegs.com STAX - Streaming API for XML Copyright c 2003-2007 The Apache Software Foundation Distributed under the Apache Licence, Version doc/licenses/apache-2.0). URL: http://geronimo.apache.org TrueZIP Virtual File System API Copyright c 2005-2007 Schlichtherle IT Services. Distributed under the Apache Licence, doc/licenses/apache-2.0). URL: http://truezip.dev.java.net

807

Public

License

(see

2.0

(see

Version

2.0

(see

wxActiveX - Library for hosting ActiveX controls within wxwidgets programs Copyright c 2003 Lindsay Mathieson. Distributed under the wxActiveX Library doc/licenses/wxActiveX). URL: http://sourceforge.net/projects/wxactivex wxMozilla - Embedding Mozilla in wxWindows Copyright c 2003-2006 Jeremiah Cornelius McCarthy. Distributed under the wxWindows Library

Licence

(see

Licence

(see

Third party software doc/licenses/wxWidgets). URL: http://wxmozilla.sourceforge.net/ wxWidgets GUI API Copyright c 1998-2005 Julian Smart, Robert Roebling et al. Distributed under the wxWindows Library doc/licenses/wxWidgets). URL: http://www.wxwidgets.org/ XML Beans

808

Licence

(see

Distributed under the Apache License, Version 2.0 (see doc/licenses/apache-2.0 and doc/licenses/xmlbeans-notice). URL: http://xmlbeans.apache.org XML parser Xerces Copyright c 1999-2002 The Apache Software Foundation. Distributed under the Apache Software License doc/licenses/apache-xerces). URL: http://xml.apache.org/xerces-j/index.html XSLT engine Xalan Copyright c 1999 The Apache Software Foundation. Distributed under the Apache Software doc/licenses/apache-xalan). URL: http://xml.apache.org/xalan-j/index.html

(see

License

(see