The Testing and Test Control Notation

Curriculum

Version 1.1(E) Date: 03/2007

Curriculum TTCN-3 Certificate

1 Document Information
Version 0.1 0.2 1.0 1.1 Date 03/2004 05/2005 08/2006 12/2006 Editors Ina Schieferdecker, Fraunhofer FOKUS Andrej Pietschker, Siemens Ina Schieferdecker, Fraunhofer FOKUS Andrej Pietschker, Siemens Ina Schieferdecker, Fraunhofer FOKUS Ina Schieferdecker, Fraunhofer FOKUS Remarks First Draft Completed Draft Revision for TTCN-3 v3 ETSI inclusion Copyright transfer to GTB in preparation of the contract with iSQI

GTB eV

Curriculum TTCN-3 Certificate

2 Keynote
The Testing and Test Control Notation (TTCN-3) is a new test specification and test implementation language that supports all kinds of black-box testing. In particular it is well suited for reactive systems covering testing of local sequential or concurrent systems up to the testing of distributed systems. TTCN's traditional stronghold is in testing communication protocols however with the release of version 3 it became a universal and adaptable testing language suited for a large variety of application domains. Today TTCN-3 is for example also used to test applications in the automotive industry through CAN and MOST busses. With the ever growing need to test e.g. Web services from a users point of view, TTCN-3 delivers the concepts and tools to master these challenges. Not all possible applications of TTCN-3 have been evaluated; however industry and research start to focus on TTCN-3 and its applications in testing as recent test conferences illustrate. TTCN-3 was developed at the European Telecommunications Standards Institute (ETSI) and is the only standardized test specification language (being also adopted at ITU). It therefore has all the benefits a standardized language possesses. Already 4 vendors support this language with compilers, even more support TTCN-3 with supplementary tools and services making the claim for vendor independence come true. With TTCN-3, test systems move from being expensive proprietary solutions designed for a single purpose to highly adaptive software test platforms supporting a variety of tests being used throughout system development and for system approval and acceptance kind of tests.

Consequences for training TTCN-3 as such requires a deep understanding and good knowledge to be used efficiently. This calls for well trained, high qualified testing specialists.

Aim of training as a TTCN-3-CERTIFICATE. It aims at making TTCN-3-Certificate holders to be knowledgeable in TTCN-3 concepts, the practical application of TTCN-3 and the integration of a TTCN-3 based test process into the system development. The curriculum covers the concepts, presentation formats, guidelines of use and concrete application examples of TTCN-3. Training for a TTCN-3-CERTIFICATE is aimed at all people involved in software testing, who wish to base their knowledge and experience on TTCN-3 on a sound foundation or extend it. The training is suitable for quality and test professionals, programmers, developers, specialists and (project-) managers who are responsible for the execution, planning or control of testing. TTCN-3-CERTIFICATE holders are capable of designing and planning the projectspecific tests and formalize them in TTCN-3. They are capable of defining adequate test strategies including test objectives, test purposes, test configurations, test suites and the realization of tests on target test platforms.
3

GTB eV

Curriculum TTCN-3 Certificate

TTCN-3-CERTIFICATE holders are capable of developing functional, conformance, interoperability, robustness, scalability, load, stress, etc. tests in TTCN3 TTCN-3-CERTIFICATE holders are aware that TTCN-3 is a powerful test technology but not the only one. They are capable of identifying those test targets which gain most from a systematic, formalized test approach with TTCN-3. They are capable of developing mixed test strategies (choosing from a set of test technologies) to have optimized test efficiency. TTCN-3-CERTIFICATE holders put into practice the TTCN-3 tests. They are capable of setting up automated tool chains for the execution of tests, creation of corresponding test reports and the interaction with the system developers. TTCN-3-CERTIFICATE holders are able to develop a TTCN-3 based testing process and to integrate the testing process with the system development process.

GTB eV

Curriculum TTCN-3 Certificate

3 Notes on Curriculum and Examination

The curriculum defines: the examination-relevant material in its contents, the minimum time in which the material must be taught in accredited training courses. The material is to be illustrated and taught using suitable examples and exercises.

The curriculum does not define: the details of the TTCN-3 technology, concepts, language features, etc. as the TTCN-3 standard is the main source for the definitions, the (chapter-)sequence in which the material has to be taught, or the timescales for examples and exercises in accredited training courses. The examination is a multiple-choice test: Word definitions, coherence as well as knowledge and understanding of test techniques defined by the TTCN-3 standard series and referenced in the curriculum are tested.

GTB eV

Curriculum TTCN-3 Certificate

4 Summary of the Curriculum

Topic Introduction Basics of TTCN-3 Duration (min) 30 60 Content The Certified Tester Program Motivation History Main Concepts Basic Syntax Basic and structured types Send templates, receive templates, inline templates Message-based and signature-based communication Port types and handling Component types Component handling Functions, altsteps, test cases, external functions Statements Operations Timing Modules Module control Import Scoping Grouping Parameterization Attributes Graphical Format Tabular Format Test system components TRI TCI Standardization process around TTCN-3 Maintenance of TTCN-3 Test purposes and test cases Functional, conformance, interoperability test aspects Performance, load and scalability aspects ASN.1, XML, IDL within TTCN-3 Tool support for TTCN-3 based testing Corresponds to 20 lectures 45min and to 3 tutorial days appr. 7 lectures a day

Language Features: Data Types and Templates Language Features: Communication Language Features: Test Configurations Language Features: Behaviour

60

60 60 180

Language Features: Modules

90

Other Presentation Formats Test Execution

45 90

TTCN-3 Standard Test Design and Development Open Data Interface Test Tools Overall

30 105

60 30 900

GTB eV

Curriculum TTCN-3 Certificate

5 Details of the Curriculum

Subject Description Time (min)

Introduction

30

Introduce the Concept of the TTCN-3-Certificate, Coherence curriculum, Training, Examination and certification, Associated institutions and international connections (certification body ISQI, training providers, ETSI).

GTB eV

Curriculum TTCN-3 Certificate

Basics of TTCN-3
Introduce the background, history, objectives, and main concepts of TTCN-3.

60

Motivation, History

15

Explain that TTCN-3 Supports Testing current system technologies Different kinds of tests functional, integration, interoperability, scalability, load, Platform-independent testing Integrated graphical test development, -documentation und -analysis Adaptable, open test environments Local and distributed test setups Avoids heterogeneous test environments that are hard to maintain and proprietary solutions which are expensive to develop and maintain Is The Testing and Test Control Notation The standardised test specification and test implementation language Developed based on the experiences from previous TTCN versions Applicable for all kinds of black-box testing for reactive and distributed systems Was developed to Enable testing of current and upcoming technologies Consolidate test concepts Widen the scope of application Is historically based on The ISO/IEC Conformance Testing Methodology and Framework With the TTCN: Tree and Tabular Combined Notation (version 1,2) Version 2++ developed by ETSI MTS Version 3 (TTCN-3) developed by ETSI MTS And currently continuously maintained by ETSI Main Concepts 45

Introduce the main aspects of TTCN-3. Being: Configuration: Dynamic concurrent test configurations with test components Communication with various communication mechanisms (synchronous and asynchronous) Control: Test case execution and selection mechanisms Improved Module concept Harmonisation with ASN.1 Extendibility through attributes, external functions, and external data
GTB eV
8

Curriculum TTCN-3 Certificate

Well-defined syntax, static and operational semantics Different presentation formats (Text, tabular, graphical)

Mention the principal structure of the TTCN-3 standard consisting of The textual core language covering all the language features (types, data, behaviour, etc.) Other presentation formats including the graphical and the tabular format Open data interface including ASN.1, IDL and XML Introduce the basic concepts of TTCN-3 including Black-box testing (giving stimuli to the SUT via well-defined interfaces, observing and evaluating the response) Test configurations (covering TSI, MTC; PTC and the port and test component handling) Test system architecture (with the test system, SUT, TSI and real test interface) Message-based ports (with FIFO queue and send and receive statements) Procedure-based ports (with FIFO queues, roles of client and server, and with call, getcall, reply, getreply, raise and catch statements) Test verdicts (list all the verdicts and their precedence, explain the only-getworse rule, explain the verdict handling within test components and the accumulation of the overall verdict result, explain the setverdict and getverdict statement) Introduce the basic elements of a TTCN-3 module covering Declarations and control Types, and templates Functions, altsteps, and test cases Module parameters and constants

GTB eV

Curriculum TTCN-3 Certificate

Language Features: Data Types and Templates

60

Give an overview of the data type system of TTCN-3 and discuss the different kinds of types and the definition of templates (including the matching mechanisms).

Basic and structured types

20

Mention that data type definitions are build from predefined and user-defined structured types. Explain Basic types Structured types Any type Configuration types Type compatibility rules Explain the different possibilities for sub-typing including range, list and length restrictions. Explain and discuss type compatibility rules. Discuss the usage of different types. Explain values and expressions and discuss the construction of expressions. Explain the TTCN-3 operators including Arithmetic operators String operators Relational operators Logical operators Bitwise operators Shift operators Rotate operators

Send templates, receive templates, inline templates

40

Explain that templates define test data they either transmit a set of distinct values or are used to test whether a set of received values matches a template specification they have the following possibilities they are a way to organize and to re-use test data, including a simple form of inheritance; they can be parameterized; they incorporate matching; they can be used with either message-based or procedure-based communication send templates contain only concrete values and no matching operations Receive templates can contain wildcards or matching operations

GTB eV

10

Curriculum TTCN-3 Certificate

Explain that templates Have global scope Can be parameterized Can be modified (achieving a simple form of inheritance) Can be unnamed (so called in-lined templates) Explain the different matching mechanisms including Matching for specific values Matching instead of values Special matching symbols inside values Matching attributes Discuss the purpose of different template features.

GTB eV

11

Curriculum TTCN-3 Certificate

Language Feature: Communication

60

Give an overview on the available forms of communication and their purpose. Explain the differences of unicast, multicast and broadcast communication in TTCN-3.

Message-based communication Explain the basic FIFO semantics of ports, message port types and their compatibility rules. Discuss the operations available on message-based ports send, receive, trigger, check. Signature-based communication Explain the The logic of an operation call Client and server role of test components Procedure port types and their compatibility rules Discuss the operations on signature-based ports call getcall reply getreply raise catch check timeout Port handling Discuss the initial status of ports, the setup of connections between ports, the differences between connected and mapped ports. Explain one-to-one and one-to-many connections (incl. from/to addressing schemes)

30

15

15

GTB eV

12

Curriculum TTCN-3 Certificate

Discuss port handling operations including connect map start stop clear

GTB eV

13

Curriculum TTCN-3 Certificate

Language Features: Test Configurations

60

Introduce component types, the role of configurations in TTCN-3, the set-up and teardown of configurations.

Basics of configurations

20

Explain that a configuration consists of a set of inter-connected test components with well-defined communication ports and an explicit test system interface which defines the border of the test system Within every configuration there is one and only one main test component (MTC) MTC is created automatically at the start of each test case execution. The behaviour defined in the body of the test case is executed on this component. During execution of a test case other components can be created dynamically. These test components are called parallel test components (PTCs). Explain and discuss Static and dynamic configurations Valid and invalid configurations Component types 20

Explain how component types are defined and that they can have locally declared variables, timers, ports, constants. that functions share data when executed via these locally declared properties of a component type.
Explain the differences between normal and alive test components.

Component handling Explain how components are created, started, and terminated including stop by itself,
GTB eV

20

14

Curriculum TTCN-3 Certificate

external stop, return from test component behaviour, completion of test component behaviour. checked if they are running. checked if they have terminated.

GTB eV

15

Curriculum TTCN-3 Certificate

Language Features: Behaviour

180

Introduce the principle behavioural elements of TTCN-3 including functions, altsteps, test cases, external functions, statements and operations. Introduce also the timing concepts.

Functions, altsteps, test cases, external functions Explain that functions are the building-blocks of the test system behaviour. functions have local declarations and a program part. external and pre-defined functions can be used.

60

Discuss the different kinds of functions. Include: Pure Data functions (without component and port behaviour). Behaviour functions (with runs on). Test case as special behaviour functions (with runs on and system). External functions. Pre-defined functions. Discuss the various predefined functions and their use.

Statements

60

Introduce basic program statements, behavioural program statements, and statements for default handling. Explain the assignments, logging and the control structures in Label and Goto If-else For loop While loop Do while loop Stop execution Returning control Explain the snapshot semantics of TTCN-3and discuss the behavioural program statements Alternative behaviour Re-evaluation of alternative behaviour Interleaved behaviour Explain the default mechanism of TTCN-3, the use of altsteps in alt statements and the default handling statements Activate a default Deactivate a default
GTB eV
16

Curriculum TTCN-3 Certificate

Operations Give an overview on the set of TTCN-3 operations including Configuration operations create connect disconnect map unmap mtc system self start (of a test component) stop (of a test component) running alive done killed Communication operations send call reply raise receive trigger getcall getreply catch check clear (a port) start (a port) stop (a port) freeze Timer operations start (a timer) stop (a timer) read (a timer) running (of a timer) timeout Verdict operations setverdict getverdict

45

GTB eV

17

Curriculum TTCN-3 Certificate

External actions Execution of test cases

Summarize those operations that have been explained previously, explain those that havent. Discuss the usage of these operations.

Timing

15

Explain that timers are used to observe progress of time. timers can be declared and used within control part, test cases, functions and altsteps, additionally component wide within component type definition. timer values are represented as floats, the base unit being seconds. timers can be started, stopped, read, checked for their status and can timeout. in addition, the execution of a test case and a call operation can be timeguarded.

GTB eV

18

Curriculum TTCN-3 Certificate

Language Features: Modules

90

Explain the module concept, the parts of a module definition, the means to reuse declarations from modules and the role of the control part.

Module definition and control Explain that Modules are the building blocks of all TTCN-3 specifications. A test suite is a (set of) module(s). A module may consist of a definition and a control part. Modules can be parameterized. Explain the possibilities within a module control part for test case selection, test case parameterization, and test case execution depending on verdicts of previous test case Import Explain the Main concepts of import Recursive and non-recursive import Explain the different forms of imports used for inclusion of A single definition All definitions Groups A number of definitions of the same kind

20

20

Scoping, Grouping

15

Explain the scope rules of a module definition part a control part of a module; component types functions altsteps test cases "blocks of statements and declarations" within compound statements formal parameters Explain the possibilities of grouping definitions and their use.

GTB eV

19

Curriculum TTCN-3 Certificate

Parameterization Introduce the possibilities and advantages of parameterization.

20

Explain the concepts of static and dynamic parameterization and of parameters by value and by reference. Explain the parameterization of a module type template function altstep testcase signature Attributes 15

Explain the principles of attributes and their use. Explain the different kinds of attributes being

display encode variant extension

attributes. Discuss the TTCN-3 concept of attributes as a language extension mechanism.

GTB eV

20

Curriculum TTCN-3 Certificate

Other Presentation Formats

45

Introduce other presentation formats. Explain that they can be mapped to the core language.

Graphical Format

20

Explain the elements of the graphical format and their mapping to the core language. Mention that the graphical format has its origins in Message Sequence Charts (MSC). Discuss the advantages and possible use of the graphical format.

Tabular Format

15

Explain the elements of the tabular format and their mapping to the core language. Mention that the tabular format has its origins in previous versions of TTCN. Discuss the advantages and possible use of the tabular format.

User-Defined Format
Motivate the usage of user-defined formats. Discuss the possibilities to include user-defined formats in TTCN-3.

10

GTB eV

21

Curriculum TTCN-3 Certificate

Test Execution

90

Discuss the architecture of a TTCN-3 based test system, present the test system components and their interfaces. Explain that the test execution interfaces provide standardised means to connect an abstract test suite to the actual SUT and to the actual test platform. Discuss the interfaces.

Test system components

Introduce the test system architecture consisting of the test execution engine (TE) platform adaptor (PA) system adaptor (SA) test management (TM) test logging (TL) codec (CD) component handling (CH)

15

TRI
Explain the interfaces of the test runtime interfaces (TRI) being TE -> PA PA -> TE TE -> SA SA -> TE

30

Discuss selected operations (like triSend), their implementation and their potential usage.

TCI
Explain the interfaces of the test control interfaces (TCI) being TE -> TM TM -> TE TE -> TL TE -> CD CD -> TE TE -> CH CH -> TE

45

Discuss selected operations (like tciEncode), their implementation and their potential usage.

GTB eV

22

Curriculum TTCN-3 Certificate

TTCN-3 Standard
Summarize the TTCN-3 standard parts and related documents (like from ITU-T).

30

Explain the role of ETSI in the standardisation of TTCN-3. Mention possibilities of contributing to the development of TTCN-3.

TTCN-3 standards
Present all current parts of TTCN-3 and outline their structure, i.e. Part 1: Core language Part 2: Tabular presentation format Part 3: Graphical presentation format Part 4: Operational semantics Part 5: TTCN-3 runtime interfaces Part 6: TTCN-3 control interfaces Discuss the new parts of TTCN-3 and outline their structure, i.e. Part 7: ASN.1 to TTCN-3 language mapping Part 8: IDL to TTCN-3 language mapping Part 9: XML to TTCN-3 language mapping Part 10: C/C++ to TTCN-3 language mapping

15

Standardization and Maintenance of TTCN-3

Explain the standardisation process. Discuss the roles of ETSI and MTS.

15

Explain the change request process for language extensions such as for new language mappings or new language features. Explain how TTCN-3 is maintained. Explain the possibilities of influencing the maintenance of TTCN-3 through change requests and active contributions in the TTCN-3 community.

GTB eV

23

Curriculum TTCN-3 Certificate

Test Design and Development

105

Outline along concrete examples the design and development of a test solution with TTCN-3. Explain the general steps of test design and development and relate them to the TTCN-3 language concepts. Outline exemplarily parts of a TTCN-3 test design and development methodology.

Test purposes and test cases

15

Discuss the general concepts of test purpose and test case and explain the TTCN-3 features for specifying test purposes and test cases. Explain that TTCN-3 has no dedicated test purpose concept. Instead, TTCN-3 documentation tags or user-defined attributes are to be used to document test purposes and to relate test purposes to test cases. Outline the structuring of a TTCN-3 test solution into TTCN-3 modules and groups. Explain the usage of groups to reflect the test suite structure. Explain the usage of modules to enable various control parts for the same set of test cases (provided in separate modules). Provide concrete examples in TTCN-3.

Functional, conformance, interoperability test aspects

45

Discuss the principal aspects of functional, conformance and interoperability testing. Present possible test configurations for functional, conformance and interoperability test solutions. Discuss which TTCN-3 language features are typically being used for functional, conformance and interoperability tests (to define test data and test behaviour). Explain how these concepts are being used. Explain along an example the specification of functional, conformance, and interoperability tests in TTCN-3.

Performance, load and scalability aspects

45

Discuss the principal aspects of performance, load, and scalability testing. Present possible test configurations for performance, load, and scalability test solutions. Discuss which TTCN-3 language features are typically being used for performance, load, and scalability tests (to define test data and test behaviour). Explain how these concepts are being used. Explain along an example the specification of performance, load, and scalability tests in TTCN-3.

GTB eV

24

Curriculum TTCN-3 Certificate

Open Data Interface

60

Outline the open data interface of TTCN-3 to link the TTCN-3 type and data system with other external type and data systems. Explain the principals of the language mapping from external languages to TTCN-3. Select one of the existing language mappings (ASN.1 to TTCN-3, IDL to TTCN-3 or XML to TTCN-3) and provide details of it by giving examples of the mapping rules and of the usage. Explain how to define own language mappings.

CORBA IDL

30
(if IDL is selected for a detailed explaination)

Explain the principal structure of the mapping rules. Discuss the usage of the mapping to test CORBA object based systems and applications. Explain the relation of this mapping to other IDL techniques. Provide the IDL specification for a concrete example and discuss selected mapping rules for this example.

ITU ASN.1

10

Explain the principal structure of the mapping rules. Discuss the usage of the mapping to test ASN.1 based interfaces and protocols.

W3C XML

10

Explain the principal structure of the mapping rules. Discuss the usage of the mapping to test XML based interfaces, protocols and applications.

Other Languages

10

Explain the approach to develop own mapping rules. Explain the central elements of the mapping such as scoping, identifiers, types and values. Use potentially an example such as Java or SQL for the explaination.

GTB eV

25

Curriculum TTCN-3 Certificate

TTCN-3 test tools

30

There are a number of tools that support different aspects of TTCN-3 based testing. Explain the various types of tools including editor, compiler, execution environment, test generator, test validator, debugger, etc. Review existing tools. Provide if possible tooling to be used for the exercises.

Different components of a TTCN-3 test platform

15

Classify the TTCN-3 tools into Test design and development (for editing, test validation, test generation, compilation) Test execution (for adaptation and execution, result generation and analysis) Test management (for test control and inclusion into the overall test process) and explain their properties. Provide concrete examples for each tool class.

Tool supported TTCN-3 based test process

15

Outline a TTCN-3 based test process in relation to the fundamental test process (as defined in ISTQB Certified Tester, Foundation Level). Explain that TTCN-3 is a test technology for test analysis and design and for test implementation and execution. Explain that TTCN-3 needs to be linked with planning and control and with evaluating exit criteria and reporting.

GTB eV

26

Curriculum TTCN-3 Certificate

6 Glossary
ASN.1 CD CH CTMF ETSI ETSI ES ETSI MTS FIFO GFT GUI IDE IDL ICS ISO ITU-T IUT IXIT MTC PA PCO PTC SA SUT TA TC TCI TE TFT TL TM TRI TSI TTCN-2 TTCN-3 XML Abstract Syntax Notation One TCI Codec Interface (Encoder/Decoder) TCI Component Handling Interface Conformance Testing Methodology and Framework European Telecommunication Standards Institute ETSI European Standard ETSI Working Group on Methods for Testing and Specification First In First Out Graphical Format of TTCN-3 Graphical User Interface Integrated Development Environment Interface Description Language Implementation Conformance Statement (defined in CTMF) International Standards Organization International Telecommunication Union Implementation Under Test Implementation Extra Information for Testing (defined in CTMF) Main Test Component TRI Platform Adapter Interface Point of Control and Observation Parallel Test Component TRI System Adapter Interface System Under Test TRI Test Adapter Interface Test Component TTCN-3 Control Interfaces TTCN-3 Executable Tabular Format of TTCN-3 TCI Logging Interface TCI Test Management Interface TTCN-3 Runtime Interfaces Test System Interface Tree and Tabular Combined Notation Test and Testing Control Notation Extended Markup Language

GTB eV

27

Curriculum TTCN-3 Certificate

7 References
[1] [2] ETSI ES 201 873-1 (V3.1.1): "Methods for Testing and Specification (MTS); The Testing and Test Control Notation; Part 1: TTCN-3 Core Language". ETSI ES 201 873-2 (V3.1.1): "Methods for Testing and Specification (MTS); The Testing and Test Control Notation; Part 2: TTCN-3 Tabular Presentation Format (TFT)". ETSI ES 201 873-3 (V3.1.1): "Methods for Testing and Specification (MTS); The Tree and Tabular Combined Notation; Part 3: TTCN-3 Graphical Presentation Format (GFT)". ETSI ES 201 873-4 (V3.1.1): "Methods for Testing and Specification (MTS); The Testing and Test Control Notation; Part 4: TTCN-3 Operational Semantics ". ETSI ES 201 873-5 (V3.1.1): "Methods for Testing and Specification (MTS); The Testing and Test Control Notation; Part 5: TTCN-3 Runtime Interface (TRI)". ETSI ES 201 873-6 (V3.1.1): "Methods for Testing and Specification (MTS); The Testing and Test Control Notation; Part 6: TTCN-3 Control Interface (TCI)".

[3]

[4] [5] [6]

GTB eV

28