Expert Level Modules Overview v1.1 PDF
Expert Level Modules Overview v1.1 PDF
Certified Tester
Expert Level
Modules Overview
Copyright Notice
This document may be copied in its entirety, or extracts made, if the source is acknowledged.
Expert Level Working Group: Graham Bath, Rex Black, Marcel Kwakernaak, Judy McKay, Andrew
Pollner, Randall Rice, Erik van Veenendaal, 2010
Revision History
Version Date Remarks
th
0.1 November 16 2010 Initial version for review
st
0.2 January 1 2011 Review comments incorporated
th
1.0 March 11 2011 Released at Prague GA
th
1.1 April 12 2013 Released at Toronto GA
• Updates resulting from decision to divide ITP and TM
syllabi into parts
• Additional text for BO TM3 to improve mapping to LOs
• Revised BOs for Test Automation
• Additional section for security testing
Revision History....................................................................................................................................... 3
Acknowledgements ................................................................................................................................. 5
1. Introduction ..................................................................................................................................... 6
1.1 Intended audience ...................................................................................................................... 6
1.2 The Testing Expert ..................................................................................................................... 6
1.3 Modular structure of the Expert Level ........................................................................................ 6
1.4 Other relevant documents .......................................................................................................... 7
1.5 Expert Level syllabi and their parts ............................................................................................ 7
2. Overview of Expert Level Modules ................................................................................................. 9
2.1 Module: Improving the Test Process.......................................................................................... 9
2.1.1 Content .................................................................................................................................. 9
2.1.2 Parts....................................................................................................................................... 9
2.1.3 Business Outcomes............................................................................................................. 10
2.1.4 Syllabus coverage of parts .................................................................................................. 11
2.2 Module: Test Management....................................................................................................... 12
2.2.1 Content ................................................................................................................................ 12
2.2.2 Parts..................................................................................................................................... 12
2.2.3 Business Outcomes............................................................................................................. 13
2.2.4 Syllabus coverage of parts .................................................................................................. 14
2.3 Module: Test Automation.......................................................................................................... 16
2.3.1 Content ................................................................................................................................ 16
2.3.2 Parts..................................................................................................................................... 16
2.3.3 Business Outcomes............................................................................................................. 17
2.3.4 Syllabus coverage of parts .................................................................................................. 18
2.4 Module: Security Testing .......................................................................................................... 19
2.4.1 Content ................................................................................................................................ 19
2.4.2 Business Outcomes............................................................................................................. 21
3. Abbreviations ................................................................................................................................ 22
4. References.................................................................................................................................... 23
4.1 Trademarks .............................................................................................................................. 23
4.2 Documents and Web-Sites....................................................................................................... 23
Acknowledgements
This document was produced by the International Software Testing Qualifications Board Expert Level
Working Group:
Graham Bath (chair)
Rex Black
Marcel Kwakernaak
Judy McKay (vice chair)
Andrew Pollner
Randall Rice
Erik van Veenendaal
The authors thank the review team and all National Boards for their suggestions and input.
The following persons participated in the reviewing, commenting and balloting of this document
(alphabetical order):
Graham Bath
Johannes Bergmann
Rex Black
Kari Kakkonen
Beata Karpinska
Marcel Kwakernaak
Judy McKay
Thomas Müller
Ingvar Nordström
Joanna Nowakowska
Andrew Pollner
Andrea Rott
Hans Schaefer
Mike Smith
Erik van Veenendaal
Xiong Xiaohong
th
This document was formally released by the General Assembly of ISTQB® on 12 April 2013.
1. Introduction
The following subjects are currently considered as potential candidates for future development (in
alphabetic order):
• Performance testing
• Static testing
• System integration
• Test design techniques
• Usability testing
Other subjects may be developed as the need arises. Additional sections will be added to this
overview document as further modules become defined.
Each Expert Level syllabus is divided into parts, with each part being examined separately. Training
courses may cover an entire syllabus or an individual part.
The syllabi and parts are defined in the following table:
The following diagram shows the overall Certified Tester syllabus structure with the Expert Level
modules and their individual parts (abbreviated names are used).
The diagram shows required certifications and the career paths currently available. These will be
developed as more Expert Level modules are added.
2.1.1 Content
The syllabus [ISTQB-ITP-Syllabus] starts with an introduction to the fundamental issues concerning
improvement to the test process. Basic questions like “why improve testing?” and “what can be
improved?” are considered and the different ways that stakeholder can view quality are described.
There are a number of different approaches which can be applied to improving the test process;
overviews of these approaches are provided.
Model-based improvement is considered in depth by first examining fundamental aspects of using
models and then describing and comparing a number of well known test improvement models.
Analytical approaches to improving the test process, such as causal analysis, are covered in as much
detail as the model-based approaches. Metrics play a significant role in analytical approaches so a
number of these are covered, together with an approach to using metrics effectively e.g., the GQM-
approach.
An expert in this field will be expected to select the right approach for a given situation. The factors to
consider are covered and the relative benefits of each approach are compared.
The task of improving test processes is itself a process which can be described in several steps,
including the setting of scope and objectives for improvement, assessing the current situation,
analyzing results, performing solution analysis, suggesting improvements and priorities, creating an
improvement plan, implementing the plan and gaining knowledge from each improvement cycle. The
syllabus details each of these steps according to a standard process definition.
Different organization forms for implementing improvements to the test process are covered, including
the impact of offshoring on those organizations. The typical roles to be found in these organizations
are described and the wide range of different skills required for effectively performing those roles are
detailed.
Management of change is essential in rolling out improvements to the test process into projects and
organizations. In particular, the human factors in the change management process are covered in the
syllabus. A wide range of different skills is covered including those needed for effective interviewing
and for performing analysis.
The syllabus concludes with a detailed consideration of critical success factors.
2.1.2 Parts
The Expert Level Improving the Test Process syllabus consists of the following parts:
The following tables show the chapters and sections of the Improving the Test Process syllabus which
relate to the two parts.
Chapter/Section Title
2 The context of improvement
3 Model-based improvement
4 Analytical-based improvement
5 Selecting test process improvement approaches
6.2 Initiating the improvement process
6.3 Diagnosing the current situation
Chapter/Section Title
6.1 Process for improvement: Introduction
6.4 Establishing a test improvement plan
6.5 Acting to implement improvement
6.6 Learning from the improvement program
7 Organization, Roles and Skills
8 Managing change
9 Critical Success Factors
10 Adapting to different Life-Cycle models
2.2.1 Content
The syllabus [ISTQB-TM-Syllabus] starts by considering the purpose of testing or the test mission, and
the relation between the test policy, the test strategy and the test objectives. The expert test manager
is considered to be able to utilize and adapt all available methods to define and meet testing
objectives. Furthermore, an expert test manager is expected to be able to design a specific method or
approach to meet those objectives and to gain and maintain early management commitment.
Managing the test team and managing testing throughout the organization are addressed extensively
and special attention is given to the various project management tasks that would apply at the expert
test management level. The test managers’ role in performing risk management (risk assessment
workshops and risk mitigation management), reviews, assessments, quality gate evaluation, quality
assurance and audits are covered in detail. How to report test results and how to interpret results by
evaluation of exit criteria is elaborated upon. The metrics are described which are presented in an Key
Performance Indicator (KPI) dashboard and play a crucial role in the management of the test process.
The expert test manager is expected to select or create, evaluate and improve the most appropriate
approach for a given situation, which implies that several right approaches can occur simultaneously,
of which no individual approach stands out significantly. It is up the expert test manager to find a
suitable approach or combination of approaches which fit the organization in order to meet or exceed
the given objectives.
Different ways to organize testing as well as different types of projects for test management, including
vendor management, are covered. Typical test management topics such as incident management,
test project evaluation and tools for reporting and test management are addressed in depth.
The syllabus concludes with an overview of special test management considerations for different
domains and project factors.
2.2.2 Parts
The Expert Level Test Management syllabus consists of the following parts:
Note that different aspects of business outcomes TM1 and TM5 are addressed in part 2 and part 3.
The following tables show the chapters and sections of the Test Management syllabus parts.
Chapter/Section Title
2.2 Mission, Policy, and Metrics of Success
2.3 Test Strategies
2.4 Alignment of Test Policy and Test Strategy with the Organization
4.6 Merging Test Strategies
5.7 Integrating Tools Across the Organization
6.4 Quality Management
8.1 Test Management Considerations for Lifecycle Models
8.2 Managing Partial Lifecycle Projects
9.3 Effectiveness, Efficiency and Satisfaction Metrics for the Test Policy
Objectives
Chapter/Section Title
4.2 Types of External Relationships
4.3 Contractual Issues
4.4 Communication Strategies
4.5 Integrating from External Sources
4.7 Verifying Quality
5.5 Creating and Building Relationships
6.2 Project Management Tasks
6.3 Project Risk Management
7.2 Tracking Information
7.3 Evaluating and Using Information - Internal Reporting
7.4 Sharing Information - External Reporting
7.5 Test Results Reporting and Interpretation
7.6 Statistical Quality Control Techniques
8.3 Release Considerations
9.2 Effectiveness, Efficiency and Satisfaction Metrics for the Test Process
9.4 Project Retrospectives
Chapter/Section Title
3.2 Building the Test Team
3.3 Developing the Test Team
3.4 Leading the Test Team
5.2 Advocating the Test Team
5.3 Placement of the Test Team
5.4 Stakeholder Communication
5.6 Advocating Quality Activities Across the Organization
5.8 Handling Ethical Issues
2.3.1 Content
The syllabus [ISTQB-TA-Syllabus] defines test automation, the scope covered, and objectives. There
are a number of different tools and technologies available to assist test projects in managing,
executing, and reporting testing. Basic questions like “why would you use automation?”, “how would
you use automation?”, and “when would you use automation?” are addressed.
Strategies for successful implementation of automated testing tools are covered extensively beginning
with an assessment and evaluation of economic benefit. Discussion of generic automated testing
architectures and how they can aid in the development, deployment, and maintainability of automated
scripts is covered along with specific planning activities surrounding implementation which affect
staffing, tools, and schedule.
An expert in the field of test automation will have the understanding to know what tools should be
applied during what phase of the testing lifecycle and which approach should be used to ensure a
successful implementation for any given situation.
Tools covered in the Test Automation Module include, but are not limited to those used for functional
testing, regression testing, and test management. Performance testing is covered briefly, as a future
Expert Level Syllabus may be dedicated to Performance Testing exclusively. The analysis and
recommendations are tool agnostic, as tools from different vendors have similar capability and
complementary features.
Techniques, based on best practices including automation frameworks, are described to ensure the
successful and efficient implementation of testing tools and the skills necessary to develop and
maintain an automated test environment.
Secondary uses of test automation are also discussed. These include, but are not limited to:
automated configuration management of test artifacts, automatic metrics gathering and test reporting,
automated creation and management of test data sets, and automatic defect reporting.
The process of automating test scripts in order to create a sustainable regression test suite can be
described in several steps. There are specific roles and responsibilities for test teams wishing to
incorporate automation into the overall testing process including the detailed consideration of critical
success factors.
2.3.2 Parts
The Expert Level Test Automation syllabus consists of the following parts:
The following tables show the chapters and sections of the Test Automation syllabus which relate to
the two parts.
Chapter/Section Title
2.2 Identifying Risks and Benefits of Test Automation
2.3 Strategic Considerations
2.4 Focusing the Automation Effort
3.2 Objectives and Goals
3.3 Planning the Effort
4.2 Succeeding at the Project Level
4.3 Controlling the Test Automation
4.4 Improving the Test Automation Process
5.0 Managing the Test Automation Team
5.2 Roles and Staffing
5.3 Training
Chapter/Section Title
1. Introduction and Objectives for Test Automation
2. Planning for Automation
3. Generic Test Architecture
4. Implementation Strategies
5. Test automation reporting and metrics
6. Transitioning manual testing to an automated environment
7. Verifying the Test Automation Solution
8. Continuous Improvement
2.4.1 Content
The syllabus [ISTQB-ST-Syllabus] is designed to prepare people to effectively design and perform
security tests that mitigate security risks for specific system and organization needs. A lifecycle
approach to security testing is reinforced by describing security test activities at each stage of a
project.
The broad objective of security testing is to test and evaluate the effectiveness of security protocols in
preventing security attacks. For example, the presence and strength of encryption is within the scope
of this syllabus, but the testing of the encryption algorithm is beyond the scope of the syllabus.
This syllabus addresses the available types of security test tools and how to understand the tool
needs for an organization so that an informed evaluation can be performed.
Human factors are a major element of security risks and testing, so specific treatment of these factors
is also covered in the syllabus.
Finally, security standards are examined since they form an important basis of security risk mitigation
and security testing.
th
The current syllabus outline (9 February 2013) is shown in the table below.
Chapter Section
Security Test Purposes, Goals and Strategies The Purpose of Security Testing
The Organizational Context
Security Testing Objectives
The Impact of Security Testing
Security Testing Strategies
Improvement of Security Testing Practices
Alignment of Security Testing
Advocating Security Testing
The Scope of Security Testing Security Risks
Understanding Stakeholder Needs In The Security
Testing Processes
Security Testing Processes Security Test Process Definition
Security Test Planning
Security Test Design
Security Test Execution
The current security testing syllabus outline (9th February 2013), continued:
Chapter Section
Testing Security Protocols Authentication
Encryption
Firewalls
Intrusion Detection
Virus Scanning
Data Obfuscation
Information Security Policies and Procedures
Security Life Cycle Phases Understanding the Security Development Lifecycle
The Role of Security Testing in Early Development
Activities
The Role of Security Testing in Implementation
Activities
The Role of Security Testing in Verification
Activities
Human Factors in Security Testing Understanding the Attackers
Social Engineering
Ethical Hacking
Security Test Evaluation and Reporting Security Test Evaluation
Security Test Reporting
Designing a Security Test Dashboard
Security Test Analysis and Reporting
Security Testing Tools Types and Purposes of Security Testing Tools
Tool Selection
Tool Implementation
Tool Maintenance and Support
Standards and Industry Trends Security Standards
Security Testing Standards
Industry Trends
3. Abbreviations
Abbreviation Meaning
GQM Goal-Question-Metric
ISTQB International Software Testing Qualifications Board
LO Learning Objective
4. References
4.1 Trademarks
The following registered trademarks and service marks are used in this document:
®
ISTQB is a registered trademark of the International Software Testing Qualifications Board