MIPRO 2010, May 24-28, 2010, Opatija, Croatia

Workflow testing

Damir Robi
Advanced Technology and Services, ICT for Government and Enterprise Segment
Ericsson Nikola Tesla d.d.
Krapinska 45, 10000 Zagreb, Hrvatska
Telefon: +385 1 365 3448 Fax: +385 1 365 3610
E-mail: [email protected]

Business process management systems become very or manual activities (people work) or both. A workflow
popular in developing enterprise systems today. Workflow management system, like IBM FileNet Business Process
activities as subset of business processes does not include only Manager, is a software system that defines, creates, and
interaction with the human user but they also interact and manages the execution of workflows through the use of
communicate with other software applications and systems software, running on one or more workflow engines,
which functionalities are included in business process. As which is able to interpret the process definition, interact
workflows become more and more complex the probability with workflow participants and, where required, invoke
that they are designed with errors increase. Consequently the the use of IT tools and applications [2]. The important
need for testing the correctness of running workflows in the achievement of workflow management is the explicit
phase of designing or immediately after becomes bigger. Such representation of process structures in process models and
workflow testing includes checking whether the expected the controlled enactment of business processes according
activities are executed and workflow data properties have to these models.
correct value. However, supporting tools for such workflow Developing workflows which some of them are quiet
testing is very poor. This paper addresses the problem of complex, we have always facing the problem of real-life
automated workflow testing in early development cycle and testing those workflows because of lack of a workflow
describes testing tool we developed for the purpose of real-life testing tools. Term real-life testing means performing tests
testing of workflows running on IBM FileNet Process Engine. on process instances running on workflow engines.
Therefore workflow tests performed by workflow „test-
infected“ developers [3] are always done manually without
I. INTRODUCTION any systematic approach or automation. Real-life testing of
workflows is left for system testing when mistakes cost
Each product that every company provides to the market much more then in earlier phases. So we decided to find
is the outcome of a number of organized activities which out how to test workflows in more efficient way.
makes business process. Business process activities can be In Section II I will present FileNet Business Process
performed by the company’s employees manually or by Manager. All tests that WorkflowTester run are performed
the help of information systems. For the last ten years on that workflow engine. In section III will be discussed
many companies were very interested in informatization of about problems which inspired us for implementing
theirs business processes and getting software for WorkflowTester. Functionality of WorkflowTester and
managing and controlling business processes called WorkflowTester as example are described in sections IV
business process management system. To increase the and V.
quality of such enterprise solutions, where the accent is
given on business process automation, the need is to II. FILENET
provide a way of real-life testing of business process
definitions in runtime. In this paper I will try to describe
FileNet, a company acquired by IBM in 2006,
our approach for such testing which we used in our
developed software to help enterprises manage their
projects. In later chapters will also be described
software and business processes. The FileNet system
WorkflowTester prototype tool we have developed in
includes a FileNet Process Designer, a FileNet Process
purpose of helping us to automate and satisfy testing
Administrator, and a FileNet Process Engine [4].
requirements. At this moment WorkflowTester supports
only IBM FileNet Business Process Manager. First, a process structure is modelled graphically with
the Process Designer and tasks are assigned to work
Business process is a set of one or more linked
queues. These work queues and the associated users are
procedures or activities which collectively realize a
created outside the Process Designer. Graphical
business objective or policy goal, normally within the
representation of the process is without formal semantics
context of an organizational structure defining functional
and without verification of correctness [Figure 1]. Then,
roles and relationships. At the highest level, workflow is
the created process definition is feeded to the Process
all those activities that combine the processes, the tools
Engine to start the execution of the workflow. The
and the people all working toward a business objective. In
execution data for individual cases is logged by the
the context of automation, ‘workflow’ has a specific
Process Engine and can be accessed with the Process
meaning where Workflow is the automation of a business
Administrator.
process, in whole or in part, during which documents,
information, or tasks are passed from one participant to Process Engine is the core of IBM FileNet BPM. It is a
another for action, according to a set of procedural rules state machine which handles all workflow routing. Two
[1]. Actions can be automated activities (software-based) types of Workflow activities are supported by Process

320 on April 19,2024 at 16:36:58 UTC from IEEE Xplore. Restrictions apply.
Authorized licensed use limited to: Pontificia Universidad Javeriana. Downloaded
 diagram, as well as executable properties used at run time.
XPDL should be input for creating test cases. In that way
we will also satisfy a requirement not to alter existing
workflow definitions. For running test cases will be used
workflow engine API wrapped and exposed via interface
as vendor specific adapter. With little coding it will be
possible to support many available workflow engines on
the market today.
Another thing we were thinking about is automation.
Since it will be made for workflow developers it must
Figure 1: Workflow created in IBM BPM Process Designer provide test automation and regression tests.

Engine: System activity and user interaction activity. IV. IMPLEMENTATION

System activities do not involve a human user; they are
executed by information systems. If a human user provides
this information, then it is a user interaction activity. A layer based approach was used for modeling
Supporting system activities and user interaction activities WorkflowTester. That means that functionalities we
lead us to the conclusion that Process engine supports wanted the WorkflowTester to have are distributed on four
human interaction workflows. Process Engine also layers: Test creation layer, Test organization layer, Test
provides Java API which enable programmers to develop execution layer and Test results layer [Figure 2]. The
applications that can perform Process Engine functions functionalities of each layer are pointed out in the
such as launching a workflow, searching for and subsequent sections.
processing work items, or working with process status.
Every Workflow designed in Process Designer can be Test Creation
exported in XPDL (XML Process Definition Language)
format which is very important because XPDL is input Testing a running workflow means sending data to and
format for WorkflowTester application. receiving data from its endpoints, according to the
business rules. WorkflowTester must provide a way for the
tester to specify such rules. Two things are most important
III. BACKGROUND for testing workflow instances. First is checking if the
workflow is in a state as it should be according to business
rule. Second, does the workflow related properties are set
to expected value? Besides defining of what to test,
After workflow is designed, the good practice is that the additional requirement must be considered also:
designer who has designed the workflow runs a test for automation. The ultimate goal of workflow testing is
that workflow. In our case until recently they were doing automated repeatable testing which means the test should
that in a way to run the process instance and test it be executable as a whole.
manually via application interface. For every human
oriented activity, designer must enter workflow relevant
data for the current work activity. When all data is entered
designer completes current activity which leads to the start
of another where he repeats activities of filling data until it
reach the last step of the workflow. In every step designer
must check if workflow under test is in the right state and
if data properties have expected value.
Disadvantage of that kind of testing approach is lack of
automation. If workflow is complex and has many possible
scenarios or alternative flows, manual testing took long
time. In most cases, developers manually check only basic
flow or basic functionality of workflow. If we have not
performed any tests after developing a workflow, it
resulted with maby errors in phase of system test when
discovering of such errors cost us much more. Regression
tests in workflow development phase did not exist at all.
So we realized that we need new testing level. It should be
a kind of unit testing [5] level which will cover workflows
and custom Java components which are integrated into
workflows. To do that, the best way is to have an
Figure 2: Testing architecture
application which will run and monitor workflow tests.
When we started to thing about such application we
wanted an external application which does not interfere
with workflow definition. It should help testers to create
test cases and to enable workflow testing in the earliest

321 on April 19,2024 at 16:36:58 UTC from IEEE Xplore. Restrictions apply.
Authorized licensed use limited to: Pontificia Universidad Javeriana. Downloaded
Test Organization
Test organization should define how test cases are
structured and what they include. WorkflowTester must
have a functionality which helps tester in creation of test
cases. Following logic, a test case can be a workflow Test Result
scenario. But such test cases can be very big and too result types
complex. For example, two different workflow scenarios
have the same sequence of ten activities and differ in last Test Execution
activity (of course not end activity). If test case is equal to execution support
workflow scenario tester will do almost the same job twice
during creation of two test cases. To avoid this repeating, Test Organization
test cases should be granulated in smaller parts, let us call organization form
it workflow blocks. Composition of two or more workflow
blocks makes one workflow scenario. In that case, in Test Creation
example above we will have three workflow blocks. One specification od test case creation
of them will be common for both scenarios.
All test cases must be integrates into one, representing
the complete test case. The true value of automated testing Figure 3: WorkflowTester components diagram
comes from executing many test cases as often as possible.
Therefore, it is necessary to be able to group tests into
composite tests (so-called test suites). Test suites should be and parameters depending on what type the specific
created separately from test cases having only the activity is (system activity, user activity, submap activity).
information which test cases will run and in what order. Those objects are then used by Test case Builder
component which deals with creation of test cases. It
Test Execution interacts with presentation layer in order of navigating user
when he builds workflow scenario for test. This
Test execution means creating a harness around the component is also responsible for saving test data in
process under test, executing the process, and handling structured format. At this moment it is XML file. Test
input and output data for a concrete process instance Runner deals with test execution and test evaluation. Test
according to the specification. Approaches to the test execution has two stages: Initialization and test run.
execution can be real-life testing and simulation testing. Initialization includes loading test cases from outer XML.
We chose real-life testing which means actually process on After that, Test Runner communicates through workflow
engine and invoking it using proprietary API. In our case engine adapter (in our case it is FileNet BPM adapter) in
FileNet API [8] is decoupled from WorkflowTester test order to retrieve workflow-related operations. User
execution engine in the way of wrapping that API with interface component consists of two main forms: Test
exposed interface so Workflow tester can be independent creator and Test Runner. As their name say, first one is for
as much as possible for running tests with other workflow test creation and second one for their running.
engines. Note that this means that all outer system which
communicates with process under test must be replaced by Usage Example
„mock“ [9] objects.
When WorkflowTester is started and workflow
definition is loaded via XPDL file, Test Creator form
Test Results
shows up. The upper part of the form is dedicated for
Execution of the tests yields results and statistics, which specific workflow path navigation. During the creation of
are to be presented to the user at a later point in time. The test scenario unambiguously are assigned previous and
most basic of all unit test results is the boolean test current step. From fall down menu with label „Next step“
execution result which all test frameworks provide: A test user chooses next activity. Menu contains only those
succeeds, or it fails. Test result report must also include activities which are followed by current activity. When
path coverage and activity coverage. Full activity coverage „Next step“ is chosen clicking on right arrow command
is reached if all activities in the workflow are executed at button previous step becomes next step and next step
least once. Path coverage covers all possible paths through becomes current step.
the whole workflow. Lower part of the form contains two tables. First table
has two columns. In left column are showed only exposed
workflow properties of current step. Right column fields
V. WORKFLOW TESTER are editable so user enters the property values which will
be sent to the process under test just before committing
current activity. In second table are showed all workflow
Figure 3 shows main components of WorkflowTester properties. Right column is also editable and contains
developed in Java [10]. Each component is described values which are compared to those in process under test.
below. If no value is set then this property will not be checked.
Those property values can be filled also when test is
Workflow is loaded by XPDL Loader component. It
running. In that case the input form with properties will
creates NxN matrix of activity objects where N is number
show up enabling the user to input test data. This feature
of activities in workflow. Objects contain specific
can be enabled by checking the checkbox „Manual input“
functions
for specific activity. This form is also used for creating

322 on April 19,2024 at 16:36:58 UTC from IEEE Xplore. Restrictions apply.
Authorized licensed use limited to: Pontificia Universidad Javeriana. Downloaded
workflow segments. The only difference between creating Once some changes have been performed,
workflow scenarios is that first and last activity must be WorkflowTester may be used to check it immediately if
marked with flag. the behavior of the workflow is still identical.
After loading of XML file with at least one workflow Although this tool has helped us in few projects in early
scenario Test runner can be started. In Test runner form discovering errors there is lot of space for its improvement
user can chose what test cases or what test suites will be as much as improving the test methods. At this moment
performed. Test results will be presented in a table in the GUI does not meet standards of simplicity and intuitivty so
form. If test case has passed then this test case gets status there should invest much more effort to improve it.
OK. Otherwise if it fails the status will be FAILED with Implementing graphical workflow process viewer will
some explanation. Test is successful if workflow has greatly improve creation of workflow test cases.
passed through all expected activities and if all test
statements are asserted. In the same form path and activity
coverage is expressed in percentage. REFERENCES

[1] Workflow management coalition. http://www.wfmc.org/

VI. CONCLUSION AND FUTURE WORK
standards/docs/TC-1011_term_glossary/v3.pdf
[2] Mathias Weske, Business Process Management: Concepts,
In this paper was presented a way of real-life testing Languages, Architestures, Springer Berlin Heidelberg, 2007
workflows in our projects which resulted creating [3] Gamma, E., Beck, K. JUnit Test Infected: Programmers
WorkflowTester tool. We make a small step towards Love Writing Tests, http://junit.sourceforge.net/doc/
testinfected/test
[4] Wei-Dong Zhu, Eric Adel, William Benjamin, Imtiaz A
Khan, Mike Marin, Mark Yingling: Introducing IBM FileNet
Business Process Manager, IBM Redbook, Aug 2008
[5] Glenford J. Myers: The Art of Software Testing, Wiley 2004
[6] Workflow management coalition,
http://www.wfmc.org/xpdl.html
[7] Stephen A. White, Derek Miers BPMN Modeling and
Reference Guide: Understanding and Using BPMN, Future
Strategies 2008
[8] Wei-Dong Zhu, Bill Carpenter, Tim Lai, Wei Liao:
Developing Applications with IBM FileNet P8 APIs;
Redbook 2009
[9] Roy Osherove: The Art of Unit Testing: With Examples in
.NET, Manning 2009.
[10] Java 2 Platform, Enterprise Edition (J2EE),
http://java.sun.com/j2ee

Figure 4: WorkflowTester form for creating test cases

workflow unit level testing. That means that we have

defined key factors typical for unit level testing in
workflow environment. The smallest testable part (unit) is
workflow itself. Test cases are identified as workflow
scenarios with predefined input data and data used for
output validation. Test automation as one of key factors is
achieved by coupling independent test cases into suites and
executing them providing the test results to the tester.
Such approach resulted with increased project quality. In a
short time WorkflowTester has proved that it was worth of
effort to develop it.

WorkflowTester contribute in increasing the quality of

projects based on BPM. It is intended to workflow
developer allowing him to perform automated tests on
running workflows. Even though bugs may not be fully
eliminated but their amount can be reduced noticeably.
Using WorkflowTester it is possible to change parts of the
workflow throughout the lifetime of the application. This
refers for example to the readability of the workflow
definition or naming conventions for activities and events.

323 on April 19,2024 at 16:36:58 UTC from IEEE Xplore. Restrictions apply.
Authorized licensed use limited to: Pontificia Universidad Javeriana. Downloaded