13

March 2011

The Magazine for Professional Testers

printed in Germany
print version 8,00 €
free digital version

Testing @ Domains –
How does Finance, Automotive,
Medical etc test?
www.testingexperience.com

Do we have to take care of the domains?

l
ia
ec
s Sp
ce
en
er
nf
Co
ISSN 1866-5705

© Alexander Raths - Fotolia.com

 © PanOptika - Fotolia.com

The key principle of testing

“Testing is context dependent”
by Nisha Dhiraj Kale

Today’s test teams have to cope with the increasing complexity of than having to change a functionality in a large system that is not
the systems under test, while often schedules, budgets and team working as requested or as designed!
sizes have not been scaled accordingly. Not every part of the tests
Defect clustering
can be automated and, due to limitations in time and budget,
only a small fraction of all use cases can be covered in manual During testing it may be observed that most of the reported de-
testing. This article describes the domain knowledge necessary fects are related to a small number of modules within a system
for a software tester and its advantages, particularly with regard that contain most of the defects found. This is the application of
to helping the testers manage the complexity issues and select the Pareto Principle to software testing: Approximately 80% of
and prioritize the tests according to the application under test. the problems are found in 20% of the modules.

Software Testing Principles: The pesticide paradox

Software testing is an extremely creative and intellectually chal- If you keep running the same set of tests over and over again,
lenging task. When testing follows the following principles given the chances are that no further new defects will be discovered
in the ISTQB® Foundation Level syllabus, the creative element of by those test cases. Because as the system evolves, many of the
test design and execution rivals any of the preceding software de- previously reported defects will have been fixed and the old test
velopment steps. cases do not apply anymore. Anytime a fault is fixed or a new
functionality added, we need to do regression testing to make
Testing shows the presence of bugs
sure the new changed software has not broken any other part of
Testing an application can only reveal that one or more defects the software. However, those regression test cases also need to
exist in the application. However, testing alone cannot prove that change and reflect the changes made in the software, in order to
the application is defect free. Therefore, it is important to design be applicable and hopefully find new defects.
test cases which find as many defects as possible.
Testing is context dependent
Exhaustive testing is impossible
Different methodologies, techniques and types of testing are re-
Unless the application under test (AUT) has a very simple logical lated to the type and nature of the application. For example, a
structure and limited input, it is not possible to test all possible software application in a medical device needs more testing than
combinations of data and scenarios. For this reason, risk and pri- a games software. More importantly a medical device software
orities are used to concentrate on the most important aspects to requires risk based testing, be compliant with medical industry
test. regulations and possibly specific test design techniques. By the
same token, a very popular website needs to go through rigorous
Early testing
performance testing as well as functionality testing to make sure
The sooner we start the testing activities, the better we can utilize the performance is not affected by the load on the servers.
the available time. As soon as the initial products, such the requi-
Absence of errors fallacy
rement or design documents are available, we can start testing. It
is not uncommon for the testing phase to get squeezed at the end Just because testing didn’t find any defects in the software, it
of the development lifecycle, i.e. when development has finished, doesn’t mean that the software is ready to be shipped. Were the
so by starting testing early, we can prepare testing for each level executed tests really designed to catch most defects, or where
of the development lifecycle. they designed to see if the software matched the user require-
ments? There are many other factors to be considered before ma-
Another important point about early testing is that when defects king the decision to ship the software.
are found earlier in the lifecycle, they are much easier and chea-
per to fix. It is much cheaper to change an incorrect requirement In software engineering, domain knowledge is knowledge about

84 The Magazine for Professional Testers www.testingexperience.com

 5. Domain knowledge is also important for defect triage: Kno-
the environment in which the target system is to operate, and
wing how the application will likely be used and how it is
every application applies different methodologies, techniques expected to perform, will tell QA whether a given defect is
and types of testing which are related to the type and nature trivial, significant, or in fact critical.
of the application. Different software products have varying re-
quirements, functions and purposes, so the same test cases and 6. Terminology: Reporting in the language of the business, re-
sulting in good rapport with the business team(s).
strategy cannot be applied across all applications. For example, a
software application in a medical device needs more testing than Are you going to test the BFSI applications (Banking, Financial
games software. More importantly a medical device software re- Services and Insurance) just for UI or functionality or security or
quires risk based testing, be compliant with medical industry re- load or stress? You should know what the user requirements in
gulators and possibly specific test design techniques. By the same banking, working procedures, commerce background, exposure
token, a very popular website, needs to go through rigorous per- to brokerage etc. are, and you should test the application accor-
formance testing as well as functionality testing to make sure the dingly. Only then you can say that your testing is enough. This is
performance is not affected by the load on the servers. where domain experts are needed.

Testing as a job requires one to be proficient with technical know- Let’s take example of my current project: I am currently working
ledge and experience of the development life cycle, functional on a stock market application, where I need to know the basic con-
domain, and technology and tools used for testing. In addition, it cept and terminology used in stock markets. If I know the functio-
requires individuals to demonstrate a high capability for logical nal domain better, I can better write and execute more test cases
analysis, deduction, reasoning, communication, and presentati- and can effectively simulate the end user actions. This is a distinct
on skills. advantage. As testers we need to apply our domain knowledge in
each and every step of the software testing life cycle.
Business domain knowledge is important as it ensures that the
tester understands the perspective and needs of the users for While testing any application you should think like an end-user.
which the software is being built. Good business domain know- The user who is going to use your application may have a good
ledge ensures that testers are more likely to identify missing re- understanding of the domain he is working in. You need to ba-
quirements and incomplete requirements or stories that might lance all these skills so that all product aspects will get addressed.
need extra details; they can identify relevant test requirements/
scenarios and create effective test cases. Communication bet- If you have the above skills, you will certainly have the advantage
ween end-users and software developers is often difficult. They of domain and testing experience, and therefore have a better
must find a common language to communicate in. Developing understanding of different issues and deliver the application bet-
enough shared vocabulary to communicate can often take a ter and faster.
while.
Case Study: Testing BFSI applications (Banking, Finan-
When we test a particular product, we have two kinds of axes cial Services and Insurance)
where Y axis denotes the technology and where the X axis deno-
tes the domain. Domain is common for all technologies. Domain Testing has always been challenging in the software develop-
experts can write the business driven scenarios to execute the ac- ment life cycle of a product. Testing financial products requires
tual business, whereas the technology experts will be able to wri- more domain knowledge, and testing in general is very critical
te scenarios on the technology, which makes their scope narrow. in the life cycle of any software development activity. Compared
to other domain products, financial products pose significantly
The advantages of domain knowledge: more challenges due to their complexity in design, development
1. Reduces the training time: A person can be productive qui- and testing, impact on end users, and safety characteristics. This
cker than a person that has zero domain/industry know- in turn means that the test engineers face multiple challenges
ledge. So it adds value to project/product. while testing the financial software to deliver high quality to the
2. Good knowledge of the functional flow (workflow), the busi- customer.
ness processes and business rules: It will help to better un-
derstand the product requirements.
3. Good knowledge of UI features: It will help the person in im-
proving the look & feel of the UI as well finding more bugs at
an initial stage at the UI front-end.
4. Good knowledge of back-end processing: Knowing how ef-
fectively/efficiently the data/code is handled.

www.testingexperience.com The Magazine for Professional Testers 85

Some of the challenges faced while testing in the financial domain are:

Id Challenge Explanation
1 Domain and System Knowledge Lack of domain and system knowledge will result in inadequacy of testing. Test engi-
neers with testing knowledge will not only suffice testing the product as a whole but
also must know the complete functionality, usage scenarios, workflows, environment,
standards used, configurations, regulations, interoperability, domain tools etc.
2 Application Workflow Testers must have good understanding of usage and workflow use cases of the product
as used by the end users at there sites. Bugs raised on invalid use cases may be rejected
which leads to waste of effort in finding and reporting the bugs.
3 Test data/ datasets Lack of availability of standard test data/datasets during testing will lead to insufficient
test coverage. Test engineers must have standard datasets classified based on various
parameters stored at a shared central repository and make it available for testing to
ensure adequate coverage of tests at various test levels.
4 Financial domain standards Insufficient knowledge of domain standards will result in poor quality of testing.
6 Specialization tests i.e., Image Financial products have to be tested for some specific specialized tests to meet the
quality, Performance, Reliability, quality goals of the product. Test engineers must be experienced fully trained and pos-
interoperability, Concurrency, sess special skills to perform these tests.
Hardware etc.,
7 Process Compliance - ISO, CMMI Financial devices are compliance to certain standards to certify that they are fit for
intended use. Test engineers must be trained on these standards to perform their job
well in qualifying the product during testing.
8 Exposure to end users/Application Lack of exposure to end user use cases, interactions with Application specialists and
specialists and Site. non-awareness of sites will result in insufficient knowledge of product and poor quality
of tests.
10 Unstable releases Developers check in software and test team is made responsible for building the soft-
ware and verifying it. Developers did not do sufficient unit testing before checking in.
The code checked in was also not reviewed in many cases. Test engineers were unable
to proceed on testing as per their plan as they faced many issues, which should have
typically been caught during reviews and unit testing.

Conclusion:
Domain knowledge plays an important role in software testing, as one of the software testing principles says “Testing is context dri-
ven”. Testing is always done differently in different contexts. Domain expertise is important in software testing because the person
who has domain knowledge can test the application better than others.

> biography
Nisha Dhiraj Kale
As an ISTQB® certified testing
professional I am currently
working as Associate Consul-
tant at Capgemini Pune for
their Client CLSA. I have 5 years
of testing experience in soft-
ware testing. During this time,
I have worked on different
projects ranging from client
server to web-based applica-
tions. I have knowledge of the
lending, capital market and
banking domains. I am an engineering graduate and also
hold a diploma in Business Management from Symbiosis
Pune.

86 The Magazine for Professional Testers www.testingexperience.com