Testing Circus

Volume 6 - Edition 11 - November 2015

Interview with
Magazine for Software Testers
David Greenlees
www.TestingCircus.com
 www.TalentPlusPlus.com

Kick Start Your Career

www.TalentPlusPlus.com

Career classes in Gurgaon

and online classes
worldwide.
 serious about software quality
At Doran Jones, our mission is to help technology organizations
improve their ability to deliver software and add business
value. We believe the best way to do this is through hands-on
delivery, working alongside our clients. Let us show you life at
the intersection of talent and opportunity.!

• Software Development • Training and Coaching

• Software Testing • Recruitment
• Urban Onshore Outsourcing

as seen in:!

www.doranjones.com!
 Testing Circus
Volume 6 - Edition 11 - November 2015

Table of Contents
Topic Author Page #
Testing Challenge – Project Balto Mike Talks 6
Solving Performance Engineering Puzzle Alexander Podelko 11
Jackie Chan and His 100,000 Friends Perze Ababa 15
A Fake Tester’s Diary, Part - 59 Fake Software Tester 18
Interview with David Greenlees Ajoy Singha 21
Hunting For ‘Hard-to-Reproduce’ Bugs Ravi Kumar BN 28
My Thoughts on EuroSTAR 2015 Patrick Prill 34
#Testers2Follow @Twitter Testing Circus Team 39
Book Worm’s Corner WoBo 41
Tips for security threat detection and prevention Santhosh Tuppad 42
Validata Launches New Test Data Generator Solution AppAloud 43
SmartBear Redefines Functional And Performance Testing AppAloud 44
Synack Launches Hydra AppAloud 45

Article Submision: [email protected]

Testing Circus Team Testing Circus India
Founder & Editor – Ajoy Kumar Singha Chaturbhuj Niwas, 1st Floor,
Team - Sector 17C, Shukrali,
Ÿ Srinivas Kadiyala Gurgaon - 122001
Ÿ Sanath Kumar India.
Ÿ Dwarika Dhish Mishra © Copyright 2010-2015. ALL RIGHTS RESERVED. Any
Ÿ Pankaj Sharma unauthorized reprint or use of articles from this magazine
Ÿ Jaijeet Pandey is prohibited. No part of this magazine may be reproduced
Edition Number : 62 (since September 2010) or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or by any
Editorial Enquiries: [email protected] information storage and retrieval system without express
Ads and Promotions: [email protected] written permission from the author / publisher.

*On the Cover Page - David Greenlees www.TestingCircus.com November 2015 - 04 -

 From the Keyboard of Editor

What is the one most important characteristic of an employee that a boss desires?
To me, it is the timely status update to the boss, informing if any promised deadline
cannot be met, especially when there is an issue which may prevent the project team
from achieving important milestones. It is very important that the red flags are
raised early enough (as soon as the employee realises about it) so that the boss can
work on alternative strategies. Remember, your boss has to answer to his boss too.
Tell him what is going wrong and most importantly, what can be done to fix the
issue.
To fix a large scale issue is entire team’s responsibility. So make sure your boss is
appraised of any serious issues. Do not hold the problem on to yourself until the last
minute or till the status meeting occurs. Inform as soon as possible. This serves two
purposes. First, your boss is aware of the problem early and what is being done to
solve the problem. Second, it gives him time to think of alternative solutions and
work on his portion of the firefighting or at least prepare for a different course of
action—and to present it to his boss. For God’s sake, don’t keep the problems to
yourself, especially when the work is time-sensitive, and many downstream
components are dependent on that particular piece of work. Do not fear breaking
the bad news.
In this edition, you will enjoy the interview with David the martial tester and the
other articles. Mike has a testing challenge for you. We have some prizes for you
too.
See you next edition. Happy testing!

- Ajoy Kumar Singha

@TestingCircus // @AjoySingha

Feedback please! [email protected]

www.TestingCircus.com November 2015 - 05 -

 Testing Challenge
– Project Balto
- Mike Talks
If you’ve seen the recent movie The Martian, then this
challenge might seem a bit familiar! There was a problem NASA was trying
to solve around testing a rocket, and I was bursting to email my solution to the author!
I ended up talking with a few people, and they came up with different angles on my solution. Wow.
In the end, I ran the scenario below as an exercise with my test team in our monthly meeting – they came
with some different approaches, and in the debriefing they had some very useful ideas to feedback into their
daily work.
Now it’s your turn – how do you think you’ll do? If you can, do talk it over with other testers you know. Even
better, please submit your answers to the [email protected] – I’m going to do a follow-up in a couple of
months, and I’d really like to use as many ideas from others as I can – this is your opportunity for a
mention!
Enjoy!
Mike Talks

Yesterday at 2:59am, we secretly landed the first

men back on the Moon since 1972. Commander
Alice Kramden and Lieutenant George Herbert
successfully landed their lander at Maginus
Crater, whilst their automated service module
remained in orbit awaiting their return.
During their stay on the Moon, they did an
excursion with their lunar rover and collected
some rock samples, looking for traces of He3,
which we hope to use to power future fusion
reactors, as well as set up a science station near
the lunar module.
Their mission completed, they boarded the lunar
module for the return visit – which is where
things started to go wrong. “There was a crash

www.TestingCircus.com October 2015 - 06 -

 The bad news is we have no rescue or resupply rocket
ready. Our expectations of such scenarios was that any
incident would be outright fatal.

Project Balto
Project Balto is our planned resupply lifeline for the two
astronauts stranded on the Moon. It’s named after the
famous Alaskan husky who ran supplies of medicine
during a diphtheria epidemia.

and then a zoom, as we tried to take off from the Moon”.

The lunar modules main engine refused to fire, and the
crew could see they were venting from their craft. Later
inspection from the outside of their craft showed that a
key valve had blown in the propellant fuel line, and the
module has vented its fuel for takeoff, stranding it on
the Moon.

We’re used to running regular supply rockets up to the

International Space Station every 3 months, but sadly
we used our the other month, and it’s replacement isn’t
constructed yet. That rocket uses a 2-stage rocket which
gets into low Earth orbit. But to get to the Moon, it
needs additional power, so we’re adding an additional
rocket stage to the base – similar to the one we used in
our manned lunar rocket.
Stage 1 – Liftoff stage
The good news is the oxygen tanks were not affected,
and the crew has food and air to last for 30 days.
Potentially 31 if everything goes favourably and
exertion and trips outside are kept to an absolute
minimum.

www.TestingCircus.com November 2015 - 07 -

This is the stage that is used at take-off, and usually lifts Stage 3 – Cargo package
a manned rocket into high Earth orbit. We’ve used these
for 6 previous launches. It’s reusable, but requires a
huge service between uses – hence it has parachutes and
beacons to help us retrieve and reuse it.
Key features
· Recovery parachute
· Recovery beacon
· Propellant tanks
· Guidance fins
· Re-entry shields
· 64 rocket engines
· Separation charges (between stages) Once in orbit, this section will attempt to land nearby
· Launch cameras the lunar lander, where the astronauts can use their
lunar rover to collect supplies.
Stage 2 – Lunar Orbital insertion Key features
Once in orbit, this stage will take the vehicle out of Earth · Retro rockets
orbit and into an orbit around the Moon. This design · Recovery beacon
· Propellant tanks
· Steering fins
· 5 rocket engines
· Guidance computer & autopilot
· Pressurized cabin
· Black box monitor
· Radio
· Webcam

Stage suppliers
Each rocket stage is supplied by a different supplier,
· STAGE 1 – Zeus Inc
· STAGE 2 – Hera Ltd
· STAGE 3 – Hermes Industrial
And each supplier is about 2 days by freight train away,

has been used over 50 times to resupply the

International Space Station, with low levels of failure.
Like Stage 1, when used to supply the International
Space Station, it is reusable.
Key features
· Recovery parachute
· Recovery beacon
· Propellant tanks
· Steering rockets
· Re-entry shields
· 18 rocket engines
· Separation charges
· Black box monitor
www.TestingCircus.com November 2015 - 08 -
Our first pass at working out a timeline for this was, Freight train
Each stage is then taken by freight train from the
supplier to our launch facility,

Rocket integration
Stage assembly
The stages of the rocket are then assembled.
Each stage is assembled on a production line.

Stage testing
The stage is then moved to a test facility where it’s tested Rocket testing
component by component, The assembled rocket is then tested to confirm the stages
are working together as expected.

www.TestingCircus.com November 2015 - 09 -

Revising timeline chances of a successful launch, as they do so, if we do
With the typical time to do delay, the crew will be dead for certain.
this, the astronauts will all be
dead by the time we’re
ready. So we’re going to
work around the clock,
which gives us the following
revised schedule,

With the typical time to do this, the astronauts will all be

dead by the time we’re ready. So we’re going to work
around the clock, which gives us the following revised
schedule,

What can you do to reduce the chance of

this happening?

In order to meet this deadline, we can’t afford the

dedicated test phases, although they increase the

Editor’s Note: We will select/publish top 3 solutions to the above problem. Send your
entries to [email protected] with subject ‘Project Balto’. We have few prizes too.

About the Author

Mike Talks was an aspiring astronaut in his youth, and a student of Astronomy at Sheffield University.
One of his favourite movies is Apollo 13 – about the Moon mission which faced disaster – because they
were so good at identifying problems, and finding solutions to them. This is testing at its most exciting
form.

www.TestingCircus.com November 2015 - 10 -

 Solving Performance Engineering Puzzle

Performance Engineering
and Load Testing:
A Changing Dynamic
- Alexander Podelko
There are many discussions about performance, but performance, concurrency, stress, endurance,
they often concentrate on only one specific facet of longevity, scalability, reliability, and similar).
performance. The main problem with that is that · Continuous Integration / Delivery /
performance is the result of every design and Deployment. Everything allowing quick
implementation detail, so you can't ensure performance deployment and removal of changes, decreasing
approaching it from a single angle only. the impact of performance issues.
There are different approaches and techniques to And, of course, all the above do not exist not in a
alleviate performance risks, such as: vacuum, but on top of high-priority functional
· Single-User Performance Engineering. requirements and resource constraints (including time,
Everything that helps to ensure that single-user money, skills, etc.).
response times, the critical performance path, Every approach or technique mentioned above
match our expectations. Including profiling, somewhat mitigates performance risks and improves
tracking and optimization of single-user chances that the system will perform up to expectations.
performance, and Web Performance However, none of them guarantees that. And,
Optimization (WPO). moreover, none completely replaces the others, as each
· Software Performance Engineering (SPE). one addresses different facets of performance.
Everything that helps in selecting appropriate A Closer Look at Load Testing
architecture and design and proving that it will
scale according to our needs. Including To illustrate that point of importance of each approach
performance patterns and anti-patterns, scalable let's look at load testing. With the recent trends towards
architectures, and modeling. agile development, DevOps, lean startups, and web
operations, the importance of load testing gets
· Instrumentation / Application Performance sometimes questioned. Some (not many) are openly
Management (APM)/ Monitoring. Everything saying that they don't need load testing while others are
that provides insights in what is going on inside still paying lip service to it – but just never get there. In
the working system and tracks down more traditional corporate world we still see
performance issues and trends. performance testing groups and most important
· Capacity Planning / Management. Everything systems get load tested before deployment. So what
that ensures that we will have enough resources load testing delivers that other performance engineering
for the system. Including both people-driven approaches don’t?
approaches and automatic self-management There are always risks of crashing a system or
such as auto-scaling. experiencing performance issues under heavy load –
· Load Testing. Everything used for testing the and the only way to mitigate them is to actually test the
system under any multi-user load (including all system. Even stellar performance in production and a
other variations of multi-user testing, such as highly scalable architecture don't guarantee that it won't
crash under a slightly higher load.
www.TestingCircus.com November 2015 - 11 -
 Fig.1. Typical response
time curve.
A typical response
time curve is shown on
fig.1, adapted from
Andy Hawkes’ post
When 80/20 Becomes
20/80 discussing the
topic. As it can be seen,
a relatively small
increase in load near
the curve knee may kill
the system – so the
system would be
unresponsive (or
crash) under the peak
load.
However, load testing doesn't completely guarantee that the system won’t crash: for example, if the real-life
workload would be different from what was tested (so you need to monitor the production system to verify that
your synthetic load is close enough). But load testing significantly decreases the risk if done properly (and, of
course, may be completely useless if done not properly – so it usually requires at least some experience and
qualifications).
Another important value of load testing is checking how changes impact multi-user performance. The impact on
multi-user performance is not usually proportional to what you see with single-user performance and often may be
counterintuitive; sometimes single-user performance improvement may lead to multi-user performance
degradation. And the more complex the system is, the more likely exotic multi-user performance issues may pop
up.
Fig.2 Single-user
performance vs. multi-
user performance.
It can be seen on
fig.2, where the
black lines represent
better single-user
performance (lower
on the left side of the
graph), but worse
multi-user load: the
knee happens under
a lower load and the
system won’t able to
reach the load it
supported before.
Another major value of load testing is providing a reliable and reproducible way to apply multi-user load needed
for performance optimization and performance troubleshooting. You apply exactly the same synthetic load and see
if the change makes a difference. In most cases you can’t do it in production when load is changing – so you never
know if the result comes from your code change or from change in the workload (except, maybe, a rather rare case

www.TestingCircus.com November 2015 - 12 -

of very homogeneous and very manageable workloads performance engineering approaches to mitigate
when you may apply a very precisely measured portion performance risks – but the exact mix depends on your
of the real workload). And, of course, a reproducible system and your goals. Blindly copying approaches
synthetic workload significantly simplifies debugging used, for example, by social networking companies onto
and verification of multi-user issues. financial or e-commerce systems may be disastrous.
Moreover, with existing trends of system self-regulation As the industry is changing with all the modern trends,
(such as auto-scaling or changing the level of services the components of performance engineering and their
depending on load), load testing is needed to verify that interactions is changing too. Still it doesn’t look like any
functionality. You need to apply heavy load to see how particular one is going away. Some approaches and
auto-scaling will work. So load testing becomes a way to techniques need to be adjusted to new realities – but
test functionality of the system, blurring the traditional there is nothing new in that, we may see such changing
division between functional and nonfunctional testing. dynamic throughout all the history of performance
Changing Dynamic engineering.

It may be possible to survive without load testing by Historical View

using other ways to mitigate performance risks if the It is interesting to look how handling performance
cost of performance issues and downtime is low. changed with time. Probably performance engineering
However, it actually means that you use customers to went beyond single-user profiling when mainframes
test your system, addressing only those issues that pop started to support multitasking, forming as a separate
up; this approach become risky once performance and discipline in 1960-s. It was mainly batch loads with
downtime start to matter. sophisticated ways to schedule and ration consumed
The question is discussed in detail in Load Testing at resources as well as pretty powerful OS-level
Netflix: Virtual Interview with Coburn Watson. As instrumentation allowing to track down performance
explained there, Netflix was very successful in using issues. The cost of mainframe resources was high, so
canary testing in some cases instead of load testing. there were capacity planners and performance analysts
Actually canary testing is the performance testing that to optimize mainframe usage.
uses real users to create load instead of creating Then the paradigm changed to client-server and
synthetic load by a load testing tool. It makes sense distributed systems. Available operating systems didn't
when 1) you have very homogenous workloads and can have almost any instrumentation or workload
control them precisely 2) potential issues have minimal management capabilities, so load testing became almost
impact on user satisfaction and company image and you only remedy in addition to system-level monitoring to
can easily rollback the changes 3) you have fully parallel handle multi-user performance. Deploying across
and scalable architecture. That was the case with Netflix multiple machines was more difficult and the cost of
- they just traded in the need to generate (and validate) rollback was significant, especially for Commercial Of-
workload for a possibility of minor issues and minor The-Shelf (COTS) software which may be deployed by
load variability. But the further you are away from these thousands of customers. Load testing became probably
conditions, the more questionable such practice would the main way to ensure performance of distributed
be. systems and performance testing groups became the
Yes, the other ways to mitigate performance risks centers of performance-related activities in many
mentioned above definitely decrease performance risks organizations.
comparing to situations where nothing is done about While cloud looks quite different from mainframes,
performance at all. And, perhaps, may be more efficient there are many similarities between them, especially
comparing with the old stereotypical way of doing load from the performance point of view. Such as availability
testing – running few tests at the last moment before of computer resources to be allocated, an easy way to
rolling out the system in production without any evaluate the cost associated with these resources and
instrumentation. But they still leave risks of crashing implement chargeback, isolation of systems inside a
and performance degradation under multi-user load. So larger pool of resources, easier ways to deploy a system
actually you need to have a combination of different

www.TestingCircus.com November 2015 - 13 -

and pull it back if needed without impacting other So it doesn’t look like the need in particular
systems. performance risk mitigation approaches, such as load
However there are notable differences and they make testing or capacity planning, is going away. Even in case
managing performance in cloud more challenging. First of web operations, we would probably see load testing
of all, there is no instrumentation on the OS level and coming back as soon as systems become more complex
even resource monitoring becomes less reliable. So all and performance issues start to hurt business. Still the
instrumentation should be on the application level. dynamic of using different approaches is changing (as it
Second, systems are not completely isolated from the was during the whole history of performance
performance point of view and they could impact each engineering). Probably there would be less need for
other (and even more so when we talk about "performance testers" limited only to running tests – due
containers). And, of course, we mostly have multi-user to better instrumenting, APM tools, continuous
interactive workloads which are difficult to predict and integration, resource availability, etc. – but I'd expect
manage. That means that such performance risk more need for performance experts who would be able
mitigation approaches as APM, load testing, and to see the whole picture using all available tools and
capacity management are very important in cloud. techniques.

About the Authors

For the last seventeen years Alex Podelko has worked as a performance engineer and architect for
several companies. Currently he is Consulting Member of Technical Staff at Oracle, responsible for
performance testing and optimization of Enterprise Performance Management and Business Intelligence
(a.k.a. Hyperion) products.
Alex periodically talks and writes about performance-related topics, advocating tearing down silo walls
between different groups of performance professionals. He collects performance-related links and
documents at www.alexanderpodelko.com, blogs at http://alexanderpodelko.com/blog, and can be
found on Twitter as @apodelko. Alex currently serves as a director for the Computer Measurement
Group (CMG) http://cmg.org, an organization of performance and capacity planning professionals.

www.TestingCircus.com November 2015 - 14 -

 Jackie Chan
and His
100,000
Friends - Perze Ababa
Once upon a time, a few jobs ago, I was asked to design limitation is to have the same person pay for all the
a performance test for a client that needed to upgrade a transactions.
workflow that was relying heavily on fax based
transactions. This particular workflow was simply I decided that based on that information, why not have
broken apart in 6 individual requests: a certain Jackie Chan pay for 100,000 of his friends. As a
bonus, these friends would be traveling to Europe.
● Customer opt-in Seemed pretty logical at that time.
● Calculation
● Payment I presented the plan to my QA Manager and the
● Data storage Engineering Head and I got a go signal to use our
● Send a confirmation email to the user pre-production infrastructure. That gave me 10 origin
● Send that same confirmation email as a PDF to a servers with ample instrumentation in place to find out
partner's print server what’s going on in all layers during the 8 hour test
window.
The performance requirement for that particular test
was straightforward. We needed the ability to handle So I fired up the script. At the end of 8 hours, we
100,000 opt-in's in an 8 hour period in the Asia-Pacific checked the results and we realized that a recent
region that covers the flows from the first 5 of the above configuration change in the request queueing server
mentioned requests. that seemingly broke 9 out of 10 transactions. I was able
to confirm that 10,000 of the transactions that were sent
As I was preparing the performance script, I realized a to the available origin server were processed properly,
couple of problems specifically dealing with test data. payment requests were honored, the correct emails
This means that I will possibly need 100,000 unique were received and all 10,000 transactions were sent to
individuals, along with their respective credit card our dummy print server.
information. I reached out to the Engineering Head of
this project and asked if there’s a way to create these We decided to look into the missing 90,000 and we
accounts beforehand. I was told that our payment found out that since 9 out of the 10 origin servers didn’t
provider only gave us one card number that we can test respond as expected, the queueing server preserved all
with and the way they’ve been able to go around this those transactions on memory. I was told that once those
9 origin servers were reachable from the queueing

www.TestingCircus.com November 2015 - 15 -

service then those transactions should be processed remaining origin servers, it just finished what it was told
accordingly. As a result, a second round of tests were to do. So a print server somewhere in Ireland printed all
scheduled 1 month after. 90,000 of these transactions that were dated 3 months
ago and an email server in Hong Kong received
Unfortunately, I left that company a couple weeks later corresponding email confirmations.
having found a better opportunity somewhere else.
When it comes to performance test planning, it’s key
The story doesn’t end here. that you as the tester understand the schedules and
timeline, it’s also important that you have the right
A few months after I started at my new company I got a people, proper tools, a reliable environment, and that
call from my former QA Manager. In the call also was you understand what the objectives that your software
the Engineering Head, and his counterparts from Asia- solution is trying to accomplish. When it comes to
Pacific and Europe. I was told that they just launched to people, you can never predict when people leave. If they
production two days before and there were 90,000 do, make sure there is continuity and plug as much
orders from Hong Kong to Ireland paid for by Jackie knowledge gaps as you can. Lastly, when running
Chan and they want to know if I’m familiar with these automation assisted tests, it’s always key to keep track
transactions. of your test artifacts. Since you know what you created,
you should also know what happens to them in the end.
I told them that I remember running a test some months Make sure you clean up after yourself.
ago, I described to them the situation and the results. As
it it turns out, they promoted that pre-production Then again, no matter how much you prepare, you can
environment without cleaning up or establishing a clean never know what will happen when you try to employ
state. Since the queueing server can now reach the 100,000 of Jackie Chan’s friends.

About the Author

Perze Ababa is the Test Engineering Manager for Johnson & Johnson Consumer, Inc., and is responsible
for the team providing testing related services and test tooling support for J&J Consumer Platform and
its websites. With over 13 years of software testing experience, Perze has tested for teams that built
multi-tiered desktop applications, websites and native applications for companies such as nytimes.com,
the now defunct ivillage.com and was recently the Director of Test Engineering for Viacom Media
Services. He has been a member of the Association for Software Testing since 2010 and have participated
in co-facilitating some BBST classes. He is also a founding member and organizer for the NYCTesters
meetup group.
Perze originally comes from Marawi City, Philippines, and is now living in New Jersey, USA with his
wife and three beautiful daughters.

www.TestingCircus.com November 2015 - 16 -

Reach out to a
larger number
of audience.

Write for Our

Readers.

Looking for a theme to write?

We have some ideas for you.
Click Here to get all ideas!

www.TestingCircus.com
 A Fake Tester’s Diary
e 5 9
Epi so d
http://www.testingcircus.com/category/a-fake-testers-diary/

"A week has 168 hours; I am asking you to bill at least

Yes Boss! 200. Do you understand?” he growled.
"No", I said.
This week, I learned “Corporate Testing Maths”. In
"Did you not have our cross-functional team
corporate Maths 24 = 72; it’s possible to have 30 hours
orientation?” he asked.
billed in a 24 hour workday.
"No", I was unsure if 24 times 7
I was summoned to my manager's
is 144 or 168; definitely nowhere
cabin; my New manager. He seemed
close to 200.....
in a foul mood and I walked in fully
prepared for one of his lectures on "Well, if you are in a cross
something. He was looking at my functional team then it means
billing sheets pondering over my you work across different
billed hours. (Or unbilled hours, as projects. And different clients.
he put it). For example, let's say that you
are flying from here to the USA
"You billed only 40 hours last week?”
for a client visit. How many
he asked.
hours would you bill?"
"Yes", I said".
"Nothing", I said. "It's my
"You are a floating resource who travelling time".
works across projects. You should
"No", he went on. "Bill client-1
have billed at least 200", he said.
for 2 days since you are
"What?” I thought I did not hear him travelling to meet them; if
right; it sounded like 200 to me. possible meet client-2 and
"Yes," he went on. "I said at least 200". client-3 and bill them for the trip
since you are visiting them as
I started wondering if he's sober after
well. While you are flying, work
all. I started calculating 24 times 7 to
on a document for client-4 and
find out how many hours are there in
client-5 and bill them too. That's 10 days of missed
a week.....
billing time in a 2 day time period. 240 hours of missed
"I am sober; not drunk; not yet", he said. billing time.”
I was blankly staring at him still trying to work out 24 "Let’s go again. How many people do you bill while
times 7. you are in a 1-hour phone call with your client?” he
"168", he seemed to read my thought. asked.
"Huh", I was blank again. "1 hour for client-1", I said.

www.TestingCircus.com November 2015 - 18 -

"No; he said. Get into the call with client-1 and bill him "Yes”, he went on. “5th question. Let's say you are
for the 1-hour. You are not going to be talking for the taking a coffee break and indulge in chit-chat with 5
entire duration of the call. While you are on the phone people from 4 projects. What will you do?” he asked.
with client-1, engage client-2 in an online discussion. "Bill all the 4 clients for 1 hour under the heading of
While you are messaging client-2, send emails to "Group Discussion on project status". A total of 4
client-3. That will make it 3 hours. If you bill client-1 hours billed for a coffee break with employees from
only for 1 hour, that makes it 2 missed hours. You can different accounts", I said.
bill 3 hours for 1 hour!!!”
"Oh yes... you are indeed a fast learner. Remember, a
I nodded; I was learning. 24 hour work day gives us, folks in Smart Billing, and
"3rd question. How many hours do you bill for a immense possibilities for billing 72 hours every day. If
meeting that lasts for 10 mins?” he asked. you do more, that’s super-awesome. That way, you
"10 Mins", I said. can bill a minimum of 200 hours in a 168 hour work
week", he signed off waving me off.
"Nope; if a call lasts for 10 mins, bill the client for 30
mins; learn to round off. 10 mins is 30 mins, 35-45 mins As I walked away, he said “That’s why you have
is 1 hour, and anything more is 2 hours. If a call goes targets on billing hours. The more you bill, the more
for beyond 2 hours, bill him for half-a day". your appraisal is.” The proverbial “carrot” was shown
to me as I walked away.
“Wow”.
Now all of the above is a lesson that I would have
"4th question. Let's say that you are writing a test learnt in schools. I learnt this story in a “promising”
scenario for login for client-1 and the entire activity company that has now taken action on their errant
spans 3 days. How much are you billing?" management. The first thing that I did after hearing
"3 days for client-1, 3 days for client-2, 3 days for this story is to ask for an exit from the project. You
client-3", I said. I wanted to impress him with my fast would also come across such individuals in many
learning skills. companies and projects; Companies are good, but
individuals are not. In every good company, there will
"No. Bill the client-1 for 6 days, client-2 for 5 days and
be such evil greedy businessmen who resort to such
client-3 for 2 days. Talk to the project managers on
tactics putting the company’s reputation at stake.
new clients and old clients and bill them accordingly.
Though many companies have checks and balances,
Don’t make the mistake of sending the test scenarios
some don’t; if you are in a company that does not have
within 3 days. Hold a meeting on day-1. Hold a
systems to prevent it, the best thing to do is ask for an
meeting on the 3rd day. Bill them for KT activities. Ask
exit from the project. Being a fake tester is fine, but
for a sign-off and bill them for the sign-off meeting too.”
you cannot be a “criminal tester”; stay away from such
“That’s 12 days of billing for a 3 day effort”, I said. projects; and stay safe. See you in December.

www.TestingCircus.com November 2015 - 19 -

More TestBash New York Photos - Here
 Interview with
DAVID GREENLEES Testers
Director of Testing
Doran Jones, Inc
New York, USA

David has been testing software and managing testing teams for almost
15 years. Many of these spent in one of Australia's largest government
departments, while more recently undertaking a consultant role in
multiple organizations prior to joining Doran Jones. While he has
worked in multiple industry sectors, the last 5 have been spent in finance
working on online banking implementations and regulatory reporting
programs.
He has published several articles, blogs regularly at
http://www.dmg.name/, http://martialtester.wordpress.com/ and
http://hellotestworld.com/, and is close to releasing his first book on the
combined subjects of software testing and martial arts. In 2012 David
founded the Australian Workshop on Software Testing
(http://ozwst.wordpress.com/), Australia's first software testing peer
conference.
* Interviewed by Ajoy Singha

1. How did you begin your career in testing? Was it time, and even though the approach they used is
planned or a chance? what many would refer to as a 'traditional' one, it's
Definitely chance. Like many testers that I've spoken given me a great grounding, especially when
to throughout the years I 'fell' into testing. thinking about how not to perform testing!
In my late teens I joined one of Australia's largest From there I moved in to the world of consulting,
federal government departments as a call center where I have performed various roles and various
operator. I saw it as a stepping stone to something industries. I made that move so that I could broaden
bigger and better, turns out it was! After 18 months my horizons and expend my testing knowledge,
of spending all day on the phone, and writing a time and it's certainly paid off.
variation on a sheet of paper when I used the rest 2. How did you end up working in New York for
room, I'd had enough. It was around that time that Doran Jones?
we had some visitors from the department's In the not so distant past I picked up a book called
Software Testing Center who were trying to recruit Lessons Learned in Software Testing; I'm sure
testers for upcoming projects. I don't recall the exact you've heard of it (if not, find it and read it!). The
words I used, but I grabbed one of them on the arm book was filled with lessons that I could directly
and asked them to get me the hell out of there! relate to, and it began to change the way I thought
Not long after their visit I was asked to be a part of a about testing. That book led to another, which lead
five week test execution exercise, after that, the only to blogs, which led to Twitter, etc, etc. I soon became
time I returned to the call center was to collect my someone who was looking in on a wonderful
belongings. I spent the next 10 years performing community of testers and I no longer wanted to be
various roles from tester through to manager of the on the outside.
entire testing center. I learned a great deal in that
www.TestingCircus.com November 2015 - 21 -
 DAVID GREENLEES

I began a blog, joined Twitter, started commenting While the above provides me with a great level of
on other's blog posts, and even started a local meet job satisfaction, what goes far beyond is the
up for testers in my home city of Adelaide. It could community work we do. We partner with a not-for-
be a very long and detailed story but I soon found profit IT training provider and recruit directly from
myself working with Anne-Marie Charrett and the them which is visibly changing people's lives. Being
guys from Let's Test to bring that wonderful a part of that is an amazing feeling, and I have also
conference to Australia. As a part of that I went to had the opportunity to manage the development of
Let's Test in Sweden in 2014 and spent quite a bit of a mobile application for a local public school. The
time with Paul Holland, the Managing Director of entire team is working on a volunteer basis on their
Testing for Doran Jones. own time and the excitement from the school and
If I recall correctly Paul was trying to teach me how their kids has already made it all worthwhile.
to play disc golf, and while he was laughing very 4. What are your challenges at work? How do you
loudly at my terrible form he simply asked me if I solve them?
would like to move to New York and work for Wow, I could write a novel!
Doran Jones. After arriving home, speaking about it
One of the biggest challenges I think most testers
with my very supportive wife, we took a holiday to
face is keeping their knowledge current. As a tester
New York in the Fall of 2014 to meet the Doran
you generally can't just focus on the technology as
Jones crew, and having the right opportunity land,
you also need to maintain an understanding of the
I'm here!
business. As a test manager this becomes even more
3. How is the work different in Doran Jones than important. You need to navigate technology,
that you did in Australia? business, office politics, your staff's needs and
Generally speaking it's the client that makes the wants, etc. etc.. Dividing your 8, or more likely 10-
difference. I've had to learn all about North 12, hours into each of these areas of focus is
American finance, and while you may think finance probably the most complex thing I have to do. Add
is the same the world over, I can tell you it's not. It's to that I also need to help Doran Jones build and be
almost an entirely different language. successful. The latter is not too difficult, as it's
What is different about Doran Jones is the approach something I really enjoy, but it does add to the
we take, to testing and to the community. We are juggling act.
context-driven testers at Doran Jones, and we give There is no one way to solve this problem, and the
that message at an organizational level, which is way I approach it could be different on any given
very different to any consultancy I've ever worked day. On my current project there are rare occasions
for. Our goal is to help our clients become better at of 'down-time' for the testers and so I pass on tasks
testing by teaching them different ways of to them when possible. It may be as simple as
performing it. Breaking in to clients who have only collating a report, but it helps them by developing
ever known and performed 'traditional' testing is a their skills while helping me by allowing me to
challenge, but it's a good one. It's fantastic to work focus on other things.
with them and see their reactions when we Another challenge I often face is educating non-
demonstrate what we can do. testing staff on what testing is. After years of being
told that testing is like a form of UAT, getting

www.TestingCircus.com November 2015 - 22 -

 DAVID GREENLEES

business stakeholders to truly understand and think the site will undergo several changes in the
appreciate what testing is can be quite a battle. In future. What they are I'm yet to decide upon.
the past I've held presentations and walked people 6. So do you often lecture your clients on usability?
through it, but recently I've found it more valuable
If by lecture you mean formally train, then not so
to actually show them. It's amazing how quickly
often. If you mean get up on my soapbox and
you can change someone's view on testing simply
preach the importance of usability, then almost
by showing them a bug that you found and how
every day!
you found it. They will often ask something like
My previous online banking projects are what really
“How did you find that?”, and this is the perfect
got me interested in the subject. When an
opening for you to further educate them. By them
application is customer facing it's a lot easier to
asking the question you have their attention; you
debate the importance of usability issues, especially
have their interest. That is key to successful
when those customers could end up being anyone
education.
with internet access! In more recent times while
5. You own Useology and TestingBullshit websites.
concentrating on regulatory reporting is been far
What are the objectives of those websites?
more of a challenge. When the end users are
Who said I own TestingBullshit? Oh, that's right, it's primarily made up of internal staff the important
linked from another one of my website. Oops. One usability issues very quickly end up defined as
day I found a really cool site called UXBullShit.com. 'training issues'.
I thought it was a wonderful idea and wanted to
In these instances I go directly to the end users and
know how it worked so I found the developer and
collect information from them. While I can 'talk for
he shared his code with me. It was a bit of fun and I
them' in project meetings, the real impact is felt
learned a bit more about GitHub, HTML, and CSS in
when they talk for themselves. Imagine yourself as a
the process. Your question has reminded me to
tester on a project stating in a meeting “I don't think
revisit that little project and add some more content,
the user will like how complicated it is to produce
so thank you. If you can't smile about your work,
that report” versus “I spoke to Jasmine (the actual
then what's the point?
end user) and she said she wouldn't use it because
Useology was sparked via my interest in both it's too hard and will take her too long”. Which
usability and psychology along with wanting to statement do you think would carry more weight?
build a website. I'll admit that it's been left far too
Usability should be front of mind for anything that
long in its current form and I would like to get back
requires human interaction, which is almost
to it so that I can add to it and turn it into something
everything, and while I'm on my soapbox I'll take
better. I actually came up with the name while
the opportunity to mention that usability is
thinking about potential business names and going
subjective. So even though we make use of
independent. If I ever did I would like to think that I
standards and heuristics to help guide us, you need
would concentrate on usability, which has been a
to remember there is no one size fits all in usability,
focus area of mine for many year now.
which is part of the reason why it's such an
The original objective of the site was to capture all interesting subject.
my research and thoughts on usability and
7. Do you love martial arts? How often do you
psychology, and while I continue to study both I
practise?
www.TestingCircus.com November 2015 - 23 -
 DAVID GREENLEES

Love would probably be an understatement. I have It's been a great journey so far, and I'd like to think I
been utterly fascinated by martial arts for as long as can release it in early 2016, probably via LeanPub,
I can remember. I have been studying at least one but I'm constantly thinking of new content. I'll have
form of martial art since I was 18, but before that I to draw a line in the sand at some point and hit the
was already unstoppable from watching Bruce Lee publish button!
movies. 9. If you have to name only one book that every
How often I train varies depending on my personal tester should read which one that would be and
situation of course, but since moving to New York I why?
have spent some time at a wonderful place called Only one? OK, Lessons Learned in Software Testing
Clockwork Jiu-Jitsu. I was training 3 times a week by Kaner, Bach, and Pettichord.
until recently. Prior to that I was boxing and kick-
Oh, and I'm going to be a good tester and break the
boxing 3-4 times a week back in Adelaide. If I had
rules by mentioning a second, Perfect Software: And
my way I would train every day.
Other Illusions about Testing by Gerald Weinberg.
It's also something that has helped me to
There has been plenty written about both of these
understand testing. Many of the philosophies in
books but as mentioned above Lessons Learned was
martial arts can be directly related to lessons in
the turning point for me. Not just because of the
testing. So much so that I'm currently writing a book
lessons in-between the covers, but also the names
on the subject. For example, take this quote from
and what the research into those names has
Bruce Lee, “Notice that the stiffest tree is most easily
provided me since discovering them.
cracked, while the bamboo or willow survives by
When talking about books for software testing many
bending with the wind.” Now think about that in
people, including myself, will tell you that they
relation to standards in software testing. I find it
don't have to be on the subject of testing to help you.
profound.
However, I do feel it's important that you have a
8. Ok, let’s talk that book you are writing on
good understanding of what testing means to you
‘Software testing and martial arts’. Give us some
before you try and take lessons from other
hints.
industries and disciplines. I'm currently reading The
Well, the biggest hint would be the name… Invisible Gorilla and it's providing me with many
Software Testing as a Martial Art. In essence it's a great lessons that I can bring to my testing, however
collection of my thoughts and experiences on how if I had read it when I was just beginning in the
studying martial arts has helped me understand industry I feel that many of those lessons would
testing and become a better tester. have been lost. So while I encourage people to read
I begin each chapter with a martial arts quote that beyond the singular subject of testing, I caution
means something to me from both a martial arts and them to be comfortable with their knowledge in it
software testing perspective, and cover a wide range first, or perhaps be prepared to come back and re-
of subject matter from the art of questioning to read the book when they are.
mental models. I also have chapters from some 10. Five testing related websites you often visit.
other great testers who are, or have been, martial
Certainly, but in no particular order:
artists.

www.TestingCircus.com November 2015 - 24 -

 DAVID GREENLEES

• Ministry of Testing - due to location and the associate expense. If that's

https://www.ministryoftesting.com/ the case for you like it was for me, then start
• Michael Bolton's Blog - something of your own in your location. With sites
http://www.developsense.com/blog/ like Meetup.com it's really easy these days to get a
small group of people together who can quickly
• Thoughts from the Test Eye -
grow into a big group. As the originator of that
http://thetesteye.com/blog/
group your reputation will grow, and your network
• StickyMinds - http://www.stickyminds.com/
greatly extended.
• Satisfice - http://www.satisfice.com/
While you need to remember the first question I
Why oh why did you limit me to five? There are asked, and be sure to chase 'fame' for the right
many more but I think these are the ones I most reasons, let's be blunt… reputation is very
frequently visit (depending on how often the important. I haven't formally interviewed for a role
bloggers post). in the last 5 years. What I've done is 'show' people
11. No problem, go ahead, add five more. what I can do. Building a solid personal brand can
Calling my bluff, well played: equal success, but it's how you build the brand that
matters.
• Test Insane Apps - http://apps.testinsane.com/
13. Do you think testing conferences are useful?
• Testing Stack Exchange -
http://sqa.stackexchange.com/ That depends greatly on the user. I've approached
conferences in few different ways. In the early days
• Software Testing Club -
of my career I took the 'standard' approach whereby
http://www.softwaretestingclub.com/
I looked at the program before going to the
• Slack Testers Community - http://www.testers.io/
conference and chose the tracks I wanted to see.
• Rob Lambert's website - Then during those tracks I stayed, no matter how
http://thesocialtester.co.uk/ unimpressed I was. I didn't want to be 'rude' by
Do you play poker, Ajoy? leaving part way through.
12. No. I don’t. Next question. How can a newbie Later in my career I've taken more of a flexible
tester become a well-known tester like David approach, whereby I often decide 5 minutes before
Greenlees? the track session begins which one I'm going to go
Firstly, ask yourself why you want to be well to, or if I'm not going to one because I'd rather stay
known. If it's to be famous among software testers where I am and continue my current conversation. I
then just speak at as many conferences as you can. also do leave part way through if I feel I need to.
However be warned, if you don't 'bring it' you'll With the new wave of more context-driven
quickly become infamous. conferences this approach is actually encouraged,
and let's the attendee feel far more comfortable in
If you want to be well known because you're good
doing so.
at what you do then just be good at what you do,
but make sure you share it! Get a Twitter account, For me, the true power of conferences is conferring.
start a blog, and be active. Many people will tell you Meeting new people and sharing ideas. That can
to also go to all the great conferences out there and happen in a track session, but it can also happen at 3
to network, but that can be very difficult for people am while sitting in a bar sharing war stories, or

www.TestingCircus.com November 2015 - 25 -

 DAVID GREENLEES

during a party in your hotel room, perhaps even in with intensely while still remaining mates, what
room 403. Remember back to Go to question 2, and more could I ask for?
how I got to work for Doran Jones! Paul Holland, Michael Bolton (not the singer),
My message here is that you, the conference Jonathan Kohl; like I said, there are many more...
attendee, are in most control of whether or not the 15. Five things nobody knows about you –
conference is useful. Conferences are what you
I thought I was an
make them, so head to them with an idea of what
open book!
you want to achieve and do whatever you need to
1) I used to DJ, but not
do in order to make that happen.
with CDs, with vinyl
14. Name few people in testing, who influenced you
(the real deal). I still
directly or indirectly in your career as a software
love listening to drum
testing professional.
and bass, techno, and
There have been so many, but OK, I'll limit myself trance!
as much as possible, in no particular order:
2) I used to be able to
• James Bach – I see James as a Grandmaster of do the splits. It was
testing. Sparring with him will leave you bruised around the time I got my black belt in Taekwondo,
and battered, but you'll learn so much in the but I didn't maintain it.
process!
3) We had pet wallabies (like small kangaroos for
• Anne-Marie Charrett – A tireless testing leader those not in the know) when I was growing up.
who helped me bring Let's Test to Australia.
4) I was crew member of the year for McDonald's
Knowledgable, patient, and a firecracker when the
when I was 15, I received a medal, was written
situation calls for it.
about in the local newspaper, and received a free
• Mark Tolfts – The first context-driven tester I got Big Mac.
to work with and I really hope to again one day.
5) I once dressed up as Aladdin for my daughter's
• Henrik Andersson – My business mentor. What Princess Jasmine party when she turned 5 (pictures
Henrik has created, and continues to create is will not be supplied).
amazing, and I'm very glad I can call on him when
16. Usually our last question. Do you read Testing
the need arises.
Circus Magazine? If yes, what is your feedback to
• Keith Klain – He's my boss, I have to mention him improve this magazine?
right? Seriously though, I've only just commenced
I do read it if I like the look of a particular article/s.
working with him, but already I can see him
In more recent times I had been focusing my efforts
shaping my future.
on Testing Trapeze and helping review articles for
• My software testing brothers from another them so please don't be angry with me.
mother, Brian Osman, Richard Robinson, and Oliver
We are not. Thank David for your time.
Erlewein – This crew from New Zealand (although
Blog URL: http://www.dmg.name/
one is trying desperately to be an Aussie) is
awesome. Three great thinkers who I can debate Twitter URL: http://twitter.com/DMGreenlees

www.TestingCircus.com November 2015 - 26 -

 “Doing business without
advertising is like winking
at a girl in the dark. You
know what you are doing
but nobody else does.”

Advertise in our Monthly Editions and Website

and reach out to thousands of real testers worldwide.

http://www.testingcircus.com/advertise
 Hunting For
‘Hard-to-Reproduce’
Bugs
- Ravi Kumar BN
Every tester likes to find bugs, report and track them to their presence once or in rare circumstances (things that
closure. An effective bug report should help developer happen only intermittently) under test.
understand the issue, reproduce it and fix it. One of the These bugs occur when a user reports an issue but
most difficult tasks in software development is to find testers or developers aren't able to reproduce the bug.
non-reproducible bugs. These bugs are difficult to find, Handling these bugs has been a challenging one for both
because you can't fix a bug if you can't reproduce and developers and testers. During testing once the tester
find it. observes an issue, he/she will try the same steps again to
As a tester, you may reach a point in your testing where see the issue again, however it might so happen that the
you have found a bug or submit bug reports for defects issue can’t be seen when tested second time or after
that are 'not reproducible'. They may be reproducible on subsequent trials. These are the type of bugs which
your computer, but not on end user's system. Or the testers often find harder to isolate. However tester logs
user supplies steps to reproduce, but you can't see the the issue as non-reproducible and to get them fix
defect locally. Many variations on this scenario of becomes mundane task. In order for non-reproducible
course. Every tester faces such bugs during testing and issues to be fixed, it requires a lot of time. The developer
could easily become a nightmare. ends up spending lot of time to reproduce the issue and
There are often situations where you hear following to get the exact steps. When developer gives up it gets
statements: marked as 'To-Be-Reproduced' and assigned back to
“I observed the issue once and the exact steps are not known”, tester. Finding these bugs may take several days, and in
“Out of 10 times, 3 times the issue was seen”, “I observed this many cases the bug never actually gets fixed by either
issue only sometimes”, “I see too many defects declared non- getting rejected or closed (stating no plan to fix).
reproducible in bug report”
These kind of responses will never help developers to It’s tough for the developers to work with such bugs and
debug & fix the issue. If issue is there, definitely it’s testers get lot of objections over such bugs. If the count
there, the only problem is that we as testers need to find is more, then it might spoil the relationships between
when, where and how this bug occurred and provide as testers and developers. These bugs cause your software
much information as possible so that developers can to lose users' trust, which eventually leads to the user
find why it occurred and fix it. choosing another application.

What are ‘Hard-to-Reproduce’ Bugs? Why do they occur?

Non-determinism is the source of many bugs. Many of
'Non-Reproducible' or 'Hard-to-Reproduce' bug are the the times application behavior is dependent on user
bugs, which are not easy to reproduce every time even input, operating environment, network, etc. There are
by following seemingly same set of steps/procedures many different reasons under which bugs are harder to
under which they were found at first place, showing

www.TestingCircus.com November 2015 - 28 -

isolate and reproduce. Following are some scenarios § Memory– Some bugs also appear due to the memory
where few things affects in order for bug to appear. location accessed outside the specific location. Or if
the accessing the
specific location is
not possible under
that particular
instant due to
operating system
resource priorities.

§ System
Process– If the
particular process
in the operating
system is not free
and it makes the operating of particular software
§ Source Code – A non-reproducible bug occurs when
harder. In such case, if you try to reproduce the same
a test both passes and fails for the same input and
steps then it becomes harder to find the bug again.
source code for different executions. It's also
possible that a test always passes during § Platform – If the bug is occurring due to the
development but fails for the tester or end-user. A platform changes and the library conflicts, then such
common issue with these bugs is that your code has bugs are harder to isolate.
two executions for the same input: one good and one
bad. However, there must be some differences § Debugger – Some bugs also vanish outside the scope
between them or the result would be the same. In of the debugger. Once the debugger is turned off
some cases, part of your code should be executed such bugs don’t appear leaving us to assume the
but it isn't for some reason. This type of analysis is a conflict happens with debugger tool or libraries.
great starting point that might lead you to the defect.
Another possibility is when the executions follow How do you hunt for such bugs?
different traces. If we find the point from which the As a tester, quite often you will be asked to reproduce
execution traces are different, we have another good some "Hard-to-Reproduce" bugs, reported by other
starting point to find the defect. testers or users of the application. Here, your aim
should be to try and get more and more information
§ Build – Another possibility is the code fails with the
regarding the bug (who knows in the process we might
release or build version of your software, but it
reproduce it too!). Testers need to improve their skills in
passes in the IDE.
order to reproduce such issues. The following five
§ Installation – A bug may appear during the points if you adapt might help you while hunting for
installation of the software under particular such bug.
operating system with specific memory. Same bug § Always be on your toes; be alert! – You should be
doesn’t appear under another system with same observant and keen for details while trying to corner
operating system and memory. This is due to the such a bug. Keep your eyes and ears open for any
fact that such bugs are often affected by current possible suspicious looking process going on in the
running processes and the libraries installed and system. You will need to have more observation
under use in the memory. skills on the application behavior under test. Look
for those tiny little changes like flickering of the
§ Operating System – Another type of bug is specific
status bar, a missing button on the toolbar, a
to the environment changes (development libraries
partially loaded dialog box, an empty dropdown
and dependencies installed on the operating system)
list, slow to respond database queries and so on. If
in the operating system.

www.TestingCircus.com November 2015 - 29 -

 you happen to miss such a change in state, it might bugs that are reproducible but only on some systems are
prove vital for your bug hunting! easy to handle:
§ Trust but Verify; be suspicious! – Never trust § By using some sort of remote software, you let the
anything without verifying. Take the bug report user tell you what to do to reproduce the problem on
(submitted by other tester or whoever), just as a the system that has it. If this fails, then close it.
reference. Don’t follow it as a word from bible, or it § Try to reproduce the problem on another system. If
won’t be long before you would find yourself this fails, make an exact copy of the users system.
moving on a wrong track! Do it yourself and see
whether things are working as they are supposed to. § If it still fails, you have no option than to try to
debug it on the user system.
§ Look for Patterns! – Bugs love patterns. It’s the
tester who has to identify the patterns to identify the Once you can reproduce it, you can fix it. Doesn't matter
bug. While trying to reproduce the bug, look for the on what system.
possible patterns that are similar to a cousin of this
As a tester, you often come across a situation when you
bug. Who knows you might be able to find a pattern
can’t reproduce some errors again. But in order for you
which could help you to reproduce this bug!
to report it, developer to reproduce and fix it, often you
§ Note down the things you are doing while testing! – have to confirm (reproduce the elusive bug) visually
Make a note of the things you are doing while trying with screenshots that you really observed this defect
to reproduce the bug. Include things like the test and give precise instructions on how to reproduce the
environment, other applications that might be bug for developers or other team members.
running simultaneously, database configuration, the
test data you are using, other instances of It does not make sense to keep wondering about why
applications that might be using your AUT software is behaving differently at different times but to
(Application Under Test) etc. Try to capture and think several parameters involved while testing. There
narrate the story around the issue and not just steps are certainly some things that we need to take care while
to the stake holders and to corner the issue. Capture testing. Some good practices that we should consider
the complete test environment details, all the before arriving at the decision that a particular
available test execution results. This includes your bug/issue is not reproducible.
test data, screen shots, applications logs, system
logs, server logs. Noting down these details can help § Test Bed: It may so happen that the test bed being
to reproduce the bug. If you are able to get the bug used itself is wrong while testing the software, so it
but have not noted down the things you did to get does make sense to define the test bed first before
the bug, then it might be difficult to recollect the even start testing the software.
exact sequence of things to reproduce it again.
§ Exact Steps to Perform: People can't capture the
§ Have patience! – Patience is the key while trying to
exact steps that they performed while testing
reproduce such a bug. As a tester you have to show
because they may have some assumptions. Avoid all
lot of patience while hunting for such a bug. These
the assumptions while testing the software. Every
bugs are not easily reproducible and hence
parameter while testing the software matters such as
sometimes may make you impatient. Sometimes
test bed, the operating system, what was the CPU
resources & time constraints might be an issue (more
and memory usage while testing the software etc.
true for tech support), bargain for the same.
If you are not confident of finding out the exact steps
How do you get them fixed? to perform to reproduce the issue, it’s highly
If a certain bug/issue is non-reproducible on one system recommended to use the desktop recording
and the same is reproducible on another system, then software (such as WebEx Recorder). Start the
the possibility of going wrong is in test bed or the test desktop recording software before you start testing,
environment where the software is being tested. Such this helps a lot. Its normal tendency of human beings
that they tend to forget to capture certain steps that

www.TestingCircus.com November 2015 - 30 -

 they actually performed while testing. The § Test Data: Provide the exact test data that was used
recording software can be used to resolve these for test or any changes made to test data to
issues. Turn on recording software while testing all reproduce the bug.
the time. You always have the option of deleting the
recording if you do not need it. In case of non- § Test Results: Provide the complete & detailed test
reproducible bugs or issues, you always have the execution results. This data will even help to
option of going back and watching what are the evaluate the changes in the tests. Find ways to affect
steps you were performing when the bug was seen. timing of your program or of your devices, Slow
down, speed up.
§ Capture the logs: Logs are very important inputs to
developers to debug the issue and find out the § Test Steps: When you realize that you can’t
possible cause for it. Every software will have logs reproduce the bug, write down everything you can
enabled for debugging purposes, if not it is highly remember. Do it now, before you forget even more.
recommended that you enable the logs in your As you write, ask yourself whether you’re sure that
software. Sometimes the logs help developers to you did this step (or saw this thing) exactly as you
find the exact root cause of the bug with no time. are describing it. If not, say so. Draw these
distinctions right away. The longer you wait, the
§ Think out of the box: There was a situation wherein more you’ll forget. Hence add the test steps that gets
one issue was because the Windows Firewall was the execution closer to the bug.
turned off while testing the software but if the
firewall is turned the issue wasn't there. There could § Similar Bugs: Check the bug tracking system. Are
be lot of such parameters to consider that no one will there similar failures? May be you can find a pattern.
usually think. So all the initial computer settings This also helps developer to relate to the root cause
may also matter. So think out of the box. Test the of previous occurrences.
software in multiple computers before concluding
§ Record Sequence: Use screenshot recording tool like
the bug is non reproducible.
WebEx or Camtasia (or any other tool specific to that
§ Discuss sufficiently: Especially for non- operating system) to record the bug.
reproducible issues, we need to discuss a lot with
§ Capture Logs: Provide the environment logs and the
user who observed the issue. Developer has to talk
software debugger logs before and after noticing
to the testing team to get more details of the issue. It
such bugs. Checking the logging software in the
may so happen that the testers will not capture each
background helps to identify the system sources
and every details of the issues that they raise though
when the bug was found.
it’s not recommended. This might give some
insights as to what is going wrong. § Test Variants: If you can’t reproduce a bug, you first
document the steps and repeat them under different
How do you report them? environment to find it again. Capture all those
Always report non-reproducible errors. If you report environments where tests were repeated and the
them well, programmers can often figure out the corresponding results.
underlying problem. To help them, you must describe
§ More Details: Capture as much information as you
the failure as precisely as possible. Below are some best
can to write new test cases for the same test steps
practices to describe the failure.
with different environment, user error and data
corruption scenarios.
§ Error Message: If you can identify a display or a
message well enough, the programmer can often § Pre-Condition: Maybe the failure was a delayed
identify a specific point in the code that the failure reaction to something you did before starting this
had to pass through. Hence do keep a screenshot or test or series of tests. Before you forget, note the
documentation of the error message that appeared tasks you did before running this test.
during the bug occurrence.

www.TestingCircus.com November 2015 - 31 -

§ Program Code: The fact that a bug is not Conclusion
reproducible is data. The program is telling you that Every bug is reproducible and if something happened
you have a hole in your logic. You are not once it can happen again! It depends on how much time
entertaining certain relevant conditions. Talk to the and effort you are ready to invest on the investigation to
programmer and/or read the code. reproduce the bug, record and track them to assess the
quality & reliability of product under test.
How do you use them?
A non- reproducible bug is a tester’s error, just like a It is true that if the so called non reproducible issues are
design bug is a programmer’s error. It’s valuable to bound to take lot of time & resources in the process. It’s
develop a system for discovering your blind spots. To suggested to reproduce each & every issue. We must
improve over time, keep track of the bugs you’re explore more for the critical issues & re-produce them.
missing and what conditions you are not attending to In the process, we learn more too.
(or find too hard to manipulate). There are several
advantages of recording, analyzing and tracking the If these bugs are not much analyzed and fixed, it’s
trend of such onetime issues: certain that today or tomorrow the same bug/issue
comes from our customers. It’s also equally important to
§ Trending during testing phase helps assess the
understand the software deployment scenarios. In-fact
quality of product under test
this should go as part of the software requirements. It’s
§ Trending at the end of testing phase helps assess very much essential to understand in what conditions
release readiness the customers are going to use the product. Testers must
§ Trending post release helps relate them to always think from this perspective and prepare their
field/customer complaints test bed according to the end user scenarios.

About the Author

Ravi Kumar BN is a Product Verification Manager for Imaging Clinical Applications & Solutions,
HealthCare IT, Philips Innovation Center, Bangalore. He is a master’s graduate from IIT Kanpur, UP,
India, and BE (CSE) from SDM College of Engg & Technology, Dharwad, Karnataka, India. He has 13
years of experience in software quality processes and testing at Honeywell. He is a six sigma, lean and
agile testing expert and a six sigma techniques and tools trainer. He is actively involved in building and
deploying testing strategies for various platforms such as ERP (Peoplesoft), CRM (Siebel, SFDC), BI
(Cognos, OBIEE), emerging technologies such as Mobility, Cloud, Analytics, Voice, Responsive Web
Design and Wearables. He has attended design thinking workshop and has expertise in deploying
design tools in problem solving and usability testing. He has authored and published few testing articles
in QAI conferences and online magazines such as Testing Experience, Testing Circus etc.

www.TestingCircus.com November 2015 - 32 -

 My Thoughts on
EuroSTAR 2015
EuroSTAR conference that was held from
November 2nd to 5th in Maastricht, The Netherlands

- Patrick Prill
The EuroSTAR conference edition 2015 was held in the But I predict the future will prove them wrong, since
MECC in Maastricht, The Netherlands from November several people were not successful yet to tell them.
2nd to 5th. This year’s conference topic, set by I want to explicitly mention a few talks that stood out for
conference chair Ruud Teunissen, originally was “Walk me:
the Talk”. The unofficial topic felt more like “The Future
of Testing” and the key message throughout many talks Paul Coyne was talking about the scientific method.
was “The Human Factor of Testing”. And this is something that really all testers ought to
know about. Paul showed us where the parallels
It all began with an intense keynote about “Trendz between science and testing are, and that the scientific
2030” by future watcher Richard van Hooijdonk. In just method is actually the thing behind testing. There is lots
45 minutes he tried to show us what will happen in the we need to invent for the future of testing, but the
next 15 years and what is already possible. He tried to foundation is already there for quite some time, we just
present to a room full of professional sceptics, that most have to acknowledge that fact and study it.
of the changes of the near future will be positive. All
people I talked to afterwards were at first benumbed by Julie Gardiner provided a 5-step program to remain
the flood of information, but more and more the relevant as a tester in the future. And that is something
feedback came through that the future will hold lots to that can’t be told too often. You have to take your career
do for testers, and that testing has to change (a lot) to and learning in your own hands. Don’t wait for your
adapt for this future. company to help you with that. Show why your testing
is adding value to the development process and remain
I went to several experience reports on various topics. relevant.
The key message was always, no matter what you try to
accomplish, you have to work as a team, establish good The most intense talk was the “Lightning strikes the
communication, do the right things (automate, test, speakers” session. Seven people with 5 minutes each,
develop, etc.), and you have to be disciplined about talking about the future of testing.
what you do, to benefit the most from current trends in Iris Pinkster’s message was “Think up new processes
software development. It was good to hear this key and realize you are a team!” Important for teams that
message coming from several speakers with different have to transition from a world of silos to a more Agile
backgrounds in different situations, and it shows approach.
exactly: context matters, and there are tools and Jeffery Payne described the key elements for testing the
approaches to deal with the situation to adapt to your Internet of Things: 1. Fault Tolerance, 2. Robustness in
context and create the product you want or need. There Error-handling, 3. Privacy. Which he also named the age
is no one solution for everything, but there a good ways of non-functional testing!
to find a good solution for your problem.
Derk-Jan de Grood asked if testing is at the right level of
Only one talk/discussion left the feeling, that not responsibility.
everyone is ready for the future and is still looking back
and trying to reestablish structures from 25 years ago. Michael Bolton challenged us to exchange “verify that”
with “challenge the idea that”, and “validate” with
www.TestingCircus.com November 2015 - 34 -
“investigate” or “look for problems”. Change your I also had the pleasure to join both evening events which
language and improve your communication. were both in caves. A fantastic location in the caves of
Rikard Edgren said that testers can bring new Château Neercanne on Tuesday, and the restaurant La
perspectives and ideas to the development lifecycle, and Caverne on Wednesday to celebrate the Awards Dinner.
asked the audience: “what are the new perspectives you The award of “EuroSTAR Tester of the Year” was
will find?” presented to James Lyndsay, “Tutorial of the Year”
went to Rob Lambert, and “Paper of the Year” went to
Rob Lambert’s hope for the future is that there will be James Thomas.
more testers to hire out there. And he gave us “10 things
to improve”. At both events the food was good, the wine was plenty
and the talks were passionate.
Kristoffer Nordström predicted that in 15 years most
testers won’t work in testing any more. For low pay you To come to an end, Ruud Teunissen and his team
get low skills, but as a tester in an Agile environment managed to get a balanced selection of tracks appealing
you need lots of skills. Kristoffer hopes for a future to the wide audience of the conference, with some really
where companies hire sapient testers and invest in the good highlights.
future of their people. And the team of EuroSTAR did a fantastic job in
And most of all, a conference is for conferring. And it organizing and running the conference. Everything
was a joy for conferring. The expo was built around the went smooth and participants could really enjoy the
TestHuddle and the TestLab, two fantastic places to event! Well done!
mingle and meet lots of awesome people. The vendors If you want to know more about the conference, you can
in the Expo held lots of fun things to try and presenting find several posts on my blog at
their newest tools, and of course giving freebies. http://testpappy.wordpress.com. And I can highly
The TestHuddle held regular soap box talks and the Test recommend the posts from Colin Cherry at
Lab offered lots of challenges and puzzles and several http://itesting.com.au, which you can also find at the
practice sessions. TestHuddle blog: https://testhuddle.com/blog/.

About the Author

Patrick Prill has over 12 years of experience in software testing. After 4,5 years as a tester he became test
manager, coordinating the work of ~50 people for another 5 years in a big national test project.
Since 3 years he is test lead for a software and consulting company for the automotive industry. The job
brought him back to a smaller test team and the hands-on experience of testing software again. This
experience and following the testing community re-lighted his fire for testing and bug hunting.
Patrick will start as speaker at two testing conferences in 2016. Patrick is living outside of Munich,
Germany and is a proud husband and father of a wonderful daughter. In the little spare time he loves
turning wooden bowls and pens.

www.TestingCircus.com November 2015 - 35 -

 .com [email protected]

Govind PK
Expert Exploratory Tester
India

236 1653
Following Followers

83 bugs reported
Level 7 (870 points)

30 more points needed to became a validater Receivable amount in this month

4 test cycles won 19,500/-

What is 99tests? Meet our community

99tests is a crowdsourced testing platform consisting over

12,000 testers who have logged around 75,000 bugs. Testers are
exposed to various types of testing and domains. 99tests is a
platform where you can earn, learn and grow by login bugs,
connecting with testers across the globe. India USA Russia

Why should I sign up with 99tests?

UK Philadelphia Austin

1 Work whenever you wish to and from any place you want to

2 Choose the products/ software you want to test

Collaborate with the testers across the globe, share your Join our community today!
3
skills and grow with them 99tests.com

“ Being in the field of software testing for a while, I always wondered whats my stand in the testing skills I
possess compared to the other testers. This was more of a positive thought as I was more keen on identifying
what I lacked and what I needed to improve.”

AWA R D E D B Y
Connect with our 4000+ fans
in our page.

http://www.facebook.com/TestingCircus
Become our fan -
https://twitter.com/_sahi
http://www.facebook.com/sahi.software

Request a free demo by sending us an email at [email protected]

http://www.sahi.co.in
 #Testers2Follow
Llewellyn Falco
Agile Coach, Creator of ApprovalTests, Co-Founder of
TeachingKids Programming. Legacy Code Expert

https://twitter.com/LlewellynFalco

Dwayne Green
Software Tester.

https://twitter.com/N00bTester

Women In Testing
Recognizing the great women in #softwaretesting.

https://twitter.com/WomenInTesting

Words From A Purple Mind

Quality Assurance, Business Analysis Web Design, SEO, and much
more.

https://twitter.com/WFAPM

https://Twitter.com/TestingCircus
www.TestingCircus.com NOvember 2015 - 39 -
 s
te ster
e
wa r
so f t
fo r
a z ine ting

cu s a g
ag
e m 010. #
tes

Cir
ngu er 2
li s h la ptemb

i n g ing
Eng nce Se
si i r cus

Test
a d s
ngC
e n
’s l itio
orld thly ed s t i
/Te
a w
u s is s. Mon m
c t o
er.c
Cir
sias
e s ting enthu i t t
://tw
T
test
and t ps
s at
ht
u
low
F ol

ter
wit
n T
s O
w U
o llo
#F
rc us
ngCi
Testi
m/
ter.co
wit
://T
ps
htt

200+ testers to follow on Twitter -

https://www.testingcircus.com/testers-in-twitter

www.TestingCircus.com NOvember 2015 - 40 -

 s
w
vie
Re
o k
Bo

BOOK
WORM’S
CORNER
Towards the end of the year is when we retrospect about our performance for that year; that’s when we think about
what we should have done, what we did not do, what we set out to do and if we met our own expectations for that
year. That’s the same for performance testing teams. They are called almost at the last minute to help and identify
problems; the bugs can be very costly to fix if found at a late stage. This book helped me to escape, when I was called
on by teams at such late stages. I hope it will help you too….
This month’s recommendations is
The Art of Application Performance Testing
Performance teams are usually engaged at a late time in the project; and they are expected to deliver a scalable sys-
tem on unstable systems. This book has helped many teams to cope and scale when they helped me when I was
called in to help out a project. Most practical things about performance testing are to understand the lifecycle of sys-
tems built for performance testing; this book would help you with that. It also helps you the various metrics that you
define for building performance testing systems so that you will understand when the systems get mature. A bigger
take away is a bunch of tools that is recommended by the author. The experience of the author is visible where they
talk about the most common issues that people would overlook if they are involved in performance testing for the 1st
time. The only time when this book does not work is when the person testing is new to performance testing and does
not realize the pitfalls, or wants to fall down and learn; if you want to get it right from the 1st time, this book is a
must-buy (and a best-buy).
Buy it here

- Wobo

www.TestingCircus.com November 2015 - 41 -

 Security Testing Tips
Part 33

Tips for security threat detection and prevention

- Santhosh Tuppad
If you are a security tester and love to find security we find such an input, we try to prevent further
vulnerabilities, cheers to you. Nevertheless, as security attacks. Not only input vectors, but also URLs can be
testers we may want to do more in terms of real-time included in detection algorithm / code. One more
where we detect the security attacks by black-hat example I would like to quote is: Tracking those
hackers or script kiddies and prevent such attacks so people who try to access /admin/ to land on admin
that our customers or users don’t have downtime or login form. Having said that, firstly it’s not
outage when the security attacks are lethal and appropriate to have admin form open to web and that
merciless. Personally, I would love to help my too such a guessable name which is /admin/ after
customers not only with security vulnerabilities hacker enters the website URL. I find many
through my exploratory testing skills, but also help applications on web which always have
them to detect any such in future when they make http://example.com/admin/ which is like BOOM!
code changes while I also help them to prevent such
attacks. Nevertheless, it’s always good to have a How do we prevent it after detection?
security tester going through the code Now, once the detection is done you
changes and then assessing it. What I need to think about how you need to
am trying to say here is, we need to deal with it. It could in terms of
have combo of both detection & blocking the IP address for a while,
prevention system while we do account-lockout policy, or notifying
frequent security assessments of the the administrator / web developer or
application or software. anyone who matter via email or SMS
or call so that the suspect attack could
How do we detect the security be blocked which may lead to
threats? downtime of application or threat to
I will be taking a simple example to users privacy (Brute-force attack
explain it to you. trying to gain unauthorized access) if
Let’s say, I have a web app where a not blocked.
hacker will try doing code injection attack and it could There are several ways to do it and the question is, do
be something like <script> or <h1> or any HTML tag. you want to start developing something like this from
In this case, I will have a javascript or AJAX detection scratch or you want to use something that is already
script which will alert the server about this activity. available as open-source or commercial. The decision
Even though we have server side truncation of this depends on various attributes here. If you are willing
kind of scripts, nevertheless it’s always good to know to go ahead with open-source framework, may be you
that someone on this planet is trying to attack our should look into AppSensor at OWASP. Thanks to all
application. the people who worked on this to give it to the world
We can always maintain a list of inputs in our database of technology. Have a good time in stopping malicious
categorized under “Possible Attacks” and whenever attackers and let’s move towards clean web.

www.TestingCircus.com November 2015 - 42 -

 Validata Launches New Test Data
Generator Solution

Validata Group, the leader in Enterprise Software right place at the right time, reducing project delay by
Testing solutions, is pleased to announce that it has efficient provisioning of quality, realistic data with all
launched, Test Data Generator, a complete end-to-end the characteristics of production, but with no sensitive
test data management solution, which offers content. It also enables to identify any gaps in the test
organisations the market leading functionality and data on demand and powerful synthetic data
flexibility to find, design and make ‘fit for purpose’ data generation to provide testers with data before testing
for use in non-production environments. starts.
It enables the creation of test data and expected results The tool’s powerful data generation engine supports
automatically avoiding time- consuming and error hierarchical overview of equivalence partitioning, and
prone validation of test results on a field by field basis. boundary values analysis, as well as advanced spot
Test data created with Test Data Generator is reusable, diagrams for effective environments comparison and
adapts to change quickly and allows faster innovation data coverage measurements for all valid combinations.
and faster release with better quality on time and Clients will benefit from reduced number of time and
within budget. resources required to provide ‘fit for purpose’ data and
The solution enables successful implementation of improved test coverage and teams’ productivity, while
continuous delivery as it delivers the right data in the allowing IT to ‘shift left’ testing and cut costs.

www.TestingCircus.com November 2015 - 43 -

 SmartBear Redefines How Functional
And Performance Testing Is Done
With Script Reuse
SmartBear Software, the leader in software quality tools effectively captures this trend by stating, “71 percent
for the connected world, announced a new version of of the most successful advanced companies have made
TestComplete that redefines how functional and it a high priority to increase software development
performance testing is done with script reuse. One of speed while maintaining quality; they regard doing this
the world’s most recognized automated testing tools, effectively as important for competitiveness compared
TestComplete is now integrated with LoadComplete, with usability, flexibility, global integration, reliability
SmartBear’s popular load testing solution for websites, and business insights.”
mobile and Web apps. Customers deploying the new “Testing cycles continue to get shorter from months to
version are able to repurpose TestComplete functional weeks and now even a few hours,” said Nikhil Kaul,
test scripts for performance testing in LoadComplete. Product Marketing Manager, Testing Products at
Organizations are under constant pressure to improve SmartBear. “Achieving higher levels of test automation
time-to-market schedules to meet accelerated delivery holds the key to delivering applications at a greater
timeframes. Delivering on these quicker release speed and higher quality. With an ability to convert
deadlines often comes with maintaining high quality functional tests into performance tests, we are ensuring
functional and performance delivery criteria. Meeting QA teams have the right tools to succeed in Agile and
such an improved time-to-delivery cannot come at the DevOps environments. The functionality ensures that
expense of quality, essentially meaning that QA performance and functional testing are not treated as
managers are under constant pressure to cut testing a stand-alone activity and same set of testing skills as
time while expanding functional and performance test well as assets can be used across disciplines.”
coverage. TestComplete’s integration with LoadComplete makes
Expanding coverage while reducing testing time can it easy for QA teams to convert functional tests into
be difficult to achieve for QA managers, especially performance tests. QA teams can increase their level of
when testers have to reinvent the wheel and create automation testing and improve time-to-market,
scripts from scratch every time a feature needs to be thereby reducing test development costs and time. The
tested for performance and functionality. Rewriting facilitated reuse of functional tests as performance tests
different scripts for performance and functional testing drives efficiency gains as the script logic needs to be
results in redundant efforts, which in turn erodes written only once, which cuts down testing time and
efficiency gains achieved during previous stages of the expands coverage.
software development lifecycle. Add to that, additional The tests created can then be run as a part of continuous
challenges arise from the fact that testers with a build process to minimize handoffs and get rapid
development background are hard to find. feedback. Most importantly, organizations can leverage
SmartBear believes that a January 2015 Forrester their existing QA resources for both functional and
Research report entitled, “Five Must-Do’s For Testing performance testing, thereby not leaving any test
Quality At Speed,” (subscription required for access) creation and execution in developers’ hands.

www.TestingCircus.com November 2015 - 44 -

 Synack Launches Innovative
Vulnerability Intelligence Platform
Hydra
Synack, the security company that harnesses the power deeper across large enterprise assets without
of a worldwide community of highly qualified and jeopardizing quality. This optimal pairing of man and
trusted researchers, announced the launch of the machine is a unique approach to combating the real
Synack Hydra Technology Platform. Hydra and ongoing threat of compromise that the enterprise
Technology is an advanced vulnerability intelligence faces on a daily basis — strategically pitting a solution
platform that together with the Synack Red Team that leverages advanced technology to scale researcher
(SRT), gives the enterprise an unparalleled adversarial intelligence against the threat of skilled blackhat
perspective of their digital assets. hackers.
Hydra combines the power of a modern vulnerability Today, Synack helps secure leading Fortune 500
scanner with the expertise and creativity found in financial, healthcare, consumer goods companies,
individual hacker toolkits to provide actionable among others, and Hydra will enhance Synack’s core
intelligence to the Synack Red Team (SRT) so that they solution in a way that can scale the unmatched
can locate, confirm and report exploitable bugs with expertise of the SRT across a growing customer base.
unprecedented efficiency and scale. Acting as a closely “Today’s vulnerability solutions are flawed,” said Mark
integrated extension of internal security teams, the Kuhr, CTO and co-founder of Synack. “Some are
Hydra-enabled SRT delivers exploitation intelligence human-centric, point-in-time penetration tests, which
that reduces windows of exposure and provides are limited to the skillsets of individual testers and
comprehensive testing coverage across large, complex project timelines. Others are solely reliant on scanner
enterprise assets. technologies, which overwhelm today’s already-
Strong security posture for a fast moving enterprise is strained IT organizations with duplicates, false
no longer an option, it’s an expectation. Security is top positives, uneven quality levels and thousands of
of mind in the court of public opinion — key business submissions that require manual review. Hydra
partners, customers, regulators, and rating agencies — Technology delivers scalable, continuous testing to the
all want to know that the enterprise they do business enterprise without the noise, so that enterprises can
with is proactively securing itself against cyber threats. secure their perimeter and ease the strain on security
Using Hydra Technology, the SRT locates and helps to organizations.”
eliminate the vulnerabilities that give way to cyber- The Hydra platform can be separated into three subsets
attacks that could ultimately threaten the way we live, of functionality — host monitoring, web application
play, travel, do business and interact with one another. analysis and mobile application analysis — all of which
Synack’s proactive testing solution seamlessly will be released in phases. Host monitoring capabilities
combines the collective human intelligence of the SRT will be available to Synack customers on October 21,
with Hydra’s advanced vulnerability intelligence with web and mobile testing capabilities to be released
technology, consistently delivering prioritized and in the first half of 2016. The technology is available as
contextualized exploitation intelligence that has led to part of the Synack subscription model. As Hydra
record-breaking find-to-fix cycles for many of their technology is a SaaS offering, there is no physical or
enterprise clients. virtual appliance to install, no software to deploy and
Hydra’s continuous monitoring capabilities are no physical infrastructure to acquire and maintain.
designed to streamline the SRT’s reconnaissance phase
of the testing process, allowing them to test faster and

www.TestingCircus.com November 2015 - 45 -

 Still relying on
reading
Testing Circus
from tweets
& facebook
updates?
Subscribe
with your
u b s c r i b e email id and
To S
C l i c k H e r e get the
magazine
delivered to
your email
every month,
Testing Circus free!
www.testingcircus.com