Bug Reporting: Applabs Center of Excellence
Bug Reporting: Applabs Center of Excellence
BUG REPORTING
TABLE OF CONTENTS
1.0 General:........................................................................3
1.1 Module Objectives:......................................................................................................3
1.2 Module Structure:.........................................................................................................3
2.0 What Is a Bug Report and Why Do We Write Them?
.................................................................................... 4
Bug reporting is one of the key phases of testing process. The good bug report actually stands as an
window for good testing process, hence writing good bug reports that are easily understandable is
indeed an art by itself.
Writing good bug reports needs not just good writing skills but also full understanding of the product
being tested and the testing process followed.
Though people can follow their own way of reporting the bugs, there are certain general standards
that need to be followed while reporting a bug.
The following module introduces you to the art of writing good bug reports, the standard practices the
needs to followed and structure that needs to be followed.
Intermittence
Inconsistent test/development environments
Disputes over “correct” behavior
But many irreproducible bug reports are poorly conceived and poorly written
Applying the following ten tips will help you achieve better bug reports:
3.2 Reproduce:
The tester should check reproducibility of a failure before writing a bug report. If the problem doesn’t
recur, he/she should still write the bug report, but she must note the sporadic nature of the behavior. A
good rule of thumb is three attempts to recreate the failure before writing the report. Documenting a
clean set of steps to reproduce the problem addresses the issue of reproducibility head-on.
Good
Bad
1.I started the SpeedyWriter editor, then I Nasty bug trashed contents of new file that I
created a new file. created by formatting some text in Arial font,
wasting my time.
2. I then typed in four lines of text, repeating
“The quick fox jumps over the lazy brown dog”
each time, using different effects each time, bold,
italic, strikethrough, and underline.
3. i highlighted the text, then pulled down the font
menu, and selected Arial.
4.This nasty bug trashed all the text into
meaningless garbage, wasting the user’s time.
5.I was able to reproduce this problem three out
of three tries.
Better Good
Steps to Reproduce Steps to Reproduce
1.I started the SpeedyWriter editor, then I created 1.I started the SpeedyWriter editor, then I created
a new file. a new file.
2.I then typed in four lines of text, repeating “The 2.I then typed in four lines of text, repeating “The
quick fox jumps over the lazy brown dog” each quick fox jumps over the lazy brown dog” each
time. time, using different effects each time, bold, italic,
strikethrough, and underline.
3.I highlighted the text, then pulled down the font
menu, and selected Arial. 3.I highlighted the text, then pulled down the font
menu, and selected Arial.
4.This nasty bug trashed all the text into
meaningless garbage, wasting the user’s time. 4.This nasty bug trashed all the text into
meaningless garbage, wasting the user’s time.
5.I was able to reproduce this problem three out
of three tries. 5.I was able to reproduce this problem three out
of three tries.
Isolation
Isolation
Also happens with Wingdings and Symbol fonts.
On the vague suspicion that this was just a On the vague suspicion that this was just a
formatting problem, I saved the file, closed formatting problem, I saved the file, closed
SpeedyWriter and reopened the file. The garbage SpeedyWriter and reopened the file. The garbage
remained. If you save the file before changing the remained. If you save the file before Arializing the
font of the contents, the bug does not occur. The contents, the bug does not occur. The bug does
bug does not occur with existing files. This only not occur with existing files. This only happens
happens under Windows 98. under Windows
Better Good
Steps to Reproduce Steps to Reproduce
1.I started the SpeedyWriter editor, then I created 1.I started the SpeedyWriter editor, then I created
a new file. a new file.
2.I then typed in four lines of text, repeating “The 2.I then typed in four lines of text, repeating “The
quick fox jumps over the lazy brown dog” each quick fox jumps over the lazy brown dog” each
time. time.
3.I highlighted the text, then pulled down the font 3.I highlighted the text, then pulled down the font
menu, and selected Arial. menu, and selected Arial.
4.This nasty bug trashed all the text into 4.This nasty bug trashed all the text into
meaningless garbage, wasting the user’s time. meaningless garbage, wasting the user’s time.
5.I was able to reproduce this problem three out 5.I was able to reproduce this problem three out
of three tries. of three tries.
Isolation Isolation
New to build 1.1.018; same test case passed Also happens with Wingdings and Symbol fonts.
against builds 1.1.007 (System Test entry) On the vague suspicion that this was just a
through 1.1.017. Also happens with Wingdings formatting problem, I saved the file, closed
and Symbol fonts. On the vague suspicion that SpeedyWriter and reopened the file. The garbage
this was just a formatting problem, I saved the remained. If you save the file before changing the
file, closed SpeedyWriter and reopened the file. font of the contents, the bug does not occur. The
The garbage remained. If you save the file before bug does not occur with existing files. This only
changing the font of the contents, the bug does happens under Windows 98
not occur. The bug does not occur with existing
files. This only happens under Windows 98.
Better Good
Summary Summary
Arial, Wingdings, and Symbol fonts corrupt new Arial, Wingdings, and Symbol fonts corrupt new
files. files.
Steps to Reproduce Steps to Reproduce
1.Started SpeedyWriter editor, then created new 1. Started SpeedyWriter editor, then created new
file. file.
2.Typed four lines of text, repeating “The quick 2.Typed four lines of text, repeating “The quick
fox jumps over the lazy brown dog” each time. fox jumps over the lazy brown dog” each time.
3.Highlighted all four lines of text, then pulled 3.Highlighted text, then pulled down the font
down the font menu, and selected Arial. menu, and selected Arial.
4.This nasty bug trashed all text into meaningless 4.This nasty bug trashed all text into meaningless
garbage, including control characters, numbers, garbage, wasting the user’s time.
and other binary junk, wasting the user’s time.
5.Reproduced three out of three tries.
5.Reproduced three out of three tries.
Isolation
Isolation
New to build 1.1.018; same test case passed
New to build 1.1.018; same test case passed
against builds 1.1.007 (System Test entry)
against builds 1.1.007 (System Test entry)
through 1.1.017. Also happens with Wingdings
through 1.1.017. Reproduced with same steps
and Symbol fonts. On vague suspicion this was a
using Wingdings and Symbol fonts. On vague
formatting problem, saved file, closed
suspicion this was a formatting problem, saved
SpeedyWriter and reopened file. Garbage
file, closed SpeedyWriter and reopened file.
remained. Saving file before changing font
Garbage remained. Saving file before changing
prevents bug. Bug does not occur with existing
font prevents bug. Bug does not occur with
files. Only happens under Windows 98.
existing files. Only happens under Windows 98,
not Solaris, Mac, or other Windows flavors
The test team needs to focus on the task of writing bug reports, and the test leads and
manager must make it clear to each member of the test team that writing good bug reports is a
primary job responsibility. Quality indicators for a well-tuned bug reporting process include:
Brevity of the bug lifecycle from opened to closed, reducing cycles where developers return
poor quality reports for more information, leading to tester rework. Improving the bug reporting
process does require an effort, but provides significant payoffs. First, a crisp process
improves the test team’s communications with senior and peer management, which enhances
the team’s credibility and professional standing, and can encourage management to invest
more resources in testing.
Second, the smooth handoff to developers promotes positive relationships. Third, shorter bug
lifecycles are more efficient, so the time invested up front writing a good bug report is repaid
in time not wasted rewriting a poor bug report. These payoffs help the development process
achieve better product quality through effective communication and efficient workflows.
The first aim of a bug report is to let the programmer see the failure with their own eyes. If you
can't be with them to make it fail in front of them, give them detailed instructions so that they
can make it fail for themselves.
In case the first aim doesn't succeed, and the programmer can't see it failing themselves, the
second aim of a bug report is to describe what went wrong. Describe everything in detail.
State what you saw, and also state what you expected to see. Write down the error
messages, especially if they have numbers in.
When your computer does something unexpected, freeze. Do nothing until you're calm, and
don't do anything that you think might be dangerous.
By all means try to diagnose the fault yourself if you think you can, but if you do, you should
still report the symptoms as well.
Be ready to provide extra information if the programmer needs it. If they didn't need it, they
wouldn't be asking for it. They aren't being deliberately awkward. Have version numbers at
your fingertips, because they will probably be needed.
Write clearly. Say what you mean, and make sure it can't be misinterpreted.
Be specific. If you can do the same thing two different ways, state which one you
used. "I selected Load" might mean "I clicked on Load" or "I pressed Alt-L". Say which
you did. Sometimes it matters.
Be verbose. Give more information rather than less. If you say too much, the
programmer can ignore some of it. If you say too little, they have to come back and
ask more questions. One bug report I received was a single sentence; every time I
asked for more information, the reporter would reply with another single sentence. It
took me several weeks to get a useful amount of information, because it turned up
one short sentence at a time.
Be careful of pronouns. Don't use words like "it", or references like "the window",
when it's unclear what they mean. Consider this: "I started FooApp. It put up a
warning window. I tried to close it and it crashed." It isn't clear what the user tried to
close. Did they try to close the warning window, or the whole of FooApp? It makes a
difference. Instead, you could say "I started FooApp, which put up a warning window.
I tried to close the warning window, and FooApp crashed." This is longer and more
repetitive, but also clearer and less easy to misunderstand.
Read what you wrote. Read the report back to yourself, and see if you think it's
clear. If you have listed a sequence of actions which should produce the failure, try
following them yourself, to see if you missed a step.
Particularly, every good bug report should have some of the following:
1. The URL of the script you received the error on, the script you were coming from, the script
you ended up at
4. Precise date and time the error occurred (use the date command on the machine that you're
working on to get this)
6. The machine that the error was generated on (may not always be the same as above)
8. What program you tried to run that gave you the error
9. What you typed to give the error (just copy this directly from your xterm, Netscape, ssh
terminal, etc.)
11. Does this error occur on other machines, or just the one that you happened to be working on?
Though, the format of the bug report varies from project to project or Client to Client or the bug
reporting tools that are used, the following are general contents of a typical bug report.
Reporter: Name of the test engineer who reported the bug. If a bug reporting tool is being used, this
can be appended automatically through the login details of the test engineer
Module: Name of the Module where the bug has been found
Severity: How damaging is the problem to progress and productivity? The following are the
severity levels that are used in general.
Blocker
Critical
Major
Minor
Normal
Trivial
The severity levels are decided, after thorough understanding of the product.
Priority:
The priority factor specifies the urgency of fixing the bug. It is not necessary that all high
severe bugs are high priority bugs and vice versa, though most of the times they are so.
The following are the Priority levels that are used in general.
High
Medium
Low
Example:
Consider a situation, where the wrong logo is displayed in the home page of the company. Such a
bug is High priority (i.e. it needs to be addressed immediately) but is of normal severity.
Typically Blocker, Critical, Major bugs are posted as High priority bugs. Normal bugs are
posted as Medium priority and Minor and Trivial are posted as Low priority bugs.
Assigned To: The Name of the person who needs to attend this bug, it could be the
developer or the project manager, who may inturn redirect the bug to the concerned
personnel.
Cc: Who else should receive e-mail updates on changes to this problem?
Summary:
Steps to Reproduce: The minimal set of steps necessary to trigger the problem. Include any special
setup steps.
Actual Results: What actually happened after performing the above steps.
Include examples and hard numbers where possible.
Expected Results: What you think should have happened. This is obviously
relevant where you are requesting an enhancement, but please include it in
any case. Maybe you have misunderstood or we have failed to explain how
things are supposed to work.
Build Date & Platform: If this is a software fault, give the date and platform
of the build in which you first encountered the bug. Does the problem occur
on any other computers or software builds to which you have access?
Additional Information: This typically contains, any other details that are not
covered in the above listed sections.
Example of bug: The following is an example of a bug, reported through a bug reporting tool
Severity
Priority
6.0 Conclusion
7.1 Exercise
Answer the Following in Short:
1. What is the purpose of a bug report?
2. What are the problems with BAD bug reports?
3. List some good practices in writing good bug reports?
4. What are the general contents of a bug report?
5. Create a format of a bug report (you can use your own creative ideas, that can make a bug
report better)?