1

SOFTWARE TESTING

1.What is 'Software Quality Assurance'?

Software QA involves the entire software development PROCESS - monitoring and

improving the process, making sure that any agreed-upon standards and procedures are
followed, and ensuring that problems are found and dealt with. It is oriented to
'prevention'.

2.What is 'Software Testing'?

Testing involves operation of a system or application under controlled conditions and

evaluating the results .The controlled conditions should include both normal and
abnormal conditions. Testing should intentionally attempt to make things go wrong to
determine if things happen when they shouldn't or things don't happen when they should.
It is oriented to 'detection'.

4. Does every software project need testers?

While all projects will benefit from testing, some projects may not require independent
test staff to succeed.

In some cases an IT organization may be too small or new to have a testing staff even if
the situation calls for it. In these circumstances it may be appropriate to instead use
contractors or outsourcing, or adjust the project management and development approach.
Inexperienced managers sometimes gamble on the success of a project by skipping
thorough testing or having programmers do post-development functional testing of their
own work, a decidedly high risk gamble.

5. Why does software have bugs?

• miscommunication or no communication
• programming errors
• time pressures
• software complexity
• changing
• poorly documented code

6. How can new Software QA processes be introduced in an existing organization?

• A lot depends on the size of the organization and the risks involved. For large
organizations with high-risk (in terms of lives or property) projects, serious
management buy-in is required and a formalized QA process is necessary.
• Where the risk is lower, management and organizational buy-in and QA
implementation may be a slower, step-at-a-time process. QA processes should be
balanced with productivity so as to keep bureaucracy from getting out of hand.
 2
• For small groups or projects, a more ad-hoc process may be appropriate,
depending on the type of customers and projects. A lot will depend on team leads
or managers, feedback to developers, and ensuring adequate communications
among customers, managers, developers, and testers.
• Other possibilities include incremental self-managed team approaches such as
'Kaizen' methods of continuous process improvement, the Deming-Shewhart
Plan-Do-Check-Act cycle, and others.

7.What is Verification? Validation?

Verification typically involves reviews and meetings to evaluate documents, plans,

code, requirements, and specifications. This can be done with checklists, issues lists,
walkthroughs, and inspection meetings.

Validation typically involves actual testing and takes place after verifications are
completed. The term 'IV & V' refers to Independent Verification and Validation.

8. What is a 'walkthrough'?

A 'walkthrough' is an informal meeting for evaluation or informational purposes.

Little or no preparation is usually required.

9. What's an 'inspection'?

An inspection is more formalized than a 'walkthrough', typically with 3-8 people

including a moderator, reader, and a recorder to take notes. The subject of the
inspection is typically a document such as a requirements spec or a test plan, and the
purpose is to find problems and see what's missing, not to fix anything. Attendees
should prepare for this type of meeting by reading thru the document; most problems
will be found during this preparation.The result of the inspection meeting should be a
written report. Thorough preparation for inspections is difficult, painstaking work, but
is one of the most cost effective methods of ensuring quality.

10. What kinds of testing should be considered?

• Black box testing

• White box testing
• unit testing
• incremental integration testing
• functional testing
• system testing
• end-to-end testing
• sanity testing or smoke testing
• regression testing
• load testing
 3
• usability testing

11. What are 5 common problems in the software development process?

• poor requirements - if requirements are unclear, incomplete, too general, and not
testable, there will be problems.
• unrealistic schedule - if too much work is crammed in too little time, problems are
inevitable.
• inadequate testing - no one will know whether or not the program is any good
until the customer complains or systems crash.
• featuritis - requests to pile on new features after development is underway;
extremely common.
• miscommunication - if developers don't know what's needed or customer's have
erroneous expectations, problems are guaranteed.

12. What are 5 common solutions to software development problems?

• solid requirements - clear, complete, detailed, cohesive, attainable, testable

requirements that are agreed to by all players. Use prototypes to help nail down
requirements.
• realistic schedules - allow adequate time for planning, design, testing, bug fixing,
re-testing, changes, and documentation; personnel should be able to complete the
project without burning out.
• adequate testing - start testing early on, re-test after fixes or changes, plan for
adequate time for testing and bug-fixing
• stick to initial requirements as much as possible - be prepared to defend against
excessive changes and additions once development has begun, and be prepared to
explain consequences. If changes are necessary, they should be adequately
reflected in related schedule changes.
• communication - require walkthroughs and inspections when appropriate; make
extensive use of group communication tools - groupware, wiki's, bug-tracking
tools and change management tools, intranet capabilities, etc.; insure that
information/documentation is available and up-to-date - preferably electronic, not
paper; promote teamwork and cooperation; use protoypes and/or continuous
communication with end-users if possible to clarify expectations.

13. What is software 'quality'?

Quality software is reasonably bug-free, delivered on time and within budget, meets
requirements and/or expectations, and is maintainable. However, quality is obviously
a subjective term. It will depend on who the 'customer' is and their overall influence
in the scheme of things. A wide-angle view of the 'customers' of a software
 4
development project might include end-users, customer acceptance testers, customer
contract officers, customer management, accountants/testers/salespeople, future
software maintenance engineers, stockholders, magazine columnists, etc.

14. What is 'good code'?

'Good code' is code that works, is bug free, and is readable and maintainable. Some
organizations have coding 'standards' that all developers are supposed to adhere to,
but everyone has different ideas about what's best, or what is too many or too few
rules. For C and C++ coding, here are some typical ideas to consider in setting
rules/standards.

15. What is 'good design'?

'Design' could refer to many things, but often refers to 'functional design' or 'internal
design'. Good internal design is indicated by software code whose overall structure is
clear, understandable, easily modifiable, and maintainable; is robust with sufficient
error-handling and status logging capability; and works correctly when implemented.
Good functional design is indicated by an application whose functionality can be
traced back to customer and end-user requirements.

17.What is the 'software life cycle'?

The life cycle begins when an application is first conceived and ends when it is no
longer in use. It includes aspects such as initial concept, requirements analysis,
functional design, internal design, documentation planning, test planning, coding,
document preparation, integration, testing, maintenance, updates, retesting, phase-out,
and other aspects.

18. What makes a good QA or Test manager?

A good QA/Test manager should:

• be familiar with the software development process

• be able to maintain enthusiasm of their team and promote a positive atmosphere,
despite what is a somewhat 'negative' process .
• be able to promote teamwork to increase productivity
• be able to promote cooperation between software, test, and QA engineers
• have the diplomatic skills needed to promote improvements in QA processes
• have the ability to withstand pressures and say 'no' to other managers when
quality is insufficient or QA processes are not being adhered to
• have people judgement skills for hiring and keeping skilled personnel
• be able to communicate with technical and non-technical people, engineers,
managers, and customers.
• be able to run meetings and keep them focused
 5

19. What's the role of documentation in QA?

QA practices should be documented such that they are repeatable. Specifications,

designs, business rules, inspection reports, configurations, code changes, test plans,
test cases, bug reports, user manuals, etc. should all be documented in some form.
There should ideally be a system for easily finding and obtaining information and
determining what documentation will have a particular piece of information.

20. What's the big deal about 'requirements'?

One of the most reliable methods of ensuring problems, or failure, in a large, complex
software project is to have poorly documented requirements specifications.
Requirements are the details describing an application's externally-perceived
functionality and properties. Requirements should be clear, complete, reasonably
detailed, cohesive, attainable, and testable.

In some organizations requirements may end up in high level project plans,

functional specification documents, in design documents, or in other documents at
various levels of detail. Without such documentation, there will be no clear-cut way to
determine if a software application is performing correctly.

21. What steps are needed to develop and run software tests?

• Obtain requirements, functional design, and internal design specifications and

other necessary documents
• Obtain budget and schedule requirements
• Determine test approaches and methods - unit, integration, functional, system,
load, usability tests, etc.
• Determine test environment requirements
• Determine testware requirements
• Obtain and install software releases
• Perform tests
• Evaluate and report results
• Track problems/bugs and fixes
• Retest as needed
• Maintain and update test plans, test cases, test environment, and testware through
life cycle

23. What's a 'test case'?

• A test case is a document that describes an input, action, or event and an expected
response, to determine if a feature of an application is working correctly. A test
 6
case should contain particulars such as test case identifier, test case name,
objective, test conditions/setup, input data requirements, steps, and expected
results.
• the process of developing test cases can help find problems in the requirements or
design of an application, since it requires completely thinking through the
operation of the application.

24. What should be done after a bug is found?

The bug needs to be communicated and assigned to developers that can fix it. After
the problem is resolved, fixes should be re-tested, and determinations made regarding
requirements for regression testing to check that fixes didn't create problems
elsewhere. If a problem-tracking system is in place, it should encapsulate these
processes. A variety of commercial problem-tracking/management software tools are
available

The following are items to consider in the tracking process:

• Complete information such that developers can understand the bug, get an idea of
it's severity, and reproduce it if necessary.
• Bug identifier (number, ID, etc.)
• Current bug status (e.g., 'Released for Retest', 'New', etc.)
• The application name or identifier and version
• The function, module, feature, object, screen, etc. where the bug occurred
• Environment specifics, system, platform, relevant hardware specifics
• Test case name/number/identifier
• One-line bug description
• Full bug description

25. What is 'configuration management'?

Configuration management covers the processes used to control, coordinate, and track:
code, requirements, documentation, problems, change requests, designs,
tools/compilers/libraries/patches, changes made to them, and who makes the changes

26. What if the software is so buggy it can't really be tested at all?

The best bet in this situation is for the testers to go through the process of reporting
whatever bugs or blocking-type problems initially show up, with the focus being on
critical bugs. Since this type of problem can severely affect schedules, and indicates
deeper problems in the software development process (such as insufficient unit testing or
insufficient integration testing, poor design, improper build or release procedures, etc.)
 7
managers should be notified, and provided with some documentation as evidence of the
problem.

27. How can it be known when to stop testing?

This can be difficult to determine. Many modern software applications are so complex,
and run in such an interdependent environment, that complete testing can never be done.
Common factors in deciding when to stop are:

• Deadlines (release deadlines, testing deadlines, etc.)

• Test cases completed with certain percentage passed
• Test budget depleted
• Coverage of code/functionality/requirements reaches a specified point
• Bug rate falls below a certain level
• Beta or alpha testing period ends

28. What if there isn't enough time for thorough testing?

Use risk analysis to determine where testing should be focused.

Since it's rarely possible to test every possible aspect of an application, every possible
combination of events, every dependency, or everything that could go wrong, risk
analysis is appropriate to most software development projects. This requires judgement
skills, common sense, and experience. (If warranted, formal methods are also available.)
Considerations can include:

• Which functionality is most important to the project's intended purpose?

• Which functionality is most visible to the user?
• Which functionality has the largest safety impact?
• Which functionality has the largest financial impact on users?
• Which aspects of the application are most important to the customer?
• Which aspects of the application can be tested early in the development cycle?
• Which parts of the code are most complex, and thus most subject to errors?

29. What if the project isn't big enough to justify extensive testing?

Consider the impact of project errors, not the size of the project. However, if
extensive testing is still not justified, risk analysis is again needed and the same
considerations as described previously in 'What if there isn't enough time for
thorough testing?' apply. The tester might then do ad hoc testing, or write up a limited
test plan based on the risk analysis.

30. How does a client/server environment affect testing?

 8

Client/server applications can be quite complex due to the multiple dependencies

among clients, data communications, hardware, and servers, especially in multi-tier
systems. Thus testing requirements can be extensive. When time is limited (as it
usually is) the focus should be on integration and system testing. Additionally,
load/stress/performance testing may be useful in determining client/server application
limitations and capabilities. There are commercial tools to assist with such testing.