STQA

Explain the guidelines to be followed for automation

in Testing. Also, explain costs incurred in testing
tools.
Testing tools can never replace the analytical skills required to conduct testing and
manual testing. It incurs some cost and it may not provide the desired solution if you
are not careful. Therefore, it is necessary that you carefully plan the automation
before adopting it. Decide which tool and how many tools are required, how much
resources are required including the cost of the tool and the time spent on training.
Discussed below are the guidelines to be followed if you have planned for
automation in testing.

1. Consider building a tool instead of buying one: If the requirement is small

and sufficient resources allow, then go for building the tool instead of buying,
after weighing the pros and cons. Whether to buy or build a tool requires
management commitment, including budget and resource approvals.

2. Test the tool on an application prototype: While purchasing the tool, it is

important to verify that it works properly with the system being developed.
However, it is not possible as the system being developed is often not available.
Therefore, it is suggested that if possible, the development team can build a
system prototype for evaluating the testing tool.

3. Not all the tests should be automated: Automated testing is an enhancement

of manual testing, but it cannot be expected that all test on a project can be
automated. It is important to decide which parts need automation before going
for tools. Some tests are impossible to automate, such as verifying a printout. It
has to be done manually.

4. Select the tools according to organizational needs: Do not buy the tools just
for their popularity or to compete with other organizations. Focus on the needs of
the organization and know the resources (budget, schedule) before choosing the
automation tool.

5. Use proven test-script development techniques: Automation can be effective

if proven techniques are used to produce efficient, maintainable, and reusable
test scripts. The following are some hints:

STQA 1
 6. Automate the regression tests whenever feasible: Regression testing
consumes a lot of time. If tools are used for this testing, the testing time can be
reduced to a greater extent. Therefore, whenever possible, automate the
regression test cases.

Following are some facts pertaining to the cost incurred in testing tools :

1. Automated script development: Automated test tools do not create test scripts.
Therefore, a significant time is needed to program the tests. Scripts are
themselves programming languages. Thus, automating test execution requires
programming exercises.

2. Training is required: It is not necessary that the tester will be aware of all the
tools and can use them directly. He may require training regarding the tool.
Therefore, it becomes necessary that in a new project, cost of training on the
tools should also be included in the project budget and schedule.

3. Configuration management: Configuration management is necessary to track

large number of files and test related artifacts.

4. Learning curve for the tools There is a learning curve in using any new tool.
For example, test scripts generated by the tool during recording must be
modified manually, requiring tool-scripting knowledge in order to make the script
robust, reusable, and maintainable.

5. Testing tools can be intrusive It may be necessary that for automation some
tools require that a special code is inserted in the system to work correctly and to
be integrated with the testing tools. Intrusive tools pose the risk that defects
introduced by the code inserted specifically to facilitate testing could interfere
with the normal functioning of the system.

6. Multiple tools are required: It may be possible that your requirement is not
satisfied with just one tool for automation. In such a case you have to go for
many tools which incur a lot of cost.

What is the need of automation in testing? Also,

Explain the guidelines to be followed while selecting
testing tool.

STQA 2
 Need of Automation in Testing
If an organization needs to choose a testing tool, the following benefits of automation
must be considered.

1. Reduction of testing effort: In verification and validation strategies, numerous test

case design methods have been studied. Executing Test cases for a complete
software manually takes a lot of testing effort and time. Thus, execution of test
suits through software tools greatly reduces the amount of time required.

2. Reduces the testers’ involvement in executing tests: Sometimes executing the

test cases takes a long time. Automating this process of executing the test suit
will relieve the testers to do some other work, thereby increasing the parallelism
in testing efforts.

3. Facilitates regression testing: If we automate the process of regression testing,

then testing effort as well as the time taken will reduce as compared to manual
testing.

4. Avoids human mistakes: Manually executing the test cases may incorporate
errors in the process or sometimes, we may be biased towards limited test cases
while checking the software.

5. Reduces overall cost of the software: As we have seen that if testing time
increases, cost of the software also increases. But due to testing tools, time and
therefore cost can be reduced to a greater level as testing tools ease the burden
of the test case production and execution.

6. Simulated testing: Load performance testing is an example of testing where the

real-life situation needs to be simulated in the test environment. Sometimes, it
may not be possible to create the load of a number of concurrent users or large
amount of data in a project. Automated tools, on the other hand, can create
millions of concurrent virtual users/data and effectively test the project in the test
environment before releasing the product.
Internal testing Testing may require testing for memory leakage or checking the
coverage of testing. Automation tools can help in these tasks quickly and
accurately, whereas doing this manually would be cumbersome, inaccurate, and
time-consuming.

7. Test enablers While development is not complete, some modules for testing are
not ready. At that time, stubs or drivers are needed to prepare data, simulate
environment, make calls, and then verify results. Automation reduces the effort
required in this case and becomes essential.

STQA 3
 8. Test case design: Automated tools can be used to design test cases also.
Through automation, better coverage can be guaranteed, than if done manually.

The big question is how to select a testing tool. It may depend on several factors.
What are the needs of the organization; what is the project environment;
what is the current testing methodology; all these factors should be considered when
choosing testing tools. Some guidelines to be followed by selecting
a testing tool are given below.

1. Match the tool to its appropriate use Before selecting the tool, it is necessary to
know its use. A tool may not be a general one or may not cover many features.
Rather, most of the tools are meant for specific tasks. Therefore, the tester needs
to be familiar with both the tool and its uses in order to make a proper selection.

2. Select the tool to its appropriate SDLC phase Since the methods of testing
changes according to the SDLC phase, the testing tools also change. Therefore,
it is necessary to choose the tool according to the SDLC phase, in which testing
is to be done.

3. Select the tool to the skill of the tester The individual performing the test must
select a tool that conforms to his skill level. For example, it would be
inappropriate for a user to select a tool that requires programming skills when the
user does not possess those skills.

4. Select a tool which is affordable Tools are always costly and increase the cost of
the project. Therefore, choose the tool which is within the budget of the project.
Increasing the budget of the project for a costlier tool is not desired. If the tool is
under utilization, then added cost will have no benefi ts to the project. Thus, once
you are sure that a particular tool will really help the project, then only go for it
otherwise it can be managed without a tool also.

5. Determine how many tools are required for testing the system A single tool
generally cannot satisfy all test requirements. It may be possible that many test
tools are required for the entire project. Therefore, assess the tool as per the test
requirements and determine the number and type of tools required.

6. Select the tool after examining the schedule of testing First, get an idea of the
entire schedule of testing activities and then decide whether there is enough time
for learning the testing tool and then performing automation with that tool. If there
is not enough time to provide training on the tool, then there is no use of
automation.

STQA 4
 Write short note on categorization of testing tools.
A single tool may not cover the whole testing process, therefore, a variety of testing
tools are available according to different needs and users. The different categories of
testing tools are discussed here:

1. STATIC AND DYNAMIC TESTING TOOLS: These tools are based on the type
of execution of test cases, namely static and dynamic:

a. Static Testing Tools: For static testing, there are static program analysers
which
scan the source program and detect possible faults and anomalies. Static
tools perform the following types of static analysis:

i. Control flow analysis: This analysis detects loops with multiple exits and
entry points and unreachable code.

ii. Data use analysis: It detects all types of data faults.

iii. Interface analysis: It detects all interface faults. It also detects functions
which are never declared and never called or function results that are
never used.

iv. Path analysis It identifies all possible paths through the program and
unravels the program’s control.

b. Dynamic Testing Tools: These tools support following:

a. Dynamic Testing activities.

b. Automated test tools enable the test team to capture the state of events
during the execution of a program by preserving a snapshot of the
conditions. These tools are sometimes called program monitors. The
perform following functions:

a. List the number of times a component is called or line of code is

executed.

b. Report summary statistics providing a high-level view of the

percentage of statements, paths, and branches that have been
covered by the collective set of test cases run.

2. Testing Activity Tools:

STQA 5
 These tools are based on the testing activities or tasks in a particular phase of
the SDLC.

1. Tools for review and inspections Since these tools are for static analysis on
many items, some tools are designed to work with specifi cations but there
are
far too many tools available that work exclusively with code.

2. Tools for test planning The types of tools required for test planning are:

a. Templates for test plan documentation

b. Test schedule and staffing estimates

3. Tools for test design and development

4. Test execution and evaluation tools: Coverage analysis, memory testing, test
management, network and performance testing.

Explain principles of ISO 9000:2000 standard.

The ISO 9000:2000 standard is based on the following eight principles:

Principle 1. Customer Focus: Success of an organization is highly dependent

on satisfying the customers. An organization must understand its customers and their
needs on a continued basis. Understanding the customers helps in understanding
and meeting their requirements. An example of customer focus is to understand how
they are going to use a system. By accurately understating how customers are going
to use a system, one can produce a better user profile.

Principle 2. Leadership: Leaders set the direction their organization should take, and
they must effectively communicate this to all the people involved in the process.
Leaders must set challenging but realistic goals and objectives. Leaders create a
positive environment and provide support for the employees to collectively realize the
organizational goal. They reevaluate their goals on a continual basis and
communicate the findings to the staff.

Principle 3. Involvement of People: In general, organizations rely on people. People

are informed of the organizational direction, and they are involved at all levels of
decision making. People are given an opportunity to develop their strength and use
their abilities. People are encouraged to be creative in performing their tasks.

STQA 6
 Principle 4. Process Approach: There are several advantages to performing major
tasks by using the concept of process. A process is a sequence of activities that
transform inputs to outputs. Organizations can prepare a plan in the form of
allocating resources and scheduling the activities by making the process defined,
repeatable, and measurable. Continuous improvement in processes leads to
improvement in efficiency and effectiveness.
Principle 5. System Approach to Management: A system is an interacting
set of processes. A whole organization can be viewed as a system of interacting
processes. In the context of software development, gathering customer requirements
for a project is a distinct process and designing a functional specification by taking
the requirements as input is another distinct process. There are simultaneous and
sequential processes being executed in an organization. A process is affected by the
outcome of some other processes, and, in turn, it affects some other processes in
the organization. It is important to understand the overall goal of the organization and
the individual sub-goals associated with each process. For an organization as a
whole to succeed in terms of effectiveness and efficiency, the interactions among
processes must be identified and analyzed.

Principle 6. Continual Improvement: Continual improvement means that the

processes involved in developing, say, software products are reviewed on a periodic
basis to identify where and how further improvements in the processes can be
effected. Since no process can be a perfect one to begin with, continual improvement
plays an important role in the success of organizations. It results in lower cost of
production and maintenance and also leads to less differences between the
expected and actual behavior of the product.

Principle 7. Factual Approach to Decision Making: Decisions may be made based on

facts, experience, and intuition. Facts can be gathered by using a sound
measurement process. A quantitative measurement program helps organizations
know how much improvement has been achieved due to a process improvement.
Principle 8. Mutually Beneficial Supplier Relationships: An organization must carefully
choose the suppliers of the components and subsystems they procure and make
them aware of the organization’s needs and expectations. The performance of the
products procured from outside should be evaluated, and the need to improve their
products and processes should be communicated to the suppliers.

Six Sigma

STQA 7
 Six sigma is a quality model originally developed by Motorola for manufacturing
processes. It derives its meaning from the field of statistics. Sigma is the standard
deviation for a statistical population. Six sigma means, if one has six standard
deviations between the mean of a process and the nearest specification limit, there
will be practically no items that fail to meet the specifications. Therefore, the goal of
this model is to have a process quality of that level.
Eventually, six sigma evolved and was applied to other non-manufacturing
processes. Today, you can apply six sigma to many fields like services, medical, and
insurance procedures, call centers including software.
Six sigma is a way to achieve strategic business results emphasizing on lower costs
with less number of defects. Six sigma processes will produce less than 3.4 defects
per million opportunities. To achieve this target, it uses a methodology known as
DMAIC with the following steps:

Define opportunities

Measure performance

Analyse opportunity

Improve performance

Control performance

This methodology improves any existing business process by constantly reviewing

and improving the process.
Six sigma is not just all about statistics. It can be applied to software problems which
affects its quality. Six sigma can be applied to analyse the customer requirements
and define business goals correspondingly.
Its approach to customer requirements, if applied to software development projects,
is fundamentally different than those typically practiced in software deployment
efforts. It does not start by asking the customers about the requirements first. But it
begins by analysing what we need to learn.
There are six sigma tools which help in identifying the prioritization of functionalities
to be delivered.

Software Quality Management

STQA 8
 Software is not similar to any physical product, so the concept of software quality
also differs from physical products. Software has many possible quality
characteristics which mainly revolve around defects.

If a software has functional bugs, then it may be possible that the software is not able
to
perform the functions. Thus, software quality may be defined in the form of delivered
defect density or defect rate, i.e. the number of defects per unit size (e.g. LOC, FP, or
other unit).
Quality needs to be planned on a larger scale and all controlling and assurance
activities are performed according to quality plans.

SQM is simply a ay to assure quality in the sw. It is the set of activities which ensure
processes, procedures as well as standards suitable for the project are implemented
correctly.

The task of quality management is to

plan suitable quality control and quality assurance activities.

define procedures and standards which should be used during software

development and verify that these are being followed by everyone.

properly execute and control activities.

If quality at some point is not under control, then it is the responsibility of quality
managers to manage the resources such that the required level of quality is
achieved.

The major role of QM is to develop a quality culture in the organization. Quality

culture means that every member of the team is aware and conscious about the
quality and is working towards a high-quality end-product.

QM involves verifying and improving the software process. The quality of

intermediate results at each stage and the final end-product is measured, reviewed,
and analysed. If the target does not seem to be achievable with the current process,
then problems and difficulties are reported to senior management, and finally the
current process can be improved based on this feedback.

MCCALL’S QUALITY FACTORS AND CRITERIA

Quality Factors:

STQA 9
 A quality factor represents a behavioral characteristic of a system. Following are the
different Quality factors:

Correctness: A software system is expected to meet the explicitly specified

functional requirements and the implicitly expected nonfunctional requirements.
If a software system satisfies all the functional requirements, the system is said
to be correct. However, a correct software system may still be unacceptable to
customers if the system fails to meet unstated requirements, such as stability,
performance, and scalability.

Reliablity: A large software system can have some functions which may not work
in all execution scenarios. However, the software may still be acceptable to
customers because the execution scenarios causing the system to fail may not
frequently occur. Customers may still consider an incorrect system to be reliable
if the failure rate is very small and it does not adversely affect their mission
objectives. Reliability is a customer perception.

Efficiency: Efficiency concerns to what extent a software system utilizes

resources, such as computing power, memory, disk space, communication
bandwidth, and energy. A software system must utilize as little resources as
possible to perform its functionalities.

Integrity: A system’s integrity refers to its ability to withstand attacks to its

security. In other words, integrity refers to the extent to which access to software
or data by unauthorized persons or programs can be controlled. Integrity has
assumed a prominent role in today’s network-based applications.

Usability: A software system is considered to be usable if human users find it

easy to use. Users put much emphasis on the user interface of software
systems. Without a good user interface a software system may fizzle out even if
it possesses many desired qualities.

Others are Maintainablilty, testability, flexibily, portability, reusability, interoperability,

Quality Criteria
Access audit: Ease with which software and data can be checked for compliance
with standards or other requirements
Access control: Provisions for control and protection of the software and data
Accuracy: Precision of computations and output
Communication commonality: Degree to which standard protocols and interfaces are
used

STQA 10
 Completeness: Degree to which a full implementation of the required functionalities
has been achieved
Consistency: Use of uniform design and implementation techniques and notation
throughout a project
Data commonality: Use of standard data representations
Hardware independence: Degree to which the software is dependent on the
underlying hardware
Modularity: Provision of highly independent modules Operability Ease of operation of
the software
Self-documentation: Provision of in-line documentation that explains implementation
of components
Simplicity: Ease with which the software can be understood.

STQA 11