International Journal of Computer Applications (0975 – 8887)

Volume 69– No.6, May 2013

Enhancing Software Quality by an SPL Testing based

Software Testing
Hosam F. El-Sofany1, Islam A. Taj-Eddin2, Hassan El-Hoimal1,
Tourki Al-Tourki1, and Amjad Al-Sadoon1
1Arab East Colleges for Graduate Studies, Department of Computer Science, Riyadh, Kingdom of Saudi Arabia
2 Faculty of Informatics and Computer Science British University in Egypt-BUE, Cairo, Egypt

ABSTRACT
Software testing is a process of providing information that a
Focuses on processes Focuses on actual
software application or program meets the business and testing.
and procedures rather Focuses on actual
technical requirements that guided its design and
than conducting testing by executing
development; and works as expected. Regardless of the actual testing on the Software with intend
environment (i.e. Structured or Object Oriented), software system. to identify
testing can also identifies important defects, flaws, or errors in bug/defect through
the application code that must be fixed. Quality control implementation of
involves the maintenance of quality in a product, particularly procedures and
by comparing the performance of a product with a set of process.
specifications. This paper introduces some basic knowledge
regarding to software testing. The paper provides some new
model that could improve quality of software testing as a
mean to increase the overall quality of the product. Process oriented Product oriented Product oriented
activities. activities. activities.
Keywords
Software testing, Software quality, Testing strategy, Software Preventive activities. Corrective process.
quality assurance, software quality control. Corrective process.

1. INTRODUCTION A subset of Software

Test Life Cycle Can be considered as Subset of Quality
The purpose of software testing is not only to detect when the subset of Quality Control
(STLC).
problems occur, but also to help in diagnosing the nature of Assurance.
the problem. Quality control involves the maintenance of
quality in a product, particularly by comparing the
performance of a product with a set of specifications. When
the product is software, an important part of quality control is
The main objective of this paper is to introduce an enhanced
software testing, where the software is run to determine
software quality model and software testing strategy to test
whether it produces the desired output [12, 13]. the different factors that effect on the quality of software as
Most people are confused with the concepts and difference well as increasing the productivity of the software by taking in
between Quality Assurance, Quality Control and Testing. consideration the software complexity that can faces the
Although they are interrelated and at some level they can be designer and implementer of the software systems.
considered as the same activities, but there is indeed a The paper is organized as follows: in section two, we present
difference between them. Mentioned below are the definitions
the software testing concepts and definitions. In section three,
and differences between them [16-18]:
we present a brief survey and a literature review, about the
software testing and software quality. In section four, we
introduce some methods for improving the quality of software
Quality Assurance Quality Control Testing testing. In section five, we introduce an enhanced model to
provide software quality assurance. The paper finally
Activities which Activities which concluded in section six.
Activities which ensure the ensure the
ensure the
implementation of
verification of identification of 2. TESTING BASIC CONCEPTS
developed software bugs/error/defects
processes, procedures in the Software.
and standards in
with respect to 2.1 Software Testing
documented (or not
context to in some cases) Software testing is the process of executing software in a
verification of requirements.
developed software controlled manner, in order to answer the question: Does the
and intended software behave as specified?. Software testing is often used
requirements. in association with the terms verification and validation.

5
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

Verification: It answers the question: Are we doing the job

Advantages Disadvantages
right?. It is the process to make sure the product satisfies the
conditions imposed at the start of the development phase.
 As the tester has knowledge  Due to the fact that a skilled
Validation: It answers the question: Are we doing the right of the source code, it tester is needed to perform
job?. It is the process to make sure the product satisfies the becomes very easy to find white box testing, the costs
specified requirements at the end of the development phase. out which type of data can are increased.
help in testing the
Software testing should not be confused with debugging. application effectively.  Sometimes it is impossible
Debugging is the process of analyzing and locating bugs when to look into every nook and
software does not behave as expected. The term bug is often  It helps in optimizing the corner to find out hidden
used to refer to a problem or fault in a computer. There are code. errors that may create
software bugs and hardware bugs. The term originated in the  Extra lines of code can be problems as many paths will
United States, at the time when pioneering computers were removed which can bring in go untested.
built out of valves, when a series of previously inexplicable hidden defects.  It is difficult to maintain
faults were eventually traced to moths flying about inside the white box testing as the use
computer [16-18].  Due to the tester's
knowledge about the code, of specialized tools like
maximum coverage is code analyzers and
2.2 Software Testing Methods attained during test scenario debugging tools are
writing. required.
There are different methods which can be used for Software
testing. This section briefly describes those methods [16-18]:
III. Grey Box Testing
I. Black Box Testing
Grey Box testing is a technique to test the application with
The technique of testing without having any knowledge of the limited knowledge of the internal workings of an application.
interior workings of the application is Black Box testing. The In software testing, the term the more you know the better
tester is oblivious to the system architecture and does not have carries a lot of weight when testing an application.
access to the source code. Typically, when performing a black
box test, a tester will interact with the system's user interface Mastering the domain of a system always gives the tester an
by providing inputs and examining outputs without knowing edge over someone with limited domain knowledge. Unlike
how and where the inputs are worked upon. black box testing, where the tester only tests the application's
user interface, in grey box testing, the tester has access to
design documents and the database. Having this knowledge,
the tester is able to better prepare test data and test scenarios
Advantages Disadvantages when making the test plan.

 Well suited and efficient for

 Limited Coverage since
large code segments.
only a selected number of Advantages Disadvantages
 Code Access not required. test scenarios are actually
performed.
 Clearly separates user's  Offers combined benefits of
 Inefficient testing, due to black box and white box  Since the access to source
perspective from the
the fact that the tester only testing wherever possible. code is not available, the
developer's perspective
has limited knowledge ability to go over the code
through visibly defined
roles. about an application.  Grey box testers don't rely and test coverage is limited.
on the source code; instead
 Large numbers of  Blind Coverage, since the they rely on interface
moderately skilled testers tester cannot target specific definition and functional  The tests can be redundant
can test the application with code segments or error specifications. if the software designer has
no knowledge of prone areas. already run a test case.
implementation,
 Based on the limited
programming language or
 The test cases are difficult information available, a
to design. grey box tester can design
operating systems.
excellent test scenarios  Testing every possible input
especially around stream is unrealistic because
communication protocols it would take an
and data type handling. unreasonable amount of
II. White Box Testing time; therefore, many
 The test is done from the program paths will go
White box testing is the detailed investigation of internal logic point of view of the user untested.
and structure of the code. White box testing is also called and not the designer.
glass testing or open box testing. In order to perform white
box testing on an application, the tester needs to possess
knowledge of the internal working of the code.
The tester needs to have a look inside the source code and find
out which unit/chunk of the code is behaving inappropriately.

6
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

IV. Black Box vs Grey Box vs White Box Description

The determination of the functionality that the

1
Black Box intended application is meant to perform.
Grey Box Testing White Box Testing
Testing
The creation of test data based on the
2
The Internal specifications of the application.
Tester has full
Workings of an Somewhat knowledge
knowledge of the
application are not of the internal The output based on the test data and the
Internal workings of 3
required to be workings are known specifications of the application.
the application
known

The writing of Test Scenarios and the execution

Another term for grey 4
of test cases.
Also known as box testing is
Also known as clear
closed box testing, translucent testing as
box testing, structural
data driven testing the tester has limited
testing or code based
The comparison of actual and expected results
and functional knowledge of the 5
testing based on the executed test cases.
testing insides of the
application

Performed by end An effective testing practice will see the above steps applied
Performed by end to the testing policies of every organization and hence it will
users and also by Normally done by
users and also by
testers and testers and developers make sure that the organization maintains the strictest of
testers and developers
developers standards when it comes to software quality.

Testing is based on A. Unit Testing

external Testing is done on the Internal workings are This type of testing is performed by the developers before the
expectations - basis of high level fully known and the
setup is handed over to the testing team to formally execute
Internal behavior of database diagrams and tester can design test
the application is data flow diagrams data accordingly the test cases. Unit testing is performed by the respective
unknown developers on the individual units of source code assigned
areas. The developers use test data that is separate from the
This is the least The most exhaustive test data of the quality assurance team.
Partly time consuming
time consuming and time consuming The goal of unit testing is to isolate each part of the program
and exhaustive
and exhaustive type of testing
and show that individual parts are correct in terms of
requirements and functionality. In an Object-Oriented
Not suited to Not suited to algorithm Suited for algorithm environment, this is usually done at the class level. The
algorithm testing testing testing
minimal unit tests include the constructors and destructors
[15].
This can only be Data domains and Data domains and
done by trial and Internal boundaries can Internal boundaries Testing cannot catch each and every bug in an application. It
error method be tested, if known can be better tested is impossible to evaluate every execution path in every
software application. The same is the case with unit testing.
There is a limit to the number of scenarios and test data that
2.3 Levels of Software Testing the developer can use to verify the source code. So after he
has exhausted all options there is no choice but to stop unit
There are different levels during the process of Testing. In this testing and merge the code segment with other units.
section a brief description is provided about these levels.
B. Integration Testing
Levels of testing include the different methodologies that can
be used while conducting Software Testing. Following are the The testing of combined parts of an application to determine if
main levels of Software Testing: Functional Testing, and they function correctly together is Integration testing. There
Non-Functional Testing. are two methods of doing Integration Testing Bottom-up
Integration testing and Top Down Integration testing.
I. Functional Testing
 Bottom-up integration: This testing begins with unit
This is a type of black box testing that is based on the testing, followed by tests of progressively higher-level
specifications of the software that is to be tested. The combinations of units called modules or builds.
application is tested by providing input and then the results
are examined that need to conform to the functionality it was  Top-Down integration: This testing, the highest-level
intended for. Functional Testing of the software is conducted modules are tested first and progressively lower-level
on a complete, integrated system to evaluate the system's modules are tested after that.
compliance with its specified requirements. In a comprehensive software development environment,
There are five steps that are involved when testing an bottom-up testing is usually done first, followed by top-down
application for functionality. testing. The process concludes with multiple tests of the
complete application, preferably in scenarios designed to
mimic those it will encounter in customers' computers,
systems and network [15, 16].

7
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

C. System Testing F. Alpha Testing

Once all the components are integrated, the application as a This test is the first stage of testing and will be performed
whole is tested rigorously to see that it meets Quality amongst the teams (developer and QA teams). Unit testing,
Standards. This type of testing is performed by a specialized integration testing and system testing when combined are
testing team. known as alpha testing. During this phase, the following will
be tested in the application:
System testing is so important because of the following
reasons [15-18]:  Spelling Mistakes
 System Testing is the first step in the Software  Broken Links
Development Life Cycle, where the application is tested
as a whole.  Cloudy Directions
 The application is tested thoroughly to verify that it  The Application will be tested on machines with the
meets the functional and technical specifications. lowest specification to test loading times and any latency
problems.
 The application is tested in an environment which is very
close to the production environment where the G. Beta Testing
application will be deployed. This test is performed after Alpha testing has been
 System Testing enables us to test, verify and validate successfully performed. In beta testing a sample of the
both the business requirements as well as the intended audience tests the application. Beta testing is also
Applications Architecture. known as pre-release testing. Beta test versions of software
are ideally distributed to a wide audience on the Web, partly
D. Regression Testing to give the program a "real-world" test and partly to provide a
preview of the next release. In this phase the audience will be
Whenever a change in a software application is made it is testing the following:
quite possible that other areas within the application have
been affected by this change. To verify that a fixed bug hasn't  Users will install, run the application and send their
resulted in another functionality or business rule violation is feedback to the project team.
Regression testing. The intent of Regression testing is to
ensure that a change, such as a bug fix did not result in  Typographical errors, confusing application flow, and
another fault being uncovered in the application. even crashes.
Regression testing is so important because of the following  Getting the feedback, the project team can fix the
reasons: problems before releasing the software to the actual
users.
 Minimize the gaps in testing when an application with
changes made has to be tested.  The more issues you fix that solve real user problems,
the higher the quality of your application will be.
 Testing the new changes to verify that the change made
did not affect any other area of the application.  Having a higher-quality application when you release to
the general public will increase customer satisfaction.
 Mitigates Risks when regression testing is performed on
the application. II. Non-Functional Testing
 Test coverage is increased without compromising Non-functional testing of Software involves testing the
timelines. Software from the requirements which are non functional in
nature such as performance, security, user interface etc. Some
 Increase speed to market the product of the important and commonly used non-functional testing
E. Acceptance Testing types are [16-18]:

This is arguably the most importance type of testing as it is A. Performance Testing

conducted by the Quality Assurance Team who will gauge It is mostly used to identify any bottlenecks or performance
whether the application meets the intended specifications and issues rather than finding the bugs in software. There are
satisfies the client's requirements. The QA team will have a different causes which contribute in lowering the performance
set of pre written scenarios and Test Cases that will be used to of software: Network delay, client side processing, database
test the application. transaction processing, load balancing between servers, and
More ideas will be shared about the application and more tests data rendering.
can be performed on it to gauge its accuracy and the reasons Performance testing is considered as one of the important and
why the project was initiated. Acceptance tests are not only mandatory testing type in terms of following aspects: Speed,
intended to point out simple spelling mistakes, cosmetic errors capacity, stability, and Scalability.
or Interface gaps, but also to point out any bugs in the
application that will result in system crashers or major errors It can be either qualitative or quantitative testing activity and
in the application. can be divided into different sub types such as: load testing
and Stress testing.
By performing acceptance tests on an application the testing
team will deduce how the application will perform in  Load Testing: A process of testing the behavior of the
production. There are also legal and contractual requirements Software by applying maximum load in terms of
for acceptance of the system [16-18]. Software accessing and manipulating large input data. It

8
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

can be done at both normal and peak load conditions. D. Security Testing
This type of testing identifies the maximum capacity of
Software and its behavior at peak time. Most of the Security testing involves the testing of Software in order to
time, Load testing is performed with the help of identify any flaws ad gaps from security and vulnerability
automated tools such as Load Runner, AppLoader, IBM point of view. Following are the main aspects which Security
Rational Performance Tester, Apache JMeter, Silk testing should ensure: confidentiality, integrity,
Performer, Visual Studio Load Test etc. Virtual users authentication, availability, authorization, non-repudiation,
(VUsers) are defined in the automated testing tool and software data is secure, software is according to all security
the script is executed to verify the Load testing for the regulations, input checking and validation, SQL insertion
Software. The quantity of users can be increased or attacks, injection flaws, session management issues, cross-site
decreased concurrently or incrementally based upon the scripting attacks, buffer overflows vulnerabilities, and
requirements. directory traversal attacks.

 Stress Testing: This testing type includes the testing E. Portability Testing
of Software behavior under abnormal conditions. Portability testing includes the testing of Software with intend
Taking away the resources, applying load beyond the that it should be re-useable and can be moved from another
actual load limit is Stress testing. The main intent is to Software as well. Following are the strategies that can be used
test the Software by applying the load to the system and for Portability testing [16-18].
taking over the resources used by the Software to
identify the breaking point. This testing can be  Transferred installed Software from one computer to
performed by testing different scenarios such as: another.
 Shutdown or restart of Network ports randomly.  Building executable (.exe) to run the Software on
different platforms.
 Turning the database on or off.
Portability testing can be considered as one of the sub parts of
 Running different processes that consume System testing, as this testing type includes the overall testing
resources such as CPU, Memory, server etc. of Software with respect to its usage over different
environments. Computer Hardware, Operating Systems and
B. Usability Testing Browsers are the major focus of Portability testing. Following
This section includes different concepts and definitions of are some pre-conditions for Portability testing:
usability testing from Software point of view. It is a black box
technique and is used to identify any error(s) and  Software should be designed and coded, keeping in mind
improvements in the Software by observing the users through Portability Requirements.
their usage and operation [16].  Unit testing has been performed on the associated
 Usability can be defined in terms of five factors i.e. components.
Efficiency of use, Learn-ability, Memory-ability,  Integration testing has been performed.
Errors/safety, satisfaction. According to this definition,
the usability of the product will be good and the system is  Test environment has been established.
usable if it possesses the above factors.
2.4 Software Specifications Documents and
 We can consider that usability is the quality requirement
which can be measured as the outcome of interactions Testing
with a computer system. This requirement can be fulfilled A hierarchy of software specifications will typically contain
and the end user will be satisfied if the intended goals are three or more levels of software specification documents, see
achieved effectively with the use of proper resources. Figure 1 [16].
 Other authors stated that user friendly system should  The Requirements Specification, which specifies what
fulfill the following five goals (i.e. Easy to Learn, Easy to the software is required to do and may also specify
Remember, Efficient to Use, Satisfactory to Use and Easy constraints on how this may be achieved.
to understand).
 The Architectural Design Specification, which
In addition to different definitions of usability, there are some describes the architecture of a design which implements
standards and quality models and methods which define the the requirements. Components within the software and
usability in the form of attributes and sub attributes such as the relationship between them will be described in this
ISO-9126, ISO-9241-11, ISO-13407 and IEEE std.610.12 etc. document.
C. Graphical User Interface Testing  Detailed Design Specifications, which describe how
User interface (UI) testing involves the testing of Graphical each component in the software, down to individual
User Interface of the Software. This testing ensures that the units, is to be implemented.
GUI should be according to requirements in terms of color,
alignment, size and other properties.
On the other hand Usability testing ensures that a good and
user friendly GUI is designed and is easy to use for the end
user. UI testing can be considered as a sub part of Usability
testing.

9
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

a test strategy to a software development project should

be detailed in the projects software quality plan.
II. Test Plans: The next stage of test design, which is the
first stage within a software development project, is the
development of a test plan. A test plan states what the
items to be tested are, at what level they will be tested,
what sequence they are to be tested in, how the test
strategy will be applied to the testing of each item, and
describes the test environment. A test plan may be
project wide, or may in fact be a hierarchy of plans
relating to the various levels of specification and testing:
 An Acceptance Test Plan, describing the plan for
Fig 1: A hierarchy of software specifications acceptance testing of the software. This would
usually be published as a separate document, but
might be published with the system test plan as a
With this hierarchy of specifications, it is possible to test single document.
software at various stages of the development, for
conformance with each specification. The levels of testing  A System Test Plan, describing the plan for system
which correspond to the hierarchy of software specifications integration and testing. This would also usually be
listed above are: Unit testing, integration testing, System published as a separate document, but might be
testing, and Acceptance testing. All four had been explained published with the acceptance test plan.
previously.  A Software Integration Test Plan, describing the
Testing does not end following the conclusion of acceptance plan for integration of tested software components.
testing. Software has to be maintained to fix problems which This may form part of the Architectural Design
show up during use and to accommodate new requirements. Specification.
Software tests have to be repeated, modified and extended.  Unit Test Plan(s), describing the plans for testing of
regression testing is of use for this case. It had been explained individual units of software. These may form part of
previously. The effort to revise and repeat tests consequently the Detailed Design Specifications.
forms a major part of the overall cost of developing and
maintaining software. The objective of each test plan is to provide a plan for
verification, by testing the software, that the software
Once each level of software specification has been written, the produced fulfils the requirements or design statements of
next step is to design the tests. The design of tests is subject to the appropriate software specification. In the case of
the same basic engineering principles as the design of acceptance testing and system testing, this means the
software. Good design consists of a number of stages which Requirements Specification.
progressively elaborate the design of tests from an initial high
level strategy to detailed test procedures. These stages are: test III. Test Case Design: Once the test plan for a level of
strategy, test planning, test case design, and test procedure testing has been written, the next stage of test design is to
design. specify a set of test cases or test paths for each item to be
tested at that level. A number of test cases will be
The design of tests has to be driven by the specification of the identified for each item to be tested at each level of
software. At the highest level this means that tests will be testing. Each test case will specify how the
designed to verify that the software faithfully implements the implementation of a particular requirement or design
requirements of the Requirements Specification. At lower decision is to be tested and the criteria for success of the
levels tests will be designed to verify that items of software test. The test cases may be documented with the test
implement all design decisions made in the Architectural plan, as a section of a software specification, or in a
Design Specification and Detailed Design Specifications. As separate document called a test specification or test
with any design process, each stage of the test design process description.
should be subject to informal and formal review.
 An Acceptance Test Specification, specifying the
The ease with which tests can be designed is highly dependent test cases for acceptance testing of the software.
on the design of the software. It is important to consider This would usually be published as a separate
testability as a key (but usually undocumented) requirement document, but might be published with the
for any software development. acceptance test plan.
The following steps can help in test design and documented  A System Test Specification, specifying the test
[16]. cases for system integration and testing. This would
I. Test Strategy: A test strategy is a statement of the also usually be published as a separate document,
overall approach to testing, identifying what levels of but might be published with the system test plan.
testing are to be applied and the methods, techniques and  Software Integration Test Specifications, specifying
tools to be used. A test strategy should ideally be the test cases for each stage of integration of tested
organization wide, being applicable to all of an software components. These may form sections of
organizations software development. Developing a test the Architectural Design Specification.
strategy which efficiently meets the needs of an
organization is critical to the success of software  Unit Test Specifications, specifying the test cases
development within the organization. The application of for testing of individual units of software. These

10
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

may form sections of the Detailed Design quality as: “The totality of features and characteristics of a
Specifications. product or service that impact its ability to satisfy given
needs.”
System testing and acceptance testing involve an
enormous number of individual test cases. In order to Bo Zhang, Changchun and Xiangheng Shen [5], defined
keep track of which requirements are tested by which test Software testing is an approach to guarantee software quality,
cases, an index which cross references between and with its effectiveness people can establish confidence in
requirements and test cases often constructed. This is the correctness of software which has been previously tested.
usually referred to as a Verification Cross Reference
Index (VCRI) and is attached to the test specification. Wesley K., Daniela de., Thelma E., Silvia R. [6] defined a
Cross reference indexes may also be used with unit Software Product Line (SPL) is a set of several software
testing and software integration testing. products that share a set of common features. These features
meet specific needs of a particular domain and are developed
It is important to design test cases for both positive from core assets previously defined. Reuse is the focus of the
testing and negative testing. Positive testing checks that SPL approach. Differently from the classical reuse
the software does what it should. Negative testing checks approach, where the software first is developed and packaged,
that the software doesn't do what it shouldn't. and then the reuse may be considered, in the SPL approach,
the development is oriented to obtain a common architecture
The process of designing test cases, including executing that will be used in all products of the SPL.
them as thought experiments, will often identify bugs
before the software has even been built. It is not Clements et al [7] say that SPL engineering comprises three
uncommon to find more bugs when designing tests than essential phases: Domain Engineering, Application
when executing tests. Engineering and SPL Management. In the Domain
Engineering, a set of core assets is developed. It has the
IV. Test Procedures: The final stage of test design is to common artifacts to the whole SPL and it may include plains,
implement a set of test cases as a test procedure, requirements, design, tests and code. In this activity, a generic
specifying the exact process to be followed to conduct architecture is defined, then, the products of the SPL are
each of the test cases. This is a fairly straight forward developed by the instantiation of this architecture according to
process, which can be likened to designing units of code the features of the product. Each specific product is generated
from higher level functional descriptions. by a single combination of features. The SPL development
For each item to be tested, at each level of testing, a test leads to a productivity increase and a reduction of costs and
procedure will specify the process to be followed in delivery time. Then, the development of a product of the SPL
conducting the appropriate test cases. A test procedure occurs in two steps: the development of components to reuse,
cannot leave out steps or make assumptions. The level of in the Domain Engineering, and the product development
detail must be such that the test procedure is from reusable components, in the Application Engineering
deterministic and repeatable. [8]. This makes the SPL testing activity more critical and
complex than test in the classical development process.
V. Test Results: When tests are executed, the outputs of
each test execution should be recorded in a test results According to Lamancha et al [9] there is no process defined
file. These results are then assessed against criteria in the for SPL testing. They also say that there are some
test specification to determine the overall outcome of a guidelines available to perform this activity. SPL testing must
test. Each test execution should also be noted in a test involve the test of core assets, of specific products and their
log. The test log will contain records of when each test interactions [10]. To perform such tasks, the existing test
has been executed, the outcome of each test execution, techniques and methods have been used. However, a
and may also include key observations made during test testing strategy to allow the application of those
execution. Often a test log is not maintained for lower methods in the SPL context is necessary.
levels of testing (unit test and software integration test). Tevanlinna et al [8] describe strategies to SPL testing.
Test reports may be produced at various points during The traditional strategy named Product by Product (PbP)
the testing process. Test reports will summaries the consists of testing each product of the SPL separately, without
results of testing and document any analysis. An the reuse of test assets. The strategy named Reusable Asset
acceptance test report often forms a contractual Instantiation (RAI) creates test specifications based on the
document within which acceptance of software is agreed. Domain Engineering, in order to predict all of the possible
3. PREVIOUS WORK ON SPL TESTING variabilities. In the Application Engineering phase, the test
assets are refined and instantiate for the specific product under
AND QUALITY CONTROL test. The main difference of both strategies is that the last
Every application or business domain faces a specific set of one is in fact product line oriented. However, unnecessary
software quality issues, and software quality must be defined test cases can be generated due to the great number of
accordingly. It is important for each software development possible combinations considering the SPL variability's.
project to define its specific meaning of software quality Strategies that are SPL oriented allow reuse of test assets
during the planning phase. Such a definition contributes to the and were proposed as an alternative to those ones that test
basis for setting objectives and practical measures of quality product by product in a separated way. Considering this fact,
progress and determination of readiness for release to the authors presented results of a case study to evaluate the
customers [1]. test strategy, named RAI (Reusable Assets Instantiation). This
Reeves and Bednar [2] stated that “no universal, strategy that is SPL oriented was compared with a traditional
Parsimonious, or all-encompassing definition or model of strategy PbP (Product by Product). The study used AGM, an
quality exists”. The American National Standards Institute SPL from the game context, and test data generation methods
(ANSI) and American Society for Quality (ANQ) define based on the use case model[6].

11
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

RAI considers that there is a common generic structure, and model do a testing after the finish from software development,
test assets can be reused for different products of SPL. This we do the testing before and after the software development,
strategy allows the test starts at the begin of the Domain which make the new model more easier to find the errors and
Engineering, because it uses the requirements as a source for modifying the code.
generating the test specification. So it is not necessary to
await the generation of products to start testing activities. 6. ACKNOWLEDGMENT
when applying RAI, deriving all the domain test assets and
The authors acknowledge the full support received from Arab
application requirements needed to test the SPL. This
East Colleges for Graduate Studies, especially they very
methodology allows us to deal with the variability of SPL and
thankful to Professor Dr. Abd Allah Alfaisal, Dean of the
instantiate test cases for a specific product[6].
Colleges, for his encouragement and support.
The result of this case study has produced evidence that the
RAI strategy is more suitable for SPL testing than the PbP REFERENCES
strategy, considering reuse. Moreover, the case study [1] Lazic L., Kolasinac A., Avdic D., “The Software Quality
constitutes an experience of SPL test which shows that the Economics Model for Software Project Optimization”,
RAI strategy is effective [6]. WSEAS Transaction on Computers, Issue 1, Vol. 8,
January 2009.
4. NEW SOFTWARE QUALITY
ASSURANCE MODEL [2] Reeves C., Bednar D., “Defining Quality: Alternative
and Implications”, Academy of Management Review 19
In [11] software complexity is added during the development (3), pp. 419-445, 1994.
stages that following the requirements phase, primarily during
the designing and coding phase. In our case to provide [3] ISO, International Organization for Standardization, "ISO
software quality assurance, we performed the testing process 9000:2000, Quality Management Systems-Fundamentals
after solution phase to ensure that design and code meet the and Vocabulary", 2000.
requirements and to achieve a best quality before development [4] ISO, International Organization for Standardization, "ISO
phase, as shown in Figure 2, Also we have modified the 9004:2000, Quality Management Systems-Guidelines for
model introduced in [11] by redesign the productivity phase performance improvements", 2000.
that comes after quality phase to ensure produce software with
high quality, fulfilled all the requirements, uncomplicated [5] Bo Zhang , Changchun and Xiangheng Shen, " The
code, ease of maintenance and modification and less errors. Effectiveness of Real-time Embedded Software Testing"
, 978-1-61284-666-8/11$26.00 2011 IEEE.
[6] Wesley K., Daniela de., Thelma E., Silvia R. "
Evaluating test reuse of a software product line
oriented strategy", 978-1-4577-1490-01111$26.00
©2011 IEEE
[7] P. C. Clements, L. G. Jones, J. D. Mcgregor and L.
M. Northrop, "Getting There From Here: A Roadmap
for Software Product Line Adoption". Communications
of the ACM. Vol. 49, N° 12. Dec., 2006.
[8] A. Tevanlinna, J. Taina and R. Kauppinen. "Product
Family Testing: a Survey". ACM SIGSOFT Sw Eng.
Notes 29(2), 2004, pp. 1-6.
[9] B. P. Lamancha, M. Usaola and M. Piattini, "Software
Product Line Testing, A systematic review".
International Conference on Soft ware, vol. 49, 2009, pp.
78 - 81.
[10] J. D. McGregor. "Testing a soft ware product line".
Technical Report CMU1SEI-2001-TR-022, Soft ware
Engineering Institute, Carnegie Mellon University,
2001.
[11] Yas A. Alsultanny, and Ahmed M. Wohaishi,
"Requirements of Software Quality Assurance Model",
2nd. International Conference on Environmental and
Computer Science, DOI 10.1109/ICECS.2009.43, IEEE,
2009.
Fig 2: New SPL Testing Based Model for software quality
assurance [12] Horch JW. Practical Guide to Software Quality
Management. 2nd edition. Artech House Publishers.
5. CONCLUSIONS ISBN 9781580535274.
In this paper we propose the difference between traditional [13] Hower R. Software QA and Testing Frequently-Asked-
testing and SPL testing strategy. The SPL testing strategy Questions." http://www. Software qatest. Com /qatfaq
starts testing at the begin of development project, so we 1.html. 2009.
modified the old model based on that concept, because the old

12
 International Journal of Computer Applications (0975 – 8887)
Volume 69– No.6, May 2013

[14] Basili, V.R. Qualitative software complexity models: A [17] Software Testing Concepts and Practices, K. Mustafa and
summary. In Tutorial on Models and Methods for R.A. Khan. Alpha Science International Ltd. Oxford
Software Management and Engineering. IEEE Computer .U.K. 2007.
Society Press, Los Alamitos, Calif., 1980.
[18] Software Testing and Computing Quality Impleme-
[15] Binder, Robert V. (1999). Testing Object-Oriented ntation, William Lewis, 2nd edition , 2004.
Systems: Objects, Patterns, and Tools. Addison-Wesley
Professional. p. 45. ISBN 0-201-80938-9.
[16] http://www.tutorialspoint.com/software_testing/

13