Iso Iec Ieee 29119
Iso Iec Ieee 29119
Iso Iec Ieee 29119
net/publication/323759544
CITATION READS
1 13,153
2 authors, including:
Shakeel Ahmed
King Faisal University
12 PUBLICATIONS 30 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Shakeel Ahmed on 14 March 2018.
New international software testing standard was formed by The covered test process of the fundamental model
International standardization organization (ISO) and executes at three different levels, those levels are
International Electrotechnical Commission (IEC) in the illustrated at Figure 2 [1]. The organizational test process
year 2013 [1]. ISO/IEC/IEEE 29119 is a standardization refers to the activities among several projects across the
series novel which covers systems and software organization. Therefore, it contributes to the creation,
development and implementation of organizational test approaches and highlight the effective ones that reveal the
policy. The test policy states management expectations of defects [4].
the organization as well as serving as direct approach for
software testing in business terms.
4. Test Processes ISO/IEC/IEEE 29119-2
Table 1: Activities and tasks example for test monitoring and control aiming to attain test completion criteria. Moreover,
process.
Activity
iteration may occur among the activities. Table 3
Code Name Task summarizes the activities and tasks of this process [5].
TMC1 Set-Up Create suitable measures and means to
update risks with the new changes.
Collect and record measures, compare 4.5 Test Environment Set-Up & Maintenance Process
TMC2 Monitor progress against test plan using measures
collected. This process is used to create and maintain the
TMC3 Control Ensure implementing the activities
required for test plan. environment where tests are performed. It may involve
Test progress against test plan and changes to test environment based on the result of the
TMC4 Report communicate results to stakeholder previous tests. The test environment may change
within specific period of time, update and
report new risks to stakeholders. depending on the configuration management processes.
The tasks and activities within this process are;
4.2 Test completion Process establishment of test environment and maintaining test
environment [5].
Test completion process is a verification test to be
performed when the test of activities is complete. It is used 4.6 Test Execution Process
as verification test to carry out the testing done on system
testing or performance testing to test the overall project. It This process is utilized to run the test procedures that are
is considered as satisfactory test leaving the testing generated as result of test design and implementation
environment in good situation. Communication to process. Test execution process may require running
stakeholder should occur when overall testing is complete. several times since all available test procedures cannot run
Table 2 below summarizes the activities and tasks at single iteration. Furthermore, when an issue is fixed, the
involved in this process [5]. test execution process should be performed again. The
activities involved within this process are; Test procedure
Table 2: Activities and tasks example for test completion process. execution, test results comparison and test execution
Code Activity Name Task recording [5].
TC1 Archive Test Assets Properly identify and store useful
test cases for later use.
Restoration of test environment Table 3: Activities and tasks for test design and implementation process.
TC2 Clean Up Test to pre-defined state upon Activity
Environment Code Name Task
completion of testing activities.
Identify Lessons Record lesson learned during the Analyze test basis to understand the
TC3 Learned project. test item requirements, combine test
TD1 Identify features into test set and prioritize by
Relevant information such as test Feature Sets
plan, test results, test status report risk value, document the feature set
TC4 Report Test and test completion report should and communicate it to stakeholders.
Completion be collected and reported to Determine the test condition for each
stakeholders. TD2 Derive Test test case; prioritize the best condition
Conditions based on risk, record test condition in
test design specification.
4.3 Dynamic Test Processes Derive test coverage items through
Derive Test applying test design techniques;
This type of testing is used to carry out the dynamic testing TD3 Coverage prioritize test coverage items based on
Items risk, record test coverage items in test
in specific phase of testing e.g. unit, system, integration design specification. Record the
and acceptance testing. Four types of dynamic test process traceability.
Determine pre-condition and input
are available; test design and implementation, test values for one or more test cases and
environment set-up and maintenance, test execution and Derive Test expected result prioritize by risk
TD4 Cases value, record test cases items in test
test incident reporting. These processes is normally design specification. Record the
considered as a part of test strategy implementation within traceability, get approval of
test plan in the test phase e.g. System testing or type stakeholder.
Distribute test cases into one or more
testing e.g. performance testing [5]. Assemble test sets based on constraints and
TD5 Test Sets execution, record test case in
procedure specification, record the
4.4 Test Design and Implementation Process traceability.
Derive test procedures from ordered
In this process, test cases and procedures are derived. test set based on pre-condition, post-
condition and dependencies, Identify
Normally, documented in test specifications, but can be Derive Test excluded test data, prioritize test
TD6 Procedures procedures based on risk, record test
executed immediately. Possibly, stored test assets can be
procedures in procedure specification,
used during this process as regression test. This process record the traceability, get approval of
can stop or reinitiate again in case of reporting new stakeholder.
incident. Also, it requires tester to apply one or more
testing techniques to derive test cases or test procedures
IJCSNS International Journal of Computer Science and Network Security, VOL.18 No.2, February 2018 115
4.7 Test Incident Reporting Process essential but considered complementary. However, they
are effective if applied in combination.
This process is used for test incident reporting as a result
of failure detection, items with unexpected or unusual The terms specification-based testing, structure-based
behavior while executing the test or in case of retest. The testing are also known as black-box testing and white-box
activities involved are; analyzing test result and create or testing (also named as clear-box testing), both black-box
update incident results [5]. testing and white-box testing are related to the visibility of
test items in software structure. In black-box testing, test
item’s visibility of the internal structure is not present
5. Test Documentation ISO/IEC/IEEE 29119- while in white-box testing the internal structure of test
3 items is visible. Grey-box testing is used as a term when
knowledge of test items is utilized from specification and
Test documentation part determines the software test structure as well. Below Table 5 shows all techniques
forms and templates that can be used by organizations, pertaining specification-based testing, structure-based
specific project or single activity of testing. It contains testing and experience-based testing [7].
definite documents that considered as an output of test
processes. These documents can have multiple versions;
such issue is related to configuration management [6]. The 7. Keyword-Driven Testing ISO/IEC/IEEE
set of documentation is useful for test practitioners, below 29119-5
Table 4 list available documents in this part.
Keyword-driven test is an approach of test specification
First two templates belong to organizational test process; which is generally used to aid test automation and creation
the three proceedings belong to management test processes of test automation framework. It can be used if no test
while the rest correspondent to dynamic test processes [1]. automation approach is planned or exist. This type of
testing can be applied to all level of testing such as
Table 4: Test documentation template list. component testing and system testing or even for test types
Seq. Documentation name Seq. Documentation name like functional testing and reliability testing. The
1 Test policy 8 Test procedure
specification advantages of applying keyword-driven testing for system
2 Organizational test 9 Test data requirement are; making system easier to use, understandable,
strategy
Test environment maintainable, reusable, automation supportive, and money
3 Test Plan 10 requirement saver.
4 Test status report 11 Test data readiness
report
Test environment Table 5: Test techniques for specification and structure based testing.
5 Test completion report 12 readiness report Specification- Experience-
based testing Structure-based based Testing
6 Test design 13 Test execution log Testing Techniques
specification Techniques Techniques
7 Test case specification 14 Test incident report Equivalence Branch / Decision
Partitioning Testing
Boundary Value Branch Condition
Analysis Testing
6. Test Techniques ISO/IEC/IEEE 29119-4 Cause-effect Branch Condition
Graphing Combination Testing
Classification Tree Data Flow Testing
This part specifies and identifies test techniques that can Method
be used with test processes in part 2. The targeted audience Combinatorial Test Modified Condition
Decision Coverage Error Guessing
of this part is testers, test managers and developers Techniques (MCDC) Testing
especially those who are in charge of software Decision Table
Testing
management and implementation. In this part, test design Random Testing
techniques are defined for specification-based testing, Scenario Testing Statement Testing
structure-based testing, and experience-based testing. In State Transition
Testing
specification-based testing, the main source of information Syntax Testing
used to design test cases is test basis, for instance user
needs, requirements, specification and models. In This part provides efficient definition and constraints for
structure-based testing the source code or model structure keyword-driven testing. It offers introduction and
is used as the source of information to develop test cases. reference approaches to implement keyword-driven testing.
In experience-based testing, the primary source of Also, requirement definition for framework of keyword-
information is the experience and knowledge of testers. driven testing is given. This enables testers to share their
Moreover, all these types of tests are used to generate the work items, test cases, test data, keywords or test
expected end results. Those test design techniques are not
116 IJCSNS International Journal of Computer Science and Network Security, VOL.18 No.2, February 2018
specifications. Moreover, this part provides interfaces processes," in ISO/IEC/IEEE 29119-2:2013(E), pp.1-68.
definitions and data exchange format from different IEEE.
vendors to ensure different system integration. [6] ISO/IEC/IEEE International Standard. 2013. “Software and
Furthermore, it includes definitions of different level of systems engineering -- Software testing --Part 3: Test
documentation," in ISO/IEC/IEEE 29119-3:2013(E), pp.1-
hierarchical keywords with explanation and advice on how 138. IEEE.
to use them. Lastly, list of examples and technical low [7] ISO/IEC/IEEE International Standard. 2015. “Software and
level keywords is provided e.g. “inputData” or systems engineering--Software testing--Part 4: Test
“checkValue” to determine test case on any particular techniques," in ISO/IEC/IEEE 29119-4:2015, pp.1-149.
technical level. It can be also combined to from business- IEEE.
level keyword if needed [8]. [8] ISO/IEC/IEEE International Standard. 2016. “Software and
systems engineering -- Software testing -- Part 5: Keyword-
Driven Testing," in ISO/IEC/IEEE 29119-5 First edition
8. Conclusion 2016-11-15 , pp.1-69. IEEE.
Software testing is an essential part of software Hesham Alaqail earned his B.Sc. in
development cycle that cannot be neglected. Actually, it is Computer Science from King Faisal
University, Saudi Arabia in 2009. He is
an activity that should start upon the creation of software pursuing his M.S. in same field at King
specifications until software deployment and further with Faisal University. Currently, Hesham is
changes and maintenance. The cost of adopting software working as Programmer Analyst at Ministry
testing approaches and practices is considerably high, but of National Guard – Health Affairs, Saudi
the final cost would be much higher in case the system Arabia. His current research interests
encounters failure especially in enterprise and safety- include Software Validation and
critical systems. The awareness of testing standard is also Verification, Machine Learning and
beneficial to purchasers, businesses and governments to Classification Techniques.
verify the quality of the market’s products on well-studied
Shakeel Ahmed received his B.Sc.
and measured basis. (Computer Science in 1997) from Kakatiya
University and M.C.A (Master of Computer
Applications in 2000) from M.K.
Acknowledgment University, India. He earned his PhD
degree in Computer Science from Indore
I would like to thank my instructor Dr. Shakeel Ahmed for University, India, in 2013. Currently he is
his assistance in understanding the testing firm in software working at King Faisal University, Saudi
quality during the academic course. Also, special thanks to Arabia as Assistant Professor in the College
my college at King Faisal University, College of Computer of Computer Sciences and Information
Science and Information Technology for facilitating all Technology. His current research interests include Mobile Ad
educational means to gain knowledge and excel. hoc networks, Software Engineering, cloud computing he has
published more than 15 papers in international journals and
conferences in the field of software engineering and cloud
References computing.
[1] Reid S. 2012. “The New Software Testing Standard”. In:
Dale C., Anderson T. (eds) Achieving Systems Safety. pp
237-255. Springer, London.
[2] Majchrzak T.A. 2012. “Software Testing”. In: Improving
Software Testing. SpringerBriefs in Information Systems.
pp 11-56. Springer, Berlin, Heidelberg.
[3] Matalonga S., Rodrigues F., Travassos G.H. 2015.
“Matching Context Aware Software Testing Design
Techniques to ISO/IEC/IEEE 29119”. In: Rout T.,
O’Connor R., Dorling A. (eds) Software Process
Improvement and Capability Determination. SPICE 2015.
Communications in Computer and Information Science, vol
526. pp 33-44. Springer, Cham.
[4] ISO/IEC/IEEE International Standard. 2013. “Software and
systems engineering --Software testing --Part 1:Concepts
and definitions," in ISO/IEC/IEEE 29119-1:2013(E) , pp.1-
64. IEEE.
[5] ISO/IEC/IEEE International Standard. 2013. “Software and
systems engineering --Software testing --Part 2:Test