Unit testing

Unit testing, a.k.a. component or module testing, is a form of software testing by which isolated source
code is tested to validate expected behavior.[1]

Unit testing describes tests that are run at the unit-level to contrast testing at the integration or system
level.

History
Unit testing, as a principle for testing separately smaller parts of large software systems dates back to the
early days of software engineering. In June 1956, H.D. Benington presented at US Navy's Symposium on
Advanced Programming Methods for Digital Computers the SAGE project and its specification based
approach where the coding phase was followed by "parameter testing" to validate component
subprograms against their specification, followed then by an "assembly testing" for parts put
together.[2][3]

In 1964, a similar approach is described for the software of the Mercury project, where individual units
developed by different programmes underwent "unit tests" before being integrated together.[4] In 1969,
testing methodologies appear more structured, with unit tests, component tests and integration tests with
the purpose of validating individual parts written separately and their progressive assembly into larger
blocks.[5] Some public standards adopted end of the 60's, such as MIL-STD-483[6] and MIL-STD-490
contributed further to a wide acceptance of unit testing in large projects.

Unit testing was in those times interactive[3] or automated,[7] using either coded tests or capture and
replay testing tools. In 1989, Kent Beck described a testing framework for Smalltalk (later called SUnit)
in "Simple Smalltalk Testing: With Patterns (https://web.archive.org/web/20150315073817/http://www.x
programming.com/testfram.htm)". In 1997, Kent Beck and Erich Gamma developed and released JUnit, a
unit test framework that became popular with Java developers.[8] Google embraced automated testing
around 2005–2006.[9]

Unit
Unit is defined as a single behaviour exhibited by the system under test (SUT), usually corresponding to a
requirement. While it may imply that it is a function or a module (in procedural programming) or a
method or a class (in object-oriented programming) it does not mean functions/methods, modules or
classes always correspond to units. From the system-requirements perspective only the perimeter of the
system is relevant, thus only entry points to externally-visible system behaviours define units.[10]
Execution
Unit tests can be performed manually or via automated test execution. Automated tests include benefits
such as: running tests often, running tests without staffing cost, and consistent and repeatable testing.

Testing is often performed by the programmer who writes and modifies the code under test. Unit testing
may be viewed as part of the process of writing code.

Testing criteria
During development, a programmer may code criteria, or results that are known to be good, into the test
to verify the unit's correctness.

During test execution, frameworks log tests that fail any criterion and report them in a summary.

For this, the most commonly used approach is test - function - expected value.

Test case
In software engineering, a test case is a specification of the inputs, execution conditions, testing
procedure, and expected results that define a single test to be executed to achieve a particular software
testing objective, such as to exercise a particular program path or to verify compliance with a specific
requirement.[11] Test cases underlie testing that is methodical rather than haphazard. A battery of test
cases can be built to produce the desired coverage of the software being tested. Formally defined test
cases allow the same tests to be run repeatedly against successive versions of the software, allowing for
effective and consistent regression testing.[12]

Test double
A test double is software used in software test automation that satisfies a dependency so that the test need
not depend on production code. A test double provides functionality via an interface that the software
under test cannot distinguish from production code.

Parameterized test
A parameterized test is a test that accepts a set of values that can be used to enable the test to run with
multiple, different input values. A testing framework that supports parametrized tests supports a way to
encode parameter sets and to run the test with each set.

Use of parametrized tests can reduce test code duplication.

Parameterized tests are supported by TestNG, JUnit,[13] XUnit and NUnit, as well as in various
JavaScript test frameworks.
Parameters for the unit tests may be coded manually or in some cases are automatically generated by the
test framework. In recent years support was added for writing more powerful (unit) tests, leveraging the
concept of theories, test cases that execute the same steps, but using test data generated at runtime, unlike
regular parameterized tests that use the same execution steps with input sets that are pre-defined.

Agile
Sometimes, in the agile software development, unit testing is done per user story and comes in the later
half of the sprint after requirements gathering and development are complete. Typically, the developers or
other members from the development team, such as consultants, will write step-by-step 'test scripts' for
the developers to execute in the tool. Test scripts are generally written to prove the effective and technical
operation of specific developed features in the tool, as opposed to full fledged business processes that
would be interfaced by the end user, which is typically done during user acceptance testing. If the test-
script can be fully executed from start to finish without incident, the unit test is considered to have
"passed", otherwise errors are noted and the user story is moved back to development in an 'in-progress'
state. User stories that successfully pass unit tests are moved on to the final steps of the sprint - Code
review, peer review, and then lastly a 'show-back' session demonstrating the developed tool to
stakeholders.

Test-driven development
In test-driven development (TDD), unit tests are written while the production code is written. Starting
with working code, the developer adds test code for a required behavior, then adds just enough code to
make the test pass, then refactors the code (including test code) as makes sense and then repeats by
adding another test.

Value
Unit testing is intended to ensure that the units meet their design and behave as intended.[14]

By writing tests first for the smallest testable units, then the compound behaviors between those, one can
build up comprehensive tests for complex applications.[14]

One goal of unit testing is to isolate each part of the program and show that the individual parts are
correct.[1] A unit test provides a strict, written contract that the piece of code must satisfy.

Early detection of problems in the development cycle

Unit testing finds problems early in the development cycle. This includes both bugs in the programmer's
implementation and flaws or missing parts of the specification for the unit. The process of writing a
thorough set of tests forces the author to think through inputs, outputs, and error conditions, and thus
more crisply define the unit's desired behavior.
Reduced cost
The cost of finding a bug before coding begins or when the code is first written is considerably lower
than the cost of detecting, identifying, and correcting the bug later. Bugs in released code may also cause
costly problems for the end-users of the software.[15][16][17] Code can be impossible or difficult to unit
test if poorly written, thus unit testing can force developers to structure functions and objects in better
ways.

More frequent releases

Unit testing enables more frequent releases in software development. By testing individual components in
isolation, developers can quickly identify and address issues, leading to faster iteration and release
cycles.[18]

Allows for code refactoring

Unit testing allows the programmer to refactor code or upgrade system libraries at a later date, and make
sure the module still works correctly (e.g., in regression testing). The procedure is to write test cases for
all functions and methods so that whenever a change causes a fault, it can be identified quickly.

Detects changes which may break a design contract

Unit tests detect changes which may break a design contract.

Reduce uncertainty
Unit testing may reduce uncertainty in the units themselves and can be used in a bottom-up testing style
approach. By testing the parts of a program first and then testing the sum of its parts, integration testing
becomes much easier.

Documentation of system behavior

Some programmers contend that unit tests provide a form of documentation of the code. Developers
wanting to learn what functionality is provided by a unit, and how to use it, can review the unit tests to
gain an understanding of it.

Test cases can embody characteristics that are critical to the success of the unit. These characteristics can
indicate appropriate/inappropriate use of a unit as well as negative behaviors that are to be trapped by the
unit. A test case documents these critical characteristics, although many software development
environments do not rely solely upon code to document the product in development.

In some processes, the act of writing tests and the code under test, plus associated refactoring, may take
the place of formal design. Each unit test can be seen as a design element specifying classes, methods,
and observable behavior.
Limitations and disadvantages
Testing will not catch every error in the program, because it cannot evaluate every execution path in any
but the most trivial programs. This problem is a superset of the halting problem, which is undecidable.
The same is true for unit testing. Additionally, unit testing by definition only tests the functionality of the
units themselves. Therefore, it will not catch integration errors or broader system-level errors (such as
functions performed across multiple units, or non-functional test areas such as performance). Unit testing
should be done in conjunction with other software testing activities, as they can only show the presence
or absence of particular errors; they cannot prove a complete absence of errors. To guarantee correct
behavior for every execution path and every possible input, and ensure the absence of errors, other
techniques are required, namely the application of formal methods to prove that a software component
has no unexpected behavior.

An elaborate hierarchy of unit tests does not equal integration testing. Integration with peripheral units
should be included in integration tests, but not in unit tests. Integration testing typically still relies heavily
on humans testing manually; high-level or global-scope testing can be difficult to automate, such that
manual testing often appears faster and cheaper.

Software testing is a combinatorial problem. For example, every Boolean decision statement requires at
least two tests: one with an outcome of "true" and one with an outcome of "false". As a result, for every
line of code written, programmers often need 3 to 5 lines of test code. This obviously takes time and its
investment may not be worth the effort. There are problems that cannot easily be tested at all – for
example those that are nondeterministic or involve multiple threads. In addition, code for a unit test is as
likely to be buggy as the code it is testing. Fred Brooks in The Mythical Man-Month quotes: "Never go to
sea with two chronometers; take one or three."[19] Meaning, if two chronometers contradict, how do you
know which one is correct?

Difficulty in setting up realistic and useful tests

Another challenge related to writing the unit tests is the difficulty of setting up realistic and useful tests. It
is necessary to create relevant initial conditions so the part of the application being tested behaves like
part of the complete system. If these initial conditions are not set correctly, the test will not be exercising
the code in a realistic context, which diminishes the value and accuracy of unit test results.

Requires discipline throughout the development process

To obtain the intended benefits from unit testing, rigorous discipline is needed throughout the software
development process.

Requires version control

It is essential to keep careful records not only of the tests that have been performed, but also of all
changes that have been made to the source code of this or any other unit in the software. Use of a version
control system is essential. If a later version of the unit fails a particular test that it had previously passed,
the version-control software can provide a list of the source code changes (if any) that have been applied
to the unit since that time.
Requires regular reviews
It is also essential to implement a sustainable process for ensuring that test case failures are reviewed
regularly and addressed immediately.[20] If such a process is not implemented and ingrained into the
team's workflow, the application will evolve out of sync with the unit test suite, increasing false positives
and reducing the effectiveness of the test suite.

Limitations for embedded system software

Unit testing embedded system software presents a unique challenge: Because the software is being
developed on a different platform than the one it will eventually run on, you cannot readily run a test
program in the actual deployment environment, as is possible with desktop programs.[21]

Limitations for testing integration with external systems

Unit tests tend to be easiest when a method has input parameters and some output. It is not as easy to
create unit tests when a major function of the method is to interact with something external to the
application. For example, a method that will work with a database might require a mock up of database
interactions to be created, which probably won't be as comprehensive as the real database interactions.[22]

Examples

JUnit
Below is an example of a JUnit test suite. It focuses on the Adder class.

class Adder {
public int add(int a, int b) {
return a + b;
}
}

The test suite uses assert statements to verify the expected result of various input values to the sum
method.

import static org.junit.Assert.assertEquals;

import org.junit.Test;
public class AdderUnitTest {
@Test
public void sumReturnsZeroForZeroInput() {
Adder adder = new Adder();
assertEquals(0, adder.add(0, 0));
}

@Test
public void sumReturnsSumOfTwoPositiveNumbers() {
Adder adder = new Adder();
assertEquals(3, adder.add(1, 2));
}

@Test
public void sumReturnsSumOfTwoNegativeNumbers() {
Adder adder = new Adder();
assertEquals(-3, adder.add(-1, -2));
 }

@Test
public void sumReturnsSumOfLargeNumbers() {
Adder adder = new Adder();
assertEquals(2222, adder.add(1234, 988));
}
}

As executable specifications
Using unit-tests as a design specification has one significant advantage over other design methods: The
design document (the unit-tests themselves) can itself be used to verify the implementation. The tests will
never pass unless the developer implements a solution according to the design.

Unit testing lacks some of the accessibility of a diagrammatic specification such as a UML diagram, but
they may be generated from the unit test using automated tools. Most modern languages have free tools
(usually available as extensions to IDEs). Free tools, like those based on the xUnit framework, outsource
to another system the graphical rendering of a view for human consumption.

Applications

Extreme programming
Unit testing is the cornerstone of extreme programming, which relies on an automated unit testing
framework. This automated unit testing framework can be either third party, e.g., xUnit, or created within
the development group.

Extreme programming uses the creation of unit tests for test-driven development. The developer writes a
unit test that exposes either a software requirement or a defect. This test will fail because either the
requirement isn't implemented yet, or because it intentionally exposes a defect in the existing code. Then,
the developer writes the simplest code to make the test, along with other tests, pass.

Most code in a system is unit tested, but not necessarily all paths through the code. Extreme programming
mandates a "test everything that can possibly break" strategy, over the traditional "test every execution
path" method. This leads developers to develop fewer tests than classical methods, but this isn't really a
problem, more a restatement of fact, as classical methods have rarely ever been followed methodically
enough for all execution paths to have been thoroughly tested. Extreme programming simply recognizes
that testing is rarely exhaustive (because it is often too expensive and time-consuming to be economically
viable) and provides guidance on how to effectively focus limited resources.

Crucially, the test code is considered a first class project artifact in that it is maintained at the same
quality as the implementation code, with all duplication removed. Developers release unit testing code to
the code repository in conjunction with the code it tests. Extreme programming's thorough unit testing
allows the benefits mentioned above, such as simpler and more confident code development and
refactoring, simplified code integration, accurate documentation, and more modular designs. These unit
tests are also constantly run as a form of regression test.
Unit testing is also critical to the concept of Emergent Design. As emergent design is heavily dependent
upon refactoring, unit tests are an integral component.

Automated testing frameworks

An automated testing framework provides features for automating test execution and can accelerate
writing and running tests. Frameworks have been developed for a wide variety of programming
languages.

Generally, frameworks are third-party; not distributed with a compiler or integrated development
environment (IDE).

Tests can be written without using a framework to exercise the code under test using assertions, exception
handling, and other control flow mechanisms to verify behavior and report failure. Some note that testing
without a framework is valuable since there is a barrier to entry for the adoption of a framework; that
having some tests is better than none, but once a framework is in place, adding tests can be easier.[23]

In some frameworks advanced test features are missing and must be hand-coded.

Language-level unit testing support

Some programming languages directly support unit testing. Their grammar allows the direct declaration
of unit tests without importing a library (whether third party or standard). Additionally, the Boolean
conditions of the unit tests can be expressed in the same syntax as Boolean expressions used in non-unit
test code, such as what is used for if and while statements.

Languages with built-in unit testing support include:

Cobra Rust[25]
D[24]
Languages with standard unit testing framework support include:

Apex MATLAB
Crystal[26] Python[29]
Erlang Racket[30][31]
Go[27] Ruby[32]
Julia[28] Swift
LabVIEW
Some languages do not have built-in unit-testing support but have established unit testing libraries or
frameworks. These languages include:

ABAP JavaScript
C++ Objective-C
C# Perl
Clojure[33] PHP
Elixir PowerShell[34]
Java R with testthat
 Scala Visual Basic .NET
tcl Xojo with XojoUnit

See also
Acceptance testing List of unit testing frameworks
Characterization test Regression testing
Component-based usability testing Software archaeology
Design predicates Software testing
Design by contract System testing
Extreme programming Test case
Functional testing Test-driven development
Integration testing xUnit – a family of unit testing frameworks.

References
1. Kolawa, Adam; Huizinga, Dorota (2007). Automated Defect Prevention: Best Practices in
Software Management. Wiley-IEEE Computer Society Press. p. 75. ISBN 978-0-470-04212-
0.
2. Benington, Herbert D. (1956). "Production of large computer programs". Proceedings of the
Symposium on Advanced Programming Methods for Digital Computers, Washington, D.C.,
June 28-29, 1956. Office of Naval Research, Department of the Navy: 15–28.
3. Benington, H. D. (1 March 1987). "Production of large computer programs (reprint of the
1956 paper with an updated foreword)" (https://dl.acm.org/doi/10.5555/41765.41799).
Proceedings of the 9th International Conference on Software Engineering. ICSE '87.
Washington, DC, USA: IEEE Computer Society Press: 299–310. ISBN 978-0-89791-216-7.
4. Donegan, James J.; Packard, Calvin; Pashby, Paul (1 January 1964). "Experiences with the
goddard computing system during manned spaceflight missions" (https://dl.acm.org/doi/10.1
145/800257.808889). Proceedings of the 1964 19th ACM national conference. ACM '64.
New York, NY, USA: Association for Computing Machinery. pp. 12.101 – 12.108.
doi:10.1145/800257.808889 (https://doi.org/10.1145%2F800257.808889). ISBN 978-1-
4503-7918-2.
5. Zimmerman, Norman A. (26 August 1969). "System integration as a programming function"
(https://dl.acm.org/doi/10.1145/800195.805951). Proceedings of the 1969 24th national
conference. ACM '69. New York, NY, USA: Association for Computing Machinery. pp. 459–
467. doi:10.1145/800195.805951 (https://doi.org/10.1145%2F800195.805951). ISBN 978-1-
4503-7493-4.
6. MIL-STD-483 Military standard: configuration management practices for systems,
equipment, munitions, and computer programs. United states, Department of Defense. 31
December 1970. pp. Section 3.4.7.2. "The contractor shall then code and test software
Units, and enter the source and object code, and associated listings of each successfully
tested Unit into the Developmental Configuration"
7. Tighe, Michael F. (1 January 1978). "The value of a proper software quality assurance
methodology" (https://dl.acm.org/doi/10.1145/1007775.811118). ACM SIGMETRICS
Performance Evaluation Review. 7 (3–4): 165–172. doi:10.1145/1007775.811118 (https://do
i.org/10.1145%2F1007775.811118). ISSN 0163-5999 (https://search.worldcat.org/issn/0163
-5999).
8. Gulati, Shekhar (2017). Java Unit Testing with JUnit 5 : Test Driven Development with JUnit
5. Rahul Sharma. Berkeley, CA: Apress. p. 8. ISBN 978-1-4842-3015-2. OCLC 1012347252
(https://search.worldcat.org/oclc/1012347252).
 9. Winters, Titus (2020). Software engineering at Google : lessons learned from programming
over time. Tom Manshreck, Hyrum Wright (1st ed.). Sebastopol, CA: O'Reilly. ISBN 978-1-
4920-8274-3. OCLC 1144086840 (https://search.worldcat.org/oclc/1144086840).
10. Beck, Kent (2002). Test-Driven Development by Example. Addison-Wesley. ISBN 978-
0321146533.
11. Systems and software engineering -- Vocabulary. Iso/Iec/IEEE 24765:2010(E). 1 December
2010. pp. 1–418. doi:10.1109/IEEESTD.2010.5733835 (https://doi.org/10.1109%2FIEEEST
D.2010.5733835). ISBN 978-0-7381-6205-8.
12. Kaner, Cem (May 2003). "What Is a Good Test Case?" (http://www.kaner.com/pdfs/GoodTes
t.pdf) (PDF). STAR East: 2.
13. Gulati & Sharma 2017, pp. 133–137, Chapter §7 JUnit 5 Extension Model - Parameterized
Test.
14. Hamill, Paul (2004). Unit Test Frameworks: Tools for High-Quality Software Development (ht
tps://books.google.com/books?id=2ksvdhhnWQsC). O'Reilly Media, Inc.
ISBN 9780596552817.
15. Boehm, Barry W.; Papaccio, Philip N. (October 1988). "Understanding and Controlling
Software Costs" (https://web.archive.org/web/20161009084506/http://faculty.ksu.edu.sa/gha
zy/Cost_MSc/R6.pdf) (PDF). IEEE Transactions on Software Engineering. 14 (10): 1462–
1477. doi:10.1109/32.6191 (https://doi.org/10.1109%2F32.6191). Archived from the original
(http://faculty.ksu.edu.sa/ghazy/Cost_MSc/R6.pdf) (PDF) on 9 October 2016. Retrieved
13 May 2016.
16. "Test Early and Often" (https://msdn.microsoft.com/en-us/library/ee330950%28v=vs.110%2
9.aspx). Microsoft.
17. "Prove It Works: Using the Unit Test Framework for Software Testing and Validation" (http://
www.ni.com/white-paper/8082/en/). National Instruments. 21 August 2017.
18. Erik (10 March 2023). "You Still Don't Know How to Do Unit Testing (and Your Secret is Safe
with Me)" (https://stackify.com/unit-testing-basics-best-practices/). Stackify. Retrieved
10 March 2023.
19. Brooks, Frederick J. (1995) [1975]. The Mythical Man-Month. Addison-Wesley. p. 64 (https://
archive.org/details/mythicalmonth00broo/page/64). ISBN 978-0-201-83595-3.
20. daVeiga, Nada (6 February 2008). "Change Code Without Fear: Utilize a regression safety
net" (http://www.ddj.com/development-tools/206105233). Retrieved 8 February 2008.
21. Kucharski, Marek (23 November 2011). "Making Unit Testing Practical for Embedded
Development" (https://www.electronicdesign.com/technologies/embedded-revolution/article/
21794376/making-unit-testing-practical-for-embedded-development). Retrieved 20 July
2020.
22. "Unit Tests And Databases" (http://wiki.c2.com/?UnitTestsAndDatabases). Retrieved
29 January 2024.
23. Bullseye Testing Technology (2006–2008). "Intermediate Coverage Goals" (http://www.bulls
eye.com/coverage.html). Retrieved 24 March 2009.
24. "Unit Tests - D Programming Language" (http://dlang.org/spec/unittest.html). D
Programming Language. D Language Foundation. Retrieved 5 August 2017.
25. Steve Klabnik and Carol Nichols, with contributions from the Rust Community (2015–2023).
"How to Write Tests" (https://doc.rust-lang.org/book/ch11-01-writing-tests.html). Retrieved
21 August 2023.
26. "Crystal Spec" (https://crystal-lang.org/api/0.23.1/Spec.html). crystal-lang.org. Retrieved
18 September 2017.
27. "testing - The Go Programming Language" (https://golang.org/pkg/testing/). golang.org.
Retrieved 3 December 2013.
28. "Unit Testing · The Julia Language" (https://docs.julialang.org/en/v1/stdlib/Test/).
docs.julialang.org. Retrieved 15 June 2022.
29. Python Documentation (2016). "unittest -- Unit testing framework" (https://docs.python.org/3/
library/unittest.html). Retrieved 18 April 2016.
30. Welsh, Noel; Culpepper, Ryan. "RackUnit: Unit Testing" (http://docs.racket-lang.org/rackunit/
index.html). PLT Design Inc. Retrieved 26 February 2019.
31. Welsh, Noel; Culpepper, Ryan. "RackUnit Unit Testing package part of Racket main
distribution" (https://pkgs.racket-lang.org/package/rackunit). PLT Design Inc. Retrieved
26 February 2019.
32. "Minitest (Ruby 2.0)" (http://ruby-doc.org/stdlib-2.0.0/libdoc/minitest/rdoc/MiniTest.html).
Ruby-Doc.org.
33. Sierra, Stuart. "API for clojure.test - Clojure v1.6 (stable)" (https://clojure.github.io/clojure/cloj
ure.test-api.html). Retrieved 11 February 2015.
34. "Pester Framework" (https://github.com/pester/Pester). GitHub. Retrieved 28 January 2016.

Further reading
Feathers, Michael C. (2005). Working Effectively with Legacy Code. Upper Saddle River,
NJ: Prentice Hall Professional Technical Reference. ISBN 978-0131177055.
Gulati, Shekhar; Sharma, Rahul (2017). Java Unit Testing with JUnit 5. Apress.

External links
Test Driven Development (Ward Cunningham's Wiki) (http://c2.com/cgi/wiki?TestDrivenDeve
lopment)

Retrieved from "https://en.wikipedia.org/w/index.php?title=Unit_testing&oldid=1272545574"