Software Quality Assurance

Lecture 6

Software Testing
+ General Techniques
+ Web Application Testing

Faculty of Information Technology

Hanoi University
 Software testing principles
1. A necessary part of a test case is a definition
of the expected output or result.
2. A programmer should avoid attempting to
test his or her own program.
3. A programming organization should not test
its own programs.
4. Any testing process should include a through
inspection of the results of each test.
 Software testing principles
5. Test cases must be written for input
conditions that are invalid and unexpected,
as well as for those that are valid and
expected.
6. Examining a program to see if it does not do
what it is supposed to do is only half the
battle; the other half is seeing whether the
program does what it is not supposed to do.
 Software testing principles
7. Avoid throwaway test cases unless the program
is truly a throwaway program.
8. Do not plan a testing effort under the tacit
assumption that no errors will be found.
9. The probability of the existence of more errors
in a section of a program is proportional to the
number of errors already found in that section.
10.Testing is an extremely creative and
intellectually challenging task.
Part 1

BLACK-BOX TESTING TECHNIQUES

 (1) Decision table testing
• Testing system behavior for different input
combinations.
• A systematic way to cover all input combinations.
 (2) Equivalence partitioning
• Identify test cases by using one element from
each equivalence class.
– Problem: how to choose equivalence classes wisely?
• Divide input value into partitions and write a test
case for each partition, ideally:
– Identify sets with the “same behavior”
– Test at least one input from each set
 (2) Equivalence partitioning
• Example:
– Condition: input x is from 1 to 999
– Valid equivalence class: (1 ≤ x and 999 ≥ x)
– Two invalid equivalence classes: x < 1 and x > 999
• For a month value:

• If value ranges are simple (e.g. 0-1, 1-999, 1000-2000),

consider testing all values to possibly discover bugs that
would not be caught otherwise.
 (3) Boundary-value analysis
• Boundary values are directly on, above, or
beneath the edges of input equivalence classes
and output equivalence classes.
– Each edge of an equivalence class is the subject of a test.
– Also consider the result (output equivalence classes).
• Guidelines:
1. If an input specifies a range of values, write test cases for
the ends of the range, and invalid-input test cases for
situations just beyond the ends. For instance, if the valid
domain of an input value is –1.0 to 1.0, write test cases for
the situations –1.0, 1.0, –1.001, and 1.001.
 (3) Boundary-value analysis
• It is a black-box testing technique.
• Guidelines:
2. Use guideline 1 for each output condition. If output of a
function ranges from 0.00 and 20.00, write test cases that
cause 0.00 and 20.00 to be produced. Also, see whether it
is possible to invent test cases that might cause a negative
output or an output of more than 20.00.
3. If the input or output is an ordered set, focus attention on
the first and last elements of the set.
 (4) Error Guessing
• Use intuition and experience to predict errors and
write test cases for them.
• It's good for testers to keep a list of typical types
of bugs as reference for testing in the future.
Part 2

WHITE-BOX TESTING TECHNIQUES

 Basis Path Testing
• A white-box testing technique introduced in 1976
for testing procedural programs.
1. Analyze execution paths in a program/module/function (using
flow chart, flow graph or text).
2. Test all execution paths.

Flow Chart Flow Graph

Text
 Code Coverage
• A measure which describes the degree of
which the source code of the program has
been tested.
• It is categorized as a white-box testing
technique.
 Types of Code Coverage
• Statement Coverage
• Decision Coverage
• Branch Coverage
• Condition Coverage
 Statement Coverage (1)
• Let A = The number of statements executed by
test cases.
• Let B = The total number of statements.
𝐴
𝑆𝑡𝑎𝑡𝑒𝑚𝑒𝑛𝑡 𝐶𝑜𝑣𝑒𝑟𝑎𝑔𝑒 =
𝐵
• If age = 5, code coverage is 100%, if age = 15,
code coverage is 66.67% (2 of 3 statements)
Statement Coverage (2)
 Statement Coverage (3)
• Statement Coverage can detect the following:
– Unused Statements
– Dead Code
– Unused Branches
– Missing Statements
• Statement Coverage is ineffective for:
– Complex logical expressions
 Decision Coverage (1)
• Measured by the number of executed decision
outcomes over the total number of decision
outcomes.
• Example:
o age = 12, coverage = 50%.
o age = 14, coverage = 50%.
Decision Coverage (2)

1. 1A-2C-3D-E-4G-5H
2. 1A-2B-E-4F
 Decision Coverage (3)
• Goal: every decision in the program must take on
all possible outcomes (true/false) at least once.

• Is also called “branch coverage”.

• Is stronger than statement coverage.

• Can often satisfy statement coverage.

 Condition Coverage
• Measured by the number of tested Boolean
outcomes over all Boolean outcomes.
• In the example below, there are 4 outcomes in
total: TT, TF, FT, FF.
o x = 5, y = 6, a = 1, b = 3: TF, coverage is 25%.
➢ Goal: Every condition in the program must take on
all possible outcomes (true/false) at least once.
 Path coverage

1. 1A-2B-E-4F
2. 1A-2B-E-4G-5H
3. 1A-2C-3D-E-4G-5H
4. 1A-2C-3D-E-4F
 Documentation Testing
• Software documentation is also a testing target.
• The goal is to ensure the document accurately
guide the end-users to use the software.
• Some points to note:
– Does the documentation accurately describe how to
accomplish each mode of use?
– Are examples accurate?
– Are terminology, menu descriptions, and system
responses consistent with the actual program?
– Are the document’s table of contents and index accurate,
and complete?
Part 3

TESTING FOR WEB APPS

 Security Testing for Web Apps (1)
• SQL Injection:
– User inputs (GET, POST) are often embedded in SQL
statements. For example:

– Certain input values can turn the SQL statement into a

different query (which retrieves sensitive data).
 Security Testing for Web Apps (2)
• How to discover SQL Injection bugs?
– Try these characters at the beginning, the end or middle if
in user input values and URL parameters:
' " ; \ \'
– Try doubling these characters (e.g. ' to '')
• How to prevent SQL Injection attack?
– Read online to know how attackers use SQL Injection.
– Always follow up-to-date standards of querying database
(prepared statement, input preprocessing, etc).
 Security Testing for Web Apps
• Cross-site Scripting (XSS) Attack:
– Example: the attacker puts a <script> tag in his
forum post. This <script> reads the cookies and
silently sends the data to the attacker's server
(e.g. by simply embedding an <img> tag). Other
forum users open his forum post and the <script>
is executed. Cookies were stolen at that moment.
 Types of XSS attacks
• Reflected
• Stored
• DOM Injection
 Security Testing for Web Apps
• How to prevent XSS attack?
– This is a long topic. First, know the types of XSS.
– Prevent all HTML tags in user input by escaping the <
and > characters (turn into &lt; and &gt;)
– Often times, tags are allowed (in rich-text input like
forum posts, news content…). In this case:
• Only allow the minimum set of tags
• Prevent HTML attributes that can lead to script
execution (such as onerror, onclick, onmouseover)
– Read more about XSS from multiple sources to update
your knowledge.
 Security Testing for Web Apps
• File upload attack:
– Upload forms can be dangerous if it allows server
scripting files such as PHP files, .htaccess files, etc.
• How to prevent?
– In server scripts, only allow specific file extensions
when handling upload form.
– Have proper security configurations for upload
directory on server (e.g. disable executing permission)
 Web Apps Performance Testing
• Monitor CPU and Memory usage when running the
web application on browser.
• Use load testing tools (e.g. loader.io) to see the web
application's response time and capability.
• Use the Development tool (F12 on Google Chrome) to
see the number of sizes of the files (CSS, images, JS,
fonts, etc).
– Look for files that are slow to load.
– Look for files that are unreasonably large in size (for example:
unscaled images, uncompressed scripts and css).
– The lower files count, the better.
 Web Apps White-Box Testing
• Code inspections:
– HTML: does HTML follow standard? Are there
redundant contents (e.g. elements from a
template which are not used and hidden).
– CSS: does CSS follow standard? Are there unused
properties?
– JavaScript: developers often focus too much on
server scripts that they forget to test browser
scripts.
 Web Apps White-Box Testing
• Code inspections:
– Server scripts: check for code quality. Do server scripts
follow design principles? Perform typical white-box testing
techniques for server scripts.
• Data inspections:
– Check that data are stored and altered correctly when the
web application is in operation.
• Web Server inspections:
– Ensure sessions are created and destroyed as expected.
– Inspect the access log and error log files.
 Midterm exam Guidelines
• Schedule: From 09:30 to 10:30 on next Saturday, March 8
(in next lecture session)
• Location: In this classroom (901C)
• Scope: Include 06 lectures and tutorials (1-6)
• Format:
+ MCQ (Multiple choice Questions)
+ 70 questions.
+ Paper – based
 References
• Glenford J. Myers, Corey Sandler, and Tom
Badgett. 2011. The Art of Software Testing, 3rd
Edition. Wiley Publishing.