1 Overview 5 Management

2 Life cycle 6 Tools

3 Static testing 7 Web testing

4 Dynamic test techniques 8 Software quality

Web testing versus traditional testing Web application components Testing practice Web testing tools

Web-based application characteristics

Application runs in two or more machines

A wide number of users accesses concurrently

Heterogeneous execution environments: hardware, network connections, operating systems, Web servers and Web browsers Fewer UI controls, fewer events

Limited keyboard event support

Slide 3

Client/Server and Web application

Client/Server application Web application

Requires installation on the clients machine

Application is a menu-driven

Do not require any installation on the client machine

URL-driven

Connected mode (connection Disconnected mode (state exists always until logout) less) Less number of network issues

Many issues like hardware compatibility, browser compatibility, version compatibility etc
Slide 4

Client/Server and Web application

Client/Server application Limited number of users Web application Unlimited number of users

Less securiry issues

Maximum securiry issues

Majorly used in Intranet areas Used in Internet areas Performance issues will not be Performance issues are very taken into consideration importance
The user interaction with the The user interaction is server is mainly through a user through a compatible web interface browser
Slide 5

Web testing versus traditional testing Web application components Testing practice Web testing tools

Web application components

Server-side components core application service components other application service components Client-side components web browsers add-on/plug-in components

Slide 7

Server-side components
Core application service components web servers

store Web pages or HTML files and their associated contents also serve advanced technology components such as Java servlets, ActiveX controls, and back-end database connectors act as data repositories for Web applications refer to a set of components that extend their services to other components or integrated application components
Slide 8

database servers

application servers

Server-side components (contd)

Other application service components search servers

catalog and index data that is published by Web servers act as intermediaries between networks and the Internet protect private networks from Internet intruders

proxy servers and firewalls

communication-related servers
e-commerce-related servers multimedia-related servers

Slide 9

Client-side components
Web browsers applications that retrieve, assemble, and display Web pages there are multiple browsers and browser versions available Add-on/Plug-in components support various forms of interactivity and animation within Web pages Macromedia Shockwave, Java applets and ActiveX controls
Slide 10

Web testing versus traditional testing Web application components Testing practice Web testing tools

Testing practice
User interface tests

Functional tests
Database tests Configuration and Compatibility testing

Security testing
Performance testing

Slide 12

User interface tests

User interface testing normally refers to a type of integration testing Often done in conjunction with other tests usability testing, functionality testing, exploratory testing, and task-oriented functional testing UI testing issues design of UI components implementation of UI components

Slide 13

UI design testing
Who is the applications target user? profiling the target user

computer experience web experience domain knowledge application-specific experience

How to evaluate design? design approach user interaction (data input) data presentation (data output)
Slide 14

UI design testing (contd)

Design approach point out as many design deficiencies early in the testing as possible, inconsistency in the design User interaction navigation methods: UI control, pages mouse/keyboard action matrices action commands: names of commands, default commands feedback and error messages

evaluated for clarity and consistency

Slide 15

Data presentation

UI implementation testing
Examines applications toward operation evaluates whether UI features work properly Do not be mistaken for functional tests

Slide 16

Usability testing
A metric that helps determining the users satisfaction when

interacting with a product or service through interfaces Factors

ease of learning efficiency of use

memorability
error frequency and severity

subjective satisfaction

Process identify any usability problems, collect quantitative data on participants' performance (e.g. time on task, error rates), and determine participant's satisfaction with the product
Slide 17

Accessibility testing
Web content must be available to and accessible by everyone navigate within a Web page and between Web pages navigate with keyboard only or other specialized input devices easily follow the contents and instructions provided Methods and tools for designing accessible products Web Content Accessibility Guidelines (W3C) several tools are available for automating: evaluation, repair, and transformation
Slide 18

Testing practice
User interface tests

Functional tests
Database tests Configuration and Compatibility testing

Security testing
Performance testing

Slide 19

Web application functional testing

Methods Functional acceptance simple tests (FASTs) Task-oriented functional tests (TOFTs) Forced-error tests (FETs) Boundary condition tests and equivalent class analysis Exploratory

Slide 20

Functional Acceptance Simple Tests

Exercises the lowest level of functionality for each command of a program Objective: check for the appropriate behaviors of UI controls based on the intended designs What to check? links (content links, image links,...) basic controls (backward and forward navigating, zoomin and zoom-out,...) action command checks (add, remove, update, create user accounts, data-entry tests...) other key features (log in/log out, e-mail notification, search, credit card validation)
Slide 21

Task-Oriented Functional Tests

Check whether the application can do useful tasks correctly How to do? create a features-to-be-tested list by dissecting product specification carefully, also consider competitive influences and market demands each entry on the list should be used to define a test case that checks whether the features requirements have been met

Slide 22

Forced-error tests
Contains negative test cases that are designed to force a program into error conditions Objective: to find any error conditions

Ways of compiling a list of error conditions (p.259) Testing process (p.260)

Slide 23

Testing practice
User interface tests

Functional tests
Database tests Configuration and Compatibility testing

Security testing
Performance testing

Slide 24

Common types of errors in database

Data integrity errors any bug that causes erroneous results to be stored missing or incorrect data in records, in tables, data might be outdated,... Output errors caused by bugs in the data retrieving and manipulating instructions, although the source data is correct See the list in ebook (p.330)

Slide 25

White-box methods
Code walk-through (peer-review process) point out any inefficiencies, redundancies, inconsistencies, or poor coding practices

executing the SQL statements or stored procedures one at a time

Testing triggers identify all the triggers analyze and catalog the conditions under which a trigger will be executed write and execute SQL statements or stored procedures
Slide 26

Inefficiency coding error example

SELECT * FROM staff SELECT id, state, salary, name FROM staff

This type of error only causes the database to do unnecessary extra work. The overall performance might be affected
Slide 27

Redundancy coding error example

A redundancy error in the ASP code can be caught by code walk-through
Send a query to the SQL database from an ASP Set RS = Conn.Execute (Select * from STAFF) Now, loop through the records If NOT RS.EOF Then Do while Not RS.EOF Notice that the If statement The code that manipulates, formats and displays records goes here is redundant because the ... condition is already Loop covered in the Do while End If

loop.

Slide 28

Black-box methods
Designing test cases executed on the browser-side

e.g. wrong data type, problematic characters: ', ,

drawbacks: the results do not sure that the data itself is

properly written; erroneous results do not necessarily mean database error

Testing for transaction logic check the database tables to ensure the transaction logic Testing for concurrency issues need to lock records to prevent concurrent updates and prevent data errors in the database
Slide 29

Testing practice
User interface tests

Functional tests
Database tests Configuration and Compatibility testing

Security testing
Performance testing

Slide 30

Configuration and compatibility testing-1

Configuration testing is designed to uncover errors related to various software and hardware combinations validate that a certain Web system installed on a dualprocessor computer operates properly Compatibility testing determines if an application, under supported configurations, performs as expected with various combinations of hardware and software releases determine which manufacturers and server brands, under the same configuration, are compatible with the Web system
Slide 31

Incompatibility issues example

Configuration and compatibility testing-2

When to run compatibility and configuration testing? should begin after the first round of functional tests has been completed and, ideally, after many functional errors have been discovered It is not practical to execute the entire set of tests on every environment testing a cross section of available hardware and software partition various operating environments Consider: server-side, client-side, network devices and connectivity How to test? (p.411)
Slide 33

Testing practice
User interface tests

Functional tests
Database tests Configuration and Compatibility testing

Security testing
Performance testing

Slide 34

Security testing
A combination of many protective measures taken to ensure the safety of the data and resources of both the owners and the users of computer systems Requires a combination of knowledge of security technologies, network technologies, programming, and real-world experience in penetrating the security of network systems Issues anatomy of an attack attacking intents common vulnerabilities and attacks testing for security
Slide 35

Anatomy of an attack
Information gathering viewing the source code

path names names and/or e-mail addresses of programmers, testers, and support staff

Network scanning scanning the system to determine which servers are alive and reachable from the Internet Attacking an active process of intrusion
Slide 36

Attacking intents
To steal

To disrupt activities
To embarrass To play a game

Slide 37

Common vulnerabilities and attacks-1

Software bugs, poor design, and programming practice buffer overflows backdoors javascript java ... Cookies

Malicious programs: virus and worm, trojan horses

Slide 38

Common vulnerabilities and attacks-2

Misuse access privilege attacks

Password cracking
Denial-of-service (DoS) attacks Information leaks social engineering keystroke capturing garbage rummaging scanning and probing

Slide 39

Testing for security

Testing the requirements and design

Testing the application code

Testing the deployment Testing with user protection via browser settings

Slide 40

Testing the requirements and design

Requirements are key understand how much the system is worth Access control need to consider access controls for system administrators, back-up utilities, the owners of the application, and users of the application servers access rights should be limited to the minimum rights required and only for the duration Client privacy issues: what information needs to be private? build a matrix of specific screens available to each class of user
Slide 41

Testing the application code

Backdoors check to make sure that all backdoors are closed in the production release ID and password testing check for the misuse of superuser accounts look for IDs and passwords in the clear when connecting to and accessing servers, directories, databases, and other resources

Slide 42

Testing the application code (contd)

Guidelines for testing information leaks test for comments in the HTML code test for error messages that reveal server name, root directory name,... turn off the directory browsing option on the Web server check to ensure that Access to Source configuration on the Web server is turned off check to ensure that the Write privilege to local path is blocked

Slide 43

Testing the application code (contd)

Testing the use of GET and POST check for the preferred use of POST instead of GET check the information in URLs try to access the file or directory shown in the URL Testing parameter-tampering e.g. www.mtesting.com/saving_acct001.pdf www.mtesting.com/saving_acct002.pdf SQL injection attacks:

SELECT * FROM bank WHERE ID=ADMIN AND PASSWORD=no OR 1 #

Slide 44

Testing the application code (contd)

Testing for buffer overflows requires testing everywhere a buffer is used send very large amounts of data to the buffer; test boundary conditions Testing for bad data, e.g. send an asterisk (*) instead of the persons last name entering </body></html> as a name Testing reliance on client-side scripting try to bypass authentication by going to the URL directly

Slide 45

Testing the deployment

Installation defaults check that the defaults provide the maximum amount of security Default passwords should never have passwords preset Program forensics using log files

Slide 46

Testing practice
User interface tests

Functional tests
Database tests Configuration and Compatibility testing

Security testing
Performance testing

Slide 47

Performance testing
Problem Solution web applications enable multiple users to access the application simultaneously evaluate the systems capability to perform critical functions during periods of normal and peak usage Often requires the aid of automated testing tools to simulate workload, collect measurement data, and present data that can be used for performance analysis Types of tests performance testing load testing stress testing
Slide 48

Types of tests 1
Performance testing an information-gathering and analysis process

metrics to collect: transactions per second, hits per second, concurrent connections, throughput,... to predict when load levels will exhaust system resources

Load testing evaluates system performance with a predefined load level minimum configuration and maximum activity levels be determined
Slide 49

Types of tests 2
Stress testing evaluates the behavior of systems that are pushed beyond their specified operational limits evaluates responses to bursts of peak activity that exceed system limitations

Slide 50

Three phases of performance testing

Planning phase

Testing phase
Analysis phase

Slide 51

Planning phase 1
Identifying baseline configuration and performance

requirements
consider hardware and software configurations, network

bandwidth, memory requirements, disk space, connectivity technologies... environment and resources, workload, acceptable response time

Determining the workload user-specific load

the number of concurrent users, the ratio of different types of users, and the behavior of different types of users TPS, ratios of specific pages requested by users...
Slide 52

application-specific load

Planning phase 2
Determining when to begin testing should be started as early in the product development process as possible Generating loads hardware-intensive approach

the use of multiple client workstations in the simulation of real-world activity

software-intensive approach

the use of a tool to simulate many users accessing the system simultaneously

Developing test cases

Slide 53

Testing phase
Generating test data by manually by programming by using existing data from the system under test Setting up a test bed of data Setting up the test suite parameters

Run tests Collect data

Slide 54

Analysis phase
Characterize the system organize the performance data gathered Analyze results and and look for the sources of performance

problems
examining the performance monitors on the servers receiving

the load hardware-related and software-related problems

Optimize the system under test add processing power? add RAM? add storage? add bandwidth?...
Slide 55

Web testing versus traditional testing Web application components Testing practice Web testing tools

Some web testing tools

Rule-based analyzers (HTML validators, link checkers)

Load/performance testing tools

GUI capture and playback tools Runtime error detectors

Web security testing tools

Java-specific testing tools Database testing tools

Slide 57