https://sites.google.

com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 1 of 8
Chapter 2.6: Testing and running a solution

2.6 (a) Types of Programming Errors

When programs are being written it is not surprising that mistakes are made, after all they are
very complicated. There are three types of error which can be made when program code is being
produced.

1. Syntax errors. These are errors in the use of the language. The rules have been broken. A
command word might be PRINT. If it was typed in as PLINT the translator program would not
recognise it and so would not be able to translate it. This is not a difficult error to spot because
when the programmer tells the translator to translate it, a message will be returned saying that a
mistake has been made and telling the programmer which word it cannot recognise. There are
other ways of breaking the rules, for instance a language might be set up to only accept a
mathematical formula if it is in the form of a variable followed by an equals sign and then a
collection of variables. So X = 3*(2+A) would be accepted by the translator, but 3*(2+A) =
X would be rejected as an error even though, mathematically, it is identical.

The following code contains many different syntax errors:

DIM X no data-type has been stated
X=1
IF X > 1 no THEN statement
Y=(2*X+3 no closing bracket
PLINT Y misspelling of a keyword
NEXT NEXT statement without a FOR (Should have been END IF)

2. Logic errors (Run Time). A logic error is a mistake in the way the program solution has been
designed. For example, an instruction in a program may tell the computer to jump to another part
of the program. This is fine as long as the instruction tells the computer to go to the right part of
the program. If it tells the computer to go to the wrong place, it is highly unlikely that the
program will produce the results that were expected. Unfortunately, such a mistake does not
usually make the program stop working; it just makes it produce wrong results. This makes it
difficult to spot that a mistake has been made, and even when the programmer realises that
something has gone wrong, finding where the error is can be very difficult. Another example of a
typical logic error is when an incorrect formula is used e.g. Total_number =
Previous_total New_number. There is nothing obviously wrong with the formula so it
will work, but it wont produce the right answer because it should have been +. Notice that this is
not an arithmetic error, the computer will do the arithmetic perfectly, and it is the logic that is
wrong.


https://sites.google.com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 2 of 8
3. Arithmetic error. This is sometimes thought of as a type of logic error but is rather different
because the computer is asked to do something that is impossible. A good example is when a
computer is asked to divide by 0, this is impossible and would cause the program to fail.

The following code will produce an arithmetic error on the fourth line because X was initialised
to zero and the first iteration will attempt to divide by X when it is this initial value:

X=0 X is initialised to zero
Y=1
FOR i=1 TO 10
Y=Y/X attempted division by zero on first iteration
X=X+i
NEXT

https://sites.google.com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 3 of 8
2.6 (b) Testing Methods

When a system is designed it is important that some consideration is given to making sure that
no mistakes have been made. A schedule should be drawn up which contains a test for every type
of input that could be made and methods of testing that the program actually does what it was
meant to do. This schedule is known as the test plan. Note that it is produced before the system is
produced.

There are a number of ways of testing a program.

1. Black box: Different values can be input for variables to determine whether the program can
cope with them. These values should include typical values, borderline values and values which
are not acceptable. For example, if a program is written which uses marks out of 100 from a
maths examination as input, the test data would include typical data like 27, 73.., borderline data
which would be 0 and 100, and unacceptable data like 34, 123, 16.345 This type of testing is
called black box testing.

2. White box testing is testing the program to determine whether all the possible paths through
the program produce the desired results. As a large program can have a very large number of
routes, when you take into account the different condition statements and loops, white box
testing is rarely carried out exhaustively.

Think of black box as a test where you cannot see into the box (program) all you see is what
comes out at the end. White box testing means that you are able to see what is happening as the
data goes through the box because it is transparent.

3. Alpha and Beta testing. When you have written a program and you sit down to test it, you
have a certain advantage because you know what to expect. After all, you wrote the program.
This can be extended to the whole software company, as the employees are all computer-minded
people. Testing carried out by people like this is known as alpha testing. Eventually, the
company will want ordinary users to test the program because they are likely to find errors that
the software specialists did not find. Testing carried out by the users of the program is called beta
testing. If you continue the course next year, in order to turn your AS grade into an A level, you
will have to write a project. This involves using a computer to solve a problem for someone.
When you have finished your project you will be expected to test whether or not it works, this is
alpha testing. You will also get marks if you persuade the person whose problem you are solving
to test it for you, this is the beta testing.

4. User Acceptance Testing is often the final step before rolling out the application. Usually the
end users who will be using the applications test the application before accepting the
application. This type of testing gives the end users the confidence that the application being
delivered to them meets their requirements. This testing also helps nail bugs related to usability
of the application.

https://sites.google.com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 4 of 8
2.6 (c) Selection of test data

If a solution is to be tested, someone has to choose what data is going to be used to do the
testing. The test data is usually chosen by the programmer because they know what they want to
test. It is important to test as many different things as possible, the important word there being
different. The problem for the programmer is that they know what inputs were expected so they
find it very difficult to think of the sort of inputs that the user of the program might try to put in.

When you are asked to think up different inputs to test a program, it must be different types of
input, not just changing numbers. Imagine a question that states that a program has been written
that will work out the mean of three numbers. You have to come up with different test data and
the reasons for doing those tests. That last bit is in bold because that is what you get the marks
for and the reasons for the tests are the things that have to be different. In this example you
would me thinking of

1, 2, 3 to test whether integers will give an integer answer
1, 2, 4 to test whether the software can cope with a recurring decimal answer
(Note that 1, 2, 4 to test a different set of integers would not get a mark because the reason for
the test is not different)
1, 2.5, 3 to test whether the program can use decimal inputs
1, 2, 3 to test whether fractions are allowed
-1, -2, -3 to test whether negative numbers can be handled
1, 2 to test what happens when only two values are input

There are many more that would be acceptable. The important thing to notice is that the numbers
themselves are almost identical but that the reasons for choosing them are very different. Also
pay attention to NORMAL, ABNORMAL and BORDERLINE data.

Example:
Software calculates the cost of parking at a council car park as follows:
Free from 5am to 8am
3 per hour or part of an hour between 8am and 5pm
Free from 5pm to 12 midnight
No parking from 12 midnight to 5am

Solution:
Suitable test data will include:
data within each of the valid parking periods;
data that crosses more than one of the valid parking periods
data at the boundaries of the valid parking periods;
data not within the valid parking periods.
The test data should be presented in a table similar to the following:
https://sites.google.com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 5 of 8


Example:
A procedure has been written which will accept any two numbers and output the
larger.
Give different sets of data, which could be used to test the procedure.

Solution:

https://sites.google.com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 6 of 8
2.6 (d) Trace tables and dry run

Students should do class practices with related past paper questions to learn dry running/desk
checking an algorithm with the help of their teachers.

https://sites.google.com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 7 of 8
2.6 (e) Debugging

Errors in computer solutions are called bugs. They create two problems. One is that the error
needs to be corrected; this is normally fairly straightforward because most errors are caused by
silly mistakes. The second problem, however, is much more complicated, the errors have to be
found before they can be corrected. Finding where the error is and identifying it, can be very
difficult and there are a number of techniques available for solving such problems.

1. Translator diagnostics. Each of the commands that are in the original program is looked at
separately by the computer translator to execute it. Each command will have a special word
which says what sort of command it is. The translator looks at the special word in the command
and then goes to its dictionary to look it up. The dictionary tells the translator program what the
rules are for that particular special word. If the word has been typed in wrongly, the translator
will not be able to find it in the dictionary and will know that something is wrong. If the word is
there, but the rules governing how it should be used have not been followed properly, the
translator will know that there is something wrong. Either way, the translator program knows
that a mistake has been made, it knows where the mistake is and, often, it also knows what
mistake has been made. A message detailing all this can be sent to the programmer to give hints
as to what to do. These messages are called translator diagnostics.

2. Debugging tools. Sometimes the program looks alright to the translator, but it still doesnt
work properly. Debugging tools are part of the software which help the user to identify where the
errors are. The techniques available include:

a) Cross-referencing. This software checks the program that has been written and finds places
where particular variables have been used. This lets the programmer check to make sure that the
same variable has not been used twice for different things.

b) Traces. A trace is where the program is run and the values of all the relevant variables are
printed out, as are the individual instructions, as each instruction is executed. In this way, the
values can be checked to see where they suddenly change or take on an unexpected value.

c) Variable dumps (check/watch). At specified parts of the program, the values of all the
variables are displayed to enable the user to compare them with the expected results.

3. Desk checking is sometimes known as a dry run. The user works through the program
instructions manually, keeping track of the values of the variables. Most computer programs
require a very large number of instructions to be carried out, so it is usual to only dry run small
segments of code that the programmer suspects of harbouring an error.

4. In section 2.1.c we discussed the splitting up of a problem into smaller and smaller parts, until
each part was a manageable size. When the parts were combined they would produce a solution
to the original problem. Because we are starting with a big problem and splitting it into smaller
problems, this was called top-down design. When the program is written, each small program is
written separately, allowing the small programs to be tested thoroughly before being combined.
This is called bottom-up programming. The technique is particularly useful for testing the
https://sites.google.com/site/computing9691/
https://sites.google.com/site/computing9691/
Page 8 of 8
finished program because it is far easier to test a lot of small programs than it is to test one large
one. One problem can arise, because the small programs have to be joined together, these joints
have to be tested too to make sure that no silly mistakes have been made like using the same
variable name for two different things in two parts of the program (tested by cross referencing).

5. Test strategies are important to establish before the start of testing to ensure that all the
elements of a solution are tested, and that unnecessary duplication of tests is avoided.

6. (This section is with respect to VB6) If you reach a point in your code that calls another
procedure (a function, subroutine, or the script associated with an object or applet), you can enter
(step into) the procedure or run (step over) it and stop at the next line. At any point, you can
jump to the end (step out) of the current procedure and carry on with the rest of the application.
You may want to step through your code and trace code execution because it may not always be
obvious which statement is executed first. Use these techniques to trace the execution of code:
Break points can be set within program code so that the program stops temporarily to
check that it is operating correctly to that point.
Step Into: Traces through each line of code and steps into procedures. This allows you to
view the effect of each statement on variables.
Step Over: Executes each procedure as if it were a single statement. Use this instead
of Step Into to step across procedure calls rather than into the called procedure.
Step Out: Executes all remaining code in a procedure as if it were a single statement, and
exits to the next statement in the procedure that caused the procedure to be called
initially.
Run To Cursor: Allows you to select a statement in your code where you want
execution to stop. This allows you to "step over" sections of code (for example, large
loops).
To trace execution from the current statement
From the Debug menu, choose Step Into, Step Over, Step Out, or Run To Cursor.
To trace execution from the beginning of the program
From the Debug menu, choose Step Into, Step Over, Step Out, or Run To Cursor.