PROJECTS FOR SCIENTIFIC PROGRAMMING IN C++ AND

OTHER LANGUAGES

VICTOR EIJKHOUT

2020
2 Introduction to Scientific Programming
Contents

I SIMPLE PROJECTS 7
1 Prime numbers 9
1.1 Arithmetic 9
1.2 Conditionals 9
1.3 Looping 10
1.4 Functions 10
1.5 While loops 10
1.6 Classes and objects 11
1.6.1 Exceptions 12
1.6.2 Prime number decomposition 12
1.7 Other 13
1.8 Eratosthenes sieve 13
1.8.1 Arrays implementation 14
1.8.2 Streams implementation 14
1.9 Range implementation 15
1.10 User-friendliness 15
2 Geometry 17
2.1 Basic functions 17
2.2 Point class 17
2.3 Using one class in another 19
2.4 Is-a relationship 20
2.5 Pointers 20
2.6 More stuff 21
3 Zero finding 23
3.1 Root finding by bisection 23
3.1.1 Simple implementation 23
3.1.2 Polynomials 24
3.1.3 Left/right search points 25
3.1.4 Root finding 26
3.1.5 Object implementation 27
3.1.6 Templating 27
3.2 Newton’s method 28
3.2.1 Function implementation 28
3.2.2 Using lambdas 28
4 Eight queens 31

3
CONTENTS

4.1 Problem statement 31

4.2 Solving the eight queens problem, basic approach 32
4.3 Developing a solution by TDD 32
4.4 The recursive solution method 34

II RESEARCH PROJECTS 37
5 Infectuous disease simulation 39
5.1 Model design 39
5.1.1 Other ways of modeling 39
5.2 Coding 40
5.2.1 The basics 40
5.2.2 Population 41
5.2.3 Contagion 42
5.2.4 Spreading 42
5.2.5 Mutation 43
5.2.6 Diseases without vaccine: Ebola and Covid-19 44
5.3 Ethics 44
5.4 Project writeup and submission 44
5.4.1 Program files 44
5.4.2 Writeup 45
5.5 Bonus: mathematical analysis 45
6 Google PageRank 47
6.1 Basic ideas 47
6.2 Clicking around 48
6.3 Graph algorithms 48
6.4 Page ranking 49
6.5 Graphs and linear algebra 50
7 Redistricting 51
7.1 Basic concepts 51
7.2 Basic functions 52
7.2.1 Voters 52
7.2.2 Populations 52
7.2.3 Districting 53
7.3 Strategy 54
7.4 Efficiency: dynamic programming 56
7.5 Extensions 56
7.6 Ethics 57
8 Amazon delivery truck scheduling 59
8.1 Problem statement 59
8.2 Coding up the basics 59
8.2.1 Address list 59
8.2.2 Add a depot 62
8.2.3 Greedy construction of a route 62
8.3 Optimizing the route 63
8.4 Multiple trucks 64

4 Introduction to Scientific Programming

 CONTENTS

8.5 Amazon prime 65

8.6 Dynamicism 65
8.7 Ethics 66
9 High performance linear algebra 67
9.1 Mathematical preliminaries 67
9.2 Matrix storage 68
9.2.1 Submatrices 70
9.3 Multiplication 70
9.3.1 One level of blocking 70
9.3.2 Recursive blocking 70
9.4 Performance issues 71
9.4.1 Parallelism (optional) 71
9.4.2 Comparison (optional) 71
10 Graph algorithms 73
10.1 Traditional algorithms 73
10.1.1 Code preliminaries 73
10.1.2 Level set algorithm 75
10.1.3 Dijkstra’s algorithm 75
10.2 Linear algebra formulation 76
10.2.1 Code preliminaries 76
10.2.2 Unweighted graphs 77
10.2.3 Dijkstra’s algorithm 77
10.2.4 Sparse matrices 78
10.2.5 Further explorations 78
10.3 Tests and reporting 78
11 Climate change 79
11.1 Reading the data 79
11.2 Statistical hypothesis 79
12 Desk Calculator Interpreter 81
12.1 Named variables 81
12.2 First modularization 82
12.3 Event loop and stack 82
12.3.1 Stack 82
12.3.2 Stack operations 83
12.3.3 Item duplication 84
12.4 Modularizing 85
12.5 Object orientation 86
12.5.1 Operator overloading 86

III APPENDIX 87
13 Style guide for project submissions 89
13.1 General approach 89
13.2 Style 89
13.3 Structure of your writeup 89
13.3.1 Introduction 90

Victor Eijkhout 5
CONTENTS

13.3.2 Detailed presentation 90

13.3.3 Discussion and summary 90
13.4 Experiments 90
13.5 Detailed presentation of your work 90
13.5.1 Presentation of numerical results 91
13.5.2 Code 91
13.6 General index 91
14 Bibliography 95

6 Introduction to Scientific Programming

 PART I

SIMPLE PROJECTS
Chapter 1

Prime numbers

In this chapter you will do a number of exercises regarding prime numbers that build on each other. Each
section lists the required prerequisites. Conversely, the exercises here are also referenced from the earlier
chapters.

1.1 Arithmetic

Exercise 1.1. Read two numbers and print out their modulus. The modulus operator is x%y.
• Can you also compute the modulus without the operator?
• What do you get for negative inputs, in both cases?
• Assign all your results to a variable before outputting them.

1.2 Conditionals

Exercise 1.2. Read two numbers and print a message stating whether the second is as divisor of the
first:
Code: Output
[primes] division:
int number,divisor;
bool is_a_divisor; ( echo 6 ; echo 2 ) |
/* ... */ divisiontest
if ( Enter a number:
/* ... */ Enter a trial divisor:
) { Indeed, 2 is a divisor of 6
cout << "Indeed, " << divisor
<< " is a divisor of " ( echo 9 ; echo 2 ) |
<< number << ’\n’; divisiontest
} else { Enter a number:
cout << "No, " << divisor Enter a trial divisor:
<< " is not a divisor of " No, 2 is not a divisor of 9
<< number << ’\n’;
}

9
1. Prime numbers

1.3 Looping

Exercise 1.3. Read an integer and set a boolean variable to determine whether it is prime by testing
for the smaller numbers if they divide that number.
Print a final message
Your number is prime

or
Your number is not prime: it is divisible by ....

where you report just one found factor.

Printing a message to the screen is hardly ever the point of a serious program. In the previous exercise,
let’s therefore assume that the fact of primeness (or non-primeness) of a number will be used in the rest
of the program. So you want to store this conclusion.

Exercise 1.4. Rewrite the previous exercise with a boolean variable to represent the primeness of the
input number.

Exercise 1.5. Read in an integer r. If it is prime, print a message saying so. If it is not prime, find
integers p ≤ q so that r = p · q and so that p and q are as close together as possible. For instance, for
r = 30 you should print out 5,6, rather than 3,10. You are allowed to use the function sqrt.

1.4 Functions
Above you wrote several lines of code to test whether a number was prime.

Exercise 1.6. Write a function is_prime that has an integer parameter, and returns a boolean cor-
responding to whether the parameter was prime.
int main() {
bool isprime;
isprime = is_prime(13);

Read the number in, and print the value of the boolean.
Does your function have one or two return statements? Can you imagine what the other possibility
looks like? Do you have an argument for or against it?

1.5 While loops

Exercise 1.7. Take your prime number testing function is_prime, and use it to write a program
that prints multiple primes:
• Read an integer how_many from the input, indicating how many (successive) prime numbers

10 Introduction to Scientific Programming

 1.6. Classes and objects

should be printed.
• Print that many successive primes, each on a separate line.
• (Hint: keep a variable number_of_primes_found that is increased whenever a new
prime is found.)

1.6 Classes and objects

Exercise 1.8. Write a class primegenerator that contains:

• Methods number_of_primes_found and nextprime;
• Also write a function isprime that does not need to be in the class.
Your main program should look as follows:
cin >> nprimes;
primegenerator sequence;
while (sequence.number_of_primes_found()<nprimes) {
int number = sequence.nextprime();
cout << "Number " << number << " is prime" << ’\n’;
}

In the previous exercise you defined the primegenerator class, and you made one object of that
class:
primegenerator sequence;

But you can make multiple generators, that all have their own internal data and are therefore independent
of each other.

Exercise 1.9. The Goldbach conjecture says that every even number, from 4 on, is the sum of two
primes p + q. Write a program to test this for the even numbers up to a bound that you read in. Use the
primegenerator class you developed in exercise 1.8Classes and objectsexcounter.1.8.
This is a great exercise for a top-down approach!
1. Make an outer loop over the even numbers e.
2. For each e, generate all primes p.
3. From p + q = e, it follows that q = e − p is prime: test if that q is prime.
For each even number e then print e,p,q, for instance:
The number 10 is 3+7

If multiple possibilities exist, only print the first one you find.
An interesting corollary of the Goldbach conjecture is that each prime (start at 5) is equidistant between
two other primes.

The Goldbach conjecture says that every even number 2n (starting at 4), is the sum of two primes
p + q:

2n = p + q.

Victor Eijkhout 11
1. Prime numbers

Equivalently, every number n is equidistant from two primes:

p+q
n= or q − n = n − p.
2
In particular this holds for each prime number:

∀r prime ∃p,q prime : r = (p + q)/2 is prime.

Exercise 1.10.
Write a program that tests this. You need at least one loop that tests all primes r; for each r you then
need to find the primes p, q that are equidistant to it. Do you use two generators for this, or is one
enough? Do you need three, for p, q, r?
For each r value, when the program finds the p, q values, print the p, q, r triple and move on to the
next r.

1.6.1 Exceptions

Exercise 1.11. Revisit the prime generator class (exercise 1.8Classes and objectsexcounter.1.8) and
let it throw an exception once the candidate number is too large. (You can hardwire this maximum, or
use a limit; section ??.)
Code: Output
[primes] genx:
try {
do { 9931
auto cur = primes.nextprime(); 9941
cout << cur << ’\n’; 9949
} while (true); 9967
} catch ( string s ) { 9973
cout << s << ’\n’; Reached max int
}

1.6.2 Prime number decomposition

Design a class Integer which stores its value as its prime number decomposition. For instance,

180 = 22 · 33 · 5 ⇒ [ 2:2, 3:2, 5:1 ]

You can implement this decomposition itself as a vector, (the i-th location stores the exponent of the i-th
prime) but let’s use a map instead.

Exercise 1.12. Write a constructor of an Integer from an int, and methods as_int / as_string
that convert the decomposition back to something classical. Start by assuming that each prime factor
appears only once.

12 Introduction to Scientific Programming

 1.7. Other

Code: Output
[primes] decomposition26:
Integer i2(2);
cout << i2.as_string() << ": " 2ˆ1 : 2
<< i2.as_int() << ’\n’; 2ˆ1 3ˆ1 : 6

Integer i6(6);
cout << i6.as_string() << ": "
<< i6.as_int() << ’\n’;

Exercise 1.13. Extend the previous exercise to having multiplicity > 1 for the prime factors.
Code: Output
[primes] decomposition180:
Integer i180(180);
cout << i180.as_string() << ": " 2ˆ2 3ˆ2 5ˆ1 : 180
<< i180.as_int() << ’\n’;

Implement addition and multiplication for Integers.

Implement a class Rational for rational numbers, which are implemented as two Integer objects. This
class should have methods for addition and multiplication. Write these through operator overloading if
you’ve learned this.
Make sure you always divide out common factors in the numerator and denominator.

1.7 Other
The following exercise requires std::optional, which you can learn about in section ??.

Exercise 1.14. Write a function first_factor that optionally returns the smallest factor of a given
input.
auto factor = first_factor(number);
if (factor.has_value())
cout << "Found factor: " << factor.value() << ’\n’;

1.8 Eratosthenes sieve

The Eratosthenes sieve is an algorithm for prime numbers that step-by-step filters out the multiples of
any prime it finds.
1. Start with the integers from 2: 2, 3, 4, 5, 6, . . .
2. The first number, 2, is a prime: record it and remove all multiples, giving
3, 5, 7, 9.11, 13, 15, 17 . . .
3. The first remaining number, 3, is a prime: record it and remove all multiples, giving
5, 7, 11, 13, 17, 19, 23, 25, 29 . . .
4. The first remaining number, 5, is a prime: record it and remove all multiples, giving
7, 11, 13, 17, 19, 23, 29, . . .

Victor Eijkhout 13
1. Prime numbers

1.8.1 Arrays implementation

The sieve can be implemented with an array that stores all integers.

Exercise 1.15. Read in an integer that denotes the largest number you want to test. Make an array
of integers that long. Set the elements to the successive integers. Apply the sieve algorithm to find the
prime numbers.

1.8.2 Streams implementation

The disadvantage of using an array is that we need to allocate an array. What’s more, the size is deter-
mined by how many integers we are going to test, not how many prime numbers we want to generate. We
are going to take the idea above of having a generator object, and apply that to the sieve algorithm: we
will now have multiple generator objects, each taking the previous as input and erasing certain multiples
from it.

Exercise 1.16. Write a stream class that generates integers and use it through a pointer.
Code: Output
[sieve] ints:
for (int i=0; i<7; i++)
cout << "Next int: " Next int: 2
<< the_ints->next() << ’\n’; Next int: 3
Next int: 4
Next int: 5
Next int: 6
Next int: 7
Next int: 8

Next, we need a stream that takes another stream as input, and filters out values from it.

Exercise 1.17. Write a class filtered_stream with a constructor

filtered_stream(int filter,shared_ptr<stream> input);

that
1. Implements next, giving filtered values,
2. by calling the next method of the input stream and filtering out values.
Code: Output
[sieve] odds:
auto integers =
make_shared<stream>(); next odd: 3
auto odds = next odd: 5
shared_ptr<stream> next odd: 7
( new filtered_stream(2,integers) ); next odd: 9
for (int step=0; step<5; step++) next odd: 11
cout << "next odd: "
<< odds->next() << ’\n’;

Now you can implement the Eratosthenes sieve by making a filtered_stream for each prime num-
ber.

14 Introduction to Scientific Programming

 1.9. Range implementation

Exercise 1.18. Write a program that generates prime numbers as follows.

• Maintain a current stream, that is initially the stream of prime numbers.
• Repeatedly:
– Record the first item from the current stream, which is a new prime number;
– and set current to a new stream that takes current as input, filtering out multiples
of the prime number just found.

1.9 Range implementation

Exercise 1.19. Make a primes class that can be ranged:

Code: Output
[primes] range:
primegenerator allprimes;
for ( auto p : allprimes ) { 2, 3, 5, 7, 11, 13, 17, 19, 23,
cout << p << ", "; 29, 31, 37, 41, 43, 47, 53,
if (p>100) break; 59, 61, 67, 71, 73, 79, 83,
} 89, 97, 101,
cout << ’\n’;

1.10 User-friendliness
Use the cxxopts package (section ??) to add commandline options to some primality programs.

Exercise 1.20. Take your old prime number testing program, and add commandline options:
• the -h option should print out usage information;
• specifying a single int --test 1001 should print out all primes under that number;
• specifying a set of ints --tests 57,125,1001 should test primeness for those.

Victor Eijkhout 15
1. Prime numbers

16 Introduction to Scientific Programming

Chapter 2

Geometry

In this set of exercises you will write a small ‘geometry’ package: code that manipulates points, lines,
shapes. These exercises mostly use the material of section ??.

2.1 Basic functions

Exercise 2.1. Write a function with (float or double) inputs x, y that returns the distance of point
(x, y) to the origin.
Test the following pairs: 1, 0; 0, 1; 1, 1; 3, 4.

Exercise 2.2. Write a function with inputs x, y, θ that alters x and y corresponding to rotating the
point (x, y) over an angle θ.
 ′   
x cos θ − sin θ x
=
y′ sin θ cos θ y

Your code should behave like:

Code: Output
[geom] rotate:
const float pi = 2*acos(0.0);
float x{1.}, y{0.}; Rotated halfway:
rotate(x,y,pi/4); (0.707107,0.707107)
cout << "Rotated halfway: (" Rotated to the y-axis: (0,1)
<< x << "," << y << ")" << ’\n’;
rotate(x,y,pi/4);
cout << "Rotated to the y-axis: ("
<< x << "," << y << ")" << ’\n’;

2.2 Point class

A class can contain elementary data. In this section you will make a Point class that models cartesian
coordinates and functions defined on coordinates.

17
2. Geometry

Exercise 2.3. Make class Point with a constructor

Point( float xcoordinate, float ycoordinate );

Write the following methods:

• distance_to_origin returns a float.
• angle computes the angle of vector (x, y) with the x-axis.

Exercise 2.4. Extend the Point class of the previous exercise with a method: distance that
computes the distance between this point and another: if p,q are Point objects,
p.distance(q)

computes the distance between them.

Exercise 2.5. Write a method halfway that, given two Point objects p,q, construct the Point
halfway, that is, (p + q)/2:
Point p(1,2.2), q(3.4,5.6);
Point h = p.halfway(q);

You can write this function directly, or you could write functions Add and Scale and combine these.
(Later you will learn about operator overloading.)
How would you print out a Point to make sure you compute the halfway point correctly?

Exercise 2.6. Make a default constructor for the point class:

Point() { /* default code */ }

which you can use as:

Point p;

but which gives an indication that it is undefined:

Code: Output
[geom] linearnan:
Point p3;
cout << "Uninitialized point:" Uninitialized point:
<< ’\n’; Point: nan,nan
p3.printout(); Using uninitialized point:
cout << "Using uninitialized point:" Point: nan,nan
<< ’\n’;
auto p4 = Point(4,5)+p3;
p4.printout();

Hint: see section ??.

Exercise 2.7. Revisit exercise 2.2Basic functionsexcounter.2.2 using the Point class. Your code
should now look like:
newpoint = point.rotate(alpha);

18 Introduction to Scientific Programming

 2.3. Using one class in another

Exercise 2.8. Advanced. Can you make a Point class that can accomodate any number of space
dimensions? Hint: use a vector; section ??. Can you make a constructor where you do not specify
the space dimension explicitly?

2.3 Using one class in another

Exercise 2.9. Make a class LinearFunction with a constructor:

LinearFunction( Point input_p1,Point input_p2 );

and a member function

float evaluate_at( float x );

which you can use as:

LinearFunction line(p1,p2);
cout << "Value at 4.0: " << line.evaluate_at(4.0) << endl;

Exercise 2.10. Make a class LinearFunction with two constructors:

LinearFunction( Point input_p2 );
LinearFunction( Point input_p1,Point input_p2 );

where the first stands for a line through the origin.

Implement again the evaluate function so that
LinearFunction line(p1,p2);
cout << "Value at 4.0: " << line.evaluate_at(4.0) << endl;

Exercise 2.11. Revisit exercises 2.2Basic functionsexcounter.2.2 and 2.7Point classexcounter.2.7, in-
troducing a Matrix class. Your code can now look like
newpoint = point.apply(rotation_matrix);

or
newpoint = rotation_matrix.apply(point);

Can you argue in favour of either one?

Suppose you want to write a Rectangle class, which could have methods such as float Rectangle::area()
or bool Rectangle::contains(Point). Since rectangle has four corners, you could store four
Point objects in each Rectangle object. However, there is redundancy there: you only need three
points to infer the fourth. Let’s consider the case of a rectangle with sides that are horizontal and vertical;
then you need only two points.

Victor Eijkhout 19
2. Geometry

Intended API:
float Rectangle::area();

It would be convenient to store width and height; for

bool Rectangle::contains(Point);

it would be convenient to store bottomleft/topright points.

Exercise 2.12.
1. Make a class Rectangle (sides parallel to axes) with a constructor:
Rectangle(Point botleft,float width,float height);
The logical implementation is to store these quantities. Implement methods:
float area(); float rightedge_x(); float topedge_y();
and write a main program to test these.
2. Add a second constructor
Rectangle(Point botleft,Point topright);
Can you figure out how to use member initializer lists for the constructors?

Exercise 2.13. Make a copy of your solution of the previous exercise, and redesign your class so that
it stores two Point objects. Your main program should not change.

The previous exercise illustrates an important point: for well designed classes you can change the imple-
mentation (for instance motivated by efficiency) while the program that uses the class does not change.

2.4 Is-a relationship

Exercise 2.14. Take your code where a Rectangle was defined from one point, width, and height.
Make a class Square that inherits from Rectangle. It should have the function area defined,
inherited from Rectangle.
First ask yourself: what should the constructor of a Square look like?

Exercise 2.15. Revisit the LinearFunction class. Add methods slope and intercept.
Now generalize LinearFunction to StraightLine class. These two are almost the same except
for vertical lines. The slope and intercept do not apply to vertical lines, so design StraightLine
so that it stores the defining points internally. Let LinearFunction inherit.

2.5 Pointers
The following exercise is a little artificial.

20 Introduction to Scientific Programming

 2.6. More stuff

Exercise 2.16. Make a DynRectangle class, which is constructed from two shared-pointers-to-Point
objects:
auto
origin = make_shared<Point>(0,0),
fivetwo = make_shared<Point>(5,2);
DynRectangle lielow( origin,fivetwo );

Calculate the area, scale the top-right point, and recalculate the area:
Code: Output
[pointer] dynrect:
cout << "Area: " << lielow.area() <<
’\n’; Area: 10
/* ... */ Area: 40
// scale the ‘fivetwo’ point by two
cout << "Area: " << lielow.area() <<
’\n’;

You can base this off the file pointrectangle.cxx in the repository

2.6 More stuff

The Rectangle class stores at most one corner, but may be convenient to sometimes have an array of
all four corners.

Exercise 2.17. Add a method

const vector<Point> &corners()

to the Rectangle class. The result is an array of all four corners, not in any order. Show by a compiler
error that the array can not be altered.

Exercise 2.18. Revisit exercise 2.5 and replace the add and scale functions by overloaded operators.
Hint: for the add function you may need ‘this’.

Victor Eijkhout 21
2. Geometry

22 Introduction to Scientific Programming

Chapter 3

Zero finding

3.1 Root finding by bisection

Figure 3.1: Root finding by interval bisection

For many functions f , finding their zeros, that is, the values x for which f (x) = 0, can not be done
analytically. You then have to resort to numerical root finding schemes. In this project you will develop
gradually more complicated implementations of a simple scheme: root finding by bisection.
In this scheme, you start with two points where the function has opposite signs, and move either the left
or right point to the mid point, depending on what sign the function has there. See figure 3.1.
In section 3.2 we will then look at Newton’s method.
Here we will not be interested in mathematical differences between the methods, though these are im-
portant: we will use these methods to exercise some programming techniques.

3.1.1 Simple implementation

Let’s develop a first implementation step by step. To ensure correctness of our code we will use a Test-
Driven Development (TDD) approach: for each bit of functionality we write a test to ensure its cor-
rectness before we integrate it in the larger code. (For more about TDD, and in particular the Catch2
framework, see section ??.)

23
3. Zero finding

3.1.2 Polynomials
First of all, we need to have a way to represent polynomials. For a polynomial of degree d we need d + 1
coefficients:

f (x) = c0 xd + · · · + cd−1 x1 + cd (3.1)

We implement this by storing the coefficients in a vector<double>. We make the following arbitrary
decisions
1. let the first element of this vector be the coefficient of the highest power, and
2. for the coefficients to properly define a polynomial, this leading coefficient has to be nonzero.
Let’s start by having a fixed test polynomial, provided by a function set_coefficients. For this func-
tion to provide a proper polynomial, it has to satisfy the following test:
TEST_CASE( "coefficients represent polynomial" "[1]") {
vector<double> coefficients = { 1.5, 0., -3 };
REQUIRE( coefficients.size()>0 );
REQUIRE( coefficients.front()!=0. );
}

Exercise 3.1. Write a routine set_coefficients that constructs a vector of coefficients:

vector<double> coefficients = set_coefficients();

and make it satisfy the above conditions.

At first write a hard-coded set of coefficients, then try reading them from the command line.

Above we postulated two conditions that an array of numbers should satisfy to qualify as the coeffi-
cients of a polynomial. Your code will probably be testing for this, so let’s introduce a boolean function
is_proper_polynomial:

• This function returns true if the array of numbers satisfies the two conditions;
• it returns false if either condition is not satisfied.
In order to test your function is_proper_polynomial you should check that
• it recognizes correct polynomials, and
• it fails for improper coefficients that do not properly define a polynomial.

Exercise 3.2. Write a function is_proper_polynomial as described, and write unit tests for it, both
passing and failing:
vector<double> good = /* proper coeficients */ ;
REQUIRE( is_proper_polynomial(good) );
vector<double> notso = /* improper coeficients */ ;
REQUIRE( not is_proper_polynomial(good) );

Next we need polynomial evaluation. We will build a function evaluate_at with the following defini-
tion:
double evaluate_at( const std::vector<double>& coefficients,double x);

24 Introduction to Scientific Programming

 3.1. Root finding by bisection

You can interpret the array of coefficients in (at least) two ways, but with equation (3.1) we proscribed
one particular interpretation.
So we need a test that the coefficients are indeed interpreted with the leading coefficient first, and not
with the leading coefficient last. For instance:
polynomial second( {2,0,1} );
// correct interpretation: 2xˆ2 + 1
REQUIRE( second.is_proper() );
REQUIRE( second.evaluate_at(2) == Catch::Approx(9) );
// wrong interpretation: 1xˆ2 + 2
REQUIRE( second.evaluate_at(2) != Catch::Approx(6) );

(where we have left out the TEST_CASE header.)

Now we write the function that passes these tests:

Exercise 3.3. Write a function evaluate_at which computes

y ← f (x).

and confirm that it passes the above tests.

double evaluate_at( polynomial coefficients,double x);

For bonus points, look up Horner’s rule and implement it.

With the polynomial function implemented, we can start working towards the algorithm.

3.1.3 Left/right search points

Figure 3.2: Setting the initial search points

Suppose x− , x+ are such that

x− < x+ , and f (x− ) · f (x+ ) < 0,

that is, the function values in the left and right point are of opposite sign. Then there is a zero in the
interval (x− , x+ ); see figure 3.1.

Victor Eijkhout 25
3. Zero finding

But how to find these outer bounds on the search?

If the polynomial is of odd degree you can find x− , x+ by going far enough to the left and right from
any two starting points. For even degree there is no such simple algorithm (indeed, there may not be a
zero) so we abandon the attempt.
We start by writing a function is_odd that tests whether the polynomial is of odd degree.

Exercise 3.4. Make the following code work:

if ( not is_odd(coefficients) ) {
cout << "This program only works for odd-degree polynomials\n";
exit(1);
}

You could test the above as:

polynomial second{2,0,1}; // 2xˆ2 + 1
REQUIRE( not is_odd(second) );
polynomial third{3,2,0,1}; // 3xˆ3 + 2xˆ2 + 1
REQUIRE( is_odd(third) );

Now we can find x− , x+ : start with some interval and move the end points out until the function values
have opposite sign.

Exercise 3.5. Write a function find_initial_bounds which computes x− , x+ such that

f (x− ) < 0 < f (x+ ) or f (x+ ) < 0 < f (x− )

How can you compute this test more compactly?

What is a good prototype for the function?
How do move the points far enough out to satisfy this condition?
Since finding a left and right point with a zero in between is not always possible for polynomials of even
degree, we completely reject this case. In the following test we throw an exception (see section ??, in
particularly ??) for polynomials of even degree:
right = left+1;
polynomial second{2,0,1}; // 2xˆ2 + 1
REQUIRE_THROWS( find_initial_bounds(second,left,right) );
polynomial third{3,2,0,1}; // 3xˆ3 + 2xˆ2 + 1
REQUIRE_NOTHROW( find_initial_bounds(third,left,right) );
REQUIRE( left<right );

Make sure your code passes these tests. What test do you need to add for the function values?

3.1.4 Root finding

The root finding process globally looks as follows:
• You start with points x− , x+ where the function has opposite sign; then you know that there is
a zero between them.
• The bisection method for finding this zero looks at the halfway point, and based on the function
value in the mid point:

26 Introduction to Scientific Programming

 3.1. Root finding by bisection

• moves one of the bounds to the mid point, such that the function again has opposite signs in the
left and right search point.
The structure of the code is as follows:
double find_zero( /* something */ ) {
while ( /* left and right too far apart */ ) {
// move bounds left and right closer together
}
return something;
}

Again, we test all the functionality separately. In this case this means that moving the bounds should be
a testable step.

Exercise 3.6. Write a function move_bounds_closer and test it.

void move_bounds_closer
( std::vector<double> coefficients,
double& left,double& right );

Implement some unit tests on this function.

Finally, we put everything together in the top level function find_zero.

Exercise 3.7. Make this call work:

auto zero = find_zero( coefficients, 1.e-8 );
cout << "Found root " << zero
<< " with value " << evaluate_at(coefficients,zero) << ’\n’;

Design unit tests, including on the precision attained, and make sure your code passes them.

3.1.5 Object implementation

Revisit the exercises of section 3.1.1 and introduce a polynomial class that stores the polynomial coef-
ficients. Several functions now become members of this class.

Also update the unit tests.

How can you generalize the polynomial class, for instance to the case of special forms such as (1 + x)n ?

3.1.6 Templating

In the implementations so far we used double for the numerical type. Make a templated version that
works both with float and double.

Can you see a difference in attainable precision between the two types?

Victor Eijkhout 27
3. Zero finding

3.2 Newton’s method

In this section we look at Newton’s method. This is an iterative method for finding zeros of a function f ,
that is, it computes a sequence of values {xn }n , so that f (xn ) → 0. The sequence is defined by

f (xn )
xn+1 = xn −
f ′ (xn )

with x0 arbitrarily chosen.

As a specific case, here we use

f (x) = x2 − n, f ′ (x) = 2x

which has the effect that, if we find an x such that f (x) = 0, we have
√
x = n.

It is of course simple to code this specific case; it should take you about 10 lines. However, we want to
have a general code that takes any two functions f, f ′ , and then uses Newton’s method to find a zero
of f .

3.2.1 Function implementation

Early computers had no hardware for computing a square root. Instead, they used Newton’s method .
√
Suppose you have a value y and you want want to compute x = y. This is equivalent to finding the
zero of

f (x) = x2 − y

where y is fixed. To indicate this dependence on y, we will write fy (x). Newton’s method then finds
the zero by evaluating

xnext = x − fy (x)/fy′ (x)

until the guess is accurate enough, that is, until fy (x) ≈ 0.

Exercise 3.8.
• Write functions f(x,y) and deriv(x,y), that compute fy (x) and fy′ (x) for the definition
of fy above.
• Read a value y and iterate until |f(x, y)| < 10−5 . Print x.
√
• Second part: write a function newton_root that computes y.

3.2.2 Using lambdas

28 Introduction to Scientific Programming

 3.2. Newton’s method

Exercise 3.9. The Newton method (HPC book, section 17) for finding the zero of a function f , that
is, finding the x for which f (x) = 0, can be programmed by supplying the function and its derivative:
double f(double x) { return x*x-2; };
double fprime(double x) { return 2*x; };

and the algorithm:

double x{1.};
while ( true ) {
auto fx = f(x);
cout << "f( " << x << " ) = " << fx << ’\n’;
if (std::abs(fx)<1.e-10 ) break;
x = x - fx/fprime(x);
}

Rewrite this code to use lambda functions for f and fprime.

You can base this off the file newton.cxx in the repository

Next, we make the code modular by writing a general function newton_root, that contains the Newton
method of the previous exercise. Since it has to work for any functions f, f ′ , you have to pass the
objective function and the derivative as arguments:
double root = newton_root( f,fprime );

Exercise 3.10. Rewrite the Newton exercise above to use a function with prototype
double root = newton_root( f,fprime );

Call the function

1. first with the lambda variables you already created;
2. but in a better variant, directly with the lambda expressions as arguments, that is, without
assigning them to variables.

Next we extend functionality, but not by changing the root finding function: instead, we use a more
general way of specifying the objective function and derivative.

Exercise 3.11. Extend the newton exercise to compute roots in a loop:

for (int n=2; n<=8; n++) {
cout << "sqrt(" << n << ") = "
<< find_zero(
/* ... */
)
<< ’\n’;

Without lambdas, you would define a function

double squared_minus_n( double x,int n ) {
return x*x-n; }

Victor Eijkhout 29
3. Zero finding

However, the newton_root function takes a function of only a real argument. Use a capture to make f
dependent on the integer parameter.
Exercise 3.12. You don’t need the gradient as an explicit function: you can approximate it as

f ′ (x) = f (x + h) − f (x) /h

for some value of h.

Write a version of the root finding function
double newton_root( function< double(double)> f )

that uses this. You can use a fixed value h=1e-6. Do not reimplement the whole newton method: instead
create a lambda for the gradient and pass it to the function newton_root you coded earlier.

30 Introduction to Scientific Programming

Chapter 4

Eight queens

A famous exercise for recursive programming is

the eight queens problem: Is it possible to posi-
tion eight queens on a chessboard, so that no two
queens ‘threaten’ each other, according to the rules
of chess?

4.1 Problem statement

The precise statement of the ‘eight queens problem’
is:
• Put eight pieces on an 8 × 8 board, no two
pieces on the same square; so that
• no two pieces are on the same row,
• no two pieces are on the same column, and
• no two pieces are on the same diagonal.
A systematic solution would run:
1. put a piece anywhere in the first row;
2. for each choice in the first row, try all po-
sitions in the second row;
3. for all choices in the first two rows, try all
positions in the third row;
4. when you have a piece in all eight rows,
evaluate the board to see if it satisfies the
condition.

Exercise 4.1. This algorithm will generate all 88 boards. Do you see at least one way to speed up the
search?
Since the number eight is, for now, fixed, you could write this code as an eight-deep loop nest. However
that is not elegant. For example, the only reason for the number 8 in the above exposition is that this is
the traditional size of a chess board. The problem, stated more abstractly as placing n queens on an n × n
board, has solutions for n ≥ 4.

31
4. Eight queens

4.2 Solving the eight queens problem, basic approach

This problem requires you to know about arrays/vectors; chapter ??. Also, see chapter ?? for TDD.
Finally, see section ?? for std::optional.
The basic strategy will be that we fill consecutive rows, by indicating each time which column will be
occupied by the next queen. Using an Object-Oriented (OO) strategy, we make a class ChessBoard, that
holds a partially filled board.
The basic solution strategy is recursive:
• Let current be the current, partially filled board;
• We call current.place_queens() that tries to finish the board;
• However, with recursion, this method only fills one row, and then calls place_queens on this
new board.
So
ChessBoard::place_queens() {
// for c = 1 ... number of columns:
// make a copy of the board
// put a queen in the next row, column c, of the copy
// and call place_queens() on that copy;
// investigate the result.....
}

This routine returns either a solution, or an indication that no solution was possible.
In the next section we will develop a solution systematically in a TDD manner.

4.3 Developing a solution by TDD

We now gradually develop the OO solution to the eight queens problem, using test-driven development.

The board We start by constructing a board, with a constructor that only indicates the size of the
problem:
ChessBoard(int n);

This is a ‘generalized chess board’ of size n × n, and initially it should be empty.

Exercise 4.2. Write this constructor, for an empty board of size n × n.

Note that the implementation of the board is totally up to you. In the following you will get tests for
functionality that you need to satisfy, but any implementation that makes this true is a correct solution.

Bookkeeping: what’s the next row? Assuming that we fill in the board row-by-row, we have an
auxiliary function that returns the next row to be filled:
int next_row_to_be_filled()

This gives us our first simple test: on an empty board, the row to be filled in is row zero.

32 Introduction to Scientific Programming

 4.3. Developing a solution by TDD

Exercise 4.3. Write this method and make sure that it passes the test for an empty board.
TEST_CASE( "empty board","[1]" ) {
constexpr int n=10;
ChessBoard empty(n);
REQUIRE( empty.next_row_to_be_filled()==0 );
}

By the rules of TDD you can actually write the method so that it only satisfies the test for the empty
board. Later, we will test that this method gives the right result after we have filled in a couple of rows,
and then of course your implementation needs to be general.

Place one queen Next, we have a function to place the next queen, whether this gives a feasible board
(meaning that no pieces can capture each other) or not:
void place_next_queen_at_column(int i);

This method should first of all catch incorrect indexing: we assume that the placement routine throws an
exception for invalid column numbers.
ChessBoard::place_next_queen_at_column( int c ) {
if ( /* c is outside the board */ )
throw(1); // or some other exception.

(Suppose you didn’t test for incorrect indexing. Can you construct a simple ‘cheating’ solution at any
size?)

Exercise 4.4. Write this method, and make sure it passes the following test for valid and invalid
column numbers:
REQUIRE_THROWS( empty.place_next_queen_at_column(-1) );
REQUIRE_THROWS( empty.place_next_queen_at_column(n) );
REQUIRE_NOTHROW( empty.place_next_queen_at_column(0) );
REQUIRE( empty.next_row_to_be_filled()==1 );

(From now on we’ll only give the body of the test.)

Now it’s time to start writing some serious stuff.

Is a (partial) board feasible? If you have a board, even partial, you want to test if it’s feasible, meaning
that the queens that have been placed can not capture each other.
The prototype of this method is:
bool feasible()

This test has to work for simple cases to begin with: an empty board is feasible, as is a board with only
one piece.
ChessBoard empty(n);
REQUIRE( empty.feasible() );

Victor Eijkhout 33
4. Eight queens

ChessBoard one = empty;

one.place_next_queen_at_column(0);
REQUIRE( one.next_row_to_be_filled()==1 );
REQUIRE( one.feasible() );

Exercise 4.5. Write the method and make sure it passes these tests.
We shouldn’t only do successfull tests, sometimes referred to as the ‘happy path’ through the code. For
instance, if we put two queens in the same column, the test should fail.

Exercise 4.6. Take the above initial attempt with a queen in position (0, 0), and add another queen in
column zero of the next row. Check that it passes the test:
ChessBoard collide = one;
// place a queen in a ‘colliding’ location
collide.place_next_queen_at_column(0);
// and test that this is not feasible
REQUIRE( not collide.feasible() );

Add a few tests more of your own. (These will not be exercised by the submission script, but you may
find them useful anyway.)

Testing configurations If we want to test the feasibility of non-trivial configurations, it is a good idea
to be able to ‘create’ solutions. For this we need a second type of constructor where we construct a fully
filled chess board from the locations of the pieces.
ChessBoard( int n,vector<int> cols );
ChessBoard( vector<int> cols );

• If the constructor is called with only a vector, this describes a full board.
• Adding an integer parameter indicates the size of the board, and the vector describes only the
rows that have been filled in.

Exercise 4.7. Write these constructors, and test that an explicitly given solution is a feasible board:
ChessBoard five( {0,3,1,4,2} );
REQUIRE( five.feasible() );

For an elegant approach to implementing this, see delegating constructors; section ??.
Ultimately we have to write the tricky stuff.

4.4 The recursive solution method

The main function
optional<ChessBoard> place_queens()

takes a board, empty or not, and tries to fill the remaining rows.
One problem is that this method needs to be able to communicate that, given some initial configuration,
no solution is possible. For this, we let the return type of place_queens be optional<ChessBoard>:

34 Introduction to Scientific Programming

 4.4. The recursive solution method

• if it is possible to finish the current board resulting in a solution, we return that filled board;
• otherwise we return {}, indicating that no solution was possible.
With the recursive strategy discussed in section 4.2, this placement method has roughly the following
structure:
place_queens() {
for ( int col=0; col<n; col++ ) {
ChessBoard next = *this;
// put a queen in column col on the ‘next’ board
// if this is feasible and full, we have a solution
// if it is feasible but no full, recurse
}
}

The line
ChessBoard next = *this;

makes a copy of the object you’re in.

Remark 1 Another approach would be to make a recursive function

bool place_queen( const ChessBoard& current, ChessBoard &next );
// true if possible, false is not

The final step Above you coded the method feasible that tested whether a board is still a candidate
for a solution. Since this routine works for any partially filled board, you also need a method to test if
you’re done.

Exercise 4.8. Write a method

bool filled();

and write a test for it, both positive and negative.

Now that you can recognize solutions, it’s time to write the solution routine.

Exercise 4.9. Write the method

optional<ChessBoard> place_queens()

Because the function place_queens is recursive, it is a little hard to test in its entirety.
We start with a simpler test: if you almost have the solution, it can do the last step.

Exercise 4.10. Use the constructor

ChessBoard( int n,vector<int> cols )

to generate a board that has all but the last row filled in, and that is still feasible. Test that you can find
the solution:
ChessBoard almost( 4, {1,3,0} );

Victor Eijkhout 35
4. Eight queens

auto solution = almost.place_queens();

REQUIRE( solution.has_value() );
REQUIRE( solution->filled() );

Since this test only fills in the last row, it only does one loop, so printing out diagnostics is possible,
without getting overwhelmed in tons of output.

Solutions and non-solutions Now that you have the solution routine, test that it works starting from
an empty board. For instance, confirm there are no 3 × 3 solutions:
TEST_CASE( "no 3x3 solutions","[9]" ) {
ChessBoard three(3);
auto solution = three.place_queens();
REQUIRE( not solution.has_value() );
}

On the other hand, 4 × 4 solutions do exist:

TEST_CASE( "there are 4x4 solutions","[10]" ) {
ChessBoard four(4);
auto solution = four.place_queens();
REQUIRE( solution.has_value() );
}

Exercise 4.11. (Optional) Can you modify your code so that it counts all the possible solutions?

Exercise 4.12. (Optional) How does the time to solution behave as function of n?

36 Introduction to Scientific Programming

 PART II

RESEARCH PROJECTS
Chapter 5

Infectuous disease simulation

This section contains a sequence of exercises that builds up to a somewhat realistic simulation of the
spread of infectious diseases.

5.1 Model design

It is possible to model disease propagation statistically, but here we will build an explicit simulation: we
will maintain an explicit description of all the people in the population, and track for each of them their
status.
We will use a simple model where a person can be:
• sick: when they are sick, they can infect other people;
• susceptible: they are healthy, but can be infected;
• recovered: they have been sick, but no longer carry the disease, and can not be infected for a
second time;
• vaccinated: they are healthy, do not carry the disease, and can not be infected.
In more complicated models a person could be infectious during only part of their illness, or there could
be secondary infections with other diseases, et cetera. We keep it simple here: any sick person is can
infect others while they are sick.
In the exercises below we will gradually develop a somewhat realistic model of how the disease spreads
from an infectious person. We always start with just one person infected. The program will then track
the population from day to day, running indefinitely until none of the population is sick. Since there is
no re-infection, the run will always end.

5.1.1 Other ways of modeling

Instead of capturing every single person in code, a ‘contact network’ model, it is possible to use an
Ordinary Diffential Equation (ODE) approach to disease modeling. You would then model the percent-
age of infected persons by a single scalar, and derive relations for that and other scalars [2].
http://mathworld.wolfram.com/Kermack-McKendrickModel.html
This is known as a ‘compartmental model’, where each of the three SIR states is a compartment: a section
of the population. Both the contact network and the compartmental model capture part of the truth. In

39
5. Infectuous disease simulation

fact, they can be combined. We can consider a country as a set of cities, where people travel between any
pair of cities. We then use a compartmental model inside a city, and a contact network between cities.
In this project we will only use the network model.

5.2 Coding
5.2.1 The basics
We start by writing code that models a single person. The main methods serve to infect a person, and to
track their state. We need to have some methods for inspecting that state.
The intended output looks something like:
On day 10, Joe is susceptible
On day 11, Joe is susceptible
On day 12, Joe is susceptible
On day 13, Joe is susceptible
On day 14, Joe is sick (5 to go)
On day 15, Joe is sick (4 to go)
On day 16, Joe is sick (3 to go)
On day 17, Joe is sick (2 to go)
On day 18, Joe is sick (1 to go)
On day 19, Joe is recovered

Exercise 5.1. Write a Person class with methods:

• status_string() : returns a description of the person’s state as a string;
• update() : update the person’s status to the next day;
• infect(n) : infect a person, with the disease to run for n days;
• is_stable() : return a bool indicating whether the person has been sick and is recovered.
Your main program could for instance look like:
Person joe;

int step = 1;
for ( ; ; step++) {

joe.update();
float bad_luck = (float) rand()/(float)RAND_MAX;
if (bad_luck>.95)
joe.infect(5);

cout << "On day " << step << ", Joe is "
<< joe.status_string() << ’\n’;
if (joe.is_stable())
break;
}

Here is a suggestion how you can model disease status. Use a single integer with the following interpre-
tation:

40 Introduction to Scientific Programming

 5.2. Coding

• healthy but not vaccinated, value 0,

• recovered, value −1,
• vaccinated, value −2,
• and sick, with n days to go before recovery, modeled by value n.
The Person::update method then updates this integer.

Remark 2 Consider a proint of programming style. Now that you’ve modeled the state of a person with
an integer, you can use that as
void infect(n) {
if (state==0)
state = n;
}

But you can also write

bool is_susceptible() {
return state==0;
}
void infect(n) {
if (is_susceptible())
state = n;
}

Which do you prefer and why?

5.2.2 Population
Next we need a Population class. Implement a population as a vector consisting of Person
objects. Initially we only infect one person, and there is no transmission of the disease.
The trace output should look something like:
Size of
population?
In step 1 #sick: 1 : ? ? ? ? ? ? ? ? ? ? + ? ? ? ? ? ? ? ? ?
In step 2 #sick: 1 : ? ? ? ? ? ? ? ? ? ? + ? ? ? ? ? ? ? ? ?
In step 3 #sick: 1 : ? ? ? ? ? ? ? ? ? ? + ? ? ? ? ? ? ? ? ?
In step 4 #sick: 1 : ? ? ? ? ? ? ? ? ? ? + ? ? ? ? ? ? ? ? ?
In step 5 #sick: 1 : ? ? ? ? ? ? ? ? ? ? + ? ? ? ? ? ? ? ? ?
In step 6 #sick: 0 : ? ? ? ? ? ? ? ? ? ? - ? ? ? ? ? ? ? ? ?
Disease ran its course by step 6

Remark 3 Such a display is good for a sanity check on your program behavior. If you include such
displays in your writeup, make sure to use a monospace font, and don’t use a population size that needs
line wrapping. In further testing, you should use large populations, but do not include these displays.

Exercise 5.2. Program a population without infection.

• Write the Population class. The constructor takes the number of people:

Victor Eijkhout 41
5. Infectuous disease simulation

Population population(npeople);

• Write a method that infects a random person:

population.random_infection();

For the ‘random’ part you can use the C language random number generator (section ??), or
the new Standard Template Library (STL) one in section ??.
• Write a method count_infected that counts how many people are infected.
• Write an update method that updates all persons in the population.
• Loop the update method until no people are infected: the Population::update method
should apply Person::update to all person in the population.
Write a routine that displays the state of the popular, using for instance: ? for susceptible, + for infected,
- for recovered.

5.2.3 Contagion
This past exercise was too simplistic: the original patient zero was the only one who ever got sick. Now
let’s incorporate contagion, and investigate the spread of the disease from a single infected person.
We start with a very simple model of infection.

Exercise 5.3. Read in a number 0 ≤ p ≤ 1 representing the probability of disease transmission upon
contact.
population.set_probability_of_transfer(probability);

Incorporate this into the program: in each step the direct neighbours of an infected person can now get
sick themselves. Run a number of simulations with population sizes and contagion probabilities. Are
there cases where people escape getting sick?

Exercise 5.4. Incorporate vaccination: read another number representing the percentage of people
that has been vaccinated. Choose those members of the population randomly.
Describe the effect of vaccinated people on the spread of the disease. Why is this model unrealistic?

5.2.4 Spreading
To make the simulation more realistic, we let every sick person come into contact with a fixed number
of random people every day. This gives us more or less the SIR model ; https://en.wikipedia.
org/wiki/Epidemic_model.
Set the number of people that a person comes into contact with, per day, to 6 or so. (You can also let this
be an upper bound for a random value, but that does not essentially change the simulation.) You have
already programmed the probability that a person who comes in contact with an infected person gets sick
themselves. Again start the simulation with a single infected person.

Exercise 5.5. Code the random interactions. Now run a number of simulations varying
• The percentage of people vaccinated, and
• the chance the disease is transmitted on contact.

42 Introduction to Scientific Programming

 5.2. Coding

Record how long the disease runs through the population. With a fixed number of contacts and proba-
bility of transmission, how is this number of function of the percentage that is vaccinated?
Report this function as a table or graph. Make sure you have enough data points for a meaningful
conclusion. Use a realistic population size. You can also do multiple runs and report the average, to
even out the effect of the random number generator.
Exercise 5.6. Investigate the matter of ‘herd immunity’: if enough people are vaccinated, then some
people who are not vaccinated will still never get sick. Let’s say you want to have this probability
over 95 percent. Investigate the percentage of vaccination that is needed for this as a function of the
contagiousness of the disease.
As in the previous exercise, make sure your data set is large enough.

Remark 4 The screen output you used above is good for sanity checks on small problems. However,
for realistic simulations you have to think what is a realistic population size. If your university campus
is a population where random people are likely to meet each other, what would be a population size to
model that? How about the city where you live?
Likewise, if you test different vaccination rates, what granularity do you use? With increases of 5 or 10
percent you can print all results to you screen, but you may miss things. Don’t be afraid to generate large
amount of data and feed them directly to a graphing program.

5.2.5 Mutation
The Covid years have shown how important mutations of an original virus can be. Next, you can include
mutation in your project. We model this as follows:
• Every so many transmissions, a virus will mutate into a new variant.
• A person who has recovered from one variant is still susceptible to other variants.
• For simplicity assume that each variant leaves a person sick the same number of days, and
• Vaccination is all-or-nothing: one vaccine is enough to protect against all variant;
• On the other hand, having recovered from one variant is not protection against others.
Implementation-wise speaking, we model this as follows. First of all, we need a Disease class, so that
we can infect a person with an explicit virus;
void infect(int);
void infect(Disease);

A Disease object now carries the information such as the chance of transmission, or how a long a person
stays under the weather. Modeling mutation is a little tricky. You could do it as follows:
• There is a global variants counter for new virus variants, and a global transmissions counter.
• Every time a person infects another, the newly infected person gets a new Disease object, with
the current variant, and the transmissions counter is updated.
• There is a parameter that determines after how many transmissions the disease mutates. If there
is a mutation, the global variants counter is updated, and from that point on, every infection
is with the new variant. (Note: this is not very realistic. You are free to come up with a better
model.)
• A each Person object has a vector of variants that they are recovered from; recovery from one
variant only makes them immune from that specific variant, not from others.

Victor Eijkhout 43
5. Infectuous disease simulation

Exercise 5.7. Add mutation to your model. Experiment with the mutation rate: as the mutation rate
increases, the disease should stay in the population longer. Does the relation with vaccination rate
change that you observed before?

5.2.6 Diseases without vaccine: Ebola and Covid-19

This section is optional, for bonus points
The project so far applies to diseases for which a vaccine is available, such as MMR for measles, mumps
and rubella. The analysis becomes different if no vaccine exists, such as is the case for ebola and covid-
19 , as of this writing.
Instead, you need to incorporate ‘social distancing’ into your code: people do not get in touch with
random others anymore, but only those in a very limited social circle. Design a model distance function,
and explore various settings.
The difference between ebola and covid-19 is how long an infection can go unnoticed: the incubation
period . With ebola, infection is almost immediately apparent, so such people are removed from the
general population and treated in a hospital. For covid-19, a person can be infected, and infect others,
for a number of days before they are sequestered from the population.
Add this parameter to your simulation and explore the behavior of the disease as a function of it.

5.3 Ethics
The subject of infectious diseases and vaccination is full of ethical questions. The main one is The
chances of something happening to me are very small, so why shouldn’t I bend the rules a little?. This
reasoning is most often applied to vaccination, where people for some reason or other refuse to get
vaccinated.
Explore this question and others you may come up with: it is clear that everyone bending the rules will
have disastrous consequences, but what if only a few people did this?

5.4 Project writeup and submission

5.4.1 Program files
In the course of this project you have written more than one main program, but some code is shared
between the multiple programs. Organize your code with one file for each main program, and a single
‘library’ file with the class methods.
You can do this two ways:
1. You make a ‘library’ file, say infect_lib.cc, and your main programs each have a line
#include "infect_lib.cc"

This is not the best solution, but it is acceptable for now.

44 Introduction to Scientific Programming

 5.5. Bonus: mathematical analysis

2. The better solution requires you to use separate compilation for building the program, and you
need a header file. You would now have infect_lib.cc which is compiled separately, and
infect_lib.h which is included both in the library file and the main program:
#include "infect_lib.h"

See section ?? for more information.

Submit all source files with instructions on how to build all the main programs. You can put these
instructions in a file with a descriptive name such as README or INSTALL, or you can use a makefile.

5.4.2 Writeup
In the writeup, describe the ‘experiments’ you have performed and the conclusions you draw from them.
The exercises above give you a number of questions to address.
For each main program, include some sample output, but note that this is no substitute for writing out
your conclusions in full sentences.
The exercises in section 5.2.4Spreadingsubsection.5.2.4 ask you to explore the program behavior as a
function of one or more parameters. Include a table to report on the behavior you found. You can use
Matlab or Matplotlib in Python (or even Excell) to plot your data, but that is not required.

5.5 Bonus: mathematical analysis

The SIR model can also be modeled through coupled difference or differential equations.
1. The number Si of susceptible people at time i decreases by a fraction

Si+1 = Si (1 − λi dt)

where λi is the product of the number of infected people and a constant that reflects the number
of meetings and the infectiousness of the disease. We write:

Si+1 = Si (1 − λIi dt)

2. The number of infected people similarly increases by λSi Ii , but it also decreases by people
recovering (or dying):

Ii+1 = Ii (1 + λSi dt − γ dt).

3. Finally, the number of ‘removed’ people equals that last term:

Ri+1 = Ri (1 + γIi ).

Exercise 5.8. Code this scheme. What is the effect of varying dt?

Exercise 5.9. For the disease to become an epidemic, the number of newly infected has to be larger
than the number of recovered. That is,

λSi Ii − γIi > 0 ⇔ Si > γ/λ.

Victor Eijkhout 45
5. Infectuous disease simulation

Can you observe this in your simulations?

The parameter γ has a simple interpretation. Suppose that a person stays ill for δ days before recovering.
If It is relatively stable, that means every day the same number of people get infected as recover, and
therefore a 1/δ fraction of people recover each day. Thus, γ is the reciprocal of the duration of the
infection in a given person.

46 Introduction to Scientific Programming

Chapter 6

Google PageRank

6.1 Basic ideas

We are going to simulate the Internet. In particular, we are going to simulate the Pagerank algorithm by
which Google determines the importance of web pages.
Let’s start with some basic classes:
• A Page contains some information such as its title and a global numbering in Google’s data-
center. It also contains a collection of links.
• We represent a link with a pointer to a Page. Conceivably we could have a Link class, con-
taining further information such as probability of being clicked, or number of times clicked,
but for now a pointer will do.
• Ultimately we want to have a class Web which contains a number of pages and their links. The
web object will ultimately also contain information such as relative importance of the pages.
This application is a natural one for using pointers. When you click on a link on a web page you go from
looking at one page in your browser to looking at another. You could implement this by having a pointer
to a page, and clicking updates the value of this pointer.

Exercise 6.1. Make a class Page which initially just contains the name of the page. Write a method
to display the page. Since we will be using pointers quite a bit, let this be the intended code for testing:
auto homepage = make_shared<Page>("My Home Page");
cout << "Homepage has no links yet:" << ’\n’;
cout << homepage->as_string() << ’\n’;

Next, add links to the page. A link is a pointer to another page, and since there can be any number of
them, you will need a vector of them. Write a method click that follows the link. Intended code:
auto utexas = make_shared<Page>("University Home Page");
homepage->add_link(utexas);
auto searchpage = make_shared<Page>("google");
homepage->add_link(searchpage);
cout << homepage->as_string() << ’\n’;

Exercise 6.2. Add some more links to your homepage. Write a method random_click for the
Page class. Intended code:
for (int iclick=0; iclick<20; iclick++) {

47
6. Google PageRank

auto newpage = homepage->random_click();

cout << "To: " << newpage->as_string() << ’\n’;
}

How do you handle the case of a page without links?

6.2 Clicking around

Exercise 6.3. Now make a class Web which foremost contains a bunch (technically: a vector) of
pages. Or rather: of pointers to pages. Since we don’t want to build a whole internet by hand, let’s
have a method create_random_links which makes a random number of links to random pages.
Intended code:
Web internet(netsize);
internet.create_random_links(avglinks);

Now we can start our simulation. Write a method Web::random_walk that takes a page, and the
length of the walk, and simulates the result of randomly clicking that many times on the current page.
(Current page. Not the starting page.)
Let’s start working towards PageRank. First we see if there are pages that are more popular than others.
You can do that by starting a random walk once on each page. Or maybe a couple of times.

Exercise 6.4. Apart from the size of your internet, what other design parameters are there for your
tests? Can you give a back-of-the-envelope estimation of their effect?

Exercise 6.5. Your first simulation is to start on each page a number of times, and counts where that
lands you. Intended code:
vector<int> landing_counts(internet.number_of_pages(),0);
for ( auto page : internet.all_pages() ) {
for (int iwalk=0; iwalk<5; iwalk++) {
auto endpage = internet.random_walk(page,2*avglinks,tracing);
landing_counts.at(endpage->global_ID())++;
}
}

Display the results and analyze. You may find that you finish on certain pages too many times. What’s
happening? Fix that.

6.3 Graph algorithms

There are many algorithms that rely on gradually traversing the web. For instance, any graph can be
connected . You test that by
• Take an arbitrary vertex v. Make a ‘reachable set’ R ← {v}.
• Now see where you can get from your reachable set:

∀v∈V ∀w neighbour of v : R ← R ∪ {w}

48 Introduction to Scientific Programming

 6.4. Page ranking

• Repeat the previous step until R does not change anymore.

After this algorithm concludes, is R equal to your set of vertices? If so, your graph is called (fully)
connected. If not, your graph has multiple connected components.

Exercise 6.6. Code the above algorithm, keeping track of how many steps it takes to reach each
vertex w. This is the Single Source Shortest Path algorithm (for unweighted graphs).
The diameter is defined as the maximal shortest path. Code this.

6.4 Page ranking

The Pagerank algorithm now asks, if you keep clicking randomly, what is the distribution of how likely
you are to wind up on a certain page. The way we calculate that is with a probability distribution: we
assign a probability to each page so that the sum of all probabilities is one. We start with a random
distribution:

Code: Output
[google] pdfsetup:
ProbabilityDistribution
Initial distribution:
random_state(internet.number_of_pages()); 0:0.00, 1:0.02, 2:0.07,
random_state.set_random(); 3:0.05, 4:0.06, 5:0.08,
cout << "Initial distribution: " << 6:0.04, 7:0.04, 8:0.04,
random_state.as_string() << ’\n’; 9:0.01, 10:0.07, 11:0.05,
12:0.01, 13:0.04,
14:0.08, 15:0.06,
16:0.10, 17:0.06,
18:0.11, 19:0.01,

Exercise 6.7. Implement a class ProbabilityDistribution, which stores a vector of floating

point numbers. Write methods for:
• accessing a specific element,
• setting the whole distribution to random, and
• normalizing so that the sum of the probabilities is 1.
• a method rendering the distribution as string could be useful too.

Next we need a method that given a probability distribution, gives you the new distribution corresponding
to performing a single click. (This is related to Markov chains; see HPC book, section 9.2.1.)

Exercise 6.8. Write the method

ProbabilityDistribution Web::globalclick
( ProbabilityDistribution currentstate );

Test it by
• start with a distribution that is nonzero in exactly one page;
• print the new distribution corresponding to one click;
• do this for several pages and inspect the result visually.

Victor Eijkhout 49
6. Google PageRank

Then start with a random distribution and run a couple of iterations. How fast does the process converge?
Compare the result to the random walk exercise above.
Exercise 6.9. In the random walk exercise you had to deal with the fact that some pages have no
outgoing links. In that case you transitioned to a random page. That mechanism is lacking in the
globalclick method. Figure out a way to incorporate this.

Let’s simulate some simple ‘search engine optimization’ trick.

Exercise 6.10. Add a page that you will artificially made look important: add a number of pages (for
instance four times the average number of links) that all link to this page, but no one links to them.
(Because of the random clicking they will still sometimes be reached.)
Compute the rank of the artificially hyped page. Did you manage to trick Google into ranking this page
high?

6.5 Graphs and linear algebra

The probability distribution is essentially a vector. You can also represent the web as a matrix W with
wij = 1 if page i links to page j. How can you interpret the globalclick method in these terms?

Exercise 6.11. Add the matrix representation of the Web object and reimplement the globalclick
method. Test for correctness.
Do a timing comparison.

The iteration you did above to find a stable probability distribution corresponds to the ‘power method’
in linear algebra. Look up the Perron-Frobenius theory and see what it implies for page ranking.

50 Introduction to Scientific Programming

Chapter 7

Redistricting

In this project you can explore ‘gerrymandering’, the strategic drawing of districts to give a minority
population a majority of districts1 .

7.1 Basic concepts

We are dealing with the following concepts:
• A state is divided into census districts, which are given. Based on census data (income, ethnic-
ity, median age) one can usually make a good guess as to the overall voting in such a district.
• There is a predetermined number of congressional districts, each of which consists of census
districts. A congressional district is not a random collection: the census districts have to be
contiguous.
• Every couple of years, to account for changing populations, the district boundaries are redrawn.
This is known as redistricting.
There is considerable freedom in how redistricting is done: by shifting the boundaries of the (congres-
sional) districts it is possible to give a population that is in the overall minority a majority of districts.
This is known as gerrymandering.
For background reading, see https://redistrictingonline.org/.
To do a small-scale computer simulation of gerrymandering, we make some simplifying assumption.
• First of all, we dispense with census district: we assume that a district consists directly of voters,
and that we know their affiliation. In practice one relies on proxy measures (such as income and
education level) to predict affiliation.
• Next, we assume a one-dimensional state. This is enough to construct examples that bring out
the essence of the problem:
Consider a state of five voters, and we designate their votes as AAABB. Assign-
ing them to three (contiguous) districts can be done as AAA|B|B, which has one
‘A’ district and two ‘B’ districts.
• We also allow districts to be any positive size, as long as the number of districts is fixed.

1. This project is obviously based on the Northern American political system. Hopefully the explanations here are clear
enough. Please contact the author if you know of other countries that have a similar system.

51
7. Redistricting

7.2 Basic functions

7.2.1 Voters
We dispense with census districts, expressing everything in terms of voters, for which we assume a
known voting behaviour. Hence, we need a Voter class, which will record the voter ID and party
affiliation. We assume two parties, and leave an option for being undecided.

Exercise 7.1. Implement a Voter class. You could for instance let ±1 stand for A/B, and 0 for
undecided.
cout << "Voter 5 is positive:" << ’\n’;
Voter nr5(5,+1);
cout << nr5.print() << ’\n’;

cout << "Voter 6 is negative:" << ’\n’;

Voter nr6(6,-1);
cout << nr6.print() << ’\n’;

cout << "Voter 7 is weird:" << ’\n’;

Voter nr7(7,3);
cout << nr7.print() << ’\n’;

7.2.2 Populations

Exercise 7.2. Implement a District class that models a group of voters.

• You probably want to create a district out of a single voter, or a vector of them. Having a
constructor that accepts a string representation would be nice too.
• Write methods majority to give the exact majority or minority, and lean that evaluates
whether the district overall counts as A part or B party.
• Write a sub method to creates subsets.
District District::sub(int first,int last);

• For debugging and reporting it may be a good idea to have a method

string District::print();

52 Introduction to Scientific Programming

 7.2. Basic functions

Code: Output
[gerry] district:
cout << "Making district with one B
voter" << ’\n’; Making district with one B voter
Voter nr5(5,+1); .. size: 1
District nine( nr5 ); .. lean: 1
cout << ".. size: " << nine.size() <<
’\n’; Making district ABA
cout << ".. lean: " << nine.lean() << .. size: 3
’\n’; .. lean: -1
/* ... */
cout << "Making district ABA" << ’\n’;
District nine( vector<Voter>
{ {1,-1},{2,+1},{3,-1}
} );
cout << ".. size: " << nine.size() <<
’\n’;
cout << ".. lean: " << nine.lean() <<
’\n’;

Exercise 7.3. Implement a Population class that will initially model a whole state.
Code: Output
[gerry] population:
string pns( "-++--" );
Population some(pns); Population from string -++--
cout << "Population from string " << pns .. size: 5
<< ’\n’; .. lean: -1
cout << ".. size: " << some.size() << sub population 1--3
’\n’; .. size: 2
cout << ".. lean: " << some.lean() << .. lean: 1
’\n’;
Population group=some.sub(1,3);
cout << "sub population 1--3" << ’\n’;
cout << ".. size: " << group.size() <<
’\n’;
cout << ".. lean: " << group.lean() <<
’\n’;

In addition to an explicit creation, also write a constructor that specifies how many people and what the
majority is:
Population( int population_size,int majority,bool trace=false )

Use a random number generator to achieve precisely the indicated majority.

7.2.3 Districting
The next level of complication is to have a set of districts. Since we will be creating this incrementally,
we need some methods for extending it.

Exercise 7.4. Write a class Districting that stores a vector of District objects. Write
size and lean methods:

Victor Eijkhout 53
7. Redistricting

Code: Output
[gerry] gerryempty:
cout << "Making single voter population
B" << ’\n’; Making single voter population B
Population people( vector<Voter>{ .. size: 1
Voter(0,+1) } ); .. lean: 1
cout << ".. size: " << people.size() << Start with empty districting:
’\n’; .. number of districts: 0
cout << ".. lean: " << people.lean() <<
’\n’;

Districting gerry;
cout << "Start with empty districting:"
<< ’\n’;
cout << ".. number of districts: " <<
gerry.size() << ’\n’;

Exercise 7.5. Write methods to extend a Districting:

cout << "Add one B voter:" << ’\n’;
gerry = gerry.extend_with_new_district( people.at(0) );
cout << ".. number of districts: " << gerry.size() << ’\n’;
cout << ".. lean: " << gerry.lean() << ’\n’;
cout << "add A A:" << ’\n’;
gerry = gerry.extend_last_district( Voter(1,-1) );
gerry = gerry.extend_last_district( Voter(2,-1) );
cout << ".. number of districts: " << gerry.size() << ’\n’;
cout << ".. lean: " << gerry.lean() << ’\n’;

cout << "Add two B districts:" << ’\n’;

gerry = gerry.extend_with_new_district( Voter(3,+1) );
gerry = gerry.extend_with_new_district( Voter(4,+1) );
cout << ".. number of districts: " << gerry.size() << ’\n’;
cout << ".. lean: " << gerry.lean() << ’\n’;

7.3 Strategy
Now we need a method for districting a population:
Districting Population::minority_rules( int ndistricts );

Rather than generating all possible partitions of the population, we take an incremental approach (this is
related to the solution strategy called dynamic programming):
• The basic question is to divide a population optimally over n districts;
• We do this recursively by first solving a division of a subpopulation over n − 1 districts,
• and extending that with the remaining population as one district.
This means that you need to consider all the ways of having the ‘remaining’ population into one district,
and that means that you will have a loop over all ways of splitting the population, outside of your
recursion; see figure 7.1Multiple ways of splitting a populationfigure.7.1.
• For all p = 0, . . . n − 1 considering splitting the state into 0, . . . , p − 1 and p, . . . , n − 1.
• Use the best districting of the first group, and make the last group into a single district.

54 Introduction to Scientific Programming

 7.3. Strategy

Figure 7.1: Multiple ways of splitting a population

• Keep the districting that gives the strongest minority rule, over all values of p.
You can now realize the above simple example:
AAABB => AAA|B|B

Exercise 7.6. Implement the above scheme.

Code: Output
[gerry] district5:
Population five("+++--");
cout << "Redistricting population: " << Redistricting population:
’\n’ [0:+,1:+,2:+,3:-,4:-,]
<< five.print() << ’\n’; .. majority rule: 1
cout << ".. majority rule: " [3[0:+,1:+,2:+,],[3:-,],[4:-,],]
<< five.rule() << ’\n’; .. minority rule: -1
int ndistricts{3};
auto gerry =
five.minority_rules(ndistricts);
cout << gerry.print() << ’\n’;
cout << ".. minority rule: "
<< gerry.rule() << ’\n’;

Victor Eijkhout 55
7. Redistricting

Note: the range for p given above is not quite correct: for instance, the initial part of the population
needs to be big enough to accomodate n − 1 voters.
Exercise 7.7. Test multiple population sizes; how much majority can you give party B while still
giving party A a majority.

7.4 Efficiency: dynamic programming

If you think about the algorithm you just implemented, you may notice that the districtings of the initial
parts get recomputed quite a bit. A strategy for optimizing for this is called memoization.

Exercise 7.8. Improve your implementation by storing and reusing results for the initial sub-populations.

In a way, we solved the program backward: we looked at making a district out of the last so-many voters,
and then recursively solving a smaller problem for the first however-many voters. But in that process, we
decided what is the best way to assign districts to the first 1 voter, first 2, first 3, et cetera. Actually, for
more than one voter, say five voters, we found the result on the best attainable minority rule assigning
these five voters to one, two, three, four districts.
The process of computing the ‘best’ districting forward, is known as dynamic programming. The fun-
damental assumption here is that you can use intermediate results and extend them, without having to
reconsider the earlier problems.
Consider for instance that you’ve considered districting ten voters over up to five districts. Now the
majority for eleven voters and five districts is the minimum of
• ten voters and five districts, and the new voter is added to the last district; or
• ten voters and four districts, and the new voter becomes a new district.

Exercise 7.9. Code a dynamic programming solution to the redistricting problem.

7.5 Extensions
The project so far has several simplifying assumptions.
• Congressional districts need to be approximately the same size. Can you put a limit on the ratio
between sizes? Can the minority still gain a majority?

Exercise 7.10. The biggest assumption is of course that we considered a one-dimensional state. With
two dimensions you have more degrees of freedom of shaping the districts. Implement a two-dimensional
scheme; use a completely square state, where the census districts form a regular grid. Limit the shape
of the congressional districts to be convex.

The efficiency gap is a measure of how ‘fair’ a districting of a state is.

Exercise 7.11. Look up the definition of efficiency gap (and ‘wasted votes’), and implement it in your
code.

56 Introduction to Scientific Programming

 7.6. Ethics

7.6 Ethics
The activity of redistricting was intended to give people a fair representation. In its degenerate form of
Gerrymandering this concept of fairness is violated because the explicit goal is to give the minority a
majority of votes. Explore ways that this unfairness can be undone.
In your explorations above, the only characteristic of a voter was their preference for party A or B.
However, in practice voters can be considered part of communities. The Voting Rights Act is concerned
about ‘minority vote dilution’. Can you show examples that a color-blind districting would affect some
communities negatively?

Victor Eijkhout 57
7. Redistricting

58 Introduction to Scientific Programming

Chapter 8

Amazon delivery truck scheduling

This section contains a sequence of exercises that builds up to a simulation of delivery truck scheduling.

8.1 Problem statement

Scheduling the route of a delivery truck is a well-studied problem. For instance, minimizing the total
distance that the truck has to travel corresponds to the Traveling Salesman Problem (TSP). However, in
the case of Amazon delivery truck scheduling the problem has some new aspects:
• A customer is promised a window of days when delivery can take place. Thus, the truck can
split the list of places into sublists, with a shorter total distance than going through the list in
one sweep.
• Except that Amazon prime customers need their deliveries guaranteed the next day.

8.2 Coding up the basics

Before we try finding the best route, let’s put the basics in place to have any sort of route at all.

8.2.1 Address list

You probably need a class Address that describes the location of a house where a delivery has to be
made.
• For simplicity, let give a house (i, j) coordinates.
• We probably need a distance function between two addresses. We can either assume that we
can travel in a straight line between two houses, or that the city is build on a grid, and you can
apply the so-called Manhattan distance.
• The address may also require a field recording the last possible delivery date.

Exercise 8.1. Code a class Address with the above functionality, and test it.

59
8. Amazon delivery truck scheduling

60 Introduction to Scientific Programming

 8.2. Coding up the basics

Code: Output
[amazon] address:
Address one(1.,1.),
two(2.,2.); Address
cerr << "Distance: " Distance: 1.41421
<< one.distance(two) .. address
<< ’\n’; Address 1 should be closest to
the depot. Check: 1

Route from depot to depot:

(0,0) (2,0) (1,0) (3,0)
(0,0)
has length 8: 8
Greedy scheduling: (0,0) (1,0)
(2,0) (3,0) (0,0)
should have length 6: 6

Square5
Travel in order: 24.1421
Square route: (0,0) (0,5)
(5,5) (5,0) (0,0)
has length 20
.. square5

Original list: (0,0) (-2,0)

(-1,0) (1,0) (2,0) (0,0)
length=8
flip middle two addresses:
(0,0) (-2,0) (1,0) (-1,0)
(2,0) (0,0)
length=12
better: (0,0) (1,0) (-2,0)
(-1,0) (2,0) (0,0)
length=10

Hundred houses
Route in order has length
25852.6
TSP based on mere listing has
length: 2751.99 over naive
25852.6
Single route has length: 2078.43
.. new route accepted with
length 2076.65
Final route has length 2076.65
over initial 2078.43
TSP route has length 1899.4
over initial 2078.43

Two routes
Route1: (0,0) (2,0) (3,2)
(2,3) (0,2) (0,0)
route2: (0,0) (3,1) (2,1)
(1,2) (1,3) (0,0)
total length 19.6251
start with 9.88635,9.73877
Pass 0
.. down to 9.81256,8.57649
Pass 1
Victor Eijkhout Pass 2 61
Pass 3
Pass 4
TSP Route1: (0,0) (3,1) (3,2)
(2,3) (0,2) (0,0)
route2: (0,0) (2,0) (2,1)
(1,2) (1,3) (0,0)
total length 18.389
8. Amazon delivery truck scheduling

Next we need a class AddressList that contains a list of addresses.

Exercise 8.2. Implement a class AddressList; it probably needs the following methods:
• add_address for constructing the list;
• length to give the distance one has to travel to visit all addresses in order;
• index_closest_to that gives you the address on the list closests to another address, presum-
ably not on the list.

8.2.2 Add a depot

Next, we model the fact that the route needs to start and end at the depot, which we put arbitrarily at
coordinates (0, 0). We could construct an AddressList that has the depot as first and last element, but
that may run into problems:
• If we reorder the list to minimize the driving distance, the first and last elements may not stay
in place.
• We may want elements of a list to be unique: having an address twice means two deliveries at
the same address, so the add_address method would check that an address is not already in
the list.
We can solve this by making a class Route, which inherits from AddressList, but the methods of which
leave the first and last element of the list in place.

8.2.3 Greedy construction of a route

Next we need to construct a route. Rather than solving the full TSP, we start by employing a greedy
search strategy:
Given a point, find the next point by some local optimality test, such as shortest dis-
tance. Never look back to revisit the route you have constructed so far.
Such a strategy is likely to give an improvement, but most likely will not give the optimal route.
Let’s write a method
Route::Route greedy_route();

that constructs a new address list, containing the same addresses, but arranged to give a shorter length to
travel.

Exercise 8.3. Write the greedy_route method for the AddressList class.
1. Assume that the route starts at the depot, which is located at (0, 0). Then incrementally con-
struct a new list by:
2. Maintain an Address variable we_are_here of the current location;
3. repeatedly find the address closest to we_are_here.
Extend this to a method for the Route class by working on the subvector that does not contain the final
element.
Test it on this example:

62 Introduction to Scientific Programming

 8.3. Optimizing the route

Code: Output
[amazon] square5:
Route deliveries;
deliveries.add_address( Address(0,5) Travel in order: 24.1421
); Square route: (0,0) (0,5)
deliveries.add_address( Address(5,0) (5,5) (5,0) (0,0)
); has length 20
deliveries.add_address( Address(5,5)
);
cerr << "Travel in order: " <<
deliveries.length() << ’\n’;
assert( deliveries.size()==5 );
auto route =
deliveries.greedy_route();
assert( route.size()==5 );
auto len = route.length();
cerr << "Square route: " <<
route.as_string()
<< "\n has length " << len <<
’\n’;

Reorganizing a list can be done in a number of ways.

• First of all, you can try to make the changes in place. This runs into the objection that maybe
you want to save the original list; also, while swapping two elements can be done with the
insert and erase methods, more complicated operations are tricky.
• Alternatively, you can incrementally construct a new list. Now the main problem is to keep
track of which elements of the original have been processed. You could do this by giving each
adress a boolean field done, but you could also make a copy of the input list, and remove the
elements that have been processed. For this, study the erase method for vector objects.

8.3 Optimizing the route

The above suggestion of each time finding the closest address is known as a greedy search strategy. It
does not give you the optimal solution of the TSP. Finding the optimal solution of the TSP is hard to
program – you could do it recursively – and takes a lot of time as the number of addresses grows. In fact,
the TSP is probably the most famous of the class of NP-hard problems, which are generally believed to
have a running time that grows faster than polynomial in the problem size.

Figure 8.1: Illustration of the ‘opt2’ idea of reversing part of a path

However, you can approximate the solution heuristically. One method, the Kernighan-Lin algorithm [8],
is based on the opt2 idea: if you have a path that ‘crosses itself’, you can make it shorter by reversing

Victor Eijkhout 63
8. Amazon delivery truck scheduling

part of it. Figure 8.1Illustration of the ‘opt2’ idea of reversing part of a pathfigure.8.1 shows that the
path 1 − −2 − −3 − −4 can be made shorter by reversing part of it, giving 1 − −3 − −2 − −4.
Since recognizing where a path crosses itself can be hard, or even impossible for graphs that don’t have
Cartesian coordinates associated, we adopt a scheme:
for all nodes m<n on the path [1..N]:
make a new route from
[1..m-1] + [m--n].reversed + [n+1..N]
if the new route is shorter, keep it

Exercise 8.4. Code the opt2 heuristic: write a method to reverse part of the route, and write the loop
that tries this with multiple starting and ending points. Try it out on some simple test cases to convince
you that your code works as intended.

Exercise 8.5. What is the runtime complexity of this heuristic solution?

Exercise 8.6. Earlier you had programmed the greedy heuristic. Compare the improvement you get
from the opt2 heuristic, starting both with the given list of addresses, and with a greedy traversal of it.

8.4 Multiple trucks

If we introduce multiple delivery trucks, we get the ‘Multiple Traveling Salesman Problem’ [3]. With
this we can module both the cases of multiple trucks being out on delivery on the same day, or one truck
spreading deliveries over multiple days. For now we don’t distinguish between the two.
The first question is how to divide up the addresses.
1. We could split the list in two, using some geometric test. This is a good model for the case
where multiple trucks are out on the same day. However, if we use this as a model for the same
truck being out on multiple days, we are missing the fact that new addresses can be added on
the first day, messing up the neatly separated routes.
2. Thus it may in fact be reasonable to assume that all trucks get an essentially random list of
addresses.
Can we extend the opt2 heuristic to the case of multiple paths? For inspiration take a look at figure 8.2Ex-
tending the ‘opt2’ idea to multiple pathsfigure.8.2: instead of modifying one path, we could switch bits
out bits between one path and another. When you write the code, take into account that the other path
may be running backwards! This means that based on split points in the first and second path you know
have four resulting modified paths to consider.

Exercise 8.7. Write a function that optimizes two paths simultaneously using the multi-path version
of the opt2 heuristic. For a test case, see figure 8.3Multiple paths test casefigure.8.3.
You have quite a bit of freedom here:
• The start points of the two segments should be chosen independently;
• the lengths can be chosen independently, but need not; and finally
• each segment can be reversed.

64 Introduction to Scientific Programming

 8.5. Amazon prime

Figure 8.2: Extending the ‘opt2’ idea to multiple paths

More flexibility also means a longer runtime of your program. Does it pay off? Do some tests and report
results.
Based on the above description there will be a lot of code duplication. Make sure to introduce functions
and methods for various operations.

8.5 Amazon prime

In section 8.4Multiple truckssection.8.4 you made the assumption that it doesn’t matter on what day a
package is delivered. This changes with Amazon prime, where a package has to be delivered guaranteed
on the next day.

Exercise 8.8. Explore a scenario where there are two trucks, and each have a number of addresses
that can not be exchanged with the other route. How much longer is the total distance? Experiment with
the ratio of prime to non-prime addresses.

8.6 Dynamicism
So far we have assumed that the list of addresses to be delivered to is given. This is of course not true:
new deliveries will need to be scheduled continuously.

Exercise 8.9. Implement a scenario where every day a random number of new deliveries is added to
the list. Explore strategies and design choices.

Victor Eijkhout 65
8. Amazon delivery truck scheduling

Figure 8.3: Multiple paths test case

8.7 Ethics
People sometimes criticize Amazon’s labor policies, including regarding its drivers. Can you make any
observations from your simulations in this respect?

66 Introduction to Scientific Programming

Chapter 9

High performance linear algebra

Linear algebra operations such as the matrix-matrix product are easy to code in a naive way. However,
this does not lead to high performance. In these exercises you will explore the basics of a strategy for
high performance.

9.1 Mathematical preliminaries

The matrix-matrix product C ← A · B is defined as
X
∀ij : cij ← aik bkj .
k

Straightforward code for this would be:

for (i=0; i<a.m; i++)
for (j=0; j<b.n; j++)
s = 0;
for (k=0; k<a.n; k++)
s += a[i,k] * b[k,j];
c[i,j] = s;

However, this is not the only way to code this operation. The loops can be permuted, giving a total of six
implementations.

Exercise 9.1. Code one of the permuted algorithms and test its correctness. If the reference algorithm
above can be said to be ‘inner-product based’, how would you describe your variant?
Yet another implementation is based on a block partitioning. Let A, B, C be split on 2 × 2 block form:
     
A11 A12 B11 B12 C11 C12
A= , B= , C=
A21 A22 B21 B22 C21 C22
Then
C11 =A11 B11 + A12 B21 ,
C12 =A11 B12 + A12 B22 ,
(9.1)
C21 =A21 B11 + A22 B21 ,
C22 =A21 B12 + A22 B22

(Convince yourself that this actually computes the same product C = A · B.)

67
9. High performance linear algebra

9.2 Matrix storage

The simplest way to store an M × N matrix is as an array of length M N . Inside this array we can decide
to store the rows end-to-end, or the columns. While this decision is obviously of practical importance for
a library, from a point of performance it makes no difference.

Remark 5 Historically, linear algebra software such as the Basic Linear Algebra Subprograms (BLAS)
has used columnwise storage, meaning that the location of an element (i, j) is computed as i + j · M
(we will use zero-based indexing throughout this project, both for code and mathematical expressions.)
The reason for this stems from the origins of the BLAS in the Fortran language, which uses column-
major ordering of array elements. On the other hand, static arrays (such as x[5][6][7]) in the C/C++
languges have row-major ordering, where element (i, j) is stored in location j + i · N .

Above, you saw the idea of block algorithms, which requires taking submatrices. For efficiency, we don’t
want to copy elements into a new array, so we want the submatrix to correspond to a subarray.
Now we have a problem: only a submatrix that consists of a sequence of columns is contiguous. For
this reason, linear algbra software treats each matrix like a submatrix, described by three parameters
M, N, LDA, where ‘LDA’ stands for ‘leading dimension of A’ (see BLAS [6], and Lapack [1]). This is

Figure 9.1: Submatrix out of a matrix, with m,n,lda of the submatrix indicated

illustrated in figure 9.1Submatrix out of a matrix, with m,n,lda of the submatrix indicatedfigure.9.1.

Exercise 9.2. In terms of M, N, LDA, what is the location of the (i, j) element?
Implementationwise we also have a problem. If we use std::vector for storage, it is not possible to take
subarrays, since C++ insists that a vector has its own storage. The solution is to use span; section ??.
We could have two types of matrices: top level matrices that store a vector<double>, and submatrices
that store a span<double>, but that is a lot of complication. It could be done using std::variant
(section ??), but let’s not.

68 Introduction to Scientific Programming

 9.2. Matrix storage

Instead, let’s adopt the following idiom, where we create a vector at the top level, and then create matrices
from its memory.
// example values for M,LDA,N
M = 2; LDA = M+2; N = 3;
// create a vector to contain the data
vector<double> one_data(LDA*N,1.);
// create a matrix using the vector data
Matrix one(M,LDA,N,one_data.data());

(If you have not previously programmed in C, you need to get used to the double* mechanism. Read up
on section ??.)

Exercise 9.3. Start implementing the Matrix class with a constructor

Matrix::Matrix(int m,int lda,int n,double *data)

and private data members:

private:
int m,n,lda;
span<double> data;

Write a method
double& Matrix::at(int i,int j);

that you can use as a safe way of accessing elements.

Let’s start with simple operations.

Exercise 9.4. Write a method for adding matrices. Test it on matrices that have the same M, N , but
different LDA.

Use of the at method is great for debugging, but it is not efficient. Use the preprocessor (chapter ??) to
introduce alternatives:
#ifdef DEBUG
c.at(i,j) += a.at(i,k) * b.at(k,j)
#else
cdata[ /* expression with i,j */ ] += adata[ ... ] * bdata[ ... ]
#endif

where you access the data directly with

auto get_double_data() {
double *adata;
adata = data.data();
return adata;
};

Exercise 9.5. Implement this. Use a cpp #define macro for the optimized indexing expression. (See
section ??.)

Victor Eijkhout 69
9. High performance linear algebra

9.2.1 Submatrices

Next we need to support constructing actual submatrices. Since we will mostly aim for decomposition
in 2 × 2 block form, it is enough to write four methods:
Matrix Left(int j);
Matrix Right(int j);
Matrix Top(int i);
Matrix Bot(int i);

where, for instance, Left(5) gives the columns with j < 5.

Exercise 9.6. Implement these methods and test them.

9.3 Multiplication
You can now write a first multiplication routine, for instance with a prototype
void Matrix::MatMult( Matrix& other,Matrix& out );

Alternatively, you could write

Matrix Matrix::MatMult( Matrix& other );

but we want to keep the amount of creation/destruction of objects to a minimum.

9.3.1 One level of blocking

Next, write
void Matrix::BlockedMatMult( Matrix& other,Matrix& out );

which uses the 2 × 2 form above.

9.3.2 Recursive blocking

The final step is to make the blocking recursive.

Exercise 9.7. Write a method

void RecursiveMatMult( Matrix& other,Matrix& out );

which
• Executes the 2 × 2 block product, using again RecursiveMatMult for the blocks.
• When the block is small enough, use the regular MatMult product.

70 Introduction to Scientific Programming

 9.4. Performance issues

9.4 Performance issues

If you experiment a little with the cutoff between the regular and recursive matrix-matrix product, you
see that you can get good factor of performance improvement. Why is this?
The matrix-matrix product is a basic operation in scientific computations, and much effort has been put
into optimizing it. One interesting fact is that it is just about the most optimizable operation under the
sum. The reason for this is, in a nutshell, that it involves O(N 3 ) operations on O(N 2 ) data. This means
that, in principle each element fetched will be used multiple times, thereby overcoming the memory
bottleneck .

Exercise 9.8. Read up on cache memory, and argue that the naive matrix-matrix product implemen-
tation is unlikely actually to reuse data.
Explain why the recursive strategy does lead to data reuse.
Above, you set a cutoff point for when to switch from the recursive to the regular product.

Exercise 9.9. Argue that continuing to recurse will not have much benefit once the product is con-
tained in the cache. What are the cache sizes of your processor?
Do experiments with various cutoff points. Can you relate this to the cache sizes?

9.4.1 Parallelism (optional)

The four clauses of equation 9.1Mathematical preliminariesequation.9.1.1 target independent areas in
the C matrix, so they could be executed in parallel on any processor that has at least four cores.
Explore the OpenMP library to parallelize the BlockedMatMult.

9.4.2 Comparison (optional)

The final question is: how close are you getting to the best possible speed? Unfortunately you are still a
way off. You can explore that as follows.
Your computer is likely to have an optimized implementation, accessible through:
#include <cblas.h>

cblas_dgemm
( CblasColMajor, CblasNoTrans, CblasNoTrans,
m,other.n,n, alpha,adata,lda,
bdata,other.lda,
beta,cdata,out.lda);

which computes C ← αA · B + βC.

Exercise 9.10. Use another cpp conditional to implement MatMult through a call to cblas_dgemm.
What performance do you now get?
You see that your recursive implementation is faster than the naive one, but not nearly as fast as the
CBlas one. This is because
• the CBlas implementation is probably based on an entirely different strategy [5], and
• it probably involves a certain amount of assembly coding.

Victor Eijkhout 71
9. High performance linear algebra

72 Introduction to Scientific Programming

Chapter 10

Graph algorithms

In this project you will explore some common graph algorithms, and their various possible implementa-
tions. The main theme here will be that the common textbook exposition of algorithms is not necessarily
the best way to phrase them computationally.
As background knowledge for this project, you are encouraged to read HPC book, chapter 9; for an
elementary tutorial on graphs, see HPC book, chapter 19.

10.1 Traditional algorithms

We first implement the ‘textbook’ formulations of two Single Source Shortest Path (SSSP) algorithms:
on unweighted and then on weighted graphs. In the next section we will then consider formulations that
are in terms of linear algebra.
In order to develop the implementations, we start with some necessary preliminaries,

10.1.1 Code preliminaries

10.1.1.1 Adjacency graph
We need a class Dag for a Directed Acyclic Graph (DAG):
class Dag {
private:
vector< vector<int> > dag;
public:
// Make Dag of ‘n’ nodes, no edges for now.
Dag( int n )
: dag( vector< vector<int> >(n) ) {};

It’s probably a good idea to have a function

const auto& neighbors( int i ) const { return dag.at(i); };

that, given a node, returns a list of the neighbors of that node.

73
10. Graph algorithms

Exercise 10.1. Finish the Dag class. In particular, add a method to generate example graphs:
• For testing the ‘circular’ graph is often useful: connect edges

0 → 1 → · · · → N − 1 → 0.

• It may also be a good idea to have a graph with random edges.

Write a method that displays the graph.

10.1.1.2 Node sets

The classic formulation of SSSP algorithms, such as Dijkstra’s algorithm (see HPC book, section 9.1.3)
uses sets of nodes that are gradually built up or depleted.
You could implement that as a vector:
vector< int > set_of_nodes(nnodes);
for ( int inode=0; inode<nnodes; inode++)
// mark inode as distance unknown:
set_of_nodes.at(inode) = inf;

where you use some convention, such as negative distance, to indicate that a node has been removed
from the set.
However, C++ has an actual set container with methods for adding an element, finding it, and removing
it; see section ??. This makes for a more direct expression of our algorithms. In our case, we’d need a
set of int/int or int/float pairs, depending on the graph algorithm. (It is also possible to use a map, using
an int as lookup key, and int or float as values.)
For the unweighted graph we only need a set of finished nodes, and we insert node 0 as our starting
point:
using node_info = std::pair<unsigned,unsigned>;
std::set< node_info > distances;
distances.insert( {0,0} );

For Dijkstra’s algorithm we need both a set of finished nodes, and nodes that we are still working on. We
again set the starting node, and we set the distance for all unprocessed nodes to infinity:
const unsigned inf = std::numeric_limits<unsigned>::max();
using node_info = std::pair<unsigned,unsigned>;
std::set< node_info > distances,to_be_done;

to_be_done.insert( {0,0} );
for (unsigned n=1; n<graph_size; n++)
to_be_done.insert( {n,inf} );

(Why do we need that second set here, while it was not necessary for the unweighted graph case?)

Exercise 10.2. Write a code fragment that tests if a node is in the distances set.
• You can of course write a loop for this. In that case know that iterating over a set gives you the
key/value pairs. Use structured bindings; section ??.
• But it’s better to use an ‘algorithm’, in the technical sense of ‘algorithms built into the standard

74 Introduction to Scientific Programming

 10.1. Traditional algorithms

library’. In this, find.

• . . . except that with find you have to search for an exact key/value pair, and here you want to
search: ‘is this node in the distances set with whatever value’. Use the find_if algorithm;
section ??.

10.1.2 Level set algorithm

We start with a simple algorithm: the SSSP algorithm in an unweighted graph; see HPC book, sec-
tion 9.1.1 for details. Equivalently, we find level sets in the graph.
For unweighted graphs, the distance algorithm is fairly simple. Inductively:
• Assume that we have a set of nodes reachable in at most n steps,
• then their neighbors (that are not yet in this set) can be reached in n + 1 steps.
The algorithm outline is
for (;;) {
if (distances.size()==graph_size) break;
/*
* Loop over all nodes that are already done
*/
for ( auto [node,level] : distances ) {
/*
* set neighbors of the node to have distance ‘level + 1’
*/
const auto& nbors = graph.neighbors(node);
for ( auto n : nbors ) {
/*
* See if ‘n’ has a known distance,
* if not, add to ‘distances’ with level+1
*/
/* ... */
{
cout << "node " << n << " level " << level+1 << ’\n’;
distances.insert( {n,level+1} );
}

Exercise 10.3. Finish the program that computes the SSSP algorithm and test it.
This code has an obvious inefficiency: for each level we iterate through all finished nodes, even if all
their neighbors may already have been processed.

Exercise 10.4. Maintain a set of ‘current level’ nodes, and only investigate these to find the next level.
Time the two variants on some large graphs.

10.1.3 Dijkstra’s algorithm

In Dijkstra’s algorithm we maintain both a set of nodes for which the shortest distance has been found,
and one for which we are still determining the shortest distance. Note: a tentative shortest distance for a
noce may be updated several times, since there may be multiple paths to that node. The ‘shortest’ path
in terms of weights may not be the shortest in number of edges traversed!
The main loop now looks something like:

Victor Eijkhout 75
10. Graph algorithms

for (;;) {
if (to_be_done.size()==0) break;
/*
* Find the node with least distance
*/
/* ... */
cout << "min: " << nclose << " @ " << dclose << ’\n’;
/*
* Move that node to done,
*/
to_be_done.erase(closest_node);
distances.insert( *closest_node );
/*
* set neighbors of nclose to have that distance + 1
*/
const auto& nbors = graph.neighbors(nclose);
for ( auto n : nbors ) {
// find ‘n’ in distances
/* ... */
{
/*
* if ‘n’ does not have known distance,
* find where it occurs in ‘to_be_done’ and update
*/
/* ... */
to_be_done.erase( cfind );
to_be_done.insert( {n,dclose+1} );
/* ... */
}

(Note that we erase a record in the to_be_done set, and then re-insert the same key with a new value.
We could have done a simple update if we had used a map instead of a set.)
The various places where you find nodes in the finished / unfinished sets are up to you to implement.
You can use simple loops, or use find_if to find the elements matching the node numbers.

Exercise 10.5. Fill in the details of the above outline to realize Dijkstra’s algorithm.

10.2 Linear algebra formulation

In this part of the project you will explore how much you make graph algorithms look like linear algebra.

10.2.1 Code preliminaries

10.2.1.1 Data structures
You need a matrix and a vector. The vector is easy:
class Vector {
private:
vector<vectorvalue> values;
public:
Vector( int n )
: values( vector<vectorvalue>(n,infinite) ) {};

76 Introduction to Scientific Programming

 10.2. Linear algebra formulation

For the matrix, use initially a dense matrix:

class AdjacencyMatrix {
private:
vector< vector<matrixvalue> > adjacency;
public:
AdjacencyMatrix(int n)
: adjacency( vector<vector<matrixvalue>>
( n,vector<matrixvalue>(n,empty) ) ) {
};

but later we will optimize that.

Remark 6 In general it’s not a good idea to store a matrix as a vector-of-vectors, but in this case we
need to be able to return a matrix row, so it is convenient.

10.2.1.2 Matrix vector multiply

Let’s write a routine
Vector AdjacencyMatrix::leftmultiply( const Vector& left );

This is the simplest solution, but not necessarily the most efficient one, as it creates a new vector object
for each matrix-vector multiply.
As explained in the theory background, graph algorithms can be formulated as matrix-vector multipli-
cations with unusual add/multiply operations. Thus, the core of the multiplication routine could look
like
for ( int row=0; row<n; row++ ) {
for ( int col=0; col<n; col++) {
result[col] = add( result[col], mult( left[row],adjacency[row][col] ) );
}
}

10.2.2 Unweighted graphs

Exercise 10.6. Implement the add / mult routines to make the SSSP algorithm on unweighted graphs
work.

10.2.3 Dijkstra’s algorithm

As an example, consider the following adjacency matrix:
. 1 . . 5
. . 1 . .
. . . 1 .
. . . . 1
1 . . . .

The shortest distance 0 → 4 is 4, but in the first step a larger distance of 5 is discovered. Your algorithm
should show an output similar to this for the successive updates to the known shortest distances:

Victor Eijkhout 77
10. Graph algorithms

Input : 0 . . . .
step 0: 0 1 . . 5
step 1: 0 1 2 . 5
step 2: 0 1 2 3 5
step 3: 0 1 2 3 4

Exercise 10.7. Implement new versions of the add / mult routines to make the matrix-vector multi-
plication correspond to Dijkstra’s algorithm for SSSP on weighted graphs.

10.2.4 Sparse matrices

The matrix data structure described above can be made more compact by storing only nonzero elements.
Implement this.

10.2.5 Further explorations

How elegant can you make your code through operator overloading?
Can you code the all-pairs shortest path algorithm?
Can you extend the SSSP algorithm to also generate the actual paths?

10.3 Tests and reporting

You now have two completely different implementations of some graph algorithms. Generate some large
matrices and time the algorithms.
Discuss your findings, paying attention to amount of work performed and amount of memory needed.

78 Introduction to Scientific Programming

Chapter 11

Climate change

The climate has changed and it is always changing.

Raj Shah, White House Principal Deputy Press Secretary
The statement that climate always changes is far from a rigorous scientific claim. We can attach a mean-
ing to it, if we interpret it as a statement about the statistical behaviour of the climate, in this case as
measured by average global temperature. In this project you will work with real temperature data, and
do some simple analysis on it. (The inspiration for this project came from [7].)
Ideally, we would use data sets from various measuring stations around the world. Fortran is then a great
language because of its array operations (see chapter ??): you can process all independent measurements
in a single line. To keep things simple we will use a single data file here that contains data for each
month in a time period 1880-2018. We will then use the individual months as ‘pretend’ independent
measurements.

11.1 Reading the data

In the repository you find two text files
GLB.Ts+dSST.txt GLB.Ts.txt

that contain temperature deviations from the 1951–1980 average. Deviations are given for each month
of each year 1880–2018. These data files and more can be found at https://data.giss.nasa.
gov/gistemp/.

Exercise 11.1. Start by making a listing of the available years, and an array monthly_deviation
of size 12 × nyears, where nyears is the number of full years in the file. Use formats and array notation.
The text files contain lines that do not concern you. Do you filter them out in your program, or are you
using a shell script? Hint: a judicious use of grep will make the Fortran code much easier.

11.2 Statistical hypothesis

We assume that mr Shah was really saying that climate has a ‘stationary distribution’, meaning that highs
and lows have a probability distribution that is independent of time. This means that in n data points,

79
11. Climate change

each point has a chance of 1/n to be a record high. Since over n + 1 years each year has a chance of
1/(n + 1), the n + 1st year has a chance 1/(n + 1) of being a record.
We conclude that, as a function of n, the chance of a record high (or low, but let’s stick with highs) goes
down as 1/n, and that the gap between successive highs is approximately a linear function of the year1 .
This is something we can test.

Exercise 11.2. Make an array previous_record of the same shape as monthly_deviation.

This array records (for each month, which, remember, we treat like independent measurements) whether
that year was a record, or, if not, when the previous record occurred:

′


y if MonDev(m, y) = maxm′ MonDev(m , y)

PrevRec(m, y) = y ′ if MonDev(m, y) < MonDev(m, y ′ )

and MonDev(m, y ′ ) = maxm′′ <m′ MonDev(m′′ , y)


Again, use array notation. This is also a great place to use the Where clause.

Exercise 11.3. Now take each month, and find the gaps between records. This gives you two arrays:
gapyears for the years where a gap between record highs starts, and gapsizes for the length of
that gap.
This function, since it is applied individually to each month, uses no array notation.

The hypothesis is now that the gapsizes are a linear function of the year, for instance measured as distance
from the starting year. Of course they are not exactly a linear function, but maybe we can fit a linear
function through it by linear regression.

Exercise 11.4. Copy the code from http://www.aip.de/groups/soe/local/numres/bookfpdf/

f15-2.pdf and adapt for our purposes: find the best fit for the slope and intercept for a linear function
describing the gaps between records.

You’ll find that the gaps are decidedly not linearly increasing. So is this negative result the end of the
story, or can we do more?

Exercise 11.5. Can you turn this exercise into a test of global warming? Can you interpret the devia-
tions as the sum of a yearly increase in temperature plus a stationary distribution, rather than a stationary
distribution by itself?

1. Technically, we are dealing with a uniform distribution of temperatures, which makes the maxima and minima have a
beta-distribution.

80 Introduction to Scientific Programming

Chapter 12

Desk Calculator Interpreter

In this set of exercises you will write a ‘desk calculator’: a small interactive calculator that combines
numerical and symbolic calculation.
These exercises mostly use the material of chapters ??, ??, ??.

12.1 Named variables

We start out by working with ‘named variables’: the namedvar type associates a string with a variable:
type namedvar
character(len=20) :: expression = ""
integer :: value
end type namedvar

A named variable has a value, and a string field that is the expression that generated the variable. When
you create the variable, the expression can be anything.
type(namedvar) :: x,y,z,a
x = namedvar("x",1 )
y = namedvar("yvar",2 )

Next we are going to do calculations with these type objects. For instance, adding two objects
• adds their values, and
• concatenates their expression fields, giving the expression corresponding to the sum value.
Your first assignment is to write varadd and varmult functions that get the following program working
with the indicated output. This uses string manipulation from sections ?? and ??.

Exercise 12.1. The following main program should give the corresponding output:
Code: Output
[structf] varhandling:
print *,x
print *,y x 1
z = varadd(x,y) yvar 2
print *,z (x)+(yvar) 3
a = varmult(x,z) (x)*((x)+(yvar)) 3
print *,a

81
12. Desk Calculator Interpreter

(To be clear: the two routines need to do both numeric and string ‘addition’ and ‘multiplication’.)
You can base this off the file namedvar.cxx in the repository

12.2 First modularization

Let’s organize the code so far by introducing modules; see chapter ??.

Exercise 12.2. Create a module (suggested name: VarHandling) and move the namedvar type defi-
nition and the routines varadd, varmult into it.

Exercise 12.3. Also create a module (suggested name: InputHandling) that contains the routines
islower, isdigit from the character exercises in chapter ??. You will also need an isop routine to
recognize arithmetic operations.

12.3 Event loop and stack

In our quest to write an interpreter, we will write an ‘event loop’: a loop that continually accepts single
character inputs, and processes them. An input of "0" will mean termination of the process.

Exercise 12.4. Write a loop that accepts character input, and only prints out what kind of character
was encountered: a lowercase character, a digit, or a character denoting an arithmetic operation +-*/.
Code: Output
[structf] interchar:
do
read *,input Inputs: 4 x 3 + 0
if (input .eq. ’0’) then 4 is a digit
exit x is a lowercase
else if ( isdigit(input) ) then 3 is a digit
+ is an operator

Use the InputHandling module introduced above.

12.3.1 Stack
Next, we are going to store values in namedvar types on a stack. A stack is a data structure where new
elements go on the top, so we need to indicate with a stack pointer that top element. Equivalently, the
stack pointer indicates how many elements there already are:

type(namedvar),dimension(10) :: stack
integer :: stackpointer=0

Since we are using modules, let’s keep the stack out of the main program and put it in the appropriate
module.

82 Introduction to Scientific Programming

 12.3. Event loop and stack

Exercise 12.5. Add the stack variable and the stack pointer to the VarHandling module.

Since Fortran uses 1-based indexing, a starting value of zero is correct. For C/C++ it would have been -1.
Next we will start implementing stack operations, such as putting namedvar objects on the stack.

12.3.2 Stack operations

We extend the above event loop, which was only recognizing the input characters, by actually incorpo-
rating actions. That is, we repeatedly
1. read a character from the input;
2. 0 causes the event loop to exit; otherwise:
3. if it is a digit, create a new namedvar entry on the top of the stack, with that value both numer-
ically as the value field, and as string in the expression field.
You may be tempted to write the following in the main program:
if ( isdigit(input) ) then
stackpointer = stackpointer + 1
read( input,’( i1 )’ ) stack(stackpointer)%value
stack(stackpointer)%expression = trim(input)

(You have already coded isdigit in exercise ??.) but a cleaner design uses a function call to a
method in the VarHandling module:

else if ( isdigit(input) ) then subroutine stack_push(input)

call stack_push(input) implicit none
character,intent(in) :: input

Note that the stack_push routine does not have the stack or stack pointer as arguments: since
they are all in the same module, they are accessible as global variable.
Finally,
4. if it is a letter indicating an operation +, −, ×, /,
(a) take the two top entries from the stack, lowering the stack pointer;
(b) apply that operation to the operanrds; and
(c) push the result onto the stack.
The auxiliary function stack_display is a little tricky, so you get that here. This uses string formatting
(section ??) and implied do loops (section ??): Also, note that the stack array and the stackpointer
act like global variables.
subroutine stack_display()
implicit none
! local variables
integer :: istck

if (stackpointer.eq.0) return
print ’( 10( a,a, a,i0,"; ") )’, ( &
" expr=",trim(stack(istck)%expression), &
" val=",stack(istck)%value, &
istck=1,stackpointer )

end subroutine stack_display

Victor Eijkhout 83
12. Desk Calculator Interpreter

Let’s add the various options to the event loop.

Exercise 12.6. Make your event loop accept digits, creating a new entry:
Code: Output
[structf] internum:
else if ( isdigit(input) ) then
call stack_push(input) Inputs: 4 5 6 0
expr=4 val=4;
expr=4 val=4; expr=5 val=5;
expr=4 val=4; expr=5 val=5; expr=6

Next we integrate the operations: if the input character corresponds to an arithmetic operator, we call
stack_op with that character. That routine in turn calls the appropriate operation depending on what the
character was.

Exercise 12.7. Add a clause to your event loop to handle characters that stand for arithmetic opera-
tions:
Code: Output
[structf] internumop:
else if ( isop(input) ) then
call stack_op(input) Inputs: 4 5 6 + + 0
expr=4 val=4;
expr=4 val=4; expr=5 val=5;
expr=4 val=4; expr=5 val=5; expr=6
expr=4 val=4; expr=(5)+(6) val=11;
expr=(4)+((5)+(6)) val=15;

12.3.3 Item duplication

Finally, we may want to use a stack entry more than once, so we need the functionality of duplicating a
stack entry.
For this we need to be able to refer to a stack entry, so we add a single character label field: the namedvar
type now stores
1. a single character id,
2. an integer value, and
3. its generating expression as string.
type namedvar
character :: id
character(len=20) :: expression
integer :: value
end type namedvar

Exercise 12.8. Add the id field to the namedvar, and make sure your program still compiles and
runs.
The event loop is now extended with an extra step. If the input character is a lowercase letter, it is used
as the id of a namedvar as follows.
• If there is already a stack entry with that id, it is duplicated on top of the stack;

84 Introduction to Scientific Programming

 12.4. Modularizing

• otherwise, the id of the stack top entry is set to this character.

Here is the relevant bit of the new stack_print function:
print ’( 10( a,a1, a,a, a,i0,"; ") )’, ( &
"id:",stack(istck)%id, &
" expr=",trim(stack(istck)%expression), &
" val=",stack(istck)%value, &
istck=1,top )

Exercise 12.9. Write the missing function and its clause in the event loop:
Code: Output
[structf] stackfind:
stacksearch = find_on_stack(stack,stackpointer,input)
if ( stacksearch>=1 ) then Inputs: 1 x 2 y x y + z 0
stackpointer = stackpointer+1 id:. expr=1 val=1;
stack(stackpointer) = stack(stacksearch) id:x expr=1 val=1;
id:x expr=1 val=1; id:. expr=2 val=2;
id:x expr=1 val=1; id:y expr=2 val=2;
id:x expr=1 val=1; id:y expr=2 val=2;
id:x expr=1 val=1; id:y expr=2 val=2;
val=2;
id:x expr=1 val=1; id:y expr=2 val=2;
id:x expr=1 val=1; id:y expr=2 val=2;

(What is in the else part of this conditional?)

12.4 Modularizing
With the modules and the functions you have developed so far, you have a very clean main program:
do
call stack_display()
read *,input
if (input .eq. ’0’) exit
if ( isdigit(input) ) then
call stack_push(input)
else if ( isop(input) ) then
call stack_op(input)
else if ( islower(input) ) then
call stack_name(input)
end if
end do

You see that by moving the stack into the module, neither the stack variable nor the stack pointer are
visible in the main program anymore.
But there is an important limitation to this design: there is exactly one stack, declared as a sort of global
variable, accessible through a module.
Whether having global data is good practice is another matter. In this case it’s defensible: in a calculator
app there will be exactly one stack.

Victor Eijkhout 85
12. Desk Calculator Interpreter

12.5 Object orientation

But maybe we do sometimes need more than one stack. Let’s bundle up the stack array and the stack
pointer in a new type:
type stackstruct
type(namedvar),dimension(10) :: data
integer :: top=0
contains
procedure,public :: display, find_id, name, op, push
end type stackstruct

Exercise 12.10. Change the event loop so that it calls methods of the stackstruct type, rather than
functions that take the stack as input.
For instance, the push function is called as:
if ( isdigit(input) ) then
call thestack%push(input)

12.5.1 Operator overloading

The varadd and similar arithmetic routines use a function call for what we would like to write as an
arithmetic operation.

Exercise 12.11. Use operator overloading in the varop function:

if (op=="+") then
varop = op1 + op2

et cetera.

86 Introduction to Scientific Programming

PART III

APPENDIX
Chapter 13

Style guide for project submissions

The purpose of computing is insight, not numbers. (Richard Hamming)

Your project writeup is equally important as the code. Here are some common-sense guidelines for a
good writeup. However, not all parts may apply to your project. Use your good judgement.

13.1 General approach

As a general rule, consider programming as an experimental science, and your writeup as a report on
some tests you have done: explain the problem you’re addressing, your strategy, your results.
Turn in a writeup in pdf form (Word and text documents are not acceptable) that was generated from a
text processing program such (preferably) LATEX (for a tutorial, see Tutorials book, section-15).

13.2 Style
Your report should obey the rules of proper English.
• Observing correct spelling and grammar goes without saying.
• Use full sentences.
• Try to avoid verbiage that is disparaging or otherwise inadvisable. The academic XSEDE has
the following guidelines: https://www.xsede.org/terminology; much longer and
more extentsive, the Google developer documentation style guide [4] is also a great resource.

13.3 Structure of your writeup

Consider this project writeup an opportunity to practice writing a scientific article.
Start with the obvious stuff.
• Your writeup should have a title. Not ‘Project’ or ‘parallel programming’, but something like
’Parallelization of Chronosynclastic Enfundibula in MPI’.
• Author and contact information. This differs per publication. Here it is: your name, EID, TACC
username, and email.

89
13. Style guide for project submissions

• Introductory section that is extremely high level: what is the problem, what did you do, what
did you find.
• Conclusion: what do your findings mean, what are limitations, opportunities for future exten-
sions.
• Bibliography.

13.3.1 Introduction
The reader of your document need not be familiar with the project description, or even the problem it
addresses. Indicate what the problem is, give theoretical background if appropriate, possibly sketch a
historic background, and describe in global terms how you set out to solve the problem, as well as a brief
statement of your findings.

13.3.2 Detailed presentation

See section 13.5 below.

13.3.3 Discussion and summary

Separate the detailed presentation from a discussion at the end: you need to have a short final section that
summarizes your work and findings. You can also discuss possible extensions of your work to cases not
covered.

13.4 Experiments
You should not expect your program to run once and give you a final answer to your research question.
Ask yourself: what parameters can be varied, and then vary them! This allows you to generate graphs or
multi-dimensional plots.
If you vary a parameter, think about what granularity you use. Do ten data points suffice, or do you get
insight from using 10, 000?
Above all: computers are very fast, they do a billion operations per second. So don’t be shy in using
long program runs. Your program is not a calculator where a press on the button immediately gives the
answer: you should expect program runs to take seconds, maybe minutes.

13.5 Detailed presentation of your work

The detailed presentation of your work is as combination of code fragments, tables, graphs, and a de-
scription of these.

90 Introduction to Scientific Programming

13.5.1 Presentation of numerical results
You can present results as graphs/diagrams or tables. The choice depends on factors such as how many
data points you have, and whether there is an obvious relation to be seen in a graph.
Graphs can be generated any number of ways. Kudos if you can figure out the LATEX tikz package, but
Matlab or Excel are acceptable too. No screenshots though.
Number your graphs/tables and refer to the numbering in the text. Give the graph a clear label and label
the axes.

13.5.2 Code
Your report should describe in a global manner the algorithms you developed, and you should include
relevant code snippets. If you want to include full listings, relegate that to an appendix: code snippets in
the text should only be used to illustrate especially salient points.
Do not use screen shots of your code: at the very least use a monospaced font such as the verbatim
environment, but using the listings package (used in this book) is very much recommended.

13.6 General index

#define, 69

Amazon
delivery truck, 59
prime, 59, 65

bisection, 23

compilation
separate, 45
connected components, see graph, connected
constructor
delegating, 34
covid-19, 44

define, see #pragma define

Dijkstra’s algorithm, 74
dynamic
programming, 56

ebola, 44
efficiency gap, 56
91
eight queens, 31

gerrymandering, 51
Goldbach conjecture, 11
Google, 47
developer documentation style guide, 89
graph
connected, 48
diameter, 49
greedy search, see search, greedy

header, 45
Horner’s rule, 25
INDEX

makefile, 45
Manhattan distance, 59
Markov chain, 49
memoization, 56
memory
bottleneck, 71

Newton’s method, 28
NP-hard, 63

operator
overloading, 86
opt2, 63

Pagerank, 47
programming
dynamic, 54

root finding, 23

search
greedy, 62, 63
Single Source Shortest Path, 49
SIR model, 42
stack, 82
pointer, 82
structured bindings, 74

variable
global
in Fortran module, 83
vector, 12

XSEDE, 89

92 Introduction to Scientific Programming

 INDEX

Victor Eijkhout 93
INDEX

94 Introduction to Scientific Programming

Chapter 14

Bibliography

95
14. Bibliography

96 Introduction to Scientific Programming

Bibliography

[1] E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Ham-

marling, A. McKenney, and D. Sorensen. LAPACK: A portable linear algebra library for high-
performance computers. In Proceedings Supercomputing ’90, pages 2–11. IEEE Computer Society
Press, Los Alamitos, California, 1990.
[2] Roy M. Anderson and Robert M. May. Population biology of infectious diseases: Part I. Nature,
280:361–367, 1979. doi:10.1038/280361a0.
[3] Tolga Bektas. The multiple traveling salesman problem: an overview of formulations and solution
procedures. Omega, 34(3):209 – 219, 2006.
[4] Google. Google developer documentation style guide.
[5] Kazushige Goto and Robert A. van de Geijn. Anatomy of high-performance matrix multiplication.
ACM Trans. Math. Softw., 34(3):1–25, 2008.
[6] C. L. Lawson, R. J. Hanson, D. R. Kincaid, and F. T. Krogh. Basic linear algebra subprograms for
fortran usage. ACM Trans. Math. Softw., 5(3):308–323, September 1979.
[7] Juan M. Restrepo, , and Michael E. Mann. This is how c̈limate is always changing.̈ APS Physics,
GPS newletter, February 2018.
[8] Lin S. and Kernighan B. An effective heuristic algorithm for the traveling salesman problem. Oper-
ations Research, 21:498–516, 1973.

97