Sums and Recurrences

P. Sam Johnson

September 5, 2015

P. Sam Johnson

Sums and Recurrences

September 5, 2015

1/32

Overview

In the lecture, we disucss a notation for sum and general techniques that
make summation user-friendly.
The following methods are discussed.
repertoire method
summation factor method
perturbation method.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

2/32

We will be working with sums of the general form

a1 + a2 + + an
where each ak is a number that has been defined somehow. This
three-dot notation has the advantage that we can see the whole
sum, almost as if it were written out in full, if we have a good enough
imagination. But it can be ambiguous and a bit long-winded.
For example, if 1 + 2 + + 2n1 is supposed to denote a sum of n terms,
not of 2n1 , we should write it more explicitly as
20 + 21 + + 2n1 .
The other notation is, notably the delimited form (sigma-notation)
n
X

ak

k=1

because it uses the Greek letter

P. Sam Johnson

(upper case sigma).

Sums and Recurrences

September 5, 2015

3/32

This notation tells us to include in the sum precisely those terms ak whose
index k is an integer that lies between the lower and upper limits 1 and n,
inclusive. In words, we sum
P over k, from 1 to n. Joseph Fourier
introduced this delimited -notation in 1820, and it soon took the
mathematical world by storm.

Joseph Fourier (1768 - 1830)

P
Each element ak of a sum is called a term. The quantity after
(here
ak ) P
is called the summand. The index variable k is said to be bound to
the
sign.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

4/32

Generalized sigma-notation
A generalized sigma-notation is even more useful than the P
delimited
form. We simply write one or more conditions under the , to specify
the set of indicies over which summation should take place. It can be
written as
X
an .
1kn

For example, we can express the sum of the squares of all odd positive
integers below 100 as follows:
X
k 2.
1k<100
k odd

P. Sam Johnson

Sums and Recurrences

September 5, 2015

5/32

But the delimited form of this sum

49
X
(2k + 1)2
k=0

is more cumbersome and less clear. Similarly, the sum of reciprocals of all
prime numbers between 1 and N is
X 1
.
p
pN
p prime

The delimited form would require us to write

(N)

X 1
,
pk
k=1

where pk denotes the kth prime and (N) is the number of primes less
than or equal to N.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

6/32

The biggest advantage of general sigma-notation is that we can manipulate

it more easily than the delimited form. For example, suppose we want to
change the index variable k to k + 1. With the general form, we have
X
X
ak =
ak+1
1kn

1k+1n

it is easy to see what is going on, and we can do the substitution almost
without thinking. But with the delimited form, we have
n
X
k=1

ak =

n1
X

ak+1

k=0

it is harder to see what has happened, and we are more likely to make a
mistake.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

7/32

On the other hand, the delimited form is not completely useless. It is

nice and tidy, and we can write it quickly because
n
X

ak

k=1

has seven symbols compared with

X

ak

1kn

having eight symbols.

P
Therefore we will often use
with upper and lower delimiters when we
state a problemP
or present a result, but we will prefer to work with
relations-under- when we are manipulating a sum whose index variables
need to be transformed.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

8/32

Formally, we write
X

ak

P(k)

as an abbreviation for the sum of all terms ak such that k is an integer

satisfying a given property P(k). (A property P(k) is any statement
about k that can be either true or false.)
For the time being, we will assume that only finitely many integers k
satisfying P(k) have ak 6= 0; otherwise infinitely many nonzero numbers
are being added together, and things can get a bit tricky.
At the other extreme, if P(k) is false for all integers k, we have an
empty sum; the value of an empty sum is defined to be zero.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

9/32

People are often tempted to write

n1
X

k(k 1)(n k)

k=2

instead of
n
X

k(k 1)(n k)

k=0

because the terms for k = 0, 1, and n in this sum are zero.

We will find it advantageous to keep upper and lower bounds on an index
of summation as simple as possible, because sums can be manipulated
P
much more easily when the bounds are simple. Indeed, the form n1
k=2
can even be dangerously ambiguous, because its meaning is not at all clear
when n = 0 or n = 1. Zero-valued terms cause no harm, and they often
save a lot of trouble.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

10/32

Kenneth E. Iverson introduced a wonderful idea in his programming

language APL and we will see that it greatly simplifies many things. The
idea is simply to enclose a true-or-false statement in brackets, and to say
that the result is 1 if the statement is true, 0 if the statement is false. For
example,
(
1 if p is a prime number
[p prime] =
0 if p is not a prime number.
Iversons convention allows us to express sums with no constraints
whatever on the index of summation, because we can rewrite
X
ak
P(k)

in the form
X

ak [P(k)].

P. Sam Johnson

Sums and Recurrences

September 5, 2015

11/32

If P(k) is false, the term ak [P(k)] is zero, so we can safely include it

among the terms being summed.
This makes it easy to manipulate the index of summation, because we
dont have to fuss with boundary conditions.
If we use Iversons convention to write the sum of reciprocal primes N as
X
[p prime] [p N]/p,
n

there is no problem of division by zero when p = 0, because our

convention tells us
[0 prime] [0 N]/0 = 0.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

12/32

The three-dots form often suggests useful manipulations, particularly the

combination of adjacent terms, since we might be able to spot a
simplifying pattern if we let the whole sum hang out before our eyes. But
too much detail can also be overwhelming.
Sigma-notation is compact, impressive and often suggestive of
manipulations that are not obvious in three-dots form. When we work with
sigma-notation,
zero terms are not generally harmful; in fact, zeros often
P
make -manipulation easier.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

13/32

Sums and Recurrences

There is an intimate relation between sums and recurrences. First we shall
see how sum is reduced to recurrence.
P
The sum Sn = nk=0 ak is equivalent to the recurrence S0 = a0
Sn = Sn1 + an , for n > 0. Therefore we can evaluate sums in closed
forms.
Example
Consider the following sum-recurrence
R0 =
Rn = Rn1 + + n,

for n > 0.

We find R1 = + + , R2 = + 2 + 3, and so on. In general the

solution can be written in the form Rn = A(n) + B(n) + C (n) where
A(n), B(n) and C (n) are the coefficients of dependence on the general
parameters , and .
P. Sam Johnson

Sums and Recurrences

September 5, 2015

14/32

Repertoire method
This method tells us to try plugging in simple funcions on Rn , hoping to
find constant parameters , and , where the solution is especially
simple.
Rn = 1
Rn = n
Rn = n 2

= 1, = 0 =
= 0, = 1, = 0
= 0, = 1, = 2

Thus Rn = + n +

A(n) = 1
B(n) = n
C (n) B(n) = n2
2
hence 2C (n) = n 2+n

n2 +n
2 .

Exercise
1

Evaluate the sum

Pn

k=0 (a

+ bk) by repertorie method.

Conversely, many recurrences can be reduced to sums.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

15/32

Summation Factor
Consider the tower of Hanoi recurrence
T0 = 0
Tn = Tn1 + 1,

for n > 0.

(1)

(1) is rewritten as
T0 /20 = 0
Tn /2n = Tn1 /2n1 + 1/2n ,

for n > 0.

Let Sn = Tn /2n . We get S0 = 0, Sn = Sn1 + 1/2n ,

for n > 0.

Hence
Sn =

n
 1 n
X
1
.
=
1

2
2k
k=1

Thus Tn =
P. Sam Johnson

2n Sn

2n

1.
Sums and Recurrences

September 5, 2015

16/32

Summation Factor : General Technique

Consider the recurrence of the form an Tn = (bn Tn1 + cn ).
We reduce the above recurrence to a sum. The idea is to multiply both
sides by a summation factor, sn (sn should not be zero).
Choose sn such that sn bn = sn1 an1 .
We get sn an Tn = sn bn Tn1 + sn cn . That is,
sn an Tn = sn1 an1 Tn1 + sn cn .
Let Sn = sn an Tn . We have a sum-recurrence Sn = Sn1 + sn cn . Hence
Sn = s0 a0 T0 +

n
X

sk ck = s1 b1 T0 +

k=1

n
X

sk c k .

k=1

Therefore the solution to the original recurrence is

n

X
1 
Tn =
s1 b1 T0 +
sk ck .
sn an
k=1

P. Sam Johnson

Sums and Recurrences

September 5, 2015

17/32

How to find the summation factor sn ?

The relation sn = sn1 an1
bn can be unfolded to tell us that the fraction
an1 an2 a1
sn =
bn bn1 b2
or any convenient constant multiple of this value, will be a suitable
summation factor for the recurrence
an Tn = bn Tn1 + cn .
Example
The tower of Hanoi recurrence has an = 1 and bn = 2. Here sn = 2n . sn
is to be multiplied both sides of Tn = 2Tn1 + 1 (n 0) if we want to
reduce the recurrence to a sum.
Caution! The summation factor method works whenever all the as and
all the bs are nonzero.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

18/32

A sorting method : Quick-sort

Quick-sort is one of the most important methods for sorting data inside a
computer. The average number of comparison steps made by quicksort
when it is applied to n items in random order satisfies the recurrence
C0 = 0
n1

Cn = (n + 1) +

2X
Ck ,
n

for n > 0.

(2)

k=0

If Cn denotes the average number of moves needed to sort n numbers (in

random order initially), it can be shown that Cn satisfies the recursion.
If (2) is rewritten as
an Tn = bn Tn1 + Cn
then the recurrence can be reduced to a sum.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

19/32

If we multiply both sides of (2) by n, we can get rid of the division.

2

nCn = n + n + 2

n1
X

Ck ,

for n > 0.

k=0

To get rid of the

sign, we have

(n 1)Cn1 = (n 1)2 + (n 1) + 2

n2
X

Ck ,

for n > 1.

k=0

Hence
nCn (n 1)Cn1 = 2n + 2Cn1 ,

for n > 1.

Therefore the original recurrence for Cn reduces to a much simpler one :

C0 = 0
nCn = (n + 1)Cn1 + 2n,

P. Sam Johnson

Sums and Recurrences

for n > 0.

September 5, 2015

20/32

A summation factor sn with an = n, bn = n + 1 and cn = 2n is

2
sn = (n+1)
n . Thus
Cn = 2(n + 1)

n
X
k=1

1
.
k +1

The sum
n
X
k=1

1
k +1

is very similar to a quantity that arises frequently in applications. We

denote
n
1 1
1 X1
Hn = 1 + + + + =
.
2 3
n
k
k=1

The letter H stands for harmonic; Hn is a harmonic number (because

the nth harmonic produced by a violin string is the fundamental tone
produced by a string that is 1/k times as long.)
P. Sam Johnson

Sums and Recurrences

September 5, 2015

21/32

Cn in terms of Hn
We can express Cn in terms of Hn as follows:
n
X
k=1

1
k +1

X
1kn

1
k +1

X
1k1n

X
2kn+1

1
k
1
k

X 1
1
1+
k
n+1

1kn

= Hn

n
.
n+1

Thus Cn = 2(n + 1)Hn 2n.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

22/32

Connection between calculus and discrete mathematics

We would like to discuss a connection between calculus (infinite calculus)
and discrete mathematics (finite calculus).
The infinite series (harmonic series)

X
1
n=1

is divergent because a subsequence

(s2n )
n=1
of sequence ofPpartial sums (sn )
n=1 is stricly increasing and not bounded
1
n
above (since
>
).
n=1 2n
2

That is, (Hn )

n=1 diverges. Also (log n)n=1 diverges, by integral test for
sequence.

But we shall prove that the sequence (Hn log n)

n=1 converges.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

23/32

Let Gn = Hn log n. Consider y = x1 .

Figure in integral for 1/x
Z n
1 1
1
1
1
1
+ + +
<
dx < 1 + + +
2 3
n
2
n1
1 x
Hn 1 < log n < Hn1 < Hn
Hence 0 < Hn log n < 1.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

24/32

Eulers constant
Gn+1 Gn = [Hn+1 log(n + 1)] [Hn log n]
1
n
=
+ log(
)
n+1
n+1
1
1
=
+ log(1
)
n+1
n+1
1
1
1
1
=

< 0.
2
n + 1 n + 1 2(n + 1)
3(n + 1)3
Therefore (Gn ) is decreasing and bounded above, hence it converges.
d
(x)]x=1 = = 0.577215664. The Gamma
Euler showed that [ dx
function is defined by
Z
e t t x1 dt (x > 0)
(x) =
0

The limit of (Gn ) is called Eulers constant and it is nothing by . The

numbers e, , are some important constants.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

25/32

Manipulation of Sums
The key to success with sums is an ability to change one
that is simpler or closer to some goal.

into another

Let k be any finite set of integers. Sum over the elements of k can be
transformed by using 3 single rules.
X

cak = c

kK

X
kK

(ak +bk ) =

kK

ak

ak +

kK

ak =

kK

P. Sam Johnson

X
p(k)K

p(k)

X
kK

bk

distributive law allows us to

move
P constants in and out of
a .
associative
P law allows us to
break a
intoP
two parts, or
to combine two s into one.
commutative law says that we
can reorder the terms in any
way as we please.

Sums and Recurrences

September 5, 2015

26/32

We now compute the general sum of an arithmetic progression

X
S=
(a + bk).
0kn

By the commutative law, we can replace k by n k, we get

X
X
S=
(a + b(n k)) =
(a + bn bk).
0nkn

0kn

These two equations can be added by using the associative law:

X
2S =
[(a + bk) + (a + bn bk)]
0kn

(2a + bk) = (n + 1)(2a + bn).

0kn

Hence
X

S=

0kn


bn 
a + (a + bn)
(a + bk) = a +
(n + 1) =
(n + 1),
2
2

the average of first and last terms times the number of terms.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

27/32

Splitting off terms

The operation of splitting off a term
X
X
ak = a0 +
ak
0kn

for n 0

1kn

is the basis of a perturbation method that often allows us to evaluate a

sum in closed form.
The idea is to start with an unknown sum and call it Sn :
X
Sn =
ak .
0kn

We write Sn+1 in two ways:

1

by splitting off its last term

by splitting off its first term.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

28/32

Splitting off terms

Sn+1 = Sn + an+1

(splitting off last term)

and
X

Sn+1 =

ak = a0 +

0kn+1

Sn+1 =

ak

1kn+1

ak

1kn+1

ak+1

1k+1n+1

Sn+1 =

ak+1

(splitting off first term)

0kn

If we express the last sum in terms of Sn , we obtain an equation whose

solution is the sum we seek.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

29/32

Exercises
2 Find the sum of the following recursions by permutation method.
(a)

ax k

(b)

0kn

k2k

0kn

(c)

kx k .

0kn

By using elementary techniques of differential calculus, find the closed

form in a completely different way. Note that the derivative of a sum
is the sum of the derivatives of its terms. Start with the equation
n
X
k=0

xk =

1 x n+1
1x

and take the derivative of both sides with respect to x, we get

n
X
k=0

kx k1 =

1 (n + 1)x n + nx n+1
.
(1 x)2

This example shows that there is a connection between calculus and

discrete mathematics.
P. Sam Johnson

Sums and Recurrences

September 5, 2015

30/32

Exercises
Find the following sums.
3

n
X

nk

(1)

k=0

P. Sam Johnson

n
X
(1)n1 k
k=0

Sums and Recurrences

n
X

(1)n1 k 2 .

k=0

September 5, 2015

31/32

References

1
6

Graham, Knuth and Patashnik, Concrete Mathematics A

Foundation for Computer Science, Pearson Education.

Marko Petkovsek, Herbert S. Wilf and Doron Zeilberger,

A = B 00 , AK Peters Ltd., Wellesley, Massachusetts.

Herbert S. Wilf, Generatingfunctionology, Third Edition, AK

Peters Ltd., Wellesley, Massachusetts.

P. Sam Johnson

Sums and Recurrences

September 5, 2015

32/32