Mathematical Preliminaries

Mathematical Preliminaries
Topics
• Set Theory

• Combinatorics

• Mathematical Functions

• Summations

• Probability Theory

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/2

Set Theory
 Set Theory
Definitions
 A Set is a collection of objects called members. If a, b, c, d are members of a set S we write :
S ={a, b, c, d}
where a, b, c, d  S ( read a, b, c, d belong to S )
 The set builder notation can be used to define a set. For example, the set
S={1, 3, 5, 7, 11, 13}
can be defined as
S = {x: where x is prime number and x<15}

 In set notation the order of members does not matter

S = {a, b, c, d} is identical to set {d, b, c, a}

 In a set all members are distinct objects , i. e, duplicate members are not allowed

 A set that contains no elements is called empty set. It is symbolically denoted by φ

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/4

 Set Theory
Subsets
 A set A is called a subset of set B if every element of A is also a member B. Symbolically:

A  B, if x A then x B
The set A is proper subset of B, if A contains only some of the elements of B. Symbolically
the relationship is represented as:
A  B
 The relationship among sets is usually shown by a picture, which is called Venn diagram.
Figures (a), (b) show Venn diagrams for a subset and proper subset

B
A=B
A

A  B
A  B
(a) A is proper subset of B (b) A is subset of B

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/5

 Set Theory
Universal Set
 The largest set for a given collection of objects is called Universal Set. All other sets are
subsets of the Universal set U. Symbolically:
φ  U,
A  U, B 
U, C U 
Example: The Universal set, U, of all small English alphabets is
U = {a, b, c,…..,x, y, z}
The subsets of the Universal set are, for example, A = {a, f, B = {p, q, r, s , t},
g}, C = {x ,y, w}, as shown in the diagram

A B
C

Subsets of Universal set of alphabets (a, b, c,…x, y ,z}

 Given a Universal set, we define complement Ac of set A as

Ac = U – A
The set Ac contains all of elements of Universal set, excluding those in the set A.
Example: If A={a, b, c}, then Ac = { d, e, f,……, x, y, z}
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/6
 Set Operations
Union
 The union of set A and set B is set of all elements x such that x is in A or x is in B.
Symbolically:
A U
B, if x 
A or x  B
 The Union operation is pictorially represented by Venn diagram, as shown below:

A B A U B
Union set
Example (1): A={ 1, 3, 4, 5}, B={ 3, 4, 7, 8, 9}. The Union of sets A and B is the
set { 1, 3, 4, 5, 7,8, 9}.
Example(2): Union operation can be used to add elements to a set. Consider sets {a, b}, {c, d}
(i) Setting S to empty set φ
S=φ
(ii) To insert elements of first set into S, we perform union of S with {a, b}
S =S U {a, b} = {a, b}
(iii) To insert elements of second set {b, c} into S, we perform union of S with {c, d}
S = S U{c, d} = {a, b, c, d}
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/7
 Set Operations
Intersection
The intersection of set A and set B is set of all elements x, such that x is in A
and x is in B. Symbolically:

A I
B, if x 
A and x B 
 The intersection operation is pictorially represented by Venn Diagram, as shown below.

A B
A I
Set intersectionB

Example: Let A={1,2, 3, 4, 5} and B={2, 4, 7, 8} then

A I B = {2, 4}

 Two sets are called disjoint, if their intersection is empty set. Thus, if A and B are disjoint,

A I B=φ
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/8
 Set Operations
Difference Set
The difference of set A and set B is set of all elements x, such that x is not in B if x is in A.
Symbolically:
B – A , if x
 A then x B 
 The difference operation is illustrated by Venn Diagram, as shown below.

A B B-A
Difference set

Example(1): If A={1,2,3,4,5} and B={4, 6, 8, 10} then A –B = {1, 2, 3, 5}

Example(2): We can use difference operation to delete elements from a set. Consider
S = {a, b, c, d, e, f }
(i) To delete element a we perform difference operation on S and set
{a}. S = S – {a} = { b, c, d, e, f }

(ii) To delete element d we perform difference operation on set S and set {d}.
S = S-{d} = { b, c, e, f }

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/9

 Set Theory
Cardinality
The cardinality of a set is the number of elements in the set. It is also called size. The
cardinality of set S is denoted by |S|.

Example: If S = {a, b, c, d, e}, then | S | = 5

 For any sets A and B, |AUB | = |A | + | B | - |A ∩B|

Consider, for example, the sets A, B
Let A = {a, b, c, d, e, f}, |A|=
6

Let B ={d, e, f, g, h, i, j ,k}, |B|=

8
| A U B | = 11

A U B = {a, b, c, d, e, f, g, h, i, j, k}, A
It follows that | A | + | B | - | A ∩ B | = 6 + 8 – 3 = 11
∩ B = {d, e, f}, | A ∩ B | = 3
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/10
 Set Theory
Cartesian Product
The Cartesian product of two sets A and B, denoted by A X B, is a set of all
ordered pairs such that the first element of the pair is in set A and the second element
is in set B. It is also called cross product. Mathematically,

and bB }
A X B = { (a , b) : a A
Example: Let A = {a, b, c} and B = {g, h}, then

A X B = {(a, g), (a, h), (b, g), (b, h), (c, g), (c, h) }

 The cardinality of Cartesian product is the product of cardinalities of sets

| A X B | = | A | .| B |

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/11

 Set Theory
Binary Relation
A binary relation R on sets A and B is a subset of the Cartesian product A X B of the sets .
In set notation, the relation is represented as:

R  AXB

 Since a cross product set can have one or more subsets there can be many binary relations

Example: Consider the sets A = {a, b, c}, B = {x, y}

Their cross product is set A X B = {(a, x), (a, y), (b, x), (b, y), (c, x), (c, y) }

(1) A binary relation R1 is

R1 = { (a, y), (b, y), (c, y) }

(2) Another binary relation R2 is

R2 = {(a, x), (a, y) }

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/12

 Binary Relation
Example
A graph G is a pair (V, E ), where V is a finite set, called vertex set, and E is a binary
relation on V, called edge set

Example: Consider the graph G = (V, E) , and V = { a, b, c }, vertex set for the graph
A binary relation is

E = {(a, a), (a, b), (a, c), (b, c), (c, b) }

The set E represents edges of graph G. The graph is shown pictorially in the diagram
below.

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/13

Combinatorics
 Permutations
Definition
A permutation is an arrangement of n objects in some order, such that each object appears
exactly once.

 Elements of a set of n objects can be arranged in the following ways

n.(n-1).(n-2)…..3.2.1
because, the first place can be filled in n ways, and the second place in n-1 ways. Thus, first
and second places together can be filled in n(n-1) ways. The first, second and third places
can be filled in n.(n-1).(n-2) ways, and so on

 The product n(n-1)(n-2)….3.2.1 is called n-factorial, which is denoted as n!

Example: The elements of Set S={ a, b, c } can be permuted in six ways because
3! = 3.2.1 = 6
The six arrangements are: abc, acb, bac, bca, cab, cba

It is assumed that 0! = 1 (factorial of zero is one ). The factorial of a negative integer is
not defined

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/15

 Factorial Function
Stirling‘s Approximation
The n! factorial increases rapidly with the increase of n. The table below shows the
growth.
n n!
2 2

4 24

8 40,320

16 20,922,789,888,000

32 2.6 x 10 35

 It can be shown that 1000! is an integer consisting of 2,500 digits.

 The analysis of algorithms sometimes involves large inputs. For large values of n, the
n! is approximated by the following formula,:

n! ≈ √ ( 2 π n ) ( n / e) n , where e = 2.718

 The above formula is referred to as Stirling’s approximation

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/16

 Combinations
Definition
The arrangement of n objects taken k ( k ≤ n ) at a time, without regard to the order,
is called k-combination.

Example: The objects a, b, c, d have following 2-combinations:

ab, ac, ad, bc, bd, cd

Observe that the combination ab ≡ ba i.e, order does not matter

The number of k-combinations of n objects is denoted by the symbol

Another notation is C(n ,k).

 It can be shown that :

C(n,k) =

 The k-combinations are called binomial coefficients because they

occur in
binomial theorem for the expansion of the expression:

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/17

Mathematical Functions
 Mathematical Functions
Use in Analysis
Several mathematical functions are used to analyze algorithms and study their behavior.
We briefly examine the properties of some these functions From analysis perspective, the
following functions are particularly useful:
i. y=└x ┘ (Floor
function)

ii. y = ┌ x ┐ (Ceil function )

iii. y = log a x (Logarithm function)

iv. y = x(Linear function )

v. y = x2 (Quadratic function)

vi. y = x3 (Cubic function)

vii. y = 2x (Exponential function with base 2)

viii. y =nn (Exponential)

When a mathematical function is used to represent the running time of an algorithm, the
function is referred to as growth function. The behavior (growth rate) of a function can be
represented pictorially by plotting a graph of (x,y) points given by the functional relation
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/19
 Floor Function
Definition
 The floor function y(x) is defined as:

y=└x ┘
where x is a real number and y is the largest integer which is smaller than or equal to
number x.

Example: └ 4.9┘= 4,

└ 4.0 ┘ = 4

 It follows from the definition : x -1 <└ x ┘ ≤ x

 The floor function is used for mapping of a real valued function into integer function

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/20

 Ceil Function
Definition
 The ceil function y(x) is defined as:

y = ┌x ┐

where x is real
number and y is the
smallest integer
larger than or equal
to x.

Example: ┌ 4.9 ┐= 5

┌ 4.0 ┐ = 4

 It follows from
the definition :
x ≤ ┌x ┐<x
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/21
 Logarithm Function
Definition
 The logarithm function y= log a x is the inverse of exponential function ay, where a is some
positive constant.

 Thus, if y = log a x ,
then
x=ay
Example: Consider, y= log 5 25 . Since 25= 52, it follows y=2 .

 The figure shows a plot of logarithm function to the base 2

 The logarithm function will frequently arise in the analysis of algorithms

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/22

 Logarithm Function
Bases
 The choice of base depends on the nature of application and data type.

The base 10 is normally used in computations involving decimal numbers. Here are some
examples
log 10 1 =0 because 1= 10 0 by
log 10 10 =1 definition
log 10 100 = 2 10= 10 1
log 10 1000000 = 6 100=10 2
 The base 2 is often used in the analysis1000000=10
of algorithms.
6 Some examples of base 2

are log 2 1=0 because 1= 20 by definition

log 2 2= 1 2=21
log 2 16= 4 16= 24
log 2 64= 6 64= 26
The logarithm to base 2 are referred to as binary logarithm. A special notation lg is used
to represent a binary algorithm, as follows
lg (n) ≡ log 2 (n)
The base e ≈ 2.718 is used in calculus for purposes of differentiation and integration It is
called natural logarithm. A special notation ln is used to represent natural logarithms.
ln (n) ≡ log e (n)

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/23

 Logarithm Function
Properties
The following properties of logarithms are often used in several applications
• log b x . y = logb x + logb y (Sum rule)

• logb x / y = logb x – logb y (Difference rule)

• logb x n = n logb x (Exponentiation rule)

• x logb y = y log x
b ( Symmetry rule )

 The last property is particularly useful for interchanging logarithmic exponential and
base of exponentiation. Here are some examples.
2log 2 n = n log 2 = n (Interchanging 2 and n)
2

8log 2 n = n log 8 = n3 (Interchanging 8 and n)

2

3log 2 n = n log 3 = n1.585 ( Interchanging 3 and n and using lg 3 = 1.585)

2

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/24

 Logarithm Function
Change of Base
Sometimes it is necessary to change the base of a logarithm. The formula for the
transformation from base a to b is as follows:

loga x = logb x / logb a

Example : lg (binary logarithm) can be converted to ln (natural logarithm):

lg x = log2 x

= log e x / loge2
= ln x / ln 2 ( ln 2 = 0.6931)

Mathematical Preliminaries/KICSIT 2020/Fahim Khan/25

 Mathematical Functions
Growth Rates
The table shows the growth rate of common mathematical functions, which are useful in the
analysis of algorithms

n lg n √n n lg n n2 n3 2n n! nn

2 1 1.4 2 4 8 4 2 2

4 2 2 8 16 64 16 24 256

8 3 2.8 24 64 512 256 40,320 16,777,216

16 4 4 64 256 4,096 65,536 20,922,789,888,000 1.845 x 1019

32 5 5.7 160 1,024 32,768 4,294,967,296 2.631 x 10 35 1.461 x 10 48

64 6 8 384 4,096 262,144 1.8 x 10 19 1.269 x 10 89 3.940 x 10 115

128 7 11 896 16,384 2,097,152 3.4 x 10 38 3.856 x 10 215 5.283 x 10 269

 The logarithm lg n has the lowest growth rate , and the exponential function n n
the highest
 The relationship of functions in terms of their growth rates is symbolically represented
as
lg n < √n < n < n lgn < n2 < n3 < 2n < n! < n n
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/26
 Growth Functions
Graph
The plot of mathematical functions lg n, n, n lg n n2, n3, 2n is shown below.

f(n)

Because of extremely rapid growth of n! and nn , these functions are not shown in the
ranges shown in the graph scale
Mathematical Preliminaries/KICSIT 2020/Fahim Khan/27