Time and Space

Complexity Analysis

Notes Target For :

Created By - Shivam Barekar

INTRODUCTION : TIME COMPLEXITY
06 February 2024 21:55

"Why did the algorithm go to therapy? It had an issue with time, and needed a safe space
to sort it out"

If I ask you, how time-efficient is this code you've written ? What do you think will be the
right way to measure it ?

What if you try and run it on your machine and then tell me that it takes 18 seconds to
run, so the time complexity is 18 seconds? Unfortunately, it's not a very nice way to
measure the efficiency because the execution time will be different for different :

1. Devices
2. Programming Languages
3. Set of Inputs
and so on...

Let’s understand why we should not judge any code on the basis of the time taken by
a machine.

If we run the same code in a low-end machine (e.g. old windows machine) and in a high-end
machine(e.g. Latest MacBook), we will observe that two different machines take different
amounts of time for the same code. The high-end machine will take lesser time as
compared to the low-end machine.

So, the time taken by a machine can be changed depending on the configuration. That is
why we should not compare the two different codes on the basis of the time taken by a
machine as the time is dependent on it.

What is Time Complexity ?

We can solve a problem using different logic and different codes. Time complexity
basically helps to judge different codes and also helps to decide which code is better. In
an interview, an interviewer generally judges a code by its time complexity.

Now, the term, time complexity, seems that it is referring to the time taken by a machine
to execute a particular code. But in real life, Time complexity does not refer to the time
taken by the machine to execute a particular code.

Definition : The rate at which the time, required to run a code, changes with respect to
the input size, is considered the time complexity. Basically, the time complexity of a
particular code depends on the given input size, not on the machine used to run the code.
 (Or)
Definition :Time complexity is a measure of how the running time of an algorithm grows as
the size of the input increases.

What's a nice way, then?

The right way is to calculate the time complexity based on the number of steps to be done
by the code for a given input.

Let's take a trivial example where I give you the value of a variable N and ask you to
calculate 1 + 2 + 3 ...... + n.

One of the ways, of course, is to iterate over 1, 2, 3, ..... to n and keep adding them to get
the answer, i.e. you do n iterations in the code.
Logically, I hope you'll be able to identify that, in this case, the time taken by the code
will be approximately proportional to the value of n. So, this algorithm is said to have an
O(n) time complexity. (also called linear time complexity) .

Another way could be using that formula we all studied in school :

Now, using the formula will lead to making the speed of code to be almost independent of
the input value of n. I mean, the multiplication and division might depend a little bit on how
significant the value of n is, but to a very small extent.

This algorithm is said to have an O(1) time complexity. (also called constant time
complexity).
DIFFERENT NOTATION
06 February 2024 21:58

Now that we know, on a high level, what time & space complexities are, it's a good time to
look at different ways to represent them and try to make sense of them practically.

Let's clear out the make sense of it part right now. It's simple: the worse the time
complexity, the more time the code takes to execute given the same input. So, given the
constraints and time limit of a problem, we'll have to calculate the worst-case time
complexity our code can have to pass, & of course, make sure we find an algorithm that
works within that time complexity.

It is similar for space complexity as well, but like I wrote in the last article, it isn't
generally cared about in more than 95% of the cases.

Diving into the Notations

Asymptotic notations are mathematical notations used to describe an algorithm's time

complexity (and space complexity).

There are mainly three asymptotic notations that we'll talk about :

Big Oh notation (O) - upper bound

Omega notation (Ω) - lower bound
Theta notation (ፀ) - exact bound

Among these three, Big O and Theta are particularly useful while analyzing algorithms.

Big Oh Notation - (O)

"Mathematically, We say that f(n) = O(g(n)), if there exist positive constants N0 and C,
i.e. f(n) <= C*g(n) ∀ n >= N0"

Let's not get into the mathematical definition for now, the crux (in easily understandable
terms) is that the Big O notation represents the upper bound of the running time of an
algorithm.
 Since it gives the worst-case running time of an algorithm, it is widely used to analyze an
algorithm as we are always interested in the worst-case scenario.

Here are some common time complexities (ordered best to worse):

1. O(1) : Constant time complexity

2. O(log⁡n) : Logarithmic time complexity
3. O(n) : Linear time complexity
4. O(nlog⁡n) : Linearithmic time complexity
5. O(n2) : Quadratic time complexity
6. O(2n) : Exponential time complexity

Omega Notation - (Ω)

"Mathematically, We say that f(n) = Ω(g(n)), if there exist positive constants N0 and C,
i.e. f(n) >= C*g(n) ∀ n >= N0"

Again, the crux is that the Ω notation represents the lower bound of the running time of
an algorithm.

Theta Notation - (ፀ)

"Mathematically, We say that f(n) = ፀ(g(n)), if there exist positive constants N0, C1 and
C2, i.e. C1*g(n) <= f(n) <= C2*g(n) ∀ n >= N0"

In simpler terms, if an algorithm, let's say has ፀ(N2) time complexity, it basically means
it's an exact bound, because the time taken by the algorithm will never grow faster than
C2*N2 and slower than C1*N2.

Summing Things Up

If an algorithm has O(N) time complexity, it means, for a sufficiently large input:

• The code may run within ፀ(LogN) time.

• It may run within ፀ(N) time.
• however, it will never take ፀ(N2) time. (because the growth of n2 is faster than that of n)

If an algorithm has Ω(N) time complexity, it means, for a sufficiently large input:

• The code may run within ፀ(N2) time.

• It may run within ፀ(N) time.
• however, it will never take ፀ(LogN) time. (because the growth of log n is slower than that
of n)

If an algorithm has ፀ(N) time complexity, it means, for a sufficiently large input:

• The code will complete execution in ፀ(N) time.

• It will never take ፀ(N2) time. (because the growth of n2 is faster than that of n)
• It will never take ፀ(N3) time. (because the growth of n3 is faster than that of n)

Basically, in case of Theta, if a program has a certain time complexity, let's say, ፀ(log n)
we can be sure that the time complexity of the program cannot grow faster or slower than
ፀ(log n).
DIFFERENT TIME COMPLEXITY
06 February 2024 21:59

Before diving in, I'd like to share that a C++ code can run around 2∗108 operations per
second if those are low-level operations (array accesses, additions, bit-shifts, multiplies,
subtractions, xors, etc.). Mods and divisions are slightly slower. Taking input & printing
output is slightly even slower.

Of course, the number of operations per second depends on the machine, but the above-
shared number is just a bare minimum that you can safely assume.

For example, my M2 Pro 14" Macbook Pro (subtle flex 😉) was able to run the code below
in just less than 0.1 sec.

"The ones marked Out of Syllabus were literally trillions of times the curent age of the
universe. So, I hope it's okay not to calculate them xD"
HOW TO CALCULATE
06 February 2024 22:00

Let’s understand this using the following example :

The time complexity for this code will be nothing but the number of steps, this code will
take to be executed. So, if we write this in terms of Big O notation, it will be like O(no. of
steps).

Let’s observe the steps for this code :

1. First, the assigning step(i = 1) will be done.

2. The second step will be the comparison i.e. i <= 5.
3. The third step will be the print statement (i.e. cout << “Raj”;).
4. The fourth step will be the increment(i.e. i++).
5. In the fifth step, the updated value of i will be again checked i.e. the comparison(i <=
5).
6. In the sixth step, the print statement will be executed and so on.

This flow will continue until the value of i becomes greater than 5(i.e. 6). In a broader
sense, we can observe that the ‘for loop’ will run 5 times and for each time three steps will
be surely executed i.e. checking/comparison, printing, and increment. So, the total steps
will be 5*3 = 15. And the time complexity in terms of Big O notation will be O(15).

Now, if we write N instead of 5, the number of steps will be then N*3 = 3N and the time
complexity will be O(3*N).

But this manual counting process is not feasible for any code. As the ‘for loop’ might run a
billion or million times and inside that ‘for loop’, there might be a large no. of operations or
some other ‘for loops’ as well. So, we have to find out a better approach to calculate the
time complexity of any given code.

Here come the three rules, that we are going to follow while calculating the time
complexity :
1. We will always calculate the time complexity for the worst-case scenario.
2. We will avoid including the constant terms.
3. We will also avoid the lower values.

Let’s discuss the rules individually:

Calculate the time complexity for the worst-case scenario :

Before discussing the point we need to understand the three terms i.e. Best Case, Worst
Case, and Average Case.

Let’s understand these three terms considering the following piece of code:

i. Best Case : This term refers to the case where the code takes the least amount of time to
get executed. For example, if the mark is 10(i.e. < 25), only the first line will be executed
and the rest of the lines will be skipped. So, the least amount of steps i.e. only 2 steps are
required in this case. This is an example of the best case.
ii. Worst Case : This term refers to the case where the code takes the maximum amount of
time to get executed. For example, if the mark is 70(i.e. > 65), the last line will be
executed after checking all the above conditions. So, the maximum amount of steps i.e. 4
steps are required in this case. This is an example of the worst case.
iii. Average Case : This term is pretty self-explanatory. This is basically the case between the
best and the worst.

Now, as we always want that our system serves the maximum number of clients, we need to
calculate the time complexity for the worst-case scenario. With this, we can actually judge
the robustness of any code or any system.

1. Avoid including the constant terms :

Let’s understand this rule considering the time complexity: O(4N3 + 3N2 + 8). Now, if we
consider the value of N as 105 the time complexity will be like this: O(4*1015 + 3*1010 + 8).
In this case, the constant term 8 is very less significant in terms of changing the time
complexity with different values of N. That is why we should avoid the constant terms
while calculating the time complexity.

If we want to think of this case in terms of code, we can consider the following code:
 Here, the first step (i.e. int x = 2) will be executed in unit time i.e. constant time. The
precise time complexity is O(3N + 1) but in this case, the constant 1 is very less
significant. So we will write the time complexity as O(3N) avoiding the constant term.

2. Avoid the lower values :

Now, in the previous example, the given time complexity is O(4N3 + 3N2 + 8) and we have
reduced it to O(4N3 + 3N2). Here, we can clearly observe if the value of N is a large
number, the second term i.e. 3N2 will also be a less significant term. For example, if the
value of N is 105 then the term 3*1010 becomes less significant with respect to 4*1015. So,
we can also avoid the lower values and the final time complexity will be O(4N3 )

Note : A point to remember is that we can actually ignore the constant coefficients as
well. For example, considering the time complexity O(4N3 ) as O(N3 ) is also correct.

"Calculating time complexity is not always a very trivial task, sometimes we may think that
a line of code runs X number of times, but when looked closely it may not be the case."

"But well, practice, and you'll become better at it."

Question 1 :

Given the following code :

In order to calculate the time complexity of the code, we need to first observe how the
loops are working. The outer loop i.e. i runs from 0 to N-1 i.e. N times and for every value
of i, the inner loop i.e. j also runs from 0 to N-1 i.e. N times. The following illustration
depicts the process :

Now, we can clearly observe the total number of steps i.e. N+N+N+N+…….+N times = N*N =
N2. So, the time complexity will be O(N2). We can ignore other constant steps as well as
the innermost block of code as it runs in constant time.

Question 2 :

Given the following code :

In order to calculate the time complexity of the code, we again need to first observe how
the loops are working. The outer loop i.e. i runs from 0 to N-1 i.e. N times and for every
value of i, the inner loop i.e. j also runs from 0 to i i.e. (i+1) times. The following illustration
depicts the process :
Now, we can clearly observe the total number of steps i.e. 1+2+3+4+…….+N. Now we know
the formula of the summation of the first N natural numbers i.e. (N*(N+1))/2 = N2/2 +
N/2. So, the precise time complexity will be O(N2/2 + N/2). Now, we should ignore the
lower values. So, the time complexity will be O(N2/2). It can be also written as O(N2)
avoiding the coefficient 1/2.

These are the basics of time complexity. Now, let’s move on to the space complexity part.
SPACE COMPLEXITY
06 February 2024 22:01

Alright; after having understood time complexity, this should be fairly easy.

We just learned that measuring time efficiency in terms of the absolute time it takes to
run a code is not ideal. Similarly, we can't just say that the space complexity of a program
is 73 bytes or 100 KB. This is because the absolute space required has more factors than
just the code itself, the input size being one of them.

What is Space Complexity?

The term space complexity generally refers to the memory space that a code uses while
being executed. Again space complexity is also dependent on the machine and so we are
going to represent the space complexity using the Big O notation instead of using the
standard units of memory like MB, GB, etc.

Definition : Space complexity generally represents the summation of auxiliary space and
the input space. Auxiliary space refers to the space that we use additionally to solve a
problem. And input space refers to the space that we use to store the inputs.
(Or)
Space complexity measures how the memory/space required by an algorithm grows as the
size of the input increases.

Let’s understand this using the following example :

The variables a and b are used to store the inputs but c refers to the space we are using
to solve the problem and c is the auxiliary space. Here the space complexity will be O(3).

Similarly, if we use an array of size n, the space complexity will be O(N).

Good coding practice:

If a question of adding two numbers like a and b is asked, one of the possible methods will
be b = a+b. In this case, the space complexity is definitely reduced as we are not using any
extra variable but this is not a good practice to manipulate the given inputs or data. In an
interview, we must be careful that we will not manipulate the given data even if the space
complexity becomes 2N instead of N. If the interviewer specifically instructs us to
manipulate, then only we should attempt this method.

Note: A company may use the same data for different purposes. That is why we should not
attempt to manipulate the given data for reducing the space complexity. So, we will never
manipulate the given data i.e. the inputs until the interviewer specifically says so.

We are now pretty much done with our concepts of time complexity and space complexity.
Now, we will briefly discuss some points about competitive programming or the online
judge.

Points to remember :

In competitive programming or in the platforms like Leetcode and GeeksforGeeks, we

generally run our codes on online servers. Most of these servers execute roughly
108 operations in approximately 1 second i.e. 1s. We must be careful that if the time limit
is given as 2s the operations in our code must be roughly 2*108, not 1016. Similarly, 5s
refers to 5*108. Simply, if we want our code to be run in 1s, the time complexity of our
code must be around O(108) avoiding the constants and the lower values.
 Know Thy Complexities!
www.bigocheatsheet.com

Big-O Complexity Chart

Horrible Bad Fair Good Excellent

O(n!) O(2^n)
O(n^2)

O(n log n)
Operations

O(n)

O(log n), O(1)

Elements
 Common Data Structure Operations
Data Structure Time Complexity Space Complexity
Average Worst Worst
Access Search Insertion Deletion Access Search Insertion Deletion
Array Θ(1) Θ(n) Θ(n) Θ(n) O(1) O(n) O(n) O(n) O(n)

Stack Θ(n) Θ(n) Θ(1) Θ(1) O(n) O(n) O(1) O(1) O(n)

Queue Θ(n) Θ(n) Θ(1) Θ(1) O(n) O(n) O(1) O(1) O(n)

Singly-Linked List Θ(n) Θ(n) Θ(1) Θ(1) O(n) O(n) O(1) O(1) O(n)

Doubly-Linked List Θ(n) Θ(n) Θ(1) Θ(1) O(n) O(n) O(1) O(1) O(n)

Skip List Θ(log(n)) Θ(log(n)) Θ(log(n)) Θ(log(n)) O(n) O(n) O(n) O(n) O(n log(n))

Hash Table N/A Θ(1) Θ(1) Θ(1) N/A O(n) O(n) O(n) O(n)

Binary Search Tree Θ(log(n)) Θ(log(n)) Θ(log(n)) Θ(log(n)) O(n) O(n) O(n) O(n) O(n)

Cartesian Tree N/A Θ(log(n)) Θ(log(n)) Θ(log(n)) N/A O(n) O(n) O(n) O(n)

B-Tree Θ(log(n)) Θ(log(n)) Θ(log(n)) Θ(log(n)) O(log(n)) O(log(n)) O(log(n)) O(log(n)) O(n)

Red-Black Tree Θ(log(n)) Θ(log(n)) Θ(log(n)) Θ(log(n)) O(log(n)) O(log(n)) O(log(n)) O(log(n)) O(n)

Splay Tree N/A Θ(log(n)) Θ(log(n)) Θ(log(n)) N/A O(log(n)) O(log(n)) O(log(n)) O(n)

AVL Tree Θ(log(n)) Θ(log(n)) Θ(log(n)) Θ(log(n)) O(log(n)) O(log(n)) O(log(n)) O(log(n)) O(n)

KD Tree Θ(log(n)) Θ(log(n)) Θ(log(n)) Θ(log(n)) O(n) O(n) O(n) O(n) O(n)

Array Sorting Algorithms

Algorithm Time Complexity Space Complexity
Best Average Worst Worst
Quicksort Ω(n log(n)) Θ(n log(n)) O(n^2) O(log(n))

Mergesort Ω(n log(n)) Θ(n log(n)) O(n log(n)) O(n)

Timsort Ω(n) Θ(n log(n)) O(n log(n)) O(n)

Heapsort Ω(n log(n)) Θ(n log(n)) O(n log(n)) O(1)

Bubble Sort Ω(n) Θ(n^2) O(n^2) O(1)

Insertion Sort Ω(n) Θ(n^2) O(n^2) O(1)

Selection Sort Ω(n^2) Θ(n^2) O(n^2) O(1)

Tree Sort Ω(n log(n)) Θ(n log(n)) O(n^2) O(n)

Shell Sort Ω(n log(n)) Θ(n(log(n))^2) O(n(log(n))^2) O(1)

Bucket Sort Ω(n+k) Θ(n+k) O(n^2) O(n)

Radix Sort Ω(nk) Θ(nk) O(nk) O(n+k)

Counting Sort Ω(n+k) Θ(n+k) O(n+k) O(k)

Cubesort Ω(n) Θ(n log(n)) O(n log(n)) O(n)