DATA STRUCTURES AND ALGORITHMS

(Ankita Bhatt-Assistant Professor EN)

What is Data Structure:

A data structure is a storage that is used to store and organize data. It is a way of arranging
data on a computer so that it can be accessed and updated efficiently.

A data structure is not only used for organizing the data. It is also used for processing,
retrieving, and storing data. There are different basic and advanced types of data structures
that are used in almost every program or software system that has been developed. So we
must have good knowledge about data structures.
Classification of Data Structure:

 Linear data structure: Data structure in which data elements are arranged sequentially
or linearly, where each element is attached to its previous and next adjacent elements,
is called a linear data structure.
Examples of linear data structures are array, stack, queue, linked list, etc.

 Static data structure: Static data structure has a fixed memory size. It is easier to
access the elements in a static data structure.
An example of this data structure is an array.

 Dynamic data structure: In dynamic data structure, the size is not fixed. It can be
randomly updated during the runtime which may be considered efficient
 concerning the memory (space) complexity of the code.
Examples of this data structure are queue, stack, etc.
 Non-linear data structure: Data structures where data elements are not placed
sequentially or linearly are called non-linear data structures. In a non-linear data
structure, we can’t traverse all the elements in a single run only.
Examples of non-linear data structures are trees and graphs.
For example, we can store a list of items having the same data-type using the array data
structure.

Array Data Structure

Data structures are an essential concept in computer science and programming. Here are
some reasons why they are important:
1. Efficient data processing: Data structures provide a way to organize and store data in a
way that allows for efficient retrieval, manipulation, and storage of data. For example,
using a hash table to store data can provide constant-time access to data.
2. Memory management: Proper use of data structures can help to reduce memory usage
and optimize the use of resources. For example, using dynamic arrays can allow for
more efficient use of memory than using static arrays.
3. Code reusability: Data structures can be used as building blocks in various algorithms
and programs, making it easier to reuse code.
4. Abstraction: Data structures provide a level of abstraction that allows programmers to
focus on the logical structure of the data and the operations that can be performed on
it, rather than on the details of how the data is stored and manipulated.
5. Algorithm design: Many algorithms rely on specific data structures to operate
efficiently. Understanding data structures is crucial for designing and implementing
efficient algorithms.
Overall, data structures are essential for managing and manipulating data in an efficient and
effective way. They are a fundamental concept in computer science and are used extensively in
programming and software development.
The structure of the data and the synthesis of the algorithm are relative to each other. Data
presentation must be easy to understand so the developer, as well as the user, can make an
efficient implementation of the operation.
Data structures provide an easy way of organizing, retrieving, managing, and storing data. Here
is a list of the needs for data.

 Efficient data access and manipulation: Data structures enable quick access and
manipulation of data. For example, an array allows constant-time access to elements
using their index, while a hash table allows fast access to elements based on their key.
Without data structures, programs would have to search through data sequentially,
leading to slow performance.
 Memory management: Data structures allow efficient use of memory by allocating and
deallocating memory dynamically. For example, a linked list can dynamically allocate
memory for each element as needed, rather than allocating a fixed amount of memory
upfront. This helps avoid memory wastage and enables efficient memory management.
 Code reusability: Data structures can be reused across different programs and projects.
For example, a generic stack data structure can be used in multiple programs that
require LIFO (Last-In-First-Out) functionality, without having to rewrite the same code
each time.
 Optimization of algorithms: Data structures help optimize algorithms by enabling
efficient data access and manipulation. For example, a binary search tree allows fast
searching and insertion of elements, making it ideal for implementing searching and
sorting algorithms.

 Scalability: Data structures enable programs to handle large amounts of data

effectively. For example, a hash table can store large amounts of data while providing
fast access to elements based on their key.
Classification/Types of Data Structures:
Data structures can be classified into two main types: primitive data structures and non-
primitive data structures.Primitive data structures: These are the most basic data structures
and are usually built into programming languages. Examples include:
Integer
Float
Character
Boolean
Double
Void
Non-primitive data structures: These are complex data structures that are built using
primitive data types. Non-primitive data structures can be further categorized into the
following types:
Arrays: A collection of elements of the same data type, stored in contiguous memory
locations.
Linked lists: A collection of elements that are connected by links or pointers.
Stacks: A collection of elements that follow the Last-In-First-Out (LIFO) principle.
Queues: A collection of elements that follow the First-In-First-Out (FIFO) principle.
Trees: A hierarchical data structure consisting of nodes connected by edges.
Graphs: A non-linear data structure consisting of nodes and edges.
Hash tables: A data structure that stores data in an associative manner using a hash function.
Heaps: A specialized tree-based data structure that satisfies the heap property.
Tries: A tree-like data structure used to store associative arrays where the keys are strings.
Sets: A collection of unique elements.
Maps: An abstract data type that stores key-value pairs.
The choice of data structure depends on the problem to be solved and the operations to be
performed on the data. Different data structures have different strengths and weaknesses
and are suitable for different scenarios. Understanding the different types of data structures
and their characteristics is important for efficient algorithm design and implementation.
1. Linear Data Structure
2. Non-Linear Data Structure.
Linear Data Structure:
 A linear data structure is a type of data structure in which data elements are arranged
in a sequential order, and each element has a unique predecessor and successor,
except for the first and last elements. Linear data structures are one-dimensional and
can be traversed sequentially from the first to the last element.
 Elements are arranged in one dimension ,also known as linear dimension.
 Example: lists, stack, queue, etc.
Non-Linear Data Structure
 A Non-linear data structure is a type of data structure in which data elements are not
arranged in a sequential order, and each element may have one or more predecessors
and successors. Non-linear data structures can represent complex relationships
between data elements, such as hierarchies, networks, and graphs.
 Elements are arranged in one-many, many-one and many-many dimensions.
 Example: tree, graph, table, etc.
Most Popular Data Structures:
1. Array: An array is a collection of data items stored at contiguous memory locations.
The idea is to store multiple items of the same type together. This makes it easier to
calculate the position of each element by simply adding an offset to a base value, i.e.,
the memory location of the first element of the array (generally denoted by the name
of the array).
2. Linked Lists: Like arrays, Linked List is a linear data structure. Unlike arrays, linked list
elements are not stored at a contiguous location; the elements are linked using
pointers.

3. Stack: Stack is a linear data structure which follows a particular order in which the
operations are performed. The order may be LIFO(Last In First Out) or FILO(First In Last
Out). In stack, all insertion and deletion are permitted at only one end of the list.

Mainly the following three basic operations are performed in the stack:

 Initialize: Make a stack empty.

 Push: Adds an item in the stack. If the stack is full, then it is said to be an Overflow
condition.
 Pop: Removes an item from the stack. The items are popped in the reversed order in
which they are pushed. If the stack is empty, then it is said to be an Underflow
condition.
 Peek or Top: Returns top element of the stack.
 isEmpty: Returns true if the stack is empty, else false.

4. Queue: Like Stack, Queue is a linear structure which follows a particular order in
which the operations are performed. The order is First In First Out (FIFO). In the
queue, items are inserted at one end and deleted from the other end. A good example
of the queue is any queue of consumers for a resource where the consumer that came
first is served first. The difference between stacks and queues is in removing. In a stack
we remove the item the most recently added; in a queue, we remove the item the
least recently added.

Mainly the following four basic operations are performed on queue:

 Enqueue: Adds an item to the queue. If the queue is full, then it is said to be an
Overflow condition.
 Dequeue: Removes an item from the queue. The items are popped in the same order in
which they are pushed. If the queue is empty, then it is said to be an Underflow
condition.
 Front: Get the front item from the queue.
 Rear: Get the last item from the queue.

5. Binary Tree: Unlike Arrays, Linked Lists, Stack and queues, which are linear data
structures, trees are hierarchical data structures. A binary tree is a tree data structure
in which each node has at most two children, which are referred to as the left child and
the right child. It is implemented mainly using Links. A Binary Tree is represented by a
 pointer to the topmost node in the tree. If the tree is empty, then the value of root is
NULL. A Binary Tree node contains the following parts.
1. Data
2. Pointer to left child
3. Pointer to the right child
6. Binary Search Tree: A Binary Search Tree is a Binary Tree following the additional
properties:

 The left part of the root node contains keys less than the root node key.
 The right part of the root node contains keys greater than the root node key.
 There is no duplicate key present in the binary tree.
A Binary tree having the following properties is known as Binary search tree (BST).
Applications of Data Structures:
 Arrays: Arrays are used to store a collection of homogeneous elements in contiguous
memory locations. They are commonly used to implement other data structures, such
as stacks and queues, and to represent matrices and tables.
 Linked lists: Linked lists are used to store a collection of heterogeneous elements with
dynamic memory allocation. They are commonly used to implement stacks, queues,
and hash tables.
 Trees: Trees are used to represent hierarchical data structures, such as file systems,
organization charts, and network topologies. Binary search trees are commonly used to
implement dictionaries and symbol tables.
 Graphs: Graphs are used to represent complex relationships between data elements,
such as social networks, transportation networks, and computer networks. They are
commonly used to implement shortest path algorithms and graph traversal algorithms.
 Hash tables: Hash tables are used to implement associative arrays, which store key-
value pairs. They provide fast access to data elements based on their keys.
 Stacks: Stacks are used to store a collection of elements in a last-in-first-out (LIFO)
order. They are commonly used to implement undo-redo functionality, recursive
function calls, and expression evaluation.
 Queues: Queues are used to store a collection of elements in a first-in-first-out (FIFO)
order. They are commonly used to implement waiting lines, message queues, and job
scheduling.
DIFFERENCE BETWEEN DATA TYPE AND DATA STRUCTURE

Data Type Data Structure

The data type is the form of a Data structure is a collection of different
variable to which a value can be kinds of data. That entire data can be
assigned. It defines that the particular represented using an object and can be
variable will assign the values of the used throughout the program.
 Data Type Data Structure
given data type only.

It can hold value but not data. It can hold multiple types of data within a
Therefore, it is dataless. single object.

The implementation of a data type is Data structure implementation is known as

known as abstract implementation. concrete implementation.

There is no time complexity in the In data structure objects, time complexity

case of data types. plays an important role.

While in the case of data structures, the

In the case of data types, the value of
data and its value acquire the space in the
data is not stored because it only
computer’s main memory. Also, a data
represents the type of data that can
structure can hold different kinds and types
be stored.
of data within one single object.

Data type examples are int, float, Data structure examples are stack, queue,
double, etc. tree, etc.

Static Data Structure vs Dynamic Data Structure

Data structure is a way of storing and organizing data efficiently such that the required
operations on them can be performed be efficient with respect to time as well as memory.
Simply, Data Structure are used to reduce complexity (mostly the time complexity) of the
code. Data structures can be two types : 1. Static Data Structure 2. Dynamic Data Structure
What is static data structure? In Static data structure the size of the structure is fixed. The
content of the data structure can be modified but without changing the memory space
allocated to it. Example of Static Data Structures: Arrays.
What is Dynamic Data Structure? In Dynamic data structure the size of the structure is not
fixed and can be modified during the operations performed on it. Dynamic data structures are
designed to facilitate change of data structures in the run time. Example of Dynamic Data
structure: Linked List.
Static Data Structure vs Dynamic Data Structure
 Static data structures, such as arrays, have a fixed size and are allocated at compile-
time. This means that their memory size cannot be changed during program execution.
Index-based access to elements is fast and efficient since the address of the element is
known.
 Dynamic data structures, on the other hand, have a variable size and are allocated at
run-time. This means that their memory size can be changed during program
 execution. Memory can be dynamically allocated or deallocated during program
execution. Due to this dynamic nature, accessing elements based on index may be
slower as it may require memory allocation and deallocation.

Aspect Static Data Structure Dynamic Data Structure

Memory Memory is allocated at

Memory is allocated at run-time
allocation compile-time

Size is fixed and cannot be Size can be modified during

Size
modified runtime

Memory Memory utilization may be Memory utilization is efficient as

utilization inefficient memory can be reused

Access time is faster as it is Access time may be slower due to

Access
fixed indexing and pointer usage

Arrays, Stacks, Queues, Lists, Trees (with variable size),

Examples
Trees (with fixed size) Hash tables
Advantage of Static data structure :
 Fast access time: Static data structures offer fast access time because memory is
allocated at compile-time and the size is fixed, which makes accessing elements a
simple indexing operation.
 Predictable memory usage: Since the memory allocation is fixed at compile-time, the
programmer can precisely predict how much memory will be used by the program,
which is an important factor in memory-constrained environments.
 Ease of implementation and optimization: Static data structures may be easier to
implement and optimize since the structure and size are fixed, and algorithms can be
optimized for the specific data structure, which reduces cache misses and can increase
the overall performance of the program.
 Efficient memory management: Static data structures allow for efficient memory
allocation and management. Since the size of the data structure is fixed at compile-
time, memory can be allocated and released efficiently, without the need for frequent
reallocations or memory copies.
 Simplified code: Since static data structures have a fixed size, they can simplify code by
removing the need for dynamic memory allocation and associated error checking.
 Reduced overhead: Static data structures typically have lower overhead than dynamic
data structures, since they do not require extra bookkeeping to manage memory
allocation and deallocation.
Advantage Of Dynamic Data Structure :
  Flexibility: Dynamic data structures can grow or shrink at runtime as needed, allowing
them to adapt to changing data requirements. This flexibility makes them well-suited
for situations where the size of the data is not known in advance or is likely to change
over time.
 Reduced memory waste: Since dynamic data structures can resize themselves, they
can help reduce memory waste. For example, if a dynamic array needs to grow, it can
allocate additional memory on the heap rather than reserving a large fixed amount of
memory that might not be used.
 Improved performance for some operations: Dynamic data structures can be more
efficient than static data structures for certain operations. For example, inserting or
deleting elements in the middle of a dynamic list can be faster than with a static array,
since the remaining elements can be shifted over more efficiently.
 Simplified code: Dynamic data structures can simplify code by removing the need for
manual memory management. Dynamic data structures can also reduce the complexity
of code for data structures that need to be resized frequently.
 Scalability: Dynamic data structures can be more scalable than static data structures,
as they can adapt to changing data requirements as the data grows.
Abstract Data Types
Abstract Data type (ADT) is a type (or class) for objects whose behavior is defined by a set of
values and a set of operations. The definition of ADT only mentions what operations are to
be performed but not how these operations will be implemented. It does not specify how
data will be organized in memory and what algorithms will be used for implementing the
operations. It is called “abstract” because it gives an implementation-independent view.
The process of providing only the essentials and hiding the details is known as abstraction.

The user of data type does not need to know how that data type is implemented, for
example, we have been using Primitive values like int, float, char data types only with the
knowledge that these data type can operate and be performed on without any idea of how
they are implemented.
Features of ADT:
Abstract data types (ADTs) are a way of encapsulating data and operations on that data into
a single unit. Some of the key features of ADTs include:
 Abstraction: The user does not need to know the implementation of the data
structure only essentials are provided.
 Better Conceptualization: ADT gives us a better conceptualization of the real world.
 Robust: The program is robust and has the ability to catch errors.
 Encapsulation: ADTs hide the internal details of the data and provide a public
interface for users to interact with the data. This allows for easier maintenance and
modification of the data structure.
 Data Abstraction: ADTs provide a level of abstraction from the implementation
details of the data. Users only need to know the operations that can be performed on
the data, not how those operations are implemented.
 Data Structure Independence: ADTs can be implemented using different data
structures, such as arrays or linked lists, without affecting the functionality of the
ADT.
 Information Hiding: ADTs can protect the integrity of the data by allowing access only
to authorized users and operations. This helps prevent errors and misuse of the data.
 Modularity: ADTs can be combined with other ADTs to form larger, more complex
data structures. This allows for greater flexibility and modularity in programming.
Overall, ADTs provide a powerful tool for organizing and manipulating data in a structured
and efficient manner.
Abstract data types (ADTs) have several advantages and disadvantages that should be
considered when deciding to use them in software development. Here are some of the main
advantages and disadvantages of using ADTs:

Advantages:

 Encapsulation: ADTs provide a way to encapsulate data and operations into a single
unit, making it easier to manage and modify the data structure.
 Abstraction: ADTs allow users to work with data structures without having to know
the implementation details, which can simplify programming and reduce errors.
 Data Structure Independence: ADTs can be implemented using different data
structures, which can make it easier to adapt to changing needs and requirements.
 Information Hiding: ADTs can protect the integrity of data by controlling access and
preventing unauthorized modifications.
 Modularity: ADTs can be combined with other ADTs to form more complex data
structures, which can increase flexibility and modularity in programming.

Disadvantages:
  Overhead: Implementing ADTs can add overhead in terms of memory and processing,
which can affect performance.
 Complexity: ADTs can be complex to implement, especially for large and complex
data structures.
 Learning Curve: Using ADTs requires knowledge of their implementation and usage,
which can take time and effort to learn.
 Limited Flexibility: Some ADTs may be limited in their functionality or may not be
suitable for all types of data structures.
 Cost: Implementing ADTs may require additional resources and investment, which can
increase the cost of development.

Overall, the advantages of ADTs often outweigh the disadvantages, and they are widely used
in software development to manage and manipulate data in a structured and efficient way.
However, it is important to consider the specific needs and requirements of a project when
deciding whether to use ADTs.

From these definitions, we can clearly see that the definitions do not specify how these ADTs
will be represented and how the operations will be carried out. There can be different ways to
implement an ADT, for example, the List ADT can be implemented using arrays, or singly
linked list or doubly linked list. Similarly, stack ADT and Queue ADT can be implemented
using arrays or linked lists.

What is an Algorithm?

The word Algorithm means “A set of rules to be followed in calculations or other problem-
solving operations” Or “A procedure for solving a mathematical problem in a finite number of
steps that frequently involves recursive operations “.Therefore Algorithm refers to a sequence
of finite steps to solve a particular problem. Algorithms can be simple and complex depending
on what you want to achieve.

Asymptotic Notation and Analysis (Based on input size) in Complexity Analysis of

Algorithms

Asymptotic Analysis is defined as the big idea that handles the above issues in analyzing
algorithms. In Asymptotic Analysis, we evaluate the performance of an algorithm in terms of
input size (we don’t measure the actual running time). We calculate, how the time (or space)
taken by an algorithm increases with the input size.

Asymptotic notation is a way to describe the running time or space complexity of an

algorithm based on the input size. It is commonly used in complexity analysis to describe how
an algorithm performs as the size of the input grows. The three most commonly used
notations are Big O, Omega, and Theta.

1. Big O notation (O): This notation provides an upper bound on the growth rate of an
algorithm’s running time or space usage. It represents the worst-case scenario, i.e.,
the maximum amount of time or space an algorithm may need to solve a problem. For
example, if an algorithm’s running time is O(n), then it means that the running time of
the algorithm increases linearly with the input size n or less.
2. Omega notation (Ω): This notation provides a lower bound on the growth rate of an
algorithm’s running time or space usage. It represents the best-case scenario, i.e., the
minimum amount of time or space an algorithm may need to solve a problem. For
example, if an algorithm’s running time is Ω(n), then it means that the running time of
the algorithm increases linearly with the input size n or more.
3. Theta notation (Θ): This notation provides both an upper and lower bound on the
growth rate of an algorithm’s running time or space usage. It represents the average-
case scenario, i.e., the amount of time or space an algorithm typically needs to solve a
problem. For example, if an algorithm’s running time is Θ(n), then it means that the
running time of the algorithm increases linearly with the input size n.

In general, the choice of asymptotic notation depends on the problem and the specific
algorithm used to solve it. It is important to note that asymptotic notation does not provide
an exact running time or space usage for an algorithm, but rather a description of how the
algorithm scales with respect to input size. It is a useful tool for comparing the efficiency of
different algorithms and for predicting how they will perform on large input sizes.

Why performance analysis?

There are many important things that should be taken care of, like user-friendliness,
modularity, security, maintainability, etc. Why worry about performance? The answer to this
is simple, we can have all the above things only if we have performance. So performance is
like currency through which we can buy all the above things. Another reason for studying
performance is – speed is fun! To summarize, performance == scale. Imagine a text editor
that can load 1000 pages, but can spell check 1 page per minute OR an image editor that
takes 1 hour to rotate your image 90 degrees left OR … you get it. If a software feature can
not cope with the scale of tasks users need to perform – it is as good as dead.

How to study efficiency of algorithms?

The way to study the efficiency of an algorithm is to implement it and experiment by running
the program on various test inputs while recording the time spent during each execution. A
simple mechanism in Java is based on use of the currentTimeMillis() method of the System
class for collecting such running times. That method reports the number of milliseconds that
have passed since a benchmark time known
as the epoch (January 1, 1970 UTC).The key is that if we record the time immediately before
executing the algorithm and then immediately after it.

long start = System.currentTimeMillis( ); // record the starting time

/∗ (run the algorithm) ∗/
long end = System.currentTimeMillis( ); // record the ending time
long elapsed = end − start; //Total time elapsed

Measuring elapsed time provides a reasonable reflection of an algorithm’s efficiency.

Given two algorithms for a task, how do we find out which one is better?

One naive way of doing this is – to implement both the algorithms and run the two programs
on your computer for different inputs and see which one takes less time. There are many
problems with this approach for the analysis of algorithms.

 It might be possible that for some inputs, the first algorithm performs better than the
second. And for some inputs second performs better.
 It might also be possible that for some inputs, the first algorithm performs better on
one machine, and the second works better on another machine for some other inputs.

Asymptotic Analysis is the big idea that handles the above issues in analyzing algorithms. In
Asymptotic Analysis, we evaluate the performance of an algorithm in terms of input size (we
don’t measure the actual running time). We calculate, how the time (or space) taken by an
algorithm increases with the input size.

For example, let us consider the search problem (searching a given item) in a sorted array.

The solution to above search problem includes:

 Linear Search (order of growth is linear)

 Binary Search (order of growth is logarithmic).

To understand how Asymptotic Analysis solves the problems mentioned above in analyzing
algorithms,

 let us say:
 we run the Linear Search on a fast computer A and
 Binary Search on a slow computer B and
  pick the constant values for the two computers so that it tells us exactly how
long it takes for the given machine to perform the search in seconds.
 Let’s say the constant for A is 0.2 and the constant for B is 1000 which means that A is
5000 times more powerful than B.
 For small values of input array size n, the fast computer may take less time.
 But, after a certain value of input array size, the Binary Search will definitely start
taking less time compared to the Linear Search even though the Binary Search is
being run on a slow machine.

Input Size Running time on A Running time on B

10 2 sec ~1h

100 20 sec ~ 1.8 h

10^6 ~ 55.5 h ~ 5.5 h

10^9 ~ 6.3 years ~ 8.3 h

 The reason is the order of growth of Binary Search with respect to input size is
logarithmic while the order of growth of Linear Search is linear.
 So the machine-dependent constants can always be ignored after a certain value of
input size.

Running times for this example:

 Linear Search running time in seconds on A: 0.2 * n

 Binary Search running time in seconds on B: 1000*log(n)

Popular Notations in Complexity Analysis of Algorithms

1. Big-O Notation
We define an algorithm’s worst-case time complexity by using the Big-O notation, which
determines the set of functions grows slower than or at the same rate as the expression.
Furthermore, it explains the maximum amount of time an algorithm requires to consider all
input values.

2. Omega Notation

It defines the best case of an algorithm’s time complexity, the Omega notation defines whether
the set of functions will grow faster or at the same rate as the expression. Furthermore, it
explains the minimum amount of time an algorithm requires to consider all input values.

3. Theta Notation
It defines the average case of an algorithm’s time complexity, the Theta notation defines when
the set of functions lies in both O(expression) and Omega(expression), then Theta notation is
used. This is how we define a time complexity average case for an algorithm.

Measurement of Complexity of an Algorithm

Based on the above three notations of Time Complexity there are three cases to analyze an
algorithm:

1. Worst Case Analysis (Mostly used)

In the worst-case analysis, we calculate the upper bound on the running time of an algorithm.
We must know the case that causes a maximum number of operations to be executed. For
Linear Search, the worst case happens when the element to be searched (x) is not present in the
array. When x is not present, the search() function compares it with all the elements of arr[] one
by one. Therefore, the worst-case time complexity of the linear search would be O(n).

2. Best Case Analysis (Very Rarely used)

In the best-case analysis, we calculate the lower bound on the running time of an algorithm. We
must know the case that causes a minimum number of operations to be executed. In the linear
search problem, the best case occurs when x is present at the first location. The number of
operations in the best case is constant (not dependent on n). So time complexity in the best case
would be Ω(1)

3. Average Case Analysis (Rarely used)

In average case analysis, we take all possible inputs and calculate the computing time for all of
the inputs. Sum all the calculated values and divide the sum by the total number of inputs. We
must know (or predict) the distribution of cases. For the linear search problem, let us assume
that all cases are uniformly distributed (including the case of x not being present in the array).
So we sum all the cases and divide the sum by (n+1). Following is the value of average-case
time complexity.

Average Case Time = \sum_{i=1}^{n}\frac{\theta (i)}{(n+1)} = \frac{\theta

(\frac{(n+1)*(n+2)}{2})}{(n+1)} = \theta (n)

There are mainly three asymptotic notations:

1. Big-O Notation (O-notation)

2. Omega Notation (Ω-notation)
3. Theta Notation (Θ-notation)

1. Theta Notation (Θ-Notation):

Theta notation encloses the function from above and below. Since it represents the upper and
the lower bound of the running time of an algorithm, it is used for analyzing the average-
case complexity of an algorithm.
.Theta (Average Case) You add the running times for each possible input combination and take
the average in the average case.
Let g and f be the function from the set of natural numbers to itself. The function f is said to be
Θ(g), if there are constants c1, c2 > 0 and a natural number n0 such that c1* g(n) ≤ f(n) ≤ c2 *
g(n) for all n ≥ n0

Mathematical Representation of Theta notation:

Θ (g(n)) = {f(n): there exist positive constants c1, c2 and n0 such that 0 ≤ c1 * g(n) ≤ f(n) ≤ c2
* g(n) for all n ≥ n0}

Note: Θ(g) is a set

The above expression can be described as if f(n) is theta of g(n), then the value f(n) is always
between c1 * g(n) and c2 * g(n) for large values of n (n ≥ n0). The definition of theta also
requires that f(n) must be non-negative for values of n greater than n0.

The execution time serves as both a lower and upper bound on the algorithm’s time
complexity.

It exist as both, most, and least boundaries for a given input value.

A simple way to get the Theta notation of an expression is to drop low-order terms and ignore
leading constants. For example, Consider the expression 3n3 + 6n2 + 6000 = Θ(n3), the
dropping lower order terms is always fine because there will always be a number(n) after
which Θ(n3) has higher values than Θ(n2) irrespective of the constants involved. For a given
function g(n), we denote Θ(g(n)) is following set of functions. Examples :
{ 100 , log (2000) , 10^4 } belongs to Θ(1)
{ (n/4) , (2n+3) , (n/100 + log(n)) } belongs to Θ(n)
{ (n^2+n) , (2n^2) , (n^2+log(n))} belongs to Θ( n2)

Note: Θ provides exact bounds.

2. Big-O Notation (O-notation):

Big-O notation represents the upper bound of the running time of an algorithm. Therefore, it
gives the worst-case complexity of an algorithm. It is the most widely used notation for
Asymptotic analysis. It specifies the upper bound of a function.The maximum time required by
an algorithm or the worst-case time complexity. It returns the highest possible output
value(big-O) for a given input. Big-Oh(Worst Case) It is defined as the condition that allows an
algorithm to complete statement execution in the shortest amount of time possible.
If f(n) describes the running time of an algorithm, f(n) is O(g(n)) if there exist a positive
constant C and n0 such that, 0 ≤ f(n) ≤ cg(n) for all n ≥ n0. It returns the highest possible
output value (big-O) for a given input. The execution time serves as an upper bound on the
algorithm’s time complexity.

Mathematical Representation of Big-O Notation:

O(g(n)) = { f(n): there exist positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0
} For example, Consider the case of Insertion Sort. It takes linear time in the best case and
quadratic time in the worst case. We can safely say that the time complexity of the Insertion sort
is O(n2).
2
Note: O(n ) also covers linear time.

If we use Θ notation to represent the time complexity of Insertion sort, we have to use two
statements for best and worst cases:

 The worst-case time complexity of Insertion Sort is Θ(n2).

  The best case time complexity of Insertion Sort is Θ(n).

The Big-O notation is useful when we only have an upper bound on the time complexity of an
algorithm. Many times we easily find an upper bound by simply looking at the algorithm.

Examples :

{ 100 , log (2000) , 10^4 } belongs to O(1)

U{ (n/4) , (2n+3) , (n/100 + log(n)) } belongs to O(n)
U { (n^2+n) , (2n^2) , (n^2+log(n))} belongs to O( n^2)

Note: Here, U represents union, we can write it in these manner because O provides exact or
upper bound

3. Omega Notation (Ω-Notation):

Omega notation represents the lower bound of the running time of an algorithm. Thus, it
provides the best case complexity of an algorithm.

The execution time serves as a lower bound on the algorithm’s time complexity.

It is defined as the condition that allows an algorithm to complete statement execution in

the shortest amount of time.

Let g and f be the function from the set of natural numbers to itself. The function f is said to be
Ω(g), if there is a constant c > 0 and a natural number n0 such that c*g(n) ≤ f(n) for all n ≥ n0

Mathematical Representation of Omega notation :

Ω(g(n)) = { f(n): there exist positive constants c and n0 such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0
}

Let us consider the same Insertion sort example here. The time complexity of Insertion Sort can
be written as Ω(n), but it is not very useful information about insertion sort, as we are generally
interested in worst-case and sometimes in the average case.

Examples :

{ (n^2+n) , (2n^2) , (n^2+log(n))} belongs to Ω( n^2)

U{ (n/4) , (2n+3) , (n/100 + log(n)) } belongs to Ω(n)
U { 100 , log (2000) , 10^4 } belongs to Ω(1)

Note: Here, U represents union, we can write it in these manner because Ω provides exact or
lower bounds.

How to Analyse Loops for Complexity Analysis of Algorithms?

The analysis of loops for the complexity analysis of algorithms involves finding the number of
operations performed by a loop as a function of the input size. This is usually done by
determining the number of iterations of the loop and the number of operations performed in
each iteration. Here are the general steps to analyze loops for complexity analysis:

Determine the number of iterations of the loop. This is usually done by analyzing the loop
control variables and the loop termination condition.

Determine the number of operations performed in each iteration of the loop. This can include
both arithmetic operations and data access operations, such as array accesses or memory
accesses.

Express the total number of operations performed by the loop as a function of the input size.
This may involve using mathematical expressions or finding a closed-form expression for the
number of operations performed by the loop.

Determine the order of growth of the expression for the number of operations performed by the
loop. This can be done by using techniques such as big O notation or by finding the dominant
term and ignoring lower-order terms.

Constant Time Complexity O(1):

The time complexity of a function (or set of statements) is considered as O(1) if it doesn’t
contain a loop, recursion, and call to any other non-constant time function.
i.e. set of non-recursive and non-loop statements

In computer science, O(1) refers to constant time complexity, which means that the running
time of an algorithm remains constant and does not depend on the size of the input. This means
that the execution time of an O(1) algorithm will always take the same amount of time
regardless of the input size. An example of an O(1) algorithm is accessing an element in an
array using an index.

Example:

 swap() function has O(1) time complexity.

 A loop or recursion that runs a constant number of times is also considered O(1). For
example, the following loop is O(1).

// Here c is a constant
for (int i = 1; i <= c; i++) {
// some O(1) expressions
}
Linear Time Complexity O(n):
The Time Complexity of a loop is considered as O(n) if the loop variables are
incremented/decremented by a constant amount. For example following functions have O(n)
time complexity. Linear time complexity, denoted as O(n), is a measure of the growth of the
running time of an algorithm proportional to the size of the input. In an O(n) algorithm, the
running time increases linearly with the size of the input. For example, searching for an
element in an unsorted array or iterating through an array and performing a constant amount
of work for each element would be O(n) operations. In simple words, for an input of size n, the
algorithm takes n steps to complete the operation.
1. // Here c is a positive integer constant
for (int i = 1; i <= n; i += c) {
// some O(1) expressions
}

2. for (int i = n; i > 0; i -= c) {

// some O(1) expressions
}
Quadratic Time Complexity O(nc):
The time complexity is defined as an algorithm whose performance is directly proportional to
the squared size of the input data, as in nested loops it is equal to the number of times the
innermost statement is executed. For example, the following sample loops have O(n2) time
complexity
Quadratic time complexity, denoted as O(n^2), refers to an algorithm whose running time
increases proportional to the square of the size of the input. In other words, for an input of
size n, the algorithm takes n * n steps to complete the operation. An example of an O(n^2)
algorithm is a nested loop that iterates over the entire input for each element, performing a
constant amount of work for each iteration. This results in a total of n * n iterations, making
the running time quadratic in the size of the input.
(1) for (int i = 1; i <= n; i += c) {
for (int j = 1; j <= n; j += c) {
// some O(1) expressions
}
}
for (int i = n; i > 0; i -= c) {
for (int j = i + 1; j <= n; j += c) {
// some O(1) expressions
}
}
Time function : 2n2
Time complexity : O(n 2)
Logarithmic Time Complexity O(Log n):
The time Complexity of a loop is considered as O(Logn) if the loop variables are
divided/multiplied by a constant amount. And also for recursive calls in the recursive function,
the Time Complexity is considered as O(Logn).
for (int i = 1; i <= n; i *= c) {
// some O(1) expressions
}
for (int i = n; i > 0; i /= c) {
// some O(1) expressions
}
// Recursive function
void recurse(int n)
{
if (n <= 0)
return;
else {
// some O(1) expressions
}
recurse(n/c);
// Here c is positive integer constant greater than 1
}
Binary search also has O(log n) complexity.
Logarithmic Time Complexity O(Log Log n):
The Time Complexity of a loop is considered as O(LogLogn) if the loop variables are
reduced/increased exponentially by a constant amount.
// Here c is a constant greater than 1
for (int i = 2; i <= n; i = pow(i, c)) {
// some O(1) expressions
}
// Here fun is sqrt or cuberoot or any other constant root
for (int i = n; i > 1; i = fun(i)) {
// some O(1) expressions
}
How to combine the time complexities of consecutive loops?
When there are consecutive loops, we calculate time complexity as a sum of the time
complexities of individual loops.
To combine the time complexities of consecutive loops, you need to consider the number of
iterations performed by each loop and the amount of work performed in each iteration. The
total time complexity of the algorithm can be calculated by multiplying the number of
iterations of each loop by the time complexity of each iteration and taking the maximum of all
possible combinations.
For example, consider the following code:
for i in range(n):
for j in range(m):
 # some constant time operation
Here, the outer loop performs n iterations, and the inner loop performs m iterations for each
iteration of the outer loop. So, the total number of iterations performed by the inner loop is n
* m, and the total time complexity is O(n * m).
In another example, consider the following code:
for i in range(n):
for j in range(i):
# some constant time operation
Here, the outer loop performs n iterations, and the inner loop performs i iterations for each
iteration of the outer loop, where i is the current iteration count of the outer loop. The total
number of iterations performed by the inner loop can be calculated by summing the number
of iterations performed in each iteration of the outer loop, which is given by the formula
sum(i) from i=1 to n, which is equal to n * (n + 1) / 2. Hence, the total time complex
for (int i = 1; i <= m; i += c) {
// some O(1) expressions
}
for (int i = 1; i <= n; i += c) {
// some O(1) expressions
}

// Time complexity of above code is O(m) + O(n) which is O(m + n)

// If m == n, the time complexity becomes O(2n) which is O(n).
How to calculate time complexity when there are many if, else statements inside loops?
As discussed here, the worst-case time complexity is the most useful among best, average and
worst. Therefore we need to consider the worst case. We evaluate the situation when values
in if-else conditions cause a maximum number of statements to be executed.
For example, consider the linear search function where we consider the case when an element
is present at the end or not present at all.
When the code is too complex to consider all if-else cases, we can get an upper bound by
ignoring if-else and other complex control statements.
Question: What is the time complexity of following function fun()? Assume that log(x)
returns log value in base 2.
void fun()
{
int i, j;
for (i = 1; i <= n; i++)
for (j = 1; j <= log(i); j++)
printf("GeeksforGeeks");
}
Time Complexity of the above function can be written as θ(log 1) + θ(log 2) + θ(log 3) + . . . . +
θ(log n) which is θ(log n!)
Order of growth of ‘log n!’ and ‘n log n’ is same for large values of n, i.e., θ(log n!) = θ(n log n).
So time complexity of fun() is θ(n log n).
What does ‘Space Complexity’ mean?
Space Complexity:
The term Space Complexity is misused for Auxiliary Space at many places. Following are the
correct definitions of Auxiliary Space and Space Complexity.
Auxiliary Space is the extra space or temporary space used by an algorithm.
The space Complexity of an algorithm is the total space taken by the algorithm with respect to
the input size. Space complexity includes both Auxiliary space and space used by input.
For example, if we want to compare standard sorting algorithms on the basis of space, then
Auxiliary Space would be a better criterion than Space Complexity. Merge Sort uses O(n)
auxiliary space, Insertion sort, and Heap Sort use O(1) auxiliary space. The space complexity of
all these sorting algorithms is O(n) though.
Space complexity is a parallel concept to time complexity. If we need to create an array of size
n, this will require O(n) space. If we create a two-dimensional array of size n*n, this will
require O(n2) space.
In recursive calls stack space also counts.
Example :
int add (int n){
if (n <= 0){
return 0;
}
return n + add (n-1);
}

Here each call add a level to the stack :

1. add(4)
2. -> add(3)
3. -> add(2)
4. -> add(1)
5. -> add(0)
Each of these calls is added to call stack and takes up actual memory.
So it takes O(n) space.
However, just because you have n calls total doesn’t mean it takes O(n) space.
Look at the below function :
int addSequence (int n){
int sum = 0;
for (int i = 0; i < n; i++){
sum += pairSum(i, i+1);
}
return sum;
}

int pairSum(int x, int y){

 return x + y;
}

There will be roughly O(n) calls to pairSum. However, those calls do not exist simultaneously
on the call stack, so you only need O(1) space.
Note: It’s necessary to mention that space complexity depends on a variety of things such as
the programming language, the compiler, or even the machine running the algorithm.
Time-Space Trade-Off in Algorithms
Space-Time tradeoff in computer science is basically a problem solving technique in which we
solve the problem:
 Either in less time and using more space, or
 In very little space by spending more time.
The best algorithm is the one which helps to solve a problem that requires less space in
memory as well as takes less time to generate the output.But in general, it is not always
possible to achieve both of these conditions at the same time.
If our problem is taking a long time but not much memory, a space-time trade-off would let us
use more memory and solve the problem more quickly. Or, if it could be solved very quickly
but requires more memory than we have, we can try to spend more time solving the problem
in the limited memory.

Arrays:
An array is a linear data structure and it is a collection of items stored at contiguous memory
locations. The idea is to store multiple items of the same type together in one place. It allows
the processing of a large amount of data in a relatively short period. The first element of the
array is indexed by a subscript of 0. There are different operations possible in an array, like
Searching, Sorting, Inserting, Traversing, Reversing, and Deleting.
(1) Array traversal
Given an integer array of size N, the task is to traverse and print the elements in the array.
Examples:
Input: arr[] = {2, -1, 5, 6, 0, -3}
Output: 2 -1 5 6 0 -3
Input: arr[] = {4, 0, -2, -9, -7, 1}
Output: 4 0 -2 -9 -7 1
Approach:-
1. Start a loop from 0 to N-1, where N is the size of array.
for(i = 0; i < N; i++)
2. Access every element of array with help of
arr[index]
3. Print the elements.
printf("%d ", arr[i])
Below is the implementation of the above approach:
// C program to traverse the array

#include <stdio.h>

// Function to traverse and print the array

void printArray(int* arr, int n)
{
int i;

printf("Array: ");
for (i = 0; i < n; i++) {
printf("%d ", arr[i]);
}
printf("\n");
}

// Driver program
int main()
{
int arr[] = { 2, -1, 5, 6, 0, -3 };
int n = sizeof(arr) / sizeof(arr[0]);

printArray(arr, n);

return 0;
}
Output:
Array: 2 -1 5 6 0 -3

Time Complexity: O(n) //since one traversal of the array is required to complete all operations
hence overall time required by the algorithm is linear
Auxiliary Space: O(1)// since no extra array is used so the space taken by the algorithm is
constant

(2) Insertion in array

(a) Insertion at the Beginning of an Array

When the insertion happens at the beginning, it causes all the existing data items to shift one
step downward. Here, we design and implement an algorithm to insert an element at the
beginning of an array.
Algorithm
We assume A is an array with N elements. The maximum numbers of elements it can store is
defined by MAX. We shall first check if an array has any empty space to store any element and
then we proceed with the insertion process.

begin

IF N = MAX, return

ELSE

N=N+1

For All Elements in A

Move to next adjacent location

A[FIRST] = New_Element

end

Implementation in C

#include <stdio.h>

#define MAX 5

void main() {
int array[MAX] = {2, 3, 4, 5};
int N = 4; // number of elements in array
int i = 0; // loop variable
int value = 1; // new data element to be stored in array

// print array before insertion

 printf("Printing array before insertion −\n");

for(i = 0; i < N; i++) {

printf("array[%d] = %d \n", i, array[i]);
}
// now shift rest of the elements downwards
for(i = N; i >= 0; i--) {
array[i+1] = array[i];
}

// add new element at first position

array[0] = value;

// increase N to reflect number of elements

N++;

// print to confirm

printf("Printing array after insertion −\n");

for(i = 0; i < N; i++) {

printf("array[%d] = %d\n", i, array[i]);
}

Output

This program should yield the following output −

Printing array before insertion −

array[0] = 2

array[1] = 3

array[2] = 4

array[3] = 5

Printing array after insertion −

array[0] = 0

array[1] = 2
array[2] = 3

array[3] = 4

array[4] = 5

(b) Insertion at end

Requires only one shift

C program:

#include<stdio.h>
#include<conio.h>
int main()
{
int arr[10], i, element;
printf("Enter 5 Array Elements: ");
for(i=0; i<5; i++)
scanf("%d", &arr[i]);
printf("\nEnter Element to Insert: ");
scanf("%d", &element);
arr[i] = element;
printf("\nThe New Array is:\n");
for(i=0; i<6; i++)
printf("%d ", arr[i]);
getch();
return 0;
}

(3) Insertion at any position

Follow the below steps to solve the problem:
 First get the element to be inserted, say x
 Then get the position at which this element is to be
inserted, say pos
 Then shift the array elements from this position to one
position forward(towards right), and do this for all the
other elements next to pos.
 Insert the element x now at the position pos, as this is
now empty.
#include <stdio.h>

int main()
{
int arr[100] = { 0 };
int i, x, pos, n = 10;

// initial array of size 10

for (i = 0; i < 10; i++)
arr[i] = i + 1;

// print the original array

for (i = 0; i < n; i++)
printf("%d ", arr[i]);
printf("\n");

// element to be inserted
x = 50;

// position at which element

// is to be inserted
pos = 5;

// increase the size by 1

n++;

// shift elements forward

for (i = n - 1; i >= pos; i--)
arr[i] = arr[i - 1];

// insert x at pos
arr[pos - 1] = x;

// print the updated array

for (i = 0; i < n; i++)
printf("%d ", arr[i]);
printf("\n");
 return 0;
}
Output
1 2 3 4 5 6 7 8 9 10
1 2 3 4 50 5 6 7 8 9 10
Time Complexity: O(N), Where N is the number of elements in the
array
Auxiliary Space: O(1)

DELETION
Deletion in array means removing an element and replacing it with the next element or
element present at next index. It involves three cases:
(a) Deletion in Array- an element at the beginning of the array:-
In this case we have to move all the elements one position forward to fill the position of the
element at the beginningof array. Though the deletion process is not difficult but moving all
elements one position forward involve movement of all the existing elements except the one
being deleted. This is the worst case scenario in deletion in a linear array.
In the example array elements from index 1 to index 8 have to moved one position forwards so
that the first element is replaced by second, second by third and so on.

(b) Deletion in Array – an element at the end of the array

In this case we don’t have to move any elements since the action here will be just removing the
last element. This is done by redefining the index of last element of linear array = N-1. This is
the best case scenario in deletion in a linear array.
In the example array no elements are moved. The last element is removed by setting the index
of last element as 7.
C code:
#include <stdio.h>
void main()
{
int position, i, n, value,ch;
printf("C Program to delete element at end of Array\n");
printf("First enter number of elements you want in Array\n");
scanf("%d", &n);
int arr[n];
for(i = 0; i < n; i++)
{
printf("Please give value for index %d : ",i);
scanf("%d",&arr[i]);
}
value=arr[n-1]; //assigning last value in value variable
printf("Element %d is deleting at %d index \n",value,n-1);
n=n-1;//here decreasing value to reduce size of array
printf("New Array after deleting element at end \n ");
for(i = 0; i < n; i++)
{
printf("%d \t",arr[i]);
}
}

(c) Deletion in Array – an element at the give position J

Let J be any location in the array for one existing element. We have to delete the element at J
position. To do this starting from J every element is moved one place forward so that the
element after index J comes to position of Jth element. This is the average case scenario in
deletion in linear array.
In the example array ,elements from index J (4) to index 8 have to moved one position forward
so that an element at index 4 is replaced by element at index 5. Similarly element at 6 th postion
comes at 5th position, element at 7th position comes at 6th position and element at 8th position
replaces element at 7th position completing the deletion process
Algorithm for Element Deletion in Array
Algorithm DeleteLA (DATA, N, ITEM, LOC)
Desc: This algorithm deletes an element at Jth position in a linear array
DATA with N elements
and stores in ITEM
If LOC=1 it means the element to be deleted is at the beginning
If LOC =N it means the element be deleted is at the end
If LOC = J it means the elements have to be deleted is at at Jth Location
Begin
Step 1: [Initialize counter I with index of element to be deleted]
I=J
Step 2: [Store the element to be deleted in ITEM]
ITEM=DATA[J]
Step 3: While I<N repeat steps 4 and 5
Step 4: [Move the current element one position forward]
DATA[I]=DATA[I+1]
Step 5: [Increment counter I]
I=I+1
Step 6: [Update total under of array elements]
N=N-1
Exit
Complexity of Deletion in Array Algorithm:
Deletion operation results deleting one element of a linear array . In the algorithm the following
is the way the steps are counted.
1. Steps 1, 2 and 6 are executed once, so they contributes 3 to complexity function f(n)
2. Step 3 is a loop control step that executes step 4 and step 5 N times (once for each
element of array DATA after the element to be deleted)
So the f(n) can be defined as
f(n)= 2*n+3 or
f(n)=2n+3
The highest order term in this function defined in the terms of input size of the algorithm is n,
so we can say that the complexity of this algorithm is O(n). It also means that the average time
of element deletion in array grows linearly with the size of array DATA.
AT ANY POSITION:
/*
* C program to delete an element from array at specified position
 */

#include <stdio.h>
#define MAX_SIZE 100

int main()
{
int arr[MAX_SIZE];
int i, size, pos;

/* Input size and element in array */

printf("Enter size of the array : ");
scanf("%d", &size);
printf("Enter elements in array : ");
for(i=0; i<size; i++)
{
scanf("%d", &arr[i]);
}

/* Input element position to delete */

printf("Enter the element position to delete : ");
scanf("%d", &pos);

/* Invalid delete position */

if(pos < 0 || pos > size)
{
printf("Invalid position! Please enter position between 1 to %d", size);
}
else
{
/* Copy next element value to current element */
for(i=pos-1; i<size-1; i++)
{
arr[i] = arr[i + 1];
}

/* Decrement array size by 1 */

size--;

/* Print array after deletion */

printf("\nElements of array after delete are : ");
for(i=0; i<size; i++)
{
printf("%d\t", arr[i]);
}
}

return 0;
}