Lecture 1

An Algorithm, Definition, Types & Characteristics

1.1 An Algorithm
An algorithm is a set of commands that must be followed for a computer to perform calculations
or other problem-solving operations. According to its formal definition, an algorithm is a finite
set of instructions carried out in a specific order to perform a particular task. It is not the entire
program or code; it is simple logic to a problem represented as an informal description in the
form of a flowchart or pseudocode.

 Problem: A problem can be defined as a real-world problem or real-world instance problem

for which you need to develop a program or set of instructions. An algorithm is a set of
instructions.
 Algorithm: An algorithm is defined as a step-by-step process that will be designed for a
problem.
 Input: After designing an algorithm, the algorithm is given the necessary and desired inputs.
 Processing unit: The input will be passed to the processing unit, producing the desired output.
 Output: The outcome or result of the program is referred to as the output.
After defining what an algorithm is, you will now look at algorithm characteristics.

1.2 How Algorithms Work

Algorithms are step-by-step procedures designed to solve specific problems and perform tasks
efficiently in the realm of computer science and mathematics. These powerful sets of instructions
form the backbone of modern technology and govern everything from web searches to artificial
intelligence. Here's how algorithms work:

 Input: Algorithms take input data, which can be in various formats, such as numbers, text, or
images.
 Processing: The algorithm processes the input data through a series of logical and
mathematical operations, manipulating and transforming it as needed.
 Output: After the processing is complete, the algorithm produces an output, which could be a
result, a decision, or some other meaningful information.
 Efficiency: A key aspect of algorithms is their efficiency, aiming to accomplish tasks quickly
and with minimal resources.
 Optimization: Algorithm designers constantly seek ways to optimize their algorithms, making
them faster and more reliable.
 Implementation: Algorithms are implemented in various programming languages, enabling
computers to execute them and produce desired outcomes.

1.3 The Need for Algorithms

You require algorithms for the following reasons:

i. Scalability

It aids in your understanding of scalability. When you have a sizable real-world problem, you
must break it down into small steps to analyze it quickly.

ii. Performance

The real world is challenging to break down into smaller steps. If a problem can be easily
divided into smaller steps, it indicates that the problem is feasible.

After understanding what is an algorithm, why you need an algorithm, you will look at how to
write one using an example.

1.4 Types of Algorithms

1. Brute Force Algorithm: A straightforward approach that exhaustively tries all possible
solutions, suitable for small problem instances but may become impractical for larger ones due
to its high time complexity.
2. Recursive Algorithm: A method that breaks a problem into smaller, similar subproblems and
repeatedly applies itself to solve them until reaching a base case, making it effective for tasks
with recursive structures.
3. Encryption Algorithm: Utilized to transform data into a secure, unreadable form using
cryptographic techniques, ensuring confidentiality and privacy in digital communications and
transactions.
4. Backtracking Algorithm: A trial-and-error technique used to explore potential solutions by
undoing choices when they lead to an incorrect outcome, commonly employed in puzzles and
optimization problems.
5. Searching Algorithm: Designed to find a specific target within a dataset, enabling efficient
retrieval of information from sorted or unsorted collections.
6. Sorting Algorithm: Aimed at arranging elements in a specific order, like numerical or
alphabetical, to enhance data organization and retrieval.
7. Hashing Algorithm: Converts data into a fixed-size hash value, enabling rapid data access and
retrieval in hash tables, commonly used in databases and password storage.
8. Divide and Conquer Algorithm: Breaks a complex problem into smaller subproblems, solves
them independently, and then combines their solutions to address the original problem
effectively.

2
9. Greedy Algorithm: Makes locally optimal choices at each step in the hope of finding a global
optimum, useful for optimization problems but may not always lead to the best solution.
10. Dynamic Programming Algorithm: Stores and reuses intermediate results to avoid redundant
computations, enhancing the efficiency of solving complex problems.
11. Randomized Algorithm: Utilizes randomness in its steps to achieve a solution, often used in
situations where an approximate or probabilistic answer suffices.

1.5 How to Write an Algorithm?

 There are no well-defined standards for writing algorithms. It is, however, a problem that is
resource-dependent. Algorithms are never written with a specific programming language in
mind.
 As you all know, basic code constructs such as loops like do, for, while, all programming
languages share flow control such as if-else, and so on. An algorithm can be written using
these common constructs.
 Algorithms are typically written in a step-by-step fashion, but this is not always the case.
Algorithm writing is a process that occurs after the problem domain has been well-defined.
That is, you must be aware of the problem domain for which you are developing a solution.

Example

Now, use an example to learn how to write algorithms.

Problem: Create an algorithm that multiplies two numbers and displays the output.

Step 1 − Start

Step 2 − declare three integers variables x, y & z

Step 3 − define values of x & y

Step 4 − multiply values of x & y

Step 5 − store result of step 4 to z

Step 6 − print z

Step 7 − Stop

Algorithms instruct programmers on how to write code. In addition, the algorithm can be written
as:

Step 1 − Start mul

Step 2 − get values of x & y

3
 Step 3 − z ← x * y

Step 4 − display z

Step 5 − Stop

In algorithm design and analysis, the second method is typically used to describe an algorithm. It
allows the analyst to analyze the algorithm while ignoring all unwanted definitions easily. They
can see which operations are being used and how the process is progressing. It is optional to
write step numbers. To solve a given problem, you create an algorithm. A problem can be solved
in a variety of ways.

As a result, many solution algorithms for a given problem can be derived. The following step is
to evaluate the proposed solution algorithms and implement the most appropriate solution.

As you progress through this "what is an Algorithm" tutorial, you will learn about some of the
components of an algorithm.

1.6 Factors to consider when designing an Algorithm an Algorithm

The following are the factors to consider when designing an algorithm:

 Modularity: This feature was perfectly designed for the algorithm if you are given a problem
and break it down into small-small modules or small-small steps, which is a basic definition of
an algorithm.
 Correctness: An algorithm's correctness is defined as when the given inputs produce the
desired output, indicating that the algorithm was designed correctly. An algorithm's analysis
has been completed correctly.

4
 Maintainability: It means that the algorithm should be designed in a straightforward,
structured way so that when you redefine the algorithm, no significant changes are made to the
algorithm.
 Functionality: It takes into account various logical steps to solve a real-world problem.
 Robustness: Robustness refers to an algorithm's ability to define your problem clearly.
 User-friendly: If the algorithm is difficult to understand, the designer will not explain it to the
programmer.
 Simplicity: If an algorithm is simple, it should be simple to understand.
 Extensibility: Your algorithm should be extensible if another algorithm designer or
programmer wants to use it.
You will now see why an algorithm is so essential after understanding some of its components.

1.7 Qualities of a Good Algorithm

 Efficiency: A good algorithm should perform its task quickly and use minimal resources.
 Correctness: It must produce the correct and accurate output for all valid inputs.
 Clarity: The algorithm should be easy to understand and comprehend, making it maintainable
and modifiable.
 Scalability: It should handle larger data sets and problem sizes without a significant decrease
in performance.
 Reliability: The algorithm should consistently deliver correct results under different conditions
and environments.
 Optimality: Striving for the most efficient solution within the given problem constraints.
 Robustness: Capable of handling unexpected inputs or errors gracefully without crashing.
 Adaptability: Ideally, it can be applied to a range of related problems with minimal
adjustments.
 Simplicity: Keeping the algorithm as simple as possible while meeting its requirements,
avoiding unnecessary complexity.

1.8 The Complexity of an Algorithm

The algorithm's performance can be measured in two ways:

a. Time Complexity

The amount of time required to complete an algorithm's execution is called time complexity. The
big O notation is used to represent an algorithm's time complexity. The asymptotic notation for
describing time complexity, in this case, is big O notation. The time complexity is calculated
primarily by counting the number of steps required to complete the execution. Let us look at an
example of time complexity.

5
 mul = 1;

// Suppose you have to calculate the multiplication of n

numbers.

for i=1 to n

mul = mul *1;

// when the loop ends, then mul holds the multiplication of the n
numbers

return mul;

The time complexity of the loop statement in the preceding code is at least n, and as the value of
n escalates, so does the time complexity. While the code's complexity, i.e., returns mul, will be
constant because its value is not dependent on the importance of n and will provide the result in a
single step. The worst-time complexity is generally considered because it is the maximum time
required for any given input size.

b. Space Complexity

The amount of space an algorithm requires to solve a problem and produce an output is called its
space complexity. Space complexity, like time complexity, is expressed in big O notation.

The space is required for an algorithm for the following reasons:

1. To store program instructions.

2. To store track of constant values.
3. To store track of variable values.
4. To store track of function calls, jumping statements, and so on.
Space Complexity = Auxiliary Space + Input Size

Finally after understanding what is an algorithm, its analysis and approches, you will look at
different types of algorithms.

1.9 Most important type of Algorithms

1. Brute Force Algorithm:

This is the most basic and simplest type of algorithm. A Brute Force Algorithm is the
straightforward approach to a problem i.e., the first approach that comes to our mind on
seeing the problem. More technically it is just like iterating every possibility available to
solve that problem.
Example:
If there is a lock of 4-digit PIN. The digits to be chosen from 0-9 then the brute force will
be trying all possible combinations one by one like 0001, 0002, 0003, 0004, and so on
until we get the right PIN. In the worst case, it will take 10,000 tries to find the right
combination.
6
2. Recursive Algorithm:
This type of algorithm is based on recursion. In recursion, a problem is solved by
breaking it into subproblems of the same type and calling own self again and again until
the problem is solved with the help of a base condition.
Some common problem that is solved using recursive algorithms are Factorial of a
Number, Fibonacci Series, Tower of Hanoi, DFS for Graph, etc.
a) Divide and Conquer Algorithm:
In Divide and Conquer algorithms, the idea is to solve the problem in two sections, the
first section divides the problem into subproblems of the same type. The second section is
to solve the smaller problem independently and then add the combined result to produce
the final answer to the problem.
Some common problem that is solved using Divide and Conquers Algorithms are Binary
Search, Merge Sort, Quick Sort, Strassen’s Matrix Multiplication, etc.
b) Dynamic Programming Algorithms:
This type of algorithm is also known as the memoization technique because in this the
idea is to store the previously calculated result to avoid calculating it again and again. In
Dynamic Programming, divide the complex problem into smaller overlapping
subproblems and store the result for future use.
The following problems can be solved using the Dynamic Programming
algorithm Knapsack Problem, Weighted Job Scheduling, Floyd Warshall Algorithm, etc.
c) Greedy Algorithm:
In the Greedy Algorithm, the solution is built part by part. The decision to choose the
next part is done on the basis that it gives an immediate benefit. It never considers the
choices that had been taken previously.
Some common problems that can be solved through the Greedy Algorithm are Dijkstra
Shortest Path Algorithm, Prim’s Algorithm, Kruskal’s Algorithm, Huffman Coding, etc.
d) Backtracking Algorithm:
In Backtracking Algorithm, the problem is solved in an incremental way i.e. it is an
algorithmic technique for solving problems recursively by trying to build a solution
incrementally, one piece at a time, removing those solutions that fail to satisfy the
constraints of the problem at any point of time.
Some common problems that can be solved through the Backtracking Algorithm are
the Hamiltonian Cycle, M-Coloring Problem, N Queen Problem, Rat in Maze Problem,
etc.
3. Randomized Algorithm:
In the randomized algorithm, we use a random number.it helps to decide the expected
outcome. The decision to choose the random number so it gives the immediate benefit
Some common problems that can be solved through the Randomized Algorithm are
Quicksort: In Quicksort we use the random number for selecting the pivot.
4. Sorting Algorithm:
The sorting algorithm is used to sort data in maybe ascending or descending order. Its
also used for arranging data in an efficient and useful manner.
Some common problems that can be solved through the sorting Algorithm are Bubble
sort, insertion sort, merge sort, selection sort, and quick sort are examples of the Sorting
algorithm.
5. Searching Algorithm:
The searching algorithm is the algorithm that is used for searching the specific key in
particular sorted or unsorted data. Some common problems that can be solved through the
Searching Algorithm are Binary search or linear search is one example of a Searching
algorithm.
7
6. Hashing Algorithm:
Hashing algorithms work the same as the Searching algorithm but they contain an index
with a key ID i.e a key-value pair. In hashing, we assign a key to specific data.
Some common problems can be solved through the Hashing Algorithm in password
verification.

1.10 Advantage and Disadvantages of Algorithms

1.10.1 Advantages of Algorithms:

 Efficiency: Algorithms streamline processes, leading to faster and more optimized

solutions.
 Reproducibility: They yield consistent results when provided with the same inputs.
 Problem-solving: Algorithms offer systematic approaches to tackle complex
problems effectively.
 Scalability: Many algorithms can handle larger datasets and scale with increasing
input sizes.
 Automation: They enable automation of tasks, reducing the need for manual
intervention.

1.10.2 Disadvantages of Algorithms:

 Complexity: Developing sophisticated algorithms can be challenging and time-

consuming.
 Limitations: Some problems may not have efficient algorithms, leading to
suboptimal solutions.
 Resource Intensive: Certain algorithms may require significant computational
resources.
 Inaccuracy: Inappropriate algorithm design or implementation can result in
incorrect outputs.
 Maintenance: As technology evolves, algorithms may require updates to stay
relevant and effective.

Next Steps

In this tutorial, you learned what an algorithm is and what its characteristics are. After that, you
took a look at why you need algorithms, how to write them, and how important they are. After
you learned about the approaches and factors of an algorithm, you learned about complexity and
types of algorithms.