In computer science, a search algorithm is an algorithm designed to solve a search problem.

Search
algorithms work to retrieve information stored within particular data structure, or calculated in the
search space of a problem domain, with either discrete or continuous values.

Although search engines use search algorithms, they belong to the study of information retrieval, not
algorithmics.

The appropriate search algorithm to use often depends on the data structure being searched, and may
also include prior knowledge about the data. Search algorithms can be made faster or more efficient by
specially constructed database structures, such as search trees, hash maps, and database indexes.[1][full
citation needed][2]

Search algorithms can be classified based on their mechanism of searching into three types of
algorithms: linear, binary, and hashing. Linear search algorithms check every record for the one
associated with a target key in a linear fashion.[3] Binary, or half-interval, searches repeatedly target the
center of the search structure and divide the search space in half. Comparison search algorithms improve
on linear searching by successively eliminating records based on comparisons of the keys until the target
record is found, and can be applied on data structures with a defined order.[4] Digital search algorithms
work based on the properties of digits in data structures by using numerical keys.[5] Finally, hashing
directly maps keys to records based on a hash function.[6]

Algorithms are often evaluated by their computational complexity, or maximum theoretical run time.
Binary search functions, for example, have a maximum complexity of O(log n), or logarithmic time. In
simple terms, the maximum number of operations needed to find the search target is a logarithmic
function of the size of the search space.

Applications of search algorithms

Specific applications of search algorithms include:

Problems in combinatorial optimization, such as:

The vehicle routing problem, a form of shortest path problem

The knapsack problem: Given a set of items, each with a weight and a value, determine the number of
each item to include in a collection so that the total weight is less than or equal to a given limit and the
total value is as large as possible.

The nurse scheduling problem

Problems in constraint satisfaction, such as:

The map coloring problem

Filling in a sudoku or crossword puzzle

In game theory and especially combinatorial game theory, choosing the best move to make next (such as
with the minmax algorithm)

Finding a combination or password from the whole set of possibilities

Factoring an integer (an important problem in cryptography)

Optimizing an industrial process, such as a chemical reaction, by changing the parameters of the process
(like temperature, pressure, and pH)

Retrieving a record from a database

Finding the maximum or minimum value in a list or array

Checking to see if a given value is present in a set of values

Classes

For virtual search spaces

See also: Solver

Algorithms for searching virtual spaces are used in the constraint satisfaction problem, where the goal is
to find a set of value assignments to certain variables that will satisfy specific mathematical equations
and inequations / equalities. They are also used when the goal is to find a variable assignment that will
maximize or minimize a certain function of those variables. Algorithms for these problems include the
basic brute-force search (also called "naïve" or "uninformed" search), and a variety of heuristics that try
to exploit partial knowledge about the structure of this space, such as linear relaxation, constraint
generation, and constraint propagation.

An important subclass are the local search methods, that view the elements of the search space as the
vertices of a graph, with edges defined by a set of heuristics applicable to the case; and scan the space
by moving from item to item along the edges, for example according to the steepest descent or best-first
criterion, or in a stochastic search. This category includes a great variety of general metaheuristic
methods, such as simulated annealing, tabu search, A-teams, and genetic programming, that combine
arbitrary heuristics in specific ways. The opposite of local search would be global search methods. This
method is applicable when the search space is not limited and all aspects of the given network are
available to the entity running the search algorithm.[7]

This class also includes various tree search algorithms, that view the elements as vertices of a tree, and
traverse that tree in some special order. Examples of the latter include the exhaustive methods such as
depth-first search and breadth-first search, as well as various heuristic-based search tree pruning
methods such as backtracking and branch and bound. Unlike general metaheuristics, which at best work
only in a probabilistic sense, many of these tree-search methods are guaranteed to find the exact or
optimal solution, if given enough time. This is called "completeness".

Another important sub-class consists of algorithms for exploring the game tree of multiple-player games,
such as chess or backgammon, whose nodes consist of all possible game situations that could result from
the current situation. The goal in these problems is to find the move that provides the best chance of a
win, taking into account all possible moves of the opponent(s). Similar problems occur when humans or
machines have to make successive decisions whose outcomes are not entirely under one's control, such
as in robot guidance or in marketing, financial, or military strategy planning. This kind of problem —
combinatorial search — has been extensively studied in the context of artificial intelligence. Examples of
algorithms for this class are the minimax algorithm, alpha–beta pruning, and the A* algorithm and its
variants.

For sub-structures of a given structure

The name "combinatorial search" is generally used for algorithms that look for a specific sub-structure of
a given discrete structure, such as a graph, a string, a finite group, and so on. The term combinatorial
optimization is typically used when the goal is to find a sub-structure with a maximum (or minimum)
value of some parameter. (Since the sub-structure is usually represented in the computer by a set of
integer variables with constraints, these problems can be viewed as special cases of constraint
satisfaction or discrete optimization; but they are usually formulated and solved in a more abstract
setting where the internal representation is not explicitly mentioned.)

An important and extensively studied subclass are the graph algorithms, in particular graph traversal
algorithms, for finding specific sub-structures in a given graph — such as subgraphs, paths, circuits, and
so on. Examples include Dijkstra's algorithm, Kruskal's algorithm, the nearest neighbour algorithm, and
Prim's algorithm.

Another important subclass of this category are the string searching algorithms, that search for patterns
within strings. Two famous examples are the Boyer–Moore and Knuth–Morris–Pratt algorithms, and
several algorithms based on the suffix tree data structure.

Search for the maximum of a function

In 1953, American statistician Jack Kiefer devised Fibonacci search which can be used to find the
maximum of a unimodal function and has many other applications in computer science.

For quantum computers

There are also search methods designed for quantum computers, like Grover's algorithm, that are
theoretically faster than linear or brute-force search even without the help of data structures or
heuristics. While the ideas and applications behind quantum computers are still entirely theoretical,
studies have been conducted with algorithms like Grover's that accurately replicate the hypothetical
physical versions of quantum computing systems

We've partnered with Dartmouth college professors Tom Cormen and Devin Balkcom to teach
introductory computer science algorithms, including searching, sorting, recursion, and graph theory.
Learn with a combination of articles, visualizations, quizzes, and coding challenges.
Intro to algorithms

What are algorithms and why should you care? We'll start with an overview of algorithms and then
discuss two games that you could use an algorithm to solve more efficiently - the number guessing game
and a route-finding game.

In computer science, a sorting algorithm is an algorithm that puts elements of a list into an order. The
most frequently used orders are numerical order and lexicographical order, and either ascending or
descending. Efficient sorting is important for optimizing the efficiency of other algorithms (such as search
and merge algorithms) that require input data to be in sorted lists. Sorting is also often useful for
canonicalizing data and for producing human-readable output.

Formally, the output of any sorting algorithm must satisfy two conditions:

The output is in monotonic order (each element is no smaller/larger than the previous element,
according to the required order).

The output is a permutation (a reordering, yet retaining all of the original elements) of the input.

For optimum efficiency, the input data should be stored in a data structure which allows random access
rather than one that allows only sequential access.

History and concepts

From the beginning of computing, the sorting problem has attracted a great deal of research, perhaps
due to the complexity of solving it efficiently despite its simple, familiar statement. Among the authors
of early sorting algorithms around 1951 was Betty Holberton, who worked on ENIAC and UNIVAC.[1][2]
Bubble sort was analyzed as early as 1956.[3] Asymptotically optimal algorithms have been known since
the mid-20th century – new algorithms are still being invented, with the widely used Timsort dating to
2002, and the library sort being first published in 2006.

Comparison sorting algorithms have a fundamental requirement of Ω(n log n) comparisons (some input
sequences will require a multiple of n log n comparisons, where n is the number of elements in the array
to be sorted). Algorithms not based on comparisons, such as counting sort, can have better
performance.

Sorting algorithms are prevalent in introductory computer science classes, where the abundance of
algorithms for the problem provides a gentle introduction to a variety of core algorithm concepts, such
as big O notation, divide-and-conquer algorithms, data structures such as heaps and binary trees,
randomized algorithms, best, worst and average case analysis, time–space tradeoffs, and upper and
lower bounds.

Sorting small arrays optimally (in fewest comparisons and swaps) or fast (i.e. taking into account machine
specific details) is still an open research problem, with solutions only known for very small arrays (<20
elements). Similarly optimal (by various definitions) sorting on a parallel machine is an open research
topic.

Classification

Sorting algorithms can be classified by:

Computational complexity

Best, worst and average case behavior in terms of the size of the list. For typical serial sorting algorithms,
good behavior is O(n log n), with parallel sort in O(log2 n), and bad behavior is O(n2). Ideal behavior for a
serial sort is O(n), but this is not possible in the average case. Optimal parallel sorting is O(log n).

Swaps for "in-place" algorithms.

Memory usage (and use of other computer resources). In particular, some sorting algorithms are "in-
place". Strictly, an in-place sort needs only O(1) memory beyond the items being sorted; sometimes
O(log n) additional memory is considered "in-place".

Recursion: Some algorithms are either recursive or non-recursive, while others may be both (e.g., merge
sort).

Stability: stable sorting algorithms maintain the relative order of records with equal keys (i.e., values).

Whether or not they are a comparison sort. A comparison sort examines the data only by comparing two
elements with a comparison operator.

General method: insertion, exchange, selection, merging, etc. Exchange sorts include bubble sort and
quicksort. Selection sorts include cycle sort and heapsort.

Whether the algorithm is serial or parallel. The remainder of this discussion almost exclusively
concentrates upon serial algorithms and assumes serial operation.

Adaptability: Whether or not the presortedness of the input affects the running time. Algorithms that
take this into account are known to be adaptive.

Online: An algorithm such as Insertion Sort that is online can sort a constant stream of input.

Stability

An example of stable sort on playing cards. When the cards are sorted by rank with a stable sort, the two
5s must remain in the same order in the sorted output that they were originally in. When they are sorted
with a non-stable sort, the 5s may end up in the opposite order in the sorted output.

Stable sort algorithms sort equal elements in the same order that they appear in the input. For example,
in the card sorting example to the right, the cards are being sorted by their rank, and their suit is being
ignored. This allows the possibility of multiple different correctly sorted versions of the original list.
Stable sorting algorithms choose one of these, according to the following rule: if two items compare as
equal (like the two 5 cards), then their relative order will be preserved, i.e. if one comes before the other
in the input, it will come before the other in the output.

Stability is important to preserve order over multiple sorts on the same data set. For example, say that
student records consisting of name and class section are sorted dynamically, first by name, then by class
section. If a stable sorting algorithm is used in both cases, the sort-by-class-section operation will not
change the name order; with an unstable sort, it could be that sorting by section shuffles the name
order, resulting in a nonalphabetical list of students.

More formally, the data being sorted can be represented as a record or tuple of values, and the part of
the data that is used for sorting is called the key. In the card example, cards are represented as a record
(rank, suit), and the key is the rank. A sorting algorithm is stable if whenever there are two records R and
S with the same key, and R appears before S in the original list, then R will always appear before S in the
sorted list.

When equal elements are indistinguishable, such as with integers, or more generally, any data where the
entire element is the key, stability is not an issue. Stability is also not an issue if all keys are different.

Unstable sorting algorithms can be specially implemented to be stable. One way of doing this is to
artificially extend the key comparison, so that comparisons between two objects with otherwise equal
keys are decided using the order of the entries in the original input list as a tie-breaker. Remembering
this order, however, may require additional time and space.

One application for stable sorting algorithms is sorting a list using a primary and secondary key. For
example, suppose we wish to sort a hand of cards such that the suits are in the order clubs (♣),
diamonds (♦), hearts (♥), spades (♠), and within each suit, the cards are sorted by rank. This can be
done by first sorting the cards by rank (using any sort), and then doing a stable sort by suit:

Within each suit, the stable sort preserves the ordering by rank that was already done. This idea can be
extended to any number of keys and is utilised by radix sort. The same effect can be achieved with an
unstable sort by using a lexicographic key comparison, which, e.g., compares first by suit, and then
compares by rank if the suits are the same.

Comparison of algorithms

In these tables, n is the number of records to be sorted. The columns "Best", "Average" and "Worst" give
the time complexity in each case, under the assumption that the length of each key is constant, and
therefore that all comparisons, swaps and other operations can proceed in constant time. "Memory"
denotes the amount of extra storage needed additionally to that used by the list itself, under the same
assumption. The run times and the memory requirements listed are inside big O notation, hence the
base of the logarithms does not matter. The notation log2 n means (log n)2.

Comparison sorts

Below is a table of comparison sorts. A comparison sort cannot perform better than O(n log n) on
average.[4]

Comparison sorts

Name Best AverageWorst Memory Stable MethodOther notes

Quicksort

log

n\log n

log

n\log n

n^{2}

log

\log n No Partitioning Quicksort is usually done in-place with O(log n) stack space.[5][6]

Merge sort

log

⁡
n\log n

log

n\log n

log

n\log n n Yes Merging Highly parallelizable (up to O(log n) using the Three Hungarians'
Algorithm).[7]

In-place merge sort — —

log

n\log ^{2}n 1 Yes Merging Can be implemented as a stable sort based on stable in-
place merging.[8]

Introsort

log

⁡
n\log n

log

n\log n

log

n\log n

log

\log n No Partitioning & Selection Used in several STL implementations.

Heapsort

log

n\log n

log

⁡
n\log n

log

n\log n 1 No Selection

Insertion sort n

n^{2}

n^{2} 1 Yes Insertion O(n + d), in the worst case over sequences that have d
inversions.

Block sort n

log

n\log n

log

n\log n 1 Yes Insertion & Merging Combine a block-based

(

O(n) in-place merge algorithm[9] with a bottom-up merge sort.

Timsort n

log

n\log n

log

n\log n n Yes Insertion & Merging Makes n-1 comparisons when the data is already sorted
or reverse sorted.

Selection sort

n^{2}

n^{2}

n^{2} 1 No Selection Stable with

(

O(n) extra space, when using linked lists, or when made as a variant of Insertion Sort instead of swapping
the two items.[10]

Cubesort n

log

n\log n

log

n\log n n Yes Insertion Makes n-1 comparisons when the data is already sorted or
reverse sorted.

Shellsort

log

n\log n

4
/

{\displaystyle n^{4/3}}

n^{3/2} 1 No Insertion Small code size.

Bubble sort n

n^{2}

n^{2} 1 Yes Exchanging Tiny code size.

Exchange sort

n^{2}

n^{2}

n^{2} 1 No Exchanging Tiny code size.

Tree sort

log

n\log n

log

n\log n

log

n\log n(balanced) n Yes Insertion When using a self-balancing binary search tree.

Cycle sort

n^{2}

n^{2}

2
n^{2} 1 No Selection In-place with theoretically optimal number of writes.

Library sort

log

n\log n

log

n\log n

n^{2} n No Insertion Similar to a gapped insertion sort. It requires randomly

permuting the input to warrant with-high-probability time bounds, which makes it not stable.

Patience sortingn

log

n\log n

log

⁡
n\log n n No Insertion & Selection Finds all the longest increasing subsequences in O(n log
n).

Smoothsort n

log

n\log n

log

n\log n 1 No Selection An adaptive variant of heapsort based upon the Leonardo

sequence rather than a traditional binary heap.

Strand sort n

n^{2}

n^{2} n Yes Selection

Tournament sort

log

⁡
n\log n

log

n\log n

log

n\log n n[11] No Selection Variation of Heapsort.

Cocktail shaker sort n

n^{2}

n^{2} 1 Yes Exchanging A variant of Bubblesort which deals well with small values at
end of list

Comb sort

log

n\log n
2

n^{2}

n^{2} 1 No Exchanging Faster than bubble sort on average.

Gnome sort n

n^{2}

n^{2} 1 Yes Exchanging Tiny code size.

Odd–even sort n

n^{2}

n^{2} 1 Yes Exchanging Can be run on parallel processors easily.

Non-comparison sorts

The following table describes integer sorting algorithms and other sorting algorithms that are not
comparison sorts. As such, they are not limited to Ω(n log n).[12] Complexities below assume n items to
be sorted, with keys of size k, digit size d, and r the range of numbers to be sorted. Many of them are
based on the assumption that the key size is large enough that all entries have unique key values, and
hence that n ≪ 2k, where ≪ means "much less than". In the unit-cost random-access machine model,
algorithms with running time of
⋅

{\displaystyle \scriptstyle n\cdot {\frac {k}{d}}}, such as radix sort, still take time proportional to Θ(n log
n), because n is limited to be not more than

2^{\frac {k}{d}}, and a larger number of elements to sort would require a bigger k in order to store them
in the memory.[13]

Non-comparison sorts

Name Best AverageWorst Memory Stable n ≪ 2k Notes

Pigeonhole sort —

n+2^{k}

n+2^{k}

2
2^{k} Yes Yes Cannot sort non-integers.

Bucket sort (uniform keys) —

n+k

n^{2}\cdot k

n\cdot k Yes No Assumes uniform distribution of elements from the domain in the array.
[14]

Also cannot sort non-integers.

Bucket sort (integer keys) —

n+r

+
n+r

n+r Yes Yes If r is

O(n), then average time complexity is

O(n).[15]

Counting sort —

n+r

n+r
+

n+r Yes Yes If r is

O(n), then average time complexity is

O(n).[14]

LSD Radix Sort

n\cdot {\frac {k}{d}}

⋅
n\cdot {\frac {k}{d}}

n+2^{d}Yes No

{\frac {k}{d}} recursion levels, 2d for count array.[14][15]

Unlike most distribution sorts, this can sort non-integers.

MSD Radix Sort —

n\cdot {\frac {k}{d}}

n\cdot {\frac {k}{d}}

2
n+2^{d}Yes No Stable version uses an external array of size n to hold all of the bins.

Same as the LSD variant, it can sort non-integers.

MSD Radix Sort (in-place) —

{\displaystyle n\cdot {\frac {k}{1}}}

{\displaystyle n\cdot {\frac {k}{1}}}

2^{1} No No d=1 for in-place,

{\displaystyle k/1} recursion levels, no count array.

Spreadsort n

⋅
n\cdot {\frac {k}{d}}

n\cdot \left({{\frac {k}{s}}+d}\right)

{\frac {k}{d}}\cdot 2^{d} No No Asymptotic are based on the assumption that n ≪ 2k, but the
algorithm does not require this.

Burstsort —

n\cdot {\frac {k}{d}}

⋅
n\cdot {\frac {k}{d}}

n\cdot {\frac {k}{d}} No No Has better constant factor than radix sort for sorting strings.
Though relies somewhat on specifics of commonly encountered strings.

Flashsort n

n+r

n^{2} n No No Requires uniform distribution of elements from the domain in the array
to run in linear time. If distribution is extremely skewed then it can go quadratic if underlying sort is
quadratic (it is usually an insertion sort). In-place version is not stable.

Postman sort —

n\cdot {\frac {k}{d}}

⋅
n\cdot {\frac {k}{d}}

n+2^{d}— No A variation of bucket sort, which works very similarly to MSD Radix Sort. Specific
to post service needs.

Samplesort can be used to parallelize any of the non-comparison sorts, by efficiently distributing data
into several buckets and then passing down sorting to several processors, with no need to merge as
buckets are already sorted between each other.

Others

Some algorithms are slow compared to those discussed above, such as the bogosort with unbounded
run time and the stooge sort which has O(n2.7) run time. These sorts are usually described for
educational purposes to demonstrate how the run time of algorithms is estimated. The following table
describes some sorting algorithms that are impractical for real-life use in traditional software contexts
due to extremely poor performance or specialized hardware requirements.

Name Best AverageWorst Memory Stable Comparison Other notes

Bead sort n S S

n^{2} — No Works only with positive integers. Requires specialized hardware for it to run in
guaranteed

(
)

O(n) time. There is a possibility for software implementation, but running time will be

{\displaystyle O(S)}, where S is the sum of all integers to be sorted; in the case of small integers, it can be
considered to be linear.

Simple pancake sort

n 1 No Yes Count is number of flips.

Merge-insertion sort

log

{\displaystyle n\log n}

comparisons

log

⁡
{\displaystyle n\log n}

comparisons

log

{\displaystyle n\log n}

comparisons Varies No Yes Makes very few comparisons worst case compared to other
sorting algorithms.

Mostly of theoretical interest due to implementational complexity and suboptimal data moves.

"I Can't Believe It Can Sort"[16]

n^{2}

n^{2}

n^{2} 1 No Yes Notable primarily for appearing to be an erroneous implementation of

either Insertion Sort or Exchange Sort.

Spaghetti (Poll) sort n n n

n^{2} Yes Polling This is a linear-time, analog algorithm for sorting a sequence of items, requiring
O(n) stack space, and the sort is stable. This requires n parallel processors. See spaghetti sort#Analysis.
Sorting networkVaries Varies Varies Varies Varies (stable sorting networks require more
comparisons) Yes Order of comparisons are set in advance based on a fixed network size.[disputed
– discuss]

Bitonic sorter

log

\log ^{2}n parallel

log

\log ^{2}n parallel

log

n\log ^{2}n non-parallel 1 No Yes An effective variation of Sorting networks.[disputed –

discuss]

Bogosort n

×
!

{\displaystyle (n\times n!)} Unbounded 1 No Yes Random shuffling. Used for

example purposes only, as even the expected best-case runtime is awful.[17]

Worst case is unbounded when using randomization, but a deterministic version guarantees

{\displaystyle O(n\times n!)} worst case.

Stooge sort

log

log

1.5

n^{{\log 3/\log 1.5}}

log
⁡

log

1.5

n^{{\log 3/\log 1.5}}

log

log

1.5

n^{{\log 3/\log 1.5}} n No Yes Slower than most of the sorting algorithms (even naive
ones) with a time complexity of O(nlog 3 / log 1.5 ) = O(n2.7095...) Can be made stable, and is also a
sorting network.

Slowsort

log

)
{\displaystyle n^{\Omega (\log n)}}

log

{\displaystyle n^{\Omega (\log n)}}

log

{\displaystyle n^{\Omega (\log n)}} n No Yes A multiply and surrender algorithm,

antonymous with divide-and-conquer algorithm.

Franceschini's method[18]

−-

log

{\displaystyle n\log n}
log

{\displaystyle n\log n} 1 Yes Yes Makes O(n) data moves in the worst case. Possesses
ideal comparison sort asymptotic bounds but is only of theoretical interest.

Theoretical computer scientists have detailed other sorting algorithms that provide better than O(n log
n) time complexity assuming additional constraints, including:

Thorup's algorithm, a randomized algorithm for sorting keys from a domain of finite size, taking O(n log
log n) time and O(n) space.[19]

A randomized integer sorting algorithm taking

log

log

{\displaystyle O\left(n{\sqrt {\log \log n}}\right)} expected time and O(n) space.[20]

One of the authors of the previously mentioned algorithm also claims to have discovered an algorithm
taking

log
⁡

{\displaystyle O\left(n{\sqrt {\log n}}\right)} time and O(n) space, sorting real numbers.[21] Further
claiming that, without any added assumptions on the input, it can be modified to achieve

log

log

log

{\displaystyle O\left(n\log n/{\sqrt {\log \log n}}\right)} time and O(n) space.

Popular sorting algorithms

While there are a large number of sorting algorithms, in practical implementations a few algorithms
predominate. Insertion sort is widely used for small data sets, while for large data sets an asymptotically
efficient sort is used, primarily heapsort, merge sort, or quicksort. Efficient implementations generally
use a hybrid algorithm, combining an asymptotically efficient algorithm for the overall sort with insertion
sort for small lists at the bottom of a recursion. Highly tuned implementations use more sophisticated
variants, such as Timsort (merge sort, insertion sort, and additional logic), used in Android, Java, and
Python, and introsort (quicksort and heapsort), used (in variant forms) in some C++ sort
implementations and in .NET.
For more restricted data, such as numbers in a fixed interval, distribution sorts such as counting sort or
radix sort are widely used. Bubble sort and variants are rarely used in practice, but are commonly found
in teaching and theoretical discussions.

When physically sorting objects (such as alphabetizing papers, tests or books) people intuitively
generally use insertion sorts for small sets. For larger sets, people often first bucket, such as by initial
letter, and multiple bucketing allows practical sorting of very large sets. Often space is relatively cheap,
such as by spreading objects out on the floor or over a large area, but operations are expensive,
particularly moving an object a large distance – locality of reference is important. Merge sorts are also
practical for physical objects, particularly as two hands can be used, one for each list to merge, while
other algorithms, such as heapsort or quicksort, are poorly suited for human use. Other algorithms, such
as library sort, a variant of insertion sort that leaves spaces, are also practical for physical use.

Simple sorts

Two of the simplest sorts are insertion sort and selection sort, both of which are efficient on small data,
due to low overhead, but not efficient on large data. Insertion sort is generally faster than selection sort
in practice, due to fewer comparisons and good performance on almost-sorted data, and thus is
preferred in practice, but selection sort uses fewer writes, and thus is used when write performance is a
limiting factor.

Insertion sort

Main article: Insertion sort

Insertion sort is a simple sorting algorithm that is relatively efficient for small lists and mostly sorted lists,
and is often used as part of more sophisticated algorithms. It works by taking elements from the list one
by one and inserting them in their correct position into a new sorted list similar to how we put money in
our wallet.[22] In arrays, the new list and the remaining elements can share the array's space, but
insertion is expensive, requiring shifting all following elements over by one. Shellsort is a variant of
insertion sort that is more efficient for larger lists.

Selection sort

Main article: Selection sort

Selection sort is an in-place comparison sort. It has O(n2) complexity, making it inefficient on large lists,
and generally performs worse than the similar insertion sort. Selection sort is noted for its simplicity, and
also has performance advantages over more complicated algorithms in certain situations.

The algorithm finds the minimum value, swaps it with the value in the first position, and repeats these
steps for the remainder of the list.[23] It does no more than n swaps, and thus is useful where swapping
is very expensive.

Efficient sorts
Practical general sorting algorithms are almost always based on an algorithm with average time
complexity (and generally worst-case complexity) O(n log n), of which the most common are heapsort,
merge sort, and quicksort. Each has advantages and drawbacks, with the most significant being that
simple implementation of merge sort uses O(n) additional space, and simple implementation of
quicksort has O(n2) worst-case complexity. These problems can be solved or ameliorated at the cost of a
more complex algorithm.

While these algorithms are asymptotically efficient on random data, for practical efficiency on real-world
data various modifications are used. First, the overhead of these algorithms becomes significant on
smaller data, so often a hybrid algorithm is used, commonly switching to insertion sort once the data is
small enough. Second, the algorithms often perform poorly on already sorted data or almost sorted data
– these are common in real-world data, and can be sorted in O(n) time by appropriate algorithms.
Finally, they may also be unstable, and stability is often a desirable property in a sort. Thus more
sophisticated algorithms are often employed, such as Timsort (based on merge sort) or introsort (based
on quicksort, falling back to heapsort).

Merge sort

Main article: Merge sort

Merge sort takes advantage of the ease of merging already sorted lists into a new sorted list. It starts by
comparing every two elements (i.e., 1 with 2, then 3 with 4...) and swapping them if the first should
come after the second. It then merges each of the resulting lists of two into lists of four, then merges
those lists of four, and so on; until at last two lists are merged into the final sorted list.[24] Of the
algorithms described here, this is the first that scales well to very large lists, because its worst-case
running time is O(n log n). It is also easily applied to lists, not only arrays, as it only requires sequential
access, not random access. However, it has additional O(n) space complexity, and involves a large
number of copies in simple implementations.

Merge sort has seen a relatively recent surge in popularity for practical implementations, due to its use
in the sophisticated algorithm Timsort, which is used for the standard sort routine in the programming
languages Python[25] and Java (as of JDK7[26]). Merge sort itself is the standard routine in Perl,[27]
among others, and has been used in Java at least since 2000 in JDK1.3.[28]

Heapsort

Main article: Heapsort

Heapsort is a much more efficient version of selection sort. It also works by determining the largest (or
smallest) element of the list, placing that at the end (or beginning) of the list, then continuing with the
rest of the list, but accomplishes this task efficiently by using a data structure called a heap, a special
type of binary tree.[29] Once the data list has been made into a heap, the root node is guaranteed to be
the largest (or smallest) element. When it is removed and placed at the end of the list, the heap is
rearranged so the largest element remaining moves to the root. Using the heap, finding the next largest
element takes O(log n) time, instead of O(n) for a linear scan as in simple selection sort. This allows
Heapsort to run in O(n log n) time, and this is also the worst case complexity.

Quicksort

Main article: Quicksort

Quicksort is a divide-and-conquer algorithm which relies on a partition operation: to partition an array,

an element called a pivot is selected.[30][31] All elements smaller than the pivot are moved before it
and all greater elements are moved after it. This can be done efficiently in linear time and in-place. The
lesser and greater sublists are then recursively sorted. This yields average time complexity of O(n log n),
with low overhead, and thus this is a popular algorithm. Efficient implementations of quicksort (with in-
place partitioning) are typically unstable sorts and somewhat complex, but are among the fastest sorting
algorithms in practice. Together with its modest O(log n) space usage, quicksort is one of the most
popular sorting algorithms and is available in many standard programming libraries.

The important caveat about quicksort is that its worst-case performance is O(n2); while this is rare, in
naive implementations (choosing the first or last element as pivot) this occurs for sorted data, which is a
common case. The most complex issue in quicksort is thus choosing a good pivot element, as
consistently poor choices of pivots can result in drastically slower O(n2) performance, but good choice of
pivots yields O(n log n) performance, which is asymptotically optimal. For example, if at each step the
median is chosen as the pivot then the algorithm works in O(n log n). Finding the median, such as by the
median of medians selection algorithm is however an O(n) operation on unsorted lists and therefore
exacts significant overhead with sorting. In practice choosing a random pivot almost certainly yields O(n
log n) performance.

If a guarantee of O(n log n) performance is important, there is a simple modification to achieve that. The
idea, due to Musser, is to set a limit on the maximum depth of recursion.[32] If that limit is exceeded,
then sorting is continued using the heapsort algorithm. Musser proposed that the limit should be

log

(
)

⌋{\displaystyle 1+2\lfloor \log _{2}(n)\rfloor }, which is approximately twice the maximum recursion
depth one would expect on average with a randomly ordered array.

Shellsort

A Shellsort, different from bubble sort in that it moves elements to numerous swapping positions.

Main article: Shellsort

Shellsort was invented by Donald Shell in 1959.[33] It improves upon insertion sort by moving out of
order elements more than one position at a time. The concept behind Shellsort is that insertion sort
performs in

O(kn) time, where k is the greatest distance between two out-of-place elements. This means that
generally, they perform in O(n2), but for data that is mostly sorted, with only a few elements out of
place, they perform faster. So, by first sorting elements far away, and progressively shrinking the gap
between the elements to sort, the final sort computes much faster. One implementation can be
described as arranging the data sequence in a two-dimensional array and then sorting the columns of
the array using insertion sort.

The worst-case time complexity of Shellsort is an open problem and depends on the gap sequence used,
with known complexities ranging from O(n2) to O(n4/3) and Θ(n log2 n). This, combined with the fact
that Shellsort is in-place, only needs a relatively small amount of code, and does not require use of the
call stack, makes it is useful in situations where memory is at a premium, such as in embedded systems
and operating system kernels.

Bubble sort and variants

Bubble sort, and variants such as the Comb sort and cocktail sort, are simple, highly inefficient sorting
algorithms. They are frequently seen in introductory texts due to ease of analysis, but they are rarely
used in practice.

Bubble sort

A bubble sort, a sorting algorithm that continuously steps through a list, swapping items until they
appear in the correct order.

Main article: Bubble sort

Bubble sort is a simple sorting algorithm. The algorithm starts at the beginning of the data set. It
compares the first two elements, and if the first is greater than the second, it swaps them. It continues
doing this for each pair of adjacent elements to the end of the data set. It then starts again with the first
two elements, repeating until no swaps have occurred on the last pass.[34] This algorithm's average time
and worst-case performance is O(n2), so it is rarely used to sort large, unordered data sets. Bubble sort
can be used to sort a small number of items (where its asymptotic inefficiency is not a high penalty).
Bubble sort can also be used efficiently on a list of any length that is nearly sorted (that is, the elements
are not significantly out of place). For example, if any number of elements are out of place by only one
position (e.g. 0123546789 and 1032547698), bubble sort's exchange will get them in order on the first
pass, the second pass will find all elements in order, so the sort will take only 2n time.

[35]

Comb sort

Main article: Comb sort

Comb sort is a relatively simple sorting algorithm based on bubble sort and originally designed by
Włodzimierz Dobosiewicz in 1980.[36] It was later rediscovered and popularized by Stephen Lacey and
Richard Box with a Byte Magazine article published in April 1991. The basic idea is to eliminate turtles, or
small values near the end of the list, since in a bubble sort these slow the sorting down tremendously.
(Rabbits, large values around the beginning of the list, do not pose a problem in bubble sort) It
accomplishes this by initially swapping elements that are a certain distance from one another in the
array, rather than only swapping elements if they are adjacent to one another, and then shrinking the
chosen distance until it is operating as a normal bubble sort. Thus, if Shellsort can be thought of as a
generalized version of insertion sort that swaps elements spaced a certain distance away from one
another, comb sort can be thought of as the same generalization applied to bubble sort.

Exchange sort

Exchange sort is sometimes confused with bubble sort, although the algorithms are in fact distinct.[37]
[38] Exchange sort works by comparing the first element with all elements above it, swapping where
needed, thereby guaranteeing that the first element is correct for the final sort order; it then proceeds to
do the same for the second element, and so on. It lacks the advantage which bubble sort has of
detecting in one pass if the list is already sorted, but it can be faster than bubble sort by a constant factor
(one less pass over the data to be sorted; half as many total comparisons) in worst case situations. Like
any simple O(n2) sort it can be reasonably fast over very small data sets, though in general insertion sort
will be faster.

Distribution sorts

See also: External sorting

Distribution sort refers to any sorting algorithm where data is distributed from their input to multiple
intermediate structures which are then gathered and placed on the output. For example, both bucket
sort and flashsort are distribution based sorting algorithms. Distribution sorting algorithms can be used
on a single processor, or they can be a distributed algorithm, where individual subsets are separately
sorted on different processors, then combined. This allows external sorting of data too large to fit into a
single computer's memory.

Counting sort

Main article: Counting sort

Counting sort is applicable when each input is known to belong to a particular set, S, of possibilities. The
algorithm runs in O(|S| + n) time and O(|S|) memory where n is the length of the input. It works by
creating an integer array of size |S| and using the ith bin to count the occurrences of the ith member of
S in the input. Each input is then counted by incrementing the value of its corresponding bin. Afterward,
the counting array is looped through to arrange all of the inputs in order. This sorting algorithm often
cannot be used because S needs to be reasonably small for the algorithm to be efficient, but it is
extremely fast and demonstrates great asymptotic behavior as n increases. It also can be modified to
provide stable behavior.

Bucket sort

Main article: Bucket sort

Bucket sort is a divide-and-conquer sorting algorithm that generalizes counting sort by partitioning an
array into a finite number of buckets. Each bucket is then sorted individually, either using a different
sorting algorithm, or by recursively applying the bucket sorting algorithm.

A bucket sort works best when the elements of the data set are evenly distributed across all buckets.

Radix sort

Main article: Radix sort

Radix sort is an algorithm that sorts numbers by processing individual digits. n numbers consisting of k
digits each are sorted in O(n · k) time. Radix sort can process digits of each number either starting from
the least significant digit (LSD) or starting from the most significant digit (MSD). The LSD algorithm first
sorts the list by the least significant digit while preserving their relative order using a stable sort. Then it
sorts them by the next digit, and so on from the least significant to the most significant, ending up with a
sorted list. While the LSD radix sort requires the use of a stable sort, the MSD radix sort algorithm does
not (unless stable sorting is desired). In-place MSD radix sort is not stable. It is common for the counting
sort algorithm to be used internally by the radix sort. A hybrid sorting approach, such as using insertion
sort for small bins, improves performance of radix sort significantly.

Memory usage patterns and index sorting

When the size of the array to be sorted approaches or exceeds the available primary memory, so that
(much slower) disk or swap space must be employed, the memory usage pattern of a sorting algorithm
becomes important, and an algorithm that might have been fairly efficient when the array fit easily in
RAM may become impractical. In this scenario, the total number of comparisons becomes (relatively)
less important, and the number of times sections of memory must be copied or swapped to and from
the disk can dominate the performance characteristics of an algorithm. Thus, the number of passes and
the localization of comparisons can be more important than the raw number of comparisons, since
comparisons of nearby elements to one another happen at system bus speed (or, with caching, even at
CPU speed), which, compared to disk speed, is virtually instantaneous.

For example, the popular recursive quicksort algorithm provides quite reasonable performance with
adequate RAM, but due to the recursive way that it copies portions of the array it becomes much less
practical when the array does not fit in RAM, because it may cause a number of slow copy or move
operations to and from disk. In that scenario, another algorithm may be preferable even if it requires
more total comparisons.

One way to work around this problem, which works well when complex records (such as in a relational
database) are being sorted by a relatively small key field, is to create an index into the array and then
sort the index, rather than the entire array. (A sorted version of the entire array can then be produced
with one pass, reading from the index, but often even that is unnecessary, as having the sorted index is
adequate.) Because the index is much smaller than the entire array, it may fit easily in memory where
the entire array would not, effectively eliminating the disk-swapping problem. This procedure is
sometimes called "tag sort".[39]

Another technique for overcoming the memory-size problem is using external sorting, for example one
of the ways is to combine two algorithms in a way that takes advantage of the strength of each to
improve overall performance. For instance, the array might be subdivided into chunks of a size that will
fit in RAM, the contents of each chunk sorted using an efficient algorithm (such as quicksort), and the
results merged using a k-way merge similar to that used in merge sort. This is faster than performing
either merge sort or quicksort over the entire list.[40][41]

Techniques can also be combined. For sorting very large sets of data that vastly exceed system memory,
even the index may need to be sorted using an algorithm or combination of algorithms designed to
perform reasonably with virtual memory, i.e., to reduce the amount of swapping required.
Related algorithms

Related problems include approximate sorting (sorting a sequence to within a certain amount of the
correct order), partial sorting (sorting only the k smallest elements of a list, or finding the k smallest
elements, but unordered) and selection (computing the kth smallest element). These can be solved
inefficiently by a total sort, but more efficient algorithms exist, often derived by generalizing a sorting
algorithm. The most notable example is quickselect, which is related to quicksort. Conversely, some
sorting algorithms can be derived by repeated application of a selection algorithm; quicksort and
quickselect can be seen as the same pivoting move, differing only in whether one recurses on both sides
(quicksort, divide-and-conquer) or one side (quickselect, decrease-and-conquer).

A kind of opposite of a sorting algorithm is a shuffling algorithm. These are fundamentally different
because they require a source of random numbers. Shuffling can also be implemented by a sorting
algorithm, namely by a random sort: assigning a random number to each element of the list and then
sorting based on the random numbers. This is generally not done in practice, however, and there is a
well-known simple and efficient algorithm for shuffling: the Fisher–Yates shuffle.

Sorting algorithms are ineffective for finding an order in many situations. Usually when elements have no
reliable comparison function (crowdsourced preferences like voting systems), comparisons are very
costly (sports), or when it would be impossible to pairwise compare all elements for all criteria (search
engines). In these cases, the problem is usually referred to as ranking and the goal is to find the "best"
result for some criteria according to probabilities inferred from comparisons or rankings. A common
example is in chess, where players are ranked with the Elo rating system, and rankings are determined by
a tournament system instead of a sorting algorithm

For other uses, see Recursion (disambiguation).

A visual form of recursion known as the Droste effect. The woman in this image holds an object that
contains a smaller image of her holding an identical object, which in turn contains a smaller image of
herself holding an identical object, and so forth. 1904 Droste cocoa tin, designed by Jan Misset

Recursion occurs when the definition of a concept or process depends on a simpler version of itself.[1]
Recursion is used in a variety of disciplines ranging from linguistics to logic. The most common
application of recursion is in mathematics and computer science, where a function being defined is
applied within its own definition. While this apparently defines an infinite number of instances (function
values), it is often done in such a way that no infinite loop or infinite chain of references can occur.

A process that exhibits recursion is recursive.

Formal definitions

Ouroboros, an ancient symbol depicting a serpent or dragon eating its own tail

In mathematics and computer science, a class of objects or methods exhibits recursive behavior when it
can be defined by two properties:

A simple base case (or cases) — a terminating scenario that does not use recursion to produce an
answer

A recursive step — a set of rules that reduces all successive cases toward the base case.

For example, the following is a recursive definition of a person's ancestor. One's ancestor is either:

One's parent (base case), or

One's parent's ancestor (recursive step).

The Fibonacci sequence is another classic example of recursion:

Fib(0) = 0 as base case 1,

Fib(1) = 1 as base case 2,

For all integers n > 1, Fib(n) = Fib(n − 1) + Fib(n − 2).

Many mathematical axioms are based upon recursive rules. For example, the formal definition of the
natural numbers by the Peano axioms can be described as: "Zero is a natural number, and each natural
number has a successor, which is also a natural number."[2] By this base case and recursive rule, one can
generate the set of all natural numbers.

Other recursively defined mathematical objects include factorials, functions (e.g., recurrence relations),
sets (e.g., Cantor ternary set), and fractals.

There are various more tongue-in-cheek definitions of recursion; see recursive humor.

Informal definition

Recently refreshed sourdough, bubbling through fermentation: the recipe calls for some sourdough left
over from the last time the same recipe was made.

Recursion is the process a procedure goes through when one of the steps of the procedure involves
invoking the procedure itself. A procedure that goes through recursion is said to be 'recursive'.[3]
To understand recursion, one must recognize the distinction between a procedure and the running of a
procedure. A procedure is a set of steps based on a set of rules, while the running of a procedure
involves actually following the rules and performing the steps.

Recursion is related to, but not the same as, a reference within the specification of a procedure to the
execution of some other procedure.

When a procedure is thus defined, this immediately creates the possibility of an endless loop; recursion
can only be properly used in a definition if the step in question is skipped in certain cases so that the
procedure can complete.

But even if it is properly defined, a recursive procedure is not easy for humans to perform, as it requires
distinguishing the new from the old, partially executed invocation of the procedure; this requires some
administration as to how far various simultaneous instances of the procedures have progressed. For this
reason, recursive definitions are very rare in everyday situations.

In language

Linguist Noam Chomsky, among many others, has argued that the lack of an upper bound on the number
of grammatical sentences in a language, and the lack of an upper bound on grammatical sentence length
(beyond practical constraints such as the time available to utter one), can be explained as the
consequence of recursion in natural language.[4][5]

This can be understood in terms of a recursive definition of a syntactic category, such as a sentence. A
sentence can have a structure in which what follows the verb is another sentence: Dorothy thinks
witches are dangerous, in which the sentence witches are dangerous occurs in the larger one. So a
sentence can be defined recursively (very roughly) as something with a structure that includes a noun
phrase, a verb, and optionally another sentence. This is really just a special case of the mathematical
definition of recursion.

This provides a way of understanding the creativity of language—the unbounded number of grammatical
sentences—because it immediately predicts that sentences can be of arbitrary length: Dorothy thinks
that Toto suspects that Tin Man said that.... There are many structures apart from sentences that can be
defined recursively, and therefore many ways in which a sentence can embed instances of one category
inside another.[6] Over the years, languages in general have proved amenable to this kind of analysis.

The generally accepted idea that recursion is an essential property of human language has been
challenged by Daniel Everett on the basis of his claims about the Pirahã language. Andrew Nevins, David
Pesetsky and Cilene Rodrigues are among many who have argued against this.[7] Literary self-reference
can in any case be argued to be different in kind from mathematical or logical recursion.[8]

Recursion plays a crucial role not only in syntax, but also in natural language semantics. The word and,
for example, can be construed as a function that can apply to sentence meanings to create new
sentences, and likewise for noun phrase meanings, verb phrase meanings, and others. It can also apply
to intransitive verbs, transitive verbs, or ditransitive verbs. In order to provide a single denotation for it
that is suitably flexible, and is typically defined so that it can take any of these different types of
meanings as arguments. This can be done by defining it for a simple case in which it combines sentences,
and then defining the other cases recursively in terms of the simple one.[9]A recursive grammar is a
formal grammar that contains recursive production rules.[10]

Recursive humor

Recursion is sometimes used humorously in computer science, programming, philosophy, or

mathematics textbooks, generally by giving a circular definition or self-reference, in which the putative
recursive step does not get closer to a base case, but instead leads to an infinite regress. It is not unusual
for such books to include a joke entry in their glossary along the lines of:

Recursion, see Recursion.[11]

A variation is found on page 269 in the index of some editions of Brian Kernighan and Dennis Ritchie's
book The C Programming Language; the index entry recursively references itself ("recursion 86, 139, 141,
182, 202, 269"). Early versions of this joke can be found in Let's talk Lisp by Laurent Siklóssy (published
by Prentice Hall PTR on December 1, 1975, with a copyright date of 1976) and in Software Tools by
Kernighan and Plauger (published by Addison-Wesley Professional on January 11, 1976). The joke also
appears in The UNIX Programming Environment by Kernighan and Pike. It did not appear in the first
edition of The C Programming Language. The joke is part of the Functional programming folklore and
was already widespread in the functional programming community before the publication of the
aforementioned books. [12] [13]

A plaque commemorates the Toronto Recursive History Project of Toronto's Recursive History.

Another joke is that "To understand recursion, you must understand recursion."[11] In the English-
language version of the Google web search engine, when a search for "recursion" is made, the site
suggests "Did you mean: recursion."[14] An alternative form is the following, from Andrew Plotkin: "If
you already know what recursion is, just remember the answer. Otherwise, find someone who is
standing closer to Douglas Hofstadter than you are; then ask him or her what recursion is."Recursive
acronyms are other examples of recursive humor. PHP, for example, stands for "PHP Hypertext
Preprocessor", WINE stands for "WINE Is Not an Emulator", GNU stands for "GNU's not Unix", and
SPARQL denotes the "SPARQL Protocol and RDF Query Language".

In mathematic

The Sierpinski triangle—a confined recursion of triangles that form a fractal

Recursively defined sets

Main article: Recursive definition

Example: the natural numbers

See also: Closure (mathematics)

The canonical example of a recursively defined set is given by the natural numbers:

The set of natural numbers is the smallest set satisfying the previous two properties.

In mathematical logic, the Peano axioms (or Peano postulates or Dedekind–Peano axioms), are axioms
for the natural numbers presented in the 19th century by the German mathematician Richard Dedekind
and by the Italian mathematician Giuseppe Peano. The Peano Axioms define the natural numbers
referring to a recursive successor function and addition and multiplication as recursive functions.

Example: Proof procedure

Another interesting example is the set of all "provable" propositions in an axiomatic system that are
defined in terms of a proof procedure which is inductively (or recursively) defined as follows:

if a proposition is an axiom, it is a provable proposition.

If a proposition can be derived from true reachable propositions by means of inference rules, it is a
provable proposition.

The set of provable propositions is the smallest set of propositions satisfying these conditions.

Finite subdivision rules

Main article: Finite subdivision rule

Finite subdivision rules are a geometric form of recursion, which can be used to create fractal-like
images. A subdivision rule starts with a collection of polygons labelled by finitely many labels, and then
each polygon is subdivided into smaller labelled polygons in a way that depends only on the labels of the
original polygon. This process can be iterated. The standard `middle thirds' technique for creating the
Cantor set is a subdivision rule, as is barycentric subdivision.

Functional recursion

A function may be recursively defined in terms of itself. A familiar example is the Fibonacci number
sequence: F(n) = F(n − 1) + F(n − 2). For such a definition to be useful, it must be reducible to non-
recursively defined values: in this case F(0) = 0 and F(1) = 1.

A famous recursive function is the Ackermann function, which, unlike the Fibonacci sequence, cannot be
expressed without recursion.[citation needed]

Proofs involving recursive definitions

Applying the standard technique of proof by cases to recursively defined sets or functions, as in the
preceding sections, yields structural induction — a powerful generalization of mathematical induction
widely used to derive proofs in mathematical logic and computer science.

Recursive optimization
Dynamic programming is an approach to optimization that restates a multiperiod or multistep
optimization problem in recursive form. The key result in dynamic programming is the Bellman equation,
which writes the value of the optimization problem at an earlier time (or earlier step) in terms of its
value at a later time (or later step).

The recursion theorem

In set theory, this is a theorem guaranteeing that recursively defined functions exist. Given a set X, an
element a of X and a function f: X → X, the theorem states that there is a unique function

In computer science

Main article: Recursion (computer science)

A common method of simplification is to divide a problem into subproblems of the same type. As a
computer programming technique, this is called divide and conquer and is key to the design of many
important algorithms. Divide and conquer serves as a top-down approach to problem solving, where
problems are solved by solving smaller and smaller instances. A contrary approach is dynamic
programming. This approach serves as a bottom-up approach, where problems are solved by solving
larger and larger instances, until the desired size is reached.

A classic example of recursion is the definition of the factorial function, given here in Python code:

The function calls itself recursively on a smaller version of the input (n - 1) and multiplies the result of
the recursive call by n, until reaching the base case, analogously to the mathematical definition of
factorial.

Recursion in computer programming is exemplified when a function is defined in terms of simpler, often
smaller versions of itself. The solution to the problem is then devised by combining the solutions
obtained from the simpler versions of the problem. One example application of recursion is in parsers for
programming languages. The great advantage of recursion is that an infinite possible sentences, designs
or other data can be defined, parsed or produced by a finite computer program.

Recurrence relations are equations which define one or more sequences recursively. Some specific kinds
of recurrence relation can be "solved" to obtain a non-recursive definition (e.g., a closed-form
expression).

Use of recursion in an algorithm has both advantages and disadvantages. The main advantage is usually
the simplicity of instructions. The main disadvantage is that the memory usage of recursive algorithms
may grow very quickly, rendering them impractical for larger instances.

In biology

Shapes that seem to have been created by recursive processes sometimes appear in plants and animals,
such as in branching structures in which one large part branches out into two or mosimilar smaller parts.
One example is Romanesco broccoli.[16]
In the social sciences

Authors use the concept of recursivity to foreground the situation in which specifically social scientists
find themselves when producing knowledge about the world they are always already part of.[17][18]
According to Audrey Alejandro, “as social scientists, the recursivity of our condition deals with the fact
that we are both subjects (as discourses are the medium through which we analyse) and objects of the
academic discourses we produce (as we are social agents belonging to the world we analyse).”[19] From
this basis, she identifies in recursivity a fundamental challenge in the production of emancipatory
knowledge which calls for the exercise of reflexive efforts:

we are socialised into discourses and dispositions produced by the socio-political order we aim to
challenge, a socio-political order that we may, therefore, reproduce unconsciously while aiming to do the
contrary. The recursivity of our situation as scholars – and, more precisely, the fact that the dispositional
tools we use to produce knowledge about the world are themselves produced by this world – both
evinces the vital necessity of implementing reflexivity in practice and poses the main challenge in doing
so.

— Audrey Alejandro, Alejandro (2021)

In business

Further information: Management cybernetics

Recursion is sometimes referred to in management science as the process of iterating through levels of
abstraction in large business entities.[20] A common example is the recursive nature of management
hierarchies, ranging from line management to senior management via middle management. It also
encompasses the larger issue of capital structure in corporate governance.[21]

In art

Recursive dolls: the original set of Matryoshka dolls by Zvyozdochkin and Malyutin, 1892

Front face of Giotto's Stefaneschi Triptych, 1320, recursively contains an image of itself (held up by the
kneeling figure in the central panel).

See also: Mathematics and art and Infinity mirror

The Russian Doll or Matryoshka doll is a physical artistic example of the recursive concept.[22]Recursion
has been used in paintings since Giotto's Stefaneschi Triptych, made in 1320. Its central panel contains
the kneeling figure of Cardinal Stefaneschi, holding up the triptych itself as an offering.[23][24] This
practice is more generally known as the Droste effect, an example of the Mise en abyme technique.

M. C. Escher's Print Gallery (1956) is a print which depicts a distorted city containing a gallery which
recursively contains the picture, and so ad infinitum.[25]

In culture
The film Inception has colloquialized the appending of the suffix -ception to a noun to jokingly indicate
the recursion of something.

This article is about sets of vertices connected by edges. For graphs of mathematical functions, see Graph
of a function. For other uses, see Graph (disambiguation).

A graph with six vertices and seven edges

In discrete mathematics, and more specifically in graph theory, a graph is a structure amounting to a set
of objects in which some pairs of the objects are in some sense "related". The objects correspond to
mathematical abstractions called vertices (also called nodes or points) and each of the related pairs of
vertices is called an edge (also called link or line).[1] Typically, a graph is depicted in diagrammatic form
as a set of dots or circles for the vertices, joined by lines or curves for the edges. Graphs are one of the
objects of study in discrete mathematics.

The edges may be directed or undirected. For example, if the vertices represent people at a party, and
there is an edge between two people if they shake hands, then this graph is undirected because any
person A can shake hands with a person B only if B also shakes hands with A. In contrast, if an edge from
a person A to a person B means that A owes money to B, then this graph is directed, because owing
money is not necessarily reciprocated.

Graphs are the basic subject studied by graph theory. The word "graph" was first used in this sense by J.
J. Sylvester in 1878 due to a direct relation between mathematics and chemical structure (what he called
a chemico-graphical image).[2][3]

Definitions

Definitions in graph theory vary. The following are some of the more basic ways of defining graphs and
related mathematical structures.

Graph

A graph with three vertices and three edges

A graph (sometimes called an undirected graph to distinguish it from a directed graph, or a simple graph
to distinguish it from a multigraph)[4][5] is a pair G = (V, E), where V is a set whose elements are called
vertices (singular: vertex), and E is a set of paired vertices, whose elements are called edges (sometimes
links or lines).

The vertices x and y of an edge {x, y} are called the endpoints of the edge. The edge is said to join x and y
and to be incident on x and y. A vertex may belong to no edge, in which case it is not joined to any other
vertex.

A multigraph is a generalization that allows multiple edges to have the same pair of endpoints. In some
texts, multigraphs are simply called graphs.[6][7]
Sometimes, graphs are allowed to contain loops, which are edges that join a vertex to itself. To allow
loops, the pairs of vertices in E must be allowed to have the same node twice. Such generalized graphs
are called graphs with loops or simply graphs when it is clear from the context that loops are allowed.

Generally, the set of vertices V is supposed to be finite; this implies that the set of edges is also finite.
Infinite graphs are sometimes considered, but are more often viewed as a special kind of binary relation,
as most results on finite graphs do not extend to the infinite case, or need a rather different proof.

An empty graph is a graph that has an empty set of vertices (and thus an empty set of edges). The order
of a graph is its number of vertices |V|. The size of a graph is its number of edges |E|. However, in some
contexts, such as for expressing the computational complexity of algorithms, the size is |V| + |E|
(otherwise, a non-empty graph could have size 0). The degree or valency of a vertex is the number of
edges that are incident to it; for graphs [1]with loops, a loop is counted twice

In a graph of order n, the maximum degree of each vertex is n − 1 (or n + 1 if loops are allowed, because
a loop contributes 2 to the degree), and the maximum number of edges is n(n − 1)/2 (or n(n + 1)/2 if
loops are allowed).

The edges of a graph define a symmetric relation on the vertices, called the adjacency relation.
Specifically, two vertices x and y are adjacent if {x, y} is an edge. A graph may be fully specified by its
adjacency matrix A, which is an n × n square matrix, with Aij specifying the number of connections from
vertex i to vertex j. For a simple graph, Aij is either 0, indicating disconnection, or 1, indicating
connection; moreover Aii = 0 because an edge in a simple graph cannot start and end at the same
vertex. Graphs with self-loops will be characterized by some or all Aii being equal to a positive integer,
and multigraphs (with multiple edges between vertices) will be characterized by some or all Aij being
equal to a positive integer. Undirected graphs will have a symmetric adjacency matrix (meaning Aij =
Aji).

Directed graph

Main article: Directed graph

A directed graph with three vertices and four directed edges (the double arrow represents an edge in
each direction)

A directed graph or digraph is a graph in which edges have orientations.

In one restricted but very common sense of the term,[8] a directed graph is a pair G = (V, E) comprising:

V, a set of vertices (also called nodes or points);

E, a set of edges (also called directed edges, directed links, directed lines, arrows, or arcs), which are
ordered pairs of distinct vertices:

{\displaystyle \phi :E\to \{(x,y)\mid (x,y)\in V^{2}\}}. To avoid ambiguity, these types of objects may
be called precisely a directed simple graph permitting loops and a directed multigraph permitting loops
(or a quiver) respectively.

The edges of a directed simple graph permitting loops G is a homogeneous relation ~ on the vertices of
G that is called the adjacency relation of G. Specifically, for each edge (x, y), its endpoints x and y are said
to be adjacent to one another, which is denoted x ~ y.

Mixed graph

Main article: Mixed graph

A mixed graph is a graph in which some edges may be directed and some may be undirected. It is an
ordered triple G = (V, E, A) for a mixed simple graph and G = (V, E, A, ϕE, ϕA) for a mixed multigraph
with V, E (the undirected edges), A (the directed edges), ϕE and ϕA defined as above. Directed and
undirected graphs are special cases.

Weighted graph

A weighted graph with ten vertices and twelve edges

A weighted graph or a network[9][10] is a graph in which a number (the weight) is assigned to each
edge.[11] Such weights might represent for example costs, lengths or capacities, depending on the
problem at hand. Such graphs arise in many contexts, for example in shortest path problems such as the
traveling salesman problem.

Types of graph

Oriented graph

One definition of an oriented graph is that it is a directed graph in which at most one of (x, y) and (y, x)
may be edges of the graph. That is, it is a directed graph that can be formed as an orientation of an
undirected (simple) graph.

Some authors use "oriented graph" to mean the same as "directed graph". Some authors use "oriented
graph" to mean any orientation of a given undirected graph or multigraph.

Regular graph

Main article: Regular graph

A regular graph is a graph in which each vertex has the same number of neighbours, i.e., every vertex has
the same degree. A regular graph with vertices of degree k is called a k-regular graph or regular graph of
degree k.

Complete graph

Main article: Complete graph

A complete graph with five vertices and ten edges. Each vertex has an edge to every other vertex.
A complete graph is a graph in which each pair of vertices is joined by an edge. A complete graph
contains all possible edges.

Finite graph

A finite graph is a graph in which the vertex set and the edge set are finite sets. Otherwise, it is called an
infinite graph.

Most commonly in graph theory it is implied that the graphs discussed are finite. If the graphs are
infinite, that is usually specifically stated.

Connected graph

Main article: Connectivity (graph theory)

In an undirected graph, an unordered pair of vertices {x, y} is called connected if a path leads from x to y.
Otherwise, the unordered pair is called disconnected.

A connected graph is an undirected graph in which every unordered pair of vertices in the graph is
connected. Otherwise, it is called a disconnected graph

In a directed graph, an ordered pair of vertices (x, y) is called strongly connected if a directed path leads
from x to y. Otherwise, the ordered pair is called weakly connected if an undirected path leads from x to
y after replacing all of its directed edges with undirected edges. Otherwise, the ordered pair is called
disconnected.

A strongly connected graph is a directed graph in which every ordered pair of vertices in the graph is
strongly connected. Otherwise, it is called a weakly connected graph if every ordered pair of vertices in
the graph is weakly connected. Otherwise it is called a disconnected graph.

A k-vertex-connected graph or k-edge-connected graph is a graph in which no set of k − 1 vertices

(respectively, edges) exists that, when removed, disconnects the graph. A k-vertex-connected graph is
often called simply a k-connected graph.

Bipartite graph

Main article: Bipartite graph

A bipartite graph is a simple graph in which the vertex set can be partitioned into two sets, W and X, so
that no two vertices in W share a common edge and no two vertices in X share a common edge.
Alternatively, it is a graph with a chromatic number of 2.

In a complete bipartite graph, the vertex set is the union of two disjoint sets, W and X, so that every
vertex in W is adjacent to every vertex in X but there are no edges within W or X.

Path graph
Main article: Path graph

A path graph or linear graph of order n ≥ 2 is a graph in which the vertices can be listed in an order v1,
v2, …, vn such that the edges are the {vi, vi+1} where i = 1, 2, …, n − 1. Path graphs can be characterized
as connected graphs in which the degree of all but two vertices is 2 and the degree of the two remaining
vertices is 1. If a path graph occurs as a subgraph of another graph, it is a path in that graph.

Planar graph

Main article: Planar graph

A planar graph is a graph whose vertices and edges can be drawn in a plane such that no two of the
edges intersect.

Cycle graph

Main article: Cycle graph

A cycle graph or circular graph of order n ≥ 3 is a graph in which the vertices can be listed in an order v1,
v2, …, vn such that the edges are the {vi, vi+1} where i = 1, 2, …, n − 1, plus the edge {vn, v1}. Cycle
graphs can be characterized as connected graphs in which the degree of all vertices is 2. If a cycle graph
occurs as a subgraph of another graph, it is a cycle or circuit in that graph.

Tree

Main article: Tree (graph theory)

A tree is an undirected graph in which any two vertices are connected by exactly one path, or
equivalently a connected acyclic undirected graph.

A forest is an undirected graph in which any two vertices are connected by at most one path, or
equivalently an acyclic undirected graph, or equivalently a disjoint union of trees.

Polytree

Main article: Polytree

A polytree (or directed tree or oriented tree or singly connected network) is a directed acyclic graph
(DAG) whose underlying undirected graph is a tree.

A polyforest (or directed forest or oriented forest) is a directed acyclic graph whose underlying
undirected graph is a forest.

Advanced classes

More advanced kinds of graphs are:

Petersen graph and its generalizations;

perfect graphs;

cographs;

chordal graphs;

other graphs with large automorphism groups: vertex-transitive, arc-transitive, and distance-transitive
graphs;

strongly regular graphs and their generalizations distance-regular graphs.

Properties of graphs

See also: Glossary of graph theory and Graph property

Two edges of a graph are called adjacent if they share a common vertex. Two edges of a directed graph
are called consecutive if the head of the first one is the tail of the second one. Similarly, two vertices are
called adjacent if they share a common edge (consecutive if the first one is the tail and the second one is
the head of an edge), in which case the common edge is said to join the two vertices. An edge and a
vertex on that edge are called incident.

The graph with only one vertex and no edges is called the trivial graph. A graph with only vertices and no
edges is known as an edgeless graph. The graph with no vertices and no edges is sometimes called the
null graph or empty graph, but the terminology is not consistent and not all mathematicians allow this
object.

Normally, the vertices of a graph, by their nature as elements of a set, are distinguishable. This kind of
graph may be called vertex-labeled. However, for many questions it is better to treat vertices as
indistinguishable. (Of course, the vertices may be still distinguishable by the properties of the graph
itself, e.g., by the numbers of incident edges.) The same remarks apply to edges, so graphs with labeled
edges are called edge-labeled. Graphs with labels attached to edges or vertices are more generally
designated as labeled. Consequently, graphs in which vertices are indistinguishable and edges are
indistinguishable are called unlabeled. (In the literature, the term labeled may apply to other kinds of
labeling, besides that which serves only to distinguish different vertices or edges.)

The category of all graphs is the comma category Set ↓ D where D: Set → Set is the functor taking a set s
to s × s.

Examples

A graph with six vertices and seven edges

The diagram is a schematic representation of the graph with vertices

In computer science, directed graphs are used to represent knowledge (e.g., conceptual graph), finite
state machines, and many other discrete structures.
A binary relation R on a set X defines a directed graph. An element x of X is a direct predecessor of an
element y of X if and only if xRy.

A directed graph can model information networks such as Twitter, with one user following another.[12]
[13]

Particularly regular examples of directed graphs are given by the Cayley graphs of finitely-generated
groups, as well as Schreier coset graphs

In category theory, every small category has an underlying directed multigraph whose vertices are the
objects of the category, and whose edges are the arrows of the category. In the language of category
theory, one says that there is a forgetful functor from the category of small categories to the category of
quivers.

An undirected graph can be seen as a simplicial complex consisting of 1-simplices (the edges) and 0-
simplices (the vertices). As such, complexes are generalizations of graphs since they allow for higher-
dimensional simplices.

Every graph gives rise to a matroid.

In model theory, a graph is just a structure. But in that case, there is no limitation on the number of
edges: it can be any cardinal number, see continuous graph.

In computational biology, power graph analysis introduces power graphs as an alternative representation
of undirected graphs.

In geographic information systems, geometric networks are closely modeled after graphs, and borrow
many concepts from graph theory to perform spatial analysis on road networks or utility grids.

In computer science, graph traversal (also known as graph search) refers to the process of visiting
(checking and/or updating) each vertex in a graph. Such traversals are classified by the order in
which the vertices are visited. Tree traversal is a special case of graph traversal.

Redundancy

Unlike tree traversal, graph traversal may require that some vertices be visited more than once,
since it is not necessarily known before transitioning to a vertex that it has already been explored.
As graphs become more dense, this redundancy becomes more prevalent, causing computation time
to increase; as graphs become more sparse, the opposite holds true.

Thus, it is usually necessary to remember which vertices have already been explored by the
algorithm, so that vertices are revisited as infrequently as possible (or in the worst case, to prevent
the traversal from continuing indefinitely). This may be accomplished by associating each vertex of
the graph with a "color" or "visitation" state during the traversal, which is then checked and
updated as the algorithm visits each vertex. If the vertex has already been visited, it is ignored and
the path is pursued no further; otherwise, the algorithm checks/updates the vertex and continues
down its current path.

Several special cases of graphs imply the visitation of other vertices in their structure, and thus do
not require that visitation be explicitly recorded during the traversal. An important example of this
is a tree: during a traversal it may be assumed that all "ancestor" vertices of the current vertex (and
others depending on the algorithm) have already been visited. Both the depth-first and breadth-
first graph searches are adaptations of tree-based algorithms, distinguished primarily by the lack of
a structurally determined "root" vertex and the addition of a data structure to record the traversal's
visitation state.

Graph traversal algorithms

Note. — If each vertex in a graph is to be traversed by a tree-based algorithm (such as DFS or BFS),
then the algorithm must be called at least once for each connected component of the graph. This is
easily accomplished by iterating through all the vertices of the graph, performing the algorithm on
each vertex that is still unvisited when examined.

Depth-first search

Main article: Depth-first search

A depth-first search (DFS) is an algorithm for traversing a finite graph. DFS visits the child vertices
before visiting the sibling vertices; that is, it traverses the depth of any particular path before
exploring its breadth. A stack (often the program's call stack via recursion) is generally used when
implementing the algorithm.

The algorithm begins with a chosen "root" vertex; it then iteratively transitions from the current
vertex to an adjacent, unvisited vertex, until it can no longer find an unexplored vertex to transition
to from its current location. The algorithm then backtracks along previously visited vertices, until it
finds a vertex connected to yet more uncharted territory. It will then proceed down the new path as
it had before, backtracking as it encounters dead-ends, and ending only when the algorithm has
backtracked past the original "root" vertex from the very first step.

DFS is the basis for many graph-related algorithms, including topological sorts and planarity
testing.

Pseudocode

Input: A graph G and a vertex v of G.

Output: A labeling of the edges in the connected component of v as discovery edges and back edges.

procedure DFS(G, v) is

label v as explored

for all edges e in G.incidentEdges(v) do

if edge e is unexplored then

w ← G.adjacentVertex(v, e)

if vertex w is unexplored then

label e as a discovered edge

recursively call DFS(G, w)

else

label e as a back edge

Breadth-first search

Main article: Breadth-first search

[icon]

This section needs expansion. You can help by adding to it. (October 2012)

A breadth-first search (BFS) is another technique for traversing a finite graph. BFS visits the sibling
vertices before visiting the child vertices, and a queue is used in the search process. This algorithm
is often used to find the shortest path from one vertex to another.
Pseudocode

Input: A graph G and a vertex v of G.

Output: The closest vertex to v satisfying some conditions, or null if no such vertex exists.

procedure BFS(G, v) is

create a queue Q

enqueue v onto Q

mark v

while Q is not empty do

w ← Q.dequeue()

if w is what we are looking for then

return w

for all edges e in G.adjacentEdges(w) do

x ← G.adjacentVertex(w, e)

if x is not marked then

mark x

enqueue x onto Q

return null

Applications

Breadth-first search can be used to solve many problems in graph theory, for example:

finding all vertices within one connected component;

Cheney's algorithm;

finding the shortest path between two vertices;

testing a graph for bipartiteness;

Cuthill–McKee algorithm mesh numbering;

Ford–Fulkerson algorithm for computing the maximum flow in a flow network;

serialization/deserialization of a binary tree vs serialization in sorted order (allows the tree to be
re-constructed in an efficient manner);

maze generation algorithms;

flood fill algorithm for marking contiguous regions of a two dimensional image or n-dimensional
array;

analysis of networks and relationships.

Graph exploration

The problem of graph exploration can be seen as a variant of graph traversal. It is an online
problem, meaning that the information about the graph is only revealed during the runtime of the
algorithm. A common model is as follows: given a connected graph G = (V, E) with non-negative
edge weights. The algorithm starts at some vertex, and knows all incident outgoing edges and the
vertices at the end of these edges—but not more. When a new vertex is visited, then again all
incident outgoing edges and the vertices at the end are known. The goal is to visit all n vertices and
return to the starting vertex, but the sum of the weights of the tour should be as small as possible.
The problem can also be understood as a specific version of the travelling salesman problem, where
the salesman has to discover the graph on the go.

For general graphs, the best known algorithms for both undirected and directed graphs is a simple
greedy algorithm:

In the undirected case, the greedy tour is at most O(ln n)-times longer than an optimal tour.[1] The
best lower bound known for any deterministic online algorithm is 10/3.[2]

Unit weight undirected graphs can be explored with a competitive ration of 2 − ε,[3] which is
already a tight bound on Tadpole graphs.[4]

In the directed case, the greedy tour is at most (n − 1)-times longer than an optimal tour. This
matches the lower bound of n − 1.[5] An analogous competitive lower bound of Ω(n) also holds for
randomized algorithms that know the coordinates of each node in a geometric embedding. If
instead of visiting all nodes just a single "treasure" node has to be found, the competitive bounds are
Θ(n2) on unit weight directed graphs, for both deterministic and randomized algorithms.

Universal traversal sequences

[icon]

This section needs expansion. You can help by adding to it. (December 2016)

A universal traversal sequence is a sequence of instructions comprising a graph traversal for any
regular graph with a set number of vertices and for any starting vertex. A probabilistic proof was
used by Aleliunas et al. to show that there exists a universal traversal sequence with number of
instructions proportional to O(n5) for any regular graph with n vertices.[6] The steps specified in
the sequence are relative to the current node, not absolute. For example, if the current node is vj,
and vj has d neighbors, then the traversal sequence will specify the next node to visit, vj+1, as the
ith neighbor of vj, where 1 ≤ i ≤ d.