This table organizing the neural network architectures

based on their founding dates:

Rank Architecture Founding Date

1 Perceptron 1958

2 Hopfield Networks 1982

3 Self-Organizing Maps (SOMs) 1982

4 Boltzmann Machines 1985

5 Backpropagation 1986

Feedforward Neural Networks

6 1980s
(FNNs)

1980s (re-popularized
7 Autoencoders
in 2006)

Radial Basis Function Networks

8 1988
(RBFNs)

Convolutional Neural Networks

9 1989
(CNNs)

10 Recurrent Neural Networks (RNNs) 1980s

1990s (popularized in
11 Bayesian Neural Networks
2018)

1990s (popularized in
12 Spiking Neural Networks (SNNs)
2018)

1993 (popularized in
13 Siamese Networks
2015)
Rank Architecture Founding Date

14 LSTM Networks 1997

15 Deep Belief Networks (DBNs) 2006

16 Extreme Learning Machines (ELMs) 2006

17 Echo State Networks (ESNs) 2002

18 Liquid State Machines (LSMs) 2002

2009 (popularized in
19 Graph Neural Networks (GNNs)
2017)

20 Residual Networks (ResNets) 2015

Generative Adversarial Networks

21 2014
(GANs)

22 Neural Turing Machines (NTMs) 2014

23 Memory Networks 2014

24 Attention Mechanisms 2014

25 Pointer Networks 2015

Spatial Transformer Networks

26 2015
(STNs)

Hierarchical Temporal Memory

27 2016
(HTM)

2016 (popularized in
28 HyperNetworks
2019)

29 Capsule Networks (CapsNets) 2017

Rank Architecture Founding Date

30 Transformers 2017

Densely Connected Networks

31 2017
(DenseNets)

Neural ODEs (Ordinary Differential

32 2018
Equations)

Self-Attention Generative
33 2018
Adversarial Networks (SAGANs)

34 Neural Architecture Search (NAS) 2019

35 Dynamic Neural Networks 2019

Hierarchical Attentive Memory

36 2019
(HAM)

Adaptive Computation Time 2016 (popularized in

37
Networks (ACT) 2019)

Various (popularized
38 Reinforcement Learning Networks
in 2015 with DQNs)
This table organizing the neural network architectures
based on their popularity:

Rank Architecture Popularity Characteristics

Convolutional
Widely used in image and video
Neural
1 recognition, classification, and
Networks
segmentation.
(CNNs)

Dominant in natural language processing

2 Transformers (NLP) tasks, such as language translation
and text summarization.

Recurrent
Popular for sequence data, including time
Neural
3 series prediction and language modeling
Networks
(especially LSTMs and GRUs).
(RNNs)

Generative
Adversarial Known for generating realistic images,
4
Networks videos, and other types of data.
(GANs)

Residual
Essential for training very deep neural
5 Networks
networks.
(ResNets)

Densely
Connected Popular for their efficiency in learning
6
Networks features with dense connections.
(DenseNets)

Used in unsupervised learning tasks like

7 Autoencoders anomaly detection, image denoising, and
data compression.

Integral to the success of Transformers

Attention
8 and used in other architectures for
Mechanisms
improving performance.

9
Rank Architecture Popularity Characteristics

Graph Neural Increasingly popular for graph-structured

Networks data in social networks, recommendation
(GNNs) systems, and biological networks.

Feedforward
Fundamental architecture, widely used as
Neural
10 the building block for more complex
Networks
networks.
(FNNs)

Commonly used in tasks requiring

Siamese
11 similarity learning, such as face
Networks
verification.

Bayesian Neural Gaining traction for applications requiring

12
Networks uncertainty estimation.

Neural
Becoming popular for automating the
13 Architecture
design of neural network architectures.
Search (NAS)

Capsule Known for preserving spatial hierarchies,

14 Networks but less widely adopted compared to
(CapsNets) CNNs.

Neural ODEs
(Ordinary Emerging popularity for modeling
15
Differential continuous-time dynamics.
Equations)

Used for tasks requiring reasoning over

Memory
16 large contexts, such as question
Networks
answering.

Spiking Neural
Popular in neuromorphic computing for
17 Networks
low-power, real-time processing tasks.
(SNNs)

Used in meta-learning and quickly

18 HyperNetworks
adapting to new tasks.
Rank Architecture Popularity Characteristics

Self-Attention
Generative
Enhanced version of GANs with self-
19 Adversarial
attention mechanisms.
Networks
(SAGANs)

Useful for combinatorial optimization

Pointer
20 problems, with specific but growing
Networks
applications.

Deep Belief Historically important but largely

21 Networks supplanted by more advanced deep
(DBNs) learning architectures.

Spatial
Useful for enhancing the spatial
Transformer
22 manipulation capabilities of neural
Networks
networks.
(STNs)

Radial Basis
Function Used in specific applications like function
23
Networks approximation and time-series prediction.
(RBFNs)

Extreme
Learning Known for fast training, but less popular
24
Machines in mainstream deep learning.
(ELMs)

Echo State
Used in reservoir computing, with niche
25 Networks
applications.
(ESNs)

Liquid State
Similar to ESNs, used in specific areas of
26 Machines
reservoir computing.
(LSMs)

Hopfield Primarily of historical interest, with niche

27
Networks applications in associative memory.
Rank Architecture Popularity Characteristics

Boltzmann Mainly of historical importance and for

28
Machines theoretical research.

Neural Turing Used in tasks requiring complex memory

29 Machines and computation, but with limited
(NTMs) adoption.

Hierarchical
Inspired by biological brains, used in
30 Temporal
niche applications.
Memory (HTM)

Dynamic Neural Emerging interest, with applications in

31
Networks adaptive and real-time processing.

Reinforcement Widely used in specific areas like game

32 Learning playing (e.g., AlphaGo), robotics, and
Networks control systems.

Adaptive
Computation Niche use cases where dynamic
33
Time Networks computation is beneficial.
(ACT)

Hierarchical
Research interest in tasks requiring
34 Attentive
hierarchical attention over sequences.
Memory (HAM)

Kohonen Primarily of historical interest, with

35
Networks specific applications in clustering.
This table organizing the neural network architectures
based on their general efficiency in terms of computational
and memory resources:

Rank Architecture Efficiency Characteristics

Feedforward
Neural Simple and computationally efficient,
1
Networks especially for smaller datasets.
(FNNs)

Radial Basis
Efficient for specific tasks like function
Function
2 approximation and time-series prediction,
Networks
but limited in scalability.
(RBFNs)

Efficient for tasks requiring similarity

Siamese
3 learning with shared weights across
Networks
subnetworks.

Convolutional
Neural Efficient for image processing tasks due to
4
Networks parameter sharing and local connectivity.
(CNNs)

Efficient for unsupervised learning tasks

5 Autoencoders like dimensionality reduction and feature
extraction.

Efficient when integrated with other

Attention
6 architectures, improving focus on relevant
Mechanisms
data without significant cost.

Memory Efficient for tasks requiring large context

7
Networks reasoning with optimized memory usage.

Neural ODEs
(Ordinary Efficient for modeling continuous-time
8
Differential dynamics with fewer parameters.
Equations)
Rank Architecture Efficiency Characteristics

Densely
Efficient in feature reuse, reducing the
Connected
9 number of parameters compared to other
Networks
deep networks.
(DenseNets)

Extreme
Learning Fast training times due to random hidden
10
Machines nodes, but less flexible.
(ELMs)

Neural Efficient in finding optimal architectures,

11 Architecture saving time and computational resources
Search (NAS) in the long run.

Hierarchical Efficient for tasks requiring temporal

12 Temporal pattern recognition with minimal
Memory (HTM) computational resources.

Graph Neural
Efficient for processing graph-structured
13 Networks
data by leveraging sparsity.
(GNNs)

Residual
Efficient in training very deep networks by
14 Networks
mitigating the vanishing gradient problem.
(ResNets)

Computationally efficient for sequential

15 Transformers data, but can be memory-intensive for long
sequences.

Capsule Efficient in preserving spatial hierarchies,

16 Networks but computationally more expensive than
(CapsNets) CNNs.

Recurrent
Efficient for sequence data, with LSTMs
Neural
17 and GRUs being more efficient than vanilla
Networks
RNNs.
(RNNs)

18
Rank Architecture Efficiency Characteristics

Echo State
Efficient in reservoir computing with fixed
Networks
recurrent layers.
(ESNs)

Liquid State
Similar efficiency to ESNs in reservoir
19 Machines
computing applications.
(LSMs)

Bayesian Efficient in providing uncertainty

20 Neural estimates, but computationally more
Networks expensive than deterministic networks.

Hopfield Efficient for specific optimization

21
Networks problems, but limited in scalability.

Generative
Efficient in generating high-quality
Adversarial
22 samples, but require careful tuning and
Networks
can be computationally expensive.
(GANs)

Self-Attention
Generative More efficient than standard GANs in
23 Adversarial capturing long-range dependencies but
Networks computationally demanding.
(SAGANs)

Boltzmann Computationally expensive due to

24
Machines stochastic nature and difficult training.

Neural Turing Efficient for tasks requiring complex

25 Machines memory operations, but computationally
(NTMs) intensive.

Spiking Neural
Efficient for low-power and real-time
26 Networks
processing, but challenging to train.
(SNNs)

Dynamic
Efficient in adaptive scenarios, but with
27 Neural
variable computational costs.
Networks
Rank Architecture Efficiency Characteristics

Efficient for specific combinatorial

Pointer
28 optimization problems, but limited in
Networks
generality.

Kohonen Efficient for clustering tasks, but with

29
Networks limited scalability.

Adaptive
Computation Efficient for tasks requiring variable
30
Time Networks computation, but complex to implement.
(ACT)

Hierarchical Efficient in tasks requiring hierarchical

31 Attentive attention, but with increased
Memory (HAM) computational complexity.

Spatial
Efficient in enhancing spatial
Transformer
32 manipulation, but with added
Networks
computational cost.
(STNs)

Reinforcement Efficient for specific applications like game

33 Learning playing and robotics, but often
Networks computationally expensive.

Deep Belief
Historically significant but less efficient
34 Networks
compared to modern architectures.
(DBNs)

Efficient in meta-learning, but

35 HyperNetworks computationally demanding for generating
network weights.