A Technical Seminar report on

NEURAL ARCHITECTURE SEARCH (NAS)

A Dissertation submitted to JNTUH, Hyderabad in partial fulfillment of the
academic requirements for the award of the degree.

Bachelor of Technology
in
Computer Science and Engineering (AI&ML)
Submitted by

D. SIMHADRI
(21H51A66E6)

Under the esteemed guidance of

Ms. Sana Afreen
(Assistant Professor, CSE(AI&ML))

Department of Computer Science and Engineering (AI&ML)

CMR COLLEGE OF ENGINEERING & TECHNOLOGY

(UGC Autonomous)
*Approved by AICTE *Affiliated to JNTUH *NAAC Accredited with A+ Grade
KANDLAKOYA, MEDCHAL ROAD, HYDERABAD - 501401.

2024-2025

.
 CMR COLLEGE OF ENGINEERING & TECHNOLOGY
KANDLAKOYA, MEDCHAL ROAD, HYDERABAD – 501401

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING (AI&ML)

CERTIFICATE

This is to certify that the Technical Seminar report entitled " Neural Architeture
Search(NAS)" being submitted by D. Simhadri (21H51A66E6) in partial
fulfillment for the award of Bachelor of Technology in Computer Science and
Engineering (AI&ML) is a record of bonafide work carried out his/her under my
guidance and supervision.
The results embodies in this project report have not been submitted to
any other University or Institute for the award of any Degree.

Ms. Sana Afreen Dr. P. Sruthi

Assistant Professor Associate Professor and HOD
Dept. of CSE (AI&ML) Dept. of CSE (AI&ML)
 ACKNOWLEDGEMENT

With great pleasure we want to take this opportunity to express my heartfelt gratitude to all
the people who helped in making this project work a grand success.
We are grateful to Ms. Sana Afreen, Assistant Professor, Department of Computer
Science and Engineering (AI&ML) for her valuable technical suggestions and guidance during the
execution of this project work.
We would like to thank Dr. P. Sruthi, Head of the Department of Computer Science and
Engineering (AI&ML), CMR College of Engineering and Technology, who is the major driving
forces to complete my project work successfully.
We are very grateful to Dr. Ghanta Devadasu, Dean-Academics, CMR College of
Engineering and Technology, for his constant support and motivation in carrying out the project
work successfully.
We extend our heartfelt gratitude to Dr. Seshu Kumar Avadhanam, Principal, CMR
College of Engineering & Technology, for his unwavering support and guidance in the successful
completion of our project and his encouragement has been invaluable throughout this endeavor.
We are highly indebted to Dr. V A Narayana, Director, CMR College of Engineering and
Technology, for giving permission to carry out this project in a successful and fruitful way.
We express our sincere thanks to Shri. Ch. Gopal Reddy, Secretary& Correspondent,
CMR Group of Institutions, and Shri Ch Abhinav Reddy, CEO, CMR Group of Institutions for
their continuous care and support.
We would like to thank the Teaching & Non- teaching staff of Department of Computer
Science and Engineering for their co-operation.
Finally, We extend thanks to our parents who stood behind us at different stages of this
Project. We sincerely acknowledge and thank all those who gave support directly and indirectly in
completion of this project work.

D. Simhadri 21H51A66E6
 Neural Architecture Search (NAS)

TABLE OF CONTENTS

CHAPTER TITLE PAGE NO.

NO.
LIST OF FIGURES ii
LIST OF ACRONYMS iii
ABSTRACT iv
1 INTRODUCTION 1
1.1 Introduction 2
2 LITERATURE REVIEW 5
2.1. Literature Review 6
3 PROPOSED WORK 8 3.1. Problem
Definition 9
3.2. Real Time Applications 9
3.3 Working Principle 12
3.4 Architecture 14
3.5 Advantages & Disadvantages 15
4 CONCLUSION 16
4.1 Conclusion and Future Enhancements 17
REFERENCES 18

CMR College of Engineering and Technology CSE (AI&ML) i

Neural Architecture Search (NAS)

List of Figures
FIGURE
NO. TITLE PAGE NO.

1.1 Neural Architecture Search 2

3.2 NAS general framework 12
3.3 Architecture of NAS 14

CMR College of Engineering and Technology CSE (AI&ML) ii

Neural Architecture Search (NAS)

List of Acronyms

ACRONYM
NO. ACRONYM FULL FORM
1 NAS Neural Architecture Search
2 GA Genetic Algorithm
3 AI Artificial Intelligence
4 EA Evolutionary Algorithm
5 RL Reinforcement Algorithm
6 DARTS Differentiable Architecture Search
7 AR Augmented Reality

CMR College of Engineering and Technology CSE (AI&ML) iii

Neural Architecture Search (NAS)

ABSTRACT

Neural Architecture Search (NAS) is a method used to automate the design of deep neural
networks, reducing human intervention in the model selection process. By employing
techniques such as reinforcement learning, evolutionary algorithms, and gradient-based
methods, NAS optimizes network structures for specific tasks. This process enhances
efficiency, accelerates model development, and often results in architectures that outperform
manually designed networks. Despite its potential, NAS faces challenges such as high
computational costs and the complexity of search spaces. Recent advancements focus on
reducing these constraints through weight-sharing mechanisms, proxy tasks, and hardware-
aware searches. The future of NAS promises more adaptable and scalable architectures that
cater to diverse real-world applications. As NAS continues to evolve, it is expected to play a
crucial role in advancing deep learning by making neural network design more efficient,
scalable, and adaptable to various real-world applications.

Deep learning has led to significant advancements in areas like image recognition, speech
processing, and machine translation. A key factor behind this progress is the design of
innovative neural network architectures. Traditionally, these architectures have been crafted
manually by domain experts, which is both time-intensive and prone to human error. To
address this, the field is increasingly shifting toward automated methods for neural
architecture search (NAS). These approaches are typically categorized based on three
components: the search space, the search strategy, and the performance estimation strategy.

CMR College of Engineering and Technology CSE (AI&ML) iv

Neural Architecture Search (NAS)

CHAPTER 1
INTRODUCTION

CMR College of Engineering and Technology CSE (AI&ML) 1

Neural Architecture Search (NAS)

CHAPTER 1
INTRODUCTION
1.1 Introduction
In recent years, the field of deep learning has witnessed remarkable progress, thanks in part to
the development of advanced neural network architectures. Neural Architecture Search (NAS)
has emerged as a powerful technique to automate the process of designing effective neural
networks. In this tutorial, we’ll look into the fundamentals of NAS, exploring its concepts,
methodologies, and practical applications.
The success of deep learning in perceptual tasks is largely due to its automation of the feature
engineering process: hierarchical feature extractors are learned in an end-to-end fashion from
data rather than manually designed. This success has been accompanied, however, by a rising
demand for architecture engineering, where increasingly more complex neural architectures
are designed manually. Neural Architecture Search (NAS), the process of automating
architecture engineering, is thus a logical next step in automating machine learning. NAS can
be seen as subfield of AutoML and has significant overlap with hyperparameter optimization
and meta-learning We categorize methods for NAS according to three dimensions:

Fig:1.1 Neural Architecture Search

Search Space: The search space defines which architectures can be represented in principle.
Incorporating prior knowledge about properties well-suited for a task can reduce the size of
the search space and simplify the search. However, this also introduces a human bias, which
may prevent finding novel architectural building blocks that go beyond the current human
knowledge.
Search Strategy: The search strategy details how to explore the search space. It encompasses
the classical exploration-exploitation trade-off since, on the one hand, it is desirable to find
well-performing architectures quickly, while on the other hand, premature convergence to a
region of suboptimal architectures should be avoided.
CMR College of Engineering and Technology CSE (AI&ML) 2
Neural Architecture Search (NAS)
Performance Estimation Strategy: The objective of NAS is typically to find architectures
that achieve high predictive performance on unseen data. Performance Estimation refers to the
process of estimating this performance: the simplest option is to perform a standard training
and validation of the architecture on data, but this is unfortunately computationally expensive
and limits the number of architectures that can be explored. Much recent research therefore
focuses on developing methods that reduce the cost of these performance estimations.
Types of NAS Approaches
There are several different approaches to Neural Architecture Search, each with its strengths
and trade-offs.
1. Reinforcement Learning-Based NAS
In Reinforcement Learning-based NAS, a controller (often an RNN) is trained to generate
architectures. The controller proposes an architecture, and after the architecture is trained and
evaluated, the result (e.g., accuracy) is used as a reward signal to improve the controller.
Example: NASNet
NASNet is a well-known RL-based NAS approach developed by Google. It uses an RNN
controller to predict neural network architectures, which are then trained and evaluated. The
reward signal from the evaluation is fed back to the controller, allowing it to generate better
architectures in subsequent iterations.
Key Contributions:
 Cell Search: NASNet searches for a small, repeatable “cell” that can be stacked
multiple times to form a larger network. This reduces the search space and
computation time.
2. Evolutionary Algorithm-Based NAS
In Evolutionary Algorithm (EA)-based NAS, a population of neural networks is evolved over
time. New architectures are generated through mutation (random changes) and crossover
(combining parts of different architectures). The best-performing architectures are selected for
the next generation.
Example: AmoebaNet
AmoebaNet is an evolutionary algorithm-based NAS method that evolves architectures using
mutation and selection. It explores the search space by creating small changes to existing
architectures and selecting the top performers for further evolution.
CMR College of Engineering and Technology CSE (AI&ML) 3
Neural Architecture Search (NAS)

Key Contributions:
 High Accuracy: AmoebaNet has achieved state-of-the-art performance on image
classification tasks, rivaling architectures designed through manual effort.
 Evolutionary Process: It leverages evolutionary algorithms to explore the vast
architecture space without requiring hand-engineering.
3. Gradient-Based NAS
Gradient-based NAS uses continuous relaxation of the architecture search space to make it
differentiable, allowing the architecture to be optimized using gradient descent. One popular
example is DARTS (Differentiable Architecture Search).
Example: DARTS (Differentiable Architecture Search)
DARTS formulates the NAS problem as a continuous optimization task by relaxing discrete
architectural choices into weighted sums. This allows the architecture search to be optimized
using standard gradient-based optimization techniques, making it more efficient than RL or
EA approaches.
Key Contributions:
 Efficiency: DARTS is more efficient than traditional NAS approaches, as it does not
require training a separate model for each architecture sampled.
 Scalability: DARTS can scale to larger tasks due to its efficient optimization process.

CMR College of Engineering and Technology CSE (AI&ML) 4

Neural Architecture Search (NAS)

CHAPTER 2
LITERATURE
REVIEW

CMR College of Engineering and Technology CSE (AI&ML) 5

Neural Architecture Search (NAS)

CHAPTER 2
LITERATURE REVIEW
2.1 Literature Review
Hyperparameter optimization is an important research topic in machine learning, and is widely
used in practice (Bergstra et al., 2011; Bergstra & Bengio, 2012; Snoek et al., 2012; 2015;
Saxena & Verbeek, 2016). Despite their success, these methods are still limited in that they
only search models from a fixed-length space. In other words, it is difficult to ask them to
generate a variable-length configuration that specifies the structure and connectivity of a
network. In practice, these methods often work better if they are supplied with a good initial
model (Bergstra & Bengio, 2012; Snoek et al., 2012; 2015). There are Bayesian optimization
methods that allow to search non fixed length architectures (Bergstra et al., 2013; Mendoza et
al., 2016), but they are less general and less flexible than the method proposed in this paper.
Modern neuro-evolution algorithms, e.g., Wierstra et al. (2005); Floreano et al. (2008);
Stanley et al. (2009), on the other hand, are much more flexible for composing novel models,
yet they are usually less practical at a large scale. Their limitations lie in the fact that they are
search-based methods, thus they are slow or require many heuristics to work well.

Neural Architecture Search has some parallels to program synthesis and inductive
programming, the idea of searching a program from examples (Summers, 1977; Biermann,
1978). In machine learning, probabilistic program induction has been used successfully in
many settings, such as learning to solve simple Q&A (Liang et al., 2010; Neelakantan et al.,
2015; Andreas et al., 2016), sort a list of numbers (Reed & de Freitas, 2015), and learning with
very few examples (Lake et al., 2015).

The controller in Neural Architecture Search is auto-regressive, which means it predicts

hyperparameters one a time, conditioned on previous predictions. This idea is borrowed from
the decoder in end-to-end sequence to sequence learning (Sutskever et al., 2014). Unlike
sequence to sequence learning, our method optimizes a non-differentiable metric, which is the
accuracy of the child network. It is therefore similar to the work on BLEU optimization in
Neural Machine Translation (Ranzato et al., 2015; Shen et al., 2016). Unlike these
approaches, our method learns directly from the reward signal without any supervised
bootstrapping.
CMR College of Engineering and Technology CSE (AI&ML) 6
Neural Architecture Search (NAS)

Also related to our work is the idea of learning to learn or meta-learning (Thrun & Pratt,
2012), a general framework of using information learned in one task to improve a future task.
More closely related is the idea of using a neural network to learn the gradient descent updates
for another network (Andrychowicz et al., 2016).

The early years of Neural Architecture Search (2016-2017) were dominated by reinforcement
learning (RL)-based approaches, with the seminal work by Zoph & Le (2016) establishing the
foundational framework. Their method employed an RNN controller that generated neural
network architectures trained through policy gradients, achieving state-of-the-art performance
on CIFAR-10. However, this breakthrough came at an extraordinary computational cost,
requiring over 2,000 GPU days to complete the architecture search. Several follow-up works
attempted to refine this approach, including MetaQNN (Baker et al., 2016) which utilized Q-
learning for architecture search, and BlockQNN (Zhong et al., 2017) that implemented a
hierarchical search strategy with RL. While these methods demonstrated promising results,
they shared common limitations of prohibitive computational requirements and limited
scalability to larger datasets or more complex architectures, hindering their practical adoption.

The field saw significant advancement with the introduction of evolutionary algorithms (2017-
2018), which offered an alternative approach to architecture search. AmoebaNet (Real et al.,
2018) became a notable example, employing tournament selection to evolve neural
architectures that matched manually designed networks while using three times fewer
parameters. Another important contribution was the hierarchical evolutionary approach (Liu
et al., 2017), which simultaneously optimized both macro and micro-structures of neural
networks. These evolutionary methods provided better interpretability compared to RL-based
approaches, as the architecture development process followed more transparent selection and
mutation operations. However, they remained computationally intensive, though somewhat
less so than their RL predecessors, and still posed challenges for widespread implementation.

CMR College of Engineering and Technology CSE (AI&ML) 7

Neural Architecture Search (NAS)

CHAPTER 3
PROPOSED
WORK

CMR College of Engineering and Technology CSE (AI&ML) 8

Neural Architecture Search (NAS)

CHAPTER 3
PROPOSED WORK
3.1 Problem Definition
Designing effective neural network architectures is a critical but complex task in deep
learning. Traditionally, this process has required extensive human expertise, trial-and-error
experimentation, and domain-specific knowledge, making it both time-consuming and
resource-intensive. As deep learning models become increasingly complex and are applied to
a wider range of domains, the manual design of neural architectures becomes a significant
bottleneck in the development pipeline.
Neural Architecture Search (NAS) aims to automate the design of neural networks by
leveraging machine learning algorithms to discover optimal architectures for a given task.
However, existing NAS techniques often face several challenges:
1. High Computational Cost: Many NAS methods require thousands of GPU hours,
making them inaccessible for most researchers and impractical for rapid deployment.
2. Inefficient Search Methods: Search algorithms like reinforcement learning and
evolutionary strategies can be slow to converge or may explore suboptimal parts of the
search space.
3. Generalization Issues: Architectures discovered on smaller datasets (e.g., CIFAR-10)
may not perform well when transferred to larger, real-world datasets (e.g., ImageNet).
4. Complex Search Spaces: The design of search spaces significantly influences NAS
outcomes, yet defining a balanced and effective space remains a challenge.
5. Lack of Interpretability: NAS-generated models often act as black boxes, making it
hard to understand their design rationale or ensure trustworthiness in critical
applications.
3.2 Real Time Applications

Neural Architecture Search (NAS) is transforming how neural networks are designed,
enabling automatic discovery of optimized models for real-world tasks. In real-time
applications, where latency, speed, and adaptability are critical, NAS offers a powerful way to
create models that are not only accurate but also efficient in execution.
CMR College of Engineering and Technology CSE (AI&ML) 9
Neural Architecture Search (NAS)
1. Real-Time Image and Video Processing
Autonomous Vehicles
 Use Case: Object detection, lane detection, pedestrian recognition, and traffic sign
recognition.
 Role of NAS: Finds efficient CNN architectures for real-time video feeds with low latency
on edge devices.
 Example: Lightweight NAS-designed models deployed on NVIDIA Jetson platforms for
real-time image inference.
Surveillance Systems
 Use Case: Real-time human detection, activity recognition, and anomaly detection.
 Role of NAS: Helps design models with fast inference speeds suitable for 24/7 operation
on low-power hardware.
 Example: NAS-based object detection models in smart CCTV systems.
2. Mobile and Edge Computing
Smartphones and Wearables
 Use Case: Face unlock, voice commands, camera enhancements, health tracking.
 Role of NAS: Optimizes model size and speed for limited hardware (battery and processor
constraints).
 Example: Google’s MNASNet uses NAS to balance accuracy and latency on mobile
devices for tasks like photo enhancement and AI assistants.
IoT Devices
 Use Case: Real-time analytics on sensors, smart home automation, industrial IoT.
 Role of NAS: Enables efficient models that run directly on the device without needing
cloud inference.
3. Natural Language Processing (NLP)
Voice Assistants (e.g., Alexa, Siri, Google Assistant)
 Use Case: Real-time speech recognition, intent detection, and response generation.
 Role of NAS: Speeds up transformer model search to fit memory and latency budgets.
Customer Support Chatbots
 Use Case: Real-time question answering, summarization, and feedback analysis.
 Role of NAS: Automates finding compact transformer variants for fast response
generation.
CMR College of Engineering and Technology CSE (AI&ML) 10
Neural Architecture Search (NAS)

4. Healthcare and Medical Diagnosis

Medical Imaging
 Use Case: Real-time detection of diseases (e.g., tumors, pneumonia, chest diseases) using
X-rays, MRIs, or CT scans.
 Role of NAS: Automatically creates CNN or hybrid models that can provide accurate
diagnoses quickly, aiding emergency medical decisions.
Remote Patient Monitoring
 Use Case: Analyzing ECG, EEG, and other sensor signals in real time.
 Role of NAS: Creates lightweight, real-time models for deployment in wearable devices.
5. Real-Time Finance and Trading
Algorithmic Trading
 Use Case: Real-time market prediction and high-frequency trading decisions.
 Role of NAS: Optimizes recurrent or temporal models (like RNNs or LSTMs) for ultra-
fast decision-making pipelines.
Fraud Detection
 Use Case: Real-time transaction analysis to detect anomalies or unauthorized behavior.
 Role of NAS: Enables deployment of robust and fast models capable of detecting patterns
in streaming financial data.
6. Gaming and Augmented/Virtual Reality (AR/VR)
AR Filters and Effects
 Use Case: Real-time face tracking and augmentation in applications like Snapchat or
Instagram.
 Role of NAS: Helps design lightweight computer vision models that run at 30+ FPS on
smartphones.
AI in Gaming
 Use Case: Real-time NPC behavior generation, content adaptation, and environment
rendering.
 Role of NAS: Enables dynamic AI models optimized for performance and minimal lag.
CMR College of Engineering and Technology CSE (AI&ML) 11
Neural Architecture Search (NAS)

3.3 Working Principle

Neural Architecture Search (NAS) is the process of automating the design of neural
networks’ topology in order to achieve the best performance on a specific task. The goal is
to design the architecture using limited resources and with minimal human intervention.
Following the work of Ren et. al 1, let’s discuss a general framework for NAS. At its core,
NAS is a search algorithm.
1. It operates on the search space of possible network topologies, which consists of a
list of predefined operations (e.g.convolutional layers, recurrent, pooling, fully
connected etc.) and their connections.
2. A controller then chooses a list of possible candidate architectures from the search
space.
3. The candidate architectures are trained and ranked based on their performance on the
validation test.
4. The ranking is used to readjust the search and obtain new candidates.
5. The process iterates until reaching a certain condition and provides the optimal
architecture.
6. The optimal architecture is evaluated on the test set.

Fig:3.1 NAS general framework

In general, the landscape of NAS algorithms is quite confusing. The most popular
categorization characterizes NAS based on three major components:
a) the search space
b) the search strategy, which involves the type of controller and the evaluation of the
candidates and
c) the performance evaluation technique.
CMR College of Engineering and Technology CSE (AI&ML) 12
Neural Architecture Search (NAS)

You can refer to this excellent review by Esken et.al for more details. Also feel free to
advice the great article by Lillian Weng as an extra resource.
However, recent approaches combine the search strategy with the evaluation step, making it
hard to distinguish algorithms between them.
For that reason, we will explore NAS based solely on the search strategy. As we progress,
we will examine different search spaces and evaluation techniques. Also note that many
implementations experiment with different types or search strategies so the following
categorization is not always strict.
Search strategy refers to the methodology used to search for the optimal architecture in the
search space. We can classify NAS algorithms by their search strategy into 5 main areas:
1. Random search
2. Reinforcement learning
3. Evolutionary algorithms
4. Sequential model-based optimization
5. Gradient optimization
Random search
The most naive approach is obviously random search, which is often used as a baseline.
Here a valid architecture is chosen at random with no learning involved whatsoever.
Reinforcement learning
NAS can be very elegantly formulated as an RL problem. The agent’s action is the
generation of a neural architecture while the agent’s reward is the performance evaluation.
The action space is of course the search space. As a result, different RL methods can be used
to solve the problem.
Early works of NAS (NAS-RL, NASNet) used a recurrent neural network (RNN) as a policy
network (controller). The RNN is responsible for generating candidate architectures. The
architecture is then trained and evaluated on the validation set. The parameters of the RNN
controller are optimized in order to maximize the expected validation accuracy. How? Using
policy gradients techniques such as REINFORCE and Proximal Policy Optimization (PPO).
Similarly, ENAS uses an RNN controller trained with policy gradients. Notably, it is one of
the first works that effectively share parameters among architectures. The intuition is that the
CMR College of Engineering and Technology CSE (AI&ML) 13
Neural Architecture Search (NAS)
architectures can be viewed as part of a large graph, an approach that has been used
extensively as we will see below. ENAS training is performed in two alternating steps: a) the
RNN controller is trained with REINFORCE and b) the shared parameters are trained in
typical gradient descent form
Modular search space
To solve the global space problem, cell-based approaches were proposed in order to
“modularize” the search space. That is, mixing different blocks of layers called
modules. NASNet is the most popular algorithm in that category. NASNet learns only two
kinds of modules or “cells” : a normal cell that performs feature extraction and a
reduction cell that downsamples the input. The final architecture is built by stacking these
cells in a predefined way.
Evolutionary algorithms
Genetic Algorithms (GA) is an alternative way to optimize the network architecture.
Evolutionary algorithms start with a population of models. In each step, some models are
sampled and “reproduce” to generate offsprings by applying mutations to them. Mutations
can be local operations such as the addition of a layer, the modification of a hyperparameter,
etc. After training, they evaluated and added back to the population. The process repeats
itself until a certain condition is met.
3.4 Architecture
 Fig:3.2 Architecture of NAS
CMR College of Engineering and Technology CSE (AI&ML) 14

Neural Architecture Search (NAS)

3.5 Advantages & Disadvantages
1. Automation of Neural Network Design
 NAS automates the process of designing neural networks, removing the need for
extensive human expertise or trial-and-error tuning.
 Saves time for researchers and engineers.
2. Discovery of High-Performance Architectures
 NAS can discover novel, high-performing architectures that may surpass manually
designed models.
 Examples: Google’s NASNet and MNASNet outperform traditional CNNs on many
benchmarks.
3. Task-Specific Optimization
 NAS allows creation of architectures optimized for specific tasks (e.g., object
detection, speech recognition) and constraints (e.g., latency, memory).
4. Scalability
 Easily adapts to different datasets or hardware platforms by tuning architectures
accordingly.
5. Hardware-Aware Design
 Some NAS methods include hardware constraints, producing models suitable for
real-time applications on mobile or edge devices.
Disadvantages
1. High Computational Cost
 Many NAS techniques are extremely resource-intensive, requiring thousands of GPU
hours.
 This makes them inaccessible for individuals or small research teams.
2. Complex Implementation
 Setting up NAS requires understanding of search strategies, evaluation methods, and
performance metrics.
 Requires careful design of search space and hyperparameters.
3. Poor Transferability
 Architectures optimized for one dataset (e.g., CIFAR-10) may not perform well on
others (e.g., ImageNet), limiting their generalization.
CMR College of Engineering and Technology CSE (AI&ML) 15
Neural Architecture Search (NAS)

CHAPTER 4
CONCLUSION
CMR College of Engineering and Technology CSE (AI&ML) 16
Neural Architecture Search (NAS)

CHAPTER 4
CONCLUSION
4.1 Conclusion and Future Enhancements
Neural Architecture Search (NAS) represents a significant advancement in the field of
automated machine learning, enabling the design of optimal neural network architectures
without extensive manual intervention. By leveraging techniques such as reinforcement
learning, evolutionary algorithms, and gradient-based methods, NAS can efficiently explore
vast search spaces to identify models that achieve superior performance on a variety of tasks.

Despite challenges like high computational cost and complexity, NAS has proven its potential
in generating state-of-the-art models for image classification, object detection, and natural
language processing. Moreover, hardware-aware and multi-objective NAS approaches have
further extended its applicability to real-time and resource-constrained environments.
As research in NAS continues to evolve, future improvements in efficiency, transferability,
and interpretability are likely to make this approach more accessible and practical. Overall,
NAS is paving the way toward fully automated AI model development, reducing human effort
while maintaining or even surpassing expert-designed architectures.

Neural Architecture Search (NAS) is transforming the field of AI by automating the design of
efficient and high-performing neural networks. From reinforcement learning-based approaches
like NASNet to gradient-based methods like DARTS, NAS is making it easier and faster to
discover innovative architectures for a wide range of tasks. While challenges like
computational cost and transferability remain, ongoing research and advancements like weight
sharing and one-shot NAS are making NAS more practical and scalable. As NAS continues to
evolve, it will play a crucial role in the future of AI development, enabling more efficient,
powerful, and accessible machine learning models.

CMR College of Engineering and Technology CSE (AI&ML) 17

Neural Architecture Search (NAS)

REFERENCES

CMR College of Engineering and Technology CSE (AI&ML) 18

Neural Architecture Search (NAS)

REFERENCES

[1]. Abdelfattah M.S, Mehrotra A, Dudziak Ł, Lane N.D. (2021). Zero-cost proxies for
lightweight NAS. arXiv preprint arXiv:2101.08134
[2]. Baker B, Gupta O, Raskar R, Naik N: (2017). Accelerating neural architecture search
using performance prediction. arXiv preprint arXiv:1705.10823
[3].Banbury C, Zhou C, Fedorov I, Matas R, Thakker U, Gope D, Janapa Reddi V, Mattina M,
Whatmough P (2021) Micronets: Neural network architectures for deploying tinyml
applications on commodity microcontrollers. Proc Machine Learn Syst 3:517–532
[4]. Barnell M, Raymond C, Smiley S, Isereau D, Brown D. (2022). Ultra low-power deep
learning applications at the edge with Jetson Orin AGX hardware. In: 2022 IEEE High
Performance Extreme Computing Conference (HPEC), pp. 1–4 IEEE
[5].Bender G, Kindermans P.-J, Zoph B, Vasudevan V, Le Q. (2018). Understanding and
simplifying one-shot architecture search. In: International Conference on Machine Learning,
pp. 550–559. PMLR
[6].Benmeziane H, Maghraoui KE, Ouarnoughi H, Niar S, Wistuba M, Wang N (2021) A
comprehensive survey on hardware-aware neural architecture search. arXiv
preprint arXiv:2101.09336
[7]. Caglar B, Broggi G, Ali MA, Orgéas L, Michaud V (2022) Deep learning accelerated
prediction of the permeability of fibrous microstructures. Composites Part Appl Sci Manufact
158:106973

CMR College of Engineering and Technology CSE (AI&ML) 19