20EE38022 BTP Report-2-1
20EE38022 BTP Report-2-1
End-evaluation Report
Bachelor of Technology
in
Electrical Engineering
by
Shubhendu Pandey
20EE38022
1. The work contained in this report has been done by me under the guidance of my supervisor.
2. The work has not been submitted to any other Institute for any degree or diploma.
3. I have conformed to the norms and guidelines given in the Ethical Code of Conduct of the
Institute.
4. Whenever I have used materials (data, theoretical analysis, figures, and text) from other
sources, I have given due credit to them by citing them in the text of the thesis and giving
their details in the references. Further, I have taken permission from the copyright owners of
the sources, whenever necessary.
1
Advanced Technology Development Center
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Kharagpur - 721302, India
CERTIFICATE
This is to certify that the project report entitled “Neuromorphic Computing Hardware for
Machine Learning using Memristors” submitted by Shubhendu Pandey (Roll No. 20EE38022) to
the Indian Institute of Technology Kharagpur towards partial fulfillment of requirements for the
award of the degree of Bachelor of Technology in Electrical Engineering is a record of bonafide work
carried out by him under my supervision and guidance during Autumn Semester, 2023-24.
2
Acknowledgements
I would like to thank my Thesis supervisor Prof. Shyamal Kumar Das Mandal and co-supervisor
Prof. Fabien Alibart for directing, guiding, and supporting me throughout the project.
I am also thankful to Prof. Debdoot Sheet, coordinator of my Bachelor Thesis Project, for pro-
viding necessary facilities during the project work.
I am grateful to my friends, who were a constant support behind me in every situation. And
last but not least, my parents for their invaluable trust and support in all my choices.
3
Abstract
Memristors are a recent innovation in semiconductor technology, distinguished by their unique
ability to alter their resistive state through the application of various voltage pulses. Initially con-
ceptualized by Chua in the 1970s, practical development didn’t occur until 2008 when HP brought
this concept to fruition. Since then, the memristor has revolutionized both the semiconductor and
artificial intelligence industries, owing to its diverse range of applications. One of the most sig-
nificant advantages of this device is its non-volatile nature, making it exceptionally well-suited for
in-memory stateful logic calculations. Additionally, it has played a crucial role in reducing time
and energy consumption in machine learning applications by a staggering factor of 100, primarily
due to its ability to facilitate parallel processing.
Given the wide-ranging applications of memristors and the ongoing robust research in this do-
main, comprehensive testing of the device is essential. However, until now, there has been a notable
absence of a System on Chip (SoC) capable of harnessing the full potential of parallelism by concur-
rently delivering parallel voltage pulses and measuring current responses. To address this limitation,
we have developed the LOTUS board.
In this research endeavor, our primary objectives include enhancing the user-friendliness and
precision of the LOTUS board. To achieve this, we plan to create a graphical user interface for
the board, optimizing its measurement capabilities. Moreover, we aim to showcase the board’s
capabilities by conducting a machine-learning experiment that leverages its unique features.
i
Contents
1 Introduction 1
1.1 The world of Memristors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Application of memristors and memristive crossbars . . . . . . . . . . . . . . . . . . 2
2 Overview 3
2.1 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Our Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4 Results 8
4.1 GUI Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2 Crossbar Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2.1 Fixing calibration errors incurred due to temperature differences . . . . . . . 9
4.2.2 Fixing Offset Current Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.2.3 Fixing errors due to parasitic components . . . . . . . . . . . . . . . . . . . 10
6 References 12
ii
Chapter 1
Introduction
However, it wasn’t until nearly four decades later that researchers at Hewlett-Packard Labs, led
by R. Stanley Williams, announced the successful creation of a practical memristor device in 2008.
This milestone marked a turning point in the history of electronics, as memristors had the potential
to revolutionize computing, memory storage, and signal processing.
In the realm of advanced electronic devices and computing, memristors have emerged as an
exciting and groundbreaking technology. What makes memristors particularly intriguing is their
ability to exhibit a distinctive electrical property known as “memristance” Memristance refers to
the property of a device to alter its resistance in response to the magnitude and direction of the
applied voltage. This property enables memristors to “remember” the charge that has previously
flowed through them, effectively acting as non-volatile memory elements. In simpler terms, unlike a
simple resistor, a memristor can attain multiple resistive values depending on the applied voltage.
The I-V curve in the figure 1.1 (b) showcases a memristive device with six different stable resistance
states.
1
1.2 Application of memristors and memristive crossbars
The memory capabilities of memristors extend beyond simple storage. They can also perform com-
putational tasks within the memory itself, offering a novel approach to computing architecture.
This in-memory computation capability holds the potential to overcome one of the most significant
challenges in traditional computing systems—the Von Neumann bottleneck. The Von Neumann
bottleneck refers to the limited bandwidth and efficiency caused by the separation of memory and
processing units.
By integrating memory and computation, memristors enable parallelism, reducing the data
movement required in conventional architectures. In-memory computation allows data to be pro-
cessed within the memory itself, eliminating the need for frequent data transfers to and from the
central processing unit (CPU). This breakthrough has the potential to revolutionize computing,
particularly in domains that heavily rely on memory-intensive tasks such as machine learning (ML).
One key application of resistive memory in ML is the implementation of neural networks. Neural
networks serve as the backbone of modern ML algorithms, and their computational demands are
constantly increasing. By leveraging the parallelism enabled by resistive memory, neural network
computations can be executed efficiently and in close proximity to the data, minimizing latency and
energy consumption. This approach proves especially valuable for tasks such as pattern recognition,
image processing, and natural language understanding.
All of this is achieved by working with memristive crossbars. The fig () shows an ideal memristor
crossbar. In a memristive crossbar, memristor devices are arranged in a two-dimensional grid,
forming rows and columns of intersecting wires. This grid-like structure allows for a multitude of
connections and interactions between the memristors.
The key characteristics and applications of memristive crossbars include:
1. Non-Volatile Memory: Memristive crossbars can store data in a non-volatile manner. This
means that the data remains intact even when the power is turned off, making them suitable
for applications where persistent data storage is essential.
3. Parallel Processing: Memristive crossbars enable parallel data processing. Multiple mem-
ristors can be accessed simultaneously, facilitating the efficient execution of operations in
applications like neural networks.
2
Chapter 2
Overview
In order to develop and iterate on these hardware designs, a profound understanding of their
behavior and underlying physical mechanism is required. For emerging types of memory for which
accurate models do not yet exist, an understanding of the material parameters of the fabricated
devices requires in-depth electrical characterizations, where hardware-in-the-loop simulations [4] are
preferred over standard simulations using complex models. Since integrated circuits and nanodevices
are often integrated with CMOS technologies, there is a growing demand for new hardware-specific
characterization instruments that are capable of handling both analog and digital signals.
One of the most promising approaches to overcoming the memory wall arising from von Neu-
mann’s architecture is the in-memory computing (IMC) paradigm [5], which is particularly suited to
artificial intelligence applications in the form of neuromorphic circuits [6] and memristive artificial
neural networks [7]. These IMC architectures leverage the analog behavior of emerging resistive
memory devices (also called memristors [8]), which can be implemented in multiple ways [9] and
in different topologies [10], such as passive and one-transistor-one-resistor (1T1R) crossbar arrays
[11]. In the absence of integrated CMOS-based control electronics, driving a memristive crossbar
requires external custom instrumentation electronics that can perform parallel voltage-current mea-
surements simultaneously, with both digital and analog signals. These instrumentation systems are
used to provide and measure electrical signals to and from the crossbar arrays, with the help of fast
programming pulses [12].
As crossbar arrays tend to have a high number of inputs and outputs, the strategy used until
recently was to multiplex a small number of these pulsed signals to the connections of the devices,
using commercially available instruments [13], [14] or custom ones [15]–[22] that integrate switching
matrices. However, these systems lack the capability to generate expansive parallel voltage vec-
tors and require the interconnection of multiple instruments in parallel to read parallel vectors of
currents. These drawbacks nullify one of the key advantages of IMC architectures, which is their
3
ability to perform parallel calculations in a single operation [23], as required for vector-matrix mul-
tiplications (VMM) with artificial neural network applications [24]
Recently, a promising FPGA-based approach [15] has been proposed to address this issue of
parallelism, with the ability to perform both writing and reading operations in parallel. Instead of
using a switch matrix, the developers of this approach implemented one measuring channel for each
input and output connection. However, as a large quantity of electronic components was necessary
to achieve this, each channel design had to be optimized in terms of power consumption and circuit
simplicity. These choices led to limited reading speeds of 500 s and constrained signal flexibility,
as timings were shared between each group of eight channels. To achieve full parallelism without
losing accuracy and speed, new system architectures need to be considered. Such a system would
represent progress towards highly parallel innovative circuits, such as sensor matrices and quantum
dot interfacing, while opening novel avenues for the exploration of neuromorphic circuits.
Figure 2.1: Top-down photography of the assembled LOTUS system, where the components are (1)
the Zynq SOM, (2) a four-channel APMU group (eight in total), (3) the DUT interface, (4) a
variable voltage supply, (5) an eight-GPIO bank, (6) the external sensor connector, and (7) the
board-to-board (B2B) interface with (8) its input and (9) its output
4
Chapter 3
We propose to develop a new graphical user interface (GUI) for the LOTUS board. This GUI
will be designed to be simple and easy to use, even for researchers with no prior experience with
electronics or programming. The GUI will allow researchers to perform the following tasks:
Benefits of the new GUI: The new GUI will offer a number of benefits to researchers like
reduced learning curve and increased productivity. Since the researchers will be able to start
using the LOTUS board immediately, without having to learn complex electronics or programming
concepts of the internal architecture of the board.
Implementation: The GUI will be implemented using the QtCreator UI designer and the
PyQt library. The backend will be designed by integrating the GUI with the Python functions
available in the PyQt library. We believe that the new GUI will be a valuable tool for memristor
researchers, and we are excited to develop it.
5
3.2 Improving the accuracy of measurements by the board
The device’s performance is currently afflicted by significant accuracy limitations attributable to a
confluence of factors. To test the working of the device under extreme conditions, we engineered
a resistive matrix characterized by a high resistance state to low resistance state ratio of 10M to
100. The large High Resistance State to Low Resistance State ratio makes the effect of
leakage currents quite significant and the task of minimizing the errors quite difficult. However, if
the device is able to perform well in these conditions, it will demonstrate its accuracy and usefulness
in a variety of real-world applications.
In other words, if the device can perform well under extreme conditions where leakage currents
are high, it is likely to be very accurate and useful in most real-world applications where leakage
currents are lower, usefulness of our device in various conditions.
The Python code for operating with the board and finding the heatmap is here
The heatmap above shows that the % error in the measurement is quite high. The probable cuase
of the inaccuaries might be temperature variations, inadequate calibration procedures, neglectance
of parasitic resistances, and leakage currents. The aim in coming months is to address these issues
and enhance the device’s overall functionality and reliability.
6
We intend to perform an inference experiment in the later stages of the project to demonstrate
the same. The process to be followed is as follows:
1. Train the model conventionally and find the weights to reduce the cost function
Figure 3.2: Model for performing inference experiment using memristor crossbar
7
Chapter 4
Results
4.1 GUI Results
In addition to reviewing the literature and identifying and deciding on the work to be done, we
have created a basic model of a graphical user interface (GUI) to help other researchers use the
LOTUS board. The GUI is still under development, and we are adding new features to make it more
user-friendly. We will continue to work on the GUI until the LOTUS board is free of calibration
and leakage errors, so that it is reliable enough for researchers to use for their work.
8
4.2 Crossbar Results
A script was written for sending the pulses from the row APMUs and sensing the current from the
column APMUs. Then the resistance was calculated and the error from the expected resistance
was found. The heatmap shown below shows that the errors are huge and work needs to be done
to reduce them. This is the immediate goal for this project.
To understand better, let’s suppose we have just 2 nodes. I apply V1 voltage at the first node
and V2 voltage at the second node. I have connected the nodes together through a resistance.
The error in the measurement of resistance can arise from errors in the measurement of voltage or
current. The first two errors correspond to deviations in the desired voltage, and the third error is
due to an error in the measurement of current.
(a) Before re-calibrating the device (b) After re-calibrating the device
We see that recalibrating reduces the error for lower resistance values considerably but not for
higher resistance values. Thus we shift our focus to errors that might be induced due to wrong
current sensing.
9
4.2.2 Fixing Offset Current Errors
Setting all nodes to 0 V should theoretically result in zero current being sensed by any of the
APMUs. However, during the experiment, it was observed that some current was still being sensed
by all nodes, as illustrated in the figure 4.3(a). To address this issue, we account for offset currents.
We calculate these offset currents by applying 0 voltage to all nodes, as demonstrated earlier. When
applying the desired voltage, we adjust our current readings by subtracting these calculated values.
This correction significantly enhances our resistance readings, as depicted in the figure 4.3(b).
(a) Offset Currents on applying 0 voltage (b) After re-calibrating the device
As can be seen in the figure the mean error has gone down to 20% but the errors in the lowest
resistance state and the high resistance state are still high.
10
Chapter 5
11
Chapter 6
References
1. X. Zou, S. Xu, X. Chen, L. Yan, and Y. Han, ”Breaking the von Neumann bottleneck:
architecture-level processing-in-memory technology,” Sci. China Inf. Sci., vol. 64, no. 6, p.
160404, Apr. 2021, doi: 10.1007/s11432-020-3227-1.
2. W. Pan, Z. Li, Y. Zhang, and C. Weng, ”The New Hardware Development Trend and the
Challenges in Data Management and Analysis,” Data Sci. Eng., vol. 3, no. 3, pp. 263–276,
Sep. 2018, doi: 10.1007/s41019-2018-0072-6.
3. K. Berggren et al., ”Roadmap on emerging hardware and technology for machine learning,”
Nanotechnology, vol. 32, no. 1, p. 012002, Oct. 2020, doi: 10.1088/1361-6528/aba70f.
8. J. M. Tour and T. He, ”The fourth element,” Nature, vol. 453, no. 7191, pp. 42–43, May
2008, doi: 10.1038/453042a.
9. L. Wang, C. Yang, J. Wen, S. Gai, and Y. Peng, ”Overview of emerging memristor families
from resistive memristor to spintronic memristor,” J. Mater. Sci. Mater. Electron., vol. 26,
no. 7, pp. 4618–4628, Jul. 2015, doi: 10.1007/s10854-015-2848-z.
10. W. Xu, J. Wang, and X. Yan, ”Advances in memristor-based neural networks,” Front. Nan-
otechnol., vol. 3, 2021, doi: 10.3389/fnano.2021.645995.
11. M. Hu, H. Li, Y. Chen, Q. Wu, G. S. Rose, and R. W. Linderman, ”Memristor crossbar-based
neuromorphic computing system: A Case Study,” IEEE Trans. Neural Netw. Learn. Syst.,
vol. 25, no. 10, pp. 1864–1878, Oct. 2014, doi: 10.1109/TNNLS.2013.2296777.
12
12. M. Park, M. Kang, and S. Kim, ”Pulse frequency dependent synaptic characteristics in
Ta/SiN/Si memristor device for neuromorphic system,” J. Alloys Compd., vol. 882, p. 160760,
Nov. 2021, doi: 10.1016/j.jallcom.2021.160760.
14. Keithley, ”Ultra-Fast I-V Module for the Model 4200-SCS.” 4225-RPM Datasheet. Accessed:
Jul. 21, 2021.
13