L1 - General Introduction
L1 - General Introduction
for AI and ML
1 J. Joshua Yang
Challenges future computing: four walls
Device:
Moore’s law
2 J. Joshua Yang
Background: Issues with classical computer hardware in the ‘big-data’ era
Architecture:
Classical computer 1. Separate processor and storage (Von
Neumann bottleneck)
2. Sequential process
3. Analog/Digital conversion
Transistors (static dopants):
• Electron manipulation: fast but volatile
• Doesn’t scale anymore: Moore’s law
Brain networks
Architecture:
Neurons, Synapses and networks
1. In-memory computing
2. Parallel comping Computing
Natural
3. Analog computing Accelerator
Intelligence
4. More neuroscience principles
Q. Xia, J. Joshua Yang, "Memristive crossbar arrays for brain-inspired computing” Nature Materials 18, 309 (2019)
3 J. Joshua Yang
Post-CMOS era
Heterogeneous integrations
(development)
Existing technologies
Extended CMOS
Completely new concepts
(research)
New technologies
Memristor
4 J. Joshua Yang
Emerging materials and devices are needed!
CMOS
100 W 100 W
1011 neurons 109 transistors
1014-15 synapses 10 transistors / synapse
Diffusive memristor è neuron 108 synapses
Drift memristor è synapse 105 neurons
Need ~ 107 chips
More efficient with emerging devices
inefficient with CMOS chips
5 J. Joshua Yang
Devices: Mobilizing dopants/ions is the way to go!
CMOS:
• Manipulating electrons (fast, but volatile)
• Immobile dopants
• Static profile control
• Failed if dopant moves
• Software learning (devices not changing)
Different from biological systems, Intelligence?
6 J. Joshua Yang
Architecture/algorithm: learning from the brain?
"Bridging Biological and Artificial Neural Networks with Emerging Neuromorphic Devices: Fundamentals, Progress, and Challenges" , Advanced Materials, 1902761, (2019, review).
7 J. Joshua Yang
History of memory devices
8 J. Joshua Yang
History of memory devices
9 J. Joshua Yang
Nonvolatile
Destructive reading
10 J. Joshua Yang
Different types of memory devices
11 J. Joshua Yang
Courtesy of Prof. Han Wang
12 J. Joshua Yang
13 J. Joshua Yang
Amount of memory is increasing
14 J. Joshua Yang
Courtesy of Prof. Han Wang
15 J. Joshua Yang
What’s in your iphone?
Memory is a big chunk!
16 J. Joshua Yang
Trend for memory: Nonvolatile memory becomes increasingly useful!
17 J. Joshua Yang
Courtesy of Prof. Han Wang
18 J. Joshua Yang
An example of a Nonvolatile
emerging memory technology
19 J. Joshua Yang
Textbooks have been rewritten
“Electronic Devices and Circuit Theory” (11th Edition) “Introductory Circuit Analysis” (12th Edition)
Apr 30, 2012 January 15, 2010
by Robert L. Boylestad and Louis Nashelsky by Robert L. Boylestad
20 J. Joshua Yang
The Fourth fundamental passive circuit element
21 J. Joshua Yang
History of Memristor: Theory
Resistor – 1827
Georg Ohm
RESISTOR CAPACITOR
v=Ri q=Cv
Capacitor - 1745
Volta / von Kleist & van Musschenbroek
INDUCTOR Benjamin Franklin
φ=Li
Inductor – 1831
Michael Faraday
Joseph Henry
22 J. Joshua Yang
History of Memristor: Theory
Introduction to Nonlinear Network Theory, NY: McGraw-Hill, 1969
1960’s – Mathematical Foundation for circuit theory of nonlinear systems Developed
RESISTOR CAPACITOR
dv = R di dq = C dv
i dq /dt = i q
dφ/dt = v
23 J. Joshua Yang
History of Memristor: Theory
M is a “memory resistor” –
it is a nonvolatile resistive memory!
RESISTOR CAPACITOR
dv = R di dq = C dv Proved that no equivalent circuit to M
could be made using any R, C and L –
is a new and unique ‘basis function’
i dq /dt = i q
dφ/dt = v
INDUCTOR MEMRISTOR
dφ = L di dφ = M dq
24 J. Joshua Yang
History of Memristor: Experiments
Nb/Nb2O5125nm/Bi
L. O. Chua and S. M. Kang, Proc. IEEE 64 209 (1976). W. R. Hiatt and T. W. Hickmott, Appl. Phys. Lett. 6, 106 (1965)
Research effort in searching such devices faded due to the rise of Si technology
Until the end of last centaury when the physical limit of Si technology is close.
25 J. Joshua Yang
Brief introduction of memristive devices
Metal 1
Metal 1
oxide
oxide V
Metal 2
Metal 2
26 J. Joshua Yang
The first functioning oxide Nanodevice
Cross-section
TEM
Nano-devices 1x17
BE Top electrode
TE
Localized region
oxide
50
n m
oxide
Conduction channel/filament
Bottom electrode
V 5nm
27 J. Joshua Yang
Image and signature of memristors
Nano-devices 1x17
BE 200
TE
100
Current ( uA )
50
n m
oxide
0
-100
-200-2
-1 0 1 2
Voltage ( V )
V
28 J. Joshua Yang
Promises
Promises 85ps ON/OFF switching
Challenges
1. Speed (85ps)
Speed (85ps) • Mechanism
2. Scalability (2nm)
3. Multilevel (>64) 1. Variability
29 J. Joshua Yang
Promises
2. Scalability
Scalability(2nm)
(2nm)
1. Variability
3. Multilevel (>64)
4. Stackability (>8 layers) Nonvolatile
2. Nonlinearity
5. CMOS compatibility memory
(selector)
6. Non-volatility (>10years)
7. Non-destructive reading 3. Synaptic dynamics
Neuromorph
8. Low cost ic 35
computing
Pi et al., Nature Nanotechnology 14, (2019).
9. …… (Q. Xia)
30 J. Joshua Yang
Promises
Promises Challenges
>64 resistance levels
1. Speed (85ps) • Mechanism
2. Scalability (2nm)
3. Multilevel
Multilevel(>64)
(>64) 1. Variability
31 J. Joshua Yang
Promises
Promises Challenges
3D integration of memristor on CMOS
1. Speed (85ps) • 8 Layers
Mechanism
2. Scalability (2nm)
1. Variability
3. Multilevel (>64)
4. Stackability (>8(8layers)
Stackability layers) Nonvolatile
2. Nonlinearity
5. CMOS compatibility memory
(selector)
6. Non-volatility (>10years)
7. Non-destructive reading 3. Synaptic dynamics
Neuromorph
8. Low cost ic computing
9. …… Lin et al., Nature Electronics 3, 225 (2020)
(Q. Xia)
32 J. Joshua Yang
(1) Emerging Memory:
exponential growth in demand
“Cloud” is really
memory and
Storage
https://www.grandviewresearch.com/industry-analysis/next-generation-non-volatile-memory-market
33 J. Joshua Yang
(1) Emerging Memory:
RRAM – fast, non-volatile, low energy, Ultra-Dense
0 nm
3
m
2µ
Switching materials
(e.g. TiO2)
34 J. Joshua Yang
(1) Emerging Memory:
Memristor – fast, non-volatile, low energy
Flash
Memristor PCM STTRAM SRAM DRAM HDD
(NAND)
Energy per bit (pJ) 0.1–3 2-25 0.1-2.5 0.0005 0.005 0.00002 1–10x10 9
as long as
Retention years years years voltage << second years years
applied
35 J. Joshua Yang
(2) Emerging logic:
Type I: Memristor/CMOS hybrid logic – FPGA
36 J. Joshua Yang
(2) Emerging Logic:
Type II: implication logic
Bertrand Russell
37 J. Joshua Yang
(2) Emerging Logic:
Type II: implication logic based NAND
Advantages:
• Denser
• Non-volatile logic
P Q S
RG
38 J. Joshua Yang
(3) Computing with Memristive devices
Natural Intelligence Original
Improved speed-energy
Gaussian Disk
efficiency
Average
Artificial
Synapses
Nature Comm. 8, 752 (2017)
Nature Materials 16, 101 (2017)
Science 364, 570 (2019)
Laplacia(2020)
Sci. Adv. 6, eaba6173 LoG Prewitt (Horizontal) Sobel (Horizontal)
Original Gaussian Disk
Nature Comm. 11,n 1861 (2020)
64 x 64 1T1R array
Artificial ML
Memristor Accelerators
Neurons
Nature Electronics 1, 137 (2018) array Nature Electronics 1, 52 (2018)
Nature Comm. 9, 3208 (2018) Nature Comm. 9,Laplacia
2385 (2018)
Nature Comm. 11, 51 (2020) LoG Prewitt (Horizontal)
Adv. Mater. 29, 1705914
n (2018)
Nature Nanotechnology 15, 776 (2020) Nature Machine Intelligence 1, 49 (2019)
Nature Electronics 2, 115 (2019). 64 x 64
Nature Machine Intelligence 1, 434 (2019)
Example: Sensing System For Robotics Nature Electronics 3, 409 (2020)
Nature 577, 641 (2020)
Intelligent
Systems
Nature Comm. 8, 882 (2017). Nature Materials 16, 396 (2017).
Nature Comm. 8, 15666 (2017). Nature Comm. 9, 417 (2018)
Adv. Mater. 29, 1604457 (2017). Nature Electronics 1, 130 (2018).
Adv. Mater. 31, 1902761 (2019) Nature Electronics 1, 548 (2018).
Nature Materials 18, 309 (2019). Nature Nanotechnology 14, 35 (2019).
Nature Electronics 3, 225 (2020) Nature Review Materials 5, 173 (2020)
39 J. Joshua Yang
Computing with memristive networks
40 J. Joshua Yang
Vector x Matrix multiplication accelerator
•Computation intensive tasks (many multiplication and addition steps)
•Only one step in dot-product engine
1. In-memory computing
2. Parallel comping
3. Analog computing
41 J. Joshua Yang
Hardware demonstration of accelerator
42 J. Joshua Yang
Image Processing (experimental demo)
Image Compression Using Analog Memristor Array (DCT) (20:3 compression ratio)
Original image Software processed Memristor array processed (Experimental)
43 J. Joshua Yang
Computing with memristive networks
44 J. Joshua Yang
The key idea in brain-inspired computing
http://www.macs.hw.ac.uk/~dwcorne/
45 J. Joshua Yang
The key idea in brain-inspired computing
http://www.macs.hw.ac.uk/~dwcorne/
46 J. Joshua Yang
The key idea in brain-inspired computing
http://www.macs.hw.ac.uk/~dwcorne/
47 J. Joshua Yang
The key idea in brain-inspired computing
Learning/training:
48 J. Joshua Yang
The key idea in brain-inspired computing
http://www.macs.hw.ac.uk/~dwcorne/
49 J. Joshua Yang
The key idea in brain-inspired computing
http://www.macs.hw.ac.uk/~dwcorne/
50 J. Joshua Yang
The key idea in brain-inspired computing
51 J. Joshua Yang
Beyond accelerator: Neural Networks capable of learning
52 J. Joshua Yang
2-layer Neural Network for in-situ Online Training
53 J. Joshua Yang
Powerful applications, but the training is demanding…
Nature Electronics 1, 52 (2018) Nature Comm. 9, 2385 (2018) Nature Electronics 3, 225 (2020)
54 J. Joshua Yang
Computing with memristive networks
55 J. Joshua Yang
The origin of the timing in bio-synapse: ‘programming’
Very Large Impulse from upper axon terminal à open NMDA receptor, Ca2+ diffuses
through à activate some bio-activities à increasing both the number and conductance of
AMPA receptorsà synapse enhanced – long term plasticity.
56 J. Joshua Yang
Similar Diffusion dynamics in artificial devices
Ag: SiNxOy
57 J. Joshua Yang
Snapshots
58 J. Joshua Yang
Experimental results: Short-term and long-term synaptic plasticity
Ag memristor
Paired pulse facilitation/depression
TaOx memristor
Ag memristor
Conductance
TaOx memristor
Ag memristor
Postsynaptic
59 J. Joshua Yang
Artificial Neurons: role of ion dynamics in bio-neurons
Action
Potential
Repolarization
Depolarization
60 J. Joshua Yang
Emulating an Ion channel in a Neuron
61 J. Joshua Yang
Experimental results: leaky integrate-and-fire neuron
Ion Channel
Conductance
Ag memristor
Plasma Membrane
62 J. Joshua Yang
The 1st fully memristive spiking neural network
8×8 Ta/HfOx
memristor synapses
8 Ag memristor neurons
(with external capacitors)
63 J. Joshua Yang
Memristor spiking neural network: a self-organizing map
64 J. Joshua Yang
Experimental results: Unsupervised learning with the 1st fully memristive NN
Convolution layer Dense layer
(Fixed) (physics driven automatic evolution)
65 J. Joshua Yang
Summary
Materials and Devices: beyond CMOS
CMOS
100 W 100 W
1011 neurons 109 transistors
1014-15 synapses 10 transistors / synapse
108 synapses
Diffusive memristor è neuron
Drift memristor è synapse
105 neurons
Need ~ 107 chips
66 J. Joshua Yang
Devices: Mobilizing dopants/ions is the way to go!
CMOS:
• Manipulating electrons (fast, but volatile, good for logic)
• Immobile dopants
• Static profile control in the fabrication stage
• Failed if dopant moves
• Software learning (devices not changing)
Different from biological systems, Intelligence?
67 J. Joshua Yang
Architecture/algorithm: learning from the brain?
How can brain be so good with such lousy
devices ?
(slow, big, stochastic, inaccurate)
"Bridging Biological and Artificial Neural Networks with Emerging Neuromorphic Devices: Fundamentals, Progress, and Challenges" , Advanced Materials, 1902761, (2019, invited review).
68 J. Joshua Yang
Outlook: Roadmap
W.G. Kim et al., SK-hynix
J. Joshua Yang et al., HP Labs
VLSI Technology Symposium
(2014) 138-139
DRAM
Flash
RAM
Flash
Hard Disk
Solid State Disk Optical disk
Floppy Disk
Time
69 J. Joshua Yang
How materials principles used in emerging
devices?
Examples
70 J. Joshua Yang
Simplified picture of switching mechanism: TiOx
Where: interface;
What: oxygen vacancies (2+, n-dopant); Schottky-like: OFF
How: drift.
w
metallic Φb
OFF
ON
Pt TiO2
V
Ohmic-like: ON
w
TiO2-X
Φb
Ti TiO2
J. Joshua Yang et al., Nature Nanotechnology 3, 429 (2008)
J. P Strachan et al., Advanced Materials 22, 3573 (2010)
F. Miao, J.P. Stranchan, J. JoshuaYang* et al., Adv. Mater. 23 5633 (2011) Vo2+ n-type dopant
71 J. Joshua Yang
Switching is a result of ionic motion under the
combined effect of electric field and Joule heating
Current
Current
Current
Current
Voltage
Voltage Voltage Voltage
Electric e- Electric Ec
Concentration Temperature
potential potential gradient
gradient Eb
gradient gradient
Et
e- e- - Ef
e- e
Cathode
J. Joshua Yang et al., Nature Nanotechnology 8, 13 (2013) Ef
e- 72 J. Joshua Yang
e-
Nonlinear
IV
thermal dominating
IV
Current
Current
Mechanisms: four Ionic transport mechanisms
Bipolar Nonpolar Nonpolar
Voltage Voltage
Drift Electromigration
Electric e- Electric
Concentrati
potential potential
gradient
Linear gradient Bi-stable
gradient
IV e- -
e-
Current
Current
e- e
e-
e-
Voltage Voltage
Fick Diffusion Thermophoresis
-
Electric Temperature Ec
Concentration
potential gradient
gradient Eb
gradient
Ef
Cathode
Ev Oxid
74 J. Joshua Yang
Nanodevice family: M/I/M with two interfaces
Pt 10
TiO2 I 0 I
TiO2-X Measured
V
V -10
-1.5 1.5 V
-V
TiO2-X
Complementary
+V
Intrinsic diode
Pt
5 1.5
switching
1.0 Measured
0
+V
-V
+V
0.5
-V
-5 Measured
0.0
-4 -2 0 2 -2 0 2
I I
V
V
75 J. Joshua Yang