See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/256377625

Integration of nanoscale memristor synapses in neuromorphic computing

architectures

Article  in  Nanotechnology · September 2013

DOI: 10.1088/0957-4484/24/38/384010 · Source: PubMed

CITATIONS READS

237 416

5 authors, including:

Giacomo Indiveri Bernabé Linares-Barranco

University of Zurich Spanish National Research Council
314 PUBLICATIONS   5,937 CITATIONS    227 PUBLICATIONS   5,334 CITATIONS   

SEE PROFILE SEE PROFILE

Robert Legenstein George Deligeorgis

Graz University of Technology Foundation for Research and Technology - Hellas
76 PUBLICATIONS   2,009 CITATIONS    97 PUBLICATIONS   1,229 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Cardiac Regeneration and Tissue Engineering View project

Sinergia View project

All content following this page was uploaded by Themis Prodromakis on 21 March 2014.

The user has requested enhancement of the downloaded file.

Home Search Collections Journals About Contact us My IOPscience

Integration of nanoscale memristor synapses in neuromorphic computing architectures

This content has been downloaded from IOPscience. Please scroll down to see the full text.

2013 Nanotechnology 24 384010

(http://iopscience.iop.org/0957-4484/24/38/384010)

View the table of contents for this issue, or go to the journal homepage for more

Download details:

IP Address: 190.0.16.58
This content was downloaded on 16/10/2013 at 16:18

Please note that terms and conditions apply.

IOP PUBLISHING NANOTECHNOLOGY
Nanotechnology 24 (2013) 384010 (13pp) doi:10.1088/0957-4484/24/38/384010

Integration of nanoscale memristor

synapses in neuromorphic computing
architectures
Giacomo Indiveri1 , Bernabé Linares-Barranco2 , Robert Legenstein3 ,
George Deligeorgis4 and Themistoklis Prodromakis5,6
1
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
2
Instituto de Microelectrónica de Sevilla, IMSE-CNM, CSIC and University of Sevilla, Sevilla, Spain
3
Institute for Theoretical Computer Science, Graz University of Technology, Graz, Austria
4
CNRS-LAAS and Universitè de Toulouse, 7 avenue du colonel Roche, F-31400 Toulouse, France
5
Center for Bio-Inspired Technology, Department of Electrical and Electronic Engineering, Imperial
College London, UK
6
Nano Research Group, School of Electronics and Computer Science, University of Southampton, UK

E-mail: [email protected]

Received 4 January 2013, in final form 6 March 2013

Published 2 September 2013
Online at stacks.iop.org/Nano/24/384010

Abstract
Conventional neuro-computing architectures and artificial neural networks have often been
developed with no or loose connections to neuroscience. As a consequence, they have largely
ignored key features of biological neural processing systems, such as their extremely
low-power consumption features or their ability to carry out robust and efficient computation
using massively parallel arrays of limited precision, highly variable, and unreliable
components. Recent developments in nano-technologies are making available extremely
compact and low power, but also variable and unreliable solid-state devices that can
potentially extend the offerings of availing CMOS technologies. In particular, memristors are
regarded as a promising solution for modeling key features of biological synapses due to their
nanoscale dimensions, their capacity to store multiple bits of information per element and the
low energy required to write distinct states. In this paper, we first review the neuro- and
neuromorphic computing approaches that can best exploit the properties of memristor and
scale devices, and then propose a novel hybrid memristor-CMOS neuromorphic circuit which
represents a radical departure from conventional neuro-computing approaches, as it uses
memristors to directly emulate the biophysics and temporal dynamics of real synapses. We
point out the differences between the use of memristors in conventional neuro-computing
architectures and the hybrid memristor-CMOS circuit proposed, and argue how this circuit
represents an ideal building block for implementing brain-inspired probabilistic computing
paradigms that are robust to variability and fault tolerant by design.
(Some figures may appear in colour only in the online journal)

1. Introduction itself [1, 2]. Neuro-computing has been very popular in the
past [3, 4], eventually leading to the development of abstract
The idea of linking the type of information processing that artificial neural networks implemented on digital computers,
takes place in the brain with theories of computation and useful for solving a wide variety of practical problems [5–9].
computer science (something commonly referred to as neuro- However, the field of neuromorphic engineering is a much
computing) dates back to the origins of computer science younger one [10]. This field has been mainly concerned

0957-4484/13/384010+13$33.00 1 c 2013 IOP Publishing Ltd Printed in the UK & the USA

Nanotechnology 24 (2013) 384010 G Indiveri et al

with hardware implementations of neural processing and The goal is to eventually integrate many of these cores onto a
sensory-motor systems built using very large scale integration single chip, and to assemble many multi-core chips together,
(VLSI) electronic circuits that exploit the physics of silicon to simulate networks of simplified spiking neurons with
to reproduce directly the biophysical processes that underlie human-brain dimensions (i.e. approximately 1010 neurons
neural computation in real neural systems. Originally, the and 1014 synapses) in real time. In the mean time, IBM
term ‘neuromorphic’ (coined by Carver Mead in 1990 [11]) simulated 2.084 billion neurosynaptic cores containing 53 ×
was used to describe systems comprising analog integrated 1010 neurons and 1.37 × 1014 synapses in software on the
circuits, fabricated using standard complementary metal oxide Lawrence Livermore National Lab Sequoia supercomputer
semiconductor (CMOS) processes. In recent times, however, (96 Blue Gene/Q racks), running 1542× slower than real
the use of this term has been extended to refer to hybrid time [16], and dissipating 7.9 MW. A diametrically opposite
analog/digital electronic systems, built using different types approach is represented by the Neurogrid system [17]. This
of technologies. system comprises an array of sixteen 12 × 14 mm2 chips,
Indeed, both artificial neural networks and neuromorphic each integrating mixed analog neuromorphic neuron and
computing architectures are now receiving renewed attention synapse circuits with digital asynchronous event routing
thanks to progress in information and communication logic. The chips are assembled on a 16.5 × 19 cm2 Printed
technologys (ICTs) and to the advent of new promising nano- Circuit Board (PCB), and the whole system can model over
technologies. Some present day neuro-computing approaches one million neurons connected by billions of synapses in
attempt to model the fine details of neural computation real time, and using only about 3 W of power [18]. As
using standard technologies. For example, the Blue Brain opposed to the neuro-computing approaches that are mainly
project, launched in 2005, makes use of a 126 kW Blue concerned with fast and large simulations of spiking neural
Gene/P IBM supercomputer to run software that simulates networks, the Neurogrid has been designed following the
with great biological accuracy the operations of neurons and original neuromorphic approach, exploiting the characteristics
synapses of a rat neocortical column [12]. Similarly, the of CMOS VLSI technology to directly emulate the biophysics
BrainScaleS EU-FET FP7 project aims to develop a custom and the connectivity of cortical circuits. In particular,
neural supercomputer by integrating standard CMOS analog the Neurogrid network topology is structured by the data
and digital VLSI circuits on full silicon wafers to implement and results obtained from neuro-anatomical studies of the
about 262 thousand integrate-and-fire (I&F) neurons and 67 mammalian cortex. While offering less flexibility in terms of
million synapses [13]. Although configurable, the neuron and connectivity patterns and types of synapse/neuron models that
synapse models are hardwired in the silicon wafers, and the can be implemented, the Neurogrid is much more compact
hardware operates about 10 000 times faster than real biology, and dissipates orders of magnitude less power than the
with each wafer consuming about 1 kW power, excluding all other neuro-computing approaches described above. All these
external components. Another large-scale neuro-computing approaches have in common the goal of attempting to simulate
project based on conventional technology is the SpiNNaker large numbers of neurons, or as in the case of Neurogrid, to
project [14]. The SpiNNaker is a distributed computer, which physically emulate them with fine detail.
interconnects conventional multiple integer precision multi Irrespective of the approach followed, nanoscale synapse
ARM core chips via a custom communication framework. technologies and devices have the potential to greatly
Each SpiNNaker package contains a chip with 18 ARM9 improve circuit integration densities and to substantially
Central Processing Units (CPUs) on it, and a memory chip reduce power dissipation in these systems. Indeed, recent
of 128 MB synchronous dynamic random access memory trends in nanoelectronics have been investigating emerging
(DRAM). Each CPU can simulate different neuron and low-power nanoscale devices for extending standard CMOS
synapse models. If endowed with simple synapse models, a technologies beyond the current state-of-art [19]. In particular,
single SpiNNaker device ARM core can simulate the activity resistive random access memory (ReRAM) is regarded as
of about 1000 neurons in real time. More complex synapse a promising technology for establishing next-generation
models (e.g. with learning mechanisms) would use up more non-volatile memory cells [20], due to their infinitesimal
resources and decrease the number of neurons that could be dimensions, their capacity to store multiple bits of information
simulated in real time. The latest SpiNNaker board contains per element and the minuscule energy required to write
47 of these packages, and the aim is to assemble 1200 of these distinct states. The factors driving this growth are attributed to
boards. A full SpiNNaker system of this size would consume the devices’ simple (two terminals) and infinitesimal structure
about 90 kW. The implementation of custom large-scale (state-of-art is down to 10×10 nm2 [21]) and ultra-low-power
spiking neural network hardware simulation engines is being consumption (<50 fJ/bit) that so far are unmatched by
investigated also by industrial research groups. For example, conventional VLSI circuits.
the IBM group led by D S Modha recently proposed a Various proposals have already been made for lever-
digital ‘neurosynaptic core’ chip integrated using a standard aging basic nanoscale ReRAM attributes in reconfigurable
45 nm silicon on insulator (SOI) process [15]. The chip architectures [22], neuro-computing [23] and even artificial
comprises 256 digital I&F neurons, with 1024 × 256 binary synapses [24–28]. However the greatest potential of these
valued synapses, configured via a static random access nanoscale devices lies in the wide range of interesting
memory (SRAM) cross-bar array, and uses an asynchronous physical properties they possess. Neuromorphic systems can
event-driven design to route spikes from neurons to synapses. harness the interesting physics being discovered in these

2
Nanotechnology 24 (2013) 384010 G Indiveri et al

Figure 1. (a) Cross section of a chemical synapse, illustrating the discharge of neurotransmitters within a synaptic cleft originating from a
pre-synaptic neuron. (b) Schematic representation of solid-state memristors where ionic species can be displaced within a device’s
insulating medium, transcribing distinct resistive states, by application of electrical stimuli on the top or bottom electrodes of the device.

new nanodevices to emulate the biophysics of real synapses a reduced one (TiO2−x ), that can facilitate distinct resistive
and neurons and reproduce relevant computational primitives, states via controlling the displacement of oxygen vacancies
such as state-dependent conductance changes, multi-level and thus the extent of the two phases. More recently however
stability and stochastic state changes in large-scale artificial it was demonstrated that substantial resistive switching is only
neural systems. viable through the formation and annihilation of continuous
In this paper we first describe how nanoscale synaptic conductive percolation channels [35] that extend across
devices can be integrated into neuro-computing architectures the whole active region of a device, shorting the top and
to build large-scale neural networks, and then propose bottom electrodes; no matter what the underlying physical
a new hybrid memristor-CMOS neuromorphic circuit that mechanism is.
emulates the behavior of real synapses, including their An example of I–V characteristics of TiO2 -based
temporal dynamics aspects, for exploring and understanding memristors is depicted in figure 2(a). In this example,
the principles of neural computation and eventually building consecutive positive voltage sweeps cause any of the cross-bar
brain-inspired computing systems. type devices [36] shown in the inset of figure 2(a) to
switch from a high-resistive state (HRS) to low-resistive
states (LRSs). When the polarity of the voltage sweeps
2. Solid-state memristors
is however inverted, the opposite trend occurs, i.e. the
device toggles from LRS to HRS consecutively (as indicated
ReRAM cells are nowadays classified as being memory
by the corresponding arrows). These measured results are
resistors [29], or memristors for short, that have first been
consistent with analogous ones proposed by other research
conceptually conceived in 1971 by Leon Chua [30]; with the
groups [37–39] and demonstrate the devices’ capacity for
first biomimetic applications presented at the same time. The
storing a multitude of resistive states per unit cell, with the
functional signature of memristors is a pinched hysteresis loop programming depending on the biasing history. This is further
in the current–voltage (I–V) domain when excited by a bipolar demonstrated in figure 2(b), by applying individual pulses
periodic stimulus [31]. Such hysteresis is typically noticed of −3 V in amplitude and 1 µs long for programming a
for all kinds of devices/materials in support of a discharge single memristor at distinct non-volatile resistive states. In
phenomenon that possess certain inertia, causing the value of this scenario, the solid-state memristor emulates the behavior
a physical property to lag behind changes in the mechanism of a depressing synapse [40, 41]; the inverse, i.e. short-term
causing it, and has been common both to large scale [32] as potentiation is also achievable by alternating the polarity of
well as nanoscale dissipative devices [33]. the employed pulsing scheme.
The development of nanoscale dynamic computation
2.1. Emerging nanodevices as synapse mimetics elements may notably benefit the establishment of neuro-
morphic architectures. This technology adds substantially
The analogy of memristors and chemical synapses is made on to computation functionality, due to the rate dependency
the basis that synaptic dynamics depend upon the discharge of of the underlying physical switching mechanisms. At the
neurotransmitters within a synaptic cleft (see figure 1(a)), in a same time it can facilitate unprecedented complexity due
similar fashion that ‘ionic species’ can be displaced within any to the capacity of storing and processing spiking events
inorganic barrier (see figure 1(b)). For TiO2 -based memristor locally. Moreover, exploiting the nanoscale dimensions and
models [33, 34] hypothesized that solid-state devices architecture simplicity of solid-state memristor implementa-
comprise a mixture of TiO2 phases, a stoichiometric and tions could substantially augment the number of cells per

3
Nanotechnology 24 (2013) 384010 G Indiveri et al

Figure 2. Characterization of a TiO2 -based solid-state memristor. (a) I–V characteristics for consecutive voltage sweeping. Positive
(negative) biasing renders an increase (decrease) in the device’s conductance. Inset of (a) depicts a 25 × 25 array cross-bar type memristors
comprising of TiO2 active areas of 1 × 1 µm2 . These cells can be programmed at distinct resistive states as shown in (b) by employing
−3 V and 1 µs wide pulses, while evaluation of the device’s states is performed at 0.9 V.

Figure 3. SEM micrograph of a large nanosized memristor array. Top inset shows a zoom-in of the top left corner where the individual
devices are distinguished. Bottom left inset shows an (AFM) image of a small part of the array. Individual devices are addressed by placing
a conductive (AFM) tip on the top electrode.

unit area, effectively enhancing the systems’ redundancy for reduce both the required set voltage as well as the read voltage
tolerating issues that could stem from device mismatch and used during operation. In this context, thickness figures of
low yields [42]. a few nanometers have been demonstrated and operating
voltages below 1 V have been shown [44] with a switching
2.2. Memristor scaling energy of a few fJ [45]. Furthermore, reducing the active
device area by down-scaling the electrodes leads to current
Resistive memory scaling has been intensively investigated scaling, as well as increased device density. Both of these
for realization of nanosized ReRAM [43]. In principle effects are favorable for high complexity circuits.
memristors may be scaled aggressively well below conven- Currently even though single memristor devices as
tional RAM cells due to their simplicity: fabrication-wise small as 10 nm × 10 nm have been demonstrated [21],
memristors typically rely on a metal insulator metal (MIM) cross-bar arrays are the most commonly used architecture [46,
structure. The memristor action occurs in the insulating 36] to organize large numbers of individually addressable
material. Scaling down the thickness of such a material will memristive synapses in a reduced space. In figure 3 we show

4
Nanotechnology 24 (2013) 384010 G Indiveri et al

Figure 4. Single memristor synapse concept. (a) One Memristor synapse with pre- and post-synaptic pulse-shaping neuron circuits.
(b) Example of a STDP weight update learning function ξ(1T), where 1T represents the difference between the timing of the post-synaptic
and pre-synaptic spikes. (c) Circuit architecture comprising three neuron layers connected by means of synaptic crossbars. (d) Hybrid
memristor/CMOS neurons and AER 2D chip architecture for spike/event routing and processing. Parts of this figure were adapted
from [24]. Reprinted with permission. Copyright 2013 Frontiers.

a large array of nanoscale memristors that we fabricated using cross-bar arrays [48] to connect large numbers of silicon
electron beam lithography. This array consists of a continuous neurons [49], and used in a way to implement spike-based
Pt bottom electrode and an active layer deposited by learning mechanisms that change their local conductance.
Sputtering. Subsequently, several arrays of nano-memristors In [25, 24] the authors proposed a scheme where neurons
with a size ranging from 20 to 50 nm were defined using can drive memristive synapses to implement a spike-timing
E-beam lithography on PMMA and lift-off of the top Platinum dependent plasticity (STDP) [50] learning scheme by
electrode. The array shown here comprises 256 × 256 devices generating single pairs of pre- and post-synaptic spikes in a
with a periodicity of 200 nm. To access each individual device fully asynchronous manner, without any need for global or
a conductive atomic force microscope (AFM) tip was used. local synchronization, thus solving some of the problems that
Such a structure has been used to study the variability of the existed with previously proposed learning schemes [51, 28].
fabricated devices. Using E-beam lithography for both the The main idea is the following: when no spike is generated,
top and bottom electrodes a fully interconnected cross-bar each neuron maintains a constant reference voltage at both
structure with similar size and pitch may be fabricated. its input and output terminals. During spike generation,
each neuron forces a pre-shaped voltage waveform at both
its input and output terminals, as shown in figure 4(a), to
3. Memristor-based neuro-computing architectures update the synaptic weight value stored in the memristor
state. Since memristors change their resistance when the
Memristive devices have been proposed as analogs of voltages at their terminals exceed some defined thresholds, it
biological synapses. Indeed, memristors could implement is possible to obtain arbitrary STDP weight update functions,
very compact but abstract models of synapses, for example including biologically plausible ones, as the one shown in
representing a binary ‘potentiated’ or ‘depressed’ state, or figure 4(b) [50]. Moreover by properly shaping the spike
storing an analog ‘synaptic weight’ value [47]. In this waveforms of both pre- and post-synaptic spikes it is possible
framework, they could be integrated in large and dense to change the form of the STDP learning function, or to

5
Nanotechnology 24 (2013) 384010 G Indiveri et al

even make it evolve in time as learning progresses [52, of Ev ≈ 4 × 1010 eps, which is about 400 eps per neuron just
25]. Fully interconnected or partially interconnected synaptic for inter-chip event exchange. In practice, inter-board traffic
cross-bar arrays, as illustrated in figure 4(c), could facilitate could be much sparser, if the system is partitioned efficiently.
hierarchical learning neural network architectures. Since there Such numbers are quite realistic for present day CMOS
is no need for global synchronization, this approach could technology, and the approach is scalable. Regarding power
be extended to multi-chip architectures that transmit spikes consumption of the communication overhead, we can use
across chip boundaries using fully asynchronous timing. For as reference some recent developments for event-based fully
example, a common asynchronous communication protocol bit-serial inter-chip transmission schemes over differential
that has been used in neuromorphic systems is based on microstrips [57, 56], where consumption is proportional to
the address event representation (AER) [53, 54]. In this communication event rate. Each link would consume in
representation, each spiking neuron is assigned an address, the order of 40 mA at 10 Meps rate (this includes driver
and when the neuron fires an address event is put on a and receiver pad circuits [57] as well as serializers and
digital bus, at the time that the spike is emitted. In this deserializers [58]). If each neuron fires at an average rate of
way time represents itself, and information is encoded in 1 Hz, and if each chip has 1 million neurons, the current
real time, in the inter-spike intervals. By further exploiting consumption of the communication overhead would be about
hybrid CMOS/memristor chip fabrication techniques [55], 4 mA per chip. If voltage supply is in the 1–2 V range,
this approach could be easily scaled up to arbitrarily large this translates into 4–8 mW per chip. For a 100 chip PCB
networks (e.g., see figure 4(d)). Following this approach the inter-chip communication overhead power consumption
each neuron processor would be placed in a 2D-grid would thus be about 400–800 mW, for 1 Hz average neuron
fully, or partially interconnected through memristors. Each firing rate.
neuron would perform incoming spike aggregation, provide
the desired pre- and post-synaptic (programmable) spike 4. Neuromorphic and hybrid memristor-CMOS
waveforms, and communicate incoming and outgoing spikes synapse circuits
through AER communication circuitry. Using state-of-the-art
CMOS technology, it is quite realistic to provide in the We’ve shown how memristive devices and nano-technologies
order of a million such neurons per chip with about 104 can be exploited to dramatically increase integration density
synapses per neuron. For example, by using present day and implement large-scale abstract neural networks. However
40 nm CMOS technology it is quite realistic to fit a neuron to faithfully reproduce the function of real synapses, including
within a 10 µm × 10 µm area. This way, a chip of about their temporal dynamic properties, passive memristive devices
1 cm2 could host of the order of one million neurons. At would need to be interfaced to biophysically realistic
the same time, for the nanowire fabric deposited on top of CMOS circuits that follow the neuromorphic approach, as
CMOS structures, present day technology can easily provide described in [10, 11]. On one hand, building physical
nanowires of 100 nm pitch [21]. This would allow to integrate implementations of circuits and materials that directly
about 104 synapses on top of the area occupied by each CMOS emulate the biophysics of real synapses and reproduce
neuron. Similarly, at the PCB level, it is possible to envisage their detailed real-time dynamics are important for basic
that a 100-chip PCB could host about 108 neurons, and 40 research in neuroscience, on the other, this neuromorphic
of these PCBs would emulate 4 billion neurons. In these approach can pave the way for creating an alternative non-von
large-scale systems the bottleneck is largely given by the spike Neumann computing technology, based on massively parallel
or event communication limits. To cope with these limits such arrays of slow, unreliable, and highly variable, but also
chips would inter-communicate through nearest neighbors, compact and extremely low-power solid-state components
exploiting 2D-grid network-on-chip (NoC) and network-on- for building neuromorphic systems that can process sensory
board (NoB) principles. For example, in [56] the authors signals and interact with the user and the environment in
proposed a very efficient multi-chip inter-communication real time, and possibly carry out computation using the
scheme that distributes event traffic over a 2D mesh network same principles used by the brain. Within this context,
locally within each board through inter-chip high speed of massively parallel artificial neural processing elements,
buses. Reconfigurability and flexibility would be ensured by memory and computation are co-localized. Typically the
defining the system architecture and topology through in-chip amount of memory available per each ‘computing node’
routing tables. Additionally, by arranging the neurons within (synapse in our case) is limited and it is not possible to
each chip in a local 2D mesh with in-chip inter-layer event transfer and store partial results of a computation in large
communication, it is possible to keep most of the event traffic memory banks outside the processing array. Therefore, in
inside the chips. At the board level, the 2D mesh scheme order to efficiently process real-world biologically relevant
would allow for a total inter-chip traffic in the order of Ev = sensory signals these types of neuromorphic systems must
4Nch × Epp , where Nch = 100 is the number of chips per use circuits that have biologically plausible time constants
board, Epp is the maximum event bandwidth per inter-chip (i.e., of the order of tens of milliseconds). In this way, in
bus (which we may assume to be around 100 Meps—mega addition to being well matched to the signals they process,
events per second), and 4 reflects the fact that each chip these systems will also be inherently synchronized with the
is connected to its four nearest neighbors [56]. With these real-world events they process and will be able to interact
numbers, the maximum traffic per board would be in the order with the environment they operate in. But these types of

6
Nanotechnology 24 (2013) 384010 G Indiveri et al

Figure 5. Neuromorphic electronic synapses. (a) Log-domain DPI circuit diagram of an excitatory silicon synapse. Red arrows show the
translinear loop considered to derive the circuit response. Input voltage spikes Vin are integrated by the circuit to produce post-synaptic
currents Isyn with biologically faithful dynamics. (b) Experimental data showing the EPSP response of the circuit for two different settings
of synaptic weight bias voltage Vw . The data was measured from the DPI synapses of 124 neurons, integrated on the same chip, with shared
common bias settings. The dashed and solid lines represent the average response, while the shaded areas (standard deviation) indicate the
extend of the device mismatch effect.

time constants require very large capacitance and resistance the thermal voltage. The currents Iw and Ith represent
values. For example, in order to obtain an equivalent RC local synaptic weight and a global synaptic scaling gain
time constant of 10 ms with a resistor even as large as terms, useful for implementing spike-based and homeostatic
10 M, it would be necessary to use a capacitor of 100 pF. plasticity mechanisms [65, 66]. Therefore, by setting for
In standard CMOS VLSI technology a synapse circuit with example, Iτ = 5 pA, and assuming that UT = 25 mV at room
this RC element would require a prohibitively large area, and temperature, the capacitance required to implement a time
the advantages of large-scale integration would vanish. One constant of 10 ms would be approximately C = 1 pF. This can
elegant solution to this problem is to use current-mode design be implemented in a compact layout and allows the integration
techniques [59] and log-domain subthreshold circuits [60, of large numbers of silicon synapses with realistic dynamics
61]. When meal oxide semiconductor field effect transistors on a small VLSI chip. The same circuit of figure 5(a) can be
(MOSFETs) are operated in the subthreshold domain, the used to implement elaborate models of spiking neurons, such
main mechanism of carrier transport is that of diffusion [60], as the ‘Adaptive Exponential’ (AdExp) I&F model [67, 49].
the same physical process that governs the flow of ions Small (minimum size, of about 10 µm2 ) prototype VLSI chips
through proteic channels across neuron membranes. As a comprising of the order of thousands of neurons and synapses
consequence, MOSFETs have an exponential relationship based on the DPI circuit have been already fabricated using
between gate-to-source voltage and drain current, and produce a conservative 350 nm CMOS technology [68]. The data of
currents that range from femto- to nano-Ampères. In this figure 5(b) shows the average response of a DPI synapse
domain it is possible to implement active VLSI analog filter circuits measured from one of such chips [68]. The data
circuits that have biologically realistic time constants and that represents the average excitatory post synaptic potential
employ relatively small capacitors. (EPSP) produced by 124 neurons in response to a single spike
sent to the DPI synapses of each neuron. The shaded areas,
4.1. A CMOS neuromorphic synapse representing the standard deviation, highlight the extent of
variability present in these types of networks, due to device
An example of a compact circuit that can produce both linear mismatch. The main role of the DPI circuit of figure 5(a)
dynamics with biologically plausible time constants as well is to implement synaptic dynamics. Short-term plasticity,
as non-linear short-term plasticity effects analogous to those STDP learning, and homeostatic adaptation mechanisms can
observed in real neurons and synapses is the differential pair be, and have been, implemented by interfacing additional
integrator (DPI) circuit [62] shown in figure 5(a). It can be CMOS circuits to control the DPI Vw bias voltage, or to the
shown [63] that by exploiting the translinear principle [64] Ith bias current [62, 69, 70]. Long-term storage of the Vw
across the loop of gate-to-source voltages highlighted in the weights however requires additional power-consuming and
figure, the circuit produces an output current Isyn with impulse area-expensive circuit solutions, such as floating gate circuits,
response of the form: or local analog to digital converter (ADC) and SRAM cells.

d Iw Ith
τ Isyn + Isyn = , (1) 4.2. A new hybrid memristor-CMOS neuromorphic synapse
dt Iτ
where τ , CUT /κIτ is the circuit time constant, κ the Nanoelectronic technologies offer a promising alternative
subthreshold slope factor [60], and UT = KT/q represents solution for compact and low-power long-term storage of

7
Nanotechnology 24 (2013) 384010 G Indiveri et al

Figure 6. Neuromorphic memristive synapse. (a) Schematic circuit implementing an array of memristive synapses, with independent inputs
and synaptic weights, but with shared temporal dynamics. (b) SPICE simulations of the circuit in figure 6(a) showing the output Isyn EPSC
in response to a pre-synaptic input spike, for four different memristor conductance values.

synaptic weights. The hybrid memristor-CMOS neuromor- Figure 6(b) shows the results of SPICE simulations of
phic synapse circuit we propose here, shown in figure 6(a), the circuit in figure 6(a), for a 180 nm CMOS process. The
exploits these features to obtain at the same time dense Ithr and Iτ current sources were implemented with p-type
integration of low-power long-term synaptic weight storage MOSFETs, biased to produce 2 pA and 10 pA respectively,
elements, and to emulate detailed synaptic biophysics for and the Vw voltage bias was set to 700 mV. The data was
implementing relevant computational properties of neural obtained by simulating the response of one input memristive
systems. branch to a single input spike, while sweeping the memristor
The circuit depicted in figure 6(a) represents a possible impedance from 1 to 7 K. In these simulations we set
implementation of a dense array of N synapses with the memristor in its LRS, and assumed we could modulate
independent weights but with the same, shared, temporal the value of the resistance to obtain four distinct analog
dynamics. Depending on their size, each memristor in states analogous to the ones measured experimentally in
figure 6(a) could represent a full synaptic contact, or figure 2(b). Of course the circuit supports also the operation
an individual ion channel in the synaptic cleft (see also of the memristor as a binary device, working in either the
figure 1(a)). If the currently accepted model of filament HRS state or the LRS one. This bi-stable mode of using
formation in memristive devices is true, then down- the memristor would encode only an ‘on’ or ‘off’ synaptic
scaled memristors should approach single filament bi-stable state, but it would be more reliable and it is compatible with
operation. While this is a severe limitation for classical neural biologically plausible learning mechanisms, such as those
network applications in which memristors are required to proposed in [71], and implemented in [69]. The circuit of
store analog synaptic weight values with some precision, it figure 6(a) shows only the circuit elements required for a
would actually provide a very compact physical medium for ‘read’ operation, i.e., an operation that stimulates the synapse
emulating the stochastic nature of the opening and closing of to generate an EPSC with an amplitude set by the conductance
ion channels in biological synapses. of the memristor. Additional circuit elements would be
The shared temporal dynamics are implemented by the required to change the value of the memristor’s conductance,
DPI circuit in the top part of figure 6(a). Indeed, if this e.g., via learning protocols. However the complex circuitry
circuit is operated in its linear regime, it is possible to controlling the learning mechanisms would be implemented
time multiplex the contributions from all spiking inputs, at the Input/Output (I/O) periphery of the synaptic array,
thus requiring one single integrating element and saving for example with pulse-shaping circuits and architectures
precious silicon real estate. The Vw bias voltage of this analogous to the ones described in section 3, or with circuits
circuit is a global parameter that sets the maximum possible that check the state of the neuron and of it’s recent spiking
current that can be produced by each memristor upon the history, such as those proposed in [61], and only a few
arrival of an input spike, while the memristor conductance additional compact elements would be required in each
modulates the current being produced by the synapse very synapse to implement the weight update mechanisms.
much like conductance changes in real synapses affect the
excitatory post synaptic currents (EPSCs) they produce. 5. Brain-inspired probabilistic computation
Larger memristor conductances, which represent a larger
number of open proteic channels in real synapses, correspond While memristors offer a compact and attractive solution for
to larger synaptic weights. long-term storage of synaptic state, as done for example in

8
Nanotechnology 24 (2013) 384010 G Indiveri et al

figure 6, they are affected by a high degree of variability P(x̄) describes the probabilistic relationships between the
(e.g., much higher than the one measured for CMOS synapses random variables x1 , . . . , xn , and if x1 , . . . , xk of this
in figure 5(b)). In addition, as memristors are scaled down, distribution are observed, then one can infer a set of
unreliable and stochastic behavior becomes unavoidable. The variables of interests xk+1 , . . . , xk+l by determining the
variability, stochasticity, and general reliability issues that posterior probability P(xk+1 , . . . , xk+l |x1 , . . . , xk ). One of the
are starting to represent serious limiting factors for advanced most popular techniques used to perform inference is belief
computing technologies, do not seem to affect biological com- propagation [77]. While this message passing algorithm
puting systems. Indeed, the brain is a highly stochastic system can be implemented by networks of spiking neurons [78],
that operates using noisy and unreliable nanoscale elements. a more promising alternative approach, also well suited
Rather than attempting to minimize the effect of variability in to model brain-inspired computation, is to use sampling
nano-technologies, one alternative strategy, compatible with techniques [79]. Probably the most important family of
the neuromorphic approach, is to embrace variability and sampling techniques in this context is Markov-Chain Monte
stochasticity and exploit these ‘features’ to carry out robust Carlo (MCMC) sampling. Since MCMC sampling techniques
brain-inspired probabilistic computation [99, 100]. operate in a stochastic manner, stochastic computational
The fact that the brain can efficiently cope with a elements are a crucial and essential feature. Recent studies
high degree of variability is evident at many levels: at have shown that probabilistic inference through MCMC
the macroscopic level trial-to-trial variability is present for sampling can be implemented by networks of stochastically
example in the arm trajectories of reaching movement tasks. spiking neurons [79, 80]. Therefore, MCMC sampling is
It is interesting to note that the variability of the end a computational paradigm optimally suited for emulating
position of the reaching movement is reduced, if the task probabilistic inference in the brain using neuromorphic
requires to hit or touch a target with high accuracy [72]. circuits and nanoelectronic synapses.
Variability is evident at the level of cortical neurons: there Within this context, it is important to see if and how
is significant trial-to-trial variability in their responses to the distribution P(x̄) can be learned from observations,
identical stimuli; it is evident also at the level of chemical i.e., how the artificial neural system can build its own
synapses, where there is a high degree of stochasticity in model of the world based on its sensory input and then
the transmission of neurotransmitter molecules [73], from the perform probabilistic inference on this model. For a relatively
pre-synaptic terminal to the post-synaptic one. The release simple model [81], it has been shown that this can be
probability of cortical synapses ranges from values of less accomplished by a local spike-driven learning rule that
than 1%–100% [74]. This indicates that stochastic synaptic resembles the STDP mechanisms measured in cortical
release may not merely be an unpleasant constraint of networks [50]. Analogous learning mechanisms have been
the molecular machinery but may rather be an important demonstrated both experimentally in neuromorphic CMOS
computational feature of cortical synapses. devices [69], and theoretically, with circuit models of
What could be the computational benefit of using memristive synapses [25].
hardware affected by variability and stochasticity in biological With regard to learning, the variability and stochasticity
and artificial computing systems? Recent advances in ‘features’ described above can provide an additional benefit:
cognitive science demonstrated that human behavior can be for many learning tasks, humans and animals have to
described much better in the framework of probabilistic explore many different actions in order to be able to learn
inference rather than in the framework of traditional ‘hard’ appropriate responses in a given situation. In these so-called
logic inference [75], and encouraged the view that neuronal reinforcement learning setups, noise and variability naturally
networks might directly implement a process of probabilistic provide the required exploration mechanisms. A number of
inference [76]. In parallel, to this paradigm shift, research in recent studies have shown how stochastic neuronal behavior
machine learning has revealed that probabilistic inference is could be utilized by cortical circuits in order to learn
often much more appropriate for solving real-world problems, complex tasks [82–84]. For example, reservoir computing
then hard logic [77]. The reason for this is that reasoning can (RC, also known under the terms Liquid State Machines and
seldom be based on full and exact knowledge in real-world Echo State Networks) is a powerful general principle for
situations. For example, the sensory data that a robot receives computation and learning with complex dynamical systems
is often noisy and incomplete such that the current state of such as recurrent networks of analog and spiking neurons [85,
the environment can only partially be described. Probabilistic 86] or optoelectronic devices [87]. The main idea behind
reasoning is a powerful tool to deal with such uncertain RC is to use a heterogeneous dynamical system (called the
situations. Of course, exact probabilistic inference is still reservoir) as a non-linear fading memory where information
computationally intractable in general, but a number of about previous inputs can be extracted from the current
approximation schemes have been developed that work well state of the system. This reservoir can be quite arbitrary
in practice. in terms of implementation and parameter setting as long
In probabilistic inference, the idea is to infer a set as it operates in a suitable dynamic regime [88]. Readout
of unobserved variables (e.g., motor outputs, classification elements are trained to extract task-relevant information from
results, etc) given a set of observed variables (evidence, the reservoir. In this way, arbitrary fading memory filters
e.g., sensory inputs), using known or learned probabilistic or even arbitrary dynamical systems (in the case when the
relationships among them. Specifically, if the distribution readout elements provide feedback to the dynamical system)

9
Nanotechnology 24 (2013) 384010 G Indiveri et al

can be learned. One long-standing disadvantage of traditional change the state of ReRAMs cells that depends on the position
RC was that readouts had to be trained in a supervised of the cell in the array. This problem is especially critical
manner. In other words, a teacher signal was necessary that in neuro-computing architectures where these cells represent
signals at each time point the desired output of readouts. In synapses, as the offsets directly affect the weight update and
many real-world applications, such a teacher signal is not learning mechanisms.
available. For example, if the task for a robot controller is to Integrating memristors as synapse elements in large-scale
produce some motor trajectory in order to produce a desired neuro-computing architectures also introduces the signifi-
hand movement, the exact motor commands that perform this cance of process variability in memristor dimensions [98],
movement are in general not known. What can be evaluated which in turn introduces a significant amount of variability
however is the quality of the movement. Recently, it has been in the characteristics of the synapse properties. In addition
demonstrated that noisy readouts can be trained with a much to their large variability, another important issue relating
less informative reward signal, which just indicates whether to these types of synapses, that is still ignored in the vast
some measure of performance of the system has recently majority of neuro-computing studies, is the effect of limited
increased [84]. Of course, such reward-based learning can in resolution in memristive states. In particular, it is not known
general be much slower than the pure supervised approach what the trade-off between desired synaptic weight resolution
(see, e.g., [89]). The actual slowdown however depends on and memristor size is. And it is not known to what extent the
the task at hand, and it is interesting that for a set of relevant multi-step synaptic weight model holds true for aggressively
tasks, reward-based learning works surprisingly fast [84]. down-scaled memristor sizes.
Since the functionality of reservoirs depends on its gen-
These scaling, integration, and variability issues are
eral dynamical behavior and not on precise implementation of
serious limiting factors for the use of memristors in
its components, RC is an attractive computational paradigm
conventional neuro-computing architectures. Nonetheless,
for circuits comprised of nanoscale elements affected by
biological neural systems are an existence proof that it is
variability, such as the one proposed in section 4.2. In fact,
if the reservoir is composed by a large number of simple possible to implement robust computation using nanoscale
interacting dynamic elements—the typical scenario—then unreliable components and non-von Neumann computing
heterogeneity of these elements is an essential requirement for architectures. In order to best exploit these emerging
ideal performance. Parameter heterogeneity is also beneficial nanoscale technologies for building compact, low-power, and
in so-called ensemble learning techniques [90]. It is well robust artificial neural processing systems it is important to
known that the combination of models with heterogeneous understand the (probabilistic) neural and cortical principles
predictions for the same data-set tends to improve overall of computation and to develop at the same time, following
prediction performance [91]. Hence, heterogeneity of compu- a co-design approach, the neuromorphic hardware computing
tational elements can be a real benefit for learning. Examples substrates that support them. In this paper we elaborated
for ensemble methods are random forests [92], bagging [93], on this neuromorphic approach, presenting an example of
and boosting [94]. a neuromorphic circuit and of a hybrid nanoelectronic-
CMOS architecture that directly emulate the properties of
real synapses to reproduce biophysically realistic response
6. Discussion and conclusions
properties, thus providing the necessary technology for
Memristors, and in particular nanoscale solid-state im- implementing massively parallel models of brain-inspired
plementations, represent a promising technology, baring computation that are, by design, probabilistic, robust to
benefits for emerging memory storage as well as revisiting variability, and fault tolerant.
conventional analog circuits [95]. Given their low-power
and small-scale characteristics, researchers are considering Acknowledgment
their application also in large-scale neural networks for
neuro-computing applications. However, the fabrication of This work was supported by the European CHIST-
large-scale nanoscale cross-bar arrays involves several issues ERA program, via the ‘Plasticity in NEUral Memristive
that are still open: the realization of nanosized electrodes re- Architectures’ (PNEUMA) project.
quires nanopatterning [96] techniques, such as electron beam
lithography (EBL) or nano-imprint lithography (NIL) [97].
References
This directly correlates to reduced electrode cross section
which results in increasing resistance. As electrode resistance
[1] McCulloch W S and Pitts W 1943 A logical calculus of the
scales with length, this can rapidly become a critical ideas immanent in nervous activity Bull. Math. Biophys.
issue for fully interconnected nanoscale cross-bar structures. 5 115–33
Furthermore, down-scaling the electrode size to reduce the [2] von Neumann J 1958 The Computer and the Brain
device active area requires simultaneous down-scaling of the (New Haven, CT: Yale University Press)
thickness of the metallizations due to fabrication concerns. [3] Rosenblatt F 1958 The perceptron: a probabilistic model for
information storage and organization in the brain Psychol.
This in turn further increases the resistance of the electrodes, Rev. 65 386–408
much like the interconnects in modern CMOS circuitry. These [4] Minsky M L 1967 Computation: Finite and Infinite Machines
factors introduce a large offset in the write voltages required to (Upper Saddle River, NJ: Prentice-Hall)

10
Nanotechnology 24 (2013) 384010 G Indiveri et al

[5] Hopfield J J 1982 Neural networks and physical systems with [26] Jo S H, Chang T, Ebong I, Bhadviya B B, Mazumder P and
emergent collective computational abilities Proc. Natl Lu W 2010 Nanoscale memristor device as synapse in
Acad. Sci. 79 2554–8 neuromorphic systems Nano Lett. 10 1297–301
[6] Rumelhart D E and McClelland J L 1986 Parallel [27] Choi H, Jung H, Lee J, Yoon J, Park J, Seong D-J, Lee W,
Distributed Processing: Explorations in the Hasan M, Jung G-Y and Hwang H 2009 An electrically
Microstructure of Cognition. Volume 1. Foundations modifiable synapse array of resistive switching memory
(Cambridge, MA: MIT Press) Nanotechnology 20 345201
[7] Kohonen T 1988 Self-Organization and Associative Memory [28] Kuzum D, Jeyasingh R G D, Lee B and Wong H S P 2012
(Springer Series in Information Sciences) 2nd edn (Berlin: Nanoelectronic programmable synapses based on phase
Springer) change materials for brain-inspired computing Nano Lett.
[8] Hertz J, Krogh A and Palmer R G 1991 Introduction to the 12 2179–86
Theory of Neural Computation (Reading, MA: [29] Chua L 2011 Resistance switching memories are memristors
Addison-Wesley) Appl. Phys. A 102 765–83
[9] Bishop C M 2006 Pattern Recognition and Machine [30] Chua L 1971 Memristor—the missing circuit element IEEE
Learning (New York: Springer) Trans. Circuit Theory 18 507–19
[10] Indiveri G and Horiuchi T K 2011 Frontiers in neuromorphic [31] Di Ventra M, Pershin Y V and Chua L O 2009 Circuit
engineering Front. Neurosci. 5 118 elements with memory: memristors, memcapacitors, and
[11] Mead C 1990 Neuromorphic electronic systems Proc. IEEE meminductors Proc. IEEE 97 1717–24
78 1629–36 [32] Prodromakis T, Toumazou C and Chua L 2012 Two centuries
[12] Markram H 2006 The blue brain project ACM/IEEE Conf. on of memristors Nature Mater. 11 478–81
Supercomputing, SC (New York: IEEE, ACM) p 53 [33] Strukov D B, Snider G S, Stewart D R and
[13] Schemmel J, Fieres J and Meier K 2008 Wafer-scale Williams R S 2009 The missing memristor found Nature
integration of analog neural networks IJCNN 2008: IEEE 459 1154–4
International Joint Conf. on Neural Networks, 2008 (IEEE [34] Prodromakis T, Peh B P, Papavassiliou C and Toumazou C
World Congr. on Computational Intelligence) pp 431–8 2011 A versatile memristor model with nonlinear dopant
[14] Jin X, Lujan M, Plana L A, Davies S, Temple S and Furber S kinetics IEEE Trans. Electron Devices 58 3099–105
2010 Modeling spiking neural networks on SpiNNaker [35] Shihong M W, Prodromakis T, Salaoru I and Toumazou C
Comput. Sci. Eng. 12 91–7 2012 Modelling of current percolation channels in
[15] Arthur J V et al 2012 Building block of a programmable emerging resistive switching elements
neuromorphic substrate: a digital neurosynaptic core arXiv:2746v1 [cond-mat.mes-hall]
IJCNN 2012: Int. Joint Conf. on Neural Networks [36] Kim K-H, Gaba S, Wheeler D, Cruz-Albrecht J M,
(Piscataway, NJ: IEEE) pp 1946–53 Hussain T, Srinivasa N and Lu W 2011 A functional
[16] Wong T M et al 2012 1014 . IBM Research Report RJ10502 hybrid memristor crossbar-array/CMOS system for data
(ALM1211-004) IBM Research, San Jose, CA storage and neuromorphic applications Nano Lett.
[17] Silver R, Boahen K, Grillner S, Kopell N and Olsen K L 2007 12 389–95
Neurotech for neuroscience: unifying concepts, organizing [37] Liu M, Abid Z, Wang W, He X, Liu Q and Guan W 2009
principles, and emerging tools J. Neurosci. 27 11807 Multilevel resistive switching with ionic and metallic
[18] Choudhary S, Sloan S, Fok S, Neckar A, Trautmann E, filaments Appl. Phys. Lett. 94 233106
Gao P, Stewart T, Eliasmith C and Boahen K 2012 Silicon [38] Yoon K J, Lee M H, Kim G H, Song S J, Seok J Y, Han S,
neurons that compute Artificial Neural Networks and Yoon J H, Kim K M and Hwang C S 2012 Memristive
Machine Learning—ICANN 2012 (Lecture Notes in tri-stable resistive switching at ruptured conducting
Computer Science vol 7552) ed A Villa, W Duch, P Érdi, filaments of a Pt/TiO2 /Pt cell Nanotechnology 23 185202
F Masulli and G Palm (Berlin: Springer) pp 121–8 [39] Yang X and Chen I-W 2012 Dynamic-load-enabled ultra-low
[19] Di Ventra M and Pershin Y V 2011 Memory materials: a power multiple-state rram devices Sci. Rep. 2 744
unifying description Mater. Today 14 584–91 [40] O’Donovan M J and Rinzel J 1997 Synaptic depression: a
[20] Rozenberg M, Inoue I and Sánchez M 2004 Nonvolatile dynamic regulator of synaptic communication with varied
memory with multilevel switching: a basic model Phys. functional roles Trends Neurosci. 20 431–3
Rev. Lett. 92 178302 [41] Chance F S, Nelson S B and Abbott L F 1998 Synaptic
[21] Govoreanu B et al 2011 10 × 10 nm2 Hf/HfOx crossbar depression and the temporal response characteristics of V1
resistive RAM with excellent performance, reliability and cells J. Neurosci. 18 4785–99
low-energy operation Int. Technical Digest on Electron [42] Gelencser A, Prodromakis T, Toumazou C and Roska T 2012
Devices Meeting pp 31–4 Biomimetic model of the outer plexiform layer by
[22] Snider G S and Williams R S 2007 Nano/CMOS incorporating memristive devices Phys. Rev. E 85 041918
architectures using a field-programmable nanowire [43] Yang L M, Song Y L, Liu Y, Wang Y L, Tian X P, Wang M,
interconnect Nanotechnology 18 035204 Lin Y Y, Huang R, Zou Q T and Wu J G 2012 Linear
[23] Hasegawa T, Ohno T, Terabe K, Tsuruoka T, Nakayama T, scaling of reset current down to 22-nm node for a novel
Gimzewski J K and Aono M 2010 Learning abilities RRAM IEEE Electron Device Lett. 33 89–91
achieved by a single solid-state atomic switch Adv. Mater. [44] Deleruyelle D, Putero M, Ouled-Khachroum T, Bocquet M,
22 1831–4 Coulet M-V, Boddaert X, Calmes C and Muller C 2013
[24] Serrano-Gotarredona T, Masquelier T, Prodromakis T, Ge2 Sb2 Te5 layer used as solid electrolyte in conductive-
Indiveri G and Linares-Barranco B 2013 STDP and STDP bridge memory devices fabricated on flexible substrate
variations with memristors for spiking neuromorphic Solid-State Electron. 79 159–65
learning systems Front. Neurosci. 7 2 [45] Chin A, Chiu Y C, Cheng C H, Zheng Z W and Liu M 2012
[25] Zamarreño-Ramos C, Camuñas-Mesa L A, Ultra-low switching power RRAM using hopping
Pérez-Carrasco J A, Masquelier T, conduction mechanism Meeting Abstracts
Serrano-Gotarredona T and Linares-Barranco B 2011 MA2012-02 pp 2574–4
On spike-timing-dependent-plasticity, memristive devices, [46] Lewis D L and Lee H-H S 2009 Architectural evaluation of
and building a self-learning visual cortex Front. Neurosci. 3D stacked RRAM caches 3DIC 2009: IEEE Int. Conf. on
5 26 3D System Integration (Sept. 2009) pp 1–4

11
Nanotechnology 24 (2013) 384010 G Indiveri et al

[47] Kudithipudi D and Merkel C E 2012 Reconfigurable biomedical signal processing BIOCAS 2006: Biomedical
memristor fabrics for heterogeneous computing Advances Circuits and Systems Conf. (Piscataway, NJ: IEEE)
in Neuromorphic Memristor Science and Applications pp 130–3
(Springer Series in Cognitive and Neural Systems vol 4) [64] Gilbert B 1996 Translinear circuits: an historical review
ed R Kozma, R E Pino and G E Pazienza (Netherlands: Analog Integr. Circuits Signal Process. 9 95–118
Springer) pp 89–106 [65] Abbott L F and Nelson S B 2000 Synaptic plasticity: taming
[48] Likharev K K 2011 CrossNets: neuromorphic hybrid the beast Nature Neurosci. 3 1178–83
CMOS/Nanoelectronic networks Sci. Adv. Mater. [66] Turrigiano G G and Nelson S N 2004 Homeostatic plasticity
3 322–31 in the developing nervous system Nature Rev. Neurosci.
[49] Indiveri G et al 2011 Neuromorphic silicon neuron circuits 5 97–107
Front. Neurosci. 5 1–23 [67] Brette R and Gerstner W 2005 Adaptive exponential
[50] Bi G-Q and Poo M-M 1998 Synaptic modifications in integrate-and-fire model as an effective description of
cultured hippocampal neurons: dependence on spike neuronal activity J. Neurophysiol. 94 3637–42
timing, synaptic strength, and postsynaptic cell type J. [68] Indiveri G and Chicca E 2011 A VLSI neuromorphic device
Neurosci. 18 10464–72 for implementing spike-based neural networks Neural
[51] Snider G S 2008 Spike-timing-dependent learning in Nets WIRN11—Proc. 21st Italian Workshop on Neural
memristive nanodevices NANOARCH 2008: IEEE Int. Nets pp 305–16
Symp. on Nanoscale Architectures, 2008 (Piscataway, NJ: [69] Mitra S, Fusi S and Indiveri G 2009 Real-time classification
IEEE) pp 85–92 of complex patterns using spike-based learning in
[52] Linares-Barranco B and Serrano-Gotarredona T 2009 neuromorphic VLSI IEEE Trans. Biomed. Circuits Syst.
Exploiting memristance in adaptive asynchronous spiking 3 32–42
neuromorphic nanotechnology systems IEEE-NANO [70] Bartolozzi C and Indiveri G 2009 Global scaling of synaptic
2009: 9th IEEE Conf. on Nanotechnology, 2009 efficacy: homeostasis in silicon synapses Neurocomputing
(Piscataway, NJ: IEEE) pp 601–4 72 726–31
[53] Deiss S R, Douglas R J and Whatley A M 1998 Pulsed [71] Brader J, Senn W and Fusi S 2007 Learning real world
Neural Networks ed W Maass and C stimuli in a neural network with spike-driven synaptic
M Bishop (Cambridge, MA: MIT Press) pp 157–78 dynamics Neural Comput. 19 2881–912
chapter 6 (A Pulse-Coded Communications Infrastructure [72] Todorov E and Jordan M 2002 Optimal feedback control as a
For Neuromorphic Systems) theory of motor coordination Nature Neurosci. 5 1226–35
[54] Chicca E, Whatley A M, Lichtsteiner P, Dante V, [73] Faisal A A, Selen L P J and Wolpert D M 2008 Noise in the
Delbruck T, Del Giudice P, Douglas R J and
nervous system Nature Rev. Neurosci. 9 292–303
Indiveri G 2007 A multi-chip pulse-based neuromorphic
[74] Volgushev M, Kudryashov I, Chistiakova M, Mukovski M,
infrastructure and its application to a model of orientation
Niesmann J and Eysel U T 2004 Probability of transmitter
selectivity IEEE Trans. Circuits Syst. I 5 981–93
release at neocortical synapses at different temperatures
[55] Likharev K, Mayr A, Muckra I and Türel Ö 2003 Crossnets:
J. Neurophysiol. 92 212–20
high-performance neuromorphic architectures for cmol
[75] Griffiths T L and Tenenbaum J B 2006 Optimal predictions
circuits Ann. New York Acad. Sci. 1006 146–63
in everyday cognition Psychol. Sci. 17 767–73
(Molecular Electronics III)
[56] Zamarreño-Ramos C, Linares-Barranco A, [76] Tenenbaum J B, Kemp C, Griffiths T L and Goodman N D
Serrano-Gotarredona T and Linares-Barranco B 2012 2011 How to grow a mind: statistics, structure, and
Multi-casting mesh AER: a scalable assembly approach abstraction Science 331 1279–85
for reconfigurable neuromorphic structured AER systems. [77] Pearl J 1988 Probabilistic Reasoning in Intelligent Systems:
application to ConvNets IEEE Trans. Biomed. Circuits Networks of Plausible Inference (San Mateo, CA: Morgan
Syst. PP(99) 82–102 Kaufmann Publishers)
[57] Zamarreño-Ramos C, Serrano-Gotarredona T and [78] Steimer A, Maass W and Douglas R 2009 Belief propagation
Linares-Barranco B 2012 A 0.35 µm sub-ns wake-up time in networks of spiking neurons Neural Comput.
ON–OFF switchable LVDS driver-receiver chip I/O pad 21 2502–23
pair for rate-dependent power saving in AER bit-serial [79] Büsing L, Bill J, Nessler B and Maass W 2011 Neural
links IEEE Trans. Biomed. Circuits Syst. 6 486–97 dynamics as sampling: a model for stochastic computation
[58] Zamarreño-Ramos C, Serrano-Gotarredona T and in recurrent networks of spiking neurons PLoS Comput.
Linares-Barranco B 2011 An instant-startup jitter-tolerant Biol. 7 e1002211
manchester-encoding serializer/deserializar scheme for [80] Pecevski D, Büsing L and Maass W 2011 Probabilistic
event-driven bit-serial LVDS inter-chip AER links IEEE inference in general graphical models through sampling in
Trans. Circuits Syst. I 58 2647–60 stochastic networks of spiking neurons PLoS Comput.
[59] Tomazou C, Lidgey F J and Haigh D G (ed) 1990 Analogue Biol. 7 e1002294
IC Design: the Current–Mode Approach (Stevenage, [81] Nessler B, Pfeiffer M and Maass W 2010 STDP enables
Herts.: Peregrinus) spiking neurons to detect hidden causes of their inputs
[60] Liu S-C, Kramer J, Indiveri G, Delbruck T and Douglas R J Proc. of NIPS 2009: Advances in Neural Information
2002 Analog VLSI: Circuits and Principles (Cambridge, Processing Systems vol 22 (Cambridge, MA: MIT Press)
MA: MIT Press) pp 1357–65
[61] Mitra S, Indiveri G and Etienne-Cummings R 2010 Synthesis [82] Legenstein R, Pecevski D and Maass W 2008 A learning
of log-domain integrators for silicon synapses with global theory for reward-modulated spike-timing-dependent
parametric control ISCAS 2010: Int. Symp. on Circuits and plasticity with application to biofeedback PLoS Comput.
Systems (Piscataway, NJ: IEEE) pp 97–100 Biol. 4 e1000180
[62] Bartolozzi C and Indiveri G 2007 Synaptic dynamics in [83] Legenstein R, Chase S M, Schwartz A B and Maass W 2010
analog VLSI Neural Comput. 19 2581–603 A reward-modulated Hebbian learning rule can explain
[63] Bartolozzi C, Mitra S and Indiveri G 2006 An ultra low experimentally observed network reorganization in a brain
power current–mode filter for neuromorphic systems and control task J. Neurosci. 30 8400–10

12
Nanotechnology 24 (2013) 384010 G Indiveri et al

[84] Hoerzer G M, Legenstein R and Maass W 2012 Emergence [94] Schapire R E 1990 The strength of weak learnability Mach.
of complex computational structures from chaotic neural Learn. 5 197–227
networks through reward-modulated Hebbian learning [95] Berdan R, Prodromakis T, Salaoru I, Khiat A and
Cerebral Cortex doi:10.1093/cercor/bhs348 Toumazou C 2012 Memristive devices as parameter
[85] Jaeger H and Haas H 2004 Harnessing nonlinearity: setting elements in programmable gain amplifiers Appl.
predicting chaotic systems and saving energy in wireless Phys. Lett. 101 243502
telecommunication Science 304 78–80 [96] Xia Q F, Yang J J, Wu W, Li X and Williams R S 2010
[86] Maass W, Natschlaeger T and Markram H 2002 Real-time Self-aligned memristor cross-point arrays fabricated with
computing without stable states: a new framework for one nanoimprint lithography step Nano Lett. 10 2909–14
neural computation based on perturbations Neural [97] Xia Q F 2011 Nanoscale resistive switches: devices,
Comput. 14 2531–60 fabrication and integration Appl. Phys. A 102 955–65
[87] Paquot Y, Duport F, Smerieri A, Dambre J, Schrauwen B, [98] Hu M, Li H, Chen Y, Wang X and Pino R E 2011
Geometry variations analysis of TiO2 thin-film and
Haelterman M and Massar S 2012 Optoelectronic
spintronic memristors ASPDAC’11: Proc. 16th Asia and
reservoir computing Sci. Rep. 2 1–6
South Pacific Design Automation Conf. (Piscataway, NJ:
[88] Legenstein R and Maass W 2007 What Makes a Dynamical IEEE) pp 25–30
System Computationally Powerful? (Cambridge, MA: [99] Querlioz D, Bichler O, Dollfus P and Gamrat C 2013
MIT Press) pp 127–54 Immunity to device variations in a spiking neural network
[89] Urbanczik R and Senn W 2009 Reinforcement learning in with memristive nanodevices IEEE Trans.
populations of spiking neurons Nature Neurosci. 12 250–2 Nanotechnology PP p 1
[90] Rokach L 2010 Ensemble-based classifiers Artif. Intell. Rev. [100] Suri M, Bichler O, Querlioz D, Palma G, Vianello E,
33 1–39 Vuillaume D, Gamrat C and DeSalvo B 2012 CBRAM
[91] Kuncheva L I and Whitaker C J 2003 Measures of diversity devices as binary synapses for low-power stochastic
in classifier ensembles and their relationship with the neuromorphic systems: auditory (Cochlea) and visual
ensemble accuracy Mach. Learn. 51 181–207 (Retina) cognitive processing applications Electron
[92] Breiman L 2001 Random forests Mach. Learn. 45 5–32 Devices Meeting (EDM) IEEE Int. (Dec 2012)
[93] Breiman L 1996 Bagging predictors Mach. Learn. 24 123–40 pp 10.3.1–10.3.4

13

View publication stats