Analog Solutions of Systems of Linear Equations

on a Configurable Platform
Aishwarya Natarajan and Jennifer Hasler
Georgia Institute of Technology, [email protected]

Abstract—Even though analog computation is better suited for Ax = b ODE

differential equation solutions (ODE, PDE), sometimes it needs Digital Analog
to solve systems of linear equations. This discussion focuses x = A-1 b PDE
on analog solutions of linear equation systems, implemented
on a configurable platform. Digital systems rely on solving Analog Solution of Linear Equations
linear equations as the fundamental numerical computation.
Systems of linear equations are used to solve static circuits V3
illustrating that at least a reduced class of analog physical linear Vdd
system computation should be possible. The analog approaches IA
utilize iterative techniques, setting up a set of ODEs to solve
the system of linear equations, rather than relying on matrix
decompositions (e.g. LU decomposition). The approach allows
V1 R1 Gm2(V3 - V1)
for multiple potential configurable circuit approaches. A set V2
of amplifier networks has been designed to demonstrate the
solutions for different matrices. These techniques provide energy-
efficient continuous-time solutions. The resulting algorithm has
Gm1(V1 - V2)
been studied considering the analog numerical analysis for the IB
solution and convergence time.
Index Terms—Analog solutions of linear equations GND
GND
Solution of linear equations is the fundamental numerical
computation for digital systems (Fig. 1), and simultaneously
not well aligned with analog computation [1]. Solutions of
linear systems through matrix decomposition is the most
straight-forward method for digital computation, and yet is the I(t) = G V(t)
most challenging option for analog approaches [1]. Solution of
linear systems is ubiquitous for digital solutions (Fig. 1). Entire
numerical software tools (e.g. MATLAB: Matrix solution
Fig. 1. Solutions of linear systems are the fundamental operations for digital
Laboratory) are dedicated to these solutions, and benchmarks computation, whereas solutions of Differential Equations (ODE, PDE) are the
for computing are based on solving systems of linear equations fundamental operations for analog computation. Analog solutions of linear
(e.g. LINPACK [2]). Linear system solution requires the systems would enable a wider range of analog capability. One approach is
recognizing that the steady state solution of linear circuits with controlled
solution to the linear algebraic matrix equation sources can represent these networks; configurable analog systems can directly
implement such models using voltage-controlled current sources.
Ax = b (1)
where A is the input matrix, b is the input vector, and x is On the other hand, Engineering students (e.g. GT’s ECE
the solution vector. The classic digital solution method is a 2040) are taught that systems of linear equations, like nodal
form of Gaussian elimination to decompose A into a product equations, are used to solve static circuits resistive circuits with
of a lower triangular matrix, L, and an upper triangular matrix, independent and dependent voltage- and current-sources (e.g.
U. Once this solution is found, the solutions can be directly Fig. 1). Therefore, the steady state of linear circuits could solve
solved, first using L and b, and then using this result and systems of linear equations (e.g. Fig. 1). Electronic circuits are
U. This computational method shows all of the strength of used to illustrate solutions (e.g. Hopfield networks [3], [4]),
symbolic digital processing, including the pivot stages, to simulating linear solutions for a reduced class of problems
get the maximum accuracy for the decomposition. Analog [5], [6], and considering specialized cases (e.g. elliptic PDEs
decomposition into L and U is extremely challenging, with [7], [8] or resistor-only concepts [9], [10], [11], [12]). This
numerous integer steps requiring intermediate value storage discussion focuses on a general algorithmic and numerical
at high resolution, and memory manipulations (e.g. pivots) to methods for analog computation of linear system of equations
retain as much final precision as possible. The lower analog through the implementation and experimental measurements
precision, particularly through short-term sampled registers, from a large-scale Field Programmable Analog Array (FPAA).
makes this process incredibly challenging [1]. [13].

978-1-7281-3320-1/20/$31.00 ©2020 IEEE

Authorized licensed use limited to: IEEE Xplore. Downloaded on October 03,2020 at 21:41:11 UTC from IEEE Xplore. Restrictions apply.
 G1,1 V1 G2,1 V1 Gm,1 V1
V1 V1 V1
fixed fixed fixed
GND GND GND
G1,2 V2 G2,2 V2 Gm,2 V2
Vdd Vdd Vdd
V2 V2 V2
fixed fixed I2 fixed
I1 Im
GND GND GND
V1 V2 Vm

C1 C2 Cm
G1,m Vm G2,m Vm Gm,m Vm
Vm Vm Vm
GND GND GND
fixed fixed fixed
GND GND GND

Fig. 2. A potential continuous-time circuit architecture to solve systems of linear equations is shown. Linear matrix solution is built from Transconductance
Amplifiers (OTA), which are elements of the Computational Analog Block (CAB) on the SoC. OTAs with Floating-Gate (FG) bias currents make up the
individual conductances allowing for a general constraint matrix to solve. The input vectors are fed into the gate input of the 9 transistor OTA, which acts as
a current source, while the offset for the reference voltages can be controlled by DACs compiled on the Array.

The paper focuses on solving systems of linear equations The dynamics of (2) are seen along the eigenspace corre-
using analog computation. The discussion moves towards iter- sponding to A. We define the eigenvalue / eigenvector of A
ative methods for solving linear equations, good methods for as
analog and digital computation, and its potential configurable A = EΛE−1 (4)
circuit approaches. The discussion then moves towards analyz-
where Λ is a diagonal matrix of eigenvalues, and E are the
ing the algorithmic issues and analog numerical analysis issues
corresponding rows of normalized (power = 1) eigenvectors
of this approach. These techniques provide energy-efficient
corresponding to the particular eigenvector. By projecting (1)
continuous-time solutions. Even if an ideal solution exists, one
and (2) on the eigenvector basis in (4), we get
needs confidence in the overall accuracy and convergence time.
dy
x = Ey, τ + Λy = E−1 b, (5)
I. P HYSICAL C OMPUTATION L INEAR E QUATION dt
S OLUTIONS → I TERATIVE M ATRIX T ECHNIQUES This ODE requires positive Λ values requiring that A be
positive definite. Non positive definite matrices can be solved
Linear systems are also solved by iterative techniques, using through multiple transformations [6], [16], such as multiplying
coupled equations that converge x to the desired solution. (2) by AT resulting in a positive definite matrix [16]. There-
Iterative digital techniques are sometimes faster and have fore, this method is general enough for these solution methods.
fewer operations than digital Gaussian elimination, particularly
for sparse matrix solutions. As one should simply use physical II. L INEAR E QUATION S OLUTION USING
techniques to solve ODE / PDE applications [14], we focus T RANSCONDUCTANCE A MPLIFIERS
on physical solution systems of linear systems from different We solve these linear system of equations employing
applications. Transconductance Amplifiers (TA) in an architecture shown
A physical approach will be aligned with iterative matrix in Fig. 2. One can achieve a larger set of matrix equations
equations. Physical computing solutions of linear systems are utilizing TAs, enabling a richer set of potential matrices. The
iterative techniques, where the iterations result from coupled linear region of an OTA is expressed as
differential equations whose steady state represents the desired
output (x). One might modify (1) to build the differential Iout = G(V + − V − ) (6)
equation For an OTA programmed with subthreshold bias currents
dx (Ibias ), the resulting coupling between nodes (k,l) would be
τ + Ax = b, (2)
dt Gk,l = κIUbiasT
. The discussion holds when utilizing above
threshold currents with a modification of the underlying func-
where τ is the time-constant for the network. Each row could
tion for G.
have a different τ , although we present the simpler case for
Figure 2 shows the resulting general network to solve (1),
clarity. Digital (sampled time) approaches would be written as
where the l row is written as
x[n] = x[n − 1] + ǫ (b − Ax[n]) , (3) dVl Xm
C = Il − Gl,k Vk (7)
dt
effectively approximating the time derivative of x. These k=1

techniques are related to gradient-descent, Jacobi, and Gauss- and by comparing with (2), al,k ∝ Gl,k and bk ∝ Ik .
Seidel iterative matrix solutions [15]. The timeconstant, τ is the load capacitance divided by a

Authorized licensed use limited to: IEEE Xplore. Downloaded on October 03,2020 at 21:41:11 UTC from IEEE Xplore. Restrictions apply.
 1.1 60
b Voltage (V)

Output V from DC (mV)

50
x2: 50.5mV
40
1
x3 30
x1: 21.2mV
200nA 100nA 100nA 100nA 20
x4: 19.2mV
10 x3: 13.7mV
1.1 100nA 200nA 100nA 100nA
k
x = b(t) 0
2 UT 100nA 100nA 200nA 100nA 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Voltage (V)

Time (ms)
100nA 100nA 100nA 200nA 100

Steady State - Output (mV)

OTA Linear Range
x4

1.05
10
x2: 50.5mV
x2

x1
x1: 21.2mV

1 x4: 19.2mV
V = 1.05, Offsets = [-12.7 29.0 -33.7 49.9] mV
ref x3: 13.7mV
1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
Time (ms) Time (ms)

Fig. 3. Analysis for the convergence of A matrix with 200nA and 100nA as the diagonal and off-diagonal elements respectively, is shown. For a 40mV b
input applied as current sources, the x(t) solution is plotted and is within the linear range of the OTAs. The time constant is 61µs curve-fitting the exponential
curves from the step responses. One of the components along the eigenvalue that is larger by a factor of 5, converges 5 times faster in 12µs.

normalizing conductance, often related to a larger diagonal sizes. Additional A matrices can be programmed and dynamics
element of A. For a particular normalizing subthreshold bias measured.
CUT
current, Ibias , τ = κI . One could have a different τ per
ref
row due to different conductance modeling and capacitances, Figure 3 analyzes the convergence and resulting eigenvalue
and potentially tuned to optimize convergence. Current sources timeconstants from the step responses. The dynamics are
set b inputs where positive current sources would go to Vdd , studied by plotting the matrix solution outputs, x(t) which
and negative current sources would go to GND. These current are the steady-state responses. In case of the matrix with
sources could also be implemented as additional TAs with 200nA diagonal elements, the effective time constant is 61µs
controlling input voltages. from the three eigenvalues, obtained through curve-fitting the
exponential curves from the step responses. Since one of the
III. M EASURED DYNAMICS ON THE S O C FPAA eigenvalues is larger by a factor of 5, it converges 5 times
This approach is implemented in an SoC FPAA device [13]. faster, in 12µs. The time constant is 47µs for the diagonal
The linear equations are solved by 9-transistor TA elements matrix with 1µA as the 1 element and 10nA as the 0 elements.
in Computational Analog Blocks (CAB) [13]. The A matrix The effective time constant is related to the minimum
is set by the conductances of the individual OTAs, that in eigenvalues, resulting in the largest value for settling to a
turn are set by Floating-Gate (FG) bias currents. A TA is solution. Other terms will settle faster, and depending on the
used to convert the input voltage (b) signal to a current. The particular problem and method, the τ for each row could be
reference voltage(s) can be controlled by DACs compiled on adapted to optimize convergence. Larger eigenvalues converge
the FPAA [13]. A 4x4 linear system solution is compiled and faster. Smaller eigenvalues that correspond to the smaller
programmed on the FPAA using 16 TAs to implement the 4x4 changes in the solution (signal power), have little changes
A matrix, and 4 TAs to implement the b matrix. The sign of overall, and therefore, not in the required precision. Hence,
each A matrix element determines the TA input sign, where it is not important to wait for them. One could visualize the
positive values are input in the - input and the reference to the adaptation of τ to accommodate for λ, because, if (2) is stable
+ input, while negative values are input in the + input and the for one set of τ , it is stable for all positive τ .
reference to the - input. The outputs, x(t) can be scanned and
buffered out to be measured. Another way to understand the ODE of the linear equation
Figure 3 shows the dynamics of the matrix solutions, solver as a dynamical system or fully coupled filters, is
demonstrated through a 4x4 network. The b vectors are inputs typically as a set of low-pass filters. Figure 4 shows the
fed to the gate input of the TA structure. In particular, the response of the network to a chirp input, sweeping between
inputs are step functions. They are step inputs, starting from frequencies from 1Hz to 20kHz over a time duration of 1ms.
zero (= fixed potential) and moving to a single input value for These approaches give a new way of approaching the solution
b. The matrix solution outputs, x(t) , show the dynamics when of linear equations as effectively filter design, consistent with
new inputs are applied to different conductance matrices. The an approach typically seen in control system implementations.
time constant is roughly the diagonal TA bias current and the The dynamics of the output at one tap show the higher
FPAA routing capacitive load, and is related to any mismatch frequency components being attenuated, which is consistent
in the resulting components, both for small and large signal with low pass behavior.

Authorized licensed use limited to: IEEE Xplore. Downloaded on October 03,2020 at 21:41:11 UTC from IEEE Xplore. Restrictions apply.
 V. A NALOG N UMERICS OF I TERATIVE L INEAR E QUATION
1.26
S OLUTIONS
Input (V)

For digital numerical analysis of linear systems, the con-

dition number of A determines the discussion of numeri-
cal accuracy. The condition number of A is related to the
1.23 ratio between the largest magnitude (λmax ) eigenvalue and
1.97
smallest (magnitude) (λmin ) eigenvalue of A, demonstrating
the eigenvalues spread. The larger the condition number, the
Output (V)

higher required starting precision is expected to counteract the

amplification of numerical errors in b. The metric is a loose
bounds on the error propagation for numerical approaches.
1.955 One might imagine that A with a moderate or high condition
5
number would be unusable for analog techniques. Typical
0 10
Time (ms) practice assumes that one loses the number of bits related
to the log2 of the condition number of A; such loss of
Fig. 4. A chirp input of 20mV offset about 1.25V is applied as the b(t) vector precision makes analog techniques nearly infeasible. This
for a set of currents to the 4x4 OTA network structure. The output response
is shown, where one could observe the higher frequencies being attenuated, assumption needs to be revisited both for the actual nature of
thereby effectively behaving like a Low-pass filter. The linear equation solver the computation, as well as within the framework of analog
can be described as a filter response, thereby connecting to control theory and numerical analysis techniques [1].
filter design concepts.
The maximum gain from b to x is λmax /λmin . Higher gain
is one reason for considering a decrease in precision for the
IV. L INEAR E QUATION S OLVER A RCHITECTURE
operation, particularly when using floating-point arithmetic.
C OMPLEXITY
Gain of errors in the input (b) or the eigenvector projected
Digital systems utilize a number of techniques for solving input ( E−1 b ) would have a similar issue for the worst case,
linear systems based on a range of potential applications. The and therefore either formulation will help. Gain, like all analog
complexity for solving diagonal matrices, whether by analog gains, will raise up signal and noise.
or digital techniques is similar. Upper or Lower diagonal
matrices (e.g. L or U) can easily be solved by analog tech- VI. S UMMARY AND D ISCUSSION
niques, linearly propagating each of the results in a similar The paper discussed solving systems of linear equations
fashion for a digital solution for L and U, retaining the using analog computation transforming the linear system so-
typical 1000× improvement for an analog system over a digital lution method to a set of ODEs. The technique is related to
system (e.g. [17]), which turns out to be a similar complexity iterative digital methods for solving linear equations. These
for a Vector-Matrix Multiplication (VMM). These operations approaches extend the energy efficient properties of analog
(analog or digital) are O(m2 ) in area and total operations for computing initially shown for vector-matrix multiplication to
mxm matrices when only considering computational elements, solutions of linear systems, where the vector-matrix multipli-
and for full architecture analysis [18] with communication / cation happens through arrays of TAs.
memory access, requiring (at least) O(m3 ) Area–Delay and The paper also starts analyzing the algorithmic issues and
Power–Delay products. analog numerical analysis issues of this approach. The dy-
The remaining question is comparing the analog computa- namics and convergences are studied for different matrices,
tion to the linear solution using Gaussian decomposition of through experimental measurements of the matrix output so-
L and U or finding A−1 . These digital computations require lutions from hardware. Analog solutions of linear systems
O(N3 ) operations. The analog techniques closely resemble typically is the most challenging algorithm for analog com-
iterative matrix solutions [15]. The relative scaling question putation [1]. Hence, finding analog algorithmic solutions for
for analog iterative techniques is eigenvalue spread with in- linear systems opens the entire range of analog computing
creasing number of nodes, which in turn, is related to the towards high-performance computing. This approach allows
propagation of the resulting iterative outputs. The particular for solution of any positive definite A matrix through the use of
method highly depends on the application, as the computa- OTA devices, and not limited as in resistive coupling networks,
tional complexities are similar in each case. originally proposed in Hopfield networks [3], [4] that could be
These computational comparisons make some particular built using two-terminal nano devices like memristors [12],
assumptions. If application comes from a directly solvable [20], [21], with simple transformations on the set of linear
physical system (e.g. PDE computations), one would utilize equations (e.g. multiplying by AT ).
those more natural techniques for that application as they
already benefit the analog techniques. Both analog and digital VII. ACKNOWLEDGEMENTS
have similar tradeoffs for sparse computation, particularly The authors wish to thank Sandia Laboratories, NM, for
with configurable analog capabilities [19]; therefore sparse partial funding that enabled the circuit characterization.
computation has similar effects on both approaches.

Authorized licensed use limited to: IEEE Xplore. Downloaded on October 03,2020 at 21:41:11 UTC from IEEE Xplore. Restrictions apply.
 R EFERENCES
[1] J. Hasler, “Starting Framework for Analog Numerical Analysis for
Energy Efficient Computing,” Journal of Low Power Electronics Appli-
cations, vol. 7, no. 17, June 2017. pp. 1-22.
[2] J.J. Dongarra, P.Luszczek, and A. Petitet, “The LINPACK Benchmark:
past, present and future,” Concurrency and Computation: Practice and
Experience, Wiley, 2003, pp. 803-820.
[3] J.J. Hopfield, “Neural networks and physical systems with emergent
collective computational abilities,” Proceedings of National Academy of
Science, vol. 79, 1982, pp. 2554.
[4] J.J. Hopfield, “Neurons with graded responses have collective compu-
tational properties like those of two- state neurons,” Proceedings of
National Academy of Science, vol. 81, 1984, pp. 3088-3092.
[5] R. Umbehauen and A. Cichocki, MOS Switched-Capacitor and
Continuous-Time Integrated Circuits and Systems, Springer-Verlag,
1989.
[6] A. Cichocki and R. Umbehauen, “Neural networks for solving systems
of linear equations and related problems,” IEEE March 1992 CAS I, vol.
39, no. 2,
[7] N. Guo, Y. Huang, T. Mai, S. Patil, C. Cao, M Seok, S. Sethumadhavan,
and Y. Tsividis, “Energy-Efficient Hybrid Analog/Digital Approximate
Computation in Continuous Time,” IEEE Journal of Solid State Circuits,
2016.
[8] Y. Huang, N. Guo, M. Seok, Y. Tsividis, and S. Sethumadhavan,
“An Analog Accelerator for Linear Algebra,” Proceedings of the 43rd
International Symposium on Computer Architecture, Seoul, June 18,
2016. pp. 570-582.
[9] R. M. Walker, “An Analogue Computer for the Solution of Linear
Simulaneous Equations,” Proceedings of the IRE – Waves and Electrons
Section, 1949, pp. 1467-1473.
[10] S. K. Mitra, “Electrical Analog Computing Machine for Solving Linear
Equations and Related Problems,” Review of Scientific Instruments vol.
26, 1955. pp.453-457.
[11] K. P. Lanneau and Lindsay I. Griffin, “Analogue Computer for Solving
Simultaneous Equations,” US Patent 2,911,146 Filed May 27, 2953,
issued, Nov. 3, 1959.
[12] Sun, Z., Pedretti, G., Ambrosi, E., Bricalli, A., Wang, W., and Ielmini, D.
, “Solving matrix equations in one step with cross-point resistive arrays”,
Proceedings of the National Academy of Sciences, 2019, 116(10), 4123-
4128.
[13] S. George, S. Kim, S. Shah, J. Hasler, M. Collins, F. Adil, R, Wunderlich,
S. Nease, and S. Ramakrishnan, “A Programmable and Configurable
Mixed-Mode FPAA SoC,” IEEE Transactions on VLSI, vol. 24, no. 6,
2016, pp. 2253-2261.
[14] J. Hasler, “Opportunities in Physical Computing driven by Analog
Realization,” IEEE ICRC, San Diego, 2016.
[15] G. E. Golub and C.F. Van Loan, Matrix Computation, 2nd ed, John
Hopkins Press, 1989.
[16] B. Ulmann and D. Killat, “Solving systems of linear equations on analog
computers,” IEEE Kleinheubach Conference, 2019.
[17] C. Schlottmann, and P. Hasler, “A highly dense, low power, pro-
grammable analog vector-matrix multiplier: the FPAA implementation,”
IEEE Journal of Emerging CAS, vol. 1, 2012, pp. 403-411.
[18] J. Hasler, “Analog Architecture Complexity Theory Empowering Ultra-
Low Power Configurable Analog and Mixed Mode SoC Systems,”
Journal of Low Power Electronics Applications, Jan. 2019. pp. 1-37.
[19] J. Hasler, “Large-Scale Field Programmable Analog Arrays,” IEEE
Proceedings, February 2020.
[20] K.-H. Kim, S. Gaba, D. Wheeler, J. M. Cruz-Albrecht, T. Hussain,
N. Srinivasa, and W. Lu, ”A Functional Hybrid Memristor Crossbar-
Array/CMOS System for Data Storage and Neuromorphic Applications”,
Nano Lett., vol. 12, 2012, pp. 389-395.
[21] S. H. Jo, and W. Lu, “Programmable Resistance Switching in Nanoscale
Two-Terminal Devices”, Nano Letters vol. 9, 2009, pp. 496-500.
[22] S.Y. Kung, VLSI Array Processors, Prentice Hall, 1988.

Authorized licensed use limited to: IEEE Xplore. Downloaded on October 03,2020 at 21:41:11 UTC from IEEE Xplore. Restrictions apply.