Fifty-Fifth Annual Allerton Conference

Allerton House, UIUC, Illinois, USA

October 3-6, 2017

Physical Layer Deep Learning of

Encodings for the MIMO Fading Channel
Timothy J. O’Shea Tugba Erpek T. Charles Clancy
Virginia Tech, Virginia Tech, Virginia Tech,
Arlington, VA Arlington, VA Arlington, VA
[email protected] [email protected] [email protected]

Abstract— We introduce a novel physical layer scheme for back from the receiver to the transmitter). Space-Time Block
Multiple Input Multiple Output (MIMO) communications Codes (STBC) are regularly used to increase robustness and
based on unsupervised deep learning using an autoencoder. extend coverage in open-loop systems [1], and in dense
This method extends prior work on the joint optimization
of physical layer representation and encoding and decoding multi-user environments, closed-loop MIMO with precoding
processes as a single end-to-end task by expanding the trans- techniques based on CSI are used to improve single and
mitter and receiver to the multi-antenna case. We introduce a multi-user throughput [2], [3]. While current day MIMO
domain appropriate wireless channel impairment model (the systems provide a substantial benefit, they are quite complex
multi-input multi-output Rayleigh fading channel), into the and not known to be optimal, leading to the obvious question:
autoencoder optimization problem in order to directly learn
a system which optimizes for it. This approach demonstrates ”can we do better?” Recent results in physical layer learning
significant potential for learning schemes which achieve and for the Single Input Single Output (SISO) channel [11]
exceed performance of current day methods which are widely have shown that autoencoders [7] can readily match the
used in existing wireless MIMO systems. We discuss how the performance of near-optimal existing baseline modulation
scheme can be easily adapted for open-loop and closed-loop and coding schemes by using an autoencoder to jointly learn
operation in spatial multiplexing modes as well as spatial
diversity modes. Each of these modes is learned and realized the system.
using the same simple and compact approach.

I. I NTRODUCTION
Multiple Input Multiple Output (MIMO) wireless systems
are widely used today in 4G cellular and wireless local area
network systems to increase throughput and coverage by
exploiting the multipath characteristics of the channel. By
encoding information across multiple antenna elements using Fig. 1. SISO channel autoencoder system
spatial-multiplexing or spatial-diversity schemes throughput
or range can be improved in various channel conditions. The idea of the channel autoencoder [8], [11] applies deep
Current schemes rely on rigid analytically obtained encod- unsupervised learning with a reconstruction loss function to
ing/beamforming and decoding techniques for these tasks jointly optimize encoding, decoding, and signal representa-
and are in general not known to be optimal. Moreover, spatial tion over some impaired communications channel. A brief
multiplexing techniques mostly rely on Channel State Infor- visualization of the channel autoencoder system is shown in
mation (CSI) estimation, quantization and feedback which Figure 1. This approach is appealing as it finds solutions
further complicates the ability of these schemes to perform for modulation and forward error correction which rival
optimally. In contrast, we introduce a scheme which can today’s best designs for small code word sizes over existing
combine many of these tasks into a single end-to-end learned channel impairment models, and offers a method to learn
estimation, feedback, encoding, and decoding process which solutions over channel impairments for which no optimal
can be jointly optimized to maximize throughput and min- solution is known [10] or a compact analytical representa-
imize bit error rate (BER) for specific channel conditions. tion may be difficult to express and optimize considering
We believe this joint system optimization process has the real world effects. Another benefit of such systems is that
potential to provide significant gains in comparison with the computational complexity of the learned encoder and
current day systems which are optimized in a more modular decoder modules can in many cases be lower compared to the
and disjoint fashion. existing methods, leading to potential power savings when
such systems are deployed.
II. BACKGROUND
Traditional MIMO communication schemes are divided III. T ECHNICAL A PPROACH
into two categories which are either open-loop (without CSI In the SISO channel autoencoder, one symbol s of k bits
at the transmitter), or closed-loop systems (with CSI fed is encoded into n sequential time samples to be transmitted

978-1-5386-3266-6/17/$31.00 ©2017 IEEE 76

over some communications channel such as a Rayleigh fad- based optimization process for such encoder and decoder
ing wireless channel. When noisy versions of these samples operations may be constructed while taking the CSI into
are received by a decoder, an estimate, ŝ is produced to best account. The methods simply differ in the fact that, in open-
reconstruct the original information. loop MIMO the encoder only has knowledge of the input bits
In this work, we consider the MIMO case where k bits when learning a representation and mapping, while in closed-
in a symbol s are encoded to form mt parallel transmit loop the encoder has both the input bits and a representation
streams of n time-samples each as illustrated in Figure 2. of the CSI available to learn a more informed mapping.
These streams undergo a multi-antenna mixing channel and
arrive at a receiver to produce mr receive streams, each
also of length n time samples, and are decoded to produce
an estimate, ŝ, of the originally transmitted symbol. Thus
there are mr ×mt unique pairwise channels, each with some
random impairment due to wireless propagation.
This is the model conventionally used in wireless systems,
where a MIMO transmission may optimize for varying
numbers of transmit and receive antennas. As an example,
in Alamouti coding, symbols are repeatedly encoded over
adjacent samples in time using different spatial modes.
Fig. 3. Channel autoencoder with CSI knowledge
By varying the number of information bits k, the transmit
streams mt , the receive streams mr and the number of time
samples n over which the information is encoded, this model A. Optimization Process
may be readily adapted to different information rates. By
selecting how many bits are encoded into a given space-time In our optimization process, we represent s as a 2k valued
block, and over what time and antenna dimension the space- integer of codeword indices which may be transmitted by the
time block spans encodings which optimize for high rate system, each encoding k bits. In the network, we present this
both spatial multiplexing and spatial diversity improvement as a one-hot input vector of length 2k with a single non-zero
can be achieved using the same MIMO channel autoencoder value of 1 for the desired codeword, and the output as a soft-
system. max classification task which approximates the probability of
each codeword (as done in [11]).
In such classification tasks, a categorical cross-entropy loss
function (CE ) may be readily used for optimization using
gradient descent to select network parameters. In this case
CE is given by 
m−1
CE (s, ŝ) = −1
m i=0 (si log(ŝi ) + (1 − si )log(1 − ŝi )),
where m is the number of codewords used during training.
Using a form of stochastic gradient descent (Adam [4]),
the weight updates can be computed using backpropagation.
In this case, we iteratively compute a forward pass: ŝ =
f (s, θ), and a backward pass, ∂θ ∂
= ∂CE (s,f
∂θ
(s,θ)
) where
network layer weights are given by θ and a weight update
∂
Fig. 2. MIMO channel autoencoder system takes the form of δw = −η ∂θ and η represents a (possibly
time varying) learning rate.
To realize and optimize such a MIMO channel autoen-
B. Channel Simulation and Network Architecture
coder we present the system shown in Figure 3 where we
leverage multi-layer neural networks (NNs) to provide the Core to the ability to optimize such an end-to-end system
mappings from information bits or codewords to digital is to accurately model or represent the MIMO channel effects
samples to transmit, and from received digital samples back within the network transfer function ŝ = f (s, θ). To that
to information bits or codewords. end, we introduce several new layers which simulate real
Here the encoder may introduce linear or non-linear de- world impairments and design constraints for each forward
pendencies and mixing between bits to form the symbols and backward pass.
transmitted over the air, and the receiver may learn a non- After the encoding to mt complex valued transmit streams,
linear mapping from received symbols back to bits which a transmit block-code tensor of shape [batch size, mt , 2, n] is
most effectively obtains this information from the mixed formed for transmission. To simulate MIMO propagation we
form over the channel. Such a system can be constructed use several custom layers to model the domain enumerated
where the encoder either has knowledge of CSI at the time below.
of encoding (closed-loop) or it does not have knowledge • enc: Learned Encoder: s → x
of CSI (open-loop). In Figure 3 we illustrate how an NN- • rnd: Random mr × mt channel response, H

77
 • mul: Complex matrix multiplication of x with H ML autoencoder implementation (Figure 4). The symbols
• norm: Normalize average power transmitted at the antenna level are then optimized during
• awg: Additive white gaussian noise N (0, σ) the training. Note that the throughput is the same for both
• dec: Learned Decoder: r → ŝ Alamouti coding and ML implementation since both take two
In terms of these basic operations, we can express the full time slots to send two symbols. In both cases it is assumed
network f as follows for the open loop MIMO encoding that no CSI is available at the transmitter (i.e. open-loop
case: MIMO).

f (s, θ) = dec(awg(norm(mul(enc(s), rnd())), σ)) (1)

and the for the closed-loop MIMO encoding case as:
f (s, θ) = (λH, dec(awg(norm(mul(enc(s, H), H)), σ)))(rnd())
(2)
where the channel information is concatenated with the input
symbols at the encoder.
Using this formulation, forward and backward gradient
passes can readily be computed on f (s, θ). In the backward Fig. 4. Symbols transmitted in Alamouti Scheme and Neural Networks
pass, the awg function becomes the identity function (it
is used only for forward passes). While the normalization Figure 5 shows the SNR vs BER results for both Alamouti
module enforces a constant average power, the noise standard STBC coding and autoencoder system. In our preliminary
deviation σ may be easily adjusted at training or test time to results we obtain slightly better performance from the Alam-
simulate varying levels of signal to noise ratio (SNR). outi STBC at low SNR and improved performance from the
autoencoder above around 15dB. We believe with additional
IV. P ERFORMANCE A NALYSIS hyperparameter tuning and longer training times at each SNR
In this section we train the autoencoder based learned level, we can also achieve the same performance as baseline
encoding model described above and evaluate the BER method for lower SNR values.
performance over a range of SNRs and compare the per-
formance to baselines widely used under different channel 2 × 1 Spatial Diversity Results Comparison
conditions. 0
10
We consider comparisons to two different baseline con- Learned 2x1
figurations, the 2 × 1 Alamouti STBC, intended to provide 10 −1 Alamouti 2x1
Bit Error Rate (BER)

spatial diversity based range extension and performance

improvements, along with the 2 × 2 MIMO system that 10−2
uses spatial multiplexing to increase throughput. The deep
learning model we developed can be extended to larger 10−3
systems with M × N antennas.
The Keras library [6] along with Tensorflow [5] as a 10−4
backend is used primarily to implement, train and evaluate
channel autoencoder models. 10−5

A. Spatial Diversity Performance 0 10 20 30

It is assumed that there are two antennas at the transmitter Signal to Noise Ratio (dB)
and one antenna at the receiver for the spatial diversity case.
As a result, the performance of the autoencoder is very Fig. 5. Error Rate Performance of Learned MIMO Encodings
comparable to the Alamouti coding [1]. Matlab is used to
simulate the communication system performance that uses Inspecting the transmitted and received constellation
Alamouti coding and these results are used as the baseline. points at high SNR for the system provides quite interesting
QPSK modulation is used to modulate the input bits and results. For the 2 × 1 system, we show the transmit constel-
Rayleigh fading is used as the channel model. lations for each antenna and time slot in Figure 6 along with
Alamouti coding groups symbols by two. It sends two the received constellations for each time slot at 20dB SNR. In
different symbols from each antenna in the first time slot. this case, the system appears to learn a scheme where power
The complex conjugate forms of the symbols are sent in the is unevenly distributed between antenna elements allocating
second time slot as shown in Figure 4. A Minimum Mean more power to antenna 0 than antenna 1 for both time slots.
Square Error (MMSE) receiver is then used to demodulate This appears to result in a multi-level amplitude and phase
and estimate bits. coding scheme emerging at the receiver which is decoded
On the other hand, the same symbols are repeated in the by the corresponding learned receiver decision surface. The
first and second time slots as inputs to the encoder for the choice of normalization method plays a large part in defining

78
the sort of constellation derived throughout the optimization As a result, transmit symbol encodings are learned learned
process, and constraints such as equal power per antenna during training which leverage CSI effectively to increase
could additionally be included as constraints if desired. throughput under specific channel conditions.
In Figure 7 we provide the performance curve comparing
BER from the learned encoding scheme compared against
the conventional system. For the baseline, average BER
results over all antennas are provided. Here, the ML approach
provides very promising results, with better performance
compared to baseline method for all SNR levels.

2 × 2 Spatial Multiplexing Code Comparison

Full CSI
−1 Baseline
10

Bit Error Rate (BER)

10−2

10−3

10−4

0 10 20 30
Signal to Noise Ratio (dB)

Fig. 7. Error Rate Performance of Learned MIMO Encodings

Fig. 6. Learned 2x1 Encoding Scheme

B. Spatial Multiplexing Performance

In the spatial multiplexing case, we seek to increase
throughput in the dense MIMO fading case rather than
extending range. Here we simulate a 2×2 closed-loop MIMO
system using a conventional point-to-point MIMO commu-
nication system to compare the results with the autoencoder
approach. Matlab is used for the baseline simulation and a
QPSK modulation is used to encode each data stream. It is
assumed that the channel estimation is perfect at the receiver
and there is no feedback error while sending the CSI back to Fig. 8. Learned 2 × 2 Encoding Scheme
the transmitter. Rayleigh fading is used as the channel model
and equal power is used at each antenna during transmission. Inspecting the constellations formed in a spatial multi-
The transmitter uses Singular Value Decomposition plexing scenario, in Figure 8, we see the learned transmit
(SVD)-based linear precoding techniques to diagonalize the and receive constellations for a 2 × 2 encoding scheme. The
channel and eliminate interference at the receiver as dis- system appears to learn a scheme where the receive constel-
cussed in [2]. It is assumed that the coherence time of the lations form relatively discernible structure. We believe in
channel is equal to the duration of a subframe. Thus, channel this case that potentially a high order QAM type equivalent
estimation is repeated every subframe. The average BER constellation is derived with significant tolerance for error
results over all symbols and subframes are computed per in recovering bits. These plots are generated at 20dB SNR,
antenna. much of the apparent noise is actually in the MIMO gain
The ML based transmitter also incorporates perfect CSI matrix, not in additive noise layer. Since we encode with
information at the transmitter within its NN model, allowing this as CSI, it leads to an almost noise like encoding in the
symbol encodings to change based on fading conditions. transmit side.

79
 We will further investigate what these constellations would [3] W. Yu, W. Rhee, S. Boyd, and J. M. Cioffi, “Iterative
look like for all symbols over a constant MIMO channel water-filling for gaussian vector multiple access chan-
matrix since it can be difficult to interpret all ”precodings” nels,” IEEE Transactions on Information Theory, vol.
merged together as shown here. 50, no. 1, pp. 145–151, 2004.
Our next step will be to improve this simulation by [4] D. Kingma and J. Ba, “Adam: A method for stochastic
introducing channel estimation and feedback error to re- optimization,” arXiv preprint arXiv:1412.6980, 2014.
flect practical implementation challenges. Moreover, since [5] M. Abadi and et al., TensorFlow: Large-scale machine
feedback of CSI (e.g., from a handset device to a base learning on heterogeneous systems, Software avail-
station) for closed-loop schemes requires protocol overhead able from tensorflow.org, 2015. [Online]. Available:
in real world systems, we will consider the case where CSI https://www.tensorflow.org/.
is constrained to be compact discrete valued information [6] F. Chollet, Keras, https : / / github . com /
encoded into compact p-bit fields. fchollet/keras, 2015.
[7] I. Goodfellow, Y. Bengio, and A. Courville, Deep
V. C ONCLUSION
learning. MIT press, 2016.
The architecture and initial results shown here provide [8] T. J. O’Shea, K. Karra, and T. C. Clancy, “Learn-
an exciting new approach to physical layer design and ing to communicate: Channel auto-encoders, domain
optimization for MIMO wireless communications schemes specific regularizers, and attention,” in 2016 IEEE
using an autoencoder. International Symposium on Signal Processing and
This is a significant departure from current day systems Information Technology (ISSPIT), 2016, pp. 223–228.
which still need to be understood and made to operate DOI : 10.1109/ISSPIT.2016.7886039.
efficiently with real world physical constraints. Preliminary [9] S. Cammerer, T. Gruber, J. Hoydis, and S. t. Brink,
results show that performance can potentially be quite com- “Scaling deep learning-based decoding of polar codes
petitive with existing schemes, especially at high SNRs. via partitioning,” arXiv preprint arXiv:1702.06901,
Further work is needed to compare these schemes to 2017.
existing baselines with error correction coding considered [10] N. Farsad and A. J. Goldsmith, “Detection algorithms
as well. Since channel autoencoders tend to learn encod- for communication systems using deep learning,”
ings equivalent to traditional joint modulation and coding CoRR, vol. abs/1705.08044, 2017. [Online]. Available:
schemes in terms of capacity, this would be a more fair http://arxiv.org/abs/1705.08044.
comparison, but would require non-matching modulation [11] T. J. O’Shea and J. Hoydis, “An introduction to
rates (with matching un-coded bit information rates). We deep learning for the physical layer,” CoRR, vol.
believe that this approach represents an exciting vein of abs/1702.00832, 2017. [Online]. Available: http://
future research considering that this approach has shown arxiv.org/abs/1702.00832.
promise in learning efficient encodings without any prior
knowledge about how to encode for such a channel, and
given that such a scheme could be learned for potentially
very different channel conditions which depart from the
MIMO Rayleigh model used here.
Lastly, these kinds of models are potentially quite at-
tractive due to the efficient computational realization which
leverages concurrent NN structure and minimal iteration,
as compared to the iterative algorithms used for detection
and error correction in many systems today. Significant
promise has been shown in the area of pure error correction
decoding in this area [9], and we believe that optimizing the
joint decoding process with MIMO symbol estimation and
decoding will in the future hold increasingly more promising
results than through isolated modular optimization of either
independently.
R EFERENCES
[1] S. M. Alamouti, “A simple transmit diversity tech-
nique for wireless communications,” IEEE Journal on
Selected Areas in Communications, vol. 16, no. 8,
pp. 1451–1458, 1998.
[2] E. Telatar, “Capacity of multi-antenna gaussian chan-
nels,” European Transactions on Telecommunications,
vol. 10, no. 6, pp. 585–595, 1999.

80