1

Interference Cancellation and Iterative

Detection for Orthogonal Time Frequency
Space Modulation
P. Raviteja, Khoa T. Phan, Yi Hong, and Emanuele Viterbo
arXiv:1802.05242v2 [cs.IT] 16 Feb 2018

Abstract

The recently proposed orthogonal time frequency space (OTFS) modulation technique was shown
to provide significant error performance advantages over orthogonal frequency division multiplexing
(OFDM) in Doppler channels. In this paper, we first derive the explicit input–output relation describing
OTFS modulation and demodulation (mod/demod) for delay–Doppler channels. We then analyze the
cases of (i) ideal pulse-shaping waveforms that satisfy the bi-orthogonality conditions, and (ii) rectan-
gular waveforms which do not. We show that while only inter-Doppler interference (IDI) is present
in the first case, additional inter-carrier interference (ICI) and inter-symbol interference (ISI) occur in
the second case. We next analyze the interferences and develop a novel low-complexity yet efficient
message passing (MP) algorithm for joint interference cancellation (IC) and symbol detection. While
ICI and ISI are eliminated through appropriate phase shifting, IDI can be mitigated by adapting the MP
algorithm to account for only the largest interference terms. The proposed MP algorithm can effectively
compensate for a wide range of channel Doppler spreads. Our results indicate that OTFS using practical
rectangular waveforms can achieve the performance of OTFS using ideal but non-realizable pulse-
shaping waveforms. Finally, simulations results demonstrate the superior error performance gains of the
proposed uncoded OTFS schemes over OFDM under various channel conditions.

Index Terms

Delay–Doppler channel, OTFS, message passing, time–frequency modulation.

The authors are with ECSE Department, Monash University, Clayton, VIC 3800, Australia. Email: {raviteja.patchava,
khoa.phan, yi.hong, emanuele.viterbo}@monash.edu.
 2

I. I NTRODUCTION

Fifth-generation (5G) mobile systems are expected to accommodate an enormous number of

emerging wireless applications with high data rate requirements such as real-time video stream-
ing, and online gaming, connected and autonomous vehicles. While the orthogonal frequency
division multiplexing (OFDM) modulation scheme currently deployed in fourth-generation (4G)
mobile systems can achieve high spectral efficiency for time-invariant frequency selective chan-
nels, it is not robust to time-varying channels with high Doppler spread (e.g., high-speed railway
mobile communications). Hence, new modulation techniques that are robust to channel time-
variations have been extensively explored.
To cope with time-varying channels, one existing approach is to shorten the OFDM symbol
duration so that the channel variations over each symbol appear inconsequential [1]. However, one
major drawback is the reduced spectral efficiency due to cyclic prefix (CP). Another approach
exploits time–frequency signaling [2], [3]. In [4] the authors have introduced the frequency-
division multiplexing with frequency-domain cyclic prefix (FDM-FDCP), which can efficiently
compensate for channel Doppler spread. In high-Doppler low-delay spread channels, FDM-FDCP
is shown to outperform OFDM at the same spectral efficiency. The performance of FDM-FDCP
under other channel conditions are yet to be studied.
A new time–frequency modulation technique called orthogonal time frequency space (OTFS)
was recently proposed in [5], [6], which shows significant advantages over OFDM in delay–
Doppler channels. The delay–Doppler domain provides an alternative representation of a time-
varying channel geometry modeling mobile terminals and reflectors [7], [8]. Leveraging on this
representation, the OTFS modulator spreads each information symbol over a two dimensional
(2D) orthogonal basis function, which spans across the entire time–frequency domain required
to transmit a frame. The set of basis functions is specifically designed to combat the dynamics
of the time-varying multi-path channels. In [5], a general framework of OTFS based on ideal
pulse-shaping waveforms was introduced. A coded OTFS system with turbo equalization was
compared with coded OFDM, showing remarkable gains. In [9], since mm-wave channels incur
high frequency dispersion, OTFS is shown to outperform OFDM significantly.
In this paper, we analyze the input–output relation describing uncoded OTFS mod/demod for
delay–Doppler channels using general pulse-shaping waveforms. The relation reveals the effects
of the inverse symplectic finite Fourier transform (ISFFT) and SFFT operations interpreted as
 3

pre- and post-processing blocks applied to a time–frequency signaling scheme. We then analyze
the cases of (i) ideal pulse-shaping waveforms that satisfy the bi-orthogonality conditions, and
(ii) practical rectangular waveforms which do not. Unlike previous works [10], [11], we assume
no CP in the second case. We show that, while only inter-Doppler interference (IDI) is present
in the ideal waveform case due to unavoidable fractional Doppler effects, additional inter-carrier
interference (ICI) and inter-symbol interference (ISI) occur in the latter case due to imperfect
bi-orthogonality in time–frequency domain of the rectangular waveforms.
The delay–Doppler channel model with a small number of paths, with varying delay and
Doppler values, provides a sparse representation of the communication channel. We then propose
a low-complexity message passing (MP) algorithm for a joint interference cancellation (IC)
and detection, which takes advantage of the inherent delay–Doppler channel sparsity. The MP
algorithm is based on a sparse factor graph and uses Gaussian approximation of the interference
terms to further reduce the complexity. The approach is similar to [12], where it was applied to
massive MIMO without the advantage of channel sparsity. The complexity and convergence of
the MP algorithm are analyzed. In the MP algorithm, while the ICI and ISI can be eliminated
by suitable phase shifting, the IDI can be mitigated by adapting the MP algorithm to account
for only the largest interference terms. Consequently, the proposed MP algorithm can effectively
compensate for a wide range of channel Doppler spreads. Further, our results show that OTFS
using practical rectangular waveforms can achieve the performance of OTFS using ideal but
non-realizable pulse-shaping waveforms. Simulations results illustrate the superior performance
gains of the proposed uncoded OTFS schemes over OFDM under various channel conditions.
The rest of the manuscript is organized as follows. Section II recalls the OTFS mod/demod and
derives the corresponding input–output relation. In Section III, we analyze the time–frequency
domain and delay–Doppler domain relations for the ideal waveform case. Section IV is dedicated
to the case of OTFS using rectangular waveforms. Section V proposes the MP algorithm for the
joint IC and detection. Simulation results are presented in Section VI followed by the conclusions
in Section VII. The proofs are relegated to Appendix at the end of the manuscript.

II. S YSTEM MODEL

In this section, we first recall the basic concepts in OTFS and then present the explicit analysis
of OTFS mod/demod. More importantly, we derive the input–output relation of OTFS mod/demod
for delay–Doppler channels.
 4

Time-Frequency Domain
x[k, l] X[n, m] Heisenberg s(t) Channel r(t) Wigner Y [n, m] y[k, l]
ISFFT h(τ, ν) SFFT
Transform Transform

Delay-Doppler Domain

Fig. 1. OTFS mod/demod

A. Basic OTFS concepts/notations

We follow the notations in [5], [13] summarized below:

– The time–frequency signal plane is discretized to a grid by sampling time and frequency
axes at intervals T (seconds) and ∆f (Hz), respectively, i.e.,

Λ = (nT, m∆f ), n = 0, . . . , N − 1, m = 0, . . . , M − 1

for some integers N, M > 0.

– Modulated time–frequency samples X[n, m], n = 0, . . . , N − 1, m = 0, . . . , M − 1 are
transmitted over an OTFS frame with duration Tf = NT and occupy a bandwidth B = M∆f .
– Transmit and receive pulses (or waveforms) are denoted by gtx (t) and grx (t). Let Agrx ,gtx (t, f )
denote the cross-ambiguity function between gtx (t) and grx (t), i.e.,
Z
′
∗ ′
Agrx ,gtx (t, f ) , grx (t − t)gtx (t′ )e−j2πf (t −t) dt′ . (1)

– The delay–Doppler plane is discretized to an information grid:

n k l
 o
Γ= , , k = 0, . . . , N − 1, l = 0, . . . , M − 1 ,
NT M∆f
where 1/M∆f and 1/NT represent the quantization steps of the delay and Doppler frequency,
respectively1.
Remark 1: Parameter choice for OTFS systems. Given a communications system with total
bandwidth B = M∆f and latency Tf = NT constraints, we may choose N, M, T , (since
∆f = 1/T ) over a time-varying channel with maximum delay τmax and maximum Doppler νmax ,
among all channel paths. We can see that T and ∆f determine the maximum supportable Doppler
(i.e., 1/T ) and delay (i.e., 1/∆f ). Hence, it is required that νmax < 1/T and τmax < 1/∆f so

1
Note that the first and second indexes, k and l, in Γ represent the Doppler and delay axis, respectively.
 5

that N and M are determined. To support a fixed data rate of NM symbols per frame, depending
on the channel conditions, we can choose a larger T and smaller ∆f , which results in a smaller
N and larger M, respectively, or vice versa.

B. General OTFS mod/demod block diagram

The OTFS system diagram is given in Fig. 1. OTFS modulation is produced by a cascade of a
pair of 2D transforms at both transmitter and receiver. The modulator first maps the information
symbols x[k, l] in the delay–Doppler domain to samples X[n, m] in the time–frequency domain
using the inverse symplectic finite Fourier transform (ISFFT). Next, the Heisenberg transform is
applied to X[n, m] to create the time domain signal s(t) transmitted over the wireless channel.
At the receiver, the time-domain signal r(t) is mapped to the time–frequency domain through
the Wigner transform (the inverse of the Heisenberg transform), and then to the delay–Doppler
domain using SFFT for symbol demodulation.

C. OTFS modulation

Consider a set of NM information symbols {x[k, l], k = 0, . . . , N −1, l = 0, . . . , M −1} from

a modulation alphabet of size Q A = {a1 , · · · , aQ } (e.g. QAM symbols), which are arranged
on the delay–Doppler grid Γ.
The OTFS transmitter first maps symbols x[k, l] to NM samples X[n, m] on the time–
frequency grid Λ using the ISFFT as follows:
N −1 M −1

1 XX nk
− ml
X[n, m] = √ x[k, l]ej2π N M (2)
NM k=0 l=0
for n = 0, . . . , N − 1, m = 0, . . . , M − 1.
Next, a time–frequency modulator converts the samples X[n, m] to a continuous time wave-
form s(t) using a transmit waveform gtx (t) as
N
X −1 M
X −1
s(t) = X[n, m]gtx (t − nT )ej2πm∆f (t−nT ) . (3)
n=0 m=0

As noted in [5], (3) is also referred to in the mathematical literature as the (discrete) Heisenberg
transform [14], parametrized by gtx (t).
 6

D. Wireless transmission and reception

The signal s(t) is transmitted over a time-varying channel with complex baseband channel
impulse response h(τ, ν), which characterizes the channel response to an impulse with delay
τ and Doppler ν [8]. The received signal r(t) is given by (disregarding the noise to simplify
notation): Z Z
r(t) = h(τ, ν)s(t − τ )ej2πν(t−τ ) dτ dν. (4)

Equation (4) represents a continuous Heisenberg transform parametrized by s(t) [5]. Since
typically there are only a small number of reflectors in the channel with associated delays and
Dopplers, very few parameters are needed to model the channel in the delay–Doppler domain.
The sparse representation of the channel h(τ, ν) is given as:
P
X
h(τ, ν) = hi δ(τ − τi )δ(ν − νi ) (5)
i=1

where P is the number of propagation paths, hi , τi , and νi represent the path gain, delay, and
Doppler shift (or frequency) associated with i-th path, respectively, and δ(·) denotes the Dirac
delta function. We denote the delay and Doppler taps for i-th path as follows:
lτi kν + κνi
τi = , νi = i (6)
M∆f NT
for integers lτi , kνi and real − 12 < κνi ≤ 12 . Specifically, lτi and kνi represent the indexes of
the delay tap and Doppler tap, corresponding to (continuous) delay τi and Doppler frequency
νi , respectively. We will refer to κνi as the fractional Doppler since it represents the fractional
shift from the nearest Doppler tap kνi . We do not need to consider fractional delays since the
1
resolution of the sampling time M ∆f
is sufficient to approximate the path delays to the nearest
sampling points in typical wide-band systems [15].

E. OTFS demodulation

At the receiver, a matched filter computes the cross-ambiguity function Agrx ,r (t, f ):
Z
′
∗ ′
Y (t, f ) = Agrx ,r (t, f ) , grx (t − t)r(t′ )e−j2πf (t −t) dt′ . (7)

The matched filter output is obtained by sampling Y (t, f ) as

Y [n, m] = Y (t, f )|t=nT,f =m∆f . (8)

 7

for n = 0, . . . , N − 1 and m = 0, . . . , M − 1. Operations (7) and (8) are referred as the Wigner
transform. In the following theorem, we characterize the relationship between time–frequency
output samples Y [n, m] and input samples X[n, m].
Theorem 1: OTFS time–frequency domain analysis. The following input–output relation of
OTFS in time–frequency domain is given by:
N
X −1 M
X −1
Y [n, m] = Hn,m [n′ , m′ ]X[n′ , m′ ], (9)
n′ =0 m′ =0

where
Z Z
′ ′
Hn,m [n , m ] = h(τ, ν)Agrx ,gtx ((n − n′ )T − τ, (m − m′ )∆f − ν)
′ ′ ′
ej2π(ν+m ∆f )((n−n )T −τ ) ej2πνn T dτ dν. (10)

Proof: The proof is given in Appendix A. 

We can see that the terms Hn,m [n′ , m′ ] include the combined effects of the transmit pulse,
channel, and receive pulse. Note that results similar to this theorem have been presented for the
case of pulse-shaped (PS) OFDM [16], [17].
Next, the SFFT is applied on the samples Y [n, m] to obtain symbols y[k, l] in the delay–
Doppler domain:
N −1 M −1

1 XX nk
− ml
y[k, l] = √ Y [n, m]e−j2π N M . (11)
NM n=0 m=0
Theorem 1 provides the basis of the study of OTFS in two special cases, namely using ideal
waveforms (Section III) and more practical rectangular waveforms (Section IV). We will obtain
explicit input-output relations using the delay–Doppler channel model (5) for both cases.

III. OTFS WITH IDEAL WAVEFORMS

The grx (t) and gtx (t) pulses are said to be ideal if they satisfy the bi-orthogonal property [5]

Agrx ,gtx (t, f )|t=nT +(−τmax ,τmax ),f =m∆f +(−νmax ,νmax) = δ[n]δ[m]qτmax (t)qνmax (f ) (12)

where qa (x) = 1 for x ∈ (−a, a) and zero otherwise. Equivalently, Agrx ,gtx (t, f ) = 0 for t ∈
(nT − τmax , nT + τmax ) and f ∈ (m∆f − νmax , m∆f + νmax ), for all values of n, m except for
n = 0, m = 0, where Agrx ,gtx (t, f ) = 1 for t ∈ (−τmax , τmax ) and f ∈ (−νmax , νmax ).
 8

Unfortunately, ideal pulses cannot be realized in practice but can be approximated by wave-
forms with a support concentrated as much as possible in time and in frequency, given the
constraint imposed by the uncertainty principle. Nevertheless, it is important to study the error
performance of OTFS with ideal waveforms since it serves as a lower bound on the performance
of OTFS with practically realizable waveforms such as rectangular waveforms, etc.

A. Time–frequency domain analysis

For ideal waveforms, the following result was given in [5] without proof. Here, we show that
it can be obtained as a special case of Theorem 1.
Proposition 1: For ideal pulses, the following result can be obtained:

Y [n, m] = Hn,m [n, m]X[n, m] (13)

where Z Z
Hn,m [n, m] = h(τ, ν)ej2πνnT e−j2π(ν+m∆f )τ dτ dν.

Proof: From (10), we observe that the value of Hn,m [n′ , m′ ] is non-zero only at n′ = n
and m′ = m for the ideal pulses satisfying the bi-orthogonal property (12). Hence, the result
in (13) follows from (9) by considering only the term with n′ = n, m′ = m in the summations. 

B. Delay–Doppler domain analysis

1) Input–output relationship: We now apply SFFT on Y [n, m] in (13) to obtain the symbols
y[k, l] in the delay–Doppler domain. The following proposition, given in [5] without proof,
describes the input–output relation in delay–Doppler domain.
Proposition 2: For ideal pulses, the following input-output relation holds:
N −1 M −1
1 XX
y[k, l] = x[k ′ , l′ ]hw [k − k ′ , l − l′ ], (14)
NM ′ ′
k =0 l =0

where hw [·, ·] is a sampled version of the impulse response function:

hw [k − k ′ , l − l′ ] = hw (ν, τ )|ν= k−k′ ,τ = l−l′ (15)

NT M ∆f

for hw (ν, τ ) being the circular convolution of the channel response with the SFFT of a rectangular
windowing function in the time-frequency domain:
Z Z
hw (ν, τ ) = h(τ ′ , ν ′ )w(ν − ν ′ , τ − τ ′ )e−j2πντ dτ ′ dν ′ , (16)
 9

N
X −1 M
X −1
w(ν, τ ) = 1 · e−j2π(νnT −τ m∆f ) . (17)
n=0 m=0

Proof: The proof is relegated to the Appendix B. 

2) Inter-Doppler interference (IDI) analysis: From (14), we can see that a received signal
y[k, l] is a linear combination of all the transmitted signals x[k ′ , l′ ], k ′ = 0, . . . , N − 1, l′ =
0, . . . , M − 1. Consequently, the input-output relation (14) can be represented as a linear system
with NM variables x[k ′ , l′ ]. Since N and M tend to be very large for practical OTFS systems, the
detection complexity can be prohibitive. In the following, by using (5) as the sparse representation
of the delay–Doppler channel, (14) reduces to a sparse linear system, where each received signal
can be approximately expressed as a linear combination of only a few transmitted signals. Such
sparsity will then be exploited in Section V to devise a low-complexity yet efficient iterative
detection algorithm based on message passing on the factor graph representation.
By substituting (5) and (17) into (16), we obtain:
P
X
hw (ν, τ ) = hi e−j2πνi τi w(ν − νi , τ − τi )
i=1
P
X
= hi e−j2πνiτi G(ν, νi ) F (τ, τi ), (18)
i=1

where we have denoted:

M
X −1
′
F (τ, τi ) , ej2π(τ −τi )m ∆f ,
m′ =0
N
X −1
′
G(ν, νi ) , e−j2π(ν−νi )n T .
n′ =0
′
Let us first evaluate F (τ, τi ) at τ = Ml−l
∆f
as:
  M X −1 ′
l − l′ j 2π (l−l ′ −l )m′ ej2π(l−l −lτi ) − 1
F , τi = e M τ i = j 2π (l−l′ −l ) (19)
M∆f m =0
′ e M τi
−1
recalling that lτi is the delay tap of i-th path with a delay τi defined in (6). From (19), we see
that 
  
M,
l − l′ [l − l′ − lτi ]M = 0,
F , τi =
M∆f 
0, otherwise,
where [·]M represents mod M operation, i.e., it evaluates to M for l′ = [l − lτi ]M and is zero
otherwise.
 10

Similarly, we can evaluate:

  ′
k − k′ ej2π(k−k −kνi −κνi ) − 1
G , νi
2π = . (20)
NT ej N (k−k −kνi −κνi ) − 1
′

′

Due to the fractional κνi , we can see that for given k, G k−k NT
, νi 6= 0, ∀n.
′

We will show that the magnitude of N1 G k−k NT
, νi has a peak at k ′ = k − kνi and decreases
as k ′ moves away from k − kνi . From (20), after some manipulations, we have:
 
1 k − k′ sin(Nθ)
G , νi = (21)
N NT N sin(θ)
π
where we set θ , N
(k − k ′ − kνi − κνi ). It can be easily shown that:
sin(Nθ) sin((N − 1)θ) cos(θ) + sin(θ) cos((N − 1)θ)
=
N sin(θ) N sin(θ)
N −1 1
≤ |cos(θ)| + . (22)
N N
Here, we used the inequality, | sin(Nθ)| ≤ N| sin(θ)|, which can be proven by induction. The
upper bound (22) is tight for small values of θ (when both sides are close to 1) and it has a
peak at the smallest value of θ when k ′ = k − kνi . As |θ| increases (due to k ′ moving away from
π
k − kνi ), the upper bound decreases with (approximate) slope of N
(k − k ′ − kνi − κνi ). Since
N is quite large in OTFS, the function decreases rapidly.
From the above analysis, we need to consider only a small number 2Ni + 1, for some Ni > 0,
′

of significant values of G k−k
NT
, νi in (20) around the peak k − kνi , i.e., [k − kνi − Ni ]N ≤ k ′ ≤
[k − kνi + Ni ]N , where Ni ≪ N. Using this approximation, we can now express the receive
signal y[k, l] in (14) as:
P
X [k−kνi +Ni ]N
X  ′ 
ej2π(k−kνi −k −κνi ) − 1
y[k, l]≈ 2π hi e−j2πνi τi x [k ′ , [l − lτi ]M ]
Nej N (k−kνi −k −κνi ) − N
′
i=1 k ′ =[k−kνi −Ni ]N

P
X Ni 
X 
ej2π(−q−κνi ) − 1
≈ j 2π (−q−κνi )
hi e−j2πνi τi x [[k − kνi + q]N , [l − lτi ]M ] . (23)
i=1 q=−Ni Ne N −N
In the simulation result section, we will demonstrate that for N = 128, by choosing Ni = 10,
no performance loss is incurred. From (23), we can see that the received signal y[k, l] is a linear
P
combination of S = Pi=1 2Ni + 1 transmitted signals. Out of 2Ni + 1 transmitted signals in i-th
path, the signal corresponding to q = 0, x [[k − kνi ]N , [l − lτi ]M ], contributes the most and all
the other 2Ni signals can be seen as interferences. Such interferences are due to the transmitted
signals that are neighbor to x [[k − kνi ]N , [l − lτi ]M ] in the Doppler domain and we refer to this
 11

interference as inter-Doppler interference (IDI). Further, the number of transmitted signals S

affecting the received signal y[k, l] in (23) is much smaller than NM in (14). Hence, the graph
(or linear system) describing (23) is sparsely-connected.
3) Special channel model cases: The above input-output expression simplifies for the follow-
ing special cases.
i) Ideal channel: Assuming h(τ, ν) = δ(τ )δ(ν), the received signal becomes

y[k, l] = x[k, l]

and behaves as an AWGN channel as expected.

ii) No fractional Doppler (i.e., κνi = 0, ∀i): Assuming that Doppler frequencies exactly
coincide with Doppler taps, the received signal can be obtained by replacing Ni = 0 in (23),
i.e.,
P
X
y[k, l]= hi e−j2πνiτi x[[k − kνi ]N , [l − lτi ]M ].
i=1
For each path, the transmitted signal is circularly shifted by the delay and Doppler taps and
scaled by the associated channel gain.

IV. OTFS WITH RECTANGULAR WAVEFORMS

Since the ideal pulses cannot be realized in practice, we now analyze the OTFS with the
rectangular pulses at both the transmitter and receiver. These pulses do not satisfy the bi-
orthogonality conditions and generate some interference which degrades the system performance.
Here, we analyze the effect of such interference and show that it can be compensated to achieve
the ideal pulses performance.
√
We assume the rectangular pulse has amplitude 1/ T for t ∈ [0, T ] and 0 at all other values,
to have unit energy.

A. Time–frequency domain analysis

For the rectangular pulses, we can see that the cross-ambiguity term in the time–frequency
relation of Theorem 1, Agrx ,gtx ((n − n′ )T − τ, (m − m′ )∆f − ν) is non-zero for |τ | < τmax
|ν| < νmax only when n′ = n and n′ = n − 1, since gtx and grx are pulses of duration T and
τmax ≪ T . Hence, the time–frequency relation (9) becomes:
n
X M
X −1
Y [n, m] = Hn,m [n′ , m′ ]X[n′ , m′ ]
n′ =n−1 m′ =0
 12

M
X −1
= Hn,m [n, m]X[n, m] + Hn,m [n, m′ ]X[n, m′ ]
m′ =0,m′ 6=m
M
X −1
+ Hn,m [n − 1, m′ ]X[n − 1, m′ ]. (24)
m′ =0

The second term in (24) can be seen as the total interferences from the samples X[n, m′ ] at
different frequencies m′ 6= m but same time slot n as the current sample X[n, m]. On the other
hand, the third term in (24) accumulates the interferences from the samples X[n − 1, m′ ] in the
previous time slot n−1. Hence, we call the second and third terms as the inter carrier interference
(ICI) and inter symbol interference (ISI), respectively. The interferences depend on the delay τ
and Doppler ν of the channel. In particular, they are affected by the value of the cross-ambiguity
function Agrx ,gtx in Hn,m[n′ , m′ ]. In the following, we focus on the cross-ambiguity function for
ICI and ISI.
1) ICI analysis: Fix n, m. We note that the cross-ambiguity function in the Hn,m[n, m′ ], m′ 6=
m term of ICI, Agrx ,gtx (−τ, (m − m′ )∆f − ν), is independent of n, and is computed for the i-th
channel path with delay τi and Doppler νi (i.e., see (5)) as:
Z
′ ′
Aici , grx (t + τi )gtx (t′ )e−j2π((m−m )∆f −νi ))(t +τi ) dt′ .
∗ ′

We discard the dependency of Aici on (m, m′ , τi , νi ) for simplicity. Since the received signal r(t)
is sampled at intervals of T /M (or 1/(M∆f )), we can compute Aici as:
M −1−lτi
1 X
e−j2π((m−m )∆f −νi )( M ∆f +τi ) .
′ p
Aici = (25)
M p=0

Recall that the pulses gtx and grx have duration T , and lτi is the delay tap defined in (6). The
amplitude of Aici is
M −1−lτi
1 X ′ p
|Aici | = e−j2π((m−m )∆f −νi ) M ∆f
M p=0
kν +κν
  M −l
τi
−j2π m−m′ − i N i
e M
−1
=  kν +κν
 .
1
−j2π m−m′ − i N i M
Me −M

Similar to the analysis of (20), we can observe that |Aici | decreases as m′ moves away from
m. It implies that the ICI becomes less as the interfering subcarriers are further away from the
interfered subcarrier. We can also see that an increase in Doppler (i.e., kνi + κνi ) increases the
 13

number of neighboring subcarriers that interfere with the present subcarrier. This is similar to
the fractional Doppler effect studied for (20).
2) ISI analysis: Similar to the ICI analysis, the cross-ambiguity function in the Hn,m [n−1, m′ ]
term of ISI, Aisi , Agrx ,gtx (T − τ, (m − m′ )∆f − ν), is computed for the i-th channel path as:
M
X −1
1
e−j2π((m−m )∆f −νi )( M ∆f +τi −T ) .
′ p
Aisi = (26)
M p=M −lτi

The amplitude |Aisi | also has similar properties of |Aici |, where it reduces as m′ moves away
from m implying that the ISI is smaller for interfering symbols further away (in the frequency
axis) from the interfered symbol.
Note that the terms that affect the ICI and ISI in the summations (25) and (26) are mutually
exclusive, i.e., p = 0 to M − 1 − lτi contributes to ICI whereas p = M − lτi to M − 1 contributes
to ISI. This property helps in differentiating the ICI and ISI effects in delay–Doppler domain,
which will be studied below.

B. Delay–Doppler domain analysis

We now characterize the input–output relation in delay–Doppler domain for OTFS with
rectangular pulses.
Theorem 2: The received signal y[k, l] in delay–Doppler domain with the rectangular pulses
can be written as
P
X Ni
X νi +κνi
 l−l  k 
τi
j2π
y[k, l]≈ hi e M N
αi (k, l, q)x [[k − kνi + q]N , [l − lτi ]M ] (27)
i=1 q=−Ni

where we have:


 1 βi (q)
N
lτi ≤ l < M
αi (k, l, q) = (28)
 [k−k +q]
 1 (βi (q) − 1) e−j2π νNi N
N
0 ≤ l < lτi
ej2π(−q−κνi ) − 1
βi (q) = 2π . (29)
ej N (−q−κνi ) − 1
Proof: The proof is relegated to the Appendix C. 
Note that the approximation error in (27) is very small and it reduces by increasing N (see
(68) in Appendix C). Theorem 2 implies that the ICI and ISI in time–frequency domain are
converted to simple phase shifts in the delay–Doppler domain. Moreover, from (23) and (27),
we can observe that the number of transmitted signals that affects a received signal is the same
 14

for both ideal and rectangular pulse cases. The only difference is that the channel is shifted by
an additional phase that depends on the location of the transmitted signal in the delay–Doppler
plane (i.e., k and l)
Special channel model cases: Let us consider the above input-output expression (27) for the
special cases mentioned in Sec. III-B3.
i) Ideal channel: The received signal becomes

y[k, l] = x[k, l],

which is the same as the ideal pulses case since the rectangular pulses satisfy the bi-orthogonal
property in (12) when the channel is ideal (i.e., τmax = 0 and νmax = 0). This can be seen easily
by observing (1) at t = nT and f = m∆f .
ii) No fractional Doppler (i.e., κνi = 0, ∀i): Equation (27) simplifies to:
P
X  l−l
τi
k
νi
j2π
y[k, l]≈ hi e M N
αi (k, l)x[[k − kνi ]N , [l − lτi ]M ],
i=1

where


1 lτi ≤ l < M
αi (k, l) =  [k−k
νi ] N


 N −1 e−j2π N
0 ≤ l < lτi
N

In this case, IDI does not appear as in the case of ideal pulses.

V. M ESSAGE PASSING ALGORITHM FOR JOINT INTERFERENCE CANCELLATION AND

DETECTION

We now propose a message passing (MP) algorithm for OTFS using the input-output relation
in (23) (or (27)).

A. Low-complexity MP detection algorithm

The received signal in vectorized form can be written as

y = Hx+z (30)

where y = {y[d]}, z = {z[d]} ∈ CN M ×1 , H = {H[d, c]} ∈ CN M ×N M , and x = {x[c]} ∈ AN M ×1 .

The elements y, x, and H are determined from (23) (or (27)) and z is the noise vector. Due to
P
mod N and mod M operations in (23), we observe that only S = Pi=1 (2Ni + 1) elements out
of NM are non-zero in each row and column of H. Recall that P is the number of propagation
 15

y[d] y[e1 ] y[eS ]

2 2
(µd;e1 ; σd;e 1
) (µd;eS ; σd;e S
)
pc;e1 pc;eS

x[e1 ] x[eS ] x[c]

fe1 ; e2 ; · · · ; eS g = Id fe1 ; e2 ; · · · ; eS g = Jc

Observation node messages Variable node messages

Fig. 2. Messages in factor graph

paths. We can see that since S is much smaller than NM, H is a sparse matrix. Let I(d)
and J (c) denote the sets of indexes with non-zero elements in the d-th row and c-th column,
respectively, then |I(d)| = |J (c)| = S for all rows and columns. Note that although (30) applies
to both ideal pulses case in (23) and rectangular pulses case in (27), with different H’s, the
number of non-zero elements S in each row and column of H remains the same for both cases.
This condition helps in compensating ICI and ISI of rectangular pulses with the same complexity
detection algorithm of ideal pulses.
Based on (30), we model the system as a sparsely-connected factor graph with NM variable
nodes corresponding to x and NM observation nodes corresponding to y. In this factor graph,
each observation node y[d] is connected to the set of S variable nodes {x[c], c ∈ I(d)}. Similarly,
each variable node x[c] is connected to the set of S observation nodes {y[d], d ∈ J (c)}.
From (30), the joint maximum a posterior probability (MAP) detection rule for estimating the
transmitted signals is given by


b = arg max Pr x y, H ,
x
x∈ANM ×1

which has a complexity exponential in NM. Since the joint MAP detection can be intractable
for practical values of N and M, we consider the symbol-by-symbol MAP detection rule for
c = 1, . . . , NM


x
b[c] = arg max Pr x[c] = aj y, H (31)
aj ∈A

1

= arg max Pr y x[c] = aj , H (32)
aj ∈A Q
Y

≈ arg max Pr y[d] x[c] = aj , H . (33)
aj ∈A
d∈Jc
 16

In (32), we assume all the transmitted symbols aj ∈ A are equally likely and in (33) we assume
the components of y are approximately independent for a given x[c], due to the sparsity of H.
(i)
That is, we assume the interference terms ζd,c defined in (34) are independent for a given c.
In order to solve the approximate symbol-by-symbol MAP detection in (33), we propose a MP
detector which has a linear complexity in NM. Similarly to [12], for each y[d], a variable x[c]
is isolated from the other interference terms, which are then approximated as Gaussian noise
with an easily computable mean and variance.
In the MP algorithm, the mean and variance of the interference terms are used as messages
from observation nodes to variable nodes. On the other hand, the message passed from a variable
node x[c] to the observation nodes y[d], d ∈ J (c), is the probability mass function (pmf) of
the alphabet pc,d = {pc,d(aj )|aj ∈ A}. Fig. 2 shows the connections and the messages passed
between the observation and variable nodes. The MP algorithm is described in Algorithm 1.
The details of the steps in iteration i in the MP algorithm are detailed below.

Algorithm 1 MP algorithm for OTFS symbol detection

Input: Received signal y, channel matrix H.
(0)
Initialization: pmf pc,d = 1/Q, c = 1, · · · , NM, d ∈ J (c), iteration count i = 1.
repeat
(i) (i)
- Observation nodes y[d] compute the means µd,c and variances (σd,c )2 of Gaussian random
(i) (i−1)
variables ζd,c using pc,d and pass them to variables nodes x[c], c ∈ I(d).
(i) (i) (i) (i−1)
- Variable nodes x[c] update pc,d using µd,c , (σd,c )2 , and pc,d and pass them to observation
nodes y[d], d ∈ J (c).
- Compute convergence indicator η (i) .
- Update the decision on the transmitted symbols x
b[c], c = 1, . . . , NM if needed.
- i=i+1
until Stopping criteria;
Output: The decision on transmitted symbols x
b[c].

Message passings from observation nodes y[d] to variable nodes x[c], c ∈ I(d): The mean
(i) (i)
µd,c and variance (σd,c )2 of the interference, approximately modeled as a Gaussian random
 17

(i)
variable ζd,c defined as:
X
y[d] = x[c]H[d, c] + x[e]H[d, e] + z[d], (34)
e∈I(d),e6=c
| {z }
(i)
ζd,c

can be computed as:

Q
(i)
X X (i−1)
µd,c = pe,d (aj )aj H[d, e], (35)
e∈I(d),e6=c j=1

and
 2

Q Q
(i)
X X (i−1)
X (i−1)
(σd,c )2 =  pe,d (aj )|aj |2 |H[d, e]|2 − pe,d (aj )aj H[d, e]  + σ 2 . (36)
e∈I(d),e6=c j=1 j=1

Message passings from variable nodes x[c] to observation nodes y[d], d ∈ J (c): The pmf
(i)
vector pc,d can be updated as:
(i) (i) (i−1)
pc,d (aj ) = ∆ · p̃c,d(aj ) + (1 − ∆) · pc,d (aj ), aj ∈ A (37)

where ∆ ∈ (0, 1] is the damping factor used to improve the performance by controlling the
convergence speed [18], and
Y  
(i)
p̃c,d (aj ) ∝ Pr y[e] x[c] = aj , H
e∈J (c),e6=d
Y ξ (i) (e, c, j)
= PQ (i) , (38)
e∈J (c),e6=d k=1 ξ (e, c, k)
2
!
(i)
− y[e]−µe,c −He,c ak
(i)
where ξ (e, c, k) = exp (i) .
(σe,c )2

Convergence indicator: Compute the convergence indicator η (i) as

NM  
(i) 1 X (i)
η = I max pc (aj ) ≥ 1 − γ , (39)
NM c=1 aj ∈A

for some small γ > 0 and where

Y ξ (i) (e, c, j)
p(i)
c (aj ) = PQ (i) (40)
e∈J (c) k=1 ξ (e, c, k)

and I(·) is an indicator function.

Update decision: If η (i) > η (i−1) , then we update the decision of the transmitted symbol as

b[c] = arg max p(i)

x c (aj ), c = 1, · · · , NM. (41)
aj ∈A
 18

We update the decision on the transmitted symbols only when the current iteration can provide
better estimates than the previous iteration.
Stopping criteria. The MP algorithm stops when at least one of the following conditions is
satisfied.
1) η (i) = 1
∗ ∗)
2) η (i) < η (i ) − ǫ, where i∗ is the iteration index from {1, · · · , (i − 1)} for which η (i is
maximum
3) Maximum number niter of iterations is reached.
We select ǫ = 0.2 to disregard small fluctuations of η. Here, the first condition occurs in the
best case, where all the symbols have converged. The second condition is useful to stop the
algorithm if the current iteration provides a worse decision than the one in previous iterations.
Remark 2: Complexity of the proposed MP algorithm. The complexity of one iteration involves
the computation of (35), (36), (37), (39), and (41). More specifically, each of (35), (36), and
(37) 2 , has a complexity order O(NMSQ). Furthermore, (39) and (41) can be computed with
a complexity order O(NMQ) 3 . Therefore, the overall complexity order is O(niter NMSQ).
In simulations, we observed that the algorithm converges typically within 20 iterations (i.e.,
see Figure 4 in the illustrative result section for more references). We conclude that the IDI
analysis, which includes the smart approximation of IDI, to exploit the sparsity of the delay-
Doppler channel representation is a key factor in reducing the complexity of the detector (due
to relatively small S). The memory requirement is dominated by the storage of 2NMSQ real
(i) (i−1) (i) (i)
values for pc,d and pc,d . In addition, we have the messages (µd,c , (σd,c )2 ), requiring NMS
complex values and NMS real values, respectively.

B. Application of MP detection algorithm for OFDM over delay–Doppler channels

Later, we will compare the performance of OTFS and OFDM over delay–Doppler channels. In
this section, we demonstrate that it is also possible to apply the above MP algorithm to OFDM

2 (i)
In computing (38), first we find the pc (aj ) in (40) which requires O(N M Q) complexity and then we obtain (38) by dividing
(40) with the term related to e = d for all d, which requires O(S) complexity for each c. Hence, the over all complexity of
(38) becomes O(N M SQ).
3
The computation of (39) and (41) require to find the maximum element out of Q elements for each c. As (40) is already
computed for (38), finding the maximum element requires O(Q) complexity for each c, which leads to an overall complexity
of O(N M Q) to compute (39) and (41).
 19

to compensate the Doppler effects.

Consider OFDM system with OFDM symbol of duration T and M subcarriers. Hence, the
received signal and noise are sampled at T /M. Then, the frequency-domain signal after fast
Fourier transform (FFT) operation is given by:

y = WHt WH x + z (42)

where (·)H denotes Hermitian transpose, W is M-point FFT matrix, and x ∈ AM ×1 is the
transmitted OFDM symbol. The elements of time-domain channel matrix Ht = {Ht [p, q]} are
given in [19] as
P
X   
τi M 2π(q−1)νi
Ht [p, q] = hi δ p − q − ej M
i=1
T M

for p, q = 1, . . . , M. Using the M × M frequency-domain channel matrix Hofdm = WHt WH ,

we can re-write (42) as:

y = Hofdm x + z. (43)

Since (43) has the form similar to (30), the MP previously developed for OTFS can also be
applied for OFDM symbol detection. We note that Hofdm is diagonally dominant and the values
of off-diagonal elements in each row decay as we move away from the diagonal entry as explained
in [19]. Hence, the Hofdm matrix is also sparse enabling the use of the proposed low-complexity
MP detection algorithm.

VI. I LLUSTRATIVE RESULTS AND DISCUSSIONS

In this section, we simulate the uncoded bit-error-rate (BER) performance of OTFS and OFDM
over delay-Doppler channels. In particular, first we study the BER performance of OTFS for
ideal pulses with the number of interference terms due to IDI Ni and MP parameter ∆. Next, we
study the BER performance of OTFS with ideal pulses and rectangular pulses, and its comparison
with OFDM.
All relevant simulation parameters are given in Table I. For both OTFS and OFDM systems,
Extended Vehicular A model [20] is adopted as the channel model for the path delays, and the
Doppler shift of the i−th path is generated using

νi = νmax cos(θi ),
 20

Parameter Value
Carrier frequency 4 GHz
No. of subcarriers (M ) 512
No. of OTFS symbols (N ) 128
Subcarrier spacing 15 KHz
Cyclic prefix of OFDM 2.6 µs
Modulation alphabet 4-QAM
UE speed (Kmph) 30, 120, 500
Channel estimation Ideal
TABLE I
S IMULATION PARAMETERS

where θi ∼ U(0, π) is uniformly distributed. In order to obtain BER values, we consider 3 × 104
different channel realizations in the Monte-Carlo simulations.
We first demonstrate the effects of IDI in OTFS. Fig. 3 shows the BER performance of
OTFS system with ideal pulses using the proposed MP detector for different number of IDI
interference terms Ni with 4-QAM signaling over the delay–Doppler channel with different
Doppler frequencies (UE speeds of 120, 500 Kmph) and SNRs. Note that ICI and ISI are not
present for the ideal pulses case. We consider the same Ni for all paths. We can see that there is a
significant BER improvement when Ni increases from 0 to 10 and saturation thereafter. Note that
Ni = 0 corresponds to the case when IDI is not taken into account. The results imply that fewer
neighboring interference terms are sufficient to consider in the MP algorithm (e.g. Ni = 10)
without incurring performance loss. We also observe that if IDI is not taken into account at all
or an insufficient number of IDI terms is considered (i.e., Ni ≤ 5), the BER performance worsens
significantly. These observations demonstrate the importance of our previous IDI analysis. Also,
note that for SNR = 18 dB, the BER performances of OTFS at different Doppler frequencies are
similar. Later, we will demonstrate that our proposed MP algorithm can effectively compensate
for a wide range of channel Doppler variations.
In Fig. 4, we illustrate the BER and average number of iterations of OTFS system with ideal
pulses using the MP algorithm, when UE speed is 120 Kmph. We vary the damping factor ∆
for Ni = 10. We consider 4-QAM signaling and SNR = 18 dB. We observe that, when ∆ ≤ 0.7,
the BER remains almost the same, but deteriorates thereafter. Further, when ∆ = 0.7, the MP
algorithm converges with the least number of iterations. Hence, we choose ∆ = 0.7 as the
optimum damping factor in terms of performance and complexity.
 21

100
OTFS, 18 dB, 120 Kmph
OTFS, 18 dB, 500 Kmph
OTFS, 15 dB, 120 Kmph
10-1

4-QAM
10-2

BER
10-3

10-4

10-5
0 5 10 15 20
Ni

Fig. 3. The BER performance of OTFS for different number of interference terms Ni with 4-QAM.

100 45
OTFS, 120 Kmph OTFS, 120 Kmph
Average no. of iterations
40
10-1
35
10-2 4-QAM, SNR = 18 dB 4-QAM, SNR = 18 dB
30
BER

25
10-3
20
10-4
15
-5 10
10
0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1
∆ ∆

Fig. 4. The variation of BER and average no. of iterations with ∆.

In Fig. 5, we compare the BER performance of OTFS with ideal pulses and OFDM systems
using 4-QAM signaling over the delay-Doppler channels of different Doppler frequencies (UE
speeds of 30, 120, 500 Kmph). We observe that OTFS outperforms OFDM by approximately 15
dB at BER of 10−4 thanks to the constant channel gain over all transmitted symbols in OTFS,
whereas in OFDM, the overall error performance is limited by the subcarrier(s) experiencing
the worse channel conditions. Moreover, OTFS exhibits the same performance at different
Doppler frequencies thanks to the IDI cancellation provided by the MP detector. Similar behavior
applies to OFDM, since the ICI can be removed by the MP detector. We can conclude that the
performance of OTFS under the proposed MP algorithm is robust to Doppler variations and is
much better than that of OFDM.
Fig. 6 shows the BER of OTFS with rectangular pulses using 4-QAM signaling for two
scenarios: one with ICI and ISI cancellations (WC) and the other without (WO). In the second
 22

100
OTFS, Ideal, 30 Kmph
OTFS, Ideal, 120 Kmph
4-QAM OTFS, Ideal, 500 Kmph
10-1 OFDM, 30 kmph
OFDM, 120 kmph
OFDM, 500 kmph
10-2

BER
10-3

10-4

10-5
5 10 15 20 25 30
SNR in dB

Fig. 5. The BER performance comparison between OTFS with ideal pulses and OFDM systems at different Doppler frequencies.

100

10-1

10-2
BER

10-3 ×10-4
3.8
OTFS, Rect., WC, 30 Kmph
OTFS, Rect., WC, 120 Kmph
OTFS, Rect., WC, 500 Kmph
10-4 OTFS, Rect., WO, 30 Kmph
OTFS, Rect., WO, 120 Kmph
3.795 OTFS, Rect., WO, 500 Kmph
OTFS, Ideal
14.2 14.3 14.4 OFDM, 500 kmph
10-5
5 10 15 20 25 30
SNR in dB

Fig. 6. The BER performance of OTFS with rectangular and ideal pulses at different Doppler frequencies for 4-QAM.

100
OTFS, Rect., WC, 120 Kmph
OTFS, Rect., WO, 120 Kmph
16-QAM OTFS, Ideal
OFDM
10-1
BER

10-2

10-3

10-4
10 15 20 25 30 35
SNR in dB

Fig. 7. The BER performance of OTFS with rectangular and ideal pulses for 16-QAM.
 23

scenario, we observe that OTFS with rectangular pulses present an error floor incurred by the
ICI and ISI. The performance degradation becomes more severe at high Doppler (e.g., 500
Kmph) due to large ICI and ISI. On the other hand, OTFS with rectangular pulses approaches
the BER performance of OTFS with ideal pulses, when ISI and ICI are mitigated. Moreover,
we can see that the proposed MP algorithm can effectively remove ISI and ICI and thus OTFS
performance remains almost constant regardless of the Doppler frequencies. These results show
that it is possible to achieve the performance of OTFS with ideal waveforms under any Doppler
frequencies even with the more practical rectangular waveforms by using our MP algorithm
together with appropriate IDI, ICI and ISI cancellation. Last, we can see that while OTFS
performance is not affected by (high) Doppler values, OFDM performance incurs error floor on
channels with high Doppler frequencies.
In Fig. 7, we compare the BER performance of OTFS and OFDM at a Doppler of 120 Kmph
using 16-QAM signaling. We observe that OTFS with ICI and ISI cancellation outperforms
OFDM by 11 dB at BER = 10−3 . We also simulate OTFS at different Doppler frequencies of
30 and 500 Kmph and we observe the BER performances are similar to that of 120 Kmph.

VII. C ONCLUSION

In this paper, we have analyzed the input–output relation describing OTFS mod/demod over
delay–Doppler channels. We have studied in detail the cases of ideal waveforms and rectangular
waveforms. In particular, we have characterized the inter-Doppler interference (IDI), inter-carrier
interference (ICI), and inter-symbol interference (ISI) using sparse representation of the channel
in the delay–Doppler domain. A low-complexity yet efficient message passing (MP) algorithm for
joint IC and symbol detection was proposed, which is suitable for large-scale OTFS with inherent
channel sparsity. In the MP algorithm, the ISI and ICI can be canceled by using appropriate
phase shifting, while the IDI can be mitigated by accounting for a small number of significant
interference terms only. The proposed MP algorithm can effectively compensate for a wide
range of channel Doppler spreads. Moreover, we have demonstrated that it is possible to achieve
the performance of OTFS with ideal yet non-realizable waveforms using practical rectangular
waveforms. Through simulations, we have shown that OTFS has significant error performance
gains over OFDM under various channel conditions.
 24

ACKNOWLEDGEMENT

This research work is support by the Australian Research Council under Discovery Project
ARC DP160100528. Simulations were undertaken with the assistance of resources and services
from the National Computational Infrastructure (NCI), which is supported by the Australian
Government.

A PPENDIX A
P ROOF OF T HEOREM 1: OTFS I NPUT–O UTPUT R ELATION IN T IME –F REQUENCY D OMAIN

The received signal after Wigner transform Y (t, f ), from (4), can be written as in (44). It can
be further expanded as in (45) and (46) using the transmitted signal s(t) in (3) and some re-
ordering of summations and integrations. Therefore, the sampled version of Y (t, f ), i.e., Y [n, m],
can be written as
N
X −1 M
X −1
Y [n, m] = X[n′ , m′ ]Hn,m [n′ , m′ ],
n′ =0 m′ =0

where Hn,m [n′ , m′ ] is given in (47). By applying the change of variable t′ − τ − n′ T → t′′ in the
inner integral and some simple algebraic calculations, we can write Hn,m [n′ , m′ ] as in (48) and
(49), respectively. Finally, we obtain Hn,m [n′ , m′ ] as in (50), by replacing the square bracket in
(49) with cross-ambiguity function in (1), which completes the proof.

Z Z Z 
j2πν(t′ −τ ) ′
Y (t, f ) = ∗ ′
grx (t − t)
h(τ, ν)s(t − τ )e dτ dν e−j2πf (t −t) dt′
′
(44)
t′ τ ν
Z Z Z (N −1 M −1 )
XX ′ ′ ′
= ∗ ′
grx (t − t) h(τ, ν) X[n′ , m′ ]gtx (t′ − τ − n′ T )ej2πm ∆f (t −τ −n T )
t′ τ ν n′ =0 m′ =0

j2πν(t′ −τ ) ′
e dτ dν e−j2πf (t −t) dt′ (45)
N
X −1 M
X −1 Z Z Z
′ ′ ′
= ′
X[n , m ] ′
h(τ, ν) ∗ ′
grx (t − t)gtx (t′ − τ − n′ T )ej2πm ∆f (t −τ −n T )
n′ =0 m′ =0 τ ν t′
 
j2πν(t′ −τ ) −j2πf (t′ −t) ′
e e dt dτ dν (46)

Z Z Z
′ ′ ′ ′
′
Hn,m [n , m ] = ′
h(τ, ν) ∗ ′
grx (t − nT )gtx (t′ − τ − n′ T )ej2πm ∆f (t −τ −n T ) ej2πν(t −τ )
τ ν t′

−j2πm∆f (t′ −nT ) ′
e dt dτ dν (47)
 25

Z Z Z
′ ′′ ′′ +n′ T )
= h(τ, ν) grx (t − (n − n′ )T + τ )gtx (t′′ )ej2πm ∆f t ej2πν(t
∗ ′′
τ ν t′′

−j2πm∆f (t′′ +(n−n′ )T +τ ) ′′
dt dτ dν e (48)
Z Z Z 
∗ ′′ ′ ′′ −j2π((m−m′ )∆f −ν)(t′′ −(n−n′ )T +τ ) ′′
= h(τ, ν) grx (t − (n − n )T + τ )gtx (t )e dt
τ ν t′′
′ ′ ′
ej2π(ν+m ∆f )((n−n )T −τ ) ej2πνn T dτ dν (49)
Z Z
′ ′ ′
= h(τ, ν)Agrx ,gtx((n −n′ )T −τ, (m −m′ )∆f −ν)ej2π(ν+m ∆f )((n−n )T −τ ) ej2πνn T dτ dν (50)
τ ν

N −1 M −1
"N −1 M −1
#

1 XX XX nk′ ml′ nk ml
y[k, l] = Hn,m [n, m] x[k ′ , l′ ]ej2π N − M e−j2π N − M (51)
NM n=0 m=0 k ′ =0 l′ =0
N −1 M −1
"N −1 M −1

#
1 XX X X −j2πnT k−k′
j2πm∆f l−l′
= x[k ′ , l′ ] Hn,m [n, m]e NT e M ∆f (52)
NM k′ =0 l′ =0 n=0 m=0
N −1 M −1
1 XX
= x[k ′ , l′ ]hw [k − k ′ , l − l′ ]. (53)
NM ′
k =0 l =0
′

N
X −1 M
X −1 Z Z 
j2πν ′ nT −j2π(ν ′ +m∆f )τ ′
hw (ν, τ ) = ′ ′
h(τ , ν )e e dτ dν e−j2πnT ν ej2πm∆f τ
′ ′
(54)
n=0 m=0 τ′ ν′
Z Z "N −1 M −1 #
XX ′ ′ )m∆f ′ ′
= h(τ ′ , ν ′ ) e−j2π(ν−ν )nT ej2π(τ −τ e−j2πτ ν dτ ′ dν ′ (55)
τ′ ν′ n=0 m=0
Z Z
′ ′
= h(τ ′ , ν ′ )w(ν − ν ′ , τ − τ ′ )e−j2πτ ν dτ ′ dν ′ . (56)
τ′ ν′

"
N −1 M −1 M −1 M −1
#
1 XX X ′ ′
X
′ ′
y[k, l] = √ Hn,m [n, m ]X[n, m ] + Hn,m[n − 1, m ]X[n − 1, m ]
NM n=0 m=0 m′ =0 m′ =0


−j2π nk − ml
e N M (57)
N −1 M −1 M −1
"N −1 M −1
#

1 XX X ′
X X
′ ′ j2π nk
′ ′ ′
− mMl −j2π nk − ml
yici [k, l] = Hn,m [n, m ] x[k , l ]e N e N M
NM n=0 m=0 ′
m =0 k =0 l =0
′ ′

N −1 M −1
"N −1 M −1 M −1

#
1 XX XX X k−k′ ml−m′ l′
= x[k ′ , l′ ] Hn,m [n, m′ ]e−j2πn N ej2π M
NM ′ ′ n=0 m=0 ′
k =0 l =0 m =0
N
X −1 M
X −1
1
= x[k ′ , l′ ]hici ′ ′
k,l [k , l ] (58)
NM
k ′ =0 l′ =0
 26

N
X −1 M
X −1 M
X −1 Z Z
′
hici ′ ′
k,l [k , l ] = h(τ, ν)Agrx ,gtx (−τ, (m − m′ )∆f − ν)e−j2π(ν+m ∆f )τ ej2πνnT
n=0 m=0 m′ =0 τ ν



k−k′ ml−m′ l′
dτ dν e−j2πn N ej2π M (59)
 
N −1 M −1 M −1 P M −1−lτi
1 X X X X X
=  hi e−j2π((m−m )∆f −νi )(p(T /M )+τi ) e−j2π(νi +m ∆f )τi ej2πνi nT 
′ ′

M n=0 m=0 m′ =0 i=1 p=0

k−k′ ml−m′ l′
−j2πn j2π
e N e M (60)
P
"N −1
#  M −1−lτi M −1
X X −j2πn
k−k′ −kνi −κνi 1 X p
j2π M
k
νi +κνi X m
= hi e N e N
e−j2π(p+lτi −l) M
i=1 n=0
M p=0 m=0
M
X −1 
′
j2π(p−l′ ) m
e M

m′ =0
P
X
= hi G ici (νi )F ici (τi , νi ) (61)
i=1

M −1−lτi
X p
j2π M
k
νi +κνi

ici
F (τi , νi ) = M e N
δ([p + lτi − l]M )δ([p − l′ ]M ) (62)
p=0

 
P M −1 M −1−l N −1
1X X X τi νi +κνi X
k 
p
j2π M
yici [k, l] = hi e N
δ([p + lτi − l]M )δ([p − l′ ]M ) G ici (νi )x[k ′ , l′ ]
N i=1 p=0
l′ =0 k ′ =0

P
" M −1−lτ Ni  j2π(−q−κν ) 
1 X X i νi +κνi X e i − 1
k 
p
j2π M
≈ hi e N
δ([p + lτi − l]M ) 2π
N i=1 p=0 q=−Ni ej N (−q−κνi ) − 1
#
x[[k − kνi + q]N , p] (63)

  
XP XNi
 1
 l−l  k +κ 
τ νi νi

 hi βi (q) e
j2π M i N
x [[k − kνi + q]N , [l − lτi ]M ] l ≥ lτi ,
yici [k, l]≈ i=1 q=−Ni N



0 otherwise.
(64)

N −1 M −1 M −1
"N −1 M −1
#

1 XX X XX (n−1)k′ ′ ′
− mMl nk
− ml
yisi [k, l] = Hn,m [n − 1, m′ ] x[k ′ , l′ ]ej2π N e −j2π N M
NM n=0 m=0 ′
m =0 ′ ′ k =0 l =0
 27

N −1 M −1
"N −1 M −1 M −1

#
1 X X −j2π k′ XX X k−k′ ml−m′ l′
= e ′ ′
N x[k , l ] Hn,m [n − 1, m′ ]e−j2πn N ej2π M
NM ′ ′ n=0 m=0 ′
k =0 l =0 m =0
N
X −1 M
X −1
1 k′
= e−j2π N x[k ′ , l′ ]hisi ′ ′
k,l [k , l ] (65)
NM
k ′ =0 l′ =0

P
"N −1
#  M −1
X X k−k′ −kν −κν 1 X j2π ( p−M
k
νi +κνi

M )
−j2πn i i
′ ′
hisi
k,l [k , l ] = hi e N e N

i=1 n=1
M p=M −lτi

M
X −1 M
X −1 
m ′
−j2π(p+lτi −l+M ) M j2π(p−l′ ) m
e e M

m=0 m′ =0
P
X
= hi G isi (νi )F isi (τi , νi ) (66)
i=1

P
" M −1 M −1
1 X X X j2π( p−M ) kνi +κνi 
yisi [k, l] = hi e M N
δ([p + lτi − l]M )δ([p − l′ ]M )·
N i=1 p=M −l
′ l =0 τi

N −1
#
X k′
G isi (νi )e−j2π N x[k ′ , l′ ] (67)
k ′ =0
 
XP M
X −1 N
X −1
1
 k +κ 
ν ν
e ( M ) N
p−M i i ′
j2π k
= hi  δ([p + lτi − l]M ) G isi (νi )e−j2π N x[k ′ , p]
N i=1
p=M −lτi k ′ =0

P
" M −1
1 X X  k +κ 
νi νi
M )
j2π ( p−M
≈ hi e N
δ([p + lτi − l]M )
N i=1 p=M −lτi
 

 
 #
 X Ni
[k−kν +q]N
N
X −1
k′

i
(βi (q) − 1) e−j2π N x[[k − kνi + q]N , p] − e−j2π N x[k ′ , p] (68)

 

q=−Ni ′
k ′ =0, 
k 6=[k−kνi +q]N ,q∈[−Ni ,Ni ]

P
" M −1 Ni
1 X X νi +κνi X
k 
[k−kν +q]N
M )
j2π ( p−M i
≈ hi e N
δ([p + lτi − l]M ) (βi (q) − 1) e−j2π N
N i=1 p=M −lτi q=−Ni
#
x[[k − kνi + q]N , p] (69)

 P N  

X X i
1 [k−kνi +q]N
 l−l  k +κ 
j2π M i
τ νi νi

 h (β (q) − 1) e−j2π N e N


i
N
i
 i=1 q=−Ni
yisi [k, l]≈ (70)

 x [[k − kνi + q]N , [l − lτi ]M ] l < lτi ,




0 otherwise.
 28

A PPENDIX B
P ROOF OF P ROPOSITION 2: OTFS I NPUT–O UTPUT R ELATION IN D ELAY–D OPPLER D OMAIN
FOR I DEAL P ULSES

The received signal y[k, l] for the ideal pulses, from (11) and (13), can be written as
N −1 M −1

1 XX −j2π nk − ml
y[k, l] = √ Hn,m [n, m]X[n, m]e N M .
NM n=0 m=0
By substituting the ISFFT equation from (2), y[k, l] can be expanded as in from (51) to (53).
k−k ′ l−l′
Here, hw [k − k ′ , l − l′ ] can be seen as the value of hw (ν, τ ) sampled at ν = NT
,τ = M ∆f
. The
value of hw (ν, τ ) can be obtained as from (54) to (56), by substituting Hn,m[n, m] from (10),
which completes the proof.

A PPENDIX C
P ROOF OF T HEOREM 2: OTFS I NPUT–O UTPUT R ELATION IN D ELAY–D OPPLER D OMAIN
FOR R ECTANGULAR P ULSES

We start with expanding y[k, l] in (11) using the Y [n, m] for rectangular pulses in (24) as in
(57). We write y[k, l] as

y[k, l] = yici [k, l] + yisi [k, l],

where yici [k, l] and yisi [k, l] contains the first term and the second term of the summation in
square brackets of (57), respectively. We analyze these ICI and ISI terms as below.
Analysis of yici [k, l]: The value of yici [k, l] can be written as in (58) using the ISFFT of X[n, m]
′ ′ ′
given in (2). Now, hici
k,l [k , l ] is expanded in (59) by using the Hn,m [n, m ] value in (10). This

can be further written as in (60) from the channel assumption in (5) and the cross-ambiguity
function in (25).
To write the expression in (60) to a simple form, let us separate the terms related to n, m, m′ ,
and p. The terms related to n are


k−k′
−j2πn
ζn = e N ej2πνinT
k−k′ −kν −κν


i i
= e−j2πn N

Here, we used the delay and Doppler taps defined in (6). Similarly, the terms related to m and
m′ are
m
ζm = e−j2πm∆f (p(T /M )+τi ) ej2πl M
 29

m
= e−j2π(p+lτi −l) M
′ ′ ′ m′
ζm′ = ej2πm ∆f (p(T /M )+τi ) e−j2πm ∆f τi e−j2πl M

′ m′
= ej2π(p−l ) M .

Finally, the terms related to p are

ζp = ej2πνi (p(T /M )+τi ) e−j2πνi τi

νi +κνi
k 
p
j2π M
=e N
.
′ ′ ici
Therefore, from the above terms, the value of hici
k,l [k , l ] can be written as in (61), where G (νi )

and F ici (τi , νi ) denote the terms in the first and second square brackets. The value of G ici (νi )
is the same as the one studied in (20) for ideal pulses case. Similar to the analysis of (19),
F ici (τi , νi ) can be written as in (62). Hence, by substituting (61) and (62) in (59), yici [k, l] can
be approximated as in (63). From (63), we can easily see that it is non-zero only if the following
conditions satisfied:

p = [l − lτi ]M and 0 ≤ p ≤ M − 1 − lτi .

These conditions are satisfied only if l ≥ lτi and p = l − lτi . Finally, with the conditions on l
and p, yici [k, l] can be obtained as in (64), where βi (q) is defined in (29).
Analysis of yisi [k, l]: Similar to yici [k, l] in (58), yisi [k, l] can be expanded as in (65). By
substituting the value of Hn,m[n − 1, m′ ] from (10), cross-ambiguity function in (26), and similar
′ ′ ′ ′
analysis of separating terms for hici isi
k,l [k , l ], the value of hk,l [k , l ] can be obtained as in (66). Here,

the summation n starts from 1 as the first symbol does not have previous symbol to experience
ISI. Therefore, the value of G isi (νi ) is equal to G ici (νi ) − 1. Using the value of G isi (νi ), yisi [k, l]
can be approximated as in (68). Further, the expression in (68) can be approximated as in (69)
by neglecting the signals x[k ′ , p] for which k ′ 6= [k −kνi +q]N , q ∈ [−Ni , Ni ], as their coefficients
are very small (1/N) for practical values of N (e.g., N = 64, 128).
Now, (69) is non-zero only if the following conditions are satisfied:

p = [l − lτi ]M and M − lτi ≤ p ≤ M − 1.

These conditions are satisfied only if l < lτi and p = l − lτi + M. With these conditions, the
value of yisi [k, l] is written in (70).
Finally, by combining (64) and (70), the value of y[k, l] in (57) can be obtained as in (27),
which completes the proof.
 30

R EFERENCES

[1] T. Wang, J. G. Proakis, E. Masry, and J. R. Zeidler, “Performance degradation of OFDM systems due to Doppler spreading,”
IEEE Trans. Wireless Commun., vol. 5, no. 6, pp. 1422–1432, June 2006.
[2] G. Matz, H. Bolcskei, and F. Hlawatsch, “Time–frequency foundations of communications: Concepts and tools,” IEEE
Signal Process. Mag., vol. 30, no. 6, pp. 87–96, 2013.
[3] K. Liu, T. Kadous, and A. M. Sayeed, “Orthogonal time–frequency signaling over doubly dispersive channels,” IEEE Trans.
on Info. Theory, vol. 50, no. 11, pp. 2583–2603, 2004
[4] T. Dean, M. Chowdhury, and A. Goldsmith, “A new modulation technique for Doppler compensation in frequency-dispersive
channels,” available online: https://web.stanford.edu/∼trdean/papers/dcfdm.pdf
[5] R. Hadani, S. Rakib, M. Tsatsanis, A. Monk, A. J. Goldsmith, A. F. Molisch, and R. Calderbank, “Orthogonal time frequency
space modulation,” in Proc. IEEE WCNC, San Francisco, CA, USA, March 2017.
[6] Ronny Hadani and Anton Monk, “OTFS: A new generation of modulation addressing the challenges of 5G,” OTFS Physics
White Paper, Cohere Technologies, 7 Feb. 2018. Available online: https://arxiv.org/pdf/1802.02623.pdf
[7] F. Hlawatsch and G. Matz, Eds., Wireless Communications Over Rapidly Time-Varying Channels. New York, NY, USA:
Academic, 2011.
[8] W.C Jakes Jr.. Microwave Mobile Communications. Wiley, New York, 1974.
[9] R. Hadani et al. “Orthogonal Time Frequency Space (OTFS) modulation for millimeter-wave communications systems,” in
Proc. 2017 IEEE MTT-S International Microwave Symposium (IMS), Honololu, HI, USA, June 2017.
[10] Li Li et al., “A simple two-stage equalizer with simplified orthogonal time frequency space modulation over rapidly
time-varying channels,” available online: https://arxiv.org/abs/1709.02505
[11] A. Farhang et al., “Low complexity modem structure for OFDM-based orthogonal time frequency space modulation,”
available online: https://arxiv.org/pdf/1710.00655.pdf
[12] P. Som, T. Datta, N. Srinidhi, A. Chockalingam, and B. S. Rajan, “Low-complexity detection in large-dimension MIMO-ISI
channels using graphical models,” IEEE J. Sel. Topics in Signal Processing, vol. 5, no. 8, pp. 1497–1511, December 2011.
[13] P. Raviteja, K. T. Phan, Q. Jin, Y. Hong, and E. Viterbo, “Low-Complexity Iterative Detection for Orthogonal Time
Frequency Space Modulation,” submitted to IEEE Wireless Communications and Networking Conference, Barcelona, April
2018.
[14] W. Mecklenbrauker, A tutorial on non-parametric bilinear time–frequency signal rtepresentations, Time and Frequency
Representation of Signals and Systems (Eds. G Longo and B. Picinbono), vol. 309, pp. 11–68, 1989.
[15] D. N. C. Tse, P. Viswanath, Fundamentals of Wireless Communications. U.K., Cambridge: Cambridge Univ. Press, 2005.
[16] W. Kozek and A. F. Molisch, “Nonorthogonal pulseshapes for multicarrier communications in doubly dispersive channels,”
IEEE J. Select. Areas Commun., vol. 16, no. 8, pp. 1579-1589, Aug. 1998.
[17] G. Durisi, U. Schuster, H. Bolcskei, and S. Shamai, “Noncoherent capacity of underspread fading channels,” IEEE Trans.
Inf. Theory, vol. 56, no. 1, pp. 367-395, Jan. 2010.
[18] M. Pretti, “A message passing algorithm with damping,” J. Stat. Mech.: Theory and Experiment, P11008, Nov. 2005.
[19] Y. Zhao, and S. G. Haggman, “Sensitivity to Doppler shift and carrier frequency errors in OFDM systems-the consequences
and solutions,” in Proc. IEEE Vehicular Technology Conf., Atlanta, GA, USA, April 1996.
[20] E. LTE, “Evolved universal terrestrial radio access (E-UTRA); base station (BS) radio transmission and reception (3GPP
TS 36.104 version 8.6. 0 release 8), July 2009,” ETSI TS, vol. 136, no. 104, p. V8.