Sampling Data Transmission and The Nyqui
Sampling Data Transmission and The Nyqui
zyxw
zyxwvut
zyxwvutsr
zyxwvutsr
zyxwvutsrqp
zyxwvutsrqpo
Sampling, Data Transmission, and
the Nyquist Rate
H. J. LANDAU
Signnl.
zyxwvut
values produces only a correspondingly small error in the recovered
zyxwvutsr
implicit in the sampling theorem, make itnatnral to ask whether the above
rates cannotbe improved, by ppssing to differently chosensampling instam@
or to badpass or mdliband (rather than badimited) signak, or to more
In tbis paper wedraw a distiactioo between re-
combwthgasignalfromitssamples,anddoiisoinastibleway,andwe
argue that only stable sampliq is meaaingfal in practice. We then prove
that :
called the Nyquist rate.
In view of the. practical importance of these results, it is
natural to ask whether the Nyquist rate prescribed above
for stable sampling or for data transmission cannot some-
how be improved upon. Since 1)and 2) refer to a very special
situation-to signals which in frequency occupy only a
single band and to their values at regularly spaced instants-
1) stablesamplingcannotbeperformedataratelowerthanthe it is conceivable that signals might be recoverable from
Nyquist, their values taken at a lower rate, if the sampling instants
2) data cannot be transmitted as samples at a rate higher than the were chosen differently; or if the signals had their fre-
Nyquist,
quencies in a union of several bands; or at thecost of more
regardless of the locatioa of sampling instants, the nature of the set of fre-
computing than is required by the simple formula (1).Were
qwacies whicb the signals occupy,or the method of comhcth. These
this possible, we would have a more efficient reduction of
cwclmions apply not merely to fiuiteswrgy, but Pt90 to bomded, signals.
continuous signals to digital form, with a consequent saving
INTRODUCTION of bandwidth. Similarly, it is conceivable thatarbitrary
HE sampling theorem states that f(t) is a signal of square-summable numbers could be prescribed at a higher
finite energy bandlimited to W Hz if and only if rate as the sample values of a signal, if the sampling instants
were properly chosen, or if the carrying signals were allowed
sin 271W(t- k/2W) frequencies in an appropriate union of bands. Were this
f ( t )= & " 271W(t - k / 2W )
with 1 < cx) ; (1) possible, we would have a more efficientwayof sending
data over given bandwidth.
thereupon The purposeof this paper is to prove that stable sampling
rm of signals cannot be performed at less than the Nyquist
rate, nor can data be transmitted as samples at a greater
rate, regardless of the location of sampling instants, the
nature of the set of frequencies which the signals occupy,
or the method of computation. Although discussed in the
This may be read in two ways, each of which has found context of finite-energy signals, these results also apply to
important applications. bounded signals.
1) Every signal of finite energy and bandwidth W Hz
FORMULATION OF THE PROBLEM AND
may be completely recovered from a knowledge of its
STATEMENT OF RESULTS
samples taken at the rate of 2W per second. More-
over-indispensable for any implementation in prac- We consider a fixed set S of N-frequency bands whose
tice-the recovery is stable, in the sense that a small total finite length (counting negative as well as positive
frequencies) we denote by m(S), and the space g ( S ) of all
Manuscript received March 3,1967; revised June 12, 1967. signals of finite energy with frequencies contained only in
The author is with Bell Telephone Laboratories,Inc., Murray Hill, N. J. S . Specifically, we make the following definition.
zy
zyxwvutsr
1702
zyxwvutsrqp
zyxwvu
zyxwvuts
zyxwvuts
Dejinition: B(S) consists of all square-integrable func-
tions f ( t ) whose Fourier transform F(w)is supported on s,
zyxw
i.e., F(w)= 0 for o 4 S.
PROCEEDINGS OF THE IEEE, OCTOBER 1967
zyxwv
positive distance d, and that the interpolation can be per-
We have in the past called such signals multiband. formed in a stable way, i.e., the function f ( t ) E B ( S )can be
We start by considering those sets of points t= {tk} at chosen so that
which the values of a signal f ( t ) E B ( S )can be used to re-
construct the signal everywhere. We assume from the outset
that these points separated
are by at least some positive
jrnlf(t)12
-m
dt 1 lak12
zyxwvu
they coincide identically. Writing this in terms of the func-
tion h(t)=f(t) - g(t), we require : to the density, on the t-axis, of sets of sampling or inter-
polation. The facts are as follows.
h E B(S), and h(tk)= 0 for all k, implies h(t) = 0. (2) 1) There exist sets of uniqueness for B(S) having arbi-
trarily low densityf2!This is occasionally reported as a
We will call a set { t k } satisfying (2) a set of uniqueness for sampling result with practical implications, but such
W). an interpretation is somewhat misleading, in view of
If ( t t } is a set of uniqueness for B(S),the samples of the lack of stability. When S is a single interval, so that
f € B ( S )taken at the instants t = { t k } do indeed determine the WS) consists of all signals of prescribed bandwidth, a
signal, but this initself does notconstitute a sampling remarkable result concerning sets of uniqueness is
theorem adequate in practice. The reason is that we know available : given a set of points { t k } , Beurling and
nothing about the effect on the reconstruction process of Malliavin[’l have shown how to determine explicitly
zy
an error in measuring the sample values. It may very well the least upper bound of those bandwidths for which
be thatanarbitrarily small misreading of thenumbers {tk} is a set of uniqueness.
f ( t k )will produce an arbitrarily large error in the recovered 2) If {tk} is a set of stable sampling for then every
function. To guard against this, we must introduce an addi-
tional requirement of stability : small errors in reading the
interval of length r must contain at least (rm(S)(27r)- ’
- A log+ r - B) of the points of {tk}, with A and B
samples shall produce no more than correspondingly small appropriate constants.’ Hence the density of {tk} is at
errors in the recovered function, or, in terms of the differ- least m(S)/2n. Stable sampling cannot be performed at
zyx
ence between the true andreconstructed signal, if the values lower than the Nyquist rate.
of a function ~ E B ( at S )the points { t r } are small, h(t) shall 3) If {tk} is a set of interpolation for B(S),no interval of
itself be small. There are many ways of measuring the size ’
length r can contain more than (rrn(S)(27z)- + C log’ r
of a signal ; we choose energy and put our condition as +D ) of the points of {tk}, with C and D appropriate
follows. constants. Hence the density of {tk} is at most m(S)/27r.
Dejinition : The set of points { tk} is a set of stable sampling Data cannot be sent as samples at higher thanthe
for B(S)if there exists a constant K such that Nyquist rate.
Wewill indicate the proofs of statements 2 ) and 3) and
close with some remarks concerning these problems for
bounded, rather than finite-energy, functions.
for all f ( t ) E B ( S ) .
We can now see that a set of stable sampling is always a DISCUSSION
AND PROOFS
set of uniqueness, whereas the reverse is not true. More- We begin with an intuitive explanation of the proof. In
over, our definition does not commit us to any particular what follows wewilluse the term “sampling” to mean
recovery scheme,and so describes the most general circum- exclusively “stable sampling.”
stances in which sampling can be performed in practice. Suppose { t k } is a set of sampling for B(S) and I is an
Takingup next the problem of transmitting data as interval containing n ofthe points of {tk}.Supposef(t)EW(S)
samples, we consider instants t = {tk} at which square- has little of its energy outside I ; we will referto such a func-
summable values can be prescribed arbitrarily for a signal tion as being well concentrated on I . If we can presume that
f ( t ) E B ( S ) Here
. too we formulate the matterindependently the samples f ( t k ) at points tk outside Z are also small, then,
of any method of computation. by virtue of our discussion of stability, replacing these
Dejinition : The set of points { tk} is a set of interpolation sample values by zero should not too drastically affect our
for W ( S )if, given any set of numbers {ak} with C lak12< co,
there exists f ( t ) E g ( S )with f(tk)=ak. With log’ r = max (log r, 0).
SAMPLING,
LANDAU:
zyxwv
TRANSMISSION,
DATA z
zyxwvutsrq
reconstruction of f ( t ) . We conclude that a function well
concentrated on I can be substantially determined by the
AND NYQUIST RATE 1703
cerns us here.
zyxwvu
nection with the sampling and interpolation which con-
zyxwvut
If S consists of intervals of length Zi, we find, after some
manipulation,
zyxwvutsrqpo
With I an arbitrary given set, we begin by determining
that function of g ( S ) which is most concentrated on I , i.e.,
that f ~ g ( Sfor
) which the concentration
fIf (t)I2 dt
N
-
1
n2
log+ zir - 1
Irn
i= 1
(3)
A = I f(t)I2 dt m(S)r
2--
271
A log+ r - B,
I zyxwvuts
-m
achieves its largest value. This is a standard problem of with constants A and B depending only on S . Thus, from
maximizing a quadratic form and leads immediately to the ( 5 ) and (8),
eigenvalue equation 1k j.k(r, s){ 1 - ik(r, s)} A log+ + B. (9)
nk4k(t) =- $k(x)w(t - dx x) , (4) It follows from (9) that the numberof eigenvalues &(r, S )
& I
lying between any two fixed bounds 6 > 0 and y c 1 can
in which the Fourier transform of w is the characteristic grow at most logarithmically with r. Specifically, each such
function of the set S . The kernel being hermitian, positive eigenvalue contributes to the left-hand side of (9) an amount
zyxwvuts
definite, and square-integrable, (4) is of the Hilbert-Schmidt no smaller than a=min {ql- 6), fil- y ) } >O; hence the
type, so that eigenfunctions and eigenvalues do exist (see number of these eigenvalues cannot exceed Ala log' r + B/a.
zyxwvutsr
p. 242, Riesz and NagyI61); we denote the latter, in nonin- Taken together with ( 5 ) and (8), this observation shows that,
creasing order, by &(I, S), k=O, 1, 2, . . . , and by &(r, S ) if asymptotically in r, the number of lk(r, S ) near 1 behaves
I is a single interval of length r. like rn(S)(271)-'r. But we can sharpen this statement by
By definition, i j -l(r, S ) is the largest concentration on an means of the following two lemmas (proved in the Ap-
interval of length r achievable by a function of W ( S )which pendix), whichserve as well to establish the connection
is orthogonal to the j - 1 eigenfunctions 4 0 ( x ) ,. . . , 4 j - 2 ( x ) between the eigenvalue problem and sampling and inter-
zyxwvutsr
previously determined ; intuitively, then, I.j- l(r, S ) repre- polation.
sents the concentration of the jth most concentrated func-
tion. Our inquiry intothenumber of independent well Lemma 1
concentrated functions therefore leads us to consider the Let S be bounded and {tk} be a set of sampling for g ( S ) ,
behavior of the sequence {ik(r, S ) } as a function of r. There whose points are separated by at least d >O. Let I be any
are two simple consequences of (4) whichyield a lot of compact set, I + be the set of points whose distance to I is
information :
1
m
&(r, S ) = trace =
less than d/2, and n(Z+) be the number of points of (tk}
contained in I +. Then &+)(I, S)I y < 1, where y depends
on S and (tk} but not on I .
k= 0 2n Lemma 2
30
1 I.z(r, S ) = double integral of (kernel12. Let S be bounded and {tk} be a set of interpolation for
(6)
k=O g ( S ) , whose points are separated by at least d >O. Let I be
any compact set, I - be the set of points whose distance to
To evaluate the latter most easily, we observe that (4) may the complement of I exceeds d/2, and n ( I - ) be the number
be converted to other equationswithout changmg the eigen- of points of { tk} contained in I - . Then An(l-) - l(r, S )2 6 >0,
values. One such transformation, obtained by introducing where 6 depends on S and {tk} but not on I .
into (4) the definition of w, multiplying both sides by e-iz", We have already remarked that the eigenvalues A k ( z , S )
and integrating over I , gives remain unchanged when we reverse the roles of the time and
zy
zyxwvutsrqponm
zyxwvu
1704
zyxwvutsrq
zyxwvutsrq
zyxwvutsr
frequency domains. It is also easy to see from (4) that they
are unaffected by a rescaling of time and frequency which
replaces Z by aZ and S by a- ' S , with a any constant.2 By
applying both of these permissible transformations, we con-
clude that i k - l(r, S ) is also the kth eigenvalue p f the prob-
lem of concentrating functions whose frequencies lie in a
single interval of length 2n (i.e., bandlimited functions) on
PROCEEDINGS OF THE IEEE,OCTOBER
zyxwvutsrqp
ber of these points contained in S + is at most (2n)-'rrn(S) so that
+ 2N, and their number in S - is at least (2n)-'rm(S)- 2 N .
zyxwvut
Thus Lemmas 1 and 2 establish the existence of absolute
constants yo and 6, such that
4(2x)-'rm(s)1+2N(r,S ) Yo < 1,
i[(2x)-'rm(S)]-2N-1(r, 2 > O;
here we use the notation [x] to denote the integer part of
x. It follows from their construction that y o and do may be
taken to be 3, but we will not prove this here.
(10)
(11)
(n(Z) - 3) - ([rm(S)(2n)-'] + 2 N ) < C'log' r
whence
n(Z) 5 rm(S)(2n)-'
Theorem 2 is established.
+ Clog' r + D.
+ D',
-m
If(')12
dt I 1 lakl2 = 1lf(tk)12.
zyxwvu
some positive distance d, and the interpolation can be per-
1zyxwvut
Proof: By writing f € W ( S ) as the inverse transform of its
This shows that when interpolation is performed in d?(S),it
is necessarily stable. Proposition 1 is established.
zyxw
Fourier transform, and applying Schwarz's inequality and Lemma 1
Parseval's theorem to this representation, we find
zyxwvutsrqp
Let S be bounded and { t k } be a set of sampling for B(S),
whose points are separated by at least d >O. Let I be any
IK, lf(t)I2 dt, (15) compact set, I + be the set of points whose distance to I
-m is less than 4 2 , and n(I+ ) be the number of points of { t k }
zyxwvutsrqp
Ipk-qkI I l / k 2 , k = 1 , 2 , . . . . Now let us prescribe the values with ck any subspace of dimension k. Thus to prove Lemma
l / k at t = p k , - l / k at t = 4 k , and 0 at every remaining point 1 , it is sufficient to show that we can keep the concentration
of { t k } ; since these values are square-summable, and { t k } is of a function of B(S)over I bounded away from 1 by im-
a set of interpolation, they will be assumed by some posing n(I +) orthogonality conditions. The following
simple device accomplishes this. Let h b ) be a square-
f(t)Eg(S).But then 2 / k = f b k ) - f ( q k ) 5 l p k - qkl lf'(sk)l with
sk apoint lyingbetween p k and q k . Hence I f ' ( s k ) l > 2 k , integrable function which vanishes for ly[ > d / 2 and whose
k = 1 , 2, . . . , contradicting ( 1 6 ) . This establishes the first Fourier transform H ( o ) satisfies IH(o)l> 1 for OES.Such a
part of the proposition. function certainly exists, since the transforms of functions
zyxwvutsrqpon
To prove the second part, we appeal toan abstract vanishing on lyl> d / 2 can be made to approximate uniformly
principle. In W(S), let usview functions as vectors and an arbitrary continuousfunction on any bounded set. Now
energy as the square of length ; this makes B(S) into a given f€&(S), we form
Hilbert space. Let d?'(S) be the subspace of g ( S ) consisting
of those functions which vanish at all the points { t k } . This
subspace is closed, since, by ( 1 5), convergence in energy of
functions of to a function h(t) implies uniform con- 1 r
zyxwvutsrq
-00
energy.
Now let us consider the transformation T which assigns and by Parseval's theorem
to a square-summable sequence { a t } that function of 8 ( S )
r m r r
which interpolates to ak at t = t k . By hypothesis, T is de-
fined on the whole space 1, of square-summable sequences,
maps 1, into the Hilbert space d?(S), and, in view of ( 1 5 ) ,
has a closed graph. Then by the closed graph theorem (see
Loomi~[~], p. 1 8 ) T is bounded, i.e., there exists a constant
= Im -m
Ig(t)12d t .
1706
zyxwvzyxw
zyxwvutsrq
zyxwvutsrq OF THE IEEE, OCTOBER
PROCEEDINGS 1967
--a:
zyxw
zyxwvutsrqpo
Now subjecting f to the n(Z +) orthogonality conditions
f ( y ) h ( t k - y ) d y = 0, tk E I+ 9
zyxwvutsr
(19) we find to see that these exist, we need only take the Fourier trans-
form of (24) and note that I H ( o ) l 2 1 for OES.The $’s are
likewise linearly independent, so that, letting C be the sub-
space of W ( S ) spanned by the functions +k with t k in I - ,
we obtain a subspace of dimension n(I -).
or Now given ~ E Cwe , form the function g of (18). In view
of (24), g is a linear combination of the $ k , hence lies in
j
zyxwvutsrq
zyxwvu
If (t)12 d t &(S) and satisfies (21).However, since the only q i k figuring
I < l - - =1y < l . in g are those with t k in I - , the values of g necessarily
jm If @)I2 d t
--a:
Kc vanish at every other t k , so that
I-m
zyxwvu
zyxwvu
Let S be a bounded set and { t k } be a set of interpolation integrals in (19) are extended over non-overlapping in-
for W(S),whose points are separated by at least d >O. Let tervals, each contained in I , so that
I be any compact set, I - be the set of points whose distance
to the complement of I exceeds d/2, and n(Z-) be the num-
ber of points of {t,} contained in I - . Then A,,,- ) - l(Z, S )
2 6 >0, where 6 depends on S and {t,} but not on I .
Proof: We again begin with the Weyl-Courant lemma in
the form
I-
J-m
with ck any subspace of dimension k. Any proof of (17) can which, applied to (22), proves Lemma 2. As in Lemma 1,
be modified to yield(22), but the following independent the constant 6 does not depend on I.
argument serves as well. Let L k - l be the subspace of B(S)
spanned by &,,. . . ,dk-2,the first k - 1 eigenfunctions. REFERENCES
Since the dimension of Lk-lis lower than that of ck, any [‘I A. Beurling and P. Malliavin, “On the closure of characters and
linear transformation of C, into & - I , in particular the zeros of entire functions,” Acta Mathematica, vol. 118, pp. 79-93, May
orthogonal projection onto L,-,, must annihilate some 1967.
H . J. Landau, “A sparse regular sequence of exponentials closed
nonzero element of ck. Thus Ckcontains a function ff0 on large sets,” Bull. Am. Math. Soc.,vol. 70, pp. 5 6 5 6 9 , July 1964.
orthogonal to Lk- and (22) follows from the definition of -, “Necessary density conditions for sampling and interpolation
of certainentire functions,” ActaMathematica, vol. 117, pp. 37-52,
‘ k - 1. February 1967.
We have shown in Proposition 1 that it is possible to [41 H. J. Landau andH. 0.Pollak, “Prolate spheroidal wave functions,
perform the interpolation in a subspace &(S) of a(S),in Fourier analysis and uncertainty.111,” Bell Sys. Tech.J., vol. 41, pp. 1295-
which every function g ( t ) satisfies 1336, July 1962.
c5] L. H. Loomis, An Introduction toAbstract Harmonic Analysb.
m New York: Van Nostrand, 1953.
F. Riesz and B. S. Nagy, Functional Analysis. New York: Ungar,
(23) 1955.
-m