(Mathematics and Its Applications 26) Shiryayev A. N. - Selected Works of A.N. Kolmogorov. Volume II - Probability Theory and Mathematical Statistics-Springer (1992)

Selected Works of A. N.
Kolmogorov
Mathematics and Its Applications (Soviet Series)
Managing Editor:
M. HAZEWINKEL
Centre Jor Mathematics and Computer Science, Amsterdam, The Netherlands
Editorial Board:
A. A. KIRILLOV, MGU, Moscow, U.S.S.R.

Yu. I. MANIN, Steklov Institute oJ Mathematics, Moscow, U.S.S.R.
N. N. MOISEEV, Computing Centre, Academy oJ Sciences, Moscow, U.S.S.R.
S. P. NOVIKOV, Landau Institute oJTheoretical Physics, Moscow, U.S.S.R.
M. C. POL YVANOV, Steklov Institute oJ Mathematics, Moscow, U.S.S.R.
Yu. A. ROZANOV, Steklov Institute oJ Mathematics, Moscow, U.S.S.R.
Volume 26
Selected W orks of
A. N. Kolmogorov
Volume II
Probability Theory and
Mathematical Statistics
edited by
A. N. Shiryayev
TransJatedfrom the Russian by G. Undquist
..
SPRINGER-SCIENCE+BUSINESS MEDIA, B.V.
Library of Congress Cataloging-in-Publication Data
Kol.ogorov, A.. N. (Andre; Nlicolaevlchl. 1903-
(Teorl1a verOlätnostel 1 uteutlch8Silaiä statlsT\ka. EngllShl
PrObabilIty theory and lIatheutlCal sTZltlstlcs .' eClud by A.N .
Shlryayev: translated FrOIl the Russ!an by G. LlndqulSt.
p. eil. -- CSelected warks of A.N . Kal.agorav : v. 2)
fMathelUtlcs lind Its appltcatlons (Sovlet S,,-II$) ; V. 261
Translation of: Teor",fi veroiätnoste1 Illateutlchesk.fi
statlstlka.
ISBN 978-94-010-5003-6 ISBN 978-94-011-2260-3 (eBook)
DOI 10.1007/978-94-011-2260-3
1. Probab,11tie:; . 2 . Hnh91U'!lcal !ta~lstlcs . 1. Shu' iäe\',
Al 'bllrt Nl kolaev l ch. 11. lltle. !Il. Serlts : Koillogcrov . A. N.
IAndrel Ntkolaevlch ). 1903- Selectlons . Engllsh. 1991 ; v. 2.
IV . Serie. : MathellltlC$ ,nd tu appllcatiOns (Klu"." Aeade.le
Publ1shersJ. SOvlet ser-Ies : 26.
QA.3.K792513 1991 v. 2
[0'273.1S1
510 s--dc20
[519.2J 91-14856
CIP
ISBN 978-94-010-5003-6
ISBN 978-94-010-5003-6
Printed on acid-free paper
This is an annotated translation from the original work

TeopHJI BepOJlTHOcTe:ß H MaTeMaTH'ieCKaJl CTaTHCTHKa
Published by Nauka, Moscow, © 1986
All Rights Reserved

© 1992 Springer Science+Business Media Dordrecht
Originally published by Kluwer Academic Publishers in 1992
Softcover reprint of the hardcover I st edition 1992
No part of the material protected by this copyright notice may be reproduced or
utilized in any fonn or by any means, electronic or mechanical,
including photocopying, recording or by any infonnation storage and
retrieval system, without written pennission from the copyright owner.
SERIES EDITOR'S PREFACE
lEt moi •...• si j'avait so comment en revenir, One service mathematics has rendered the
je n'y serais point alle:' human race. It has put common sense back
Ju1e. Veme where it belongs, 01\ the topmost shelf next
to the dUlty canister labelled 'discarded non-
Tbe series is divergent; therefore we may be sense'.
able to do something with it. Erie T. Bell
O. Heaviside
Mathematics is a tool for thought. A highly necessary tool in a world where both feedback and non-
linearities abound. Similarly, all kinds of parts of mathematics serve as tools for other parts and for
other sciences.
Applying a simple reweiting rule to the quote on the right above one finds such statements as:
'One service topology has rendered mathematical physics .. .'; 'One service logic has rendered com-
puter science .. .'; 'One service category theory has rendered mathematics .. .'. All arguably true. And
all statements obtainable this way form part of the raison d'tftre of this series.
This series, Mathematics and fts Applications, started in 1977. Now that over one hundred
volumes have appeared it seems opportune to reexamine its scope. At the time 1 weote
''Growing specialization and diversification have brought a host of monographs and
textbooks on increasingly specialized topics. However, the 'tree' of knowledge of
mathematics and related fields does not grow only by pUUing forth new branches. It
also happens, quite often in fact, that branches which were thought to be completely
disparate are suddenly seen to be related. Further, the kind and level of sophistication
of mathematics applied in various sciences has changed drastically in recent years:
measure theory is used (non-trivially) in regional and theoretical economics; algebraic
geometry interacts with physics; the Minkowsky lemma, coding theory and the structure
of water meet one another in packing and covering theory; quantum fields, crystal
defects and mathematical programming profit from homotopy theory; Lie algebras are
relevant to filtering; and prediction and electrical engineering can use Stein spaces. And
in addition to this there are such new emerging subdisciplines as 'experimental
mathematics', 'CFD', 'completely integrable systems', 'chaos, synergetics and large-scale
order', which are almost impossible to fit into the existing classification schemes. They
draw upon widely different sections of mathematics."
By and large, all this still applies today. It is still true that at first sight mathematics seems rather
fragmented and that to find, see, and exploit the deeper underlying interrelations more effort is
needed and so are books that can help mathematicians and scientists do so. Accordingly MIA will
continue to try to make such books available.
If anything, the description 1 gave in 1977 is now an understatement. To the examples of
interaction areas one should add string theory where Riemann surfaces, algebraic geometry, modu-
lar functions, knots, quantum field theory, Kac-Moody algebras, monstrous moonshine (and more)
all come together. And to the examples of things which can be usefully applied let me add the topic
'finite geometry'; a combination of words which soundslike it might not even exist, let alone be
applicable. And yet it is being applicd: to statistics via designs, to radar! sonar detection arrays (via
finite projective planes), and to bus connections of VLSI chips (via difference sets). There seems to
be no part of (so-called pure) mathematics that is not in immediate danger of being applied. And,
accordingly, the applied mathematician needs to be aware of much more. Besides analysis and
numerics, the traditional workhorses, he may need all kinds of combinatorics, algebra, probability,
and so on.
In addition, the applied scientist needs to cope increasingly with the non linear world and the
v
vi SERIES EDITOR'S PREFACE
extra mathematical sophistication that this requires. For that is where the rewards are. Linear
models are honest and a bit sad and depressing: proportional efforts and results. It is in the non-
linear world that infinitesimal inputs may result in macroscopic outputs (or vice versa). To appreci-
ate what I am hinting at: if electronics were linear we would have no fun with transistors and com-
puters; we would have no TV; in fact you would not be reading these lines.
There is also no safety in ignoring such outlandish things as nonstandard analysis, superspace
and anticommuting integration, p-adic and ultrametric space. All three have applications in both
electrical engineering and physics. Once, complex numbers were equally outlandish, but they fre-
quently proved the sllortest path between 'real' results. Similarly, the first two topics named have
already provided a number of 'wormhole' paths. There is no telling where all this is leading -
fortunately.
Thus the original scope of the series, which for various (sound) reasons now comprises five sub-
series: white (Japan), yellow (China), red (USSR), blue (Eastern Europe), and green (everything
else), still applies. It has been enlarged a bit to include books treating of the tools from one subdis-
cipline which are used in others. Thus the series still aims at boots dealing with:
- a central concept which plays an important role in several different mathematical and/or
scientific specialization areas;
- new applications of the results and ideas from one area of scientific endeavour into another;
- inftuences wh ich the results, problems and concepts of one field of enquiry have, and have had,
on the development of another.
The roots of much that is now possible using mathematics, the stock it grows on, much of that goes
back to A.N. Kolmogorov, quite possibly the finest mathematician of this century. He solved out-
standing problems in established fields, and created wh oie new ones; the word 'specialism' did not
exist for hirn.
A main driving idea behind this series is the deep interconnectedness of all things mathematical
(of which much remains to be discovered). Such interconnectedness can be found in specially writ-
ten monographs, and in selected proceedings. It can also be found in the work of a single scientist,
especially one like A.N. Kolmogorov in whose mind the dividing lines between specialisms did not
even exist.
The present volume is thesecondof a three volume collection of selectedscientific papers of A.N.
Kolmogorov with added commentary by the author himself, and additional surveys by others on the
many developments started by Kolmogorov. His papers are scattered far and wide over many dif-
ferent journals and they are in severallanguages; many have not been available in English before. If
you can, as Abel recommended, read and study the masters themselves; this collection makes that
possible in the case of one of the masters, A.N. Kolmogorov.
Tbe shortest path between two truths in the N ever lend books. for no one ever returns
real domain passes througlt the complex them; the only books I have in my library
domain. are books that other folk have lent me.
J. Hadamard Anatole France
La physique ne nous donne pas seulement The function or an expert is not to be more
l'occasion de resoudre des problemes ... elle right than other people, but to be wrong for
nous fait pressentir 1a solution. more sophisticated reasons.
H. Poincare David Butler
Bussum, December 1991 Michiel Hazewinkel

Contents
Series Editor's Preface . . . . . . . . . . v
From the publishers of the Russian edition Xl
A few words ab out A.N. Kolmogorov . . . Xlll
1. On eonvergenee of series whose terms are determined by random events 1

2. On the law of large numbers . . . . . . 11
3. On a limit formula of A. Khinehin . . . . 13
4. On sums of independent random variables 15
5. On the law of the iterated logarithm . . . 32
6. On the law of large numbers .43
7. General measure theory and probability ealculus 48
8. On the strong law of large numbers . . . . . 60
9. On analytieal methods in probability theory. . . 62
10. The waiting problem . . . . . . . . . . . . 109
11. The method of the median in the theory of errors 115
12. A generalization of the Laplaee-Lyapunov Theorem 118
13. On the general form of a homogeneous stoehastie proeess 121
14. On eomputingthe mean Brownian area. . . . . . 128
15. On the empirieal determination of a distribution law 139
16. On the limit theorems of probability theory . . . . 147
17. On the theory of eontinuous random proeesses. . . 156
18. On the problem of the suitability of foreeasting formulas found by
statistical methods. . . . . . . . . . . . . . . . . 169
19. Random motions 176
20. Deviations from Hardy's formulas under partial isolation 179
21. On the theory of Markov ehains . . . . . . . . . . . 182
22. On the statistical theory of metal erystallization . . . . 188
23. Markov ehains with a eountable number of possible states . 193
24. On the reversibility of the statisticallaws of nature 209
viii CONTENTS
25. Solution of a biological problem . . . . 216

26. On a new confirmation of Mendel's laws 222
27. Stationary sequences in Hilbert space 228
28. Interpolation and extrapolation of stationary random sequences 272
29. On the logarithmic normal distribution of particle sizes under
grinding . . . . . . . . . . . . . . . . . . 281
30. J ustification of the method of least squares . . . 285
31. A formula of Gauss in the method of least squares 303
32. Branching random processes . . . . . . . . . 309
33. Computation of final probabilities for branching random processes 315
34. Statistical theory of oscillations with continuous spectrum 321
35. On sums of a random number of random terms . . . . . . . . 331
36. A locallimit theorem for classical Markov chains . . . . . . . 338
37. Solution of a probabilistic problem relating to the mechanism of bed
formation . . . . . . . . . . . . . . . . . . . . . . . . . 364
38. Unbiased estimators . . . . . . . . . . . . . . . . . . . , 369
39. On differentiability of transition probabilities of time-homogeneous
Markov processes with a countable number of states . . . . . . 395
40. A generalization of Poisson's formula for a sampie from a finite set 404
41. Some recent work on limit theorems in probability theory . 406
42. On A.V. Skorokhod's convergence . . . . . . . . . . . 419
43. Two uniform limit theorems for sums of independent terms 429
44. Random functions and limit theorems 442
45. On the properties of P. Levy's concentration functions . . 459
46. Transition of branching processes to diffusion processes and related
genetic problems. . . . . . . . . . . . . . . . . . . . . 466
47. On the classes ~(n) of Fortet and Blanc-Lapierre. . . . . . . . . 470
48. On conditions of strong mixing of a Gaussian stationary pr6cess . . 471
49. Random functions of several variables almost all realizations of which
are periodic . . . . . . . . . . . . . . . . . . . . . . . . 478
50. An estimate of the parameters of a complex stationary Gaussian
Markov process . . . . . . . . . . . . . . . . . . . . . . 479
51. On the approximation of distributions of sums of independent terms
by infinitely divisible distributions . . . . . . . . 484
52. Estimators of spectral functions of random processes 505
53. On the logical foundations of prob ability theory . . 515
CONTENTS ix
Comments . . . . . . . . . . . . . . . . . . . . . . . . 520
A.N. Kolmogorov. On the papers on prob ability theory and mathe-
matical statistics . . . . . . . . . . . . . . . . . . . . . 520
Analytical methods in prob ability theory (No. 9) (A.D. VentseQ 522
Markov processes with a countable number of states (No. 10)
(B.A. Sevast'yanov) . . . . . . . . . . . . . . . . . . 528
Homogeneous random processes (No. 13) (V.M. Zolotarev) . 528
Homogeneous Markov processes (No. 39) (A.A. Yushkevich) 530
Branching processes (Nos. 25, 32, 33, 46) (B.A. Sevast'yanov) 538
Stationary sequences (No. 27) (Yu.A. Rozanov) . . 539
Stationary processes (No. 48) (V.A. Statulyavichus) . . . . . 542
Statistics of processes (No. 50) (A.N. Shirgaev) . . . . . . . 544
Spectral theory of stationary processes (No. 34) (A.M. Yaglom) 545
Spectral representation of random processes (Nos. 47, 49)
( Yu. G. Balasanov and I. G. Zhurbenko) . . . . . . . . 551
Brownian motion (Nos. 14, 19, 24) (A.M. Yaglom) 554
Markov chains with a countable number of states (No. 23)
(A.A. Yushkevich) . . . . . . . . . . 559
Wald identities (No. 35) (A.A. Novikov) . . . . 567
S-Convergence (No. 42) (A. V. SkorokhorI) . . . 569
Uniform limit theorems (Nos. 43, 51) (T. V. Arak) 570
Concentration functions (No. 45) (V.M. Kruglov) 571
Empirical distributions (No. 15) (E. V. Khmaladze) 574
The method of least squares (Nos. 30, 31) (M.B. Malyutov) 583
Unbiased estimators (No. 38) (Yu.K. Belyaev and Ya.P. Lumel'skia) 585
Statistical prediction (No. 18) (A.M. Yaglom) 587
On inter-bed washout (No. 37) (A.B. Vistelius) . . . . . . . . . 591
From the Publishers of the Russian Edition
In accordance with the decision of the Praesidium of the USSR Academy of
Sciences, the first book of the selected works of Academician A.N. Kolmogorov
"Mathematics and Mechanics" ("Nauka", Moscow) came out in 1985. As for
the second book of Kolmogorov's works planned for publication, the editorial
board decided to divide it into two parts: the first contains the papers on
prob ability theory and mathematical statistics, and the second those on infor-
mation theory and the theory of algorithms. So the articles marked with two
asterisks in the list of references of the first book should be naturally divided
into those given in the present second book and those prepared for publication
in the third book.
The second and third books of the selected works of Kolmogorov were
prepared for publication by Yu.V. Prokhorov and A.N. Shiryaev.
A few words about A.N. Kolmogorov*
The remarkably broad creative interests of A.N. Kolmogorov, the wide range
and variety offields ofmathematics he worked in through different periods ofhis
life - all this makes Andrei Nikolaevich distinguished among mathematicians
in our country and all over the world. This diversity of interests makes hirn
unique among the mathematicians of our time. In many fields of mathematics
he obtained truly fundamental and principally important results. The problems
were often most difficult to solve and required great creative endeavour. This
is true for the results obtained by Andrei Nikolaevich in his young years on the
theory of sets and functions, both the descriptive and the metrical theories; for
example, the theory of operations on sets developed by hirn and his celebrated
example of a divergent Fourier series.
This was followed by papers on general measure theory, both the abstract,
that is, "the general theory proper", and the geometrie theory. After that
Kolmogorov started his fundamental work in various branches of probability
theory, work that made hirn, beyond any doubt, the most outstanding among
the researchers in this field all over the world.
Along with this, Kolmogorov wrote his first papers on mathematicallogic
and the foundations ofmathematics. Later, these were supplemented by studies
in information theory.
Andrei Nikolaevich made a very important contribution in topology. It
suffices to say that simultaneously with the outstanding American topologist
J .W. Alexander and quite independently of him, Kolmogorov came to the no-
tion of cohomology and laid the foundation of the theory of cohomology oper-
ations, obtaining the results that transformed topology as a whole. The deep
* Uspekhi Mat. Nauk 38:4 (1983), 7-8.

xiv A FEW WORDS ABOUT A.N. KOLMOGOROV
connections between topology and the theory of ordinary differential equa-

tions, celestial mechanics and, later, the general theory of dynamical systems
are weIl known. These connections were established back in the first papers
by H. Poincare. The ideas of Kolmogorov in this whole vast area of mathe-
matics, furt her developed by his numerous students, have largely determined
the present state of this field. Finally, the studies by Kolmogorov in mechanics
proper are worth mentioning, in particular, his famous work on the theory of
turbulence, which reached out directly to the realm of experimental natural
sciences. All this, which is by no means everything that could be said about
Kolmogorov as a scientist, shows that he is one of the most outstanding repre-
sentatives of modern mathematics in the broadest sense of the word, including
applied mathematics. Andrei Nikolaevich is undoubtedly recognized as such
by the international scientific community. In particular, this is reflected in the
fact that he has been elected a member of many Academies and scientific soci-
eties, and many universities have made hirn an honorary member - no other
Soviet scientist has ever received such recognition. Among these are the Paris
Academy of Sciences, the London Royal Scientific Society, the Academy of Nat-
ural Sciences "Leopoldina" , the Polish Academy of Sciences, the East German
Academy, the Polish Mathematical Society, the London Mathematical Society,
the USA National Academy of Sciences, the American Philosophical Society
founded by B. Franklin, and the Universities of Paris, Berlin, and \Varsaw.
A.N. Kolmogorov was born on 25th April, 1903 in Tambov, where his
mother Mariya Yakovlevna Kolmogorova was staying on her way from the
Crimea. Mariya Yakovlevna died giving birth to her son, and he was raised
by her sister Vera Yakovlevna Kolmogorova, who really replaced his mother
in his life. A.N. Kolmogorov loved her as his mother all her life, up to her
death in 1950 in Komarovka at the age of 87. On his mother's side, A.N. Kol-
mogorov comes from nobility: his matern al grandfather, Yakov Stepanovich
Kolmogorov, was marshaI of the nobility in the province of Uglich. Andrei Niko-
laevich's father was a son of a priest and worked as an agronomist with a higher
special education, or, as they said in those years, as a "learned agronomist" .
My friendship with A.N. Kolmogorov played a very special role in my life:
in 1979 it was fifty years since we had become friends, and through more than
half a century, this friendship has never been in question. We have never had
a single misunderstanding of any kind on any matter of importance for our life
A FEW WORDS ABOUT A.N. KOLMOGOROV xv
and outlook. Even when our opinions on some question differed, we always
respected each other's viewpoint and never lost our deep sympathy for each
other.
As already mentioned, Kolmogorov has a lot of students in various fields of
mathematics, and some of them have hecome famous in their areas. The older
students of Kolmogorov are Sergei Mikhailovich Nikolskii (h. 1905) and the late
Anatoli Ivanovich Mal'tsev (h. 1910), hoth Academicians. Next in age are Boris
Vladimirovich Gnedenko (h. 1912), Academician of the Ukrainian Academy
of Sciences and expert in prohahility theory of world-wide recognition, Acad.
Mikhail Dmitrievich Millionshchikov (1913-1973) and Acad. Izrael' Moiseevich
Gel'fand (h. 1913), elected as a foreign memher of the USA National Academy
of Sciences and Paris Academy of Sciences. Much younger, though also helong-
ing to the older generation of Kolmogorov's students, are Acad. Aleksander
Mikhailovich Ohukhov (h. 1918) and Corresponding Memher Andrei Sergeevich
Monin (h. 1921).
They are followed hy Vladimir Andreevich Uspenskii, Vladimir Mikhail-
ovich Tikhomirov, Vladimir Mikhailovich Alekseev, Yakov Grigor'evich Sinai
and Vladimir Igorevich Arnol'd, Corresponding Memher since 1984.
The largest group of Kolmogorov's students works in prohahility theory
and mathematical statistics. It includes Acad. Yurü Vasil'evich Prokhorov,
Corresponding Memher Login Nikolaevich Bol'shev, Acad. ofthe Uzhek Acad-
emy of Sciences Sagdi Khasanovich Sirazhdinov, Acad. of the Ukrainian Acad-
emy of Sciencesand Corresponding Memher of the USSR Academy of Sciences
Boris Aleksandrovich Sevast'yanov, Juri Anatol'evich Rozanov, Al'hert Niko-
laevich Shiryaev and Igor' Georgievich Zhurhenko. Of course, this list is hy no
means completej from the very title of my note it is clear that this is neither a
juhilee review of the life and activities of A.N. Kolmogorov, nor a traditional
''juhilee article" j so it does not claim to he complete in any respect.
P.S. Aleksandrov
1. ON CONVERGENCE OF SERIES WHOSE TERMS
ARE DETERMINED BY RANDOM EVENTS *
Jointly with A.Ya. Khinchin
Consider aseries
(1)
whose terms are random variables; denote the values taken by Yn (their number
. fi me
IS ·t or, POSSI·bly, count a ble ) by Yn(1) , Yn(2) , ... , Yn(i) , ... , an d th e correspon d·mg
·l·t·les by pn(1) , Pn(2) , ... , pn(i) , ... , WI·th"
prob a b11 (i)
L....i Pn =.
1 Fur ther, denote by
an = Ly~i)p~)
i
the expectation of Yn, and by
bn = L {y~i) - a n }2 . p~)
i
the expectation of the square of the deviation Yn - an.

We first prove that, assuming the convergence of the series Ln an and
Ln bn, the probability P of the convergence of the series (1) is 1. A partic-
ular case of this statement (namely when Yn takes only two values, +cn and
-Cn with the same probability ~) was established by Rademacher in function-
,
theoretic terms. 1 In §1 we prove a general theorem using a generalization of
the method suggested by Rademacher. In §2 we prove it again by other meth-
ods that bring ab out the result more quickly. In §3 we establish that, for one
important and broad dass of cases, convergence of the series L an, L bn is not
only a sufficient, but also a necessary condition for P = 1 to hold; this is the
case when the values taken by Yn are uniformly bounded. Finally, in §4 we
indicate a necessary and sufficient criterion for P = 1 to hold in general.
§1 is due to A.Ya. Khinchin, §§2, 3 and 4 to A.N. Kolmogorov.
§1
Let us reduce the problem to the function-theoretic form more suitable for uso
For this, we construct a system of functions
<P 1 (z ) , <P2 ( Z ), ••. , <Pn ( z ), ...
* Über Konvergenz von Reihen, deren Glieder durch den Zufall bestimmt werden',
Mat. Sb. 32 (1925), 668-677.
1 Math. Ann. 87 (1922), 135.
1
2 ON CONVERGENCE OF SERIES WITH RANDOM TERMS
by means of the following recurrence: assume, without lass 0/ generality, that
for all n and i; divide the interval ° ~ x ~ 1 from left to right into parts of
lengths
(1) (2) (i)
P1 ,PI , ... , P1 , ...
and assume that on each of these subintervals 4>1 (x) equals a constant, namely
4>1(X) = z~i) = y~;) - a1
on the ith interval. Then we have
1 1
4>1(x)dx = 0, 1 1
{4>1(x)}2dx = b1.
If the function 4>n-1 (x) is already determined, then the function 4>n (x) is
determined as follows: each interval on which 4>n-l (x) is constant is divided
into intervals whose lengths (from left to right) are proportional to the values
Pn(1) 'Pn(2) , ... , Pn(i) , ...
and on the ith subinterval we set
1 1
Then
1
4>n(x)dx = 0, 1
{4>n(x)}2dx = bn.
The values of the functions 4>n (x) at the ends of the subintervals are unimpor-
tant and can be taken arbitrarily.
Assuming that I:an converges, the probability that (1) converges is equal
to that of the convergence of
(2)
which in our case is given by the (Lebesgue) measure of the set on which this
series converges. Hence, our main result can be interpreted as folIows: 2 con-
vergence of L bn is a sufficient condition for the almost everywhere convergence
of (2).
2 Clearly, this statement is the same as that stated in the introduction. Gen-
eral considerations of this kind of relation can be found in Steinhaus's work
(Fund. Math. 4 (1923), 286-310).
ON CONVERGENCE OF SERIES WITH RANDOM TERMS 3
The proof given below generalizes Rademacher's proof for a special case. 3
Set
n
'll"n(x) = TI {1 + tPk(X)},
k=l
We have
Let eS (a, b) be an inter val of constancy of the function tPn (t), taken to be
a ~ x < b. Clearly
and, consequently,
since 'll"n(t) is constant on (a, x). From the conditions
L.....
" " zn(i) Pn(i) -- 0 ( n - 1, 2 , ... )
i
we obtain
Iz~)p~)1 = IE z~i)p~)1 (n,j = 1,2, ... )
iti
and the Schwarz inequality implies
Iz~j)p~)1 ~ E{z~i)Fp~)Ep~) ~ v;;;.J1-p~),

itj i~i
(')
Izn' I ~
Il
Vbn
J1- Ci)
PnCi)
(n,j = 1,2, ... ).
pn
Hence, if on (a, b)
then
(4)
3 See footnote 1.
On the other hand we have
and
where Iz~1d is the maximal value of l<Pn+l(t)1 for a < t ~ x. If
then by our hypotheses the inequality p~+t) ~ P~11 holdsj therefore a fortiori
r V
we have
a
n
k=l
rr 1 - pUn+t}
IJa <Pn+l(t) dt l < p~k\/bn+l Un;~l
Pn+l
(6)
From (5) and (6) it follows that
11 x
<Pn+l(t)dt l ~ 2v'bn+1plh)p~2) ... pW n )Vl- p~+t), (7)
n+l >
since this follows from (6) for pUn+l) n+l < 12·
- 12 and from (5) for pUn+l)
Now, (3), (4) and (7) imply
ILln(x)1 = Ifn+l(X) - fn(x)1 ~
~ 2v'bn+1Vl- p~+t) ft {p~k) + .jb;Vl- p~k)}

k=l
or , substituting 1 - p(h)
k -- w k'
2
If we set
00
II(1+h)=B,
k=l
then
n
l6. n (x)1 < Bbn +1 + 2Bvn +1 II (1 - Vk). (8)
k=l
Since for every positive integer N, putting Vo = 0, we have

N n N+l
1- I>n+l II (1- Vk) = II (1- Vk) > 0,
n=O k=l k=l
the right-hand side of (8) contains the general term of a convergent series.
Hence, the limit
lim fn(x)
n_oo
= f(x)
exists.
In the special case considered by Rademacher, f(x) turned out to be a
monotone function, which is not necessarily so for the general case considered
here.
However, monotonicity was used only to prove differentiability of f(x), so
it suffices to demonstrate that f( x) is a function of bounded variation. This is
easily proved as folIows. In any case the total variation of f(x) is not greater
than the upper bound ofthe total variations of all the fn(x) and, consequently,
it is not greater than
But
n n
:sII II (1 + b k) < vB,

k=l k=l
where the admissibility of changing the order of taking the product and inte-
grating follows directly from the particular structure of the functions <p k (x).
This gives the desired statement. Thus, the derivative f'(x) exists almost
everywhere. Let x be an arbitrary point of (0,1), other than a sub division
point, at which f'(x) exists, and let (an, bn ) denote the interval containing x
on which <Pn(x) is constantj since f(a n ) = fn(a n ), f(b n ) = fn(b n ), we obtain
since 4
fn(b n ) - fn(a n ) = 1I"n(x)
bn - an
because 1I"n(x) is constant on (an,b n ).
Hence, lim 1I"n(x) exists almost everywhere. On the other hand, the series
n-+oo
converges almost everywhere, since otherwise the series
would diverge, which contradicts the assumption.

But the convergence almost everywhere of the sequence 1I"n(x) and the se-
ries L:{<p(x)F implies that L:<Pn(x) converges almost everywhere, as required.
§2
We now prove this assertion in a different, much simpler way. We set

n
E <Pk(X) = sn(X).
k=l
If the series (2) diverges at every point of some set of positive measure,
then there exists a set E of positive measure mE and a positive constant A
such that
4 If lim (b n - an) > 0, then we have for sufliciently sm all f >0

n-+oo
1
f (x) = 2f[f(x + f) - f(x - f)] =
I
lim ~[fn(x + f) - fn(x - f)] =

n-+oo 2f
lim lrn(X).
n-+oo
for each positive integer and each x E E, for a suitable p = p(x, n) > n.
Since sp(x )-sn(x) is constant on every interval on which f/;p(x) is constant,
by construction of our functions, we can find a finite number of non-overlapping
intervals Ö such that:
1) E ö (the sum of the lengths of all Ö's) is greater than ! mE;
2) Each Ö is an interval on which f/;p(x) (p> n) is constant and
If now k is the largest of these numbers p (there is a finite number of

them) and ö is an interval on which sp(x) (p ~ k) is constant, then, by the
construction of our functions,
consequently,
L
k
p=n+l
bp = 1
0
1
{Sk(X) - Sn (x)}2dx ~ A 2 Lö > !A 2mE.
This implies the divergence of E bn , contrary to the assumption.
§3
Thus, convergence of E bn is a sufficient condition for convergence of E f/;n (x)

almost everywhere. In general this condition is not necessary, as can easily be
demonstrated by an example.
Namely, if f/;n(x) is equal to 0, +Cn , -Cn on sets ofmeasure l-Tn , !Tn , !Tn
respectively and E Tn converges, then f/;n(x) does not vanish almost everywhere
only for a finite number of subscripts n, and therefore Ef/;n(x) converges al-
most everywhere, whatever Cn . Since bn = C~Tn, by suitably choosing the Cn ,
E bn can be made to diverge.
For the corresponding series (1) we obviously have P = 1, so the con-
vergence of E bn is not necessary for P = 1. It is also easy to show that
the convergence of E an is not necessary for P = 1. To show this, put
y~l) = Cn , y~2) = 0, p~l) = Tn , pe,;) = 1 - Tn , with ETn convergent and
E Cn T n divergent.
However, there is an important special case, when our condition is also

necessary. In fact, if the absolute values of our functions tPn(x) are uniformly
bounded, then convergence almost everywhere of E tPn (x) implies the conver-
gence of E bn . The situation with (1) is similar. If P = 1 and if the random
variables Yn (n = 1, 2, ... ) are uniformly bounded, then both E an and E bn
must converge.
In order to prove this general result we first establish the following: if the
tPn(x) are uniformly bounded and ifthere exist a set E ofpositive measure and
a constant K such that
(9)
for any k and all pairs of points (Xl, X2) belonging to E, then E bn converges.
Denote by EIt; the union of those intervals of constancy of tPlt;(x) (hence of
sIt;(x» that contain the points of E. Obviously, EIt; contains E and Elt;+l. It is
also dear that (9) holds for any pair of points of EIt;. Set
then
f f {SIt;+1(X1) - SIt;+1(X2)}2dx 1dx 2 = f f {Slt;+l(Xt}-

lE/C+l lE/c+l lE/c lE/c
-SIt;+1(X2)}2dx1dx2 - 2 f f {SIt;+1(X1) - SIt;+1(X2)}2dx1dx2-

lE/c+l lF/c
- f f {SIt;+1(X1) - SIt;+1(X2)}2dx1dx2'
JF/clF/c
But tPlt;+l(X) is orthogonal to both sIt;(x) and 1 on EIt;, hence
f f {SIt;+1(X1)-SIt;+1(X2)}2dx1dx2 = f f {SIt;(xt}-SIt;(X2)}2dx1dx2+
lE/c+l lE/c+l lE/c lE/c
+ f f {tPlt;+l(Xt} - tPIt;+1(X2)}2dx1dx2-
lE/clE/c
-2 f f {SIt;(X1) - SIt;(X2) + tPIt;+1(X1) - tPIt;+1(X2)}2dx1dx2-

lE/c+l lF/c
- f f {SIt;(x!) - SIt;(X2) + tPIt;+1(X1) - tP1c+1(X2)}2dx1dx2 ~

lF/cJF/c
~ f f {SIt;(xt} - SIt;(X2)}2dx1dx2 + 2bIt;+1(mEIt;)2-

lE/clE/c
-(K + 2M)2{2mEIt;+1 . mFIt; + (mFIt;?l,
where M is the upper bound of l~n(z)1 and mE denotes the measure of E. In

the inequality obtained, we set k = 0, 1, ... ,n - 1, add up and note that
n-l
I:{2mEk+l . mF + (mF )2} $ l.
k k
k=O
As a result we have
n-l
[ [ {Sn(Zt} - Sn(Z2)}2dz 1 dz 2 ~ I: 2bk+l(mEk)2 - (f( + 2M)2,
JEn JEn k=O
n-l
I: 2bk+l(mEk)2 < 2(f( + 2M)2
k=O
and a fortiori,
n-l
(mE)2 I: bk+l < (f( + 2M)2,
k=O
which clearly proves the convergence of E bn .

We now suppose that the random variables Yn are uniformly bounded and
that P = 1, which implies convergence almost everywhere of E(a n + ~n(z)).
Since an is independent of z, for E ~n (z) the assumptions of the auxiliary
statement just proved hold. 5 Consequently E bn converges and by §1 or §2,
E ~n(z) converges almost everywhere. But this entails the convergence of
E an, which proves all we need.
§4
Consider, together with (1), another similar series
Ul + U2 + ... + Un + ... , (10)
denote by Q the prob ability of its convergence, and by Tn the probability of

the event
5 The uniform boundedness of the Yn clearly implies that of the an and conse-
quently that ofthe tPn(X).
10 ON CONVERGENCE OF SERlES WITH RANDOM TERMS
When E Tn converges we call the series (1) and (10) equivalent. 6 Clearly, in
this case Q P. =
Now we claim that a necessary and sufficient condition for Q = 1 to hold
for (10) is the existence of an equivalent series (1) for which E an and E bn
converge. Clearly, only the necessity of the condition needs to be proved.
=
Thus we suppose that Q 1 and define the series (1) as follows. Let Yn =
Un if IUn I < 1 and Yn = 0 otherwise. Hence, T n coincides with the prob ability
=
of lun(x)1 ~ 1. Since Q 1, this immediately implies the convergence ofETn ,
so that (1) and (10) are in fact equivalent. Thus, the constructed series. (1) is
already uniformly bounded, therefore according to §3 the corresponding series
E an, E bn converge, and the required result is proved.
Note also that our proof not only justifies the existence of (1) but also
gives an extremely simple rule for constructing this series.
Moscow, 3 December 1925
6 This means that there is a certain relation between Un and 'Yn.

2. ON THE LAW OF LARGE NUMBERS *
Let E l , E 2 , ••• , E n , ... be a sequence of independent trials. The reasoning given
below is true only for a finite number of distinct possible outcomes Eil), Ei 2 ) , •••
of the trial Ek. However, the theorem to be proved holds also in the general
case.
Let Fn be a variable wh ich depends on the first n trials. If for any positive
f the prob ability of
where D n is the expectation of F n , tends to 1 as n tends to infinity, then F n is

said to satisfy the law of large numbers, or that Fn is stable.
The simplest sets of conditions sufficient for the stability of Fn are ex-
pressed in terms of restrictions imposed on the variance of F n .
It is natural to consider the upper bound of the difference
-F.n (E1Cid , ... , EU"-l) Ch

k-l , E k ) , ECi"+l)
k+l , ... , ECi
n
n ») ,
when i 1 , i 2 , ... ,in and ik run independently through all possible values, as the
variance of Fn with respect to Ek. Denote this variance by Onk. With this
definition we can formulate the following theorem.
Theorem. If the sum
tends to zero as n increases, then F n satisjies the law of large numbers.
Corollary. The condition

Onk = o(l/Fn). (1)
is also sufficient for Fn to be stable.
Proof of the theorem. Denote by B n the expectation of the square of the

deviation F n - D n . A well-known condition for the stability of F n is
(2)
* 'Sur la loi des grands nombres', C. R. Acad. Sei. Paris 185 (1927), 917-919.
Presented by E. Borel.
11
12 ON THE LAW OF LARGE NUMBERS
To use this eondition, a more effeetive expression for B n is needed. For

this, we denote by Dnk the expectation of F n assuming that the results of the
first k trials are known. Let Znk = Dnk - Dnk-l. Thus, Znk is the inerement
of the expectation of Fn when the result of the trial Ek is known. Clearly we
have
Fn - Dn = Znl + Zn2 + ... + Znn. (3)
It ean be proved that the expectation ZniZnk (i 'I k) is zero.

Let k > i. Assume that the result of E l , E 2 , ••• , E k - l is known. Under
this assumption Dnk = D nk - l ; therefore, Znk = 0 and sinee Zn; is a eonstant,

ZniZnk is also zero. This is true for any results of the trials E l , E 2 , .•. ,Ek - l .
Consequently, the expeetation Zni Znk itself is also zero.

Thus, denoting by ßnk the expeetation Z~k' we obtain
Bn = ßnl + ßn2 + ... + ßnn. (4)
It ean be proved that
(5)
Inequality (5) permits us to derive our theorem beeause of (2).
Remark. In the classical Chebyshev ease the trial Ek eonsists in determining

X k , and Fk is equal to the arithmetie mean of Xl, X 2 , ... ,Xn . Denoting by dk
the expectation of X k and by bk the expectation of (Xn - dn )2, we obtain
Condition (1) turns into the well-known eondition
maxlXnl = o(.;ri).
The above is a generalization of Chebyshev's ease, where F n is the sum of
independent random variables, sinee it eoneerns sums of dependent variables.
Very often the summands here are functions of eertain other independent vari~
ables. We think that in this ease the above method for arbitrary functions in
independent variables would be very natural.
31 Oetober 1927
3. ON ALIMIT FORMULA OF A. KHINCHIN *
Let
(1)
be a sequence of independent randorn variables. Without loss of generality

assurne that the expectation of Zn is zero. Denote by bn the expectation of the
square of Zn and by Sn and B n the sums
n n
Sn = LZk, B n = Lbk.
k=l k=l
A.Ya. Khinchin [1] suggested that under broad assurnptions the probabil-
ity that
·
I Imsup Sn = 1 (2)
V2Bn log log B n
is 1. Khinchin himself proved this formula for some important special cases [2].
Our aim is to formulate sufficiently general conditions for this formula to hold.
These conditions are as folIows:
(I) Bn -+ 00,
(II) mn-boundedness from above, IZnl:S; m n = o( Bn

log log B n
)
.
Clearly the first condition is necessary, except for the case Zn = O. 1f the
Zn are uniformly bounded, then the second condition follows fram the first; in
this case this condition is necessary and sufficient.
If we want to use only probabilistic relations that can be effectively ob-
served, the meaning of formula (2) can be explained as folIows:
10 • Whatever the positive numbers 11 and [j, there is an integer n such

that the prob ability that all the inequalities
hold is not less than 1 - 11, whatever pis.
* 'Sur une formule limite de M.A. Khintchine', C. R. Aead. Sei. Paris 186 (1928),
824-825. Presented by J. Hadamard.
13
14 ON ALIMIT FORMULA OF A. KHINCHIN
2°. For any 'TJ, 6 and m, there exists an integer P such that the probability
that all the inequalities
Sk< J2Bk loglogB k (1-6) (k=m,m+1, ... ,m+p)
hold simultaneously does not exceed 'TJ.

A complete proof of statements 1° and 2° using only the conditions (I)
and (11) will be published elsewhere. 1
Applied to the problem of recurrent trials, our theorem yields: let Pn be
the probability of an event f at the nth trial and J.l( n) the number of events f
during the first n trials. Then the prob ability that
(3)
is 1, provided the series EPn(1- Pn) diverges.
References
1. A.Ya. Khinchin, Basic laws 0/ probability theory, Moscow, 1927 (in Rus-
sian).
2. A.Ya. Khinchin, Math. Ann. 99 (1928), 152.
1 See article No. 5 of this publication.

4. ON SUMS OF INDEPENDENT RANDOM VARIABLES *
§1. Notation
P(A) denotes the probability of an event A;
PB(A) or P(AIB) denotes the conditional prob ability of event A with
respect to an event B;
E~ denotes the expectation of a randorn variable ~;
EB~ or E(~IB) denotes the conditional expectation of ~ with respect to

an event B;
E(~ : B) = P(B) . EWB) or, wh at is the same, is equal to the Lebesgue
integral
over B;
D~ = E(~ - E~)2 denotes the variance of a randorn variable ~.
If 6 , ... , ~n are randorn variables, we set
k
Sk = L~;' S = Sn;
;=1
k
(k=~k-E~k; Tk=L(;' T=Tn;

;=1
(We assurne that E~k and E~; exist; 1 ::; k ::; n.) Clearly E(k = 0, T k
Sk - ESk . If 6, ... , ~k are independent, then
k
DSk = LD~i.
;=1
§2 Some theorems on finite sums
Assurne that 6, ... ,~n are rnutually independent.
Theorem 1. Let R > O. Then
* 'Über die Summen durch den Zufall bestimmter unabhängiger Grössen', Math.
Ann. 99 (1928), 309-319; Bemerkungen ... - Math. Ann. 102 (1929), 484-488.
15
16 ON SUMS OF INDEPENDENT RANDOM VARIABLES
Theorem 2. Let f > 0, m a non-negative integer, and
R=fD+M.
Then
Theorem 3. Let R > O. Then
Theorem 4.
P{T > O} > ~ D - M.
- 16 D+ M
Theorem 5.
P{T> -3M} 2: 418.
Theorem 6.
Proo/ 0/ Theorem 1. Let
P{m:xITkl2: R} = P(A) = LP(Ak).

k=l
Since the variables (i with i >" k do not occur in the definition of Ak, the
hypothesis that 6, ... ,en are independent implies
n
E(T 2 ;Ak) = E(T;;Ak) + L E((l;Ak) 2: E(T;;Ak) 2: R 2 p(Ak),
i=k-1
hence ET 2 2: R 2 P(A), which proves the desired inequality.

ON SUMS OF INDEPENDENT RANDOM VARIABLES 17
Remark. This inequality is an identity, for example, in the following case: n = 1

and
Proo/ 0/ Theorem 2. We set
A;k = {11J I< iR, j< k; ITkl ~ iR}
and
n
Ai = LAik.
k=l
On the set A ik we have
Clearly
(jl ~
P
P(Ai+1IAik ) = P{u ~ (i+ 1)R}:S P{dPfX<J L €D}.
_P- j=k+l
As in the proof of Theorem 1, we note that the variables (j for j > k do not
occur in the definition of Aik. Therefore,
and consequently
P(Ai+lIAi):S 1/€2, P{u ~ mR} = P(A m ) =

= P(Am-lIAm)P(Am-dAm-2) ... P(AI) :S 1/€2m,
which proves Theorem 2.
Pro%/ Theorem 3. Let
Bk = {ISil:SR, i:Sk},
B = B u = {v:S R},
Ck = Bk-l \Bk.
Clearly
n
P(B) +L P(Ck) = 1. (1)

k=l
Put
On the set B" we have
IT" - a,,1 = IS" - E(S"IB,,)I ~ 2R,

la" - a"-ll = IE«("IB,,)I ~ M.
Now consider the expression
E[(T" - a,,?; B,,-tl =

=E[(T" - a,,)2; B,,] + E[(T"_l - a"_l - (a" - a,,-d+
+(,,)2; c,,] ~ E[(n - a,,)2; B,,] + P(C,,) . 4(R + M)2. (2)
On the other hand
E[(n - a,,)2; B"-l] =

=E[«T"_l - a"-l) - (a" - a,,-d + (,,)2; B,,-d =
=E[(T"_l - a"_1)2 + (a" - a,,_d 2 +(;; B,,-tl ~
~ E[(T"_l - a,,_d 2; B,,-d + P(B)E(;. (3)
Comparing (2) and (3) we obtain
E[(T"_l - a"_1)2; B"-l] + P(B)E(i ~

~ E[(T" - a,,)2; B,,] + P(C,,) . 4(R + M)2.
Setting k = 0,1, ... ,n in this inequality, summing the corresponding inequali-

ties and taking into ac count (1), we obtain
n
P(B)D 2 ~ E[(Tn - an )2; B n ] + LP(C,,), 4(R+ M)2 ~
n
"=1
~ P(B)· 4R2 + LP(C,,), 4(R+ M)2 ~ 4(R+ M)2,
"=1
Remark. If we take z" instead of (", the statement of the theorem will be valid
with v replaced by u.
Proo/ 0/ Theorem 4. We set
Gm = {mR < T ~ (m + 1)R},
where R = 8D + M and set

00
Clearly
Bo = {T > O}.
By Theorem 2,
00 00
E(T; B o) = L E(T; Gm) ~ L(m+ 1)RP(Gm ) =

m=O m=O
00 00 1
=R L P(Bm) ~ R[P(Bo) + L 82m] ~
m=O m=l
~ R[P(Bo) + 312]'
On the other hand, it is easy to prove that
E(T; B o) ~ ~D.
Thus
~D ~ R [P(B o) + 312] = (8D + M) [P(B o) + 312];

D 1 1 D-M
P(B o) ~ 2{8D+M) - 32 ~ 16 D+M'
as required.
Pro%/ Theorem 5. Let

D~~M.
Then by Theorem 4
T 1 D-M 1
P{ > O} ~ 16 D + M ~ 48'
If
D>~M
- 2
then by Theorem 1,
P{ITI :5 3M} :5 D 2/9M 2 < 1/4.
In both cases these estimates prove the desired statement.
Proolol Theorem 6. We set
m:x IS - L(il.
n
W=
i=k
Then from Theorem 3,
P{W> R+ 3M} ~ 1- 4(4M + R)2/D2
and applying Theorem 5 to E?=k (i we find that
P{s> RIW > R+3M} ~ 1/48.
This immediately implies the desired statement.
§3. Convergence of series
Below necessary and sufficient conditions are considered for the convergence of
series in independent random variables, previously established by other meth-
ods in our joint paper with A.Ya. Khinchin (see paper 1 of this volume).
Consider two sequences of random variables
TI = (Tll, TJ2, ••• ,TIn, ...), ii = (fit, ii2, . .. , iin, ... ).

The sequences TI and ii are called equivalent if 1
00
L P{Tli #; iid < 00. (4)

i=l
In what follows we assume that each of the sequences TI and ii consists of

independent random variables.
1 This definition is due to A.Ya. Khinchin.

Theorem 7. 1) The probability P of convergence of

00
L:'7i
i=l
(5)
is 1 if there is a sequence ij = (ijn) equivalent to '7 = (77n) such that the series
00
L:Eiji (6)
i=l
and
(7)
(where (i = iji - Eiji) converge.

2) P is zero if there is no equivalent sequence for which the senes (6) and
(7) converge simultaneously.
Proof. 1) According to Theorem 1; for every positive y,
Since the series (7) converges, for every { > 0 we can find some (possibly large)
m such that for n ~ m the right-hand side of (8) is smaller than f.
By definition,
p
P lim lim lim p{ max IL: 'lA,1 < y}.

= ,,-+On-+ooN-+oo n'5.p'5.N k=n
Therefore (8) implies that the prob ability of convergence of

00
(9)
is 1. Since the series (6) converges, the same is true for the series
00
L: ijn. (10)
n=l
Since in the final analysis the probabilities of convergence of equivalent series

coincide, P = 1.
2) We set
_ { 1/n,
1/n = (11)
0,
First we assume that 1/ = (1/n) and Ti = (Tin) are equivalent , that is,
00
L P{I1/nl > 1} < 00. (12)

n=l
In this case one of the series (6) or (7) should diverge. If the series (7) diverges,
then by Theorem 3 we see that for any Y and each n,
as N -+ 00. This means that the probability of convergence of (10) is O. By

the equivalence of 1/ = (1/n) and Ti = (Tin) this implies that P = O.
If (7) converges, then the prob ability of convergence of (9) is 1, according
to the first part of the theorem. Therefore, if (6) diverges, then P = O.
Now consider the case when 2:::'=1 P{I1/nl > 1} = 00. In this case we
immediately have
N
p{ max IYkl< 1} = II[l- P{I1/kl
n<k<N -
> 1}] -+ 0
- - k=n
for every n as N -+ 00, hence P = O.

Thus, our theorem is proved for all possible cases.
Remark. If it is known that there exists a certain sequence Ti = (Tin) satisfy-

ing the conditions of the theorem, then the sequence determined by (11) also
satisfies the conditions of this theorem.
§4. A generalized law of large numbers
Consider the system of random variables
1/11,1/12, ... ,1/1ml'

1/21, 1/22, ... , 1/2m2'
1/n1, 1/n2, ... ,1/nmn'

denoted by lI1Jnkll for short. Suppose that in every row the variables are inde-
pendent of one another, whereas in different rows they may be dependent.
We say that the means
are stable if there exists a sequence of numbers d1 , d 2 , ••. , such that for any
(: > 0,
We will give a necessary and sufficient condition for the stability of the means.
We say that two systems l11Jnk 11 and 117Jnk 11 are equivalent if
and
Clearly for equivalent systems the means are simultaneously either stable or
unstable.
Theorem 8. A necessary and sufficient condition lorthe stability 01 the means

U n, n ~ 1, is the existence 01 some system 117Jnkll equivalent to II1JnkIL lor which
(13)
Prool 01 the sufficiency. Theorem 1 implies that for any (: > 0,
Therefore, (13) implies that
and, from the equivalence of II1Jnk 11 and 117Jnk 11 we obtain
(14)
which proves the sufficiency.
Proo/ 0/ the necessity. Suppose that the me ans U n , n ~ 1 are stable. For any
random variable TJnk there exists a constant fnk such that
(15)
It is easy to prove that for any positive f > 0,

mn
I:P{ITJnk - fnkl ~ Tnnf} -+ 0, (16)
k=l
as n -+ 00.
Now set
Tink = {TJnk' (17)
fnk, if ITJnk - fnkl > m n;
_ () _ { TJnk,
TJnk f - ~
Jnk, if ITJnk - fnkl > mnf.
By (16) the systems IITJnkll, IlTinkll and IlTink(f)1I are equivalent, and the means
ii n and iin(f) are stable. Note that
Therefore by Theorem 6,
mn
P{liin(f) - dnl > f} = p{IE 71nk(f) - mndnl > fmn} ~
k=l
1 [ 324m 2 f 2 ]
~ 48 1- L~l E«(~k(f» .
For any f °
> the left-hand side of this inequaIity converges to zero. Hence,
(18)
For f ~ 1,
and, by (16),
Therefore (13) follows from (18).
Remark. If there exists a system lliink 11 satisfying the condition of the theorem,
then the system determined by (17) and (15) is such a system.
We say that the means O'n, n ~ 1, have the property of normal stability if
for any f > 0,
Theorem 9. A necessary and sufficient condition for normal stability of the

means O'n, n ~ 1, is the existence of an equivalent system lliinkll for which (13)
holds and
The proof of the theorem follows directly from (14) which was established
for any stable system.
Re mark. For the case ofnormal stability, instead of lliinkll determined by (17),
(15), one can take a system
_ {'TJnk' if I(nkl :::; m n ,

'TJnk =
E'TJnk, if I(nkl > m n ·
§5. The law of large numbers
The condition of stability of the means
O'n = ('TJl + ... + 'TJn)/n,

where 'TJl, 'TJ2, ... , are independent random variables, is a special case of Theo-
rems 8 and 9 if we set m n = n,
'TJnk = 'TJk.
Normal stability of such means is usually called the law of large numbers.
In this case we have
Theorem 10. In order that for any f > 0,
(19)
it is necessary and sufficient that there exist a system lliink 11 such that
n
1. LP{i1nk:P 'l]nk} --+ O.
k=l
1 n
2. - L[Ei1nk - E'I]nk] --+ O.
n
k=l
3.
Remark. If there is a system IIi1nk 11 satisfying the above conditions, then as

such a system we can take the system
_ { 'l]k, if I(kl ~ n,
'l]nk = E
'l]k, if I(kl > n.
Theorem 11. In order that
it is necessary and sufficient that

n
1. LP{I(kl > n} --+ O.
k=l
2. -n1 k=l
L E((k;
n
I(kl ~ n) --+ O.
3.
Moscow, 24 December 1927
REMARKS
The purpose of these remarks is, on the one hand, to refine and improve certain
results (§§1, 2) and on the other hand, to study an important special case of
the law of large numbers (§3).
§1
V.V. Nemytskii pointed out to me that the inequality
E(T;Bo) ;::: ~D,

used in the proof of Theorem 4 is false. Indeed, it is easy to show that
that is, the reverse inequality holds.

In this connection Theorems 4, 5 and 6 of this paper must be replaced by
the following.
Theorem 4*. Let M ~ D. Then
Theorem 5*.
Theorems 7-11 remain unchanged.

Theorem 6 used only for proving Theorem 8 should be replaced by Theo-
rem 6* (below).
Pro%/ Theorem 4*. Let

E = {ISI ~ D/2}
and let E be the complementary event. If IMSI ~ 2D, then
as required.
If IMSI < 2D, then we introduce the event
Ern = {3mD ~ ITI < 3(m + l)D, ISI ~ D/2}.

By Theorem 2 we have
On the set
5
E' = LErn
rn=O
we have
ITI ~ 18D, ISI ~ 20D.
Hence
Since
E(S2., E) <
- 1D
4
2
,
it follows that
00
n 2 ~ ES2 ~ E(S2 j E+ E' + 2: E m ) ~

m=6
< 1n 2 + 400n2 P(E) + ~ (m + 1)2 25n 2 <

-4 ~ ~m -
m=6
~ ~n2 + 400D 2P(E),
whence
1
P(E) ~ 1600'
which proves Theorem 4" .
Pro%/ Theorem 5". If M > D or R > tD, then the required inequality holds,
since then its right-hand side is negative.
If, however, M ~ D and R ~ ! D, then the required inequality follows
from Theorem 4" .
§2
In proving Theorem 8 I used the formula

mn
2:P{I'7nk - /nkl ~ m n· n} -+ O. (16)
k=6
For a rigorous proof of this formula we need the following
Theorem 6". Let E 1 , ... ,En be a sequence 0/ independent events and let U
be an event such that
Then
Proof If there exists k such that
then
as required.
Now for all k let
Let us show that in this case there exists a k such that
For this we put
and let Fi be the complementary event. Since Fi and Ei are independent, for
i ~ k we have
P(FiIEi) = P(Fi ) ~ P(F,.) ~ ~u,
P(U FilEi ) ~ lu,
k k
P(U) ~ Ep(UFiEi) ~ lu Ep(Ei) ~ luP(Fk) ~ ~u2,
i=l i=l
as required.
Prool 01 Formula (16). We set
Fk = {I (E
i#
7Jni + Ink) / mn - dn I~ ~}.
Clearly
For sufficiently large n, in view of the stability of the means we have
P(U) ~ t,
hence
If the event E k takes place and F k does not, then U takes place. Therefore
But the events Ek and Fk are independent, so for sufficiently large n,
P(Fk!Ek) = P(Fk) ~ ~,
P(U!Ek) ~!.
By Theorem 6*
1 2
P(U) ~ 36P (E1 + ... + Emn )
and, since P(U) ---+ °as n ---+ 00,
as desired.
§3
Here we study a special case of the law of large numbers, namely the case
when the independent random variables 171,172, ... have the same distribution
P{171 < x} = F(x). Then we have
Theorem 12. The means (Fn = (171 + ... + 17n)/n, n ~ 1, are stable if and
only if
nP{!171! > n} ---+ 0, n ---+ 00.
Proo! Set
_ { 17n, if !17n! ~ K,
17nk = 0, if !17n! > K.
It is easy to see that if nP{ 11]11 > n} - °as n - 00, then the systems
(r,nk) and (1]n) are equivalent and
1 ~ -2 1 ~ -2
n 2 L.J E(nk ~ n 2 L.J E1]nk - 0, n - 00.
k=l k=l
Hence by Theorem 8, the means O'n, n ~ 1, are stable. Conversely, if the

means O'n, n ~ 1, are stable, then as has been shown, formula (16) holds. In
the present case this formula takes the form
mP{I1]l - I1 ~ w} - 0,
where 1 is a constant. This formula immediately implies the required condition.

If the random variables 1]n have finite expectation,
I: IxldF(x) < 00,
then the condition of Theorem 12 holds, since then
It can be shown that in this case we have normal stability. Thus the
following theorem holds:
Theorem 13. The stability ofthe means O'n = (1]1 + ... + 1]n)/n,
n ~ 1, of a
sequenee of independent identieally distributed random variables with singular
distribution is normal if 2 and only if
EI1]l 1< 00.
The latter statement was earlier proved by A.Ya. Khinchin (G.R. Aead.
Sei. Paris 188 (1929), 477).
8 February 1929
2 The 'only if' part is trivial and follows from the fact that the definition of normal
stability makes sense only if the expectation exists.
5. ON THE LAW OF THE ITERATED LOGARlTHM *
We consider a sequence of independent random variables
Zl, Z2, .•• , Zn, .•• , (1)
with zero expectations Ezn . Further let
Following A.Ya. Khinchin we say 1 that the sequence (1) obeys the law of
iterated logarithm if the prob ability that
·
1lmsup Sn = 1 (2)
.j2Bn In In B n
is 1. In one important special case this law was established by Khinchin him-
self; 2 we will prove that it is applicable under the following conditions:
I. Bn --> 00,
II. IZnl ~ m = o( Vln!nBJ.

n
If the case Zn == 0 is excluded, the necessity of the first condition can be

easily derived. 1f the absolute values of all the Zn are uniformly bounded, then
the second condition follows from the first; in this case the first condition is
necessary and sufficient.
Without using the probabilities of relations that cannot be observed di-
rectly, for example, the event (2), the law of the iterated logarithm can be
formulated as follows:
10 • For arbitrary positive numbers "I and 6 there exists a positive integer
n such that the probability of at least one of the inequalities
for any p remains smaller than "I.
* 'Über das Gesetz des iterierten Logarithmus', Math. Ann. 101 (1929), 126-135.
1 See his book Basic laws 01 probability theory, Moscow, 1927 (in Russian).
2 Math. Ann. 99 (1928), 152.
32
ON THE LAW OF THE ITERATED LOGARITHM 33
2° . For arbitrary 7],6 and m there exists a non-negative integer p such

that the probability that all the inequalities
hold simultaneously is smaller than 7].
§1. Lemmas
Let
In the following lemmas x denotes a positive number. Since in this section

n is taken to be fixed, we shall omit it in our notation.
Lemma I. 11 xM $ B, then
Lemma 11. 11 xM ~ B, then 3
Prool 01 Lemmas land 11. Let a > 0, aM $ 1. Then
$ 1+ a;Tc (1 + a~) < exp [a;Tc (1 + a~)], (3)
Ee Gs = TI Ee GZk < exp [a~B (1 + a~)], (4)

Tc=1
3 See: S. Bernshtein, 'On a variation of Chebyshev's inequality and the error of
Laplace's formula', Uchen. Zap. Nauch.-Issl. Kafedr. Ukrain. Otd. Mat. 1 (1924),
38-48 (in Russian). (See also: Collected worb, Vol. 4, pp. 71-79.)
34 ON THE LAW OF THE ITERATED LOGARITHM
(5)
Formulas (4) and (5) imply that
W(z) a2B(1 + 2
< exp [-az + -2- aM)] .
For zM ~ B we set a = z/ B in the following formula and obtain Lemma I; for

zM > B we set a = 1/M and obtain
In both cases the condition aM ~ 1 is satisfied.

Lemma I immediately implies
Lemma IH. If zM ~ B, then
Lemma IV. If
zM/B = w < 1/256, (6)

z2/B = A > 512, (7)
then
W(z) > e-(X2/2B)(1+ f ),
where f denotes the mazimum of32..j(lnA)/A and 64y'w.
Proof. Let 6 = ~f, then
62 = max(64w, 161~A), (8)
and also, because of (6) and (7),
62 < 14' (9)

6 <~, (10)
6> 26 2 • (11)
We further set
a = z/(B(1 - 6)).
Clearly,
z = aB(1- 6), (12)

z/B < a < 2z/B, (13)
aM < 2w < 1/128, (14)
a2 B>A>512. (15)
Since for every positive u we have
1+ u > e u (1-u),
it follows from (3) that
(16)
The condition aM ~ 1 assumed here is satisfied in our case in view of (4).

Further, by (4) we have
(17)
Because of (14) and (8),

aM < 62 /4,
so that we finally obtain
a 2 B ( 1 - 4"
Ee os > exp[-2- 62 )] . (18)
-1: 1:
On the other hand we have
Ee os = eOlldW(y) =a eOIlW(y)dy =
=a ({O
1-00
+ l
0
OB (1-6) + l OB (1+6)
oB(1-6)
1
+ 80B
oB(1+6)
+ (>0) =
180B
(19)
Since W(y) ~ 1, it follows that
aJI ~a 1 0
-00
eaYdy ~ l. (20)
By Lemma II and (14), for all y ~ B/M we have
W(y) < e- y / 4M < e- 2ay .
By Lemma III we also have for 8aB ~ y ~ B/M
Therefore we obtain
aJs < a foo e-aYdy< l. (21)
lsaB
Since by (18), (9) and (15),
Ee aS > 8,
equations (20) and (21) imply
(22)
To estimate aJ2 and aJ4 we apply Lemma I. Because of (6) and (8), for y ~ 8aB
we have the inequalities
1
0< w < 862,
W (y) < exp [- : ; (1 - ~62) ].
Therefore we obtain
a(J2 +J4)<a( lo
aB (1-6)
+
jSaB
aB(l+6)
y2 1
)exp[aY-2B(1-862)]dy.
In both integration intervals the quantity
u(y) = ay - -y2 ( 1 - _6 2 1)
2B 8
does not exceed
Hence we have
Since by (7), (8) and (15),
In(32A) < 21nA ~ ~A62,

(23)
In(32a 2 B) < (a 2 B /8)6 2 ,
we finally obtain
(24)
From (19), (22) and (24) it follows that
ah > "21 Ee aS >"21 exp [a-2-

2
B( 1 - 6)] . (25)
On the other hand, since W(y) is monotone decreasing, (12) implies
(26)
From (25) and (26) we conclude that
W(x) > 4a12 B exp [a

2
B + 36) ] .
--2-(1
Since, along with (23) and for similar reasons, the formula
holds, finally we have
W(x) > exp[- a:B (1 + 46)] > exp [ 2B(;~ 6)2 (1 + 46)] >
> exp [- ; ; (1 + 86)] = exp [- ; ; (1 + f)] .

Lemma V.
W(x) ~ 2W(x - ,fiB).
Proof Let E denote the event
and E" the event

Si< U, i< k
S" = U ~ z.
Clearly,4
E=El+E2+ ... +En, W(z)=P{E}.
Since the Zi with i > k do not occur in the definition of the event E", we
have
PE" {lu,,1 ~ V2B} $ ~.

Since S = U + 0'", the following inequalities also hold:
PE,,{S~U-V2B}~ !'
W(z - V2B) = P{S ~ z - V2B} ~
~ W(Z)PE{S ~ U - V2B} ~ !W(z).
This proves our assertion.
§2. Proof of the first part (1°) of the main theorem
It suflices to eonsider the ease

6 <~. (27)
Clearly, by Conditions land 11 there exists an no such that
B no > e, (28)
JInlnBno > 4/6 (29)
and for any n ~ no

M~/Bn < 6/16, (30)
4 The + sign is used instead of U in order to stress the fact that the events in
question are pairwise incompatible.
M JlnlnBn ~ (31)
n Bn < 4'
After no, nl, ... , nk-l have been defined, we choose nk so that
(32)
(33)
By (30) and (32) we have
and hence, by (28),

(34)
Suppose further that

x(t) = v'2tlnlnt.
By (32) we have
(35)
The truth of at least one of the inequalities
implies
(36)
Denote by Vk the probability of (36). Then clearly the convergence of the

serles
(37)
suffices for Statement 10 to hold.

By (35) we have
(38)
while according to Lemma V, the latter prob ability does not exceed
Note that (29) implies
so that we have
and by Lemma I,
where, by (31),
()= MnkJlnlnBnk <~.
2 Bnk 8
This implies that
Vk < 2exp[-lnlnBnk (1+ ~)] < 2(lnBnk )-(1+6/ 4 ) <
< 2[kln(1 + ~)r(1+6/4) < Ck-(1+6/ 4 ),
which proves the convergence of the series (37).
§3. Proof of the second part 2° of the main theorem
As before, suppose that (27) holds. Conditions I and 11 and the first part of
the main theorem just proved imply that there exists an no such that, first (28)
holds, secondly, for every n ~ no,
(39)
and thirdly, the probability that all inequalities
IBnl < 2x(Bn ), (n = no, no + 1, ... , no + p) (40)
simultaneously hold is greater than 1 - 1]/2 for any p.

Choose nk so that
(41)
Then because of (27) and (39), we have
(42)
Set
Then (41) implies that
ßk > Bnk (l - 6/4), X(ßk) > X(B nk )(l- 6/4), (43)
X(ßk)(l- 6/4) > X(B nk )(l- 6/2). (44)
On the other hand, (41) also implies
(45)
Since Snk = (Tk + Snk_l' it follows that if (40) holds, then the inequality
(46)
together with (44) and (45), implies
(47)
Now we prove that for sufficiently large p the probability that at least one
of the inequalities (46) holds for k = 1,2, ... ,p is greater than 1 - 1//2. This
clearly implies that the corresponding prob ability for the inequalities (47) is
greater than 1 - 1/, which proves the second part of the main theorem.
Since the (Tk are mutually independent, for our purpose it suffices to prove
the divergence of the series
00
LVk (48)
k=l
of probabilities of the inequalities (46).

To estimate the probabilities Vk we make use of Lemma IV, setting
x = X(ßk)(l- 6/4).
For sufficiently large k, conditions (6) and (7) hold, since

as a consequence of (43); therefore
lim X2ßk
(ßk) >
- lim(llnlnB
2 nk
) -
-
00
.
From the latter formulas we can see that f tends to zero as k ---+ 00. Thus
for sufficiently large k we have
(1 + f)(l + 8/4) < 1
and, because of (42),
vk>exp[-X:~k)(l-~)] =exp[-lnlnßk(l-~)] =
= (In ßk)-(1-6/4) > (ln B nk )-(1-6/4) > Ck-(1-6/4),
which proves the divergence of (48).

Moscow, 24 November 1927
6. ON THE LAW OF LARGE NUMBERS *
By the law of large numbers in mathematical prob ability theory we usually
mean either Poisson's theorem or Chebyshev's theorem, with their generaliza-
tions to the case of dependent trials. However, this law can be given a broader
interpretation, more in agreement with its natural philosophical treatment, if it
is used whenever it can be asserted with prob ability elose to one that a certain
variable differs only very slightly from a certain apriori given constant. 1 In a
previous paper 2 I gave a theorem which, as I think, corresponds most elosely to
this intuitive concept. In this paper I would like to come back to this subject,
formulate a somewhat more general statement, and also elarify some questions
arising in this connection.
§1. For an accurate formulation of the problem, consider the sequence of real
numbers 3
where X n depends on the result of successive trials 4
Let us agree to say that X n is stable 5 or that X n obeys the law o/large numbers,
if there is a sequence of constants
such that for every positive 7],
(1)
* 'Sur la loi des grands nombres', Atti. Accad. Naz. Lincei. Rend. 9 (1929), 470-
474. Presented by G. Castelnuovo on 3rd March 1929.
1 See: S.N. Bernshtein, Probability theory, p.142, where a more general definition
is given.
2 C. R. Acad. Sei. Paris 185 (1927), 917.
3 Similar results can be obtained by considering vectors. V. Glivenko has informed
me that similar arguments hold even for vectors in Hilbert space of functions.
4 We assume throughout that these trials are independent.
5 Concerning the definition of stability, see my paper: 'Über die Summen durch
den Zufall bestimmter unabhängiger Grössen', Math. Ann. 99 (1928), 309 (Paper
4 in this volume.)
43
In most of the important cases dn can be taken to be the expectation E(Xn ).

If it turns out that
(2)
for every positive 71, then we say that X n has normal stability. It can be proved
that if X n - E(Xn ) is uniformly bounded, then (2) follows from (1), that is, in
this case the stability can only be normal.
According to Chebyshev we have
(3)
where
Thus the condition
Bn --+ 0 (Markov's condition) (4)
is a sufficient condition for normal stability of X n . If the absolute values of

X n - E(Xn ) are uniformly bounded by a constant M, then, as can easily be
shown,
(5)
Hence, in this case Markov's condition is not only sufficient, but also necessary
for the stability of X n which, as has already been mentioned, can only be
normal in the situation considered.
§2. In formulating Markov's condition we touched upon normal stability only.

However, this condition is interesting since in most cases it is easier to calculate
B n than Pn(71).
In the general case, we denote by Ek(Xn ) the expectation of X n and
assume that the results ofthe first k trials E~ n ) , E~ n) , ... ,E~n) are known. Then
we have the following fundamental formula
n
B~ = L:E[Ek(Xn ) - E k - 1 (X n )]. (6)
k=l
Indeed, it is dear that

ON THE LAW OF LARGE NUMBERS 45
therefore, setting
we have
X n - E(Xn ) = Znl + Zn2 + ... + Znn.
More generally, denoting by Ek(Y) the expectation of Y and assuming that
the results of the trials E~n), E~n), . .. ,Ein) are known, we have
Since Zni is constant for i < k, if the results of the first k - 1 trials are fixed,
we also have
and finally,
This latter equality immediately implies 6
2 - ß2
B n- ß2 ß2 (7)
nl+ n2+"'+ nn'
which differs from (6) only in notation.

Notice the meaning of ßnk: it is the mean squared deviation of the ex-
pectation of X n when the result of the trial Ein) is known. Therefore, ßnk is
a natural measure of the dependence of X n on the result of the trial 7 Ein).
This agrees with the concept of the law of large numbers formulated in the
beginning of the paper. However, now we measure the dependence of X n on
each trial Ein) using the moments ßnk. We have proved that the law of large
numbers can be efficiently applied to X n if the sum of the squares of ßnk IS
infinitesimally smalI. This is so if, in particular
ßnk = o(l/Fn). (8)
§3. Now we consider the case when the trials Ein) determining the value of X n
are independent. Denote by E k (Y) the expectation of Y when the results of
6 In my article (see footnote 5) this formula is proved only for the case of inde-
pendent trials.
7 It is assumed that the trials Ein) are sequentially carried out in accordance with
the order of their indices k.
all trials &ln) except &k n) are known. In the case under consideration, that is,
when the trials are independent, we have
(9)
(10)
Thus, for normal stability of X n the condition 8
(11)
is sufficient.
Formula (9) can be proved as folIows:
EdEk(Xn)] = Ek-l(Xn),
Ek[Xn - E k(X n )]2 ~ {Ek(Xn ) - E k [E k (Xn )]}2 = [Ek(Xn ) - E k- 1(XnW,
a~k = E[Xn - Ek(XnW ~ E[Ek(Xn) - Ek-l(XnW = ß~k·

In the general case of dependent trials the first of these formulas does not
hold.
Let M(k)(Xn ) be the least upper bound of all possible values of X n when
the results of all trials &;(n) except &k n ) are known, and let m(k)(Xn ) denote
the corresponding greatest lower bound. We set
Thus, Ünk is the maximal deviation of X n if only the result of the trial &k n) is
unknown. We see that
(12)
(13)
This implies the condition for stability of X n in the case of independent

trials, given in a previous paper of mine:
(14)
8 This condition is especially interesting, since the definition of the moments O'nk
[k
does not involve the order of the trials n ).
ON THE LAW OF LARGE NUMBERS 47
Finally we remark that in the case considered by Chebyshev,
where Xk depends on the trial c~,.) and these trials are independent, we have
which leads to the classical condition for stability of X,.:

7. GENERAL MEASURE THEORY AND PROBABILITY CALCULUS *
Probability theory has become a topic of interest in modern mathematics not
only because of its growing significance in natural sciences, but also because of
the gradually emerging deep connections of this theory with many problems in
various fields of pure mathematics. It seems that the formulas of prob ability
calculus express one of the fundamental groups of general mathematical laws.
However, it would have been imprudent to assert that these facts indi-
cate the dependence of notions used in pure mathematics on the not ion of
randomness dealt with in probability theory. For example, the fact that the
distribution of the digits of decimal expansions of irrationals can be studied
using formulas of probability calculus should not be interpreted as indicating
dependence of these expansions on chance.
On the contrary, I believe that these facts indicate the possibility of con-
structing a very general and purely mathematieal theory whose formulas can
be applied both to probability calculus and to many other fields of pure and
applied mathematics. To outline the contents of this theory, it suffices to single
out from prob ability theory those elements that bring out its intrinsic logical
structure, but have nothing to do with the specific meaning of the theory.
This way we come, first of all, to general measure theory. The general
not ion of measure of a set includes the notion of probability as a special case. A
set of arbitrary elements considered from the viewpoint of the measure defined
on its subsets will be called purely metne spaee, though perhaps it is an abuse
of the term "space". In particular, in prob ability calculus we speak of the space
of elementary events of a given problem and of the probabilities of various sets
of these events.
Function theory in purely metne spaees is a further development of general
measure theory. This theory studies the properties of functions that only de-
pend on the measures of the sets on which the functions take a certain range of
values. For instance, such are the properties of orthogonality of two functions
or completeness of a system of orthogonal functions. If we consider variables
depending on events (random variables) as functions defined on the space of el-
ementary events, then all the relevant statements of probability theory appear
to be particular cases of statements of such a general theory.
* Trudy Kommunist. Akad. Razd. Mat. 1 (1929), 8-21 (in Russian).
48
GENERAL MEASURE THEORY AND PROBABILITY CALCULUS 49
The strength of methods in probability theory as applied to problems

in pure mathematics is largely based on the notion of independence of random
variables. This notion has not yet achieved a purely mathematical formulation,
though it is not so difficult to give one. We then define "independence" of
a system of functions. This independence turns out to be a purely metric
property.
Properties of sets that at first glance are quite different when considered
merely from the viewpoint of the definition of measure established for their
subsets, often turn out to be identical.
For example, it can be proved by elementary means that there exists a
one-to-one map of a cube of arbitrary dimension onto an interval so that all
the (L)-measurable sets ofthe cube are mapped into sets whose linear measures
equal the measure of the initial sets. We say therefore that the cube of arbitrary
dimension is metrically equivalent to an interval. Apart from the notion of
metric equivalence, we define the more general notion of isometry of two spaces.
It is difficult to foresee whether we will need an early substantial develop-
ment of the outlines of the theory or whether it will suffice merely to indicate
that such is possible in principle. The latter is necessary anyway in order to
give a correct outlook on the connections between various fields of mathemat-
ics. Besides, this gives us a method for transferring reasonings from one field
to another field in which the statements of the general theory could be used.
I. ABSTRACT DEFINITION OF MEASURE
Consider a set A with elements a. We say that A is endowed with a measure

M if a certain measure M(E) is assigned to some of its subsets E. The set
A together with the measure M is ametrie space. We now give the axioms
satisfied by all the measures we shall be considering.
First, we assurne that the measure of a set is areal number, positive or
zero.
Axiom I.
M(E) ~ o.
Secondly, we assurne that iftwo sets do not intersect, then the measure oftheir
sum is equal to the sum of their measures.
50 GENERAL MEASURE THEORY AND PROBABILITY CALCULUS
Axiom 11. If
then
Here we assurne that the existence of measures for two of the sets involved
implies the existence of a measure for the third set.
Based on this axiom the general formula
(1)
can be proved. One should not assurne, however, that the existence ofmeasures
of two intersecting sets implies the existence of a measure for their sum or
difference: there are certain important measures without this property.
Since the product of two sets can be defined as
(2)
(1) can be used for deriving all the various relations among sums, differences
and products of a finite number of sets.
For convenience we include the empty set in the domain of sets to be
considered, and denote it by o. If for a certain measure at least one set has a
measure, then Axiom 11 implies that
M(O) = O. (3)
Finally, the third axiom intro duces a quite arbitrary restrietion on the measures
considered: the measure of the whole space is 1.
Axiom 111.
M(A) = 1.
This greatly simplifies the discussion and the general case can be easily
derived from this particular case. In many applications, in particular, in prob-
ability calculus, this restrietion is due to the essential nature of the subject.
Let us now consider some examples of measures satisfying these axioms.
1) The Lebesgue measure of point sets in an n-dimensional cube with
side equal to 1.
2) The density of linear sets at 0 can be considered as their measure.

3) If a set A consists of positive integers, then the set density can be
considered as its measure, that is, the limit of the ratio of the number of
elements not greater than n, to n, as n --+ 00.
4) If A is a set of elementary events of a problem in prob ability calculus,

then the measure of a set of elementary events is the prob ability that at least
one of them occurs.
H. CLOSEDNESS OF A MEASURE
§1. A given measure M determines the system (E) of sets E having a measure.
Suppose we know about this system that it includes the empty set 0 and the
whole space A, and also that if
and two of these sets belong to (E), then so does the third one. Clearly, the
complement of a set belonging to (E) also belongs to (E) since
E + E = A, E . E = O.
§2. A measure is said to be complete, if it consists of (that is, is defined on)

all the subsets of the space A.
A complete measure in which all one-element sets have zero measure is not
known. To prove the existence of such a measure without using the axiom of
choice seems to be a problem of great difficulty.
§3. We say that a measure M contains a measure M' given on the same set A
if (E) contains (E') and both measures coincide on the sets belonging to (E').
One should not assurne that any measure is contained in some complete
one; counterexamples are elementary. Still, for none of the measures usually
employed in mathematics has it been proved that it is impossible to consider
it as contained in a complete measure. On the contrary, using the axiom of
choice, S. Banach proved that linear Lebesgue measure is contained in a certain
complete measure. 1 In view of the metric equivalence of spaces of arbitrary
dimension the same is true for n-dimensional Lebesgue measure. But already
1 Fund. Math. 4.
in the three-dimensional case, for a complete measure containing the Lebesgue

measure the principle of equality of the measure of congruent sets cannot hold:
this is proved by Hausdorff's example on the decomposition of a sphere into
three sets, each of these sets being congruent to the sum of the two others to
within a countable set.
Even if the main set A is countable, the construction of complete measures
is quite difficult. For example, the generalized density problem for sequences
of real numbers, that is, to find a complete measure containing the measure 3)
of Part I, has difficulties similar to those encountered in constructing 2 a point
set that is not (L)-measurable.
§4. Accordingly, when posing problems in probability calculus, it should be

required to indicate for which events the probabilities are assumed to exist. It
is clear, for example, that for geometrical probabilities it is not wise to assurne
the existence of the hitting probability of a point for each point set in space.
§5. In most cases, however, the system (E) can be assumed to possess certain
closedness properties. We say that a system (E) is finitely c10sed if it contains
an sums of pairs of sets belonging to this system. Clearly, since
(4)
a closed system contains an the differences, and because of (2), it also contains
the products of sets belonging to this system.
§6. For each system (E) there is a certain minimal system F(E) that is closed
and contains (E). If one can assign to an sets of the system F(E) a measure
coinciding with the measure given for the sets of (E) and satisfying our three
axioms, then we say that the initial measure is c1osable.
It is not known whether every measure is closable. If closure is possible,
then it is not necessarily closable in only one way; examples of the latter are
elementary.
It would seem that it is very difficult to find a measure that closes the
measure given above under 3).
It is also doubtful if a measure connected with some problem in probability
calculus need be closed.
2 My results in C. R. Aead. Sei. Paris (1925) point out the similarity of these two
problems.
§7. A system (E) is called countably closed if it contains aJI possible countable
sums of its elements. A countably closed system contains countable products
of its elements, since
II E = L:E
n n (n = 1,2, ...). (5)
The study of countable closedness is closely connected with normality of mea-

sures, and in fact makes sense only for normal measures.
IH. NORMALITY OF A MEASURE
§8. A measure is normal if for sets having a measure the condition
implies
M(E) = L: M(En) (n = 1,2, ... ).
Of the measures mentioned above, those in 2) and 3) are not normal.
Measures corresponding to problems in prob ability calculus also need not be
normal.
§9. For a finitely closed normal measure it is easy to prove the formulas
n-1
M(~En) = ~M(En- ~Ek)' (6)
k=1
n
M(II En) = limM(II Ek)' n = 1,2, ... , (7)
k=1
which are true under the single condition that the sets E n and either their sum
or product have a measure.
§10. Moreover, for finitely closed normal measures the following Proposition
on countable coverings holds: if
E C ~En (n = 1,2, ... ),
then
M(E):5 ~M(En) (n = 1,2, ... ).
Indeed, set
n-l
E~ = E(En - 2:Ek) (n = 1,2, ... ).

k=l
Then clearly
E~ . E:.r, = 0, n # m;
2: M(En) ~ 2: M(E~) = M(E) (n = 1,2, ... ).
§11. A set E is called measurable with respect to a measure M if for any f
there exist two sequences of sets from (E),
such that
E C 2:En, E C 2:E~,
2: M(En) + M(E~) ~ 1 + f (n = 1,2, ... ).

By the generalized measure of a measurable set E we mean the infimum of the
sums
2: M(En) (n = 1,2, ... )
over the coverings of E consisting of sets from (E).
§12. Thus we obtain a certain new measure L(M) which measures any set
that is measurable with respect to M. If M is finitely closed and normal, then
L(M) satisfies Axioms 1-111, is normal and countably closed. The methods for
proving this are the same as those used for studying Lebesgue measure. Thus,
for any normal finitely closed measure there exists a countably closed measure
containing it.
§13. The measure L(M) is not the minimal countably closed measure con-
taining M. It is not too difficult to determine the minimal such measure B (M)
contained in L(M).
However, the measure L(M) has another remarkable property: a set that is
measurable with respeet to M has the same measure with respeet to any normal
measure eontaining M as with respeet to L(M).
§14. In particular, in prob ability calculus, if a system of events with certain

probabilities is finitely closed and normal, then without contradictjon we can
uniquely define the prob ability of the countable product or the countable sum
of events with probabilities. It would seem that only after this has been done
can an argument involving probabilities of events such as convergence of series
of random variables, etc., be considered justified.
IV. METRIC EQUIVALENCE AND ISOMETRY OF SPACES

§15. Two metric spaces are called metrieally equivalent if they can be put
into a one-to-one correspondence so that sets with a measure correspond to
sets with a measure, and measures of corresponding sets are equal.
§16. The cube of arbitrary dimension with side equal to 1 and with Lebesgue
measure on its subsets is metrically equivalent to an interval of length 1, also
with Lebesgue measure. The proof of this is elementary.
§17. Two sets EI and E 2 in a metric space have the same metrie type if
In many cases when we can neglect sets of measure zero only the metric
types, not sets themselves, need be involved in the argument. 3
If a measure is finitely closed, then the type of the sum, product or differ-
ence of two sets with a measure depends only on their types and consequently
we can speak of the sums, products and differences of the types themselves.
If a measure is countably closed and normal, then so are the countable
sums and products.
§18. Two finitely closed spaces are called isometrie if a one-to-one corre-
spondence can be established between the metric types of their subsets with
measures in such a way that the· sum of two types corresponds to the sum of
the corresponding types, and the measures of corresponding types are equal.
3 Of course, everything that follows can be presented without introducing the

notion of metric type as based on the "subdivision" axiom. Instead the notion
of metric "equality" of sets can be used.
56 GENERAL MEASURE THEORY AND PROB ABILITY CALCULUS
Obviously, the products and differences of two types also correspond to

the products and differences of the corresponding types. If both measures
are normal and countably closed, the same is also true for countable sums
and products. Note, however, that a countably closed normal measure can be
isometrie to a non-normal measure.
§19. A measure has a eountable basis ifthe system (E) has a countable sub-
system (I) (a basis) such that any set of (E) is measurable for the basis (1),
that is, this set and its complement each have a covering by sets of (1) in which
the sum of the measures of the covering sets is greater than their measure by
an arbitrary small value. Of course, this definition is interesting only when the
space is normal.
The Lebesgue measure on an interval has a countable basis consisting
of segments with rational ends points. Normal spaces with the cardinality
of the continuum and with a countable basis are not known (except those
determined using the axiom of choice). Such aspace of cardinality 2C can be
easily constructed.
§20. A measure M that is normal, finitely closed and with a eountable basis
is isometrie to the measure M l , wh ich is the Lebesgue measure 01 the interval
(0,1).
§21. In conclusion, since many properties ofpurely metric spaces depend only
on relations between the metric types of their subsets, which can be identical
for spaces that are most different, even for spaces of different cardinalities, it
would be worth trying to set forth the theory of such spaces considered as
systems of metric types that can be added, multiplied, etc., without assuming
the existence "elements" of the space.
v. PURELY METRIC FUNCTION THEORY
§22. We will consider only functions that take real values at all points of a
purely metric space. Such a function is called measurable if the set of points at
which it takes values belonging to an interval is always measurable.
§23. Clearly, the set on which a function takes a certain value a is also mea-
surable, since
E[b < 1 < a] + E[I = a] + E[a < I< c] = E[b < I< cl·
If the measure in the space considered is countably closed, then the set
on which a measurable function takes a (B)-measurable set of values has a
measure, since it can be obtained by countable additions and multiplications
of sets that obviously have a measure. But one cannot claim that the same
is true for an (L)-measurable set of values of a function, even if the measure
considered is of the type L(M).
§24. A family of functions [f] is called equimeasurable if, for an arbitrary

choice of intervals corresponding to the functions, the set on which every func-
tion takes values in its corresponding interval is measurable.
For a finitely closed measure a finite family of measurable functions is
measurable. For a countably closed measure the same is true for a countable
family of functions.
For a countably closed measure the set on which the values of a sequence
of functions determine the points of a (B)-measurable set in a countable di-
mensional space has a measure. This applies, for instance, to the set of points
of convergence of aseries of functions.
All this can be directly applied to random variables in probability theory.
§25. The integral of a function f over a set E can be defined in a purely

metric theory only with respect to the Lebesgue measure as
. L
hm
n_oo
oo
rn
-M
n n
rn + 1) E,
[(rn-<f:5--n
]
m=-oo
provided that the series converges absolutely and the limit exists. When con-
sidering problems connected with the notion of integral we assurne the space
to be normal.
§26. It can be proved in the usual way that for a bounded measurable function
the integral exists.
§27. For an integral defined in such a way the following relations hold:
4. lim LIn = L', if 11 < 12 < ... < In ... -+ f.
§28. The notation of expectation whieh is dose to that of the Lebesgue inte-
gral, has long been used in probability theory. If prob ability is eonsidered as
a measure on the set of elementary events, then the expeetation of a random
i z,
variable Z is
D(Z) =
and its expectation under a hypothesis E is
DE(Z) = M~E) L Z.
By analogy with the notion of relative probability we introduee the nota-

tion
then
m=-QO
§29. Based on this definition of integral the theory of orthogonal lunctions in

purely metrie spaees ean be developed. For example, for a system of bounded
orthogonal functions to have at most eountable eardinality it is suffieient that
the measure eonsidered have a eountable basis.
It would be interesting to see to what extent purely metrie function theory
ean be presented in terms invariant for isometrie spaees. In this ease the very
definition of a function should be different.
VI.INDEPENDENCE
§30. Two partitions of a spaee A into non-interseeting parts
A= LF', A= LF"
are independent if for any measurable sets E' and Eil eomposed from the ele-
ments of eertain sets F' and F" respeetively,
M(E' . Eil) = M(E') . M(E").

§31. The (finite or infinite) product of partitions [F] is the partition of the
space into the products of elements of the given partitions, with one element
taken out of every partition.
A finite or infinite number of partitions [F] are mutually independent if
any two products of these partitions without common elements are indepen-
dent. Mutual independence of partitions does not imply independence of these
partitions.
§32. These general definitions allow a number of simplifications in particular

cases. For instance, if a measure is finitely closed and the partitions (F') and
(F") consist of a finite number of elements with a measure, then they are
independent if
M(F' . F") = M(F') . M(F")
for each pair F' and F".

If the measure is normal, the same is true for partitions into a countable
set of parts.
§33. Every function defines a partition of the space into the sets at which
it takes a certain value. Functions are called mutually independent if their
corresponding partitions are independent.
§34. Independence of partitions of a set (E) and independence of functions

on a set are defined in exactly the same way.
For functions ft, f2, ... , In, independent on a set E, the following relation
holds:
DE(ftf2 ... In) = DE(ft)DE(f2) ... DEUn).
8. ON THE STRONG LAW OF LARGE NUMBERS *
Let
Xl, X2, ••• , X n , ... (1)
be a sequence of independent random variables with zero expectations E(xn ).

Following Cantelli and Khinchin we say that (1) satisfies the strong law 0/ large
numbers (SLLN) if the probability of convergence to zero of the means
_
O"n -
Sn _
- -
Xl + X2 + ... + X n
-------
n n
is equal to 1.
We will now prove that the SLLN holds ifthe second moments E(x~) = bn
exist and the senes
(2)
converges. This condition cannot be replaced by any weaker one: if for some
sequence of constants bn (2) diverges, then we can construct a sequence (1)
of independent random variables satisfying E(x n ) = 0, E(x~) = bn but not
satisfying the SLLN.
To prove this, we can use the lemma expressed by the formula
(3)
which was proved elsewhere (for R ~ 0).1
* 'Sur la loi forte des grands nombres',

C. R. Acad. Sei. Paris 191 (1930), 910-912.
1 'Über die Summen durch den Zufall bestimmer unabhängiger Grössen', Math.
Arm. 99 (1928), 310, Satz 1 (Paper 4 in this volume).
60
ON THE STRONG LAW OF LARGE NUMBERS 61
Clearly, according to (3),
n=l
00 1 00 1 2
= P{limsup Iunl > f} ~ L: Pm ~ f2 L: (2 m )
(4)
m=O m=O
~ f12 ?= L:. (2~ f

00 00 n<2 41
L:. bn ~
,=0 m=' n=2'
i +1
1 00 1 2 n <2 Soobn
~ f2 L:(2i-1) L: bn ~ f2 L: n 2 '
i=O n=2i n=l
But P does not change if a finite number of first terms in (1) are replaced
by 0. If (2) converges, then the last term in (4) can be made arbitrarily smalI.
Hence P = 0. Since this is true for any f > 0, in the case under consideration,
the SLLN holds.
To prove the second part of our statement, assume that (2) diverges.
For bn /n 2 ~ 1 let Zn = n, Zn = -n, Zn = ° with probabilities bn /2n 2 ,
bn /2n 2 , 1 - bn /n 2 respectively, while for bn /n 2 ;::: 1 we set Zn = A and
Zn = -A with probabilities ! and !. It can easily be seen that E(zn) = 0,
E(z~) = bn and the SLLN does not hold.
9. ON ANALYTICAL METHODS IN PROBABILITY THEORY*
The object of investigation
A physical process (a change of a certain physical system) is called stochastically

determined if, knowing astate X o of the system at a certain moment of time
to we also know the prob ability distribution for all the states X of this system
at the moments t > to.
I systematically consider the simplest cases of stochastically determined
processes, and primarily, processes continuous in time (this is what makes the
method essentially new: so far, a stochastic process has usually been considered
to be a discrete sequence of separate "events").
If the set ~ of different possible states of the system is finite, then a
stochastic process can be characterized using ordinary linear differential equa-
tions (Chapter 11). If astate of the system depends on one or several continu-
ous parameters, then the corresponding analytic apparatus reduces to partial
differential equations of parabolic type (Chapter IV) and we obtain various
distribution functions, the normal Laplace distribution being the simplest.
INTRODUCTION
1. In order to subject social or natural phenomena to mathematical treat-

ment, these phenomena should first be schematized. The fact is that mathe-
matical analysis can only be applied to studying changes of a certain system if
every possible state of this system can be completely determined using known
mathematical techniques, for example, by the values of a certain number of
parameters. This mathematically defined system is not a reality itself, but a
scheme that can be used to describe reality.
Classical mechanics makes use only of the schemes for which the state y of
a system at time t is uniquely determined by its state x at any preceding time
to. Mathematically this can be expressed by the formula
y = I(x, to, t).
If such a unique function 1 exists, as is always assumed in classical me-

chanics, then we say that our scheme is a scheme 01 a purely deterministic
* Über die analytischen Methoden in der Wahrscheinlichkeitsrechnung, Math. Ann.

104 (1931), 415-458.
62
ON ANALYTICAL METHODS IN PROBABILITY THEORY 63
process. These purely deterministic processes also include processes when the
state y is not completely determined by giving astate z at a single moment of
time t, but also essentially depends on the pattern of variation of this state z
prior to t. However, usually it is preferred to avoid such a dependence on the
preceding behaviour of the system, and to do this the notion of the state of the
system at time t is generalized by introducing new parameters. 1
Outside the re alm of classical mechanics, along with the schemes of purely
deterministic processes, one often considers schemes in which the state z of the
system at a certain time t o only determines a certain probability of a possible
event y to occur at a certain subsequent moment t > to. If for any given
to, t> to, and z there exists a certain probability distribution for the states y,
we say that our scheme is a scheme 0/ a stochastically determined process. In
the general case this distribution function can be represented in the form
P(t o, z, t, (1;)
where (1; denotes a certain set of states y, and P is the prob ability of the fact
that at time tone of the states y belonging to this set will be realized. Here
we face a complication: in general, this probability cannot be determined for
all sets (1;. A rigorous definition of a stochastically determined process which
enables one to avoid this complication is given in §l.
As in the case of a purely deterministic process, we could also have consid-
ered here schemes in which the probability P essentially depends not only on
the state z but also on the past behaviour of the system. Still, this influence
of the past behaviour of the process can be bypassed using the same method
as in the scheme of a purely deterministic process.
Note also that the possibility of applying a scheme of either a purely de-
terministic or a stochastically determined· process to the study of some real
processes is in no way linked with the quest ion whether this process is deter-
ministic or random.
2. In probability theory one usually considers only schemes according to which

any changes of the states of a system are only possible at certain moments
1 A well-known example of this method is to introduce, in addition to positions

of points, the components of their velocities when describing astate of a certain
mechanical system.
64 ON ANALYTICAL METHODS IN PROBABILITY THEORY
t 1 ,t 2 , •.. ,tn , ... which form a discrete series. As far as I know, Bachelier 2
was the first to make a systematic study of schemes in which the probability
P(to, x, t, Q:) varies continuously with time t. We will return to the cases studied
by Bachelier in §16 and in the Conclusion. Here we note only that Bachelier's
constructions are by no means mathematically rigorous.
Starting from Chapter 11 of this paper we mainly consider above-mentioned
schemes that are continuous with respect to time. From the mathematical
point of view these schemes have an important advantage: they allow one to
introduce differential equations for P with respect to time and lead to simple
analytic expressions which in the usual theory can be derived only as asymptotic
formulas. As for the applications, first the new schemes can be directly applied
to real processes, and secondly, from the solutions of differential equations
for processes continuous with respect to time new asymptotic formulas for
continuous schemes can be derived, as will be shown later in §12.
3. We do not start with the complete system ofaxioms of probability theory.

Let us indicate, however, all the prerequisites we will use in our further dis-
cussion. We do not make any special assumptions about the set 21 of possible
states x. Mathematically, 21 can be considered as an arbitrary set consisting of
arbitrary elements. All assumptions concerning the system ~ of sets and the
function P(to,x,t, Q:) are given in §1. In what follows the theory is developed
as a purely mathematical one.
CONTENTS
Chapter I. Generalities:
§1. General scheme of a stochastically determined process.
§2. The operator F1(x, Q:) * F2 (x, Q:).
§3. Classification of particular cases.
§4. The ergodic principle.
Chapter 11. Finite state systems:

§5. Preliminary remarks.
§6. Differential equations of a continuous stochastic process.
2 I. 'Theorie de la speculation', Ann. Ecole Norm. Super. 17 (1900), 21; 11. 'Les
probabilites a plusieurs variables', Ann. Ecole Norm. Super. 27 (1910), 339; IH.
Calcul des probabilites, Paris, 1912.
ON ANALYTICAL METHODS IN PROB ABILITY THEORY 65
§7. Examples.
Chapter IH. Countable state systems:

§8. Preliminary remarks. Discontinuous schemes.
§9. Differential equations of a process continuous in time.
§1O. Uniqueness of solutions and their calculation for a process homogenous
in time.
Chapter IV. Continuous state systems, the case of one parameter:

§11. Preliminary remarks.
§12. Lindeberg's method. Passage from discontinuous to continuous schemes.
§13. The first differential equation for processes continuous in time.
§14. The second differential equation.
§15. Statement of the uniqueness and existence problem of solutions of the
second differential equation.
§16. Bachelier's case.
§17. A method for transforming distribution functions.
§18. Stationary distribution functions.
§19. Other possibilities.
Conclusion.
CHAPTER I. GENERALITIES
§l. General scheme of a stochastically determined process
Let (5 be a system that can be in states z, y, z, ... , and ~ a system of sets ~

formed from the elements z, y, z, . ... A process of variation of the system (5
is stochastically determined with respect to ~ if for any choice of state z, set ~
and moments t l and t2 (tl< t2) the prob ability P(tl,z,t2,~) ofthe fact that,
if z takes place at tl, then one of the states of ~ takes place at t2 exists. If
P(tl,z,t2,~) is defined only for t2 > tl ~ to, then we say that the process of
variation is stochasticany determined for t ~ to.
Regarding the system ~, we assume that it is first additive (that is, it
contains all the differences, as weIl as finite or countable sums ofits elements),
and secondly contains the empty set, the set ~ of an possible states z, y, z, ...
and an the one-element sets. If the set 2( is finite or countable, then clearly
~ consists of an the subsets of~. In the most important case when ~ is
uncountable, the assumption that ~ contains all the subsets of ~ does not hold
for any of the schemes known at present.
Of course we assume that
(1)
and for the empty set sn,
We further assume that P(t 1 ,:1:, t 2, IE) is additive as a function of IE, that is,
for any decomposition of IE into a finite or countable number ofnon-intersecting
summands IEn the following identity holds
(2)
n
To formulate further assumptions on P(tl,:1:, t2, IE) we need the notion of

measurability of a function 1(:1:) with respect to the system ~ and the definition
of abstract Stieltjes integral. We given them here in a form suitable for our
needs. 3
A function 1(:1:) is called measurable with respect to the system ~ iffor any
choice ofreal numbers a and b the set 1E~(a < 1(:1:) < b) of all:1: for which 1(:1:)
satisfies the inequality in parentheses, belongs to~. It can easily be shown
that if the system ~ is additive and 1(:1:) is measurable with respect to ~, then
the set IE of all :1: for which 1(:1:) belongs to a given Borel-measurable set is
contained in ~.
Now let 1(:1:) be measurable with respect to ~ and bounded, and let </J(IE)
denote a non-negative additive function defined on ~; then, as is known, the
sum
I:m</J(m
n n
~1(:1:)< m+l)
n
m
tends to a weH defined limit as n --+ 00. This limit will be caHed the integral
[ 1(:1:)</J(d~).
lr~3:
3 Concerning these notions, as weH as additive sets of systems, etc., see, for ex-
ample, M. Frechet, 'Sur l'integrale d'une fonctionneHe etendu a. un ensemble
abstrait', Bull. Soc. Math. France 43 (1915),248.
ON ANALYTICAL METHOnS IN PROBABILITY THEORY 67
This notation differs from the usual one only in the specification of the
variable of integration and the place of the differential inside the parentheses.
In what follows we assume that P(t l , X, t2, ~), as a function of the state x,
is measurable with respect to the system~. Finally, P(it, x, t2, ~) must satisfy
the fundamental equation
(3)
for arbitrary t l , t 2, t 3, (tl< t 2 < t 3). If ~ is a finite or countable set of elements

Xl,X2,"" xn , ... , then
and on the right-hand side we have the expression for the total prob ability
P(t l , x, t3, ~); therefore (3) is satisfied in this case. In case ~ is uncountable
we take (3) as a new axiom.
The above requirements completely define a stochastically determined pro-
cess: the elements x, y, z, ... of an arbitrary set ~ can be considered as charac-
teristics of astate of a certain system, and an arbitrary function P(tl, x, t2, ~)
satisfying the above requirements as the corresponding probability distribution.

A non-negative function F(~) defined on ~, additive and such that
F(~) = 1, (4)
will be called a normal distribution function. All the requirements imposed

on P(tl,X,t2,~)can now be formulated in the following way: P(t l ,x,t 2, ~),
as a function of ~, is a normal distribution function; as a function of x it is
measurable with respect to the system ~ and satisfies the integral equation (3).
Suppose now that at t = to we have a normal distribution function Q(to, ~)
which gives the probability of the fact that the system <5 at to is in one of the
states belonging to ~. The distribution function Q(t, ~) for t > to is determined
by means of the second fundamental equation
Q(t,~) = f P(to, x, t, ~)Q(to, d~). (5)

Jv. x
We clearly have
Q(t,~) = l Q(to, d~) = Q(to,~) = 1, (6)

= [ [ P(tl, x, t2, ~)P(to, y, tl, d!!))Q(to, d!!') =

Jg;; Jg",
= Jg;;
[ P(to, y, t2, ~)Q(to, d!!') = Q(t2, ~). (7)
Formula (5) is considered as the definition of Q(t, ~), not as a new re-
quirement imposed on 6. Note, however, that (5) implies (3) as a particular
case.
Let F 1 (x, ~) and F 2 ( x, ~) be two normal distribution functions which, consid-

ered as functions of x, are measurable with respect to j. Set
It is easy to see that F(x,~) satisfies the same conditions of measurability and
additivityas Fl(X,~) and F2(X,~) and (4) also holds:
consequently, F(x, ~) is also a normal distribution function.

Further , the operator F1 * F2 is associative,
(9)
which can easily be seen by the following simple calculation:
F1 * (F2 * F3)(X,~) = [ [ F3(z, ~)F2(Y, d!!')F1(x, d!!) =

Jgy Jg~
= [ F3(Z,~) [ F2(Y, d!!')F1(x, d!!) = (F1 * F2) * F3(x, ~).
Jg~ Jgy
By contrast, F1 * F2 is not, in general, commutative.
Now we define the unit function I'(x, ~), which for any normal distribution
function F( x, ~) satisfies
p. * F(x, ~) = F * p.(x, ~) = F(x, ~). (10)
To this end it sufIices to set p.(x, ~) = 1 when x belongs to ~ and p.(x, ~) = 0

otherwise. We then have
p. * F(x, ~) = JglI
f F(y, ~)p.(x, fAl) = F(x, ~),
F * p.(x, ~) = JglI
f I'(y, ~)F(x, d9.) = f F(x, d~) = F(x, ~).
J~
The prob ability P(tl, x, t2,~) has been defined so far only for t2 > tl; now
set for any t
P(t,x,t,~) = I'(x, ~). (11)
In view of (10), this new definition does not contradict the fundamental equa-
tion (3), since (3) can be written as
(12)
§3. Classmcation of particular cases
If the changes in the state of the system 6 take place only at certain moments
which form a discrete series
to < tl < t2 < ... < tn < ... --+ 00,
then obviously
(13)
for all moments t' and t" such that
Introducing the notation
P(t m, x, tn,~) = Pmn(x, ~), (14)

Pn-l,n(X,~) = Pn(x, ~), (15)
we have
(16)
Hence in this case the process of change of (5 is totally deterrnined by the
elementary distribution functions Pn(x, IE).
Now let P1(x, IE), P2 (x, IE), ... , Pn(x, IE) be arbitrary normal distribution
functions which are assumed to be measurable as functions of x; further, let
to < tl < ... < t n < ... be a certain sequence of moments of time. Defining
Pmn(x, IE) and P(t', x, t", IE) by (16), (14) and (13), we also obtain normal
distribution functions which satisfy the equations
Pmn(x, IE) * Pnp(x, IE) = Pmp(x, IE) (m< n < p), (17)
and hence the equation
P(t',x,t", IE) * P(t",x,t lll , IE) = P(t',x,t lll , IE) (t' < t" < t lll ).
But this latter equation is none other than the fundamental equation (12) or
(13). Thus we see that every sequence of arbitrary normal distribution func-
tions Pn(x, IE), measurable as functions of x, characterizes a certain stochasti-
cally deterrnined process.
The schemes with discrete time defined above are those usually considered
in prob ability theory. If all the distribution functions Pn(x, IE) coincide,
Pn(x, IE) = P(x, IE), (18)
we have a homogeneous scheme with discrete time; in this case (16) yields
Pn n+p(x, IE) = P(x, IE) * P(x, IE) * ... * P(x, IE) = [P(x,
I , .,.. .J
IE)]~ = PP(x, IE).
p times
(19)
As far back as 1900 Bachelier considered stochastic processes continuous
in time. 4 There are good grounds for giving schemes with continuous time
a central place in probability theory. It seems that most important here are
schemes homogeneous in time, in which P(t,x,t + T, IE) depends only on the
difference t2 - tl:
P(t, x, t + T, IE) * P( T2, x, IE) = P( T, X, IE). (20)
4 See the first of the papers cited in footnote 2.

The fundamental equation in this case takes the form
(21)
Another series of particular cases is obtained under special requirements on

the set ~ of elementary states z. Here one should distinguish the cases of finite
or countable sets ~; in the continuous case the classification is performed with
respect to the number of parameters determining the state of the system. The
subsequent sub division of the material in this paper is based on distinguishing
such particular cases.
§4. The ergodie principle
Without special assumptions on the set 2( of all possible states z, we can only
prove several general theorems, namely those dealing with the ergodic principle.
We say that a stochastic process obeys the ergodie principle if for any t(O), z, y
and ~
lim [P(t(O) , z, t,~) - P(t(O), y, t, ~)] =
t_oo
o. (22a)
For a scheme with discrete time (22a) is clearly equivalent to the following:
(22b)
and in the latter case the following theorem holds:
Theorem 1. If for any z, y, and ~
(23)
and the series 00
(24)
n=l
diverges, then the ergodie principle (22b) holds and the limit in (22b) is uniform
with respect to z, y and ~.
Proof Let
SUpPkn(Z,~) = Mkn(~),
z:
For i <k we clearly have

and, similarly,
(26)
By (23), for any x and Y we have
Pk(X, e:) - >'kPk(Y, e:) ~ 0,
Pk-1,n(X, e:) =[ Pkn(z, "e:)Pk(x, dl2l) =

JfJ.z
=[ Pkn(Z, e:)[Pk(x,dl2l) - >'kPk(y,dl2l)]+
JfJ.z
+>'k [ Pkn(Z, e:)Pk(y,dl2l) ~
JfJ.z
~ mkn(e:) [ [Pk(x,dl2l) - >'kPk(y,dl2l)] + >'kPk-l,n(Y, e:) =
JfJ.z
=mkn(e:)(1- >'k) + >'kPk-l,n(Y, e:),
Pk-l,n(Y, e:) - Pk-l,n(X, e:) ~ (1- >'k)[Pk-l,n(Y, e:) - mkn(e:)];
hence by (25),
Since (27) holds for any x and y, we also have
Setting k = m+ 1, m+ 2, ... , n successively in (28) and multiplying all the

resulting equalities we find
n
Mmn(e:) - mmn(e:) ~ II (1- >'k). (29)

k=m+l
The right-hand side of (29) tends to zero as n ~ 00; this proves the
theorem.
For a homogeneous scheme with discontinuous time the following holds:
Theorem 2. If for any x, Y and e:
P(X, e:) ~ >.P(y, e:) (>' > 0), (30)
then pn(x, e:) converges uniformly to a certain distribution function Q(e:).

Proof. We have
Mn.n+p(~) = supPP(z,~) = Mp(~),
mn.n+p(~) = inf PP(z,~) = mp(~),
An = A,
and, by (29),
Mp(~) - mp(~) $ (1- A)P. (31)
But (25) and (26) imply that for q > p,
pq(z,~) = POq(z,~) $ Mq_p.q(~) = Mp(~), (32)

pq(z,~) ~ mp(~); (33)
therefore,
(34)
Our theorem now follows immediately from (31) and (34).

Important particular cases of Theorem 2 were proved by Gostinskii and
Hadamard. 5 As has been shown by Hadamard, in these particular cases Q(~)
satisfies the integral equation
Q(~) = r
J!lz
P(z, ~)Q(d~). (35)
. For the most general stochastically determined scheme one has:
Theorem 3. If for some sequence
to < ti < ... < t n < ... -+ 00

and any z, y and ~,
and if the series E::i An diverges, then the ergodie principle (22a) holds and
the convergence in (22a) is uniform with respect to z, y, ~.
Proof. For a given t(O) let
sup P(t(O), z, t, ~) = M(t, ~),

:z:
inf P(t(O), z, t, ~)
:z:
= met, ~).
5 C. R. Acad. Sei. Paris 186 (1928), 59; 189; 275.
If
then, in the same way as in the proof of Theorem 1, we obtain the analogous
formula to (29),
M(t, ~) - m(t, ~) ~ rr
k=m+l
n
(1 - Ak)'
Since n -+ 00 together with t, it follows that M(t, ~) - m(t, ~) -+ 0 as t -+ 00,
which proves the theorem.

Finally in case of a scheme homogeneous in time one has the following
theorem, analogous to Theorem 2:
Theorem 4. If there exists u such that for any x, y, ~,
P(u, x,~) ~ AP(U, y,~) (A > 0), (37)
then P( T, x,~) converges uniformly to a certain distribution function Q(~) as

T -+ 00.
CHAPTER 11. FINITE STATE SYSTEMS
§5. Preliminary remarks
Let us now assurne that 2l is formed from a finite number of elements
In this case set

(38)
Since for any set ~ we obviously have
P(tl,Xi,h,~) = L Pik(it,t2), (39)

XkC~
we can confine ourselves to the probabilities Pij(tl, t2)' The fundamental equa-
tion (3) now takes the form
L Pij(tl,t2)Pjk(t2,t3) = Pik(tl,t3), (40)

j
whereas (1) can be written as
LPij (t b t 2 ) = l. (41)
j
Any non-negative functions Pij(tl,t2) satisfying the conditions (40) and (41)
determine some stochastically determined process of variation of the systems
6.
In this case the operator is defined as folIows:
Fil, = Fg) * Fi~) = L FS) FR), (42)

j
hence the fundamental equation (40) reduces to
(43)
For a scheme with discontinuous time we set
p.pq ( ZI,. zJ.) -- p(pq)

ij ,
Then the probabilities Pi~) satisfy
(44)
and, conversely, arbitrary non-negative values p;~) satisfying (44) can be con-
sidered as the corresponding values of the probabilities of a certain stochasti-
cally determined process.
The probabilities P;~q) can be calculated by the formula
(45)
For a homogeneous scheme with discontinuous time we have
p~!,q) = [P.J·]q-p = p.q.-p

IJ • • IJ·
If all the Pij are positive, then obviously the conditions of Theorem 2 (§4)
hold, hence Pi~ tend to a certain limit Qj as q --+ 00. The integral equation
(35) transforms in our case into the system of equations
Qi=LQjPji (i=l, ... ,n). (46)

j
These results were obtained by Gostinskii and Hadamard. 6
§6. Differential equations of a continuous stochastic process
By (11) we have
~i(t, t) = 1, Pij (t, t) = 0, i -I j. (47)
If the variations of our system <5 are possible at any time t, then it is natural
to suppose that
lim Pii(t,t+ß)=l, limPij(t,t+ß)=O, i-lj, (47a)

~_oo ~_o
that is, for small time intervals the probability of a change in the state of
the system is small. This assumption is contained in the hypothesis of the
continuity of the functions Pij(t 1 , t2) with respect to t 1 and t 2.
Now assume that the functions Pij (8,t) are continuous and differentiable
with respect to t and 8 for t -I 8. We do not require differentiability of these
functions at t = 8. It would be imprudent to assume apriori the existence of
a derivative at these special points. 7
For t > 8 we have
If the deterrninant
s= IPij (8, t)1
is non-zero, then the equations
'"' D ( ) Pilc(t, t + ß) n ( ) Pu(t, t + ß) - 1 (i=l, ... ,n)

L...J r ij8,t ß +rik 8,t ß =O:il:
j#
can be solved:
Pkk(t,t + ß) -1 _ Au
(49)
ß S'
6 See Footnote 5.
7 Compare with the functions F(s,x,t,y) considered in Chapter 4, which neces-
sarily have points of discontinuity at t = s.
Since by (48) (Xik tend to the limit values OPik(S,t)/ot as ß --+ 0, the values
(49) tend to weIl defined limits 8
r1m Pkk(t, t ß+ ß) - 1 - A ()
-Ht, (50a)
· Pjk(t,t+ß) A ()
I1m ß = jk t , j i' k. (50b)
In fact it is evident from the relation
lim3 = 1,
, ..... t
(51)
which holds by (47) and the continuity of 3, that 3 may be non-zero under a
proper choice of S < t.
From (48) and (50) we immediately obtain the first system 0/ differential
equations for the function Pik(S, t):
OPik(S,t)
o "" () ( ) =Piks,t
=L..,..AjktPijS,t ( ) *Aik ()
t . (52)
t .
J
In this case, by (47) and (50),
Ajk(t) = [OPjk(t, u)] , (53)
ou u=t
Ajk ~ 0, j i' k, AH ~ 0, (54)

and, by (41) and (50),
LAjk = O. (55)
k
The equations (52) were established only for S < t; however, (47) and (53)
show that these equations are valid also for t = s.
For S < t we have
OPik(S, t) = lim Fh(s + ß, t) - Pik(S, t) =
os .1 ..... 0 ß
= li~o![Pik(s+ß,t)- L~j(s,S+ß)Pjk(S+ß,t)] =
j
· [~i(s,S+ß)-lp' ( A)
= - .111m
..... 0
A
~
ik S +~, t +
+ L Pij(S,~ + ß) Pjk(s + ß, t)] (56)

#i
8 We could equally weIl have taken the opposite approach: to assurne apriori that
the conditions (47a) and (50) hold and to derive from this the continuity and
differentiability of the function Pij (s, t) with respect to t.
and, by (50), we obtain the second system 0/ differential equations
8 t)
8Pil.(S,
S
= - "'"
L.JAij(S )Pj" (s,t) = -Ai" (S) * Pi" (s,t) .
.
(57)
J
If the functions Aij(s) are eontinuous, then clearly the equations (57) are also
true for s = t.
Now assurne that at to we know the distribution function
Q(to, z,,) = Q,,(to), L Q,,(to) = 1,

"
of the probabilities that the system (5 at t o is in the state z". Then equation
(5) takes the form
Q,,(t) = L Qi(tO)Pi,,(to, t).
By (52) the functions Q,,(t) satisfy
dQ,,(t) = LAj,,(t)Qj(t) (k = 1, ... , n). (58)

dt .
J
If the functions Ai,,(t) are eontinuous, then the functions Pi"(S, t) form
the unique system of solutions of (52) satisfying the initial eonditions (47);
eonsequently, the eonsidered stoehastie proeess is totally determined by all the
Ai,,(t). The real meaning of the functions Ai,,(t) ean be illustrated in the
following way: for i # k Ai,,(t)dt is the probability of passing from the state Zi
to the state z" during the time from t to t + dt, whereas
Au(t) = - LA"j(t).
j#
It ean also be shown that if we have any eontinuous functions Ai" (t) satisfying
the eonditions (54) and (55), then the solutions Pi"( s, t) of the differential
equations (52) under the initial eonditions (47) are non-negative and satisfy the
eonditions (40) and (41); in other words, they determine a stoehastie proeess.
Indeed, by (52) and (55) we have
:t ~Pi"(S,t) = L[~Aj,,(t)]Pij(S,t) = 0, (59)
and, by (47),
Thus (59) implies (41).

For tl < t2 we now assume that
PIk (tl ,t) = Fh(tl, t), if tl ~ t ~ t2, (60)
PIk(tl,t) = LPii(tl,t2)Pjk(t2,t), ift2 < t. (61)

i
The functions PIk (t l, t) are continuous and satisfy the differential equations
(52); consequently, (60) holds for any t and not merely for t ~ t 2; but then
(61) with t = t3 coincides with (40),
It remains to show that the solutions Pik (tl ,t) are non-negative. For this
we assume that for fixed s,
.,p(t) = min PikeS, t).

Choosing appropriate i and k we clearly have
D+.I,(t)
0/
= 8Pik8t(S, t), P.ik ( S, t ) = ol,()
0/ t ,
and if .,p(t) ~ 0, then by (54),
~ LAjk(t).,p(t) = R(t).,p(t).
j#
Since .,p(s) = 0, .,p(t) is clearly greater than any negative solution of the
equation
dy/dt = R(t)y,
and therefore it cannot be negative itself.
§7. Examples
In scheInes homogeneous in time the coefficients Aik(t) appear to be indepen-
dent of the time t; in this case the process is completely determined by the n 2
constants Aik. Equations (52) now take the form
dPäk(t)
dt
= "L...J.k
A- P... (t).
'3 , (62)
j
and solving these equations is not difficult. If all the A ik are non-zero, then
the conditions ofTheorem 4 (§4) hold and consequently, Pik(t) tends to a limit
Qk as t -+ 00. The quantities QlI: satisfy the equations
L: QlI: = 1, L: Ajll:Qj = 0 (k = 1, ... , n).

lI: j
For example, let
n = 2, A 12 = Au = A, Au = A22 = -A,
that is, the probabilities of transition from the state Xl to the state X2 and the
reverse transition from X2 to Xl are the same. The differential equations (62)
in our case give
= P21 (t) = ~(1- e- 2At ),
P12 (t)
Pu(t) = P22 (t) = ~(1 + e- 2At ).
We see that Pill:(t) tends to the limit QlI: = ~ as t -+ 00.
The following example shows that approaching the limit can be accom-
panied by oscillations damping with time:
= 3, A 12 = A23 = A3l = A,
n
A21 = A32 = A 13 = 0, Au = A22 = A33 = -Aj

2 1
Pu(t) = P22 (t) = P33 (t) = 3e-3/2At cos at + 3'
Pl2 (t) = P23 (t) = P31 (t) = e- 3/ 2At ()asinat - ~cosat) +~,
P21 (t) = P32(t) = P13(t) = -e- 3/ 2At (Jasinat + ~cosat) +~,
y'3
a=T A .
Similar damping oscillations for schemes with discontinuous time were
found by Romanovskii.
Chapter IH. COUNTABLE STATE SYSTEMS

§8. Preliminary remarks. Discontinuous schemes
If 2l consists of a countable set of elements
all the notations and results of §5 of Chapter 11 remain valid. The convergence
of the series
L: Pik(tl, t2) = 1,
k
is assumed, and from this we derive convergence of the series (40), (42), (46);
by contrast, we do not require that the series
should converge.
We now make a few remarks on schemes with discontinuous time, in par-
ticular homogeneous ones. The conditions of our theorems concerning ergodic
principles for schemes with a countable set of states fail in most cases, but
nevertheless the principle itself often appears to be satisfied.
Consider, for example, agame studied recently by S.N. Bernshtein: in any
separate trial agambier wins only one rouble with probability A and loses it
with probability B (B > A, A + B :::; 1), the latter, however, provided only
that his cash is non-zero; otherwise he does not lose anything.
If we denote by Xn the state in wh ich the cash of our gambIer is n - 1
roubles, then the conditions of the game can be written as follows:
Pn,n+l = A, Pn+1,n = B (n = 1,2,3, ... ),

P11 =l-A, Pnn =l-A-B (n=2,3,4, ... ),
Pij = 0 otherwise.
It can easily be proved that
which implies the ergodic principle in this case.

Note that the fact that the limits
lim p? = Aj
p_oo ')
exist, implies the ergodic principle only if

It can be shown that always A ~ 1 and that for A < 1 the ergodic principle
fails.
If all the Aj exist and are zero, then there arises the question of the asymp-
totic expression for PZ as p -+ 00. If such an expression exists independently
of i:
PZ = A} + O(A}),
then we say that the loeal ergodie prineiple holds. This principle seems to be
of great significance in the case of a countable set of possible states.
Now let all possible states x be enumerated by the integers (-00 <n<
+00). All the notation and formulas of §5 are then true, but now the sums run
over all the integers. We consider the case
P.'.J. = pT?J-'.
in more detail. Clearly in this case we have
PZ = PI-i' pr H = LPf pr-i, Pkm+n -_ '"' n

pmpk-;·
L..;
i i
If the series
a = LkPk , b2 = Lk2Pk
k k
are absolutely convergent, then there arises the quest ion on the conditions of
applicability of the generalized Laplace formula
p _
Pk -
1 [(k - pa)2]
bJ27rp exp - 2pb2
(1)
+ 0 JP . (63)
All we know is that it holds in the Bernoulli case, when
Po = 1 - A, P1 = A, (64)
and the other Pk vanish. Lyapunov's theorem is of no help for our problem, as
is clear from the following example:
P+1 -p
- -1
_1
- 2' Pk = 0, k f ±1,
where (63) is inapplicable. In order for (63) to hold, it is necessary 9 that for
any integer m there exists k such that
k"l- 0 (mod m), Pk f O.

9 More details on this question can be found in R. von Mises Wahrscheinlichkeits-
rechnung, Berlin, 1931, especially the chapter on "local" limit theorems. (Re-
mark by Russian editor.)
Note also that only for a = 0 does formula (63) actually give an asymptotie
expression for pe for a given k. In this ease it follows from (63) that for a given
(65)
and given i and j,

p_ 1
Pij - by'21rp
(1)
+ 0 ..;p . (66)
By (66) we obtain the ergodie principle in the ease eonsidered.

We obtain special approximation formulas for pr;
when the probabilities
Pu, that is, the probabilities of the facts that the state of the system does not
vary at any partieular moment, are very close to one. For example, in the
Bernoulli ease, for small A the approximate Poisson formula ean be used:
pP '" Akpk e-Ap (67)

k k!
A general method for deriving such formulas ean be obtained by using
differential equations for processes eontinuous in time, as is shown in §10 for
formula (67).
§9. Differential equations of a process continuous in time
As in §6, we assurne that the funetions Pij(S, t) are eontinuous and have deriva-
tives with respect to t and S for t f; s. In the ease of a eountable set ofpossible
states formulas (48) and (56) still remain valid; but to prove the possibility of
changing the order of the sum and the limit in these formulas and thus arrive at
the differential equations (52) and (57), we have to introduee new restrictions,
namely:
A) the existenee of limit values in (50);
B) uniform eonvergenee in (50b) with respeet to j for a given k;
C) uniform eonvergenee of the series
" " Pjk(t, t + ~) _ 1 - Pjj (t, t + ~) (68)

~ ~ - ~
with respect to ~ (the fact that this series eonverges follows immediately from
(41».
In §6, for a finite number of states we dedueed eondition A) from the
differentiability of Pij(S, t) for t f; S; by eontrast, in the ease of a eountable set
of states this condition does not seem to follow from this property of Pij . With
regard to condition B), note that uniform convergence in (50b) with respect to
k for a given j follows from the obvious inequality
Note, further , that we do not require uniform convergence in (50b) for any j
and k, nor do we require uniform convergence in (50a) with respect to k; these
requirements would have been inconvenient for applications.
Since the factors Pij(s, t) in (48) form an absolutely convergent series, we
can, in view of conditions A) and B), change the order of the signs lim and
E in this formula and obtain (52). Then the variables Ajk(t) clearly satisfy
the formulas of the last condition; moreover, since the factors Pjk(S +..6., t) are
uniformly bounded, we can change the order of the sum and limit signs in (56),
which suffices for deducing (57).
§10. Uniqueness of solutions and their calculation for

a process homogeneous in time
In the present case, (52) takes the form
(69)
with constants Ajk. We will prove that if the series
L IAjkl = Br l ),
LBY)IAjkl = B~2),
j
(70)
~ B(n)IA·
L...J j Jk
I -- B(n+l)
k ,
j
B(n)
L _k_,
n.
x n, k = 1,2, ... , lxi ~ () (> 0), (71)
n
converge and the initial conditions
(72)
hold, then the equations (69) have the unique system of solutions PiA:(t) satis-
fying the conditions of our problem.
Indeed, since always Pij(t) :5 1, (69) and (70) imply
therefore (69) can be differentiated term by term
cF Puc(t) _ ~ A- dPij(t) - !!P.. () A-

dt2 - L..J JA: dt - dt .A: t * .A:.
In a similar way the general relations are obtained:
(73)
(74)
From (73) and the assumption of convergence of the series (71) it follows
that the functions PiA: are analytic. Further, by (69) and (74) we find that
(75)
in particular, for t = 0 we have, by (72),
(76)
which implies that the analytic functions ~A:(t) are uniquely determined by the
constants AiA:. Formulas (76) and (75) serve also for calculating the solutions
of the system (69) using Taylor series.
For example, if
A i ,i+l = A, A ii = -A,
Aij = 0 otherwise,
then we easily obtain
(At)n-m
Pmn(t) = (n _ m)!e- At , n ~ m,
Pmn(t) = 0, m > n,
that is, the formula of the Poisson distribution: for k =n - m, p =t the
resulting formula coincides with (67).
If the ergodie principle holds and Pik(t) -+ Qk as t -+ 00, then obviously

the constants Qk satisfy the equations
LQk = 1, LAikQi = 0 (k = 1,2, ... ). (77)

k
If, for example
Ai,i+l = A, Ai+l,i = B, B > A,

Au = -A, Au = -(A + B), i> 1,
Aij = 0 otherwise,
then we easily obtain from (77)
Qn = (1 - AI B)(AI Bt- 1.
As a second example, we set
Ai,i+l= A, Ai+l,i = iB,

Au = -A - (i - l)B,
A ij = 0 otherwise,
so that from (77) we have
1
Qn+l = n!
(A)n
Be,
-AlB
which again is Poisson's formula.
CHAPTER IV. CONTINUOUS STATE SYSTEMS,

THE CASE OF ONE PARAMETER
§11. Preliminary remarks
Suppose now that the state of the system considered is determined by the
values of a certain real parameter x; in this case we denote by x both the state
of the system and the value of the parameter corresponding to this state. If cE y
is the set of all states x for which x ::; y, then we set
As a function of y, F(t1' x, t 2, y) is monotone and right continuous and satisfies

the boundary conditions
(78)
For the function F(t1, x, t2, y) the fundamental equation (3) transforms into:
(79)
Thus we have to use integral distribution functions of random variables and

ordinary Stieltjes integrals.
Integral (79) exists according to Lebesgue 10 if F(t2, y, t3, z) is Borel-meas-
urable with respect to y. In what follows we assurne that the system J (see §1)
coincides with the system of all Borel sets, which implies Borel-measurability
of F(t1, x, t2, y) as a function of x. In this case, as is known, the additive
set function P(t1, x, t2, ~), for all Borel sets ~, is uniquely determined by the
corresponding function F(t1' x, t2, y).
A function F(y), monotone and right continuous, such that
F( -00) = 0, F( +00) =1
is called a normal distribution function. If F 1 (x, y) and F2 (x, y), as functions
of x, are Borel measurable and, as functions of y, are normal distribution
functions, then the same is true for the function
This operator EB, like *, obeys the associative law; using this law, the funda-
mental equation (79) can be expressed as
(81)
with F 1 (x, y) = V1 (y-x), F 2 (x, y) = V2 (y-x). Then, as can easily be shown,
10 H. Lebesgue, Ler;ons sur I'integration et la recherche des fonctions primitives,

Gauthier-Villars, Paris, 1928.
where
(83)
The associative law also holds for the operator 0 while for normal distribution
functions the commutative law holds as weIlj if Vl(x) and V2(x) are considered
as distribution functions of two independent random variables Xl and X 2 ,
then Vl(X) 0 V2(X), as is known, is the distribution function of the sum 11
X = Xl +X2 •
If F(tl, X, t2, y) is absolutely continuous as a function of y, then we have
(84)
In this case the non-negative function f(tl,x,t2,y) is Borel measurable with
i:
respect to x and y and satisfies
i:
f(tl,x,t2,y) = 1, (85)
f(tl,X,t3,Z) = f(tl,x,t2,y)f(t2,y,t3,Z)dy. (86)
Conversely, if (85), (86) hold for f(tl, x, t2, y), then the function F(tl, x, t2, y)
defined by (84) satisfies (78) and (79): hence, such a function determines the
scheme of a stochastic process. This function f(tl,x,t2,y) will be called the
differential distribution function for the random variable y.
i:
Note also that the following mixed formulas hold:
i:
F(tl,x,t3,Z) = F(t2,y,t3,Z)f(tl,x,t2,y)dy, (87)
f(tl.X,t3,Z) = f(t2,y,t3,Z)dF(tl,X,t2,Y). (88)
When the scheme is discontinuous in time, the functions
are consideredj they satisfy the equations
Fm.n+1(x,y) = Fmn(x, y) EI:) Fn+l(x,y), (89)

Fkn(x, y) = Fkm(x, y) EI:) Fmn(x, y) (k< m < n). (90)
11 See P. Levy, Calcul des probabilites, Paris, 1927, p.187.
If
i:
then, in addition, we have
i:
fm,n+1(X,Z) = fmn(x,y)fn+1(y,z)dy, (91)
fkn(X,Z) = fkm(X, Y)fmn(Y, z)dy (k< m< n). (92)
§12. Lindeberg's method.

Passage from discontinuous to continuous schemes
As we nüted in §3, probability theory usually deals only with schemes that are
discontinuous in time. Für these schemes, the main problem is to find approx-
imate expressions for the distributions F mn (x, y) für large n - m, or what is
essentially the same, to construct asymptotic formulas for Fmn(x, y) as n -+ 00.
The Laplace-Lyapunov theorem is the most important result achieved in this

direction. Now we will consider in more detail the proof of this theorem given
by Lindeberg 12 with the purpose üf outlining his main idea in as general a form
as püssible and thus obtaining a general method for constructing asymptotic
expressions for F mn (x, y).
Let
i: i:
Fn(x,y) = Vn(y - x),
an(x) = (y - x)dFn(x, y) = ydVn(y) = 0,
b~(x) = i: (y - x)2dFn(x,y) = i: y 2dVn(y) = b~,

2 -- bm+1
B mn 2 +
2
bm+2+ ... +
2
bn·
The Laplace-Lyapunüv theorem states that under certain additional assump-

tions, for constant m and as n -+ 00 we have
y - X)
Fmn(x,y) = ( B mn + 0(1)
12 Math. Z. 15 (1922), 211.
uniformly with respect to x and y, where
1 jZ
~(Z) = -- e- Z 2 /2dz.
V2i -00
Along with the stochastic process with discontinuous time determined by

the functions Fn(x, y), we will consider another one, with continuous time; we
suppose that it is characterized by the function
-(' ") =~ (Y-X)

Ft,x,t,y .~ .
v t" - t'
Further , let
to = 0, tn = Bgn ,
Fmn(x,y) = F(tm,x,tn,y), Fn(x,y) = Fn-1,n(x,y).
Clearly we have
-Fn(x,y) = ~ (y-
b;:-X) ,
i:
i:
än(x) = (y - x)dFn(x, y) = 0,
b~(x) = (y - x?dF n(x, y) = b~.
The first and second moments än(x) and b~(x) of the distribution F n(x, y)
coincide with the corresponding moments an(x) and b~(x) of the distribution
Fn(x, y). From this Lindeberg deduced that
Fmn(x, y) - F mn(x, y) -+ 0 as n -+ 00,
after which the Laplace-Lyapunov theorem follows directly from the obvious
identity
-Fmn(x,y) = ~ (y-
-B X) .
mn
In the general case of arbitrary functions Fn(x, y) we can only apply Linde-
berg's method ifwe know a function F(t',x,t",y) characterizing a continuous
stochastic process, and which, for a certain sequence of instants of time
to < tl < t2 < ... < tn < ...

gives the moments än(x), b~(x) which coincide with an(x), b~(x), or are dose
to them. A general method for constructing such functions F is obtained by
using differential equations of continuous processes, considered in the sections
below. To pass from F to F we can use the following:
Transition Theorem. Let thefunctions Fn(x,y) and Fn(x,y) determine two
i: i:
stochastic processes with discontinuous time. If
= an(x) = än(x), (93)
i: i:
(y - x)dFn(x,y) (y - x)dFn(x,y)
(y - x)2dFn(x, y) = b~(x), (y - x)2dFn(x, y) = b~(x),
i: i:
(94)
Iy - xI 3dFn(x,y) = cn(x), Iy- xI 3dFn(x,y) = cn(x), (95)
lan(x)-än(x)1 ~ Pn, Ibn(x)-b~(x)1 ~ qn, cn(x) ~ rn, cn(x) ~ f n , (96)

and ifthere exists a function R(x) such that
R(x)=O, forx~O,
o ~ R( x) ~ 1, for 0 < x < I, (97)
R(x) = 1, for I ~ x,
and for
i
(98) "
the inequalities
(99)
I()x3Ukn x,z) I~ K n
{)3 ( (3)
, (k=O,1, ... ,n),
hold, then the relation
holds, where
In applying this theorem to the case when the moments a(x), b(x), c(x) are
unbounded as x increases, it is often possible to eliminate this unboundedness
by introducing a new properly chosen variable x' = <jJ(x).
Proo/ 0/ the Transition Theorem. By (98) we have
Uk-l,n(X, y) = Fk-l,n(X, y) EB R(y - x) =

=Fk(X, y) EB Fk+l(X, y) EB ••• EB F n(x, y) EB R(y - x) =
= Fk(x,y) EB Ukn(X,y) (101)
and by (93)-(95), (99),
= 1: [Ukn(X,y) + :xUkn(X,y)z~x+
ö2 (z-x)2 ö3 (z-x)3]_
+ öx2Ukn(x,y) 2 + öx3Ukn (e,y) 6 dFk(X,Z) =
Ö _
= Ukn(X, y) + öx Ukn(X, y)ak(x)+
2 -2 -
~U ( )bk(x) ÖK(3)Ck(X)
+ öx2 kn X, Y 2 + n 6' (102)
Setting
(103)
we obtain a formula similar to (102),
Ö
Vk-l,n(X, y) = Ukn(X, y) + öx Ukn(X, y)ak(x)+
~U ( )b~(x) OJ{(3)Ck(X) 10 I ::; 1.

+ öx2 kn x, Y 2 + n 6' (104)
From (102) and (104) and using (96) and (99) it follows that
Now let
Wkn(X, y) = FOk(X, y) EB Ukn(X, y) =

=F1(x, y) EB Fz(x, y) EB ••• EB Fk(X, y) EB Ukn(X, y) =
= FO,k-l(X, y) EB Vk-l,n(X, y). (106)
Then by (105) we have
= IFo,k-l(X,y) EB Vk-l,n(X, y) - FO,k-l(X, y) EB Uk-l,n(X,y)1 ~
~ [ : IVk-l,n(Z,y) - Uk_l,n(Z,y)ldFo,k_l(X,Z) ~
~ sup IVk-l,n(Z,y)-Uk-l,n(Z,y)1 ~ K~1)Pk+~K~Z)qk+~K~3)(rk+fk)' (107)
But
Wnn(x, y) = Fon(x, y) EB R(y - x) = [: R(y - z)dFon(x, z)
and
WOn(X, y) = Fon(x, y) EB R(y - x) =[: R(y - z)dFon(x, z).
Taking into account (97) we obtain
Wnn(X, y) ~ [Yoo dFon(x, z) = Fon (x, y),
Wnn(x, y + 1) ~ [Yoo dFon(x, z) = Fon(x, y),

(108)
Won(X, y) ~ [Y~' dFon(x, z) = Fon(x, y -1),
WOn(x, y + 1) ~ j-00
Y+l
dFon(x, z) = Fon(x, y + 1),
Formula (100) now follows immediately from (107) and (108). The details of
the proof can be found in Lindeberg's paper referred to above.
§13. The first differential equation for processes

continuous in time
If the state of our system 6 can be changed at any moment t, then it is

natural to assume that significant changes of the parameter x during small
time intervals will occur very seldom, or, more exactly, that for any positive f,
P(t,x,t+ß, Iy-xl>f)-O asß-O. (109)
In most cases we may assume that the stricter condition
holds, at least for the first three moments m(l), m(2) and A general study
m(3).
of the possibilities that arise under these assumptions is of great interest; some
remarks to this end will be given below in §19.
In the following sections we also assume that the following important con-
dition holds:
(111)
This condition will certainly hold if in the definition of m(3)(t, x, ß) via (110)
only infinitesimally small differences y - x playa significant role for infinitesi-
maHy small ß or, more precisely, if
(112)
Strictly speaking, only in this case is our process continuous in time. For-
mula (111) also implies that
m(2)(t x ß)
--;~.....:'---,-'-,-7- - 0 as ß - O.
m(1)(t,x,ß)
Finally, we will also assume that for s i= t all the partial derivatives of the
function F( s, x, t, y) up to the fourth order exist, and that these derivatives for
constant t, y are uniformly bounded with respect to sand x for t - s > k > O.
From (78) and (110) we conclude that for s = t the function F(s,x,t,y) is, by
contrast, discontinuous. The function
()
f(s,x,t,y) = ()yF(s,x,t,y), (113)
clearly satisfies (84)-(86) and, at given t, y, has for t - s > k > 0 deriva-
tives up to the third order that are uniformly bounded with respect to s and
t. All further calculations are made for this differential distribution function
f(s, z, t, y).
I:
Set
I:
a(t, z, d) = (y - z)f(t, z, t + d, y)dy, (114)
= = m(2)(t, z, d),
I:
b2(t, z, d) (y - z)2 f(t, Z, t + d, y)dy (115)
c(t, z, d) = Iy - zl3 f(t, Z, t + d, y)dy = m(3)(t, z, d). (116)
I:
By (85) and (86) we have
f(s, z, t, y) = f(s, z, s + d, z)f(s, d, z, t, y)dz =
= I: f(S,Z,S+d,Z)[f(s+d,z,t,y)+
8
+ 8z f (S + d,z, t,y)(z - z)+
82 (z - z)2
+8z2f(s+d,z,t,y) 2 +
83 (z - z)3]
+ 8e3f(s + d,e, t, y) 6 dz =
8
= f(s+d,z,t,y)+ 8z f (s+d,z,t,y)a(s,z,d)+
82 f( d )b 2(s, Z, d) 8c(s, z, d)
+ 8z 2 S + ,z, t, y 2 + 6 ' 181< c, (117)
where for s + d < T < t, C can be chosen independently of d. From (117) we

immediately obtain
f(s+d,z,t,y)-f(s,z,t,y) _
d
- -!...f( d t )a(s,z,d)
- 8z s + ,z" y d
8 2 f( A ) b2 (s, Z, d) lI c(s, Z, d)
- 8z 2 S + u, z, t, y 2d - 17 6d . (118)
First we prove that if for given x and s the determinant
::c /(s, x, t', y') {):c "X

~/(s t" , y")
D(s , x , t' , y' " t" y") = IlJ (119)
{ii'J" /(s, x, t', y') ()~ /(s " X t" , y")
(ii'J"
does not vanish identicaHy for any t', y' , t" , y", then the ratios
a(s,x,t::..)jt::.. and b2 (s,x,t::..)j2t::..
tend to weH defined limits A(s,x) and B 2 (s,x) as t::.. -+ O.

Thus, let t', y', t", y" be chosen so that (119) is non-zero; in this case, for
any sufficiently smaH t::.. we also have
D(s + t::.., x, t', y', t", y") i= 0,

so that the equations
>.(t::..) :x /(s + t::.., x, t', y') + JJ(t::..) :x /(s + t::.., x, t", y") = 0,
(120)
>.(t::..) ::2/(S + t::.., x, t', y') + JJ(t::..) ::2/(S + t::.., x, t", y") =1
have a unique solution. In this case >'(t::..) and JJ(t::..) tend to >'(0) and JJ(O) as
t::.. -+ O. Further, by (118) we obtain
>.(t::../(s + t::.., x, t', yl- /(s, x, t', y') +
+JJ(t::../(s + t::.., x, t", y") - /(s, x, t", y") =

t::..
b2 (s,x,t::..) _ (0' Ol/)C(s,x, t::..) (121)
2t::.. + 6t::..·
The left-hand side of formula (121) tends to
n= >'(0) :s/(s, x, t', y') + JJ(O) :s /(s, x, t", y")

as t::.. -+ 0, whereas on the right-hand side the second term is infinitesimally
smaH as compared with the first one, by (111); therefore this term tends to the
limit
2
B 2( S,X ) -_ 1·1m b (s, x, t::..) -__ u.
t"\
(122)
"'-_0 2t::..
ON ANALYTICAL METHOnS IN PROBABILITY THEORY 97
It follows immediately from (122) and (111) that
c(s, x, t:..)j t:.. -+ 0 as t:.. -+ O. (123)
By (122) and (123), formula (118) for t:.. = 0 becomes:
1· [8
A1.T f( )a(s,x,t:..)] 8 f( s,x,t,y-
= - 8s )
o 8x s,x,t,y t:..
82
- 8x2f(s,x,t,y)B 2(s,x).
Since 8f(s, x, t, y)j8x does not vanish identically for any t and y, the following
limit also exists:
A(s, x) = lim a(s,x,t:..) =

A_O t:..
_ -8f(s, x, t, y)j8s - B 2(s, x)8 2f(s, x, t, y)j8x 2
(124)
8f(s, x, t, y)j8x
Passing to the limit in (118), (122), (123) and (124) we obtain the first
fundamental differential equation
8 8 82
8s f (s, x, t, y) = -A(s, x) 8x f(s, x, t, y) - B 2(s, x) 8x 2 f(s, x, t, y). (125)
When the determinant D(s , x , t' , y' , t" , y") vanishes for any t' , y' ,t" , y" ,
then the limits A( s, x) and B 2 ( s, x) do not in general exist, as is dear from the
following example:
f( s x t y) = 3y 2 e-(y3-11: 3)2/4 (t-.).

(126)
, , , 2 ../rr(t - s)
Here, for x = 0 we have
It can be shown, however, that these singular points (s, x) form a nowhere
dense set in the (s, x)-plane.
The practical significance of these very important quantities A(s, x) and
B(s, x) is as folIows: A(s, x) is the mean rate of variation of the parameter x
over an infinitesimally small time interval and B(s, x) is the differential variance
of the process. The variance of the difference y ~ x for the time interval t:.. is
b(s,x,t:..) = B(s,x)~ + o(VX) = O(VX); (127)

and the expectation of this difference is
a(s,x,~) = A(s,x)~ + o(~) = O(~). (128)
It is worth mentioning that the expectation m(1)(t,x,~) of Iy - xl, like the

variance b( s, x, ~), is a quantity of order -...IX.
As will be shown in the following sections, the functions A(s, x) and B(s, x)
in some cases uniquely determine our stochastic scheme.
§14. The second differential equation
In this section we retain all the requirements imposed on f(s,x,t,y) in §13

and, in addition, assurne that f(s, x, t, y) has continuous derivatives up to the
fourth order. Then, from (120) it clearly follows that if the determinant (119)
is non-zero, then A(O) and 1'(0) have continuous derivatives with respect to s
and x up to the second order; by (122) and (124) the same is also true for
B 2(s, x) and A(s, x).
Now, assurne that for a certain t we are given an interval a ~ y ~ b such
that at each point of this interval the determinant D(t, y, u', z', u", Zll) does
not vanish identically for any u', z' , u", Zll. Next, let R(y) be a function that
is non-zero only on the interval a < y < b, is non-negative and has bounded
derivatives up to the third order. In this case we have
l a
b
()
{)/(s,x,t,y)R(y)dy= ()
()t l a
b
f(s,x,t,y)R(y)dy=
lim 1 JOO [f(s,x,t+~,y)-f(s,x,t,y)]R(y)dy=

= .1-0.u. A
-00
lim A{
= .1_0 1 JOO R(y) JOO f(s,x,t,z)f(t,z,t+~,y)dzdy-
.u. -00 -00
JOO 1
i:
- f(s, x, t, Y)R(Y)dY} = lim A X
-00 .1-0.u.
X {I: f(s, x, t, z) f(t, z, t +~, y) [R(z) + R'(z)(y - z)+
+R"(z) (y ~ z)2 + R11I (e) (y ~ Z)3] dydz-

OO 1jOO
- J
-00
f(s,x,t,z)R(z)dz} = .1-0.u.
lim A
-00
f(s"x,t,z)x
) ( A) nI,( z )b 2(t,z,.6.)
[R'( zat,z,.u+.n + BC(t,z,.6.)]d_
i:
X 2 6 z-
= f(8, x, t, z)[R'(z)A(t, z) + R"(z)B 2(t, z)]dz =
= l b
f(8, X, t, y)[R'(y)A(t, y) + R"(y)B 2(t, y)]dy, (129)
IBI ~ sup IRIII (e) I·

The passage to the limit with respect to .6. in deriving these formulas is justified
by the fact that a(t,z,.6.)/.6., b2(t,z,.6.)/2.6. and c(t,z,.6.)/.6. tend uniformly
to A(t,z), B 2(t,z) and 0 respectively, and the integral ofthe factor f(8,X,t,Z)
with respect to z is finite.
Integrating by parts we obtain
l a
b
f(8, X, t, y)R'(y)A(t, y)dy = - lb ()
a
8[f(8, X, t, y)A(t, y)]R(y)dy.
Y
(130)
In exactly the same way, integrating by parts twice, we obtain
I a
b
f(8, X, t, y)R"(y)B 2(t, y)dy = I a
b
:22 [/(8, x, t, y)B 2(t, y)]R(y)dy, (131)
Y
since R(a) = R(b) = R'(a) = R'(b) = O. Formulas (129)-(131) immediately

imply that
l b
;/(8,x,t,y)R(y)dY= l b
{-:y[A(t,y)f(8,x,t,y)]+
{)2 2 }
+ {)y2 [B (t,y)f(8,X,t, y)] R(y)dy. (132)
However, since the function R(y) can be chosen arbitrarily only if the above
conditions are fulfilled, we easily see that for points (t, y) at which the deter-
minant D(t, y, u', z', u", Zll) does not vanish identically the 8econd fundamental
differential equation also holds:
() () {)2 2
()tf(8,x,t,y)=-{)y[A(t,y)f(8,X,t,y)]+ {)y2[B (t,y)f(8,X,t,y)]. (133)
This second equation could also have been obtained without using the
first one, using the methods described in §13 directly; then, however, new and
more stringent restrietions (that we omit here) would have to be imposed on

f(s,:c, t, y). In that case we start from the formula, similar to (118),
1
Ll [f(s,:c, t, y) - f(s,:c, t - Ll, y)] =
= f(s,:c,t - Ll,y) Ll
1[ 1 00
-00 f(t - Ll,z,t,y)dz -1]+
o
+-;;-f(s,:c,t
vy
- Ll,y) Ll 11 00
-00
f(t - Ll,z,t,y)(z - y)dz+
2
o0 2f (s,:c,t-Ll,y)2Ll
y
1 1 00
-00
f(t-Ll,z,t,y)(z-y) 2 dz+
+6Ll
00
1
1 _oof(t-Ll,z,t,y)lz-YI 3 dz. (134)
Then we prove that
lim
.:l.-Ou.
11
A
00
-00
f(t - Ll,z,t,y)lz - yl3dz = 0
1
and that the limits
. ~
1 yl 2 dz = -B2 (t,y),
00
hm f(t - Ll,z,t,y)lz - (135)
.:l.-o2u. -00
lim 11
A
.:l.-ou.
00
-00
f(t - Ll, z, t, y)(z - y)dz = -A(t, y), (136)
li~oLl1 ( 1 00
-00 f(t - Ll,z,t,y)dz -1) = N(t,y),
- (137)
exist. Thus we would have obtained our second equation in the followingform
o -
o/(s,:c,t,y) = N(t,y)f(s,:c,t,y)+
- y) Oyf(s,:c,
+A(t, 0 0 2 f(s,:c, t, y).
t, y) + -B2 (t, y) oy2 (
138)
To show the identity of this equation with the one derived before, we would
have to prove that
1J2(t, y) = B 2 (t, y), (139)

- 0 2
A(t, y) = -A(t, y) + oy B (t, y), (140)
- 0 0 2
N(t, y) = oy 2
A(t, y) + oy2 B (t, y). (141)
§15. Statement of the uniqueness and existence problem of

solutions of the second differential equation
In order to define the function f(8, z, t, y) uniquely by the differential equations

(125) and (133) we have to set up some initial conditions. For the second
equation (133) the following approach can be adopted: according to (85), the
function f(8, z, t, y) satisfies the condition
1: f(8,z,t,y)dy = 1 (142)
for every t > 8 and, in view of (110), we also have
1: (y - z?f(8,z,t,y)dy -+ 0, as t -+ 8. (143)
The main question regarding the uniqueness of solutions is as follows: under

what conditions can we assert that for given 8 and z there exists a unique
non-negative function f(8, z, t, y) ofthe variables t, y defined for all y and t >8
and satisfying (133) and the conditions (142), (143) ? In certain important
particular cases such conditions may be described: these are, for example, all
the cases considered in the following two sections.
Now given the functions A(t, y) and B 2 (t, y), the question is whether there
exists a non-negative function f(8, z, t, y) such that, on the one hand, it sat-
isfies (85) and (86) (as was indicated in §11, these requirements are needed
for f(8, z, t, y) to determine a stochastic system), and on the other hand, after
passing to the limit via formulas (122) and (124), it gives these functions A(t, y)
and B 2 (t, y).
To solve this problem, we can, for example, first determine some non-
negative solution of our second differential equation (133) satisfying the condi-
tions (142), (143) and then check if it is indeed a solution to our problem. In
doing this, the following two general questions arise:
1) Under what conditions does there exist such a solution of equation
(133)?
2) Under what conditions does this solution really satisfy (85) and (86)?
There are good grounds for assuming that these conditions are of a sufli-
ciently general character.
§16. Bachelier's case
We now assume that I(s, x, t, y) is a function ofthe difference y-x, depending

arbitrarilyon s and t, that is, that our process is homogeneous with respect to
the parameter
I(s,x,t,y) = v(s,t,y - x). (144)
In this case, clearly A(s, t) and B(s, t) depend only on s, so that the differential
equations (125) and (133) are now expressed as
{}I {}I 2 {}2 I
{}s = -A(s) {}x - B (s) {}x 2 ' (145)
{}I = -A(t) {}I + B 2 (t) {}2 I. (146)

ät {}y {}y2
For the function v(s, t, z), we obtain from (145) and (146):
{}v _ A( ) {}v _ B 2 ( ) {}2 v

{}s - s {}z S {}z2' (147)
{}v {}v 2 {}2 V

{}t = -A(t) {}z +B (t) {}z2' (148)
Equation (148) was found by Bachelier, 13 but strictly speaking, was not
proved.
If we have A(t) = 0 and B(t) = 1 identicaHy, then (133) (respectively
(146» turns into the heat equation
(149)
for which the only non-negative solution satisfying (142), (143) is given, as is
weH known, by Laplace's formula
(150)
In general we assume that
x' =x -1 6
A(u)du, y' =y -l t
A(u)du,
s' = 1 6
B 2 (u)du, t' = l' B 2 (u)du.
13 See papers Nos. 1 and 3 in footnote 2.

Then (146) turns into
and the conditions (142), (143) retain the same form in the new variables
s', x',t', y' as in the variables s,x,t,y. Bence in the general case, the function
(151)
is the only solution of equation (146) satisfying our conditions.
§17. A method of transforming distribution functions
Let
s' = ljJ(s), t' = ljJ(t), x' = 1jJ(s,x), y' = 1jJ(t,y)
I(s, x, t, y) = (I}1/J(t, y)joy)!'(s', x', t', y'), (152)
and assurne that ljJ(t) is a continuous, nowhere decreasing function, whereas

1jJ(t, y) is arbitrary with respect to t and has a continuous positive derivative
with respect to y. If I(s,x,t,y) satisfies (85) and (86), then the same is also
true, as can easily be demonstrated, for the function f' with respect to the new
variables s', x', t', y'; in other words, our transformation gives a new function
f'(s',x',t',y') which, like I(s,x,t,y), determines a stochastic scheme.
If ljJ(t) and 1jJ(t, y) have appropriate derivatives, then under transition to
the neW variables (125) and (133) turn into
of' A'(' ,)of' '2(' ,)02f' (153)

os' = - s ,x ox' - B s, x ox'2'
of'
ot'
= -~[A'I']
oy'
+ ~[B'2/']
oy'2 '
(154)
where we have set

A'(' ') _ (021jJ(t, y)joy2)B 2(t, y) + (o1jJ(t, y)joy)A(t, y) + o1jJ(t, y)jot
t ,y - oljJ(t)jot '
B,2(t' ') = (o1jJ(t,y)joy)2B 2(t,y) (155)

,y oljJ(t)jot'
With the help of the above transformation the solutions of (133) can be
obtained for many new types of coefficients A(t, y) and B 2(t, y). For example,
let
A(t, y) = a(t)y + b(t), B 2(t, y) = c(t); (156)
we set
</J(t) = J c(t)e- 2Ja(t) dt dt,
'IjJ(t,y) = ye- Ja(t)dt - J b(t)e- Ja(t)dt dt

(157)
and obtain in the new variables s', x', t', y', I' the simplest heat equation:
(158)
In this case the initial conditions (142) and (143) remain valid for /'( s', x', t', y')
as weIl; therefore the formula
I' = 1 e-(yl-:z;1)2/4(t ' _'/)

(159)
J1r(t' - s')
together with (157) and (152) gives the unique solution I(s, x, t, y) of (133)
with coefficients of the form (156) satisfying our conditions. It is easy to see
that in this case the function I(s, x, t, y) is of the form
_1_
.,;:;rpe -(y-a)2/ 4 ß , (160)
where a and ß depend only on s, x and t, but not on y.

It is an important problem to find all possible types of coefficients A(t, y)
and B 2 (t, y) such that for any s, t, x we always obtain a function of the form
(160), that is, the Laplace distribution function.
As a second example consider the case
A(t, y) = a(t)(y - c), B 2(t, y) = b(t)(y - c)2. (161)
This time, setting
</J(t) = J b(t)dt, 'IjJ(t) = ln(y - c) + J [b(t) - a(t)]dt (162)
we again obtain for I'(s', x',t', y') equation (158) for which the solution (159)
is already known. Note that here it suffices to consider only the values x > c,
y > c, since as x or y varies from c to +00 the variable x' (hence y') runs through
all the values from -00 to +00. Certain complications arising in connection
with this when transferring the conditions (142) and (143) to f' can easily be
eliminated.
In particular, for
A(t, y) = 0, B 2(t, y) = y2 (163)
we have the formula

1 { (In y +t-Inx-s)2}
f(s,x,t,y) = exp - 4() . (164)
YV7r (t-s) t-s
For applications the most important is the case when A(t, y) and B 2 (t, y)
depend only on y, but do not depend on the time t. The next step in this
direction would be to solve our problem for coefficients of the form
A(y) = ay + b, B 2(y) = cy2 + dy + e. (165)
§18. Stationary distribution functions

If at time t a the differential function of a probability distribution g(ta, y) is
known, then, as for the general formula (5), the distribution function g(t, y) is
I:
determined for any t > ta by the formula
g(t, y) = g(ta, x)f(ta, x, t, y)dx. (166)
Clearly, g(t, y) satisfies the equation
og 0 02 2
öt = - öy [A(t, y)g] + oy2 [B (t, y)g]. (167)
We now assume that the coefficients A(t, y) and B2(t, y) depend onlyon y
(the process is homogeneous in time) and study the functions g(t,y) which in
this case do not change with time. It is dear that for such functions we have
(168)
If we assume that 9 and g' tend to 0 so rapidly as y -. ±oo that the entire
left-hand side of (168) tends to 0, then dearly C = 0 and we have
(169)
Moreover the function g(y) must also satisfy the condition
(170)
In most cases it appears possible to prove that, if there exists a station-

ary solution g(x), then f(s,x,t,y) tends to g(y) as t --+ 00 and for arbitrary
constants sand Xj thus, g(y) appears to be not only a stationary, but also the
limiting solution.
If the coefficients A and B 2 are of the form (165) , then (169) turns into
the Pearson equation
g' y- p
(171)
g - qo + qlY + q2y2 '
with
d-b e d e
p=---, qo=--, ql = ---, q2 = - - - . (172)
a- 2c a - 2c a - 2c a - 2c
Hence we can construct stochastic schemes for which any of the functions of
the Pearson distribution is a stationary solution.
§19. Other possibilities
The theory presented in §§13-18 is essentiaHy determined by the assumption

(111). If we get rid of this assumption, then even when the condition (110)
is retained, a number of new possibilities appear. For example, consider the
scheme determined by the distribution function
F(s,x,t,y) = e-a(t-·)u(y - x) + (1- e-a(t-.» iYoo u(z)dz, (173)
where u(z) = 0 for z < 0 and u(z) = 1 for z ~ 0, and u(z) is a continuous
i:
non-negative function for which
u(z)dz =1
and the moments
i: u(z)lzlidz (i = 1,2,3)
are finite. It can easily be shown that the function F(s,x,t,y) satisfies (78)
and (79), as weH as (110).
This scheme can be interpreted as follows: du ring an infinitely small time
inter val (t, t + dt) the parameter y either remains constant with prob ability
1- adt, or takes a value y', z < y' < z + dz with prob ability au(z)dt dz. Thus
a jump is possible in any time interval, and the distribution nmction of the
values of the parameter after the jump does not depend on the values of this
parameter prior to the jump.
This scheme could also be generalized in the following way: imagine that,
during an infinitely small time interval (t, t + dt) the parameter y retains its
former value with probability 1- a(t,y)dt and turns into y', z < y' < z + dz
I:
with probabiIity u(t, y, z)dt dz. Clearly we assume that
u(t, y, z)dz = a(t, y). (174)
In this case for g(t, y) the integro-differential equation
8 g(t, y) = -a(t, y)g(t, y) +

-8
t
1 00
-00
g(t, z)u(t, z, y)dz (175)
should hold.
If we wish to consider not only jumps but also continuous changes in y,
then it is natural to expect that g(t, y) satisfies
8 g(t, y) = -a(t, y)g(t, y) +

-8
t
1 00
-00
g(t, z)u(t, z, y)dz-
8 82
- 8y [A(t, y)g(t, y)] + 8 y 2 [B 2 (t, y)g(t, y)], (176)
provided (174) holds and the coefficients A(t, y) and B 2 (t, y) are as indicated
in §13.
CONCLUSION
If the state of the system under consideration is determined by n real param-

eters Z1. Z2, ... , Zn, then under certain conditions similar to those of §13 we
have the following differential equations for the differential distribution function
I(s, Z1. ... , Zn, t, Y1,··· ,Yn):
81 81 82I
aS
= - L:: Ai(S,
n
i=l
Z1. ... , Zn)a-:- -
Z,
L:: L:: Bij(S,
n n
i=l j=l
Zl, ... , zn) 8 .8 .'
Z, Z,
(177)
108 ON ANALYTICAL METHOnS IN PROBABILITY THEORY
For the case when A;(t, Yl," . ,Yn) and B;j(t, Yl,' .. ,Yn) depend only on
t, these equations were discovered and solved by Bachelier. 14 In this case the
solutions satisfying the conditions of our problem have the form
f = Pex p { - ~ ~P;j(Yi - Xi - qi)(Yj - Xj - qj)}, (179)
with P,Q,Pij and qi depending only on sand t.

It is also possible to consider mixed schemes, where the state of the system
is determined by parameters some of which are discrete and others continuous.
Moscow, 26 July 1930
14 See item 11 in footnote 2.

10. THE WAITING PROBLEM *
§1. The purpose of this paper is to show several applications of the general
equati<?ns which I studied in my memoir submitted to "Mathematische An-
nalen" 1 (see No. 9 of the present publication). For this I give a new solution
to the "Waiting problem" dealt with in an extensive memoir by Polyachek. 2
The essence of the problem is as folIows. (Naturally, the same mathe-
matical problem can arise when studying other real phenomena, therefore such
expressions as "telephone lines" or "conversation" are used here only for the
sake of illustration.)
Assume that, in a telephone station, there are n lines over which telephone
conversations can take place. At any moment there are m clients that either
have a conversation or await their turn; the latter takes place only for m > n,
and the length of the waiting line is m - n. Theoretically, m can take all
non-negative integer values:
m = 0, 1,2, ....
We denote by
the probabilities of these values at time t. Clearly,
L: Qm(t) = l.
00
(1)
m:O
If we know m at time to, then
(2)
The first problem to be solved is to calculate the probabilities Qm(t) for

t > to if we know their initial values Qm(tO), which, in particular, can take
the form (2). The second problem is to determine the expectation and the
distribution law for the waiting time for a client arriving at time t.
* Sur le probleme d'attente', Mat. Sb. 38:1/2 (1931), 101-106.

1 This article, however, does not suggest that the reader has read this memoir,
apart from the proof of several auxiliary statements.
2 Math. Z. 32 (1930), 64-100.
109
110 THE WAITING PROBLEM
§2. We proceed from the following two assumptions on the character of the
random events studied.
I. At any infinitely small time interval (t, t + dt) a new dient arrives at the
station with probability o:(t)dt. More precisely, at any interval (t, t + A) the
probability of a new dient arriving at the station is
o:(t)A + o(A),
whereas the probability for more than one dient to arrive is
o(A);
here, as usual, o(A) is an infinitesimal with respect to A; we assume that the

function o:(t) is continuous. The probabilities relating to the interval (t, t + A)
do not depend on the number of dients arriving earlier or on the times of their
arrival.
11. The prob ability of finishing a conversation maintained by a dient at a mo-
ment t during the time interval (t, t + dt) equals ßdt; it does not depend on the
length ofthe conversation prior to the moment t. This gives us the exponential
distribution for the length of a conversation: the probability that this length is
between t and t + dt is
p(t)dt = ße-fjtdt. (3)
In this case the expected length of a conversation is
D= 1 00
o
ßte-tfjdt
1
= -.
ß
(4)
Instead of distribution (3) Polyachek assumes that the length of a conver-

sation is always equal to some constant D; this assumption is as arbitrary as
ours.
§3. Denote by Qmp(t, t+A) the conditional prob ability that at time t+A there
are p clients at the station if at time t there were m dients. The probability
that within the inter val (t, t + A) more that one dient will arrive, or more than
one on-going conversation will be finished, is o(A). Therefore
Qmp(t, t + A) ~ w(A) = o(A), Ip - ml > 1, (5)

THE WAITING PROBLEM 111
where w(Ll) does not depend on m and p. For Ip - ml ~ 1 we have

Qm,m+l(t, t + Ll) = a(t)Ll + o(Ll), (6)
Qm,m-l(t, t + Ll) = mßLl + o(Ll), if m ~ n, (7)
Qm,m-l(t, t + Ll) = nßLl + o(Ll), if m ~ n, (8)
therefore
Qmm(t, t + Ll) = 1- (a(t) + mß)Ll + o(Ll), if m ~ n, (9)
Qmm (t, t + Ll) = 1 - (a(t) + nß)Ll + o(Ll), if m ~ n. (10)
By the formula for the total probability we have
L
00
Qp(t + Ll) = Qm (t)Qmp (t + Ll). (11)

m=O
Substituting the value of Qmp(t, t + Ll) from (5)-(10) into (11) we prove that
the limit
Q' (t) = lim Qp(t + Ll) - Qp(t) (12)
p .a._o Ll
always exists and the following equations hold:
Q~(t) = ßQ1(t) - a(t)Qo(t), (13)
Q~(t) = (i + I)ßQi+l(t) - (a(t) + iß)Qi(t) + a(t)Qi-l(t), 1 ~ i < n, (14)
QHt) = nßQi+l(t) - (a(t) + nß)Qi(t) + a(t)Qi_l(t), n ~ i. (15)
Thus we obtain for the functions Qi(t) a countable infinite system of dif-
ferential equations. It suffices to find a solution Q~m)(t) with initial conditions
of type (2). A general solution can be found with the help of the formulas for
the total probability,
=L
00
Qi(t) Qm(to)Q~m)(t). (16)

m=O
§4. Consider the simplest and most important case when the function a(t) is
constant:
a(t) = a. (17)
In this case the equations (13)-(15) have constant coefficients. In the paper
mentioned above I proved the existence and uniqueness of the solution far these
systems (under the assumption that the Qm(t) are non-negative and satisfy
(1), which holds in our problem). Therefore the functions Qm(t) are uniquely
determined by the equations (13)-(15) and their initial values Qm(tO). It is
not difficult to find approximate solutions for these equations; one method for
obtaining such a solution is given in my paper mentioned above. But from the
practical viewpoint it is preferable to go directly to the limit solution.
This solution only exists if
nß> 0'. (18)
If the reverse inequality

nß<O' (19)
holds, then the waiting line in the station grows indefinitely. The case
nß=O' (20)
is only of theoretical value, so we confine ourselves to the study of the case

(18).
First we find the constants Qm(t) = Qm satisfying the equations (13)-(15)
and (1). In this case
ßQl - O'Qo = 0, (21)
(i + 1)ßQi+l - (0' + ßi)Qi + O'Qi-l = 0, 1 $ i < n, (22)
nßQi+1 - (0' + nß)Qi + O'Qi-l = 0, n $ i, (23)

00
(1)
From (21)-(23) it is easy to obtain
Qi = WiQO,
Wi = ~a. (-ßO') i, 0 $ i $ n, (24)
1 (O')i
Wi = n!ni-n ß ' n $ i, (25)
THE WAITING PROBLEM 113
from which, by (1) we have
(26)
LWi,
00
ü=
i=O
(27)
Formulas (24)-(27) give the unique stationary solution of our problem.

It can be proved that for any other solution the functions Qm(t) tend to
Qm as t -+ 00, that is, the stationary solution is also the limit solution.
The following remark concerns formula (24)-(27). We say that the prob-
ability distribution {Q~)(t)} exceeds the distribution {Q~)(t)} iffor any m,
(28)
where
(29)
If the initial conditions {Q~)(to)} exceed {Q~)(to)}, this relation persists at

any moment t > t o. Let Qm(t) be the solution of our problem with initial
conditions
Qo(to) = 1, Qm(tO) = 0, m> O. (30)
These initial conditions exceed all other possible conditions. Therefore at any
moment the solution Qm(t) exceeds any other solution, and in particular, the
stationary solution.
Thus we see that if the coefficients a and ß remain constant for a suf-
ficiently long time, then the desired probabilities are given by the formulas
(24)-(27). For example, if n = 3, a/ß = 2, then we have the table:
m o 1 2 3 4 5 6
1/9 2/9 2/9 4/27 8/81 16/243 32/729
1/9 1/3 5/9 19/27 65/81 211/243 665/729
In this case we see that if on the average two out of three lines are busy,
then there is a free line with probability R 2 = 5/9, whereas the prob ability
that m > 6, that is, the probability that more than three dients are awaiting
their turn is equal to 1 - R6 = 64/729 '" 1/11.
§5. If the limit solution is taken as the exact solution, then it is simple to
calculate the expectation E of the waiting time. For this, the average length
waiting time
1 1
u= :L:(m-n)Qm=-- -
(a)n+1 (1 -1-a)-2
- (31)
m>n
On!n ß nß
should be divided by nß:
E = u/nß. (32)
Thus, in our example, for n = 3, a/ß = 2 we find
8 8 8 8
u= 9' E= 9.3ß = 27ß = 27 D .
This means that the average waiting time is 8/27 of the average length of a
conversation.
§6. To be able to calculate the distribution law for the waiting time we have
only to solve the following problem: under the assumption that at time to there
are m dients at the station and a new dient A appears, we have to calculate
the distribution law of the waiting time for this dient. We are interested only
in the case m ;::: n, since otherwise there are free lines available. Set m = n + p
and denote by Pq(t), q = 0,1, ... ,p, the probability that in the interval (to, t)
precisely q conversations finish; denote by P(t) the prob ability that this number
exceeds p. Clearly, at time to,
Po(to) = 1, Pq(to) = 0, q > 0, P(to) = O. (33)
The functions Po, PI, ... , Pp, P satisfy the equations
P~(t) = -nßPo(t),
P;(t) = nß(Pq-l{t) - Pq{t)), q = 1,2, ... ,p, (34)
P'{t) = nßPq(t).
Thus we have found a finite system of linear equations, which can be easily
solved. Hence we obtain the probability P{t) that the waiting time for dient
A is at most t - to.
Paris, 24 November 1930
11. THE METHOD OF THE MEDIAN IN THE THEORY OF ERRORS*
Under the assumption that the error distribution is a normallaw the method
of the arithmetic mean, as is weH known, is the best method for calculating
the true value of an observable. The method of the median in this case is less
effective, though not much less, as Haag has shown. However, if the hypothesis
of normal distribution does not hold, then the problem arises of finding the
best method for the given distribution law. In particular, in many cases when
it is considered necessary to rule out "abnormal observations" it would be
methodologically better to study a general distribution law and to find a more
appropriate method for calculating the true value.
In this paper I show how, knowing the law of error distribution, one can
determine the degree of accuracy of the method of the median and compare
it with the accuracy of the method of the arithmetic mean. Which of the two
methods is preferable depends on the nature of the distribution law adoptedj
however, by Theorem 2, when the distribution law is unknown and can deviate
markedl-y /rom the normallaw, it is sa/er to use the method 0/ the median.
The method of study, in its essential features, is due to Haag [1] (who
applied it, however, only to the normal distribution law).
Let the probability that the error of a single observation lies in (z, z + dz)
be /(z)dz. We shaH assume that /(z) is continuous and !(m) :f:. 0 for the
median m:
m
1 1
-00 !(z)dz = 2·
Furthermore, let
a= 1: 00
z/(z)dz
be the expectation of the error. Denote by Z1, Z2, ... ,Zn the results of n suc-
cessive observations, let an be their arithmetic mean and m n their median. We
=
restrict ourselves to the case of n 2k + 1, when m n =
Zk+1. assuming that
the Zi are numbered in increasing order. The differences (an -a) and (mn -m)
are (as we shall see) of order O(I/fo), so that it is natural to set
an = (an - a).,fii, I'n = (mn - m).,fii

and study the distribution laws for an and I'n. As is known, for an we have
a continuous distribution law: the prob ability that an lies in (a, a + da) is
* Mat. Sb. 38:3/4 (1931), 47-50.
115
116 THE METHOD OF THE MEDIAN IN THE THEORY OF ERRORS
equal to tPn(a)da, where tPn(a) can be calculated by well-known formulas. If

the variance
U2 = [:00 (x _ a)2 f(x)dx
exists, then tPn(a) converges to the normal distribution law
with variance u.
The probability that rn n lies in (t, t + dt) is
= (2k + I)! (1- 02(t))k f ( )d

U n(t ) (k!)2 4 t t,
J:
where 1
O(t) = 2 f(s)ds.
The probability that J-ln lies in (J-l, J-l + dJ-l) is
J-l = (t - rn)V2k + 1, t = rn+ V2k+ l'
Theorem 1. As n increases, 'ljJn(J-l) tends to the normal distribution law
with variance
Um = ~f(rn).
Indeed,
./.'f'n (J-l ) = ..;2iC+1(2k)!
(k!)24k
{I _ 02( )}kf( )
t t .
By Stirling's formula
Further , it is clear that

f(t) ~ f(rn).
1 See the derivation of the similar formula (4) in [1].

THE METHOD OF THE MEDIAN IN THE THEORY OF ERRORS 117
It only remains to consider the factor {I - (J2(t)}". First, we have

I' I'f(m)
(J(t) = m + y'2kTI
2k + 1
= y'2kTI(1
2k + 1
+ f),
where -+ 0 as n -+ Hence
;i
f 00.
(J2(t) = 1'2 m ) (1 + f'),
{l- (J2(t»" = e- p2J2 (m)/2(1 + f'/),

where f' and f" also tend to O. Comparing the above expressions we obtain
tPn(l') -+ Jgf(m)e- p2J2 (m)/2,

which proves our theorem.
Thus we see that the variances of m n - m and an - aare infinitesimally
small as n -+ 00. Hence, m is the systematic error in the method of the
median, while a is the systematic error in the method of the arithmetic mean.
The accuracies of both methods are estimated by the relation
>. = um/u = !f(m)u.
For a normal distribution fex) the ratio>. was calculated by Haag; it is
>'g = .;:;J2 ,... 5/4.

It is easy to construct examples of distribution laws for which >. takes any
values between 0 and 00. However, we have:
Theorem 2. For distribution laws with one maximum the ratio>. can take any
values in the interval
0< >. < -13,
but it can not exceed 0.
The maximum value 0 is attained if
fex) = a, x ~ 1/2a, fex) = 0, x> 1/2a,

but this distribution law does not satisfy the continuity condition: >. = 0 if
u = +00.
References
1. J. Haag, C. R. Acad. Sci. Paris 179 (1924), 1388.
2. E. Borel, Traite du calcul des probabilites et ses applications, Paris, 1924.
12. A GENERALIZATION OF THE LAPLACE-LYAPUNOV THEOREM *
Let Xl, X2, ••• be independent random variables with expectations
and suppose that the ratios dn/b n are smaller than a certain fixed constant
(1)
Set
An important problem in probability theory is to study the dependence of

the sums Sn as n -+ 00, on the properties of t n as Jl. -+ O. On the basis of the
now classical law of Lyapunov 1 we have
181 ::; 1, (2)
and R( t n , Jl.) -+ 0 uniformly with respect to Jl. if t n is greater than some constant
T.
Thus, for fixed n we have a formula for the asymptotic behaviour of Sn.
Consider the following problem. Let a(t) and b(t) be functions ofthe parameter
t. What is the probability that all the inequalities
(3)
hold? Assurne that a(t) and b(t) are continuously differentiable and that
a(t) < b(t), a(O) < 0 < b(O).
Then we can obtain an asymptotic solution of our problem similar to (2).

All the sums Sb S2, ... , Sn (up to Sn) can be divided into the following
three classes K, K 1 and K 2 :
K contains those sums for which all the inequalities (3) hold;
* 'Eine Verallgemeinerung des Laplace-Liapounoffschen Satzes', Izv. Akad. Nauk

SSSR, OMEN (1931), 959-962. Presented by S.N. Bernshtein.
1 A.M. Liapounoff, Bull. Acad. Sei. St. Petersburg 13 (1900), 359.
118
A GENERALIZATION OF THE LAPLACE-LYAPUNOV THEOREM 119
K 1 contains those sums for which there exists a k such that all inequalities
a(ti) < Si < b(ti), i = 1,2, ... , k - l.

Sk ~ a(tk)
hold;
K 2 contains those sums for which there exists k such that the inequalities
hold. Denote the probabilities corresponding to these sets by Pn , PA l ), PA 2).

Finally, denote by Pn(x, y) the probability of the event
X< Sn < y.
Clearly
The inequalities
t > 0, a(t) < s < b(t)
single out a region G in the (s, t)-plane. Denote by g(so, to; s, t) the Green's
function for the heat equation
(4)
in G and set
g(s,t) = g(O,O; s,t), og(s,t)fos = u(s,t),
Vl(t) = -u[a(t), t], V2(t) = u[b(t), t].
Theorem. The following asymptotic formulas hold
Pn(x, y) = 11/ g(s, tn)ds + (JR(tn,jt), (5)
(6)
t
PA 2) = Jo v2(t)dt + (J2R2(jt),
n
(7)
I(JI ~ 1, l(Jll ~ 1, 1(J21 ~ 1,

120 A GENERALIZATION OF THE LAPLACE-LYAPUNOV THEOREM
where R, R 1 , R 2 tend to zero uniformly with respect to Pi furthermore, for

R(t, p) this convergence is uniform with respect to t ift is greater than a certain
constant.
If, instead of (3) we have one-sided inequalities, for example,
a(tk) < Sk, k = 1,2, ... ,n, (8)
then a similar result holds, whieh ean be easily obtained from the above by
passing to the limit.
Göttingen, 20 January 1931
13. ON THE GENERAL FORM OF A HOMOGENEOUS
STOCHASTIC PROCESS *
(The problem of Bruno de Finetti)
Let X(A) be a real variable which varies randomly as a function of time A, or

more exactly, in such a way that the probability distribution
depends only on d = A2 - Al and does not depend either on Al or on X(At}

or on the behaviour of X(A) for A < Al. As usual P denotes the prob ability of
the relation inside the braces. Then, as is weH known, the distribution function
cI> a (x) satisfies the functional equation I
(1)
In the present paper, generalizing certain results of Bruno de Finetti,2 we

give a solution to equation (1) under the sole assumption that the first and
second moments
are finite.
For non-specialists, we note that cI> a (x) is non-decreasing and left contin-
uous and that cI>a(-oo) = 0, cI>a(oo) = l.
Let
* This article was presented on 6 March 1932 by the member of the Italian
Academy of Sciences, G. Castelnuovo in the journal Atti R. Accad. Naz. Lincei.
Sero Sesta. Rend. and in the same year published in two parts in Italian. The first
part was called 'Sulla forma generale di un processo stocastico omogeneo' 15:10,
805-808). The translation of this part is the title of the present article. The
second part was called 'Ancora sulla forma generale di un processo omogeneo'
(15:12, 866-869).
I See, for example, my paper: 'Über die analytischen Methoden in der Wahrschein-
lichkeitsrechnung', Math. Ann. 104 (1931), 415-458. (Paper 9 in this volume.)
2 See: B. de Finetti, 'Le funzioni caratteristiche de legge istantanea', Atti Accad.
Naz. Lincei. Rend. 12 (1930), 278-282.
121
122 ON THE GENERAL FORM OF A HOMOGENEOUS STOCHASTIC PROCESS
be the characteristic function of 4> ~ (X). From the weH known properties of
characteristic functions we have
Since tP~(t) is continuous with respect to Ll, 3 we have
(2)
Since the distribution 4>~(x) is uniquely determined by tP~(t), in order to

describe it, it suffices to find the general form of tPl (t).
According to de Finetti, we have 4
Moreover,
-[tP~(t)
1
Ll
1 eitxd4>~(x) =
- 1] = Ll
1 [
00
- 1]
1 xd4>~(x) + 1
-00
= -1 [it itx)d4>~(x)]
1:
00 00
(e,tx
. - 1- =
Ll -00 -00
! [itm~ + (e itx -1- itx)d4>~(x)] =

1:
=
= itml+! (e itx - 1 - itx)d4>~(x).

Now let
(3)
The function F~(x) is non-decreasing with respect to x; moreover, F~( -00) =

o and
1
F~(oo) = Ll 1 00
-00
1 (m~
x 2 d4>~ (x) = Ll 2
+ (T~2 ) = Llm 2l + (Tl·
2
3 If D. :5 l/n, then u~ = U~/n - U~/n_~ :5 U~/n = (l/n)u? Therefore u~ -+

o as D. -+ 0 and "p~(t) -+ 1 accordingly. Taking into account the equality
"p~1+~2(t) = "p~1(t)"p~2(t) we conclude that "p~(t) is continuous with respect
to D..
4 ["p~(t) _ l]/D. is bounded for each fixed t, hence the equality "pl(t) = 0 is
impossible, so that log"pl (t) is defined for any t.
ON THE GENERAL FORM OF A HOMOGENEOUS STOCHASTIC PROCESS 123
For any inter val (a, b) not containing 0, a i= 0, b i= 0, we have
where
p(x,t) = (e itz -1- itx)/x 2 for x i= 0,
(4)
p(x, t) = -t 2 /2 for x = o.
Since for any given t the function p(x, t) is finite and continuous (including at
x = 0), we have
We now choose a sequence of positive numbers Al, A2, ... , An, ... tending
to zero. As is weIl known, given a sequence of functions
we can select a subsequence
that converges to some function F( x) at all points of continuity of the latter.

Of course, F(x) is non-decreasing and its extremal values are
F(-oo) ~ limFd(-oo) = 0,
(5)
F(oo) ~ limFd(oo) = (T~ (as A ~ 0).
Taking into account the fact that p(x, t) ~ 0 for fixed t as x ~ ±oo, we finally
I: I:
obtain
p(x, t)dFdn" (x) ~ p(x, t)dF(x),
!
I:
log tPl(t) = lim [tPd(t) - 1] =
= lim[itml + (e itz -1- itx)dcJd(x)] =
= lim[itml + I: ~ 0);
I:
p(x, t)dFd(x)] (as A
log tPl(t) = itml + p(x, t)dF(x). (6)

Since the second moment u? is finite, we have
Moreover, (6) gives
implying that
F(oo) - F(-oo) = u?;
therefore, taking (5) into account,
F(-oo) =0, F(oo)=u?
Let us assume that F( x) is left continuous (though the values of F( x) at

points of discontinuity are of no importance). We have already mentioned that
F(x) is non-decreasing.
Formulas (6), (4) and (2) together with the well-known formula
clia(x) - clia(O) = -211"1 1 00
-00
1- e- itx
.
zt
tPa(t)dt (7)
give the gen~ral solution of our problem.

The function F(x) is completely determined by the distribution function
I:
clia(x). Indeed, the integral
p(x,t)dF(x) = 10gtPl(t) = itml

can be differentiated twice with respect to t and taking into account the equality
82
-p(x
8t 2 t) - _e itx ,
,-
1
we have
fj2
= -!i2logtPl(t) = x(t),
00
eitxdF(x)
1
-00 vt
= -211"1 1-
.zte-it:r: X(t)dt.
00
F(x) - F(O)
-00
Fr~m the latter equality we can determine F(x) to within an additive constant
which, in turn, is determined by the condition F( -00) = o.
As already mentioned, from any sequence F~n(x) with ß n ---+ 0 we can
always select a subsequence F~nk (x) converging to F(x). Hence, F~(x) ---+
F(x) as ß ---+ 0 and
(8)
However, (8) is only true at those points where F(x) is continuous.

For x < 0, (3) implies that
Further (8) implies that at each point of continuity x < 0 of F(x) we have
(9)
Similarly, for x > 0 we have
(10)
The meaning of Pl(X) and P2(x) can be explained as follows. In general

X(A) does not have to change continuously with time A; this change can also
have jumps and P2(x)dA is the probability that during the time interval dA
the function made a positive jump greater than x. Similarly, P1 (x)dA is the
prob ability of a negative jump with absolute value greater than lxi.
It foHows from (9) and (10) that
F(x) = [~y2dPl(Y), x< 0, (U)
ul- F(x) = F(oo) - F(x) = 100

y 2dP2(y), x> O. (12)
Clearly, these formulas, as weH as aH the others in which P1 (x) and P2 (x)
occur hold only when F(x) is continuous.
°
Thus, the behaviour of F(z) outside z = depends only on the probability
distribution ofthejumps of X(A). At the same time, the discontinuity of F(z)
at the origin is associated with the continuous variation of X(A). To make this
clear we set
O(z) = F(z), z ~ 0,
O(z) = F(z) - u~, z > 0,
where u~ denotes the jump of F(z) at z = 0. Clearly, dF(z) = dO(z) outside
a neighbourhood of the origin. We can write (11) and (12) as follows
O(z) = 11C
00 y2 dP1 (y) , z < 0, (13)
u~ - O(z) = 1 00
y2 dP2 (y), z > 0, (14)
where u~ = u~ - u~. Since O( z) must be continuous at the origin, u~ = 0(00)
1: 1:
and hence O(z) is completely determined by P1(z) and P2 (z). We have
p(z,t)dF(Z) = u~p(O,t) + p(z,t)dO(z).
Since p(O, t) = -t 2 /2, we obtain the foHowing final

Main formula
log'l/Jt(t) = imlt - 17;t 2 +

2
1 00
-00 p(z,t)dO(z). (15)
= 0, that is, if X(A)

If O(z) can vary only continuously as a function of
time, then u~ = u~. We have
We show in conclusion that for any function F(z) (non-decreasing, left

continuous and with values F( -00) =
0, F(oo) =
u~ < 00) (7), (6) and (2)
give a solution to our problem.
In other words, we are going to prove that under the above conditions
4.>.1(z) given by (7) really is a distribution function for aH A > 0.
First we consider a step function T(x) with jumps
only at a finite number ofpoints Xl, X2, ... , Xn. Denote by CT~ = Wl +W2+ ...+Wn
the sum of these jumps (it is constant on every interval that does not contain
any of the Xk). Suppose also that X = 0 is not one of the points Xk. Then 5
1 00
-00
p(x,t)dT(x) = LPk(e it 1: k
k
- 1- itxk),
log~l(t) = itP+ LPk[n(t) -1],

k
n(t) = exp(itxk) i
Clearly n(t) = exp(itxk) and exp(itP..6.) are characteristic functions, which

means that by (7) they generate distribution functions. The results of de Finetti
imply that in this case exp{..6.pk[Xk(t) - In
and ~/l.(t), which is a product of
characteristic functions, are characteristic functions.
Under an appropriate choice ofta and for f > 0 one can always approximate
1:
any function F(x) by a step function T(x) so that the inequality
11: p(x, t)dF(x) - p(x, t)dT(x) 1 < f (16)
holds for all Itl < ta. We now form a sequence Tn(x) offunctions of the form
T(x) such that the difference in (16) tends to 0 for all t, uniformlyon every
bounded interval. Then the corresponding functions ~~)(t) converge to tP/l.(t),
which implies that tP/l.(t), being a limit of characteristic functions, is itself a
characteristic function.
5 When we replace F(x) by T(x) we write ~/l. instead of tP/l..

14. ON COMPUTING THE MEAN BROWNIAN AREA *
Jointly with M.A. Leontovich
In this paper we solve the following problem posed by S.1. Vavilov. Determine
the expectation of the area covered during a given time by the projection onto a
plane of a moving Brownian particle of finite size. The parts of the area covered
by this projection several times must be counted only once. This problem will
be reduced to the following more general problem: determine the probability
that an infinitesimal Brownian particle moving in a region G and situated at
t =
0 at a given point (x, y) hits the boundary R of this region at least once
during the time t. In §1 we explain a method for solving this problem, and in
§§2, 3 this method is applied to computing the mean Brownian area. §§1, 2 of
this paper are written by A.N. Kolmogorov, §3 by M.A. Leontovich.
§1. First we consider the motion of an infinitesimal particle on aplane with

Cartesian coordinates x and y. (The analogous three-dimensional problem can
be studied in the same way.) P(x, y; t) is the prob ability that the particle
situated at time t = 0 at a point (x, y) of G hits the boundary R of this region
at least once during time t. Further, let PL(X,y;t) be the probability that the
particle situated at time t = 0 at the point (x, y) hits the boundary R at least
on ce during time t in such a way that it first hits apart L of R. Then, clearly,
PL(x,y;t) ~ P(x,y;t).
Now let p(x,y;e,'fJ;t)ded'fJ be the prob ability that the particle which was
e
at the point (x, y) at the moment t = 0 is in the region (e, + dei 'fJ, 'fJ + d'fJ) at
time t and does not hit the boundary during time t. Then, clearly,
Lp(x,y;e''fJ;t)d1,d'fJ+ P(x,y;t) = 1. (1)
Moreover, it can easily be seen that
P(x, y; t + r) = P(x, y; r) + L p(x, y;e, 'fJ; r)p(e, 'fJ; t)ded'fJ, (2)
PL(X, y;t + r) = PL(X, y; r) +L p(x, y;e, 'fJ; r)PL(e, 'fJ;t)ded'fJ, (3)
* 'Zur Berechnung der mittleren Brounschen Fläche', Phys. Z. Sow. 4 (1933),1-13.
128
ON COMPUTING THE MEAN BROWNIAN AREA 129
We now make the following assumptions:

I. The functions P(Z,y;t) and PL(Z,y;t) can be expanded into Taylor
series in a neighbourhood of every interior point of G, so that
p(e, 77;t) = P(z, y;t) + 8P(;~y;t) (e - z) + 8P(;~Yit) (77 - y)+
2P(Z'Yi t )(t_)2 82P (z,Yi t )(t_)( - )

+1{8
2 8z 2 .. Z +2 8z8y .. z 77 Y +
+ 82P~:; y; t) (77 _ y)2} + 8{(e _ z)2 + (77 _ y)2}3/2, (4)
PL(e, 77;t) = PL(Z,Yi t ) + 8PL~~Yit)(e - z) + 8PL~~Yit)(77 - y)+
1{82PL(Z'Yi t )(t_ )2 282PL(Z,Yit)(t_ )( _ )

+2 8z 2 .. Z + 8z8y .. z 77 Y +
+ 8 2PL8~; Yi t) (f/ _ y)2} + 8{(e - z)2 + (77 _ y)2}3/2, (5)
where 181 $ M, 10'1 $ M', M and M' being independent of eand 77 (but may
depend on z, y and t)i
11. At every interior point of G,
lim P(z, Yi T) = o. (6)

.,. ..... 0 T
Since PL $ P, this also implies that
lim PL(Z, Y; T) = o. (7)

.,. ..... 0 T
IH. The function p(Z,Yie, 77i t ) satisfies the following conditions:
lim{.!. { p(Z,Yie, f/iT)(e - z)ded77}

.,. ..... 0 T Ja = A 1 (z,y), (8)
lim{.!. { p(Z,Yie,f/iT)(f/-y)ded77} =A 2(z,y), (9)

.,. ..... 0 T Ja
(10)
130 ON COMPUTING THE MEAN BROWNIAN AREA
lim{2 1 f p(X,y;e,TJ;r)(TJ-y)2cJedTJ}=B 22 (X,y), (11)

,.-0 r JG
lim{ 2 1 f p(x, y;e, TJ; r)(e - x)(TJ - Y)cJedTJ} = B I2 (X, y), (12)
,._0 r JG
lim{!r JGf p(x, y;e, TJ; r)[(e -

,._0 x)2 + (TJ - y?]3/2dedTJ} = 0, (13)
where Al, A 2, B u , B 12 , B 22 are functions defined by these relations.

Under the assumptions 1-111 the functions P(x,y;t) and PL(X,y;t) are
solutions to the following differential equation:
8P 82p 82p 8 2p 8P 8P
7ft = B u 8x2 + 2Bl2 8x8y + B 22 8 y2 + Al 8x + A 2 8y . (14)
We prove this for P(x, y; t), since for PL(X, y; t) the proof is quite similar. By
(2) and (4) we have
P(x,y;t+r) P(x,y;t)
r r
-_P(x,y;r)
r
+r !1 ( G
·)P(C.. ,TJ,·t)d'Cd_P(x,y;t)_
p x,y,.c.. ,TJ,r .. TJ
r
-
-_P(x,y;r)
r . . )'/cd
+:;:l {JGf P( x,y,e,TJ,r ..... TJ-l }( .)+
P x,y,t
+-r11G p(x,y;e,TJ;r)(e-x)dedTJ 8P(x,y;t)

8
x
+
1 f ( )( 2 8 2p(x,y;t)
+2rJG Px ,y;e,TJ;r e-x) dedTJ 8x 2 +
1 f 8 2 P(x, y;t)
+2rJG P(x,y;e,TJ;r)(e- x )(TJ-y)cJe dTJ 8x8y +
1 f 2 82p(x,y;t)
+2r JGP(x,y;e,TJ;r)(TJ- y) dedTJ 8 y2 +
+ 9" f p(x, y;e, TJ; r){(e - x)2 + (TJ - y)2}3/2dedTJ, (15)

r JG
where 19"1 $ M". By (6) and (13) the first and the last terms on the right-hand
side of (15) tend to 0 as r -+ O. But, since by (1)
1- L p(x, y;e, TJ;t)dedTJ = P(x, y;t),

the second term on the right-hand side of (15) also tends to 0 as T --+ O. The
coefficients of the derivatives of P with respect to :c and y have the limits
Al, A 2, B 11 , 2B 12 , B 22 ; therefore the right-hand side of (15) tends to the right-
hand side of (14) as T --+ O. The left-hand side of (15) also has the limit oplot.
This proves (14).
Equation (14) satisfied by P is adjoint to the Fokker equation. This equa-
tion can also be derived under much weaker assumptions (cf. [1]-[4]).
We now assume that, in addition to I-lU the following two conditions hold:
IV. For fixed t (where t > 0) the probability P(:c, y; t) tends to 1 as (:c, y)
approaches the boundary of G, and P(:c, y; t) == 0 at t = 0 for any interior point
(:c, y) of G;
V. For fixed t (where t > 0) the prob ability PL(:C, y;t) tends to 1 as (:c, y)
tends to an interior point of the part L of the boundary and tends to 0 as (:c, y)
tends to a point of the boundary outside L, and PL(:c, y; t) == 0 at t = 0 for
every interior point (:c, y) of G.
Taking into ac count that both P and PL are bounded (namely, they are
non-negative and not greater than 1), it can easily be shown that both P and
PL are uniquely defined by (14) and the conditions IV and V.
§2. Computation of the area covered by a Brownian particle of finite

size. Assume that the projection of a Brownian particle can be represented by a
disk ofradius 1 (under an appropriate scaling). In what follows we only consider
such projections, so for simplicity we call them particles. Note, however, that an
entirely similar treatment can be given in the corresponding three-dimensional
problem of computing the volume filled in by a Brownian particle.
The centre of the particle obeys the laws stated in §l. Symmetry consider-
ations make it natural to assume that Al = A 2 = B 12 = 0 and B 11 = B 22 = D.
Then (14) takes the form
oP =D(02P +02P).
ot o:c 2 oy2
By (10) the diffusion coefficient D is defined by the relation

where E denotes the expectation. It ean easily be seen that D eoineides with
the analogous eonstant for a three-dimensional Brownian particle.
Our problem is to determine the expectation 0/ the area covered by succes-
sive positions of a particle at times from t = 0 to a given t. As the origin we
take the point that coincides with the particle's eentre at t = O. Denote by
W(x, Yj t) the probability that during time ta given point (x, y) is at least onee
eovered by this particle. When the distanee from (x, y) to the origin is at most
1, then clearly, W(x,Yjt) = 1. 1fthis distanee exeeeds 1, then (x,y) is eovered
by the particle at least onee if and only if the route of the particle's eentre falls
at least onee inside the disk with radius 1 and eentre at (x, y). By symmetry
eonsiderations the probability of such an event is also equal to the probability
=
that the eentre of the Brownian particle situated at (x, y) for t 0 will during
time t touch the boundary of the disk S of radius 1 with eentre at the origin at
least onee. This probability ean be eomputed using the eonsiderations of §1.
Thus, inside S we have
W(x,Yjt) = 1, (16)
while outside this disk W(x, Yj t) must satisfy the differential equation
8W = D(8 2 W 8 2 W) (17)
8t 8x 2 + 8y2 •
Moreover, on the boundary of S the boundary eondition
W(x, Yjt) =1 (18)
must hold, as weH as the initial eondition
W(x, Yj 0) = O. (19)
To determine the expeetation of the Brownian area, we must reason as

foHows. Let ~ be the region eovered by the particle during time t and F the
area of this region. Assume that c5(x, Yj~) = 1 for (x, y) in the interior of ~,
and c5( x, Yj ~) = 0 otherwise. Clearly,
E{c5(x,Yj~)} = W(x,Yjt),
and the Brownian area F ean be defined by the relation
F= JJc5(x, Yj ~)dxdy.
This implies that
E(F) = 11 E{c5(x,y;~)}dxdy= 11 W(x,y;t)dxdy. (20)
§3. To solve the desired problem with boundary conditions we pass to polar
coordinates (r,<p). Clearly, W depends only on the position vector r, and not
on the angle <p. We now choose the time unit in such a way that D takes the
value 1. In this case our problem with boundary conditions for W(r, t) takes
the form
8W 82W 18W
- - - - + r-8r
8t - 8r 2
-' (21)
W(I, t) = 1; W( 00, t) = 0; (22)
W(r,O) = 0, r> 1. (23)
To solve this problem we use the Laplace transformation. Multiply (21)

by e- vt and integrate over t from 0 to 00. Integrating by parts and using the
initial condition (23) for
U(r, v) = 1 00
W(r, t)e-vtdt (24)
we obtain the following differential equation:
d2U 1 dU
-+---vU=O. (25)
dr 2 r dr
The boundary conditions (22) for W yield the following conditions for U:
U(I,v) = I/v; U(oo,v) = O. (25')
A solution of (25) under the conditions (25') is of the form
1 K(ry'v)
U(r,v) =~ K(y'v) ,
where
K( x ) -- i1r (l)(. )
2 H 0 'x,
and H~l) is the first Hankel function of order zero (see [5], §17.71).
By (24), in order to find W(r, t) we have to solve the linear integral equa-
tion of the first kind
(JO ( -vt 1 K(rVv)
Jo W r,t)e dt = -;;- K(Vv) . (26)
The right-hand side of this equation for Re v > 0 is an entire function of v.

Indeed, by a known theorem (see, for example, [6], §15.7), K(x) does not vanish
for I arg x I ::;
/2. A solution of our integral equation when the right-hand side
7r
satisfies this condition is well-known (see, for example, [7], §6.7), and is of the
following form:
_ 1 l(X+ioo evt K(rVv)
W(r,t) - -2. - K(Vv) dv, (27)
7rt a-ioo V V
where the integration is taken over a straight line parallel to the imaginary axis
and a > O. This gives the solution of our problem with boundary conditions.
Equation (27) holds for r > 1, whereas if r ::; 1, then by (16) we have
W(r,t) = 1. (27')
For the expectation we have from (20)-(23) and (27'),
E(F) = 27r 1 00
W(r, t)r dr = 7r + 27r 100
W(r, t)r dr =
= 7r _ 27r
Jo
r oW(l,
or
t) dt, (28)
and from (27)

'l(X+ioo evt K'(Vv)
E(F) = t . 3/2 K(Vv) dv + 10 , (28')
(X-aoo v v
where
' l a+ioo 1 K'(Vv)
10 = 7r - t . K(v'V) dv.
3/2
(X-aoo v v
Now we derive an asymptotic formula for E(F) for large t. It is known
(see [5], §17.71) that
K(x) = -Jo(ix) In "'I; + x2 P(x),

K'(x) = - Jo(ix) _ iln "'Ix JMix) + x[2P(x) + xP'(x)],
x 2
where, = eC , Cis Euler's constant (that is, , = 1.7810 ... ) and P(x) is an
entire function.
Hence,
K'(x) 1
K(x) x In(,x/2) + xR(x)j
R(x) = iln(,x/2)J~(ix)/2 - 2P(x) - P'(x)x + P(x)/ln(,x/2).
Jo(ix) In(,x/2) - x 2P(x)
Since K'(x)/K(x) does not have singularities for largxl < 7r/2 (except x = 0)
and tends to 0 as lxi -+ 00 (which follows from the weIl known asymptotic
formulas K(x) "" (7r/2)1/2 e-X /x 1/ 2 and K'(x) "" _(7r/2)1/2e -X /x 1/ 2), R(x)
does not have other singularities in the same region, except at x = 2/,. As
x-+O
R( x) -+ _1. + A + --o-_B--:--:-
2 ln(,x /2) ln(,x /2) ,
hence IR(x)1 is bounded.
By (28') we may set
E(F)=lo +lt+h, (29)
where
",+ioo evt
12 = i l -R( -.[ü)dv.
",-ioo v
To avoid the singularity of R( v'V) at v = 4;'2 we choose as the path of inte-

gration a straight line that intersects the real axis between 0 and 4;'2 j in other
words, we assurne that a < 4/,2. It is then easy to estimate 12. Setting vt = e
we obtain
12 =i l
",+ioo evt
. -R(-.[ü)dv=i
"'-'00 v
l"'t+ioo
.
",t-.oo..
c
e€
R(
If-)cJe=
t
where ß can be considered as being independent of t. The expression

is bounded for any t. This can easily be seen by applying the Second Mean
Value theorem to its real and imaginary parts.
Thus we see that
(30)
The interval 10 does not depend on t, so that
10 = 0(1). (31)
It merely remains to estimate the integral
where z = v-y2/4, T = 4th 2, a1 = -y2 a /4. Transforming the integrand of h

using the identity
__
1 = e -21nz - 1 + _1 = -2
z2 In z In z In z a I n z
1
1 1
e-2alnzda+_,
we obtain
---.,--)
Fig. 1
In the term proportional to f e~: ~z the integration can be carried out along
the imagiriary axis. It is then easy to show (for example, by applying the Second
Mean Value theorem to the real and imaginary parts ofthe integral), that this
term is of order 0(1). In the expression f e-2alnz+zT dz the integration path
can be deformed so that it coincides with the loop (5 encircling the negative
part of the real axis (see Fig. 1). This expression can then be transformed in
the following way:
We use the relation (see [5], §12.22)
[ (-2a 'd( = 21ri

Je e r(2a) ,
from which we obtain
21r1'2 [1 e2a In T
lt = -r- Jo r(2a) da + 0(1).
The integral on the right-hand side of the latter relation is given by
1 e2alnT _ r212 e-ylnTdy _ r 2 {1 1 e-ylnTdy
1 --da--
o r(2a) 2 0 r(2 - y)
--
2 0 r(2 - y)
+
+.! [1 (1- y)e-YlnTdy}.

r Jo r(2 - y)
Hence, by (29), (30) and (31) it follows that
[1 e- ylnT dy
E(F) = 41rt Jo r(2 _ y) + 0(1).
For large In r the following asymptotic relation holds:
[1 e- Y lnT dy 1 1
Jo r(2-y) "'Inr = In( 4t h 2) ,
and therefore
(32)
For 0 < Y < 1 we have

1 1
1 < r(2 _ y) < 0.88'
and therefore,
1 l1e-YlnTdY 1
-Inr < < -=-::-::-:--
0 r(2-y) 0.88Inr·
In order to obtain an asymptotic expansion for E( F) we can, by successive
integration by parts, represent the latter integral as
dy _
[1 e-yln T 1 { 1 \)(2) 1 \)2(2) - \)'(2)r(2) }
Jo r(2 - y) - In r . r(2) 1 + In r r(2) + In 2 r r(2) + . .. ,
where \)(x) = r'(x - l)jr(x - 1). Substituting here the numerical values of r
and \) we obtain
41rt { 0.423 0.467 }
E(F) = In 126t
. 1 + I n 1.26t - In 2 1.26t + ... + 0(1).
If times and lengths are measured in arbitrary units, then t must be replaced
by Dt/(1'2, and F by F/(1'2, where (1' is the particle's radius. Thus, finally we
obtain
411"Dt { 0.423 0.467 }

E(F) = In(1.26Dt/(1'2) 1 + In(1.26Dt/(1'2) - ln 2(1.26Dt/(1'2) +... + 0(1),
which is the asymptotic solution of our problem.
References
1. A.N. Kolmogoroff, 'Über die analytischen Methoden in der Wahrschein-

lichkeitsrechnung', Math. Ann. 104 (1931), 415-458. (Paper 9 of this vol-
urne)
2. A.N. Kolmogoroff, 'Zur Theorie der stetigen zufälligen Prozesse', Math.
Ann. 108 (1933), 149-160. (Paper 170fthis volume)
3. A.N. Kolmogoroff, 'Eine Verallgemeinerung des Laplace-Liapounoffschen
Satzes', Izv. Akad. Nauk SSSR OMEN, (1931), 959-962. (Paper 12 of this
volume)
4. A.N. Kolmogoroff, 'Uber die Grenzwertsätze der Wahrscheinlichkeitsrech-
nung', Izv. Akad. Nauk SSSR, OMEN (1933), 366-372. (Paper 16 of this
volume)
5. E.T. Whittaker and G.N. Watson, A course ofmodern analysis, Cambridge
Univ. Press, 1927.
6. G.N. Watson, A treatise on the theory of Bessel functions, Cambridge
Univ. Press, 1922.
7. R.H. Fowler, Statistical mechanics, Cambridge Univ. Press, 1929.
15. ON THE EMPIRICAL DETERMINATION
OF A DISTRIBUTION LAW *
We consider the possibility of determining a distribution law based on a finite

number of tests.
§1. Let XI, X 2 , ••• ,Xn be the results of n mutually independent observations,
ordered increasingly, that is, Xl ~ X2 ~ ••• ~ X n and let
F(x) = P{X ~ x}
be the distribution law corresponding to this sequence. The empirical distri-

bution law is the function Fn(x) defined by the relations
Fn(x) = 0, x< Xl;

Fn(x) = kin, Xk ~ x < Xk+l, k = 1,2, ... ,n-I;
Fn(x) = 1, Xn ~ X.
Hence, nFn(x) is the number of the X k not exceeding x. A natural quest ion
is: does Fn (x) approach F( x) for large n ? A theorem related to this question
was formulated by von Mises [1] and is called the w2-method. However, the
fundamental statement that the prob ability of the inequality
D = sup IFn(x) - F(x)1 < f
tends to 1 as n -* 00 for any f has not, in my view, been explicitly formulated

until now, though it could have been proved in various simple ways.
I shall obtain this statement as a direct consequence of the theorem proved
below. I should add that the setting of the problem considered below is moti-
vated by re cent studies of Glivenko [2].
Theorem 1. For any continuous distribution /unction F(x), the probability

n(A) 0/ the inequality
D = sup IFn(x) - F(x)1 < Alyn

* 'Sulla determinazione empirica di una legge di distribuzione', Giorn. Ist. Ital.
Attuar. 4:1 (1933), 83-91.
139
140 ON THE EMPIRICAL DETERMINATION OF A DISTRIBUTION LAW
tends to
(1)
-00
uniformly in ~ as n -+ 00.
Below the table of some values of ~(~) is givenj these were calculated by
N. Kozhevnikov.
~ ~(~) ~ ~(~) ~ ~(~)
0.0 0.0000 1.0 0.7300 2.0 0.99932

0.2 0.0000 1.2 0.8877 2.2 0.99986
0.4 0.0028 1.4 0.9603 2.4 0.999973
0.6 0.1357 1.6 0.9888 2.6 0.9999964
0.8 0.4558 1.8 0.9969 2.8 0.99999966
As is dear from the table, D ~ 2.4/,fii can be considered to be practically

certain. Moreover, it turns out that
~(0.83) '" 0.5.
is small, then (1) converges very slowlYj in this case the following
If ~
asymptotic formula may be used
At ~ = 0.6 this formula yields
~(0.6) '" 0.1327,
instead of the value

~(0.6) '" 0.1357
computed using the exact formula (1).
§2. Lemma. The probability function ~n(~) does not depend on the distribu-
tion function F( z) if the latter is assumed to be continuous.
ON THE EMPIRICAL DETERMINATION OF A DISTRIBUTION LAW 141
Proo/. Let X be a random variable with continuous distribution law F(x);

clearly, corresponding to the random variable Y = F(X) there is the distribu-
tion law F(O)(x) for which
F(O)(x) = 0, x ~ 0;
F(O)(x) = x, 0 ~ x ~ 1;
F(O)(x) = 1, x ~ 1.
If Fn(x) and F~O)(x) are the empirical distribution laws for X and Y for n
observations, then
Fn(x) - F(x) = FAO)[F(x)] - F(O)[F(x)] = FAO)(y) - F(O)(y),

sup IFn(x) - F(x)1 = sup IFAO)(y) - F(O)(y)l.
This implies that the function n(.~) corresponding to an arbitrary continuous

distribution function F( x) is identical to the function corresponding to F(O) (x).
Therefore, in proving our theorem, we can confine ourselves to the case F(x) =
F(O)(x).
In what follows we will write Fn(x) instead of F~O)(x) and confine ourselves
to x for which 0 ~ x ~ 1 and F(O)(x) = x. Our problem then reduces to finding
the probability n (A) of the inequality
sup IFn(x) - xl< A/..fii, 0 ~ x ~ 1. (2)
Let A be of the form
where J-l is an integer. Substituting it into (2) we obtain
(3)
The values Fn(x) are multiples of l/n; for example, let Fn(x) = i/n and
x = j/n + f (0 ~ f < l/n). Taking into account the fact that Fn(x), being a
distribution function, is monotone, we immediately obtain
i-j
Fn(x) - x = - - - f,
n
j) j i-j
Fn ( - - -
n n
~ Fn(x) - (x - f) = --,
n
j+1) j+1
Fn ( -n- --n-~Fn(x)-
(X+-;;-f
1 ) = i-j-1
n .
142 ON THE EMPffiICAL DETERMINATION OF A DISTRIBUTION LAW
Therefore the inequality
i-i
IFn(z)-zl= I--Cl I'
~-
n n
holds if and only if at least one of the following inequalities holds:
Fn ( -i) -
i
-
i-i
~ - - ~ --,
I'
n n n n
This implies that (3) may be replaced by the following:
Now let Pilc be the probability of simultaneous fulfillment of the relations
IFnU In) - jfnl < I'/n, i = 0,1, ... ,kj

(5)
IFn(k/n) - k/nl = i/no
Note that
(6)
For k = 0 we clearly have
P oo = 1, PiO = 0 (i # 0) (7)
More generally,
Pilc =0 (8)
for lil ~ 1', since in this case the inequalities (5) are contradictory. Furthermore,
Pik+l = L PjkQ~~), lil < 1', (9)

j
where Q~~) denotes the probability of Eik+l under the condition that Ejk holds,
that is, the prob ability of
(10)
subject to the condition
Fn(kln) = (k + j)ln. (11)
Relation (11) means that, of the results of the n observations Xl, X 2, ... , X n
exactly n - k - j belong to the interval kin< z :5 1; (10) can therefore be true
only when i - j + 1 of the results of these n - k - j observations belong to the
interval kin< z :5 (k + l)ln.
Provided that X m is uniformly distributed, we obtain the following ex-
pression for our desired probability:
Q(k)
.. -_
JI
(n - + j) (
i - j
k-
1
1 -1-)n-k-i-l ( - 1 )i-i+
n- k n- k
1
.
(12)
Formulas (7)-(9), (12) and (6) enable us to find the prob ability cf>n(A) for the
case A = 1'1.,fii.
These formulas can be replaced by other, more convenient ones. For this
we set
(n-k-i)!nn -k
Rik = (n -
k) n-",-In.I ePik.
L·
(13)
Conditions (7) and (8) turn into the following:
Roo = 1, Rio = 0, i i= 0; (14)

Rik = 0, lil ~ 1'. (15)
Simplifications reduce (9) to
(16)
Finally, from (6) and (13) we obtain
(17)
Formulas (14)-(17) also enable us to find cf>n(A) for the case A = 1'1.,fii.
§3. Now let Yl, Y2 , ••• , Yn be a sequence ofindependent random variables with
distribution law given by the formula
{ i-I}
1 .
P Yk = - - = "1e-l, i= 1,2, .... (18)
I' z.
144 ON THE EMPffiICAL DETERMINATION OF A DISTRIBUTION LAW
Setting
we easily see that the probability Rö" that the relations
(19)
hold simultaneously satisfies the same conditions (14)-(16) that Rö" does, in
other words, Rö" = Rö". This allows us to give an asymptotic expression for
Rö" as n -+ 00. For this we state the following general theorem.
Theorem 2. Let Yl. Y 2 , ... , Y n be a sequence 0/ independent random variables

whose values can only be multiples 0/ a constant f.
Let
E(Y,,) = 0, E(Yl) = 2b", E(IY,,31) = d",

S" = Y1 + Y2 + ... + Y",
t" = b1 + b2 + ... + b"
and let a(t), b(t) be continuously differentiable /unctions satis/ying
a(t) < b(t), a(O) < 0 < b(O).
Denoting by Rön the probability that the relations
Sn = if
hold simultaneously and by u(u, T, S, t) the Green's /unction o/the heat equation
in the domain G dejined by the inequalities
a(t) < s < b(t),
we have
Rön = f· {u(O, 0, if, tn) + Ll},
where ,6. tends uniformly to 0 as l -+ 0 if the following conditions hold:
a) a(t) and b(t) remain constant and the t n are bounded:
b) there exists a constant C > 0 such that
c) there exists a constant K >0 such that for any k there exists an ile
for which
d) there exists a constant A such that
Apart from these restrictions, the YIe, as weil as their number n and the
integer i, can depend on l in arbitrary fashion.
This theorem falls into the same scope of ideas as the one given in [3].
However, the assertion of the above theorem is stronger: the theorem in
[3] allows us only to assert that
L: Rän = l
i=q
i=p
qE
pE
11.(0,0, z, tn)dz + ,6.',
where ,6.' tends to 0 as l -+ 0 under the conditions a) and b). The condition
c), which is essential in our new theorem, had already been used by von Mises
in similar considerations.
In our case
l = 1/,.,.,
E(YIe) = 0, E(Yl) = 2ble = 1/,.,.2, E(Yt) = die = C/,.,.3,
dle/ble = C/,.,. = Cl, t n = n/2,.,.2 = 1/>..2, a(t) = -1, b(t) = +1,
~n = l{ 11. ( 0,0,0, ;2) + ,6.},

146 ON THE EMPIRICAL DETERMINATION OF A DISTRIBUTION LAW
1 +00 k (s-2k)2
U(O, 0, s, t) = 2..jirt ~) -1) exp [- 4t ],
-00
+00
= ~)_1)ke-2k2>.2 + R = 4>(A) + R.
-00
In this formula the remainder R tends uniformly to 0 as n ~ 00 if A is

greater than some AQ > O. For in this case f = 1/J.l = l/(A.,jn) tends to 0 as
n~oo.
Thus, Theorem 1 is proved for values A of the form J.l/ yn, provided that
A > AQ. Since the limit function 4>(A) is continuous and its limit value is
4>(0) = 0, it is easy to see that these restrictions are essential. We prove
Theorem 2 elsewhere.
References
1. R. von Mises, Wahrscheinlichkeitsrechnung und ihre Anwendung in der

Statistik und theoretischen Physik, Fr. Denticke, Leipzig, Vienna, 1931.
2. V. Glivenko, 'Sulla determinazione empirica della leggi di probabilita',
Giorn. Ist. Ital. Attuar. 6:1 (1933), 92-99.
3. A. Kolmogoroff, 'Eine Verallgemeinerung des Laplace-Liapounoffschen
Satzes', Izv. Akad. Nauk SSSR, OMEN, 1931, 959-962 (Paper 12 in this
volume).
16. ON THE LIMIT THEOREMS OF PROBABILITY THEORY *
As is well known, the fundamental ideas of P. Laplace [1] and P.L. Chebyshev
[2] were subsequently developed in papers by A.A. Markov [3] and A.M. Lya-
punov [4], culminating in a very general statement (the so-called Lyapunov
theorem) on the limit of probability distributions for sums of a large number
of small independent random variables. Further studies by Markov [5] and
S.N. Bernshtein [6] have demonstrated that in many cases a similar statement
also holds for sums of independent variables. These generalizations are of spe-
cial importance for applied purposes, but in principle they do not go much
further; in all cases studied by these authors only summands that are near
to each other are strongly dependent, whereas if the sum is decomposed into
sufficiently long partial sums, the later ones will be almost independent. Of
much greater consequence is the two- (and multi- )dimensional generalization
of Lyapunov's theorem to the case of sums of ralildom vectors, which was first
rigorously proved by Bernshtein [6].
What still remains after all these studies is to determine the limits of
various kinds of probabilities related to the whole set of partial sums of a given
sequence of random variables: these limits are unknown even in the simplest
case of independent summands. In any case, the early results of Laplace and
Poisson [7] on the prob ability of a gambler's ruination belong to this field; they
were further elaborated by P. Levy [8]. Recently I published a general statement
of this kind [9]. In the meantime some partial two-dimensional problems of
similar type were studied by R. Lüneburg [10]. Both these papers clarify the
connection between these problems and differential heat equations [11].
Several months ago I.G. Petrovskii in Moscow found a general method of
reducing problems in prob ability theory on sums of small random variables to
corresponding differential equations in a very general situation. In what follows
this method is applied to proving Lyapunov's theorem (§1) and my theorem
mentioned above (§§2, 3). The applications to random walk problems can be
found in a yet unpublished work by Petrovskii, where the setting given by
Lüneburg is considerably generalized.
Apparently this method also can be used when the distribution law for each
term Xk+l depends on the sums of all previous terms Bk = Xl +X2+' . . +Xk. This
* 'Über die Grenzwertsätze der Wahrscheinlichkeitsrechnung', Izv. Akad. Nauk

SSSR, OMEN, 1933, 366-372. Presented by S.N. Bernshtein.
147
148 ON THE LIMIT THEOREMS OF PROBABlLITY THEORY
case is crucial for a rigorous mathematical justification of diffusion theory, in

which the external forces as weIl as the diffusion coefficients are point functions.
For this justification it was necessary to prove that the distribution laws of sums
can be approximately represented by the corresponding solutions of Fokker-
Planck differential equations [12], [13].
§1. Let
be independent random variables with expectations
and suppose that the quotients d" : b" are uniformly bounded by a fixed con-
stant
(1)
By the weIl known inequality for moments, b~ :5 dZ, (1) implies that
b,,:5J.'2. (2)
We now set
Z1 + Z2 + ... + Z" = S", Sn = 5,
b1 + b2 + ... + b" = t", t n = T.
Let T be fixed (and n variable); then Lyapunov's theorem asserts that
P{a < 5 < b} = .~

2v1fT
1 a
6
e-· 2 / 4T ds + (JR(J.'), 1(J1:5 1, (3)
where P denotes the prob ability ofthe inequality in parentheses and R(J.') tends
to 0 together with 1'.
To prove this, consider the probability
P,,(s) = P{a < 5 - s" + s < b}
so that a < 5 < b under the assumption s" = s. Clearly, the desired probability
P{a < 5< b} coincides with Po(O); P,,(s) satisfies the equation 1
P,,(s) = J
PH1(S+z)dTH1(Z), k=O,1,2, ... ,n-1, (4)
1 Formula (4) is proved in the same way as that for the distribution function of
the sum of two independent variables; see, for example, P. Levy, Calcul des
probabilites, Paris, 1927, p.187.
ON THE LIMIT THEOREMS OF PROBABILITY THEORY 149
where TH1 is the distribution function of Xk+l, and the integral is taken from
-00 to +00. Moreover,
Pn(s) = 1, if a < s < b,

(5)
Pn(s) =0 if s ~ a or s ~ b.
The initial conditions (5) and equations (4) uniquely define Pk(S) for all k
(0 ~ k ~n).
We call Pk(s) an upper function if the inequalities
Pk(s) ~j Pk+l(s + X)dn+l(X), (6)

P~(s) ~ Pn(s) (7)
hold. It is not difficult to see that
(8)
holds also for k < n. Lower functions are defined similarly. We now wish
to construct a function u*(s, t) which for any sufficiently small J-! leads to the
upper function
(9)
For this we first determine a four times continuously differentiable function
v(s) satisfying the following inequalities:
v(s) =0, s~a-f, O~v(s)~l, b~s~b+f,
o ~ v(s) ~ 1, a- f ~ S ~ a, v(s) = 0, b+ f ~ s,
v(s)=l, a<s<b. (10)
Now let
ü(s,t) = 1 jv(s+x)e-x2/4(T-t)dX, (11)

2J1f(T - t)
u*(s, t) = ü(s, t) + f(T - t), (12)
where f > 0 in both (10) and (12). The function u(s, t) satisfies (for t ~ T) the
differential equation
(13)
150 ON THE LIMIT THEOREMS OF PROBABILITY THEORY
and its derivatives up to the fourth order with respect to s and the second
order with respect to t are bounded. Let M be the least upper bound of these
derivatives. If P:(s) is defined by (9), then we obtain
P:(s) = u·(s, T) = ü(s, T) = v(s) ~ Pn(s).

Thus, (7) is fulfilled. Therefore it suffices to prove (6) for small p. For this we
consider the difference
~ = Pk(s) - j P:+1(s + z)dT1:+1(Z) = {(T - tk) + ü(s, tk)-
-{(T - t1:+t) - j ü(s + z, t1:+1)dTk+l(Z) = {(tk+1 - tk)+
+ü(s, tk) - j{ ü(s, tk+1) + 8s8 ü(s, t1:+dz + 8s2ü(s, t1:+1)2+

~ ~
+OMlz31}dT1:+1(z) = (b1:+1 + Ü(S,tk) - ü(s,t1:+d-
-::2 ü(s, t1:+ l)b1:+1+ 0' M d1:+1, 101 ~ 1, 10'1 ~ 1. (14)
But since
ü(s, tk) - ü(s, t1:+d = - 8t8 ü(s, t1:+d(t1:+1 - tk) + 0" M(t 1:+ 1 - tk? =
=- ! ü(s, t1:+1)b1:+1 + 0'" Mb1:+1p2, 10"1 ~ 1, 10"'1 ~ 1,

and in view of (1) and (13), (14) implies that
::2
~ = {b1:+1 + {- :t ü(s, t1:+1) = ü(s, t1:+d }b1:+1 + Oiv Mbk+11'+
+0'" Mb k +11'2 = b1:+1 ({ + Oiv M p + 0'" M 1'2), 10iv I ~ 1. (15)
Formula (15) immediately implies that if I' is sufficiently small, then
~~o,
which proves (6). Therefore P: (s) is an upper function for a sufficiently small
I' and
P{a < Sn< b} = Po(o) ~ P;(o) = u*(O,O) =

={T+_1_jv(z)e-:c2/4Tdz=_1_1b e-:c 2/ 4T dz+R, (16)
2M 2M Q
with R arbitrarily small for an appropriate choice of f. Consideration of lower

functions gives the reverse inequality. Thus our main formula (3) is completely
proved.
§2. Now let a(t) and b(t) be four times continuously differentiable functions
of t satisfying the conditions
a(t) < b(t), a(O) < 0 < b(O). (17)
What is the probability P that
a(tk) < 8k < b(tk), k = 1,2,3, ... ,n (18)
holds ? The inequalities
o ~ t < T, a(t) < 8 < b(t) (19)
determine a certain region G in the (8, tl-plane. Let U(8, t) be a bounded

solution of the equation
(13)
in G with boundary conditions
u(8,T) = 1, a(T) < 8< b(T),

u[a(t), t] = 0, o ~ t < T, (20)
u[b(t) , t] = 0, o ~ t < T.
Our purpose is to show that
P = u(O, 0) + (}R(p), I(}I ~ 1, (21)
where R(p) tends to zero together with p.

For this consider the more general case in which the inequalities
(22)
hold for all i, k ~ i ~ n. The desired probability is Po(O). If
(23)
then Pk(S) satisfies the same equation
(4)
as in §l. Conditions (5) are now replaced by the following:
Pn(s) = 1 if a(T) < s < b(T),

(24)
Pn(s) = 0 if s ~ a(Tk) or s 2: b(Tk).
Equations (4) and (24) uniquely determine Pk(s) for all k (0 ~ k ~ n).
We now call P:(s) an upper function if (23) implies
(25)
and
P~(s) 2: 1 for a(T) < s < b(T),
(26)
P~(s) 2: 0 for s ~ a(Tk) and, correspondingly, s 2: b(Tk).
As in §1, we prove that for k < n the inequality
holds.
To construct an upper function we need a function u(s,t) which is defined
in G as a solution of the differential equation (13) with the following boundary
conditions:
ü(s, T) = 1, a(T) < s < b(T),
ü{a(t), t} = v(t), 0 ~ t ~ T, (27)
ü{b(t), t} = v(t), 0~t ~ T,
where v(t) is a four times continuously differentiable function such that
v(t) = 0, 0~t ~T - f,
o ~ v(t) ~ 1, T- f ~ t ~ T, (28)
v(t) = 1, v'(T) = v"(T) = O.
Since ü(s, t) has continuous derivatives up to the fourth order (including on the
boundary of G), this function can be extended beyond G so that the derivatives
with respect to s up to the fourth order are bounded on the whole (s, t)-plane
and the inequality
ü(s,t) > -f (29)
holds everywhere. We now set
u*(S, t) = u(s,t) + f + f(T - t), (30)

P;(s) = u*(s, tk). (9)
Apart from obvious changes, the proof of the fact that P;(s) is an upper
function for any sufficiently small fJ is the same as in §1. Thus for the indicated
fJ we obtain
P = Po(O) ~ P;(O) = u*(O, 0)
and u* (0,0) -+ u(O, 0) as f -+ O. To complete the proof of (21) we merely have

to make similar estimates of the probability P from below, which is done using
the same method with lower functions.
§3. Now let p(1) (respectively, p(2») denote the prob ability of the existence
of k such that
a(ti) < Si < b(tä), i = 1,2, ... , k - 1,

(31)
Sk ~ a(tk),
and
a(ti) < Si < b(ti), i = 1,2, ... , k - 1,

(32)
b(tk) ~ Sk,
respectively. Finally, let P(x, y) be the probability that all the inequalities
a(tk) < Sk < b(tk), k = 1,2,3, ... , n - 1,

(33)
x< sn< y,
hold with a(T) ~ X < Y ~ b(T).

Then the following limit relations hold:
p(1) = u(I)(O,O) + 0(1) R(I)(fJ), 10(1)1 ~ 1, (34)
p(2) = u(2)(0, 0) + 0(2) R(2)(JI), 10(2)1 ~ 1, (35)
P(x, y) = Uz,y(O, 0) + OR(fJ), 101 ~ 1, (36)

where R(1)(Jl), R(2)(Jl) and R(Jl) tend to 0 together with Jl, and where
U(l)(S " t) U(2)(S , t) and UX,1I (S , t)
are bounded solutions of (13) determined from the following boundary condi-
tions:
u(l)(s,T) = 0, a < s< b,
u(1){a(t),t} = 1, 0 ~ t < T, (37)
u(1){b(t),t} = 0, 0 ~ t < T,
u(2)(s,T) = 0, a < s < b,

u(2){a,(t),t} = 0, 0~t < T, (38)
u(2){b(t), t} = 1, 0 ~ t < T,
Ux,y(s,T) = 0, a< s < x, ux,y{a(t),t} = 0, 0 ~ t < T,
Ux,y(s, T) = 1, x< s < y, ux,y{b(t), t} = 0, 0 ~ t < T. (39)
Ux,y(s,T)=O, y<s<b,
The proof is the same as in §2. In [9] other expressions 2 are derived for the
same values u(l)(O, 0), u(2)(0,0) and UX,y(O, 0).
References
1. P. Laplace, Theorie analytique des probabiliUs, 3rd ed. Paris, 1886.

2. P.L. Tchebycheff, Sur deux theoremes relatifs aux probabiliUs, Oeuvres de
Tchebycheff, St. Petersburg, 1907, vol. 2, pp. 481-491.
3. A.A. Markov, 'The law oflarge numbers and the method ofleast squares',
Izv. Fiz.-Mat. Obshch. Kazan. Univ. 8 (1898) (in Russian).
2 In [9] and [14] multiple differentiability of a(t) and b(t) is not assumed. If a(t)
and b(t) are continuously differentiable only once, then in the proof one must
approximate them by four times continuously differentiable functions. The proof
of the fact that the remainders in (3), (21), (34)-(36) tend to zero uniformly with
respect to T when T is greater than some fixed constant is simple if the random
variable is multiplied by JTo/T and in this way, the case of arbitrary T reduces
to the case T = To. A simple argument then shows that in (34) and (35) the
remainders tend to zero together with IL even without assuming that T> To.
4. A.M. Liapounoff, 'Sur une proposition de la theorie des probabilites', Bull.

Acad. Sei. St.-Petersburg 13 (1900), 359; 'Nouvelle forme du theoreme sur
la limite de probabilite', Mem. Acad. Sei., St. Petersburg 12:5 (1901).
5. A.A. Markov, 'Generalizing limit theorems in probability calculus to sums
of variables related by a chain', Zap. Akad. Nauk 22 (1910) (in Russian).
6. S.N. Bernshtein, 'Generalizing a limit theorem in prob ability theory to
sums of independent variables', Uspekhi Mat. Nauk 10 (1944), 65-114 (in
Russian); Math. Ann. 97 (1926), 1-59 (in German).
7. S.-D. Poisson, Recherches sur la probabilite, Paris, 1837.
8. P. Levy, 'Nuove formule relative al giuoco di testa e croce', Giorn. Ist. Ital.
Attuar. 2 (1931), 3.
9. A.N. Kolmogoroff, 'Eine Verallgemeinerung des Laplace-Liapounoffschen
Satzes', lzv. Akad. Nauk SSSR, OMEN (1931), 959-962 (Paper 12 of this
volume).
10. R. Lüneburg, 'Das Problem der Irrfahrt', Math. Ann. 104 (1931), 700.
11. R. Courant, K. Friedrichs and H. Lewy, 'Über die partiellen Differenzen-
gleichungen', Math. Ann. 100 (1928), 32-74.
12. A. Fokker, Ann. Phys. 43 (1914), 812.
13. M. Planck, Sitzungsber. Preuss. Akad. Wiss. (1917).
14. A.N. Kolmogoroff, Math. Ann. 104 (1931), 415; in particular, §§12-14
(Paper 9 of this volume).
17. ON THE THEORY OF CONTINUOUS RANDOM PROCESSES*
Let <5 be a physical system with n degrees of freedom; this means that the
admissible states :c of <5 form a Riemannian manifold mof dimension n. The
process of variation of <5 is said to be stochastically determined if under an
arbitrary choice of :c, the region ~ (in m) and times t' and t" (t' < t"), the
prob ability P(t',:c, t", ~) that the system in state :c at time t' will be in one of
the states of ~ at time t" is defined. It is further assumed that the prob ability
P(t',:c,t",~) can be given by the formula
P(t',:c, t", ~) = l J(t',:c, t", y)dVII , (1)
where dVy denotes the volume element. Here J(t',:c,t",y) has to satisfy the
following fundamental equations:
L J(t',:c,t",y)dVy = 1, (2)
J(tl,:c,t3,y) =L !(tlJ:c,t2,Z)!(t2,z,t3,y)dVz, tl < t2 < t3. (3)
The integral equation (3) was studied by Smolukhovskii and then by other
authors. 1 In the paper 'Über die analytischen Methoden in der Wahrschein-
lichkeitsrechnung' 2 I have proved that, under certain additional conditions,
!(t',:c, t", y) satisfies certain differential equations of parabolic type. 3 But in
A.M. there was no answer to the question 4 as to what extent !(t',:c,t",y) is
uniquely determined by the coefficients A(t,:c) and B(t,:c). In this paper the
theory is developed in the general case of a Riemannian manifold m and the
question of uniqueness is answered affirmatively for a closed manifold m.
§1. The first differential equation

Let m be a Riemannian manifold of dimension n. Because of the assumptions
made, !(t',:c, t", y) is defined for t' < t" and any pair of points :c, y. Moreover,
* 'Zur Theorie der stetigen zufälligen Prozesse', Math. Ann. 108 (1933), 149-160.
1 See bibliography in: B. Hostinsky, 'Methodes generales ducalcul des proba-
bilites', Mem. Sei. Math. 52 (1931).
2 Math. Ann. 104 (1931), 415-458. Referred to in the present paper as A.M. (see
No. 9 of this book).
3 These differential equations were introduced by Fokker and Planck independently
of Smolukhovskii's integral equation. See: A. Fokker, Ann. Phys. 43 (1914), 812;
M. Planck, Sitzungsber. Preuss. Acad. Wiss. (1917) 10 May.
4 See A.M. §15.
156
ON THE THEORY OF CONTINUOUS RANDOM PROCESSES 157
we assume that f(t',:c, t", y) has continuous derivatives up to the third order
with respect to all the arguments (t', t" and the coordinates :Cl, :C2, ... , :Cn ,
Yl, Y2, ... ,Yn of the points :c and y) and satisfies the continuity condition
Im f(t,:c, t + Ll, z)p3(:c, z)dVz

=>f;:.:..-,,..;-----:-~---=-2';---':--:::-:- -+ 0 as Ll -+ 0, (4)
Jm f(t,:c, t + Ll, z)p (:c, z)dVz
where p(:c, z) denotes the geodesic distance 5 between :c and z.

We choose a coordinate system z = (Zl, ... , Zn) in a neighbourhood ~ of
:c. Then we set
l f(s,:c,s + Ll, Z)(Zi - :Ci)dV" = ai(s,:c,Ll), (5)
l f(s,:c, s + Ll, Z)(Zi - :Ci)(Zj -:Cj )dV" = bij(S,:c, Ll), (6)
1 f(s,:c, s + Ll, z)p2(:c, z)dVz = ß(s,:c, Ll), (7)
1 f(s,:c, s + Ll, z)p3(:c, z)dVz = v(s,:c, Ll). (8)
Our purpose is to prove that the ratios
tend to limits Ai(S,:c) and Bij(S,:c) as Ll -+ 0, independently of~. Below this

is proved under the assumption that all N = n + n(n + 1)/2 functions
8
-8 f(s,:c,t,y),
:Ci
of y and t (for fixed sand:c) are linearly independent, that is, that t l , Yl, t2, Y2, . ..
. . . ,tl:, YI:, ... ,tN, YN can be chosen so that the determinant
(9)
is non-zero. 6
5 See A.M., §13, formula (112).
6 See A.M., §13, determinant (119).
158 ON THE THEORY OF CONTINUOUS RANDOM PROCESSES
In 2l we have
while outside 2l we clearly have
2( ) 0' 3( )
PX,z=-px,Z, 10'1 ~ C',
where C' and C are constants independent of z. Hence
ß(s,X,a) = Lf(s,x,s+a,z)p2(x,Z)dVz =
= L:9;j l fes, x, s + a, z)(Z; - x;)(Zj - Xj )dVz +
+ lf(s,x,s+a,z)0 p3 (x,z)dVz +
+ [ f(s,x,s+a,z)0'l(x,z)dVz =
Jm-~
L: 9;jb;j(s, x, a) + 0"v(s, x, a), 10"1 ~ C". (10)
But since, by the continuity condition (4),
ß(s,X,a)
---'----'-
v(s, x, a)
- +00 as a- 0, (11)
formula (10) implies that
(12)
Now, for fixed x, y, S, T, t, s < T < t, we consider only a so small that

s +a < T. Then f( s + a, z, t, y) and its derivatives with respect to Z up to
the fourth order are uniformly bounded and continuous in 2l (we assurne that
2l is compact). Hence, for every point Z in 2l we have
{)
fes + a, z, t, y) - fes + a, x, t, y) = L:(z; - Xi) {)x/(s + a, X, t, y)+
+~ L:(Zi - x;)(Zj - Xj) {)X~;Xj fes + a, X, t, y) + 0 p3(x, z), 101 ~ C, (13)

where C does not depend on ß or z. On the other hand, the fundamental

equation (3) implies that
f(s,x,t,y) = Lf(s,x,S+ß,Z)f(s+ß,z,t,y)dVz =
= L f(s, x, s + ß, z)f(s + ß, x, t, y)dVz+
+ l f(s, x, s + ß, z){f(s + ß, z, t, y) - f(s + ß, x, t, y)}dVz+
+ f f(s,x,s+ß,z){f(s+ß,z,t,y)-f(s+ß,x,t,y)}dVz =
JfJt-1l
(14)
By (2),
h = Lf(s,x,s+ß,z)f(S+ß,x,t,y)dVz =
= f(s + ß, x, t, y) L f(s, x, s + ß, z)dVz = f(s + ß, x, t, y). (15)
Then (13), (5) and (6) imply that
12 = l f(s, x, s + ß, z){f(s + ß, z, t, y) - f(s + ß, x, t, y)}dVz =
= l f(s, x, s + ß, Z){~(Zi - Xi) 8~i f(s + ß, x, t, y)+
82
+~ ""(Zi - xi)(zi - xi) 8 8 f(s
L...J Xi xi
+ ß, x, t, y)+
+0 p3(x, z) }dVz = ~ ai(s, x, ß) 8~/(s + ß, x, t, y)+

82
+~ ""bii (S,X,ß)8 8 f(s+ß,x,t,y)+
L...J Xi xi
+ l f(s, x, s + ß, Z)0 p3(X, z)dVz. (16)
Finally, since throughout 9t - ~ we have
l(x,z) > K > 0,
where K does not depend on z, in 9t - ~ we can set
f(s + ß, z, t, y) - f(s + ß, x, t, y) = 0'l(x, z).

Then
I3 = [ f(s,x,s+Ll,zHf(s+Ll,z,t,y)-f(s+Ll,x,t,y)}dVz=
1m- fA
= [ f(s,x,s+Ll,z)e'l(x,z)dVz , 1.
le'I:::;C'=K (17)
1m-fA
Substituting (15)-(17) into (14) we finally obtain
Ö
f(s, x, t, y) = f(s + Ll, x, t, y) + L ai(s, x, Ll) ÖXi f(s + Ll, x, t, y)+
ö2
+! L bij(s, X, Ll) ÖXiÖXj f(s + Ll, x, t, y)+
+ lf(s,x,s+Ll,z)e llp3 (x,z)d Vz , le"l:::;c". (18)
If we also take into account the obvious equality
1 f(s, x, s + Ll, z)e"l(x, z)dVz = e/ll 1m f(s, x, s + Ll, z)p3(x, z)dVz =

= elllv(s, x, Ll), lellll:::; CIII ,
then (18) can be rewritten as folIows:
f(s + Ll, x, t, y) - f(s, x, t, y) _ '"' ai(s, x, Ll) Ö f( A )
Ll - - L.J Ll ÖXi s + u.,x,t,y -
_L bij ( s,
x, Ll) ö2
A !I !I
_ f( A
s+u.,x,t,y
) _ 0111
\7
v( S , x, Ll)
A • (19)
2u. UXiUXj u.
The left-hand side in (19) tends to :.f(s,x,t,y) as Ll-+ O.
Suppose that the deterrninant DN(s,x) is non-zero for tl,yl,t2,Y2, ...
... ,tN,YN. Then DN(s + Ll,x) =F 0 for sufliciently srnall Ll. Hence, there
(20)
If we rnultiply (19) by Ak(Ll) with t = tk and y = Yk and surn all the N

equalities thus obtained, then we have
LAk(Ll)f(s + Ll,x,tk,Yk) - f(S,x,tk,Yk) =
k Ll
__ '"' . ai(s, x, Ll) _ '"' .. bij(S, x, Ll) _ '"' \ (A)O/ll v(s, x, Ll) (21)
- L.J a, Ll L.J a'J 2Ll L.J Ak u. \7 k Ll .
i i,j k
If tends to zero, then the

~ Ak(~), as solutions of (20), tend to the solution
Ak(O) of the equations
(22)
Hence, the left-hand side of (21) has a finite limit
= Lk
f)
Ao Ak(O)Jl /(s, z, tk, Yk) (23)
vS
as ~ -0.
In particular, if we set eri = 0, erij - gij, then
Egijbij(S,Z,~) ' " A (~)elllv(s,z,~) _ A as ~ _ O. (24)

2~ +~ k k ~ 0
By (12), the second term in (24) is infinitesimally small as compared with the
first one (since the Ak(~) are hounded). Hence we have
(25)
But (25) and (12) imply
v(S,z,~)/~ - 0 as ~ - O. (26)
and erij in (21) to zero,

If we now equate all hut one of the coefficients eri
then a similar passage to the limit using (26) shows that all the limits
· ai(s,z,~)
Ai ( s, z ) = 11m A 0 (27)
~ as ~ - ,
B I).. ( s, z ) -li bij(S, z, ~) A 0 (28)

- m 2~ as ~ - ,
exist and do not depend on the choice 7 of~. Then (27), (28), (26) and (19)
immediately imply the first differential equation
7 See A.M., §13, formulas (122)-(124).

Certainly the condition that DN(8, X) does not vanish identically can be
replaced by the direct requirement that the limits (27) and (28) exist, since
(28) implies the existence of a finite limit (25) and therefore of (26).
At certain exceptional points the limits (27) and (28) need not exist. This
was illustrated in A.M. 8 by the following example: !R is the ordinary number
axis and
3 y2 [ (y3 _ x3)2]
/(8, X, t, y) = 2J7r(t _ 8) exp - 4(t _ 8) ; (30)
for X = 0 we easily obtain
b(8,X,/l.)j2/l. -+ +00 as /l. -+ O.
Hence there is no finite limit B(8, x).
§2. The second differential equation
Assume now that in a neighbourhood ~ of the point Yo for a given t the limits
A;(t, y) and B;j(t, y) exist uniformly and that v(t, y, /l.)j /l. tends uniformly to
oin ~. Suppose further that R(y) is a non-negative function vanishing outside
~ with bounded derivatives up to the third order. Then for y E ~, Z E ~ we
have
82
+~ ""(Yi - Zi)(Yj - Zj ) -88 R(z)+
L...J Zi Zj
+e' l(y, z), le'l ~ C', (31)
whereas for y E !R - ~ and Z E ~,
R(y) = R(z) + eil l(y, Z), le"l ~ C". (32)
Finally, for y E !R -~, Z E !R - ~
R(y) = O. (33)
8 See A.M., §13, formula (126).

If in the corresponding regions R(y) is replaced by (31)-(33), we obtain
l R(y):/(s,x,t,y)dVy =
= :t l R(y)/(s,x,t,y)dVy = ! 1m R(y)/(s,x,t,y)dVy =
= lim ! 1m R(y)[/(s, x, t + ß, y) - /(s, x, t, y)]dVy =
= lim ! {1m R(y) 1m /(s, x, t, z)/(t, z, t + ß, y)dVzdVy-

- 1m R(y)/(s, x,t, Y)dVy} =
= lim! {1m /(s,x,t,z) 1m R(y)/(t,z,t + ß,y)dVydVz-
- 1m R(z)/(s, x, t, z)dV z} =
= lim ! { l /(s, x, t, z) 1m R(z)/(t, z, t + ß, y)dVydV + z
+l /(s, x, t, z) l ["I)Yi - Zi) O~i R(z)+
2
0
+~ L(Yi - z;)(Yj - Zj) OZ;OZj R(z) ] /(t, z, t + ß, y)dVydVz +
+ l/(s,x,t,z) 1m 8'''p3(y,z)f(t,z,t+ß,y)dVydVz-
- l R(z)/(s,x,t,z)dVz } = lim! { l /(s,x,t,z)R(z)dVz +
+ l/(S,x,t,z)[Lai(t,z,ß)o~;R(Z)+
2
0
+~ L b;j(t, z, ß) OZ;OZj R(z) ] dVz +
+8 l /(s, x, t, z)v(t, z, ß)dVz - l /(s, x, t, z)R(z)dVz } =

Replacing z by Y in the right-hand side of the equation we obtain
Now assume that Ai(t, z) and Bij(t, z) are twice continuously differentiable
in ~. Then we set
Q(t, y) = Igij (t, y)1
and after integration by parts, we obtain
f f(s, z, t, y)Ai(t, y) (){) R(y)dv;, =

1m Yi
=f f(s,z,t, y)Ai(t, y)Q(t, y){){) R(y)dy1 dY2 .. . dYn =

~ ~
=- f (){). [I(s, z, t, y)Ai(t, y)Q(t, y)]R(y)dy1 dY2 . .. dYn. (35)
~ ~ .
Double integration by parts (since all the derivatives vanish on the boundary
of~) yields
lm
f(s,z,t,y)Bij(t,y)-{)
Yi Yj
()2
() R(y)dV" =
l
= -()
()2
() [I(s, z, t, y)Bij(t, y)Q(t, y)]R(y)dy1dY2 ... dYn.
m Yi Yj
(36)
Formulas (34)-(36) immediately imply that
k R(y)Q(t,y)!f(s,z,t,y)dY1 dY2 ... dYn =
= k R(Y){ - L ()~i [Ai(t, y)Q(t, y)f(s, z, t, y)]+

+ L {)2 }
()Yi{)Yj [Bij(t, y)Q(t, y)f(s, z, t, y)] dy1 dY2 ... dYn.
Since R(y) is arbitrary, apart from the above conditions, it is easy to conclude
that at interior points of ~ the second differential equation
8
Q(t, y) 8t fes, x, t, y) = - L: 8
8Yi [Ai(t, y)Q(t, y)f(s, x, t, y)]+
82
+ "" ~[Bij(t, y)Q(t, y)f(s, x, t, y)]
L..J uy;uYj
(37)
also holds.
If at time to the differential function of the probability distribution is given,
that is, a non-negative function g(t o, y) of y satisfying the condition
1m g(to, y)dVy = 1, (38)
then for arbitrary t > t o the distribution function g(t, y) is given by the formula
g(t,y) = 1m g(to,x)f(to,x,t,y)dV.,. (39)
The function g(t, y) satisfies the equation 9
(40)
§3. Uniqueness
Under a change of the coordinate system the coefficients Ai (s, x) and Bij (s, x)
are transformed in the following way:
(41)
(42)
Here we always have
· bii( s, x, .6.)
Bu= 1Im 2.6. = 1·Im 21.6.}f1.f f( s,x,s+u.,z
A)( Zi-Xi )2dVz2:0. (43)
Hence the quadratic form

(44)
9 See A.M., §18, formulas (169) and (170).

is non-negative. This is crucial in the proof of the following theorem. 10
Uniqueness Theorem 1. If~ is closed, then (40) has at most one solution
g(t, y) with given continuous initial condition g(t o, y) = g(y).
Proof. Clearly it suffices to consider the initial condition g(to, y) = 0 and prove
that g(t, y) = 0 also for t > to. We can transform (40) into the form
(45)
Now set
v(t, y) = g(t, y)e-et.
The function v(t, y) satisfies the equation
{)v
-() = 2: B ij -{)2(()v) + 2: Si -()
()v
+ Tv - cv. (46)
t Yi Yj Yi
For fixed t o and t 1 the constant c can be chosen so large that
T(t,y) - c < 0
for all y and t, to ::; t ::; t1. Under these conditions v(t, y) cannot have a
positive maximum at any point (t,y), t o < t < t 1, since at such a maximum
~-o ,
{)Yi -
(T - c)v < 0,
which contradicts (46). Neither can there be a negative minimum of v(t, y)

within these limits. Since v(to, y) = 0 at t = to, we obtain for to < t < t1,
v(t,y) < maxv(t 1,y) = e-et, maxg(t 1,y)
g(t,y) < e-e(t,-t)maxg(t 1,y).
Since c was arbitrarily large, it follows that
g(t, y) = o.
10 See: E. Rothe, 'Über die Wärmeleitungsgleichung', Math. Ann. 104 (1931),
353-354 (uniqueness proof).
Uniqueness Theorem 2. Let 9t be closed. Then there is at most one non-

negative continuous solution /(s, x, t, y) /or (2) and (3) that satisjies (29) with
given twice continuously differentiable coefficients Ai(t, y) and Bij(t, y), and
the continuity condition (4).
The continuity condition (4) can be replaced by the /ollowing, weaker one:
L /(s, x, t, y)p2(x, y)dVy -+ 0 aB t -+ s. (47)
Proo/. Assume that two different functions 11 (s, x, t, y)

and h(s, x, t, y) satisfy
all our conditions. Then we can choose s and a continuous function g( x) such
that
gl(t, y) = L g(x)l1(s, x, t, y)dV:e,
g2(t,y) = L9(X)h(s,x,t,y)dV:e
are also different. By (2) and (47), gl(t, y) and g2(t, y) tend to g(y) aB t -+ s.
Since the functions gl(t, y) and g2(t, y) satisfy (40), this contradicts Uniqueness
Theorem 1.
§4. An example
The following example, which is interesting also for applications, demonstrates

that the quadratic form (44) need not be positive definite: let 9t be the usual
Euclidean plane and let
A simple computation shows that
§5. The limit solution
Let 9t be closed and /(s, x, t, y) everywhere positive and dependent only on

the difference t - s:
/(s, x, t, y) = t/J(t - s, x, V). (49)
Then general ergodie theorems 11 imply the existence of the limit probability
distribution. In other words, for any distribution ge', y) determined by (38)
and (39) and any region ~ the relation
l ge', y)dV!I -+ P(~) as t -+ +00, (50)
holds, where P(~) does not depend on g(to,y). It can easily be proved that
ge', y) is uniformly continuous for large t. From this we deduce that 12
P(~) = -l g(y)dV!I' (51)
ge', y) -+ g(y) as t -+ +00. (52)
Clearly, g(y) and P(~) do not depend on g(to,Y).

Now, let g(y) be the solution of the equations
8 82
- ' " ~[Ai(y)Q(y)g(y)] + '" ~[Bij(y)Q(y)g(y)]
L..J UYi L..J UYiUYj
= 0, (53)
1m g(y)dVy = 1. (53a)
= =
Setting g(to, y) g(y) it can easily be seen that ge', y) g(y) also for t > t o
(see (40) and Uniqueness Theorem 1). From this we deduce that the solution
01 (53) and (53a) (il it exists) is uniquely determined and coincides with the
limit function g(y).
As a particular case, (52) implies
I(s, %, t, y) -+ g(y) as t -+ +00. (54)
Klyazma, near Moscow, 12 April 1932
11 See A.M., §4, Theorem IV.

12 See footnote 1.
18. ON THE PROBLEM OF THE SUITABILITY OF FORECASTING
FORMULAS FOUND BY STATISTICAL METHODS *
§1. A number of authors (Exner, Baur, Vize) have made attempts, by means
of statistical analysis of the data gathered during the last 30-50 years, to ob-
tain formulas relating the deviation il.y of some meteorologieal factor y, to be
forecast, from its mean jj (averaged over many years) , with similar deviations
il.Zl, il.z 2 , ••• , il.ZA: of various faetors that ean be determined beforehand. So
far only linear formulas have been used for this purpose:
(1)
By ehoosing appropriate faetors Zl, Z2, ••• ,ZA:, whose number is somewhere
between three and seven, and eoeffieients a1, a2, ... ,aA: it ean be shown that in
many eases the eorrelation eoeffieient between the actuaUy observed il.y and
its value eomputed by formula (1), ealculated from the data during the same
30-50 years that served for deriving the formula, reaches 0.60-0.75. However,
when verifieation of the formula was based on the observations over years other
than the years used in deriving the formula (Vize), the eorrelation eoeffieient
between the eomputed and the observed il.y turned out to be 0.30-0.40. This
correlation eoefficient has no practical value, espeeially since forecasts of the
degree of reliability ean be obtained in a much simpler way.
The theory for obtaining regression equations of type (1) is based on the
foUowing assumptions. It is assumed that y, Zl, Z2, ••• , ZA: are random variables
with a eertain distribution law w(y, Zl, Z2, ••• , ZA:) whieh does not change over
the years. It is assumed further, that the probabilities that y, Zl, Z2, ••• , ZA: take
certain values in a given year do not depend on the values taken by these vari-
ables in previous years. Without the first condition (of stability) the problem of
formulating a regression equation has no meaning at aU. If the first eondition is
fulfiUed, then there exist determinate eoefficients a1, a2, ... ,aA: that minimize
the expectation
The relation
6 = E(il. y 2) - E(u 2 )
E(il. y 2)
* Zh. GeoJiz. 3 (1933), 78-82.
169
170 FORECASTING FORMULAS FOUND BY STATISTICAL METHODS
is called the trae correlation coefficient. The second condition (of indepen-
dence) is essential for proving that the coefficients a1, a2, ••• ,aA: of the empirical
regression equation (1) computed from the observations during a sufficiently
large number of years n are dose to the theoretical coefficients ab a2, ••• ,aA:,
and also for proving that the empirical correlation coefficient R is dose to the
theoretical one, 6, also for sufficiently large n.
There can certainly be some doubts as to whether these two conditions
can be applied to meteorological phenomena. Moreover, if we consider the
existence of secular or periodic multi-year dimate ßuctuations an established
fact, the first condition is dearly wrong. This fact is often used for explaining
the discrepancy mentioned above: it is supposed that the regression equa-
tion (1) does indeed reßect the regularities that took place during the period
under study with great accuracy, but these regularities themselves are subject
to change as a result of long-term climatic ßuctuations. I am going to prove
that these excessively high empirical correlation coefficients are quite explica-
ble also under the assumption of complete stability and independence of the
studied factors from year to year, in other words, that the methods used by
the above researchers inevitably lead to a certain "blow-up" of correlation co-
efficients. This is discussed in §2. In §3 we consider whether stability and
independence in meteorological series are sufficient to make statistical deter-
mination of regression equations possible and useful; in §4 we give some ideas
on the techniques for finding regression equations.
§2. The mathematical apparatus needed for solving our problem was devel-
oped quite recently by Fisher [1] in his study on the distribution law for the
correlation coefficient under multiple correlation (see also the review by Rider
[2]). It is assumed that the distribution law w(y, Xl, X2, • •• , XA:) is normal.
Suppose that we are given a regression equation for a certa,in variable y.
Assurne that y is related to Xl> X2, ••• , XA: in such a way that the true correlation
coefficient is 6. Fisher's analysis makes it possible to compute the distribution
law for the empirical correlation coefficient R from 6 and the number of ob-
servations n (number of years during which observation took place). However,
this distribution law by itself still gives no answer to our question. Indeed,
Xl, X2, ••• ,XA: can be chosen so that the correlation coefficient 6 is as large as
possible; then although at most 5-7 values are introduced into the regression
equation, the stock of values from which these 5-7 values can be chosen is very
FORECASTING FORMULAS FOUND BY STATISTICAL METHODS 171
large.
We assume therefore that there are i groups of values
(1) (1) (1)

Xl ,x2 , ••• ,x k ,
(2) (2) (2)
Xl 'X 2 , ... ,x k ,
(i) (i) (i)

Xl 'X 2 '''''X k ,
of k values each. Assume for simplicity that y is related to each of these groups
with true correlation coefficient b. For any A Fisher's formulas allow us to
compute the prob ability that in each individual case the empirical correlation
coefficient R exceeds A. Let this prob ability be p. It is natural to assume that
with probability
P=l_(l_p)i (2)
the inequality R > A holds for at least one of the groups: this is so under the
assumption of independence of the deviations of R from b corresponding to
different groups. 1 If, for example, i = 14 and p = 1/20, then
P = 1 - (1 - 1/20)14 ~ 1/2.
The values of A corresponding to p = 1/20 for given b, k and R can be computed

using Fisher's tables. For this we set
m=n-k-1, ß=v'ffitanh- 1 b,
then find B from ß and m via Fisher's tables 2 and, finally, find A by the formula
B = v'ffitanh- 1 A.
For example, set n = 42 and k = 5. Then, after appropriate calculations,

we find:
b 0.20 0.30 0040 0.50
A 0.56 0.61 0.66 0.72
1 This assumption is quite arbitrary, but in reality the number of groups from
which to choose might be even greater than i = 14 taken for the calculation in
the example; this, I hope, justifies the arbitrariness of our assumption.
2 Tables on p.665 in [1].
The immediate significance of these results is as folIows: by computing the

five-term regression formula from 42 observations for 0 = 0.20; 0.30; 0.40; 0.50
in one twentieth of the cases we find that R exceeds the corresponding .\.
If, however, from the 14 regression formulas we choose the one with the
largest R, then R exceeds the computed values of .\ with prob ability greater
than !.
In conclusion we discuss Fisher's general formulas. The probability that
R 2 lies between v and v + dv is, according to Fisher ,
xv(l:-2)/2(1_ v)(m-2)/2 1"

o
dt/J 100 . t/Jd
-00 (coshz-oRcost/J)m+A: dv =
l:-2
sm Z
= 1 [!(m+k l -2)]! (1_o2)(m+A:)/2 x

[2(m - 2)]![2(k - 2)]!
xF [!(m + k), ~(m + k), !k, 02v] v(l:-2)/2(1 - v)(m-2)/2dv,
where F(p, q, r, z) denotes the hypergeometric function. These relations can be

simplified in special cases and replaced by relations asymptotic in m, but even
after simplification they are still too complicated to be used directly.
§3. The established fact of "blowing up" of the correlation coefficient does
not necessarily mean that these statistical regression formulas cannot be used.
We must make the number k of values involved in the regression equation and
the stock of values from which they are chosen as low as possible; in this case
the risk of obtaining an artificially "blown-up" correlation coefficient can be
significantly diminished.
More essential for evaluating future prospects for the statistical estab-
lishment of forecasting equations is to find out to what extent meteorologi-
cal series satisfy the stability and independence conditions. The very exis-
tence of secular changes or periodic fluctuations does not in itself preclude
the use of studies stemming from assumptions on stability and independence,
if the role of these secular or periodic fluctuations in forming the deviations
ßy, ßZl, ßZ2,"" ßZl: is insignificant.
The simplest method for checking the stability of the series and the inde-
pendence of its terms is the following: let ßy( i) be the value of the deviation ßy
FORECASTING FORMULAS FOUND BV STATISTICAL METHODS 173
for the ith year. We need to compute, over a long period of n years, the correla-
tion coeflicients between Ay(i) and Ay(i+ 1), between Ay(i) and Ay(i+ 2), etc. If
these correlation coeflicients deviate from zero within the bounds correspond-
ing to theoretical computations made under the assumption of independence
and stability, this confirms our hypothesis. Indeed, secular change, or peri-
odic fluctuation with period of four years or more, inevitably lead to positive
correlation between Ay(i) and Ay(i+ 4 ), whereas short fluctuations during two
or three years lead to positive correlation between A.y(i) and A.y(i+2) or Ay(i)
and Ay(i+ 3). In the case of stability and independence the expectation of the
square of each of the above correlation coeflicients is approximately 1/(n - 3),
for n not too smalI.
Such computations were made for the average monthly temperatures in
Leningrad. For each month the correlation coeflicients between Ay(i) and
Ay(i+ 1 ), Ay(i+2) and Ay(i+3) were computed from the data over a hundred
years. The mean square of these three correlation coeflicients over twelve
months turned out to be 0.065, 0.064 and 0.110, which is in good agreement
with the theoretical values:
1/..,fn - 3 = 11m = 0.102.
Now consider two series y(i) and x(i). If y is related to x with positive corre-
lation, then .6.yAx has positive expectation. To study the stability of corre-
lation between y(i) and x(i) we must form the correlation coeflicients between
Ay(i) - .6.x(i) and .6.x(i+l) - Ay(i+l), .6.x(i+2) - Ay(i+ 2), etc. If the double
series is stable and the pairs y(i), x(i) relating to different years are indepen-
dent, then the square of this correlation coeflicient has expectation 1/(n - 3),
as in the first case. In this way the stability of the correlation between av-
erage temperatures of two successive months in Leningrad was studied (it is
known to be a significant positive correlation): the mean square of the cor-
relation coeflicient between Ay(i) - Ax(i) and Ay(i+l) - Ax(i+ 1) over twelve
pairs of adjacent months was 0.071, which is also in good agreement with the
theoretical value 0.102. Similar treatment can be given to the stability of the
variability of y by forming the correlation coeflicients between (Ay(i»2 and
(Ay(i+ 1»2, (Ay(i+2»2, etc.
Only such systematic studies of the stability of meteorological series can
solve the problem, whereas separate remarks to the effect that a certain corre-
lation coeflicient appeared to be unstable can only distort our ideas on the real
situation. If some separately chosen correlation coefficient between two meteo-

rological factors has very different values for two adjacent periods oftwenty-five
years, this by no means proves that the general regularities of the climate that
occurred during the first twenty-five years changed over the second twenty-
five year period. Rather , this discrepancy can be considered to be of a purely
random nature.
Clearly, a few figures are far from sufficient for the solution of our problem,
but it is quite possible that further studies will reveal fairly good agreement
with the stability and independence hypotheses. The point is that in order
to prove secular climate changes or multi-year periodic fiuctuations, highly
averaged values are usually studied, based on observations made by individual
stations, for which the absolute values of the deviations ll.y are comparatively
small and therefore the role of long-term fiuctuations is more significant.
§4. We now assurne that the requirements of stability and independence for
our series are fulfilled. If the number of observation years is large enough,
then under our assumption, even quite complex regression equations selected
on the basis of these observations, would refiect with sufficient accuracy the
regularities that are true for the entire series. However, for n = 30 or even 50 or
even 100 the situation changes considerably: if too many values are introduced,
from which we make our choice for entering in our equation, then the danger is
that there randomly appear nice combinations whose use for forecasting wöuld
be quite unjustified.
On the contrary, if serious theoretical observations show that the value y
to be forecast can to a considerable degree be determined, for example, by three
values x!, X2, X3, then the computation of the regression formula (1) which re-
lates ll.y to ll.Xl, ll.x2 and ll.x3, based on 50 years' observations, should be con-
sidered reliable. Here we mean theoretical considerations dynamically justified
and not ''theories'' of purely statistical origin. However, even purely statistical
techniques might give a hint of where to look for non-random correlation rela-
tions. Such an attempt on a broad scale was made by Baur [3]. He computed
correlation coefficients between average monthly temperatures in Iceland and
atmospheric pressure in previous months at various points of the Earth. It ap-
peared that for the stations on the Southern hemisphere the absolute values of
the correlation coefficients do not exceed on the average (over different stations
and twelve months) the values predicted theoretically under the assumption of
FORECASTING FORMULAS FOUND BY STATISTICAL METHODS 175
independence of both events (temperature in Iceland and pressure at a given

station in the Southern hemisphere). This makes these rarely occurring corre-
lation coefficients considerably exceeding the general norm quite doubtful. In
contrast to this, the pressure at the stations of the Northern hemisphere show
correlation with sub se quent temperatures in Iceland, which is systematically
much greater than it would have been at random. Baur remarks, however,
that perhaps the influence of pressure anomalies at distant points of the Earth
tells something about temperatures in Iceland only over times longer than one
month. Baur does not give sufficient proof of this hypothesis. The quest ion
can be solved only by systematically continuing Baur's studies on the average
value of correlation coefficients, not by looking for new special cases of high
correlation.
These studies should be guided by Fisher's result by virtue of which, for
independent x and y the empirical correlation coefficient r has a distribution
law which, after the transformation
z = tan- 1 r,
turns into the normal law for z with centre at the origin and variance
(1 = 1/..;n:::3,
where n is the number of observations.
References
1. R.A. Fisher , 'The general sampling distribution ofthe multiple correlation

coefficient', Proc. London Roy. Soc. A 121 N787 (1928), 654-673.
2. P.R. Rider, 'A survey of the theory of small samples', Ann. 0/ Math. 31:4
(1930), 577-628.
3. F. Baur, 'Statistische Untersuchungen über Auswirkungen und Beding-
ungen der grossen Störungen der allgemeinen atmosphärischen Zirkula-
tion', Ann. Hydrogr. Maritimen Meteor. 53:1 (1925), 1-6; 53:8, 241-258.
19. RANDOM MOTIONS *
On the theory of Brownian motion
In my two earlier papers [1], [2], I developed a general theory of continuous

random processes. Under very general conditions, it was proved that ifthe state
of a physical system at every given moment is fuHy determined by n parameters
Xl, X2, ••• , X n , and if these n parameters continuously 1 change with time t, then
the corresponding distribution functions satisfy the Fokker-Planck differential
equation. In the general case of such random processes the increments .6.x; of
the parameter Xi are of the same order as (.6.t) 1/2 . This implies that in the
general case .6.xi : .6.t -+ 00 as.6.t -+ 0, so that we cannot speak about adefinite
rate of variation of X;. We will now show how to apply this general theory to
random motions, for which we assurne that not only the system's coordinates,
but also their derivatives with respect to time vary continuously.
Let ql, q2, ... , qn be the coordinates of a system with n degrees of freedom.
Assurne that if we know q and 4 and the time t, then we can determine the
probability density
of q' and 4' for any t' > t. We assurne, moreover, that G does not depend on
the behaviour of the system before time t.
It is natural to assurne that 2
EI.6.qi - 4i.6.tl = o(.6.t), (1)

E(.6.q;)2 = o(.6.t), (2)
where .6.t = t' - t and Eis the expectation symbol. Relations (1) and (2) imply
E(.6.qi) = 4i.6.t + o(.6.t), (3)
E(.6.q;.6.qj) ~ VE(.6.qt}2E(.6.%)2 = o(.6.t) (4)
* 'Zufällige Bewegungen (Zur Theorie der Brownschen Bewegung)', Arm. 0/ Math.

35 (1934), 116-117.
1 On the precise meaning of the condition of continuity of random processes, see
[1], §13.
2 Since t:..qi should be of the same order as t:..t.
176
RANDOM MOTIONS 177
In [2], under very general assumptions it was proved that in addition to

(2)-(4) the following relations hold:
E(~qi) = fi(t, q, q)~t + o(~t), (5)
E(~qi)2 = kii(t, q, q)~t + o(~t), (6)
E(~qi~qj) = kij(t, q, q)~t + o(~t), (7)
where fand k are continuous functions. In many problems of physics the

assumptions (5)-(7) also allow direct verification. In particular, (2) and (6)
imply that
(8)
Under the assumptions (2)-(8), which are quite natural from the point of
view of physics, it is dear that G is a fundamental solution of the following
differential equation of Fokker-Planck type 3
{f, (' , .') }

{)g
{)t'
' " ., {) ' " {)
= - L.J qi {)q~ g - L.J {)q~ i t ,q ,q g +
(9)
Thus, for n = 1 we obtain

{f(' , .')} {)2 {k(' , .') }
{)g
{)t' = -q., {)q'
{)
g-
{)
{)q' t , q , q g + {)q'2 t ,q ,q g . (10)
If fand kare constants, then the fundamental solution of (10) can be repre-
sented by
2V3 {[4' - q - f(t' - t)F

g = 7rk2(t' - t)2 exp - 4k(t' - t)
3k-q- ti'ti(t'-t)f
- k(t' _ t)3 }. (11)
3 The proof can be found in [2].

178 RANDOM MOTIONS
Clearly ä4 is of the order (ät)1/2, that is, it behaves like äz in the case
of a general continuous process. However, for äq we obtain 4
(12)
It can be proved that the latter relation also holds in the case of the general
equation (9).
References
1. A.N. Kolrnogoroff, 'Über die analytischen Methoden in der Wahrschein-

lichkeitsrechnung', Math. Ann. 104:3 (1931), 415-458 (No. 9 in this vol-
urne).
2. A.N. Kolrnogoroff, 'Zur Theorie der stetigen zufälligen Prozesse', Math.
Ann. 108:1 (1933), 149-160 (No. 17 in this volurne).
4 This result means that for any constant K, however large, the probability of
IAq - qAtl ;::: K(At) is sm aller than any fixed f > 0 uniformly in At.
20. DEVIATIONS FROM HARDY'S FORMULAS
UNDER PARTIAL ISOLATION *
The profound mathematical studies by R. Fisher [1] and S. Wright [2] deal.
with the evolution of gene concentration in a population in which free cross-
ing dominates. The purpose of this paper is to give a method for obtaining
similar results for a population consisting of a large number of partial popu-
lations weakly connected with each other. Mathematical analysis was applied
to the following scheme: a population with a constant number of individuals
N consisting of s partial populations of n individuals each (N = sn) with free
crossing in each partial population and in which in every generation on the av-
erage k "wandering" individuals are isolated from every population; regardless
of their origin, wandering individuals randomly join any of the partial popula-
tions, where they take part in creating the next generation. This scheme was
indicated as a possible one by N.P. Dubinin and D.D. Romashov. A number of
other, not less interesting, schemes of restricted crossing do not yet succumb
to mathematical treatment.
§1. The case of no selection. Denote by p the concentration of a certain

gene in the large population and by p the concentration of the same gene in a
partial population.
In this section we assurne that the studied gene is not subject to selection.
The formulas below are asymptotic, which are true as np, njk 2 and s tend to
infinity. In practice, they are applicable if np, njk 2 and s are sufficiently large.
Denote by tl.p the increment of the concentration p in a certain partial
population over one generation. After Fisher and Wright, we assurne that the
expectations of tl.p and (tl.p 2) satisfy the formulas
k
A = E(tl.p) = -(p - p),
n
B = E(tl.p)2 = 2n
pq,
where q =1- p. Since s is large, the variation of the total concentration

p will proceed much more slowly than those of the partial concentrations p.
Therefore, p can temporarily be taken constant. The concentrations p in par-
tial populations deviate from p in either direction. After sufficiently long time
the fluctuations of p around p ·result in a certain stationary prob ability distri-
bution for the concentrations p. This stationary distribution u(p) satisfies the
* Dokl. Akad. Nauk SSSR 3 (1935), 129-132. Presented by S.N. Bernshtein.
179
180 DEVIATIONS FROM HARDY'S FORMULAS UNDER PARTIAL ISOLATION
differential equation
1 02 0
"2 ap2 (Bu) - ap (Au) = O.
Solving this equation we obtain
p4k p -1 q4kq-1
u(p) = B(4kp,4kq) . (1)
Wright obtained (1) using a different method. It can be proved that (1)
not only gives the prob ability distribution for the concentration p but, for a
sufficiently large number s of partial populations, the actually observed distri-
bution of partial populations over concentrations p will also be given by (1).
Under these assumptions Hardy's formula holds for every partial popula-
tion, that is, the concentrations of individuals of the types AA, Aa and aa are
equal to q2, 2pq and p2. The concentrations of individuals of the type AA, Aa
and aa in a large population can be computed by the formulas:
AA = 10 1 q2u(p)dp,
Aa = 2 10 1pqu(p)dp,
aa = 10 1 p2 u(p)dp.
The computations give
- 4k -2 1_
AA = 4k + 1 q + 4k + 1 q,
- 4k
Aa = 2 4k + 1 pq, (2)
_ 4k -2 1_
aa = 4k + 1p + 4k + 1p.
Formulas (2) give the solution to the problem raised in the title of the
paper.
Due to the absence of selection, the expectation E(Äp) of the increment of
the total gene concentration (in the large population) over one generation is 0,
while the intensity of random concentration fluctuations in the large population
is E(Äp)2. This latter value is computed by the formula
E(Äp)2 11
= -s 0
1
E(Äp)2 u(p)dp = -2
sn
11 0
1
pqu(p)dp.
DEVIATIONS FROM HARDY'S FORMULAS UNDER PARTIAL ISOLATION 181
Computations give
4k --
E(ilp? = 4k+ 1 ;;. (3)
In the case of free crossing we have, according to Fisher and Wright,
E(ilp)2 = pij/2N.
§2. The effect of selection. Formula (1), Hardy's formula for partial popu-
lations and its substitute (2) for a large population remain valid in the presence
of selection when the selection coefficient a is much less than l/n. We consider
only this case: accordingly, the formulas below are merely asymptotic formu-
las, true as na ---+ O. Of special interest is the case of a recessive gene. In this
case the mean increment of gene concentration due to selection is ap2q for any
partial population. Therefore, for the total concentration we have
or
4k
E(ilp) = a (4k + 2)(4k + l)pij(4k p + 1). (4)
Formula (4) can be used to confirm, for the above scheme, the general
statement on the existence of an optimum of partial isolation for the selec-
tion of recessive genes, which was suggested and quantitatively justified by
A.A. Malinovskii. It is easy to compute the k corresponding to the highest
rate of selection. For small p this optimal k is
ko = tV2 = 0.35.
Mathematical Institute of Moscow State University
18 June 1935
References
1. R.A. Fisher, The genetic theory 0/ natural se1ection, Oxford, 1930.

2. S. Wright, Genetics, London, 1931, pp. 97-157.
21. ON THE THEORY OF MARKOV CHAINS *
The eonsiderations given below, though simple, are, I believe, new and interest-
ing for eertain physieal applieations, espeeially in the analysis of reversibility of
statisticallaws of nature made by Sehrödinger 1 for one partieular ease. In what
foHows it is a matter of indifferenee whieh of the two following assumptions is
made: either the time variable t runs through aH real values, or only through
the integers. The classical understanding of Markov ehains eorresponds to the
seeond possibility.
1. The notion of a Markov ehain
Consider a physieal system whieh at any given time t ean be in one of the
states of a finite set Ei, E 2 , ... , EN. Assume that for any pair of states Ei
and Ej and every pair of moments t and s, t ~ s, the eonditional probability
Pij(t, s) that the state Ej takes plaee at time s under the assumption that at t
the system was in state Ei, is defined. A signifieant, but not always explieitly
stated assumption is that the eonditional prob ability Pij (t, s) is independent of
the prehistory of the system before t. This assumption holds in dedueing the
fundamental equation of the theory of Markov ehains,
Pik(S,t) = L~j(S,U)Pjk(U,t), s~ U ~ t, (1)

j
with the eonditions
(2)
L Pij(t, s) = 1, (3)
j
(4)
where Dij = 0 is the Kroneeker symbol.

2. Absolute probabilities
So far we have only eonsidered eondition transition probabilities Pij (t, s). There
arises the question whether it is possible, knowing the transition probabilities
* 'Zur Theorie der Markoffschen Ketten', Math. Ann. 112 (1936), 155-160.
1 Berliner Berichte (1931), 144.
182
ON THE THEORY OF MARKOV CHAINS 183
Pij(t, S), to give the probabilities of various states at each instant t? If we

assurne that the process of change for our system starts at a certain moment to,
then the various states E l , E 2 , •.• ,EN are associated at time to with arbitrary
probabilities Ql(tO), Q2(tO), ... , QN(tO), Qk(tO) ~ 0, Ek Qk(tO) = 1. For each
moment t > to we obtain the following expressions for the probabilities Qk(t)
of the states Ek (k = 1,2, ... ,N):
Qk(t) = L Qi(tO)Pik(to, t).

i
Then for to ~ t ~ s we automatically also obtain the formula
(5)
The introduction of absolute probabilities is not so trivial if we do not assurne

a certain beginning of the process, but instead try to find the absolute prob-
abilities Qk(t) at all times t, -00 < t < +00. But even here the following
assertion also holds:
For arbitmry tmnsition probabilities Pik(t, s) given for all t and s (t ~ s),
it is possible to determine in at least one way for all t, -00 < t < +00,
the absolute probabilities Qk(t) corresponding to these transition probabilities
Pik(t, s).
Purely analytically, the theorem means the following. For each choice of
Pik(t, s) corresponding to (1)-(4), the infinite system of equations (5) has at
least one solution satisfying the additional conditions
Qk(t) ~ 0, L Qk(t) = 1. (6)

k
To prove this theorem we first note that for each to, as we have already
seen, there exists at least one system Q~to)(t) determined for all t ~ to and
satisfying (5) and (6) for s ~ t ~ to. Using a diagonal process, we can extract
from the sequence
to = -1,-2,-3, ...
a subsequence
such that An -00, so that for each k and each integer t the quantities
-+
Q~~n)(t) defined at each fixed t for all sufficiently large n te nd to a certain
184 ON THE THEORY OF MARKOV CHAINS
limit Qk(t) as n -+ 00. From (6) it is easy to deduce that for all real non-
integer t there exist limit values Qi;(t). These limit values, as can be proved
by passing to the limit, satisfy (5) and (6). This proves our theorem.
Of special interest is the case when the absolute probabilities are uniquely
defined by the transition probabilities Pik(t, s). A necessary and sufficient
condition for such uniqueness is the following:
For arbitrary fixed k and s, ~k(t,S) tends to a certain limit Qi;(s), in-
dependent 01 i, as t -+ -00. 11 this condition holds, then it is precisely these
limits Qi;( s) that lorm the desired unique system 01 absolute probabilities.
Let us first prove sufficiency. Let Qk(t) be certain absolute probabilities
compatible with the transition probabilities Pik(t,S). Then
(5)
but since Pik(t, s) -+ Qi;(s) as t -+ -00, the right-hand side of (5) tends to
I: Qi(s)Q;(t) = Qk(s)
i
as t -+ -00, which implies that
Now suppose that our condition fails. Then we can choose i 1 and i 2 and
two sequences,
t~ > t~ > ... >t~ > ... -+ -00,
t~ > t~ > ... >t~ > ... -+ -00,
such that Pilk(t~, s) and Pi2k(t~, s) tend to limits Ql,(s) and QT:(s) respectively,
as n -+ +00, for arbitrary k and s; furthermore, Ql,(s) and QT:(s) are not
identically equal for all s. Then it is easy to prove that both Ql,(s) and Qr:(s)
can be taken as absolute probabilities, implying the necessity of the condition.
3. Inverse probabilities
We now assume that there is a certain fixed system of absolute probabili-

ties Qk(S) with all the Qk(S) positive. In this case, along with the transition
probabilities Pik (t, s) we can also define the inverse conditional probabilities.
Namely, we denote by IIik(t, s) the conditional prob ability of the state Ei at

time tunder the assumption that at a certain later time s, s ~ t, the state Ek
is observed. Clearly,
Qi(t)
IIik(t,s) = Qk(t) Pik(t,S). (7)
It is easy to derive the following formulas analogous to (1)-(5).
IIik(s,t) = EIIij(s,u)IIjk(u,t), s ~ u ~ t, (1*)

j
IIik(s, t) ~ 0, (2*)
EIIik(s,t) = 1, (3*)
k
IIik(t,t) = 6ik, (4*)
Qi(S) = E Qk(t)IIik(s, t), s ~ t. (5*)

k
Note that if we want to bypass a certain new principle of "independence of the
future", then (1*) should be derived from the above formulas, not directly, as
for (1). This latter principle can then be deduced from (1*).
4. Inversion of the laws of nature
We now assurne that the transition probabilities Pik(S,t) depend only on the
difference t - s:
Pik(S,t) = P;k(t - s).
As is known, in this case there exists at least one system of absolute probabil-
ities independent of time t:
For a fixed system of such probabilities Qk the inverse probabilities IIik(s,t)

depend also only on t - s:
The question is: under what conditions do we have the equality
(8)
186 ON THE THEORY OF MARKOV CHAINS
Since
it follows that (8) holds if and only if
(9)
It should be noted, however, that necessary and sufficient conditions for

(8) to hold can be expressed even without using Q", in terms of the transition
probabilities Pi,,(r) only. Indeed, von Mises 2 proved that all states can be
divided into dasses so that the following conditions hold: 1) Pi,,(r) can be non-
zero only when Ei and E" belong to the same dass; 2) for any pair Ei and E"
from the same dass, the ratio Qi : Q" is uniquely determined by the transition
probabilities Pi,,(r). This implies that if Ei and E" belong to the same dass,
then (9) is true regardless of the absolute probabilities Qj; if, however, Ei and
E" are taken from different dasses, then (9) trivially follows from the equations
Pi,,(r) = P"i(r) = O. We also see that (9) does not depend on the system Q".
A necessary and sufficient condition, independent of Q", for (8) to hold
may be formulated as folIows:
Relation (8) holds if and only if for arbitrary r, q, k1 , k 2 , ••• , k q ,
In the discrete case (when t runs only through the integers) it suffices to require
(10) only for r = 1. The proof is elementary, and we leave it to the reader.
In particular, if the transition probabilities are symmetrie:
(11)
then (10) holds. Hence, the symmetry condition (11) is sufficient for (8).
5. Conclusion
These almost trivial facts find many physical applications. Here we confine
ourselves to an example which is different from Schrödinger's original exam-
pIe. Suppose that a cirde is divided into a very large number M of identical
intervals. A large number L of mobile particles move along this cirde so that
2 R. von Mises, Wahrscheinlichkeitsrechnung, Berlin, 1931, §16.

eaeh partide, independently of others, passes at every step into the neighbour-
!.
ing interval, either to the right or the left, eaeh possibility with prob ability
There are M L = N various possible patterns of L particles on M intervals.
The absolute probabilities Qk eorresponding to these N possibilities are all
equal to each other and to 1/N. The transition probabilities Pik (r) are clearly
symmetrie, and therefore, the invertibility equation (8) holds. If we eonsider
the "maeroseopie" distribution of the particles along the eircle, then with prob-
ability very close to one, it will be uniform. If it is known (though a prior;
this is highly unlikely) that at a eertain time to a eonsiderable deviation from
this uniform distribution takes plaee, then with prob ability very dose to 1 we
ean assert that this non-uniformity for t > to will die out approximately in ac-
eordanee with the diffusion differential equation. Formula (8) now implies the
same probability of uniformity for t < to with the same differential equation,
but with the opposite sign of the time variable.
Mathematieal Institute of Moseow State University
20 May 1935
22. ON THE STATISTICAL THEORY OF METAL CRYSTALLIZATION *
This paper gives a rigorous solution to the problem of the rate of a erystalliza-
tion proeess under eertain sehematie, but still suffieiently general assumptions.
The study of the proeess of erystal growth after random formation of erys-
tallization eentres is of signifieant irnportanee for metallurgy. In this eonnection
it is fairly diffieult to take aeeount of eollisions between the erystal grains ap-
pearing around separate erystallization eentres. These eollisions disturb the
grain form by preventing the growth of erystals in eertain direetions. In papers
by F. Göler and G. Saehs [1], G. Tammann [2], B.V. Stark, I.L. Mirkin and
A.N. Romanovskii [3], and others only rough approximation formulas for the
growth of erystal matter are given. In this paper I give, under rather wide
assumptions, an exact formula for the prob ability pet) that a randomly taken
point P of the volume filled with a erystallized substanee is inside the erystal
body within the erystallization period. With a suffieient approximation rate it
ean be eonsidered that the amount of substanee erystallized over time t is also
pet). In eonclusion I determine the number of erystallization eentres formed
throughout the entire erystallization proeess.
I extend my gratitude to I.L. Mirkin who interested me in this problem
and kindly provided me with all the neeessary material.
§1. Mathematical setting of the problem
Let V be a eertain volume. Initially (t = 0) it is oeeupied by "the mother

phase". After time t a eertain part Vi(t) of the volume V is oeeupied by
erystallized substanee. The volume Vi(t) grows with time t as follows.
1. In the free part V - Vi new erystallization eentres appear. For any
volume V' < V - Vi the probability of forming one erystallization eentre during
the time between t and t + tJ..t is
a(t)V' tJ..t + o(tJ..t),
and that of more than one eentre is o(tJ..t) , where o(tJ..t) is an infinitesimal
with respeet to tJ..t. These probabilities do not depend on the distribution
of the erystallization eentres formed prior to time t provided that it ean be
guaranteed (see later) that at the time t there is no erystal bulk in V'.
* Izv. Akad. Nauk SSSR Sero Mat.3 (1937), 355-360. Presented by S.N. Bernshtein.
188
ON THE STATISTICAL THEORY OF METAL CRYSTALLIZATION 189
2. This crystal bulk grows around newly-formed crystallization centres

and the whole crystal at a linear rate
c(t, n) = k(t)c(n),
depending on time t and direction n. We assume that the ends of the vectors of
length c( n) measured in the direction n from the origin form a convex surface.
Under these conditions an essential restrietion is that although c(t, n) may
depend on n, this dependence should be the same at all points. In other
words, the formulas given below hold either under the simplifying assumption
of uniform growth in all directions, or for crystals of arbitrary shape similarly
oriented in space.
§2. Determination of the probability pet)
The quantity c is defined by the equality
c3 = ~ ( c3 (n)du,
47r Js
where we integrate over the surface of the unit sphere S with centre at the
origin. Clearly, at t > to the volume of the crystal growing freely round a
centre formed at the moment t o is given by
~c 3 (1: k(r)drf·
Now consider an arbitrary point P of V, lying at a distance greater than
maxc(n) l t
k(r)dr
from the boundary of V.

In order for P to get into the crystallized bulk by the time t it is necessary
and sufficient that a crystallization centre is formed at a certain time t' < t at
some point P' whose distance from P is less than
c(n)lt k(r)dr,
t'
where n is the direction P' P. For a fixed t' the volume occupied by the points
P' satisfying our conditions is
V'(t') = ~c3(l,t k(r)drf.

190 ON THE STATISTICAL THEORY OF METAL CRYSTALLIZATION
The probability offorming a crystallization cent re during time tJ..t' in V'(t')

IS
a(t')V'(t')tJ..t' + o(tJ..t'),
and the probability that this does not happen is
•
q(t) = II{1 - a(ti)V'(ti)tJ..t'} + 0(1).
i=l
Therefore the probability that the point P does not belong to the crystallized
bulk at the moment t is
•
q(t) = II{1 - a(ti)V'(ti)tJ..t'} + 0(1), (1)
i=l
where t = stJ..t', ti = itJ..t', and 0(1) is infinitesimal if tJ..t' is infinitesimal.
Taking the logarithm of (1) we obtain
logq(t) = L• a(ti)V'(ti)tJ..t' + 0(1) = - 10ft a(t')V'(t')dt' =

i=l 0
t
= - 4; c3 10 a(t')( 1 t'
t
k(r)dr) 3 dt'. (2)
For the desired probability
p(t) = 1 - q(t)
for P to belong to the crystallized bulk we finally obtain
411"
p(t) = 1- eXP{-Tc3n}, (3)
1
where
n = 10t
t
a(t')( t' k(r)dr) 3 dt'. (4)
§3. Conclusions
When V is large enough as compared with the sizes of the individual grains,
we may set Vl(t) = Vp(t) or, in view of (3),
V(t) = V( 1- exp { _ 4; c n}) ,
1 3 (5)
ON THE STATISTICAL THEORY OF METAL CRYSTALLIZATION 191
where n is defined by (4). The formula (5) for the volumeV1 (t) ofthe substance
crystallized over the time t gives a solution of the first problem posed in the
introduction. If O'(t) and c(t, n) do not depend on time, then we can set
O'(t) = 0', k(t) = 1.

In this case
(4a)
and (5) gives

(5a)
For a sufficiently large volume V the following formula holds for the number
N(t) of crystallization centres formed over the period t:
N(t) =V 1 1
O'(r)q(r)dr. (6)
For a constant O'(t) = 0' and k = 1 we obtain from (6)

(7)
or
(6a)
where
For t = +00 (6) and (6a) give the total number of crystallization cent res
throughout the whole process. In particular, for constant O'(t) = 0' and k = 1
we obtain
(7a)
Note also the special case when all the crystallization centres are formed
at the very beginning, with an average of ß centres per unit volume. The
corresponding formulas are derived from the general formulas by passage to
the limit. Instead of (4) we obtain
t
n = ß(}o k(r)dr) ,
3
(4b)
192 ON THE STATISTICAL THEORY OF METAL CRYSTALLIZATION
formula (5) still holds and (6) is replaced for any t > 0 by the trivial identity
N = Vß. If we assume, moreover, that k = 1 (that is, c(t, n) is independent of
t), then we obtain
n = ßt 3 , (4c)
Vl(t) = V( 1- exp{ _ 4; c3ßt 3 }). (5c)
Mathematical Institute of Moscow State University

20 April 1937
References
1. F. Göler and G. Sachs, Z. Phys. 77 (1932), 281.

2. G. Tammann, Z. Andrg. Chemie 214 (1933), 407.
3. B.V. Stark, 1.1. Mirkin and A.N. Romanovskii, 'Metals and thermal treat-
ment', Mosk. lnst. Stal. 7 (1935), 5-38 (in Russian).
23. MARKOV CHAINS WITH A COUNTABLE NUMBER
OF POSSIBLE STATES *
In [1] I made some general assertions eoneerning the asymptotie behaviour

of transition from one state to another in an unbounded number of steps for
Markov ehains with a eountable set of possible states.
This paper gives a broader exposition of the same questions, eomplete
with proofs of the main theorems announeed earlier without detailed proofs,
and some new facts. The way of dealing with these questions used in this
paper may be of some interest also for Markov ehains with a finite number of
possible states. This is indieated in the note by W. Doeblin [2] whieh has sinee
appeared. As I know from his letter, W. Doeblin independently proved some
of the theorems on Markov ehains with a eountable number of states that I
published.
§1. Notation
Denote by Ei the various possible states of the system under study, where i
runs through all positive integers. We note that the following aeeount is also
applieable for the ease when i takes a finite number of values; in this ease a
number of the statements ean be simplified. The probabilities Pij of transition
from Ei to Ej in one step are, as usual, assumed to be subjeet to the eonditions
PI).. >
_ 0 (1)
(2)
The probabilities Pi)n) of transition from Ei to Ej in n steps are determined

inductively from the equalities
(3)
Pi)n+!) = 'L Pi<;')P1:j, (4)

1:
where
for i =j
for i f. j
* Eul. MGU Mat. Mekh. 1:3 (1937), 1-16 (in Russian).
193
194 MARKOV CHAINS WITH A COUNTABLE NU MB ER OF POSSIBLE STATES
Further, let KI;) be the probability of passing, in n steps, from Ei to Ej

without visiting Ej in a fewer number of steps. Clearly,
In particular, K~n) is the probability of returning to Ei for the first time in n

steps. We now set
I: K&n) = Lij.
00
(6)
n=l
Clearly, Lij ~ 1. If Lij = 1 then, starting from Ei the system will inevitably,
sooner or later, visit the state E j • The expectation of the number of steps
needed in this case for the transition from Ei to Ej is
I: nK&n).
00
Mij = (7)
n=l
In particular, M ii is the expectation of the number of steps prior to the first

return to Ei, provided we started from Ei. The expectation M ij can be both
finite and infinite.
§2. Inessential states, classes and subclasses of essential states
Astate Ei is called in essential if there exist a j and an n such that Pi~n) >0
and ~(;") = 0 for all m, that is, there is a transition from Ei to E j without
return to Ei. All other states are called essential. Clearly, if two states Ei and
Ej are essential and if there exists an n with Pi~n) > 0 then there also exists an
m with PjSm ) > O. Ifthere are such n and m, then the essential states Ei and Ej
are called communicating. If Ei communicates with Ej and Ej communicates
with Ek, then Ei also communicates with Ek. Therefore all essential states
fall into c1asses S(O:), such that states belonging to one dass communicate and
those belonging to different dasses do not communicate. It is also dear that
for an essential state Ei and an inessential state Ej, Pi~n) is always O. Thus,
on ce having fallen into astate of S(O:), our system can never leave this dass of
states.
Consider now an essential state Ei. Let IDti be the set of indices n for
which p;~n) > O. Since Ei is essential, the set IDt; is non-empty. If n and m
occur in IDt;, then so does n + m. Let di be the greatest common divisor of all
numbers in IDti . The set IDti consists only of multiples of di. It can easily be
MARKOV CHAINS WITH A COUNTABLE NUMBER OF POSSIBLE STATES 195
shown that all sufliciently large multiples of di occur in roti . The number c4 is
called the period of the state Ei.
It can easily be shown that all states be/onging to the same dass s(a) have
the same period which we denote by d(c:r) and call the period 0/ s(a). Indeed,
for two states Ei and Ej of the same dass s(a) let there exist n and m such
that Pi)n) > 0 and Pj~m) > 0 (such n and m do exist, as indicated above). Then
p};d j ) > 0 for sufliciently large k. Hence, for sufliciently large k,
that is, all sufliciently large numbers of the form kdj +n +m occur in roti ,
which is only possible if dj is divisible by di. Conversely, di is also divisible by
d j and therefore di = dj •
For two states Ei and Ej belonging to the same dass s(a), we simulta-
neously have Pi)n) > 0 and Pi)m) > 0 only if n == m(modd(c:r». Therefore,
having chosen a certain state Ei o of dass s(a), we obtain for any state Ei of
the same dass a well-defined number ß(Ei) = 1,2, ... , d(c:r) such that Pi~~) >0
is possible only for n == ß(Ei)(mod d(c:r». All the states Ej with given ß(Ej)
belong to the subdass S~a). Thus, s(a) is divided into d(c:r) subdasses ~a).
With every step our system inevitably goes from the states of S~a) into one of
the states of S~~l and in the case ß = d(c:r), into one ofthe states of S~a). Thus
if Ei and Ej belong respectively to subdasses S~a) and S~a), then Pi)n) =1= Oonly
if n == 'Y - ß (mod d(c:r». On the other hand, for sufliciently large n satisfying
the latter congruence we actually have Pi)n) > O.
§3. Recurrent and non-recurrent classes
Apart from the prob ability Lij that, having started from the state Ei, we
visit at least once, sooner or later, the state Ej, we introduce the prob ability
(lij that, starting from the state Ei we visit the state Ej an infinite number of
times. Clearly,
(8)
On the other hand, we now prove the following lemma:
Lemma 1. Lii = 1 implies (lii = 1.

196 MARKOV CHAINS WITH A COUNTABLE NUMBER OF POSSIBLE STATES
Proof. Denote by the prob ability that, starting from astate Ei, we return
TC.I:)
to it no fewer than k times. Clearly, always
For L;; = 1 these formulas imply that n ii = 1.

In this and the following sections we shall be concerned only with rela-
tions that hold within each dass of essential elements; in other words, it will
be assumed that an indices referring to states run only through the values cor-
responding to the states of one dass. In the statements of the theorems this
will be indicated by the words "within one dass" .
Theorem 1a. Within one dass either nij < 1 or nij = 1.

Proof. Clearly it suffices to establish that for any i,j, k,
1) nij = 1 implies ni.l: = 1;
2) nji = 1 implies n.l:i = 1.
If these two points have been established, then for any i, j, i', j', nij =1
implies nij' =
1 and ni/jl 1. =
Now let us prove 1). Assurne that nij = 1, that is, starting form Ei
we return to E j an infinite number of times with prob ability 1. Consider the
interval between the sth and the (s + l)th revisits. Since all states under
consideration belong to one dass, the probability of the event !V, that in the
interval between the sth and the (s + l)th visit to Ej the system will get to a
fixed state E.I: is positive. It can easily be seen that this prob ability does not
depend on s, and that the events !V, for different s are independent. However,
given an infinite sequence of possible independent events !V. with the same
positive probability, an infinite number of these events will indeed take place
with probability 1, that is, ni.l: = 1 as required.
It now remains to prove 2). For this we use the formula
n .. - ' " pCn)n.l:·

J' -~ j.l: " (9)
.I:
which holds for any i, j, k, n. Since L.I: PjC;) =

1, we have: nji 1 implies =
n.l: i = 1 for all k for which PjC;) > O. But for any j and k (within one dass)
such an n does indeed exist. This proves 2) and Theorem 1a.
If an the nij = 1, then by (8) an the Lij = 1. If, however, an the nij < 1,
then by Lemma 1, an the Lii < 1. This implies:
Theorem lbo Within one dass either all the Lii < 1 or all the Lij 1. =
It should be noted that when all the Lii < 1, there might still be some
Lij = 1 (i i= j).
If all the Lij = 1 and all the (lij =
1, then the dass is called recurrent.
If, conversely, all the Lii < 1 and all the (lij < 1, then the dass is called non-
recurrent. It is easy to see that if astate Ej is of non-recurrent dass and Ei is
arbitrary, then
lim p~!,,) =
n-++oo 'J
o.
§40 Positive and zero classes
The considerations of this section refer to phenomena observed within each of

the recurrent dasses. As a result of these considerations the recurrent dasses
will be divided into positive and zero dasses. Note, however, that in the fol-
lowing sections all non-recurrent dasses will also be referred to as zero classes.
The expectations Mij (see (7)) are of primary importance in the study of
recurrent dasses. In this connection we will consider the me ans
(10)
Lemma 2ao For any Ei /rom a recurrent dass with finite Mii
while if Mii = +00, then

lim 1r~?) =
n-+oo U
o.
Proof Starting from Ei we return to this state an infinite number of times with
probability 1. Let the first return to Ei take place at the nl th step, the second
return at the n2th step and the kth return at the nkth step. The differences
form a sequence of random variables independent of each other with the same
distribution law: Zk = s with prob ability KI:). Clearly, Mii is none other than
the expectation of each of the Zk.
Assume first that the expectation Mi; of the random variables XTe is finite.
Then, according to Khinchin's theorem [3], the sequence {XTe} satisfies the law
of large numbers, that is, for any f > 0 there exists a ko such that for k ~ ko
the prob ability of the inequality
Te
"x· - M··I" = InkTe - M··I" ->:2
I.!kL..JJ
j=l
is less than f. Let f< ! and n ~ no = 2koMii. Set

k' = ~(1- f),
Mii
k" = ~(1
Mii
+ f).
It can easily be seen that k' ~ k o, k" ~ k o. Therefore, with probability greater
than 1 - f we can assert that
which implies (since Mii ~ 1)
InTel - n(l - f)1 < k'f ::; nf,
hence,
Similarly, with probability greater than 1 - f we have
Thus if n ~ no, then with prob ability greater than 1- 2f we have the inequality
that is, the number tPn of returns to the state Ei within the first n steps is
between k' and k". It can easily be seen that the expectation of the frequency
tPn In of returns to Ei within the first n steps is 11'}~). Since for n ~ no we have
k' In < tPnln < k" In with prob ability greater than 1 - 2f:
1 tPn 1
-k'n = -(1
Mi;
- f) < -
n
< -(1
Mi;
+ f)
k"
= -,
n
and since always 0 ~ tPnln ~ 1 and Mii ~ 1, we finally obtain for n ~ no,
1 1I"~~) _ _1_ 1 < EltPn __1_ 1 < _f_ + 2f

SI Mii - n Mii - Mii '
which implies that
Now consider the case Mu = +00. Then, whatever M < +00 and f> 0,
there exists a ko such that for k ~ k o the prob ability that the inequality
A:
.!.kL..J)
" z, = <M
nA:
k-
j=l
holds is less than f. To prove this it suffices to introduce a new sequence of

random variables z~, independent of each other, smaller than the corresponding
variables ZA: (z~ ~ ZA: for every k) with equal expectations, for example, 2M,
and apply the law oflarge numbers in the above statement to this new sequence.
Setting k = [nIM] + 1> nlM we see that k ~ k o for n ~ koM and hence,
with probability greater than 1 - f,
nA: tPn 1 1
-<-+-.
k > M, nA: > kM ~ n,
n - M n
Hence, for n ~ koM
(n)
11" ..
SI
= E (tPn)
-M < -1
- M
+ -n1 + f,
which brings us to the condusion that in this case
lim 11"~~) = O.
n-++oo 11
Theorem 2. Within one class either all the Mii are infinite or all finite.
Proof. For any two states Ei and Ej of the same class there exist k and m such
that Pi)A:) > 0, Pj~m) > O. Furthermore it is dear that for any n,
p~?+A:+m)
))
> p~~) p~~) p~~)
-)1 U I)'
This inequality directly implies that

·
I1m (n) p(m)p(A:) I' (n)
1I"jj ~ ji ij 1m 1I"u .
n-+oo Q-+OO
Hence, the limits of 11"~;) for all i (corresponding to a given dass) are either
zero or positive. Therefore, according to Lemma 2a, the Mii are also either all
infinite or all finite.
It only remains to prove that the finiteness of all the M" implies the
finiteness of all the Mij. We denote by R~;) the probability that, starting from
Ei we visit Ej (j f i) in n steps without visiting Ei meanwhile. Then 1
n
M ii = L mK~;n) + LRr;)(Mji + n).
m=l j~i
But within one dass for any i and j we can find an n for which R~;) > O.
Hence Mji = +00 would imply Mii = +00, which proves our theorem.
The dasses with all the Mi; finite are called positive, and with all the
M ii = +00 are called zero. Note that in zero dasses some of the Mij (i f j)
may be finite.
Theorem 3. In a zero dass Pi)n) --4 0, as n --4 +00, tor any Ei, Ej in the
given dass.
To prove Theorem 3 we need the following:
Lemma 2b. In a zero dass, tor any Ej the quantity
~~,m) = ~(P~?+1)
7rJJ JJ
+ p~?+2)
JJ
+ . . . + p~?+m»
J1 (11)
m
tends to 0 unitormly with respect to n as m --4 +00.

Proot. Denote by H(') the probability that, with initial state Ej, starting from
the (n + 1)th step, the state Ej appears first at the (n + s)th step regardless
1 Proof. For r > n,
Therefore
"r "L..J R~?) K(~-n)

00 n
Mi; = "
L..J mK~~)
SI
= "L..J mK~~)
SI
+ L..J 'J J'
=
m=l m=l r>n j~i
n 00 n
= L mKim ) + L R~;) ~)n + s)Kj:) = L mKim ) + L R~;)(Mji + n).
m=l j~i .=1 m=l j~i
MARKOV CHAINS WITH A COUNTABLE NUMBER OF POSSffiLE STATES 201
of whether or not there was areturn to E; within the first n steps. Then
(12)
Choose an ro such that for r ~ ro we always have 1/"~i) < f. (This is possible
by Lemma 2a.) Choose mo > ralf. Also let m ~ mo. Then for m - s :$ ra we
have (m - s)/m < f, while for m - s ~ ro we have 1/"~j-.) < f.
Since always (m - s)/m :$ 1, 1/"~j-.) :$ 1, it foIlows that for all m ~ mo,
we have
m - S (m-.)
- - 1 / " ..
m JJ
<f.
Since
we see from (12) that 1/"~j,m) < f + 11m as soon as m ~ mo. Since f > 0 is
arbitrary and does not depend on n, our lemma is proved.
Proo/ 0/ Theorem 3. First note that to prove this theorem it suflices to prove
it for i = j, that is, to prove that the probabilities p;~) tend to 0 as n -+ +00.
To see this, we choose an m such that p;~m) > O. Then, clearly,
p~?+m) > p~~) p~!,,)

J1 - JI IJ'
Let n tend to +00 for m constantj since p;~+m) -+ 0 , it follows that

pä)n) -+ 0 as weIl.
Now assurne that
limsup p;~) = ~ > O.
n_+oo
If this assumption leads to a contradiction, Theorem 3 has been proved.
Choose an integer a such that
For any f> 0 there exists no such that n ~ no implies Pj~) ~ A + f. For some
6 > 0 choose mo ~ a such that
L KJj) < 6.
m>mo
Then the assumptions
n ~ no + mo, P (n) ,
.. >"-TJ (TJ > 0)
JJ -
imply that
In fact,
atm<mo
But for n ~ no + mo + sa the inequality
implies that
Pj~-a) ~ A - (TJ + f + 6)/A = A - TJ1,
Pj~-2a) ~ A - (TJ1 + f + 6)/A = A - TJ2,
where
TJ < TJ1 < TJ2 < ... < TJ.·
Now, for any s we can choose TJ > 0, f> 0, 6 > 0 so that TJ. < A/2. By
choosing no and mo suitably we obtain for all n ~ no + mo + sa such that
Pj~n) > A - TJ the inequalities
P11~?) > "'/2, p(n-a)

jj
'/2 ... , p(n-.a)
>", jj
'/2
>",
lI'Jr· a ,.a) > (I/sa) . S· A/2 = >'/2a.
Since >'/2a is constant, s is arbitrarily large and n can be assumed arbitrarily

large for fixed s, we arrive at a contradiction to Lemma 2b.
§5. Asymptotic relations within a positive dass
Theorem 3 completes in a first approximation the study of zero classes. As for

positive classes, we have the following general theorem.
Theorem 4a. In a positive dass S(O:), for any Ei from a subdass S~o:) and
Ei from a subdass S~o:), the probability Pi)n) tends to a limit
Pj = d(o:)/Mjj,
which is independent of i, when n -+ 00 runs through the values n - r-
ß (mod d(o:)).
Remark. If n ~ r- ß(mod d(o:)), then, as has already been noted, Pi)n) = O.

To prove Theorem 4 we need several lemmas.
Lemma 3. In a positive dass for any i,j and f > 0 there exists m such that for
any n the probability that, starting /rom Ei, we visit Ei at least once between
the nth and the (n + m)th step is greater than 1 - f.
Proof. The probability that, starting from Ei, we do not visit Ei within the
time interval indicated is given by
00 n+m-1 n-1 00 n-1
L...J K~~)
"" '3
+ ""
L...J KY?)
13
""
L...J p~~)
'3
+ ""
L...J K~~) "" p~~) <
13 L...J '3 -
p=n+m p=m+1 1:=n+m-p p=n+m 1:=1
00 00
<
- ""
L...J K~~)
~
+ ""
L...J pKY?)
33
= U(m).
p=m p=m+1
But if 00
Mn = LpKjr)
p=1
is finite, then u(m) -+ 0 as m -+ +00. Since u(m) does not depend on n, our
Lemma is proved.
204 MARKOV CHAINS WITH A COUNTABLE NUMBER OF POSSffiLE STATES
Lemma 4a. In a positive class consisting 0/ one subclass,
· . fP(n) 0
IImIn
n_+oo iJ' > .
Proo! Ifthe dass consists of one subdass, then there exists k o such that always
pN) > 0 for k ~ ko. In accordance with Lemma 3, we choose m such that for
given i and j the number f from Lemma 3 can be taken equal to !. We set
, =' f{P~~o) p~~o+l) p~~o+m)}
" In 1) 'JJ ' ... , 11 .
Clearly, A > O. Now let n' > m + k o . Set n' = n + m + k o . The probability
that, starting from Ei, we visit Ei between the nth and the (n + m )th step is
greater than !.
Suppose that the first arrival into Ei between the nth and the (n + m)th
step takes place after the (n+s)th step (s < m). Then the conditional probabil-
ity of revisting Ei in n' = n + m + k o steps is Pg o+m - 8 ) ~ A. This inequality
holds for any s (1 $ s $ m). Therefore, the total probability p;)n that, l
)
starting from Ei, we visit Ei in n' = n + m + ko steps, satisfies the inequality

p~~') > lA
'J 2'
which, since n is arbitrary, proves our Lemma.
Lemma 4b. In a positive class consisting 0/ one subclass
lim p~~)
n--+oo U
= 11M;;.
Proo/. First we prove the existence of the limit of Pi)n) as n -+ +00. Let
lim inf p(n)

n-+oo U
= a rlmsup p(n)
ii = b.
n-+oo
By Lemma 4a we have b ~ a > O. Let f > O. Choose m satisfying the conditions
of Lemma 3 for j = i. Then choose k o such that for k ~ k o the inequalities
(k)
a-f<P;; <b+f
hold. Let n >m+ k o be such that p;)n) < a + f, and n' >n+ k o such that
Pi)n l
) > b - f (such n and n' always exist). Put n' - n = k. Then
p~~')
11
= p(k) p~~)
n 11
+ A(l) p~~-l)
11
+ A(2) p~~-2)
n'..
+ + A(n) p~.o)
11 ,
where A(6) is the probability that, starting from Ei, we visit after the kth step
Ei for the first time only in (k + s) steps. Clearly,
n
pHt) + I:A(') :$ 1.
.=1
Moreover, by Lemma 3,
m
pHt) + I:A(') > 1- f.
6=1
These two inequalities imply that
Note also that for s :$ m we have n - s > k, and hence Pt~n-.) a - f, Pt~n') > b - fand b - a:$ 1 we obtain
b - f :$ b + 2f - (a - f)(b - a) :$ b + 3f - a(b - a),

a(b - a) :$ 4f, b - a :$ 4f/a.
Since a > 0 and f > 0 is arbitrary, it follows that b - a = O. This proves the
existence of the limit of ~~n) equal to b = a. Lemma 2a directly implies that
this limit is I/Mii'
Proo/ 0/ Theorem 4a. Consider, together with the given Markov chain, a new
chain determined by the elementary transition probabilities
P-IJ.. -_ p(d)
ij ,
where d is the period of the dass considered. Clearly, for all states of our dass
P?) = p~~d).
IJ IJ
With respect to the new Markov chain our dass of states forms one single
subclass. Therefore, according to Lemma 4b,
lim p.<.nd) = lim "P?) = _1_ = ~

n-++oo" n-++oo SI Mii Mii·
Thus Theorem 4a is proved for i = j. To prove it for the general case, let q
be the minimal number of steps in which we can pass from Ei to Ej (dearly
q == / - ß (mod d)). Then,
n
p.(~d+q) = """ K~':'1d+q) p~'.'d-md).
'J L...J 'J JJ
m::O
In this case,
m::1
and Pj~d-md) tends to d/Mjj for constant m and n -+ +00. This implies that
An important supplement to Theorem 4a is the following.
Theorem 4b. In a positive dass the sum of the limits Pj over all states in a
subdass is 1 for every subdass.
Theorem 4b follows directly from the next lemma:
Lemma 5. In a positive dass there exists for any f > 0 a finite system of
states Eh, Eh, ... ,Eile such that for any Ei from the same dass and for all
sufficiently large n,
k
I: Pi):) > 1 - f.
6::1
Proof. Choose i o arbitrarily. According to Lemma 3 there exists an m such

that for any n the prob ability that, starting from E io , we visit E io at least once
between the nth and the (n + m)th step is greater than 1 - f/3.
Clearly, we can always choose a system of states Eh, Eh , ... ,Eile such
that for any r ~ m,
k
I: p .(r). 'oJ.
f
> 1--.
3
6::1
We will prove that this system of states satisfies the conditions of the Lemma.
For this we take some fixed i and choose q such that
L.J K~~)
"
110
> 1- ~.
3
t=l
Now let n > m + q. Set n = q' + m, q' > q. With prob ability greater than
1 - f./3, starting from Ei we visit E io in the first q' steps. No matter at what
step ~ q' we first visit Ei o' with prob ability greater than 1 - f./3 we return
to Ei o between the q'th and the (q' + m)th step. If this happens at some
(q' +m- r)th step, then with probability greater than 1 - f./3 we are in one
of the chosen states Ei. after q' + m steps. Thus with probability greater than
(1- f./3)3 > 1- f., having started from Ei we arrive at one of our chosen states
Ei. in n = q' + m steps. This proves the Lemma and with it Theorem 4b.
Remark. Theorem 4a implies that not only does 7r~;) tend to I/Mi; (Lemma
2a), hut also that for any Ei from the same dass as Ei, the 7r]7) tend to the
same limit. By Theorem 4h, the sum E(1/Mii) taken over the states of one
sub dass is equal to l/d (where d is the period of the dass), and the same sum
taken over all the states of one dass is equal to 1.
§6. Asymptotic behaviour of probabilities in other cases
We disregard combinations of states Ei and Ei for which Pi)n) = 0 for all

n, due to the considerations of §2, and note that if Ei is inessential, then
always Pi)n) -+ 0 as n -+ +00. We consider he re the most difficult case: Ei
is inessential, while Ei is essential and belongs to a certain dass S(Ot). For an
inessential state Ei let NI Ot) he the prob ability that, having started from Ei,
we visit sooner or later one of the states of S( Ot). Clearly,
since, having once arrived in one of the states of the dass S(Ot) it is impossible
to leave this dass. In the case when we enter the dass S(Ot) at the initial state
Ei, we denote by no the number of steps before the first visit to one of the
states Ei of the dass S(Ot) and by ßo the number of the sub dass S~:) to which
this first state Ei belongs. Now let Ni~~) be the probability that, given the
initial state Ei, we visit the dass s(a) so that no == ßo +'Y (mod d(a». Clearly,
d(a)
L....J N~a)
'"
',1'
= N~a).
,
-y=1
The following result can be proved.
Theorem 5. In the ease 0/ an in essential Ei and an essential Ej /rom a

subclass S~a), the probability Pi~n) tends to Nt;L-yPj as n -+ +00 and runs
through the values n == ß(mod d(a».
Thus, we see that for fixed i and j the dependence of the probability
Pi~n) on niß all cases (essential or inessential) is asymptotically periodic. The
averages 1r}j) however, always have certain limits 1rij as n -+ +00.
Komarovka, 22 December 1936
References
1. A. Kolmogoroff, 'Anfangsgründe der Theorie der Markoffschen Ketten mit

unendlich vielen möglichen Zuständen', Mat. Sb. 1 (1936),607-610.
2. W. Doeblin, 'Sur les chaines discretes de Markoff', C. R. Aead. Sei. Paris
203 (1936), 24-26.
3. A. Khintchine, 'Sur la loi des grands nombres', C. R. Aead. Sei. Paris 188
(1928), 477-479.
24. ON THE REVERSIBILITY OF THE
STATISTICAL LAWS OF NATURE *
1. Statement of the problem
Consider an n-dimensional manifold R. Let let, x, y)dy1 dY2 ... dYn be the prob-
ability of transition in time t > 0 from a point x to a point 1J with coordinates
1Ji, i = 1,2, ... , n, such that Yi < 1Ji < Yi + dYi. Assume that let, x, y) is
differentiable up to a certain sufliciently high order and satisfies the following
conditions:
!(t,x,y)?O, (1)
11··· 1R
let, x,y)dy1 dY2 .. . dYn = 1, (2)
!(s+t,x,y) =11··· 1 R
!(s,x,z)!(t,z,y)dz1 dz2 ... dzn, (3)
where x is an interior point of a domain G (cf. [1], [2]). Given let, x, Y), p(x)
determines a stationary probability distribution compatible with let, x, y) if and
only if
p(x) ? ° (5)
11···1 R
p(X) dX l dx 2 ... dxn = 1, (6)
p(y) =11··· 1 R
p(x)!(t, x, y) dX l dx 2 ... dxn. (7)
A stationary distribution is called ergodie if let, x, y) -+ p(y) aB t -+ 00

for any x and y. Formula (7) implies that an ergodic stationary distribution is
always the only stationary distribution, that is, if there exists an ergodic sta-
tionary distribution Po(x), then no other ergodic stationary distribution p(x)
other than Po(x) can exist. Note that if R is closed, then the existence of
* 'Zur Umkehrbarkeit der statistischen Naturgesetze', Math. Ann. 113 (1937),

766-772.
209
210 ON THE REVERSIBILITY OF THE STATISTICAL LAWS OF NATURE
an ergodie (and, consequently, the only possible) stationary distribution di-

rectly follows from the single condition that f(t, x, y) > 0 for any x and y and
sufficiently large t (cf. [2],§5).
We suppose that f(t, x, y), as weH as a certain stationary distribution
p(x), are given beforehand. Assurne, moreover, that p(x) > 0 for any x. In
this case, knowing the position of y at the end of the time interval of duration
t, we can also determine the conditional distribution of the probabilities of the
initial position of x (for given y). Denoting by h(t, x, y) the density of this
conditional probability distribution, we have
h(t, x, y)p(y) = p(x)f(t, x, y). (8)
Clearly, (8) uniquely defines h(t, x, y).

The question on the reversibility ofthe laws ofnature mentioned in the title
of this paper 1 can be formulated in the following way: under what conditions
does the following relation hold:
h(t,x,y) = f(t,y,x)? (9)
This paper deals with the special case in which the function f(t, x, y) satisfies
the following Fokker-Planck equations:
{) f _ " " ; {) f "" " " ; j {)2 f

{)t - L.,..A (x){)x; + L.,..L.,..B (x) {)x;{)x i I
(10)
; ; j
{) f = - ""
8t {). "" "" {)2
L.,.. '8i{A'(y)f} + L.,.. L.,.. {) ;{) j {B'J.. (y)f} (11)
; y ; j y y
(see [2]).
For simplicity we will furt her confine ourselves to the case of a closed
manifold R. In this case, (11) implies that every stationary distribution p(y)
satisfies the equation
- " " '8i{A'(Y)p}

L.,.. {). + "L.,.." "L.,.." ~{B'J(y)p}
{)2 .. = O. (12)
; y ; j y yJ
1 In [3] this problem is discussed for the case of Markov chains with a finite number
of statesj cf. also [4].
ON THE REVERSffiILITY OF THE STATISTICAL LAWS OF NATURE 211
In addition to the propositions given above we further assume that the

quadratic form Bii is everywhere strictly positive definite. The case of a de-
generate form Bii is very interesting for many physical questions (see, for
example, [5]), but we shall not consider it here. The assumption of positive
definiteness of Bii impIies that I(x, y, t) > 0 for any x, y and t > 0, and hence,
the existence of a unique stationary distribution p(x), where p(x) > 0 for any
x. This unique stationary distribution p(x) is the unique solution of (12) such
that
JJ... J
R
P(X) dX l dx 2 ... dXn = 1. (13)
Under these assumptions, necessary and sufficient conditions for (9) to

hold can be directly expressed in terms of properties of the coefficients Ai and
Bii of the equations (10) and (11). Namely, under the above assumptions,
(9) holds il and only il the vector a(y) defined below, whose components are
expressed in terms 01 Ai(y) and Bii (y) is the gradient 01 a scalar potential. If
Bii (y) = hii , this condition becomes especially simple, since then a(y) is none
other than the vector with components Ai(y).
Let us once again state our assumptions:
1) the function I(t, x, y) is differentiable a sufficient number of times and
satisfies (1)-(4), (10), (11);
2) the manifold R is closed;
3) the form Bii (y) is positive definite. 2
2. Invariant form of the Fokker-Planck equations
Now we consider a more general case than that discussed in §l. Namely, we
now assume that the transitions from x to y in the time between the moments
sand t > s have prob ability distribution described by the prob ability density
I(s,t,x,y). In this case I(s,t,x,y) must satisfy the following conditions (cf.
[1], [2]):
I(s,t,x,y) ~ 0, (14)
JJ... JR
I(s, t, x, y)dy 1 dY2 ... dYn = 1, (15)
2 As already mentioned, this implies that p(x) > 0 for any x.

212 ON THE REVERSIBILlTY OF THE STATISTICAL LAWS OF NATURE
!(s,t,x,y) = 11 ···1 R
!(s,u,x,z)!(u,t,z,y)dz1 dz2 ... dzn, (16)
s< u < t,
11··· 1
G
!(s,t,x,y)dy1 dY2 ... dYn- 1 ast-s, (17)
with x belonging to the interior of G.

In this case the Fokker-Planck equations take the form:
8!(s,t,x,y) _ -LAi( s,x )~!( s, t ,x,y)_

j::I - j::I.
vS l vx
i
2
- ""
L..J ""..
L..J BI, (s, x) 8x8i 8xj !(s, t, x, y), (18)
i j
8!(s,t,x,y) "" 8 {Ai( ) ( )}

8t =-~8yi t,y!s,t,x,y +
I
2
+ """"
~ ~ 8y8i 8yj {B"(t,
.. y)!(s,t, x,y)}. (19)
I ,
The coefficients Bij(s,x) form a contravariant tensor ofrank two, whereas the
coefficients Ai(s,x) transform according to the following more complex law:
_.. 8 2 xi
A'=x'Ak + B km •
k 8x k 8x m
(From now on the summation sign is omitted.)
Let us assume that the quadratic form Bij (s, x) is positive definite every-
where and for all s, and choose it to be the principal metric form on R (note
that our metric depends on s). We then set
(20)
where r~k is the Christoffel symbol corresponding to Bij (s, x). The contravari-
ant vector a i coincides with Ai in every geodesic coordinate system (at x). In
a geodesic coordinate system chosen at the point x, (18) may be written in the
following way:
(21)
where (x) indicates the argument over which the derivative is taken, ßi is the
covariant derivative, and ß is the Laplace operator, that is, ß = ßißi.
ON THE REVERSIBILITY OF THE STATISTICAL LAWS OF NATURE 213
The latter equation, by virtue of its invariance, must hold also in any
other coordinate system. When the coordinates z change, f( s, t, z, y) does not
vary; if, however, the coordinates y change, this function transforms as a scalar
density.
Now set
fes, t, z, y) = VIBi;(t, y)It/J(s, t, z, y). (22)
Clearly, t/J(s, t, z, y) is now invariantwith respect to changes in both z and y;
it also satisfies (21):
(23)
As for (19), it takes the following form:
0;: = _d~lI) {li(t, y)t/J} + d (1I)t/J + l 0 log I~ (t, y)1 t/J. (24)
3. Solution of the problem

Now let us return to the case when the conditions indicated at the end of §1
hold. Then the quadratic form Bi; (z) is time-independent and depends only
on the coordinates of z on R. We set
li(z) = Ai(z) - ril:(z), (25)
f(t, z, y) = Vi Bi; (y)It/J(t, z, y), (26)
p(z) = VIBi;(z)I1r(z), (27)
h(t, z, y) = VIBi;(z)I.,p(t, z, y). (28)
Then to find the conditions when (9) holds, we merely have to the find condi-
tions under which the equivalent relation
.,p(t, z, y) = t/J(t, y, z) (29)
holds. In view of the results of §2, t/J(t, z, y) and 1r(z) satisfy the equations
ot/J/ot = Qi(Z)d~:C)t/J + d(:C)t/J, (30)
ot/J/ot = _d~II){Qi(y)t/J} + d(II)t/J, (31)
_d~:C){Qi(z)1r} + d(:C)1r = O. (32)

214 ON THE REVERSmILITY OF THE STATISTICAL LAWS OF NATURE
Sinee, moreover,
.p(t,:c, y) = 4J(t,:c, Y)-1r(:C )/1r(Y),
(29) ean be rewritten as
4J(t,:c,Y)1r(:C) = 4J(t,y,:C)1r(Y). (33)
Assurne that (29) (and henee (33» holds. Interchanging :c and y and using
(31), we obtain the following equation for 4J(t,y,:c):
(34)
(Hereafter all the derivatives are taken either with respeet to t or :c, but never
with respect to y, and o:i depends only on :c.) The same equation must be
satisfied by .p(t,:c, y) and therefore by .p(t,:c, Y)1r(Y) = 4J(t,:c, Y)1r(:C):
(35)
or
1r ~~ = -1ro:i l:1i4J - 4Jl:1i(o:i1r) + 1r1:14J + 2(l:1i 1r)(l:1i4J) + 4J1:11r. (36)
Multiplying (32) by -4J and (30) by -1r and adding the resulting inequalities
to (36), we have
or
(37)
It ean easily be seen that l:1i4J does not vanish identieally (that is, for any
t > 0) in any domain of R. Therefore, on an everywhere dense set of points :c
and henee, by eontinuity, everywhere,
that is,
(39)
Putting log 1r = P we see that P is a potential of 0:. This proves the
neeessity of the eondition stated in §l.
Assurne now that, on the eontrary, there exists P such that
ON THE REVERSIBILITY OF THE STATISTICAL LAWS OF NATURE 215
we must prove that in this case (29) holds.

It can easily be seen that e P satisfies (32). Since, however, 11" is the only
solution of (32) such that
we clearly have
(40)
11···1
where
J = eP VIBij(x)ldxldx2 ... dx n . '
R
Now (40) successively implies (39), (38), (37), (36) and (35), while (35) implies
that .,p(t, x, y) satisfies
(41)
that is, the equation that was earlier denoted by (34) holds for tfJ(t, y, x) as weH.
The initial values for tfJ(t,y,x) and .,p(t,x,y) clearly coincide at t = o. Hence,
tfJ(t,y,x) = .,p(t,x,y), which proves our theorem. 3
Moscow, 1 July 1936
References

lichkeitsrechnung', Math. Ann. 104:3 (1931), 415-458.
2. A.N. Kolmogoroff, 'Zur Theorie der stetigen zufälligen Prozesse', M ath.
Ann. 108:1 (1933), 149-160.
3. A.N. Kolmogoroff, 'Zur Theorie der Markoffschen Ketten', Math. Ann.
112:1 (1936), 155-160.
4. B. Hostinsky and J. Potocek, 'Chaines de Markoff inverses', Bull. Intern.
Aead. Teheque Sei. 36 (1935), 64-67.
5. A.N. Kolmogoroff, 'Zufällige Bewegungen', Ann. 0/ Math. 35:1 (1934),
116-117.
3 Cf. the uniqueness proof in [2]. The existence of continuous derivatives of the
coeflicients Ai and Bij follows from the existence of sufliciently high derivatives
of f. The condition of continuity of (29) in [2] for the case of a closed manifold
R is equivalent to our condition (4).
25. SOLUTION OF A BIOLOGICAL PROBLEM *
R.A. Fisher [1] gave an interesting application of iteration theory to the laws of
breeding a new gene in an unbounded population. Recently J.F. Steffenson [2]
gave a detailed exposition of the probability that all offspring from an individual
die out. Both these questions are, however, mathematically identical. After
giving a brief statement of the problem and recapitulating the known results, I
we will give several additions to the results by Fisher and Steffenson in certain
directions.
1. Mathematical statement of the problem
The following assumptions are essential for our mathematical discussion. Let
Fo, FI , ... , Fn , be a sequence of generations and let Fo have K o individuals with
a feature M (M-individuals); we assurne that crossing between M-individuals
is impossible; each M-individual from Fn +1 is assumed to be an offspring from
a certain M-individual in Fn . We are given the probabilities Pk that an M-
individual from generation F n has exactly k M-offspring in Fn +l , and these
probabilities are independent of the fate of other branches of M-offspring.
Pk
Our goal is to determine the probabilities n ) (n ~ 1) that the number
K n of M-individuals in the nth generation is equal to k. Here pJn) is the
probability that all the M-offspring die out. Clearly, pJn) can only increase
with n. The main result of this paper is an asymptotic formula for pJn) for
large n.
We assurne that the three first factorial moments
a = PI + 2P2 + ... + kPk + ...

b = 2P2 + 6P3 + ... + k( k - 1)Pk + ...
c = 6P3 + 24p4 + ... + k(k - 1)(k - 2)pl: + ...
are finite.
2. Biological explanations
K n denotes the number of M-individuals in the nth generation that have

reached a certain age. Consequently, the probabilities Pk, which are assumed
* Izv. NIl Mat. Mekh. Tomsk. Univ. 2:1 (1938), 7-12 (in Russian).
I References to previous works can be found in Stefl"enson's paper.
216
SOLUTION OF A BIOLOGICAL PROBLEM 217
to be prescribed, take into account the effect of selection. If, for instance, a
grown-up M-individual always has a very large number m of M-offspring in
the next generation which, however, have a very insignificant probability alm
of reaching the age set for determining K n , then we obtain
or, approximately, by Poisson's formula
(1)
In this case it is easy to compute that
Most important for the fate of an M-generation is the first moment a,

that is, the average number of M-offspring of an M-individual in the next
generation. If K n is very large, then we have approximately
We should distinguish three cases which are essentially different from each
other: a greater than, smaller than, or equal to L
If a < 1, then it is dear that the M-offspring will finally disappear. In what
follows we will show that this also holds when a = 1, b> O. If a > 1, then the
probability Po ofthe M-offspring dying out is #:-1 (Po = limPJn) as n -+ (0).
The probability that the offspring do not die but propagate unrestrictedly is
I-Po.
The exception a = 1, b = 0 is only possible when PI = 1, PA: = 0 (k #:- 1).
In this case, dearly K n always equals K o and consequently, pJn) is always O.
A specific problem that led to Fisher's studies is the following: in a very
large stationary population consisting only of individuals of BB type there
appears a small number K o of individuals of Bb type (generation Fa). If in
the further generations FI, F2, . .. Bb-offspring remain comparatively small in
number, then Bb x Bb crossing is virtually excluded. Under this assumption
our scheme can be applied to Bb-offspring.
If a = 1, then Bb-individuals are as viable as BB-individuals. However,
due to random fluctuations of K n they should surely die out (here the case a =
218 SOLUTION OF A BIOLOGICAL PROBLEM
1, b = 0 is impossible). Bb-individuals certainly disappear if a < 1. If a > 1,

tben tbere is a positive probability 1 - Po tbat Bb-individuals will propagate
unrestrictedly (a > 1 corresponds to tbe assumption tbat Bb-individuals are
more viable tban BB-individuals). In tbe real case, bowever, even for a = 1
random fluctuations of K n could be so substantial tbat tbe crossing Bb x
Bb attains a non-zero prob ability. Tben one must also take into ac count tbe
viability of bb-individuals.
3. The general theory
Tbe main result of Fisber and Steffenson consists in reducing tbe computation
of tbe probabilities pin) to determining tbe coefficients of known power series.
For tbis purpose tbey assurne tbat
q(x) = Po + PlX + P2x2 + ... + PleX Ie + ... , (2)
Q(n)(x) = p~n) + pfn)x + p~n)x2 + ... + pin)x le + ... (3)
Since Pie ~ 1, pin) ~ 1, it follows tbat q(x) and Q(n)(x) are analytic functions
for Ix I < 1. Let qn (x) be tbe ntb iterate of q( x):
Tbe basic formula of tbe tbe wbole tbeory is: 2
(4)
In particular,
and consequently,
p~n) = [qn(o)]Ko, (5)
Po = lim p~n) = lim[q(n)(O)]KO = AKo , n -+ 00. (6)
Tbe limit
A = lim qn(o), n -+ 00,
2 See [2]. The reader can easily reproduce the proof.

can be determined using general iteration theory. It can be proved that A

equals the minimal non-negative root of the equation 3
q(A)-A=O. (7)
This root does exist, since
q(1) = Po + Pl + ... + PI: + ... = 1, q(1) - 1 = O.
It can further be proved that for a > 1 the required smallest root A is
smaller than 1. Therefore in this case we have a positive prob ability
that the M -offspring propagate unrestrictedly. If, on the other hand, a :::; 1,
(and if a = 1, then b is also greater than zero), then A = 1 and consequently,
Po = 1. In this case M -offspring must necessarily die out. Therefore it is
desirable to study the asymptotic behaviour of the prob ability
that the offspring will survive up to the nth generation. Neither Fisher nor
Steffenson did this under general assumptions.
4. Asymptotic behaviour of R(n) for a:::; 1

Let
In our case r n converges to zero as n --+ 00. Hence, as n --+ 00 we have

asymptotically
(8)
It remains to study the asymptotic behaviour of r n . First note that
q'(1) = a,
3 See [2].
220 SOLUTION OF A BIOLOGICAL PROBLEM
while for 0 ~ x ~ 1,
o ~ q"(x) ~ q"(1) = b, 0 ~ qlll(X) ~ qlll(1) = c.

Now let a< 1. Expanding q(1- y) in a Taylor series we obtain
q(1 - y) = q(1) - q'(1)y + O(y2) =1- ay + O(y2).
Set y = rn-l = 1 - qn-l(o). Then
(9)
As can easily be seen, (9) implies that asymptotically
(10)
where C is a suitable eonstant. Finally, by (8),
(11)
Thus we see that for a < 1 the prob ability Rf..n) deereases asymptotieally as a
geometrie progression with eommon ratio a.
Let a = q'(1) = 1. Then
(12)
It follows from (12) that asymptotieally (for b > 0)
rn '" 2/nb. (13)
Finally, by (8),
R(n) '" 2Ko/nb. (14)
Thus we see that for a = 1, R(n) tends to zero eonsiderably more slowly.
Formula (14) shows that for a = 1 and large n the probability R(n) of survival
until the nth generation is inversely proportional to the seeond moment b. In
particular, if the probabilities Pie satisfy the Poisson formula (1), then b = a2 =
1 and consequently,
R(n) _ 2Ko/n.
This formula has been given by Fisher . As a second example we consider

the case when every Bb-individual gives exactly two offspring after the crossing
Bb x BB. Then Po = t,
Pt = ~, P2 = t, Pie = 0 (k > 2), which implies that
= =
b ~ and R<n) 4Ko/n.
5. An approximate formula
Now let us return to the case a > 1. Assume that
w =a-l
is small in absolute value compared to b > O. Then
is also small, and we can approximately set
Equation (7) gives
q(A) - A - 1- aJ.' + ~bJ.'2 - 1 + J.' = ":'wJ.' + ~bJ.'2 = 0,
J.' - 2w/b, (15)
A-I-2w/b. (16)
If w is small compared to Ko, then (15) and (16) imply approximately
1- Po - 2Kow/b,
that is, the prob ability 1 - Po of unrestricted propagation of M -offspring is

directly proportional to the selection coefficient and inversely proportional to
the second moment b.
References
1. R.A. Fisher , The genetic theory 0/ natural selection, Oxford, 1930.

2. J .F. Steffenson, 'Deux problemes du calcul des probabilites', Arm. [nst.
H. Poincare 3 (1933), 331-334.
26. ON A NEW CONFIRMATION OF MENDEL'S LAWS *
In the discussion on genetics that took place in the autumn of 1939 much at-
tention was paid to checking whether or not Mendel's laws were really true. In
the basic discussion on the validity of the entire concept of Mendel, it was quite
reasonable and natural to concentrate on the simplest case, which, according
to Mendel, results in splitting in the ratio 3:1. For this simplest case of crossing
Aa x Aa, with the feature A domin.ating over the feature a, it is weIl known
that Mendel's concept leads to the condusion that in a sufficiently numerous
progeny (no matter whether it consists of one family or involves many separate
families resulting from various pairs of heterozygous parents of type Aa)<the
ratio between the number of individuals with the feature A (that is, the indi-
viduals of the type AA or Aa) to the number of individuals with the feature a
(aa type) should be dose to the ratio 3:1. T.K. Enin [1], [2], N.I. Ermolaeva [3]
and E. Kol'man [4] have concentrated on checking this simplest consequence of
Mendel's concept. However, Mendel's concept not only results in this simplest
conclusion on the approximate ratio 3:1 but also makes it possible to predict
the average deviations from this ratio. Owing to this it is the statistical analysis
of deviations from the ratio 3:1 that gives a new, more subtle and exhaustive
way of proving Mendel's ideas on feature splitting. In this paper we will try to
indicate what we think to be the most rational methods of such checking and
to illustrate these methods on the material ofthe paper by N.I. Ermolaeva [3].
In contrast to the opinion of Ermolaeva herself, this material proved to be a
brilliant new confirmation of Mendel's laws. 1
§1. General considerations on the role of a random event in

hereditary phenomena
Let us first assume a viewpoint independent of Mendelism. Suppose that the

crossing of two individuals a and ß give a progeny of n individuals. In this
case, usually each of the individuals a and ß produces a number of gametes
which is much greater than the number of offspring n. Let a produce gametes
* Dokl. Akad. Nauk SSSR 27 (1940), 38-42 (in Russian).

1 My attention was drawn to the work of N.1. Ermolaeva by A.S. Serebrovskii.
222
ON A NEW CONFIRMATION OF MENDEL'S LAWS 223
and ß gametes
ßl, ß2, ... , ßk l •
It depends on many factors which exactly of the large number k1 , k 2 of possible

gamete pairs will be actually used for producing the progeny. We divide these
factors into intern al ones, determined by biological properties of the individ-
uals (t and ß and gametes (ti and ßi, and externaiones independent of these
biological properties. Take, for example, pollination of plants. Which pollen
grains will get onto the stigma and which will be lost? What will be the dis-
tribution of the pollen grains on the stigma ? All this depends on innumerable
factors that are extraneous with respect to the biologicallaws of behaviour. In
studying the biologicallaws of behaviour, these fertilization factors should be
considered as random and treated using the techniques of probability theory.
We can choose n pairs
in 8 = n.I(k 1 !l)~(k _ )1 different ways. In accordance with the above our further
n. 2 n.
study sterns from the assumption that for each of these possible choices there is
only a certain probability ofits actual fulfillment as determined by the biological
factors.
The derivation of Mendel's laws is based on the simplest assumption that
the probabilities corresponding to any of these 8 possible choices are equal
(and, consequently, all equal to 1/8) (see §2). From the biological viewpoint
this assumption implies the same viability of gametes, the absence of selective
fertilization, and equal viability (at least up to the moment of counting the
offspring) of the individuals resulting from any pair combination of gametes
(ti , ßi. For simplicity, we call this the independence hypothesis (the probability
of obtaining some set of gametes used for producing the progeny is assumed to
be independent of the biological features of these gametes).
Like any other hypothesis on the independence of some phenomenon of
certain other ones, our hypothesis taken as an absolute dogma not allowing any
corrections is wrong: there are a number of well-known examples of deviations
from this hypothesisj sorne are quantitatively insignificant, while others are
quite considerable.
It is quite clear that a viewpoint that totally rejects the role of biologi-
cal external random events in the selection of the gamet es that take part in
224 ON A NEW CONFIRMATION OF MENDEL'S LAWS
fertilization would be equally without foundation.

It can be seriously disputed, however, which of the following two viewpoints
is correct.
1. In most cases the independence hypothesis can weIl serve as a first
approximation to the real state of affairs (the proponents of Mendel's and
Morgan's genetics).
2. Selective fertilization and unequal viability always play so decisive a
role that the considerations based on the independence hypothesis are fruitless
for biology (Acad. T.D. Lysenko's school).
§2. Splitting in the ratio 3:1.
Let us now return to the more special Mendelian assumption for the case of
crossing Aa x Aa and domination of the feature A. In this case we assume
that each of the parents forms as many type Agametes as those of type a,
gamete pairs of AA and Aa types give offspring with feature A and gamete
pairs of aa type, offspring with feature a. These assumptions, together with
the assumption that k1 and k 2 are much greater than n, and the independence
hypothesis, imply the following:
1. The probability that there are exactly m individuals with feature A in
a group of n offspring (all the rest of them having feature a) is
Pn(m)
n!
= m!(n - m)!
(3)m
4"
(l)n-m
4" . (1)
Now assume that a large number r of Aa x Aa crossings is made, and each

crossing resulted in a family consisting of
individuals, of which
respectively have the feature A. The question is how one can best check whether
or not this result is in agreement with Mendel's assumption.
Ifthe number ofindividuals in every family is very small (for example, less
than 10), then it is reasonable to check (1) directly using Pearson's x2-criterion.
If each of the families is sufficiently numerous, then it is better to apply
another method. In this case (1) implies the following:
ON A NEW CONFffiMATION OF MENDEL'S LAWS 225
11. The normalized deviations
~ = (m/n - 3/4) : U n ,
where
approximately obey the Gaussian law with variance 1, that is, the prob ability
of
is approximately equal to
P(z) = _1_ft! e-e/ 2 cJe. (2)

.../2i -00
Here U n = (Vä/4)Vii is the mean square deviation of the frequency m/n from
3/4. We see that this mean square deviation is proportional to I/Vii. Hence
it is only for very large families that Mendel's theory predicts that m/n is very
dose to 3/4. For example, we can affirm that
Im/n - 3/41 < 0.01,
with prob ability 0.99 only when n > 12,000.

On the other hand, by examining a large number of families of medium
size we can achieve a much finer verification of Mendel's assumptions by con-
sidering the_ distribution of the devia~i()ns~. In particular, (2) implies that
the prob ability of I~I $ 1 is approximately 0.68. Hence, according to Mendel's
theory, among a sufficiently large number offamilies there should be about 68%
with ~ $ 1 and about 32% with I~I > 1. In Ermolaeva's work only aseries of
98 families with splitting in the colouring of a flower and leafaxil (Table 4 from
[3]) and the series of 123 families with splitting in the colouring of cotyledons
(Table 6 from [3]) can be used to verify this consequence of Mendel's theory.
The rest of the series contain too few families (or groups of families ) to provide
a reliable verification. The results are given in the following table. 2
2 The summary Table 1 in Ermolaeva's paper [3] presents some figures other than
those given here, since she takes into account certain families (2 from the first
series and 4 from the second serles) that, for reasons we do not know, were not
included in her Tables 4 and 6. Our summary Table includes only the families
represented in Tables 4 and 6. However, the conclusions given below remain the
same if drawn from the data given in the summary Table 1 of Ermolaeva.
226 ON A NEW CONFIRMATION OF MENDEL'S LAWS
Splitting
(in colour) Splitting
No. of families of the flower (in colour) Theoretical, %
and leaf axil of cotyledons
total 98 100% 123 100% 100
withlLlI ~ 1 66 67% 85 69% 68
withlLlI> 1 32 33% 38 31% 32
For this number of families in the series the correspondence with the theory
should be considered quite good. Due to some strange misunderstanding, Er-
molaeva herself claims in her work that the presence of a noticeable percentage
of families with ILlI > 1 disproves Mendel's theory.
In a similar way we could verify the coincidence of the percentage of fam-
ilies for which ILlI ~ a for a "# 1. For example, the theory predicts that
approximately 50% of the families should have ILlI ~ 0.674. However, it is best
just to check whether the actually observed distribution of the deviations Ll is
elose to the theoretical Gaussian distribution. To do this, we draw on the same
plot the theoretical curve y = P( x) in accordance with (2) and the empirical
step curve
y = q(x)/r,
where r denotes the total number of families in a given series and q(x) is the
number of families in the series for which Ll ~ x. The results of this verification
for the two series from Ermolaeva's paper are shown in Figs. 1 and 2. It is elear
that in both cases the coincidence between the theoretical and empirical curves
is sufficiently good. To evaluate whether the observed discrepancy between
these two curves is admissible for the given size of the series we should use the
formulas derived earlier (see [5]). Using these formulas for the cases given in
Figs. 1 and 2, we find that
Al = 0.82, A2 = 0.75,
ON A NEW CONFffiMATION OF MENDEL'S LAWS 227
-J -,2 ..;./ .z J -J -,2 J
Fig.1 Fig.2
which is quite satisfactory.

So we see that according to Mendel's theory, for the given family sizes we
could not have expected the frequencies m/n for the individual families to be
doser to their mean value 3/4 than those obtained by Ermolaeva.
If, for some sufficiently large series of families the deviations of m/n from
3/4 had been systematically smaller than required by the theory, then this
would have disproved the applicability of the above assumptions to this series
to the same extent as systematic overestimation of theoretically predicted devi-
ations would do. Hints to the fact that the frequencies m/n are systematically
too dose to 3/4 can be found in T.K. Enin's work [1]. However, the material
of this work is not broad enough (25 families as compared with two se ries of 98
and 123 families in Ermolaeva's work), and give reason for other doubts (the
author himself thinks that they are not sufficiently homogeneous). Therefore
we will not discuss them here.
Kol'man's paper referred to in the beginning of this note does not con-
tain any new facts; it only analyses Enin's data and is based on a complete
misunderstanding of the circumstances set forth in this paper.
20 February 1940
References
1. T.K. Enin, Dokl. Akad. Nauk SSSR 24 (1939), 176-178 (in Russian).
2. T.K. Enin, Dokl. VASKhNIL 6 (1939), (in Russian).
3. N.!. Ermolaeva, Yarovizatsiya 2:23 (1939), 79-86 (in Russian).
4. E. Kol'man, Yarovizatsiya 3:24 (1939), 70-73 (in Russian).
5. V. Romanovskii, Mathematical statistics, Moscow-Leningrad, 1939 (in
Russian).
27. STATIONARY SEQUENCES IN HILBERT SPACE*
Introduction
Definition 1. A sequence {x(t)} of elements in a complex Hilbert space H,

where t runs through all integers from -00 to +00, is called stationary if the
scalar products
B:r;:r;(k) = (x(t + k), x(t» (0.1)
do not depend on t.
Definition 2. Two stationary sequences {x(t)} and {y(t)} are called jointly
stationary if the scalar products
By:r;(k) = (y(t + k),x(t» (0.2)
do not depend on t.
The definitions of By:r;(k) and B:r;:r;(k) immediately imply that
B:r;y(k) = By:r;(-k), (0.3)

B:r;:r;(k) = B:r;:r;(-k). (0.4)
This kind of stationary and jointly stationary sequence is of great impor-

tance in prob ability theory and mathematical statistics. In terms of probability
theory they are studied in detail in H. Wold's book [1] and H. Cramer's paper
[2].
In §3 and §7 the main results of Wold and Cramer are given in terms of
the geometry of Hilbert space.
All the new problems that are studied and solved in this paper have also
arisen from prob ability theory and mathematical statistics. The application of
our results to problems of extrapolation and interpolation of stationary series
of random variables is illustrated in detail in my paper [7].
The following theorem shows a very simple connection between stationary
and jointly stationary sequences and unitary operators.
Theorem 1. Let the sequences
* Byull. Moskov. Gos. Univ. Mat. 2:6 (1941), 1-40 (in Russian).
228
STATIONARY SEQUENCES IN HILBERT SPACE 229
be stationary and pairwise jointly stationary, and let HII:11I:2 ••. lI:n be the smallest
c10sed linear subspace 0/ H containing all the elements 0/ these sequences. Then
the equations
UXj'(t) = xj'(t + 1), I-' = 1,2, ... , n, -00 < t < +00,
uniquely define a unitary opemtor
mapping HII:11I:2 ..• lI:n onto itself.

This theorem allows us to obtain a number of basic properties of stationary
and jointly station\ry sequences as direct consequences of the spectral theory
of unitary operators. The same is true for all the theorems of §§3-6.
The results of §§8-10 are more original and basically new; §§1-2 are of an
auxiliary nature.
Theorem 1 follows directly from Definitions 1 and 2 and Lemma 1 proved
in §1.
§1. Two lemmas
Let T be an operator defined on a certain subset M of Hand taking values in

H. The operator T is called an isometry if for any x, y E M,
(x, y) = (Tx, Ty). (1.1)
Let H M be the smallest closed linear subspace of H containing M. Then
Lemma 1. The opemtor T can be extended as an isometry onto the whole 0/

H M. This extension is unique.
For the proof we choose an everywhere dense sequence of elements of M,
Zl,Z2,···'Zn'···· (1.2)
We eliminate from this sequence all the elements Zn that depend linearlyon
the preceding Zk. Then we have a sequence
Znl' Zn2' .. ·' Zn",··· , (1.3)

230 STATIONARY SEQUENCES IN HILBERT SPACE
for whieh the set R of all elements z of the form
(1.4)
eontains all elements of the sequenee (1.2) and is dense everywhere in HM.
Sinee the elements of R ean be uniquely represented in the form (1.4) (by the
linear independenee of the elements in (1.3», the formula
(1.5)
uniquely defines an operator T* on R sueh that
(z', z") = (T* z', T* z")
for any z, and z" in R, that is, T* is an isometry on R. It ean be extended by

eontinuity to an isometry onto the whole of HM.
Sinee all the elements of (1.2) are eontained in R, (1.5) implies that for all
elements of (1.2),
by eontinuity this also holds for the whole set M.

Thus, the existenee of the required extension is proved.
By (1.5) any other isometrie extension of T eoineides with T* on R. By
eontinuity this eoineidenee also holds on the whole of HM. Consequently, the
isometrie extension of T onto the whole spaee HM is unique, whieh proves the
Lemma.
Lemma 2. A sequence
satisfying the conditions
(Um, u n ) = cmn ; m, n, = 1,2, ... , (1.6)
exists on the space H if and only if for any k, ml, m2, ... , mk the matrix
IIcm,mj 11 is hermitian and non-negative, that is, if 1
S = 2: Cm,m/.i(i ~ 0 (1.7)
i.i=l
1 Here and in what follows, a ~ 0 means that aisreal and non-negative.

for any complex numbers el,6,··· ,ek.

To prove the necessity of (1.7) consider an element
of H. It is easy to compute that
lIull2 = (u, u) = S. (1.8)
Since always lIull 2~ 0, (1.8) implies (1.7).

We now prove the sufficiency of (1.7). In accordance with (1.7), assume
that for any n,
n
Sn = L: Cijei(j ~ o.
i,j=1
We find a transformation
n
71i = L: a~j)ej
j=1
that reduces Sn to the form

m
Sn = L: l'1d 2 •
i=1
Choose in H an orthonormal system of vectors
(n) (n) (n)

ZI ,z2 , ... 'Zm
and set m
u~n) = L:a~7)zj, i = 1,2, ... , n.
j=1
Then it can easily be computed that
(u1~n) ' u<n» - c ..

J - IJ·
Having done this for all positive integers n, we construct a sequence
using the following inductive process:

1) We set
2) assuming Ul, U2, ••. , UN to be already known and to satisfy the require-
ment (1.6) for all m, n ::; N, we set
n = 1,2, ... ,N.
The operator TN is an isometry on the set MN = {u~N+l), ... , uW+l)}.

By Lemma 1 it can be extended to an isometry on HMN' Since HMN and
TMNHMN are finite dimensional, TN can be extended to an isometry on the
whole of H (see, for example, Theorem 2.49 from M. Stone's book [3]). Having
chosen such an extension, we set
Clearly, the elements Ul. U2, ... , UNH satisfy (1.6) for all m, n ::; N + 1.
§2. Operator calculus in H
In accordance with Theorem 1 we denote by Hz; the minimal closed linear

subspace of H containing all the elements of a stationary sequence {x(t)} and
consider the unitary operator Uz; defined by the formula
Uz;x(t) = x(t + 1). (2.1)
Let
(2.2)
be the spectral representation 2 of Uz;. For -1r ::; A ::; +1r we set
(2.3)
2 See M. Stone's book [3] on spectral representations of linear operators. In wh at

follows we will only use spectral families E(A) corresponding to unitary oper-
ators for which E(-r) =0 and E(+r) =
1. All the theorems from Stone's
book used below (formulated there for the more general case of spectral families
corresponding to spectral representations of self-adjoint operators of the form
A = f~oo AdE(A)) are applicable to them.
After Stone ([3], Chapter 6, §1), we denote by L; the dass of all complex
functions </J, measurable with respect to Fxx(A), defined on the segment -'Ir ~
A~ 'Ir, and for which

(2.4)
is finite. Two functions </JI and </J2 from L; are considered to be identical if
almost everywhere with respect to Fxx(A).

To every function </J from L; there corresponds (see [3] Theorem 6.1) the
operator
(2.5)
whose domain 6(</J) consists of all the elements Z of Hx for which the integral
(2.6)
is finite. The finiteness of (2.4) implies that x(O) belongs to 6(</J) for any </J from
i:
L;. Therefore to each </J in L; there corresponds a certain element
ZtP = T(</J)x(O) = </J(A)dEx(A)X(O) (2.7)
of the space H x .
These definitions allow us to state the following two Lemmas, which are
almost immediate corrolaries of Theorems 6.1 and 6.2 of Stone [3].
Lemma 3. The map </J T(</J) is a one-to-one correspondence between L;

-jo
and the dass Tx 01 all operators T representable in the lorm (2.5), where </J
belongs to L;.
Under this map
a</J -jo aT( </J), (3a)

</JI + </J2 -jo T( </JI) + T( </J2); (3b)
(3c)
(3d)
if n(l') = 1 for I' ~ A and n(l') = 0 for I' > A, then
(3e)
Lemma 4. The map

,p - z~
maps the class L~ in one-to-one fashion onto the whole space H:c. Under this
map
a,p - az,p, (4a)
(4b)
if ,pl,p2 belongs to L~, then
,pl,p2 - T(,pdz~2 = T(,p2)Z~1I (4e)

eit>.,p _ U;z,p, (4d)
I:
n(,p) - E:c(A)Z~,
(Z~l' Z,p2) =
(4e)
I:
,pl(A),p2(A) dF:c:c(A), (4f)
(E:c(A)Z~l' Z~2) = ,pl(A),p2(A) dF:c:c(A). (4g)
The properties (3a), (3b) and (3e) of the map ,p - T,p follow from Theo-
rem 6.2 of Stone [3]. The properties (3d) and (3e) ean be derived by eomparing
I:
(2.5) with the formulas
I: I:
U; = eit"dE:c(A), (2.8)
E:c(A) = dE:c(I') = n(l') dE:c(I'). (2.9)
The fact that the map ,p - T(,p) is one-to-one follows from the fact that for
,pl "I ,p2 (in the sense of equality in L~) the differenee
has norm
and consequently
To prove Lemma 4 we note the following: the set !m of all elements of

Hz: representable in the form (2.7), where I/J belongs to L;, is, according to
Theorem 6.2. of Stone [3], a closed linear subspace of Hz:; since
(2.10)
and eit>· belongs to L;, the set !m contains all the x(t), which implies that
In view of this remark, the fact that I/J - z~ is a one-to-one correspondence

and its properties (4a), (4b) and (4f) follow directly from Theorem 6.2 ofStone
[3].
The properties (4c), (4d) and (4e) of the correspondence I/J - z~ follow
from the corresponding properties (3c), (3d) and (3e) of the map I/J - T(I/J).
Finally, (4e) and (4f) imply (4g):
(Ez:(A)Z~ll Z~2) = i f.x>.(J1.)1/J1(J1.)]~2(J1.)dFz:z:(J1.) =
i:
7r
7r
= I/Jl(A)~2(A)dFz:z:(A).
This proves Lemmas 3 and 4.
§3. Basic spectral properties
Theorem 2. For any stationary sequence {x(t)} the values Bz:z:(k) can be
represented as
(3.1)
where Fz:z: is areal continuous function, non-decreasing from the right, defined
on the interval -1f ~ A~ 1f, and such that
(3.2)
The function Fz:z:(A) satisfying these conditions is defined uniquely by Bz:z:.

Here we note also that F:I::I:(A), which in what follows will be called the
spectral function of the sequence {x(t)}, coincides with the function defined by
(2.3), as will be dear from the text below.
Theorem 3. Iftwo stationary sequences {x(t)} and {y(t)} are jointly station-
ary, then
(3.3)
where F:l:y(A) is an (in general complex) function continuous from the right, of
bounded variation, defined on the interval -11" :::; A :::; 11", for which
(3.4)
The function F:l:y(A) satisfying these conditions is uniquely defined by B:l:Y.

Since any stationary sequence is jointly stationary with respect to itself,
Theorem 2 can be considered as a particular case of Theorem 3, except for the
statement that F:I::I:(A) is real and non-decreasing.
1:
To prove Theorem 3, we consider the spectral representation
U:l:Y = ei>'dE:l:y(A) (3.5)
of the operator U:l:Y whose existence follows from Theorem 1 for
n=2, Xl=X, X2=Y.
For -11" :::; A :::; 11", we set
(E:l:y(A)X(O), y(O» = F:l:y(A) (3.6)
It is known (see Stone [3], p.177) that F:l:y(A) is a function, continuous

from the right and of bounded variation. The equality
(3.7)
implies that
Finally,
B:l:y(k) = (x(k), y(O» = (Ukx(O), y(O» = (1: eik>.dE(A)X(O) , Y(O») =
= 1: eik>.d(E(A)X(O),y(O» = 1: eik>'dF:l:y(A).
STATIONARY SEQUENCES IN HILBER!' SPACE 237
Thus, F:r:y(~) defined by (3.6) satisfies all the conditions of the theorem.
For the particular case of one sequence {z(tn = {y(tn the function
(3.8)
is real and non-decreasing (see Stone [3], p.189). This justifies the correspond-
ing statement of Theorem 2.
Elementary properties of Fourier series for functions of bounded variation
(see, for example, A. Zygmund [4], p.13) imply that
(3.9)
where
w:r:y (~) = B :r:y (O)~ _ '"
L-,
B:r:y(k) e- iA:>.
ik ' (3.10)
k;tO
and the constant C can be found from the condition
(3.11)
Formulas (3.9) and (3.10) show that F:r:y(~) is uniquely defined by B:r:y. Thus,
Theorems 2 and 3 are completely proved.
Note also that (3.8) directly implies that
(3.12)
Theorem 4 (H. Cramer). 11 the sequences
are stationary and jointly stationary, then lor any -'11' ~ a ~ ß ~ '11' the
increments
(3.13)
lorm a non-negative hermitian matrix, that is,

n
E äF:r:,.:r:II{./Jell ;::: 0, (3.14)

/J,II=l
Conversely, if the functions
F/JI/()..), p., v = 1,2, ... ,n,
which are continuous /rom the right and are defined on -1r ~ ).. ~ 1r, satisfy
the conditions
n
L: ~F/JI/f./J(1/ ~ 0, (3.15)
1',1/=1
(3.16)
then there exists in H a system of stationary and jointly stationary sequences
for which
(3.17)
To prove this first part of the theorem we consider the spectral represen-
tation
of the operator
of Theorem 1 and setting

n
Z = L:f./JX/J(O),
1'=1
we consider on the inter val -1r ~ ).. ~ 1r the function
n n
~()..) = IE()..)zI2 = (E()")z, z) = (E()") L: f./Jx/J(O), L:f.//XI/(O») =

1'=1 1/=1
n n
= L: (f./JE()..)x/J(O),f.l/xl/(O» = L: f./J(I/(E()..)x/J(O), XI/(O» =

1',1/=1 1',1/=1
= L: Fx,.x,,()..)f./Jf.//.
1',,,=1
Since </J().) is real and monotone non-decreasing (see Stone [3], p.189), it
follows that for a <ß
n
</J(ß) - </J(a) = ~</J = L: ~FII:,.II:"'("f,,, ~ 0,

,.,,,=1
as required.
Passing to the second part of the theorem, we set
According to Lemma 2, in order that there exist elements Z,.(t) in H corre-

sponding to any integer t and any J.1. = 1,2, ... ,n such that
(Z,.(t'), z,,(t")) = B,.,,(t' - t"),
it suffices to show that

r
B= L: B,.p,.q(tp - tq)f,p(q ~ 0
p,q=l
for any r, J.1.p, tp, f,p. This is always the case since, setting
where the sum E(") is taken over all p for which J.1.p = J.1., we have, by (3.15),
For the elements Z,.(t) of H, whose existence is thus established, we have
Therefore the sequences

are stationary and jointly stationary. It is easy to see that in this case
Theorem 4 is completely proved.

For n = 1 the second part of Theorem 4 implies
Theorem 5. For any real non-decreasing function F(A), continuous from

the right, defined on -'Ir ~ A~ 'Ir and such that F( -'Ir) = 0, there exists a
stationary sequence {x(t)} with
For n = 2 (3.15) is equivalent to the set of conditions
(see Cramer, [2], p. 227). In this case Theorem 4 gives
Theorem 6. If the sequences {z(t)} and {y(tn are stationary and jointly
stationary, then for any -'Ir ~ a < ß ~ 'Ir,
(3.18)
Conversely, if the functions F ll (A), F 22(A) and F12(A) defined on -'Ir ~
A~ 'Ir are continuous from the right and satisfy the conditions
(3.19)
then there erist stationary and jointly stationary sequences {z(tn and {y(tn
for which Fu(A) = Fl1(A), F!I!I(A) = F22(A), FZ!I(A) = F12 (A).
In what follows we require the following strengthened version ofTheorem 6:
Theorem 7. If F l1 (A), F22 (A) and F 12 (A) satisfy all the conditions of the
second part of Theorem 6, the stationary sequence {z(tn is such that
and the orthogonal complement HeHr; of Hr; in Hisinfinite dimensional,

then there exists a sequence {y(t)} in H, stationary and jointly stationary with
{x(t)}, fOT which
To prove Theorem 7, we note that, by Theorem 6, the conditions assumed

with respect to F11 (A), F22 (A) and F12 (A) imply the existence of stationary
and jointly stationary sequences {x·(t)} and {y·(t)} for wh ich
For all integers t we set
Tx·(t) = x(t).
It can easily be verified that T is an isometry on the set {x*(t)}. By Lemma 1,
Textends as an isometry onto the whole space Hr;o. Then clearly,
Since the orthogonal compiement HeHr; is infinite dimensional, T can

be extended as an isometry from Hr; onto the whole space Hr;I/ (see Stone [3],
Theorem 2.49). Having chosen such an extension, we set
Ty*(t) = y(t).
It is easy to see that the sequence {y(t)} has all the properties required
by Theorem 7.
§4. Sub ordinate sequences
Definition 3. A stationary sequence {y(t)} is said to be subordinate to a sta-

tionary sequence {x(t)} if {x(t)} and {y(t)} are jointly stationary and all the
elements of {y(t)} belong to Hr;.
Clearly, if {y(t)} is sub ordinate to {x(t)}, then
Hr;I/ = Hr;, (4.1)

Ur;I/ = Ur;. (4.2)
Using the notions introduced in §2 we will prove the following theorem.
Theorem 8. To every sequence {y(t)} subordinate to {x(t)} there corresponds
1:
a unique function 4>~x)(A) of dass L; for which
14>~X)(A)12dFxx(A),
1:
Fyy(A) = (4.3)
Fyx(A) = 4>~X)(A)dFxx(A). (4.4)
The map
is a one-to-one correspondence from the dass Yx of all sequences {y(t)} subor-

dinate to {x(t)} onto the dass of L; functions 4>(A). For two sequences {Yl(t)}
and {Y2(t)} /rom Yx we have
(4.5)
The assertion of Theorem 8 that the function 4>~x) (A) is unique is to be

understood in the sense of identity offunctions 4>( A) from the dass L;, given in
§2. The one-to-one correspondence follows from (4.4) by virtue of the equality
(4.6)
We will prove the existence of functions 4>~x)(A) for 'any sequence {y(t)} from
the dass Yx using Lemma 4 in §2. If {y(t)} belongs to Yx , then y(O) belongs
to H x , and according to Lemma 4, there exists a function 4>~x)(A) of dass L;
such that
y(O) = z tP~X). (4.7)
By (3.6) and (4g) of Lemma 4 we have
1:
Fy1y2 (A) = (Ey1y2 (A)Yl(O),Y2(O)) = (Ex(A)Yl(O),Y2(O)) =
= 4>~~)4>~:)(A)dFxx(A)
for any sequences {Yl(t)} and Y2(t)} in Y x , which proves (4.5). In the particular
case Yl = y, Y2 = X we obtain (4.4), while for Yl = Y2 = X we obtain (4.3).
The one-to-one correspondence between {y(t)} and q,~X)(A) also follows

from Lemma 4.
An important addition to Theorem 8 is
Theorem 9. If two sequences {z(t)} and {y(t)} are stationary and jointly
stationary, then {y(t)} is subordinate to {z(t)} if and only if there exists a
i:
function q,(A) in L~ for which
i:
Fyy(A) = 1q,(AWdFxx(A), (4.8)
Fyx(A) = q,(A)dFxx(A). (4.9)
The necessity of the condition follows from Theorem 8. Let us prove

its sufficiency. If there exists q,(A) in L~ satisfying (4.8) and (4.9), then by
Theorem 8 there exists a sequence {y*(t)} subordinate to {z(t)} for which
Therefore
By Lemma 1 we can determine an isometry Ton Hxyo such that for any
integer t,
Tz(t) = z(t), Ty*(t) = y(t).
Clearly
(4.10)
Since {y*(t)} is sub ordinate to {z(t)},
(4.11)
Equalities (4.10) and (4.11) imply that
which means that {y(t)} is subordinate 3 to {z(t)}.
3 This condusion makes it dear that {y(t)} coincides with {x(t)}.

Definition 4. Stationary sequences {z(t)} and {y(t)} are called equivalent if

each of them is subordinate to the other, that is, if they are jointly stationary
and if
H:t:=H".
Theorem 10. A sequence {y(t)} subordinate to {z(t)} is equivalent to {z(t)} if

and only if tP~:t:)(A) is non-vanishing almost everywhere with respect to F:t::t:(A).
If this is the case, then
(4.12)
almost everywhere with respect to F:t::t:(A) and F",,(A).
1o. We prove the necessity of the condition. Assume that {y(t)} is sub-
ordinate to {z(t)}, and {z(t)} is subordinate to {y(t)}. Then it follows from
(4.3) and the formula
(4.13)
obtained by interchanging the indices, that F:t::t:(A) and F",,(A) are absolutely
continuous with respect to each other. Therefore the notions "almost every-
where with respect to F:t::t:" and "almost everywhere with respect to F",," co-
incide. By (4.3) and (4.13),
and hence, almost everywhere with respect to F:t::t:,
(4.14)
Formula (4.14) shows that tP~:t:)(A) ::f:. 0 alrnost everywhere with respect to
F:t::t:, as required.
20 • We now prove the sufficiency ofthe condition. Assume that {y(t)} is
subordinate to {z(t)} and tP~:t:)(A) ::f:. 0 alrnost everywhere with respect to F:t::t:.
Since, according to (4.3), F",,(A) is absolutely continuous with respect to
F:t::t:(A), the function tP~:t:)(A) is also absolutely continuous almost everywhere
with respect to Fyy('x). Therefore we can write
J-,..A1</J~:>:)('x)12
dFyy('x) = JA 1</J~:>:)(,X)I2dF:>::>:('x) = JA dF:>::>:('x) = F:>::>:('x)
-,.. 1</J~:>:)('x)12 -,.. ,
(4.15)
JAdFyy('x) -JA 1</J~:>:)('x)I2dF:>::>:('x) -JA .I.(:>:)dF:>: >: (')_

" -
1:
( ) - ( ) - 'l'y
-,.. </Jy:>: (,X) -,.. </Jy:>: (,X) -,..
= </J~:>:)('x)dF:>::>:('x) = Fy:>:('x) = F:>:y('x). (4.16)
Formulas (4.15) and (4.16) show that the function
(4.17)
1:
satisfies the conditions
1:
F:>::>:('x) = 1</J(,X1 2 dFyy ('x), (4.18)
F:>:y('x) = </J('x)dFyy('x). (4.19)
By Theorem 9 this implies that {x(t)} is subordinate to {y(t)} , as required.
30. By Theorem 8, (4.15) and (4.16) imply that
§5. Decomposition of stationary sequences into

orthogonal summands
Definition 5. Two sequences {Yl (t)} and {Y2(t)} are called mutually orthogonal
if
(5.1)
for all integers k and t.

Clearly, two stationary and jointly stationary sequences {Yl (t)} and {Y2 (t)}
are mutually orthogonal if and only if
(5.2)
or, equivalently, if
(5.3)
Theorem 11. Iftwo sequences {Yl(t)} and {Y2(t)} are stationary and mutually
orthogonal, then
a) the sequence {z(t)} = {Yl(t) + Y2(t)} is stationary;
b) {Yl(t)} and {Y2(t)} are jointly stationary both with respect to each other
and with respect to {z(t)}:
(cd Bxx(k) = B'/1!Il (k) + B y2y2 (k),
(C2) B Y1X (k) = B y1y1 (k),
(C3) B Y2X (k) = B y2y2 (k),
(dd Fxx(A) = Fy1y1 (A) + Fy2y2 (A),
(d 2 ) FY1X (A) = Fy1y1 (A),
(d 3 ) FY2X (A) = Fy2y2 (A).
Theorem 11 is proved by simple computations based on (0.2), (3.9)-(3.11).
In the conditions of Theorem 11, by (d 1 ) and the non-negativeness of the
increments t1Fxx , t1FY1YlI t1FY2Y2 (for non-negative t1A) we have:
(5.4)
Therefore the functions
(5.5)
i:
are bounded and uniquely defined (in the sense adopted in [3]) by the formulas
,p~~)(A)dFxx(A),
i:
Fy1y1 (A) =
(5.6)
Fy2y2 (A) = ,p~~)(A)dFxx(A).
Theorem 12. Under the conditions of Theorem 11, either both sequences
{Yl (t)} and {Y2(t)} are subordinate to {x(t)} or neither iso The first of these
two cases takes place if and only if
(5.7)
almost everywhere with respect to FII:II:.
Proof. 1°. If {Yl(t)} is subordinate to {z(t)}, then by virtue of the relation
Y2(t) = z(t) - Yl(t),

{Y2(t)} is also subordinate to z(t). Similarly, if {Y2(t)} is sub ordinate to {z(t)},
then {Yl(t)} is also subordinate to {z(t)}.
2°. Ifboth sequences {Yl(t)} and {Y2(t)} are subordinate to {z(t)}, then,
comparing (5.6) and (4.3), we see that almost everywhere with respect to FII:II:'
(5.8)
but if we take into account (d 2 ) and (da), then comparing (5.6) with (4.4), we
see that almost everywhere with respect to Fil:II: ,
.1.(11:)
'l'Yl
= ..J.(II:)
'l'Yl'
.1.(11:)
'l'y~
= 'l'Y2
..J.(II:)
.
(5.9)
Comparing (5.8) and (5.9), we conclude that t/J~~) and t/J~~) are equal to 0 or
1 almost everywhere with respect to FII:II:. Since, moreover, almost everywhere
with respect to Fil:II:
.1.(11:) .1.(11:) _ dFY1Y1 dFY~Y2 _ dF1I:1I: _ 1
'l'Yl + 'l'y~ - dF1I:1I: + dF1I:1I: - dF1I:1I: - , (5.10)
it follows that (5.7) holds.

3°. Suppose that (5.7) holds. Then, by (5.10), t/J~~) and t/J~~) are equal
to 0 or 1 almost everywhere with respect to Fn .
Hence, almost everywhere with respect to Fil:II: we have
(5.11)
Formulas (5.11), (5.6), (d 2 ) and (da) imply that
FY1Y1 (A) = I: 1t/J~~)(A)12dFII:II:(A),
FY11I:(A) = I: t/J~~)(A)dFII:II:(A),
Fy2y2 (A) = I: 1t/J~~)(AWdFII:II:(A),
(5.12)
FY~II:(A) = I: t/J~~)(A)dFII:II:(A).
By Theorem 9, the existence of tP~~) and tP~~) satisfying (5.12) implies that
the sequences {Yl(t)} and {Y2(t)} are subordinate to {x(t)}.
Theorem 13. Suppose that for a stationary sequence {x(t)}, the orthogo-
nal complement He H x of the space H x in H is infinite dimensional. Then
corresponding to any representation of Fxx(A) as a sum
(5.13)
of two real non-decreasing functions F 1(A) and F 2(A) continuous from the right
and for which F 1(-1r) = F 2(-1r) = 0, there is at least one representation of
{x(t)} as
x(t) = Yl(t) + Y2(t), (5.14)
where {Yl(t)} and {Y2(t)} are mutually orthogonal and have spectral functions
(5.15)
To prove this we set
The functions FU(A), F 22 (A) and F 12 (A) satisfy the conditions ofthe sec-
ond part of Theorem 6. Therefore, by Theorem 7 there exists in H a sequence
{Yl(t)} that is stationary and jointly stationary with respect to {x(t)} and is
such that
Setting
Y2(t) = x(t) - Yl(t),
and using simple computations we see that {Yl(t)} and {Y2(t)} are mutually
orthogonal and satisfy (5.15).
Theorem 12 gives conditions that shouldbe imposed on F 1 (A) and F2(A)
in the representation (5.13) in order to make {Yl(t)} and {Y2(t)} sub ordinate to
{x(t)}, that is, to ensure that they lie in H x • Under these additional conditions
the requirement that the dimension of the complement He H x be infinite is
superfluous,4 and it is possible to strengthen the conclusion of Theorem 13 so

that the representation (5.14) is uniquely 5 determined by (5.13).
We obtain the following theorem:
Theorem 14. For any stationary sequence {x(tn and for each representation
of the spectral function Fxx(.~) in the form
where F1(A) and F 2(A) are real non-decreasing functions of A, continuous from
the right with F 1 (-1r) = F2(-1r) = 0, and almost everywhere with respect to
Fxx ,
dFl + dF2 = 0, (5.16)
dFxx dFxx
there corresponds one representation {x(tn in the form
x(t) = Yl(t) + Y2(t),
for which {Yl (tn and {Y2(tn are mutua//y orthogonal and
The sequences {Yl (tn and {Y2(tn are subordinate to {x(tn.
§6. Sliding summation
Definition 6. We say that a sequence {x(tn is obtained from a stationary

sequence {u(tn by sliding summation if we can choose coefficients an such
that
L anu(t -
+00
x(t) = n), (6.1)

-00
where the series on the right-hand side converges in norm.
4 Indeed, consider an extension H* of the space H for which H* e H is infinite

dimensional. By Theorem 13 the desired sequences {Yl(t)} and {Y2(t)} exist in
H* while under the additional condition of Theorem 12 they belong to H x .
5 This follows from the fact that according to Theorem 8, the functions <p~~) =
dFt/dFxx and <p~~) = dF2/dFxx uniquely define sequences {Yl(t)} and {Y2(t)}
sub ordinate to x(t).
Clearly, a sequence {x(t)} obtained by sliding summation from a stationary

sequence {u(t)} is always stationary and subordinate to {u(t)}. In this section
we consider a particular case for which the converse statement is also true.
Definition 7~ A sequence {u(t)} of elements of H is called fundamental if it is
stationary and satisfies the conditions
Buu(O) = 1, Buu(k) = 0 for k :F O. (6.2)

By (3.9) and (3.10) it is easy to compute that for every fundamental se-
quence {u(t)} ,
FUU(A) = (A + 7r)/27r. (6.3)
Conversely, every stationary sequence {u(t)} with a spectral function of the
form (6.3) is fundamental.
The elements of a fundamental sequence {u(t)} form a complete orthonor-
mal system in H u . Therefore, each element z of H u can be uniquely represented
as
L
+00
Z= Ctu(t), (6.4)
t=-oo
where
L
+00
c~ = IIzII 2 < +00. (6.5)

t=-oo
The coefficients Ct in (6.4) are determined by the equalities
Ct = (z, u(t)). (6.6)
Theorem 15. A stationary sequence {x(t)} is obtained /rom a fundamental

sequence {u(t)} by slidiag summation if and only if {x(t)} is subordinate to
{u(t)}. If this condition holds, then the unique representation of {x(t)} in the
form (6.1) is given by the formula
L
00
x(t) = Bxu(n)u(t - n). (6.7)

n=-oo
The necessity of the condition of the theorem has already been proved. If
this condition holds, then every element x(t) of the sequence {x(t)} belongs to
H u and, by (6.4) and (6.6), is representable in the form
L L
00 00
x(t) = (x(t), u(s))u(s) = Bxu(t - s)u(s)

3=-00
STATIONARY SEQUENCES IN HlLBERT SPACE 251
or, setting t - s = n, in the form (6.7).

Taking into account the uniqueness of the representation of any element
in the form (6.4), we see that each representation of {x(t)} in the form (6.1) is
the same as (6.7).
Note also that a comparison of (6.7) with (6.4) and (6.5) yields
L
00
IBxu (n)1 2 = IIx(t)11 2 = Bxx(O). (6.8)

n=-oo
Formulas (4.3), (4.4) and (6.3) imply that
Fxx(A) = 2~ 1~ 14>1x )(AWdA, (6.9)
Fxu(A) = 2~ 1~ 4>1u)(A)dA, (6.10).
Formula (6.9) shows that 4>1u )(A) is a square integrable function. It follows
from (3.3) and (6.10) that
Bxu(n) = -2
1
7r
1 -lr
lr
einA 4>1u )(A)dA. (6.11)
Thus the Bxu(n) are the Fourier coefficients of 4>1u )(A), that is,
L
00
4>1u )(A) '" Bxu(n)e-inA. (6.12)

n=-oo
With regard to the above formulas we note that, by (6.3), the words "al-
most everywhere with respect to F uu " mean "exduding a set of Lebesgue mea-
sure zero" and the dass L~ coincides with the dass L 2 of square integrable
functions in the usual sense of Lebesgue.
Theorem 16. In order that a stationary sequence {x(t)} can be obtained by

sliding summation from some fundamental sequence it is necessary that the
spectral function Fxx(A) be absolutely continuous.
If He H x is infinite dimensional, then this condition is also sufficient.
The necessity of the condition follows from (6.9). To prove its sufficiency
we consider an arbitrary function ,(A) in L 2 for which, almost everywhere on
-7r ~ A~ 7r,
(6.13)
and define Fll (A), F22 (A) and F12 (A) by the formulas
Fll (A) = F.,.,(A),
It is easy to see that these functions satisfy the conditions of the second
part of Theorem 6. By Theorem 7, if H e H., is infinite dimensional, then
there exists a sequence {u(t)}, stationary and jointly stationary with {x(t)} ,
for which
(6.14)
(6.15)
Formula (6.14) shows that {u(t)} is a fundamental sequence. The absolute

continuity of F.,.,(A) and formula (6.13) imply that
(6.16)
By Theorem 9, we conclude from (6.15) and (6.16) that {x(t)} is subordi-

nate to {u(t)}. By Theorem 15 this implies that {x(t)} can be obtained from
{'!l(t)} by sliding summation.
Theorem 17. A stationary sequence {x(t)} can be obtained by sliding summa-

tion from a fundamental sequence subordinate to {x(t)} if and only if F.,.,(A)
.is absolute1y continuous and the function
f.,.,(A) = dF.,.,(A)/dA (6.17)
is almost everywhere positive on -'Tr :::; A :::; 'Tr.
If these conditions are satisfied, then:

A. Every fundamental sequence from wh ich {x(t)} can be obtained by slid-
ing summation is subordinate to {x(t)}.
B. The sequence {x(t)} can be obtained by sliding summation from any
fundamental sequence subordinate to it.
C. A sequence {'!let)} subordinate to {x(t)} is fundamental if and only if
(6.18)
almost everywhere on -1r ~ A ~ 1r.

Regarding (6.18), note that if Fzz(A) is absolutely continuous and Izz(A) is
positive almost everywhere, then the dass of sets of measure zero with respect
to F zz coincides with the dass of sets of Lebesgue measure zero. Therefore,
functions of dass L;, in particular f/J~z), are defined to within a set of measure
zero in the sense of Lebesgue.
Proolol Theorem 17.

10 • The necessity of the absolute continuity has already been established
in Theorem 16. Let us prove that Izz must be positive almost everywhere. If
a fundamental sequence {u(t)} is subordinate to {x(t)}, then by Theorem 9,
- - = Fuu(A) = jA
A+1r If/Ju(z) (A )1 2dFzz ()
A, (6.19)
21r _ ...
which implies after differentiation with respect to A, that almost everywhere
(6.20)
It is dear from (6.20) that Izz is positive almost everywhere.

20 • Now assurne that Fzz is absolutely continuous and Izz is positive
almost everywhere. By Theorem 16, the absolute continuity of F zz on its own
implies that {x(t)} can be obtained by sliding summation from a fundamental
sequence {u(t)} belonging to an extension H' of Hz. By (6.9), we have almost
everywhere
Hence by Theorem 10, {u(t)} is sub ordinate to {x(t)} (that is, actually
belongs to Hz itself). Thus, we have proved the sufficiency of the conditions of
the Theorem and with it, the additional assertion A.
A sequence {u(t)} subordinate to {x(t)} is fundamental if and only if
that is, if almost everywhere
(6.21)
This proves C.
By (6.21) 1~~)(.\)12 :F 0 almost everywhere for any fundamentalsequence

{u(tn sub ordinate to {z(tn. By Theorem 10 this implies that {z(tn is sub-
ordinate to {u(tn, and hence, by Theorem 15, {z(tn is obtained from {u(tn
by sliding summation. This proves B.
§7. Wold decomposition
For a stationary sequence {z(tn we denote by Hz(t) the smallest cl08ed linear
subspace of Hz containing all the z(s) for s $ t, and let S~ be the intersection
of all the Hz(t). Clearly,
u; Hz(t) = Hz(t + k), (7.1)

U;Sz =S~. (7.2)
Definition 8. A stationary sequence {z(tn is called singular if
(7.3)
Clearly, for a singular sequence {z(tn,
(7.4)
for any t. Conversely, if (7.4) is true for some t, then {z(tn is singular. Indeed,
if (7.4) holds for one t, then by (7.1), it also holds for all other t, whence it
follows that S~ = Hz.
Let sz(t) be the projection of z(t) onto S~. It is easy to see that {sz(tn
is a singular stationary sequence subordinate to {z(tn for which
(7.5)
The sequence {s~(tn is called the singular component of {z(tn.

Now ass urne that {z(tn is non-singular. Then z(t) can be uniquely rep-
resented as
z(t) = ~(t) + ~(t), (7.6)
where ~(t) belongs to H~(t - 1) and ~(t) :F 0 is orthogonal to H~(t - 1). Set
u~(t) = ~(t)/II~(t)lI. (7.7)

Clearly the elements
Uz(t), uz(t - 1), ... , uz(t - n) ...
form a complete orthonormal system in the space
Since z(t) belongs to Hz(t), it follows that z(t) can be uniquely represented in
the form
E c~z)uz(t -
00
z(t) = sz(t) + n). (7.8)

n=O
Since {uz(t)} is stationary and subordinate to {z(t)}, the coefficients
(7.9)
in (7.8) do not depend on t. It follows from (7.6) and (7.7) that
Co = 1I~(t)1I > o. (7.10)
The representation of a non-singular sequence in the form (7.8) is called

the W old decomposition. The above arguments make it dear that
s(t) = Sz(t), u(t) = uz(t), Cn -- c(z)

n
have the following properties:

Wl) the sequence {s(t)} is singular and subordinate to {z(t)}j
W2) {u(t)} is a fundamental sequence subordinate to {z(t)}j
W3) u(t) belongs to Hz(t)j
W 4) the sequences s(t) and u(t) are mutually orthogonalj
Ws) co> o.
The following theorem shows that the decomposition (7.8) is uniquely de-
termined by these five properties.
Theorem 18. If a stationary sequence {z(t)} is represented as

00
z(t) = s(t) + Ecnu(t - n), (7.11)

n=O
and i!{s(t)} and {u(t)} and the coefficients Cn satisfy the conditions W 1 )-W 5 ),
then {x(t)} is non-singular and
s(t) = sx(t), u(t) = ux(t), Cn = c~x).
Proof. Since x(t') and an the u(t' - n) belong to Hx{t') for n = 0,1,2, ... (by
W 3 ), then so does
L:
00
s(t') = x(t') - Cn u(t' - n).

n=O
Since Hx(t') is contained in Hx(t) for t' ~ t, all the s(t') belong to Hx(t) for
t'~ t. Hence H,(t) is contained in Hx{t). Since S, = H,(t) (by W 1 )), S, is
contained in all Hx(t) and hence in Sx. This implies that all the s(t') belong
to Sx and hence to an Hx(t).
We rewrite (7.11) as
x(t) = (t) + cou(t), (7.12)
where
L:
00
(t) = s(t) + Cn u(t - n).

n=l
Since all the s(t) and an the u(t - n) for n = 1,2,3, ... belong to Hx{t - 1),
it fonows that (t) belongs to Hx(t - n). Since for t' < t an the x(t') can be
written as linear combinations of elements s(t") and u(t") with t" < t and u(t)
is orthogonal to an s(t") and u(t") for t" < t, we find that u(t) is orthogonal
to an x(t') for t' < t, and hence to the space Hx(t - 1).
Since (t) belongs to Hx(t - 1) and u(t) is orthogonal to Hx(t - 1), com-
paring (7.12) with (7.6) we see that
(t) = e(t), cou(t) = ~(t). (7.13)
Comparing (7.13) with (7.7) and taking W5) into account we obtain
u(t) = ux(t). (7.14)
By W2) and W 4 ), (7.11) implies that
cn = [x(t), u(t - n)]. (7.15)

From (7.14) , (7.15) and (7.9) we obtain
Cn = c~z). (7.16)
Comparing (7.8) with (7.11) and taking into account (7.15) and (7.16) we finally
obtain
S(t) = sz(t).
§8. Regular sequences
Theorem 19. A stationary sequence {z(t)} can be represented in the form

00
z(t) = ~ cnu(t - n), (8.1)

n=O
where {u(t)} is a fundamental sequence, if and only if
Sz(t) = O. (8.2)
If z(t) = 0, then the condition sz(t) = 0 is fulfilled and the representation

(8.1) is possible (with all Cn = 0). Now we have only to consider the case
z(t) I O. The sufliciency of the condition sz(t) = 0 in this case is also dear: if
sz(t) = 0 and z(t) = 0, then the sequence z(t) is non-singular and, according
to (7.8) of the previous section, can be represented in the form
00
z(t) = ~ c~z)uz(t - n). (8.3)

n=O
Let us prove the necessity of (8.2). For this purpose note that if there
exists a representation (8.1), then the space Hz(t) is contained in the space
Hu(t) and Sz in Su. Therefore, if the equality
Su = 0 (8.4)
is true for every fundamental sequence, then Sz = 0 and hence sz(t) = o.

Definition 9. A stationary sequence {z(t)} is said to be regular if
z(t) I 0, sz(t) = O.
If {U{t)} is a fundamental sequence, then (8.1) is a particular case of the

representations studied in detail in §6. Therefore, in accordance with (6.7),
(6.8) and (6.12),
Bxu(n) = Cn for n ~ 0, Bxu(n) = 0 for n < 0, (8.5)
L: ICn
00
12 = Iz(tW = Bxx(O), (8.6)

n=O
L: cne- in >..
00
~~u)(A) = (8.7)
n=O
By (8.6), the series
L cn(n
00
r;«() = (8.8)
n=O
represents an analytic function in the disk 1(1 < 1. By (8.7), its boundary
values on I( I = 1 are given by the formula
(8.9)
The condition z(t) :j; 0 implies that r:«() cannot vanish identically. There-
fore,6 almost everywhere on 1(1 = 1,
(8.10)
Since by (6.9)
(8.11)
almost everywhere, (8.10) implies that almost everywhere
fxx(A) > O. (8.12)
By Theorem 17 we conclude that {u(t)} is subordinate to {z(t)}.
Theorem 20. 1f a regular sequence {z(t)} can be represented in the form (8.1)
with a fundamental sequence {u(t)}, then {u(t)} is subordinate to {z(t)}.
6 See, for example, 1.1. Privalov's monograph [5], p.39.

A stationary sequence {x(t)} subordinate to a fundamental sequence {u(t)}

can be represented in terms of {u(t)} in the form (8.1) if and only if the function
'Y1u )«(), dejined for 1(1 = 1 by
(8.13)
coincides almost everywhere on the circle 1(1 = 1 with the boundary values of
the analytic function d U
)«(), dejined for 1(1 < 1 by
(8.14)
1fthis condition holds, then the coefficients Cn of(8.1) can be determined from
(8.8).
The first part of the theorem, the necessity of the conditions in the second
part of the theorem, and the assertion on the form of dependence between the
coefficients Cn and the function r~u)«() have been proved above.
It remains to prove the sufficiency of the conditions of the second part of
the theorem.
If these conditions hold, then for the boundary values of d u )«() we have
the Fourier series expansion
L cne- in>..
00
'Y~u)(e-i>') = ~1u)(A) '" (8.15)

n=O
Comparing (8.15) with (6.12) and (6.7) we conclude that x(t) can be rep-
resented in the form (8.1) with coefficients Cn determined from (8.15) or, what
is the same, from (8.8). This completes the proof of Theorem 20.
Theorem 21. For any regular sequence x(t), the function
= r~U.,)«() = L
00
r x«() c~x)(n (8.16)

n=O
does not vanish inside the disk 1(1 < 1.

Assurne that, on the contrary, r x «() vanishes at a (0 with 1(1 < 1. We
construct a sequence {u(t)} subordinate to {ux(t)} such that
~(uz)(A) = e- i >' - (? . (8.17)

u 1 - (oe-I>'
This sequence exists by Theorem 8 since the function
is bounded and is therefore of elass L!,s = L2.

It is easily checked that for any A
(8.18)
Now
d dA+'Ir 1
fu,suzCA) = dA FU,su,s(A) = dA ~ = 2'1r'
Therefore according to assertion C of Theorem 17, (8.18) implies that {u(t)}
is a fundamental sequence.
i:
By (4.5),
Fss(A) = J~ q,~uz)(A)4>~uz)(A)dFuzuz(A) = 2~ q,~uz)(A)4>~uz)(A)dA.

(8.19)
It follows from (4.6) that almost everywhere
.I.(U)(A)
'l's
= dFuu(A)
dFsu(A) = 2 dFsu(t)
'Ir dA
= .I.(uz)(A)J;(uz)(A)
'l's 'l'u . (8.20)
By (8.18),
Therefore (8.20) can be written as
(8.21)
Now, (8.9) and (8.16) imply that
q,~uz)(A) =f~uz)(e-i>') = fs(e- i>'). (8.22)
From (8.21) , (8.22) and (8.17) we conelude that
(8.23)
It is easy to see that

coincides almost everywhere with the boundary values of the function
(8.24)
for '(I = 1.
Since (0 is a zero of r :r:«), it follows that r~u)«) is analytic for '(I < 1.
By Theorem 20 this implies that 7
L: cnu(t -
+00
z(t) = n), (8.25)

n=O
where the coefficients Cn are determined from (8.8). The function
coincides almost everywhere on the cirele 1(' = 1 with the boundary values of
the function
r(US)(I") = (- (0 (8.26)
U .. 1- (0('
which is analytic in the disk '(I < 1. Therefore, according to Theorem 20
L: dnu:r:(t -
00
u(t) = n), (8.27)

n=O
where the dn are the coefficients of the Taylor series of the function rLus)«).
Formula (8.27) shows that u(t) belongs to Huz<t). Since by property W 3 ) of
the sequence {u:r:(t)} (see §7), H(us)(t) lies in H:r:(t), we see that u(t) also lies
in H:r:(t).
This implies that for n > 0 all the u(t - n) belong to H:r:(t - n). Therefore,
comparing (8.25) and (7.6) we obtain
Cou(t) = ~(t),
and, by (7.7) and (7.10),
7 For Theorem 20 to be applicable it is essential that r~u)(c) can be represented as

a Cauchy integral in terms ofthe boundary values 'Y~u)(e-i~). This holds, as we
can see, for example, from Remark C on p. 94 of Riesz's paper [6], since, clearly
r~u)(c), as weIl as r:r:(C), belong to the class H1 in the sense ofthat paper.
It can easily be verified that the sequence {u(t)} defined by the formula
must correspond to the function
(8.28)
The contradiction between (8.28) and (8.17) proves Theorem 21.
Theorem 22. A stationary sequence {z(t)} is regular if and only if the fol-
lowing conditions hold on -7r $ A $ 7r:
1) Fxx(.~) is absolutely continuous;
2) fxx(A) is positive almost everywhere;
3) log fxx(A) is summable.
If all these conditions hold, then
(8.29)
(x) 00
Qx«() = a~ + ~)ar) + ib~x»(k, (8.30)

k=l
where the coefficients ak and bk are determined from the expansion

(x) 00
!logfxx(A) '" a~ + ~)a~x)coskA+b~x)sinkA). (8.31)

k=l
Since every regular sequence {z(t)} can be obtained by sliding summa-

tion from a subordinate fundamental sequence {ux(t)} of it, the necessity of
conditions 1 and 2 follows from Theorem 17. Let us prove the necessity of con-
dition 3. If {x(t)} is regular, then by Theorem 21, r x «() is analytic and does
not vanish in the disk 1(1 < 1. Denote by Qx«() the branch oflog(r x «()/v'21r)
obtained by analytic continuation from the real value
r x(O) c~x)
log . Jn= = log. Jn=.
V 27r v27r
Since r x«() has no zeros, the function Qx«() is uniquely defined at all
points of the disk 1(1 < 1. For the real part of Qx«() we have, by (8.29),
(8.32)
Let Re+ Qx(() be the function equal to ReQx(() for ReQx(() > 0 and
equal to 0 otherwise. Since (8.32) implies
it follows that for any p <1
Since
we find that
1 : IRe Qx(pe-iA)ld,X = 1)2Re+ Qx(pe- iA ) - Re Qx(pe-iA)]d'x ~

~ 2K - 211'ReQx(O).
The latter inequality implies that the boundary values
of Re Qx( e- iA ) are integrable with respect to'x. This proves the necessity of
the integrability of log fxx('x), that is, condition 3 of our theorem.
Before passing to the proof of the sufficiency of the conditions of the the-
orem, let us show that for any regular sequence {x( t)} the function r x (() is
determined by (8.29)-(8.31). Since, according to what was proved above, the
function ~logfxx('x) is integrable for a regular sequence {x(t)}, it can be ex-
panded in a Fourier series (8.31). Formulas (8.33) and (8.31) imply that
(x) 00
ReQx(eiA).-v a~ + ~)a~x)cosk'x-br)sink'x). (8.34)

1:=1
Since
(8.35)
is real, the imaginary part ImQ1:(ei>·) is given by

00
ImQ1:(ei>.) -l:(a~1:) sin kA + b~1:) eos kA). (8.36)

"=1
Formulas (8.34) and (8.36) imply (8.30). Finally, (8.29) follows from the
definition of Q1:«().
Now let us prove the suffieieney of the eonditions of the theorem. For this
we assurne that these eonditions hold, and we set
(1:) 00
Q«() = a~ + l:(a~1:) + ib~1:»)(", (8.37)

"=1
r«() = .,fiie Q ({), (8.38)
where the eoeffieients a~1:) and b~1:) are determined from the expansion (8.31).
Then 8
(8.39)
where
P (0) _ 2. 1 - p2 (8.40)
P - 211" 1 + p2 - 2p eos 0 .
The inequality between the geometrie mean and the arithmetie mean 9 implies
that
(8.41)
Sinee 11:1:(>') is integrable and J~7r Pp(J-l- >')dJ-l = 1, by (8.41) we have
8 See [5], p.15.

9 We apply this inequality in the following form: if
m= 1 6
P(x)f(x)dx, logs = 1 6
P(x) log f(x)dx, 1
6
P(x)dx = 1,
P(x) ~ 0, f(x) ~ 0,
then s ::; m.
The boundedness of the integral on the left-hand side of (8.42) thus estab-
lished guarantees 10 that r(e) can be represented as the Cauchy integral of its
boundary values
r(e- i >.) = cf>(A).
Formulas (8.38) and (8.31) imply that
1cf>(A)1 2 = lf(e-i>.)1 2 = 211"exp(2ReQ(e- i >'» = 211"/:c:c(A). (8.43)
Therefore,
11" 1
1 -11" 1cf>(A)12dF:c:c(A) = 1
and consequently l/cf>(A) belongs to the dass L~. Let us find a sequence {u(t)}
subordinate to {z(t)} for which
Equation (8.43) and Theorem 17C imply that {u(t)} is a fundamental sequence
equivalent to {z(t)}. By Theorem 10,
Thus, {u(t)} satisfies all the conditions of the second part of Theorem 20.
Hence {z(t)} can be represented in terms of {u(t)} in the form (8.1), which
proves the sufficiency of the conditions of Theorem 22. It is easy to see that
the sequence {u(t)} so constructed in fact coincides with {u:c(t)}.
In condusion we note the formula 11
c~:c) = "f2;e Q z<O) = "f2;exp +=

a<:c)
"f2;exp (411"
1 111"
-11" 10g/:c:c(A)dA) , (8.44)
which follows from (8.30) and (8.35).
9. Spectral characterization of singular sequences

and singular components
By the results in §7, every non-singular stationary sequence {z(t)} can be

represented as
(9.1)
10 See Remark C on p.94 of [6].

11 This formula holds not only for regular, but also for all non-singular sequences,
as will be dear from Theorem 23 of the following section.
where
= L: C~")U.,(t -
00
r.,(t) n). (9.2)

n=O
Let us call {r.,(t)} the regular component 12 of {x(t)}. Clearly, it is always

subordinate to {x(t)} and is regular.
For any stationary sequence we set
A.,.,(") = i),7f I.,.,(")d", (9.3)
D.,.,(") = F.,.,(") - A.,.,("). (9.4)
We will distinguish the following three cases:

1) 1.,.,(") = 0 on a set of positive measure;
2) 1.,.,(") = 0 only on a set of measure zero but log 1.,.,(") IS non-
integrable;
3) 1.,.,(") = 0 only on a set of measure zero and log 1.,.,(") is integrable.
Theorem 23. In cases 1) and 2) the sequence {x(t)} is singular. In case 3)

{x(t)} is non-singular,
F,x''''('') = D.,.,("), Frxrx (") = A.,.,(") (9.5)
and the coefficients c~.,) in (9.2) are determined lrom (8.16), where r .,«) is
defined by (8.29)-(8.31).
Proof. Assume that {x(t)} is non-singular. Then by the regularity of {r.,(t)}

and Theorem 22, Irxrx (") is positive almost everywhere and log Irxrx (") is
integrable. Since {s.,(r)} and {T.,(t)} are orthogonal to each other (see §7),
Theorem 11 implies that
(9.6)
and consequently, almost everywhere
1.,.,(") = 1,,,,,,,,(") + Ir",r",("). (9.7)
12 Since the sequence identically equal to zero is singular, but not regular, the
singular component of a regular sequence equals 0 and a singular sequence has
no regular component.
It follows from (9.7) that almost everywhere
(9.8)
Therefore, every set of positive Lebesgue measure has positive measure also
with respect to F.,.,(A).
Since {S.,(t)} and {r.,(t)} are subordinate to {z(t)}, Theorem 12 implies
that
dF6Z6z (A) dFrzrz(A) = 0
dF.,., (A) dF.,., (A)
almost everywhere with respect to F.,.,(A), and hence, in the sense ofLebesgue.
Since by (9.8) almost everywhere
it follows that almost everywhere
that is,
(9.9)
(9.10)
Formula (9.10) and the facts that frzrz(A) is positive almost everywhere
and log frzrz (A) is integrable imply that for a non-singular sequence case 3)
always holds.
Let us show that, conversely, in case 3) {z(t)} is non-singular. According
to Theorem 14 in case 3) {z(t)} can be uniquely represented as a sum
z(t) = s(t) + r(t)
of orthogonal sequences subordinate to {z(t)} for which
By Theorem 22, {r(t)} is regular. Therefore,
= s(t) + l: C~)Ur(t -
00
z(t) n). (9.11)

n=O
268 STATIONARY SEQUENCES IN HIT.BERT SPACE
Formula (9.11) shows that in this ease HIIJ(t) is eontained in the spaee 13
S. $ Hr(t). Representing z(t + 1) as
z(t + 1) = Ot + ß,
where
E C~)Ur(t -
00
Ot = c~r)Ur(t + 1), ß = set + 1) + n + 1),

n=1
it is easy to see that ß belongs to SIIJ $ Hur(t) and Ot ::fi 0 is orthogonal to this
spaee. Therefore z(t + 1) does not He in SIIJ $ Hur(t) and henee does not He in
HIIJ(t). This implies that {z(t)} is non-singular.
We have proved (9.10) for any non-singular {z(t)}. Therefore r llJ «() deter-
mined in aeeordanee with (8.29)-(8.31) eoincides with rr,.«(). To prove that
the eoefficients c~lIJ) ean be obtained from the expansion (8.16) of r IIJ«()' it only
remains to establish that
(9.12)
For this it suffiees to show that (9.2) satisfies Wt}-W5) (with zero singular
eomponent). Clearly Wl), W2), W4) and W5) hold. Let us prove that W3)
holds for (9.2). For this we note that by (9.8) and Theorem 17A, {UIIJ(t)} is
subordinate to {rllJ(t)}. From (9.1) and (9.2) it is dear that HIIJ(t) is eontained
in SIIJ $ Hr,.(t). Sinee UIIJ(t) belongs to HIIJ(t) and is orthogonal to SIIJ (see §7),
ulIJ(t) belongs to Hr,.(t), whieh means that (9.2) satisfies W3).
For a non-singular sequenee, (9.5) follows immediately from (9.1), (9.10)
and the absolute eontinuity of Fr,.r,. (the latter is ensured by the regularity of
{rllJ(t)}, see Theorem 16).
§10. Minimal sequences

For any stationary sequenee {z(t)} denote by HIIJ(t) the smallest dosed linear
subspaee of H I J eontaining aH the z(s) for s::fi t. Sinee
(10.1)
only one of two eases is possible: either aH the HIIJ(t) eoincide with H I J or all
the HIIJ(t) are different from HIIJ.
13 6 1 Efl ~ denotes the minimal linear closed subspace of H containing both 61

and ~.
Definition 10. A stationary sequence is called minimal Ü
(10.2)
Each element z(t) of the sequence {z(tn can be uniquely represented as

asum
z(t) = v(t) + 6(t), (10.3)
where v(t) belongs to H:t:(t) and 6(t) is orthogonal to H:t:(t). Set
dz = 16(t)l. (10.4)
Clearly, the condition

dz> 0 (10.5)
is necessary and sufficient for {z(tn to be minimal.
Theorem 24. A stationary sequence {z(tn is minimal il and only ill:t::t:(A)

is positive almost everywhere and the integral
1 11'
-11'
d>'
I:t::t:(>')
(10.6)
is finite.
11 these conditions hold, then
2 ( )2 111' d>' (10.7)

d:t: = 211" : -11' I:t::t:(>')'
First we prove the sufficiency of the conditions of the theorem. Let
<p(>') = 1 : I:t::t:(>') (1 11'

-11"
d>' )1/2
1:t::t:(A) ,
if I:t::t:(>') is defined and finite, and
<p(>') = 0,
if I:t::t:(>') is not defined (and is infinite). Then

Hence 4>(.\) belongs to the dass L~ and the equality
defines a certain sequence {y(t)} subordinate to {x(t)} for which, according to

(4.4),
1:
By (3.3), for 8 =I t,
(y(t), X(8)) = Byz;{t - 8) = ei (t-8)"'dFyz;(.\) = o.

Hence the element y(t) is orthogonal to the space Hz;(t). Since
the space Hz;, which contains y(t), cannot coincide with Hz;(t). This proves the
minimality of {x(t)}.
To prove the necessity ofthe conditions of the theorem, assurne that {x(t)}
is minimal. It is easy to see that {6(t)} is stationary and subordinate to {x(t)}.
Then
B6z;(k) = (6(t + k), x(t)) = 0 for k =I 0, (10.8)

B6z;(0) = (6(t),x(t)) = 16(tW = d~. (10.9)
By (3.9)-(3.11) this implies that
(10.10)
At the same time, by (4.4),
(10.11)
Formulas (10.10) and (10.11) imply that
(10.12)
and consequently, the derivative
(10.13)
is finite almost everywhere with respect to Fxx(.~). Since by (10.12) A is ab-

solutely continuous with respect to F.,.,(A), the derivative dAjdF.,.,(A) is finite
almost everywhere with respect to A, that is, in the sense of Lebesgue. There-
fore the derivative
is non-vanishing almost everywhere. Taking this into account, it follows from

(10.13) that almost everywhere
According to Theorem 8, the integral
1: 14>~")(AWdF.,.,(A) = 1:127rf~:(A) 2
1 dFxx(A) = (:;)21: f.,~~A)
(10.14)
is finite. This proves the necessity of the conditions of the theorem. Noting
that
(10.15)
we obtain (10.7) from (10.14) and (10.15).
References
1. H. Wold, A study in the analysis 0/ stationary time series, Uppsala, 1938.

2. H. Cramer, Ann. 0/ Math. 41 (1940), 215-230.
3. M. Stone, 'Linear transformations in Hilbert space', Amer. Math. Soe.
Colloq. Publ. 15 (1932).
4. A. Zygmund, Trigonometrie senes, Cambridge Univ. Press, 1979, 2 vols.
5. LI. Privalov, The Cauehy integral, Saratov, 1919 (in Russian).
6. F. Riesz, Math. Z. 18 (1923), 87-95.
7. A.N. Kolmogorov, Izv. Akad. Nauk SSSR Sero Mat. 5 (1941), 3-14 (in
Russian) (No. 28 in this book).
28. INTERPOLATION AND EXTRAPOLATION OF STATIONARY
RANDOM SEQUENCES *
Spectral conditions are established for the possibility of extrapolating and in-
terpolating stationary random sequences by a sufficiently large number of terms
with any prescribed accuracy.
Introduction
For each integer t (-00 < t < +00) let x(t) be areal random variable whose
square has finite expectation. The sequence of random variables x(t) will be
called stationary if the expectations 1
m = Ex(t)
and
B(k) = E[(x(t + k) - m)(x(t) - m)]
do not depend on t. Without loss of generality we may set
m = Ex(t) = o. (1)
Then
B(k) = E[x(t + k)x(t)]. (2)
Since
B(-k) = B(k), (3)
it suffices to consider the second moments B(k) only for k ~ o.
The problem of linear extrapolation of a stationary sequence satisfying (1)
is to select for given n > 0 and m ~ 0 real coefficients a, for which the linear
combination
of random variables
x(t -1), x(t - 2), ... ,x(t - n)
* Izv. Akad. Nauk SSSR Sero Mat. 5 (1941), 3-14.

1 The expectation of a random variable y will be denoted by Ey.
272
INTERPOLATION OF STATIONARY RANDOM SEQUENCES 273
gives the closest possible approximation to the random variable z(t + m). It is
natural to take the expectation
n n n
(T2 = E(z(t + m) - L)2 = B(O) - 2 I:B(m + s)a. + I: I: B(p - q)apaq
p=lq=l
as the measure of accuracy of such an approximation.
If the second moments B(k) are known, then it is easy to find the coeffi-
dents a. for which (T2 takes the smallest value. This smallest value of (T2 will
be denoted by (T~ (n, m).
Clearly, (T~ (n, m) cannot increase with n. Therefore the limit
lim (T~(n, m) = (T~(m) (4)

n-oo
exists. To find this limit is the first problem we solve in this paper.
As for the interpolation problem, we will consider only the case of esti-
mating z(t) from
z(t + 1), z(t + 2), ... , z(t + n),
z(t - 1), z(t - 2), ... , z(t - n).
For this case, let (TJ(n) be the minimal expectation
where Q is a linear form

Q = alz(t + 1) + a2z(t + 2) + ... + anz(t + n)+
+ a_lz(t - 1) + a_2z(t - 2) + ... + a_nz(t - n)
with constant real coeffidents a •.
Since (TJ(n) does not increase with n, the limit
(5)
exists.
Our second goal is to determine (TJ. The solution to both these problems
was announced in my paper [1].2 It uses notions relating to the spectral theory
of stationary random processes.
2 There is amisprint in formula (1) of [1]. Th, correct form of (1) is:
lim tTJ(n)
n_oo
=r : 1'" d(~).
0 8 A
274 INTERPOLATION OF STATIONARY RANDOM SEQUENCES
The spectral theory of stationary processes was constructed by A.Ya. Khin-

chin [2] for the case of continuous variation of the time argument t. The theory
for the case we are interested in now, that is, of a discrete stationary sequence,
is given in detail in Wold's book [3]. Of principal importance here is the
following 3
Theorem 1. For any stationary sequence {x(t)} the second moments B(k)
can be represented in the form
B(k) = - 11"" COSUdW(A),

'Ir 0
(6)
where W(A) is the non-decreasing real function defined by the formula
W(A) = B(O)A + 2 f: B~k)

k=l
sin kA. (7)
The derivative
W(A) = dW(A)jdA
of the non-decreasing function W(A) exists almost everywhere, is non-negative
and summable. Since
logw(A) ~ W(A),
it follows from the summability of W(A) that the integral
p = -11""
0
'Ir
log w(A)dA (8)
is either finite or equal to _00. 4 Further, we prove the following:
Theorem 2. If P = -00, then u1{m) = 0 for all m ~ O. If the integral P is

finite, then
u1{m) = eP (1 + r; + r~ + ... + r~), (9)
where the ri are determined from the formulas
e a1 (+a 2(2+ ... = 1 + r1( + r2(2 + ... , (10)
ak = -11""
0
'Ir
cosUlogw(A)dA. (11)
Since W(A) ~ 0, the integral
R =;: Jo
1 r W(A)
dA
(12)
is either finite or equal to +00. 5 Then we prove the following:

3 See [3], §17.
4 Ir w(>.) = 0 on a set of positive measure, we set P = -00.
5 Ir w(>.) = 0 on a set of positive measure, we set R = +00.
Theorem 3. If R = +00, then uj = O. If the integral is finite, then
uj = I/R. (13)
In my paper [4] I constructed a theory of stationary sequences of elements

of a complex Hilbert space. In §1 of the present paper it is shown that stationary
random sequences in the sense defined above can be considered as a special case
ofthe stationary sequences considered in [4]. This allows us to obtain the above
Theorems 1, 2 and 3 as simple corollaries of the results of [4].
In what follows, references to formulas indexed by two numbers (for ex-
ample, (8.44)) apply to formulas from [4].
§1. Stationary random sequences and Hilbert space geometry
Our discussion will be based on the axioms and the construction of the basic
not ions suggested in my book [5], with one difference: our random variables
can take not only real, but also complex values. 6
Consider the set Jj of all random variables of a certain Borel probability
field (F, P) with finite expectation of the squared absolute value, regarding
equivalent random variables (that is, random variables that differ from each
other with prob ability zero) as identical. We introduce in Jj the scalar product
(x,y) = E(xy). (14)
The norm in Jj is given by
(15)
Summation of elements in Jj and their multiplication by a complex number is

understood as usual.
It is easy to verify that under these definitions the set Jj satisfies the axioms
A, Band E from M. Stone's book [6], that is, all the axioms of an abstract
unitary space.
Now let {x(t)} be a stationary sequence of real random variables x(t) of
the probability field (F, P) in the sense adopted in the Introduction, satisfying
6 A complex function x(e) defined on the set E of elementary events e is called

e
a random variable if for any choice of real numbers a and b the set of all for
which the real and imaginary parts of x(t) satisfy Re x(e) < a and Im x(e) < b
respectively, belongs to the system F.
the additional eondition (1). Then, by (2) and the fact that z(t) is real, we
have
B(k) = E[z(t + k)z(t)] = (z(t + k), z(t».
Since, by definition, B(k) does not depend on t, {z(t)} is a stationary sequenee
of elements of" in the sense of [4].
In [4] I eonsider stationary sequenees in a Hilbert space, that is, in the
spaee satisfying not only the Axioms A, B and E, but also C and D from [6].
However, this restriction is inessential. Indeed, denoting by Hz the smallest
closed linear subspaee of" eontaining all elements of {z(t)}, it is easy to show
that Hz is separable, that is, satisfies Axiom D. A separable unitary space is
either a Hilbert spaee itself (that is, satisfies not only Axioms A, Band E, but
also Axiom C), or is finite dimensional, and in this latter ease ean be extended
to a Hilbert spaee H.
Thus, all the results obtained in [4] may be applied to {z(t)}, setting
B(k) = Bzz(k) = (z(t + k),z(t». (16)
§2. Proof of Theorem 1

For the ease of real random variable z(t), by (3) and (16) we have
(17)
Therefore from (3.10) we obtain
Wzz ('\) = Bz:z:(O)'\ - L

kjlfO
;1
Bz k ) e-iA:>. =
~ Bzz(k) .
= B zz (0),\ + 2 L..J k sm kA. (18)
k=1
It follows from (18) that
(19)
Finally, from (3.1), (3.9) and (19) we obtain
B(k) = Bzz(k) = l:7r eiA:>.dFzz (,\) =

1
2~ -7r ~ 0
11
= - j+7r eiA:>.dWzz ('\) = - 7r eoskAdWzz ('\). (20)
It is dear from (18) and (3.9) and Theorem 2 of [4] that Wxx(A) is a
non-decreasing real function. Together with (18) and (20) this shows that the
function
W(A) = Wxx(A)
satisfies the requirements of Theorem 1.
§3. tT~(m) in the general case
After [4], we denote by Hx(t - 1) the smaHest dosed linear subspace of H x

containing the elements
z(t - 1), z(t - 2), ... ,z(t - n), ....
For any m ~ 0 the element z(t + m) can be uniquely represented as
z(t + m) = e(t - 1, m) + ~(t - 1, m) (21)
where e(t-1, m) belongs to Hx (t-1) and ~(t-1, m) is orthogonal to H x (t-1).

It is easy to show that in the case of a stationary sequence of real random
variables 7
(22)
In the general case of stationary sequences in the sense of [4] we take (22) as
the definition of uj;(m).
If {z(t)} is singular, then Hx(t - 1) = Hx and hence
(23)
7 As is known, 116(t -1, m)1I equals "the distance" of the point x(t + m) from the
space Hx(t -1), that is, the greatest lower bound of the distances IIx(t+m) -yll
for all y in Hx(t - 1). Since the elements of the form
L = alx(t - 1) + a2x(t - 2) + ... + anX(t - n)
are everywhere dense in Hx(t -1), 116(t - 1, m)1I also equals the greatest lower
bound of the distances
IIx(t + m) - LII = vElx(t + m) - L12.
If all the x( s) are real random variables, then the greatest lower bound of (*)
does not change when we consider only real coefficients ak, in which case it
clearly coincides with u E ( m )
If {Z(t)} is non-singular, then, by (7.8),
2: c~~)u~(t + m -
00
z(t + m) = s~(t + m) + n). (24)

n=O
Since s~(t+m) and u~(t+m-n) belongto H~(t-l) for n > m and u~(t+m-n)
are orthogonal to H~(t - 1) for n:::; m, comparing (21) and (24) we obtain
~(t - 1, m) = c~~)u~(t + m) + c~~)u~(t + m - 1) + ... + c~)u~(t). (25)
Since the elements u~(t+i) are pairwise orthogonal and normalized, (25) implies
that
/T~(m) = 1I~(t _1)11 2 = (c~~»2 + (c~~»2 + ... + (c~». (26)
§4. u!(m) in the real ease
It is easy to derive Theorem 2 !rom (26) for stationary sequences of real random
variables. This is carried out in this section.
By (3.9) we have
(27)
Formula (19) implies that

w(-~) = w(~). (28)
Using (27) and (28) we obtain
1 +11'
-11' logf~~(~)d~=2
r logw(~)d~-27rlog27r.
Jo (29)
Together with Theorem 23 from [4], (29) shows that the equality
1111'
P = - logw(~)d~ = -00
7r 0
is necessary and sufficient for the singularity of the sequence {z(t)}. We have
already seen in §3 that in this and only in this case
/TMm) = O. (30)
If {x(t)} is non-singular, then (8.44) and (29) imply that
1 1+11"
(c~X))2 = 211" exp (211" r
_11" logfxx(,x)d,x) = exp (;1 Jo logw,xd,x) = eP • (31)
Under the same assumption, namely if {x(t)} is non-singular, 8 (8.31), (27)
and (28) imply that
a~X) 1 1+11" coskxlogfxx(,x)d,x = -1111" coskAlogw(,x)d,x

= -2
11" _11" 11" 0
(32)
b~X) = 1 1+11" sinkxlogfxx(,x)d,x = -2
-2 1 1+11" sinkAlogw(,x)d,x = O.
11" _11" 11" _11"
Formulas (8.16), (8.29), (8.30) and (32) imply that
~c~x)(n = r x«) = rx(o)~:i~~ = rx(O)eQ$«)-Q$(O) =
(2: a~X)(k).
00
= c~x) exp (33)

k=l
Setting
(2: a~X)(k) = 1 + rl( + r2(2 + ... ,
00
exp (34)
k=l
and comparing (33) and (34), we have
c(x)/c(x)
n 0
-- r n· (35)
O"k(m) = (c~x))2(1 + r~ + r~ + ... + r!). (36)
Formulas (30), (31) and (36) now complete the proof of Theorem 2.
§5. Definition of a}
After [4], we denote by Hx(t) the smallest closed linear subspace of H x con-
taining the elements
x(t + l),x(t + 2), ... ,x(t + n), ... ,

x(t - 1), x(t - 2), ... ,x(t - n), . .. .
8 As indicated in Theorem 23 from [4], (8.16), (8.29)-(8.31) are applicable to any
non-singular sequence.
The element z(t) can be uniquely represented in the form (10.3),
z(t) = lI(t) + 6(t),
where lI(t) belongs to H:t:(t) and 6(t) is orthogonal to H:t:(t). It is easy to show
that for a stationary sequence of real random variables
(37)
By Theorem 24 from [4],
(38)
where ~ = 0 if the integral in the denominator on the right-hand side of (38)

is infinite. Formulas (37), (38), (27) and (28) imply that
CT~ = d! = 'Ir: 1
0
11' d>' 1
w(>.) = R'

Steklov Mathematical Institute, 26 November 1940
References
1. A. Kolmogoroff, 'Sur l'interpolation et extrapolation des suites station-

naires', C. R. Acad. Sei. Paris 208 (1939), 2043-2045.
2. A. Khintchine, 'Korrelationstheorie der stationären stochastischen Pro-
zesse', Math. Ann. 109 (1934), 604-615.
3. H. Wold, A study in the analysis 0/ stationary time se ries, Uppsala, 1938.
4. A.N. Kolmogorov, 'Stationary sequences in Hilbert space', Byull. Moskov.
Gos. Univ. Mal. 2:6 (1941), 1-40 (in Russian) (No. 27 in this volume).
5. A.N. Kolmogorov, Basic notions 0/ probability theory, ONTI, Moscow,
1936.
6. M. Stone, 'Linear transformations in Hilbert space', Amer. Math. Soc.
Colloq. Publ. 15 (1932).
29. ON THE LOGARITHMIC NORMAL DISTRIBUTION OF
PARTICLE SIZES UNDER GRINDING *
In arecent paper [1] N.K. Razumovskii indicates many cases when the loga-
rithms of particle sizes (gold grits in gold placers, rock particles under grinding,
etc.) obey approximately the Gauss distribution law. The aim of this paper
is to give a fairly general scheme of the random process of particle grinding,
for which in the limit (when grinding does not stop) the Gauss law for the
logarithms of particle sizes can be established theoretically. Perhaps similar
considerations will help to explain also why the Gauss distribution is applica-
ble to the logarithms of mineral contents in separate sampIes (Razumovskii's
paper is mainly devoted to this question).
Let us study the general number of particles N(t) and their distribution
in size at successive times t = 0, 1, 2, ....
Let N(r, t) denote the number of particles with sizes p ~ rat time t (in
what follows it is immaterial whether p denotes diameter, weight or any other
characteristic of a particle's size, provided the size of each particle obtained
after grinding a particle of size r does not exceed r).
We denote by Q( k) the expectation of the number of particles of size p ~ kr
formed during the period from t to t + 1 from one particle which at time t had
size r. We set
A = Q~I) 1110gkdQ(k), (1)
B2 = Q~I) 1
1
(logk - A)2dQ(k). (2)
Under certain assumptions given below it can be proved that for sufficiently
large t the ratio
N(e#:,t)/N(t) (3)
is arbitrarily close to
1 j#:
';2rlB -00 exp
- At?}de
{(e 2B t2
(4)
with prob ability arbitrarily close to 1.
* Dokl. Akad. Nauk SSSR 31 (1941),99-101.
281
282 LOG-NORMAL DISTRIBUTION OF PARTICLE SIZES
The most essential assumption used to derive this relation is that the
probability that a particle is ground into a certain number of parts of certain
relative sizes per unit time does not depend on the size of the initial particle.
To formulate rigorously the required assumptions we introduce our nota-
tion. Let Pn be the probability of obtaining exactly n particles from one particle
during the period between t and t + 1 and let
be the conditional distribution law for the ratios ki = ri/r of the sizes of
the resulting n particles to the size of the original particle. The n particles
resulting from grinding are supposed to be enumerated in increasing order of
size: rl ::; r2 ::; r3 :::; ... :::; r n .
In accordance with this, Fn (al, a2, ... , an) is defined only for 0 :::; al <
a2 :::; ... :::; an :::; l.
Clearly,
00
Q(k) = LPn{Fn(k, 1, 1, ... ,1,1) + Fn(k, k, 1, ... ,1,1) + ...

n=l
... + Fn(k, k, k, ... , k, 1) + Fn(k, k, k, ... , k, k)}.
We assurne that
a) the probabilities Pn and the distributions F n do not depend on the
absolute size of a particle, on its prehistory (that is, what kind of previous
grinding it resulted from), and on the fate of other particlesj
b) the expectation Q(1) of the total number of particles obtained between
t and t + 1 from one particle, is finite and greater than 1j
c) the integral
(5)
is finitej
d) at initial time t = 0 there is a certain number of particles N(O) with
arbitrary size distribution N(r,O).
Under these assumptions:
1) The expectation of the total number of particles N(t) at the moment t
lS
N(t) = N(0)Qt(1)j (6)

LOG-NORMAL DISTRIBUTION OF PARTICLE SIZES 283
2) for sufficiently large t, (3) is arbitrarily dose to the ratio
N(eIC,t) N(eIC,t)
N(t) = N(O)Q'(l) = T(z, t) (7)
of the corresponding expectations 1 with probability arbitrarily close to 1.

In view of all these comments our problem reduces to estimating T(z, t).
Assumptions a) and b) imply that
(8)
Setting
Q(k) = Q(1)8(logk), (9)
from (7) and (8) we obtain
T(z, t + 1) = 1~ T(z - e, t)d8(e). (10)
It is easy to deduce from (7) and (9) that 8(z) and T(z, 0) = N(e IC , O)jN(O)
satisfy all the requirements for distribution functions. 2 In view of the recur-
rence relation (10), the same is true for the functions 3 T(z,t) for any integer
t > O. Condition (c) implies that
(11)
is finite. By (10) and (11) and Lyapunov's theorem we have as t -+ 00,
T(z, t) -+
1
.;2irt
jIC exp{
(e - At)2}
2B2 de (12)
2mB -00 t
1 This point should not be dismissed as being trivial. For N(t) and N(r, t) taken
separately, the corresponding statement, that is, that the ratios N(t)/ N(t) and
N(r, t)/ N(r, t) are dose to 1, would be wrong.
2 In this case we set for z > 0,
S(z) = S(O) = 1.
Therefore in all integrals involving dS the upper limit 0 can be replaced by +00
without chan ging the value of the integral.
3 In our problem S(z) and T(z, t) are not probability distributions, but rather
express certain expectations. This does not prevent us from invoking Lyapunov's
theorem, considered as a theorem in pure analysis.
284 LOG-NORMAL DISTRIBUTION OF PARTICLE SIZES
uniformly with respect to x, where
are clearly the same as the A and B defined by (1) and (2).
It would be interesting to study mathematical schemes in which the rate
of grinding decreases (or increases) with the decrease of particle size. Then it
would be natural to consider first the cases when the grinding rate is propor-
tional to a certain power of particle size. If this power is non-zero, then the
logarithmic normal law is no longer applicable.
Steklov Mathematical Institute, 17 December 1940
References
1. N.K. Razumovskii, Dokl. Akad. Nauk SSSR 28:8 (1940) (in Russian).
30. JUSTIFICATION OF THE METHOD OF LEAST SQUARES *
From the purely practical viewpoint, the standard literature on the method
of least squares has one essential drawback: it does not give any indications
on the use of the Student distribution and X2 laws for evaluating the reliabil-
ity of results obtained (for computing the probability that the errors exceed
certain limits). On the other hand, the use of the Gauss law for a small num-
ber of observations results in a very high, and in practice, quite noticeable,
overestimation of this reliability (see below §§9, 10).
In addition, the standard handbooks do not explain adequately the tech-
niques involved in the method ofleast squares; this is in evidence in the teaching
courses in universities and pedagogical institutes where the students are sup-
posed to have a good knowledge of linear algebra. The point is that usually all
the basic results of the method of least squares are obtained in a very clumsy
purely computational way, whereas the use of proper general methods of modern
linear algebra (for example, the notion of orthogonality) gives the same results
much more transparently. The presentation is most lucid when it is performed
with the help of notions of n-dimensional vector geometry.
The aim of this paper is to indicate by means of an example of a very
simple problem on the method of least squares, under the assumption that all
observations are equally valid, how one may obviate both these drawbacks. The
reader is supposed to have a knowledge of linear algebra in geometrie vector
presentation and the fundamentals of probability theory. Greek characters,
excluding 7r and r, denote random variables.
I. LIST OF MAIN RESULTS REQUIRING JUSTIFICATION
§1. Statement of the problem
We assurne that the variables
are related by a homogeneous linear dependence

n
y = Eajxj. (I)
j=l
* Uspekhi Mat. Nauk 1:1 (1946), 57-70 (in Russian).
285
286 JUSTIFICATION OF THE METHOD OF LEAST SQUARES
The coefficients aj are unknown. To determine them, we find by experiment

the values n
Yr = LajZjr, r= 1,2, ... ,N. (11)
j=l
We ass urne that the aj are uniquely determined by the values of Zjr and Yr,
that is, the rank of the matrix rank IIZjrll is greater than or equal to n. This
implies that N ~ n. In the experimental determination of the Yr certain errors
are inevitable. Instead of the true values of Yr we experimentally obtain the
values
7]r = Yr + 6. r . (111)
Given Zir and 7]r, we have to determine the best rational approximate values
G:i of the coefficients ai.
§2. Dogmatic presentation of Gauss's results
Let
n
7]; = LG:jZjr, (IV)

j=l
!r = 7]r - 7];. (V)
According to Gauss, the G:j are determined from the condition
(VI)
Here, and in what follows, we use Gauss's notation

N
[ab] = Larbr.
r=l
Requirement (VI) is equivalent to the system of equations

n
L[ZiZj]G:j = [Zi7]], i = 1,2, ... , n. (VII)
j=l
From these "normal equations" G:i is found.

The resulting approximate values of G:i do not contain any "systematic
error" , that is,
(VIII)
JUSTIFICATION OF THE METHOD OF LEAST SQUARES 287
The accuracy of these approximations is determined by the formula for the

mean square error, the variance D(aj):
(IX)
where s is the mean square error of the experimental values of Yr (it is assumed
to be independent of the number r) and qir is determined from the equations
n
L: [Xi Xj ]qjA: = eiki i, k = 1,2, ... , n. (X)
j=1
Here
for i'" k,
for i = k
Formula (IX) is a particular case (for i = j) of the formula
(XI)
Practical applications ofthis formula are discussed in §11.

If s is not known apriori, it is usually taken to be approxim~tely equal to
(J' = J[a]/(N - n). (XII)
This stratagem is based on the fact that
(XIII)
(XIV)
The latter formula shows that for large N - n the ratio (J' : s is indeed elose to 1
with prob ability elose to 1. Together with formula (IX) this makes it possible
to consider q;j(J'2 as an approximate value of Daj:
(XV)
Remark. If N = n, then the system of equations

n
"Ir = L:ajXjri r = 1,2, ... ,N = n
j=1
is solvable and (VI) implies that
[u] = O.
In this case the denominator on the right-hand side of (XII) vanishes and (1 is
undefined.
§3. Continuation of the dogmatic presentation of Gauss's results
Formulas (VIII), (IX), (XIII) and (XIV) give but a rough indication ofthe size
of the error resulting from the replacement of aj by O:j and s by (1. The final
solution of this problem should have consisted in producing distribution laws
for the deviations O:j - aj and (1 - s. Gauss did this (under the assumption
that the errors ~r are independent and obey the Gauss distribution law with
mean equal to zero) for O:j - aj. Namely, he found that
has the normal distribution
(XVI)
§4. More recent results
The deviations of aj from O:j can be evaluated more accurately with the help
of the distribution law
(XVII)
where
2) _ 1 fh m-l _h 2 /2
Hm(h - 2(m-2)/2f(m/2) Jo h e dh. (XVIII)
Tables ofthe function H m and its inverse are widely used. In §1O we point
out a possible modification in these tables which seems to be desirable from
the viewpoint of practical application of the method of least squares.
The error resulting from replacing aj by O:j for an unknown s is estimated
using the theorem stating that
obeys the Student distribution law
p{ ..;q;;u
aj - aj < t} = SN-n(t), (XIX)
where 1
r((m+1)/2)jt ( t 2 )-<m+1)/2
Sm(t) = y'miil'(
m1r m /2) -00
1 + -
m dt. (XX)
11. THE PROOFS
§5. Formula VII

The fact that requirement (VI) is equivalent to the system of normal equations
(VII) is quite elementary and can be established in many different ways. We
will obtain it by vector methods for the sake of uniformity of style.
For this purpose we note that from the viewpoint of modern linear algebra
the Gaussian brackets [be] are just the sealar produet of two n-dimensional
vectors
Regarding
as the components of the N -dimensional vectors
y, Xi, 1/, A, 1/*, {,
we express (II)-(V) in the form

n
Y= Lajxj, (1)
j=l
1/ = y+A, (2)
n
1/* = LajXj, (3)

j=l
1/ = 1/* + {= L ajxj + {. (4)

j=l
1 At the end of this paper I give the table of the function inverse to Sm(t) which
is a reduced version of the table published by E.N. Pomerantseva. I believe it
satisfies the basic needs for a practical use of the method of least squares (see
§9).
Since the choice of the Cl:i is at our disposal, (3) merely asserts that
TJ" E L, (5)
where
L=L(Zl,Z2, ... ,Zn)
is the linear subspace generated in the N -dimensional vector space V N by the
In the language of vectors, requirement (VI),
[u] = min (6)
together with (5) means the following:

In (4) TJ" is the orthogonal projection of TJ onto Land f is the complemen-
tary vector orthogonal to L.
This implies that
[fZi] = 0, i = 1,2, ... , n. (7)
By (7), taking the vector product of (4) and Zi we obtain
L:[ZiZj ]Cl:j = [ZiTJ], i = 1,2, ... ,n. (8)

j
These are the normal equations (VII) for determining Cl:j. The corresponding
determinant
(9)
is the Gram determinant of the vectors Zt, Z2, •.. , Zn. Since, by the assumption
of §1, IIZjrll has rank n, the vectors Zi are linearly independent and G :I O.
Therefore (8) uniquely determine the Cl:j.
§6. Formulas (VIII)-(XII)

We need certain prerequisites from prob ability theory in order to justify for-
mulas (VIII)-(XII). These are as follows:
A) The true errors Ar are random variables;
B) EAr = 0,
C) = s2, where 8 2 is finite and does not depend on r;
EA~
D) E{ArA r .} = 0 for r:l r'.
We define a system of vectors
in L that are 6n -orthogonal to the system Xl, X2, ... , X n ; this means that
[XiUj] = eij, (10)

Uj E L. (11)
Since each of the systems {Xl, X2, ... , X n } and {U1' U2, ... , U n } is a basis
in L, it follows that
n n
Ui = Lqikxk, Xi = LCikUk.
k=l k=l
Taking the scalar product of the first of these equalities and Uj, and the scalar
product of the second and Xj, and using (10) we obtain
(12)
that is,
n
Ui = L qijXj = L[UiUj]Xj, (13)
j=l
n
Xi = L[XiXj]Uj. (14)
j=l
By (13)-(14), IIqijll is the inverse to II[XiXj] 11 , that is, the qij are indeed
determined from (X).
Taking the scalar product of (1) and Ui we obtain
ai = [yu.]. (15)
Similarly, (3) implies that

(16)
But since I': is orthogonal to L, we have

and (16), together with (4), implies that
(17)
Together with (2), formulas (15) and (17) give
(18)
whence from B), C) and D) we obtain
(19)
N N n
= L: L: E{A6A&'}Ui6 Uj&' = s2L: Ui,Uj' = [Ui Uj]s2,

,=16'=1 .=1
that is, in view of (12),
(20)
Formulas (19) and (20) are none other than formulas (VIII) and (XI). As has
already been mentioned, (XI) gives (IX) for i = j.
Weset
A*=1]*-Y· (21)
Formulas (2), (4) and (21) imply that
A=A*+f. (22)
Since A* belongs to L together with 1]* and y, and f is orthogonal to L, it

follows that (22) is the decomposition of A into its orthogonal projection onto
Land the orthogonal complement to this projection.
In our N -dimensional vector space we choose an orthogonal basis
br = (b r !, br2 , ... , brN), r = 1,2, ... , N,

[brbr,] = err"
such that the first n vectors belong to L and the remaining N -n are orthogonal
to L.
Setting
(23)
we obtain
(24)
r=l
n
A* = L:Lirbr, (25)
r=l
N
l = L: Lirbr. (26)
r=n+l
Clearly, by C) and D)
for r = r',
(27)
for r i= r'.
Formula (26) and B) imply that
El= 0 (28)
and (26) and (27) imply that
E[u] = (N - n)s2. (29)
Formula (29) and (XII) (this formula merely serves as adefinition of u) imme-
diately imply (XIII).
§7. The hypothesis of the Gaussian distribution and

the independence of the true errors
For our further derivations, the assumptions B), C) and D) given at the begin-
ning of §6 should be replaced by the following stronger ones:
G) The errors Ar obey the Gauss distribution
where 82 doe8 not depend on r;

U) The errors dl, d2, ... , dN are N mutually independent random vari-
ables.
It is weIl known that G) implies B) and Cl, and B) together with U)
implies 0).
Assumptions G) and U) imply that the system dl, d2,' .. ,dN of random
variables obeys the N -dimensional distribution law with prob ability density
(30)
or, in vector notation,
(31)
Formula (31) gives the prob ability density of a random vector d with
(32)
as the volume element.

In the coordinate system
Ir = [tbr), (33)
Li r = [dbr ), (34)
where the basis vectors br satisfy the usual orthogonality condition
(35)
the volume element (32) retains the form
(32')
Therefore in this new coordinate system !(t) can also be considered as the
probability density of d. In other words, the prob ability density of the random
variables Li l , Li 2 , ••• , LiN is of the form
(30')
Formula (30') shows that the random variables Ar are independent and
obey the same distribution law
- r < t} = -1-
P{6.
-/2is
1
t
-00
e- t 2/2• 2 dt, (36)
as do the 6. r .
It is important to be aware of the fact that the independence of Ar is
derived from both G) and U). This conclusion is no longer the case when the
Gaussian distribution for the errors of the first kind is replaced by some other
one.
§8. Proof of formulas (XIV)-(XX)

By (18) ai is a linear function (with non-random coefficients) of the random
variables 6.r . Therefore G) and U) imply that ai obeys a Gaussian probability
distribution. Normalizing this distribution in accordance with (VIII) and (IX)
we obtain (XVI).
Now we derive the distribution law for
(37)
For this we note that, by (26),

N
[u]= L (38)
(39)
The independence of the random variables Ar implies that of the random vari-
ables Ar/s. By (36) each ofthese latter variables obeys the normal distribution
law with prob ability density Ja;e- t2 / 2 • Therefore their (N - n)-dimensional
distribution law is characterized by the prob ability density
exp{ -~ L
N
(21r)-(N-n)/2 t~}.
r=n+l
The prob ability of the inequality
X<h
is given by integrating this density over the (N - n )-dimensional volume for

which
N
R2 = L: t~ < h 2 •
r=n+l
Thus we obtain 2
_ 1 fh hN- n - 1 -h2/2dh (40)

- 2(N-n-2)/2r«N _ n)/2) Jo e .
This is formula (XVIII).

From (40) it follows that
EX 2 = N - n, (41)
DX 2 = E{x2 - (N - n)}2 = 2(N - n). (42)
By (37), formulas (41) and (42) are equivalent to (XIII) and (XIV).
It only remains to consider the distribution law for
Tj = (aj - aj)/..;qjju. (43)
For this it is helpful to write Tj in the form
Tj = 'r;/x, (44)
where
"/j =.JN - n(aj - aj)/..;qjjs. (45)
We now note that by (15), (16), (21) and (25),
n
aj - aj = [~·Uj] = L: UjrAr , (46)
r=l
2 For the derivation of (40) it should be noted that the area of the surface of a
sphere of radius R in m-dimensional space is
where
(47)
Comparing (46) with (37)-(38) we see that the ~r entering in the expres-
sion for lkj - aj are only those ~r that do not occur in the expression for X2 •
Therefore, the independence of the ~r implies that lkj - aj is independent of
X. By (45), 'Yj and X are also independent. In accordance with (XVI), the
probability density of 'Yj is
1 -c2/2(N-n)
V 21r (N _ n) e .
Together with (40) this implies that the 2-dimensional prob ability density of
'Yj and X is
1 hN - n- 1 { c2 h2 }
V1r(N - n) ·2(N n 1)/2r«N _ n)/2) exp - 2(N - n) - 2 .
Integrating this prob ability density over the region where c/h < t, we
obtain
P{ Tj < t} = p{ ~ < t} =
1
= X
V1r(N - n)2(N-n-l)/2r«N - n)/2)
X 11 hN - n - 1 exp{ - 2(Nc~ n) - ~2 }dcdh. (48)
We now change the variables c and h to 8 and u via the formulas

c c2 h2
8 = h' u = 2(N _ n) + 2·
Since
8(8, u) = 1 +~, h2 _ 2u
8(c,h) N-n -1+82 /(N-n)'
(48) implies that
P{tj < t} =
= V1r(N - 1
1 00 N-n-l
u -2-e- n du
1t 82
1+ - -
- N-n±l
2
=
n)r«N - n)/2) 0 0 ( N - n) d8
=
r«N - n + 1)/2)
V1r(N - n)r«N - n)/2)
1 0
t( 2
1 +t- -) -(N-n+1)/2 dt.
N - n
(49)
Formulas (XIX) and (XX) are nothing but another way of expressing (49).
In. REMARKS ON THE PRACTICAL SIGNIFICANCE OF

IMPROVING THE STANDARD ACCOUNT OF
THE METHOD OF LEAST SQUARES
§9. On the use of the Student distribution law
In accordance with the general principles of modern mathematical statistics 3

the two values a] and a'j such that for any acceptable hypothesis concerning
the unknown values aj we have
P{a'.J < a· < a'l} >

- J- J -
w (50)
will be called the "confidence limits" for aj corresponding to the "confidence

coefficient" w.
If s is known and only the al, a2, ... ,an are unknown, then by (XVI), the
formulas
a] = aj - tViijs, a'j = aj + tViijs, (51)
satisfy this requirement, where t is defined by the equation
G(t) = (1 + w)/2. (52)
The values of t corresponding to various w are given in the second last line
(m = 00) of Table 1.
If s is known, then in accordance with (XV), it is customary to consider
(T as an approximate value of s. However, for the confidence limits
(53)
to satisfy (50), t should be defined not by (52) but, in accordance with (XIX),
(XX), by
SN-n(t) = (1 + w)/2. (54)
The values oft corresponding to various w according to (54) for different values
of N - n = mare given in the same Table 1.
3 See [1] or [2].
Table 1. Values of t satisfying the equation Sm(t)=P
m P =0.75 P 0.90
= P 0.95
= P =0.975 P 0.99
= P 0.995
= P = 0.9975 P 0.999
= P 0.9995
=
1 1.000 3.078 6.314 12.076 31.821 63.657 127.321 318.309 636.619
2 0.816 1.886 2.920 4.303 6.965 9.925 14.089 22.327 31.600
3 0.765 1.638 2.353 3.182 4.541 5.841 7.453 10.214 12.922
4 0.741 1.533 2.132 2.776 3.747 4.604 5.597 7.173 8.610
5 0.727 1.476 2.015 2.571 3.365 4.032 4.773 5.893 6.869
6 0.718 1.440 1.943 3.447 3.143 3.707 4.317 5.208 5.959
7 0.711 1.415 1.895 2.365 2.998 3.499 4.029 4.785 5.408
8 0.706 1.397 1.860 2.306 2.896 3.355 3.833 4.501 5.041
9 0.703 1.383 1.833 2.262 2.821 3.250 3.690 4.297 4.781
10 0.700 1.372 1.812 2.228 2.764 3.169 3.581 4.144 4.587
11 0.697 1.363 1.796 2.201 2.718 3.106 3.497 4.025 4.437
12 0.69 1.356 1.782 2.179 2.681 3.055 3.428 3.930 4.318
13 0.694 1.350 1.771 2.160 2.650 3.012 3.372 3.852 4.221
14 0.692 1.345 1.761 2.145 2.624 2.977 3.326 3.787 4.140
15 0.691 1.341 1.753 2.131 2.602 2.947 3.286 3.733 4.073
16 0.690 1.337 1.746 2.120 2.583 2.921 3.252 3.686 4.015
17 0.689 1.333 1.740 2.110 2.567 2.898 3.222 3.646 3.965
18 0.688 1.330 1.734 2.101 2.552 2.878 3.197 3.610 3.922
19 0.688 1.328 1.729 2.093 2.539 2.861 3.174 3.579 3.883
20 0.687 1.325 1.725 2.086 2.528 2.845 3.153 3.552 3.849
21 0.686 1.323 1.721 2.080 2.518 2.831 3.135 3.527 3.819
22 0.696 1.321 1.717 2.074 2.508 2.819 3.119 3.505 3.792
23 0.685 1.319 1.714 2.069 2.500 2.807 3.104 3.486 3.768
24 0.685 1.318 1.711 2.064 2.492 2.797 3.092 3.467 3.745
25 0.684 1.316 1.708 2.060 2.485 2.787 3.078 3.450 3.725
26 0.684 1.315 1.706 2.056 2.479 2.779 3.067 3.435 3.707
27 0.684 1.314 1.703 2.052 2.473 2.771 3.057 3.421 3.690
28 0.683 1.313 1.701 2.048 2.467 2.763 3.047 3.408 3.674
29 0.683 1.311 1.699 2.045 2.462 2.756 3.038 3.396 3.659
30 0.683 1.310 1.697 2.042 2.457 2.750 3.030 3.386 3.646
00 0.67449 1.21855 1.64485 1.95996 2.32634 2.57582 2.80703 3.09023 3.29053
w 0.5 0.8 0.9 0.95 0.98 0.99 0.995 0.998 0.999
As m --+ 00, t determined from (54) converges to t determined from (52)

with s replaced by u, but this convergence is quite slow, especially for w dose
to 1. For example, by (52), t = 3.090 for w = 0.998, which approximately
corresponds to the dassical "three sigma" mle, while (54), for example, yields
t = 5.959 for m = 6.
§10. Estimation of s from tT.
The equality
(55)
is satisfied most simply by choosing k1 and k 2 so that
In accordance with (XVII), (XVIII) this is achieved if k1 and k 2 are determined

from
H (N-n) _ l+w (57)
N-n k1 - 2 '
From the viewpoint of the method of least squares, it would appear desir-
able to have a table which would give k1 and k 2 directly from m =N - n and
w. This is done in our Table 2 for w = 0.98.
The sharp asymmetry between the upper and lower bounds of the estimate
is worth noting. For example, for w = 0.98 and m = 3,

k1 = 0.514, k2 = 5.111,
that is, u twice as large as s are found only with prob ability 0.01 but with
probability 0.01 s is 5 times larger than u. Therefore for small m it would be
unreasonable to use confidence limits for s of the form (1 ± k')u, because then
one would often have to take the lower bound negative! Only for very large m
can we assurne that
(58)
where t is determined from (52) by the Gaussian law.

Tahle 2. Values of k satisfying the equation Hm (m / k) = P (ku::; s} = P.
P P P
m m m
0.01 0.99 0.01 0.99 0.01 0.99
1 0.388 79.750 8 0.631 2.204 40 0.792 1.343
2 0.466 9.975 9 0.644 2.076 50 0.810 1.297
3 0.514 5.111 10 0.657 1.977 60 0.824 1.265
4 0.549 3.669 15 0.708 1.694 70 0.835 1.241
5 0.576 3.003 20 0.730 1.556 80 0.844 1.222
6 0.597 2.623 25 1.751 1.473 90 0.852 1.207
7 0.615 2.377 30 0.768 1.416 100 0.858 1.195
§11. Remarks on the value of the matrix IIq;jll

The value of the covariance matrix
(59)
gives much more exhaustive information on the nature of the errors
than the values of its diagonal elements
(60)
alone. For example, to determine the mean square error made when replacing
ai hy ai in the expression
where the coefficients Ci are given, knowledge of the variances D( as) is not
enough, hut it is quite sufficient to know the covariance matrix 8ij. Setting
we have
E{ß - b}2 = LLSijCiCj.
i j
Usually the diagonal elements of the matrix IIqij 11 are introduced as the recip-
rocals of the "weights"
This points to the essential role of the entire matrix IIqij 11.
References
1. Yu. Neiman, 'Statistical estimation in problems of classical probability

theory', Uspekhi Mat. Nauk 10 (1944), 207 (in Russian).
2. A.N. Kolmogorov, 'Determination of the cent re of scattering and the mea-
surement accuracy from a limited number ofobservations', Izv. Akad. Nauk
SSSR Sero Mat. 6 (1942), 3-32 (in Russian).
31. A FORMULA OF GAUSS IN THE METHOD OF LEAST SQUARES *
Jointly with A.A. Petrov and Yu.M. Smirnov
This paper refines certain estimates from Gauss's theory of the method of least
squares.
In §39 of "Theoria combinationis observationum erroribus minimis obnox-
iae" , Gauss computes the mean square error made in replacing 1'1' by
M ,U + .U' + >,">." + ...

7r-P = 7r-P
Gauss shows that this mean square error equals
0-~ 0-~ 2
- ( )2 [p - ~(all' + bß + e)' + ...) ]. (1)
7r-P 7r-P L...J
In §40, Gauss derives from (1) simple estimates ofthe mean square error.
He derives these estimates from the inequalities
(2)
Gauss overlooked the fact that the upper estimate in (2) can be essentially
sharpened, and (2) may be replaced by
PP/7r::; ~(all' + bß + C')' + ... )2::; p. (3)
Therefore the conclusions of §40 are unexpectedly weak: the lower estimate
suggested by Gauss for the least square error is sometimes even negative.
Our aim is to prove the inequalities (3) and to sb.ow that they cannot be
improved.
Let us first recapitulate the problem in modern notation dose to that in Kol-
mogorov's paper [1].
Let Yr, ai and Xir where r = 1,2, ... , N; i = 1,2, ... , n; n < N, satisfy
the N equations
n
Yr = ~aiXir. (4)
i=l
* Izv. Akad. Nauk SSSR Sero Mat. 11:6 (1947), 561-566.
303
304 A FORMULA OF GAUSS IN THE METHOn OF LEAST SQUARES
The xir are assumed to be exact and instead of Yr we are given
TJr = Yr + ß r , (5)
where the ß r are independent random variables such that
(6)
(E denotes the expectation). Let the rank of the matrix IIXiril be n.

Under the above assumptions the method of least squares reeommends
that one takes the ai determined from the eondition
N n
I)TJr - L:
a i X ir)2 = min (7)
r=l i=l
as approximate values of the unknown ai.

As is known, this problem has the unique solution
N
ai = L: UirTJr· (8)
r=l
The eoeffieients Uir in (8) have the following geometrie meaning: in N-dimen-
sional space the veetors
Ui=(UibUi2, ... ,UiN), i=1,2, ... ,n,

form abiorthogonal system with the vectors
Xi = (XibXi2,'" ,XiN), i = 1,2, ... ,n,
whieh means that the vectors U1, U2, ... , Un are uniquely determined by the fol-
lowing eonditions: they belong to the linear n-dimensional subspace L spanned
by Xl, X2, ... ,Xn and satisfy the biorthogonality eonditions
for i = j, (9)
for i i= j.
Setting
1 N
(T2 =- - - " f~,
N-nL.....!
(10)
r=l
A FORMULA OF GAUSS IN THE METHOD OF LEAST SQUARES 305
where
n
lr = 'Ir - L (kiZir, (11)

i=1
we have
(12)
Gauss's problem mentioned in the beginning of our paper is to determine
(13)
In our notation Gauss's formula (1) is expressed in the form
(14)
where
N n 2
0= L(LZirUir) . (15)
r=1 i=1
§2. Estimation of 0
Expression (15) has a simple geometrie meaning. Namely, if we denote by

e; = (e;1' e;2' ... ,e; N) the projeetion of the unit eoordinate vector er onto L,
then
N N N 2
0= L le;1 = L [L(e;k?] .
4 (16)
r=1 r=1 k=1
Indeed, the projeetion z" of any veetor z onto L ean be written in the form
n
Z" = LCiZi.
i=1
Computing the sealar produet (ZUi) we obtain, by (9):
If Z = er, then the latter formula gives Ci = Uir. Therefore

n
e; = LUirZi, (17)
;=1
306 A FORMULA OF GAUSS IN THE METHOD OF LEAST SQUARES
or, in coordinate form,

n
e;k = L UirXik· (18)

i=1
Similarly we obtain
r
e; = LXirUi, (19)
i=1
e;k = LXirUik. (20)

N N n n n
le;1 2 = L(e;k)2 =L L L(UirXikXjrUjk) =L UirXir· (21)
k=1 k=1 i=1 j=1 i=1
Formulas (21) together with (15) immediately give (16).

Now let us introduce in N -dimensional space a new orthogonal coordinate
system, placing the first n axes in the space L, and choosing the other N - n
axes to be orthogonal to this space.
In this new coordinate system let the vectors er be given by
Clearly, in this new coordinate system
e; = (Or1,Or2, ... ,Orn,0,0, ... ,0).

Therefore,
(22)
The matrix \lOrk 11 is orthogonal. It is easy to see that the statement of

our problem does not impose any other restrietions on the form of this matrix:
under an appropriate choice of the matrix \lXir \I of rank n we can make \lOrk 11
an arbitrary orthogonal matrix.
Thus, the problem of estimating the possible values of 0 can be put as
folIows: which values can (22) take for n < N, for an orthogonal N x N matrix?
Since
n, N
LO;k ~ LO;k = 1,
k=l k=l
A FORMULA OF GAUSS IN THE METHOD OF LEAST SQUARES 307
it follows that always
Therefore (22) implies

N n
o ~ I:I:0~k = n.
r=lk=l
The inequality
(23)
thus established turns into an equality if IIOrkll is the identity matrix
On the other hand, since always
(22) implies that
In the resulting inequality
(24)
an equality can be attained for any N and n < N, as has been established by
A.1. Mal'tsev [2] at my request.
§3. Conclusions
In §2 it was established that
(25)
and both bounds in (25) can be attained. To estimate Du 2 we obtain by (14)
r- 84 n (/4 - 384 )
.:...".".----
2 84
<Du <.:...".".--
r-
N-n N N-n - - N-n'
(26)
/4 _ 84 /4 _ 84 n (38 4 _ /4)
.:...".".--
N-n -
< Du 2 <
- N-n
+ -N N-n
,
308 A FORMULA OF GAUSS IN THE METHOD OF LEAST SQUARES
and the bounds in (26) can be attained for any N and n < N in certain
particular cases.
If r - 38 4 = 0, then from (26) we obtain the formula
(27)
which is weH known for the case of the normal Gaussian distribution of the
errors ~r.
If n/N ~ 0 and r- 84 > 0, then asymptoticaHy
(28)
In fact, in Gauss's memoir only the asymptotic estimates
are given for Du 2 as n/N ~ o.

Note also that in the degenerate case 14 = 84 (as is weH known, the case
r < 8 4 is impossible) one obtains from (26) the formula
(29)
5 May 1947
References
1. A.N. Kolmogorov, 'A justification ofthe method of least squares', U8pekhi

Mat. Nauk 1:1 (1946),57-70 (in Russian).
2. A.1. Mal'tsev, 'A remark on the work by A.N. Kolmogorov, A.A. Petrov
and Yu.M. Smirnov "A Gauss formula from the method ofleast squares"',
Izv. Akad. Nauk SSSR Sero Mat. 11 (1947), 567-568 (in Russian).
32. BRANCHING RANDOM PROCESSES *
Jointly with N.A. Dmitriev
Consider a set of objects (for example, molecules) of n types Tl. T2,.'" Tn and
=
aSsume that with prob ability P:(t 1 ,t2) P(T" -+ Saltltt2)' during the time
interval (tl, t 2) one object of type T" turns into the set
consisting of (ll objects of the first type, (l2 objects of the second type, (li ob-
jects of type i, etc. A random process consisting of this kind of transformation
is called a branching process if the probabilities P:(tl,t2) are uniquely deter-
mined by the times tl < t2, the number k of the original type, k = 1,2, ... , n,
and the n-dimensional vector (l = «(ll, (l2, ... , (ln) with integer coeflicients,
(li = 0,1,2, ....
It is essential here that we assume the probabilities to be independent 0/:
1) how the original object of type T" appearedj it is only assumed that it
exists at time tl j
2) the fate of other objects of types Tl. T2, ... ,Tn different from T" at time
tl and the objects originating from them at times t > tl.
This probability-theoretic scheme finds numerous applications in biology,
chemistry and elementary particle physics. In particular, in chemistry it can
can be used to describe the initial stages of the most varied chemical re ac-
tions. At the initial stage of a chemical reaction the concentration of certain
types T{, T~, ... ,T:n of molecules may be considered high but approximately
constant, whereas the concentrations of other types T{', T~', ... ,T:: are variable
but very low. Under these assumptions meeting between two types T" is vir-
tually impossible, while the results of meeting between one molecule of type
T" with one or several molecules of the types T' approximately obey the above
requirements as regards the number of resulting molecules of types T".
In chemical and physical quest ions it is natural to use a "continuous time"
version of this scheme, assuming that the probabilities P:(tl, t2) are differ-
entiable with respect to t l and t 2 • The differential equations derived under
* Dokl. Akad. Nauk SSSR 56:1 (1947),7-10.
309
310 BRANCHING RANDOM PROCESSES
this assumption in §3 of this paper for the special case of "monomolecular"

reactions (Pf(tl,t 2) > 0) only for al + a2 + ... + an = 1) have been given
before by M.A. Leontovich [1]. In biological questions another, "discrete time"
approach is more natural, where the "time" t takes only integer values and
indicates the number of the generation. In this version the results given below
were obtained for the case n = 1 by R.A. Fisher [2]. Fisher's studies were
continued by J.F. Steffenson [3] and one of the authors of this paper [4].
§2. The basic functional equation
By the general principles of probability theory, the Pf(tl, t2) satisfy the con-
ditions
(I)
I: Pk(t1, t 2) = l. (II)
0<
Taking these probabilities to be defined for any tl ~ t2, it is natural to

assume that for tl = t2,
Pk(t, t) = Ef = { ~
if ak = 1, ai = 0, i t k,
(III)
otherwise.
Finally, in view of the above assumption, for any
the Pf(tl, t2) satisfy the equation
pt(t 1,t3) = I:Pf(tbt2)pt(t2,t3)' (IV)

0<
where
is the probability that the set
is transferred into the set

BRANCHING RANDOM PROCESSES 311
during the time interval (tl, t 2).

The probabilities Pg (t1, t2) can be expressed in terms of the probabilities
Pf(t1, t2) as
n O'i
P~(t1,t2) = I: II II Pf(i,8)(t 1,t2)' (IVa)
;=18=1
where the sum is taken over all sets of vectors
ß(i,s) = (ß1(i,s),ß2(i, s), ... ,ßn(i,s))
with non-negative integer coefficients ß,.(i, s) such that 1

n O'i
I:I:ß(i,s) = ß· (IVb)
;=1 8=1
Formulas (I)-(IV) are a complete translation of the not ion of "branching

random processes" defined in §1 in terms of probability theory, into purely
analytical terms.
Formulas (IV) and (IVa) imply that the Pg(tl, t2) satisfy the main equa-
tion of Markov processes,
PJ (t1, t3) = I: Pg (tl, t2)P; (t2, t3). (1)

ß
Equation (1) follows immediately from the probabilistic assumptions of §l.

In essence, (IV) is but a particular case of (1). This shows that "branching
random processes" are, in essence, merely a particular case of Markov processes
with a countable set of states. However, for this particular case we have a
more effective analytical apparatus than the one that can be developed for the
general case of Markov processes with a countable number of states. For this
we introduce the generating functions
F,.(tl, t2; Xl, X2, ... , Xn ) = I: Pf(it, t2)xrlx~2 ... x~n. (2)
a
It is convenient to express the n functions F,. as a single vector-valued

function
1 For (Xi = 0 the corresponding product in (IVa) and (IVb) is set to be 1, and the
corresponding surn o.
of the vector argument x = (Xl, x2, ... , xn). The reason for introducing the
function F(tl, t2; x) is that by means of this function (IV) can be expressed in
the following way:
(A)
Apart from the basic functional equation (A) for F(t1, t2; x) (111) also gives the
boundary value
F(t,t;x) = x (B)
Naturally F(t1,t2;X) is considered to be defined only for t1 ::::; t2. The

argument x has a purely formal meaning, although in any case, (I) and (11)
imply that Fk(t l , t2; x) is defined and analytic with respect to the arguments
Xl,X2, ... ,Xn for lXii< 1, s= 1,2, ... ,n.
Especially interesting is the case of processes homogeneous in time, that
is, processes satisfying the condition
(3)
In this case for

Fk(t; x) = L: Pk'(t)xf' X~2 ... x~n (4)
a
we obtain, instead of (A) and (B):
F(t + T; x) = F(t; F(T; x»,

F(O;x)=x.
§3. Differential equations for the case of continuous tUne
We now assurne that
Pk'(t,t +~) = Ef + ~pf(t) + o(~), (V)
where o(~) is an infinitesimal with respect to~. In this case it is natural to

introduce the generating functions
(5)
BRANCHING RANDOM PROCESSES 313
Then for lXii< r < 1, i = 1,2, ... , n, we obtain
F(t, t + ß; x) = x + ßf(t; x) + o(ß). (6)
Setting tl = t', t2 = t' + ß, t3 = t" in (A) and passing to the limit as ß ---+ 0,
we obtain for t' < t" the following differential equation for F(t', t"; x):
ßF/ßt' = -f(t';F). (C)
This differential equation together with the boundary value (B) can serve as
a basis for computing this type of processes. Indeed, PI:(t) can usually be
determined directly from the conditions of the problem, while the purpose
of the mathematical theory is to determine the probabilities pr (tl, t2), and
sometimes, their asymptotic behaviour as t 2 ---+ +00. Since fk can easily be
found from the given pI: and the desired pr are obtained from the power series
expansions of the Fk in the Xi, to determine the probabilitiespr and study
their asymptotic behaviour we only have to solve (C) with the boundary value
(B) and investigate the asymptotic behaviour of the solutions.
For a process homogeneous in time the prob ability densities PI: are con-
stants and the Fk(t; Xl, X2,' .. , Xn) are related to the functions
fk(Xl,X2, ... ,x n ) = LPl:xrlx~2 ... X~n (7)

a
by the equations
dFl _ dF2 _ ... _ dFn = dt,

h(Fl ,F2, ... ,Fn ) J2(Fl ,F2, ... ,Fn ) fn(Fl ,F2, ... ,Fn )
(C)
which has to be solved for t > 0, regarding Xi as constants, and with the initial
value
(B')
This method is often much more efficient than directly dealing with the
infinite systems of differential equations of Markov processes with a countable
set of states derived from (1) under assumptions similar to (V). Here we will
confine ourselves to one simple example of using this method (the same problem
has been solved by N. Arley [5] using infinite systems):
n = 1:
p~ = a, pt = -(a + b), p~ = b, p~ = 0 for r > 2,

/(x) = a - (a + b)x + bx 2 = (1- x)(a - bx).
dF a + ce(a-b)t
-:-----:-:-----,-
(1 - F)(a - bF)
= dt, F - --..,..--:-:-
- b + ce(a-b)t '
a+c bx - a
F(O;x) = -b- = x, C=--,
+c 1-x
a(1-x)+(bx-a)e(a-b)t 0 I 2 2
F(t; X) = b(1- X) + (bx _
a)e(a-b)t = PI (t) + PI (t)x + PI (t)x + ...
o a 1 - e(a-b)t
Pd t ) = b 1 _ (a/b)e(a-b)t'
a) 2 (1 _ e(a-b)t)k-I
pk(t) - ( 1 _ _ (a-b)t
I - b [1- (a/b)e(a-b)t]k+ 1 e ,
where k = 1,2, ....

20 February 1947
References
1. M.A. Leontovich, 'Basic equations of kinetic gas theory from the point of
view of the theory of random processes', ZhETF, 5 (1935), 211-231 (in
Russian).
2. R.A. Fisher , The genetic theory 0/ natural selection, Oxford, 1930.
3. J.F. Steffenson, 'Deux problemes du calcul des probabilites', A nn. Inst. H.
Poincare 3 (1933), 331-334.
4. A.N. Kolmogorov, 'Solution ofa biological problem', Izv. NIl Mat. Mekh.,
Tomsk Univ. 2:1 (1938),7-12 (No. 25 in this volume).
5. N. Arley, On the theory of stochastic processes and their application to
the theory of cosmic radiation, Dissertation, Copenhagen, 1943.
33. COMPUTATION OF FINAL PROBABILITIES
FOR BRANCHING RANDOM PROCESSES *
Jointly with B.A. Sevastyanov
The terminology and notation in this paper are elose to those in [1]. In what fol-
lows, we consider discrete schemes homogeneous in time t: t runs only through
1,2,3, ... and can be interpreted as the "generation number" of the partiele in
question. Accordingly,
(1)
is the probability that one partiele of type Tk gives al, a2, ... , an partieles of
the types Tl, T 2 , ••• , Tn , respectively in t generations. All further computations
are based on the generating functions
(2)
of the probabilities
pI: = Pk(l) (3)
of the various transitions during one generation. Using these probabilities, we

define by induction the gene rating functions
Fk(t;X) = LPk(t)xrlx~· ... x~n (4)

'"
for all positive integers t:
(5)
In accordance with the general theory given in [1],
(0,0, ... ,0) _

Pk -
F
Jk
(0 , 0 , ... , 0) (6)
denotes the probability of one particle of type Tk dying in one generation

without producing any new partieles of the considered types Tl, T 2 , ••. , T n .
In what follows it is useful to eliminate this possibility and to consider only
schemes in which
fk(O,O, ... ,O) = O. (7)
* Dokl. Akad. Nauk SSSR 56:8 (1947), 783-786.
315
316 FINAL PROBABILITIES FOR BRANCHING RANDOM PROCESSES
If (7) does not hold in a eertain scheme, it is always possible to introduee

an additional ghost type T n +1 and eonsider that the death of a particle of any
other type means its transition into a particle of type T n +1 , while the particles
of type T n +1 remain unehanged. Analytieally this is equivalent to transition
from the initial system of generating functions 11, 12, ... ,fn to the new system
1,,(zl, ... , Zn, Zn+!) = 1,,(zl, ... , zn)+

+(Zn+1 - 1)/,,(0,0, ... ,0), k = 1,2, ... , n,
ln+1(Zl, ... ,Zn,Zn+1) = Zn+!. (8)
In what follows we assurne that (7) is already satisfied for the initial system
of types Tb T2, .. . , Tn and functions 11,12,·· ., In.
A group of types T"l , T"3' ... , T"m is ealled closed if a particle of any other
type produees only particles belonging to the types of the group. The system
of all types Tl, T 2, ... , Tn is deeomposable if it ean be divided into two closed
groups. It is natural to eonfine ourselves (and we shall do so) to indeeomposable
systems.
A group of types is ealled final if a) it is closedj b) eaeh particle of any type
in the group always produees exaetly one particlej and e) it does not eontain
any smaller group with the properties a) and b).
It is easy to see that two final groups do not have eommon elements.
Therefore, in general, the entire system Tb T2, ... , T n eonsists of a eertain
number offinal groups lli r = {Trb Tr2, ... , T rnr }, r = 1,2, ... , s, and a eertain
number of types T Ol , T 02 , ... , T ono which do not belong to final groups. Clearly,
no + n1 + ... + n, = n.
The proeess is eonsidered to be eompleted if there only remain particles of
the types from final groups. This way of understanding is quite natural, sinee
the progeny of a particle of a type belonging to a final group eonsists, in any
future generation, of one particle of a type belonging to the same final group,
and transitions from one type to another within a final group are governed by
the weIl known law of Markov ehains in their simplest form, eorresponding to
the assumption that all "states" are "essential" and form one "class" (see, for
example, [2]).
Denote by
(9)
FINAL PROBABILITIES FOR BRANCHING RANDOM PROCESSES 317
the probability that the evolution of the progeny from one particle of type TA:
eventually terminates in the final state, where ßr particles belonging to the
types of each of the final groups Wr remainj
(10)
is the total probability that the evolution of the progeny from one particle
eventually terminates (in the above sense).
We introduce the generating functions
(11)
In accordance with the double index of particle types we sometimes denote

by </Jrm the functions among the </JA: corresponding to the type TA: = Trm . Since
q~ ~ 0, by (10), the functions (11) are in every case defined and analytic with
respect to all variables for
o ~ Ur < 1, r = 1,2, ... , s. (12)
When some of the variables attain the value 1, analyticity may be lost, but
continuity remains. In particular,
(13)
It is easy to derive from prob ability considerations the following relation,

which is of basic importance to us:
(14)
Moreover, it can be derived from the definition of a final group that
</Jrm=Ur , m=1,2, ... ,nr , r=1,2, ... ,s. (15)
Equations of the system (14) for which the number k corresponds to the
type of a certain final group are corollaries of (15). Therefore, finally to deter-
mine </JA: we have the system of equations
(16)
</Jrm=Ur , m=1,2, ... ,nr , r=1,2, ... ,s.
The junior author of this paper has proved the following theorem on
uniqueness of the solution of (16).
Theorem. System (16) together with the restrictions 0 ~ tPA: < 1, k = 1,

2, ... ,n, uniquely determines the values of tPl. tP2, ... ,tPn for any given 0 ~
Ur< 1, r= 1,2, ... ,s.
Remark 1. The proof will be published elsewhere. It is based on properties

of the /J: which follow from the assumption that the systems of types are
indecomposable, and from the definition of final groups, assumption (7) and
the relations
(17)
a
These relations, however, are not fully used: analyticity of fA: is inessential
in the proof.
Remark 2. All these considerations are applicable also to computing final prob-
abilities qffor branching processes with continuous time. It suffices to count
not over time literally, but over "generations" of particles, introducing for the
types that do not undergo any further transformation additional ghost trans-
formations of their particles into themselves (with prob ability 1). This remark
will be clarified with an example (see below).
Remark 3. In most applications to chemical chain reactions each final group

consists of one type (final re action products). In this case the theory simplifies.
Example. Let two positive transition probability densities be given in a scheme

with continuous time and two types Tl and T 2 (see [1],§3); we set
p~2,0) = p(T1 -+ 2T1 ), p~O,l) = p(T1 -+ T2 ),

and let all other transitions be forbidden (in particular, particles of type T 2 do
not turn into their own type). Counting over generations, we set
_ (2,0)
(2,0)
P1 = P{T1 -+ 2T1 } = -_7:(2:-:,0~)..!.+-_7:(0:-:,1")
P1 = p,
P1 P1
_ (0,1)
(0,1) _ P{T 2'Tl} - P1 - 1
P1 - 1 -+ .L2 - -(2,0) + -(0,1) - - p,
P1 P1
FINAL PROBABILITIES FOR BRANCHING RANDOM PROCESSES 319
and additionally
(here the p~ have the meaning of probabilities introduced in this paper). In

our example there is one final group of one type '11 1 = {T2}' For 4>l(Ut), 4>2(U1)
we obtain
Solving these equations, we formally obtain
(18)
When 0 ~ U1 < 1, 0 ~ 4>1 < 1 there remains only one branch of the curve
(18) (with minus sign).
Let us complete the computations for p = ~. In this case we obtain for
the coefficients of the expansion 4>1 (ud = q~O) + qP)U1 + qF)u~ + ...
q1(O) -- 0 , (1) _ 1 (2) _ 1 (m) _ 1·3·5 ... (2m - 3)
q1 - 2' q1 - '8' ... , q1 - 2m . m! ' m~2,
that is, asymptotically

(19)
Note that although
that is, the process will inevitably terminate, the expectation E1 = Lm mq~m)
of the number of particles of type T 2 obtained from one particle of type Tl is
infinite. This causes a peculiar phenomenon of non-stability of the number of
particles of type T 2 generated by a given, though perhaps very large, number of
particles of type Tl. To make this clear, we denote by J.ln the number of particles
of type T 2 generated by n particles of type Tl. Clearly, J.ln = K1 + K2 + ... + K n
where K; denotes the number of particles of type T 2 generated by the ith particle
of type Tl. The variables Ki are independent and have prob ability distribution
P{K; = k} = q~k).
This together with (19) implies that (see [3]) the distribution law for
(20)
is Sn (x) = P {en < x}, which tends to a certain limit distribution law for the
quantity
S(x) = 111: s(x)dx. (21)
The limit distribution law (21) can be found from the logarithm of its
characteristic function,
logX(t) = log 1
o
00
s(x)eitll:dx = -1-
2...(i
1
0
00
(e iut -
du
1)--.
u3 / 2
(22)
Thus for large n, J.ln is of order n 2 , but the ratio J.ln/n 2 varies from case
to case.
Moscow, 12 April 1947
References
1. A.N. Kolmogorov and N.A. Dmitriev, 'Branching random processes', Dokl.

Akad. Nauk SSSR 56:1 (1947),7-10 (in Russian) (No. 32 in this volume).
2. A.N. Kolmogoroff, 'Anfangsgründe der Theorie der Markoffschen Ketten
mit unendlich vielen möglichen Zustanden', Mat. Sb. 1:4 (1936), 607-610.
3. B.V. Gnedenko, 'On the theory of domains of attraction of stable laws',
Uch. Zap. Moskov. Gos. Univ. 30 (1939), 61-82 (in Russian).
34. STATISTICAL THEORY OF OSCILLATIONS WITH
CONTINUOUS SPECTRUM *
1. The subject discussed in this sketch can be characterized in very general
terms not connected with any special field of mechanics or physics. For sim-
plicity, we confine ourselves to considering only changes in time of a certain
finite number of variables
6(t),6(t), ... ,e,(t).
Naturally the argument t (time) is assumed to be real, while er(t), in general,

will be complex. If the er(t) are changing periodically with period w, then the
process of their variation is usually studied by means of Fourier series
er(t) "" L: a~n)ei2,..ntlw. (1)

n
Owing to this, an oscillatory variation of er(t) with period w can be expanded

in harmonie oscillations of period
w,w/2,w/3,w/4, ....
A natural generalization of periodic oscillations are almost periodic oscillations,
ofthe form
er(t) "" L:a~n)ei>.nt. (2)
n
Here the periods
are, in general, incommensurable. Naturally, a further generalization of (2) is

to pass from sums with respect to a sequence of particular discrete frequencies
An to integration with respect to a continuous frequency A ,
(3)
or, in a completely general form combining the discrete (2)and continuous (3)
cases, to represent oscillatory processes by Stieltjes integrals
(4)
* In: Collected papers on the 30th anniversary 0/ the Great October Sodalist Rev-
olution, Vol. 1, Akad. Nauk SSSR, Moscow-Leningrad, 1947, pp. 242-252.
321
322 STATISTICAL THEORY OF OSCILLATIONS WITH CONTINUOUS SPECTRUM
However, if we restrict ourselves to the methods of classical analysis, then

the above approach meets with an essential difficulty. The classical Fourier
integrals (3) can only represent functions that tend, in the mean, to constants as
t -+ 00. Integrals (4) can also represent only functions that can be decomposed
into two components,
e(t) = 7J(t) + ((t),
where 7J(t) is almost periodic (that is, has discrete spectrum), and ((t) tends, in
the mean, to a constant as t -+ 00. Thus, the classical tool of Fourier integrals
for the case of a continuous spectrum leads only to damped O8cillations. This is
what is said in textbooks on physics that want to retain mathematical rigour. 1
Nevertheless, the idea that even in the non-damped case, oscillations with
continuous spectrum are possible and can be expressed by integrals of the
form (3) is maintained by many, sometimes quite outstanding, researchers in
mechanics and physics. Studies of this kind, despite the lack of mathematical
rigour, in many cases bring about correct and useful results.
In essence, the prerequisites for a rigorous mathematical justification of
the theory of non-damped O8cillations with continuous spectrum are contained
in the spectral theory of operators. 2 A mathematically rigorous spectral the-
ory for functions e(t) of a real argument t which was directly oriented to the
mathematical justification of physical notions on non-damped oscillations with
continuous spectrum was first created by Norbert Wiener 3 in 1925. Wiener's
concept, however, does not bring us to either of the representations (3) or
(4). Only the spectral theory of stationary random processes suggested by
A.Ya. Khinchin in 1934 [1] completely clarified the above questions. In partic-
ular, it leads to a mathematically rigorous and, at the same time, physically
reasonable justification of representing non-damped statistically stationary 08-
cillations in the form (4). Apparently, this justification is general enough for
practical applications. Still, even in Khinchin's concept, the representation of
non-damped oscillations in the form (3) has no rigorous mathematical justifica-
tion. I believe that such astate of affairs should be considered final: there seems
1 See, for example, [15], §54.

2 See the presentation ofthis theory in the papers by A.1. Plesner and V.A. Rokhlin
[16], [17].
3 A systematical presentation of Wiener's theory can be found in Chapter 4 of his
book [18]; see also [19].
STATISTICAL THEORY OF OSCILLATIONS WITH CONTINUOUS SPECTRUM 323
to be no reasonable way to ascribe definite derivatives <Pr(A) to the functions

~r(A) in the case of continuous spectrum.
The purpose of this paper is to review certain Soviet and foreign studies
developing the concept of Khinchin.
2. All the very simple and complete results stated below are obtained by a
radieal cha.nge in viewpoint: the er(t) are considered as random variables in
the sense of prob ability theory. Thus, the main object of study is not a certain
individual oscillatory process, but the law of the prob ability distribution in the
function space of various possible versions of such a process.
Formally, the theory may be presented starting from the following defini-
tions. 4
An s-dimensional random process is a collection of complex random vari-
ables er (t) given for aH real t and r = 1,2, ... ,s.
It is assumed that the expectations
are finite and that the process is stochastically continuous, which means that
as ~ -+ o.
A random process {el(t),6(t), ... ,e.(t)} is called stationary if for any
t, t1, t 2 , ••• ,tn , the sn-dimensional distribution law of sn random variables
el(t + t 1 ), 6(t + t 2), ... , el(t + tn),

6(t + tl), e2(t + t2), .. . , 6(t + tn),
does not depend on the shifts in t.

Stationarity implies, in particular, that the expectations 5
(5)
4 The notions of probability theory used in what follows are rigorously introduced,
for example, in my book [20].
5 e
In (5), denotes the complex conjugate of e.
do not depend on t. The independence itself of the expectations (5) on t is

called stationarity in the wider sense. The account given here and in the next
sections is based solelyon the hypothesis of stationarity in the wider sense.
The main result of A.Ya. Khinchin, in the form given by Harald Cramer
[2], is as folIows:
The functions Bqr(r) can be represented as
(6)
where for any Al < A2 the matrix IIFqr(ß>.)1I of the increments
is hermitian and non-negative.

To understand the meaning of Khinchin's spectral functions we must con-
sider the discrete case, when (6) turns into the sums
Bqr(r) = La~~)ei>'nT. (7)

n
This case was studied by E.E. Slutskii [3]. He showed that under the assumption
(7) there exists an expansion
er(t) ...... L o:~n)ei>'nT (8)

n
where the o:~n) are certain random variables uniquely 6 determined by the given
er(t). In this case,
(9)
In particular,
(9')
It is natural to assume that in the general case we also have
Fqr(ß>.) = E{4>q(ß>.)4>r(ß>.)}, (10)

Frr(ß>.) = EI4>r(ßr W, (10')
6 Uniqueness is to within equivalent variables, that is, variables that are equal
with probability 1.
where the <Pr(Ll),) are the increments of the spectral function occurring in the
representation of er(t) in the form (4).
This assumption appears to be correct. The further development of Khin-
chin's theory of stationary random processes follows almost automatically from
the reduction of it to the spectral theory of one-parameter groups of unitary
operators, as is indicated in [4]-[7] and presented in the following section.
3. The random variables er(t) may be considered as elements of a Hilbert
space, in which the scalar product is given by the formula
(11)
It can easily be shown that any stationary process {6(t),6(t), ... ,e8(t)} in
the corresponding Hilbert space generates a one-parameter group of unitary
operators {ur} satisfying the relation
(12)
for all real t and and all r = 1,2, ... , s. As is weIl known, the operators ur
I:
T
can be represented as
ur = ei)'r dE()'), (13)
where E().) is the resolution 0/ the identity. Setting

(14)
we obtain (provided the integration sign is properly understood) the basic
formula (4). A probabilistic interpretation of (4) has been given in subsequent
papers by H. Cramer [8], M. Loeve [9], A. Blanc-Lapierre and R. Fortet [10],
[11] in a manner more general than in [6] and [7]. To understand the real
meaning of the basic spectral functions <p r ().) it is natural to consider their
increments
andjumps
at distinct points ).. Naturally, <Pr(Ll),) and a r ().) are random variables, as is
er(t), but, unlike er(t), they do not depend on the time t. If for some ). the
jump a r ().) is non-zero, then er(t) contains a strictly periodic component
(15)
In particular, O'r(O) is the average (over time) of
. 1 jT
O'r(O) = Thm
..... oo
2T
-T
er(t)dt. (16)
The real meaning of the increments C)r(A.\) for sufficiently short intervals A.\
is similar. Namely, the component
(17)
corresponding to the interval A.\ of the spectrum can be arbitrary closely ap-
proximated by
(17')
where A is an arbitrary point of A.\, on any finite interval -T ~ t ~ +T ofthe

time axis, provided that A.\ is sufficiently short. It is natural, however, that in
the case of a continuous spectrum the components
are not strictly periodic for any finite fixed interval A.\ (unless they are identi-
cally zero). Their time behaviour on short intervals A.\ is similar to the oscil-
lations of a pendulum with weak damping generated by chaotically distributed
small random pushes.
The components er(t, A.\) for non-intersecting intervals A~ and A~ are
not correlated to each other, that is,
(18)
if A~ and A~ do not intersect. For one interval A.\,
(19)
In particular,
(19')
that is, Frr(A.\) is just the average value (with respect to probability) of the
square of the spectral component er(t, A.\) of er(t).
4. The use of the abstract tool of operators in Hilbert space could have pro-
duced the impression on the reader that the spectral components er(t, A.\)
are some mathematical fiction far from possible direct experiment. This is
not true: with any desired degree of approximation they can be isolated from
any statistically stationary oscillatory process by me ans of appropriate filters.
Namely, with any desired accuracy a device can, in principle, be constructed
which associates with any given random stationary function el(t) the random
1
stationary function
e2(t) = 00
6(t - T)S(T)dT, (20)
where s( T) is subject to the sole requirement that
If we set
(21)
then for sufficiently large positive T
is arbitrarily elose to
with probability elose to 1.

Note that (20) can be approximately realized in the form
(22)
where
d ~ d!'
L = Co + Cl dt + C2 dt 2 + ... + Cn dt n (23)
and all the eigenvalues of L have negative real part. Under these assumptions,
if el(t) is statistically stationary, then 6(t) tends to a statistically stationary
function as t - +00. This statistically stationary limit behaviour of 6(t) is
determined by the formula
K (z) = Co + Cl Z + ... + Cn zn • (24)

Formula (24) is of interest in its own right and not merely for the experi-
mental determination of the spectral components er(t, dA). It implies that
(25)
(26)
Formulas (24), (25) and (26) generalize the usual resonance theory to the case
of an arbitrary statistically stationary external force. Usually, a non-statistical
presentation of this theory can be applied only to the case when the external
force has a discrete spectrum, that is, when 6(t) is ofthe form (2). In physical
literature, however, for the case of continuous spectrum this kind of reasoning
is widely used, though, of course, without rigorous mathematical justification.
5. In applications, for the case of a continuous spectrum, we usually have
(27)
that is, (6) may be replaced by
(28)
This passage to spectral densities Iqr(>') enables us to do without Stieltjes

integrals in the part of the theory that does not involve ~r(>').
The simplest and most important in applications is the case when the dis-
tribution laws of any finite number of variables er(t) are Gaussian. In this case
so are the ~r(dA). Under the simplest and most typical case of a continuous
spectrum (the applicability of (27) and (28) and Gaussian distributions) the
random functions ~r(>') are not differentiable and the passage from (4) to the
formulas of type (3) seems impossible. On the intervals of the >.-axis where
the spectral density Irr (>') is continuous and non-zero, the nature of variation
of ~r(>') is the same as the time dependence of the coordinates of a Brownian
particle, when neglecting inertial forces, that is, if for small d = >'2 - >'1 the
increments of ~r(>') are of order.Ji5.. Here is a new case where continuous
nowhere differentiable functions of Weierstrass type intrude into mathematical
physics. For Brownian motion, more refined arguments taking into ac count the
inertial forces bring us back to differentiable functionsj here non-differentiability

is inherent in the idea of a continuous spectrum.
Beyond the mechanics and physics of oscillations, the ideas of spectral
analysis of statistically stationary process are applied mainly in the slightly
different form of the spectral theory of stationary sequences (for example, in
meteorology). Apart from the papers [4], [5] mentioned above, the book ofthe
Swedish mathematician Herman Wold [12] is devoted to this theory. Important
additions to Wold's theory were made by V.N. Zasukhin [13]. The clear under-
standing of the fact that the existence of a spectrum follows automatically from
statistical stationarity and does not necessarily indicate that the process be-
ing studied results from superposition of strictly periodic components was very
important for a critical reconsideration of the so-called periodographics, which
claim to playamajor role in meteorology and even in economy. E.E. Slutsky's
papers in this field (especially [14]) have become classical in statistics.
References
1. A. Khintchine, 'Korrelations theorie der stationären stochastischen Pro-

zesse', Math. Ann. 109:4 (1934), 604-615.
2. H. Cramer, 'On the theory ofstationary random processes', Ann. of Math.
41:1 (1940), 215-230.
3. E, Slutsky, 'Sur les fonctions aIeatoires presque periodiques et sur la decom-
position des fonctions aleatoires stationnaires en composantes', Aetual. Sei.
738 (1938), 35-55.
4. A.N. Kolmogorov, 'Interpolation and extrapolation of stationary random
sequences',Izv. Akad. Nauk SSSR Sero Mat. 5:1 (1941), 3-14 (in Russian)
(N o. 28 in this volume).
Gos. Univ. Sero Mat. 2:6 (1941), 1-40 (in Russian) (No. 27 in this volume).
6. A.N. Kolmogorov, 'Curves in Hilbert space invariant with respect to a one-
parameter group of motions', Dokl. Akad. Nauk SSSR 26:1 (1940), 6-9 (in
Russian).
7. A.N. Kolmogorov, 'Wiener's spiral and some other interesting curves in
Hilbert space', Dokl. Akad. Nauk SSSR 26 (1940), 115-118 (in Russian).
8. H. Cramer, 'On harmonie analysis in certain functional spaces', Ark. Mat.
Astron. Fys. B 28:12 (1942), 1-7.
9. M. Lohe, 'Analyse harmonique generale d'une fonction aleatoire', C. R.

Acad. Sei. Paris 220:12 (1945), 380-382.
10. A. Blanc-Lapierre and R. Fortet, 'Sur la decomposition spectrale de fonc-
tion aleatoire stationnaire d'ordre deux', C. R. Acad. Sei. Paris 222:9
(1946), 467-468.
11. A. Blanc-Lapierre and R. Fortet, 'Resultats sur la decomposition spectrale
des fonctions aleatoires stationnaires d'ordre 2', C. R. Acad. Sei. Paris
222:13 (1946), 713-714.
12. H. Wold, A study in the analysis 0/ stationary time se ries, Uppsala, 1938;
2nd. ed. 1954.
13. V.N. Zasukhin, 'On the theory of multidimensional stationary random
processes', Dokl. Akad. Nauk SSSR 33:5 (1941), 435-437.
14. E.E. Slutsky, 'Summation of random causes as a sour ce of cyclic pro-
cesses', Voprosy Kon"yunktury 3:1 (1927), 34-64 (in Russian). English
transi: Econometrica 5:2 (1937), 105-'-146.
15. E.V. Shpol'skii, Atomic physics, Gostekhizdat, Moscow-Leningrad, 1944
(in Russian).
16. A.I. Plesner, 'Spectral theory of linear operators', Uspekhi Mat. Nauk 9
(1941),3-124 (in Russian).
17. A.I. Plesner and V.A. Rokhlin, 'Spectral theory of linear operators',
Uspekhi Mat. Nauk 1:1 (1946), 192-216 (in Russian).
18. N. Wiener, The Fourier integral and certain 0/ its applications, New York,
1933.
19. N. Wiener, 'The harmonie analysis ofirregular motion', J. Math. Phys. 5
(1926), 99-122, 158-191.
20. A.N. Kolmogorov, Basic notions 0/ probability theory, ONTI, Moscow-
Leningrad, 1936; 2nd ed. Nauka, Moscow, 1974 (in Russian).
35. ON SUMS OF A RANDOM NUMBER OF RANDOM TERMS *
J ointly with Yu.V. Prokhorov
Wald and a number of other American authors have given interesting theorems
concerning the sums
of the first v random variables from an infinite sequence
where the number v of terms is a random variable (see [1]-[3], where references
to earlier literat ure can be found). In their method of proof these theorems
go back to the work of one of the authors of the present paper [4], where for
estimating the prob ability
P{ max I(n - Anl > h}

l:5 n :5N -
he considered sums (v with index v equal to the first number n for which
The inequality proved in [4] (see also [5], p.154) can easily be derived from
Theorem 5 of the present paper.
Further we give very simple proofs for theorems of Wald type relating to
the first and second moments. Our conditions for the applicability of basic
identities are somewhat broader than those of Wald and Wolfowitz. Our gen-
eralization of the conditions for the applicability of these identities is important
for certain applications.
In what folIows, v denotes a random variable that can take only non-
negative integer values, and
n=O,1,2,3, ....
The event that v = n will be denoted by

Sn = {v = n},
* Uspekhi Mat. Nauk 4:4 (1949),168-172.
331
332 ON SUMS OF A RANDOM NUMBER OF RANDOM TERMS
and the probability of this event by
Pn = P(Sn).
Moreover, we set
00
Pn =P{lI=n}= 2: Pm·
m=n
Expectations of random variables,
will be understood in the sense of the abstract Lebesgue integral over the set
of elementary events U. Accordingly, the expectations, when they exist, are
always finite, and the existence of E(7J) implies the existence of E(I7JI). The con-
ditional probability distributions and conditional expectations are understood
in the sense explained in [6].
Of basic importance for all theorems of Wald type is the assumption
(w) For n > m the random variable en
and the event Sm are independent.
According to [6], (w) means that for n > m the conditional distribution of
en under the condition Sm coincides with the unconditional distribution
Theorem 1. I/ condition (w) holds and the expectations
E(lI) and E(e) = a, E(lenl) = c,

exist, where a and c do not depend on n, then the expectation 0/ (v exists and
equals
E«v) = aE(lI). (1)
Since 00 00
E(lI) = 2:P n = 2: P
n n,
n=l n=l
Theorem 1 is an obvious consequence of the following more general statement:
Theorem 2. I/ condition (w) holds, the expectations

ON SUMS OF A RANDOM NUMBER OF RANDOM TERMS 333
exist and the series

(I)
converges, then the expectation 0/ (11 exists and equals

00
E«(II) = L:PnAn, An = E«(n) = al + a2 + ... + an. (11)

n=l
Pro%/ Theorem 2. In view of (I), the series on the right-hand side of (11)
converges absolutely. Applying the Abel transformation to it, we obtain
00 00
L:PnAn= L: Pnan. (2)

n=l n=l
We recall that
Pn = P{/I ~ n}.
The event {/I ~ n} is complementary to the event {/I < n} which, by condition
(w), is independent of en . But the independence of an event implies the
independence of its complementary event. Therefore, denoting by E(7JIA) the
conditional expectation of 7J under condition A, we obtain
00 00 00
L:PnAn = L: P{/I ~ n}an = L: P{/I ~ n}E(enl/ln ~ n) =

n=l n=l n=l
(3)
Since the fact that en is independent of the event {/I ~ n} implies that len I is
also independent of this event, it follows from (I) that
00
= L: P{/I ~ n}E(lenlll/l ~ n) =
n=l
L: P{/I ~ n}E(lenD = L: Pncn <

00 00
= 00. (4)
n=l n=l
Since (4) is finite, we can change the order of summation:
Since (m = (v holds on Sm,
(6)
As for the latter equality, we note that the "necessary event" U is the union of
the events Sm, including So. However, the omission ofthe term with m = 0 on
the left-hand side of (6) is inessential, since, in accordance with the accepted
rule, we consider the sum (0 of an empty set of terms to be identically equal
to zero.
Comparing (3), (5) and (6), we obtain (11) as required.
When considering the second moments we will assume that
is a vector with two components e~ and e~. Condition (w) is now understood
in the sense that for n > m the two-dimensional conditional distribution of
en under the condition Sm coincides with the unconditional distribution of the
same vector. In addition to (w) we also assume the condition
(z) The vectors 6,6,6, ... are independent.
By contrast, the dependence among the components of the same vector
can be arbitrary. We now consider the particular case when e~ == e~:
Theorem 3. If(w) and (z) hold and the expectations
exist, where a i and bij do not depend on n, then
(III)
exists.
Naturally, in (III) (~ denotes
Theorem 3 can easily be derived from the more general Theorem 4 if we

note that the finiteness of
00 00
E(v3/ 2) = LPn n 3 / 2 = L Pn (n 3 / 2 - (n - 1?/2)

n=l 1'1=1
is equivalent to the convergence of 2:::=1 Pn yn.

In the statement of Theorem 4 we use the notation
B nij -- bij
1 + bij2 + ... + bn'
ij
Theorem 4. If (w) and (z) hold, the expectations
exist and the series

00
L Pn ( J bh1 B~2 + Jb~2 BA1 ) (7)

n=l
converges, then
00
E{(~ - A~)(; - A~)} = LPnB~2. (IV)

n=l
exists.
Proof of Theorem 4. By passing from the variables e~ to *e~ = e~ - a~, we can

reduce the general case to the case
(8)
We consider this case in what folIows. Setting
(9)
we have identically
(10)
From (8) (9) and the independence of en of (n-1 (which follows from (z))
we obtain
(11)
Under condition (8) the equality (IV) to be proved turns into

00
E(Xv) = L:PnB~2. (12)

n=l
Comparing (10), (11) and (12) with the statement of Theorem 2, we see that
(12) is simply (11) applied to the sums
Xv = <PI + <P2 + ... + <Pv.

It remains to prove that the conditions of Theorem 2 hold. Special computa-
tions are required to verify (I) which, applied to the sums Xv, is written in the
form
00
L: PnE(I<Pnl) < 00. (13)

n=l
Using the Cauchy-Bunyakovskii-Schwarz inequality we obtain from (9) the es-
timate
~ v'b~lb~2 + v'b~l B~2 + v'b~2 BAI ~ 2 ( v'b~l B~2 + v'b~2 BAI) (14)
for E(I<Pn 1). Comparing (7) and (14) we see that (13) holds, which completes
the proof of Theorem 4.
In the particular case e~ == e; we can abandon the vector notation and
write
Cl _ c 2 - C
<"n-<"n-<"n, A n=al + a2 + ... + an,
Then from Theorem 4 we obtain
Theorem 5. If (w) and (z) hold for a sequence of random variables en, the
expectations E(en) = an, E(en -an)2 = bn exist and the series E::'=l Pnv'bnBn
converges, then
00
E«v - A v)2 = L:PnBn (V)

n=l
exists.
References
1. A. Wald, 'On cumulative sums of random variables', Ann. Math. Statist.

15 (1944), 283-296.
2. A. Wald, 'Differentiation under the expectation sign of the fundamental
identity in sequential analysis', Ann. M ath. Statist. 17 (1946).
3. J. Wolfowitz, 'The efficiency of sequential estimates and Wald's equation
for sequential processes', Ann. Math. Statist. 18 (1947), 215-231.
4. A.N. Kolmogoroff, 'Über die Summen durch den Zufall bestimmter un-
abhängiger Grössen', Math. Ann. 99 (1928), 309-319.
5. S.N. Bernshtein, Probability theory, GTTI, Moscow-Leningrad, 1946 (in
Russian).
6. A.N. Kolmogorov, Fundamentals 0/ probability theory, ONTI, Moscow-
Leningrad, 1936 (in Russian).
36. A LOCAL LIMIT THEOREM FOR CLASSICAL MARKOV CHAINS *
This paper studies the limiting distributions of the number of visits to various
states for a Markov chain with a constant transition prob ability matrix.
§1. Statement of the problem and review of results
Consider a classical Markov chain, that is, a random Markov process with
discrete time, a finite number of states (s > 1)
and constant probabilities
(1.1)
of transition from astate €(t) = ea at time t to astate €(t + 1) = eß at time

t + 1. Naturally, we assume that the usual conditions hold:
(1.2)
If the states e a are the unit vectors
el = (1,0'00.,0),
e2=(0,1, ... ,0),
(1.3)
e, = (0,0, 00.,1)
in s-dimensional coordinate vector space, then the components
of the vector
/let) = €(1) + €(2) + ... + €(t) (1.4)
are equal to the number of visits to the states
* Izv. Akad. Nauk SSSR Sero Mat. 13 (1949), 281-300.
338
A LOCAL LIMIT THEOREM FOR CLASSICAL MARKOV CHAINS 339
at times t' = 1, 2, ... , t respectively. We will adhere to these vector represen-

tations throughout. We suppose that at time t = 0 p(t) is equal to the zero
vector
p(O) == 0 = (0,0, ... ,0). (1.5)
The only possible values for p(t) are the vectors m = (m 1 ,m 2 , ••• ,m')
with integer components such that
m= Lmß =t. (1.6)

ß
For any integer vector m with m ~ 0 we set
(1.7)
Thus the function Wa(m) ofthe vector argument m comprises the conditional
distributions for E(O) = ea of all vectors p(t) for t = 1,2,3, .... This is possible
because, since by virtue of the relation.
Ji(t) = Lpß(t) = t, (1.8)

ß
these distributions are at most (s - l)-dimensional. Naturally, the sum of the

probabilities Wa(m) pertaining to one variable p(t) is 1:
(1.9)
Our entire analysis is devoted to the darification of the limiting behaviour

of the probabilities Wa(m) as m -+ 00. The study of all limit distributions of
sums of random variables "linked into a chain" (according to Markov's termi-
nology) can easily be reduced to this problem.
The most noteworthy is the case when, in terms of [1], all states e a form
one dass, that is, when the following condition holds
(A) For any two states ea and eß there exists a sequence 0/ states (e a , e.n ,
e""2' ... , e""k' eß) along which all the transition probabilities pi}, p~:, ...
. .. , P~:-l' P~k are positive.
The most general case can be reduced to the case (A). This is done in §7.
The following Lemmas 1-3 hold only under condition (A). In the state-
ment of these lemmas we denote by Ea the conditional expectations under the
340 A LOCAL LIMIT THEOREM FOR CLASSICAL MARKOV CHAINS
hypothesis f(O) =e Q • Although these lemmas are proved in [1]-[3], Lemmas 2

and 3 are again proved in §2 of the present paper.
Lemma 1. The system of equations
(1.10)
Q Q
has a unique solution.
Lemma 2.
A~(t) == E'YpQ(t) = tqQ + 0(1) (1.11)
as t -+ 00.
Lemma 3. The second moments
(1.12)
are of the form

(1.13)
as t -+ 00, where the constants bQß are determined solely from the matrix of
initial probabilities ~.
It follows from the general properties of seeond moments of any system
of random variables that IIbQß 11 is symmetrie and the eorresponding quadratie
form
b(x) = L: bQßxQxß (1.14)
Q,ß
is non-negative. Then it ean easily be derived from (1.8) that
L: bQ'Y = L: b'Yß = o. (1.15)

'Y 'Y
Formula (1.15) implies that
b(x) = b(x - x). (1.16)
Therefore it suffiees to eonsider the form b(x) in the (8 - l)-dimensional spaee

N of veetors x with
x = o. (1.17)
It is natural to consider the case when

(B) the form b(x) is positive in the space N of vectors x for which x= 0
as the general "non-degenerate" case of our problem.
By (1.15) the determinant of the form b(x) is always 0;
W~ßI = 0, (1.18)
and all the principal minors of the matrix IIbaßII are equal to one another:
(1.19)
The fact that b( x) is non-negative implies, of course, that always
(1.20)
condition (B) is equivalent to the requirement that
Ll> o. (1.21)
Under condition (B) the form b(x) has an inverse form c(x) in the space
N which can, for example, be expressed as
bl l b12 bl ,3-l xl
b2l b22 b2,3-l x2
1
c(x) =-- ..... (1.22)
Ll
b3- l ,1 b3- l ,2 b3- l ,3-l x 3- l
Xl x2 x 3- l 0
or in a similar alternative way by choosing some other index 'Y instead of s - 1.
Setting
1
p(x) = e- l / 2c (x), (1.23)
v's(27r)3-lLl
~(t) = (J.l(t) - tq)/..fi, (1.24)
we can state a "non-degenerate" integral limit theorem.
Theorem 1. Under conditions (A) and (B), for any 'Y and any rectifiable
domain G ofthe space N,
p{~(t) E G I (0) = e1'} --+ !aP(X)dX (1.25)

as t -+ 00, where
(1.26)
denotes the volume element in N.

This theorem is proved in §4.
However, to formulate the simplest local limit theorem, we must make a
certain additional analysis of possible directions of transition from one state to
another.
A chain is a sequence of states
(1.27)
for which all transition probabilities
are positive. Achain (1.27) is called a cycle if
(1.28)
We then say that an s-dimensional vector z is cyclic if there exists a cycle (1.27)
such that
(1.29)
Finally, the fundamentallattice is the set of all vectors m representable as
(1.30)
where Zl, Z2, ••• ,Zn are cyclic vectors and al, a2, ••• ,an are arbitrary integers
(the number of terms n is also arbitrary). We denote the fundamentallattice
by Z. Clearly Z consists exclusively of vectors with integer components and
forms a group with respect to addition. We now state our additional condition.
(C) the fundamentallattice Z coincides with the set Q of all integer vectors
of s-dimensional coordinate space.
The results of §§6, 7 imply that both conditions (A) and (C) are necessary
for the limiting behaviour of the probabilities W-y (m) to be independent of the
indices 'Y. Thus, the case when both conditions hold is the only case when we
can count on obtaining a locallimit theorem in an ideally simple formulation.
This makes us regard the results of §3 and §5 as definitive in a certain sense:
Corollary 3 of Lemma 10 (§3). Conditions (A) and (C) imply (B).
Theorem 3 (§5). Ifl (A) and (C) hold, then whatever the choice of indices "
(1.31)
as m --+ 00, where

x = (m - mg)/vm. (1.32)
A very interesting quest ion is the following: are (B) and (C) equivalent
if (A) holds? If the answer is yes, then the conditions of applicability of the
Local Theorem 3 and the Integral Theorem given above would coincide. 2
In §6 we give a fuH analysis ofthe complications wh ich arise when (C) does
not hold while (A) still holds. As mentioned earlier, in §7 we consider the case
when (A) does not hold. The results of this section are stated in a somewhat
more complicated way, though the question on the limiting behaviour of the
probabilities W-y( m) is essentiaHy solved for the most general case as fuHy as
for the case when (A) and (C) hold.
§2. Doeblin's method
Local theorems, which are the main subject of this paper, will be derived in §5
and §6 using a certain strengthening of the method developed by Doeblin for
1 Unlike (1.23), in (1.31) there is no factor 8 under the root sign, sinee in the
(8 - 1)-dimensional space of veetors m with given m (for example, in the space
N) integer points are distributed with density I/Vi. Note also that in the text
of §5 the statement of Theorem 3 differs from that given here in that the way
in whieh the probabilities tend to their asymptotie expression is indieated more
precisely.
2 Let L be the linear dosure of the fundamental lattiee Z, that is, the set of
all veetors representable in terms of eydic ones by (1.30) with arbitrary real
eoefficients ak. In §3 (see Corollary 2 of Lemma 10) it is proved that under
eondition (A), requirement (B) is equivalent to (B'):
(B') The space L eoincides with the entire 8-dimensional veetor space R.
The question whether (B) and (C) are equivalent would be solved if we eould
show that
(*) under eondition (A) the fundamentallattiee Z always eoincides with the set
of all integer points from L.
Assumption (*) is very likely to be true. If it were proved, this would lead to a
eertain improvement in the results of §6.
(Assumption (*) was later proved by Rosenknop in Moseow and Chulanovskii in
Leningrad. )
proving the integral limit theorem for the case of an infinite number of states
[4]. To make dear how this method was developed, in this section we briefly
present Doeblin's method in its original form, which is only suitable for proving
integral theorems.
The presentation of this paragraph will be brief, since the results given
here are essentially known. Here, as in §§3-6, (A) will be assumed to hold
without any special indication. Moreover, throughout this section 'Y will be
assumed to be fixed and
f(O) = e-y. (2.1)
Under this hypothesis the conditional probabilities and expectations will be

denoted by
Let
0= r(O) < r(l) < r(2) < ... < r(n) < '"
be the sequence of all times t when the state e-y is observed. For n ~ 1 we set
6(n) = p(r(n)) - p(r(n - 1)), (2.2)

A(n) = 6(1) + 6(2) + ... + 6(n) = p(r(n)). (2.3)
For n = 0 set
A(O) = O. (2.4)
The components 60t (n) of 6( n) denote the number of visits to the state eOt at
times t satisfying the inequalities
r(n - 1) < t ~ r(n),
that is, between the (n - 1)th and the nth returns to the original state e-y
(induding the very moment of the nth return). Clearly, we always have
P(n) = 1, (2.5)
while for the sums A( n) we have
(2.6)
Doeblin's method is based on a simple observation: the random vectors

o( n) are independent and identically distributed. Because of this, the limit
theorems established for sums of independent summands may be applied to
the sums A(n). Relation (2.3) allows transition from the sums A(n) to the
sums p(t). Further we give two versions of this transition: one is based on
Lemma 6 and gives most accurate results for estimating the second moments
B;;ß(t), while the other is based on Lemma 7 and is convenient for deriving
integral limit theorems. In [1] I proved
Lemma 4. The variables o<>(n) have the finite expectations
(2.7)
Since the terms o( n) are independent
(2.8)
and since
7'(n) = X(n), (2.9)
(2.7) and (2.8) imply that
(2.10)
By the methods of [1] it is easy to prove
Lemma 5. There exist constants C and D > 0 such that for any k and for
c5( n) = 7'( n) - 7'( n - 1) the following inequality holds:
(2.11)
Lemma 5 implies
Corollary. The second moments
(2.12)
are finite.
By analogy with (2.8), for the second moments .\Q(n) we obtain
(2.13)
Let Tt denote the smallest of the numbers T(n) that are ~ t and let lI(t)
be the corresponding number n.
Setting
"t
.\(t) = .\(lIt) = !-'(Tt) = L:c5(n), (2.14)
n=l
we obtain
.\l = IIt, (2.15)

Xt = Tt· (2.16)
In the same way as for Lemma 5 we prove
Lemma 6. There exist constants C and D > 0 such that for any k,
(2.17)
Since always
(2.18)
(2.17) implies that
(2.19)
The weIl known Wald identities (see [5] concerning the conditions of their
applicability) are applicable to the following sums with random upper limit IIt:
"t
.\t = L:c5(n).
n=l
From these identities we obtain
E-y.\f = a;E-r(lIt), (2.20)
E-y{(.\f - IIta;)(.\f - IIta~)} = b;ßE-y(lIt). (2.21)

From (2.16) and (2.20) we obtain
(2.22)
By Lemma 6,
E-yTt = t + 0(1) (2.23)
as t -+ 00. Therefore, from (2.22) and (2.20) we obtain as t -+ 00
E-yVt= q-yt + 0(1), (2.24)

E-yAf = qf + 0(1). (2.25)
Using (2.19) we infer from (2.25) that
(2.26)
Thus we obtain a new proof of Lemma 2 stated in §1.

Noting that identically
Af - Ttqa = (Af - Vta~) - qa ~)At - Vta~), (2.27)

.p
and using (2.21), we then obtain
E-y{(Af - Ttqa)(A~ - Ttqß)} =
= E-y(Vt)[b~ß - L:(qßb~.p + qab~ß) + L: qaqßb~"']. (2.28)

.p .p.",
Using Lemma 6 we can change the left-hand side of (2.28) to
with accuracy to within 0(1) as t -+ 00. Together with (2.24) this leads to the
formulas
(1.13)
where
baß = qV[b~ß - L:(qßb~.p + qab~ß) + L: qaqßb~"']. (2.29)
.p .p.",
Thus we have proved Lemma 3 from §1, at the same time showing the
connection between the coefficients baß and the moments b~ß (the assertion of
independence of baß of the index 'Y should be proved separately, but because
of (A), this is quite simple).
Note here the conversion 3 of (2.29) (although we shall not need it in what
follows):
(2.30)
We introduce the vectors
-
7J(n) = V(riY 1~
~[A(n) - A(n)q] = Vii L..J il(k), (2.31) .
k=1
where
il(n) = JqY[6(n) - 6(n)q]. (2.32)
Using (2.7), (2.12) and (2.29) we obtain
E-yila(n) = E-y7Ja(n) = 0, (2.33)

E-yila(n)ilß(n) = E-y7Ja(n)7Jß(n) = baß. (2.34)
Since the il(n) are independent and identically distributed, by (2.33) and
(2.34) the vectors 7J(n) have a Gaussian distribution as n -+ 00 with matrix of
second moments IlbaßII. Applying to the sums
n"
L il(k)
k=n'
a well known strengthening of Chebyshev's inequality (see [2], p.154), it is easy

to prove the following
Lemma. If a random variable Vn taking only positive integer values satisfies

the condition
3 Formula (2.30) can be proved directly in a way similar to that given for (2.29)
starting not from (2.27) but from the identity
which is trivial in view of (2.15).

as n -+ 00, where C is a constant, then for any h >0
(2.35)
as n -+ 00.
In order to pass from the vectors '1( n) to the vectors
e(t) = (Jt(t) - tq)/Vt (1.24)
introduced in §1 we can use
Lemma 7. If nt is the integer part oftq'Y, then for any H > 0,
(2.36)
as t -+ 00.
In the proof of Lemma 7 we can pass from e(t) to '1(t) via the vectors e( Tt)
and
(2.37)
To estimate Vt we must use the relation
(2.38)
obtained from (2.24) and (2.28) by setting 0' = ß = 'Y in (2.28).

By Lemma 6, (2.28) implies that for sufficiently large C and t the probab-
ility
P'Y{IVt - ntl > C..;nt}
can be made arbitrarily small. This enables us to use (2.35), setting h = H/3
and to write
P'Y{I'1(vt} - '1(nt)1 > ~H} -+ 0 as t -+ 00. (2.39)
Lemma 6, (2.38), the definition of nt and (2.37) imply that
(2.40)
Finally, by Lemma 6,
(2.41)
Combining (2.39)-(2.41) we obtain (2.36).

An application of Lemma 7 to the derivation of integral theorems will be
given in §4.
§3. Cyclic vectors and the fundamental lattice
Apart from the probabilities W-y(m) we shall also need the probabilities
(3.1)
W~(m) is the conditional prob ability under the hypothesis (0) = e-y of the
combination of the two events:
1) visiting the states eßl (ß = 1,2, ... , s) m ß times at the times t =
1,2, ... , m respectivelYj and
2) visiting the state e a at the final moment t = m.
For m = 0 we set
a
W-y (0) -
_ {10 for 'Y = a,
for 'Y # a.
(3.2)
Clearly, the probabilities W-y(m) can be expressed in terms ofthe probabilities

W~ (m) by the formula
(3.3)
a
The probabilities W~ (m) are of special interest when the upper and lower
indices are equal. They can be used to express the probability distributions for
the vectors A( n) considered in the previous section,
(3.4)
Since always
(3.5)
the distribution of A(n) under the hypothesis (0) = e-y is completely deter-
mined by the values W.J(m) with m-Y = n and for any n ~ 0
E W.J(m) = 1. (3.6)
m"Y=n
Clearly, for any cyclic vector z for which z-Y > 0, we have
W.J(z) > O. (3.7)

Clearly, the converse is also true: if for some "y (3.7) holds, then,z is cyclic. 4
Especially important for us are the cyclic vectors with z'Y = 1. They are
the only values of vectors c5(n) which (naturally, provided f(O) = ea ) have
positive probability. They are considered in
Lemma 8. The minimal additive vector group containing all cyclic vectors
with z'Y = 1 coincides with the fundamental lattice Z.
To prove this lemma it suffices to establish that any cyclic vector z can be
represented as a linear combination with integer coefficients of cyclic vectors z
with z'Y = 1. To this end we consider three cases:
1) if z'Y = 1, then our assertion is already provedj
2) if z'Y > 1, then the cycle generating z can be divided into z'Y cycles,
from the visit to e'Y up to the next (along the cycle) subsequent visit to e'Y' and
then z can be represented as
z = zl + z2 + ... + Zz"l,
where the z"correspond to the partial cyclesj
3) if Z'Y = 0, that is, the z-generating cycle
does not cointain e'Y at all, then by (A), we can find chains
so that they can contain e'Y as indicated: the first chain as first element, and
the second chain as last element. Then the chains
(e'YO,e'Yl'···' e'YO,eßl'···' eßj' e'Y,e a1 , · · · ,e ai , e'Yo),

(e'Yo' eßl , ... , eßj' e'Y' ea1 , ••• , eai , e'Yo)
are cycles, and for the corresponding cyclic vectors Zl and Z2 we obtain
Zl - Z2 = z, zl = 1, z~ = 1.
4 Note here also the interesting identity (a.lthough we sha.ll not need it in what
folIows):
This completes the proof of our lemma.

Denote by .c the smallest linear subspace of s-dimensional vector space
containing all cyclic vectors (and consequently, the fundamentallattice Z). The
dimension r of this space is called the rank of the Markov chain considered.
Denote by .co the intersection of .c with the (s - 1)-dimensional space N of
vectors x with ii = O. Since .c is not contained in N (there exist cyclic vectors
z with z> 0), it follows that dim.c = r - 1.
Lemma 9. The space .c is equal to the linear span of the values of the vectors
6( n) which have (under the hypothesis e(O) = e-y) positive probability, while the
space .co is equal to the linear span of the possible values of the vectors
A(n) = v'ifY[6(n) - 6(n)q). (2.32)
The first part of Lemma 9 follows directly from Lemma 8. The second
part can be proved in the following way:
1. Lemma 4 and the first part of Lemma 9 imply that q belongs to .c.
2) It can easily be shown that ~(n) = O. Therefore the possible values of
A( n) belong to .c.
3) Since the possible values of
1 -
6(n) = vq-Y
.r,;::yA(n) + 6(n)q
generate the whole space .c obtained by attaching to .co a vector q that does
not belong to .co, the possible values of A(n) generate the entire space .co.
Since
E-yA(n) = 0, (2.33)
E-yACl(n)Aß(n) = bClß , (2.34)
it follows almost immediately from Lemma 9 that
Lemma 10.
for x E .co,
(3.8)
for x orthogonal to .co.
Lemma 10 brings us the following conclusions (some of them were already
mentioned in §1):
Corollary 1. The rank of IIbaßII is r - 1.
Corollary 2. Condition (B) is equivalent to the condition

(Bd r = s.
Corollary 3. Condition (C) implies (B).

To eondude this seetion we will show that the determination of the fun-
damental lattiee Z and the verifieation of (C) are purely arithmetie problems
whieh allow simple algorithmie solutions. It ean easily be seen that when de-
termining the eydes it suffiees to eonsider, instead of II~II, the matrix IIO~II
where
for ~ > 0,
(3.9)
for ~ = O.
The sequenee
is a eyde if and only if
ll'Yl -
u'Yo -
ll'Y2 -
U'Yl -
-
•.. -
ll'Yo -
u'Yk_l -
1. (3.10)
A eyde will be ealled simple if it eontains eaeh state at most onee. Simple
eydie vectors corresponding to simple eydes are eharacterized by the fact that
all their eomponents are at most 1:
(3.11)
Sinee there are a finite number of simple eydes and all of them ean easily
be found, the following lemma gives a very effective way for determining the
fundamentallattiee.
Lemma 11. All the vectors in Z are representable in the form (1.30) with
simple cyclic vectors Zi.
For the proof it suffiees to note that any eyde in whieh a eertain state is
found more than on ce ean be divided into two eydes. Repeating such division,
any eyde ean be divided into simple eydes.
Let
be the system of all simple cyclic vectors. Then the above considerations im-
mediately give the following result:
Lemma 12. The lattice Z is the minimal additive group generated by the
vectors h, 12, ... ,Ihl while the space .c is the linear span 01 this system 01
vectors. The rank r 01 the M arkov process equals the rank 01 the matrix
and (C) is equivalent to the condition

(Cd The rank 0111/;11 equals s, and the greatest common divisor olthe sth
order determinants that can be lormed lrom its rows (1;, 1;, ... ,I;) is equal to
1.
Example. Let s = 5 and

0 1 0 1 1
0 0 1 0 0
1I0~11 = 1 0 0 1 0
0 1 0 0 1
0 1* 1 0 0
It can easily be verified that in this case the simple cycles consist only of
three or four states. Here is the complete table of them
(el,e2,e3,el) h = (11100)
(el,e5,e3,ed 12 = (10101)
(e2,e3,e4,e2) h = (01110)
(e3,e4,e5,e3) 14 = (00111)
(el,e4,e2,e3,el) 15 = (11110)
(el,e4,e5,e3,el) 16 = (10111)
(el,e5,e2,e3,el)* h = (11101)*
(e2,e3,e4,e5,e2)* fs = (01111)*
The vectors Ig corresponding to the cycles are given in the right column. Since
h 1 1 1 0 0
12 1 0 1 0 1
h 0 1 1 1 0 = -1,
14 0 0 1 1 1
15 0 1 1 1 1
r = 5 and (e) holds.

If in the matrix Beeil of this example e~ =
1 is replaced by e~ 0, then =
only the first six of the simple cycles /g will remain and it can be verified that
the rank is lowered to r = 4.
§4. Integral limit theorems
By Lemma 10 the distribution ofthe vectors ~(n) and '7(n) is totally concen-
trated on the space .c. Since the form b(z) on this space is positive, it follows
that as n -+ 00, '7( n) obeys a Gaussian distribution which is non-degenerate
on .co and corresponds to b(z). We denote by p(z) the probability density of
this Gaussian distribution. Then by Lemma 7 we have the following result.
Theorem 2. Under condition (A), tor any 'Y and any domain G the intersec-
tion 0/ whose boundary with .co has measure 0 in .co we have
p-y{e(t) E G} -+ f p(z)dz, (4.1)

JGnt.o
as t where dz is the volume element in .co.
-+ 00,
In the particular case when (B) holds, .co coincides with N and Theorem
2 gives Theorem 1 stated in §1.
§5. The basic identity and local limit theorem in

the non-degenerate case
The version of Doeblin's method which allows us to reduce locallimit theorems

for Markov chains to local limit theorems for independent terms is based on
the following identity:
W-y(m) = L W;(m -/)W-y(1). (5.1)

/'"'1=0
The sum on the right-hand side extends over all integer vectors I with non-
negative components satisfying the conditions
(5.2)
l"Y=0. (5.3)
Identity (5.1) is self-evident from the probabilistic viewpoint.

It is easily checked that the sum of the probabilities W-y(l) over all vectors
I satisfying (5.3) is given by
L 1
W-y(l) = Eyc5(n) = - . (5.4)
/"'=0 q-Y
Under the additional restrietion (5.2) this sum may become somewhat smaller,
but because ofthe absolute convergence of (5.4) this reduced sum tends to l/q-Y
as the m tend to infinity.
Q
If (A) and (C) hold, then by Lemma 8, the minimal additive group gen-
erated by the values of the vectors c5(n) with positive prob ability (under the
hypothesis (0) = e.,) consists of all integer vectors, for any 'Y. For the differ-
ences of the values of the vectors c5( n) having positive prob ability, m-y is always
zero (since for the possible values of c5(n) themselves m-y is always 1), that is,
they belong to the group Q-y of all integer vectors m with m-y = O. It is easy
to see that in fact they generate the entire group: if the vectors
ml - mo, m2 - mo, ... , mk - mo, ... ,
where mo, ml, ... , mk, ... are all the possible values of c5(n), did not generate
the whole group Q-y, but only a proper subgroup of it, then after adding one
vector mo they could not generate the whole group Q, whereas the group
generated by the vectors
clearly coincides with the group generated by the vectors
that is, in view of (C), with the whole group Q of integer vectors. As a result,
we have the following:
Lemma 13. Under conditions (A) and (C) the minimal group generated by
the differences 0/ the values 0/ the vectors c5(n) coincides with the group Q-y 0/
all integer vectors m with m-Y = O.
From Lemma 4, the corollary of Lemma 5 and Lemma 13, and using the
limit theorem [6] applied to the sums
..\(n) = 15(1) + 6(2) + ... + c5(n) ,

we obtain
Lemma 14. 11 (A) and (e) hold, then
(m'Y)(6-1)/2W;(m) = P'Y(y) + 0(1), (5.5)
as m'Y -+ 00, where

(5.6)
P'Y(y) is the Gaussian probability density in the space Q'Y corresponding to zero
mean values and covariance matrix IIb~.8I1, and the estimate of the remainder
term is uniform lor
(5.7)
with any fixed C.

Noting that under condition (5.7) and as m'Y -+ 00,
we can express (5.5) in the form
(5.8)
This estimate must now be applied to W,J(m - I) in (5.1). Because of the

convergence of (5.4), on the right-hand side of (5.1), as m -+ 00 we can confine
ourselves to the terms for which the ratios
are arbitrarily dose to 1. Then from (5.1), (5.4) and (5.8) we obtain
(5.9)
as m -+ 00, where (5.7) can be replaced by
(5.10)
Since under condition (5.10) and as m -+ 00,
x= rm '" .;q'f(y
m-mq 1 _
- yq), (5.11)
y ~ .;q'f(x - x'Yq/q'Y), (5.12)

(5.9) can be rewritten in the form
(5.13)
Formula (5.13) acts uniformly provided that (5.10) holds. Taking this into
ac count and keeping in mind the integral Theorem 1, we obtain 5
(5.14)
Finally, the results of this section can be stated as follows:
Theorem 3. If conditions (A) and (C) hold, then as m -+ 00 and for any 'Y
(5.15)
uniformly under condition (5.10).
§6. The case when condition (C) does not hold
In this section we retain condition (A), but discard (C). Since now the funda-
mentallattice Z does not coincide with the lattice Q of all integer vectors, it
is natural to consider residues of Q modulo Z. In more detail, this means the
following. Two vectors ml and m2 will be considered congruent modulo Z if
(6.1)
All vectors with integer components are divided into classes of congruent vec-
tors modulo Z. These classes are residue classes modulo Z.
Lemma 15. Vectors m such that for fixed a and ß
W!(m) > 0 (6.2)
are congruent modulo Z.
5 Of course, (5.13) can be proved directly from the relations between the moments
b~ß and baß. Then the proof of the local Theorem 3 stated below would become
independent of integral theorems. Such a presentation, more consistent from the
algebraic viewpoint, would however be somewhat cumbersome. The factor .JS
in (5.14), as mentioned in §1, is connected with the fact that the integer points
in N are distributed with density 1/.JS.
Lemma 15 can also be stated in the following way:
Lemma 15'. All vectors m satisfying (6.2) belong to the same residue class
modulo Z. This residue class will be denoted by D~.
To prove Lemma 15, we assume that Wg(mt} > 0 and Wg(m2) > O. Then
there exist chains
for which
By condition (A) there exists achain eß, e X1 ' ex~' ... ' e Xk = ea . Clearly,
=eß, eX1 ' eX2 ' ... , e Xk = ea ),
(e a , e«>l' e«>2'· .. , e«>i
(ea,e~1,e~2,···,e~j = eß,eXl,eX2, ... ,eXk = ea )
are cycles. We denote by Zl and Z2 the corresponding cyclic vectors. Since

ml - m2 = Zl - Z2 E Z, our lemma is proved.
Clearly, always 6
D~=Z (6.3)
D~+DJ = D~. (6.4)
In complete analogy with Lemma 14, the general local limit theorem [6]
gives
Lemma 16. 1f (A) holds and r > 1, then
for m rt. Z,
(6.5)
for mEZ,
m - m'Yq/q'Y
y= ...;r;:rr , (5.6)
as m'Y -+ 00, the estimate of the remainder is uniform for
(5.7)
6 Residue classes are added in accordance with the usual algebraic ruIes.
P'Y(Y) is the Gaussian density corresponding to mean value 0 and matrix 0/

second moments IIb~ßII in the space .cl' 0/ vectors l' E .c with y'Y = 0, and w'Y
is the density 0/ the points 0/ Z in .cl"
In the same way that we replaced (5.5) by (5.8), we can rewrite (6.5) in
the form
(6.6)
For each residue dass D in the quotient group Q modulo Z we denote by

/'Y(D) the sum of all the W'Y(l) with /'Y = 0 and I E D:
/'Y(D) =L W'Y(l). (6.7)
/'Y=O
According to (5.4),
1
"L.J /'Y(D) = -. (6.8)
q'Y
D
Since
(3.3)
it follows by Lemma 15 that /'Y(D) is positive only for D that coincide with
some D~, that is, only for a finite number of residue classes D.
Treating (6.6), (5.1) and (6.7) in the same way as (5.9), we see that, as
m'Y -+ 00when (5.7) holds and m lies in D,
(m)(r-l)/2W'Y(m) = (q'Y)-(r-l)/2w'Yf'Y(D)p'Y(Y.) + 0(1), (6.9)
where
m. - mJqjq'Y
y. = y:;n:r , (6.10)
m. = m- UD (6.11)
and UD is a fixed vector in D.

It is easy to show that y. E .cl" It is necessary to replace y by Y., since y
may not belong to .cl" in which case the density p(y) is not defined for it.
In exactly the same way that we derived Theorem 3 in §5, we obtain from
(6.9):
Theorem 4. If(A) holds and r > 1, then as m -+ 00 and for m in the residue
class D we have
(m)(r-l)/2W-y(m) = w/-y(D)p(:c.) + 0(1), (6.12)
where 7
(6.13)
W is the density o/the points 0/ Z in the space .co, p(:c) is the Gaussian density
in .co corresponding to mean value 0 and matri:c 0/ second moments 116'1<,811 and
the estimate 0(1) is uniform under the condition
(5.10)
§7. The ease when eondition (A) does not hold
In the most general case the set of states el, e2, ... ,e. splits into a certain num-
ber of "dasses" Kl, K 2 , ••• , K n of "essential" states and a set R of "inessential"
states (see [1]). Condition (A) holds within each dass Kj: transitions from a
state ea E Kj to astate e,8 E Kj for i =I j are impossible as are transitions
from astate belonging to one of the Kj to astate in Rj on the other hand,
there is always the possibility of passing in a certain number of steps from a
state e a E R to astate in at least one of the Kj. Clearly transitions of the
latter kind are irreversible: having reached astate of dass K j , our system is
not able to come out of the states of this dass.
Let K be the union of all dasses K j • For ea E K we denote by Oa the set
of integer vectors m with non-negative components satisfying the conditions
ma = 1, m,8 = 0 for ß =I er, ß E K. (7.1)
Since from any state e-y E R transition to some state e,8 E K eventually occurs,
the following lemma holds:
Lemma 17. If e-y ER, then
I: I: W;(m) = 1. (7.2)
aEKmEOa
7 The definition of x. depends on the choice of the vectors UD in (5.7), hut since
f-y(D) > 0 only for a finite numher of residue classes D, this arhitranness is
immaterial in the limit theorems.
Consider the following sets of vectors m with non-negative integer compo-

nents:
M': the components m Ct of the vector m vanish for all 0: such that eCt E K;
M": there are components m Ct > 0 of m with e belonging to more than
Ct
one dass Ki;

Mi: there are components m Ct > 0 of m with e Ct E K i , but there are no
components m Ct > 0 with e E Kj,
Ct j i= i.
In the same way as Lemmas 5 and 6, we can prove
Lemma 18. There exist constants C and D >0 such that for any 'Y and any
mEM ' ,
(7.3)
We can now easily complete the study of the limit behaviour of W-y(m) as
m -4 00:
(I) If m E M' we can apply Lemma 18;

(11) If m E M", then for any 'Y
(7.4)
(111) If m E M(i), then m can be uniquely represented in the form
(7.5)
where ml E M' and m2 has components m 2 > 0 only for e Ct E K;. Then
(7.6)
It is easy to study the limit behaviour ofthe probabilities WCt (m2 - e Ct ) using
Theorem 4, since within the dass K condition (A) holds.
15 March 1949
References
1. A.N. Kolmogorov, 'Markov chains with a countable number of possible

states', Byull. Moskov. Gos. Univ. Mat. Mekh. 1:3 (1937), 1-16 (in Rus-
sian) (No. 23 in this volume).
2. S.N. Bernshtein, Probability theory, 4th ed., GTTI, Moscow-Leningrad,

1946 (in Russian).
3. M. Frechet, Recherches theoriques modernes sur la theorie des probabilites,
Paris, 1937.
4. W. Doeblin, 'Sur deux problemes de M. Kolmogoroff concernant les chaines
denombrables', Bull Soc. Math. France 66 (1938), 210-220.
5. A.N. Kolmogorov and Yu.V. Prokhorov, 'On the sums of a random number
of random terms', Uspekhi Mat. Nauk 4:4 (1949), 168-172 (in Russian).
6. D.G. Meizler, O.S. Parasyuk and E.L. Rvacheva, 'A multidimensionallocal
limit theorem in prob ability theory', Dokl. Akad. Nauk SSSR 60 (1948),
1127-1128 (in Russian).
37. SOLUTION OF A PROBABILISTIC PROBLEM RELATING TO
THE MECHANISM OF BED FORMATION *
A.B. Vistelius and O.V. Sarmanov drew my attention to a mathematical prob-

lem that they were confronted with in the statistical study of bed thickness in
geological sediments. The scheme of bed formation considered by them can be
understood from the example given in Fig. 1. The horizontal axis represents
time, and the vertical axis, the thickness of sediments and washouts. The nth
bed is deposited over the time interval (tn-l; sn) and its thickness (before sub-
sequent washouts) is en = 9n - h n- 1 • During time intervals (sn;t n ) washouts
reach the depth "In = 9n - h n ·
----------------------------
~.......~~~~-. "0_1
~ ----------------- "'---l---i----!-- hn
ffr
gz
Fig.l
As a result of these alternating sedimentations and washouts certain beds

can be washed out several times (bed 1 is washed out twice during the first and
the second washouts), and certain beds can even entirely disappear (bed 2 is
washed out completely after the second washout). Ifwe ass urne that the mean
thickness of sediments is greater than the depth of washouts, then each bed
would undergo a significant risk of washing out only during a small number
of alternating sedimentations and washouts. Therefore we may speak of the
probability of the "final" existence of a bed and of the conditional distribution
of the probabilities of the "final" thickness of the remaining beds. It is these
conditional distributions that we should compare with actually observed statis-
tical distributions of bed thickness in a certain profile. According to the data
* Dokl. Akad. Nauk SSSR 65:6 (1949), 793-796 (in Russian)
364
SOLUTION OF A PROBABILISTIC PROBLEM ON BED FORMATION 365
of A.B. Vistelius, these distributions are often highly asymmetrie, as if "cut

off" to the left of the zeroth abscissa.
Let 6n = en-7]n = h n -hn- 1 be the difference between the initial thickness
of the nth bed and the depth of the washout that occurs immediately after this
bed has been formed. In our further mathematical constructions we will assurne
that an infinite sequence of random variables 61 ,62 , ... ,6n , ••• is given and we
make the following assumptions: 1) the random variables 6n are independent
and have the same distribution law P{6n < x} = G(x);
2) the expectation M = E6n = J~oo xdG(x) is positive;
3) the distribution of the 6n is continuous, that is, it can be expressed in
terms of a corresponding probability density g( x) by the formula
G(x) = i~ g(x)dx.
The third assumption could have been omitted, thereby complicating fur-
ther the analytical tools. But in view of the applied significance of the prob-
lem, we decided to make the presentation less cumbersome by avoiding the
use of Stieltjes integrals. The second assumption guarantees 1 that the sums
dr) = 6 n + 6n +1 + ... + 6n +r
tend to +00 as the second index tends to +00,
therefore the greatest lower bounds tPn = inf«(~O), (~1), ... ,(~r), ... ) are finite
and will be attained at a certain (random) finite number r.
It can easily be seen that if tPn ~ 0 the nth bed is washed out completely,
while if tPn > 0, it retains some final thickness tPn. Therefore the problem is
reduced to determining the prob ability p = P{ tPn > O} and the conditional
distribution of tPn under the hypothesis tPn > O. It is clear from assumption 1)
that both p and this distribution do not depend on n.
Assumption 3) implies that the distribution of the random variables tPn
is continuous, that is, it can be characterized by a certain probability density
f(x). Clearly, p = Jooo f(x)dx and the conditional distribution of tPn under the
1 By a well-known theorem of A.Ya. Khinchin, 1) and 2) imply that our sums obey
the strong law of large numbers, that is,
,(r) }
P { lim _n_=M =1.
r-+oo r +1
366 SOLUTION OF A PROBABILISTIC PROBLEM ON BED FORMATION
hypothesis <Pn > 0 is given by the prob ability density
/*(z) = /(z)/p for z > 0, /*(z) = 0 for z < o. (1)
To find /*(z) and p it suffices to know the function s(z) = /(z)/p. Then
1:
/* is determined by (1) and p is computed by the formula
p =1: s(z)dz. (2)
Theorem. Under the assumptions 1), 2), 3) the Junctional s(h) = /(z)/p
(-00 < z < +00) is the unique solution oJ the integral equation
s(z) = g(z) + 1°00 g(z - y)s(y)dy. (3)
Clearly (3) is equivalent to
/(z) = pg(z) + 1~ g(z - y)/(y)dy. (4)
The proof of (4) is as folIows. From the definition of <Pn it can easily be
derived that
By assumption 1), the quantities 6n and <Pn+l are independent. Therefore,

representing J(z) as
/(z) = ph(z) + (1 - p)h(z), (6)
where h(z) and h(z) are the conditional densities for <Pn under the hypotheses
<Pn+l > 0 and <Pn+l < 0 respectively and noting that the conditional density
for <Pn+l under the hypothesis <Pn+1 < 0 is 2
/**(z) = J(z)/(1 - p) for z < 0; /**(z) = 0 for z > 0 (7)
2 Here we set p < 1. When p = 1, g(x) = 0 for x < 0, f(x) = g(x) and (4) holds
trivially.
SOLUTION OF A PROBABILISTIC PROBLEM ON BED FORMATION 367
we obtain
= g(z),
1
I1(Z) (8)
Ja(z) = . 1 00
-00g(z - y)/",*(y)dy = 1-
1P -00
0
g(z - y)f(y)dy. (9)
Formula (4) now follows immediately from (6), (8) and (9).
To complete the proof of the theorem we now only need to establish the
uniqueness of the solution to (3). Consider the iterated kerneis
= g(z - = 0 for y > 0;

I:
K1(z, y) y) for y< 0; K1(z, y)
Kr(z,y) = K1(z,z)Kr_1(z,y)dz.
All these kernels, as weIl as the functions
I:
So(z) = g(z),
sr(z) = Kr(z, y)g(y)dy =
=JrJ.f. ..jU1+U2+
[ ... +u/c<0 g(Ul)" .g(ur)g(z - Ul - ••• - Ur)dUl" . dUr
k=1,2, ... ,r
are non-negative. A formal solution to (3) is of the form

00
s(z) = LSr(Z). (10)

r=O
To see that it really is a solution, and the unique one, it s1,lffices to prove
the convergence of the series E~o Ir, where
The convergence of (11) can be derived from the probabilistic meaning of

its terms. Denote by A~) the event {<Pn = d;)}, which can alternatively be
written in the form
A~) = { bnH + bn+k+l + ... + bn+r ~ 0, k = 1,2, ... ,r.

<Pn+r+1 ;::: o.
368 SOLUTION OF A PROBABILISTIC PROBLEM ON BED FORMATION
Clearly,
P{A~)} =
= P{ 4>n+r+l ~ O}P{ 6n +k + 6n +k+l + ... + 6n +r :::; 0; k = 1, ... ,r} =

=plr .
Since the events A~), r = 1,2, ... , form (to within probability 0) a com-
plete system of pairwise incompatible events, it follows that
(12)
This essentially solves the problem posed at the beginning of the article:
formula (10) gives the function s(x). Using this function we can compute I*(x):
1* (x) = s(x) for x > 0; I*(x) = 0 for x < 0, (13)
while the probability pis given by (12). We can estimate the remainder terms
of the series (10) and (12). The use of these series for computing numerical
resuIts is somewhat cumbersome but entirely possible.
Fig.2
Figure 2 plots /(x) and I*(x) for the case
.1.( ) __1__ .,2/2

g(x) = 4>(x - a), 'I' x - . rn=e
v27r
for a = 1 (this corresponds to p = 0.82). The fact that I*(x) is obtained by

"cutting off" s(x) via formula (13) explains the qualitative features observed
in reality.
2 March 1949
38. UNBIASED ESTIMATORS*
The article considers a number of problems on finding unbiased estimators

cP(Zl, Z2, ... , Zn) for various functionals f(P), which depend on the distribution
law P of observables Zl, Z2, ... , Zn. Some of these problems are connected with
questions of statistical control and inspection of industrial mass production.
§1. General definitions and theorems
Suppose that we are given a system ~ of "admissible" prob ability distributions
P(A) = P{z E A}
for a random point Z in a certain space X, and a functional f(P) defined on
~.
Definition 1. A function cP( z) defined on X is called an unbiased estimator

for f(P) if for any distribution P in ~ 1
EP cP = f(P). (1)
For example, if X is the n-dimensional space of points
and ~ consists of distributions P given by the prob ability densities

n
p(zla) = (211')-n/2 exp{ -l ~)Zk - a)2},
k=l
then a well-known unbiased estimator for
f(P) = a
* Izv. Akad. Nauk SSSR Sero Mat. 14:4 (1950), 303-326.

1 In (1) and in what follows EP denotes the expectation corresponding to the
distribution P:
EP =[ q,(x)P(dx).
369
370 UNBIASED ESTIMATORS
is the arithmetie mean
while
<p = i 2 - 1/n
ean serve as an unbiased estimator for
f(P) = a2 ,
ete.
We shall see that in many important eases there are no unbiased estima-
tors. In these eases the following definition is helpful.
Definition 2. Functions <p+(x) and <p-(x) are ealled the upper and lower
estimators for f(P) respeetively if for any distribution P from ~
EP <p+ ~ f(P), (2)
EP <p- ::; f(P). (3)
On the other hand, in most problems where unbiased estimators do exist,

there are many of them. Thus in the example eonsidered above, any linear
form
with eoeffieients satisfying the eondition
Cl + C2 + ... + Cn = 1
ean serve as unbiased estimator for a.
The exeessive diversity of unbiased estimators ean be signifieantly de-
ereased if we confine ourselves to unbiased estimators that are expressed in
terms of properly chosen suffieient statisties of the problem. To formulate the
appropriate general theorems, I shall have to give a somewhat generalized def-
inition of a sufficient statistie. Sinee in this definition a sufficient statistie ean
be not only scalar, but also veetor-valued, we do not need to introduee the
UNBIASED ESTIMATORS 371
not ion of "a system of sufficient statistics": in our general definition such a
system of statistics (Xl, X2, ... , XI) can be considered as a single statistic
The following, somewhat abstract formulations will be made more concrete in

§2 and §3 which the reader should refer to should he find this formulation too
complicated.
Let X( x) be a function given on X and with values in some other space
H. Consider the conditional probability distributions
P(Alh) = P{x E Alx(x) = h} (4)
and the conditional expectations 2
Er 4J = L 4J(x)P(dxlh) (5)
for a fixed value

X(x) = h.
Definition 3. A function X(x) is called a sufficient statistic for the system

of distributions ~ if the conditional prob ability distributions P(Alh) do not
depend on the choice of P from ~.
Since, in accordance with [1], the conditional probabilities P(Alh) are

only defined to within a set of values h taken by X(x) of prob ability 0 (for the
distribution P for x), the exact meaning of Definition 3 is as folIows: X(x) is
called a sufficient statistic for ~ if there exists a function Q(Alh) of A S;; X
and hE H that for any P from ~ can be taken for P(Alh).
Theorem 1. If X(x) is a sufficient statistic for~, then for any 4J(x) whose
expectation EP is finite for all P E ~,the conditional expectations Er 4J do not
depend on P E ~.
As with Definition 3, the exact meaning ofTheorem 1 needs to be clarified.
It is as folIows: for any function 4J(x) whose expectation EP 4J is finite for all P
in ~, there exists an M(h) that for any P E ~ can be taken for Er 4J.
2 The integrals in (5) should be understood in the sense of (10) and (11) of §4,
Chapter 5 of my book [1].
To prove Theorem 1 it suffices to set
M(h) = L </>(x)Q(dxlh), (6)
where Q(Alh) is taken from the explanation to Definition 3 and the integral is
understood in the sense of the footnote to (5).
The following two theorems give a generalization of Blackwell's results [3].
Theorem 2. If </>(x) is an unbiased estimatoT fOT f(P) and x(x) a sufficient

statistic fOT 'iJ, then 3
</>*(x) = Ex(x)</>(x) = M[x(x)] (7)
is also an unbiased estimatoT for f(P).
Theorem 3. FOT any P E 'iJ for which the variance oP </> exists, under the
conditions of Theorem 2 we have the inequality
(8)
To prove Theorems 2 and 3 it suffices to use the identities
E(E x </» = E</>, (9)

O</> = O(Ex </» + E(</> - EX</»2, (10)
which hold for any functions </>(x) and X(x) provided that E</> and O</> exist.
Theorems 2 and 3 can be considered as a substantiation of the natural ten-
dency to use only unbiased estimators that are expressed via sufficient statis-
tics of the problem: Theorem 2 shows that in doing this we do not restrict the
number of problems for which there exist unbiased estimators, and Theorem 3
demonstrates that when we pass from an unbiased extimator </> to the averaged
estimator </>* expressed in terms ofthe statistic X, we can decrease the variance.
It can be shown that the estimator </>* is always "no worse" than the estimator
</> generating it, even when using other methods for comparing the "quality" of
estimators.
3 The superscript P to Ein (7) is omitted, since by Theorem 1 q,* can be chosen
independent of P.
Moreover, as we shall see, Theorem 2 gives an effective tool for finding

unbiased estimators with low variancesj often, by averaging "bad" but easily
found estimators tP one can get "good" estimators tP... In a rather wide dass
of cases unbiased estimators expressed in terms of properly chosen sufficient
statistics appear to be uniquely determined by the functional f(P) to be esti-
mated. This is discussed in an interesting work by Halmos [4] which represents
one of a few attempts to approach unbiased estimators from the general view-
point of sufficient statistics. Some results of Halmos will be given in §10 of this
paper.
§2 Main formulas for the discrete case
If the set X of possible values of z is finite.,
and the distributions pI of the system ~ are uniquely determined by one or

several parameters 0, 4 then it is natural to introduce the probabilities
represent the functional f(P) by the function f(O) and set
for the possible values of the estimator tP. The condition that tP is an unbiased
estimator of f(O) is then expressed as
n
L tPkPk = f(O). (1)

k=l
Formula (1) shows that in the case when 0 runs through an infinite number
of values, only few functions f(O), namely those expressed by linear forms (1),
allow unbiased estimators tP. If the Pk(O) are linearly independent, then for
each function f(O) which allows unbiased estimators there exists only one such
estimator.
4 In the case of several parameters 81. 82, ... , 8n , we shall denote by 8 the vector
(81, 82, ... ,8n ).
If the possible values of z are enumerated by pairs of indices,

X = {ahk}, 1 $ h $ m, 1 $ k $ nh,
then it is natural to give a similar enumeration to the probabilities Phk(O) and
the values t/Jhk of the estimator t/J. Then the first index h will be a sufficient
statistic of the problem if and only if Phk (0) can be written in the form
(2)
where the qhk do not depend on 0,
nh
qhk ~ 0, Lqhk = 1,
k=l
m
Ph(O) ~ 0, LPh(O) = 1.
h=l
In case (2), in accordance with Theorems 2 and 3 we obtain from the unbiased
estimator
for /(0), the unbiased estimator

nh
t/J"(ahk) = t/J'h = L qhkt/Jhk (3)

k=l
with variance
for any 0.
The proof of Theorems 2 and 3 which, in the general case, though ex-
tremely brief and simple, was based on a somewhat difficult general theory of
conditional probabilities and expectations, now becomes, for the special case
under consideration, purely arithmetic, since the expectations and variances
under consideration can be expressed by the known formulas:
m nh m
E8t/J = LLPhk(O)t/Jhk, E8t/J" = LPh(O)t/J'h,

h=lk=l h=l
m nh
08t/J = L LPhk(O)(t/Jhk - E8t/J)2,

h=lk=l
m
E8t/J" = LPh(O)(t/J'h - E8t/J")2.

h=l
§3. Main formulas for the eontinuous ease
Let the distributions pB from ~ be given by prob ability densities
p(xIO).
In this case the condition for ljJ(x) to be an unbiased estimator for 1(0) is
L
written in the form
ljJ(x)p(xIO)dx = 1(0), (1)
that is, the determination of an unbiased estimator ljJ(x) for a given function
I( 0) reduces to the solution of an integral equation.
Suppose further that a function X(x) is given with values in aspace H.
This function divides X into subsets Vh on which X takes constant values,
x(x) = h.
We assurne that the volume element dx can be expressed in the form
where dh is the volume element in Hand dXl is a suitably defined volume

element in Vh. Then X is a sufficient statistic of the problem if and only if the
probability densities p(xIO) can be expressed in the form
p(xIO) = p(h(x)IO)q(x), (2)
where q( x) does not depend on 0 and
q(x) ~ 0, f q(X)dXl = 1,
JVh
p(h) ~ 0, [P(hIO)dh = 1.
In case (2), in view of Theorems 2 and 3, we can obtain from an unbiased

estimator
1jJ= ljJ(x)
for 1(0), the unbiased estimator
1jJ* = 1jJ*(x) = 1 V X (.?:)

ljJ(x)q(X)dX l (3)
with variance
(4)
for any 0, this variance depending only on x( z):
<p* = s[x(z)].
§4. Unbiased estimators of random variables

In this section we further generalize the problem of §lj this generalization will
be used in §5. Given a system i;J.l of admissible probability distributions
P(A) = P{(z, y) E A}
for the pairs consisting of an "observable" point zEH and a random variable Y
that cannot be observed directly, a function <p( z) is called an unbiased estimator
of Y for all P in i;J.l if
(1)
The question of finding such unbiased estimators for random variables does not
involve new difliculties, since it is equivalent to finding unbiased estimators for
f(P) = EPYj
only now, to characterize the accuracy of the estimator <p for Y it is natural to
consider the expression
instead of the variance oP <p.

In the case when z and Y are independent for any P in i;J.l we have
(2)
and since oP y does not depend on the choice of <p, the problem of finding an
unbiased estimator <p for y with least value of EP(<p-y)2 is equivalent to finding
an unbiased estimator for EP y with minimal variance oP <p.
§5. Inspection by qualitative features based on a single sampie:

the case of destroying the product item while testing
In order to show, by means of a sufliciently specific example, the value of the

theory of unbiased estimators in inspecting mass production based on selected
data, we consider two problems from this field in this section and in §7. In this
paper we shall not go into details on the practical application of the suggested
method. In this section we assurne that we destroy the industrial product item
by testing and therefore, in principle, this testing can only be of a sampIe
character. The inspection system under consideration is as folIows.
Random sampling of n items is made from a batch containing N items.
These n selected items are tested and the number x of "defective" items in the
sampIe is found. If x :::; c, then the N - n items not included in the sampIe are
accepted. If x ~ d = c + 1, then the whole batch is rejected.
If there are y defective items in the batch before testing, then the number
of accepted defective items will be
y-X for x :::; c,
y* = {
o for x ~ d.
We set
q=y/N, q*=y*/N.
The aim of inspection is to guarantee that the q* are small enough, without
causing unnecessary, that is, for small q, too frequent rejection of the entire
batch, and without excessively increasing the sampIe size n.
To satisfy all these requirements, appropriate values of n and c must be
chosen. The main characteristic of the inspection system with given n and c is
the conditional prob ability
L(q) = P{x :::; clq}
of obtaining x :::; c for a given q, that is, of accepting the batch. To compute
L(q) we use the conditional probabilities
n! (N - n)!N n ( 1)
Pm(q)=P{x=mlq}= m!(n-m)! N! q q- N x
x(q- ~) ... (q- m;l)(1_q)(l_q_ ~) ...

n-m-1)
... ( 1-q- N (1)
of obtaining x = m for a given q. It can easily be seen that
L(q) = L Pm(q). (2)

m~c
If ,it is necessary to reject batches with q < qo and to accept those with
q > qo, ~hen the ideal operative characteristic for L(q) would be the function
for q < qo
L(q) = {~ for q > qo,
given in Figure 1. This form of L(q) can be achieved only for n = N, which in
case the tests are of a destructive character, would make the entire operation
meaningless. However, we come dose to the ideal characteristic in Figure 1 if
we choose a sufliciently large c and take
n "" c/qo.
If N is very large, this does not lead to an excessive increase in the ratio n/N
which, of course, should remain sufficiently small. Figure 2 gives operative
characteristics for the case 5
N = 10,000, n = 1,000, c = 20, qo "" c/n = 0.02.

Unfortunately, quite satisfactory operative characteristics even for N of
the order 1,000-10,000 can often be obtained only at the expense of increasing
the ratio n/N. Therefore, we often use inspection systems with operative
characteristics that do not faU steeply enough as q grows.
lJIJ
/
tJ f/J !l Il, tJ!
Fig.1 Fig.2.
In these cases it becomes especially important to estimate the proportion of

actually accepted defective items based on the test results. When production
5 Sampling inspection. Statistical Research Group, Columbia Univ., 1948.

runs smoothly enough this proportion is often much smaller than the proportion
of defedive items qo which has been used for computing the inspedion system.
Subsequent estimations of the proportion of defective items in the products
to be checked based on test results and the proportion of defective items that
were accepted using this inspedion system, is certainly very important also in
the case of quite satisfactory operative characteristics. This is the problem we
will solve now.
Assume that the inspection system with given n and c is applied to a large
number S of batches. The ratios q and q* introduced above will be denoted by
qr and q; for the rth bateh. Moreover, Sm will be the number of batches with
X = m (where m is the number of defedive items found in a bateh). Clearly,
S = So + SI + ... + Sn,
while
S' = So + SI + ... + Sc
denotes the number of accepted batches. The total number of items in S batches
lS
R=sN,
while the number of accepted items is
R' = s'(N - n).
The total number of defective items is
y = LYr = NLqr,
r=l r=1
while the number of accepted defective items is
3' 3
y' = LY~ = NLq;·

r=1 r=1
The success of the entire inspection procedure is measured by how much the
ratio
q:nean = y' / R'.
is less than the ratio

Since
where
y' 1 3
*
qmean = R = -; ~ *
L...Jqr,
r=l
while the ratio R/ R' becomes known after acceptance and is dose to 1 for any
normal production run, we can take it that our problem is to estimate qmean
and q~ean.
If the number S of batches is large enough, and if tfo(x) and tfo*(x) are
unbiased estimators of q and q* respectively with respect to x, then by the
law of large numbers, qmean and q~ean can be estimated using the approximate
formulas
(3)
(4)
The accuracy of these formulas is determined by the standard methods, since

the terms on the right-hand si des of the equalities
1 3
qmean - tfomean = - L[qr - tfo( x r )],
S r=l
q~ean - tfo~ean = ~ t[q; - tfo*(xr)]

r=l
are independent. This is discussed at the end of the next section.

An unbiased estimator for q is weIl known:
tfo(x) = x/no (5)
It leads to the estimate
1
L
n
qmean ....., tfomean = - mSm (6)
ns m=l
for qmean.
Since always 6
(7)
it follows that qmean can also serve as an upper estimator (see Definition 2
of §1) for q~ean' However, this well-known estimator does not indicate the
decrease in the number of defective items as a result of inspection. To obtain an
idea about more efficient estimators of q~ean we consider the general question:
which functions /(x) have unbiased estimators with respect to the number of
defective items x in a sampIe of n items. This quest ion is solved very simply
in accordance with §2. For any function ifJ(x),
n
/(q) = EqifJ(x) = L ifJ(m)Pm(q) (8)

m=O
is a polynomial in q of degree at most n. Since in this case the n+ 1 polynomials

Pm(q) are linearly independent, any polynomial /(q) of degree at most n can be
uniquely expressed in the form (8). In other words, polynomials /(q) 0/ degree
at most n, and only these polynomials, have unbiased estimators with respect
to x, and such an estimator is unique1y determined by the polynomial /(q).
It can easily be verified that
Q(q) = Eqq* = L (q - ~)Pm(q) (9)

m:5 c
is a polynomial of degree n + 1. Therefore an accurate unbiased estimator ifJ* (x)

for q* with respect to x does not exist. However, in the following section we shall
see that the problem of finding such an estimator allows, under rat her general
assumptions, a very simple approximate solution. Moreover, the follo~ing two
remarks are of interest.
1. For any n' < n there exists an unbiased estimator for the number of
defective items x in a sampIe of n items for q* that would be obtained from
inspection over sampIes of n' items (with any c).
2. By approximating Q(q) by polynomials Q+(q) and Q_(q) of degree at
most n such that
we can obtain upper and lower estimators ifJ+ (x) and ifJ*- (x) respectively for q* .
6 More preClse
. Iy, *
qmean < 1 ~8
_ qmean - 6N L.....r=l Xr·
This method promises acceptable practical results for many cases in which
the approximate formulas of the next section are inapplicable.
§6. The case c «: n «: N

If n is small compared to N, then we may set
. {q
q =
for x ~ c,
o for x ~ d,
in a sufficiently good approximation, and replace (1), (2), (9) of §5 by
I
Pm(q) = m.n
I( n~ m.
)l qm (1- qt- m , (1)
L(q) = L Pm(q), (2)
Eqq" - Q(q) = qL(q). (3)
Assuming that we can use (1) for the variance
of the estimator (5) of the previous section, we obtain the unbiased estimator
1jJ2 = x( n - x) , (4)
n 2 (n - 1)
found by Girshick, Mosteller and Savage [5]. From this we obtain the unbiased
estimator
1 n-l
ß2 = sn
2 2( n- 1) L
m(n - m)sm (5)
m:1
for the variance of <Pmean.
For m small compared with n the probabilities Pm(q) are non-zero only
for small q, and (1) can be replaced by the simple approximate formula
(6)
Since by (6),
m+1
qPm(q) = --Pm+1(q),
n
under the assumption that c is small compared with n, (2) and (3) give
1
Q(q) = -
n
L mPm(q). (7)
m~d
Thus when we can use (3), (6) and (7), the function Q(q) is a linear
combination of the Pm(q) and, in accordance with §2, we are able to construct
an unbiased estimator for Q(q) (and consequently, for q*) with respect to x.
This estimator has the form
4J*(x) = {~/n for x ~ d = c + 1, (8)

for x ~ d+ 1.
This gives the following unbiased estimator for q~ean:
(9)
It is instructive to compare (9) with formula (6) in §5. Comparing these

formulas we see that for large s the difference
*
4Jmean - 4Jmean w mSm
= -1 '"'
ns
(10)
m>d
is approximately equal to qmean - q~ean or (when R/ R' is elose to 1) to qmean -
q:nean' that is, the decrease of the proportion of defective items in the process
of inspection.
We can quite effectively estimate the accuracy of the approximate estima-
tor (9). The independence of the sampies from different batches (this indepen-
dence is implicit in the notion of a "random sampie") implies that
(11)
Since
(q - x/n)2 for x ~ c,
{
(q~ean - 4J~ean)2 = ~x/n)2 for x = d = c+ 1 (12)
for x ~ d+ 1,
it follows that
or, after appropriate transformations,

d
Eq(q~ean - 4J~ean)2 = :2 [L: mpm(q) + d(d + l)Pd+1(q)]. (13)

m=l
Therefore an unbiased estimator of Eq(q* - </J*)2 is

x/n2 for x ~ d,
{
"pZ(x) = ~(d + 1)/n 2 for x = d + 1, (14)
for x ~ d+ 2,
while
1 •
Ä~ = s2 ~ "p2(X r ) = (ns)2
1
[f d
mSm + d(d + l)sd+1] =
</J:nean d( d + 1)
= -ns- + (ns )2 Sd+l (15)
can serve as an unbiased estimator of E(q:nean - </J:nean)2. For large s
and according to Lyapunov's theorem,
P{lq:nean - </J:neanl ~ tÄ} '" v211'

~ 1 0
t
e- t2 / 2 dt. (16)
In conclusion, we consider a numerical example in which (6) of §5 and (4),

(9), (15) and (16) of this section are applied. Let
N = 1000, n = 50,
s = 200, c = 1,d = 2.
m o 1 2 3 4 5 6 7 8 9 10 11
143 27 12 9 3 1 2 1 1 1
</Jmean = 0.0133, Ä* '" 0.0010,
Ä '" 0.0011, qmean '" 0.0133 ± 0.0011,

</J':nean = 0.0051, q':nean '" 0.0051 ± 0.0010,
or, under the 3u-rule,
Iqmean - </Jmeanl < 3Ä, Iq:nean - </J:neanl < 3Ä,

0.0100 < qmean < 0.00166, 0.0021 < q':nean < 0.0081.
I believe that for the chosen s, N, n and c the use of the formulas introduced
under the assumptions c <t:: n <t:: N and large s is already correct. This
question, however, deserves a detailed study.
§7. Inspection by qualitative features based on a single sampie:

the case when the tests do not affect the product items
If a test does not affect the items, we may accept an inspection system different
from that considered in §5.
A random sampie of n items is made from a batch consisting of N items.
The n items chosen are tested, and the number z of "defective" items in the
sampie is found. If z :5 c, then the z detected defective items are replaced by
good ("non-defective") ones, and the whole batch is accepted. If z ~ d = c+ 1,
then the whole batch is checked, all defective items thus detected are replaced
by good ones, and only after that is the entire batch accepted.
As in §5,
for z :5 c,
q
* ={qo- ziN
for z ~ d,
but now q* has a simpler meaning: it is the proportion of defective items
remaining in the batch after the above-described procedure. Formulas (1)-(9)
of §5 hold. Here Q(q) has the new meaning of expectation of the proportion
of defective items in an accepted product * under the assumption that prior
to inspection the proportion of defective items in each batch was equal to
q. Therefore we can now make an a priori of the worst possible
es~imate
average quality of a product accepted under this inspection system: whatever

the distribution of defective items over the batches prior to inspection, in the
product accepted after inspection the proportion of defective items is, on the
average, at most
QL = max Q(q). (1)
°S9S 1
In the setting of §5 this apriori estimate is impossible. In Figure 3 the function
Q(q) is plot ted 7 for
N =
1000, c 1 =
n = 100, d = 2.
=
The maximum QL 0.0035 is attained here for q 0.009. =
In addition to the number of defective items z, in this new setting of simple
inspection we can also observe the number
z- {o for z :5 c,
qN-z for z ~ d,
* Here a product can also be a set of batches (Translator's note).
7 Grant, Statist'cal quality control, 1946, p.353.
L(;)
4D~~
Il, /Ja
-
Fig.3
of defective items found during overall checking of N - n items not included

in the sampie, should such checking take place. This gives new possibilities for
obtaining unbiased estimators of q and q* .
It can easily be seen that the sum
= x+z = {;N
for x ~ e,
u
x~d
is a sufficient statistic for the problem.

Now we shall see that for any 0 <e~ n and for any funciion f(q) there
exists a unique unbiased estimator 4;( u).
Indeed, the requirement
EQ4;(u) =L 4;(m)Pm(q) + 4;(qN)[1- L(q)] = f(q) (2)

m~c
comprises N + 1 equations corresponding to the possible values of q:
q = 0,1/N,2/N, ... ,1.
Since for q = 0, 1/N, ... , e/N the factor 1- L(q) in (2) vanishes, for u ~ e
the function 4;( u) is uniquely determined by the system of d equations
L4;(u)pu(:) =f(~), m=0,1, ... ,e. (3)

u~c
For u ~ d, setting q = u/N, we obtain from (2)
4;(u) = [f(u/N) - L 4;(m)Pm(u/N)] / [1- L(u/N)]. (4)

m~c
In particular, for f(q) = Q(q) formulas (3), (4) give an unbiased estimator for
Q(q), that is, for q*.
§8. Estimators of f(a) for a normal distribution with given tT
Let Xl, X2, ••• ,Xn be independent and obey the normal distribution with prob-
ability density
(1)
We assume that u is known. Then
(2)
is a sufficient statistic for the problem. The prob ability density for x can be
expressed in the form
p(xla, u) = G(x - a, T), (3)
where
G(z t) = _1_ e- z 2/4t (4)

, 2Viii '
T = u 2 j2n. (5)
In our case (for unbiased estimators of the form </J(x) for the function f(a)) the
i:
main equation (1) of §3 can be expressed in the form
</J(x)G(x - a,T)dx = f(a). (6)
i:
For t > - T we set
</J(z, t) = </J«()G«( - z, T - t)d(. (7)
Clearly,
</J(z, 0) = f(z). (8)
Without neglecting solutions of the problem that can be of practical in-

terest, we may confine ourselves to functions </J(z) such that
1) </J(z, t) is analytic for all real z, t > -T with respect to the variables z
and tj
2) </J(z,t) has the limits
</J(z, -T) = </J(z) (9)

for almost all z.

As is known, for t > - T the function <p( z, t) satisfies the heat equation
(10)
and, under the above assumption, is uniquely defined for t > -T by its values
fez) at t = o.By (9) this implies that <p(z) is uniquely determined by fez). 8
Thus, if the problem of finding unbiased estimators for f(a) is solvable,
then, under restrietions 1) and 2) its solution is unique.
By (8)-(10) the problem offinding unbiased estimators for f(a) is reduced
to "the inverse heat conductivity problem", which is considered, for example,
in [6]-[8]. In what follows we will note only the following concerning this. If at
I:
a certain time To > T, fez) is already representable as
fez) = G«( - z,To)dF«(), (11)
I:
then the unbiased estimator <p(x) is given by the formula
<p(z) = G«( - z,To - T)dF«(). (12)
In particular, for the prob ability density itself,
p(xla, u) = G(x - a, u 2/2) (13)
at some fixed point x we obtain an unbiased estimator for n > 1 in the form
<p:c(x) = ~ e-(x-:c)2/2q~, u~ = (1- ~)u2. (14)
v2~uo n
Integrating (14) for any fixed set A on the x-axis, we obtain for
P(A) = P{x E Ala,u} = i e-(:c-a)'/2 q2dx (15)
the unbiased estimator 9

<PA(X) = __1_ [ e-(:c-x?/2q~dx. (16)
y'2iuo JA
8 Since by (3) x hits a set of measure zero with prob ability zero, the possible
ambiguity of tjJ(z) on a set of measure zero does not bother uso In all similar
cases it is legitimate to understand the uniqueness of a solution as uniqueness
to within cases having probability zero for all admissible distributions P.
9 It was Yu.V. Linnik who drew my attention to (14) and the possibility of obtain-
ing (16) from it by integration. Before this remark by Yu.V. Linnik I derived
estimators of type (16) for special kinds of sets A directly from (6).
Formula (16) is applicable for n > 1. When n = 1, we can use instead the
unbiased estimator for P(A) with respect to i = Zl of the form
for Zl E A,
(17)
for Zl 'EA.
Similarly, for n > 1 we can also give for P(A) an unbiased estimator:
(18)
equal to l/nth of the number of points Zm belonging to the set A. This

estimator has the advantage of being an unbiased estimator of P(A) also in the
case when the dass ~ of admissible distributions
P(A) = P{z E A}
is enlarged so as to indude the dass of all one-dimensional distributions P(A).

However, in the case of normal distributions with unknown a and given (1', the
estimator tP~ is far less efficient than t/JA.
§9. Estimators of f(a,tT) for a normal distribution with

unknown a and tT
We shall proceed from (1) of §8 and retain the assumption of independence

of Zl, Z2, ... ,zn, but ass urne that both parameters are unknown. From the
formula
p(Zl. Z2,···,
(
1
znl a , (1') =."fiFi
21r(1')n
[n { _ a) 2+ s2}]
exp --22
(1'
(Z - (1)
for the n-dimensional prob ability density it is dear that there are two sufficient
statistics for the problem, namely i and
(2)
The two-dimensional prob ability density for i and s is expressed by
(3)
where
1rn(n-l)/2
(4)
Kn- 2 = r«n - 1)/2)
is the (n - 2)-dimensional volume of a sphere of radius 1 in (n - 1)-dimensional
i: i:
space. The main equation (1) of §3 now takes the form
<fo(i,s)p(i, sla, u)dsdi = f(a,u)·v (5)
We will confine ourselves to finding, with the help of Theorem 2 of §1, the
solution of (5) for the case
f(a,u) = P(A) = L p(xla,u)dx. (6)
As already mentioned in §8, for P(A) there exists an unbiased estimator
for Xl E A,
for X2 'EA.
To obtain an unbiased estimator for P(A) of the form <fo(i, s) it remains
to compute the integral
(7)
where VX ,$ is the set of points (Xl, X2, ••• , x n ) in n-dimensional space satisfying
the equations
Xl + X2 + ... + X n
-=----=-----"- = i,
n
which is an (n - 2)-dimensional sphere of radius
p= vns; (8)
dv denotes the volume element in VX ,$' and q is the constant
1
(9)
For n > 2,
(10)
where <Px is an unbiased estimator with respect to i and s for the prob ability
density p(xla, 0") at x. To determine <Px we consider the volume of the annular
zone
cut out from the sphere VX ,<1 by the hyperplanes
Xl = X, Xl = + dx.
X
When
this annulus disappears and

<Px = 0;
whereas for
its radius is
(11)
its width is
{j = f6---dx
n -1 p'
p
and its volume is
<Px dx -_ K n-3 (')n-3~

q
P u -- K n-3 Jn 1P(,)n-
n-
P 4d x.
Using (9)-(11) we finally obtain
<Px = { ;n-2
Kn_3 1 1 1 1 x-x 2 -~-
-/n-l .. { - n=I (-8-)} ,
n-~
if I(x:x) I < Vn=1,

if I(x:x) I > Vn=1.
(12)
By introducing the function
for Itl ~ .;n=l, (13)

for It I ~ .;n=l,
where
c _ K n -3 Vn _ r«n - 1)/2) (14)
n - Kn - 2 n- 1- y'27rr«n - 2)/2).;n=l'
we can write 1>x as

1>x = ~fn (X ~ X). (15)
As n ~ 00, the functions fn(t) converge to
foo(t) = ~e-t2/2. (16)
v 27r
§10. Unbiased esthnators of characteristics of an arbitrary
distribution based on independent observations
The examples that we have been considering have given the reader some idea
of the variety of specific problems on finding unbiased estimators. It seems to
me that this field of mathematical statistics deserves m~re attention than it
has received so far. Concluding this article, which does not claim more than
to demonstrate by way of examples the fact that this field is underdeveloped, I
would like to point out a connection between the general approach to unbiased
estimators and sufficient statistics given in §1, and the results of Halmos' work
[4] quoted above.
Suppose that we are given a system of distributions !,po in a certain space
X o. As admissible distributions in the space X of ordered systems
where Xm E X o (m = 1,2, ... ,n), we will consider only those distributions

that are obtained in the usual way from the assumption that all the x m are
independent and obey the same distribution 0 E !,po.
The question: which functionals f(O) defined on !,po allow an unbiased
estimator of the form
1>(XI,X2, ... ,Xn),
is equivalent to the question: which functionals f( 0) defined on !,po can be
represented in the form
f(O)= r r ... Jxr
Jx o Jx o o
1>(XI,X2, ... ,xn )O(dxI)O(dx 2) ... O(dx n ). (1)
It is natural to call functionals of the form (1) functionals of degree ~ n. In

many ways they are similar 10 to polynomials of degree ~ n. For example, it
can easily be seen that they have the following property:
10 It can easily be seen that any functional /(8) that is representable in the form
(1) for n = nl is also representable in the form (1) for n = n2 > nl. It is natural
to call the functionals /(8) representable in the form (1) for n = no but not
representable in the form (1) for any smaller n /unctionals 0/ degree no.
(*) For any fh and (J2 the expression
may be expressed as a polynomial of degree ~ n in A for all A such that
(J = A(J1 + (1 - A)(J2.
Under eertain restrietions on ~o the eonverse is true: (*) implies representabil-

ity in the form (1), that is, (*) may be taken as adefinition of funetionals of
degree ~ n (cf. the definition of polynomial operations given in [9]).
This evident eriterion for the existenee of unbiased estimators for f( (J) has
led Hairnos to some interesting eonclusions. For example, if ~o eonsists of all
1:
distributions (J on the number ws for whieh the absolute moment
(3, «(J) = IW d(J
I: I:
where s is a positive integer, is finite, then the eentral moment
p,«(J) = [t - m«(J)]'d(J, m«(J) = td(J,
has an unbiased estimator of the form
if and only if
s ~ n.
(Unbiased estimators for s ~ n are well-known, see [2],§27.6.)

It is then easy to see that, in aceordanee with the general definition of
§1 for our problem of estimating f«(J) in terms of Xl, X2, •• . , Xn, there exists a
suffieient statistic X in the form of a system of values (x 1 , X2, ... , X n) eonsidered
independently of the order of the points X m (but eounting multiplieities, if
eertain X m eoineide). The functions rjJ( Xl, X2, ..• , Xn ) expressed in terms of this
suffieient statistie X are, in fact, symmetrie functions in Xl, X2, ... , Xn . This
evident remark brings us, by Theorems 2 and 3 of §1, to one of Halmos' results:
ifthere exists an unbiased estimator ofthe form rjJ(X1' X2, ... , xn) for f«(J), then
for f«(J) there also exists asymmetrie unbiased estimator rjJ*(X1, X2, ... , xn)
with varianee not exeeeding the varianee of rjJ(X1' X2, •.. , xn).
However, Halmos went even furt her than these evident results and proved
that under certain not too restrictive limitations imposed on the system '+lo,
the symmetrie unbiased estimator is unique (see [4]).
30 March 1950
References

Leningrad, 1936.
2. H. Cramer, Mathematical methods 0/ statistics, Princeton Univ. Press,
1946.
3. D. Blackwell, 'Conditional expectation and unbiased sequential estima-
tion', Ann. Math. Statist. 18 (1947), 105-110.
4. P. Halmos, 'The theory of unbiased estimation', Ann. Math. Statist. 17
(1946), 34-43.
5. M.A. Girshick, F. Mosteller and L.J. Savage, 'Unbiased estimates for cer-
tain binomial sampling problems with applications', Ann. Math. Statist.
17 (1946), 13-23.
6. A.N. Tikhonov, 'Uniqueness theorems for the heat equation', Dok!. Akad.
Nauk SSSR 1 (1935), 294-300 (in Russian).
7. A.N. Tikhonov, 'Uniqueness theorems for the heat equation', Mat. Sb. 42
(1935),199-216 (in Russian).
8. P. Levy, 'Sur un probleme de caIcul des probabilites lie a celui du re-
froidissement d'une barre homogene' Ann. Scuola Norm. Super. Pisa Sero
2 1 (1932), 283-296.
9. S. Mazur and W. Orlicz, 'Grundlegende Eigenschaften der polynomischen
Operationen', Stud. Math. 5 (1934),50-68, 179-189.
10. P. Halmos and L.J. Savage, 'Application ofthe Radon-Nikodym theorem to
the theory of sufficient statistics', Ann. Math. Statist. 20 (1949), 225-241.
39. ON DIFFERENTIABILITY OF TRANSITION PROBABILITIES OF
TIME-HOMOGENEOUS MARKOV PROCESSES WITH
A COUNTABLE NUMBER OF STATES *
The transition probabilities ~(t) we are interested in are defined for all real
t ~ 0 and satisfy the relations
~(t) ~ 0, (I)
L~(t) = 1, (H)
fJ
for 0: f:. ß,
(IH)
for 0: = ß,
L~(t)p~(t') = pl(t +t'). (IV)
fJ
In addition to these relations of an algebraic character we will assume that the
continuity condition holds:
~(t) -+ ~(o) = e~ as t -+ O. (V)
Although (V) requires continuity of the functions ~(t) only at t = 0, it

can easily be seen that the conditions (I)-(V) imply that these functions are
continuous for all t ~ O.
In the case of a finite number of states, that is, when 0:, ß and 'Y in (1)-
(V) run only through 1,2, ... , n, (I)-(V) imply (see [2]) that the ~(t) are
differentiable for all t 0 (naturally, only on the right for t
~ = 0), that is, in
particular, that the limits
d ] . 1- pa(t)
aa = - [ -p~(t)
dt t=O
= hmo
t .... t
a, (1)
a fJa-- [ ~~(t)]
dt t=O
= lim ~(t)
t .... o t
for ß f:. 0:, (2)
exist. These limits satisfy the relations
(3)
* Uch. Zap. MoskOfJ. Gos. UnifJ. Mat. 148:4 (1951), 53-59.
395
396 DIFFERENTIABILITY OF MARKOV TRANSITION PROBABILITIES
and have the weIl-known probabilistic meaning of prob ability densities of leav-
ing the ath state (for the aa) and prob ability densities of transition from the
ath state to the ßth state (for the a~). The transition probabilities J{(t) them-
selves are uniquely determined by the transition densities a~ (in terms of which
the aa are defined by (3)) as solutions of any of the two systems of differential
equations
~P!(t) = -aap~(t) + L a~~(t), (4)

"Ita
(5)
with initial values (111).

After my work [1] the systems (4) and (5) became the main apparatus
for studying transition probabilities ~(t) for a countable set of states as weIl,
that is, when the indices a, ß, 'Y in (I)-(V) run through the positive integers
1,2,3, . ... However, in the general case even the preliminary questions of the
existence of the limits (1) and (2), the differentiability of the transition proba-
bilities J{(t) for t > 0 and the applicability of formula (3) and the differential
equations (4) and (5) remained open for a long time. Apparently, if we confine
ourselves to the assumptions (I)-(V) and introduce no additional restrictions,
then the case with a countable number of states is as follows:
(A) the limits aa always exist, but there are some examples when they
are equal to 00;
(B) the a~ always exist for ß i' a and are finite;
(C) always
L a~ ~ aa, (6)
ßta
but even for finite aa there are examples when
(7)
(D) the J{(t) always have finite derivatives for t ~ 0;

(E) if all the aa are finite, then (4) holds if the equations (3) hold;
(F) there are examples when the system (5) is not satisfied, even if all
the aa are finite and all the equations (3) hold;
DIFFERENTIABILITY OF MARKOV TRANSITION PROBABILITIES 397
(G) there are examples of two different systems of functions ~(t) satis-
fying (I)-(V) with identical and finite aa and a~ satisfying (3).
Of the above statements, only (D) remains conjectural. The others were
partially proved by Doob (see [3], [4]), and are established completely in the
present paper.
Proo/ 0/ (A). The existence of finite or infinite limits was proved by Doob. We
shall give another, quite elementary, proof (Doob 's proof makes use of measures
in function spaces).
Set
- = 1"ImIDf 1 - p~(t) .
aa (8)
t-O t
Clearly, äa ~ O. If äa = 00, then
-
aa= 1- p~(t)
l'Im-....;;...=~ (9)
t-O t
and aa = ii a is the desired limit.

If ii a < 00, then for any t > 0,
(1 - p~(t))/t $ äa . (10)
Indeed, take arbitrary t > 0 and t: > 0 and choose h such that
(1 - p~(h))/h $ ii a + t:/2, (11)

p~(s) > 1- (t:/2)t (12)
for s < h. We write t in the form
t = nh+s,
where n 0 is an integer and s < h. Then
r(1-
~
p~(t) ~ {p~(h)}np~(s) ~ {1- (ii a + i)h i t) ~
~ {1- n ( ii a + i) h} (1 - i t ) ~ { 1 - (ä a + i) t } x
x ( 1 - it) ~ 1 - (ii a + t:)t,
(1 - p~(t))/t $ ii a + t:. (13)

Since f > 0 is arbitrary, (13) implies (10). Formulas (10) and (8) imply
(9). Thus the existence of the limit a~ = -aO' = -aO' is also proved for finite
aO'.
Whether the case aO' = 00 is possible remains an open question in Doob's
work. The affirmative answer is given by the following:
Lemma. Suppose that for a 2: 2 the aO' >0 are finite and are such that
1
L:-
00
< 00. (14)

aO'
0'=2
Then there exists a Markov process satisfying (I)-(V), with given aO' for a 2: 2,
al = 00,
ß
al = 1 for ß 2: 2,
a~ = aO'}
a~ = 0 for a 2: 2, a f. ß 2: 2.
The ~(t) satisfying these requirements may be constructed in the follow-

ing way. Setting
pt(t) = 4>(t), (15)
we assume that for ß 2: 2,
d ß ß
dt Pl (t) = 4>(t) - aßPl (t),
which, together with

pf(o) = 0,
gives
(16)
The requirement
L:pf(t) = pt(t) + L:pf(t) = 1
ß ß~2
leads to the equation
(17)
Condition (14) implies that the kernel
= L: e-aßT
00
k(r)
ß=2
has finite integral

1 o
00
k( r)dr = L: -.
00
a{:J
(:J=2
1
Therefore, 1 (17) has a continuous solution </J(t) for t ~ 0, which can be obtained
in the usual way via the Fourier transformation. It can easily be shown that
this solution is continuous and satisfies the conditions
</J(O) = 1, </J'(O) = -00.

To complete our construction, it suffices to attach to (15) and (16) the ~(t),
for 0' ~ 2 satisfying the equations 2
~p~(t) = aapr{t) - aap~(t),

~p~(t) = p~(t) - aap~(t),
~~(t) = p~(t) - a{:J~(t),
which can easily be integrated one by one (with initial values (111». The prob-
abilities ~(t) so constructed satisfy all the requirements (I)-(V).
Proof of (B).
Lemma. Suppose that for t ~ H,
1 - p~(t) ~ f, 1 - ~(t) ~ f. (18)
Then for
nh ~t ~ H,
where n > 0 is an integer,
p!(t) ~ np!(h)(1 - 3f). (19)
1 See E.C. Titchmarsh, Introduction to the theory 0/ Fourier integrals, Oxford

Univ. Press, Oxford, 1948, §11.5.
2 Here the second and third equations are taken from (5), and the first one from
(4).
Proof. Let 3
ri~ß
i=1.2 •...• k-l
Qk = L: p~l(h)p~~(h) ... p~:+l(h) ... P;k-l (h).

ri~ß
199-1
It can easily be seen that

n
1 - p~(t) ~ ~(t) ~ L: PkPg(t - kh) ~ (1 - t)P. (20)

k=l
P ~ t/(l- t). (21)
On the other hand,

k-l
Qk ~ p~(kh) - L: Pi ~ p~(kh) - P, (22)
i=l
which, together with (18) and (21) gives
(23)
Finally,
n
~(t) ~ L: Qk_l~(h)~(t - kh), (24)

k=l
which, together with (18) and (23), gives
~(t) ~ n~(h)(l- t - 1~ J(1- t) ~ n~(h)(l- 3t),

as required.
3 To understand the meaning of inequalities (20), (22) and (24) the reader should
remember the probabilistic meaning of Pk and Qk' These inequalities, however,
can easily be proved also in a purely algebraic way, based on (I)-(IV).
Now assume that the conditions of the lemma are satisfied for a given H
and consider an h ~ H. Let n be the integer part of H/h and let t = nh. Then
n ~ H/2h
and, by (19),
1 ~ p!(t) ~ np!(h)(1 - 3e),
that is,
~(h) < 1 < 2
h - nh(1 - 3e) - H(1 - 3e) .
Hence
ä~ = limsup ~(t) < 00. (25)
t ..... o t
For any t ~ H choose h such that
P!(h)/h ~ ä~(1 - e), (26)

hit ~ e, (27)
and denote by n the integer part oft/ho Then
nh ~ t(l- e)
and, by (19), for e< 1,
~(t) > nh ~(h) (1 _ 3e) > ~(h) (1 _ 3e)(1 _ e) >

t -t h - h -
~ ä~(1 - 4e)(1 - e). (28)
For any e such that 1 > e > 0 (28) is proved for t ~ H, where H is sufficiently
small. This, together with (25), implies that
~(t) _ -p _ p
II· m - - - aa - aa·
t-O t
Proof of(C). Inequality (6) is elementary and can be found in the work ofDoob
[3], [4].
Here is an example where (7) holds for a = 1, though all the aa are finite.
1) For 0 ~ 3 the probabilities p~(t), ~(t) for 3 ~ ß < 0 and p~(t) are
determined from the differential equations
!p~(t) = -aap~(t),
d
dt~(t) = aß+l~+l(t) - aß~(t), ß = 0 - 1,0 - 2, ... ,3,
!p~(t) = a3P~(t)
(with the usual initial values (111)), where the aa > 0 are chosen so that
1
L-<oo.
00
(29)
a=3 aa
2) For 0 ~ 2 and ß = 1 or ß > 0, ~(t) == O.

3) p~(t) == 1.
4) pHi) = e- t .
5) For ß > 1,
where
~(t) = a-oo
lim ~(t).
Since for these ~(t)
al = - [ddt p~(t)] t=G = 1,

[ddt pf(t)] t=o = 0 for all ß > 1,
it follows that
Laf < al·
ß#
It can be verified that (I)-(V) hold in this example.
Statement (E) was proved by Doob (see [4], VIII). For the above example
equation (4) fails. Indeed, in this example al = 1 and af = 0 for ß :F 1.
Therefore for 0 = 1, (4) are of the form
dpf (t)/ dt = -pf (t)

and, together with pf(O) = 0 yields
pf(t) = 0
for all ß i 1, which is absurd.
Proof of(F) and (G). This can be found in Doob's work (see [4], Theorem 2.2).
References
1. A.N. Kolmogorov, 'On analytic methods in prob ability theory', Uspekhi

Mat. Nauk 5 (1938), 5-41.
2. W. Doeblin, 'Sur l'equation matricielle A(t+.) = A(t)A(')', Bull. Sei. Math.
62 (1938), 21-32.
3. J.L. Doob, 'Topics in the theory of Markoff chains', Trans. Amer. Math.
Soe. 52 (1942), 37-64.
4. J .L. Doob, 'Markoff chains - denumerable case', Trans. Amer. Math. Soe.
58 (1945), 455-473.
40. A GENERALIZATION OF POISSON'S FORMULA
FOR A SAMPLE FROM A FINITE SET *
A box contains N balls, M of which are white and N - M black. If n balls

are taken out of the box at random, then the prob ability of obtaining m white
balls is given by
(1)
Although (1) is elementary, computations based on it are difficult, and it

is replaced by the approximate relations
Pn(m/N,M) '" P~(m/N,M), (2)
where P" is some expression that is easier to compute or use than the precise
formula for P. In what follows we indicate conditions when (2) becomes precise
in the limit, meaning that
n
I:lp-rl-+O. (3)
m=O
Most often the Laplace-type approximation is used:
P*(m/N M) = _1_ e -(m-a)2/2q 2 (4)

n 'v'2iu '
where
a = np, (5)
u 2 = np(1 - p)(N - n)/(N - 1), (6)
p=M/N. (7)
The most widely known conditions for the applicability of approximation

(4) are as folIows: 1
..\ = n/N < ..\0 < 1, a = np -+ 00, n(1- p) -+ 00, (8)
where ..\0 is assumed to be constant.
* Uspekhi Mat. Nauk 6:3 (1951), 133-134.

1 For Bernoulli's scherne, cf. P.A. Kozulyaev, Uch. Zap. Moskov. Gos. Univ. Mat.
15 (1939), 179-182.
404
GENERALIZATION OF POISSON'S FORMULA FOR A FINITE SAMPLE 405
The condition A ::; AO < 1 is not restrictive since for A ~ ~ we can consider
the "complementary" sampie from the remaining
n' = N - n
balls, for which A' ::; ~.

Therefore in all cases when n - 00 it only remains to consider the cases
of bounded a = np and bounded n(l- p). The second of these cases is reduced
to the first one by calling white balls black and black balls white. The aim of
the present note is to show that if
n- 00, a = np ::; c (9)
the following generalized Poisson approximation is correct:
P*( /N M) = a(a - A)(a - 2A) ... (a - (m - l)A) -wa (10)

n m , m!(l _ A)m e,
where
w = -Oog(l - A)]/A. (11)
Approximation (10) is also correct under more general conditions. Namely,
for its applicability in the sense of (3) it suffices that
p-O. (12)
Using the easily verified identity
a(a - A)(a - 2A) ... (a - (m - 1)A) e-wa = ~Am(l- At-m (13)

m!(l- A)m M
and the well-known duality
Pn(m/N,M) = PM(m/N,n) (14)
the problem is reduced to proving that the binomial approximation
(15)
is applicable in the sense of (3) under the condition
A-O. (16)
41. SOME RECENT WORK ON LIMIT THEOREMS
IN PROBABILITY THEORY*
Introduction
In the middle of the 1940's it seemed that the topic of limit theorems of clas-
sical type (that is, problems of the limiting behaviour of distributions of sums
of a large number of terms that are either independent or connected in Markov
chains) was basically exhausted. The monograph written by me together with
B.V. Gnedenko [1] was intended to summarize the results ofthe previous years.
In reality, however, starting from the late 1940's more papers in these classi-
cal fields appeared. This can be explained by several circumstances. First, it
became clear that from a practical viewpoint the accuracy of remainder terms
obtained so far was far from sufficient. Secondly, certain problems that were
solved earlier only under complicated and very restrictive conditions unexpect-
edly obtained very simple complete solutions (in the sense of necessary and
sufficient conditions). These include, for example, the problem of "localiza-
tion" of limit theorems, which turn out to have an exhaustive solution both
for the case of identically distributed independent summands and for the case
of the distribution of the number of separate states visited in a homogeneous
Markov chain.
Naturally, these results have been a stimulus for seeking similar complete
results for a number of other cases. Finally, the very statements of the problems
have become more refined and transparent, owing to the introduction of suitable
distances between distributions and the ideas of computing least upper bounds
of residue terms borrowed from the theory of best approximation.
This paper briefly presents the results of some recent papers that illustrate
these new tendencies. The results given in the monograph [1] are supposed to
be known to the reader, though sometimes they are mentioned in order to make
the presentation more coherent.
§1. Estimates of the proximity of distributions and various kinds of

convergence of distributions
If eisa random variable of a certain set X, then the function
P(A) = p{e E A},
* Vestn. Moskov. Gos. Univ. 10 (1953), 29-38.
406
SOME RECENT WORK ON LIMIT THEOREMS IN PROBABILITY THEORY 407
defined for an appropriate system of subsets A of X and giving the probabil-

ity that ebelongs to A, is called the prob ability distribution of the random
variable e. When X is ametrie space (for example, the number axis R l , or
n-dimensional coordinate space Rn), we assurne that P( A) is only defined on
the B-sets. As is known (in the one-dimensional case), P(A) is uniquely deter-
mined by the corresponding distribution function
F(x) = p{e < x}.
It is most natural to consider two distributions Pl and P2 to be elose if the prob-

abilities Pl(A) and P2(A) are elose for all admissible A. This understanding of
being elose corresponds to the distance between distributions
Pl(Pl , P2) = sup IPl(A) - P2(A)I,

A
where the supremum is taken over all A for which the probabilities are defined.
In the one-dimensional case, Pl (Pl , P2 ) is equal to half the total variation of
the difference F l - F2 :
Convergence of distributions Pn to the distribution P determined by the re-

quirement
will be called convergence in variation. This convergence is the strongest of

all convergences with a direct probabilistic meaning: if two distributions are
elose enough in terms of Pl, then for all real problems dealing with a limited
(though quite large) number of tests, the distributions P l and P2 are virtually
equivalent. 1
In the one-dimensional case we are sometimes concerned only with the
probabilities P(Ll) for intervals. A new distance p'(Pl , P2) might be defined
by the formula
p'(Pl , P2) = sup IPl(Ll) -
~
P2(Ll)I,
1 Let p n be the distribution of a system (6,6, ... , en) of independent random

variables ek each of which obeys the distribution P. Then it is easy to prove that
Pl(Pf,pr) :5 npl(Pl,P2), which shows that when npl(Pl,P2) is sufficiently
small, the prob ability of any given outcome of n independent tests under the
distribution Pl differs only slightly from the probability of the same outcome
under the distribution P2.
408 SOME RECENT WORK ON LIMIT THEOREMS IN PROBABILITY THEORY
where the supremum is taken over all intervals ~. However, it is easy to prove
that
Therefore the use of pi is basically equivalent to the use of
Clearly,
Convergence in the sense that
will be called strong convergence of distributions. Strong convergence is weaker

than the convergence in variation.
In practice, random variables are usually measurable and are needed only
to within a certain accuracy. From this point of view the distances PI and
P2 introduced above sometimes overestimate the practical significance of the
difference between two distributions. For example, for the degenerate distribu-
tions
I, if xE A,
Ex(A) = {
0, if x ~ A,
for a i= b we always have
whereas for small b - a it is natural to consider the distributions Ea and Eb as

being elose to each other.
This remark can be made more precise as folIows. Let p(t7) be the distri-
bution of the variable
TJ = e+ 6,
where eobeys the distribution P and 6 (the measurement error) is independent
of e and obeys the normal distribution with mean 0 and variance 0'2. We
introduce two new types of convergence by the requirements
a) Pn --+ P, if for any 0' >0
b) Pn -+ P, iffor any (T >0
These two, formally different, types of convergence appear in fact to be equiv-

alent to the weIl known weak convergence 2 of distributions.
We confine ourselves to distributions on the straight line and somewhat
arbitrarily introduce errors that obey normal prob ability distributions. One
might think, however, that in a more general setting weak convergence is in
good agreement with the idea of eloseness of distributions which takes into
ac count practical equivalence (or identity) of elose points of the set X.
Weak convergence is weaker than convergence in variation (in any met-
ric space X) and also weaker than strong convergence with respect to the
distance P2 (on the number axis). Weak convergence of distributions in any
metric space can be defined by a properly introduced distance. Levy's distance
L(Pn , P) (see [1], §9) can serve for this purpose on the number axis. However,
all such distances are artificial and non-invariant with respect to the simplest
transformations of X (for instance, shifts (translations) on the number axis).
This distinguishes weak convergence from convergence in variation and strong
convergence, defined above (for the number axis) , which are connected with
quite natural definitions of distance between distributions. 3
§2. A catalogue of possible limiting distributions.

Distribution families sufficient for uniform approximation
For a long time general limit theorems were being found ''hy feeI" . Only after
the limiting distribution for a particular case was computed was the problem
raised of finding general conditions for convergence to this distribution. The
first example of a different approach I am aware of is Levy's theorem on limiting
distributions of the successive sums
2 We say that in an arbitrary metric space X distributions Pn weakly converge

to a distribution P if Ix
fdPn - Ix
fdP for any bounded continuous function
f(x).
3 Clearly, the distance Pi is invariant with respect to all one-to-one transformations
of the set X onto itself. The distance P2 is invariant with respect to one-to-one
transformations of the straight line onto itself.
of the terms of a sequence
of independent identically distributed terms (see [1], §33). The theorem states
that the distributions of the variables
can only converge weakly to a stable distribution, and to any such distribution
(under an appropriate choice of the distribution of the terms en). It was only
much later that W. Doeblin and B.V. Gnedenko fully studied conditions under
which this convergence holds at all or there is convergence to a certain definite
stable law (see [1], §35).
Even more important is A.Ya. Khinchin's theorem, which characterizes a
dass of possible limit laws for the distributions of sums
of independent "infinitesimally smalI" terms (see [1], §24). By virtue of Khin-

chin's theorem, this dass coincides with the dass of infinitely divisible distri-
butions. 4 The theorems of Levy and Khinchin deal with weak convergence.
It is dear that passing to a stronger convergence can only narrow the dass of
possible limit laws. In reality, when we pass to strong convergence in the sense
of the distance P2 or even to convergence in variation, this dass of laws remains
the same.
There is the perfectly justifiable view that for sums of independent terms
Khinchin's theorem indicates the most general dass of limiting distributions
that are of interest. Soon after this theorem was established, B.V. Gnedenko
completed the study of conditions for convergence to any infinitely divisible
distribution (see [1], §25-27), which essentially exhausted the quest ion of the
limiting behaviour of the sums of an increasing number of independent terms
each of which is "negligible in the limit" .
For schemes with independent tests similar questions are not yet suffi-
ciently studied. Arecent paper by R.L. Dobrushin [2] can serve as an example
4 The fact that we do not include in the expression for Cn the non-random term
An (cf. (1) on p.119 in [1]) does not make any difference: Khinchin's theorem
holds in this form as weIl.
SO ME RECENT WORK ON LIMIT THEOREMS IN PROBABILITY THEORY 411
of a completely definitive study for a somewhat particularized scheme. He

studied the case of sums
where each enk takes only two values, 0 and 1, and are connected within each
sequence in a simple homogeneous Markov chain with matrix of transition
probabilities
IPnqn 1- Pn I
1- qn,
depending only on the number n of the sequence, but not on the number k of
the test in the sequence. If the enk are independent (that is, when Pn = qn)
then only the degenerate distribution f n , the normal distribution, the Poisson
distribution and distributions obtainable from these three by linear transfor-
mations 5 can serve as limit distributions for
For arbitrary Pn and qn the problem becomes much more difficult and in order
to enumerate all possible limit distributions of 7Jn a number of new ingenious
considerations are required.
The study of similar questions for the case of Markov chains with any finite
number of states (not necessarily homogeneous) was started by Koopman (see,
for instance [14]). It would be very interesting to obtain as complete results as
those of R.L. Dobrushin for s = 2, at least for the homogeneous case with an
arbitrary number of states s.
In estimating the proximity of distributions with the help of some distance
p(P1 , P2 ) between distributions we can raise the question of convergence not to
an individual distribution, but to a whole elass of distributions. For instance,
we can ask if the distribution of the sum
of independent identically distributed summands ek should necessarily be elose

to an infinitely divisible distribution in the sense of some distance p as n -+ 00.
5 This result by P.A. Kozulya.ev is contained in general theorems of §26 and §27
in [1].
More exactly, the quest ion is whether for any f > 0 there exists an N such that
for any n ~ N there exists an infinitely divisible distribution S such that for
the distribution P of the sum ( we have
p(P, S) < f.
So far there are few papers in which problems of this sort are solved in
a setting essentially non-reducible to limit theorems of the usual type. Note
that in Dobrushin's work mentioned above, the question of approximating the
distributions of the sums (n is solved (for the case he considered) also in the
sense of this kind of uniform approximation (as n -+ 00) with respect to the
distance Pl.
Dobrushin's result is quite complicated because of case-by-case checking
of many particular distributions. The principle underlying his result can be
elarified by the example of a particular case of independent variables enk con-
sidered earlier by Yu. V. Prokhorov [3]. In this case we speak about uniform
approximation in the sense of the distance Pl of the binomial distribution B np ,
given by the formula
Denote by L np the approximation to B np used in the local Laplace theo-

rem, P np the approximation to B np in Poisson's sense (used for p elose to zero),
and Pnp the approximation obtained by Poisson's approximation of n-( (wh ich
is natural for p elose to 1). It is reasonable to use each ofthese approximations
when the corresponding distance
is smaller. Prokhorov's theorem states that
f np = min{Pl (Bnp , L np ), Pl(Bnp , Pnp)'Pl(Bnp , Pnp )}

tends to zero uniformly with respect to p as n -+ 00. Thus, the family of
distributions L np , P np and Pnp is sufficient for uniform approximation of B np
as n -+ 00.
In a simpler setting, when the approximating distributions are linear trans-

forms of the normal distribution given by the distribution function
<l>(x) = -1- 1:& e-:& ~ /2dx,

.,f2i -00
and P2 is taken for the distance, these theorems on uniform approximation are
not essentially new (see §4). However, a deliberate search for families sufficient
for uniform approximation in more complicated cases has been started quite
recently, and there still remains much to be done in this direction.
§3. Local theorems and convergence in variation
Any distribution P(A) on the straight line can be expressed as
P(A) = i p(x)dx + tJl(A),
where p( x) is the corresponding prob ability density and tJl( A) is the singular
part of the distribution. If tJl == 0, then P is continuous. If at least one of the
distributions P1 and P2 is continuous, then
Therefore for a continuous limit distribution P, convergence in variation
i:
is equivalent to convergence in mean of the densities
IPn - pldx -+ O.
A number of researchers have considered local theorems of convergence

of probability densities to the probability density of the limit distribution. 6
They have usually been concerned with conditions of uniform convergence of
distributions. As was mentioned above, this is excessive from the probabilistic
viewpoint. In Yu.V. Prokhorov's work [4] it is shown that passing to the con-
vergence in variation (that is, in the case of a continuous limit distribution, to
the convergence of densities in the mean) we arrive at very simple necessary
and sufficient conditions of convergence for the case of identically distributed
independent summands. Prokhorov demonstrated that under the conditions
of Levy's theorem given above for the convergence of the distributions of the
6 See [1], §46 and §47 and the Hungarian translation of this book, in which the
formulations of local theorems are improved.
variables «(n - An) / B n to a stable distribution, two requirements are necessary

and sufficient: 1) weak convergence to S; and 2) for some n the distribution of
(n must have a non-zero continuous component.
Together with local theorems for densities, local theorems of discrete arith-
metic type for ''lattice'' distributions (see [1], §§48-50) are usually considered.
Local theorems of this type are in connection with transferring from a contin-
uous distribution with density p(x) to the "lattice"
Xk = kh+ a.
This operation can be performed, for example, by ascribing the prob ability
to each point Xk of this lattice.

If in the setting of Levy's theorem the en are lattice-like with step h, then
are lattice-like with step

hn = h/B n .
B.V. Gnedenko [5] has shown that when the step h for en is maximal and
the distributions of 1]n converge weakly to a stable distribution S, then the
variation al distance between the distribution Fn of 1]n and the distribution Sn
obtained by transferring S to the lattice of possible values of 1Jn tends to zero
as n -> 00:
The theorems of Prokhorov and Gnedenko are limited to the case of iden-
tically distributed summands. But in any case they show that more general
and complete results on "the localization of limit theorems" are possible than
it seemed before. To conclude this section, note also the papers on arithmetic
local theorems [6]-[10]. The first three of them, [6]-[8], deal with multidimen-
sional generalizations of Gnedenko's result mentioned above. The fourth, [9],
gives an exhaustive answer to the question of approximating in variation the
distribution of the number of distinct states visited in a homogeneous Markov
chain with a finite number of states. The fifth, [10], contains quite deep, though
less complete, results for inhomogeneous Markov chains.
§4. Precise estimates of remainder terms

and improvement öf approximations
In [3] Prokhorov considers not only
f np = min{Pl(Bnp , Lnp ),Pl(Bnp , Pnp ),Pl(Bnp , Pnp )},

but also
fn = sup f np
O<p<1
and shows that
where
A= (~) 1/3 ~(1 + 4e- 3 / 2 )2/3 e-l/6 = 0.42 ....
Thus, in terms of the variational distance he found an asymptotically precise
estimate of the largest deviation (for given n and variable p) of the binomial
distribution from the most suitable of the approximations L np , Pnp and Pnp .
This somewhat narrow result can serve as an example of the technique for
estimating remainder terms that have taken shape in the papers of the last
decade. There are no complete results of the same kind for sums of arbitrarily
distributed independent summands. Below we consider problems of this kind
connected with the so-called Lyapunov's ratio.
For simplicity we assurne that the sum
'=6 +6+ ···+en

of independent summands ek is normalized, that is, it has expectation 0 and
variance 1. Then "Lyapunov's ratio" takes the form
n
L = EElek - Eek13 .
k=1
It is weH known that as L --+ 0 the following limit relation for the distribution
function F( z) of the sum , holds:
F(z) --+ ~(z) = -1- jZ e- z 2 /2dz.

V2i -00
416 SOME RECENT WORK ON LIMIT THEOREMS IN PROB ABILITY THEORY
We introduce the new variables
p(/, z) - sup IF(z) - ~(z)l,

L=/
p(/) = supp(/,z) = sUPP2(F,~).

z L=/
Cramer showed that
where Cl and C2 are positive constants. Therefore it is natural to consider

p(/)/I, its least upper bound
C = sup p(1)
1
and its limsup,
c* = limsup p(/).
/ ..... 0 1
The estimates
are best possible as regards the order of L.

Clearly
c* ~ c.
After Berry's work [15], where a more particular problem was considered,
Esseen [16] proved that
1/...;2; ~ c ~ 15/2.
Of great interest is the quest ion whether more precise estimates or computa-
tions of c and c* are possible, especially for the functions
( _
c z) - s~p
p(1, z) _
- 1 - - sup
IF(z) -
L
~(z)1
'
c*(z) = limsup p(1, z) = limsup IF(z) - ~(z)l.

/ ..... 0 1 L ..... O L
If we impose certain restrietions on the character of the terms ek, then
the values of c and c*, c(z) and c*(z) may decrease. Assuming symmetry of
the distribution of the terms and imposing certain other restrictions, whose
necessity is not dear, Yu.V. Linnik [11] proved that
* 1
c = ...;2;'
Perhaps these formulas hold under the single restriction of symmetry of the
distributions of the e. The first of these formulas might also be true in the
general case. Even the conjecture
1
c = ...j2-i
has not been disproved.

It is also very important to study from the same viewpoint (of obtaining
uniformly precise or asymptotically precise estimates of remainder terms) ap-
proximations for the distributions of sums of a large number of independent
summands, which make use of the corrections to normal distribution that de-
pend on the highest moments and are expressed via Chebyshev-Hermite poly-
nomials. Nothing has been done in this direction so far. On the other hand,
a remarkable paper by S.H. Sirazhdinov [13], in which local theorems with
specifications depending on the higher moments and giving remainder terms of
order I/nOt with arbitrary large exponents aare extended to the distribution of
the number of distinct states visited during n steps in a homogeneous Markov
chain with a finite number of states.
3 September 1953
References
1. B.V. Gnedenko and A.N. Kolmogorov, Limit distributions tor sums 0/
independent random variables, Gostekhizdat, Moscow-Leningrad, 1949 (in
Russian).
2. R.L. Dobrushin, Izv. Akad. Nauk SSSR Sero Mat. 17 (1953), 285 (in Rus-
sian).
3. Yu.V . Prokhorov, Uspekhi Mat. Nauk 8:3 (1953), 135-142 (in Russian).
4. Yu.V. Prokhorov, Dokl. Akad. Nauk SSSR 83 (1952), 797-800 (in Rus-
sian).
5. B.V. Gnedenko, Dokl. Akad. Nauk SSSR 71 (1950), 425-428 (in Russian).
6. D.G. Metzler, O.S. Parasyuk and E.L. Rvacheva, DoH. Akad. Nauk SSSR
60 (1948), 1127-1128 (in Russian).
7. D.G. Metzler, O.S. Parasyuk and E.N. Rvacheva, Ukr. Mat. Zh. (1949),
9-20 (in Russian).
8. E.L. Rvacheva, Proc. Inst. Math. Mekh. Uzbek. Akad. Nauk No.10, Part I
(1953) 106-121 (in Russian).
9. A.N. Kolmogorov, Izv. Akad. Nauk SSSR Sero Math. 13 (1949), 281-300
(in Russian).
10. Yu.V. Linnik and N.A. Sapogov, Izv. Akad. Nauk SSSR Sero Mat. 13
(1949),533-566 (in Russian).
11. Yu.V. Linnik, Izv. Akad. Nauk SSSR Sero Mat. 11 (1947), 111-138 (in
Russian).
12. Yu.V. Prokhorov, Izv. Akad. Nauk SSSR Sero Mat. 16 (1952), 281-292 (in
Russian).
13. S.Kh. Sirazhdinov, Dokl. Akad. Nauk SSSR 84 (1952),1143-1146 (in Rus-
sian).
14. B.O. Koopman, 1rans. Amer. Math. Soc. 70 (1951), 277-290.
15. A.G. Berry, 1rans. Amer. Math. Soc. 49 (1941), 122-136.
16. C.G. Esseen, Acta Math. 77 (1945), 1-125.
42. ON A.V. SKOROKHOD'S CONVERGENCE *
When describing the course of areal process in time with the help of a function
I(t) of time t that takes values in an appropriate "phase space" X, it is often
natural and correct to assurne that I has only discontinuities of the first kind
(jumps). To study such processes in detail it is useful to introduce in a set D
of functions with discontinuities of the first kind (for the sake of being specific
we will consider such functions on the unit interval 0 ~ t ~ 1) an appropriate
topology.
The topology of uniform convergence, natural when studying continuous
processes, appears to be too strong when studying processes with discontinu-
ities of the first kind. For example, it is natural to require that the sequence
of functions
for t < tn ,
for t > tn ,
where t n -+ ta as n -+ 00, converge to the function
for t < ta,
for t > ta,
since In differs from I for large n only by a small shift in the jump from the
state Xl to the state X2' As is known, this convergence does not hold in the
space of uniform convergence for Xl I X2'
On the other hand, the topology in D should not be too weak, since we
wish to retain the most essential properties of I E D when passing to the limit.
For example, we wish that under the assumption that In converges to I, the
conditions
imply that
I(t + 0) - I(t - 0) = c.
Even for these two assumptions we need a new topology in D, adapted
especially to the needs of studying processes with discontinuities of the first
kind. This kind of topology is suggested in the papers [1], [2] by A.V. Skoro-
khod. In these papers Skorokhod defines a certain convergence in D (we will
call it S-convergence) for the case of real functions I(t) (that is, when X is the
* Teor. Veroyatnost. i Primenen. (Probability Theory and its Applications) 1:2

(1956), 239-247 (in Russian).
419
420 ON A.V. SKOROKHOD'S CONVERGENCE
real line) and notes that this convergence turns the set D into a topological
Hausdorff space. Further, in §2, a new definition of S-topology is given which
can be used under the assumption that X is an arbitrary metric space and it is
shown that the topological space SD arising in this way is a metric space. The
distance s(l,g) introduced in §2 for this purpose and the corresponding uni-
form topology seem to be quite natural and convenient in applications, despite
the fact that even for a complete space X the metric space sD is not complete.
At the end of §2 a simple necessary and sufficient condition for S-compactness
is given.
The results of §3 (as weIl as Theorem IV of §2) are essentially based on the
assumption that the phase space X is complete (for simplicity we will assurne
this from the very beginning).
In §3 the completion of the metric space sD is given a specific interpreta-
tion using D-curves in the space R = T x T. This interpretation is used for
proving the possibility of introducing into D another distance s· (I, g) inducing
the same Skorokhod convergence and topology, but turning D into a complete
metric space s· D. When presenting the results of this paper at the seminar on
prob ability theory at Moscow University, I raised the problem of the simplest
possible explicit construction of the distance s· (I, g). This problem was solved
by Yu.V. Prokhorov [3].
§1 contains some prerequisites necessary for the considerations of §3. The
contents of §2 could be as weIl presented without introducing the set e, using
instead a useful technique for representing a process with discontinuities of the
first kind, for example, using functions of a real variable t that are continuous
from the right:
f+(t) = f(t + 0).
§1. The class of functions D = D x

In the description ofreal processes using functions f(t) with discontinuities
of the first kind we can usually assurne that the real phenomena taking place at
the time t of the jump are fully described by the limit values from the left and
from the right: f(t - 0) and f(t + 0). It is only by mathematical convention
that the value f(t) is defined either by f(t) = f(t - 0) or f(t) = f(t + 0), or
for real functions, by
f(t) = ![!(t - 0) + f(t + 0)].

ON A.V. SKOROKHOD'S CONVERGENCE 421
Actually, it is more logical to consider functions f(O) of the symbols 0 of the

type t - 0 or t + 0, and only in the case f(t - 0) = f(t + 0) denote their
common value as f(t), thus considering f(t) undefined if f(t - 0) # f(t + 0).
In the presentation of the theory of D-curves in §3 this approach has also
formal advantages: certain additional efforts made in §1 are rewarded by a
more concise and graphie presentation in §3.
Let e be the set of symbols 0 of one of the two types:
0= t - 0, 0 < t ~ 1,
o= t + 0, 0~t < 1.
An order relation on e is defined by
1) t ± 0 < t' ± 0, if t < t',
2) t - 0< t + O.
We say that
On ~ t - 0,
iffor any f: > 0 there exists n(f:) such that for n ~ n(f:),
t - f: + 0 ~ On ~ t - 0,
and that
iffor any f: > 0 there exists an n(f:) such that for n ~ n(f:),
This convergence turns the set e into a compact topological space (see [5]).
The fact that this space is not metrizable is no hindrance to its use in further
constructions.
By definition, the dass D x consists of functions f(O) defined on e, with
values in ametrie space X and continuous in the sense that
implies
It can easily be proved that for any f E D = D x the function
f+(t) = f(t + 0)
is defined for all t in the half-interval 0 ~ t < 1, is right continuous and has a
left limit,
f+(t - 0) = f(t - 0),
at every point t ofthe half-interval 0 <t ~ 1. It is dear that the correspondence

between the functions fE D and f+ with these properties is one-to-one.
For any set M ~ e we define the "oscillation" of f E D on M by setting
wf(M) = sup p(x,x').

x,x'EM
The "intervals" of the space e, that is, the sets

[0' ,0"] = {O; 0' ~ 0 ~ O"}
are of special interest in what folIows. In particular, the interval [t - 0, t + 0]
consists of two points: its end points t - 0 and t + O. For this interval
wf[t - O,t + 0] = If(t + 0) - f(t - 0)1.
It is easy to prove
Theorem 1. A function f(O) defined on e

and with values in X belongs to
the class D if and only if for any € > 0 there exists a sequence
o = to < t 1 < ... < t n = 1,
such that for all k = 1,2, ... , n,
Wj(tk-1 + 0, tk - 0] < L
§2. S-convergence and the distance s(f, g)
Definition 1. Two functions fE D and g E D are called €-equivalent if there

exist rand sequences
0= to < t1 < ... < t r = 1, 0 = t~ < t~ < ... < t~ =1

such that for k = 1,2, ... , r, the following inequalities hold:
sup{p(f(O),g(O')) $ f: 0 E [tl:-1 + 0, tl: - 0], 0' E [t~_1 + O,t~ - on.

In what follows f-equivalence is denoted by
I.!.. g.
Definition 2. A sequence of functions In E D is called S-convergent to I ED

iffor any f > 0 there exists n(f) such that for n ~ n(f),
In.!.. I·
In what follows S-convergence is denoted by
s
In ~ f.
This is the convergence Skorokhod introduced for the case when X is the real
line.
We now set
s(f, g) = inf f.
':'9
Theorem 2. The set D with the distance s(f, g) is ametrie space.

In fact, it can easily be seen that s(f, g) is real and non-negative for any
I E D and 9 E D. Theorem 1 implies that
s(f,t) = o.
It is fairly easy to show that
s(f,g) = 0
only if I = g. The symmetry of s(f, g) is dear from the definition. The

inequality
s(f,h) $ s(f,g) + s(g,h)
holds because of the following lemma, which is easily proved:

Lemma. 11
t'
I~g, 9 '" h,
then
The following theorem is ohvious.
Theorem 3. For the convergence

s
In --+ I
it is necessary and sufficient that
s(fn, f) --+ O.
If X contains two different points Xl -::F X2, then the metric space sD is
not complete, since for
tn < ta < 1, t n --+ ta
the sequence of functions
for 0 + 0 ~ () ~ t n - 0,
for t n + 0 ~ () ~ ta - 0,
for ta + 0 ~ () ~ 1 - 0
satisfies the Cauchy criterion hut does not converge. In this connection the
following theorem is interesting.
Theorem 4. A sequence In E D satisfying the Cauchy criterion
converges to I E D il and only il the lollowing condition holds:

(*) lor any e > 0 there exists 6 > 0 such that lor any n and t E (0; 1)
wln[O + 0,6 - 0] < e, wln [l- 6 + 0, 1- 0] < e,

w* In[t + 0, t + 6 - 0] < f,
where
w* f[(),()'] = infmax{wl[(),t - O],wl(t + 0, ()')}.
t
It should be noted that the operations w" I are similar to the '"'(J that were
already used by E.B. Dynkin in [4] for studying the functions of the dass D.
Theorem 4 is dosely connected with
Theorem 5. A set M ~ D is compact in the space SD 01 Skorokhod conver-

gence if and only if the following two conditions hold:
1) there exists a compact set K ~ D such that all values f(O) of the
function I E M at the points 0 E e belong to K i
2) for any f >0 there exists 0 > 0 such that for f E M we have
WJ[O+O,O-O]<f, wJ[1-o+0,1-0]<f,
W" J[t + 0, t + 0 - 0] < f.
§3. Graphs of functions of dass D and D-curves

in the space R = T X T
Let T be the unit interval
T={t;0~t~1}
of the real line and denote by R the direct product
R=T X T = {(t, x); tE T, x EX}.
We will consider this direct product as ametrie space with distance
p«t,x),(t',x'» = max{\t' -t\,p(x,x')}.
The set rJ of points from R of any of the two types
(t,/(t - 0» or (t, f(t + 0»

will be called the graph of the function I E D. It can easily be seen that the
function
l/JJ(t - 0) = (t, f(t - 0», l/JJ(t + 0) = (t, I(t + 0»
defined on e and with values in R is continuous in the sense of §1. It maps
e onto r J, and the order on e (the relations 0 < 0' defined in §1) induces a
certain direction of travel along r J .
Quite apart from the graphs of functions I

E D = D x , we now consider
mappings of e into the space R that are are continuous in the sense of §1, that
is, functions <fo(O) of dass DR. Two functions <fo E DR and 1/J E DR are called
equivalent if there exist monotone mappings Xl and X2 (that is, mappings X
satisfying the condition X(O') ~ X(O) when 0' > 0) of e onto itself such that
With respect to this equivalence relation DR is divided into dasses ~,

which we call D-curves in R. It is natural to intro du ce the distance between
D-curves by the formula
p(~, (f) = in( sup p(<fo(O), 1/J(O)).

"'E'" 9C0
t/JE,'fi
The theory of D-curves is quite similar to the theory of continuous curves.

The distance p(~, (f) turns the set ~ of all D-curves in R into a complete metric
space.
Theorem 6. A lunction I E D is uniquely determined by the corresponding

curve
and
s(l,g) = p(j,g).
Theorem 6 gives a new interpretation ofthe distance s(l, g) and S-converg-
ence: S-convergence of a sequence of functions In to I is equivalent to con-
vergence in the sense of the distance p(~, (f) of the curves in to the curve
1.
In what folIows, it is useful to introduce the notations T",(O), e",(O) for the
components of the function
<fo(O) = (T",(O),e",(O)).
Denote by A the set of D-curves i corresponding to the functions I E D, and

by Ä the closure of A in ~. It can easily be proved that
if and only if
(Al) r",(fJ) maps () onto the whole set T;
(A2) () < ()' implies that r",((}) ~ r",((}').
By (Ad, (A2) for a function t/J E ~ E A the sets r;l(t) of those () for which
r",((})=t
are intervals of the space e, that is,
When the parameter () runs over r;l(t), the value r",(t) remains constant, while
~"'((}) runs over some sequence of points of the phase space X. Therefore the
curves ~ E Ä may be considered as the graphs of special generalized processes
with discontinuities of the first kind whose behaviour at time t can, in gen-
eral, be more complex than simple transition from the state I(t - 0) to the
state I(t + 0). Such generalized processes can naturally appear as limiting re-
strictions similar to condition (*) of Theorem 4 that prevent the accumulation
of several vanishingly small jumps at one point t. Possibly, introducing this
kind of generalized process will appear useful in certain special studies on limit
properties of random processes. Without developing this idea any furt her I use
the analysis of the construction of Ä only as auxiliary means for proving the
possibility of introducing, in the space SD, a distance s*(I,g) different from
s(l, g) and turning SD into a complete metric space. For this purpose I will
prove the following lemma:
Lemma. The set A is a set 01 type G 6 in the space (j.
Proof It can easily be shown that for given t E T and ~ E Ä the value of
is the same for all t/J E ~, that is, it characterizes the properties of the curve
~, not of its parametric representation t/J; we denote this value by w;(t). The
condition
(I') w;(t) =0 for all tE T

is neeessary and suffieient for ~ E Ä to belong to A. Therefore
(1)
n
where An is the set of ~ E Ä for whieh

1
supw1(t) ~-.
tET 'I' n
The set Ä is closed in i by definition. It ean be proved that the sets An are
also closed in i. Our lemma now follows immediately from the closedness of
the sets Ä and An and (1).
By a well-known theorem of P.S. Aleksandrov this lemma implies that we
ean introduee a new distanee p. (i, g) in A that is topologieally equivalent to
the distanee p(j, g) but turns it into a eomplete metrie space.
Setting
s·(/,g) = p.(j,g)
and keeping Theorem 6 in mind, we ean see that the following is true:
Theorem 7. In the set D a distance s·(/,g) can be introduced that is topolog-

ically equivalent to s(/,g) but turns D into a complete metric space.
Moseow, 25 July 1956
References
1. A.V. Skorokhod, Dokl. Akad. Nauk SSSR 104 (1955), 364-367 (in Rus-
sian).
2. A.V. Skorokhod, Dokl. Akad. Nauk SSSR 106 (1956), 781-784 (in Rus-
sian).
3. Yu.V. Prokhorov, Teor. Veroyatnost. i Primenen. (Probability Theory and
its Applieations) 1:2 (1956), 175-237 (in Russian).
4. E.B. Dynkin, Izv. Akad. Nauk SSSR Sero Mat. 16 (1952), 563-572 (in
Russian).
5. P.S. Urysohn, Works on topology and other fields 0/ mathematics, Gostekh-
izdat, Moseow, 1951, Vol. 2 (in Russian).
43. TWO UNIFORM LIMIT THEOREMS FOR SUMS OF
INDEPENDENT TERMS *
In what follows
where
e=6 +6 + ···+en
and the random variables ek are independent. Let ~ be the family of degenerate
distributions of the form
for x ::; a,
E(x) = {~ for x > a,
and let e be the family of infinitely divisible distributions. The purpose of the
present paper is to prove the following two theorems:
Theorem 1. There exists a constant C such that if the inequalities
hold for all f. > 0, L > 21 > 0, where Ek E ~, k = 1,2, ... ,n, then there exists
q; E e for which
q;(x - L) - 6::; cI>(x) ::; q;(x + L) + 6,
where
Theorem 2. There exists a constant C such that for identically distributed

summands ek with arbitrary distribution functions Fk(X) = F(x) there exists
q; E e such that for any x,
* Teor. Veroyatnost. i Primenen. (Probability theory and its applications) 1:4

(1956), 426-436 (in Russian).
429
430 TWO UNIFORM LIMIT THEOREMS FOR SUMS OF INDEPENDENT TERMS
The proof of each theorem makes use of a number of ideas borrowed from
papers of P. Levy (see [1], §48), W. Doeblin [2] and Yu.V. Prokhorov [3], and
is based on the following lemmas, in which
Ga,u (x) = _I_lx

. '2=
e-(x-a)2/ 2u2 dx
' G
u = Go,U'
V ~7rU -00
QF(l) = sup[F(x+I+O)-F(x)] is the "concentration function" for the distribu-

x
tion F(x) introduced by P. Levy and Cl, C2 , C3 , C4 , C5 , C6 , C7 , c' are absolute
constants.
Lemma 1. If
where 1
s = ~)1- QF,,(r)],
k
then
Lemma 2.
Lemma 3.
Lemma 4. If
then
F * Go,u(x -I) - TJ ~ F(x) ~ F * Go,u(x + I) + TJ·
Lemma 5. Let
Pml ... mn --rr k

(k)
Pm",·
1 Throughout, the index k in the sums L:k and products TIk or TI; runs through
all integers 1 :5 k :5 n.
TWO UNIFORM LIMIT THEOREMS FOR SUMS OF INDEPENDENT TERMS 431
where
1-Pk form =0
{
p~) = ~k fior m = 1, qm(k) -- ~
m! e -Pk , m = 0, 1, ...
for m > 1.
IfO ~ Pk ~ 1 then 2
L
ml· .. mn
IPml ... mn - qml ... mnl ~ es LP~'
k
Lemma 6. If 0 ~ P ~ 1, then
Lemma 7. Let
(1'2 =L (1'~, (1'~ = (1'2 + (1'5.

k
Then the inequalities

lek I ~ I, k = 1, 2, ... , n,
imply that for al/ x
The proof of Lemma 1 will be published elsewhere. 3 Lemmas 2-5 are

quite elementary and can easily be proved by the reader. Lemma 6 is due to
Yu.V. Prokhorov (see [3]). Lemma 7 easily follows from the known estimate
2 Here and in what folIows, l:ml ... mn denotes summation over all sets of positive
integers ml, ... , mn.
3 See: A. Kolmogorov, 'Sur les proprietes des fonctions de concentrations de
M.P. Levy', Ann. Inst. H. Poincare 16:1 (1958), 27-34. No. 45 in this book.
if the additional normal summand with variance O'~ is represented as a sum of

several normal summands, each of them with sufficiently small variance.
1. Pro%/ Theorem 1, general part. It suffices to prove the theorem for the
case of continuous and strictly increasing functions Fk(x). Indeed, assume that
F/c, t, land L are fixed so that the conditions of the theorem hold. Choose I'
and L' so that
L > L' > 2/' > 21
and set t' = 2t. Using only the qualitative part of Lemma 4, it is easy to
establish that for a sufficiently small s the distributions
satisfy the conditions of the theorem with L replaced by t', I', L'. It is easy
t, I,
to prove that the Fk(x) are continuous strictly increasing functions. If for the
corresponding function ~' = ~ * GIVn we prove the inequality
w(x - L') - 6' ~ ~'(x) ~ w(x + L') + 6',
I'g'
where
6' = C' max (L'
- log - e'l/5)
I' , ,
then, using Lemma 4 with = ns 2 and I = L - L', we easily find that
0'2
for sufficiently small sand C = 2C' the original function ~(x) satisfies the
inequality stated in the theorem.
In accordance with this, we take y = F/c(x) continuous and strictly in-
creasing, which allows us to consider the inverse functions x = Fk-1(y), which
are uniquely defined for all y in the interval 0 < Y < 1, continuous, strictly
increasing, and running through all real values of x.
We set
~/c(P) = F/c-l(l- P) - F/c-l(p).
Under these assumptions the ~/c(P) are continuous and strictly decreasing.
They run through all positive values ~, 0 < ~ < 00, when P runs through
o P > O. Therefore, provided
that
L fk(2/) > 2f- 4/ 5
k
(here 1 and f are the quantities occurring in the conditions of the theorem and
we assume without loss of generality that f ~ 1), the equation
L fk(AO) = 2f- 4/ 5
k
has the unique solution

AO > 2/.
Now define A and ero in the following way:
A) 11
L fk(2/) > 2f- 4/ 5 , (A.1)
k
then
(A.2)
B) 11
L fk(2/) ~ 2f- 4/ 5 , (B.1)
k
then
A = 21, ero
1 (
= ..,fiL L) -1/2 .
logT (B.2)
To determine the required infinitely divisible distribution q;, we set
and introduce the random variables

}Jk = {0 if F;;1(fk) < ek < F;;1(1- fk),
1
otherwise.
We then set
er 2 = L(1- 2fk)eri,
k
F~O)(x) = Piek < XI}Jk = O},

F~1)(x) = Piek < XI}Jk = 1}.
cI> =II *[2l kFi 1 ) + (1 - 2lk)FiO)] = EIl * [pkFi 1 ) + (l-llk)FiO)] =

k k
'"
L...J Pml ... mn II * [F(l)]m"
k * II *[F(O)]l-m"
k'
ml ... mn k k
where the probabilities
are expressed via the formulas from Lemma 5 by setting
Now, expressing the qml ... mn in terms of Pk = 2lk according to the formulas of
the same Lemma 5, we set
w= [exp L2lk(Fi 1) - E o] * G{J,I7. = L qml ... mn II * [Fi1)]m" * G{J,I7.'

k ml ... mn k
where
In the formulas for cI> and W, Eo denotes the unit distribution
for z ~ 0,
Eo(z) = {~ for z > 0,
and the powers are understood in the sense of convolution
FO = E o, F1 = F, F2 = F*F etc.
Clearly, by passing from ek to
the ak are replaced by ak =°

and cI> and W by cI>'(z) cI>(z + a) and W'(z) = =
w(z + a). Therefore it is dear that we may restrict ourselves to the case
ak = 0, k = 1, ... ,nj a = 0.
We shall also need the random variable
and its eonditional varianees
for fixed 1'1, ... ,I'n. Clearly,
O'~l ... mn = ~)1 - mk)O'~.

k
We now set 4
Noting that
~= L: Pml ... mn ~~~ ... mn * ~~~ ...mn'

ml· .. mn
w= L: qml ... m,. ~~~ ... mn *G t7 . ,
ml· .. mn
we introduee the distributions
~1 = ~ * G = t70 L: Pml ... mn ~~L.mn * ~~L . mn * G t70 '
mt···mn
~2 = L:
mt .. ·mn
Pml ... mn~~~ ...mn * Gt7ml ... mn * G t70 '
~3 = L: Pml ... mn ~~~ ... mn *G t7 ••
mt···mn
It ean easily be seen from the eondition of the theorem that
and, sinee A ~ 2/,

(3)
4 The definition of i)~~ ... mn presupposes that mk = 0 or 1. The definition of

i)~~ ...mn applies to any non-negative mk.
In both cases A) and B),

L..J
" "fk - < 2C 4 / 5 (4)
k
(5)
Using (5) and Lemma 5 we obtain
Iw - <)31 ~ L IPml ... mn - qml ... mn 1~ C5 L p~ = 4C5 L f~ ~ 8C5 f 1/ 5 .

ml .. ·mn k k
(6)
By Lemma 3 and assuming that
(7)
we have
(8)
Therefore
(9)
where E' is the sum of those Pml ...mn for which (7) does not hold. Clearly,
(10)
where
p2 = L(1 -l'k)O'I.
k
Ep 2 = L(1- 2fk)O'I = 0'2,

k
(11)
Dp2 = L2fk(l- 2fk)O'f,
k
and since O'l ~ A2, fk ~ f,
D p2 ~ 2f L(1- 2fk)O'f ~ 2fA 2 L(I- 2fk)O'~ = 2fA 2 O' 2 • (12)

k k
Using Chebyshev's inequality, from (10)-(12) we obtain
(13)
Since always 0"0 ~ A, (3/5 ::; (1/5, (9) and (13) imply
(14)
By Lemma 7 and the fact that lek I ::; A for J.lk = 1, we have
(15)
which implies that

1<1>1 - <1>21::; C7 A/0"0. (16)
From (6), (14) and (16) we obtain
(17)
where
C. = 8C5 + C3 + 4.
2. Case A). The end of the proof of Theorem 1 differs according to cases A)
and B). Let us first consider case A). In this case, by (A.2), (17) turns into
(18)
We now recall that
A = AO > 2/, (19)

'L>k =:~:::>k(AO) =2(-4/5. (20)
k k
(21)
and, consequently, the quantity

n
S = ~)1- QF,,(A)]
k=1
is bounded by
(22)
Lemma 1 and formulas (22) and (A.2) imply that 5
C1/TO 1/5
Q~ ( TO) ~ A.,fi ~ 2C1 f . (23)
According to Lemma 2, (23) implies that
Finally, from (18) and (24) we obtain
(25)
Formula (25) now immediately implies (for case A)) the conclusion of the the-
orem, and the shifts by L are not needed; they are needed only for case B).
3. Case B). In this case, by (B.2) inequality (17) takes the form
Iw- 4>1 I ~ C.f

1/5
+ 23/2 C7 L1 Yf:L
log T' (26)
According to Lemma 4, by (B.2) for
(27)
we have
(28)
Now set
C = C. + 23 / 2 C7 + 2C1C2 + C4 / log 2.
Since L > 2/, log(L/I) > log 2, it follows from (26) and (28) that
w(x - L) - 8 ~ 4>(x) ~ w(x + L) + 8, (29)
where
_ f:L
8 - Cmax Lylog(I
I,f 1/5) .
5 It is easy to verify that by (A.2), (22) and under the assumption that f ~ 1, the
additional condition R 2 2: r 2 log s holds for R = 0'0 and r = A.
As we see from (25), inequality (29) proved here for case B), holds also for
case A). This completes the proof of Theorem 1.
4. Proo/ 0/ Theorem 2. The proof of Theorem 2 is essentially the same as

that for Theorem 1, case A), where
f = ein.
Naturally, fk and Pk = 2fk do not now depend on k. Since
the Pk must be equal to
The construction involving transition from to
is now superfluous and would even have somewhat complicated the achievement
of the final result. Instead of referring to Lemma 5 it is better to refer to the
simpler Lemma 6. Since these simplifications are possible, we now give a proof
of Theorem 2 that is independent of Nos. 1, 2. We denote by F( x) = P {ek < x}
the general distribution function of the terms ek. The conditions
uniquely define the generalized inverse function F- 1 to within a countable set

of values y corresponding to the intervals where F is constant. Without loss of
generality we mayassume that
where the random variables are independent and uniformly distributed on

°
'f/k
the interval < Y < 1.

We set P = ~n-1/5,
O, if pI2 < 'f/k < 1- p12,

J.tk = { 1
otherwise,
a = (1 - p)E{6IJtk = O}, u 2 = (1 - p)D{ek IJtk = O},

Fo(x) = Piek < XIJtk = O}, F1(x) = Piek < XIJtk = I}.
Clearly,
~ = EIl *(jJkFl + (1 - Jtk)Fo] = L: C!pk(l - pt- kFt * F~-k.

k k
We give the desired infinitely divisible distribution q; in the form
.T, '"' (np)k -nPFk G

'!l' = L...J k l e 1 * na,u.
where u; = nu 2 + u3, u3 = 2n 1/ 5 A, A = F- 1 (1- p/2) - F- 1 (p/2).

As in No. 1, it suffices to consider the case a = O. Set
~l = ~ * Guo = L: C!pk(l - pt- k Ft * F~-k * Guo ,

k
~2 = L: C!pk(l- pt- kFt * G Uk * Guo , ,..2 _ n-k

vk -
_ _ ,..2
1-p
v ,
~3 = L: C!pk(l- p)n-k Ft * G u •.
k
By Lemma 6
(30)
Lemma 3, under the assumption that
--k- n I <n
I-n1-p -
4/5
(31)
implies that
(32)
therefore
(33)
where 'L' is the sum of those C!pk(l - p)n-k for which (31) fails. Clearly
L:' = P{lx - ~(1 - p)1 > n 4 / 5 },
where X = 'Lk Jtk· Therefore, using Chebyshev's inequality,
,",' < DX = np(l- p) < n-4/5 < n- 1/ 5 . (34)

L...J - n8/5 n8/5 - -
From (33) and (34) we obtain
(35)
Since for Pk = 1 and a = 0 we have lek I ~ A, by Lemma 7 we have
which implies that

(36)
Clearly,
QF(A/2) ~ 1- p/2.
By Lemma 1 this implies that
(37)
By Lemma 2, i~ follows from formula (37) that
(38)
From (30), (35), (36), (38) we obtain
which completes the proof of Theorem 2.

Moscow, 12 November 1956
References
1. P. Levy, Theorie de I'addition des variables a/eatoires, Gauthier-Villars,

Paris, 1937.
2. W. Doeblin, 'Sur les sommes d'un grand nombre des variables aIeatoires
independantes', Bull. Sci. Math. 63 (1939), 23-32; 35-64.
3. Yu.V. Prokhorov, 'On the sums of identically distributed random vari-
ables', Dokl. Akad. Nauk SSSR 105:4 (1955), 645-647 (in Russian).
44. RANDOM FUNCTIONS AND LIMIT THEOREMS * 1
Jointly with YU.V. Prokhorov
Without claiming to present an exhaustive review of the literat ure on the ques-
tion, we intend to give an outline of the first steps of the theory of random
functions, possible basic methods for a systematic construction of this the-
ory and basic problems concerning functional methods in limit theorems; in
addition, we report certain comparatively new results.
§1. Preliminary remarks on random elements of arbitrary nature
We will proceed from the axiomatics of prob ability theory given in [1]. A family
of {n, 4>, P} consisting of
1) a set n whose elements ware called "elementary events";
2) alT-algebra 4> of subsets of n;
3) a measure P(A) defined on 4> and satisfying the additional requirement
P(A) = 1,
will be called a prob ability field (the terminology corresponds to [2]). Some-
times it is also useful to assume that
(w') The measure P(A) is complete, that is P(A) = 0, B ~ A implies that
BE 4>.
In studying random variables, random vectors and other "random ele-

ments" of an arbitrary nature it is sometimes useful to leave the nature of the
basic set of elementary events w undetermined and to consider specific random
* 'Zufällige Funktionen und Grenzverteilungssätze', In: Bericht über die Tagung

Wahrscheinlichkeitsrechnung und mathematische Statistik, Berlin, 1956, pp.113-
126.
1 The papers presented by Kolmogorov and Prokhorov at the conference are dose
to those of M. Frechet and R. Fortet. In his talk given on the last day of the
conference Kolmogorov presented new results obtained by Prokhorov in between
his talk and that of A.N. Kolmogorov. Therefore the authors deemed it appro-
priate to publish the joint text of their two papers. §4 (without Theorems IV
and V) and §6 represent Prokhorov's contribution, while §§1-3, 5 and Theorems
IV and V from §4 constituted Kolmogorov's paper. Theorems IV and V from §4
were proved by Prokhorov between his talk and that of Kolmogorov.
442
RANDOM FUNCTIONS AND LIMIT THEOREMS 443
elements e as functions of w. For instance, it is customary to call any real

function ew such that the set
{w : ew < a}
belongs to <I1 for any real a, a (real) random variable (see [1]).
An arbitrary function ew with values in a set X determines a probability
field {X, <I1e, Pd where <I1e consists of all A ~ X for which
Since in any problem in prob ability theory the basic set n is considered to
be quite definite while its elements ware implicit in the formulations of more
specific problems, the index w at ew
is usually omitted, and we speak about a
"random element e of the set X". Note that under the assumption (w') the
condition
(6) if Pe{A) = 0, B ~ A, then B E <I1 e
holds automatically.
If X is a topological space, then it is natural to require for a random
element eE X that certain topologically very simple sets A ~ X are contained
in <I1e. If Xis a metric space (this space is the main one in what folIows), then
these requirements are undoubtedly quite sensible. Namely, it is reasonable to
confine ourselves to random variables satisfying the condition
(e2) any open set G ~ X belongs to <I1 e.
As is weIl known, (e2) implies that any Borel set A X belongs to <I1 e.
~
If X is the real axis, then (6) is equivalent to the usual requirement in the
definition of random variables. We can further simplify the problem if, m
addition,
(6) For any A E <I1e, Pe(A) = infG2A Pe(G),
where the infimum is taken over all open sets G containing A.
For (6) to hold for any random element ein a separable complete metric
space X satisfying (6), it is necessary and sufficient that
(w") the measure P on n is perfect (see [3]).
The quest ion whether it is worth giving a systematic construction of prob-
ability theory using perfect measures remains debatable until such a systematic
444 RANDOM FUNCTIONS AND LIMIT THEOREMS
construction is actually carried out. We note, however, that this limitation by

no means restricts the scope of real applications of probability theory, since
we have the following theorem: any complete normed Boolean algebra can be
realized as the algebra of metric types of perfect measures (cf. [4]).
Requirements (6) and (6) can be generalized in different directions for
topological spaces of a more general nature (cf. [2], §§52-54).
§2. Random functions
For the sake of definiteness we will consider complex functions x(t) of an argu-
ment t E T, where T is, in general, an arbitrary set. (Mostly studied here are
"random processes" where T is the real axis.) Two approaches to introducing
the notion of a "random function" of this kind are possible.
(I) A random function e(t) defined on T is a system of complex random

variables e(t) defined at every t E T. If in this case the basic prob ability
field {n,~, P} satisfies (w'), then the distributions Pe(t), as weIl as the n-
dimensional distributions Pe(tl),W~), ... ,e(tn) satisfy (6) and (6).
(11) We choose in advance a certain function space ~ consisting of func-

tions x(t) defined on T and a metric is introduced (the distance satisfying the
usual axioms of a metric space). A random element of the metric space ~
satisfying (6) and (6) is called a random function e(t) of type ~.

A systematic study ofrandom functions in a very important particular case
was started by N. Wiener in 1923 (see [5]). The first ofthe concepts given above
was most extensively developed by E.E. Slutskii, starting from 1928 (see [6]).
The second concept for arbitrary Banach space was developed by V.1. Glivenko
from 1928 on (see [7]-[9]). The subsequent abundant literature can be found
in modern presentations of the question in [10], [11].
If ~ is such that for any t o E T and any open set C in the complex plane
the set
{x(t) : x(to) E C} (1)
of functions x(t) for which x(to) belongs to C is a Borel subset of ~, then

every random function in the sense of (11) clearly has a corresponding random
function in the sense of (I).
For a random function in the sense of (I) the inverse problem can be posed:
Is it possible to consider this kind ofrandom function e(t) as actually belonging
to a certain function space (for example, in the case T = [a, b]) as belonging to
the space qa,b] of continuous complex functions with the metric
p(x(t), y(t)) = max Ix(t) -

a9~b
y(t) I) ?
This problem can be stated precisely using equivalence of random functions in

the sense of (I). Two random functions el(t) and 6(t) (in the sense of (I)) are
called equivalent if for any t E T,
P{6(t) t 6(t)} = O. (2)
The quest ion posed above can now be interpreted in the following way: Given
a random function 6 (t) in the sense of (I) and aspace of Borel measurable sets
(I), does there exist a random function 6(t) of type ~ for which (2) holds for
any t E T? From the point of view of applications, this approach developed
by E.E. Slutskii (see [12], [13]) is apparently quite sufficient and allows us to
avoid a number of complications of set theory which will be discussed below
in §3. From the practical viewpoint the most interesting are the questions:
Is it possible to consider a random function as a) continuous, b) having only
discontinuities of the first kind? In the first direction the following result
by A.N. Kolmogorov (see [12]) has long been known: in order that a random
function e(t) in the sense of (I) given on T = [a, b] be equivalent to a random
function C(t) of type qa,b], it suffices that
(3)
for some a > 0, 0: > 1 and K. In this case C(t) satisfies with probability 1 a
Lipschitz condition
IC(t + r) - C(t)1 ::; K*lrl<>;;l-.5, (4)
for any °> 0, where K* is a random variable. (Note also that for any f >0
there exists K~, depending only on f,o,o:,a,K, such that P{K* > K~} < f.
This will be used in §6.)
The conditions for the representability of e(t) as a function with discon-
tinuities of the first kind only are especially essential when e(t) represents a
Markov process. Among the recent papers on this question we point out those
by E.B. Dynkin [14] and Kinney [15].2
As is weIl known, elements of a number of important function spaces are
not individual functions, but metric types z(t) of these functions with respect
to the measure I' introduced in T (a metric type is a dass of all functions
that differ from a certain fixed function z(t) only on a set of I'-measure 0).
We introduce the notion of asymptotic continuity of z(t) at to so that for the
metric type z(t) the "true values" of z(t) are uniquely determined at almost all
points to E T by asymptotic continuity, and the space ~ is such that the true
values of a random function of type ~ at the points where it is defined are with
probability 1 random variables. In this setting the same problem on the relation-
between the two not ions of a random function, (I) and (11), may be raised and
solved, perhaps in a different form. Now, using the approach (I) we naturally
require that random variables e(t) be defined only almost everywhere on T
and regard two random functions (in the sense (I)) 6(t) and 6(t) as being
equivalent if (2) holds for almost all t. Then corresponding to each random
function C(t) in the sense of (11) there is a random function e(t) in the sense
of (I) which is defined to within equivalence and is equal to the true value of
f"(t) at t for almost all t with probability 1. The complete solution of the
inverse problem under conditions when a random function in the sense of (I) is
equivalent to a measurable function was given in the paper by Ambrose [16] for
the case when T is the real axis and I' is Lebesgue measure. A necessary and
sufficient condition is for e(t) to be asymptotically stochastically continuous
almost everywhere (with respect to 1'). If this condition is satisfied, we can
further consider our problem in the space ~ of measurable functions (more
exactly, in the space of metric types) and without any difficulties in principle
compute the probability of the summability of e(t), its square summability,
"essential boundedness" , etc.
§3. Finite-dimensional distributions of values of a

random function
Let us return to random functions e(t) in the sense of (I) defined for all t E T.
2 See also a later work by N.N. Chentsov, Teor. Veroyatnost. i Primenen. (Trans-
lated as Theory Probab. Appl.) 1:1 (1956), 155-161 (in Russian). (Note of Rus-
sian editor.)
Each such function e automatically defines a probability field {nT, lPe, Pd

where nT is the set of all complex functions defined on T.
The formula
(tk = X(tk), k = 1,2, ... ,n,
defines a mapping 1I"t l.t2 ....• t n(x) from nT into the n-dimensional complex vector
space of vectors
For fixed tt, t2, ... , t n this mapping assigns to each random function e(t) a
random vector
Usually, in specific problems involving random functions the corresponding

distributions
play an important role.

Since Pl1h ..... tn IS defined on the Borel sets B, Pe is defined on their
pre-images in nT:
B' -- 11"-1
tl,t2, ... ,t n
(B) .
These are the so-called cylindrical Borel sets in nT.

Let IPT be the Borel closure of a system of such sets (the smallest O"-algebra
containing these sets). Clearly, for any random function e(t) we have
The values Pe(B) for B E IPT are uniquely defined by the finite-dimen-
sional distributions. In [1] it was proved that, conversely, arbitrary "compat-
ible" finite-dimensional distributions Ptlh ..... tn given for all finite systems of
elements tl, t2, ... ,tn in T and defined on the Borel sets in an n-dimensional
complex vector space give rise to a probability field {nT, IPT, P*}. The follow-
ing formulation is a slight variation of this result: If the distributions Pt1 .t2 ..... t n
are defined on all systems of finite elements of T, are compatible and satisfy
(e2) and (e3), then there exist random functions e(t) in the sense of (I) with
It is essential, however, that the function p{ which, generally speaking,

is uniquely defined by the finite-dimensional distributions P;lh, ... ,tn , is not
defined beyond 4>T. At the same time, the class 4>T is too narrow for applica-
tions: for uncountable T it does not include the set of functions x(t) such that
Ix(t)1 ::; K for all t E T, or, if T = [a, b], the set C of continuous functions, etc.
Passing to the concept (II), in aH cases of practical importance studied so
far we were able to recover p{ uniquely from the finite-dimensional distributions
ptt" ... ,t n (the meaning ofthis statement for spaces ~ consisting ofmetric types
x(t), rat her than of individual functions, needs some clarification which we will
not go into now) and at the same time, when the space ~ is properly chosen,
all the sets needed for specific problems appear to belong to 4>{.
§4. Weak convergence of distributions
In this section we will confine ourselves to the consideration of random elements

etaking values in a complete separable metric space X and satisfying the
conditions (6) and (6). In the study of these sequences
of random elements from X, there arises the quest ion of the convergence of the
corresponding distributions p{n to a distribution P on X. Here convergence
naturally means "weak convergence" of distributions in the usual functional
analytic sense. Recall that a sequence Pn of distributions on X weakly con-
verges to a distribution P, if for any function f(x), x E X that is continuous
and bounded on X,
[f(X)dPn ~ [f(X)dP.
If X is the real line, then weak convergence appears to be equivalent to
the well-known convergence "in general" of the corresponding distribution func-
tions. Denote by Pn P the weak convergence of distributions. The meaning
~
of weak convergence is adequately clarified by the foHowing two theorems, of

which the first is weH known and the second can easily be proved.
I) Pn ~ P if and only if
for any set A whose boundary has P-measure 0 (that is, for any continuity set
of the distribution P);
II) Pn => P if and only iffor any functional I( x) continuous almost every-
where with respect to P, the sequence of distribution functions
converges to the distribution function
FJ(O:) = P{x : I(x) < o:}

at every continuity point of the latter.
Now let
be a sequence of random elements of X for which
(5)
Further , let
TI; Tl1, Tl2,···, Tin,···
be a sequence of random elements of a complete metric space Y with TI = f(e)

and Tin = In(en), where I and In are continuous mappings of X into Y. Then,
under rather general assumptions (for example, that In -+ I uniformlyon any
compact set K C X), (5) implies that
P"n => P".
This simple remark is useful when proving a number of limit theorems.

Finally, the following theorem may be of use for studying limit theorems
for a sequence of random variables (see [17]):
111) For a family !m of distributions in X to be compact it is sufficient 3

that for any f > 0 there exists a compact set K, such that P(K,) > 1 - f for
all P E!m.
We consider an example illustrating Theorem III. It is known that a set
K C C, where C is the space of functions x(t), continuous on the interval
3 Yu.V. Prokhorov proved also the necessity of this condition.

[0, 1], x (0)= 0, is compact if and only if there exists a function 4>( r), tending
to zero as r ---+ 0, for which
sup Ix(t + r) - x(t)1 ::::; 4>(r).

t,xEK
Now using the continuity condition with probability 1 of a random process

(see (3) and (4) in §2) we obtain the following sufliciency condition for the
compactness of a family rot of distributions in C:
There exist constants a > 0, a> 1, K > 0 such that for all PE rot
where Epe denotes
Sometimes it is relatively simple to establish convergence
Pn(A) ---+ P(A), A E S, (6)
for some system S of subsets of X. Weak convergence Pn => P can be derived

from (6) using, for example, the following theorem:
IV) A system S of subsets X is called a "uniqueness system" if any distri-

bution P defined on S is uniquely defined on X.
In order that Pn => P it is necessary and suflicient that the following two
conditions hold:
a) Pn(A) ---+ P(A) on a "uniqueness system" S;
b) the family {Pn } is compact.
This theorem implies, for example, that if a sequence of processes {en(t)}
defined on [a, b] is such that
a) the finite-dimensional distributions of the processes en(t) converge to
the finite-dimensional distributions of some process e(t),
b) there exist constants a, a, K(a > 0, a> 1, K > 0) independent of n
such that
then for the corresponding distributions in qa,bj we have

V) Let S be a system of open sets A ~ X such that:

a) S is a basis in X;
b) Al ES and A 2 ES imply Al n A 2 ES. Then the convergence
Pn(A) - P(A), A E S,
implies that
Pn~P.
Examples illustrating the use of these general results can be found in the
last section of the present paper.
§5. Characteristic functionals
If eis a random element from a Banach space X, 4 then for the corresponding
distribution p( we can define an analogue of the usual characteristic function.
For any element f of the space X* dual to X we define the value of the char-
acteristic functional H by the formula
This definition, as wen as the basic properties of characteristic functionals, was

given in the note by A.N. Kolmogorov [18]. Recently interest in distributions
in function spaces has grown considerably, and it seems that the apparatus of
characteristic functionals will find broader applications in probability theory
and in certain fields of mathematical physics (see, for example, Hopf [19]). In
view of this we consider it useful to take note of the results of [18] and point
out certain unsolved problems in this connection. For example, in [18] it has
been established that
1. p( is uniquely determined by H(f, p().
2. When summing independent random variables with values in X, the
corresponding characteristic functionals are multiplied.
3. If
then
4 In what follows we shall consider separable spaces only.

Ix /I12 ...
where
E(/I, 12,···, In) = IndPe
are multilinear functionals which are natural generalizations of the moments.

Clearly, the functions
are uniquely determined by H(f, Pe).

For the purposes of probability theory it is essential to have necessary
and sufficient conditions for weak convergence of distributions in terms of the
corresponding characteristic functionals. However, these conditions are not
yet known. Certain applications of characteristic functionals for deriving some
limit theorems are studied in work by R. Fortet and E. Mourier (see [20]-[23]
and Fortet's talk at the present conference). Here we note only one result
following from Theorems III and IV of §4.
Let X be a separable Banach space and let {Pn } be a sequence of dis-
tributions on X. Convergence Pn ::} P holds if and only if the following two
conditions hold simultaneously:
a) H(f, Pn ) - H(f, P) for every I E X*j
b) for any f. > 0 there exists a compact set K, such that Pn(K,) > 1 - f
for all n.
The quest ion of sufficient conditions for a functional H(f) to be character-
istic acquires new aspects in the infinite-dimensional case. Here, we only make
the following remark. In accordance with [18] it is natural to call a distribution
P in X normal if its characteristic functional has the form
where ml is a linear and Q(f) a quadratic functional of I.

For a Hilbert space this brings us to the formula
H(f) = ei(m,J)-~(J,sJ),
where m is an element of X and S is asymmetrie bounded linear operator
replacing the matrix of second moments.
However, the condition that
(f,Sj)"? 0
be non-negative, which is suflicient for obtaining the characteristic function in

the finite-dimensional case is not suflicient here. 5 In particular, the functional
H(J) = e-tu,J) which is obtained for m = 0, Sj == j, is not characteristic.
Note also that the necessary and suflicient conditions for a functional to be
characteristic given in [23] can hardly be considered final, since they are of an
extremely complex character. 6
§6. Applications to limit theorems
6.1. Passage from sums of random variables to processes. The notion

that limit distributions for sums of random variables are exact for properly
chosen random processes took shape long ago (see, for example, [25]-[29] and
especially A.Ya. Khinchin's monograph [30]). Here we will give the results
of more recent works touching on this group of questions. Note that posing
and solving the problems on passing from sums of random variables to pro-
cesses seems impossible without the methods which essentially have to do with
functional analysis in their most general form.
Let
be random variables with zero expectations Een,k and finite variances Oen,k
such that
We denote by
Sn,k = en,l +en,2 + ... +en,k, k = 1,2, ... , n, (7)
the "accumulated sums" of the random variables en,k, and by tn,k the quantities
OSn,k = L:: Oen,j'

j=l
5 It is sufficient if the trace of S is finite (which is also necessary).

6 The modern situation is reviewed in: N.N. Vakhaniya, V.1. Tarieladze and
S.A. Chobanyan, Probability distributions in Banach spaces, Nauka, Moscow,
1985 (in Russian) (Note of Russian editor).
We assign to the sequence of sums {Sn,k}, k = 1,2, ... , n, a random ele-

ment ~n of the space C of functions x(t) continuous on the segment [0,1] and
vanishing at t = 0, by setting
for tn,k :::; t < tn,H1'

It can be shown (see [17]) that if the sums Snn satisfy Lindeberg's condition
used in the ordinary central limit theorem, then
(8)
as n --- 00, where e(t) is a Wiener random process, that is, a process continuous
with probability 1 and satisfying the conditions:
1) = 0 with probability 1;
~(O)
2) the increments of {(t) are independent on non-intersecting intervals;

3) {(t + Ll) - {(t) has normal distribution with mean 0 and variance Ll.
Formula (8) and Theorem 11 of §4 imply a number of limit relations. Thus,
for instance, by considering the functional
fex) = max x(t), x(t) E C,

0$t~1
we obtain for a ~ 0,
P{ max en(t)
0$t$1
> a} = P{ l~k~n
max Sn k > a} --- P{ max ~(t) > a} =
' 0~t~1
Given continuous functions a(t) and b(t) on [0,1] such that a(O) < 0 < b(O)
and a(t) :::; b(t), consider the functional fex) equal to 1 if a(t) < x(t) < b(t) for
an t and 0 otherwise. We find that
P{for an t, a(t) < ~n(t) < b(t)} --- P{for all t, a(t) < ~(t) < b(t)}
and hence
P{for an k, a(tn,k) :::; Sn,k :::; b(tn,k)} --- P{for all t, a(t) < ~(t) < b(t)}.
The latter result was obtained by A.N. Kolmogorov in 1931 under somewhat
different assumptions (see [28], [29]). This remark allows us to obtain similar
limiting relations for the joint distribution of any finite number of functionals,
for example
P{ min Sn k <:c, max Sn k < y} -+ P{ min e(t) <:c, max e(t) < y}.
lSkSn' lSkSn' 0St9 099
Statement (8) contains theorems of Erdös, Kac and Donsker (see [31]-[33])
as particular cases. According to these theorems, the distributions of functions
of "accumulated sums" converge to the distributions of the corresponding func-
tionals of Wiener's process. The method by Erdös and Kac in its final form
given by Donsker [33] is first to establish the convergence of
(9)
for "strips", that is, for the sets A whose elements satisfy a(t) < :c(t) < b(t) for
all t, where a(t) and b(t) are step functions (see Figure).
'!/tJ
/1 11' b
1
alt) 1
1
Then by approximating functionals f that are continuous almost everywhere

with respect to p~ by characteristic functionals (in the set-theoretic sense)
of "strips", Donsker established the convergence of the distribution functions
Fjn)(a) = p{f(en) < a} which, by Theorem II of §4, is equivalent to weak
convergence (8). To pass from (8) to (9) we can also use Theorem V of §4, since
the system S of all "strips" satisfies the conditions of this theorem. Another
method for proving (8) based on Theorem III of §4 is given in [17].
Remark. It is known that by using the method of upper and lower functions
(see [30]) for the case when the process en(t) is constructed by the accumulated
sums of terms that are independent or connected in a Markov chain, and if the
process e(t) is a properly chosen Markov process satisfying the Fokker-Planck
differential equations, we can obtain a statement of the following type: for any
piecewise smooth functions a(t) and b(t)
P{for all t, a(t) < ~n(t) < b(t)} - P{for all t, a(t) < ~(t) < b(t)}. (10)
Clearly, using Theorem III of §4, we can infer from the convergence (10)
that Pen => Pe. Nowadays the method of upper and lower functions is suc-
cessfully used by 1.1. Gikhman [34]-[39], who has applied it to a number of
problems concerning sums of random variables and certain problems in math-
ematical statistics (see below).
6.2. Approximation of the empirical distribution to the theoretical

one. Let ~ be a random variable with uniform prob ability distribution on [0,1],
let Fn(t) be the empirical distribution function constructed by n independent
observations of ~ and let
Denote by ~(t) the Gaussian process (that is, the process all finite-dimen-
sional distributions of which are Gaussian), continuous with probability 1 and
with correlation function
E{~(t),~(s)} = t(l- s), °t

~ ~ s ~ 1.
Using the higher-dimensional Laplace theorem it is easy to show that

the finite-dimensional distributions of the process ~n(t) converge to the finite-
dimensional distributions of the process ~(t) as n - 00.
This makes it possible to derive convergence of distributions to the dis-
tribution f(~) for a broad dass offunctionals f(~n). This can be done by all
the methods described in §6.1. (See, for example, the papers by Donsker [45],
Gikhman [38], Fortet and Mourier [23]. The latter makes use of the method of
characteristic functionals when ~n and ~ are considered as random elements in
the space L2[0, 1]. The paper by Donsker [45] is dose to [33].)
Based on this result we can obtain a number of new non-parametric tests
(see, for example, [46]), generalizing the known tests of Kolmogorov [42] and
Smirnov [43], [44].
References

2. P. Halmos, Measure theory, Van Nostrand, Princeton, N.J .,1950.
3. B.V. Gnedenko and A.N. Kolmogorov, Limit distributions tor sums 0/
independent random variables, GTTI, Moscow, 1949 (in Russian).
4. A. Kolmogoroff, 'Algebres de Boole metriques completes', In: VI Zjazd
Matematykow Polskich Warszawa, 1948, pp. 22-30.
5. N. Wiener, Publ. Mass. Inst. Techn.,2 (1923), 60.
6. E.E. Slutsky, C. R. Acad. Sei. Paris, 187 (1928), 370-372.
7. V.I. Glivenko, Atti Accad. Naz. Lincei. Rend. 8 (1928), 480-483.
10. J .L. Doob, Stochastic processes, New York, 1953.
11. A. Blanc-Lapierre and R. Fortet, Theorie des /onctions a/eatoires, Paris,
1953.
12. E.E. Slutsky, G. Ist. Ital. Attuar. 8 (1937), 183-199.
13. E.E. Slutskii, Trudy Sredneaz. Univ. 31 (1939), 1-15 (in Russian).
14. E.B. Dynkin, Izv. Akad. Nauk SSSR Sero Mat. 16 (1952), 569-572 (in
Russian).
15. J.R. Kinney, Trans. Amer. Math. Soc. 74 (1953), 280-302.
16. W. Ambrose, Trans. Amer. Math. Soc. 47 (1940), 66-79.
17. Yu.V. Prokhorov, Uspekhi Mat. Nauk 8:3 (1953), 165-166 (in Russian).
18. A.N. Kolmogoroff, C. R. Acad. Sei. Paris 200 (1935), 1717-1718.
19. E. Hopf, J. Rat. Mech. and Analysis 1 (1952),87-123.
20. E. Mourier, C. R. Acad. Sei. Paris 231 (1950), 28-30.
21. R. Fortet and E. Mourier, C. R. Acad. Sei. Paris 238 (1954), 1557-1559.
22. R. Fortet and E. Mourier, Bull. Sei. Math. 78 (1954), 14-30.
23. R. Fortet and E. Mourier, Ann. Ecole Norm. Super. (1954), 267-285.
24. E. Mourier, 'Elements aleatoires dans un espace de Banach' , These Doct.
Paris, 1954.
25. L. Bachelier, Les methodes du calcul des probabiliUs, Paris, 1939.
26. R. Lüneburg, Math. Ann. 104 (1931), 700.
27. I.G. Petrovsky, Math. Ann. 109 (1934),425-444.
28. A.N. Kolmogorov, Izv. Akad. Nauk SSSR Sero Fiz.-Mat. (1931),959-962
(in Russian).
29. A.N. Kolmogorov, Izv. Akad. Nauk SSSR Sero Fiz.-Mat. (1933), 363-372
(in Russian).
30. A.Ya. Khinchin, Asymptotic laws 0/ probability theory, ONTI, Moscow-
31. P. Erdös and M. Kac, Bull. Amer. Math. Soc. 52 (1946), 292-302.
32. P. Erdös and M. Kac, Bull. Amer. Math. Soc. 53 (1947), 1011-1020.
33. M. Donsker, Mem. Amer. Math. Soc. 6 (1951), 1-12.
34. 1.1. Gikhman, Ukmin. Mat. Zh. 6:1 (1954), 28-36. (in Russian).
35. 1.1. Gikhman, Mat. Sb. Kiev. Gos. Univ. 1953, No. 7, 8-15 (in Russian).
36. 1.1. Gikhman, Dokl. Akad. Nauk SSSR 82:6 (1952), 837-840 (in Russian).
37. 1.1. Gikhman, Dokl. Akad. Nauk SSSR 56 (1947), 961-964 (in Russian).
38. 1.1. Gikhman, Ukmin. Mat. Zh. 5 (1953), 413-433 (in Russian).
39. 1.1. Gikhman, Dokl. Akad. Nauk SSSR 91:4 (1953), 1003-1006 (in Rus-
sian).
40. M. Kac, In: Proc. Second Berkeley Sympos. 1951, pp. 189-215.
41. G. Maruyama, Nat. Sci. Rept Ochanomicu Univ. 1 (1953).
42. A.N. Kolmogoroff, G. Ist. Ital. Attuar. 4 (1933), 83-91.
43. N.V. Smirnov, Rev. Math. Moscow 6 (1939), 3-26.
44. N.V. Smirnov, Byull. Moskov. Gos. Univ. Sero A 2:2 (1939), 3-14 (in
Russian).
45. M. Donsker, Ann. Math. Statist. 23 (1952), 277-28l.
46. T. Anderson and D. Darlong, Ann. Math. Statist 23 (1952), 193-212.
45. ON THE PROPERTIES OF P. LEVY'S
CONCENTRATION FUNCTIONS * 1
In what follows
denotes the sum of independent random variables 6:,
Q(/) = sup P{z

x
~ e~ z + I}
e
the concentration function of the sum and Q" the concentration function of
the e".
Theorem 48.2 from the well-known monograph by P. Levy , Theorie de
l'addition des variables aIeatoires', can be formulated in the following way:
For any (, 1 ~ ( > 0, and ß > 0 there exist two constants 6 >0 and N >0
such that tor n ~ N the inequality
implies
(1)
In my recent work I happened to need a generalization of (1). The following

theorem [1] suited my requirements:
Theorem. There exists a constant C such that 2
where
n
s = ~)1 - Q,,(1)),
"=1
imply that
Q(L) ~ CL/l..jS. (2)
* 'Sur les proprietes des fonctions de concentrations de M.P. Levy', Ann. Inst.
H. Poincare 16:1 (1958), 27-34.
1 In two issues of the Proc. Inst. Statistics Paris Univ. articles in honour of Paul
Levy have been published. This paper was submitted too late to be included in
these issues.
2 Here and in what follows logarithms to the base 2 are considered.
459
460 ON THE PROPERTIES OF P. LEVY'S CONCENTRATION FUNCTIONS
In this paper I give a proof of this theorem. Another refinement of (1)

was given with a detailed proof by W. Doeblin in [2]. His result is not clearly
formulated in [2], and there is a misprint. 3 It seems evident, however, that in
terms of concentration functions the result by W. Doeblin may be formulated
in the following way:
For any f > 0 and ß > 0, there exist constants 6 > 0 and N > 0 such that
for
the inequality
implies
Q(6L) ~ ß. (3)
It would be interesting to find inequalities that include (2) and (3) as par-
ticular cases. In general, I would like to point out that the further development
of elementary methods for direct probabilistic computations, so brilliantly de-
veloped in France by P. Levy and W. Doeblin, seems to remain as urgent as the
development of classical or functional analytic methods. In any case, I could
not prove the results of [1] without these elementary direct methods. It is quite
possible that mathematicians that have a better command of the subtle prop-
erties of characteristic functions will sooner or later be able to prove and even
generalize theorems from [1] using purely analytical methods, as already has
happened with results initially proved by direct methods of probability theory.
It seems, however, that today we are still in aperiod when the competition
between these two trends is bringing about the best results. While P. Levy
who masters these two techniques equally weIl, makes use of these two kinds
of method in his own work, it is very desirable that, after the premature death
of Doeblin, the younger generation of probabilists should not forget the direct
methods despite their admiration (admittedly, quite justified) for the power of
the methods which make use of distributions in function spaces.
We will use the following notation:
F(x) = p{e < x}, a = Ee, (72 = Oe.
3 It says L~ I instead of Li ll.

ON THE PROPERTIES OF P. LEVY'S CONCENTRATION FUNCTIONS 461
The following lemma is weIl known.
Lemma 1. There exists a constant Cl such that
where ak = Eek (k = 1,2, ... , n), implies the inequality
Ga,q being the normal distribution function with pammeters a, u.

The following lemma is a generalization of Lemma 48.2 from 'Theorie de
l'addition des variables aIeatoires'.
Lemma 2. There exists a constant C 2 such that
p{ek = xA:} = p{ek = -xA:} =~, Xk ~ I (k = 1,2, ... , n),
L ~ Iv'logn(n ~ 2)
imply that
To prove this lemma, it is necessary to distinguish between the following

three different cases:
1°. L > Iyn.
2°. L ~ Iyn and moreover, there exists an r ~ 1 such that the number n r
of numbers Xk such that
(4)
satisfies the inequalities
3°. L ~ Iyn and there is no r satisfying the properties indicated in 2°.

In the first case
Q(L) ~ 1< Lj/...fii,.
In the second case it is natural to consider the sum e. of those ek satisfying
(4). Since all the terms of this sum satisfy the inequalities
lekl < I. = 4r l, U. = v'Oe. ~ 4r - l /...jn,: ~ ~..;n,

according to Lemma 1 for the corresponding function F. (z) we have
Since the concentration function Q/J,u(L) of the normal distribution func-

tion G/J,u satisfies the inequality
we have in the second case
(6)
where Q.(L) denotes the concentration function of e•.

In the third case, for each r ~ 1 one of the following two inequalities is
satisfied:
either n r < n/4r , (7)

or nr < n/4Iogn. (8)
The sum of the n' numbers n r satisfying (7) is not greater than
1 n
nE 4 = a·
00
r
r=1
Since the sum of all the n r equals n, the sum of the n" numbers n r satis-
fying (8) is not smaller than !n. Clearly, among all these nr there are at least
2 log n non-zero numbers. Since the number of r satisfying
is smaller than logn, there exists some number, not less than logn, ofnumbers
r for which
Taking from these reither only even, or only odd numbers in increasing
order we obtain the sequence
for which
and study their sum f,'. Since
a more or less elementaryargument, which is left to the reader, shows that the
sum f,' can belong to an inter val of length L only for one single quite specific
> 1) of the signs of the terms f,k ö' In other words, with
n-'.
combination (if n
probability at most 2- 8 ~
Hence, in the third case, because of the condition L ~ 1 we have
Q(L) ~ Q'(L) ~ l/vn ~ L/lvn. (9)
Comparing (5), (6) and (9) in these three cases we see that Lemma 2 is
proved.
Now let us prove the theorem. The relations
(10)
determine Uk(Y), 0 < Y < 1. (Since Fk(Z) is the distribution function of a

random variable f,k, it uniquely determines the values of Uk(Y) for all y, 0 <
y< 1 except perhaps on a countable set of points y.) 4 It can be considered as
the inverse function
whose definition must be made suitably precise for all Y corresponding to dis-
continuity points of Fk(Z), If we assurne that the random variables 7]k are
uniformly distributed on (0,1) and are independent, then f,k = Uk(7]k) are also
independent and have distributions Fk(Z), Clearly, it may be assumed without
4 There is no uniqueness for the points y corresponding to the intervals of con-

stancy of Fk(x), hut clearly Uk(Y) can he well-defined on the whole interval (0,1)
so that (10) holds.
loss of generality that the initially given random variables ek are expressed in
this way.
It may be assumed that all the Qk(/) are smaller than 1, since by excluding
from our discussion those terms for which Qk(1) = 1 and obtaining (2) for the
sum of all other terms we can include the excluded terms again, and this will
not enlarge Q( L). We set
4fk = 1 - Qk(1),
x~ = 'Uk(fk), x~ = 'Uk(1- fk)'
Clearly,
X~ - x~ > I.
Denote by k 1 , k 2 , ••• ,km the indices k for which either 1/k < fk or 1/k >
1- fk.
We fix the quantities
if 1/k r < fk r ,
if 1/k r > 1 - fk r .
It is easy to see that the joint conditional distribution of the random variables
eki is such that they remain independent and each of them has the distribution
where
Xr = ~[Ukr(1- Zr) - Ukr(Zr)],
a r = ~[ukr(1- Zr) + Ukr(Zr)].
Applying (for fixed kr and Zr) Lemma 2 to the e~ = ek r - ar for which le~1 ~
I' = ~/, we obtain for L ~ I' y10g m the inequality
(11)
Inequality (11) is proved for the conditional function Q(L) (when m, kr , ar , Zr

are fixed and L ~ I' y10g m.) This immediately implies the inequality
for the absolute function Q(L), where
p=1-PHs~m~s}.
We now merely have to estimate this latter probability. Since
Em = L:2fk = !s, Dm = L: 2fk(1 - fk) ~ !s,

k k
it follows from Chebyshev's inequality for k = tS that

Dm 8
1- P{ }
~m is
~ s ~ P{lm - Eml ~ k} ~ k2 ~ -;.
Thus we finally obtain
or, since Q(L) ~ 1, and by our assumptions L ~ I,
Q(L) ~ CL/hIS,
where C = 8 + 4C2 •
References
1. A.N. Kolmogorov, 'Two uniform limit theorems for sums of independent

terms', Teor. Veroyatnost. i Primenen. 1:4 (1956),426-436 (in Russian).
(Translated as Theory Probab. Appl.)
2. W. Doeblin, 'Sur les sommes d'un grand nombre des variables aIeatoires
independantes', Bull. Sei. Math. 63 (1939), 23-32; 35-64.
46. TRANSITION OF BRANCHING PROCESSES TO
DIFFUSION PROCESSES AND RELATED GENETIC PROBLEMS *
Survey Report
The abstract theory of "branching processes" with the "number of particles"

taking arbitrary non-negative real values (see [1]) was obviously developed to
inelude the limiting laws of behaviour of ordinary branching processes for a
large number of partieles. Apparently, the most important particular case of
these processes is a diffusion process. In the simplest one-dimensional case the
corresponding Fokker-Planck equation is of the form
UT = ~b(xu).,., - a(xu).,. (1)
A more complex related problem was considered by R.A. Fisher in his

remarkable book [2]. In a population of N individuals a gene A can be present
in v copies, 0 ~ v ~ 2N. For e = v/2N, 0 ~ e~ 1, Fisher obtains the
Fokker-Planck equation
uT = ~b[x(l- x)u].,., (2)
which, even for large N, is only reasonable for x not too elose to 0 or 1. On
the other hand, near x = 0, v obeys the laws of a branching process, and
near x = 1 so does 2N - v. Similar problems may appear also in physics and
chemistry.
FeIler's paper [3] may serve as an introduction to the whole range of prob-
lems. Apart from [2], the corresponding genetic problems are discussed in [4],
[5] (without mathematical rigour). Basic information on branching processes
is given in Sevastyanov's review [6].
It is interesting that, from the viewpoint of the theory of diffusion processes
here we have adegenerate case: the diffusion coefficient bx/2, or bx(l - x)/2,
vanishes on natural boundaries. In [3] FeIler stresses the difficulties that appear
because of this. However, I do not agree with hirn when (on p.238) he says
that certain results obtained by biologists are meaningless: these results are
true under proper interpretation. I think that in this field there are many
subjects that deserve further development. In what foIlows I only treat certain
very simple results obtained by other methods and give a correct interpretation
* Teor. Veroyatnost. i Primenen. 4:2 (1959), 233-236 (in Russian). (Translated

as Theory Probab. Appl.)
466
BRANCHING PROCESSES, DIFFUSION PROCESSES & GENETIC PROBLEMS 467
of certain results of Fisher which were tackled by FeIler, from the diffusion
viewpoint.
Suppose that during one step of a process one partiele turns into k particles
with prob ability Pk(N),
L: kpk(N) = 1 + o:(N), L: k(k - 1)Pk(N) = b(N),

k k
L:k(k - 1)(k - 2)Pk(N) = c(N).

k
We let N tend to infinity and suppose that
Pk(N) -+ Pk, (3)

No:(N) -+ a, (4)
b(N) -+ b, (5)
c(N) -+ c, (6)
where a, band c are finite.

The meaning of assumption (4) is that for the number of particles of order
N and 0: of order 1/N, the role of the number of particles in the process of
systematic increase (or decrease) and the role of random scattering are of the
same order. If the order of 0: is greater, then for a Iarge number of particles
the process becomes almost deterministic, whereas if 0: is of a smaller order,
it can be neglected. "Borderline" phenomena for 0: elose to zero are studied
in Sevastyanov's work [7]. It is here that the idealized diffusion model can be
expected to bring about some results.
If we now set v = Ne, then a natural time variable for studying the
behaviour of eis given by T which is related to the number of steps t of the
process by the relation t = NT. Under these assumptions the process e(T) gives
a diffusion process with Fokker-Planck equation (1) in the limit as N -+ 00.
At a = 0 equation (1) has the solution

(7)
with singularity at T = 0, = O.
X In a certain sense it describes the behaviour
of one partiele that appeared at time T = 0, meaning that if for TO > 0 we
1
choose c so that
00
U(To,x)dx = 1,
468 BRANCHING PROCESSES. DIFFUSION PROCESSES & GENETIC PROBLEMS
then, provided that e(rO) > 0, the conditional distribution of e(r) for r > ro
converges to the distribution
p{e(r) <x}=O, x~O,
p{e(r) < x} = P(r) + 1:Z: u(r, x)dx, x> 0,
P(r) = p{e(r) = O} = 1-1 00

u(r,x)dx
as N -+ 00.
The general solution of the stationary equation at a = 0, that is, of the

equation
(xu):z::z: = 0
has the form
This is not a probability distribution. However, the solution
u(x) = ct/x
has the following statistical meaning: if at each step a new particle appears
independently of time with prob ability p, and p( Xl, X2) is the number of those
initial particles whose progeny at a certain fixed time t satisfies
then the expectation EJl(Xl' X2) tends to the limit
as N -+ 00.
The solution
ct/x + c2/(1- x)
of the stationary equation
(x(1 - x)u):z::z: =0
in Fisher's theory can be interpreted in a similar way. This interpretation was
even verified in an experiment and appeared to be true.
18 November 1958
BRANCHING PROCESSES. DIFFUSION PROCESSES & GENETIC PROBLEMS 469
References
1. M. Irzhina, Czechoslovak Math. J. 8 (1958), 83.

2. R.A. Fisher, The genetical theory 0/ natural selection, Oxford, 1930.
3. W. FeIler, In: Proc. Second Berkeley Sympos., 1951, pp. 227-246.
4. J. Mah~cot, Les mathimatiques de l'hiredite, Paris, 1948.
5. S. Wright, Bull. Amer. Math. Soc. 48 (1942), 223-246.
6. B.A. Sevastyanov, Uspekhi Mat. Nauk 6:6 (1951), 37-99 (in Russian).
7. B.A. Sevastyanov, Teor. Veroyatnost. i Primenen. 4:2 (1959), 121-135 (in
Russian). (Translated as Theory Probab. Appl.)
47. ON THE CLASSES ~(n) OF FORTET AND
BLANC-LAPIERRE *
If for areal process e(t) there exist continuous absolute moments of order
n such that
(1)
then for k ~ n the moments
may be represented as
where the J.l~k) are generalized functions.

A process belongs to the dass ~(n) (see [1]), if aH the J.l~k) are of ''finite
measure type", that is, if
e tl, ... ,tk ) --

m (k)( JJ
... e 1\,
ei(t,A)M(k)(d')
where M?)(A) is a finite complex measure in k-dimensional space.

It is weH known that when n =
2, for a strictly stationary process e(t) the
existence of the moments Elel and Elel 2 implies that it is of the dass ~(2).
It can be shown by an example that similar statements fail for n > 2.
References
1. A. Blanc-Lapierre and R. Fortet, Theorie des fonctions aleatoires, Paris,

1953.
* Teor. Veroyatnost. i Primenen. 5:3 (1960), 373 (in Russian). (Translated as

Theory Probab. Appl.)
470
48. ON CONDITIONS OF STRONG MIXING OF A GAUSSIAN
STATIONARY PROCESS*
Jointly with Yu.A. Rozanov
As is weH known, two u-algebras of events mt' and mt" are said to be indepen-
dent iffor any A' E mt', A" E mt", P(A',A") = P(A')P(A").
M. Rosenblatt [1] has suggested a natural measure of dependence between
two u-algebras of events:
a(mt', mt") = sup IP(A' n A") - P(A')P(A")I.

A'E!JJl',AIE!JJl"
For a stationary random process e(t) the measure a(mt~, mti+T) (where
mt! denotes the u-algebra of events generated by e( u), s ~ u ~ t) depends
only on rand will be denoted by a(r). If a(r) - 0 as r - 00, then it is said
that the process e(t) has the strong mixing property. In this paper we indicate
which properties of the spectral function F( A) of the process guarantee the
strong mixing condition for Gaussian processes.
1. For any two systems {O = ~, and {17} = ~" with finite second moments
we introduce the index
If~' and ~" are, respectively, the families of aH variables with finite second
moments measurable with respect to the u-algebras mt' and mt", then, by def-
inition (see [2]), p(mt', mt") = p(~', ~") is the maximal correlation coefficient
between mt' and mt".
Clearly,
(1)
Now let {O and {17} be two sets of random variables which have (for
any finite set el, ... ,em,171, ... ,17m) Gaussianjoint distributions, and let mte
and mt'1 be the u-algebras generated by the events (e Er') and (17 Er")
respectively, where r' and r" are arbitrary Borel sets on the line, and let He
and H'1 be the closed (with respect to the mean square) linear spans of {O and
{17}·
(1960), 222-227 (in Russian).
471
472 CONDITIONS OF STRONG MIXING OF A GAUSSIAN STATIONARY PROCESS
Theorem 1. 1
(2)
Theorem 2. The maximal correlation coefficient satisfies the lollowing in-

equalities:
(3)
Proolol Theorem 1. Clearly, we may restrict ourselves to the case when {e}
and {'11} consist of a finite number of variables.
Further, in He and H" we can choose el, ... ,ern and '11, ... , 'In SO that only
those ek and 'I1k with the same indices are dependent, and each e E {e}, '11 E {'I}
is a function of 6, ... , ern and '11, ... , 'In respectively. We may also assurne that
Eek = 0, E'I1j = 0, Dek = D'I1j = 1, k = 1, m, j = 1, n.
Then the quantities I = 1(6, ... ,ern) and 9 = g('11, ... ,'In) may be rep-
resented as
rn n
1= L: Ik(el, ... ,ek), g= L:9j('I11"",'1j),

1 1
where
Ik = E(flel, ... ,ek) - E(f16,··· ,ek-d,
gj = E(gl'l11,'''' 'I1j) - E(gl'l11,"" 'I1j-d·
Note that for k ~ j,
and similarly for j ~ k
which implies that E/kgj = 0 for k =1= j and, taking m ~ n for the sake of
definiteness,
rn rn
Elg = L:E/k9k =L:E[E(fk9kI6, ... ,ek-l,'11"",'I1k-l)].
1 1
1 Theorem 1 is a multidimensional generalization of the result by Sarmanov [2],

e
according to which (2) holds for any two variables and " with Gaussian joint
distribution.
CONDITIONS OF STRONG MIXING OF A GAUSSIAN STATIONARY PROCESS 473
The variables 6: and TJk with Gaussian distribution do not depend on 6, ... ,ek-1 ,
TJ1, ... , TJk-1, and Sarmanov's result [2] implies that
where
Hence,
m
IE/gl ~ p L: Eakbk ~ p,
1
which proves our theorem.
Pro%/ Theorem 2. 2 Take an arbitrary > 0 and ee EHe, TJe E HrJ with
f
Eee = ETJe = 0, oee = ßTJe = 1 such that r = EeeTJe > p - f. Consider the
events
Ac = {ee > O} E 9Re and Be = {TJe > O} E 9RrJ'
We have ([9], p.321)
and, clearly ,
2~ sin- 1 r = P(AeBe) - P(Ae)P(Be) ~ 0'.
Further, if 0' > ~, then the inequality p ~ 211'0' is trivial. If, on the other hand,
0' ~ i, then
p- f ~ r ~ sin 211'0', p ~ 211'0' + f, P ~ 211'0',
which, together with (1) and (2) proves our theorem.

Theorems 1 and 2 imply, in particular, that a Gaussian stationary process
e(t) satisfies the strong mixing condition if and only if the maximal correla-
tion coefficient p(9R:' oo , 9Rif.r) -+ 0 as T -+ 00 (by Theorem 1 it coincides
with p(T) = p(H:oo,Hnr), where H: oo and Hnr are the linear closures with
2 This simple proof was suggested to the authors by Yu.V. Prokhorov.

respect to the mean square of the quantities ~(u), u ~ t and ~(v), v ~ t

respectively). Moreover, (cf. [4], Introduction),
a(r) ~ p(r) ~ 211"a(r). (3')
2. Let ~(t) be a process that is stationary in the wide sense and let
(stationarity implies that p(r) does not depend on t).

If the spectral function F(~) of our process is not absolutely continuous,
then p(r) = 1 for all r (see [5], [6]).
Let F(~) be absolutely continuous and f(~) the spectral density of the
process ~(t), f(~) = F'(~).
Theorem 3. For integer time
(4)
where inf", is taken over a/l functions <fJ( z) that can be analytica/ly continued
inside the unit disc; for continuous time
(4')
where inf", is taken over a/l functions <fJ(z) that can be analytically continued
into the lower half-plane.
The proof of this theorem is based on a general lemma taken from func-
tional analysis.
Lemma. Let L be a Banach space and L* its dual.

Let H be a subspace of Land HO the set of linear functionals that vanish
on H. Then
(5)
for any h* E L*.
Proof Since (h* - hO)(h) = h*(h), it follows that h*(h) ~ IIh* - hOIl for h E
H, IIhll = 1, therefore sup h*(h) ~ inf IIh* - hOIl. Further, according
hEH,lIhll=! hOEHO
to the Hahn-Banach extension theorem there exists a functional hi, coinciding

with h" on H, with norm IIhili not exceeding sup h"(h). Then h" -hi =
hEH,lIhll=l
h~ E HO and
IIh" - h~1I = IIh~1I = sup h"(h),
hEH,lI h ll=l
which proves (5).

Let us come back to our theorem. Clearly
J
where
Pj(A) = L c{e- iAt{, IpjI2f(A)dA ~1
tÜO
and the integration is carried out from -11" to 11" for integer time, and from
-00 to 00 for continuous time. Using certain properties of boundary values of
analytic functions it can be shown that in fact
(6)
J
where
p(A) =L Cke-iAtk, Ip(A)lf(A)dA ~ l.
tk~O
Let us take for the space L the space offunctions h(A) that are integrable
=
with weight f(A), that is, IIhll f Ih(A)lf(A)dA < 00, and as subspace H the
linear closure of the functions p( A) of the form
p(A) =L Cke-iAtk.
tk~O
Every linear functional h" on L has the form
h*(h) = J
h*(A)h(A)f(A)dA,
where IIh*1I = esssup

A
Ih"(A)I.
The subspace HO is clearly the subspace of linear functionals hO corre-
sponding to functions hO(A) such that f e-itAhO(A)f(A)dA = 0 for all t ~ o.
Hence the function 4>(e- iA ) = hO(A)f(A) has an analytic continuation to the
unit disc for integer time, and for continuous time 4»(A) = h(A)/(A) may be
analytically continued into the lower half-plane.
Taking as h* the linear functional corresponding to the function h*(A) =
e-i>.T, we obtain (4) and (4') from (5).
Theorem 4. I/there exists 4»o(z) that has an analytic continuation inside the
unit disc /or integer time (to the lower hai/-plane /or the case 0/ continuous
time) with boundary value 4»p(e- i >.) (respective1y, 4»p(A)) , such that the ratio
/ No is a uni/ormly continuous function 0/ A with 1/Nol ~ ( > 0 for almost
all A, then
per) -+ 0 (7)
as r -+ 00. If there exists an analytic function 4»o(z) such that 1/Nol ~ ( > 0
and the derivative (f No)<k) is uni/ormly bounded, then
p(r)::; cr- k •
Proof Let 4»(z) be a polynomial of degree at most [r/2] for integer time (or
an analytic function of exponential type with type at most r /2 for continuous
time). We have
i~fess:uP[I/-e-i>'T4»I/(\)] ::;qll~~",ess:up[l:o -e i>'T,p117 1]::;

::; ~infesssuplL - ei>.T,pI-+ 0 (8)
('" >. 4»0
as r -+ 00 (since //4»0 can be uniformly approximated by functions ,p( z) of the
above type (see [7], p.207; [8]).
For example, (7) always holds ifthe spectral density /(A) is continuous and
does not vanish at any A, A ::; r (integer time) or if /(A) is uniformly
-7r ::;
continuous over the whole line, does not vanish and for sufficiently large A
satisfies the inequality
m/A k ::; /(A) ::; M/A k - 1 (9)
for some positive m, M and integer k > 0 (for the case of continuous time).
It then follows from [5], [6] that if p( r) -+ 0 as r -+ 00, then the spec-
tral density cannot vanish "too strongly": namely, it must be positive almost
everywhere and satisfy the inequality
j log/(A)d\
1 + A2 ,,>
_
00. (10)
As follows from Theorem 4, weaker vanishing of the spectral density f(>")

compatible with (10) does not by itself contradict the strong mixing condition
(equivalent to (7)). If f(>") is rational with respect to ei>· (for integer t) or
with respect to >.. (for continuous t), then p(r) decreases exponentially fast;
for example, if f(>") = cj(a 2 + >..2) (e(t) is a Markov Gaussian process), then
p(r) = e- aT •
Apparently, the strong mixing condition might fail if the spectral den-
sity f(>") has a discontinuity (even if it is everywhere greater than a positive
constant for integer t).
The authors thank Yu.V. Prokhorov for the remarks on the manuscript,
which undoubtedly has improved it.
References
1. M. Rosenblatt, 'A centrallimit theorem and a strong mixing condition',

Proe. Nat. Aead. Sei. 42:1 (1956), 43-47.
2. O.V. Sarmanov, 'Maximal correlation coefficients', Dokl. Akad. Nauk SSSR
121:1 (1958), 52-55 (in Russian).
3. O.V. Sarmanov and V.K. Zakharov, 'Maximal coefficients of multiple cor-
relation', Dokl. Akad. Nauk SSSR 130:2 (1960), 269-271 (in Russian).
4. V.A. Volkonskii and Yu.A. Rozanov, 'Some limit theorems ofrandom func-
tions, 1', Teor. Veroyatnost. i Primenen. 4:2 (1959), 186-207 (Russian).
Gos. Univ. Sero Mat. 2:6 (1941), 1-40 (in Russian).
6. M.G. Krein, 'On an extrapolation problem of A.N. Kolmogorov', Dokl.
Akad. Nauk SSSR 46 (1945), 339-342 (in Russian).
7. N.!. Akhiezer, Theory of approximation, F. Ungar, 1956 (translated from
the Russian).
8. S.N. Bernshtein, 'On best approximation of continuous functions on the
real axis', Dokl. Akad. Nauk SSSR51 (1946), 327-330; 485-488 (in Rus-
sian).
9. H. Cramer, Mathematieal methods 0/ statistics, Princeton Univ. Press,
1946.
49. RANDOM FUNCTIONS OF SEVERAL VARIABLES ALMOST ALL
REALIZATIONS OF WHICH ARE PERIODIC *
A set
of n linearly independent veetors is ealled a frame in Eudidean spaee En. A

function k(t) of an n-dimensional veetor is ealled R-periodie if there exists a
frame
R* = (Ir, ... ,1~)
eongruent to R sueh that
and if f is periodie in each variable zk with period 211'.

We give neeessary and suffieient eonditions for R-periodieity for almost all
realizations of a random function e(t) in terms of "speetral moments".
In partieular, this implies that a stationary isotropie non-eonstant random
function eannot be Gaussian for n = 1. This may be of interest when studying
potentials and solutions of Sehrödinger's equation in erystals and media with
"dose order" .

(1960), 374 (in Russian).
478
50. AN ESTIMATE OF THE PARAMETERS OF A COMPLEX
STATIONARY GAUSSIAN MARKOV PROCESS *
Jointly with M. Arato and Ya.G. Sinai
§1. Here we consider a two-dimensional stationary random process whose com-

ponents e(t) and TJ(t) satisfy the stochastic differential equations
de = -~edt - WTJdt + d</J,

(1)
dTJ = wedt - ~TJdt + d.,p,
where </J(t) and .,p(t) are two independent Wiener processes with
Ed</J = Ed.,p = 0, E(d</J)2 = E(d.,p)2 = adt.

Setting ( = e + iTJ, X = </J + i.,p, , =~ - iw, we can write the system
(1) as one equation
d( = -,(dt + dX. (la)
The complex correlation function of our process is of the form
C(r) = A(r) + iB(r) = E[(t)(t + r)] = (J'2 exp( -~Irl- iwr), (2)
where (J'2 = a/~.

If a process is observed on the interval [0,1'], then we can determine the
empirical correlation function
c(r) = a(r) + ib(r) = --1rr l 0

T
-
T
(t)(t + r)dt. (3)
With probability 1 the right-hand derivative of the empirical correlation

function at 0 is
'( ) 1 2 1 2 •
C 0 = -a - T S ! + rS2 - zr,
where the parameter a introduced above characterizes the intensity of the

"white noises" </J'(t) and .,p'(t), and
* Dokl. Akad. Nauk SSSR 146:4 (1962), 747-750 (in Russian).
479
480 PARAMETERS OF A COMPLEX STATIONARY GAUSSIAN MARKOV PROCESS
In the expression for r the integration is over the angle defined from
«t) = 1«t)lei8 (').
The figure shows the empirical correlation function for Chandler variations
of the coordinates of the earth's pole. 1
§2. The parameter a is determined precisely from the realization. Now it

remains only to consider the problem of estimating the parameters A and w.
We denote by P the probability measure in the space of realizations of our
process on [0, Tl. In the same space we introduce the standard measure
V=LxW
where L is ordinary Lebesgue measure on the «O)-plane and W is two-dimen-

sional Wiener measure in the space of increments «t) - «0) with the same
characteristics as those taken for the random process X(t). It can be shown
that (cf. [2], [3])
A2 + w2
dP
dV = CAexp [- A
2a Ts~ - ;s~ + AT+ ;Tr ,
W ]
(4)
where C is a constant. Formula (4) shows that the system of three statistics
s~, s~, r is a sufficient system of statistics of the problem. Differentiating
dP A2 +w 2 A W
L=logdV =c'+logA- 2a TS~-;S~+AT+;Tr
1 The instantaneous axis of the earth's rotation moves with respect to the small
axis of the earth's ellipsoid (so called free nutation). These movements have a
periodic component with aperiod of one year. After eliminating this component
there remain Chandler's movements with tendency to fluctuate with period of
about 14 months, but which are not strictly periodic and have large and mainly
smooth variations of the amplitudes (waves of about 10-20 years). The figure
shows that the Chandler component of pole movement is in good agreement with
the hypothesis at the beginning of this paper.
The figure was obtained by processing the data of Table 6 from the book by
A.Ya. Orlov [1]. The component with one year period is singled out from the
coordinates x(t), y(t) of Table 6, and the remainder is taken to be e(t) and '1(t).
The nodes on the figure indicate the points corresponding to the increments of T
in 0.1 year. The figure shows straight away that the period 21r/w approximately
equals 14 months. The regular pattern of the spiral obtained might suggest that
the parameter A can also be estimated very precisely. This, however, is not true,
as will be explained at the end of our paper.
PARAMETERS OF A COMPLEX STATIONARY GAUSSIAN MARKOV PROCESS 481
with respect to wand A we obtain the equations
öL w 2 T
- = --TS 2 + - r = 0, (5)
öw a a
-öL
ÖA
= -A1 - A 2 s~
-Ts
a 2 - -a +T =° (6)
for determining the maximum likelihood estimates wand i From (5) we obtain
It can be shown that
(w - w)/u(w), u 2 (w) = a/Ts~
has a (0,1)-normal distribution (this result is precise, not asymptotic). Equa-

tion (6) always has a unique positive solution.
T = 0, 1, ... , n; n = 0, 1, ... , 156
Setting AT = "-, AT = k, we obtain for k the equation
where h 1 = si/aT, h 2 = sVaT.

482 PARAMETERS OF A COMPLEX STATIONARY GAUSSIAN MARKOV PROCESS
The distributions of the statistics h 1 and h 2 and consequently k, depend

only on the parameter K. Since the distribution of K. is continuous, for any
0', 0< 0' < 1, and K, 0 < K< 00, we can find k such that
P{k> klK} = 0'. (7)
Taking the inverse of the dependence k = ka(K) we obtain = Ka(k) K
(it has been established that ka(K) is monotone increasing from 0 to 00 as

K increases from 0 to 00, so that taking the inverse is possible and unique).
Clearly,
(8)
We have initiated computations of Ka(il:) for 0' = 0.1; 0.05; 0.025; 0.01;
0.005; 0.001; 0.9; 0.95; 0.975; 0.99; 0.995; 0.999. The results thus obtained
will be published when the computations are completed.
For small k, (8) is equivalent to the relation
P{k< UK} = exp( -1/u), (9)
that is, k/ K has a X2 distribution with two degrees of freedom. For large k, (8)
is equivalent to
(10)
that is, the estimate k is asymptotically normal with variance
(11)
§3. For the case mentioned in the beginning of the paper, that is, the movement
of the earth's pole, we have 2
w = 5.274, k = 3.6, 211": W = 1.191, u(211": w) = 0.006
2 The introduction of Wiener processes 4> and ,p, that is, perturbations of "white
noise" type in (1) is, of course, a gross generalization in the case of the movement
of the earth's poles. It would be more correct to write
e' = -~{ - Wfl + J, fl' = w{ - ~11 + g.
However, data from [1] show that the values J(t) and g(t) at times t separated by
several years are actually independent, so the replacement of J and 9 by "equiv-
alent white noise" is possible. Apparently, the error in determining the intensity
PARAMETERS OF A COMPLEX STATIONARY GAUSSIAN MARKOV PROCESS 483
based on observations over T = 60 years.

The asymptotic formula (11) gives
0- 2 (,,) = 3.6.
Since K is clearly positive and (10) gives a negative estimate for k a when
er < 0.03, it is evident that the asymptotic formula (10) is still unsuitable.
Our computation gives the estimates
KO.90 = 5.5; KO.95 = 6.2; KO.975 = 7.8;
KO.I0 = 1.27; KO.05 = 0.82; KO.025 = 0.46,
which, as regards A, correspond to the estimates

KO.90 = 0.09; KO.95 = 0.10; KO.975 = 0.13;
KO.I0 = 0.021; KO.05 = 0.014; KO.025 = 0.008,
Moscow State University, 20 February 1962
References
1. A.Ya. Orlov, Service 0/ latitude, Akad. Nauk SSSR, Moscow, 1958 (in
Russian).
2. Ch. Striebel, Ann. Math. Statist. 30:2 (1959), 559.
3. M. Arato, Dokl. Akad. Nauk SSSR 145:1 (1962) (in Russian).
4. E.P. Fedorov, Proe. Poltava Gravimetrie Observatory 2:3 (1948) (in Rus-
sian).
5. N.I. Panchenko, In: Proc. 14th Astronomical Conference of the USSR,
Akad. Nauk SSSR, Moscow, (in Russian).
6. W.H. Munk and G.J.F. Macdonal, The rotation o/the earth, Cambridge,
1960. (Monographs on mechanies and applied mathematics.)
7. H. Jeffreys, Monthly Notiees Roy. Astronom. Soc. Geophys. Suppl. 100
(1942), 139.
8. A.M Walker and A. Young, Monthly Notices Roy. Astronom. Soc. Geophys.
Suppl. 115 (1955), 443; (1957) 117, 119.
a of this equivalent white noise is sma.ll enough not to affect the estimates of
A substantia.lly. The value of wis computed via the discrete analogue of for-
mula (*) obtained using the maximum likelihood method for the "discrete-time
scheme".
Concerning the estimation of the parameters A and w for the earth 's movement,
= =
see also [6]. The results of [6] are dose to ours: A 1/15; 211': W 1.193. Close
values were given by Jeffreys [7], but [5], [8] give sharply different values: A = 0.3
=
and A 0.01.
51. ON THE APPROXIMATION OF DISTRIBUTIONS OF SUMS OF
INDEPENDENT TERMS BY INFINITELY DIVISIBLE DISTRIBUTIONS *
Introduction
In what follows e= 6 + ... + en is the sum of n independent real terms

Fk(x) = p{ek < x}, H(x) = p{e < x},
x --
G172 () ~1
y211'u
1'"
-00
e _",2/ 2q 2 d x, u> 0,
for x 0,
= Go(x) = {~
~
E(x)
for x > 0,
l) = {D} is the set of infinitely divisible distribution functions D(x), and

Cl, C 2 , ••• are positive constants.
We will prove the following two theorems, which are strengthened versions
of those given in my paper [1].
Theorem 1. There exists Cl such that tor identically distributed ek, there
= =
exists tor any F(x) Fk(x), k 1,2, ... , n, a distribution D E l) such that
(0.1)
tor all x.
Theorem 2. There exists C 2 such that tor any (; > 0, L > 21 > 0, the
inequalities
(0.2)
tor all x and k = 1, ... ,n imply the existence ot D E 1) tor which

D(x - L) - 6 ~ H(x) ~ D(x + L) + 6 (0.3)
tor all x, where

6 = C 2 max(± (log~) 1/2, (;1/3). (0.4)
The history of the problem is as follows.
* Trudy MoskolJ. Mat. Obshch. (Proe. Moscow Math. Soc.) 12 (1963),437-451 (in
Russian).
484
APPROXIMATION BY INFINITELY DIVISIBLE DISTRIBUTIONS 485
1. It follows from the dosedness (with respect to weak convergence) of the dass
of infinitely divisible distributions introduced by Bruno de Finetti [2] that if
the distributions of sums
(0.5)
whose terms are independent and identically distributed within each series,
converge weakly, then the limit distribution is infinitely divisible.
It is tempting to interpret this result as folIows: the distribution 0/ the sum
0/ a large number 0/ identically distributed independent terms is dose to an in-
finitely divisible distribution. However, prior to my work [1] this interpretation
was not quite convincing.
Even for a sequence
of independent identically distributed terms the "totally divergent" case is pos-

sible (according to Doeblin [3]) when for no normalization
and for no sequence

nl < n2 < ... < nk < ...
can the distributions of the sums e(n,,) converge to anything other than de-
generate distributions E(z - a). This, of course, can be achieved by choosing
sufficiently small factors An.
Only in 1955 Yu.V. Prokhorov [4] proved that for the case of a sequence of
identically distributed independent variables en there always exists a sequence
of infinitely divisible distribution functions
which approximate the distributions Hn(z) of the sums
e(n) = 6 + 6 + ... + en
in the sense that

sup IHn(z) - Dn(z)l- 0 (0.6)
z:
486 APPROXIMATION BY INFINITELY DIVISIBLE DISTRIBUTIONS
as n ---+ 00. Still, the work left open the question of whether the convergence in
(0.6) is uniform with respect to the distribution function F(x) of the variables
en.
In terms of the supremum metric
p(F', F") = sup IF'(x) - F"(x) I

x
the question is whether or not the function 1
1jJ(n) = supp(Hn ,!»

F
tends to 0 as n ---+ 00. The answer to this question was given in my work [1]:
it was proved that
(0.7)
In 1960 Prokhorov [5] strengthened this result by showing that
(0.8)
Our Theorem 1 states that 2
(0.9)
It was natural to try to estimate 1jJ(n) from below. These estimates were
made by the student of Prokhorov, I.P. Tsaregradskii, Prokhorov himself and
L.D. Meshalkin. The latest result of Meshalkin [6] is
(0.10)
2. For sums
1 The supremum is taken over all distribution functions F.

2 After this paper was finished I found out that in 1961 F.M. Kagan had obtained
a result that is somewhere between (0.8) and (0.9):
t/J(n) = O(n- 1 / 3 1ogn).
Later this result was reported by F.M. Kagan at the Meeting on Probability
Theory and Mathematical Statistics in Fergana (September 1962).
all terms of which are independent within each series but have different dis-
tributions, A.Ya. Khinchin in 1937 [7] established a sufficient condition for
the limit distribution of e(k) to be infinitely divisible. This is the condition of
infinitesimal terms: there exist
for which the distributions Fki of the eki satisfy the condition
Our Theorem 2 is an attempt to give a uniform character to this result

of Khinchin. The essen ce of Theorem 2 may be clarified by using the "Levy
distance"
PL(F', F") = inf f,
where the infimum is taken over all f satisfying the condition
F'(x - f) - f ~ F"(x) ~ F'(x + f) + f.

It is easy to see that Theorem 2 implies
Corollary. 1f
sUPPL(Fi,E) ~ 1/,
i
then
3. As is weH known, the most powerful means of proving limit theorems on the
distribution of sums of a large number of independent terms is the apparatus
of characteristic functions. Now "direct" probabilistic methods in this area
can very sei dom compete with the possibilities of the analytic apparatus of
characteristic functions.
Our Theorems 1 and 2 give an interesting example of another state of
affairs. An essential element for proving these theorems is Lemma 1, which
refers to the "concentration functions" introduced by P. Levy. I strengthened
theorems of Levy and Doeblin on the properties of concentration functions
([8]) specificaHy to prove the new versions of Theorems 1 and 2 given in [1].
Further progress in deriving estimates for concentration functions is due to

B.A. Rogozin (see [9], [10]). He also makes use of elementary direct probabilistic
and set-theoretic methods (Erdös' theorem on the subsets of a finite set). 3
Mathematicians whose attention I drew to this problem have so far failed
to prove theorems of types 1 and 2 without turning to these particular methods.
Throughout this paper, as in [1], I make use of the methods of reasoning
proposed by Doeblin (see, for example, [3]). As can be seen from the above, the
passage from 1/5 to 1/3 in Theorem 1 was carried out by Prokhorov. In order
to eliminate the factor log2 n from Prokhorov's estimate (0.8) it was required
a) to use the more accurate estimates of concentration functions obtained by
Rogozinj b) to make some changes in Prokhorov's proof, namely to introduce
Lemmas 5 and 6. 4
The passage from 1/5 given in [1] to 1/3 in the proof of Theorem 2 was
made by similar techniques borrowed from Prokhorov's work [5], which also
makes use of Lemmas 5 and 6.
4. Apart from the distance
p(F', F") = sup IF'(z) - F"(z)l,

It:
it is natural to consider the "variation al distance"
Pv(F',F") = !Var[F'(z) - F"(z)] =sup[F'(A)

A
- F"(A)],
where A is an arbitrary Borel-measurable set on the straight line.

As is weIl known,
p(F', F") $ Pv(F', F").
Therefore, for the function
we have
(0.11)
3 This result of Rogozin (now know as the Kolmogorov-Rogozin inequality) was
proved by Esseen in 1966 by the method of characteristic functions (Editor's
note).
4 The first of these steps was taken by F.M. Kagan somewhat before this paper
was written (see footnote 2).
We do not know whether tPv(n) -+ 0 as n -+ 00.
§1. Eight Lemmas
After P. Levy, we introduce for any distribution function F(x) its "concentra-
tion function"
QF(l) = sup[F(x + I + 0) - F(x)].
x
Lemma 1. If xr - x'k > land
for k = 1,2, ... , n, L ~ I > 0 then
Lemma 2. If U > 0, I > 0,
then for any distribution function F(x)
Lemma 3. If U > 0, Ul > 0, then
Lemma 4. If
J xF(dx) =0, J x 2 F(dx)=u 2 , h~u>O,

then
L sup IF(x) -
00
E(x)1 ~ es.
r:-oo rh:5x:5(r+l)h
Lemma 5. If E{k = 0, lekl ~ I, Oe = (7'2, h ~ (7' > 0, then
Lemma 7. For any positive integer n and 0 ~ p ~ 1,
Lemma 8. Let 5 0 ~ Pk ~ 1,
I-Pk for m = 0,
Pk(m) =
{
~k for m = 1, qk(m) = ~e-Pk,
for m > 1,
Then
L Ip(m) -
m
q(m)1 ~ C 12 LP~'
k
Lemmas 2 and 3 are proved by direct eomputation. Lemma 1 is a di-

reet eorollary of Theorem 1 from Rogozin's paper [9]. Lemma 7 is proved in
Prokhorov's work [11], Lemma 4 follows from the estimate (F is the distribution
of e):
(Chebyshev's inequality).
Lemmas 5 and 6 are elose to the known estimate
(1.1)
5 Here and in what folIows, m = (mb"" mn) is an n-dimensional veetor, Lm

denotes summation over the veetors m with non-negative integer components.
which follows from Lyapunov's theorem in Esseen's formulation ([12], p.216):
(1.2)
Lemma 6 may be derived from (1.2) if the additional normal term with
variance 0"5 is represented as a sum of a large number of terms with sufficiently
small variances.
The proof 6 of Lemma 5 is somewhat more difficult.
§2. Proof of Theorem 1.
1. In what follows we consider n > 1. It can easily be seen that this requirement
is inessential.
2. We suppose further that the ek = F k- 1 (1Jk) are non-decreasing functions of

the independent variables 1Jk with uniform distribution on [0,1]:
p {1Jk ::; Y} = Y for 0 ::; y ::; 1.
It can easily be shown that in an appropriate extension of the basic prob-

ability field (O,!.m, P) such 1Jk do exist. 7
3. We set
p = n- 1/ 3 ,
for p/2 < 1Jk < 1 - p/2,
~k = {~ otherwise.
a = E{ekl~k = O}, 0"2 = D{ekl~k = O},

A(x) = p{ek < Xl~k = O}, B(x) = p{ek < Xl~k = 1}.
All these constructions hold when passing from 6 to
e~ =6 - a,
6 Lemma 5 can easily be derived from the following estimate due to A. Bikyalis:
where Cis an absolute constant (see Litov. Mat. Sb. 6:3 (1966), 323-346) (Edi-
tor's note).
7 The functions Fk- 1 must be suitably defined. This is left to the reader.
onlya' = 0 appears instead of a, and the functions A(x) and B(x) are replaced
by
A'(x) = A(x + a), = B(x + a).
B'(x)
Therefore we need only consider the case a = 0, to which we shall restrict

ourselves in what folIows.
4. In the decomposition
F(x) = pB(x) + (1 - p)A(x)
the support of the distribution A lies in the interval [x-, x+],
of length
while the support of the distribution B lies outside this interval, with each
of the rays (-00, x-] and [x+, 00) having probability ~ in the distribution B.
Lemma 1 may be applied to the distributions B rn (throughout, the powers
of a distribution are understood in the sense of convolution), which gives the
estimate
(2.1)
5. We approximate the distribution
H = [pB + (1 - p)Ar = L C: prn(l - pt-rn B rn * An-rn

rn
by an infinitely divisible distribution D differently according to the two cases:

A) A~ Vnu. B) A< Vnu.
ease A. We set
D = enp(B-E) =L (np~rn e- np B rn ,
tIM
m.
H1 = L C: prn(l - p)n-rn B rn .
rn
According to Lemma 7,
(2.2)
By Lemma 4 and estimate (2.1), for h = -' we have S
IBm * An - m - Bml ~ J IAn-m(z - z) - E(z - z)IBmdz ~
~ QBm(-') Er sup
r>'~!I~(r+l)>'
IAn-m(y) - E(y)1 ~ C5 CS m- 1 / 2 • (2.3)
Therefore
IH - H11 ~ EC:'pm(l- pt-mlB m * A n - m - Bml ~ Cs Cs n- 1/ 3 + 2E',

m
where
(2.4)
Noting that
EI' = np = n2 / 3 , (2.5)
DJ.' = np(l - p) ~ n 2 / 3 , (2.6)
we obtain by Chebyshev's inequality
E' ~ P{IJ.' - n 2/ 3 1 > ~n2/3} ~ 4n- 2/ 3 ,

which, together with (2.4), gives
IH - H11 ~ (2Cs Cs + 8)n- 1/ 3 • (2.7)
From (2.6) and (2.2) we obtain (0.1)
Case B. We set
D -- enp(B-E) * Gn(1-p)q2 -- ""' (npr -nPB m

L..J --,-e * Gn(1-p)q2,
m
m.
H2 = E C:'pm(l- pt-m B m * G (1_p)q2. n

m
S The variance of the distribution A n - m is (n - m)/T 2 , so that the conditions of

Lemma 4 are satisfied for h = " 2: Vn/T.
According to Lemma 7 we have
(2.8)
The difference H - H 1 is estimated using Lemma 5, where we now assurne

that h = ,fiiu:
~
,fiiu
C5 - - m
-1/2
C g -A- = C 5 C9 m -1/2 .
A ,fiiu
This estimate is quite similar to estimate (2.3) for case A. As in case A we

obtain
(2.9)
The difference H 1 - H 2 is estimated using Lemma 3:
where
I n-m
n(l-p)
-ll>fu
C7
Using Chebyshev's inequality we obtain from (2.5) and (2.6) for n > 1 and an
appropriate choice of C 16 the estimate
11
"L...J <
- C17 n- 1 / 3 ,
which leads to
(2.10)
From the inequalities (2.8)-(2.10) we obtain (0.1), wh ich completes the

proof of Theorem 1.
§3. Proof of Theorem 2
1. We suppose without lass of generality that f < 1.

2. We show that it suffices to consider the case of continuous and strictly
increasing functions Fk (x).
Assurne that Theorem 2 is proved for continuous and strictly increasing
functions with constant C~ and consider the sum
with arbitrary Fk(x) satisfying (0.2). Let L > 2/. Choase I' and L' such that_
L> L' > 2/' > 2/, I'/L' ~ ~I/L. (3.1)
By Lemma 2 we can choose 0"0 to be so small that for any distribution function
F(x) we have
F * GI1~(x -)..) - f ~F(x) ~ F * GI1~(x +)..) + f, (3.2)
F * Gnl1~(x - A) - {/ ~F(x) ~ F * Gnl1~(x + A) + 6', (3.3)
where
)"=1'-1, A=L-L',
I' L'
C~ max[L' (log l')
1/2
6' = ,(2f)1/3].
Set
According to (0.2) and (3.2) we obtain
E(x -I') - 2f ~ FHx) ~ E(x + I') + 2c
Since the functions F~ are continuous and strictly increasing, there exists an
infinitely divisible distribution D' for which
D'(x - L') - 6' ~ H(x) ~ D(x + L') + 6'. (3.4)
Noting that
(3.3), applied to F(x) = H(x), and (3.4) imply that

D'(x - L) - 26' $ H(x) $ D'(x + L) + 26'. (3.4')
Since under a suitable choice of C 2 we have, by virtue of (3.1),
26' $ 2C~max[~, (log ~:r/2,(2()1/3] $ c2max[~(log*r/2,(1/3],

formula (0.3) follows from (3.4').
3. In accordance with No. 2 we take Fk(X) to be continuous and strictly

increasing. Then
is defined for an p, 0 < P < 1, continuous and strictly decreasing. It takes an

positive values. Therefore, the inverse function Pk(A) is defined for 0 < A < 00,
is continuous, strictly decreasing and takes an values in the interval 1 > P > o.
The function
S(A) = LPk(A)
k
is also continuous and strictly decreasing. It takes all values in the interval
n > S > O. Therefore for 0 < ( < 1 there exists a unique solution AO of the
e quat ion
S(A) = (-2/3.
4. We set
AO, if AO ~ I,
A= {
I, if AO < I,
S = s(A) = LPk'
k
for x;;
< (k < xt,
otherwise,
ak = E{ekll'k = O},
(j~ = D{ekll'k = O},
(j2 = L(I- Pk)(j~.

k
APPROXIMATION BY INFINITELY DIVIsmLE DISTRmUTIONS 497
Setting
we represent Fk (z) as
where the support ofthe distribution Ak lies in the interval [z;, ztl, while that
of the distribution Bk lies outside this intervalj here the prob ability of the rays
(-00, z;l and [zt, 00) is !.
Using the notation of Lemma 8 and setting 9
B( m ) = rr *Bmr.
k k , A(m) -_ rr:A~-mr.,
,. ,.
we obtain
The construction of the approximating infinitely divisible distribution differs

according to the three cases A, B and C:
A B C
).0 ~ 1 ).0 ~ 1 ).0< 1
).0 ~ (T ).0 < (T
). = ).0 ). = ).0 ). = 1
s =C 2/ 3 S = e- 2/ 3
5. Since always A ~ 1, it follows that
(3.5)
9 The B(m) are defined for any negative mk, while the A(m) are defined only
when the mk take the values 0 and 1.
This is the only instance in our proof when we use condition (0.2) of the theo-
rem. Since the definition of Pk(A) and all the other variables essential for our
constructions is invariant with respect to the shifts
we need only consider the case
6. We now make certain calculations that we shall need in the sequel:
r is equal to the number of variables 6 such that x'k < ek < xt. It is easily
checked that
Er = s, Dr = L:Pk(l- Pk) < s.
k
Therefore, according to Chebyshev's inequality,
P{lr - si ~ C} = L:
It(ID)-'I~C
p(m) < ;2' (3.6)
7. Suppose further that
This is the sum of those ek for wh ich x'k < ek < xt. In view of the assumption
that the ak = 0, for any m
E( = 0, E«IJ.t = m) = O.
In what follows we shall be interested in the conditional variance
For the random variable p2 = u 2 (J.t) it is easy to calculate
D p 2 = L:Pk(l- Pk)U:.
k
Since
~)1- Pl:)o-~ = 0- 2 ,
I:
we have the inequality
Therefore
0- 2 A2 f
L: p(m) ~ 4C2 • (3.7)
10"2(m)-0"21~c
8. Finally, we note that by Lemma 1,
QB(m)(L) ~ C5(L/Ao)t(m). (3.8)
We now proceed to the proof of Theorem 2 for the cases A, Band C.
Case A. In this case
is approximated by
D = exp L: PI: (BI: - E) = L: q(m)B(m).

I: m
To proceed from H to D we also consider
According to Lemma 8 and (3.5),
ID - H11 ~ L: Ip(m) - q(m)1 ~ C 12 L:p~ ~ C12 f L:Pk ~ C12 1/ f 3• (3.9)

m k k
On the other hand, according to Lemma 4 with h = A = AO ~ 0- and (3.8),
IB(m) * A(m) - B(m)1 ~ JIA(m)(x - z) - E(x - z)IB(m)(dz) ~

~ QB(m)(AO) L:r sup
r>'o~y~(r+l)>'o
IA(m)(y) - E(y)1 ~ C5Cs[t(m)r1/2. (3.10)
Therefore
where
~' = ~ p(m).
t(m)<t c2 / 3
By (3.6) and noting that in our case s = (.2/3, we have
that is, (3.11) implies that
(3.12)
Prom (3.9) and (3.12) we obtain
(3.13)
ease B. In this case we set
D = exp ~Pk(Bk - E) * Gu = ~q(m)B(m) * G

2 U 2,
k m
m m
The inequality
(3.14)
is derived in the same way as (3.9) in case A.

By Lemma 5 for h = U > A = Ao and (3.8) we obtain
IB(m) * A(m) - B(m) * G u 2(m)1 ~

~ J IA(m)(x - z) - G u 2(m)(X - z)IB(m)(dz) ~
~ QB(m)(U) ~ sup IA(m) - Gu2(m) I ~

r ru:$y:$(r+1)u
(3.15)
Exactly as we obtained (3.12) from (3.10) in case A we now obtain from (3.15)
(3.16)
It now remains to estimate the difference H 1 - H 2 • By Lemma 3,
if
Using (3.7) and taking into account the fact that now A < (1', we obtain the
estimate
L:" = L: p(m) $ G20 e1/ 3 • (3.17)
1";2L11~%:
Therefore in the same way that we derived (2.10) in the proof of Theorem 1
we obtain
IH1 - H 2 1 $ (G19 + G20 )e 1/ 3 •
It follows from (3.14), (3.16) and (3.17) that
(3.18)
Inequalities (3.12) and (3.18) show that in cases A and B the estimate
(0.3), which is the essence of Theorem 2, may be replaced by the following
stronger one:
(3.19)
Case C. In this case we have not managed to obtain an estimate of the type
(3.19). We set
1 ( L)-1/2
(1'0 = ..,ß.L log T (3.20)
and introduce the auxiliary distribution
H' = H*G ..o '
By Lemma 2, for
(3.21)
we have
H'(z - L) - TJ:S H(z) :S H'(z + L) + TJ. (3.22)
We now show that the infinitely divisible distribution
D =L q(m)B(m) * Gq~, u? = u2 + u~,

m
satisfies the inequality
ID - H'I :S C23 [f 1/ 3 + ~ (log~) 1/2]. (3.23)
Using (3.23) together with (3.21) and (3.22) we obtain (0.3).

It now only remains to prove (3.23). For this we introduce the distributions
H 1' -- ",",p(m)B(m)
LJ *G q~(m),
_21(m) -_ _
v v
2(m) + v-02,
m
H~ = Lp(m)B(m) * Gq~.
m
The inequality
(3.24)
is proved in the same way as (3.9) and (3.14) in cases A and B, only now
which leads to the estimate
LP~ :S f LPk < f1/3.

k k
By Lemma 3,
(3.25)
where E III
runs over those p(m) for which
(3.26)
For suitable C24 , inequality (3.25) implies that

Therefore (3.7) and the inequalities 10 U2 < UI, A = I $ Uo imply that
Thus from (3.25) we obtain
(3.27)
Finally, according to Lemma 6 we obtain from (3.20)
whence we have
IH' - H~I = v'2C102I (log T
L) 1/2 . (3.28)
Formulas (3.24), (3.27) and (3.28) immediately imply (3.23). This completes
the proof of Theorem 2.
Steamboat "Sergei Kirov"
Red Sea-Persian Gulf
13-24 March 1962
References
1. A.N. Kolmogorov, 'Two uniform limit theorems for sums of independent

terms', Teor. Veroyatnost. i Primenen. (Probability theory and its appli-
cations) 1:4 (1956), 426-436 (in Russian). (Translated as Theory Probab.
Appl.)
2. B. de Finetti, 'Le funzioni caratteristiche di legge instantanea', Atti Aeead.
Naz. Lineei. Rend. Sero 6 12:7/8 (1930), 278-282.
3. W. Doeblin, 'Sur les sommes d'un grand nombre des variables aleatoires
independantes', Bull. Sei. Math. 63 (1939), 23-32; 35-64.
4. Yu.V. Prokhorov, 'On sums of identically distributed random variables',
Dokl. Akad. Nauk SSSR 105:4 (1955), 645-647 (in Russian).
10 The second inequality is obtained from (3.20):
I ( L) 1/2
I =..fiT, log T 0"0 :5 0"0·
5. Yu.V. Prokhorov, 'On a non-uniform limit theorem of A.N. Kolmogorov',

Teor. Veroyatnost. i Primenen 5:1 (1960), 103-113 (in Russian).
6. L.D. Meshalkin, 'On approximating distributions of sums by infinitely di-
visible laws', Teor. Veroyatnost. i Primenen. 6:3 (1961), 257-275 (in Rus-
sian). (Thanslated as Theory Probab. Appl.)
7. A. Khinchin, 'Zur Theorie der unbeschränktteilbaren Verteilungsgesetze',
Mat. Sb. 2:1 (1937), 79-119.
8. A.N. Kolmogorov, 'Sur les proprietes des fonctions de concentrations de
P. Levy', Ann. Inst. H. Poincare 16:1 (1958), 27-34.
9. B.A. Rogozin, 'On an estimate of a concentration function', Teor. Veroyat-
nost. i Primenen 6:1 (1961), 103-105 (in Russian). (Thanslated as Theory
Probab. Appl.)
10. B.A. Rogozin, 'On increasing the variance of sums of independent ran-
dom variables', Teor. Veroyatnost. i Primenen. 6:1 (1961), 106-108 (in
Russian). (Thanslated as Theory Probab. Appl.)
11. Yu.V. Prokhorov, 'Asymptotic behaviour of the binomial distribution',
Uspekhi Mal. Nauk. SSSR 8:3 (1953), 135-142 (in Russian).
12. B.V. Gnedenko and A.N. Kolmogorov, Limit distributions for sums of
independent random variables, Addison-Wesley, 1954 (translated from the
Russian).
52. ESTIMATORS OF SPECTRAL FUNCTIONS
OF RANDOM PROCESSES *
Jointly with I.G. Zhurbenko
The paper deals with the asymptotic behaviour of efficient statistics of a spec-
tral density as the sampie size increases. A certain new statistic introduced
by A.N. Kolmogorov is compared with other well-known statistics of a general
density, as weIl as with asymptotically optimal statistics with respect to the
mean square deviation. The influence on various statistics of high peaks of
the spectral density at neighbouring frequencies is discussed. A new dass of
statistics is obtained by applying the shift operator to a periodogram calculated
from smoothed data.
Let X(t), t = 0, ±1, ... , be a stationary random process with expecta-
tion EX(t) = 0, covariance function C(t) and spectral density f(>'), where
f(>'), -00 < >. < 00, is a function with period 211". For dimensional reasons it
is natural to consider all quadratic forms
N
1 'W" b(N)
211"N 8 ,t X(s)X(t) (1)
l,t=1
with arbitrary coefficients b~~) as the admissible dass of statistics. As was

shown by Grenander and Rosenblatt [1], the asymptotic behaviour as N -+ 00
of the first two moments will not be worse if instead of the dass of statistics
(1) we consider the narrower dass of statistics of the form
(2)
which can be represented as
(3)
where
1 N-Itl
BN(t) = N L: X(s)X(s + Itl).
8=1
* Proc. 2nd European Conference on Statistics, Oslo, August 14-18, 1978. First
pu blica tion.
505
506 ESTIMATORS OF SPECTRAL FUNCTIONS OF RANDOM PROCESSES
The statistics iN(>') can also be written in the from
(4)
where
As N ~ 00, the asymptotic minimum of the shift l:l.iN(>') = EiN(>') - 1(>')

and the variance and consequently also the mean square deviation "V iN(>') =
E(jN(>') - 1(>.))2 of the statistics of the dasses (2) and (3) are, according to
[1], not greater than those of the statistics of dass (1). The function N(X)
is called the "spectral window" of the statistic iN(>') and will be discussed in
detail below.
The asymptotic properties of the estimators iN(>') depend on the smooth-
ness of the spectral density 1(>').
We say that
X(t) E /C(>" 0', C, Cd = /C,
if for any JJ and certain given >., 0 < 0' ~ 2, C 2: 0, Cl > 0, one of the
inequalities
1/(>' + JJ) - 1(>')1 ~ CIJJI'\ 0< 0' ~ 1,
holds, or if I' (>') exists and
1/'(>' + JJ) - 1'(>')1 ~ CIJJlcr-l, 1 < 0' ~ 2,
and the 4th order spectral density 14(Xl, X2, X3, X4) of the process X(t) IS
bounded:
We say that a sequence offunctions N(X) is of dass T, N(X) E T, iffor

any integer N > 0 N(X) is a continuous even periodic function with period
211" such that N(X) ~ 0 uniformly in the region f ~ lxi< 11" for f> 0 and if
the following relations hold:
ESTIMATORS OF SPECTRAL FUNCTIONS OF RANDOM PROCESSES 507
Moreover, the following smoothness conditions hold uniformly in :F:

1. For an integrable I(x) such that I/(x)1 ~ Ixl a , 0< a ~ 2, a relation of
the form
holds, where Feller's kernel 4>'jy(z) is defined as follows:
(4a)
2. There exists a sequence of numbers A = A(N) > 0, A(N) -+ 00 as
i:
N -+ 00, such that
c)N(X)c)N(X + Y + z)dx =
= i: c)N(X)c)N(X + y)dx + o(i: c)'jy(x)dx)
uniformly with respect to y for all Izl < A/N.

We define the dass :F(G) C:F ofsequences offunctions c)N(X) ofthe form
(5)
for some AN > 1 such that AN -+ 00 and AN / N -+ 0 as N -+ 00. Let

G(x), -00 < x < 00 be even, piecewise differentiable and satisy the conditions
(6)
for some a > O.

The problem of finding the optimal statistics in the sense of the mean
square deviation is solved under the above natural restrictions on the smooth-
ness of c)N(X). In this case we find not only the type of asymptotics of the
mean square deviation, as it was done by E. Parsen [2], but also the values
of the coefficients in these asymptotics, which allow us to make a numerical
comparison of the most often used statistics of the spectral density proposed
by various authors. A considerable weakening of the conditions used in this
case is the requirement of smoothness of the spectral density 1(>') at one single
point as compared to the uniform smoothness
E
00
Itla B(t) < 00,

t=-oo
where B(t) is the correlation function of the process X(t).

Note that the smallest possible order of the mean square deviation of the
estimator iN(>') is N- 20:o/(1+2a o ), where ao is the maximal a for which the
latter condition is fulfilled. Unfortunately, however, ao is the characteristic of
smoothness of 1(>') simultaneously with respect to all frequencies [3], [4], [7].
Theorem 1. 11 X(t) E K(>', a, C, Cl)' <IlN(x) E :F, then lor the statistic iN(>')
we have, as N -+ 00
(7)
where
I, >. == 0 (mod 71"),
7](>') = { (8)
0, >. t 0 (mod 71").
The minimum in (7) is attained at
(9)
where
lxi ~ 1,
lxi> 1,
and
( aNC2 ) 1/(1+20:)
(10)
AN = 7I"J2(>')(1 + a)(1 + 2a) .
Proof. Isolating the main term of the mean square deviation \7 iN( >.) we obtain,
in accordance with Lemmas 1 and 2 in [3], the variational problem of finding
the minimum of the functional
over aH positive 4>N(X) E :F.

In solving the problem of finding the conditional extremum we see that
the solution 4>N(X) must satisfy the equation
which implies that one should look for a solution of the form (9). Taking into
ac count the initial conditions we obtain (10) and with it (7) and (8).
Theorem 1 gives an explicit form of the optimal window in the sense of
the mean square deviation of the estimator iN(A) for a given smoothness of
the spectral density f(A) under study at the point A.
In order to compare asymptotic properties of various periodogram estima-
tors of the form (2) and (4) we use the following
Theorem 2. Let 4>N(X) E:F be dejined by (5) and suppose that G(x) satisjies
(6) for some a such that 0 < a:::; 2. Then for X(t) E K, we have
. ~ ( C2 )
mf sup VfN(A)""f 2(A) f2(')
1l2a 2a
N-l+2ag(a) (11)
AN X(t)EI< A
as N -+ 00, where
C;:a)(2aVdd2a(211'V2)1:l!~a,
i: i:
g(a) =
(12)
V1 = IxlaG(x)dx, V2 = G2 (x)dx,
with asymptotic maximum attained at
(12a)
The proof of this theorem foHows from Lemmas 1 and 2 of [3].

The function g(a) can be calculated by (12) for various estimators of the
spectral density, namely, those of Tukey, Parsen, Bartlett et al. (see, for exam-
pIe, [3]), as weH as for the optimal statistics given by (10). The results of the
calculations are given in [3], [7]. A common drawback of periodogram estima-
tors of the spectral density iN (A) is the weak decrease (of order O( N-l)) of
the dependency on high frequencies and non-stationary phenomena. Earlier,
one of the authors (A.N. Kolmogorov) suggested that these drawbacks could
be eliminated by using a "time window" followed by averaging periodograms
over various time intervals.
Let aM(t), t = 0, ±1, ... , be a non-negative function vanishing outside
[0, M]. From a sampie {X (Q), ... ,X (Q + M)} we construct the function
L
00
W~(.\) = aM(t - Q)X(t)e iAt .

t=-oo
We determine the statistic fN(.\) of the spectral density f(.\) of the random
process X(t) in the following way:
T-1
fN(.\) = ~ L IW~k(.\W·
k=O
It makes use of N = T(L - 1) + M + 1 sampie values of the process X(t).

Let us estimate the spectral density. We determine the "data window" as
folIows:
aM(t) = aK ,p(t) = J.l(K, P) ( K(P12 -

2 1)) 1/4 K
P- CK,p(t), (13)
~
where M = K(P-1) and the coefficients CK,P are determined from the relation
Since
L
00
cPM,P(X) = cPM(X) = aM(t)e itx ,

t=-oo
it follows from (13) that the "spectral window" of the estimator in quest ion is
given by
I'f'K,P
A. ()1 2 = 2(K p)(K(P2 -1))1/2(Sin 2(PX/2))K (13a)
x J.l, 12~ p2 sin2(x/2) ,
where J.l(K, P) ~ 1 as K ~ 00, P ~ 00 (see [3]). From the definition of

spectral densities of the 2nd and 4th order we obtain the following equalities
for the estimator IN(>"):
(14)
f( >")1.1, ( )1 2 1.1, ()12sin2(TL(x + y)/2) d d

X Y+ 'l'M X 'l'M Y sin 2(L(x + y)/2) X y+
+ ;2k2 f(x)f(y)cpM(X + >,,)cpM(-X + >")cpM(Y - >,,)cpM(-Y - >..)x
sin 2(TL(x + y)/2) d d

(15)
X sin 2(L(x + y)/2) X y,
where TI = [-11",11"],
L
00
c5*(x) = c5(x + 2k1l"), (15a)

k=-oo
c5(x) being the Dirac delta-function.
Theorem 3. 1f X(t) E K, then for the statistic IN(>") for>.. i= 0 (mod 11") with
coefficients defined by (13) we have
as the sampie size N = (L(T - 1) + K(P - 1) + 1) --+ 00, where
K(a) = 1+ 2a E(av'24 r2 (a + 1) 1211') 1/{1+2a),

2a VB..[;3 2
the asymptotic equality (14) being attained when
L N(1-2a)/(1+2a) P
N1/(1+ 2a ) --+ 0(1), P --+ 0(1), N1/(1+2a) --+ 0(1),
K p 2( N av'24 r2 (a + 1) 12aC2) -2/(1+211') = 0(1). (17)

..[;3J2(>") 2
Proof A direct application of Laplace's method for estimating the integral on

the right-hand side of (14) with kernel tPK,P(Z) calculated via (13a) gives
sup tl, 2iN(). = 1'4(K,P) ( K(PI22-1») -a c2r2C:W +2 1) 1 + O(K- 1).

x(t)e" 7/"
The main term of OiN().) gives an expression which, using the method of
steepest descent, can be reduced to the following form:
1=1'4(K P)f2().)V7/"2 K (P2- 1) (1+0( L )+O(~)). (18)

, TLV6 VK(P2 -1) K
The difference between (18) and the second term of (15) is estimated by
11 - 121$ L )Ha ( 1 + 0 (
C\Tp..(K { K(P2-1)})
exp - L2 + 0 (1))
K ' Q < l.
If Q > 1, then 11 - 12 1is of order P..(K(TL)-2. In a similar way we can show

that the third term in (15) has smaller order. The first term in (15) under the
conditions of the theorem is estimated in the following way:
=
Setting K(P2 -1) lIN" and finding the minimum of V iN().), we obtain the
asymptotic equality (16). This proves our theorem.
As has been shown by calculations, (see [3], [7]), the mean square devia-
tion appears to be closest to the optimal statistic in the case of Kolmogorov's
statistic iN().) as compared to the estimators of Tukey, Parsen, Bartlett, Abel
and others. As follows from Theorem 3, the statistic iN().) by comparison
with other statistics, is insensitive to changes in the parameters, which can be
chosen in broad ranges. The degree of dependence on high frequencies of this
statistic is of order N-2K(1+2a), where K can be arbitrarily large as N -+ 00.
This enables us to carry out a spectral analysis in the required frequency strip
in the presence of strong noises and non-stationary processes concentrated at
other frequencies, for example, in the presence of a trend. At the same time it
is easy to check stationarity in the frequency strip under study.
We now consider the effect of strong peaks in the spectral density fez) at
a frequency ). + tl, close to ).. Assume that
fez) = fa(z) + /6(z),

where fOl(x) is the spectral density ofthe process XOI(t) EI\, and /6(x) is defined
by an equation of the form
f6(x) = 6· 6*(x -..\ -.6.) + 66*(x +..\ + .6.), .6."t 0 (mod 211'), (19)
where 6 > 0 is areal number and the function 6*(x) is given by (15a). We say
that X(t) E I\, (..\,.6.,6) if the spectral density of X(t) is determined by (19),
the semi-invariant spectral density of the 4th order is bounded and EX(t) = O.
The asymptotics of the mean square deviation of the statistic with optimal
choice ofparameters in the presence of a chosen 6 at frequency ..\+.6. is described
by the following theorem (see [5]-[7]):
Theorem 4. Let X(t) E I\, (..\,.6.,6), let the kerneis cJ.>N(X) E :F be determined
by (5), let G(x) satisfy (6) for a ~ 2 and let the parameter AN of the statistic
iN(..\) be chosen in accordance with (12a). Then as N -+ 00
C 2 ) 1/(1+201)
sup "VfN(..\) = f2(..\) N- 201 /(1+ 201 ) x
A (
-2-
X(t))EI«>.,A,5) f (..\)
x(l + 7](..\))201/(1+ 201 )g(a)(1 + 0(1))+
+26 2 (1: cJ.>N(X)tfoÄr(x + .6.)dx + 1'1f'lf cJ.>N(X)tfoÄr(x + 2..\ + .6.)dx + f

+O(N 1: cJ.>N(x)tfoÄr(x + .6.)dx, (20)
where
..\ == 0 (mod 11'),
7](..\) = { 0,I, ..\"t 0 (mod 11')
(21)
and the functions tfoÄr(x) and g(a) are defined by (4a) and (12a) respectively.
Theorem 5. Let X(t) E I\, (..\,.6.,6) and let the statistic IN(..\) be determined
by the coefficients aM(t) found via (13) under the conditions (16) as N =
L(T-1)+M(P-1)+1-+00. Then
- ( C ) 1/(1+201)
sup "VfN(..\) = K(a)f2(..\) f2(..\) x
X(t)EI«>.,A,5)
XN- 201 /(1+ 201 )(1 + 7](..\))201/(1+ 201 )(1 + 0(1)) + 26 2(ltfoM(.6.W+
+ltfoM(.6. + 2..\W)2 + O(N- 0I /(1+201)ltfoM(.6.W), (22)

where l<Pn(x)1 2 , K(a) andTJ()..) are defined by(13a), (16a) and(21) respectively.
The proofs of Theorems 4 and 5 are given in [5]-[7].
Note that the order of the remainder terms in (20) for the statistic iN()..)
is not less than
(23)
if .6. ~ N- c/(1+2a)(l+c). For iN()..) the order of the remainder terms in (22) is
not less than
which can be taken arbitrarily small with respect to (23) for comparatively
small k as N --+ 00. Under these conditions
According to (23) the statistics iN()..) have fixed order of dependence on the
choice of 6, which cannot be diminished by choosing a suitable spectral window
N(X).
References
1. U. Grenander and M. Rosenblatt, Statistical analysis 0/ stationary time

series, Wiley, New York, 1957.
2. E. Parsen, 'On asymptotically efficient consistent estimates of the spectral
density function of a stationary time series', J. Roy. Statist. Soc. B, 20
(1958), 303-322.
3 I.G. Zhurbenko, 'On the efficiency of estimators ofthe spectral density of a
stationary process', Teor. Veroyatnost. i Primenen. 25:3 (1980), 476-489
(in Russian). (Translated as Theory Probab. Appl.)
4. I.G. Zhurbenko, 'The study of statistics of the spectral density of a sta-
tionary random process', Sibirsk Mat. Zh. 22:5 (1981), 40-65 (in Russian).
5. I.G. Zhurbenko, 'On the efficiency of estimators of the spectral density
of a stationary random process, 11', Teor. Veroyatnost. i Primenen. 28:2
(1983), 388-396 (in Russian). (Translated as Theory Probab. Appl.)
6. I.G. Zhurbenko, 'On a limit theorem for statistics of the spectral density
with time shift', Ukrain. Mat. Zh. 32:4 (1980), 463-476 (in Russian).
7. I.G. Zhurbenko, The spectral density 0/ time se ries, Moskov. Gos. Univ.,
1982, (in Russian).
53. ON THE LOGICAL FOUNDATIONS OF PROBABILITY THEORY *
When we talk about random events in the everyday sense of the word, we
mean phenomena in which we do not find any regularities that would allow us
to predict their behaviour. Generally speaking, there are no reasons to suggest
that events random in this sense obey any prob ability laws. Consequently we
should distinguish between randomness in the wider sense (the absence of any
regularity) and stochastic random events (which are the subject of probability
theory).
The problem is to describe the reasons why mathematical probability the-
ory can be applied at all to phenomena of the real world. My first attempt
to ans wer questions of this kind was made in [1] (published in an edition of
methodological character).
Since a random event is defined as the absence of regularity, we should
first define the notion of regularity. The natural means for this is given by the
theory of algorithms and recursive functions. The first attempt to apply it to
probability theory was made by Church [2].
The purpose of my report is to familiarize the readers with this field, at
least in a first approximation.
Paying tribute to tradition, I begin with the classical definition of proba-
bility as the ratio
P=m/n
of the number of favourable out comes m to the total number of out comes n.
This definition reduces the problem of calculating the prob ability to combina-
torial problems.
However, this definition cannot be applied in many practical situations.
This brought to life the so-called statistical definition of probability:
p ~ I'/N, (1)
where N is the total number of trials, which is assumed to be sufficiently

large, and I' is the number of successes. Strictly speaking, this definition is not
mathematical in its initial form. This is why we write the sign of approximate
equality in (1).
* In: Lee tu re Notes in Mathematies, 1021 (1983), pp.I-5.
515
516 ON THE LOGICAL FOUNDATIONS OF PROBABILITY THEORY
The first attempt to make definition (1) more exact was made by R. von
Mises. But before we describe his approach, let us discuss (from the viewpoint
of the dassical definition of prob ability ) why stability of frequencies is so often
observed in natural phenomena.
Consider all 0-1 sequences of length n containing exactIy m ones and as-
sume that all such sequences are equally probable. Suppose that some method
of dividing any sequence of length n into two subsequences has been chosen.
Then for each sequence it is important to compare the frequencies of ones in
both subsequences by calculating the difference
where n1 and n2 are the lengths of the subsequences and 1'1 and 1'2 the number
of ones in them, so that n1 + n2 =
n, 1'1 + 1'2 =
m. We would like to think
that this difference would "almost always be small" in the sense that for any
t>O
Pclass{ll'l
n1
- 1'n22 < t} -+ 1 as n1, n2 -+ 00.
1
Certainly, to make this assertion a theorem, we should narrow the dass

of possible rules for choosing subsequences (for example, forbid the rule ac-
cording to which all zeros are placed into one subsequence and all ones into
the other). All necessary darifications of the not ion of an admissible rule of
choosing subsequences based on von Mises' ideas are given in [3]. The not ion of
an acceptable selection rule is crucial in von Mises' frequency approach to the
notion of prob ability. According to von Mises, an infinite sequence Xl, X 2 , •••
of zeros and ones is called a Bernoulli sequence if:
1) the limit
exists;
2) this limit is preserved under transition to a subsequence chosen with
the help of an admissible rule:
lim .!
m-oo m '"L..J X n J· = P.
iSm
Von Mises gave only general characteristics and several examples of ad-
missible rules. His instructions basically mean that the choke of every next
ON THE LOGICAL FOUNDATIONS OF PROBABILITY THEORY 517
term of the subsequence should not depend on the value of the term but only
on the values of the previous terms. Of course, this definition is not exact, but
no rigorous definition should be expected since the very notion of ''rule'' did
not have a mathematical definition at that time. The situation changed essen-
tially when the not ions of algorithm and recursive function appeared, used by
Church [2] to darify von Mises' definition. In [3] I proposed a broader dass
of admissible selection rules than that given by Church. According to [3], a
selection rule is given by an algorithm (or, if you like, a Turing machine). The
next term of the sequence is chosen as folIows: the input information consists
of a finite sequence of numbers n1, n2, ... ,nj: and values X n1 , X n2 , ••• ,Xnk of
the corresponding terms of the original sequence. The output consists, first,
of the number nH1 of the next considered element of the sequence (it should
not coincide with any of the n1, n2, ... , nj:, on whose order no restrictions are
imposed), and secondly, indications as to whether the element X nk +1 should
be chosen only for inspection or is to be induded in the chosen subsequence.
At the next step the input of the algorithm consists of the longer sequence
n1, n2, ... , nH 1; naturally, the algorithm starts working from the empty input.
In comparison with [2] the dass of admissible selection rules is wider, since
the order of terms in the subsequence need not coincide with their order in
the initial sequence. Another, even more important, difference is in the strictly
finitary character of the entire concept mentioned above and in the quantitative
estimate of stability of the frequencies.
Passing to finite sequences inevitably leads to restrictions on the complex-
ity of the selection algorithm. An exact definition of the complexity of a finite
object and examples of its use in probability theory were proposed in [3], [6].
The results of frequency and complexity approaches are compared in [4].
Now let us return to the original idea according to which "randomness"
consists in the absence of "regularity" and see how the notion of complexity of
a finite object enables us to make this idea precise. The notion of complexity
is dealt with in a number of papers that may be divided into two groups: on
the complexity of calculations and on the complexity of definitions. We will
deal with the second group.
We take the definition of complexity from [6]. We define the conditional
complexity of a constructible object with respect to a certain algorithm A under
the condition that the constructible algorithm Y is known. More precisely,
518 ON THE LOGICAL FOUNDATIONS OF PROBABILITY THEORY
define the eonditional eomplexity KA(XIY) of the objeet X for a known Y as

the length of the shortest programme by me ans of which an algorithm A ean
derive X from Y:
KA(XIY) = min{l(p)IA(p, Y) = X}.
Here l(p) is the length of the sequenee of zeros and ones eneoding the pro-
gramme. There exists an "optimal" algorithm A, that is, one such that for any
algorithm Al there exists a eonstant C such that for all X and Y
If Al and A 2 are optimal algorithms, then the eorresponding eomplexity fune-

tions differ at most by an additive eonstant (independent of X and Y).
We ean now define a random, or more precisely, a .6.-random objeet in a
given finite set M (here .6. is a number). Namely, we say that X E M is a
.6.-random element of M if
where IMI is the number of elements of M. The objeets from M that are
.6.-random for relatively small .6.'s will be ealled random in M. We obtain a
definition of a random finite objeet whieh ean be regarded as definitive.
Taking as M the set D n of aH 0-1 sequenees of length n we eome to the
eondition
It ean be proved that the sequenee satisfying this eondition for suflieiently
small .6. possesses the property of stability of frequeneies when passing to sub-
sequenees. Henee, von Mises' requirements on random sequenees ean be eon-
sidered as a partieular ease of our requirements.
Further results in this direetion may be found in [5], [7]-[12].
References
1. A.N. Kolmogorov, 'Probability theory', In: Mathematics, its contents,

methods and significance, Akad. Nauk. SSSR, Moscow, 1956 (in Russian).
2. A. Church, 'On the concept of a random sequence', Bull. Amer. Math.
Soc. 46:2 (1940), 130-135.
ON THE LOGICAL FOUNDATIONS OF PROBABILITY THEORY 519
3. A.N. Kolmogorov, 'On tables ofrandom numbers', Sankhya, Sero A 25:4,

369-376.
4. A. Shen, 'A frequency approach to the definition of the not ion of random
sequence', In: Semiotics and informatics, VINITI, No.18, Moscow, 1982,
pp. 14-42 (in Russian).
5. D. Loveland, 'A new interpretation of the von Mises concept of random
sequence', Z. Math. Logik und Grundlag. Math. 12:4 (1966), 279-294.
6. A.N. Kolmogorov, 'The logical foundations of information theory and prob-
ability theory', Problemy Peredachi Informatsii 5:3 (1969), 3-7. (Trans-
lated as Problems Inform. 1Tansmission)
7. R.J. Solomonoff, 'A formal theory of inductive inference', Inform. and
Control7:1 (1964), 1-22.
8. P. Martin-Löf, Algorithms and random sequences, Univ. Press, Erlangen,
1966.
9. P. Martin-Löf, 'The definition of random sequences', Inform. and Control
9:6 (1966), 602-619.
10. V.V. Vyugin, 'Algorithmic entropy (complexity) of finite objects and its
applications to the definition of randomness and quantity of information',
In: Semiotics and informatics, VINITI, No.16, Moscow, 1981, pp.14-43 (in
Russian).
11. G. Chaitin, 'On the length of the programs for computing finite binary
sequences', J. ACM 13:4 (1966), 547-569.
12. A.K. Zvonkin and L.A. Levin, 'Complexity of finite objects and substanti-
ation of the notions of information and randomness by means of the theory
of algorithms', Uspekhi Mat. Nauk 25:6 (1970), 85-127 (in Russian).
Comments
ON THE PAPERS ON PROBABILITY THEORY AND

MATHEMATICAL STATISTICS
A.N. Kolmogorov
Here is a classification of my works on probability theory in terms of their

contents and the time when they were written.
I. Limit theorems for independent and weakly dependent random vari-

ables derived by the methods of metric function theory. Papers Nos. 1-6, 8.
In terms of the methods used, these works, dealing with various forms of
the law of large numbers and the law of the iterated logarithm, are adjacent
to the work by me and D.E. Men'shov on metrical function theory. They were
done in close contact with A.Ya. Khinchin. Of the relevant results obtained
by other researchers the first to be mentioned are the necessary and sufficient
conditions of applicability of the strong law of large numbers for sums of inde-
pendent random variables obtained by Yu.V. Prokhorov.
11. Axiomatics and logical foundations of prob ability theory. Paper No. 7.
Paper No. 7, published in 1929, was the first result of my reflections on
the logical structure of probability theory. Here probability theory is presented
as one of the fields of application of general measure theory. But the concept
developed had not yet revealed the set-theoretic implications of conditional
prob ability, a notion that is fundamental in probability theory. Only after this
difficulty was overcome and the theory of distributions in infinite products was
constructed did it become possible to speak of a set-theoretic justification of
probability theory as a whole which was given in my monograph "Fundamen-
tal notions of probability theory" that came out in 1933 in German and in
1936 in Russian. A modernized presentation of these concepts, developed by
A.N. Shiryaev and myself, is given in the second edition of this monograph
(Nauka, Moscow, 1974).
520
PAPERS ON PROBABILITY THEORY AND MATHEMATICAL STATISTICS 521
III. Markov ehains (Markov proeesses with diserete time). Papers

Nos. 21, 23, 36.
The fundamental signifieanee of Markov proeesses was understood in 1906-
1907 by A.A. Markov. From a strictly finite viewpoint the essenee ofthe matter
is eompletely eharacterized already by the example of finite diserete Markov
ehains. For homogeneous ehains with a finite number of states, Markov estab-
lished a fundamental theorem on the existenee of the limit of transition proba-
bilities. A whole range of problems on this was taken up by V.1. Romanovskii
(Tashkent) and later by many researehers of the Moseow probability sehool.
For a homogeneous ehain with a finite number of states the most exhaustive
results were obtained by S. Kh. Sirazhdinov, a student of Romanovskii and Kol-
mogorov, who determined the asymptoties of multidimensional distributions of
the number of distinet states visited in a Markov ehain.
IV. Markov proeesses. Papers Nos. 9, 10, 13, 14, 17, 19,24,39.
In 1929 I foeused attention on the theory of Markov proeesses with eon-
tinuous time. In No. 9 (one-dimensional ease) and in No. 17 (multidimensional
ease) this theory was developed in classieal terms without explieitly using tra-
jectory spaces. The first versions of the modern eoneept of a Markov proeess
with eontinuous time were developed by J. Doob and E.B. Dynkin (see the
eomments by A.D. Ventsel). Papers Nos. 14, 19, 24 (see the eomments by
A.M. Yaglom) deal with various applieations of the diffusion-type theory of
Markov proeesses. Paper No. 39 (see the comments by A.A. Yushkevieh) dis-
eusses the ease of a diserete set of states.
V. Limit theorems on the eonvergenee of Markov ehains to Markov pro-

eesses with eontinuous time. Papers Nos. 9, 12, 16, 41.
,
The first eoneeption of this sort of theorem was presented in 1931 in §12
of article No. 9. Two examples of these theorems are given in papers Nos. 12,
16. Further work in this direetion was done by other researehers (invarianee
prineiple), see the review article No. 41.
VI. Stationary proeesses. Papers Nos. 27, 28, 47, 48, 50, 52.
I beeame interested in the speetral theory of stationary random proeesses
after the appearanee of the work of A.Ya. Khinehin and E.E. Slutskii. A first
idea on the range of problems involved ean be obtained from Paper No. 34. The
hypothesis on representing oseillatory proeesses by Stieltjes integrals (see (4) in
522 COMMENTS
No. 34) was completely confirmed during the following 40 years. Apparently,
even in teaching (in particular, to engineers) it should be more explicitly stated
that the spectral decomposition of a process does not allow, in general, a more
concrete explanation.
VII. Branching processes. Papers Nos. 25, 32, 33, 46.

A number of researchers have studied particular cases of ''branching pro-
cesses" . A review of the early literature can be found in the paper by J. Steffen-
son referred to in my paper No. 25. In his well-known monograph, R.A. Fisher
makes extensive use of the fact that the evolution of the number of genes in a
population obeys the scheme of abranching process when the gene concentra-
tion is low. Fisher proposed to treat this problem using generating functions.
On further development of the theory of branching processes see B.A. Sev-
astyanov's monograph "Branching processes" (Nauka, Moscow, 1971).
VIII. Various applications. Papers Nos. 14, 18, 22, 26, 29, 37.
Paper No. 14 solves a problem posed by S.1. Vavilov. In this paper §§1
and 2 are written by me, and §3 by M.A. Leontovich. As for paper No. 18,
the considered "blow-up" of the empirical correlation coeflicients under a small
number of observations is quite typical in many applied works (see also the
comments by A.M. Yaglom).
IX. Mathematical statistics. Papers Nos. 11, 15, 30, 31, 38, 50.
See the comments by E.V. Khmaladze, E.V. Malyutov and A.N. Shiryaev.
ANALYTICAL METHODS IN PROBABILITY THEORY (No. 9)

(A.D. Ventsel)
The title to this article, which in modern language would rather read "Analyt-
ical methods in the theory of Markov processes" may serve as the title for an
entire branch of this theory. The essence of this branch is that Markov pro-
cesses come into consideration only for the purposes of translating the problem
in question into the language of transition probabilities P( s, Z, t, E) of a Markov
process or other related analytical objects; after that the problem is solved as
a purely analytical one. In paper No. 9 of this volume, random processes as
ensembles of realizations (trajectories) or as objects described by a system of
finite-dimensional distributions are not considered explicitly but only as mo-
tivations of certain definitions and assumptions. Thus, having introduced the
ANALYTICAL METHODS IN PROB ABILITY THEORY 523
decompositions (111), (112), Kolmogorov writes: "Strictly speaking, only in

this case is the process continuous in time". In terms of the modern theory of
Markov processes this could be reformulated as follows: (111) (112) are elose to
the conditions imposed on the transition function necessary for the continuity
of the trajectories of the corresponding Markov process (or, more exactly, for
the existence of modifications of it with continuous trajectories).
The main themes of the paper are: the general not ion of a stochastically
determined (Markov) process, differential characteristics of Markov processes,
and differential equations related to these processes, that is, ergodic properties.
As for the notion of a Markov process, we can say that as there appeared
more and more methods for solving specific problems dealing with the be-
haviour of Markov processes over a time interval, trajectories became more
and more explicitly involved in the schemes under consideration. Thus, in
1933 there appeared papers by Kolmogorov and Leontovich [1] and Andronov,
Vitt and Pontryagin [2], which derived differential equations for the functions
connected with a diffusion process, under the condition that the process will
not leave a certain domain during a given time. General theorems ineluding
not only only transition functions, but also trajectories as explicit objects for
consideration were presented in the books by Doob [3] and Dynkin [4]. Both
theories were based on the notion of conditional probability and conditional
expectation with respect to au-algebra; but in Doob's theory the main object
is a random process on a certain prob ability space satisfying the independence
condition of the past and the future for a fixed present, whereas in Dynkin's
case, it is a whole family of such processes that start at an arbitrary time
in an arbitrary point of space (which is expressed by considering a family of
probability measures p.,:c and prob ability spaces (O,M.,P.,:c)).
At first the analytic trend in the theory of Markov processes was the only
dominating one; then, in the 1950's, direct probabilistic methods started to
be used that dealt with constructing realizations of random processes from
realizations of other simpler processes, as weH as with stochastic equations
and random changes of time. Analytic and direct methods complemented each
other. There appeared the opposite trend, in which the theory of Markov
processes is applied with the purpose of obtaining purely analytical results.
Starting from the 1970's we cannot speak about a pure analytical trend in the
theory of Markov processes, but only of strong prevalence of analytical methods
524 COMMENTS
in certain works.
The central problem in the analytical trend in the theory of Markov pro-
cesses with continuous time is to obtain differential characteristics of processes
and the mechanisms, realizing the connection between them and the processes
themselves. Let us review the development of methods of answering this ques-
tion and some achievements in the analytical trend of the theory of Markov
processes.
In No. 9 differential characteristics are introduced by formulas (50) for
processes with a discrete state space and by formulas (114), (124), (115), (122)
for continuous (diffusion) processes on the realline. The existence ofthe limits
(50), (122), (124) (which are in fact differential characteristics for s = t) is
established under the assumption of differentiability of the transition proba-
bilities Pij(S,t), (or the transition prob ability densities f(s,x,t,y» at s <t
(additionaIly, provided that the determinant (119) does not vanish). In both
cases differential equations are derived: inverse ones by differentiating with
respect to the first time argument s (formulas (57), (125» and direct ones:
by differentiating with respect to the second argument t (formulas (52), (133».
The existence and uniqueness problem is raised for a solution satisfying natural
conditions at s =t and the conditions (1), (3) (particular cases are (40), (41)
and (85), (86» required for the existence of the corresponding process. In the
continuous case it is solved only when the equation reduces to the classical heat
equation. In §19 differential characteristics for purejump Markov processes are
given and a direct differential equation is suggested (175) (as weIl as (176) for
a process with diffusion between the jumps).
The problem of finding transition probability densities for a diffusion pro-
cess in its analytical formulation is the problem of finding the fundamental
solution of a parabolic differential equation. Existence and uniqueness theo-
rems were obtained under general conditions already in the 1930's (FeIler [5]);
they were also obtained for pure jump processes and processes with diffusion
between jumps.
The differential characteristics of Markov processes in No. 9 are introduced
separately for different particular classes of processes; these characteristics are
functions of the corresponding arguments. The next step was to consider a
linear (sometimes unbounded) operator, instead of a set of functions, as a
differential characteristic of a Markov process. This step was taken when the
ANALYTICAL METHODS IN PROBABILITY THEORY 525
theory of sernigroups of linear operators was applied to Markov processes. The

infinitesimal operator of the sernigroup of operators related to the process was
introduced as a characteristic of a Markov process (more precisely and more
narrowly, of its transition function). The theory of semigroups enabled us to
establish, under very weak restrictions, the uniqueness ofthe transition function
corresponding to a given infinitesimal operator and allowed us to find necessary
and sufficient conditions for the existence of such an operator. The existence
problem for differential characteristics was solved: according to the general
theory an infinitesimal operator has a dense domain.
The theory of semigroups is applied directly only to Markov processes
homogeneous in time, that is, processes with transition function that depends
only on the difference between times:
P(s, x, t, E) = Po(t - s, x, E).
In order to be able to apply this theory to non-homogeneous processes, these

processes have to be reduced to homogeneous ones by introducing a wider state
space.
The first applications of semigroup theory related to semigroups of op-
erators connected with transition functions. These operators mapped initial
distributions into distributions at subsequent times (see No. 9 (5)), which is con-
nected with an analogue of the direct Kolmogorov equations (see (52), (133)).
It turned out to be more fruitful, however, to use sernigroups of operators that
h
act on the functions
Tt/(x) = Po(t,x,dY)f(Y),
rather than on the initial distributions. This is connected with an analogue of

the inverse equations (57), (125).
The possibility of considering the values of a process at random times,
such as the moment of the first exit from a set, appeared to be essential for
the connection between Markov processes and their infinitesimal operators, in
particular for deriving the equations for the expectations related to the process.
This was provided by introducing the strong Markov property. Using the strong
Markov property, Dynkin introduced a new type of differential characteristic
for a Markov process, a characteristic operator (see [4]), that is, not a new type
of characteristic (because under natural broad assumptions the characteristic
526 COMMENTS
operator coincides with the infinitesimal one), but rather a new aspect of the
connection between this operator and a Markov process.
New methods made it possible to solve the problem of giving a full de-
scription of all one-dimensional diffusion processes homogeneous in time (that
is, strong Markov processes with continuous trajectories). At the same time
a fruitful setting of new problems was found, in particular, on the behaviour
of a Markov process given inside a domain, after going out onto its boundary
(analytically it reduces to finding boundary values that restriet the given linear
operator to an infinitesimal operator of the semigroup of contracting operators
preserving positivity).
A further development of analytical methods, following the work of Hunt
[7], was based on considering, on the one hand, general non-negative additive
functionals of Markov processes, and on the other hand, from the analytical
viewpoint, excessive functions (non-negative functions superharmonie with re-
spect to a given semigroup of operators). By considering the extremal points of
the set of these functions, ways were found of constructing an ideal boundary of
the domain corresponding to a given Markov process (the Martin boundary).
A new approach to the question of obtaining differential characteristics of
a Markov process was started by Fukusima (see [8]). Instead of an infinites-
imal operator, he considers the corresponding bilinear Dirichlet form as such
a characteristic. However, a complete and closed theory is only obtained for
the case when the semigroup consists of operators symmetrie with respect to
a certain measure (in terms of probability theory, for Markov processes invert-
ible in time). Important results were obtained dealing with possible extensions
concerning the exit of a Markov process given inside a domain (in particular,
Brownian motion with "reflection in the normal" at an arbitrary, non-smooth,
boundary has been considered).
In the 1960-1970's, especially after the works by Stroock and Varadhan [9],
the approach to the connection between linear operators and Markov processes
based on the not ion of martingale and the "martingale problem" became quite
popular. Instead of the strong Markov property, it uses the preservation of the
martingale property for Markov random moments (the strong Markov property
appears to be an automatie consequence of the uniqueness of the solution to the
martingale problem). In papers by Krylov [10] and Stroock and Varadhan [9]
the existence and uniqueness problem for a diffusion process corresponding to
ANALYTICAL METHODS IN PROB ABILITY THEORY 527
given diffusion and transition coefficients posed in No. 9 was solved practically
without any limitations.
Perhaps the analytical trend in the theory of Markov processes will now be
developed on a new, higher level based on considering, as main analytical ob-
jects, distributions in function spaces, rat her than transition probabilities and
related operators. This was started in papers by Stroock and Varadhan, and
others. Analytical methods give especially wide possibilities for establishing
various limit theorems for random processes.
References
1. A.N. Kolmogoroff and M.A. Leontovich, 'Zur Berechnung der mittleren

Brounschen Fläche', Phys. Z. Sow. 4 (1933), 1-13.
2. A. Andronov, A. Witt and L. Pontryagin, 'On statistical considerations of
dynamical systems', ZhETF 3:3 (1933), 165-180 (in Russian).
3. J .L. Doob, Stochastic processes, Chapman and Hall, 1953.
4. E.B. Dynkin, Markov processes, Springer-Verlag, 1965 (translated from the
Russian).
5. W. Feiler, 'On the theory ofstochastic processes (Existence and uniqueness
theorems), Uspekhi Mat. Nauk 5 (1938), 57-96.
6. W. FeIler, 'Parabolic differential equations and corresponding transforma-
tion semigroups', Matematika 1:4 (1957), 105-153.
7. G.A. Hunt, Markoff processes and potentials 1, 2, 3, Ill. J. of Math., 1
(1957),44-93; 316-368; ibid. 2 (1958), 151-213.
8. M. Fukusima, Dirichlet forms and Markov processes, North-Holland, Am-
sterdam, 1980.
9. D. Stroock and S. Varadhan, Multidimensional diffusion processes, Springer-
Verlag, Berlin, 1979.
10. N.V. Krylov, 'On stochastic integral equations', Teor. Veroyatnost. i Pri-
menen. 14:2 (1969), 340-348 (in Russian).
MARKOV PROCESSES WITH A COUNTABLE NUMBER OF STATES
(No. 10)
(B.A. Sevastyanov)
An example is given of the application of the equations for Markov processes

with a countable number of states to one of the original problems in queuing
theory. Namely, the problem of a queue in a service system with n lines under
Poisson flow of demands with exponential service time is studied. This result
has later become classical and is included in textbooks. (See, for example,
W. FeIler, An introduction to probability theory and its applications, Vois. I, 11,
Wiley, 1950.)
HOMOGENEOUS RANDOM PROCESSES (No. 13)

(V.M. Zolotarev)
The paper is divided into two parts (the first ends at formula (7» because of a
formal restriction on the length of a paper in a volume of the journal Atti della
Reale Accademia ....
Abrief history preceding this paper is as folIows. In 1929-1930 several
papers of the well-known Italian mathematician B. de Finetti came out, in
which he initiated the study ofproperties ofrandom processes X(A) that were
later called homogeneous processes with independent increments. It turns out
that
1/;(t, A) = Eexp{itX(A)} = [1/;(t, 1)]".
for any A > O.

In this connection, in his last paper (referred to in the paper by A.N. Kol-
mogorov) de Finetti poses the problem of describing all distributions whose
characteristic functions 1/;(t) are such that for any A > 0, [1/;(t)]" is the charac-
teristic function of a distribution. The class 1!5 of these distributions is called
the class of infinitely divisible laws. In the same paper it is shown that, if prop-
erly understood, 1!5 includes all distributions whose characteristic functions are
of the form
where "'I, (1', c are real constants and F(x) is a distribution function.
528
HOMOGENEOUS RANDOM PROCESSES 529
A.N. Kolmogorov, who was aware of de Finetti's work and maintained a

correspondence with him, became interested in the problem of describing the
dass 18. In No. 13 he described only the part of 18 corresponding to laws with
finite variance. De Finetti's problem was solved completely in 1934 by the
French mathematician P. Levy [1].
Despite its seemingly intermediate character, the result of Kolmogorov
played a fundamental role in the search for a complete description of the dass
18.
First of all the representation (15) obtained by Kolmogorov convinced de
Finetti that 18 could be described in terms of characteristic functions. Equally
important was the fact that the method of proving (15) used by Kolmogorov
appeared to be very promising. In 1937 the Soviet mathematician Khinchin
used this method in a somewhat elaborated form to reproduce the results by
Levy [2]. The advantages of the Kolmogorov-Khinchin method were so obvious
that in modern probability theory courses the canonical Levy-Khinchin repre-
sentation (together with its modification, the Levy representation) is proved
by this very method [3]. It is also effective when describing the analogues of
infinitely divisible distributions in Banach spaces and locally compact groups
[4].
It should also be mentioned that A.N. Kolmogorov was aware ofboth forms
of the canonical representation of characteristic functions of infinitely divisible
laws that are currently in use. Indeed, the form of the representation (15)
corresponds exactly to the Levy-Khinchin canonical representation and Levy's
canonical representation can be derived from it if we use (13), (14) for O(x)
involving Pi(x) and P2 (x). The latter two are, in fact, the spectral functions
in the Levy representation.
The paper by Kolmogorov contains another important result that the au-
thor did not care to emphasize. This is a probabilistic interpretation of the
spectral functions Pi and P2 • Here we should clarify the meaning of the state-
ment that "P2 (x)dA is the prob ability that over time dA a positivejump greater
than x took place". It is understood that dA is small and that P2 ( x )dA repre-
sents this probability with accuracy o(dA).
References
1. P. Levy, 'Sur les integrales dont les elements sont des variables aleatoires
530 COMMENTS
independantes', Ann. Scuola Norm. Super. Pisa, Sero 2 3 (1934), 337-366.

2. A.Ya. Khinchin, 'A new derivation of a formula ofP. Levy', Byull. Moskov.
Gos. Univ. 1:1 (1937) (in Russian).
3. M. Loove, Probability theory, Princeton Univ. Press, 1963.
4. H. Heyer, Probability measures on locally compact groups, Mir, Moscow,
1981.
HOMOGENEOUS MARKOV PROCESSES (No. 39)

(A.A. Yushkevich)
This paper by A.N. Kolmogorov, together with those by Doob and Levy quoted
below, laid the foundations of the theory of Markov processes dealing with
homogeneous processes with a countable number of states. The peculiar effects
taking place in these processes made them aseparate branch of the general
theory. At the same time, studies of the countable case helped to work out
important concepts having a wider scope of application, such as the strong
Markov property and boundary conditions for Markov processes.
The densities a~ of transition from a to ß and the densities aa = -a~
of leaving a, as weH as the direct and inverse systems of differential equations
PI(t) =
P(t)A and PI(t) = AP(t), rar the transition probabilities ~(t) of a
homogeneous Markov process with a finite or countable set of states E were
introduced by Kolmogorov in his fundamental work [1] (No. 9 of the present
volume), in which he assumed differentiability of the ~(t) and, when E is
infinite, certain conditions of uniform convergence (A and pet) denote the
matrices with entries a~ and ~(t), t ~ 0). If Eisfinite, it is easy to show that
aa = =
E ß # a~, and for the initial value P(O) I (I is the identity matrix)
each of the systems of Kolmogorov differential equations has the unique solution
pet) = eAt • After Doeblin [2] established that for finite E differentiability of
the transition probabilities foHows from stochastic continuity of the process,
that is, from the condition P( +0) = I, the case of a finite set of states was
essentially exhausted.
Processes with a countable state space proved to be more difficult. The
analysis of their infinitesimal characteristics, smoothness of transition func-
tions, behaviour of trajectories, possible "pathologies", etc., accounted for a
whole trend notable in its time.
HOMOGENEOUS MARKOV PROCESSES 531
In the papers [3], [4], which came out earlier and were related to his notion
of separable process, Doob studied the transition matrix P(t) based on an
examination of the jumps in the trajectories. Assuming that P(+O) =I (this
condition is considered to hold throughout) Doob took the following steps: 1)
he proved the existence of finite or infinite derivatives a~ of the functions ~(t)
at the origin; 2) he showed that the inverse system holds when there exists a
first jump after 0, that is, when aa =
Eß~a a~ < 00 (we then say that the
matrix A is conservative) and found a similar connection between the direct
system and the last jump before t; and 3) by continuing the process in different
ways after accumulating the jumps, he obtained examples of non-uniqueness of
a process with a given matrix A.
In No. 39 of [5], by making use ofremarkably simple and concise purely al-
gebraic or analytical techniques (in terms of wh ich probabilistic considerations
can be reflected), Kolmogorov 1) proved the existence of finite densities a~ for
ß =t=. Cl'; 2) proved the existence of the densities aa ::; 00; 3) constructed an ex-
ample of a process where al = 00; and 4) constructed an example of a process
where Eß~a a~ < aa < 00 (we will denote these examples by Kl and K2).
Further, Kolmogorov conjectured [5] that the ~(t) are always differentiable at
t > O.
In a paper that came out at the same time, Levy [6], who was interested
primarily in the asymptotics of ~(t) as t -+ 00, classified the processes con-
sidered from the viewpoint of the behaviour of trajectories, treating them on
a less formal level than Doob. In particular, Levy established the effects ob-
served in the examples Kl and K2. Levy suggested calling the states Cl' with
aa < 00 stable, and those with Cl'a = 00 instantaneous ones. The theorem [6]
stating that not all states are instantaneous turned out to be erroneous. Levy
continued his analysis in [7], [8].
Further studies of countable homogeneous Markov processes were consid-
erably influenced by Kolmogorov's works and ideas. At Moscow University this
subject was discussed at the seminars of Dynkin and Kolmogorov. Kolmogorov
suggested the study of the differentiability of ~(t) as the subject for diploma
work to Yushkevich, who managed with the help of trajectory analysis to prove
the continuous differentiability of ~(t) under the assumption that at least one
of the states Cl' or ß is stable, and to construct an example of a process with
infinite second derivative of ~(t) at certain t > 0 (the idea of the example was
532 COMMENTS
suggested by A.N. Kolmogorov). This paper was published six years after [9].
Yushkevich also noticed the gap in the proof of the above-mentioned theorem
of Levy on instantaneous states, which was connected with the absence of a
definite value of the trajectory at the moment of a discontinuity of the second
kind (see [28]).
These questions were discussed at the Mathematical Congress in Ams-
terdam, where Kendall and Reuter [10] gave a detailed analysis of examples
K1 and K2 from the viewpoint of semigroups. Kolmogorov was present at the
congress. From conversations, some participants got the wrong impression that
in [9] an example is constructed that refuted the hypothesis of differentiability
of ~(t) [12]. Soon Austin [11], [12], published a purely analytical proof of
the continuous differentiability of ~(t) for t > 0 in the case of a stable state
fr. Then Chung [13] proved the same for t ~ 0 using a probabilistic method
elose to [9]. Later Austin generalized his proof to the case of a stable state ß
[14]. Improvements of this proof were proposed by Reuter [15], who further
developed the semigroup approach, as weIl as by Jurkat [16] and Chung [17],
[18]. Finally a complete purely analytical proof of Kolmogorov's conjecture was
found by Ornstein [19]. Smith [20] gave a negative ans wer to Chung's quest ion
on whether or not the derivative of p~(t) tends as t ~ 0 to its value -00 at
t = 0 for an instantaneous state fr. On the other hand, Orey [21] showed that
the total prob ability pa(t) of the state fr at time t need not be differentiable for
a suitable initial distribution and time t > O. Subtle questions on the structure
of the functions ~(t) as t ~ 0 were also dealt with by Kendall [22], Blackwell
and Freedman [23] and Reuter [24]. Studies on the dass of possible functions
f(t) = were one of the factors that stimulated Kingman to develop the
p~(t)
theory of regenerative events [25] ..
Soon after No. 39 came out, Kendall partially expanded Kolmogorov's
results on the existence of densities aa and a~ to Markov jump processes with
arbitrary set of states [26] and, by combining the ideas of examples Kl and K2,
constructed an example of a process with al 00 and = af =
0 for ß =1= 1 [27]. A
positive answer to the intriguing question whether there exists a process whose
states are all instantaneous, was given independently by Dobrushin [28] and
FeIler and McKean [29]; other examples were soon suggested by Kendall [30]
and Blackwell [31]. Later Reuter returned to the studies of K1 and showed that
in this example the infinitesimal matrix A uniquely determines the transition
matrix P(t). All matrices A corresponding to processes with only instantaneous

states were described in subsequent works by Williams [32], [33].
The results on differentiability of transition probabilities and trajectory
structures of homogeneous Markov processes with a countable number of states
are given in detail in the book by Chung [17] and Freedman [34], in which also a
detailed description of examples of various "pathologies" can be found. Another
book by Freedman [35] is devoted exclusively to the construction of examples
with instantaneous states by passing to the limit from processes with a finite
number of states.
The range of problems originating in paper No. 39 promoted the develop-
ment of the notions of Markov moment and the strong Markov property. This
property was formulated and proved by Doob [4], Yushkevich [9] and Chung
[36], [13] for various classes of Markov moments that are encountered in ho-
mogeneous Markov processes with a countable number of states. On the other
hand, in the study of Markov processes in an arbitrary metric space by means
of characteristic operators, the strong Markov property was first required by
Dynkin [37], [38] and was introduced by him as a quite plausible assump-
tion [37]. In private conversations with Yushkevich, Kolmogorov accepted the
possibility of Markov processes that are not strongly Markov, and Yushkevich
constructed appropriate examples mentioned in [37] and included in [39]. Then
Dynkin, comparing his results with those of FeIler, which were obtained in a
purely analytical way, conjectured that any FeIler process without discontinu-
ities of the second kind is a strong Markov process [37]. Yushkevich proved
this by replacing the second condition by the assumption of right continuity of
the trajectory [39]. In parallel, the strong Markov property for particular cases
was established by Hunt [40] and Ray [41] and was introduced in its general
form by Blumenthai [42]. In order that a homogeneous Markov process with a
countable set ofstates E have the strong Markov property, the space E must, in
general, be compactified in an appropriate way by introducing fictitious states.
This was done much later by Doob [43].
Kolmogorov's paper No. 39 also became one of the starting points In
the study of boundaries and boundary conditions for Markov processes. If
a matrix A is conservative, then the inverse system of differential equations
P'(t) = AP(t) holds but, in general, its solution is not unique for the ini-
tial value P(O) = I. General results by FeIler imply that the probabilities
534 COMMENTS
of transition during a finite number of jumps form the minimal solution for
this system. The problem arises of constructing possible continuations of the
minimal process beyond an accumulation point of the jumps, preserving the
Markov property and the infinitesimal matrix A, so that the process does not
terminate. After FeUer [44], the preliminary question as to when accumulation
of jumps (in other words, going away to infinity) can take place over finite
time, was studied by Dobrushin for the countable case [45]. The furt her con-
struction requires constructing exit boundaries, entrance boundaries (if any)
and "glue together" the exits to the entrances, or to the initial states by means
of boundary conditions. Without mentioning the vast range of corresponding
research for diffusion-type processes and their generalizations, we merely note
that for a countable state space such a programme was sketched by Dynkin
in his talk at the AU-Union Meeting on Probability Theory in Leningrad in
1955. Apparently, it could only be realized for the case with a finite number of
exits. Without attempting to give a complete review, we mention the relevant
papers by Dobrushin [46], FeIler [47], Reuter [15], [48], Neveu [49], Chung [50],
Williams [51], [52] and Dynkin [53].
References
lichkeitsrechnung', Math. Ann. 104 (1931), 415-457.
2. W. Doeblin, 'Sur l'equation matricieIle A(t+6) = A(t)A(6) et ses applica-
tions aux probabilites en chaine', Bull. Sei. Math. 62 (1938), 21-32; 64
(1940),35-37.
3. J .L. Doob, 'Topics in the theory of Markoff chains', Trans. A mer. M ath.
Soc. 52 (1942), 37-64.
4. J .L. Doob, 'Markoff chains - denumerable case', Trans. Amer. Math. Soc.
58 (1945), 455-473.
5. A.N. Kolmogorov, 'On differentiability of transition probabilities in time-
homogeneous Markov processes with a countable number of states', Uch.
Zap. Moskov. Gos. Univ. Mat. 148:4 (1951), 53-59 (in Russian).
6. P. Levy, 'Systemes markoviens et stationnaires. Cas denombrable', Ann.
Sei. Ecole Norm. Super. 68 (1951),327-381.
7. P. Levy, 'Processus markoviens et stationnaires du cinquieme type (infinite
denombrable des etats possibles, parametre continu), C. R. Acad Sei. Paris
236 (1953), 1630-1632.
8. P. Levy, 'Remarques sur les etats instantanes des processus markoviens et

stationnaires, a une infinite denombrable d'etats possibles', G. R. Aead.
Sei. Paris 264 (1967), 844-848.
9. A.A. Yushkevich, 'On differentiability of transition probabilities of homo-
geneous Markov processes with a countable number of states', Diploma
work, Moskov. Gos. Univ., Ueh. Zap. Moskov. Gos. Univ. Mat.,186:9
(1959), 141-159 (in Russian).
10. D.G. Kendall and G.E.H. Reuter, 'Some pathological Markov processes
with a denumerable infinity of states and the associated semigroups of
operators on 1', In: Proe. Intern. Gongr. Math. Amsterdam 3 (1954), 377-
415.
11. D.G. Austin, 'On the existence ofthe derivative ofMarkofftransition prob-
ability functions', Proe. Nat. Aead. Sei. USA 41 (1955), 224-226.
12. D.G. Austin, 'Some differentiation properties of Markoff transition proba-
bility functions', Proe. Amer. Math. Soe. 1 (1956), 756-761.
13. K.L. Chung, 'Some new developments in the theory of Markov chains',
Trans. Amer. Math. Soe. 81 (1956), 195-210.
14. D.G. Austin, 'Note on differentiating Markoff transition functions with
stable terminal states', Duke Math. J. 25 (1958), 625-629.
15. G.E.H. Reuter, 'Denumerable Markov processes and the associated con-
traction semigroups on I', Acta Math. 91 (1957),1-46.
16. W.B. Jurkat, 'On semigroups of positive matrics. I, 11', Seripta Math. 24
(1959), 123-131; 207-218.
17. K.L. Chung, M arkov ehains with stationary tmnsition probabilities, Springer-
Verlag, Berlin, 1960.
18. K.L. Chung, 'Probabilistic methods in Markov chains', In: Proe. Fourth
Berke/ey Sympos. Math. Statist. Probab. Univ. Calif. Press, Vol. 2, 1961,
pp.35-56.
19. D. Ornstein, 'The differentiability of transition functions', Bull. Amer.
Math. Soe. 66 (1960), 36-39.
20. G. Smith, 'Instantaneous states of Markov processes', Trans. Amer. Math.
Soe. 110 (1964), 185-195.
21. S. Orey, 'Non-differentiability of absolute probabilities of Markov chains',
Quart. J. Math. 13 (1962), 252-254.
22. D.G. Kendall, 'On the behaviour of a standard Markov transition function
536 COMMENTS
near t = 0', Z. Wahrscheinlichkeitstheorie 3 (1965), 276-278.

23. D. Blackwell and D. Freedman, 'On the local behaviour of Markov transi-
tion probabilities', Ann. Math. Statist. 39 (1968), 2123-2127.
24. G.E.H. Reuter, 'Remarks on a Markov chain example of Kolmogorov', Z.
Wahrscheinlichkeitstheorie 13 (1969), 315-320.
25. J .F.C. Kingman, 'The stochastic theory of regenerative events', Z. Wahr-
scheinlichkeitstheorie 2 (1964), 180-224.
26. D.G. Kendall, 'Some analytical properties of continuous stationary Markov
transition functions', Trans. Amer. Math. Soc. 78 (1955), 529-540.
27. D.G. Kendall, 'Some furt her pathological examples in the theory of denu-
merable Markov processes', Quart. J. Math. 7 (1956), 39-56.
28. R.L. Dobrushin, 'An example of a countable homogeneous Markov process
in which all states are instantaneous', Teor. Veroyatnost. i Primenen. 1
(1956),481-485 (in Russian). (Translated as Theory Probab. Appl.)
29. W. FeUer, and H.P. McKean, Jr., 'A diffusion equivalent to a countable
Markov chain', Proc. Nat. Acad. Sei. USA 42 (1956), 351-354.
30. D.G. Kendall, 'A totally unstable denumerable Markov process', Quart. J.
Math. 9 (1958), 149-160.
31. D. Blackwell, 'Another countable Markov process with only instantaneous
states', Ann. Math. Statist. 29 (1958), 313-316.
32. D. Williams, 'The Q-matrix problem for Markov chains', Bull. Amer.
Math. Soc. 81 (1975), 1115-1118.
33. D. Williams, 'The Q-matrix problem. 2: Kolmogorov backward equations',
Lecture Notes in Math. 511 (1976),216-234; 505-520.
34. D. Freedman, Markov chains, Holden-Day, San Francisco, 1971.
35. D. Freedman, Approximating countable Markov chains, San Francisco,
1972.
36. K.L. Chung, 'Foundations of the theory of continuous parameter Markov
chains', In: Proc. Third Berkel~y Sympos. Math. Statist. Probab. Univ.
Calif. Press, Vol 2, 1956, pp. 29-40.
37. E.B. Dynkin, 'Infinitesimal operators of Markov random processes, Dokl.
Akad. Nauk SSSR 105 (1955), 206-209 (in Russian).
38. E.B. Dynkin, 'Infinitesimal operators of Markov processes', Teor. Veroy-
atnost. i Primenen. 1 (1956), 38-60 (in Russian). (Translated as Theory
Probab. Appl.)
39. E.B. Dynkin and A.A. Yushkevich, 'Strict Markov processes', Teor. Veroy-
atnost. i Primenen. 1 (1956), 149-155 (in Russian). (Translated as Theory
Probab. Appl.)
40. G.A. Hunt, 'Some theorems concerning Brownian motion', 1Tans. Amer.
Math. Soc. 81 (1956), 294-319.
41. D .. Ray, 'Stationary Markov processes with continuous paths', 1Tans.
Amer. Math. Soc. 82 (1956), 452-493.
42. R.M. Blumenthal, 'An extended Markov property', 1Tans. Amer. Math.
Soc. 85 (1957), 52-72.
43. J .L. Doob, 'Compactification of the discrete state space of a Markov pro-
cess', Z. Wahrscheinlichkeitstheorie 10 (1968), 236-251.
44. W. FeIler, 'On the integro-differential equations of purely discontinuous
Markoff processes', 1Tans. Amer. Math. Soc. 48 (1940), 488-515; Errata
- Ibid. 58 (1945), 474.
45. R.L. Dobrushin, 'On the regularity conditions for time-homogeneous
Markov processes with a countable number of possible states', Uspekhi
Mat. Nauk 7:6 (1952), 185-191 (in Russian).
46. R.L. Dobrushin, 'Some classes of homogeneous countable Markov pro-
cesses', Teor. Veroyatnost. i Primenen. 2 (1957), 377-380 (in Russian).
47. W. FeIler, 'On boundaries and lateral conditions for the Kolmogoroff dif-
ferential equations', Ann. 0/ Math. 65 (1957), 527-570.
48. G.E.H. Reuter, 'Denumerable Markov processes. 11, 111', J. London Math.
Soc. 34 (1959), 81-91.
49. J. Neveu, 'Sur les etats d 'entree et les etats fictifs d 'un processus de
Markoff', Ann. [nst. H. Poincare 17 (1962), 324-337.
50. K.L. Chung, 'On the boundary theory for Markov chains', Acta Math. 110
(1963), 19-77; 115 (1966), 111-163.
51. D. Williams, 'The process extended to the boundary', Z. Wahrschein-
lichkeitstheorie 2 (1962), 332-339.
52. D. Williams, 'On the construction problem for Markov chains', Z. Wahr-
scheinlichkeitstheorie 3 (1964), 227-246.
53. E.B. Dynkin, 'General boundary conditions for Markov processes with a
countable set of states' , Teor. Veroyatnost. i Primenen. 12 (1967), 222-257
BRANCHING PROCESSES (Nos. 25, 32, 33, 46)
(B.A. Sevast'yanov)
The general notion of and the term "branching random process" itself that
immediately became commonly accepted, were first explicitly coined by A.N.
Kolmogorov at his seminar at Moscow University in 1946-1947. Various prob-
lems relating to simple models of branching processes had been considered pre-
viously. In particular, one ofthese problems was solved by A.N. Kolmogorov in
his earlier work, No. 25. Rowever, it was the publication of papers Nos. 31, 32,
that triggered an intensive development of the theory of branching processes.
By now there are several monographs that deal with branching processes (books
by Rarris [1], Sevast'yanov [2], Athreya and Ney [3], and others). The model
of branching processes described in papers Nos. 31, 32 are Markov branching
processes with several types of particle and with continuous and discrete time.
This model appeared to be very efficient, both in terms of the number of re-
sults obtained and in its possible applications to biology, chemistry, physics and
technology. A special case of this model is a branching process with immigra-
tion, when, in addition to multiplying particles, there also appear immigrating
particles (see, for example, [2]).
Later on, more complex models of branching processes were developed
which took into account the following parameters: dependence ofmultiplication
on the age of particles (the BeIlman-Rarris process), a particle's position in a
certain space, a particle's dependence on energy, random environment, etc.
The asymptotic formulas of the probability of continuing the process ob-
tained in No. 25, as weIl as related limit theorems proved by Yaglom [4], were
later generalized by other authors for other models, including non-Markov ones.
Papers [5]-[7] deal with the convergence of branching processes to diffusion pro-
cesses, which is discussed in the review paper No. 46.
References
1. T.E. Rarris, The theory 0/ branching processes, Springer-Verlag, Berlin,

1963.
2. B.A. Sevast'yanov, Branching processes, Nauka, Moscow, 1971 (in Rus-
sian).
3. K.B. Athreya and P.E. Ney, Branching processes, Springer-Verlag, Berlin-
Reidelberg-New York, 1972.
538
STATIONARY SEQUENCES 539
4. A.M. Yaglom, 'Some limit theorems of the theory of random branching

processes', Dokl. Akad. Nauk SSSR 56:8 (1947), 795-798 (in Russian).
5. M. Jirina, 'On Feller's branching diffusion processes', Casop. Pestov. Mal.
94:1 (1969), 84-90.
6. J. Lamperty, 'The limit of a sequence of branching processes', Z. Wahr-
scheinlichkeitstheorie 7:4 (1967), 271-288.
7. V.P. Chistyakov, 'On the convergence of branching processes to diffusion
processes', Teor. Veroyatnosl. i Primenen. 15:4 (1970), 727-730 (in Rus-
sian). ('franslated as Theory Probab. Appl.)
STATIONARY SEQUENCES (No. 27)

(Yu.A. Rozanov)
Paper No. 27 has by now become classical and opened the main road in the
theory of stationary (and related) random processes, giving a proper setting 1
for prediction problems and solving them for stationary processes with dis-
crete time (stationary sequences). This work is remarkable because of its deep
connection with various questions of approximation theory, spectral theory of
operators in Hilbert space and the theory of analytic functions. The two central
notions in this theory are the regularity of a random process and subordination
of one process to another.
A stationary sequence can be regarded as asequence
z(t) = ei>.f, t = 0,±1, ... ,

in the Hilbert space L2 with measure F(d>') on -11" ~ >. ~ 11". Let H:t:(n) be the
closed linear span of all z(s), s ~ n, and let in(t) be the best approximation
for z(t) by functions from H:t:(n)j regularity me ans that in(t) -+ 0 as n -+ -00.
A well-known result of No. 27 consists in establishing the following regularity
i:
criterion: F(d>') is absolutely continuous and the density f(>') = F(d>')/d>'
satisfies the condition
logf(>')d>' > -00.
1 Prediction problems themselves are considered in another work by A.N. Kol-

mogorov: 'Interpolation and extrapolation of stationary random sequences',
Izv. Akad. Nauk SSSR Se,.. Mat. :; (1941), 3-14 (see No. 28 of this volume).
540 COMMENTS
The general model of a stationary sequence is:
x(t) = Utx(O), t = 0, ±1, ... ,

where Ut = f~7r eiAt E( d>') is a group of unitary operators in Hilbert space. The
fact that y(t), t = 0, ±1, ... is sub ordinate to this x(t) means that y is obtained
from x by a linear transformation
y(t) = Ax(t)
J
with the operator
A= tjJ(>')E(d>')
in H", = H",(oo). What is surprising here is that this linear dependence of

y and x may be expressed in terms of their correlation dependence (the only
previously known fact of this sort was the trivial fact of linear dependence
of random variables x and y with correlation coefficient 1). A description of
the spectral structure of subordinate sequences gives, in particular, a spectral
description of cydic subspaces for the unitary group Ut in the space H",.
Of particular importance here are subordinate sequences y with the non-
anticipating condition
for which conditions under which
have been established. For example, for
in L 2 with F(d>') = d>' the answer is the following: Hy(O) = H",(O) holds if
and only if
where tjJ(>') is a so-called outer function of the well-known analytic dass H 2 =

H",(O). (In No. 27
o
tjJ(>.) = I: c/ce iAk = r(e- iA )
-00
STATIONARY SEQUENCES 541
is determined by the condition r(z) -I 0, Izl < 1.) It should be mentioned

that many questions for the dass H 2 are easily solved within the scheme of
subordinate sequences (and processes ) proposed in No. 27. This is the case
with the well-known question of invariant subspaces in H 2 , namely: Every
subspace invariant with respect to multiplication by e- iA ,
IS given by H = Hy(O) for an appropriate regular subordinate sequence

y(t) = eiAt <p(>..), ")1 2 .
The studies related to No. 27 are reviewed in detail, for example, in the
monographs [1]-[5]. We will mention some of them.
For some time paper No. 27 remained unknown to many researchers who
in the 1940's dealt with dosely related questions, among them Wiener (see his
paper on filtering stationary processes). Here we should also mention Beurling's
problem concerning "translations" in the analytic dass H 2 , etc.
A direct generalization of regularity condition (*) for stationary processes
with continuous time was made in 1949 by Krein in the form
1-00
00 logf(>")d'
1
+ >..2 1\ >
_
00.
Later Krein found the connection between the prediction problem and the
inverse spectral problem for the string equation. A direct generalization of the
regularity criterion for the non-degenerate multidimensional case was given in
i:
1941 by Zasukhin in the form:
logdetf(>")d>" > -00.

542 COMMENTS
The problems of predicting processes with rational spectrum, important in

applications, were considered by Yaglom. In the general case, the regularity
condition is equivalent to factorizability of 1(>'):
I( >.) = (1/2'1I}/>( >. )4>( >.),
where 4>(>') is an operator-valued function from the corresponding analytic dass

H2. The factorization problem of matrix-valued (operator-valued) functions
1(>') was solved by Roz anov , Wiener and Mazani, Matveyev and others. In
particular, Lax showed that in the infinite-dimensional case even a very strong
I:
generalized version of (*) in the form
log/(>')d>' > -cl
(where c> 0 is a constant and I the identity operator) does not guarantee the
required regularity of 1(>'). A general criterion for regularity was proposed by
Rozanov.
References
1. Yu.A. Rozanov, Stationary random proeesses, Fizmatgiz, Moscow, 1963 (in

Russian).
2. C. Goffman, Banach spaees 01 analytic lunetions, Prentice-Hall, 1962.
3. B. Szökefalvi-Nagy and Ch. Foias, Harmonie analysis 01 operators on
Hilbert space, North-Holland, 1970 (translation from the French).
4. Yu.A. Rozanov, Theory 01 rene wal proeesses, Nauka, Moscow, 1974 (in
Russian).
5. H. Dym and H.P. McKean, Gaussian proeesses, lunction theory and inverse
speetral problems, Academic Press, New York, 1976.
STATIONARY PROCESSES (No. 48)

(V.A. Statulyavichus)
The authors solve the problem for a Gaussian stationary process X(t) under
the strong mixing (SM) condition introduced by Rosenblatt [1]. It is proved
that in this case the SM function 0:( r) is equivalent to the maximal correlation
coefficient p(r) (o:(r) ~ p(r) ~ 21r0:(r)) and the representation (4) for p(r) is
STATIONARY PROCESSES 543
found in terms of the speetral density of the proeess !(>'), whieh exists in view
of the SM eondition. Thus, the problem is solved eompletely. For example, in
the ease of integer time the proeess X t possesses the SM property if and only
if
where the supremum is taken over all one-sided trigonometrie polynomials

p(ei~) = E~o eteit~ (m 2: 0) and 11·1100 is the essential supremum norm. This
implies that if !(>') is eontinuous, !(-'lr) = !('Ir) , and bounded everywhere,
exeept perhaps at zero, then Xt has the SM property. Ibragimov [2] obtained
further results on the form of !(>') when X t possesses the SM property. Helson
and Sarason [3] reeonsidered this problem as a quest ion in harmonie analysis.
Generalizations were also obtained for nmctions X t with values in a spaee of
matriees [6]. In the ease of a random walk on a eompact abelian group Rosen-
blatt [4] showed that a stationary proeess has the SM property if and only
if
where T is the transition operator of the random walk. A.N. Kolmogorov also
posed the problem of finding an effeetive eriterion for eomplete regularity of
a Gaussian stationary proeeSSj it was solved by Volkonskü and Rozanov [7].
A student of A.N. Kolmogorov, Leonov [8], studied various SM eonditions of
random proeesses using higher semi-invariants.
References
1. M. Rosenblatt, 'A eentrallimit theorem and a strong mixing eondition',

Proe. Nat. Aead. Sei. USA 42 (1956), 43-47.
2. I.A. Ibragimov, 'On speetral functions of eertain classes of stationary Gaus-
sian proeesses', Dokl. Akad. Nauk SSSR 137 (1956), 1046-1048 (in Rus-
sian).
3. H. Helson and D. Sarason, 'Past and future', Math. Seand. 21 (1967), 5-16.
4. M. Rosenblatt, Markov proeesses. Strueture and asymptotie behaviour,
Springer-Verlag, Berlin-Heidelberg-New York, 1971.
5. Yu.A. Rozanov, Stationary random proeesses, Fizmatgiz, Moseow, 1963 (in
Russian).
544 COMMENTS
6. I.H. More, and L.B. Page, 'The dass W of operator valued weight func-
tions', J. Math. Mech. 19 (1970), 1011-1017.
7. V.N. Volkonskii and Yu.A. Rozanov, 'Some limit theorems for random
functions. I, 11', Teor. Veroyatnost. i Primenen. 4:2 (1959), 186-207; 6:2
(1961),202-214 (in Russian). (Translated as Theorg Probab. Appl.)
8. V.P. Leonov, Some applications 0/ higher semi-invariants to the theorg 0/
stationarg mndom processes, Nauka, Moscow, 1964 (in Russian).
STATISTICS OF PROCESSES (No. 50)

(A.N. Shiryaev)
In this work the estimators jT and WT for the parameter A (damping coef-
ficient) and w (frequency) in a complex stationary Gaussian Markov process
are constructed using the maximum likelihood method. It turned out that the
normalized ratio
WT-W
JJ: (ei + 71;)ds

does not depend on wand A and is exactly normally distributed with parameters
o and 1 (when a = 1). For a proof of this interesting fact (not mentioned in
the paper) see [1] and the monograph [2], Chap. 17, §4.
The paper under review is also interesting because it shows how Kol-
mogorov interprets a physically posed problem (in this case the problem of
studying the movements of the earth's rotation axis) in strict terms of statistics
of random processes and, based on results for the densities of certain measures
with respect to others, finds estimators of the parameters considered.
Nowadays the statistics of diffusion type processes is a big chapter in the
statistics of random processes. A significant part of the monograph [2] deals
with the statistics of these processes. In particular, it considers the problem of
the paper No. 50, as weIl as a number of related problems.
References
1. A.A. Novikov, 'Sequential estimators of the parameters of a diffusion pro-

cess', Teor. Veroyatnost. i Primenen. 16:2 (1971), 394-396 (in Russian).
2. R. Sh. Liptser and A.N. Shiryaev, Statistics 0/ mndom processes, Nauka,
Moscow, 1974 (Translation: Reidel, Dordrecht, to appear).
SPECTRAL THEORY OF STATIONARY PROCESSES (No. 34)
(A.M. Yaglom)
Paper No. 34 by A.N. Kolmogorov published in 1947 is the first popular review
of the spectral theory of stationary random processes, one of the most impor-
tant sections of the mathematical theory of random functions which had been
developed shortly before (with an active contribution by Kolmogorov himself),
but at that time hardly known to anyone outside the narrow cirele of experts.
The first similar review of this theory outside the Soviet Union appeared two
years later (see [1]). The editions in Russian dealing with this theory include
a scientific monograph [2], the rather elementary book [3] and large sections in
many textbooks (see, for example, [4]-[7]).
Kolmogorov starts his review by giving elassical results by Khinchin [1] 1
who was the first to define a stationary random process e(t) and proved that
its correlation function B(T) = E{e(t + T)e(tn can always be represented as
a Fourier-Stieltjes integral (see formula (6) which relates to the somewhat
more general case of a multidimensional process e(t) = {el(t), 6(t), . .. ,e.(tn).
However he paid main attention to the question of substantiating the possibility
of representing the stationary process e(t) as a Fourier-Stieltjes integral of the
form (,.0 and on the physical meaning of such a representation. He remarks
that the spectral representation (4) can be easily derived from the well-known
theorem in functional analysis on the spectral decomposition of one-parameter
groups of unitary operators in Hilbert space, and the first deduction of this
type was given in [6]-[7] in 1940. However, for simplicity he does not stress
the point that these works solved a more general (and more complex) problem
on the spectral representation not only for stationary processes, but for the
wider class of processes with stationary increments.
A random process e(t) is called a process with stationary increments (in
the wide sense) if for any t, T, 8, Tl, T2 the expectations
E{{(t + T) - {(tn = a(T),

(1)
E{[e(t + 8 + Td - {(t + 8)][{(t + T2) - {(tm = 0(8; Tb T2)
exist and do not depend on t. (It is clear that any process stationary in the
wide sense is always a process with stationary increments, but a process with
1 Italicized figures in square bra.ckets indicate the reference in Kolmogorov's paper.

Italicized numbers in parentheses refer to the corresponding formula of No. 34.
545
546 COMMENTS
stationary increments need not be stationary.) The results of the paper [6] 2
imply that any process with stationary increments ~(t) allows a spectral rep-
1:
resentation of the form
~(t) = (ei>.t - l)d~(A) + ~o + 6t, (2)
where ~o,6 are random variables and the random function ~(A) is such that
its increments ~(~~) = ~(A;) - ~(AD and ~(~~) = ~(A~) - ~(An on non-
intersecting intervals ~>. = [A;, A~] and ~~ = [A~, At] are uncorrelated to each
other (that is, = 0), while the real monotone non-decreasing
E{~(~~)~(~~)}
function F(A) of the argument A given by E{I~(~>.W} = F(~>.) satisfies, for
any ( > 0, the condition
(3)
In the particular case of a stationary process ~(t) (considered for discrete time,
in particular, by Kolmogorov's [4], [5]), 6 = 0 and, furthermore, not only
1:
must (3) hold, but also the more restrictive condition
dF(A) < 00. (4)
In this case, dearly f~oo d~(A) = ~~ is a random variable of finite variance;
1:
therefore (2) may be rewritten as
~(t) = ei>'td~l(A)
(where ~l(A) = ~(A) + (~o -~~)K(A) and K(A) is a "jump function" equal to
ofor A < 0 and to 1 for A ~ 0). Applied to stationary processes this brings us
back to (4) and to the results (applied to the multidimensional process ~(t»
of the paper No. 34 by Kolmogorov.
The spectral theory of random processes with stationary increments de-
veloped in [6]-[ 7] can be applied to the more general dass of random processes
~(t) with stationary increments of order n > 1 (see [9], [10], where one can also
2 The results of [6] dealing with geometry of Hilbert space were obtained a little
later also by von Neumann and Schoenberg [8], who noted, however, that the
same results can at the same time be interpreted as certain facts concerning
random processes e(t) with stationary increments.
SPECTRAL THEORY OF STATIONARY PROCESSES 547
find precise definitions of this dass of random processes). For these processes
the spectral representation is
where eO,el,." ,enare random variables, and the increments of the random
function cI(>') on non-intersecting intervals ~~ and ~~ are again uncorrelated
to each other, whereas the function F(>') determined by E{lcI(~A)12} = F(~A)'
satisfies, for any f > 0, the condition
(6)
Formulas describing spectral representations of processes with stationary

increments of a certain order can be derived, for example, from results re-
lated to another generalization of the not ion of a stationary random process,
namely from the spectral theory of generalized stationary random processes.
By analogy with the notion of a distribution or generalized function which
has become widely used in modern analysis, a generalized random process is
a random linear functional e(q;) (that is, a linear functional whose values are
random variables) given on a certain linear space D of "good" (that is, sufli-
ciently smooth) functions q; = q;(t), for example, on the space D oo of infinitely
differentiable functions (introduced by L. Schwartz), each of which vanishes
identically outside some finite interval (only such aspace D = D oo will be con-
sidered in what folIows). Clearly, an ordinary random process e(t) can always
i:
be associated with a generalized process
e(q;) = e(t)q;(t)dt. (7)
From this viewpoint ordinary random processes may be considered as a partic-

ular case of generalized processes. In principle, however, there can also exist a
generalized process e(q;) for which the "values at the point" e(t) do not exist
(that is, such that e(q;) cannot be described by (7)). A typical example ofsuch
a process e(q;) is the "white noise" process, very often encountered in various
applications.
Generalized random processes were introduced in [11], [12] (see also [13],
Chapter 111). In [11]-[13] the notion of a stationary (in the wide sense) general-
e e
ized random process (q;) is defined also and a spectral theory for these (q;) is
548 COMMENTS
developed. To define generalized stationary processes one only has to consider

the operation TT in D given by TTt/J(t) = t/J(t + T) and then require e(t/J) to
satisfy
E{e(TTt/J)} = E{e(t/J)},
(8)
E{e(TTt/Jl)e(TTt/J2)} = E{e(t/Jde(t/J2)}
for any T. For simplicity we suppose that E{e(t/J)} = m(t/J) = 0 for all t/J. In this
case a generalized stationary random process e(t/J) has a spectral representation
ofthe form
(9)
where ~(>.) = f~oo ei>.tt/J(t)dt is the Fourier transform of t/J(t), and the random
function <1>(>') has the same properties as <1>( >.) featuring in the spectral decom-
position (.4) of an ordinary random process e(t), with the only difference that
now F(>.) is such that E{I(Ll A)I2} = F(Ll A ) satisfies the condition
1 00
-00
dF(>')
(1 + >,2)m < 00
(10)
for some integer m ~ 0 (for ordinary stationary processes e(t) we always have
f~oo dF(>') < 00, that is, m = 0). It can easily be seen that in the general case
when the process e(t/J) is ordinary, (that is, it is given by (7)), (9) immediately
implies also the more usual spectral representation of the process e(t) of the
form (4).
Generalized random processes are in certain aspects simpler than ordinary
processes; in particular, whereas an ordinary random process e(t) has deriva-
tive df.(t)fdt = e'(t) only under certain special conditions, a generalized process
e(t/J) is always differentiable, its derivative e'(t/J) being e'(t/J) = -e(t/J') (if a pro-
cess.e(t/J) is given by (7) and e'(t) exists, then clearly e'(t/J) = f~ooe'(t)t/J(t)dt).
Since generalized processes are differentiable, a generalized random process
with stationary increments of given order n can be defined simply as a process
e(t/J) (non-stationary in general) whose nth derivative en(t/J) is a generalized
stationary process. Using this definition, it is easy to derive from (9) and (10)
the general spectral representation of a generalized random process with sta-
tionary increments of order n, which includes (5) and (6) as particular cases
(relating to processes e(t/J) of the form (7)).
SPECTRAL THEORY OF STATIONARY PROCESSES 549
Another generalization of the spectral representation of a stationary ran-

dom process e(t) as a Fourier-Stieltjes integral (4) concerns homogeneous ran-
dom fields in higher-dimensional spaces R!', that is, random functions e(x), x =
(Zl, ... ,zn) E Rn, of many variables related to stationary processes. A random
field e(x), x E Rn, is called homogeneous (in the wide sense) iffor any r ERn,
(so that Ee(x) =

const, Ee(xd(X2) = B(X1 - X2». Such a field allows a
spectral representation of the form
(12)
where <fI(~k) is a random function of an n-dimensional interval ~k in the

space of vectors k, whose values on non-intersecting intervals ~k1 and ~k2
are uncorrelated, and F(~k) = E{I<fI(~kW} is integrable over the whole space
(that is, JRn F(dk) < 00). We note that as far back as 1941 Obukhov [14],
[15] used the spectral decomposition (12) of a homogeneous random field e(x)
in his important work on statistical turbulence theory, referring to the paper
[6] by Kolmogorov. A rigorous proof of this formula and some of its gener-
alizations (concerning, in particular, fields with homogeneous increments and
generalized homogeneous fields) can now be found, for example, in [16] (see
also [13], §III.5). A number offurther examples of "generalized spectral repre-
sentations" of various classes of random functions related to the representation
of stationary random processes e(t) as Fourier-Stieltjes integrals (4) is given in
particular in [17], [18].
Apreeise formulation of Zasukhin 's results [13] mentioned by Kolmogorov
and a rigorous proof of these results can be found in Rozanov's book [2], Chap-
ter 11. For a modern presentation and furt her development of Slutskii's results
[14] see the papers by Moran [19], [20] and Kendall and Stuart [21], §47.15.
References
1. J .L. Doob, 'Time series and harmonie analysis', In: Proc. First Berkeley
Sympos. Math. Statist. Probab., 1949, pp.303-344.
2. Yu.A. Rosanov, Stationary random processes, Fizmatgiz, Moscow, 1963 (in
Russian).
550 COMMENTS
3. A.M. Yaglom, Correlation theory 0/ stationary mndom /unetions, Gidro-

meteoizdat, Leningrad, 1981 (in Russian).
4. J .L. Doob, Stoehastie proeesses, Chapman & Hall, 1953.
5. A.D. Ventsel, Course on the theory 0/ mndom proeesses, Nauka, Moscow,
1975 (in Russian).
6. 1.1. Gikhman and A.V. Skorokhod., Introduetion to the theory 0/ random
proeesses, 2nd ed., Nauka, Moscow, 1977 (in Russian).
7. 1.1. Gikhmann and A.V. Skorokhod, The theory 0/ random proeesses, Vol.
I, Nauka, Moscow, 1971 (in Russian).
8. J. von Neumann and I.J. Schoenberg, 'Fourier integrals and metric geom-
etry', Trans. Amer. Math. Soe. 50:2 (1941), 226-251.
9. A.M. Yaglom, 'Correlation theory ofprocesses with random stationary nth
increments', Mat. Sb. 37:1 (1955), 141-196 (in Russian).
10. M. Pinsker, 'The theory of curves in Hilbert space with stationary nth
increments', Izv. Akad. Nauk SSSR Sero Mat. 19:5 (1955), 319-345 (in
Russian).
11. I.M. Gel'fand, 'Generalized random processes', Dokl. Akad. Nauk SSSR
100:5 (1955), 853-856 (in Russian).
12. K. Ito, 'Stationary random distributions', Mem. Coll. Sei. Univ. Kyoto
(A) 28:3 (1954), 209-223.
13. I.M. Gel'fand and N.Ya. Vilenkin, Generalized /unctions, Vol. 4, Appliea-
tions 0/ harmonie analysis. Rigged Hilbert spaees, Academic Press, New
York-London, 1964.
14. A.M. Obukhov, 'On the energy distribution in the spectrum of a turbulent
flow', Dokl. Akad. Nauk SSSR 32:1 (1941), 22-24 (in Russian).
15. A.M. Obukhov, 'On the energy distribution in the spectrum of a turbu-
lent flow', Izv. Akad. Nauk SSSR, Geography and Geophysies senes 5:4/5
(1941),453-466 (in Russian).
16. A.M. Yaglom, 'Some classes of random fields in n-dimensional space re-
lated to stationary random processes' , Teor. Veroyatnost. i Primenen. 2:3
(1957),292-338 (in Russian). (Translated as Theory Probab. Appl.)
17. K. Karhunen, 'Über lineare Methoden in der Wahrscheinlichkeitsrechnung',
Ann. Aead. Sei. Fenn. A 1:37 (1947), 3-79.
18. A.M. Yaglom, 'Spectral representations for various classes of random func-
tions', In: Proe. Third All-Union Math. Congress, Moseow, Izd. Akad.
SPECTRAL REPRESENTATION OF RANDOM PROCESSES 551
Naull: SSSR, Vol. 1, 1963, pp. 250-273 (in Russian).

19. P.A.P. Moran, 'The spectral theory of discrete stochastic processes', Bio-
metrill:a 36:1 (1949), 63-70.
20. P.A.P. Moran, 'The oscillatory behaviour ofmoving averages', Proc. Gam-
bridge Phil. Soc. 46:2 (1950), 272-281.
21. M.J. Kendall and A. Stuart, The advanced theory 0/ statistics. Design and
analysis and time se ries, Griffin, 1983.
SPECTRAL REPRESENTATION OF RANDOM PROCESSES (Nos. 47, 49)

(Yu.G. Balasanov and LG. Zhurbenko)
Let T(1e) be the dass of random processes {(t) such that
and S(k) the dass of processes {(t) E T<k) such that for all 1 $ I $ 11:, -00 <
T< 00,
Then the example constructed in the commented papers shows that although
S(2) C c)(2), for 11: > 2 there exist processes of dass S(k) that are not of dass
c)(k) •
A suitable definition of the dasses of random processes that have spectral

representation of higher orders was discussed in a number of subsequent works
ofvarious authors. We first mention [1], where, at Kolmogorov's suggestion, the
dasses ,6.(k) are introduced and studied. They are defined as certain subdasses
of the dasses S(k) n c)(k). For these dasses it is more natural to consider the
semi-invariant spectral measure F?)(.~l'.'" Ak) defined by the relation (see
[1], [2], [9])
where
552 COMMENTS
instead of the moment spectral measure MP'\)\l"'" Ak) (defined by the rela-
tion (see [1], [2], [9])
In particular, the classes il(k) are charac1erized by the fact that for e(t) E il(k)
the se mi-invariant measures FP), 1 ~ I ~ k, are absolutely continuous with
=
respect to Lebesgue measure on the sets Al + .. .+A/ 0 (mod 211'). The study
of the classes il(k) was then continued in [2]. A generalization of il(k) for vector
random processes is obtained in [3]. Similar questions for random fields were
considered in [4].
The work initiated by A.N. Kolmogorov in which the classes il(k) were
defined and studied, initiated aseries of studies in a new branch, called the
theory of higher spectra of stationary random processes and their statistical
analysis. The results of this branch are now of fundamental character and are
used extensively for solving various applied problems in astronomy, geophysics,
studies of liquid and gas turbulence, etc. (see, for example, [5]-[8]).
A fundamental contribution to the further development of higher-order
spectral analysis was made in [9] where the mathematical apparatus of higher
moments and semi-invariants, necessary for further research, was developed
(see [10], [11], etc.). Higher-order spectral theory for homogeneous random
fields was considered by Yaglom [12]-[14]. Upper estimates for higher spectral
densities and their derivatives under different mixing conditions can be found
in [11] and are essential in statistical analysis.
Numerous works dealt with constructing and studying higher spectral den-
sities (for example, [15]-[17], etc.). However, until recently, all the statistics
had the same essential drawback: they did not allow one to construct an esti-
mator of the higher semi-invariant spectral density for all arguments. Though
the semi-invariant density of nth order is defined at all points Al + ... + An =
o (mod 211'), the statistics proposed gave no ans wer for the subsets Ak l + ...
... + Ak = 0
p (mod211'), 1 ~ ki ~ n, i =
1, ... ,p, for p < n. Naturally
statistical estimators were ullstable in a neighbourhood of these sets. This
drawback has recently been eliminated by using statistics constructed by the
time translation method [18].
SPECTRAL REPRESENTATION OF RANDOM PROCESSES 553
References
1. A.N. Shiryaev, 'Some questions of the spectral theory of higher moments' I,

Teor. Veroyatnost. i Primenen. 5:3 (1960), 293-313 (in Russian). (Trans-
lated as Theory Probab. Appl.)
2. V.P. Leonov and A.N. Shiryaev, 'Some questions of the spectral theory of
higher moments' 11, Teor. Veroyatnost. i Primenen. 5:4 (1960), 460-464
3. D.R. Brillinger, 'An introduction to polyspectra', Ann. Math. Statist. 36
(1965), 1351-1374.
4. N.N. Leonenko, 'Some questions of the spectral theory of higher moments
of random fields', Dokl. Akad. Nauk Ukrain. SSR 9 (1975), 777-782 (in
Russian).
5. P.I Kuznetsov, R.L. Stratonovich and V.1. Tikhonov, 'Some random func-
tions and their transmission through non-linear systems', Avtomatika i
telemekhanika 14:4 (1963) (in Russian).
6. A.S. Monin and A.M. Yaglom, Statistical hydromechanics Vol. 1, Nauka,
Moscow, 1965 (in Russian).
7. A.S. Monin and A.M. Yaglom, Statistical hydromechanics Vol. 2, Nauka,
8. D.R. Brillinger, 'Identification of polynomial systemS by polyspectra', J.
Sound and Vibr. 2:3 (1970), 301-313.
9. V.P. Leonov and A.N. Shiryaev, 'On a technique of computing semi-invari-
ants', Teor. Veroyatnost. i Primenen. 4:3 (1959), 342-355 (in Russian).
10. V.P. Leonov, Some applications 0/ higher semi-invanants to the theory 0/
stationary random processes, Nauka, Moscow, 1964 (in Russian).
11. I.G. Zhurbenko, Spectral analysis o/time senes, Izdat. Moskov. Gos. Univ.,
1982 (in Russian).
12. A.M. Yaglom, 'Some classes ofrandom fields in n-dimensional space related
to stationary random processes', Teor. Veroyatnost. i Primenen. 2:3 (1957)
292-338 (in Russian). (Translated as Theory Probab. Appl.)
13. A.M. Yaglom, An introduction to the theory 0/ stationary random func-
tions, Prentice-Hall, Englewood Cliffs, 1962.
14. A.M. Yaglom, 'Second-order homogeneous random fields', In: Proc. Fourth
Berkeley Sympos. Math. Statist. Probab., 1961, pp. 593-622.
554 COMMENTS
15. M.A. Rosenblatt and J.S. Van Ness, 'Estimation ofthe bispectrum', Ann.
M ath. Statist. 36 (1965), 1120-1136.
16. D.R. Brillinger and M.A. Rosenblatt, 'Asymptotic theory of estimates of
kth order spectra', In: Advanced Seminar on Spectral Analysis 0/ Time
Sen es, Wiley, New York, 1967, pp. 189-232.
17. I.G. Zhurbenko and N.N. Trush, 'On estimators of spectral densities of
stationary processes', Litov. Mat. Sb. 19:1 (1979), 65-82 (in Russian).
18. I.G. Zhurbenko, 'Statistical estimators of higher spectra', Teor. Veroyat-
nost. i pnmenen. 30:1 (1985),66-77 (in Russian). (Translated as Theory
Probab. Appl.)
BROWNIAN MOTION (Nos. 14, 19, 24)

(A.M. Yaglom)
Papers Nos. 14, 19, 24 are closely related to the important papers Nos. 9
and 17 dealing mainly with the general theory of continuous Markov ("Stochas-
tically determined" in Kolmogorov's terms) random processes. Starting from
the well-known works by Einstein and Smoluchowski [1], [2], processes of this
kind have been widely used in physics to describe Brownian motion of both
individual particles and systems with many degrees of freedom; we recall in
this connection that, as it turned out, the physicists Fokker and Planck, who
studied Brownian motion, came to some of the conclusions given in No. 9 even
earlier. It was therefore natural to expect that the results of No. 17 can also be
directly applied to many problems concerning Brownian motion. Some concrete
examples of these applications are given in Nos. 14, 19, 24.
Paper No. 14, written together with M.A. Leontovich
, and published in a
physics magazine, is more related to physics than the other two. It solves the
following problem in the theory of Brownian motion posed by S.1. Vavilov: to
find the mean E(F) ofthe area F on the plane (x,y) covered by the projection
of a spherical particle of fixed radius u in the process of Brownian motion over
time t. Based on partial differential equations for the transition probability
densities of a multidimensional continuous Markov process found in No. 17, the
authors obtain for E( F) a very precise asymptotic formula (applicable when
Dtfu 2 ~ 1, where D is the corresponding diffusion coefficient, which, due to
Einstein's results [1], [2] for a spherical particle ofradius u immersed in a liquid
BROWNIAN MOTION 555
or gaseous medium of viscosity J.I. is given for the absolute temperature T by

the relation D = koT/61fuJ.I., where k o is the Boltzmann constant).
Note that the solution given in the paper is based on the following impor-
tant result established in the same paper. Let PL(:C, Yj t) denote the prob ability
of a particle located at (:c, y) at time t = 0 and performing Brownian motion
in the two-dimensional plane (the case of Brownian motion in n-dimensional
space can be solved similarly for any n) crossing the boundary r of a given
domain G (containing the point (:c, y» at least once within a given time t, so
that the first crossing will be within a given part L of r (where L can coincide
with r). In this case PL(:c, Yjt) satisfies a certain partial differential equa-
tion (the ''first Kolmogorov equation", see (14», and the following conditions:
PL(:c,YjO) = 0 for any interior point (:c,y) ofG and PL(:c,Yjt) -+ 1 for any
t > 0, when (:c,y) tends to a point of L, L C r, and PL(:C,yjt) -+ 0 for t > 0
and (:c, y) tending to a point of r\L. These equations and conditions uniquely
determine PL(:C, Yj t). Clearly, this general conclusion is of considerable interest
even beyond the specific problem of Brownian motion solved in No. 14. It was
obtained independently at the same time by Pontryagin, Andronov and Vitt
[3]. This and some related results are now widely used in many applications
and are included in a number of textbooks (see, for example, [4] §30, [5] §1.4,
[6] §§26, 27).
A much more general question in the theory of Brownian motion is dis-
cussed by Kolmogorov in the short article No. 19. As is weIl known, in the
classical theory of Brownian motion developed by Einstein and Smoluchowski
[1], [2] the inertia of Brownian motion is neglected, that is, actually a particle is
assumed to have zero mass. Therefore, in Einstein and Smoluchowski's theory
a Brownian particle does not have finite velocitYj thus, for example, for the
particular case of Brownian motion of a free particle, Wiener had rigorously
proved (see [7], Chapter IX) that with probability 1 the Brownian trajectory
is a continuous but nowhere differentiable curve. We note also that in the
Einstein-Smoluchowski approximation, Brownian motion of a physical system
is modelled by a continuous Markov process in its coordinate space (for the
motion of a free particle it is a random Wiener process).
It is clear that the non-differentiability of Brownian trajectories in the
Einstein-Smoluchowski theory is closely related to the idealization introduced
(neglecting inertia), which makes the theory inapplicable for an analysis of the
556 COMMENTS
motion over very small time intervals A.t. For the simplest case of Brownian
motion of a free particle a refined theory taking into account the particle's
inertia was developed in 1930 by Uhlenbeck and Ornstein [8] (see also Doob
[9] and Chandrasekhar [10], Chapter II)j in this refined theory trajectories of
particles are differentiable (but do not have second derivative, so that a par-
ticle's acceleration is infinite). In fact, the same generalization of the classical
theory of Brownian motion is discussed in No. 19 which, however, considers
the general case of Brownian motion with n degrees of freedom, instead of the
particular case of motion of a free particle. According to Kolmogorov, inertia is
taken into account by considering the state of the system as given by the values
of n positions Q1, ••• , qn and their derivatives with respect to time (velocities)
41, ... , 4n. Brownian motion is modelIed here by a continuous Markov process
in the 2n-dimensional phase space of positions and velocities. The main equa-
tion for the transition probability density of this Markov process (equation (9)
of No. 19) turns out in this case to be adegenerate parabolic-type equation
(since its right-hand side contains only second derivatives with respect to ve-
locities, but not with respect to two position coordinates or a position and a
velocity coordinate). Soon after No. 19 was published, Kolmogorov's student,
Piskunov, began a study of the mathematical theory of such equations (11).
The Uhlenbeck-Ornstein theory of one-dimensional Brownian motion of a free
particle is derived from Kolmogorov's general theory for the particular case
n = 1 (so that the basic equation of the theory acquires the form (10), see
No. 19), where f = =
-o:q (and 0: ß/m, where m is the particle's mass, and ß
the viscous friction coefficient equal to 67rui' for a spherical particle of radius
u), and k = const = koT/mß.
Finally, paper No. 24 discusses the interesting question of the meaning
and conditions of statistical reversibility of Brownian motion. It is weIl known
that in the thermodynamical sense processes of Brownian motion (or diffusion)
are irreversible: in the presence of a large number of diffusing particles these
processes always result in a levelling out of the distribution of the particles
available as t increases, whereas the distribution becomes more and more in-
homogeneous as t decreases. Apparently Schrödinger [12] was the first to draw
attention to the fact that an essential role is played he re by the presence of
a well-defined initial condition at t = 0 and the assumption (always made)
that the distribution of particles tends to a homogeneous one (in general, to a
BROWNIAN MOTION 557
certain stationary ergodic distribution) as t - 00, whereas no constraints are

usually imposed on the behaviour ofthe process as t - -00. In this connection
he considered a diffusion process during the time interval to < t < t1 under the
condition that both the initial distribution at t = to and the final distribution at
t = t1 are fixed and pointed out that in this setting, diffusion is to some extent
reversible. In particular, assuming that diffusion takes place in a finite space
interval (or in a finite volume), and the prob ability distribution tends to the
same homogeneous (that is, stationary and ergodic) distribution both as t - 00
and t - -00, then the changes in the probability distribution for to :5 t < 00
and -00 < t :5 to corresponding to a given value of the distribution at the
time to are described by absolutely identical relations, that is, the diffusion is
completely reversible in time.
In his paper Schrödinger considered only the simplest one-dimensional dif-
fusion described by the diffusion equation with constant coefficient D, whereas
in No. 24 Kolmogorov studies the general case of an arbitrary n-dimensional
Brownian motion described by a continuous Markov random process on a cer-
tain n-dimensional manifold R of positions (Xl, ... , X n ) = X of a physical sys-
tem. If for this Markov process the "absolute (unconditional) prob ability den-
sity" p(x) is given for all t, which is the strictly positive density of a stationary
prob ability distribution, (when there exists an ergodic distribution, it is natural
to take it as p(x», then along with the usual prob ability density of the transi-
= =
tion from a given state at t t o to a certain state at t t1 > t o we can also
determine the "inverse transition probability density", that is, the conditional
=
prob ability density for the state at t to, given the state at a later moment
t = t1 > to. After this it is natural to raise the problem of necessary and
sufficient conditions for these two transition probability densities to be equal
to each other. These necessary and sufficient conditions are found in No. 24.
They are derived with the help of an invariant tensor form (derived in the same
paper) of the general equations for the transition prob ability density of the n-
dimensional Markov process given in No. 19. This prob ability density might
be helpful for problems other than that on statistical reversibility considered
here, and the desired conditions themselves appeared to be quite general and
satisfied in the particular case considered by Schrödinger.
We note, however, that in No. 24 the problem of statistical reversibility of
Brownian motion is still not treated in its most general form, since it is assumed
558 COMMENTS
there that the matrix IIBij (z) 11 ofthe coefficients ofthe second derivatives in the
equations for the transition probability density is strictly positive definite for all
z. At the end of §1 the author points out that the case of degenerate IIBij (z)1I is
in fact of considerable physical importance, since it appears, in particular, when
considering Brownian motion in the phase space of positions and velocities of
a physical system (see No. 19). But he immediately adds that the degenerate
case is not considered in this paper. Later Kolmogorov asked a postgraduate
of his, the author of the present comments, the quest ion on the conditions of
statistical reversibility of Brownian motion in the phase space of positions and
velocities. In this case the very definition of statistical reversibility differs from
that given in No. 24, since reversing the motion of a physical system entails
changing the sign of all the velocities (see [13]). However, this difference is
very simple and has little effect. To obtain necessary and sufficient conditions
of statistical reversibility, we must now rewrite the main equations for the
transition probability (equation (9) of No. 19 and the adjoint equation) in
invariant tensor form. (This acquires an especially dear physical meaning when
the coefficients of the metric quadratic form in the position space are taken
to be the coefficients of the quadratic form of the velocities that gives the
system's kinetic energy.) If we now confine ourselves to the most interesting
case, when the coefficients of second derivatives with respect to the velocities
in the equation for the transition probability densities depends only on the
positions and the forces affecting the system in question depend linearly on
the velocities, we can show (see [13]) that for a Brownian motion in the phase
space to be statistically reversible it is only necessary that the corresponding
stationary prob ability distribution have the same form as the canonical Gibbs
distribution. Thus, in this case the statistical reversibility condition has dear
physical meaning. If we then pass to the limit corresponding to letting the
system's inertia te nd to zero (that is, let all coefficients of the quadratic form
of velocities tend to zero, which gives the kinetic energy), then we again come
to the Einstein-Smoluchowski model of Brownian motion as a Markov process
in the space of positions onlYi then the conditions for statistical reversibility of
the Brownian motion in the phase space found in [13] by passing to the limit
again revert to the conditions for statistical reversibility found in No. 24 by
Kolmogorov. The latter circumstance sheds additional light on the physical
meaning of the latter conditions.
MARKOV CHAINS WITH A COUNTABLE NUMBER OF STATES 559
References
1. A. Einstein and M. Smoluehowski, Brownian motion, Colleeted papers.

2. A. Einstein, 'Works on the theory of Brownian motion, In: A. Einstein,
Colleeted scientific papers, Vol. 3, Nauka, Moseow, 1966, pp.75-91j 108-
127j 149-151; 155-163.
3. L.S. Pontryagin, A.A. Andronov and A.A. Vitt, 'On statistical eonsidera-
tion of dynamie systems', ZhETF 3:3 (1933), 165-180 (in Russian).
4. A.A. Sveehnikov, Applied methods 0/ random /unction theory, 2nd ed.,
Nauka, Moseow, 1968 (in Russian).
5. V.I. Tikhonov, Blow-outs 0/ random processes, Nauka, Moseow, 1970 (in
Russian).
6. V.I. Tikhonov and M.A. Mironov, Markov processes, Sovyetskoye Radio,
(in Russian).
7. R. Paley and N. Wiener, 'Fourier transforms in the eomplex domain "
Amer. Math. Soc. (1934).
8. G.E. Uhlenbeck and L.S. Ornstein, 'On the theory of the Brownian mo-
tion', Phys. Rev. 36 (1930), 823-841.
9. J.L. Doob, 'Brownian motion and stoehastie equations', Ann. 01 Math.
43:2 (1942), 351-369.
10. S. Chandrasekar, Stochastic problems in physics and astronomy, Inost. Lit.,
Moseow, 1947.
11. N.S. Piskunov, 'Boundary value problems for an equation of elliptie-para-
boHe type', Mat. Sb. 7 (1940), 385-424 (in Russian).
12. E. Sehrödinger, 'Über die Umkehrung der Naturgesetze', Sitzungsber.
Preuss. Akad. Wiss., Phys.-Math. KI., 12 März (1931), 144-153.
13. A.M. Yaglom, 'On statistical reversibility of Brownian motion', Mat. Sb.
24 (1949), 457-492 (in Russian).
MARKOV CHAINS WITH A COUNTABLE NUMBER OF STATES (No. 23)

(A.A. Yushkevieh)
A.N. Kolmogorov's note [1] and its enlarged version [2] (No. 23 of the present
edition) eontain the foundations of the theory of eountable homogeneous Markov
ehains; these works marked the beginning of the systematie study of Markov
ehains with a eountable and, in general, an infinite set of states. To obtain a
560 COMMENTS
dassification of the states and asymptotics of the transition probabilities Pi)n)

as n -+ 00 is an indispensable part of any detailed presentation of countable
Markov chains [5], [8], [12], [31], [39], [40]. In an adapted form these results are
induded in textbooks for non-mathematicians.
In the papers cited A.N. Kolmogorov proved that 1) a countable space of
states E is subdivided into a set of inessential states and dasses of essential
states (interacting with each other), while the dasses, in their turn, are subdi-
vided into periodic subdasses; 2) within one dass the probability of an infinite
number ofreturns is the same for all states and equals either 1 (recurrent dass),
or 0 (non-recurrent dass); 3) within one dass the mean recurrence time Mjj
is either finite for all the states j (positive dass) or infinite for aU the states
(zero dass); 4) in a non-periodic dass the limit
lim p~~)
n-oo I)
= M-:-.11 1 (1)
exists and the sum of the limits in a positive dass equals 1; 5) therefore, in
general, the dependence of Pi)n) on n is asymptotically periodic.
For a finite E aU dasses are recursive and positive: the division of E into
inessential states, dasses and subdasses and the asymptotic periodicity of the
Pi)n) for this case were independently obtained by Doeblin [3] (it was Markov
who established the existence of the limits (1) independent of i for a finite chain
with strictly positive P;j and consisting, evidently, of one non-periodic dass).
Kolmogorov's work was continued by many researchers. Erdös, FeUer and
PoUard [4], striving to "make Kolmogorov's results more accessible", proved
(1) as a renewal theorem that later acquired a number of generalizations (see
[5], Chap. XIII). FeIler [5] was the first to present the foundations ofthe theory
of countable homogeneous Markov chains for beginners; he pointed out that
in a positive dass the limits (1) form the only stationary distribution. 1 In
[6] he found a number of refinements of (1), for example, an expression for
E: (Pj~) - M j } 1) for a finite second moment MH) of the time of return to j,
and later in [7] he gave a new proof to the renewal theorem.
Sarymsakov [8] reproduced Kolmogorov's dassification and proposed a ma-
trix method for calculating the limits (1); he also considered the centrallimit
theorem for chains with a countable space E and with E = [a, b].
1 FeUer first divided the states into recurrent and non-recurrent ones, and in his
remarks on terminology, erroneously ascribes the same meaning to Kolmogorov's
terms "essential and inessential states".
For the zero dass, when the limits in (1) are 0, it is interesting to compare
the orders of convergence to zero of the transition probabilities for different
states, and also to compare future distributions for various initial states. Doe-
blin [9] established that within one dass the finite non-zero limit
lim
E n1 p~~)
n-oo ~n p(m)
I)
(2)
L..,l k
always exists (Doeblin's result is actually important for the zero recurrent dass,
since for a positive dass it follows from (1), while for a non-recurrent dass it
follows from the convergence of the series obtained by replacing n by 00 in (2)).
Chung [10] found that for a non-recurrent dass the limit (2) is JLjJL/ l where
JLi is the average number of visits to i during one excursion from a fixed state
h to h, and Derman [11] showed that under the same conditions JL = {JLd is a
u-finite invariant measure which is unique to within a constant factor. These
and related results are discussed in detail in the monograph by Chung [12].
Orey [13] showed that in a recurrent dass under different initial states
future distributions converge to each other in variation:
(3)
Another proof was given by Blackwell and Freedman [14] (in a positive dass
(3) it follows directly from Kolmogorov's result (4)).
Kingman and Orey [15] considerably strengthened Doeblin's result on the
limit (2) by proving an "individual" limit theorem for ratios: if in a non-periodic
recurrent dass
N
"p~~) > ( Vi
L.Ju (4)
1
for some N and ( > 0, then

p~~+m)
lim ') = JLj (5)
n-oo p(n) JLI
kl
As was pointed out by Molchanov [16], condition (4) is equivalent to the simpler
condition Pi\M) > 8 for any i. Molchanov generalized (5) to a-recurrent chains,
that is, chains such that for some a ~ 1,
00
",np.(~)
"L.J.... I)
= 00 v., J. ,
w;
n=1
562 COMMENTS
but for each positive ß < a the corresponding series converges; if in this chain
Pi\N) > €Pi\M) for some N > M, € > 0 and all i, then the limit in (5) exists
and equals a-ml';t/JiJi, 1 t/J; 1 , where I' = al'P is (to within a constant factor)
a unique a-harmonic measure, and t/J = aPt/J a unique positive function. The
proof of Molchanov's theorem, which makes use of the Martin boundary, can
be found in [17].
The asymptotics of the transition probabilities Pi; (t) of a homogeneous
Markov process with a countable state space and continuous time was studied
by Levy [18].
For an uncountable phase space E Kolmogorov's sub division of E into the
set of inessential states, classes and subclasses was made by Doeblin [19] and
improved by Doob [20], Chung [21] and Orey [22]. The well-known condition
of Doeblin assumed in these works guarantees the presence of a stationary
distribution and the analogy with the case of positive classes. Adefinition of
probability for a general space E was proposed by Harris [23]. In chains that
are recurrent in Harris's sense there exists a unique CF-finite measure, which
is invariant to within a constant factor. For these chains an analogue of the
result on the existence (at i = k) of the limit (2), equal to 1';1',1, holds (the
Chacon-Ornstein theorem [24], later improved by a number of authors). These
problems are discussed in detail in the monograph by Revuz [25].
Kolmogorov's papers [1], [2] have stimulated not only further studies on
the classification of states and the asymptotics of transition probabilities, but
to a significant extent, also the development of the whole theory of Markov
chains with a countable or arbitrary phase space.
The first of these deals with the recurrent, primarily positive case and con-
sists in extending the known results on sums of independent random variables
or finite chains to additive functionals of the Markov chains considered. This
includes the law of large numbers and related ergodic theorems, the central
limit theorem and its refinements and corresponding asymptotic expansions,
the multidimensional limit theorem for the number of visits to a given set of
states, the law of the iterated logarithm, convergence of normalized increasing
sums to a Wiener process (also called the invariance principle), convergence to
non-Gaussian limit laws, etc. The ergodic theorem for the ratio of two func-
tionals is proved for recurrent chains, while the other results require stronger
assumptions of an ergodic character such as finiteness of the second moments
Mj 7) or the above-mentioned condition of Doeblin. The first significant re-

sults in this field were obtained by Doeblin [19]. Far-reaching refinements of
these results were obtained by Nagaev [26], [27]. Leading roles were played
by Dobrushin [28] and Statulyavichus [29] in applying these results to inho-
mogeneous chains. More detailed references and information can be found in
the book by Chung [12] (a lot of references to papers by Soviet authors were
added in translation), and arecent monograph by Sirazhdinov and Formanov
[30] that contains some old and many new results. The law of the iterated
logarithm, the invariance principle and ergodie theorems for ratios are given in
the book by Freedman [31], specifically for countable Markov chains.
Another trend that originated in the study of non-recurrent Markov chains
(and processes) is Martin's theory ofpotentials and boundaries. The connection
between Dirichlet 's problem for the Laplace equation in a bounded domain G
and Wiener's process in G considered before the time of reaching the boundary
BG of the domain G has been known for a long time. This relationship allows us
to introduce basic notions of classical potential theory in terms of Wiener's pro-
cess or its transition functions. By analogy, the not ions of harmonie function,
potential Green's function and excessive (positive superharmonie) functions are
transferred to other non-recurrent Markov processes. Basic works in developing
the potential theory for Markov processes were done by Hunt [32] and Dynkin
[33]. Potential theory for countable Markov chains is presented in the book by
Henneken and Tortra [34].
The exit boundary of a non-recurrent Markov chain is constructed for de-
scribing the possible behaviour of trajectories as n -+ 00 (with prob ability 1 a
trajectory is attracted to one of the boundary points). The entrance boundary
plays a dual role. FeIler was the first to construct a boundary for a count-
able Markov chain [35]. However, it was another, more meagre boundary that
gained recognition. It was proposed independently by Doob [36] and Watanabe
[37]. It is based on potential theory and is called Martin's boundary (in the
case of Wiener's process in a plane domain G, Doob's construction coincides
with that of Martin [38], which allows us to represent all functions harmonie in
G by a formula similar to Poisson's integral for the disc). The probabilistic con-
struction of the Martin boundary for a non-recurrent countable Markov chain
was made by Hunt [39] and improved by Dynkin [40]. Potential theory and
Martin's boundary for Markov chains with a common phase space is presented
564 COMMENTS
in the book by Revuz [25].

A new method for studying reeurrent and non-reeurrent eountable Markov
ehains, based on the notion of the fundamental matrix, was proposed by Ke-
meny and Snell. These authors (and, independently, Orey) transferred the
theory of potentials and boundaries to recurrent Markov chains. These results,
as weIl as ergodie theorems and Martin boundaries are presented in detail in
the monograph [41] which contains a comprehensive historical review and bib-
liography.
References
1. A.N. Kolmogoroff, 'Anfangsgründe der Theorie der Markoffsehen Ketten

mit unendlich vielen möglichen Zuständen', Mat. Sb. 1:4 (1936), 607-610.
2. A.N. Kolmogorov, 'Markov chains with a countable number of possible
states', Byull. Moskov. Gos. Univ. Math. i Mekh. 1:3 (1937), 1-16 (in
Russian).
3. W. Doeblin, 'Sur les chaines discretes de Markoff', G. R. Acad. Sei. Paris
203:1 (1936), 24-26.
4. P. Erdös, W. FeIler and H. PoIlard, 'A theorem on power series', Bull.
Amer. Math. Soc. 55 (1949), 201-204.
5. W. FeIler, An introduction to probability theory and its applications, Vol.
1, Wiley, New York, 1950.
6. W. FeIler, 'Fluctuation theory of recurrent events', Trans. Amer. Math.
Soc. 67 (1949), 98-119.
7. W. FeIler, 'A simple prooffor renewal theorems', Gomm. Pure Appl. Math
14 (1961), 285-293.
8. T.A. Sarymsakov, Foundations olthe theory 01 Markov processes, Gostekh-
izdat, Moscow, 1954 (in Russian).
9. W. Doeblin, 'Sur deux problemes de Kolmogoroff concernant les chaines
denombrables', Bull. Soc. Math. France 66 (1938), 210-220.
10. K.L. Chung, 'An ergodie theorem for stationary Markov chains with a
countable number of states', In: Proc. Intern. Gongr. Math. Gambridge
(Mass), Vol. 1, 1950, p. 568.
11. C. Derman, 'A solution to a set of fundamental equations in Markov
chains', Proc. Amer. Math. Soc. 5 (1954), 332-334.
12. K.L. Chung, Markov chains with stationary transition probabilities,

Springer-Verlag, Berlin, 1960.
13. S. Orey, 'An ergodie theorem for Markov ehains', Z. Wahrscheinlichkeits-
theorie 1 (1962), 174-176.
14. D. Blaekwell and D. Freedman, 'The tail u-field of a Markov ehain and a
theorem of Orey', Ann. Math. Statist. 35 (1964), 1291-1295.
15. J.F.C. Kingman and S. Orey, 'Ratio limit theorems for Markov ehains',
Proc. Amer. Math. Soc. 15 (1964), 907-910.
16. S.A. Molchanov, 'Limit theorems for the ratio of transition probabilities
of Markov ehains', Uspekhi Mat. Nauk 22:2 (1967), 124-125 (in Russian).
17. S.A. Molchanov, Some problems in the theory of Martin boundaries for
Markov processes, Candidate ofSeienees Dissertation, Moseow State Univ.,
1967 (in Russian).
18. P. Levy, 'Systemes markoviens et stationnaires. Cas denombrable', Ann.
Sci. Ecole Norm. Super. Sero 3 68 (1951), 327-381.
19. W. Doeblin, 'Sur les proprietes asymptotiques de mouvement regis par
eertains types de ehaines simples', Bull. Math. Soc. Roum. 39:1 (1937),
57-115; 39:2, 3-61.
20. J .L. Doob, Stochastic processes, Wiley, New York, 1953.
21. K.L. Chung, 'The general theory of Markov proeesses aeeording to Doe-
blin', Z. Wahrscheinlichkeitstheorie 2 (1964), 230-254.
22. S. Orey, Limit theorems for M arkov chain transition probabilities, London,
1971.
23. T.E. Harris, 'The existenee ofstationary measures for eertain Markov pro-
cesses', In: Proc. Third Berkeley Sympos. Math. Statist. Probab., Vol. 2,
1956, pp. 113-124.
24. R.V. Chaeon and D.S. Ornstein, 'A general ergo die theorem', 111. J. Math.
4 (1960), 153-160.
25. D. Revuz, Markov chains, Amsterdam, 1975.
26. S.V. Nagaev, 'Certain limit theorems for homogeneous Markov ehains',
27. S.V. Nagaev, 'Refinement of limit theorems for homogeneous Markov
ehains', Teor. Veroyatnost. i Primenen. 6:1 (1961), 67-86 (in Russian).
566 COMMENTS
28. R.L. Dobrushin, 'A central limit theorem for non-homogeneous Markov
chains', Teor. Veroyatnosl. i Primenen. 1:1 (1956), 72-89 (in Russian).
29. V.A. Statulyavichus, 'Locallimit theorems and asymptotic expansions for
non-homogeneous Markov chains', Litov. Mal. Sb. 1:1 (1961), 231-314 (in
Russian).
30. S.Kh. Sirazhdinov and Sh.K. Formanov, Limit theorems for sums of ran-
dom vectors connected into a Markov chain, Fan, Tashkent, 1979 (in Rus-
sian).
31. D. Freedman, Markov chains, Holden-Day, San Francisco, 1971.
32. G.A. Hunt, 'Markoff processes and potentials', 111. J. Math. 1 (1957),44-
93; 316-369; 2 (1958), 151-213.
33. E.B. Dynkin, Markov processes, Fizmatgiz, Moscow, 1963 (in Russian).
34. P.L. Henneken and A. Tortra, Probability theory and some of its applica-
tions, Mir, Moscow, 1974.
35. W. FeIler, 'Boundaries induced by non-negative matrices', 7rans. Amer.
Math. Soc. 83 (1956), 19-54.
36. J .L. Doob, 'Discrete potential theory and boundaries', J. M ath. M ech. 8
(1959), 433-458.
37. T. Watanabe, 'On the theory of Martin boundaries induced by countable
Markov processes', Mem. Colloq. Sei. Univ. Kyoto (A) 33 (1960), 39-108.
38. R.S. Martin, 'Minimal positive harmonie functions', Trans. Amer. Math.
Soc. 49 (1941), 137-172.
39. G. A. Hunt, 'Markoff chains and Martin boundaries', Ill. J. Math. 4 (1960),
313-340.
40. E.B. Dynkin, 'The boundary theory of Markov processes (Discrete case)',
Uspekhi Mat. Nauk 24:2 (1969), 3-42 (in Russian). (Translated as Russian
Math. Surveys)
41. J.G. Kemeny, J.L. SneIl and A.W. Knapp, Denumerable Markov chains,
Princeton Univ. Press, 1966; New York, 1976.
42. A.N. Shiryaev, Probability, Nauka, Moscow, 1980 (in Russian).
WALD IDENTITIES (No. 35)
(A.A. Novikov)
The paper commented on here anticipates in many aspects numerous studies of

generalized Wald identities performed later with the help of martingale theory.
In [1],[2] Wald obtained, among other things, the following relation for
the first exit time 1/ of sums of independent identically distributed random
variables elc from a finite interval (a,b). Let en = E~=l elc and
1/ = inf{n ~ 1; en ~ (a, b)}.

Then EI/ < 00 and if Elelc I < 00, then
(1)
M oreover, if Ee~ < 00, then
(2)
These relations enabled hirn to obtain approximate formulas for the aver-
age observation time in the problem of sequential testing of two simple alter-
native hypotheses.
In paper No. 35 the author tried to get rid of the condition of identical
distribution of the random variables elc, which was quite important in Wald's
method [1], [2]. But much more important was the generalization of (1) and
(2) for arbitrary stopping times 1/. Later, results of this type (with arbitrary
stopping times) acquired a fundamental role in various problems of statistical
sequential analysis [3], the theory of controlled random processes [4], boundary
conditions for random processes [5], [6], etc.
Interestingly, the paper [35] did not use the not ion of stopping times (or
Markov times, see [3]) for a sequence of random variables, and Theorems 1
and 2 hold for some non-Markov times. However, in the proof of Theorems 3,
4 and 5 a condition dose to the Markov property of the time 1/ was assumed
implicitly. This was pointed out by Seits and Winkelbauer in [7] (in which also
a refined formulation of Theorems 3, 4 and 5 is given).
When 1/ is a Markov stopping time with respect to the family of u-algebras
Fn = u(6, ... ,en), Fa = {0, n} (that is, the event {I/ = n} E Fn for any n =
1,2, ... ), the results of Kolmogorov's paper have the following generalizations.
567
568 COMMENTS
Let ek be a sequence of random variables with finite expectation ev = L:~=1 ek

and suppose that
(3)
for some a E [1,2]. Then

v
E(ev - L E(ekIFk-d) = O. (4)
k=l
v 2 v
E[ev - LE(ekIFk-d] = E[LE(eiIFk-d]. (5)
k=l k=l
Identity (4) was proved by Burkholder and Gundy [8] under condition (3)
with a = 2; its generalizations for the case 1 :::; a :::; 2 are given in the book
by Chow and Teicher [9]. Similar results for the case of continuous time were
obtained by Novikov [10], [11]. Identities with moments of en of all orders are
considered in the work by Hall [12] and in [10], [11].
Wald identities (1) and (2) for the case ofidentically distributed summands
are trivial consequences of (4) and (5). However, as follows from (3), the first
of these relations holds only if
References
1. A. Wald, 'On cumulative sums of random variables', Ann. Math. Statist.

15 (1944), 285-287.
2. A.Wald, 'Differentiation under the expectation sign in the fundamental
identity of sequential analysis', Ann. Math. Statist. 17 (1946), 493-497.
3. A.N. Shiryaev, Statistical sequential analysis, Nauka, Moscow, 1976 (in
Russian).
4. 1.1. Gikhman and A.V. Skorokhod, Controlled processes, Naukova Dumka,
Kiev, 1977 (in Russian).
5. A.A. Borovkov, Probabilistic processes in queuing theory, Nauka, Moscow,
1972 (in Russian).
S-CONVERGENCE 569
6. A.A. Novikov, 'A martingale approach to problems ofthe first hitting time
on non-linear boundaries', Trudy. Math. Inst. Steklov 158 (1981), 130-152
(in Russian). (Translated as Proc. Steklov Inst. Math.)
7. I. Seits and K. Winkelbauer, 'A note to the article by Kolmogorov and
Prokhorov "On sums of a random number of random summands"', Czecho-
slovak Math. J. 3 (1953), 89-91.
8. n.L. Burkholder and R.F. Gundy, 'Extrapolation and interpolation of
quasilinear operators on martingales', Acta Math. 124 (1970), 249-304.
9. Y.S. Chow and H. Teicher, Probability theory, Springer-Verlag, New York,
1978.
10. A.A. Novikov, 'On the stopping times of a Wiener process', Teor. Veroyat-
nost. i Primenen. 16:3 (1971), 458-465 (in Russian). (Translated as Theory
Probab. Appl.)
11. A.A. Novikov, 'On discontinuous martingales', Teor. Veroyatnost. i Prime-
nen. 20:1 (1976), 13-28 (in Russian). (Translated as Theory Probab. Appl.)
12. W.J. Hall, 'On Wald's equation in continuous time', J. Appl. Probab. 8:1
(1971), 59-68.
S-CONVERGENCE (No. 42)

(A.V. Skorokhod)
In paper No. 42 [1] the following, more convenient metric, equivalent to the
metric S introduced in §2, was proposed:
rD(J, g) = ~EA
inf sup [P(J(t) , g(t)) + '\(t) -
0:5t:51
t],
where J, 9 E D x j p is ametrie in X j A is the set of all continuous invertible

= =
mappings'\ of [0,1] onto itselfsuch that '\(0) 0, '\(1) 1. In this metric Dx
is separable if X is separable, but is not complete.
Billingsley [2] proposed a new kind of metric that, while inducing the same
convergence, turns Dx into a complete space (naturally, if X is complete). It
is defined in the following way:
do(f, g) = inf sup [P(J(t),g(t)) + Iin ,\(t) - ,\(s) I]

~EA 0<t<1 t - S
05-51
(we give a simpler equivalent definition).

570 COMMENTS
This convergence on the space of functions defined on [0,00] without dis-

continuities of the second kind, similar to uniform convergence of continuous
functions on all finite intervals, is generalized in [3].
References
1. 1.1. Gikhman and A.V. Skorokhod, Introduction to the theory 0/ random

processes, Nauka, Moscow, 1965 (in Russian).
2. P. Billingsley, Convergence 0/ probability measures, Wiley, New York, 1968.
3. C. Stone, 'Weak convergence of stochastic processes defined on semi-infinite
time intervals', Proc. Amer. Math. Soc. 14 (1963), 694-696.
UNIFORM LIMIT THEOREMS (Nos. 43, 51)

(T.V. Arak)
The main theorems of paper No. 51 are now known as the Kolmogorov uniform
limit theorems. As is mentioned in the introduction, Theorem 1 (or, equiva-
lently, relation (0.9)) is the third refinement of the corresponding theorem from
No. 43. Later, in 1965 Le Cam [1] showed that in Theorem 1 under suitable
centralization we can take D = exp(n(F - E)), Cl = 132. In 1973 Ibragimov
and Presman [2] gave a new proof with Cl = 8. In parallel with estimates
uniform over the set of all distributions, estimates for various narrower classes
of distributions of summands were studied. In particular, Meshalkin proved
that
sup p(B n , lJ) = O(n- 2 / 3 ),
O~p~l p
where B; is the binomial distribution with parameters n and p.

The final solution was obtained by Arak [3], [4]: There exist positive (ab-
solute) constants C and C such that
(cf. (0.9) and (0.10)).

Using new methods for estimating uniform distances, Zaitsev and Arak [5]
also found a definitive quantitative refinement of Theorem 2.
Higher-dimensional versions of Kolmogorov's uniform limit theorems were
studied by Sazonov, Presman and Zaitsev.
CONCENTRATION FUNCTIONS 571
References
1. 'On the distribution ofsums of independent random variables', In: Bern-

oulli, Bayes, Laplace (anniversary volume), L. le Carn, J. Neyman eds.,
Springer-Verlag, Berlin-Heidelberg-New York, 1965, pp. 179-202.
2. I.A. Ibragimov and E.L. Presman, 'On the rate of attraction of distribu-
tions of sums of independent random variables with accompanying laws',
3. T.V. Arak, 'On the rate of convergence in Kolmogorov's uniform limit
theorem' I, 11, Teor. Veroyatnost. i Primenen. 26:2 (1981), 225-245; 26:3,
4. T.V. Arak, 'A refinement of the lower estimate for the rate of convergence
in Kolmogorov's uniform limit theorem', Teor. Veroyatnost. i Primenen.
27:4 (1982), 767-772 (in Russian). (Translated as Theory Probab. Appl.)
5. T.V. Arak and A.Yu. Zaitsev, 'Estimates of the convergence rate in Kol-
mogorov's second uniform limit theorem', Dokl. Akad. Nauk SSSR 267:1
(1982), 14-18 (in Russian). (Translated as Soviet Math. Dokl.)
CONCENTRATION FUNCTIONS (No. 45)

(V.M. Kruglov)
This remarkable work by A.N. Kolmogorov initiated numerous deep studies on

concentration functions. There is now available a monograph [1] devoted specif-
ically to concentration functions. The chapters on inequalities of Kolmogorov
type constitute the core of this book. It also gives numerous generalizations
and applications of the Levy concentration function and a fairly detailed bib-
liography. In what follows we discuss in detail some results connected with
Kolmogorov's inequality.
The not ion of concentration function was introduced by P. Levy in his
celebrated monograph, published in 1937, "Theorie de l'addition des variables
aIeatoires", referred to in Kolmogorov's work. In the same work Levy gave
a number of very important properties of these functions. It took some time
before the notion of concentration function becarne recognized and sufficiently
known. Apparently, this is the reason why five years later it was introduced
once more as a characteristic of prob ability distributions by Littlewood and
572 COMMENTS
Offord. 1 Studies on refining and generalizing Kolmogorov's inequality were

done in several directions. Two of them were pointed out by Kolmogorov him-
self: one is to obtain an inequality unifying Levy's inequality with a refinement
of it given by Kolmogorovj the other is to analyze concentration functions.
As Kolmogorov says in his paper, his interest in concentration functions and
their properties was aroused by the problem of approximating convolutions of
distributions by infinitely divisible distributions.
The second line of approach relates to a result of Kolmogorov that he did
not draw special attention to and is stilllittle known. In any case, we will state
it.
Let Q be a concentration function. We set
. { Q(x) x }
KQ = mf K: K Q(y) ~ y' y~ x >0 .
If KQ= 1, then Q = pE+(l-p)R, where 0 ~ p ~ 1, Eis the unit distribution

with 0 as support and R is an absolutely continuous distribution.
The first generalizations of Kolmogorov's inequality were obtained by
Rogozin in 1961. In particular, in one of his papers he showed that the con-
dition L 2 ~ Flog s in Kolmogorov's theorem can be omitted. Further gen-
eralizations and refinements of Kolmogorov's inequality were made by Siegel,
Kesten, Le Cam, Miroshnikov, Petrov, Postnikova, Rogozin, Rosen, Sazonov,
Sevast'yanov, Teodorescu, Hengartner, Enger, Esseen and others. The first
generalization of Kolmogorov's inequality was obtained along the lines indi-
cated by Kolmogorov by invoking combinatorial methods. It was followed by a
tendency to use predominantly analytical methods. In the paper by Postnikova
and Yudin published in 1978 one inequality of Kolmogorov type was proved by
analytical methods making use of certain results of additive number theorYj
their inequality can be derived from Kesten's inequalities proved by hirn in
1969 using mainly combinatorial methods.
The latest result of this kind is the inequality by Miroshnikov and Rogozin
[2], [3]. Denote by Q the concentration function of the sum e= 6 + ... + en of
independent random variables, let Qj be the concentration function of ej and
let e?) be the symmetrization of the random variable ej.
1 Here we do not give a reference to their paper and to a number of other pa-
pers mentioned in wh at follows. The interested reader can find them in the
monograph [1].
CONCENTRATION FUNCTIONS 573
There exists an absolute constant C such that

n 1
Q(L) ~ CL (2: E(min{lej3)I, hj })2Qj 2(1j))-2
j=l
for any positive 11 , ... ,In, L, 2L ;::: max Ij.

This inequality was proved by analytical methods. Moreover in the gen-
erality in which it is proved, and without comparing absolute constants, the
Miroshnikov-Rogozin inequality is the strongest one obtained so far.
Of the numerous generalizations of Levy's concentration function we will
give only the one indicated by Enger. Let E be a convex set in Eudidean space
R d. Denote by PE (x) the Minkowski functional determined by E. We call
the integral concentration function of the random vector e, where P is the

distribution of the symmetrization e(3).
Recently Miroshnikov [4] proved an inequality for complete concentration
functions similar to the one given above.
There exists an absolute constant C such that
n 1
U(e, LE) ~ CL (2: E(min{pE(ej')), I })2U- 2(ej, LE))-2
j
j=l
for any positive 11 , ••. , In, L, 2L ;::: max Ij . Here e and ej denote random vectors
with values in R d ; the convex set E belongs to a special dass of sets which, in
particular, indudes the sets satisfying the condition
m
PE(X) = 2: IA;xl, xE R d,
;=1
where Al,'" ,Am are linear operators in R d •

Note that the definition of an integral concentration function is of the same
nature as that ofthe average concentration function ofKawata [5]. As a suitable
generalization from the one-dimensional [1], p.155, to the multidimensional
case, it shows a similarity in nature with the integral concentration function,
regarded as integral transforms with certain kerneis.
574 COMMENTS
References
1. W. Hengartner and R. Teodorescu, Concentration functions, Academic

Press, 1973.
2. A.L. Miroshnikov and B.A. Rogozin, 'Inequalities for concentration func-
tions', Teor. Veroyatnost. i Primenen. 25:1 (1980), 178-183 (in Russian).
3. A.L. Miroshnikov and B.A. Rogozin, 'Remarks on an inequality for concen-
tration functions of sums of independent random variables', Teor. Veroyat-
Probab. Appl.)
4. A.L. Miroshnikov, 'Inequalities for an integral concentration function', In:
Limit theorems for sums of random variables, Trudy Inst. Math. 3 (1984),
159-170 (in Russian).
5. T. Kawata, 'The mean concentration function of a chance variable', Duke
Math. J. 8 (1941), 666-677.
EMPIRICAL DISTRIBUTIONS (No. 15)

(E.V. Khmaladze)
1. This paper is the one in which Kolmogorov's classical statistic
D n = ynsup IFn(t) - F(t)1
t
is introduced and in which its limit distribution, the Kolmogorov distribution,
is established.
By 1933, the empirical distribution function Fn was an attractive object
which had received little mathematical attention. There were no precise state-
ments on the convergence of Fn to the distribution function F (see No. 3),
and there were no correct results concerning the behaviour of the deviations
of F n from F (see No. 2), except for Pearson's theorem of 1900 on the limit
distribution of X2 statistics (see [1]).
2. Let us briefly describe two well-known works by Cramer and von Mises
illustrating the degree of formalization of the statements concerning deviations
of Fn from F obtained by 1933.
In his book [2], published in 1931, von Mises ([10], pp. 316-327) introduced
a statistic that he called the "w 2-statistic":
w~ = j[Fn(t) - F(tWdt.
EMPIRICAL DISTRIBUTIONS 575
Discussing the "possible deviations" of w~ for large n, von Mises writes (see, for
example, p.320 in [2]), that the theoretical interval ofvalues for w~ (theoretische
Wert) is the interval
Cn = (Ew~ - 0.674JOw~, Ew~ + 0.674JOw~),

where 0.674 is the third quartile of the standard normal distribution 4>, that
is, 4>(0.674) - 4>(-0.674) = 0.5. Apparently, this statement must be based
on the idea of asymptotic normal distribution of the random variable (w~ -
Ew;)/JOw; which is clearly wrong.
It is interesting to note, however, that for the uniform distribution function
on [0, 1],
lim P{w~ E Cn}
n ...... oo
= 0.60,
which is not all that far from 0.5.
In Cramer's paper [3] of 1928, where he also proposed considering the
statistic w;
for checking the hypothesis concerning F, there is no question
raised on the limit distribution of the statistic w;. Instead, the following plan
is carried out: assume that F is dose to the normal distribution function 4>1',<1
with mean J.l and variance (1'2, that is, F can be represented by a small but
unknown number of terms of the Charlier series:
How many terms of this sum are essential ? To answer this question, Cramer
suggested that one should consider the integrals
.6.~n = J [Fn(t) - 4>1',<1(t)]2dt,
.6.~n = J [Fn(t) - 4>1',<1(t) - Pl(t)~I',<1(tWdt

etc. (up to the fourth order in Cramer's paper). If .6.in decreases "strongly" as
compared to .6.Ll,n, then the ith term in the decomposition of Fis "essential".
This technique is applied in the paper to many specific data, with parameters
J.l and (1'2 being estimated by the same data.
3. It follows from Theorem 1 of No. 15 that sup IFn(t) - F(t)/ --+ 0 in
t
probability as n --+ 00. A stronger statement on the convergence of sup IFn(t)-
t
576 COMMENTS
F(t)1 to zero with probability 1 as n -+ 00 (Glivenko's theorem) was published

by Glivenko simultaneously with Kolmogorov's paper in the same issue of the
journal (see [4]).
As regards the connection between Kolmogorov's theorem and Glivenko's
theorem, it is interesting to note that 11 years later, in 1944, Smirnov proved
in the fundamental work [6] that Kolmogorov's statistic satisfies the inequality
for large deviations
which implies Glivenko's theorem.

4. Apparently, the first table of values of the Kolmogorov distribution
function was published by Kolmogorov himself.
In 1939 Smirnov published six-digit tables of the Kolmogorov distribution
function as an appendix to his paper [7] (see also [5], p. 117). Later these
tables were published separately in 1948 in [8] and then reproduced in [9].
Very detailed seven-digit tables of the Kolmogorov distribution function were
computed in 1965 and published on pages 267-277 in [5]. The step in these
tables is fixed and equals 0.001 and the greatest argument value is 3.000. For
this argument the Kolmogorov distribution function equals 0.9999999695.
5. A remarkable lemma given in the paper, stating that the distribution
of the statistic D n does not depend on Fn , directed the further development
of the theory of non-parametric goodness-of-fit tests in accordance with the
following principle: only functionals of Fn and F whose distribution does not
depend on F should be chosen as statistics of goodness-of-fit tests.
Here it is worth recalling the following: in 1900 in constructing the X2
statistic,
Pearson [1] had chosen the coefficients 1/ilF(ti) so that the limit distribution
ofthis quadratic form does not depend on the probabilities ilF(ti) = F(ti+1)-
F(ti) and consequently, on the division points ti and the distribution function
F. Subsequently many papers appeared that established that these statistics
were asymptotically normal. Clearly it suffices to normalize these statistics to
EMPffiICAL DISTRIBUTIONS 577
obtain random variables with the standard limit distribution. In 1931, perhaps
under the influence of this trend, von Mises [2] considered the statistics
and discussed only two weight functions A : A1(t) = n/f(t) where f is the
density of a distribution function F and A2(t) = l/E[J[Fn (t) - F(t)]2dt]. The
choice of Al was regarded as being analogous to the X2 statistic but unsuitable,
since the integral in W~(A1) "usually" diverges, whereas A2 was considered suit-
able, since it standardized the means of the w 2 statistics. The limit distribution
of the W~(A2), however, clearly depends on F.
Then, in 1933, the above-mentioned lemma was formulated, and in 1937
Smirnov in [10], with reference to Glivenko, finally introduced the statistics
W~(A), where A(t) = ~[F(t)]f(t), with distribution independent of F. Since
then, or perhaps since 1952, after the papers by Anderson and Darling [11],
this principle became standard in constructing non-parametric tests.
When a statistical hypothesis fixes a certain family F of distribution func-
tions instead of a specific distribution function F, the realization of this prin-
ciple becomes somewhat diflicult. Abrief description of papers dealing with
this problem and a very incomplete bibliography, to which Gikhman's paper
[12] should be added, can be found, for example, in [13], §2.1. In the same §2 it
is described how to construct functionals of F n and F whose limit distribution
does not depend on F.
When F is a continuous distribution in the multi dimensional space R m ,
m> 1, the lemma does not hold: it is false that the random variable F(X1 , ••.
. . . , X m ) has the same distribution for all F if the random vector (Xl, ... , X m )
has continuous distribution function F. Therefore there seems to be no stan-
dard way for constructing non-parametric goodness-of-fit tests for distributions
in multidimensional space, unlike the one-dimensional case. Still, we point out
two modern [14], [15] and one old [16] paper on this subject.
6. The convolution (16) and the recurrence formula (9) of paper No. 15
were later extensively used for calculations for finite n. In particular, in 1950
Massey [17] was the first to use (16) to calculate small tables for the distribution
function of the Kolmogorov statistic for 5 < n < 80. Quite recently (see [18])
a recurrence formula similar to (9) was used for calculating the probability of
F n remaining within domains with various curvilinear boundaries.
578 COMMENTS
Speaking of tabulating cI>n, we note that modern tables usually contain

the table of values of distribution functions cI>n and the percentage points for
n ~ 100. The known asymptotic formulas, however, fit even for n 2: 30. This
side of the matter is discussed in the explanation part of [9], where appropriate
references are also given.
7. We introduce the so-called homogeneous empirical process V n , vn(t) =
yn[Fn(t)-t] and Brownian bridge v, v(t) = w(t)-tw(1), where w is a Wiener
process, t E [0,1], and we consider the following functional of the empirical
process:
Let
Pn(A, h) = p{Vt E [0,1] : IFn(t) - tl ~ )nh(t)},
cI>n,K(A, K) = P{K(vn ) ~ A}.
Clearly, Pn(A, h) = cI>n,K(A, K), that is, the calculation of the probability that
the empirical process remains within a certain boundary is equivalent to cal-
culating the distribution functions for the functionals K(v n ). These problems
were solved in different ways, however.
The first problem, namely, that of studying the prob ability of remaining
within a certain domain, is reviewed in the well-known papers by Gikhman and
Gnedenko [19] and Borovkov and Korolyuk [20]. In particular, they describe
Gikhman's work on limit theorems for the probability of remaining within the
boundary for processes converging to diffusion processes (these works are very
elose in their ideas to Kolmogorov's paper) and the papers by Gnedenko and
Korolyuk that make use of random walk methods.
We recall the development of the second problem. In No. 15 the limit for
the distribution function of the Kolmogorov statistic was established: cI>nK ~ cI>
for h = 1. However, the probabilistic "construction" of Kolmogorov distribu-
tion function cI>, namely the equality
cI>(A) = P{K(v) < A}, (1)
remained unelear for a long time. In any case, it remained unelear of what use
such a "construction" could be.
With time other statistics were discovered which, as was noticed later,
can be conveniently represented as functionals of an empirical process. In
EMPmICAL DISTRIBUTIONS 579
particular, in 1939, Wald and Wolfowitz [22] considered the so-called weighted
Kolmogorov statistic, that is, the functional K(v n ) with weight function h '# 1,
while in 1937 Smirnov [10] considered the ",2 statistic, that is, the functional
",2(vn ) = 1
1
[vn (t)]2dt. (2)
It was natural to single out what the convergence of the distributions of various
functionals of V n had in common, namely, convergence of V n to v in distribution.
The discovery of the fact that it is natural to represent various statistics
as functionals of an empirical process was only one important feature of the
matter. Another was to foresee that general efficient methods could be worked
out for proving convergence in distribution of processes V n to a process v and,
in general, for proving weak convergence in function spaces.
Both methods were ripe in the late 1940's: on 30 November 1948 (as re-
ported in 'Uspekhi Matematicheskikh Nauk" (Russian Mathematical Surveys),
4:2 (1949), p. 173), A.N. Kolmogorov gave a lecture for the Moscow Mathe-
matical Society caHed "Measures and distributions in function spaces", which
discussed, among other matters, the problems of determining weak convergence
of measures in function spaces. On 29 January 1949, Smirnov gave a talk on
the "Cramer-von Mises test" [23] (see also [5], p.200), where again, after an
interval of 12 years, he obtained the limit distribution of ",2 statistics, this time
using the representation (2) and the corresponding Parseval identity. Finally,
in September 1949, Doob's well-known paper [24] appeared. Together with the
description of the general viewpoint, to the effect that the convergence ~n => ~
is a consequence of the convergence of V n to v in distribution, he also proved
(1).
This started the useful development of the theory of weak convergence in
function spaces that presently constitutes the basis for the limit theorems in
non-parametric statistics.
I recommend the small monograph by Durbin [25] and the review article
by Gaensaler and Stute [26] for an acquaintance with the state of affairs in this
field.
8. Let us make several general conduding remarks.
A. Kolmogorov's statistic, as weH as the statistic K(vn ) (see No. 7),
stands apart from the other statistics of non-parametric goodness-of-fit tests in
that it leads to confidence sets that are easy to visualize. Thus, for example,
580 COMMENTS
the confidence set {F' : n J[Fn(t) - F'(t)j2dF'(t) < A} constructed by means

of the statistic w 2 is not graphie and therefore inconvenient. By contrast, the
confidence set
{F' : Vnsup IFn(t) - F'(t)1 < A} =

= {F': Fn(t) - Jn < F'(t) < Fn(t) + Jn}
constructed with the help of Kolmogorov's statistic is easy to perceive and is
widely used. This property of Kolmogorov's statistic was observed very early:
back in 1939 Wald and Wolfowitz [22] introduced the confidence sets
relating to weighted Kolmogorov statistics.

B. In the same year 1939, two papers by Smirnov [27], [7] appeared. They
gave distributions for the statistics D;t = Vnsup[Fn(t) - F(t)] and D n1n2 =
y'nsup IF1n1 (t) - F2n2 (t)l, n = nln2j(nl + n2), namely,
P{D;t < A} ---+ 1 - e- 2A2 , P{Dn1n2 < A} ---+ C)(A). (3)
Here Fl n1 and F2n2 are the empirical distribution functions constructed from
two independent sequences of independent identically distributed random vari-
ables.
It is interesting that both statements (3), wh ich are the best known state-
ments in non-parametric statistics, were given as simple consequences of general
theorems on the number of intersections of the curves Fn and F + Ajy'n and
the curves F1n1 and F2n2 ± Ajy'n respectively.
In 1944 Smirnov [6] gave the distribution of the statistic D;t for finite n.
In 1955 Korolyuk [28] obtained the distribution of the statistic D n1n2 for finite
nl and n2 and in 1962 Borovkov [29] established the asymptotic expansions for
the distributions and, in particular, for the probabilities of large deviations of

D n1n2 . The specific use of certain results from [29] and [21] was demonstrated
in [9], p. 84.
Apparently, Massey [30] was the first to give a table for the distribution
of the two-sample Smirnov statistie for finite nl and n2. Tables of percentage
points of D n1n2 for nl < n2 :::; 50 are published by Borovkov, Markov and Sy-
chev [31]. A remarkable example of the application of the Kolmogorov statistic
EMPffiICAL DISTRIBUTIONS 581
to the verification of the hypothesis on the distribution function F was given

in 1940 by Kolmogorov himself [32].
References
1. K. Pearson, 'On the criterion that a given system of deviations from the
probable in the case of a correlated system of variables is such that it can
be reasonably supposed to have arisen from random sampling', Phil. Mag.
5:50 (1900), 157.
2. R. von Mises, Warscheinlichkeitsrechnung und ihre Anwendungen in der
Statistik und theoretischen Physik, Franz Deuticke, Leipzig, Vienna, 1931.
3. H. Cramer, 'On the composition of elementary errors. Second paper: sta-
tistical applications', Skand. Aktuarietidskr. 1/2 (1928), 141-180.
4. V. Glivenko, 'Sulla determinazione empirica delle leggi di probabilita' , G.
Ist. Ital. Attuar. 4:1 (1933), 92-99.
5. N.V. Smirnov, Probability theory and mathematical statistics: Collected
works, Nauka, Moscow, 1970 (in Russian).
6. N.V. Smirnov, 'Approximation of distribution laws ofrandom variables by
empirical data', Uspekhi Mat. Nauk No.lO (1944), 179-206 (in Russian).
7. N. V. Smirnov, 'An estimate of the discrepancy between empirical distribu-
tion curves in two independent sampies', Byull. Moskov. Gos. Univ. Math.
and Mech. 2 (1939), 3-14 (in Russian).
8. N.V. Smirnov, 'Tables for estimating the goodness of fit of empirical dis-
tribution', Ann. Math. Statist. 19:2 (1948), 279-281.
9. L.N. Bolshev and N.V. Smirnov, Tables ofmathematical statistics, Nauka,
10. N.V. Smirnov, 'On the distribution of von Mises' w2 -test', Mat. Sb. 2:1
(1937), 973-993 (in Russian).
11. T.W. Anderson and D.A. Darling, 'Asymptotic theory of certain "goodness
of fit" criteria based on a stochastic process', Ann. Math. Statist. 23:1
(1952), 193.
12. 1.1. Gikhman, 'Remarks on Kolmogorov's goodness offit test', Dokl. Akad.
Nauk SSSR 91:4 (1953), 715-718 (in Russian).
13. E.V. Khmaladze, 'Certain applications of martingale theory in statistics',
Uspekhi Mat. Nauk 37:6 (1983), 194-212 (in Russian). (Translated as Rus-
sian Math. Surveys)
582 COMMENTS
14. P.J. Bickel and L. Breiman, 'Sums of functions of nearest neighbour dis-
tances, moments bounds, limit theorems and a goodness of fit test', Ann.
Probab. 11 (1983), 185-214.
15. M.F. Schilling, 'Goodness of fit testing in R m based on the weighted em-
pirical distribution of certain nearest neighbour statistics', Ann. Statist.
11:1 (1983), 1-12.
16. M. Rosenblatt, 'Remarks on multivariate transformations', Ann. Math.
Statist. 23:3 (1952), 470-472.
17. F.J. Massey, Jr., 'A note on the estimation of a distribution by confidence
limits', Ann. Math. Statist. 21:1 (1950), 116-118.
18. V.F. Kotel'nikova and E.V. Khmaladze, 'Calculating the prob ability of an
empirical process remaining within a curvilinear boundary, Teor. Veroyat-
Probab. Appl.)
19. 1.1. Gikhman and B.V. Gnedenko, 'Mathematical statistics', In: Mathemat-
ics in the USSR during jorty years: 1917-1957, Vol. 1, Fizmatgiz, Moscow,
1959 (in Russian).
20. A.A. Borovkov and V.S. Korolyuk, 'On results of asymptotic analysis con-
cerning boundary problems', Teor. Veroyatnost. i Primenen. 10:2 (1965),
21. V.S. Korolyuk, 'Asymptotic analysis of the distribution of maximal devi-
ations in a Bernoulli scheme', Teor. Veroyatnost. i Primenen. 4:4 (1959),
22. A. Wald and J. Wolfowitz, 'Confidence limits for continuous distribution
functions', Ann. Math. Statist. 10:2 (1939), 105-118.
23. N.V. Smirnov, 'On the Cramer-von Mises test', Uspekhi Mat. Nauk 4:4
(1949),196-197 (in Russian).
24. J. Doob, 'Heuristic approach to the Kolmogorov-Smirnov theorems', Ann.
Math. Statist. 20:3 (1949), 393-403.
25. J. Durbin, 'Distribution theory for tests based on the sampIe distribution
function', Regional Gon! Sero in Appl. Math., 9th issue, SIAM, 1973.
26. P. Gaensaler and W. Stute, 'Empirical processes: A survey of results for
independent and identically distributed random variables', Ann. Probab.
7:2 (1979), 193-243.
THE METHOD OF LEAST SQUARES 583
27. N.V. Smirnov, 'On the deviations ofthe empirical distribution curve', Mat.
Sb. 6 (48):1 (1939),3-24 (in Russian).
28. V.S. Korolyuk, 'On the divergence of empirical distributions for the case of
two independent sampIes', Izv. Akad. Nauk SSSR Sero Mat. 19:1 (1955),
81-96 (in Russian).
29. A.A. Borovkov, 'On the problem of two sampIes', Izv. Akad. Nauk SSSR
Sero Mat. 26:4 (1962), 605-624 (in Russian).
30. F.J. Massey, Jr., 'Distribution tables for the deviation between two sampIe
cumulatives', Ann. Math. Statist. 23:3 (1952),435-441.
31. A.A. Borovkov, N.P. Markov and N.M. Sycheva, Tables for N. V. Smirnov's
test of uniformity of two samples, Izdat. Sibirsk. Otdel. Akad. Nauk SSSR,
Novosibirsk, 1964 (in Russian).
32. A.N. Kolmogorov, 'On a new proof of Mendel's laws', Dokl. Akad. Nauk
SSSR 27 (1940), 38-42 (in Russian).
THE METHOD OF LEAST SQUARES (Nos. 30, 31)

(M.B. Malyutov)
The method of least squares is one of the most popular statistical methods
for parameter estimation in applications. Suffice it to say that specialists in
geodesy, navigation, artillery, etc., even nowadays write their own textbooks
on this method (which do not always meet the standards of modern statistics).
At the same time, before No. 30 was published, the mathematical presen-
tations of this method were not based on modern geometrical ideas of higher-
dimensional geometry and were essentially no different from the initial method
of Gauss. It is in No. 30 that the connection of the method with orthogonal pro-
jection onto a subspace in RN was explicitly used and it was discovered which
properties of the method depend on the assumption of normality of the distri-
bution of measurements and which are true for orthogonal observations. For
applications it is important to present in detail the confidence intervals for the
parameters and for the unknown variance of measurements and to supplement
them with the tables of Student and x2-distributions. .
Paper No. 30 played an important role in developing the mathematical
theory of the method of least squares in the USSR and putting into some
order the applications of this method. In particular, its influence can be seen
in the books by Linnik [1] and Romanovskii [2]. The latter, for instance,
584 COMMENTS
reproduces with a detailed proof Kolmogorov's report "The real meaning of

results of the analysis of variance" (Proceedings of the 2nd All-Union Meeting
in Mathematical Statistics), which generalized the results of No. 30 to the
theory of the analysis of variance and criticized certain not sufficiently rigorous
formulations by Fisher .
Paper No. 30 remained unnoticed abroad and the modern presentation of
the method of least squares was published only in 1959 by Scheffe [3].
Paper No. 31 clarifies a formula of Gauss for the method of least squares.
Gauss found a formula for the variance Ds 2 of the standard estimator s2 of the
variance tr2 for independent measurements Yi in the linear regression model.
This formula includes the fourth moment Itr 4 , and therefore Gauss obtained
upper and lower bounds for Ds 2 that hold for any I. However, the lower bound
was rough (sometimes even negative). In No. 31 a best possible lower bound
for D s 2 is found.
The presentation is essentially based on the geometric theory of the method
ofleast squares, developed by A.N. Kolmogorov previously in No. 30 (for exam-
pIe, an important role is played by the orthogonality of certain matrices). With
small modifications this work is reproduced in the book by Linnik mentioned
above. In the subsequent literature main emphasis is laid on constructing
asymptotic confidence intervals for tr 2 , where instead of 1 an estimator of it
with respect to the same sampIe is used [4].
References
1. Yu.V. Linnik, The method 0/ least squares and the /oundations 0/ the
mathematical and statistical theory 01 observation processing, Fizmatgiz,
2. V.1. Romanovskii, Mathematical statistics, Vol. 2, Izdat. Akad. Nauk
UzSSR,1963.
3. H. Scheffe, Analysis 01 variance, Wiley, New York, 1959.
4. H. Drügas, 'Asymptotic confidence intervals for the variance in the lin-
ear regression model', In: Statistics and probability: Essays in honour 0/
G.R. Rao, North-Holland, Amsterdam, 1982, pp.233-239.
UNBIASED ESTIMATORS (No. 38)
(Yu.K. Belyaev and Ya.P. Lumel'skii)
The notion of unbiased estimator is fundamental in mathematical statistics.

Although this not ion was known earlier, No. 38 gives the first systematic treat-
ment of the properties of unbiased estimators and various methods for con-
structing them in terms of sufficient statistics.
Kolmogorov's paper and also [1] graphically demonstrate the applied value
of unbiased 'estimators in problems of statistical control. Kolmogorov was the
first to apply unbiased estimators to the determination of the effectiveness of
practically used schemes of sampie control with respect to an alternative fea-
ture. The construction of unbiased estimators for detecting any missed defec-
tive products, as weIl as for the apriori distribution of the number of defective
items in the batches subject to control, is considered in [2]-[4]. Unbiased es-
timators were obtained for control with respect to a qualitative feature witb
known classification errors [5]. The construction ofunbiased estimators for con-
trol with respect to alternative qualitative features, that is, for classifying into
k quality groups, was considered in detail in [3], [4], where a vast bibliography
is given. The unbiased estimators of the main control indices are included in
GOST 24660-81 (State Standards).
The unbiased estimator of the normal distribution density obtained by
A.N. Kolmogorov is widely used in the problem of control based -on a quanti-
tative feature [6]. Subsequently this result was generalized to the multidimen-
sional normal distribution [7]-[9] and to problems of statistical classification
[10].
A.N. Kolmogorov drew attention to the paper [11] in which unbiased esti-
mators were constructed for binomial random walks. Later, problems of obtain-
ing unbiased estimators and the completeness of the families of distributions
generated by the first entrance schemes for various schemes of random walks
were considered in [12]-[14], [5].
The upper and lower bounds introduced by Kolmogorov can be effectively
used also in the cases when unbiased estimators do not exist. This is the
case with estimating the number of missed defective items for the binomial
distribution and a one-step control scheme. Upper and lower bounds for various
functions of an unknown parameter, as weIl as estimators with minimal bias,
were obtained in [15]-[18].
585
586 COMMENTS
References
1. A.N. Kolmogorov, Statistical inspection control/or the admissible number

0/ de/ective items equal to zero, Leningrad, 1951, pp. 1-24 (in Russian).
2. S.Kh. Sirazhdinov, 'One-step statistical inspection control', Proc. Inst.
Math. Mekh. UzSSR Acad. Sei. 15 (1955), 41-56 (in Russian).
3. Yu.K. Belyaev, Probability methods 0/ sampie control, Nauka, Moscow,
1975 (in Russian).
4. Ya.P. Lumel'skii, Statistical estimators tor results 0/ quality control, Izdat.
Standart., Moscow, 1979 (in Russian).
5. Yu.K. Belyaev, 'Sequential estimators under sampie control with respect
to qualitative features with classification errors', Dokl. Akad. Nauk SSSR
231:3 (1976), 521-524 (in Russian). (Translated as Soviet Math. Dokl.)
6. G.J. Liberman and G.J. Resnikoff, 'Sampling plans for inspection by vari-
ables', J. Amer. Statist. Assoe. 50:270 (1955), 457-516.
7. Ya.P. Lumel'skii, 'Unbiased suflicient probability estimates for the multi-
dimensional normallaw', Vestnik Moskov. Gos. Univ. Mat. No.6 (1968),
14-17 (in Russian).
8. Ya.P. Lumelskii and P.N. Sapozhnikov, 'Unbiased estimators for proba-
bility densities', Teor. Veroyatnost. i Primenen. 14:2 (1969), 372-380 (in
Russian). (Translated as Theory Probab. Appl.)
9. S.G. Ghurye and J. Olkin, 'Unbiased estimation of some multivariate prob-
ability densities and related functions', Ann. Math. Statist. 40:4 (1969),
1261-1271.
10. R.A. Abusev and Ya.P. Lumel'skii, 'Unbiased estimators and problems of
classifying multidimensional normal sets', Teor. Veroyatnost. i Primenen.
11. M.A. Girshick, F. Mosteller and L.J. Savage, 'Unbiased estim~tes for cer-
tain binomial sampling problems with applications', Ann. Math. Statist.
11:1 (1946), 13-23.
12. Yu.V. Linnik and LV. Romanovskii, 'On the theory of sequential estima-
tion', Dokl. Akad. Nauk SSSR 194:2 (1979), 270-272 (in Russian). (Trans-
lated as Soviet Math. Dokl.)
13. Ya.P. Lumel'skii, 'Random walks satisfying generalized urn schemes', Dokl.
Akad. Nauk SSSR 209:6 (1973), 1281-1284 (in Russian). (Translated as
Soviet Math. Dokl.)
STATISTICAL PREDICTION 587
14. Yu.K. Belyaev and A.N. Mal'kov, 'Completeness offamilies ofprobability

measures in sampling by first entrance schemes', Dokl. Akad. Nauk SSSR
225:1 (1975),27-30 (in Russian). (Translated as Soviet Math. Dokl.)
15. S.Kh. Sirazhdinov, 'On estimators with minimum bias for the binomial
distribution', Teor. Veroyatnost. i Primenen. 1:1 (1956), 168-174 (in Rus-
sian). (Translated as Theory Probab. Appl.)
16. L.N. Bolshev, 'On probability estimators', Teor. Veroyatnost. i Primenen.
17. Yu.K. Belyaev and Ya.P. Lumel'skii, 'Estimators of polynomials based on
limited plans in schemes of random walks', Izv. Akad. Nauk SSSR Sero
Tekhn. Kibernet. No.l (1979), 61-67 (in Russian).
18. Yu.K. Belyaev and T.N. Dugina, 'On estimating the total number of de-
fective items accepted under inspection with errors', Elektron. Tekhn. Sero
8 No.5 (83) (1980), 24-29 (in Russian).
STATISTICAL PREDICTION (No. 18)

(A.M. Yaglom)
The first attempts at the statistical prediction of future values of certain me-
teorological parameters using linear regression equations to give a future value
of the quantity !1y of interest in the form of a simple linear combination of
the !1xl, ... !1xk known from observations made in the past or present were
started in the 1920's. (In addition to Bauer's paper of 1925 referred to by
A.N. Kolmogorov, we can also refer to later works [1]-[6] containing a num-
ber of additional references.) At first sight this prediction method seems to
be quite simple: it requires only preliminary estimates of a certain number of
correlation coefficients which determine the unknown coefficients al, ... , ak in
the regression equations and does not require cumbersome calculations such
as those involved in "dynamic weather forecasting" based on numerical solu-
tion of partial differential equations approximately describing the dynamics
of the atmosphere. Here, the only problem is to choose appropriate predic-
tors !1xl, ... , !1xk, that is, atmospheric characteristics in the past and present
whose values are used for predicting !1y. It turns out, however, that to choose
appropriate predictors is not that simple.
Indeed, it might seem that it is better to take as many predictors as possi-
ble, since in that case the forecast uses very broad initial information and will
588 COMMENTS
be accurate with high prob ability. Unfortunately, the empirical correlation co-
efficients ri and rij between il.y and il.zi, i = 1, ... , k, and between the pairs
il.zi and il.Zj, j = 1, ... , k, used for calculations are not exact and depend
on the volume (and quality) of the available empirical data. Therefore, the
coefficients al, ... , al: for which the linear combination a1il.z 1 + ... + al:il.zk
corresponding to these data gives the best approximation of il.y are not ex-
act, that is, they are not the best for the whole sampIe of random variables
(il.Zl, ... , il.Zl:, il.y) and applied to the subsequent independent sampIe, these
values may be appreciably less appropriate. To avoid this, the choice of predic-
tors should satisfy a number of special conditions. Here we will only consider
the two most important of them.
First, in order that the accuracy of the determination of the coefficients
al, ... ,ak should not be too low, the total number k of predictors should be
rather small as compared to the volume of the sampIe used for determining
the coefficients ri, rij and ai. Thus, for example, if the order of the sample's
volume is one or two hundred, then k should not exceed several units. This
condition was intuitively clear even for the first researchers of statistical weather
forecasting (see, for example, Kolmogorov's remark to the effect that when
using a sampIe with 30-50 average annual values, k is usually chosen to vary
from 3 to 7); later, the well-known American meteorologist Lorenz called it "the
taboo of statistical forecasting" . However, apart from this, there is another
essential requirement, which is clarified in No. 18 and which is by no means
always taken into account, even now. This second requirement is to forbid
searching among a large number of systems of predictors (even if each of these
systems contains only a small number of values) in order to choose the best of
these systems. The point is that if we try a large number of various systems
of predictors, then with high prob ability at least in one of them the empirical
value of the cumulative correlation coefficient between these predictors and il.y
is much larger than its true value. In this case it is quite possible that this
system of predictors will be chosen for forecasting. However, as applied to a
subsequent independent sampIe the empirical correlation coefficient of il.y and
il.u =
a1il.zl + ... + akil.zk will most probabably be essentially smaller than
that of the initial sampIe. Kolmogorov had good reason to assert that this
"blow-up of the maximal empirical correlation coefficient" will very often take
place when searching for a large number of systems of predictors, which readily
STATISTICAL PREDICTION 589
explains the failures of numerous attempts to use linear regression equations

for long-term weather forecasting. As a "model example" that allows exact
calculations, he analyzed the case when all the values under consideration have
a multidimensional normal probability distribution, the initial sampie has fixed
size N and we check i independent systems of k predictors, for each of which
the joint correlation coefficient with ay takes the same value p. Here A is such
that the maximal value of the empirical composite correlation coefficient for
our i groups of predictors to exceed A with a certain given probability (say, ~),
increases rapidly as i increases, and at moderately high values of i exceeds p
(see, for example, the table for the case N =42, k =5 and i =
14). We note
also that the accurate derivation of Fisher's results used by Kolmogorov in his
calculations can be found, for example, in the weIl-known monograph [7].
The considerations given in Kolmogorov's paper question the advantage
of certain new methods of statistical weather forecasts proposed long after the
paper was published. A typical example here is the "screening procedure" that
was advocated, for example, by Miller [4], [8]. The essence of the method is
to consider first a very large aggregate of predictors which is "screened" in the
following way. First we take out the predictor (denote it by aXl) that corre-
sponds to the largest empirical correlation coefficient with ay to be forecastj
then we take the one of those remaining (say aX2) that makes the largest con-
tribution to the joint correlation coefficient of the pair (aXl' aX2) with aYj
then we take the one of those remaining (say, aX3) that corresponds to the
largest joint correlation coefficient of the three values (aXl, aX2, aX3) with
ay, and so on. Usually, after several steps the addition to the joint correlation
coefficient obtained at the next step appeared to be insignificant, so the pro-
cedure could weIl be terminated. Usually, the number of selected predictors is
small enough not to contradict "the taboo" but, in view of the considerations
published by Kolmogorov in 1933, in the case of a large initial set of possi-
ble predictors it is quite possible that for the selected k of them the empirical
value of the joint correlation coefficient will only occasionally be large, so that
on passing to independent sampies, we obtain disappointing results.
The same is true for another widely used method of selecting only a small
number of the most significant predictors, the method of empirical orthogonal
functions, proposed in [9], [10] (see also [6], [11], [12]), which actually coincides
with the method of principal components of multidimensional statistical anal-
590 COMMENTS
ysis (cf. [7], Chap. 11). In this method, first a linear combination Ul = dXl
of predictors with the largest variability (that is, variance) is chosen from the
initial set of all possible "virtual predictors", then only linear combinations of
predictors uncorrelated with Ul are considered, and the combination U2 = dX2
with greatest variance is chosen, etc. Usually after selecting a small number k
of linear combinations Ul. U2, •.• , Uk the variance of all the others is so small
that they might as weIl not be considered. In cases when the original number
of admissible predictors is comparatively small (as, for example, in Obukhov's
work [10] where the number is 5), this method results in a notable simplification
of the calculations (for example, allowing one to confine oneself to considering
only the two most significant linear combinations ofthe five variables). If, how-
ever, we first consider a large number of predictors (for example, all the values
of all or several meteorological fields for a large network of stations or at the
no des of a regular network), then we can again apply Kolmogorov's considera-
tions, which suggest that the combinations are optimal only for the sampie for
which they were calculated, and on passing to an independent sampie the re-
sults become much worse. It is quite possible that this accounts for some of the
disappointing conclusions on statistical forecasting using empirical orthogonal
functions contained in [4].
In conclusion, we should emphasize that Kolmogorov's work clearly demon-
strates the need for a thorough study of the selective properties of systems of
predictors used for developing methods of statistical weather forecasting. This
study was started by A.N. Kolmogorov in 1933; unfortunately, many problems
remain open even now.
References
1. F. Baur, 'Der gegenwärtige Stand der meteorologischen Korrelationsfor-

schung', Meteor. Z. 47 (1930), 42-52.
2. F. Baur, Ein/ühring in die Grosswetterkunde, Wiesbaden 1948.
3. G.P. Wadsworth, ' Application of statistical methods to weather forecast-
ing', In: Gompendium 0/ meteorology, Boston, 1951, pp. 849-855.
4. E.N. Lorenz, 'Prospects for statistical weather forecasting', Statist. Fore-
casting Proj. Final Rept., Cambridge (Mass.), 1959.
5. A.M. Yaglom, 'Statistical methods of extrapolating meteorological fields',
In: Proc. All-Union Meteor. Gongr., Vol. 2, (Dynamic Meteorology),
ON INTERBED WASHOUT 591
Gidrometeoizdat, Leningrad, pp. 221-234 (in Russian).

6. P.N. Belov, 'Statistical methods of numerical weather forecasting', In:
Numerical methods 0/ weather /orecasting, Gidrometeoizdat, Leningrad,
1975, pp. 259-281; 386-387 (in Russian).
7. T. Anderson, Introduction to multivariate statistical analysis, Wiley, 1958.
8. R.G. Miller, 'Statistical prediction by discriminant analysis', Meteorol.
Monogr. 4:25 (1962).
9. E.N. Lorenz, 'Empirical orthogonal functions and statistical weather pre-
diction', Statist. Forecasting Proj. Sci. Rept. No. 1, Cambridge (Mass.) ,
1976.
10. A.M. Obukhov, 'On statistically orthogonal expansions of empirical fimc-
tions',Izv. Akad. Nauk SSSR Sero Geojiz. No. 3 (1960),432-439 (in Rus-
sian).
11. A.V. Mesherskaya, L.V. Rukhovets, M.I. Yudin and N.!. Yakovleva, Natu-
ral components 0/ meteorological jields, Gidrometeoizdat, Leningrad, 1970
(in Russian).
12. M.I. Fortus, 'The method of empirical orthogonal functions and its appli-
cation in meteorology', Meteorology and Hydrology No. 4 (1980), 113-119
(in Russian).
ON INTER-BED WASHOUT (No. 37)

(A.B. Vistelius)
The problem 80lved by A.N. Kolmogorov in this paper was posed while studying
productive rock-masses (strata) of the Apsheronskii peninsula and red-rock
oil-bearing sediments on Cheleken. These strata are comp08ed exclusively of
terrogeneous formations. They include, on the Apsheronskii peninsula, sparse
conglomerate beds, while at Cheleken the strata consist completely of sands,
aleurites and clays. The depressions in lower beds are filled by the shelves of the
higher ones. Moreover, the bases of thick sand beds often contain fragments
composed of clays that are identical in their appearance to the clays of the
clay bed underlying the sand beds. These fragments are rounded, and they are
considered by all researchers as the traces of washout of the lower bed when
the upper one was formed. This pattern is especially typical of Cheleken.
These features of the strata leave no doubt that when they were formed, the
lower beds were washed out during the formation of the upper ones ("Inter-
592 COMMENTS
bed washout"). So there were sound reasons for posing the problem solved
by Kolmogorov and it has retained its importance to this day. At the same
time one should bear in mind that when Kolmogorov's paper was published, a
number of geologists believed that the thicknesses of beds in a profile do not
depend on the material of the beds. In general, this opinion turned out to be
false. Moreover, when the paper was published, geologists did not operate with
random variables, probability distribution functions and sequences of values of
a random variable. It was the period in which the foundation of a number of
geological disciplines had just been laid, based on the notion of the stochastic
character of the values studied. This fundamental restructuring later gave rise
to mathematical geology and was largely as a result of Kolmogorov's paper, as
weIl as to personal advice and re marks by him during 1945-1950.
For the application of No. 37 to the solution of the problem relating to the
mechanism of bed formation, three requirements must be met:
a) numerical solutions of Kolmogorov's equation must be found;
b) the applicability ofKolmogorov's axioms (which, from the modern view-
point, does not have a universal character at aIl) to specific profiles should be
confirmed;
c) a model of bed-formation that gives G( x) should be available. To obtain
reasonable results, this model should be derived from bed-formation conditions
and should allow testing based on data of observations.
Condition a) is in no way restrictive. Kolmogorov's equation coincides
with the Wiener-Hopf equation, and there is extensive literature on numerical
methods for solving this latter one. In any case, even in the late 1940's there
were quite a number of methods for solving this problem. An appropriate
algorithm was chosen immediately after Kolmogorov's paper was published,
and it was used to compute a number of examples. These data were not
published, since no function could be taken for G(x).
The axioms adopted by Kolmogorov (condition b» are as follows:
1) the random variables On and ~n+l are independent, and all the On have
the same distribution law P{on < x} = G(x);
2) the expectations EO n= J~oo xdG(x) are positive;
3) the distribution of On is continuous, that is, it can be expressed in terms
of a corresponding probability density g(x) by the formula G(x) = J~oo g(x)dx.
Recall that On is the difference between the thickness of the sediment
and the subsequent depth of the washout of this sediment directly after its
accumulation is finished. The sequence of beds in the profile may have certain
beds with thickness contained in the sequence bn . It is not dear how to identify
the layers with thicknesses from the sequence bn during field work.
Items 2) and 3) of these axioms do not give rise to any doubts from a
geological viewpoint. However, in analyzing item 1), the following facts discov-
ered after publishing No. 37 have to be taken into account. Apparently, it is
reasonable to assume that: 1) in specific profiles the beds that had undergone
a washout and belong to the sequence bn can dominate; 2) when the layers of
sand beds are deposited, the values T/n playa decisive role in the final deter-
mination of the thickness of a bed. For a sequence of day beds, in many cases
T/n = O.
If this is true, then by studying the strata profiles during field work we can
estimate to what extent item 1) of these axioms is true. The following relevant
data were obtained by studying the profiles of red-rock and productive strata
and the flysch of north-western Caucasus, Kakhetia and Southern Urals.
For beds of different compositions the average bed thicknesses are usu-
ally different. Thus for red-rock strata in Cheleken it was found out that the
sand layers have average thickness of 84.11 cm. with standard error 188.34
and asymmetry +5.0 (for number of beds n = 318); for aleurites 15.22 with
16.93 and +2.7 (n = 471), and for days 27.66 with 53.23 and +8.0 (n = 792)
respectively.
Sampie correlation functions for sequences of bed thicknesses are divided
into two types, Band F (Vistelius [2]). For type B all autocorrelation coefli-
cients are positive and monotone decreasing as the distance s between the beds
increases (in the number of beds). For type F, the odd autocorrelation coefli-
cients are negative and the even ones are positive. Both sequences of even and
odd autocorrelation coeflicients rapidly and monotonely decrease in absolute
value as s increases. Hence, if the beds with thicknesses from the sequence bn
really dominate in the profile, then the assumption of the independence of the
bn can be questioned.
It is interesting to note the following. Let
E(x" X,+r la,=i, a,+r=j) - E(x, la,=j, a,+r=j )E(x,+rla,=j, a,+r=j) = 0,
where adenotes bed composition, i,j denote elements ofthe set ofbed compo-
sitions, x denotes the bed's thickness, s denotes the number of the bed starting
594 COMMENTS
from the foot of the profile, and r denotes the distance between beds. In this
case, for simple Markov chains consisting of two states (for example, sand and
day), each of which corresponds to a random value (bed thickness) with dis-
tinct expectations for each state (that is, the average thickness of sand beds
differs from that of day beds), only three types of correlograms are possible:
belonging either to type B, to type F, or identically zero (Vistelius [4]).
The latter statement shows that in the sequences of beds that were washed
out only once after sedimentation, and without total destruction, the sequence
of bed composition is a simple Markov chain, and the correlogram is of type
F. Hence for these sequences, bed sections could be found in which successive
beds have the same g(z) and the 6n are independent. Something similar was
observed in flysch profile in the Southern Urals. Here a sequence of 1530
layers (described by Bezhaev) is dose to a homogeneous Markov chain, and its
correlogram is of type F with the following sampIe autocorrelation coefficients:
-0.23 0.17 - 0.19 0.07 - 0.09 0.05 0.02
(Vistelius [3]). The table below gives estimates of the correlation coefficients
between bed thicknesses next to each other (8 + 1) and every second one (8 + 2)
for fixed bed compositions.
Sampie autocorrelation coefficient8 between the thickne88e8

0/ beds 0/ fixed composition
Composition Composition
Composition oft he (s + 1)th bed of the (8 + 2)th bed
of the 8th bed
'Ir er 'Y 'Ir er 'Y
0.22 0.24 -0.04 0.09 0.28

'Ir - --
43
--
153
- -
46
--
94
--
57
0.17 0.07 0.18 0.27

er - - --
543
--
125
--
357
--
61
0.3 -0.11 0.35 0.06 -0.01 0.19

'Y -
195
- -
502
--
87
--
21
- -
94
--
667
where 11" are sand, er aleurite, and 'Y clay beds, whose comp08itions are given
by the table's entries. The numerators of the entries give autocorrelation coef-
ficients between the beds of the corresponding compositions and denominators
the number of the bed pairs observed. Dashes mean that the number of the
beds compared was less than 5.
It can be seen from the table that in a first approximation the beds com-
p08ed of sands and aleurites make it possible to take a sampie in such a way
that the 6n are independent, provided they dominate in the profile.
Summing up it can be said that, when embarking on the analysis of bed
sequence we should very thoroughly check condition 1) of Kolmogorov's axioms.
Here we may encounter both cases satisfying the axioms (especially if we use
a special testing procedure and study not all the beds, but only some special
types of them, for example, sands), and those contradicting it. One should
also study the robustness of the model, that is, how violations of the axioms
propagate on the final geological conclusions.
While the problem of verifying the axioms finally reduces to that of the
robustness of the model and the necessity of developing special procedures for
selecting beds that satisfy Kolmogorov's conditions, the question of choosing
g(z) in specific studies is much more difficult. Since there are non-observable
values among the 6n , g( z) can only be found from a model of layer formation
based on appropriate lithological (sedimentological) assumptions. Attempts to
construct such a model have not resulted in anything really worth while so far.
Kolmogorov's paper was ahead of its time and only in 1962, when geol-
ogists became aware of the importance of mathematics for their science and
mathematical geology took its proper place, did this work draw the attention
of researchers. This paper was referred to in the first book on the application
of mathematical methods in geology (Miller and Kahn [7]), and was mentioned
later in all serious publications on this subject (Agterberg [1]). Unfortunately,
these textbooks gave only general references to this paper, without analyzing it.
In 1975 the monograph by Schwarzacher [8] was published. It was the first to
analyze Kolmogorov's work. Schwarzacher considered the corresponding model
for a discrete case using the method ofrandom walks. He also imposed stronger
independence conditions.
The literat ure on specific studies based on Kolmogorov's paper can be
subdivided into two types. In one of them no analysis of the correspondence
596 COMMENTS
between Kolmogorov's axiomatics and specific conditions of profile formation is

given. They make no attempt to justify the introduction of a density g(x). In
other words, they do not use the statement of the problem, but the algorithm
that comes out of Kolmogorov's paper. In doing this, they forget that this
algorithm should be used only after a thorough study of the conditions of
profile formation. Among these works there is a publication that recommends
a special "Kolmogorov index" for estimating washout (Mizutani and Hattori
[6]).
In 1979 the first paper appeared that discussed Kolmogorov's axiomatics
and gave data on the type of G( x) found under special conditions (Dacey
[5]). Dacey assumed that the en and 1/n are independent identically distributed
random non-negative variables which are also mutuaHy independent. Under
these conditions it appears that if en and 1/n have exponential distributions
in the continuous case (or geometrie distribution in the discrete case) , then
the residues of beds after washout also obey this distribution. Dacey does
not give any geological data to confirm the importance of his case for geology.
Still, his paper must be singled out since it contains a competent analysis of
Kolmogorov's work.
As we have already mentioned, Kolmogorov's paper was very much ahead
of its time, and still some time was to pass before geologists were to realize the
importance of this outstanding result and the approach as a whole. There are
a number of branches of sedimentology (lithology) that cannot be developed
without working out the whole range of problems posed by this work. Hope-
fuHy this will be understood, which will greatly contribute to progress in some
important branches of geology.
References
1. F.P. Agterberg, Geomathematics, Elsevier, Amsterdam, 1974.

2. A.B. Vistelius, Materials on lithostratography 0/ productive strata 0/ Azer-
baidzhan, Izdat. Akad. Nauk SSSR, Moscow, 1961 (in Russian).
3. A.B. Vistelius, Red-rock deposits 0/ the ehe/eken peninsula (Lithology),
Nauka, Leningrad, 1966 (in Russian).
4. A.B. Vistelius, Fundamentals 0/ mathematicallithology, Nauka, Leningrad,
1980 (in Russian).
5. M.F. Dacey, 'Models of bed formation', Internat. Assoe. Math. Geol. J.

11:6 (1979), 655-668.
6. S. Mizutani and I. Hattori, 'Stochastic analysis of bed-thickness distribu-
tion of sediments', Internat. Assoe. Math. Geol. J. 4:2, 123-146.
7. R.L. Miller and J .S. Kahn, Statistical analysis in the geological sciences,
Wiley, New York, 1962.
8. W. Schwarzacher, Sedimentation models and qualitative stratigraphy, EI-
sevier, Amsterdam, 1975.

(Mathematics and Its Applications 26) Shiryayev A. N. - Selected Works of A.N. Kolmogorov. Volume II - Probability Theory and Mathematical Statistics-Springer (1992)

Uploaded by

Copyright:

Available Formats

(Mathematics and Its Applications 26) Shiryayev A. N. - Selected Works of A.N. Kolmogorov. Volume II - Probability Theory and Mathematical Statistics-Springer (1992)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

(Mathematics and Its Applications 26) Shiryayev A. N. - Selected Works of A.N. Kolmogorov. Volume II - Probability Theory and Mathematical Statistics-Springer (1992)

Uploaded by

Copyright:

Available Formats

Selected Works of A. N.

A. A. KIRILLOV, MGU, Moscow, U.S.S.R.

TransJatedfrom the Russian by G. Undquist

Printed on acid-free paper

This is an annotated translation from the original work

Published by Nauka, Moscow, © 1986

All Rights Reserved

Bussum, December 1991 Michiel Hazewinkel

A few words ab out A.N. Kolmogorov . . . Xlll

1. On eonvergenee of series whose terms are determined by random events 1

25. Solution of a biological problem . . . . 216

* Uspekhi Mat. Nauk 38:4 (1983), 7-8.

connections between topology and the theory of ordinary differential equa-

the expectation of Yn, and by

the expectation of the square of the deviation Yn - an.

<P 1 (z ) , <P2 ( Z ), ••. , <Pn ( z ), ...

by means of the following recurrence: assume, without lass 0/ generality, that

4>1(X) = z~i) = y~;) - a1

on the ith interval. Then we have

Pn(1) 'Pn(2) , ... , Pn(i) , ...

and on the ith subinterval we set

since 'll"n(t) is constant on (a, x). From the conditions

Iz~j)p~)1 ~ E{z~i)Fp~)Ep~) ~ v;;;.J1-p~),

On the other hand we have

where Iz~1d is the maximal value of l<Pn+l(t)1 for a < t ~ x. If

From (5) and (6) it follows that

ILln(x)1 = Ifn+l(X) - fn(x)1 ~

~ 2v'bn+1Vl- p~+t) ft {p~k) + .jb;Vl- p~k)}

Since for every positive integer N, putting Vo = 0, we have

:sII II (1 + b k) < vB,

converges almost everywhere, since otherwise the series

would diverge, which contradicts the assumption.

We now prove this assertion in a different, much simpler way. We set

4 If lim (b n - an) > 0, then we have for sufliciently sm all f >0

lim ~[fn(x + f) - fn(x - f)] =

If now k is the largest of these numbers p (there is a finite number of

This implies the divergence of E bn , contrary to the assumption.

Thus, convergence of E bn is a sufficient condition for convergence of E f/;n (x)

However, there is an important special case, when our condition is also

f f {SIt;+1(X1) - SIt;+1(X2)}2dx 1dx 2 = f f {Slt;+l(Xt}-

-SIt;+1(X2)}2dx1dx2 - 2 f f {SIt;+1(X1) - SIt;+1(X2)}2dx1dx2-

But tPlt;+l(X) is orthogonal to both sIt;(x) and 1 on EIt;, hence

-2 f f {SIt;(X1) - SIt;(X2) + tPIt;+1(X1) - tPIt;+1(X2)}2dx1dx2-

- f f {SIt;(x!) - SIt;(X2) + tPIt;+1(X1) - tP1c+1(X2)}2dx1dx2 ~

~ f f {SIt;(xt} - SIt;(X2)}2dx1dx2 + 2bIt;+1(mEIt;)2-

where M is the upper bound of l~n(z)1 and mE denotes the measure of E. In

which clearly proves the convergence of E bn .

Consider, together with (1), another similar series

Ul + U2 + ... + Un + ... , (10)

denote by Q the prob ability of its convergence, and by Tn the probability of

6 This means that there is a certain relation between Un and 'Yn.

where D n is the expectation of F n , tends to 1 as n tends to infinity, then F n is

-F.n (E1Cid , ... , EU"-l) Ch

Theorem. If the sum

tends to zero as n increases, then F n satisjies the law of large numbers.

Corollary. The condition

Proof of the theorem. Denote by B n the expectation of the square of the

To use this eondition, a more effeetive expression for B n is needed. For

Fn - Dn = Znl + Zn2 + ... + Znn. (3)