UNCLASSIFIED
AFOSR TN 57648
ASTIA Document No. AD 136 634
Doeblin's theory of
Markov processes I
K. L. Chung
Syracuse University
Research Report 14
Contract No. AF 18 (600) 760
File No. 3.5
September 1957
Mathematics Division
Air Force Office of Scientific Research
UNCLASSIFIED
DOEBLIN'S THEORY OF MARKOV PROCESSES, I.
K. L. Chung
For the general setup of a Markov process with discrete time
parameter, general state space and stationary transition probabilities
we refer to Doob [2; Ch. 5 5]. Our study is based on the most general
theory developed by Doeblin [1]. A version of this work was previously
given by the author in lecture notes at Columbia University in 1951.
The present attempt aims at a better synthesis with a number of new
results added. It is selfcontained and will be given in several parts
of which this is the first instalment.
Among the notation to be used we note the following
X is the state space; x, y, z are points of X; E, F, C, D are sets
of a given Borel field of subsets of X; E = X E.
P is the underlying probability measure. The Markov process is
{tn' n > 0}. {rn E}, for example, is the event that n belongs
to E. The generic sample point is w, omitted when dispensable.
p(n)(x, E) = P{m+n E m = x} is the nstep transition probability
frox the point x to the set E; p(l) is abbreviated to p which is
assumed to be measurable in x for each E and a measure in E for each x.
co
L(x, E) = P( [n E] m = x .
n=m+l
Q(x, E) = P{limsup [n c ] m = x}.
Q(x, E; F) = Ptlimsup [n c E]n limsup [fn F]m = x}.
Simple measurability questions will be passed muster. Results will
be recorded under Roman numerals for reference. After each recorded
result the proof is given in the following paragraph.
For an arbitrary E we define four sets:
E = (x: L(x, E) = 0}
El = {x:
f =
E = Ix:
L(x, E) = 1}
Q(x, E) = 0}
Eo= {x: Q(x, E) = 1}.
Definition 1. A nonempty set E such that p(x, E) = 1 for every x c E
is called stochastically closed (cl.)
I. If x c E, then p(x, E 0) = 1.
either both empty or both cl.
We have:
The sets Eo and E E are
0 = L(x, E) =
PE
p(x, dy) +
E0
L(y, E) p(x, dy)
+ I L(y, E) p(x,dy)
E F
where the integrand in the last integral is positive.
Hence: EU EF
p(x,
= 0,
E l p(x, E E) = i.
II. If x E E1, then p(x, ElUE) = 1. If E is cl'., so is E1
$o E
We have!
1 = L(x, E) = p(x, E) + p(x, E1 E) + L(y, E) p(x, dy)
E1 E
where the integrand in the last integral is less than 1.
Hence:
p(x, E1 ) = 0.
& K EC A_ J
p(x, ElE) =1.
/t
,~~
III. EE is cl. or empty.
Let x c E ; then
0 Q (x, E) = Q(y, E) p(x, dy) a
X
The integrand in the last integral is
IV. Eco is cl. or empty.
Q(y, E) p(x, dy) > 0.
Ef
positive; hence p(x, E ) = 0
Let x c E then
1 = Q(x, E) =
Q(y, E) p(x, dy) +
SQ(y, E) p(x, dy)
Eo0
The integrand in the last integral is
V. If E = U E, then Eo = nEO
n n n n
less than 1; Hence p(x, E 0) = 0.
J 4
Clearly Eo e E so that EO C 0/ EO On the other hand if
n n n
x QF En, then for every n we have L(x, En) = 0;
consequently
L(x, E) = L(x,
and x & EO.
Thus (f Eo E.
n n 
Remark. Anplausible, but false identity is (E )1 = E .
Take:
1 1
X = {xn, n > 0 p(x, xo) = 1, p(xl' xo) = 9 p(x1, x2 29
(x, Xn+I) = 1 , n, x1) = n > 2 E = {xl}.
n n
Then:
E0 = {Xo, }, E
= X. We have, L(x1, E) 1
T
r /* ^..~
o00
n (1
k=2
 ) <
k
Or o
F:
L ^[ (X\ 
T
;tM33 ~1\~ ~tf.
JE ) < 2 L(x, E ) = 0
n n n n
O) xtr/titC*vtv/^
LCt^JQ 'X. <^~
/ik^r b^^^XS
1
' 2
Xc" C7
I :I
00oo
and if n > 2, L(xn, E) = L(x xl) = 1 (1 ) < 1
k=n k
Hence: (E0) = {Xo}.
Definition 2. A set E such that Q(x, E) = 0 for every x c X is called
inessential (iness); otherwise essential (ess.) &n essential set which
is the denumberable union of inessential sets is called improperly
essential (imp. ess.); otherwise absolutely essential (abs. ess.).
The next two theorems are basic for the sequel. VI is given by
Blackwell [3], VII in the lecture notes mentioned in the beginning
paragraph. Both were proved by simple, direct arguments. Here they
will be proved by the martingale convergence theorem.
VI. If sup Q(x, F) < 1
xcE
then Q(x, E/ F) = 0 for every x: X.
Putt
A E = limsup n E
For any E the eventAE is invariant under the shift n n+l'
We have, with a Baire function )n'
P{AE AFIo' "...," n P{AEAFIn = n ()
By the martingale theorem [2;,p331]
(1 if w A/\E AF
(I) lira l(Wn (w)) = othEri
nico Yn (0 otherwise
for almost all w. Because of the invariance we have
P AE AF~ln =x = PE AFIo(V =
so that all on F (The conditional probabilities are taken to
4Y < I 4r t
be the versions uniquely determined by p(x, E); see [21 p 191].) Now
if w c0AE, then there is a sequence tnk} going to infinity such that
Q(x, F < 1.
This is consistent
this means Q(x, E;
with (1) only if P(/E A
F) = 0.
F) = 0. If we set o = x
VII. If inf L(x, F) > 0
xCE
fR I Att then Q(x, E) = Q(x, E> F) for every x c X.
00
Let Mn = k c F1, and
k=n+l
for a fixed N > 0,
Pt/NE MNI o', = nn
Then for almost all w,
(1
lim ( ) = (0
nr+O n in(0
if w CA.E N
otherwise
We have if n > N,
n(x) > P{/\EMI O. = j
= PA\E 1 ^n = x~ = P /E IV/ = x4 > L(x, F).
It follows as in the preceding proof, if nk > N,
nk (nk (wo ) _> gi L(x, F) > 0.
This is consistent with (2) only if P(/E.AE MN) = 0. This
being time for every N we have:
Q(x, E; F) == imP(AE MNI = x ) = AE = x) = Q(x, E).
n*oo EMNIo=x FOI o=)
'I
(nk ( ))
r /d / .^7 2 I
p 7 2
~3Zc4 S *
t4,
CL" 22 > aO
24
7/o^l /? / /,, > 7 ? z 4/ . r., 7 a ,
i // f / ,,  j 1/, / 1
J/,./ , T
/1L h / l f, .f7 ^ J =^ .A / ?., 7 ^ J ,z
i LCL / 1 / ?A. 7 I /f L)
7//L ( () ) "> O ai e. /t 
~r4M ~~7
.7~1~7 k
I'/y^ 
L~~ L~/ cto L1 c ~70
>1
VIII. If E is ess. and in L(x, F) > 0 then F is ess.
Since E is ess. there is an x such that Q(x, E) > 0.
By VII, Q(x, F) Z Q(x, E0 F) = Q(x, E) > 0. Hence F is ess.
IX. If E is abs. ess. and inf L(x, F) > 0 then F is abs. ess.
xcE
It is sufficient to prove that whenever F = U Fk there
k=l )
n
o
exists an no such that 19 Fk is ess. We note the simple
k=l
identity:
n
L(x, F) = lim L(x, UL Fk).
n+oo k=l
Let x c E, L(x, F) o o( O; hence there exists a finite mo(x)
such that:
mo(x)
L(x, Fk) > > 0.
k=l
oo
Let E = {xcE: m (x) = n}, then E =Z2 E Since E is abs. ess.
n o n;;j n
there exists an n such that E is ess. By the definition of
0 n
E we have:
0o n
0 ~
inf L(x, (J Fk)> / o
Hence by VIII, U( Fk is ess.
k=l
X. If there exists a set F such that
sup Q(x, F) < 1, inf L(x, F) > 0,
then E is iness.
For every x we have by VI and VII, Q(x, E) = Q(x, EA F) = 0.
Hence E is iness.
XI. Cf Xis abs. es~g.. andIiL .rOthen E is.
Let; >
E = {x L(x, E)
then we have
ic<
oo
X= E+. . E +E. U nF
1I ? L ,' I L(.,( n=2 v = i
i 4 i)
Since Q(x, E) L(x, E), eachE is iness, by X and so their
n
union is not abs. ess. It follows that El is abs. ess. Now
infl L(x, E) 1,.... .
rhenee E is abs. ess. by IX.i
S,, 4 .. .XII. If E is abs. ess. then for any F the set
L, E(FO + F)
is abs. ess. in particular not empty,
Let:
SE = x: Q(x, F) 1 L(x, F) 1
n n n
It is clear that
2 E = } EE + E(F + FC)
n=1
S' Each EE is iness. by X, since E is abs. ess. it follows that
n
E(Fo + Fj is abs. ess.
XIII. If E is abs. ess., then EEcO is abs. ess; in particular Eo0 / 0.
Applying XITwith F = I we see that EE0 + EEO is abs. ess. Now
EEO is obviously iness., hence EEco is abs. ess.
^ C 7 C o i 0 u" kC] CcA i ,*
1 _8.
XIV. If C is cl., and C does not contain any cl. set, then C is not,
abs. ess. In pr TF~culrO + Co is not abs. ess.
rs Since C is cl., any point in (C) must by definition belong to C.
S. 4 .. But (C)oo is cl. if not empty by IV, hence (5)oo = 0 and consequently
)C j ) ' CJ C is not abs. ess. by XIII. To prove the second assertion we
dR C,tAf observe that any cl. set contained in C must be contained in Co,
CL hence C + Co does not contain any cl. set.
Definition 3. A cl. set which does not contain two disjoint cl. sets. is
SC P 0 called indecomposable (indecomp.); otherwise decomposable (decomp.) An
indecomposable set which is not properly contained in any indecomposable
Z,; set is called maximal indecomposable (max. indecomp.).
XV. If E i/ indecomp., then (Eo)is max. indecomp.
Suppose it is decomp.; let C and D be two disjoint cl. sets contained
in it. For any x c C we have x E since Eo (E0)0 = 0; hence
L(x, E) > 0. Since C is cl. this implies that CE / 0. Similarly
DE / 0. Thus CE and DE are disjoint cl. sets contained in E and E
is decomp. We have thus proved that if E is indecomp., so is (E 0).
Now suppose F is cl. and containS(EO) properly. Let x e F (EO),
then L(x, Eo) > 0. Since Eo is cl. this implies FEof 0 and so is cl. Thus
F contains the disjoint cl. set E and FEo and is decomp.
We have thus proved that any cl. set properly containing (Eo) is
decomp. Hence (E O) is max. indecomp.
XVI. Two max. indecomp. sets are either identical or disjoint.
Let E and F1e two instinctt indecomp. sets. The E U F is cl. and
contains .both properly. Hence it is decomp. and contains two disjoint
cl. sets C and D. Since E is indecomp. at least one of EC and ED is
empty. Suppose EC is empty. Then F > C and since F also contains EF
which is either cl. or.empty we must have EF = 0 since F is indecomp.
XVII. If X is indecomp. and E is abs. ess., then EO = 0.
By XIII we have Eco / 0. By I and IV, EO and Eo are cl. if not empty.
Since E E = 0 and X is indecomp. we must have E = 0.
XVIII. If X is abs. ess. and indecomp., then every sequence of cl. sets has
a cl. and abs. ess. intersection.
4/
) VCI
Let (Cn, n > 1} be a finite or infinite sequence of cl. sets.
Then D =
n
n
() Ck is not empty since X is indecomp. We have
k=l
x = 5 U (nf D ).
n n n n
Each D being cl., t is not abs. ess. by X and so is the
n n
union U D Since X is abs. ess. it follows that C( D is abs.
n n n n
ess. In particular it is not empty and so is cl.
Remark. In an indecomp. but imp. ess. X it is possible that every
sequence of cl. sets has a nonempty intersection. Take any
denumerable X which forms one nonrecurrent class (of a Markov chain)
so that the only cl. set is X itself.
XIX. In an abs. ess. and indecomp. space X an abs. ess. set E is
characterized by any one of the following properties:
E = 0, E / 0, Ef = 0,
Fhat E = 0 is characteristic follows from XI and XVII.. Next, each
of the three sets E0, E o, E is either cl. or empty, by I, IV, III.
Applying XIgwith X, E for the E, F there we see that at least one
of the two sets EO and E is not empty. Hence exactly one is not
empty since Eo E00 = 0 and X is indecomp. Hence E0 / 0 is
equivalent to Eo = 0. Finally since EO E f Ef = 0 implies EO = 0;
on the other hand since E E00 = 0, Ef 0 implies Eco = 0OO.C
Remark. Let us use the symbols "*" and "4" to denote "implies" and
"does not imply" respectively; and the symbol "A" to denote 'E is
abs. ess." The complete situation is as follows (where + stands the
required example is trivial):
4i
Abs. ess. and
Arbitrary X Abs. ess. X Indecomp. X idec
indecomp. X
SA EO = 0
4 j Ef = 0
E0/ 0 A
E = 0 A
E = 0 + A
A+E = 0
A E = 0
Ec / 0 A
0
E = 0 A
Ef = 0 A
Xj1uz
E OD + A
.i~ i ~"
10.
XX. If X is indecomp. and E is abs. ess. then
3) E p(n) (x, E)
n
is infinite for every x c X. If X is abs. ess. and (3) is positive
for every x c X, then E is abs. ess.
If (3) is finite for some x, then by the BorelCantelli lemma
Q(x, E) = 0 so the Ef / 0. Since X is indecomp., Ec = 0.
Hence E is not abs. ess. by XIII. If (3) is positive for every x,
then E = 0. Hence E is abs. ess. if X is, by XI.
Remark.
every x
of {Yn,
x.I
7
("; 4.
It is possible in an indecomp. X that (3) is infinite for
but E is iness. Consider the following example: X consists
n >1) and x nk 1 < k < n, n 1}.
p(yl, 2) = 2;
p(Yn Yn+) = 1 PI
n
P(y'1 Xnl) = Pn 3
t n
It is clear
Let:
p(y, y1) , n 2;
n
T
^,~ ~ {'(, ^ '2
""G '7 "
P(xnk, xn, k+l) = 1, 1 < k < n;
P(Xnn, y1) = 1
that X forms one nonrecurrent class.
E= {Xnk, 1 < k < n, n > 1).
We have:
p() (Y1, E) >
CO ok
k=n p 2 k2
k=n Pk2 ^n k2
so that (3) is infinite for x = y. Since L(x, y ) > 0 for every x
it follows that (3) is infinite for every x. To see that E is iness.
we verify easily that inf L(x, x) = p > 0, s Q(x, xll) >' and
and apply ?
C
h__.
~h"~ IL jl~l. X'I C)
Definition 4. & set E such that Q(x, E) < 1 for some x c X is
called habtitbe (B" P
Yro&i4fe
L(x,G) < I
XXI. If E isM& it is ess.
For Q(x, E) < 1 implies Q(x, E)> 0, since Q(x, E) + Q(x, E) 1 1.
XXII. If C is cl. and C is ess., then C is I
Since 6 is ess. there exists an x such that Q(x, O) > 0.
Since C
is cl. this implies that Q(x, C)jl Hence C is sawr.
XXIII. &ny imp. ess. set is contained in a(B L and imp. ess. set.
If X is imp. ess. the proposition is trivial. Now suppose X is abs.
ess. and E is imp. ess. Then Eo / 0 by XI., and E E is cl. by I.
The set# E J .. .
fo.~t ~h O
XXIV.
,, I P 3 p
is not abs. ess. 4 It contains E and is ~b by XXII.
If E is q' then u ^ E
f RIU ^4b C
inf L(x, E) = 0.
xgE
We have for any E,
1 = Q(x, E + E) = Q(x, E) + Q(x, E) Q (x, E( E)
If inf L(x, E) > 0 then by VII we have Q(x, E) = Q(x, E{ E) so
xcE
that the above identity implies Q(x, E) = 1 for every xcX.
Thus E is not gggq S c
4`1 I
cFC (
S5) 3 K(
v
k) A. t14 I C', . ^, .4
CP1 C' '4 L14 L 1A 1(
BIBLIOGRAPHY
W. Doeblin;
J. L. Doob:
D. Blackwell:
E16ments d'une theorie general des
chatnes simples constantes de Markoff.
&nn. Sci. Ecole Norm Sup. (3) Vol. 57
Fasc. 2. (1940). pp 61111.
Stochastic Processes, Wiley and Sons.
1953.
The existence of normal chains, Bull.
Amer. Math. Soc. Vol. 51 (1945), pp.
465468.
UNCLASSIFIED
Bibliographical Control Sheet
1. Originating agency and monitoring agency
0. A.: Syracuse University, Syracuse, New York
i.. A. :Aathematics Division, Office of Scientific Research
2. Originating agency and/or monitoring agency report number:
0. A.; Report No. 14
M. A.: AFOSR TN 57648
3. Title and classification of title: Doeblin's theory of :Aarkov
processes
4. Personal author(s)s K. L. Chung
5. Date of report: September, 1957
6. Pages: 12
7. Illustrative material: None
8. Prepared for Contract No.: AF 18 (600) 760
9. Prepared for Project File No. 5 3.5
10. Security Classification: Unclassified
11. Distribution Limitations: None
12. Abstract: This report is the first installment of a general theory of
iMaarkov processes based on the work of Doeblin.
AIR FORCE
Commandant
Headquarters Air University
Maxwell Air Force Base,
Alabama
Applied Mathematics and Stat:
Laboratory
Stanford University
Stanford, California
The Rand Corporation
Technical Library
1700 Main Street
Santa Monica, California
Director of Advanced Studies
Air Force Office of Scientif:
Research
Post Office Box 2035
Pasadena 2, California
Department of Mathematics
Yale University
New Haven, Connecticut
Human Factors Operations Resi
Laboratories
Air Research and Development
Command
Bolling Air Force Base
Washington 25, D.C.
Department of Commerce
Office of Technical Services
Washington 25, D.C.
Library
National Bureau of Standards
Washington 25, D.C.
Headquarters, USAF
Director of Operations
ATTNs Operations Analysis
Division, AFOOP
Washington 25, D.O.
DISTRIBUTION LIST
OFFICE OF SCIENTIFIC RESEARCH
MATHEMATICS DIVISION
1 MARCH 1967
Commander
European Office, ARDC
60 Rue Ravenstein
Brussels, Belgium
Department of Mathematics
University of California
Berkeley, California
Commander
Air Force Flight Test Center
ATTN: Technical Library
Edwards Air Force Base,
California
Commander
Western'Development Division
ATTN: WDSIT
E. 0. Box 262
Inglewood, California
Chief of Naval Operations
Department of the Navy
ATTN: OPO3EG; Mr. Arthur Kaufman
Washington 25, D.C.
Office of Naval Research
Department of the Navy
ATTN: Code 432
Washington 25, D.C.
Director National Security Agency
ATTN: Dr. H.H. Campaign
Washington 25, D.C.
National Applied Mathematics
Laboratories
National Bureau of Standards
Washington 25, D.C.
Commander 2
Air Force Office of Scientific Research
ATTN: SRDB
Washington 25, D.C.
Commander,Air Force Office of 2.
Scientific Research
ATTN: SRE
Washington 25, D.C.
Commander
Air Force Armament Center
ATTN: Technical Library
Eglin Air Force Base, Florida
Department of Mathematics
Northwestern University
Evanston, Illinois
Department of Mathematics
University of Chicago
Chicago 37, Illinois
Department of Mathematics
Purdue University
Lafayetto, Indiana
Mathematics and Physics Library
The John Hopkins University
Baltimore, Maryland
Department of Mathematics
Harvard University
Cambridge 38, Massachusetts
Department of Mathematics
Massachusetts Institute of
Technology
Cambridge, Massachusetts
Commander
Air Force Cambridge Research Center
ATTN: Electronic Research Library
L. G. Hanscom Field
Bedford, Massachusetts
Willow Run Research Center
University of Michigan
Upsilanti, Michigan
Department of Mathematics,
Institute of Technology
Engineering Building
University of MAinnesota
Minneapolis, Minnesota
Department of Mathematics
University of Missouri
Columbia, Missouri
National Science Foundation
1520 "'tP St., N.W.
Washington 25, D.C.
Commander
Air Force Missile Test Center
ATTN: Technical Library
Patrick Air Force Base, Florida
Institute for Air Weapons Research
Museum of Science and Industry
University of Chicago
Chicago 37, Illinois
Department of Iathematics
University of Illinois
Urbana, Illinois
Institute for Fluid Dynamics and
Applied Mathematics
University of Maryland
College Park, Maryland
Commander
Headquarters Air Research and
Development Command
ATTN Technical Library
P.O. Box 1395
Baltimore 3, Maryland
Commander
Air Force Cambridge Research Center
ATTN: Geophysics Research Library
L. G. Hanscom Field
Bedford, Massachusetts
Department of Mathematics
Wayne University
ATTN: Dr. Y. W. Chen
Dotroit 1, Michigan
Department of Mathematics
Folwell Hall
University of Minnesota
Minneapolis, Minnesota
Department of 'Mathematics
Washington University
St. Louis 5, 1:issouri
Linda Hall Library
ATTN: M!r. Thomas Gillis
Document Division
5109 Cherry Street
Kansas City 10, Missouri
Commander
Strategic Air Command
ATTN: Operations Analysis
Offutt Air Force Base
Omaha, Nobraska
Library
Institute for Advanced Study
Princeton, New Jersey
Commander
Holloman Air Development Center
ATTNS Technical Library
Holloman Air Force Base,
New Mlexico
Professor J. Wolfowitz
Mathematics Department
White Hall
Cornell University
Ithaca, New York
Mathematics Research Group
New York University
ATTN: Professor I'. Kline
45 Astor Place
New York, New York
Department of Mathematical Stat
Fayerweather Hall
ATTN: Dr. Herbert Robbins
Columbia University
New York 27, New York
Institute for Aeronautical
Sciences
ATTN: Librarian
2 East 64th Street
New York 21, New York
Department of Mathematics
Duke University
Duke Station
Durham, North Carolina
Commander
Air Technical Intelligence
Center
ATTNt ATIAE4
WrightPattorson Air Force Base
Ohio
Commander
USAF Institute of Technology
ATTN: Technical Library, MCLI
WrightPatterson Air Force Base
Ohio
Lstics
The James Forrestall Research Center
Library
Princeton University
Princeton, New Jersey
Department of Mathematics
Fine Hall
Princeton University
Princeton, New Jersey
Commander
Air Force Special Weapons Center
ATTN: Technical Library
Kirtland Air Force Base
Albuquerque, New Mexico
Department of Mathematics
Syracuse University
Syracuse, New York
Department of Mathematics
Columbia University
ATTNs Professor B. 0. Koopman
New York 27, New York
Commander
ATTNs Technical Library
Rome Air Development Center
Griffiss Air Force Base
Rome, New York
Institute of Statistics
North Carolina State College of A & E
Raleigh, North Carolina
Department of Mathematics
University of North Carolina
Chapel Hill, North Carolina
Office of Ordnance Research 2
Box CM
Duke Station
Durham, North Carolina
Commander
Wright Air Development Center
ATTN: Technical Library
WrightPatterson Air Force Base
Ohio
Commander
Wright Air Development Center
2 ATTN: ARL Technical Library, WCRR
WrightPatterson Air Force Base
Ohio
Chief, Document Service Center 10
Armed Services Technical
Information Agency
Knott Building
Dayton 2, Ohio
Commander
Arnold Engineering Development
Center
ATTN: Technical Library
Tullahoma, Tennessee
Department of Mathematics
Rice Institute
Houston, Texas
Department of Mathematics
University of Wisconsin
Mladison, Wisconsin
Department of Mathematics
Carnegie Institute of Technology
Pittsburgh, Pennsylvania
Department of Mathematics
University of Pennsylvania
Philadelphia, Pennsylvania
Defence Research Laboratory
University of Texas
A ustin, Texas
Commander
Air Force Personnel and Training
Research Center
Ladapand Air Force Base
San Antonio, Texas
Mathematics Research Center
AT'IN R,, E. Langer
University of Wisconsin
Madison, Wisconsin
Doeblin's theory of Markov processes, II
by K. L. Chung
This is a sequel to Part I under the same title (AFOSR TN 57648). The same
notation as there will be used except that the transition probability function will
be denoted by P rather than p The basic Borel field of the state space X
will be denoted by F In what follows k is a positive integer.
The properties of a set in F such as "closed" and "essential" were defined
with reference to the basic transition probability function P(*,') Ifithe
latter is replaced by its kth iterate P(k)(.,.) then the corresponding property
will be prefixed by "(k)_. Thus the previously defined concepts are the
P(1) versions, with the prefix "p(1)" omitted from the terminology. The
results we have proved so far have their P(k) versions which need no new
proofs. In terms of the process, we shall be considering i nk+r n } fcr
a fixed k and r in lieu of n n 0 .
3 (k) (k) (k)
XXV. A set i P(k) iness., p(k) imp. ess., or P abs. ess.
according as it is iness., imply ess., or abs. ess.
If a set is iness., it is clearly P~k)_ iness. If E is ess. there
exists an x c Z such that Q(x,S) > 0 Then for each k there exists an r ,
1 i r k such that
P c nk+r E for infinitely many values of n = x > 0 .
= ) nk+r 0
Hence there exists a y c E and an integer n0 such that
P '~ c for infinitely many values of n jk = y > 0.
T = hnk+r show> kn +r
This shows that E is p(k)_ uss. The other assertions follow easily.
Definition 5, For an arbitrary set E in F we set
2
A(s) = x : P(x, E) = 1 }
Let A (E) = E Al(E) = A(E) and define A'(E) for each j 2 1 by
AJ() = A(1(E)) .
AJ() is called the jth antecedent of .
We have A(E) c F for any E c F since P(.,E) is Fmeasurable for each
EcF.
We set P()(x,E) = 1 if x c E and = 0 if x .
XXVI. If P(k)(x,E) = 1 then we have
p(kj)(x, A (E)) = 1 1 j k .
We have
S= P(k)(x,E) = [ + ] P(y,E)P(k(x,dy) ,
A(E) ^AE)
where in the second integral the integrant is less than one. Hence the assertion
follows for j = 1 andthe general case then follows from this by induction on j
XXVII. We have for each j ,
A(E) = {x : P((x,E) = 1 .
The assertion is true for j = 1 iby definition. Assume for the sake of
induction that it is true for a certain j then if x c A+1 (E) = A(A(E)) we
have P(x, AJ(E)) = 1 and consequently
P(j+l)(x,E) = P()(y,E)P(x,dy) = 1 FP(x,dy) = 1
A ) A E)
by the induction hypothesis. Hence A=J+l(E) C { x : P(J+l)(x,E) = 1} by induc
5
tion. Conversely, if P(j+1)(x,E) = 1 then by XXVI we have P (x,Aj(E))= 1 ,
and so by definition x c A(Aj(E)) = AJ+ (E) .
Definition 6. A sequence of k sets Ej 1 j kj is said to form a
kcycle iff
Sc A(Ej+) 1 j ki ,
and
Ek C (El)
k
The union U E. will also be called the cycle when no confusion is likely and
j=l 3
each E. 1 7 j k a member of the cycle. The cycle is called clean iff the
E. 's are disjoint. (The word "disjoint" means always "pairwise disjoint.")
J
Note that in general the members of a cycle need not be distinct.
XXVIII. Each member of a kcycle is P(k) cl. andtbhe cycle itself is
cl. If E is P(k)cl., then the sequence Ak (E) 1 j k forms
a cycle.
It follows from XXVII that E C Ak(E) if and only if E is P(k)cl. Now
if E C F then L(E) C j(F) Hence by the definition of a cycle we have
.j c k(EjJ) C *** C kj()C kj+ 1) C C(S )
Thus each E. is P (k)cl. Furthermore we have
k k1 k
U Sj LE (E,) = ( t E)
j=1 3 jo= j+l j=1
hence the cycle is F(1)cl.
If E is P(k)cl,, then,
4
(EB) = g C k(s) = A ( k (E)) ,
Akj(E) = A(Aklj()) i j k;
and consequently k{ k'j(E) 1 k ) forms a cycle.
Definition 7. The cycle in the second part of Prop. XXVIII is said to be
generated by E .
XXIX. If E is P(k)cl. or P (kindecomp. or P(k)max. indecomp., then
so is Aj(E) for each j 0 .
It is sufficient to prove the assertion for j = 1, since the general case
then follows by iteration. Let F = A(E) If E is P(k)cl., then F is
A(E)_c . th. Fp(k)_cl F is
(k)
F ( cl. by the preceding proofC Next, suppose that E is P cl and F is
P(k)decomp.; we are going to show that E is P(k)decomp. There exist dis
joint P(k)cl. subsets F1 and F2 of F. Define
kl
S= (F ) n = 1, 2, .
n n
Then E1 and E2 are disjoint P(k)cl sets. If x Fn we have
1 = P(k)x, Fn) = P(k)(y,F) P(x,dy)
since P(x, E) = 1 by the definition of F Hence there exists a y c E with
P (k (y F ) = 1 and consequently y c E by definition. Thus E En / 0
n n n
for n = 1, 2. Each E ) E is P(k)6l. and so E is P(k)decomp. as was
n
to be shown.
Finally, suppose that E is P(k)max. indecomp. Then F is P(k)
indecomp. as just proved. Let F ce P()cl. and contain F properly. Define
k1 i
E = A (F) Then E E If x c F then P(x, E) = 1 by XXVI If
x F then P(x, E) < 1 by the definition of F Since F F is nonempty
we see by choosing an x in this difference that E contains $ properly.
5
Hence 9 is P(k)decomp. and so must be F by what has been proved. Therefore
F is P(k)max. indecomp,
Notation. If k1 and k2 are two positive integers, we write kljk2 iff
k1 is a divisor of k2 .
XXX. Letd. k A P(d)cl. set is P(k)cl. A P(k)cl. and P(k)indecomp.
set is P d)indecomp. A P(d)max. indecomp. and P(k)indecomp. set is
P(k)max. indecomp. A P d)cl. and P(k)max. indecomp. set is P (d)max. indecomp.
Without loss of generality we may suppose d = 1 since we may consider
P (d,)( ) in lieu of P(.,*) as the basic transition probability function. The
first two assertions are trivial.
Let E be P(1)max. indecomp. and P(k)indecomp. and let F be a
(k)
P(k)cl. set which contains E properly. We are going to show that F is
P(k)decomp. Let G be the kcycle generated by F Then G is P(1)cl. and
contain E properly. Hence G contains two disjoint P 1)cl. sets A and B.
If x c A then b y the property of a cycle we have P(j)(x, F) = 1 for some j ,
1 j k Since A is P(l)cl. this implies A ( F / 0 By the same token
B n F / 0 The two sets A r/ F and B n F are disjoint and P(k)cl. Hence
F is P(k)deccmp. as was to be shown.
To prove the last assertion in XXX let E be P(1)cl. and P(k)maxo
indecomp. Then E is P(1)indecomp. by the second assertion in XXX Let F
be P(l)cl. and contain E properly; we are going to show that F is P(1)
decomp. Since F is P(k)cl. it must be P(k)decomp. Let A and B be dis
joint, P(k)cl. sets contained in F Since E is P(k)indecomp. at least one
of A ,\ E and B A E9 is empty. Suppose A t E = 0 and let C be the kcycle
generated by A Then 0 ( E = 0 by the property of a cycle. Hence C and E
are disjoint, P(1)cl. sets contained in F and F is P(1)decomp. as was to
be shown.
4\A2% C C. 4 S" kP '< A ?)z ( Y, c I
AA C 0 .
npY A n C
6
In Prop. XXXI to XXXIV the state space X is assumed to be P )indecomp.
XXXI. There are at most k disjoint P(k)cl. sets Let m n ,
be disjoint, P(k)cl. sets. By XXVIII each of them generates a kcycle
(1) (1) n
C which is P cl. Since X is P indecomp., C = 1 C is nonempty,
m m=l m
m=1
Let x c C then by the property of a kcycle for each m 1 m n there
exists an integer jm 1 jm k such that P (x, Em) = 1 Since the
E 's are disjoint the j 's must be distinct. Therefdaec n k ,
XXXII. Each P(k)cl. set contains a P(k)indecomp. set and intersects a
P(k)max. indecomp. set. The number of distinct P(k)max. indecomp. sets is the
J.*;r`t (k)
maximum number of P cl. sets.
If there were a P(k)cl. set which does not contain any P(k)indecomp.
subset then the set itself is P(k)decomp. andhhence contains two disjoint
P(k)cl. sets each of which is P(k)decomp. Hence by induction there would be
an infinite number of P(k)cl. sets, contradicting XXXI Now by the P(k)
version of XV each P(k)indecomp. set is contained in a P(k)max. indecomp.
set; hence each I(k)cl. set intersects a P(k)max. indecomp, set. Two dis
joint P(k)cl. sets cannot intersect the same P(k)max. indecomp. set, proving
the second assertion.
XXXIII For each k let 6(k) be the number of distinct P(k)max.
indecomp. sets contained in X ; then 6(k)k These 6(k) sets form a clean
6(k) (k)_
cycle Ii 1 i 6(k)l X k I. does not contain any P(k)cl. sets
i=l
and is not abs. ess.
By XXXII there exists a P(k)max. indecomp. set I Set
Ii A (I) ,i .
By XXVIII Iki liik is the kcycle generated by I We have
I = C Ik But by XXIX each I. is P(k)max. indecomp. Hence by the
0 iC 1
7
P(k)version of XVI 1 = I and consequently I. = I. if i j (mod R) ,
Let d be the least positive integer such that I = Id Then I. / I. for
0 i < j d1 for otherwise we should have
ki ki kjI
I = k = A (li) = Ak (I) = Ak+ (E) . ,
S k = 1 0= o k+ji= ji
(k)
contradicting the definition of d By the P (k)version of XVI the I. ,
1
1 i d are disjoint and so form a clean cycle. We have now I. = I. if and
only if i = j (mod d) hence dfk .
This d is the 6(k) asserted in the proposition. For if there is a
P(k)max. indecomp. set J distinct from the I. +s then it is disjoint from
their union C s ,to
d
their union C =i Ii. As before, there is an integer e such that elk
and i A(7) 1 j e is a cycle. Let D = e A(J) Since X is
j=1
indecomp., C 0 D / 0 and consequently A (I) (' A3(J) / 0 for some i
and j But then A'(I) = AJ(J) because both sets are PF (max. indecomp. and
it follows that
J = Ak(J) Ak+ij
Thus J is one of the I. 's and therefore d = 6(k) .
1
By XXXII any P(k) cl. set must intersect one of the I. 's Hence
X C does not contain any P(k) l. set. Then X C is not abs. ess. by the
P(k)version of XIV and XXV .
Definition 8. 6(k) is called the cyclic index belonging to k and the
6(k)cycle described in XXXIII is called the cycle belong to k .
Notation. For two positive integers k and k' we denote their least com
mon multiple by k v/k' and their greatest common divisor by k A k' .
XXXIV. For arbitrary k and k' we have
(1) 6(k V k') 6(k) V 6(k')
(2) 6(k A k') = 6(k) A 6(kt)
Let D. 1 i 6(k) and f i 1 i 6(k')} be the cycles belonging
to k and k' respectively.
We show first that
(5) 6(k) > k A 6(k')
Writing = k A 6(k') and 6(k') = qd we set
q1
Fr U md+r
m=O
The sets IFr 1 r dj, are clearly disjoint and P cl., hence P (k)cl.
It follows from XXXII that there are at least d distinct P k)max, indecomp.
sets; hence 6(k) a d which is (3).
Next, we show that
(4) if klk' then 6(k) 6(k') .
&(k) 6(k') ()
Since X is indecomp., and J D. and E. are both P(1cl., we have
i=l i=l
Di E / 0 for some i and j By relabelling we may suppose that D1\ J10.
Define D. and E. for all i> 1 by setting D. = D. iff i 5 j (mod 8(k))
and E. = E. iff i j (mod 6(k)) Then it follows from the properties of
cycles that the sets D. ri E. 1 i (k) V 6(k') are disjoint and
1 1.
P (k k')cl., hence P(k')cl. if kIk' Hence
6(k) V 6(k') 6(k')
by XXXII and consequently (4) is true.
We can now prove that
if klk' then 6(k) = k A (k') .
9
For 6(k)k by .XXIII ; together with (4) this implies
6(k) k A6(k')
Together with (5) this implies (5).
Let k V k' = / then we have by (5):
6(k) = k A s(/) ,
6(k') = kA A s(3) .
Since 6(/)IJ it is a simple arithmetical fact that
(7) (k /\6()) V (k' A s(/)) =(k V k') ,A (/) = X s6() = 6(/) .
Substituting from (6) into (7) we obtain (1).
Finally, let k A k' = d ; then by (5):
6(d) = d A (6(k) AS(k'))
Since 6(k)k and 6(k') k' by XXXIII ($) reduces to (2).
XXXV. We have for an arbitrary k ,' 
(9) 6(6(k)) = 6(k)
and the cycle belonging to 5(k) coincides, member for member, with that
belonging to k .
writing d = 6(k) we observe that each I. in XXXIII is P(d)cl. and
1
F(k)max. indecomp. Hence it is d)max. indecomp. by the last assertion in
XXX Thus 6(d) d and since 6(d)jd we have 5(d) = d The rest follows.
The equation (9) also follows from (2) if we substitute 6(k) for k' there
and us,; the frct th.t 6(k)lk .
XXXVI. To each prime number p there corresponds an e which is either a
nonnegative integer or "infinite," such that
6(p") = pmin(n,ep)
for ea.ich n 1 .
finee e = e to be the least nonnegative integer such that 6(p ) e+
(6)
r/* ~7 i

10
or oo if such an integer does not exist, Then 6(pn) = p for 0 n e .
If e = co there is nothing more to prove Let e < oo and assume, for the sake
of induction that
(pn) e < <
6(p) = p for e n a
where a is an integer. By XXXIII we have (pa+) = p for some nonnegative
integer b and by (4) we have b e On the other hand we have by XXXV ,
6(p ) = (6(p+l)) = (pa+l) = p
Hence b e by the definition of e and the induction hypothesis. It follows
that b = e (pa) = p and the induction is complete.
XXXVII Let
f
k =TT P
be the primefactorization of k then
min( f, e )
6(k) =IT p
where e is as given in XXXVI .
This is an immediate consequence of XXXVI and equation (1) in XXXIV .
th
Definition 9. The set C in F is called a k consequent of x iff
P(k)(x, C) = 1 The sequence 0 Ck k 11 is called a consequent sequence of
x iff for each k 1 Ck is a kth consequent of x .
Each x has as a consequent sequence the sequence each term of which is X .
XXXVIII. Given a consequent sequence CGk k 1l of x there exists a
consequent sequence D k > 1 of x such that DkC Gk and Dk C A=(Dk+1)
Let
co
D = (C ) .
k = k+j
j=O
ii^ ~ J1.
110
Then D C A(Ck) = C k ; and
O .
(Dkj+l (Ck+l+j) 0 id
k+ += k+l+j) k+) k j
j=0 j=1
Since P(k+)(x, Ck+j) = 1 for each j 0 we have by XXVI ,
P(k)(x, AJ(Ck+j)) = 1
and consequently
c0
P(k)(x, .~ (C )) = 1
j=0
This proves that Dk is a k consequent of x for each k 1 .
Definition 10. For each x we define approbability measure x(*) as
follows: for each E c F,
00
x(E) = 1 p(n)(x, E) .
n=l 2"
It is clear that T () is a probability measure, and that T(E) = 0 if and
x X
only if L(x, E) = 0 or equivalently x c E
th I
Definition 11. A k consequent C is called minimal iff C is minimal
with respect to the measure [I namely iff there does not exist a kth con
x
sequent D with xf(D) < J~(C) A minimal consequent sequence is one in
which each member is minimal.
XXXIX. For each x and each consequence iCk k 1 of x
a minimal consequent sequence Dk k 1 such that Dk C k
There always exists a consequent sequence of x namely the
members of which are X. Writing for a moment C c C (x) iff
consequent of x we set
ak = inf
C C C (x)
, there exists
for each k 1.
sequence all
C is a kth
TFx(C)
12
Then there exists a C in C (x) with (C ) < a + Let
k,n =k x k,n k n
oo00 th
Dk = Ck. rn Ck,n ; then Dk is a k consequent of x and (Ck) = ak
n=l
Clearly I Dk k > l1J is a minimal consequent sequence and Dk C Ck for each
k 1 .
XXXX. In an indecomp. space X two minimal kth consequents of a given x
differ by a set which is not abs. ess.
Let Ck and Dk be two minimal kt consequent of x then I (Ck D )
= 0 and so by a previous remark (Ck Dk) 0 0 Consequently Ck Dk is
not abs. ess. by XXVII .
XXXXI. Let X be indecomp., x an arbitrary point of X and Cn nl
an arbitrary consequent sequence of x There exists a not abs. ess. set F
(depending on x ) and for each y c X F there exists a positive integer m(y)
such that {Cm(y)+n, 1 is a consequent sequence of y .
We have for each pair of integers m and n with m < n :
( Cnm) )P
1 = P(n)(x, ) = (nm)(y, )P()(x, dy) .
X
Hence there is a set F in F with P(m)(x, F ) = 0 and such that if
m,n =m,n
y c X F then
m,n
p(m)(y, C) = 1 .
00 )(
Let F U F Then P ((x, F ) = 0 ; and if y c X F the above
m m,n m m
n=m+l i o
equation holds for every n m+l Let F = r' F then F c F and
m=l
P(m)(x, F) = 0 for every m 1 Consequently F / 0 and F is not abs. ess.
by XVII If y c X F then there exists a positive integer m = m(y) such
that y c X F and p(k)(y, C ) = 1 for every k & 1. This proves the
proposition.
15
In propositions XXXXII to XXXXV the space X is assumed to be abs. ess.
and indecomp.
XXXXII. Let X be abs. ess. and indecomp. For each x there exists a
finite positive integer k(x) such that if Ck ,k k 1 is any consequent
sequence of x then there exist m and n both less than k(x) + 1 such that
C m C is abs. ess.
m n
It.is sufficient to prove this for a fixed minimal consequence Ok k l1
For then the conclusion will remain valid with the same m and n for any con
sequent sequence of x by XXXX Furthermore we may suppose on account of
XXXIX and XXXVIII that 0k C (Ck+l) Hence C = U 0k is cl. and con
k=l
sequently abs. ess. by XVIII Set
00
Dk = k (Ckr f k+j )
k j=
If y c Dk then F()(y, Ck+j) = 1 and hence P(J)(y, Dk) = 0 for each j 1 ,
since Dk ^ Ck+j = 0 Thus L(y, Dk) = 0 and Dk C Dk Such a Dk is
O0
clearly iness. and consequently D = Dk is not abs. ess. But
k=l
00 00
C D = U U (Ck Ck+j) .
k=l j=l
It follows that at least one Ck Ck+j id abs. ess., as was to be proved.
For each x let Ok(x) k > 1} be a minimal consequent sequence of x and
let
h(x) = min{(mn : Cm(x) r\ n(x) is abs. ess. .
,according to XXXXII h(x) < k(x) < oo, and h(x) is independent 6f the choice
6f the minimal consequence sequence.
XXXXIII. There exists an integer H and a set F which is in = and not
H
abs. ess. such that h(x) is equal to H for all x c X FH H
Let x be an arbitrary point and let Cn n 1 be a minimal consequent
14
sequence of x such that G C A (C ) for each n 1 and k 1 Such a
n n+k
choice is possible by XXXVIII Let h(x) = Z then by definition there exists
an integer j such that
C. 0i C. j is abs. ess.
Let y be in this intersection, then by the choice of C n we have for
each k 1 ,
p(k)(y, Cj+k ) j+k+) = 1
Hence by IX ,
C.j+k/ Cj. k+ is abs. ess., for each k 0 ,
or
(1) On J C n+ is abs. ess., for each n j .
According to XXXXI there exists a set F in F which is not abs. ess. such
that if y c X F then iCm(y)+n, n } is a consequent sequence (but not
necessarily minimal) of y for some m(y) 1 Hence we have h(y) f by the
definition of h() .
'We now prove that the function is bounded on X For otherwise let
xn n l 1 be points of X such that lim h(x ) = oo. Byr, what we have
n  oo
proved, for each x there exists a set F in F which is not abs. ess. and
n x =
n
such that
h(y) h(x ) if y c X F .
n
00
oo
Since X is abs. ess., X U F is not empty; and if y is in this set,
x
n=l n
h(y) would be oo which is impossible. Hence we may set
max h(x) = H < oo .
eettht2) on X FH as was to be proved.
A H
Remark. It has not been shown the function h is Fmeasurable or not,
wbf+l('r
15
but this information will not be needed below.
Definition 12. The integer H is called the overlapping index, an the set
F
X FH (in/ )tho overlapping core of the abs. ess. and indecomp. space X
XXXXIII. For each x in X FH there exists an integer v(x) such that
for an arbitrary consequent sequence iO n n 1 of x ,
Cn 0n+H is abs. ess. for n v(x) .
This is merely a restatement of (1) in XXXXIII .
XXXXIV, For each k 6(k)H Consider the cycle ii 1 i < 6(k)
/ 6(k)
belonging to k and/set I. = I. if i s j (mod 6(k)) Then C = U I. is
il 2i=l
abs. ess. by XXXIII since X is abs. ess. (cf. XIX). If x e0 o (X FH) ,
S then f I i /1 is a consequent sequence of x and we have by XXXXIII,
c
Ii ) Ii+H / 0
for some i But the cycle is clean according to XXXIII hence 6(k)IH ,
Definition 15. Let D = max 6(k) ; D is called the maximum cyclic indet
ce (n D k> I
and the cycle elongrg to D is called the maximum cycle.
We have DIH according to XXXXIV It is not known whether D = H in
general; this is the case where X is countable.
XXXXV In the notation of XXXVI we have
Sep >
where 0 e < oo for each prime p' Furthermore, we have for each k 1 ,
p
(1) 6(k) = kAD
This is immediate from XXXVI XXXVII and XXXXIV A more direct proof of
(1) is as follows. Let 6(k') = D then by (5) of' XXXIV ,
1 4
S in)< //=~
V/,i f < 
el / c
16
(2) 6( kAD .
On the other hand, by (1) of XXXIV ,
6(k V k') = 6(k) V D
hence 6(k)ID by the definition of D Since 6(k)k it follows that 6(k)kA D
and so there must be equality in (2).
Example 1. X= 1, 2, 3, 4, 5 ) .
P(n, n+l) = 1 for n = 1, 2, 5 ; P(4,1) 1 ; P(5,1) = P(5,2) = .
Each fn n = 1, 2, 5, 4, is P(4)max. indecomp.; {l,3} and {2,4} are
P(2)indecomp., but {2,4) is not P(2)max. indecomp. since {2, 4, 5j is. This
example shows that the cycle belonging to a divisor of k is not necessarily
obtained by the obvious grouping from the cycle belonging to k
Example 2. X = 1, 2, 5, 4, 5, 6, 7, 8 .
P(l, 5) = P(l, 6) =
P(2, 5) = P(2, 6) = F(2, 7) = P(2, 8) =
P(5, 7) = P(4, 8) = P(5, 3) = P(6, 4) = P(7, 1) = P(8, 2) = 1 .
Here the maximum index D = 2 and the maximum cycle is composed of
{1, 2, 5, 4} and {5, 6, 7, 8 It is easily verified that H = 2 and FH .
The minimal consequent sequence for f6j is
S6 } 41 8} 2 {5,6,7,8} {1,2,5,4 5,6,7,8J, ***
If we denote this sequence of sets by Cn n Oj it is to be noted that
C1 COn = 0 for n = 2, 3, 4 but C2 n 4 / 0
Example 5. X = 1 2, 3, 4, 5, 6, 7, 8 .
P(l, 2) = P(1, 4) = P(3, 4) = P(3, 6) ;
P(2, 5) = P(4, 5) = P(5, 6) = P(6, 7) = P(7, 8) = P(8, 1) = 1
The minimal consequent sequence for { 11 is:
17
r
{il {2, 4j, 54 {4, 6 {5, 7} t6, 8 7, 1 ,
12, 4, 8) 1, 5~, 5 2, 4, 6) 3, 5, 7} ,(4, 6, 8} 5, 7, 1 ,
12, 4, 6, 8) (1, 3, 5, 7) { 2, 4, 6, 8 ***
Here in notation similar to the above: C1 / 3 / 0 C3 r) 0C 5 0 but
C, n1 C = 0 .
