Research report, Doeblin's Theory of Markov Processes, 1957


Material Information

Research report, Doeblin's Theory of Markov Processes, 1957
Physical Description:
Chung, Kai Lai
Physical Location:
Box: 1
Folder: Research report, Doeblin's Theory of Markov Processes, 1957


Subjects / Keywords:
Mathematics -- History -- 20th century

Record Information

Source Institution:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
System ID:

This item is only available as the following downloads:

Full Text


AFOSR TN 57-648

ASTIA Document No. AD 136 634

Doeblin's theory of
Markov processes I

K. L. Chung

Syracuse University

Research Report 14

Contract No. AF 18 (600) 760

File No. 3.5

September 1957

Mathematics Division

Air Force Office of Scientific Research



K. L. Chung

For the general set-up of a Markov process with discrete time
parameter, general state space and stationary transition probabilities
we refer to Doob [2; Ch. 5 5]. Our study is based on the most general
theory developed by Doeblin [1]. A version of this work was previously
given by the author in lecture notes at Columbia University in 1951.
The present attempt aims at a better synthesis with a number of new
results added. It is self-contained and will be given in several parts
of which this is the first instalment.

Among the notation to be used we note the following-

X is the state space; x, y, z are points of X; E, F, C, D are sets
of a given Borel field of subsets of X; E = X E.

P is the underlying probability measure. The Markov process is
{tn' n > 0}. {rn E}, for example, is the event that n belongs

to E. The generic sample point is w, omitted when dispensable.

p(n)(x, E) = P{m+n E |m = x} is the n-step transition probability

frox the point x to the set E; p(l) is abbreviated to p which is
assumed to be measurable in x for each E and a measure in E for each x.
L(x, E) = P( [n E] m = x .

Q(x, E) = P{limsup [n c ] m = x}.

Q(x, E; F) = Ptlimsup [n c E]n limsup [fn F]|m = x}.

Simple measurability questions will be passed muster. Results will
be recorded under Roman numerals for reference. After each recorded
result the proof is given in the following paragraph.

For an arbitrary E we define four sets:

E = (x: L(x, E) = 0}

El = {x:

f =
E = Ix:

L(x, E) = 1}

Q(x, E) = 0}

Eo= {x: Q(x, E) = 1}.

Definition 1. A non-empty set E such that p(x, E) = 1 for every x c E
is called stochastically closed (cl.)

I. If x c E, then p(x, E 0) = 1.
either both empty or both cl.

We have:

The sets Eo and E E are

0 = L(x, E) =


p(x, dy) +


L(y, E) p(x, dy)

+ I L(y, E) p(x,dy)


where the integrand in the last integral is positive.

Hence: EU EF


= 0,

E l p(x, E E) = i.

II. If x E E1, then p(x, ElUE) = 1. If E is cl'., so is E1

$o E

We have!

1 = L(x, E) = p(x, E) + p(x, E1 E) + L(y, E) p(x, dy)

E1 E

where the integrand in the last integral is less than 1.

p(x, E1 ) = 0.

& K EC A_ J

p(x, ElE) =1.



III. EE is cl. or empty.

Let x c E ; then

0 Q (x, E) = Q(y, E) p(x, dy) a


The integrand in the last integral is

IV. Eco is cl. or empty.

Q(y, E) p(x, dy) > 0.


positive; hence p(x, E ) = 0

Let x c E then

1 = Q(x, E) =

Q(y, E) p(x, dy) +

SQ(y, E) p(x, dy)


The integrand in the last integral is

V. If E = U E, then Eo = nEO
n n n n

less than 1; Hence p(x, E 0) = 0.

J 4

Clearly Eo e E so that EO C 0/ EO On the other hand if
n n n

x QF En, then for every n we have L(x, En) = 0;


L(x, E) = L(x,

and x & EO.

Thus (f Eo E.
n n -

Remark. Anplausible, but false identity is (E )1 = E .


1 1
X = {xn, n > 0 p(x, xo) = 1, p(xl' xo) = 9 p(x1, x2 29

(x, Xn+I) = 1 -, n, x1) = n > 2 E = {xl}.
n n


E0 = {Xo, }, E

= X. We have, L(x1, E) 1

r- /* ^..~

n (1

- ) <

Or o

L- ^[ (X\ -

;tM33 ~1\~ ~tf.

JE ) < 2 L(x, E ) = 0
n n n n

O) xtr/ti-tC*vtv/^
LCt^-JQ 'X. <^~
/ik^r b-^^^XS

' 2

Xc" C7

I -:I-

and if n > 2, L(xn, E) = L(x xl) = 1 (1 ) < 1
k=n k

Hence: (E0) = {Xo}.

Definition 2. A set E such that Q(x, E) = 0 for every x c X is called
inessential (iness); otherwise essential (ess.) &n essential set which
is the denumberable union of inessential sets is called improperly
essential (imp. ess.); otherwise absolutely essential (abs. ess.).

The next two theorems are basic for the sequel. VI is given by
Blackwell [3], VII in the lecture notes mentioned in the beginning
paragraph. Both were proved by simple, direct arguments. Here they
will be proved by the martingale convergence theorem.

VI. If sup Q(x, F) < 1

then Q(x, E/ F) = 0 for every x: X.

A E = limsup n E

For any E the eventAE is invariant under the shift n n+l'

We have, with a Baire function )n'

P{AE AFIo' "...," n P{AEAFIn = n ()

By the martingale theorem [2;,p331]

(1 if w A/\E AF
(I) lira l(Wn (w)) = othEri
n-ico Yn (0 otherwise

for almost all w. Because of the invariance we have

P AE AF~ln =x = PE AFIo(V =

so that all on F (The conditional probabilities are taken to
4Y -< I 4-r t

be the versions uniquely determined by p(x, E); see [21 p 191].) Now
if w c0AE, then there is a sequence tnk} going to infinity such that

Q(x, F < 1.

This is consistent
this means Q(x, E;

with (1) only if P(/E A
F) = 0.

F) = 0. If we set o = x

VII. If inf L(x, F) > 0

fR I Att then Q(x, E) = Q(x, E> F) for every x c X.

Let Mn = k c F1, and

for a fixed N > 0,

Pt/NE MNI o', = nn

Then for almost all w,

lim ( ) = (0
nr+O n in(0

if w CA.E N

We have if n > N,

n(x) > P{/\EMI O. = j

= PA\E 1 ^n = x~ = P /E IV/ = x4 > L(x, F).

It follows as in the preceding proof, if nk > N,

nk (nk (wo ) _> gi L(x, F) > 0.

This is consistent with (2) only if P(/E.-AE MN) = 0. This

being time for every N we have:

Q(x, E; F) == imP(AE MNI = x ) = AE = x) = Q(x, E).
n-*oo EMNIo=x FOI o=)


(nk ( ))

r /d / .^7 -2 I

-p 7 2

~3Zc-4 S *


CL"- 22 > aO


7/o^l /? / /,, > 7 -? z 4/- .- r., 7 a ,

i // f / -,--, -- j 1/, / 1

J-/,.-/ ,- T
/1L h / l f, .f7- ^ J =^ .A / ?., 7 --^ J ,z

i LCL / 1 / ?A. 7 I /f L)

7/-/-L ( () ) --"-> O ai e. /t --

~r4M ~~7
.7~1~7 k

I'-/y^ --

L~~ L~/ cto L1 c ~70


VIII. If E is ess. and in L(x, F) > 0 then F is ess.

Since E is ess. there is an x such that Q(x, E) > 0.

By VII, Q(x, F) Z Q(x, E0 F) = Q(x, E) > 0. Hence F is ess.

IX. If E is abs. ess. and inf L(x, F) > 0 then F is abs. ess.

It is sufficient to prove that whenever F = U Fk there
k=l )
exists an no such that 19 Fk is ess. We note the simple


L(x, F) = lim L(x, UL Fk).
n+oo k=l

Let x c E, L(x, F) o o( O; hence there exists a finite mo(x)

such that:

L(x, Fk) > > 0.
Let E = {xcE: m (x) = n}, then E =Z2 E Since E is abs. ess.
n o n;;j n

there exists an n such that E is ess. By the definition of
0 n

E we have:
0o n
0 ~
inf L(x, (J Fk)> /- o

Hence by VIII, U( Fk is ess.

X. If there exists a set F such that

sup Q(x, F) < 1, inf L(x, F) > 0,

then E is iness.

For every x we have by VI and VII, Q(x, E) = Q(x, EA F) = 0.
Hence E is iness.

XI. -Cf Xis abs. es~g.. andIiL .rOthen E is.

Let; >
E = {x L(x, E)

then we have
X--= E--+.- -. E +-E-.- U nF
1I ? L ,' I L(.,( n=2 v = i
i 4 i)
Since Q(x, E) L(x, E), eachE is iness, by X and so their
union is not abs. ess. It follows that El is abs. ess. Now

infl L(x, E) 1,....-- .

rhenee E is- abs-. ess. by IX.i

S,, 4 .. .XII-. If E is abs. ess. then for any F the set

L, E(FO + F)

is abs. ess. in particular not empty,

SE = x: Q(x, F) 1 L(x, F) 1
n n -n

It is clear that

2 E = } EE + E(F + FC)

S' Each EE is iness. by X, since E is abs. ess. it follows that
E(Fo + Fj is abs. ess.

XIII. If E is abs. ess., then EEcO is abs. ess; in particular Eo0 / 0.

Applying XITwith F = I we see that EE0 + EEO is abs. ess. Now

EEO is obviously iness., hence EEco is abs. ess.

^ C -7 C o i 0 -u-" kC] CcA- i ,*
1 _8.
XIV. If C is cl., and C does not contain any cl. set, then C is not,
abs. ess. In pr T-F~culrO + Co is not abs. ess.

rs Since C is cl., any point in (C) must by definition belong to C.
S. 4 .. But (C)oo is cl. if not empty by IV, hence (5)oo = 0 and consequently
)C j ) -' C-J C is not abs. ess. by XIII. To prove the second assertion we
dR C,-tAf observe that any cl. set contained in C must be contained in Co,
CL hence C + Co does not contain any cl. set.

Definition 3. A cl. set which does not contain two disjoint cl. sets. is
S-C P 0 called indecomposable (indecomp.); otherwise decomposable (decomp.) An
indecomposable set which is not properly contained in any indecomposable
-Z,; set is called maximal indecomposable (max. indecomp.).

XV. If E i/ indecomp., then (Eo)is max. indecomp.

Suppose it is decomp.; let C and D be two disjoint cl. sets contained
in it. For any x c C we have x E since Eo (E0)0 = 0; hence
L(x, E) > 0. Since C is cl. this implies that CE / 0. Similarly
DE / 0. Thus CE and DE are disjoint cl. sets contained in E and E
is decomp. We have thus proved that if E is indecomp., so is (E 0).
Now suppose F is cl. and containS(EO) properly. Let x e F (EO),
then L(x, Eo) > 0. Since Eo is cl. this implies FEof 0 and so is cl. Thus
F contains the disjoint cl. set E and FEo and is decomp.
We have thus proved that any cl. set properly containing (Eo) is
decomp. Hence (E O) is max. indecomp.

XVI. Two max. indecomp. sets are either identical or disjoint.

Let E and F1-e two instinctt indecomp. sets. The E U F is cl. and
contains .both properly. Hence it is decomp. and contains two disjoint
cl. sets C and D. Since E is indecomp. at least one of EC and ED is
empty. Suppose EC is empty. Then F > C and since F also contains EF
which is either cl. or.empty we must have EF = 0 since F is indecomp.

XVII. If X is indecomp. and E is abs. ess., then EO = 0.

By XIII we have Eco / 0. By I and IV, EO and Eo are cl. if not empty.
Since E E = 0 and X is indecomp. we must have E = 0.

XVIII. If X is abs. ess. and indecomp., then every sequence of cl. sets has
a cl. and abs. ess. intersection.


Let (Cn, n > 1} be a finite or infinite sequence of cl. sets.

Then D =

() Ck is not empty since X is indecomp. We have

x = 5 U (nf D ).
n n n n

Each D being cl., t is not abs. ess. by X and so is the
n n

union U D Since X is abs. ess. it follows that C( D is abs.
n n n n

ess. In particular it is not empty and so is cl.

Remark. In an indecomp. but imp. ess. X it is possible that every
sequence of cl. sets has a non-empty intersection. Take any
denumerable X which forms one nonrecurrent class (of a Markov chain)
so that the only cl. set is X itself.

XIX. In an abs. ess. and indecomp. space X an abs. ess. set E is
characterized by any one of the following properties:

E = 0, E / 0, Ef = 0,

Fhat E = 0 is characteristic follows from XI and XVII.. Next, each
of the three sets E0, E o, E is either cl. or empty, by I, IV, III.
Applying XIgwith X, E for the E, F there we see that at least one
of the two sets EO and E is not empty. Hence exactly one is not
empty since Eo E00 = 0 and X is indecomp. Hence E0 / 0 is
equivalent to Eo = 0. Finally since EO E f Ef = 0 implies EO = 0;
on the other hand since E E00 = 0, Ef 0 implies Eco = 0OO.-C

Remark. Let us use the symbols "-*" and "4-" to denote "implies" and
"does not imply" respectively; and the symbol "A" to denote 'E is
abs. ess." The complete situation is as follows (where + stands the
required example is trivial):


Abs. ess. and
Arbitrary X Abs. ess. X Indecomp. X idec
indecomp. X

SA EO = 0

4 j Ef = 0

E0/ 0 A
E- = 0 A
E = 0 + A

A-+E = 0
A E = 0

Ec / 0 A
E = 0 A
Ef = 0 A


E OD -+ A

.i~ i ~-"


XX. If X is indecomp. and E is abs. ess. then

3) E p(n) (x, E)

is infinite for every x c X. If X is abs. ess. and (3) is positive
for every x c X, then E is abs. ess.

If (3) is finite for some x, then by the Borel-Cantelli lemma
Q(x, E) = 0 so the Ef / 0. Since X is indecomp., Ec = 0.
Hence E is not abs. ess. by XIII. If (3) is positive for every x,
then E = 0. Hence E is abs. ess. if X is, by XI.

every x
of {Yn,

("--;--- 4--.

It is possible in an indecomp. X that (3) is infinite for
but E is iness. Consider the following example: X consists
n >1) and x nk 1 < k < n, n 1}.

p(yl, 2) = 2;

p(Yn Yn+) = 1 PI

P(y'1 Xnl) = Pn 3
t n

It is clear

p(y, y1) -, n 2;

^,~ ~ {'(, ^ '2
-""G '7 "

P(xnk, xn, k+l) = 1, 1 < k < n;

P(Xnn, y1) = 1

that X forms one nonrecurrent class.

E-= {Xnk, 1 < k < n, n > 1).

We have:

p() (Y1, E) >

CO ok
k=n p 2 k2
k=n Pk2 ^n k2

so that (3) is infinite for x = y. Since L(x, y ) > 0 for every x
it follows that (3) is infinite for every x. To see that E is iness.
we verify easily that inf L(x, x) = p > 0, s Q(x, xll) >' and

and apply ?



~h"~ IL jl~l. X'I C)

Definition 4. & set E such that Q(x, E) < 1 for some x c X is

called habtitbe (B" P

L(x,G) < I

XXI. If E isM&- it is ess.
For Q(x, E) < 1 implies Q(x, E)> 0, since Q(x, E) + Q(x, E) 1 1.

XXII. If C is cl. and C is ess., then C is I

Since 6 is ess. there exists an x such that Q(x, O) > 0.

Since C

is cl. this implies that Q(x, C)jl Hence C is sawr.

XXIII. &ny imp. ess. set is contained in a(B L and imp. ess. set.
If X is imp. ess. the proposition is trivial. Now suppose X is abs.
ess. and E is imp. ess. Then Eo / 0 by XI., and E E is cl. by I.
The set# E J .. .
fo.~t ~h O


,, I P 3 p
is not abs. ess. 4 It contains E and is ~b by XXII.

If E is q' then u ^ E

f RIU ^4-b C

inf L(x, E) = 0.

We have for any E,

1 = Q(x, E + E) = Q(x, E) + Q(x, E) Q- (x, E( E)

If inf L(x, E) > 0 then by VII we have Q(x, E) = Q(x, E{ E) so
that the above identity implies Q(x, E) = 1 for every xcX.
Thus E is not gggq S c

4`1- I

cFC (

S5) 3 K(-


k) A. t14 I C', .- ^, -.4

CP1 C' '4 L-14 L -1A 1(


W. Doeblin;

J. L. Doob:

D. Blackwell:

E16ments d'une theorie general des
chatnes simples constantes de Markoff.
&nn. Sci. Ecole Norm Sup. (3) Vol. 57
Fasc. 2. (1940). pp 61-111.

Stochastic Processes, Wiley and Sons.

The existence of normal chains, Bull.
Amer. Math. Soc. Vol. 51 (1945), pp.


Bibliographical Control Sheet

1. Originating agency and monitoring agency

0. A.: Syracuse University, Syracuse, New York
i.. A. :Aathematics Division, Office of Scientific Research

2. Originating agency and/or monitoring agency report number:

0. A.; Report No. 14
M. A.: AFOSR TN 57-648

3. Title and classification of title: Doeblin's theory of :Aarkov

4. Personal author(s)s K. L. Chung

5. Date of report: September, 1957

6. Pages: 12

7. Illustrative material: None

8. Prepared for Contract No.: AF 18 (600) 760

9. Prepared for Project File No. 5 3.5

10. Security Classification: Unclassified

11. Distribution Limitations: None

12. Abstract: This report is the first installment of a general theory of
iMaarkov processes based on the work of Doeblin.


Headquarters Air University
Maxwell Air Force Base,

Applied Mathematics and Stat:
Stanford University
Stanford, California

The Rand Corporation
Technical Library
1700 Main Street
Santa Monica, California

Director of Advanced Studies
Air Force Office of Scientif:
Post Office Box 2035
Pasadena 2, California

Department of Mathematics
Yale University
New Haven, Connecticut

Human Factors Operations Resi
Air Research and Development
Bolling Air Force Base
Washington 25, D.C.

Department of Commerce
Office of Technical Services
Washington 25, D.C.

National Bureau of Standards
Washington 25, D.C.

Headquarters, USAF
Director of Operations
ATTNs Operations Analysis
Division, AFOOP
Washington 25, D.O.

1 MARCH 1967

European Office, ARDC
60 Rue Ravenstein
Brussels, Belgium

Department of Mathematics
University of California
Berkeley, California

Air Force Flight Test Center
ATTN: Technical Library
Edwards Air Force Base,

Western'Development Division
E. 0. Box 262
Inglewood, California

Chief of Naval Operations
Department of the Navy
ATTN: OPO3EG; Mr. Arthur Kaufman
Washington 25, D.C.

Office of Naval Research
Department of the Navy
ATTN: Code 432
Washington 25, D.C.

Director National Security Agency
ATTN: Dr. H.H. Campaign
Washington 25, D.C.

National Applied Mathematics
National Bureau of Standards
Washington 25, D.C.

Commander 2
Air Force Office of Scientific Research
Washington 25, D.C.

Commander,Air Force Office of 2.
Scientific Research
Washington 25, D.C.

Air Force Armament Center
ATTN: Technical Library
Eglin Air Force Base, Florida

Department of Mathematics
Northwestern University
Evanston, Illinois

Department of Mathematics
University of Chicago
Chicago 37, Illinois

Department of Mathematics
Purdue University
Lafayetto, Indiana

Mathematics and Physics Library
The John Hopkins University
Baltimore, Maryland

Department of Mathematics
Harvard University
Cambridge 38, Massachusetts

Department of Mathematics
Massachusetts Institute of
Cambridge, Massachusetts

Air Force Cambridge Research Center
ATTN: Electronic Research Library
L. G. Hanscom Field
Bedford, Massachusetts

Willow Run Research Center
University of Michigan
Upsilanti, Michigan

Department of Mathematics,
Institute of Technology
Engineering Building
University of MAinnesota
Minneapolis, Minnesota

Department of Mathematics
University of Missouri
Columbia, Missouri

National Science Foundation
1520 "'tP St., N.W.
Washington 25, D.C.

Air Force Missile Test Center
ATTN: Technical Library
Patrick Air Force Base, Florida

Institute for Air Weapons Research
Museum of Science and Industry
University of Chicago
Chicago 37, Illinois

Department of Iathematics
University of Illinois
Urbana, Illinois

Institute for Fluid Dynamics and
Applied Mathematics
University of Maryland
College Park, Maryland

Headquarters Air Research and
Development Command
ATTN- Technical Library
P.O. Box 1395
Baltimore 3, Maryland

Air Force Cambridge Research Center
ATTN: Geophysics Research Library
L. G. Hanscom Field
Bedford, Massachusetts

Department of Mathematics
Wayne University
ATTN: Dr. Y. W. Chen
Dotroit 1, Michigan

Department of Mathematics
Folwell Hall
University of Minnesota
Minneapolis, Minnesota

Department of 'Mathematics
Washington University
St. Louis 5, 1:issouri

Linda Hall Library
ATTN: M!r. Thomas Gillis
Document Division
5109 Cherry Street
Kansas City 10, Missouri

Strategic Air Command
ATTN: Operations Analysis
Offutt Air Force Base
Omaha, Nobraska

Institute for Advanced Study
Princeton, New Jersey

Holloman Air Development Center
ATTNS Technical Library
Holloman Air Force Base,
New Mlexico

Professor J. Wolfowitz
Mathematics Department
White Hall
Cornell University
Ithaca, New York

Mathematics Research Group
New York University
ATTN: Professor I'. Kline
45 Astor Place
New York, New York

Department of Mathematical Stat
Fayerweather Hall
ATTN: Dr. Herbert Robbins
Columbia University
New York 27, New York

Institute for Aeronautical
ATTN: Librarian
2 East 64th Street
New York 21, New York

Department of Mathematics
Duke University
Duke Station
Durham, North Carolina

Air Technical Intelligence
Wright-Pattorson Air Force Base

USAF Institute of Technology
ATTN: Technical Library, MCLI
Wright-Patterson Air Force Base


The James Forrestall Research Center
Princeton University
Princeton, New Jersey

Department of Mathematics
Fine Hall
Princeton University
Princeton, New Jersey

Air Force Special Weapons Center
ATTN: Technical Library
Kirtland Air Force Base
Albuquerque, New Mexico

Department of Mathematics
Syracuse University
Syracuse, New York

Department of Mathematics
Columbia University
ATTNs Professor B. 0. Koopman
New York 27, New York

ATTNs Technical Library
Rome Air Development Center
Griffiss Air Force Base
Rome, New York

Institute of Statistics
North Carolina State College of A & E
Raleigh, North Carolina

Department of Mathematics
University of North Carolina
Chapel Hill, North Carolina

Office of Ordnance Research 2
Box CM
Duke Station
Durham, North Carolina

Wright Air Development Center
ATTN: Technical Library
Wright-Patterson Air Force Base

Wright Air Development Center
2 ATTN: ARL Technical Library, WCRR
Wright-Patterson Air Force Base

Chief, Document Service Center 10
Armed Services Technical
Information Agency
Knott Building
Dayton 2, Ohio

Arnold Engineering Development
ATTN: Technical Library
Tullahoma, Tennessee

Department of Mathematics
Rice Institute
Houston, Texas

Department of Mathematics
University of Wisconsin
Mladison, Wisconsin

Department of Mathematics
Carnegie Institute of Technology
Pittsburgh, Pennsylvania

Department of Mathematics
University of Pennsylvania
Philadelphia, Pennsylvania

Defence Research Laboratory
University of Texas
A us-tin, Texas

Air Force Personnel and Training
Research Center
Ladapand Air Force Base
San Antonio, Texas

Mathematics Research Center
AT'IN R,, E. Langer
University of Wisconsin
Madison, Wisconsin

Doeblin's theory of Markov processes, II

by K. L. Chung

This is a sequel to Part I under the same title (AFOSR TN 57-648). The same

notation as there will be used except that the transition probability function will

be denoted by P rather than p The basic Borel field of the state space X

will be denoted by F In what follows k is a positive integer.

The properties of a set in F such as "closed" and "essential" were defined

with reference to the basic transition probability function P(*,') Ifithe

latter is replaced by its kth iterate P(k)(.,.) then the corresponding property

will be prefixed by "(k)_. Thus the previously defined concepts are the

P(1)- versions, with the prefix "p(1)-" omitted from the terminology. The

results we have proved so far have their P(k) versions which need no new

proofs. In terms of the process, we shall be considering i nk+r n } fcr

a fixed k and r in lieu of n n 0 .

3 (k) (k) (k)
XXV. A set i P(k) iness., p(k) imp. ess., or P abs. ess.

according as it is iness., imply ess., or abs. ess.

If a set is iness., it is clearly P~k)_ iness. If E is ess. there

exists an x c Z such that Q(x,S) > 0 Then for each k there exists an r ,

1 i r k such that

P c nk+r E for infinitely many values of n = x > 0 .
= ) nk+r 0

Hence there exists a y c E and an integer n0 such that

P '~ c for infinitely many values of n jk = y > 0.
T = hnk+r show> kn +r

This shows that E is p(k)_ uss. The other assertions follow easily.

Definition 5, For an arbitrary set E in F we set


A(s) = x : P(x, E) = 1 }

Let A (E) = E Al(E) = A(E) and define A'(E) for each j 2 1 by

AJ() = A(-1(E)) .

AJ() is called the jth antecedent of .
We have A(E) c F for any E c F since P(.,E) is F-measurable for each
We set P()(x,E) = 1 if x c E and = 0 if x .

XXVI. If P(k)(x,E) = 1 then we have

p(k-j)(x, A (E)) = 1 1 j k .

We have

S= P(k)(x,E) = [ + ] P(y,E)P(k-(x,dy) ,
A(E) ^-A-E)
where in the second integral the integrant is less than one. Hence the assertion
follows for j = 1 andthe general case then follows from this by induction on j

XXVII. We have for each j ,

A(E) = {x : P((x,E) = 1 .

The assertion is true for j = 1 iby definition. Assume for the sake of
induction that it is true for a certain j then if x c A+1 (E) = A(A(E)) we
have P(x, AJ(E)) = 1 and consequently

P(j+l)(x,E) = P()(y,E)P(x,dy) = 1 FP(x,dy) = 1
A ) A E)
by the induction hypothesis. Hence A=J+l(E) C { x : P(J+l)(x,E) = 1} by induc-


tion. Conversely, if P(j+1)(x,E) = 1 then by XXVI we have P (x,Aj(E))= 1 ,

and so by definition x c A(Aj(E)) = AJ+ (E) .

Definition 6. A sequence of k sets Ej 1 j kj is said to form a

k-cycle iff

Sc A(Ej+) 1 j k-i ,


Ek C (El)
The union U E. will also be called the cycle when no confusion is likely and
j=l 3
each E. 1 7 j k a member of the cycle. The cycle is called clean iff the

E. 's are disjoint. (The word "disjoint" means always "pairwise disjoint.")
Note that in general the members of a cycle need not be distinct.

XXVIII. Each member of a k-cycle is P(k)- cl. andtbhe cycle itself is

-cl. If E is P(k)-cl., then the sequence Ak- (E) 1 j k forms

a cycle.

It follows from XXVII that E C Ak(E) if and only if E is P(k)-cl. Now

if E C F then L(E) C j(F) Hence by the definition of a cycle we have

.j c k(EjJ) C *** C k-j()C kj+ 1) C C(S )

Thus each E. is P (k)-cl. Furthermore we have

k k-1 k
U Sj LE (E,-) = ( t E)
j=1 3 jo= j+l j=1

hence the cycle is F(1)-cl.

If E is P(k)-cl,, then,


(EB) = g C k(s) = A ( k- (E)) ,

Akj(E) = A(Aklj-()) i j k;

and consequently k{ k'j(E) 1 k ) forms a cycle.

Definition 7. The cycle in the second part of Prop. XXVIII is said to be

generated by E .

XXIX. If E is P(k)-cl. or P (k-indecomp. or P(k)-max. indecomp., then

so is Aj(E) for each j 0 .

It is sufficient to prove the assertion for j = 1, since the general case

then follows by iteration. Let F = A(E) If E is P(k)-cl., then F is
A(E)_-c .- th. Fp(k)_cl F is
F ( cl. by the preceding proofC Next, suppose that E is P cl and F is

P(k)-decomp.; we are going to show that E is P(k)-decomp. There exist dis-

joint P(k)-cl. subsets F1 and F2 of F. Define

S= (F ) n = 1, 2, .
n n

Then E1 and E2 are disjoint P(k)-cl sets. If x Fn we have

1 = P(k)x, Fn) = P(k)(y,F) P(x,dy)

since P(x, E) = 1 by the definition of F Hence there exists a y c E with

P (k (y F ) = 1 and consequently y c E by definition. Thus E En / 0
n n n
for n = 1, 2. Each E -) E is P(k)-6l. and so E is P(k)-decomp. as was
to be shown.

Finally, suppose that E is P(k)-max. indecomp. Then F is P(k)

indecomp. as just proved. Let F ce P()-cl. and contain F properly. Define
k-1 i
E = A (F) Then E E If x c F then P(x, E) = 1 by XXVI If

x F then P(x, E) < 1 by the definition of F Since F F is nonempty

we see by choosing an x in this difference that E contains $ properly.


Hence 9 is P(k)-decomp. and so must be F by what has been proved. Therefore

F is P(k)-max. indecomp,

Notation. If k1 and k2 are two positive integers, we write kljk2 iff

k1 is a divisor of k2 .

XXX. Letd.- k A P(d)-cl. set is P(k)-cl. A P(k)-cl. and P(k)-indecomp.

set is P d)-indecomp. A P(d)-max. indecomp. and P(k)-indecomp. set is

P(k)-max. indecomp. A P d)-cl. and P(k)-max. indecomp. set is P (d)-max. indecomp.

Without loss of generality we may suppose d = 1 since we may consider

P (d,)( ) in lieu of P(.,*) as the basic transition probability function. The

first two assertions are trivial.

Let E be P(1)-max. indecomp. and P(k)-indecomp. and let F be a
P(k)-cl. set which contains E properly. We are going to show that F is

P(k)-decomp. Let G be the k-cycle generated by F Then G is P(1)-cl. and

contain E properly. Hence G contains two disjoint P 1)-cl. sets A and B.

If x c A then b y the property of a cycle we have P(j)(x, F) = 1 for some j ,

1 j k Since A is P(l)-cl. this implies A ( F / 0 By the same token

B n F / 0 The two sets A r/ F and B n F are disjoint and P(k)-cl. Hence

F is P(k)-deccmp. as was to be shown.

To prove the last assertion in XXX let E be P(1)-cl. and P(k)-maxo

indecomp. Then E is P(1)-indecomp. by the second assertion in XXX Let F

be P(l)-cl. and contain E properly; we are going to show that F is P(1)

decomp. Since F is P(k)-cl. it must be P(k)-decomp. Let A and B be dis-

joint, P(k)-cl. sets contained in F Since E is P(k)-indecomp. at least one

of A ,\ E and B A E9 is empty. Suppose A t E = 0 and let C be the k-cycle

generated by A Then 0 (- E = 0 by the property of a cycle. Hence C and E

are disjoint, P(1)-cl. sets contained in F and F is P(1)-decomp. as was to

be shown.
4\A2% C C. 4 S" kP '-< A ?)z ( Y, c I

AA C 0 .
npY A n C


In Prop. XXXI to XXXIV the state space X is assumed to be P )-indecomp.

XXXI. There are at most k disjoint P(k)-cl. sets Let m n ,

be disjoint, P(k)-cl. sets. By XXVIII each of them generates a k-cycle
(1) (1) n
C which is P -cl. Since X is P -indecomp., C = 1 C is nonempty,
m m=l m
Let x c C then by the property of a k-cycle for each m 1 m n there

exists an integer jm 1 jm k such that P (x, Em) = 1 Since the

E 's are disjoint the j 's must be distinct. Therefdaec n k ,

XXXII. Each P(k)-cl. set contains a P(k)-indecomp. set and intersects a

P(k)-max. indecomp. set. The number of distinct P(k)-max. indecomp. sets is the
J.*;r`t (k)
maximum number of P -cl. sets.

If there were a P(k)-cl. set which does not contain any P(k)-indecomp.

subset then the set itself is P(k)-decomp. andhhence contains two disjoint

P(k)-cl. sets each of which is P(k)-decomp. Hence by induction there would be

an infinite number of P(k)-cl. sets, contradicting XXXI Now by the P(k)

version of XV each P(k)-indecomp. set is contained in a P(k)-max. indecomp.

set; hence each I(k)-cl. set intersects a P(k)-max. indecomp, set. Two dis-

joint P(k)-cl. sets cannot intersect the same P(k)-max. indecomp. set, proving

the second assertion.

XXXIII For each k let 6(k) be the number of distinct P(k)-max.

indecomp. sets contained in X ; then 6(k)|k These 6(k) sets form a clean
6(k) (k)_
cycle Ii 1 i 6(k)l X k I. does not contain any P(k)-cl. sets
and is not abs. ess.

By XXXII there exists a P(k)-max. indecomp. set I Set

Ii A (I) ,i .

By XXVIII Ik-i l-iik is the k-cycle generated by I We have

I = C Ik But by XXIX each I. is P(k)-max. indecomp. Hence by the
0 iC 1


P(k)-version of XVI 1 = I and consequently I. = I. if i j (mod R) ,

Let d be the least positive integer such that I = Id Then I. / I. for

0 i < j d-1 for otherwise we should have

k-i k-i kjI
I = k = A (li) = Ak (I) =- Ak+ (E) .- ,
S k = 1 0= o k+j-i= j-i

contradicting the definition of d By the P (k)-version of XVI the I. ,
1 i d are disjoint and so form a clean cycle. We have now I. = I. if and

only if i = j (mod d) hence dfk .

This d is the 6(k) asserted in the proposition. For if there is a

P(k)-max. indecomp. set J distinct from the I. +s then it is disjoint from

their union C s ,to
their union C =i Ii. As before, there is an integer e such that elk

and i A(7) 1 j e is a cycle. Let D = e A(J) Since X is
indecomp., C 0 D / 0 and consequently A (I) (' A3(J) / 0 for some i

and j But then A'(I) = AJ(J) because both sets are PF (-max. indecomp. and

it follows that

J = Ak(J) Ak+i-j

Thus J is one of the I. 's and therefore d = 6(k) .
By XXXII any P(k) cl. set must intersect one of the I. 's Hence

X C does not contain any P(k)- l. set. Then X C is not abs. ess. by the

P(k)-version of XIV and XXV .

Definition 8. 6(k) is called the cyclic index belonging to k and the

6(k)-cycle described in XXXIII is called the cycle belong to k .

Notation. For two positive integers k and k' we denote their least com-

mon multiple by k v/k' and their greatest common divisor by k A k' .

XXXIV. For arbitrary k and k' we have

(1) 6(k V k') 6(k) V 6(k')

(2) 6(k A k') = 6(k) A 6(kt)

Let D. 1 i 6(k) and f i 1 i 6(k')} be the cycles belonging

to k and k' respectively.

We show first that

(5) 6(k) > k A 6(k')
Writing = k A 6(k') and 6(k') = qd we set

Fr U md+r

The sets IFr 1 r dj, are clearly disjoint and P -cl., hence P (k)-cl.

It follows from XXXII that there are at least d distinct P k)-max, indecomp.

sets; hence 6(k) a d which is (3).

Next, we show that

(4) if klk' then 6(k) 6(k') .
&(k) 6(k') ()
Since X is indecomp., and J D. and E. are both P(1-cl., we have
i=l i=l
Di E / 0 for some i and j By relabelling we may suppose that D1\ J10.

Define D. and E. for all i-> 1 by setting D. = D. iff i 5 j (mod 8(k))

and E. = E. iff i j (mod 6(k)) Then it follows from the properties of

cycles that the sets D. ri E. 1 i (k) V 6(k') are disjoint and
1 1.
P (k k')cl., hence P(k')-cl. if kIk' Hence

6(k) V 6(k') 6(k')

by XXXII and consequently (4) is true.

We can now prove that

if klk' then 6(k) = k A (k') .


For 6(k)|k by .XXIII ; together with (4) this implies

6(k) k A6(k')

Together with (5) this implies (5).

Let k V k' = / then we have by (5):

6(k) = k A s(/) ,

6(k') = kA A s(3) .

Since 6(/)IJ it is a simple arithmetical fact that

(7) (k /\6()) V (k' A s(/)) =(k V k') ,-A (/) = X s6() = 6(/) .

Substituting from (6) into (7) we obtain (1).

Finally, let k A k' = d ; then by (5):

6(d) = d A (6(k) AS(k'))

Since 6(k)|k and 6(k') k' by XXXIII ($) reduces to (2).

XXXV. We have for an arbitrary k ,' -

(9) 6(6(k)) = 6(k)
and the cycle belonging to 5(k) coincides, member for member, with that

belonging to k .

writing d = 6(k) we observe that each I. in XXXIII is P(d)-cl. and
F(k)-max. indecomp. Hence it is d)-max. indecomp. by the last assertion in

XXX Thus 6(d) d and since 6(d)jd we have 5(d) = d The rest follows.

The equation (9) also follows from (2) if we substitute 6(k) for k' there

and us,; the frct th-.t 6(k)lk .

XXXVI. To each prime number p there corresponds an e which is either a

nonnegative integer or "infinite," such that

6(p") = pmin(n,ep)

for ea.ich n- 1 .

finee e = e to be the least nonnegative integer such that 6(p ) e+


r/* ~7 i


or oo if such an integer does not exist, Then 6(pn) = p for 0 n e .

If e = co there is nothing more to prove Let e < oo and assume, for the sake

of induction that
(pn) e < <
6(p) = p for e n a

where a is an integer. By XXXIII we have (pa+) = p for some nonnegative

integer b and by (4) we have b e On the other hand we have by XXXV ,

6(p ) = (6(p+l)) = (pa+l) = p

Hence b e by the definition of e and the induction hypothesis. It follows

that b = e (pa) = p and the induction is complete.

k =TT P

be the prime-factorization of k then
min( f, e )
6(k) =IT p

where e is as given in XXXVI .

This is an immediate consequence of XXXVI and equation (1) in XXXIV .

Definition 9. The set C in F is called a k-- consequent of x iff

P(k)(x, C) = 1 The sequence 0 Ck k 11 is called a consequent sequence of

x iff for each k 1 Ck is a kth consequent of x .

Each x has as a consequent sequence the sequence each term of which is X .

XXXVIII. Given a consequent sequence CGk k 1l of x there exists a

consequent sequence D k >- 1 of x such that DkC Gk and Dk C A=(Dk+1)

D = (C ) .
k = k+j

ii^ ~ J1.-


Then D C A(Ck) = C k ; and

O .
-(Dkj+l (Ck+l+j) 0 id
k+ += k+l+j) k+) k j
j=0 j=1

Since P(k+)(x, Ck+j) = 1 for each j 0 we have by XXVI ,

P(k)(x, AJ(Ck+j)) = 1

and consequently
P(k)(x, .~ (C )) = 1
This proves that Dk is a k consequent of x for each k -1 .

Definition 10. For each x we define approbability measure x(*) as

follows: for each E c F,
x(E) = -1 p(n)(x, E) .
n=l 2"

It is clear that T () is a probability measure, and that T(E) = 0 if and
x X
only if L(x, E) = 0 or equivalently x c E
th I
Definition 11. A k consequent C is called minimal iff C is minimal

with respect to the measure [I namely iff there does not exist a kth con-
sequent D with xf(D) < J~(C) A minimal consequent sequence is one in

which each member is minimal.

XXXIX. For each x and each consequence iCk k 1 of x

a minimal consequent sequence Dk k 1 such that Dk C k

There always exists a consequent sequence of x namely the

members of which are X. Writing for a moment C c C (x) iff

consequent of x we set

ak = inf
C C C (x)

, there exists

for each k 1.

sequence all

C is a kth



Then there exists a C in C (x) with (C ) < a +- Let
k,n =k x k,n k n
oo00 th
Dk = Ck. rn Ck,n ; then Dk is a k consequent of x and (Ck) = ak
Clearly I Dk k > l1J is a minimal consequent sequence and Dk C Ck for each

k 1 .

XXXX. In an indecomp. space X two minimal kth consequents of a given x

differ by a set which is not abs. ess.

Let Ck and Dk be two minimal kt consequent of x then I| (Ck D )

= 0 and so by a previous remark (Ck Dk) 0 0 Consequently Ck Dk is

not abs. ess. by XXVII .

XXXXI. Let X be indecomp., x an arbitrary point of X and Cn n-l

an arbitrary consequent sequence of x There exists a not abs. ess. set F

(depending on x ) and for each y c X F there exists a positive integer m(y)

such that {Cm(y)+n, 1 is a consequent sequence of y .

We have for each pair of integers m and n with m < n :

( Cn-m) )P
1 = P(n)(x, ) = (nm)(y, )P()(x, dy) .
Hence there is a set F in F with P(m)(x, F ) = 0 and such that if
m,n =m,n
y c X F then
p(m)(y, C) = 1 .
00 )(
Let F U F Then P ((x, F ) = 0 ; and if y c X F the above
m m,n m m
n=m+l i o
equation holds for every n m+l Let F = r' F then F c F and
P(m)(x, F) = 0 for every m 1 Consequently F / 0 and F is not abs. ess.

by XVII If y c X F then there exists a positive integer m = m(y) such

that y c X- F and p(k)(y, C ) = 1 for every k & 1. This proves the



In propositions XXXXII to XXXXV the space X is assumed to be abs. ess.

and indecomp.

XXXXII. Let X be abs. ess. and indecomp. For each x there exists a

finite positive integer k(x) such that if Ck ,k k 1 is any consequent

sequence of x then there exist m and n both less than k(x) + 1 such that

C m- C is abs. ess.
m n sufficient to prove this for a fixed minimal consequence Ok k l1

For then the conclusion will remain valid with the same m and n for any con-

sequent sequence of x by XXXX Furthermore we may suppose on account of

XXXIX and XXXVIII that 0k C (Ck+l) Hence C = U 0k is cl. and con-
sequently abs. ess. by XVIII Set
Dk = -k (Ckr f k+j )
k j=

If y c Dk then F()(y, Ck+j) = 1 and hence P(J)(y, Dk) = 0 for each j 1 ,

since Dk ^ Ck+j = 0 Thus L(y, Dk) = 0 and Dk C Dk Such a Dk is
clearly iness. and consequently D = Dk is not abs. ess. But
00 00
C D = U U (Ck Ck+j) .
k=l j=l
It follows that at least one Ck Ck+j id abs. ess., as was to be proved.

For each x let Ok(x) k > 1} be a minimal consequent sequence of x and


h(x) = min{(m-n : Cm(x) r\ n(x) is abs. ess. .

,according to XXXXII h(x) < k(x) < oo, and h(x) is independent 6f the choice

6f the minimal consequence sequence.

XXXXIII. There exists an integer H and a set F which is in = and not
abs. ess. such that h(x) is equal to H for all x c X FH H

Let x be an arbitrary point and let Cn n 1 be a minimal consequent


sequence of x such that G C A (C ) for each n 1 and k 1 Such a
n n+k
choice is possible by XXXVIII Let h(x) = Z then by definition there exists

an integer j such that

C. 0i C. j is abs. ess.

Let y be in this intersection, then by the choice of C n we have for

each k 1 ,

p(k)(y, Cj+k ) j+k+) = 1

Hence by IX ,

C.j+k/ Cj. k+ is abs. ess., for each k 0 ,


(1) On J C n+ is abs. ess., for each n j .

According to XXXXI there exists a set F in F which is not abs. ess. such

that if y c X F then iCm(y)+n, n } is a consequent sequence (but not

necessarily minimal) of y for some m(y) 1 Hence we have h(y) f by the

definition of h(-) .

'We now prove that the function is bounded on X For otherwise let

xn n l 1 be points of X such that lim h(x ) = oo. Byr, what we have
n -- oo
proved, for each x there exists a set F in F which is not abs. ess. and
n x =
such that

h(y) h(x ) if y c X F .
Since X is abs. ess., X U F is not empty; and if y is in this set,
n=l n
h(y) would be oo which is impossible. Hence we may set

max h(x) = H < oo .

eett-ht-2-) on X- FH as was to be proved.
Remark. It has not been shown the function h is F-measurable or not,


but this information will not be needed below.

Definition 12. The integer H is called the overlapping index, an the set
X FH (in/ )tho overlapping core of the abs. ess. and indecomp. space X

XXXXIII. For each x in X FH there exists an integer v(x) such that

for an arbitrary consequent sequence iO n n- 1 of x ,

Cn 0n+H is abs. ess. for n v(x) .

This is merely a restatement of (1) in XXXXIII .

XXXXIV, For each k 6(k)H Consider the cycle ii 1 i < 6(k)
/ 6(k)
belonging to k and/set I. = I. if i s j (mod 6(k)) Then C = U I. is
il 2i=l
abs. ess. by XXXIII since X is abs. ess. (cf. XIX). If x e0 o (X FH) ,

S then f I i /1 is a consequent sequence of x and we have by XXXXIII,
Ii ) Ii+H / 0

for some i But the cycle is clean according to XXXIII hence 6(k)IH ,

Definition 15. Let D = max 6(k) ; D is called the maximum cyclic indet
ce (n D k>- I
and the cycle elongrg to D is called the maximum cycle.

We have DIH according to XXXXIV It is not known whether D = H in

general; this is the case where X is countable.

XXXXV In the notation of XXXVI we have
Sep >

where 0 e < oo for each prime p' Furthermore, we have for each k 1 ,

(1) 6(k) = kAD

This is immediate from XXXVI XXXVII and XXXXIV A more direct proof of

(1) is as follows. Let 6(k') = D then by (5) of' XXXIV ,

1 4
S in)<- /-/=-~

V/,i f < -
el / c


(2) 6( kAD .

On the other hand, by (1) of XXXIV ,

6(k V k') = 6(k) V D

hence 6(k)ID by the definition of D Since 6(k)|k it follows that 6(k)kA D
and so there must be equality in (2).

Example 1. X= 1, 2, 3, 4, 5 ) .
P(n, n+l) = 1 for n = 1, 2, 5 ; P(4,1) 1 ; P(5,1) = P(5,2) = .
Each fn n = 1, 2, 5, 4, is P(4)-max. indecomp.; {l,3} and {2,4} are
P(2)-indecomp., but {2,4) is not P(2)-max. indecomp. since {2, 4, 5j is. This
example shows that the cycle belonging to a divisor of k is not necessarily
obtained by the obvious grouping from the cycle belonging to k

Example 2. X = 1, 2, 5, 4, 5, 6, 7, 8 .
P(l, 5) = P(l, 6) =
P(2, 5) = P(2, 6) = F(2, 7) = P(2, 8) =-
P(5, 7) = P(4, 8) = P(5, 3) = P(6, 4) = P(7, 1) = P(8, 2) = 1 .
Here the maximum index D = 2 and the maximum cycle is composed of
{1, 2, 5, 4} and {5, 6, 7, 8 It is easily verified that H = 2 and FH .
The minimal consequent sequence for f6j is

S6 } 41 8} 2 {5,6,7,8} {1,2,5,4 5,6,7,8J, ***
If we denote this sequence of sets by Cn n Oj it is to be noted that
C1 COn = 0 for n = 2, 3, 4 but C2 n 4 / 0
Example 5. X = 1 2, 3, 4, 5, 6, 7, 8 .
P(l, 2) = P(1, 4) = P(3, 4) = P(3, 6) ;
P(2, 5) = P(4, 5) = P(5, 6) = P(6, 7) = P(7, 8) = P(8, 1) = 1
The minimal consequent sequence for { 11 is:

{il {2, 4j, 54 {4, 6 {5, 7} t6, 8 7, 1 ,

12, 4, 8) 1, 5~, 5 2, 4, 6) 3, 5, 7} ,(4, 6, 8} 5, 7, 1 ,
12, 4, 6, 8) (1, 3, 5, 7) { 2, 4, 6, 8 ***
Here in notation similar to the above: C1 / 3 / 0 C3 r) 0C 5 0 but
C, n1 C = 0 .