Group Title: Department of Computer and Information Science and Engineering Technical Reports
Title: Asynchronous parallel discrete event simulation
Full Citation
Permanent Link:
 Material Information
Title: Asynchronous parallel discrete event simulation
Series Title: Department of Computer and Information Science and Engineering Technical Reports
Physical Description: Book
Language: English
Creator: Fishwick, Paul A.
Lin, Yi-Bing
Publisher: Department of Computer and Information Sciences, University of Florida
Place of Publication: Gainesville, Fla.
Copyright Date: 1995
 Record Information
Bibliographic ID: UF00095326
Volume ID: VID00001
Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.


This item has the following downloads:

1995167 ( PDF )

Full Text


Asynchronous Parallel Discrete Event Simulation


Abstract- Complex models may have model components
distributed over a network and generally require significant
execution times. The field of parallel and distributed simu-
lation has grown over the past fifteen years to accommo-
date the need of simulating the complex models using a
distributed versus sequential method. In particular, asyn-
chronous parallel discrete event simulation (PDES) has been
widely studied, and yet we envision greater acceptance of
this methodology as more readers are exposed to PDES in-
troductions that carefully integrate real-world applications.
With this in mind, we present two key methodologies (con-
servative and optimistic) which have been adopted as solu-
tions to PDES systems. We discuss PDES terminology and
methodology under the umbrella of the personal communi-
cations services application.
Keywords-parallel algorithm, distributed simulation, syn-
chronization, virtual time, network communications

OUR purpose is to introduce the basic technical con-
cepts of distributed simulation of event-based models
(so called discrete event models), and to tie these generic
concepts to a specific application: personal communica-
tions services (PCS). Several introductory articles have
been presented in the literature such as F Lii!.. I., [1], Ni, .1
et al. [2] and Richter et al. [3]. These papers have helped to
disseminate the .... !1, ......... parallel discrete event simu-
lation (PDi. ) !, 1 i1... .- for a wide readership. Our ap-
proach is similar but stresses a single real world application
for discussing the ii 11i 1 .1 .1._- of PDi.S By defining the
i. ir 1,, 1,1 and all PDi.S I, i 1 ',,!,,,- within the context
of the PCS application, this paper serves both as a tutorial
to PDi.S and as an introduction to PCS simulation mod-
eling. PCS is a rich enough application to illustrate most
basic PDi.S concepts.
The processing elements in PDi.S can either be of a par-
allel or distributed nature. An MIMD machine with mul-
tiple asynchronous elements performing message passing is
an example of a parallel machine. Distributed elements
normally refers to local or wide area networks composed
of inter-connected set of heterogeneous workstations and
computers. PDi.S is used for one of two reasons: 1) one
wants to execute a model faster than is possible in a se-
quential machine, or 2) one must model in a distributed
fashion because of a constraint that a process (i.e., com-
putation) must be distributed rather than localized to a
single processor. One author (Lin) has demonstrated var-
ious speedups possible on a distributed memory architec-
ture for the PCS application [4], [5]. There is no question

Yi-Bing Lin is Professor with the Dept. of Computer Science and
Information Engineering, National Chiao Tung University. E-mail:
Paul A. Fishwick is Associate Professor with the Dept. of Com-
puter and Information Sciences, University of Florida. E-mail: fish-
wick( .

that PDi.S speeds up otherwise serial computations dur-
ing a simulation. The second reason for PDi.S (distributed
model constraint) is based on a situation where models for
-i. I', components are stored in 1.! -1 i11- [!. i. I loca-
tions. The other author (1 i- I- I:I is building a I."'..l.l- I"
distributed simulation of a process plant where each plant
component is ultimately co-located with the manufacturer
of that component.
The paper proceeds as follows. I ii -1 in Section II, we
define our terms within the PDi.S area and demonstrate
the generic approach to distributed simulation. In Sec-
tion III, we introduce the PCS application and demon-
strate the need for i 1! i !.. i - 1.,! of incoming messages
to a given process. There are two key approaches to i,-
chronization. Method 1, defined in Section IV, is termed
the conservative method since it ensures that the causal re-
lation among time consecutive events will be maintained
at all times during the simulation. Method 2 is defined
in Section V, and identifies the optimistic method. In this
approach, the causal relation can be broken with subse-
quent fixing of state variables. We close in Section VI with
directions for the future of PDi.
Throughout this paper, we use three font -l1. to rep-
resent different concepts. The typewriter I 1.. -1 1.. rep-
resents attributes or methods (e.g., S.- ri.'l I -t ag ()) of ob-
jects. The italic I ..i -- 1.. represents variables such as LP
or p. The serif Il.- -. .- represents event I- i such as


A Basic T, .... 7.., .,

We begin by defining terms which are commonly found
in the simulation and PDi.S fields. These terms will be
revisited in Section 3 when we assign the terms to the
PCS application. The study of ;o! 11 -;i, 1 -- -!. i, to
be simulated begins with the creation of a model. Such
a model can be in one of several I I," [6]: 1) concep-
tual, 2) declarative, 3) functional, 4) constraint, 5) spatial
or 6) multimodel. One begins with a conceptual model
which describes qualitative terms and class hierarchies for
the -l. i. In i ,ii ways, the conceptual model ".,-
i the definition of attributes, methods and general
characteristics of each -1. i- component without going so
far as to ascribe dynamics to components. The next four
model I- i" reflect an orientation to -I. 11, construction;
a -I. i i be constructed as a Petri net [7], queuing
model [8] or as a cellular automaton [9] for instance. The
last model I (multimodel) permits the integration of ba-
sic model I- i, to create a model composed of component
models [10], [11] where each component model represents
a level of abstraction for the -- -1. i


The PCS area, to be discussed in Section 3, uses a spa-
tial model in that the -- -1. I is viewed as a hexagonal dis-
cretization of a large two-dimensional space representing
an area where cellular communications are to be imple-
mented. Spatial models can be executed in several ways
including time 1 ",., event scheduling and parallel and
distributed. Our approach will be to use a parallel and
distributed approach to model execution, while using the
concept of event scheduling within each process. Speaking
of process, we must define this term appropriately. Model
components for a PCS implementation will be a collection
of hexagonal cells. Other model I- i" such as a queu-
ing model, are composed of other components (facilities).
A logical process (LP) is defined as a set containing basic
model components, so a PCS logical process will be a set
of hexagons, or just one hexagon. A /!,,.', ,,1 process or
processor is a set of logical processes mapped in a way that
conforms to the architecture of the parallel/distributed sys-
An LP contains several objects:
Local Virtual I ... (LVT): time associated with the
LP. The LP does not know another LPs time unless
communicated via a message.
Future Event List (FEL): event list used when there
are internal events posted within the LP itself.
Event: an item within the FEL.
Message: an item sent from one LP to another.
The FEL is composed of events, where an event combines
the following objects: 1) time stamp, 2) token, 3) event
I I." The time stamp reflects when the event is to oc-
cur. An event's occurrence correlates with the execution
of an event routine for that LP. The token is associated
with whatever is flowing through the network of LPs. For
the PCS application, portables (i.e., mobile phones) flow
through the -1i. i An event I i" specifies what will
happen to the token (arrival, boundary crossing, depar-
ture, incoming call). An LP has input channels and output
channels where each channel has a first-in/first-out (FIFO)
buffer associated with it. A message is equivalent to an
event that must be moved from one LP to another. Mes-
sages which simply enter an FEL and are processed are
generally called events. When an event must be issued to
another LP, it becomes a message. The relationship among
the above terms are shown in I I,..i- 1.

I !,11., 1 about here.]
Messages arrive in one of several input channel buffers
and are routed directly to the LPs FEL. Note that simple
LPs ii ,- involve a calculation such as 1) taking the times-
tamp from an incoming message, 2) adding a value to this
timestamp, and 3) sending the new message to the output
buffers. Such an LP would not have ;,~i need of an FEL
and would be a "--p1 distributed simulation. This kind of
technique, however, is wasteful of the computing elements
since there will be a large price to 1-, in communications
overhead among inter-LP communication. A simple addi-
tion is not -it! n. -! iI to warrant a distributed approach. On
the other hand, if the processing element can be made to

do work then the communications overhead becomes less
critical. The kind of work ideally suited in simulation is a
sequential simulation within the LP, composed of the usual
FEL and event routines. Thus, the distributed simulation
is 1i 1'. I1 in form with sequential simulation coinciding -
and -- !. i.. i I with- distributed simulation. The LVT
of this more substantial LP is updated by removing the
highest priority event (lowest timestamp) from the FEL
and executing the associated event routine. Some (or all)
of these event routines will contain scheduling commands
to place events with new times back into the FEL. Some
event routines will involve messages to be issued through
the output buffer(s) to a target LP.

B. Object Oriented Implementation
A PDi.S consists of several PDi.S objects or LPs. These
LPs execute 1 .! !.. 1-1- with coordination to complete
a simulation run. To implement the objects in an LP (as
described in Section II-A), the attributes and methods of
the LP are classified into four categories (see Table I):
[Table 1 about here.]
A clock mechanism indicates the progress of the LP.
An attribute LVT represents the timestamp of the
event that just occurred in the LP. The Lv i-ip.'a.r.: ()
method updates LVT to advance the ". I.. !:" of the LP.
A FEL mechanism processes the events occurring in
the LP. The FEL is basically a priority queue with one
attribute and three methods. An attribute eventList
maintains the events to occur in the future. The
Enqueue() method inserts a time-tagged event into
eventList so that eventList maintains its ordered
sequence. The Dequeue () method deletes the event
with the minimum timestamp in eventList. The
Cancel() method deletes the event with a specified
timestamp in eventList.
A ,, ?.,!. .... "', i ',. mechanism interacts with other
LPs to coordinate the execution of PDi. The
ReceiveMessage() method receives messages from
other LPs (these messages will be inserted into the
FEL for oi!.... --ii!-_. The method ExecuteMessage()
executes events in the FEL. The S.nli.'-ll cag0()
method sends output message (generated by the ex-
ecution of events) to their destination LPs.
It is probably more appropriate to consider
ExecuteMessage() as a method of the FEL. However,
this method is affected by the PDi.S -- 1 1,,!I i..l,,
mechanisms to be described later. Thus the method
is classified as part of the synchronization mechanism.
An application mechanism represents a sub-model for
a specific simulation application to be simulated by
the LP (to be elaborated).

C. PDES Implementation l,, i.f.' ....
PDi.S -I. i! have been implemented in 1itl, i. I par-
allel architectures such as BBN Butterfly [12], [13], [14],
Sequent [15], [16], [17], JPL Mark III [18], Simulated i ,-
ford Dash Multiprocessor [19], Transputers [20], [21], CM-


1/('.!-, [22], KSR [23], and iPSC/860 [24]. PDi.S has
also been implemented in workstations connected by a lo-
cal area network [4] which is widely available in both the
industrial and the academic environments.

We use personal communication service (PCS) network
simulation to illustrate PDi.S functionality. A PCS net-
work _-j], [26] provides low-power and high-quality wire-
less access for PCS subscribers or portables. The service
area of a PCS network is populated with a number of radio
ports. Every radio port covers a sub-area or cell. The port
is allocated a number of channels (time slots, frequencies,
spreading codes or a combination of these). A portable
occupies a channel for an i !,,i ..! /outgoing call. If all
channels are busy in the radio port, the call is blocked.
In PCS network planning, PCS network modeling (usually
conducted by simulation experiments) is required to inves-
tigate the usage of radio resources. Since PCS network
simulation is time-consuming, PDi.S effectively speeds up
the process of PCS network simulation. Specifically,
The size of the PCS network under study is usually
large (e.g., thousands of cells). A I- i.1 1 sequential
PCS simulation run takes over 20 hours, while the cor-
responding PCS PDi.S takes less than 3 hours using
8 processors [4].
Another popular parallel approach, the parallel inde-
pendent replicated simulation [27], [28], [29] (running
multiple simulation replications concurrently) does not
work for PCS simulation. In most cases, the PCS de-
signer only is interested in the behavior of the PCS
network at the engineered workload (e.g., the work-
load at which the blocking probability is .). To cal-
ibrate the simulation at the engineered workload, the
setup of input parameters for the next simulation run
is dependent on the previous run.
Now we describe the PCS model and its mapping to the
corresponding PDi.S For demonstration purposes, we de-
scribe a simplified PCS model without considering the de-
tails of the radio signal propagation issues (such as Rayleigh
fading, co-channel interference, and so on). We assume
that there are S cells in the PCS network, and on the aver-
age, there are n portables in a cell. Every port is allocated
some number of channels. A portable resides at a cell for
a period of time which is a random variable with some
distribution (e.g., exponential [30], [31], [32]). Then the
portable moves to a neighbor cell based on some routing
function (e.g., equal routing probabilities for all neighbors).
The call arrivals to a portable is a random process (e.g.,
Poisson), and is independent of the portable's movement.
A call is connected if a channel is available. Otherwise, the
call is blocked. When a portable moves from one cell to
another while a call is in progress, the call requires a new
channel (in the new cell) to continue. This procedure of
changing channels is called / ...... l/for automatic link trans-
fer (ALT). Several handoff schemes have been proposed in
the literature [33], [34], ,;,] In this paper, we consider
the simplest scheme called non-prioritized scheme. In this

scheme, if no channel is available in the new cell, then the
call will be dropped or forced terminated immediately.
The PCS example is probably more realistic to the reader
if we add some geometry to these moving vehicles (porta-
bles). Unfortunately, whether a vehicle moves from one
cell to another cannot be simply determined by the I'1! --
ical movement of the vehicle. We also need to consider
the radio propagation. It is possible that the connection
to a vehicle changes from one port to another even if the
vehicle is stationary the change of radio signal strength
ii result in re-connecting the vehicle to a .t!!. i. II port.
According to the PCS network measurement methods, we
determine that the movement (in the sense of port con-
nection) of a vehicle is best characterized by the residence
time' distribution and the destination cell routing proba-
',il' The reader ii image that this movement model
is equivalent to a simple path approach where a vehicle
moves straight with an angle. The angle determines the
destination cell and the residence time is the product of a
constant speed and the diameter of the cell2.
To map the PCS model into PDi.S the cells in the PCS
network are represented by cell objects derived from the
PD1.S objects (i.e., LPs). These LPs are then mapped
to processors for execution (see 1 ioi,- 2). A cell LP
has the following attributes and methods (i.e., the ap-
plication mechanism of a general LP): A constant at-
tribute channelNo represents the total number of chan-
nels in a radio port. An attribute i.'Tl-h~r'.alri,:, repre-
sents the number of idle radio channels. A portableList
collects the information of all portables reside in the cell.
There are five methods in the cell object: CallArrival (),
CallCompletion(), PortableMoveln(),
Port abI .I l iu r, (), and Handoff (). These methods will
be elaborated later.

I ,1 ii.- 2 about here.]
The portables in the PCS network are represented by
the portable objects. A portable object consists of four at-
The busy attribute indicates the status of the portable.
If busy=YES then the portable is in a conversation.
The callArrivalTime attribute represents the next
call arrival time.
The callCompletionTime attribute represents the
completion time of the current phone call when
busy=YES. If buC -=11i1 the callCompletionTime at-
tribute is meaningless.
The port at.ll I:.-eiurTime attribute represents the
time when the portable moves out of the current cell.
There are two categories of events in a PD. S An internal
event is scheduled and executed at the same LP (the event
represents the interaction between a cell and a portable
within the cell in our PCS example), and an external event
is scheduled by one LP and is executed by another LP.

1Residence time refers to the time that a portable resides within a
2 But note that our movement model is practical it is used to ap-
proximate real radio systems, while the simple path approach cannot.


Thus, after its creation, an internal event is inserted in
the FEL by using the Enqueue () method, and an external
event is considered as a message, and is sent to the des-
tination LP by using the S-.ri.'T It ag 0() method. In the
PCS PDi.S there are three internal event I- i" and one
external event 1- 1" The internal event I- i" are described
CallArrival: Either the port (the cell) or the portable
initiates a call setup. A radio link is required to con-
nect the port and the portable. If no radio link is
available or the portable is already busy with another
conversation, the call is dropped.
CallCompletion: A phone call completes, and the radio
link between the port and the portable is disconnected.
PortableMoveOut: The portable moves out of a cell. If
the portable is in a conversation, the radio link be-
tween the portable and the port is disconnected.
We treat the CallArrival event I- i" as an internally gen-
erated event based on a probability distribution. This is
just an abstraction of the actual situation where arrivals
are sent from outside the LP to one of the LPs input chan-
nels. Therefore, a more detailed simulation would involve
" . I 11! !, i. i n. -- ... reflecting the true nature of
incoming calls. The use of a probability function is an ab-
straction for this il![lI. 1' i!i process.
The external event I- i" is described below.
PortableMovein: A portable moves in a new cell. If the
portable is in a conversation, then a new radio link
between the cell (port) and the portable is required.
If no radio channel is available, the call is forced ter-
In PDi. S the execution of a PortableMoveOut event at
a logical process LPA always results in the scheduling
of a PortableMovein event for the destination logical pro-
cess LPB. This event I- i- is external (to LPB), and the
scheduling of the event requires communication between
LPA and LPB.
An event/message m is of the format

m = (timeStamp, p, event lq,')

where event l.,I, represents the -I I" of the event,
timeStamp represents the (simulated) time when the event
occurs, and p is the pointer which points to the corre-
sponding portable p. The execution of the event mes-
sage m at a cell object LP is described as follows. The
LP.ExecuteMessage() method invokes different methods
according to the event 1- 1"' of m (the Pascal-like '
statement is used in the following definition).

FeuvnteMeaasora () J

end case

The methods invoked in ExecuteMessage() are described
below. When the event I of m is CallArrival, the follow-
ing action is taken.

CallArrival(p) {
if p.busy=YES then
/* A call is already in progress when the new */
/* call arrives at LVT. In other words, a busy line */
/* occurs and the new call arrival is ignored. */
update the busy line statistic;
else /* I.e., i tu;. -= ll */
if i.l- 'har Ir-lil:, = 0 then
/* The call arrival is blocked. */
update the blocking statistic;
else /* I.e., i.l ~: I-'iarT rsli:> 0. */
i 'I,-'Ti ar r lli <-- idl 'Tihatrir llr io -1;
p.busy +- YES;
generate the call holding time t, and
p.callCompletionTime +- LVT + t;
end if
end if
generate the next inter-call arrival time t' and compute
the next call arrival time as
p.callArrivalTime +- LVT + t'.
invoke S.: -I u I I I- -.i E -, r r. (p);
/* Schedule a new event (to be described). */
Note that the busy line and call blocking statistics are out-
put measures of the PCS simulation (not shown in our
PD1.S example) 3
When the event I "' of m is CallCompletion, the follow-
ing action is taken.

CallCompletion(p) {
/* Release occupied channel at call completion. */
idl I-,'t'ian llIo+-i l'ill-'t'ia nrlr l Io+1;
tLu -=lItI
invoke S.: u I -L I .i I (p);
/* Schedule a new event (to be described). */

When the event -I I". of m is PortableMoveln, the following
action is taken.

PortableMoveln(p) {
ifp.busy=YES then /* A handoff occurs. */
invoke Handoff (p); /* To be described. */

L v iiEI a r. (m.timeStamp) end if
/* I.e., LVT+- m.timeStamp. */ generate the portable residence time t and compute tL
case (m.event l.,i, ) of next move time p.port at-l ll : i urT I+- LVT+t;
CallArrival: invoke CallArrival (m.p); invoke S.:r-u. u -- I I -.i E-L r- r, (p)
CallCompletion: invoke CallCompletion(n.p); 3Call blocking is a major performance measure of a PCS net-
PortableMovein: invoke PortableMoveIn(m.p); work. A PCS Network is usually engineered at 1% or 2% blocking
PortableMoveOut: invoke Port abl. I::- i -Lu r(mT.p); probabilities.



/* Schedule a new event (to be described). */

The method Handoff ( is described below.

Handoff(p) {
if i. l-~''hiari-ll:,O0 then
/* No channel is available to connect the */
/* handoff call i.e., the handoff fails. */
update the forced termination statistic
(not shown in our PD1.S example);
,. tu '-=IIL
else /* The handoff succeeds. */
il Ill'-'Tia' tteillli, i:1 -e'ha rtteilll Oi -1;
end if
If the event I I- of m is PortableMoveOut, the following
action is taken.

Port t.lel I-.:, -.Lur(p) {
if p.busy=YES then
i "ie'-'t atrt-ll I I .-- il,' ''t artti I Io+ 1;
/* When a communicating portable moves */
/* to a new cell, it releases the occupied */
/* channel of the old cell. */
end if
determine the destination cell (LP') to which the
portable moves ;
generate an output message
m' = (LVT,p, PortableMoveln);
invoke S- .ll IIs s ag- (rm', LP');
/* A PortableMovein event is scheduled for LP'. */

Note that the timestamp of m' is the same as that of m.
In our implementation, the execution of the event mes-
sage m results in the scheduling of exactly one future event
m'. When m is executed, one or more attributes of the cor-
responding portable are updated. Then the next event for
the portable is determined based on the updated values of
the attributes. If the event 1'- of m is PortableMoveOut,
then a PortableMovein event message with the same times-
tamp is scheduled for the destination LP as described in
the definition of Port all I I:. -lut: (). For the other three
event I- i the message m' = (ts,p, eventT(,,, ') is deter-
mined by invoking S.: h u .i. E L ri r ():

.: t u eI -l E-.iLEi rL (p) {
if i t.u = ii then
/* The portable is idle at LVT. The next */
/* event occurring to p is either a call arrival */
/* or a cell crossing movement. */
ts +- min(p. callArrivalTime,
p.port -at1-l 11.: -e u rTime);
if ts = p.callArrivalTime then
m'.eventT(,!, +-CallArrival;
else m'.eventT,!, +-PortableMoveOut;

end if
else /* I.e., p.busy=YES. The portable is busy at */
/* LVT. The next event occurring to p is either a */
/* call arrival, a call completion or a cell crossing */
/* movement. */
ts +- min(p. callArrivalTime,
p.port a tl e I I.:. I u lrTime);
if ts = p.callArrivalTime then
m'.eventT.l,! +-CallArrival;
else if ts = p. callCompletionTime then
m'.eventT.l +-CallCompletion;
else m'.eventTl, +-PortableMoveOut;
end if
end if
Consider the example illustrated in i !,L.- 3. In this fig-
ure, a portable is represented by a car (although in i! !ii
PCS -. i!,- portables are carried by pedestrians). A call
arrival is represented by a phone connected to the car. A
call completion is represented by a cross (disconnection) on
the phone line.
At time 0, portable pi is at cell A. At time 10, a phone
call for pi occurs. The call completes at time 13, and the
portable moves to cell B at time 16. At time 20, another
phone call for pi arrives. At time 24, pi moves to cell C
(and a handoff occurs).
IJ 1,1.- 3 about here.]
In PDi.S cells A, B, and C are simulated by logical pro-
cess LPA, LPB, and LPc respectively. At the beginning of
the simulation, the attributes of p, are

busy = 10l
callArrivalTime = 10,
callCompletionTime =?
port able. Il ,l urT iilT = 16

and an event

mi = (10, p, CallArrival)

is scheduled and inserted in the FEL of LPA. When the
LVT of LPA advances to 10, rnm is executed by invoking
LPA.CallArrival(pl). Suppose that an idle channel ex-
ists. The call is connected and the call holding time for the
conversation is generated (which is 3, or the call comple-
tion time is 10+3=13). The next call arrival time is also
generated (which is 20 in I ,.- 3). Thus the attributes of
pi are modified as

busy = YES,
callArrivalTime = 20,
callCompletionTime= 13,
port a-lll. II i url Ti ie = 16

and a new event

m2 = (13,pi, CallCompletion)


is scheduled and inserted in LPA's FEL. When the LVT
of LPA advances to 13, rn2 is executed. The method
LPA.CallCompletion(pl) is invoked and the attributes of
pi are modified as

busy= IIL,
callArrivalTime = 20,
callCompletionTime =?,
port abt.eIl h.OluruTime= 16

and a new event

ms = (16,p, PortableMoveOut)

is scheduled. At LVT 16, m3 is executed. The method
LPA.Port ab.l-ll:.- -ilur.(pi) is invoked to determine the
destination cell (which is B in I i,..- 3), and a message

m4 = (16,1p, PortableMovein)

is sent from LPA
to LPB by invoking LPI Sn.- I sa (.. LPB). Note
that the portable pi migrates to LPB when mf4 is sent. (In
GIT/Bellcore's PCS implementation [4], a message is part
of a portable object, and sending a message automatically
migrates the corresponding portable object.) When LPB's
LVT advances to 16, it executes m4. The next portable
move time is generated (which is 24). The attributes of pl
are modified as

busy = 0lii
callArrivalTime = 20,
callCompletionTime =?,
port at.-Ill- .-eurTime = 24

and a new event me = (20, pi, CallArrival) is scheduled.
A PDI.S is correct if the following rule is satisfied.
Local Causality Constraint: Every LP processes
events in nondecreasing timestamp order.
The ii11. ir problem of PDi.S is that the logical processes
are executed at hi!!. ii I speeds. Consider the scenario in
1 i ., 4 that portable pi moves from cell A to cell B at
time 20 with an ongoing phone call (i.e., a handoff call),
and portable p2 moves from cell C to cell B at time 13 with
an ongoing phone call (see 1 i,.- 4(a)).

1I1 i,.,- 4 about here.]
Consider the PDi.S scenario in ii.- 4(b). LPA sends
a PortableMovein event (message) mi (for pi) with times-
tamp 20 to LPB. Later LPc sends r2 (for p2) with times-
tamp 13 to LPB. If LPB executes mr before r2 arrives,
then the modifications to LPLf i 'hai. r li:, is out of the
timestamp order, and the local H L- ,l i- rule is violated.
Thus the simulation result is not correct.
To solve this problem, the executions of the logical pro-
cesses must be -- i 1!,..!i 1I The remainder of this paper
describes two popular;. !, 1ii. .! ,- G.- !,! .! -. 11 !mech-
anisms, the conservative and the optimistic methods.

The conservative simulation [36] is conservative in the
sense that it does not execute an event before it ensures
that the local ,L- li- rule is satisfied. The conservative
simulation follows two rules: the input waiting rule and the
output waiting rule. It also assumes that
the messages are received in the order they are sent
(the FIFO communication -,.'; ,, i, i', and
the communication channels among LPs are fixed and
never change during the simulation. In I i ,.i- 4(b),
LPA (LPc) has one output channel directed to LPB,
and LPB has two input channels (one from LPA and
one from LPc).

A. Basic S,., !..... ",,',, .1. Mechanism
In a conservative simulation, every logical process LP
repeats the following two steps.
Step 1. LP waits to select an input message m from
its input channels (extra data structures are required to
implement input channels in a logical process) by invok-
ing LP.ReceiveMessage(). This method is implemented
based on the input waiting rule to be described. The
method inserts m into LP's FEL.
Step 2. Let ts be the timestamp of m.
LP.ExecuteMessage() is invoked to process all events
in the FEL with timestamps no larger than ts in non-
decreasing timestamp order. The execution iii invoke
LP.S'-r.L.'rT IJ. ag () to send output messages. This method
is implemented based on the output waiting rule to be
described. If the termination condition is satisfied (e.g.,
LP.LVT>5000), then exit the loop. Otherwise go to Mi. p
The waiting rules are described as follows.
The input waiting rule. An LP does not process i!,
input message until it has received at least one message
from each of its input channels. The input message with
the smallest timestamp is selected for processing. I i ,5-. 5
shows how the input message is selected for the PCS sim-
1I ii,.. 5 about here.]
1 !,!..- 5(a) illustrates a PCS -- -i. i where 6 portables
P1,P2,P3,P4,P5, and pG move from cells B, C, D, E, F,
and G to cell A at times 30, 10, 26, 4, 12, and 14, respec-
tively. In the PDi.S model (see 1 iii.- 5(b)), the Portable-
Moveln events of pi,..., P6 are represented by the messages
mi,..., mT sent to LPA. By the input waiting rule, m4 is
the next message to be executed in LPA.
Assume that all messages sent from one LP to another
are in non-decreasing timestamp order (this property will
be guaranteed by the output waiting rule to be described
next), then the input waiting rule ensures that the times-
tamp of the selected message is no larger than o- input
messages to be processed in the future.
The output waiting rule. An LP does not send an
output message to another LP until it ensures that no out-
put messages with smaller timestamps will be scheduled
(at LP) in the future. Assume that all input messages are


handled in non-decreasing timestamp order (the property
is guaranteed by the input waiting rule). The output wait-
ing rule is satisfied if an LP only sends output messages
with timestamps no larger than its current LVT value.
Consider the following PCS example. Portables pl,p2
and ps move into cell A at times 10, 20, and 30, and move
out of the cell at times 29, 24, and 36, respectively (see
1 ,,iiL. i,, ,i). This situation occurs since a portable, once
inside cell A, ii i- take a dramatically different from other
portables. Some portables ii .- i n the same ,1!. --n 1
location for a period while other portables continue moving
towards an I li ii. I~ cell to A.

1 1,o1,.- 6 about here.]
In PDi.S miI, m2, and ms are input messages represent-
ing the arrivals of pl,P2 and ps, respectively (see p 1
in I i ,iL 6(b)). When mi is processed, a move event
mT for p, is scheduled with timestamp 29 (see I p1 2 in
I ,i ,.. 6(b)). In the conservative simulation, mr cannot
be sent to the destination LP immediately, or the output
waiting rule iii be violated. In our PCS PDi.S imple-
mentation, the portable move is simulated by two I -
of events: a PortableMoveOut event and a PortableMoveln
event. In I iiLi 6(b), .,-' and .,.' represent the Portable-
MoveOut event and PortableMoveln event of portable pi,
respectively. When the event .,.' is scheduled, it is in-
serted in LPA's FEL. When the LVT of LPA advances to
the portable "!i.. - Ii!i,. (i.e., the timestamp of ,-'), ','
is processed, which results in sending the PortableMoveln
event .,.-" (with the timestamp of i,.') to the destination. In
1 6 (b), m"' and m'" are sent after li. (3) and before
l. 1' (4); i.e., when LPA is sure that next input message to
be handled has timestamp larger than m' and m'r. Note
that .,-' is sent before mrn is.
Since the output waiting rule is guaranteed by using the
two "--i ..-" event I i. the conservative S.-ri.'ll T- g- 0)
method simply sends the output message to the destina-
tion. Note that for other applications, a L,!. I II conserva-
tive' r!'~t il l- ag- () method i i' be required to implement
the output waiting rule.
The correctness of the conservative simulation can be
proved by induction on the interaction of the two waiting

B. Deadlock and Deadlock Avoidance
The input waiting rule i!! i result in deadlock (LPs are
waiting for input messages from each other and cannot
progress) even if the simulated -- -. i! is deadlock free.
Consider a three-cell PCS network (see 1 ,LIi.- 7(a)).

I 1,1,.- !7 about here.]
There is one portable in the network, and the portable
moves in the path A -+ B -+ C -+ A. At time 0, the
portable is in cell A. The portable moves form cell A to
cell B at time 8. In the conservative simulation, a Portable-
MoveOut event mi is scheduled in LPA initially (see Fig-
ure 7(b)). By the input waiting rule, LPA waits for an in-
put message from LPc before it can process mi. Similarly,

LPc does not send out ;~,i output message before it re-
ceives an input message from LPB, and LPB does not send
out ,i output message before it receives an input message
from LPA (i.e., before mi is processed). Thus the PDi.S is
in the deadlock situation. Two deadlock resolutions have
been proposed: deadlock avoidance [36] and deadlock de-
tection/recovery [37], [38]. It has been shown [39] that the
cost of deadlock detection/recovery is much higher than
deadlock avoidance. This article will focus on the deadlock
avoidance mechanism.
In a PCS network, a portable is expected to reside in
a cell for a period of time before it moves. Assume that
every portable resides in a cell for at least six time units
before it moves to a new cell. The information that "a
portable resides in a cell for at least 6 time Lii is used
in the deadlock avoidance mechanism to predict when an
LP will receive an input message, and "6 time ,!Lii- is
referred as the lookahead value. The lookahead information
is carried by the control messages called null messages. A
null message does not represent I event in the simulated
- -1 i, Instead, it is used to break deadlock as well as to
improve the progress of a conservative simulation.
In I i,.i.i 7(b) at the beginning of PDi.S, the LVTs
of the three LPs are 0, and a PortableMoveOut event mi
with timestamp 8 is in LPA's FEL. At time 0, LPA sends a
null message with timestamp 0+6=6 (the LVT value plus
the lookahead value) to LPB (see 1 i,oii, 7(c)). The null
message implies that no portable will move in cell B earlier
than time 6. Thus, the LVT of LPB advances to 6 when
the null message arrives (I i,ti.- 7 (d)). Since no portable
arrives at cell B before time 6, it implies that no portable
will move out of cell B before time 12 and LPB sends a null
message with timestamp 12 to LPc. After the sending of
several null messages, LPA will eventually receive a null
message with timestamp larger than 8 (see 1 i,,!i.- 7 (e)),
and by the input and output waiting rules, mi is sent from
LPA to LPB and the deadlock is avoided (see I 1,1t.. 7 (f)).

C. Exploiting Lookahead
It is important to exploit the lookahead to improve the
progress of a conservative simulation. Experimental studies
have indicated that the larger the lookahead values, the
better the performance of the conservative simulation [39].
Based on the techniques proposed in [40], [41], [42], we give
three PCS examples for lookahead exploration. The first
two examples assume single cell entrance and exit. The
single entrance/exit PCS model has been used in modeling
highway cellular phone --. in- [43]. The results can be
easily generalized for multiple entrances and exits. The
techniques introduced can be combined to exploit greater
1. Lookahead Method 1 (FIFO): In a large scale PCS
network, a cell i!! only cover a street, and the porta-
bles leave the cell in the order they move in (the FIFO
property; see 1 !,tii.- 8(a)).
I io,.. 8 about here.]
Consider the corresponding FIFO LP for cell A in
PDI. The lookahead for the LP can be derived by


a presampling technique proposed by Ni. ., [41]. The
idea is to presample the residence times of the arrival
If the FEL is not ,!11 then the next departure time
can be easily computed. In the PCS PDi.S the move-
out timestamp of a portable is computed and stored
in port at-lll :. OlutrTime of the portable object at the
time when the PortableMovein event is processed. The
FIFO property guarantees that the next departure
time is the minimum of the port atl-lll :. -luiTime
values of portable objects in the FEL. Thus, the pre-
computed next departure times can be used as the
If the FEL of the LP is. ii>.1 at timestamp LP.LVT,
then the lookahead can be generated by the same pre-
sampling technique. Since the portable will arrive at
the cell later than LP.LVT, it will leave the cell later
than LP.LVT + t (where t is the presampled portable
residence time). The FIFO property guarantees that
after time LP.LVT, no portables will depart earlier
than LP.LVT+t, and the LP ii, send null messages
with this timestamp to the downstream LPs.
2. Lookahead Method 2 (.1 .. '..... Inter-Boundary
Crossing I .... Consider the example in I ,.iI! 8(b)
where the FIFO portable movement property in the
previous example does not hold. In practice, the in-
ter arrival times to the cell (for the portables from
the same entrance) cannot be arbitrary small. In-
stead, a minimum cell crossing time T is assumed.
Let pi (i = 1,2,3,...) be the ith portable arrival af-
ter time LP.LVT. The portable residence time for pi
is ti. Then the departure time of pi is later than
LP.LVT + (i 1)r + ti, and the next departure time at
the cell after LP.LVT is later than LP.LVT + A where

A= min (i- 1)T + t (1)
Since T > 0, there exists j such that

r > min [(i 1)T + ti =A

In other words, to compute A it -i thI, to consider the
first j presampled residence timestamps in (1). Fig-
ure 9 ,1-.1 1- a situation where we ,i.1.., formula
I 1,1,i.- 9 about here.]
Four portables arrive using times 10, 14, 19 and 22.
Let T = 3 so that we know that no two consecutive
portable arrivals will be less than 3. The residence
times for the portables are placed in parentheses in
I !, ,i. 9. The variable j is increased by 1 until the
above i,1, .I!L1 1i is satisfied. Suppose that LPA needs
to send a null message to its downstream before it
receives the PortableMovein event for pi. The resi-
dence times of the subsequent arriving portables are
pre-samples as tl = 9, t2 = 4,t3 = 1,t4 = 5... Our
algorithm proceeds as follows:

(a) For j = 1, 1 x 3 > min[0 + 9] = 9 ? No.
(b) Forj = 2,2 x 3>min[0 +9,3 + 4]= 7? No.
(c) Forj =3, 3 x 3 >min[0+9,3 +4,6 +1]= 7? Yes.
From this procedure, we derive A = 7 by using the
first three pre-sampled residence times.
3. Lookahead Method 3 (.1 i.... -i Residence I ... If
the FIFO portable movement property does not pre-
serve, and T does not exist (or is too small to be use-
ful), then the technique proposed in the previous ex-
ample ii, 1 not work. In a PCS simulation, the total
number N = S x n of portables is an input parameter.
To compute the next lookahead value for an LP, it suf-
fices to sample the next N portable residence times,
and (1) is re-written as [42]
A = min ti
I The last two examples 11i require a large number of op-
erations to generate a lookahead value. In [40], 0(1) al-
gorithms have been proposed to generate the lookahead
When the ExecuteMessage() method processes a null
message in
an LP, it invokes a method ComputeLookahead( to com-
pute the timestamp of the output (null) messages. The
ComputeLookahead() method Ii implement the looka-
head exploiting techniques described above. Then the new
null message is sent to some or all output channels by in-
voking the I[ z t ag. () method.

The optimistic simulation [44] is optimistic in the sense
that it handles the arrival events aggressively. When a mes-
sage m arrives at an LP, LP.ReceiveMessage() simply in-
serts m in the input queue (the optimistic simulation termi-
!, .1 -_- for the FEL). The logical process assumes that the
events already in its input queue are the "I1 i, next events.
The ExecuteMessage() method proceeds to execute these
events in timestamp order, and -nr.l I a sag () is invoked
whenever an output message is scheduled When a mes-
sage arrives at the LP, the timestamp of the message 11i!
be less than some of the events already executed. (This
arrived message is referred to as a '..,l. .,.) The opti-
mism was l i l-If .1 and therefore a method Rollback()
is invoked by ExecuteMessage() to cancel the erroneous
computation. To support rollback, data structures such as
the state queue and the output queue are required (to be
Several strategies for cancelling incorrect computation
were -! - .1 by F, ii i. .. [45]. Two popular cancellation
strategies called ,*,*,.I, ,', cancellation [44] and lazy can-
cellation [46] are described in this section.

A. Cancellation st ,... y ,
Consider the example in I ,,.i 10.

I ,11I. 10 about here.]
For -iii ,.1, i assume that cell C has one radio chan-
nel (i.e., LPc.channelNo=l in PDiL.). In this example,


portable p2 moves from cell B to cell C at time 10 (event
1), and make a phone call at time 13. The call is completed
at time 21. Portable 1 moves from cell A to cell C at time
16 (event 2), and attempts to make a phone call at time
20. Since the only radio channel is used by portable 2, the
call attempt from portable 1 is blocked. Portable 1 moves
from cell C to cell D at time 24. 1 i,.. 11, 12, and 13

I 1,ii.- 11 about here.]
illustrate the data structures of LPc (the logical process
corresponding to cell C) assuming that message mi (the
message that represents event 2) arrives at LPc earlier
than message me (the message that represents event 1)
does. In LPc, a state queue and an output queue are main-
tained to supported rollback. In our example, the state
variable (attribute) for LPc is the number of idle channels
LPc. i.'l 'ti'arrlli,.:. The state variable is checkpointed
and saved in the state queue from time to time. The snap-
shots in the state queue are used to recover the state of
LPc when rollbacks occur. The output queue records the
anti-messages of the output messages that have been sent
from LPc. The anti-messages are used to annihilated false
messages sent in the incorrect computation.
In I !,.i.i 11(a), LPc receives mi that is inserted in
LPc's input queue. Initially, the output queue of LPc
is ,iil, and the value of LPc. i. l-'ti a Tire- Iio at times-
tamp 0 is saved. After mr is executed, the -1. ill state
at timestamp 16 is checkpointed, and a call arrival event
(message rn2) is scheduled for LPc itself (see 1 ,.i.- 11l(b)).
Note that after its execution, mi is kept in the input queue
(this message !1 i~ be re-executed if a rollback occurs). A
pointer in the input queue indicates the next event to be
executed. The anti-message m2 of rn2 is saved in LPc's
output queue. The message m2 is identical to rn2 except
that it includes a destination field (in the original optimistic
or Time Warp algorithm [44], the sender and the destina-
tion are recorded in both the output message and the cor-
responding anti-message for flow control). To summarize,
the ExecuteMessage() method for the optimistic simula-
tion saves the -- -1i, i state after an event execution (note
that the state 1i i- be saved after several event executions),
and the executed event is not deleted from the input queue.
The S- .r.'-l IT z agti () method saves the anti-messages in the
output queue when it sends an output message.
After rn2 is executed, the number of idle channel is decre-
mented by 1, and

LPc. i.-1ie,-!are!lrlo = 0

is saved in the state queue. A PortableMoveOut event ms
is scheduled at timestamp 24, and its anti-message .*- is
stored in the output queue (see 1 i,.ii- 12(a)).

I 1,.11. 12 about here.]
When ms is executed, a PortableMoveln message m4 is
sent to LPD (see 1 i ,. 12 (b)). After m4 is sent, the strag-
gler rn (the event that p2 moves in LPc at timestamp 10)
arrives. Since LPc.LVT=24, the out-of-order execution is
detected (see 1 iL,-, 13 (a)) by LPc.ReceiveMessage(),

and LPc.Rollback() is invoked. Two strategies for can-
celling incorrect computation are described below.
Aggressive Cancellation. When a straggler arrives,
aggressive cancellation assumes that the out-of-order com-
putation, as well as all other computations that 11 i
have been affected by this computation are not correct.
Thus, the out-of-order computation is recomputed, and
LPc.Rollback() cancels the affected computations imme-
diately by sending anti-messages. In our example, a roll-
back of LPc at timestamp 10 occurs. In I i,..ii 13 (b), the
anti-messages m2, a, and mn are

I ,11.. 13 about here.]
deleted from the output queue, and are sent to their des-
tinations to annihilate false messages m2, m3, and m4, re-
spectively. After the rollback (see 1 i,..i!- 13 (c)), messages
r2 and mn (and r4 in LPD) are removed from the in-
put queue. The state of LPc at timestamp 0 is re-stored.
Then LPc.ExecuteMessage() resumes the simulation by
executing rn.
Lazy Cancellation. It is possible that the erroneous
computation still generated correct output messages. In
that case, it is not necessary to cancel the original message
that was sent. In lazy cancellation, logical processes do
not immediately send the anti-messages for ;,1 rolled back
computation. Instead, they wait to see if the reexecution
of the computation causes ,i!! of the same messages to be
regenerated. If the same message is recreated, there is no
need to cancel the original. Otherwise, an anti-message
is sent. In our example, lazy cancellation applies to three
1. If portable p2 arrives at cell C (LPc) at time 10 and
leaves cell C at time 28 without making i; phone call
(see 1 !,Ii.- 14(a)) then the arrival of me in I !,,. .- 13
(a) will not affect the executions of mi, rn2, and ms.
('i..,I- that in PDi.S^ whether a call for p2 occurs in
the interval [10,28] can be detected in the portable
object.) Thus messages mi, rn2, and m3 do not need
to be reexecuted after rnm is executed. This is called
jump forward or lazy reevaluation [1].
I ,,. 14 about here.]
In this case, LPc.ReceiveMessage() simply inserts
rn in the input queue, and the pointer of the input
queue points to mr. LPc.ExecuteMessage() executes
rn and the pointer jumps directly after m3 without
re-executing mi, m2, and ms.
2. The call for P2 does not block the call for pi if
p2'S call completes before pi's call arrives (see Fig-
ure 14 (b)) or p2'S call overlaps pi's call but LPc has
two or more radio channels (i.e., LP.channelNo > 2;
see 1 i,.i.- 14(c)). In these cases, the channel uti-
lization (not shown as a state variable in our ex-
ample) changes, but the subsequent messages (i.e.,
m2, n3, and r4) scheduled due to the execution of
mi are not affected. Thus, messages mi,rn2, and
m3 are re-executed to reflected the correct channel
utilization. No anti-messages need to be sent (i.e.,
mn .,_, and m4 are not sent out). Like the previ-


ous case, LPc.ReceiveMessage() simply inserts m5
in the input queue. After m5 has been executed,
LPc.ExecuteMessage() will re-execute mi,m2, and
m3 without re-generating ,i i output messages.
If lazy cancellation does succeed most of the time, then
the performance of the optimistic simulation is improved
by eliminating the cost of cancelling the computation which
would have to be reexecuted. If lazy cancellation fails, then
the performance degrades, because erroneous computations
are not cancelled as early as possible. In our PCS simula-
tion, we iii exploit situations that lazy cancellation does
not fail (as described above), and a logical process can be
switched between aggressive cancellation and lazy cancel-
lation to reduce the rollback cost.

B. Memory Management
To support rollback, it is necessary to save the "-I -..i '
(the already executed elements in the input, the output,
and the state queues) of a logical process. However, it
11 i,- not be practical to save the whole history of a logical
process because memory is likely to be exhausted before
the simulation completes. Thus, it is important that we
only save "!.. !. ]i ln-1.. of logical processes to reduce
the memory usage.
Memory management for the optimistic simulation is
based on the concept of global virtual time (GVT). The
GVT at (execution) time t is the minimum of the times-
tamps of the 1. - t. executed messages (these messages are
either in the input queue or are in transit) in the optimistic
simulation at time t. (Several other operational definition
of GVT are given in [47], [48].) It has been pointed out [44]
that at i,!! given time t, a logical process cannot be rolled
back to a timestamp earlier than the GVT at t. Therefore
the storage for all messages with timestamps smaller than
the GVT value can be reclaimed for other usage. The pro-
cess of reclaiming the storage for the obsolete elements is
called fossil collection.
The GVT computation is not trivial in a distributed sys-
tem because it 1!! be l!ii !i. ll to capture the messages in
transit. Several GVT algorithms have been developed in
the -- -I. !,- with the FIFO communication property [49]
or without the FIFO communication property [50], [51].
In GIT/Bellcore PCS PD1.S (where eight workstations
are connected by a local area network), all logical processes
are frozen during GVT computation. By utilizing the low
level communication mechanism, all transient messages are
guaranteed to arrive at their destinations before the GVT
computation starts. The fossil collection procedure works
as follows. A coordinator initiates the procedure by freez-
ing the execution of every logical process. After all tran-
sient messages arrive at their destinations, every logical
process reports its local minimum value (the minimum of
the timestamps of all unprocessed messages in the input
queue) to the coordinator. The coordinator then compute
the GVT value as the minimum of the received local mini-
mums. The GVT value is broadcast to all logical processes
for fossil collection.
To illustrate the storage reclaimed in fossil collection,

consider the example in I !,,Li!- 15. In this example, we
ignore the phone call events and assume that all Portable-
Moveln/PortableMoveOut events must be executed in their
timestamp order in the optimistic simulation. We further
assume that the state variable of a logical process is the
number of portables move in the corresponding cell after
time 0.

LI L,111. 15 about here.]
Portable 1 moves from cell C to cell A at time 4 and moves
from cell A to cell B at time 60. Portable 2 moves from
cell C to cell B at time 10. Portable 3 moves from cell B to
cell A at time 20. Portable 4 moves from cell A to cell C
at time 8. Portable 5 moves from cell A to cell C at time
7. Portable 6 moves from cell A to cell B at time 1 and
moves from cell B to cell C at time 15.
1 i_'i.- 16 illustrates the
elements in the input/output/state queues of LPA, LPB,
and LPc after all transient messages arrive at their desti-
nations, and the GVT value (which is 8 = min(60, 20, 8))
is found.

I ,,.11. 16 about here.]
1 o..- 17 illustrates the
elements in the input/output/state queues of LPA, LPB,
and LPc after the fossil collection procedure is completed.

L ,11o. 17 about here.]
All messages with timestamps smaller than 8 were fossil
collected. Note that fossil collection for the state queue is
not exact the same as that for the input/output queues.
In the state queue, the element with the largest times-
tamp smaller than the GVT value (i.e., 8) must not be
removed (see 1 i,.i.- 17). The other elements with times-
tamps smaller than 8 are removed.

C. Performance Evaluation

The performance of an optimistic PCS PDi.S implemen-
tation has been investigated in [4]. In this study, a version
of Time Warp has been developed that executes on 8 DEC
5000 workstations connected by an Ethernet.
In the experiments, speed was used as the output mea-
sure where the sequential simulator used the same priority
queue mechanism as that of PD1.S for managing the pend-
ing set of events, but did not have the state saving, rollback
and fossil collection overheads associated with the PDi.S
implementation. 1024 cells are simulated for 2.5x105 sim-
ulated seconds. I i,,..- 18 shows the performance of the
optimistic PD. S

I !,11o.- 18 about here.]

The figure indicates good performance of PDi.S for the
PCS application. PDi.S is particularly t! !t I when the
number of portables is large, the cell residence time is long,
and the call interarrival time is short.



This tutorial describes the asynchronous parallel dis-
crete event simulation (PDi.S) mechanisms and optimiza-
tion techniques by examples of personal communications
services (PCS) network simulation. We described the con-
servative and the optimistic PDi.S mechanisms and several
optimizations tailored for the PCS simulation. The per-
formance of the optimistic method was ',,i. [- discussed.
Since the conservative optimizations (tailored for PCS) in-
troduced in this paper are new and were not previously
reported, no performance studies have been conducted. In-
vestigating the performance of these optimizations will be
one of our future research directions.
The optimization techniques described in the paper are
general and apply to other simulation applications such as
battlefield simulation, VLSI simulation, queueing network
simulation and computer architecture simulation. How-
ever, these optimization techniques ii, i need to be tailored
for specific applications. i 1!- studies have devoted to this
issue (see [1], [2], [52], ;], [54] and references therein). The
PCS example can be seen as being a member of a larger
class of simulation model where one first discretizes the
spatial domain into a grid, and then simulates moving en-
tities from one grid cell to another. In this sense, the PCS
problem is isomorphic to the problems of particle/n-body
An important research direction that has not been fully
exploited is the building of user-friendly PDi.S environ-
ments. Such an environment should provides convenient
tools to develop simulation application. Methods should
also be provided to tailor general optimization techniques
to fit a specific simulation application. We anticipate that
these user-friendly environments can be constructed by the
object-oriented models described in [6].


C. Carothers and Y.C. Wong provided useful comments
to improve the !, II of this paper.

[1] Fujimoto, R.M., "Parallel Discrete Event Simulation", Com-
munications of the ACM, vol. 33, no. 10, pp. 31-53, October
[2] Nicol, D. M. and Fujimoto, R. M., "Parallel simulation today",
Annals of Operations Research, vol. 53, pp. 249-286, December
[3] Richter, R. and Walrand, J.C., "Distributed Simulation of Dis-
crete Event Systems", Proceedings of the IEEE, vol. 77, no. 1,
pp. 99-113, January 1989.
[4] Carothers, C., Fujimoto, R.M., Lin, Y.-B. and England, P., "Dis-
tributed Simulation of PCS Networks Using Time I ', Proc.
International I I I- on Modeling, Analysis and Simulation
of Computer and Telecommunication Systems, pp. 2-7, 1994.
[5] Carothers, C., Lin, Y.-B. and Fujimoto, R.M., "A Re-dial Model
for Personal Communications Services Network", To appear in
45th Vehicular Technology Conference, 1995.
[6] Fishwick, P.A., Simulation Model Design and Execution: Build-
ing D ..' Worlds, Prentice Hall, 1995.
[7] Peterson, J.L., Petri Net !' and the Modeling of Systems,
Prentice-Hall, Inc., Englewood Cliffs, N.J., 1981.
[8] Law, Averill M. and Kelton, David W., Simulation Modeling &
Analysis, McGraw Hill, 1991, Second edition.
[9] Toffoli, T. and Margolus, N., Cellular Automata Machines: A
New Environment for Modeling, MIT Press, 2nd edition, 1987.

[10] Fishwick, P.A. and Zeigler, B. P., "A Multimodel Methodol-
ogy for Qualitative Model Engineering", ACM Transactions on
Modeling and Computer Simulation, vol. 2, no. 1, pp. 52-81,
[11] Fishwick, P.A., "A simulation environment for .li i; .... i',
Discrete Event Dynamic Systems: I and Applications, vol.
3, pp. 151-171, 1993.
[12] Fi.1i.. M., Di Loreto, M., Presley, M., Wieland, F. and Jeffer-
son, D., "An Ant Foraging Model Implemented on the Time
.W Operating System", Proc. 1991 SCS Multiconference on
Distributed Simulation, pp. 21-26, March 1991.
[13] Hontalas, P., Beckman, B., Diloreto, M., Blume, L., Reiher, P.,
Sturdevant, K., Warren, L., Wedel, J., Wieland, F. and Jefferson,
D., "Performance of the C. .iii11; Pucks Simulation on the Time
Wr. Operating Systems (Part 1: Asynchronous Behavior &
Sectoring)", Proc. 1989 SCS Multiconference on Distributed
Simulation, pp. 3-7, March 1989.
[14] Fujimoto, R.M., "Time -ir. on a Shared Memory .l,,ii...
cessor", Proc. 1989 International Conference on Parallel Pro-
cessing, vol. Volume III, pp. 242-249, August 1989.
[15] Ayani, R. and Rajaei, H., "Parallel simulation of a general-
ized cube i....i;i ... interconnection network", Proc. 1990 SCS
Multiconference on Distributed Simulation, pp. 60-63, January
[16] Thomas, G.S. and Zahorjan, J., "Parallel simulation of perfor-
mance Petri Net: F i .. i;,,. the domain of I..11 .i simulation",
Proc. 1991 Winter Simulation Conference, pp. 564-573, 1991.
[17] Reed, D.A. and Malony, A., "Parallel Discrete Event Simulation:
The ( I. .1.I 1, i Approach", Proc. 1988 SCS Multiconference
on Distributed Simulation, pp. 8-13, February 1988.
[18] Wieland, F., Hawley, L., Feinberg, A., Di Loreto, M., Blume, L.,
Reiher, P., Beckman, B., Hontalas, P., Bellenot, S. and Jeffer-
son, D., "Distributed Combat Simulation and Time .r 1 The
Model and Its Performance", Proc. 1989 SCS Multiconference
on Distributed Simulation, pp. 14-20, March 1989.
[19] Soule, L. and Gupta, A., "An Evaluation of the ( I.....I 1 .-
Bryant Algorithm for Digital Logic Simulation", ACM Transac-
tions on Modeling and Computer Simulation, vol. 1, no. 4, pp.
308-347, 1991.
[20] Beazner, D., Lomow, G. and Unger, B., "A i., .11.1 simulation
environment based on Time I i*', To appear in International
Journal in Computer Simulation, 1995.
[21] Turner, S. and Xu, M., "Performance evaluation of the bounded
Time i I algorithm", The 6th I I on Parallel and Dis-
tributed Simulation, 1992.
[22] Lubachevsky, B., "Efficient Distributed Event-Driven Simu-
lations of lill;i. Loop Networks", Communications of the
ACM, vol. 21, no. 2, March 1989.
[23] Ghosh, K., Panesar, K., Fujimoto, R.M. and Schwan, K.,
"PORTS: A ..i .11. i optimistic, real-time simulator", Proc. 8th
SI I- on Parallel and Distributed Simulation, 1994.
[24] Gaujal, G., Greenberg, A.G. and Nicol, D.M., "A sweep algo-
rithm for massively I.. .11. simulation of circuit-switched net-
works", Journal of Parallel and Distributed Computing, vol. 18,
no. 4, pp. 484-500, 1993.
[25] Cox, D.C., "Personal communications A viewpoint", IEEE
Commun. Mag., vol. 128, no. 11, pp. 8-20, 1990.
[26] Cox, D.C., "A radio system proposal for widespread low-power
tetherless communications", IEEE Trans. Commun., vol. 39,
no. 2, pp. 324-335, February 1991.
[27] (.1 .... P.W and H. ;i. 11.. i . P., "Analysis of Initial Transient
Deletion for Parallel Steady-State Simulation", SIAM Journal
on Scientific and Statistical Computing, vol. 13, no. 4, pp. 904-
922, 1992.
[28] H. i.1 I i P., "Discrete Event Simulations and Parallel Pro-
cessing: Statistical Properties", SIAM Journal on Scientific and
Statistical Computing, vol. 9, no. 6, pp. 1114-1132, November
[29] Lin, Y.-B., "Parallel Independent F. 1.1;. ...- Simulation on A
Network of Workstations", To appear in SIMULATION, 1995.
[30] ..... W .C., "Packet reservation i....l i ; .1. access in a :.. I .....1i
tan microcellular radio environment", IEEE J. Select. Areas
Commun., vol. 11, no. 6, pp. 918-925, 1993.
[31] ..... V.C., "Dynamic allocation of packet reservation .... il .i.
access carriers", IEEE Trans. Veh. Technol., vol. 42, no. 4, 1993.
[32] Lin, Y.-B., "Determining the user locations for personal com-
munications networks", IEEE Trans. Veh. Technol., vol. 43, no.
3, pp. 466-473, 1994.


[33] Lin, Y.-B., Mohan, S. and ... i I A., "Channel Assignment
S. for Hand-off and Initial Access fora PCS Network",
IEEE Personal Communications Magazine, vol. 1, no. 3, pp.
47-56, 1994.
[34] Lin, Y.-B., Mohan, S. and ... i.I A., "Queueing Priority
Channel Assignment : I. . for Handoff and Initial Access
for a PCS Network", IEEE Trans. Veh. Technol., vol. 43, no. 3,
pp. 704-712, 1994.
[35] Lin, Y.-B., .. 1.. I A. and Harasty, D., "Sub-rating Channel
Assignment :' .1. for Hand-offs", To appear in IEEE Trans.
Veh. Technol., 1995.
[36] ( I ... I K.M. and Misra, J., "Distributed Simulation: A Case
Study in Design and Verification of Distributed Programs",
IEEE Trans. on Software Engineering, vol. SE-5, no. 5, pp. 440-
452, September 1979.
[37] ( I, ....I K.M and Misra, J., ... I........ Distributed Simu-
lation via a Sequence of Parallel Computations", Communica-
tions of the ACM, vol. 24, no. 11, pp. 198-206, April 1981.
[38] Misra, J., "Distributed Discrete-Event Simulation", Computing
Surveys, vol. 18, no. 1, pp. 39-65, March 1986.
[39] Fujimoto, R.M., "Performance Measurements of Distributed
Simulation : 1 i .1- ', Proc. 1988 SCS Multiconference on Dis-
tributed Simulation, pp. 14-20, February 1988.
[40] Lin, Y.-B. and Lazowska, E.D., I i.i..i.... Lookahead in Par-
allel Simulation", IEEE Trans. on Parallel and Distributed Sys-
tems, vol. 1, no. 4, pp. 457-469, October 1990.
[41] Nicol, D.M., "Parallel Discrete-Event Simulation of FCFS
Stochastic Queueing Networks", Proc. ACM SIGPLAN Sympo-
sium on Parallel Programming: Experience with Applications,
Languages and Systems, pp. 124-137, 1988.
[42] ..... D.B. and Lazowska, E.D., "Parallel Simulation of
Queueing Networks: Limitations and Potentials", Proc. 1989
ACM SIGMETRICS and Performance '89 Conference, pp. 146-
155, 1989.
[43] Kuek, S.S. and ..... W.C., "Ordered Dynamic Channel As-
signment Scheme with Reassignment in Highway Microcells",
IEEE Trans. Veh. Technol., vol. 41, no. 3, pp. 271-277, 1992.
[44] Jefferson, D., "Virtual Time", ACM Transactions on Program-
ming Languages and Systems, vol. 7, no. 3, pp. 404-425, July
[45] Fujimoto, R.M., "Optimistic Approaches to Parallel Discrete
Event Simulation", Transactions of the Society for Computer
Simulation, vol. 7, no. 2, pp. 153-191, June 1990.
[46] Gafni, A., "Rollback Mechanisms for Optimistic Distributed
Simulation", Proc. 1988 SCS Multiconference on Distributed
Simulation, pp. 61-67, February 1988.
[47] Jefferson, D., "Virtual Time II: The Cancelback Protocol for
: ..i -,. Management in Time .!i.', Proc. 9th Annual ACM
Symposium on Principles of Distributed Computing, pp. 75-90,
August 1990.
[48] Lin, Y.-B., I ..... Management Algorithms for Parallel Simu-
lation", Information Sciences, vol. 77, no. 1, pp. 119-140, 1994.
[49] Lin, Y.-B., "Determining the Global Progress of Parallel Simu-
lation", Information Processing Letters, vol. 50, 1994.
[50] Samadi, B., Distributed Simulation, i. and Perfor-
mance Analysis, PhD thesis, Computer Science Department,
University of California, Los Angeles, 1985.
[51] Mattern, F., "Efficient Distributed Snapshots and Global Vir-
tual Time Algorithms for Non-FIFO Systems", Journal of Par-
allel and Distributed Computing, vol. 18, no. 4, pp. 423 434,
[52] Fujimoto, R.M., "Parallel Discrete Event Simulation: Will the
Field Survive?", ORSA Journal on Computing, vol. 5, no. 3,
[53] Arvind, D., F .....i; R. and Lin, Y.-B., Ed., Proc. 8th Work-
shop on Parallel and Distributed Simulation. ACM, 1994.
[54] Bailey, M. and Lin, Y.-B., Ed., Proc. 9th II I on Parallel
and Distributed Simulation. ACM, 1995.

Yi-Bing Lin received his BSEE degree from
National ( I .... Kung University in 1983, and
his Ph.D. degree in Computer Science from the
University of 1....... in 1990. Between
1990 and 1995, he was with the ....1i;. .1 Re-
search Area at Bell Communications Research
(Bellcore), Morristown, NJ. In 1995, he was ap-
pointed full professor of Department and Insti-
tute of Computer Science and Information En-
gineering, National Chiao !, I. University. His
current research interests include design and
analysis of personal communications services network, distributed
simulation, and performance : ..I...1 i. He is a subject area editor
of the Journal of Parallel and Distributed Computing, an associate
editor of the International Journal in Computer Simulation, an as-
sociate editor of SIMULATION, a member of the editorial board of
International Journal of Communications, a member of the editorial
board of Computer Simulation Modeling and Analysis, Program Co-
Chair for the 8th I on Distributed and Parallel Simulation,
and General Chair for the 9th I on Distributed and Parallel

Paul A. Fishwick is an associate professor
in the Department of Computer and Informa-
tion Sciences at the University of Florida. He
received the BS in Mathematics from the Penn-
sylvania State University, MS in i. 1.11..1 Sci-
ence from the C..11. _. of William and Mary,
and PhD in Computer and Information Science
from the University of Pennsylvania in 1986.
He also has six years of industrial/government
production and research experience working at
.I.-.. I News i ii.I.iii I,, ._: and Dry Dock Co.
(doing CAD/CAM parts definition research) and at NASA Langley
Research Center (studying engineering data base models for struc-
tural engineering). His research interests are in computer simulation
: ...... 1.1 and analysis methods for complex systems. He is a senior
member of the IEEE and the Society for Computer Simulation. He is
also a member of the IEEE Society for Systems, Man and Cybernetics,
ACM and AAAI. Dr. Fishwick founded the comp. simulation Internet
news group (Simulation Digest) in 1987, which now serves over 15,000
subscribers. He was chairman of the IEEE Computer Society techni-
cal committee on simulation (TCSIM) for two years (1988-1990) and
he is on the editorial boards of several journals. 1, l.1i.- the ACM
Transactions on Modeling and Computer Simulation, IEEE Trans-
actions on Systems, Man and Cybernetics, The Transactions of the
Society for Computer Simulation, International Journal of Computer
Simulation, and the Journal of Systems Engineering.


1 Ai! iJ.i,! of a logical process (LP). ............... .. ................... ... .. 14
2 Cells, logical processes, and processors. A PCS cell is represented by a logical process (LP) in PDi.S More
than one LP ii be mapped to a processor for execution. .................. ...... 15
3 A simple PCS example. .................. ................... ........ 16
4 PDi.S !!,,m .,!i Ii, plproblem .................. ... ........... ...... 17
5 The input waiting rule. In (a), the number below a car represents the time when the portable crosses the
cell boundary . . . . . . . . . . . . . . . . . . . . . . .. . . . 18
6 The output waiting rule. .................. ................... ....... 19
7 Deadlock and deadlock resolution. .................. .............. ... . . .. 20
8 Examples for lookahead exploiting. .................. .................. .. .. 21
9 Portables entering and leaving cell A ....... . . . . . . .................. ... 22
10 A PCS example for optimistic PDi.S Events 1 and 2 will be represented by messages mr and rnm respectively
in the optimistic PD1.S (see the next figures). .................. ... .......... 23
11 The data structures of LPc before/after rollback. .................. ........... 24
12 The data structures of LPc before/after rollback (cont.). .................. . ..... ..25
13 The data structures of LPc before/after rollback (cont.). .................. ........ 26
14 ,i it ,.i,- when lazy cancellation applies (in these situations, t2 > t). .................. . .27
15 An PCS example for fossil collection in optimistic PDi1. ... ................ ......... ..28
16 The optimistic PD1.S before fossil collection. .................. ... ........... . 29
17 The optimistic PD1.S after fossil collection. . . . . . . . ... . . . . . . . . 30
18 Speedup of the Optimistic PD1.S (The call holding time is exponentially distributed with mean 3 minutes.
Eight processors are used in the parallel simulation.) .................. ........... .. 31


Output Channel

Fig. 1. Anatomy of a logical process (LP).



> Cells

(Hexagonal PCS Network Model)

Logical Processes

(Parallel Simulation Software)

Processors (Physical Processes)

(Multiprocessor Hardware)

Fig. 2. Cells, logical processes, and processors. A PCS cell is represented by a logical process (LP) in PDES. More than one LP may be
mapped to a processor for execution.



S10 13 16 20 tim


portable (cross-boundary) movement call arrival

Fig. 3. A simple PCS example.

call completion


(a) The PCS model.

(b) The PCS PDES.

Fig. 4. PDES synchronization problem.


(a) The PCS model. (b) he PCS PES.

Fig. 5. The input waiting rule. In (a), the number below a car represents the time when the portable crosses the cell boundary.


P3 P2 P P3 PI P2

30 20 10 36 29 24

portable arrivals portable departures

(a) The portables do not depart in the order they arrive.

LVT: 0 LVT: 10
FEL: empty FEL: (m'1,29)

(m3,30) (m2,20) (m,10) (m3,30) m2,20)
LAStep 1. Before m is processed LPte 2 After m is recessed

Step 1. Before m, is processed Step 2. After m, is processed

LVT 20
FRL: (m'2,24) (m'1,29)


Step 3. After m2 is processed

LVT: 30
FRL: (m'3,36)

P (m"1,29) (m"2,24)

Step 4. After m3 is processed

(b) The PCS PDES

Fig. 6. The output waiting rule.


LVT: 0
FEL: emt

LVT: 0
FEL: empty

(b) Deadlock in PDES.

(a) The three-cell PCS model.

LVT: 0

LVT: 6
FEL: emn

LVT: 0
FEL: (ml,8)

LVT: 0
FEL: empty

(c) LP4 sends a null message to LPB.

(d) LPB sends a null message to LPc.

LVT: 6

LVT: 6
FEL: empty

FEL: (m1,8)

LVT: 12
FEL: empty

(e) LPc( sends a null message to LP4.

LVT: 12
FEL: empty

(f) mi is processed and sent to LPB.

Fig. 7. Deadlock and deadlock resolution.


P3 P2 PI

portable arrivals

P3 P2 PI

portable departures

(a) The portables depart in the order they arrive.


portable arrivals

(b) Non-zero inter-boundary crossing time (t > r).

Fig. 8. Examples for lookahead exploiting.


P4 P3 P2 Pi

22(5) 19(1) 14(4) 10(9)

portable arrivals

P4 P3 Pi P2

27 20 19 18

portable departures

Fig. 9. Portables entering and leaving cell A.


Portable \ //
`0 16' 20 2 time

/ event 1
Portable 2 B en

10 '13 21 time


portable movement call arrival call completion call blocking

Fig. 10. A PCS example for optimistic PDES. Events 1 and 2 will be represented by messages m5 and mi respectively in the optimistic
PDES (see the next figures).


(LPA m




4=^r^^ 1








Pi M2]
16 '720

P, P,
m. arrival

tinostap. idle
no. idle ch.

no. idle ch

I null I
L -- -- J

(a) Before mi is executed.

(b) After mi is executed.

Fig. 11. The data structures of LPC before/after rollback.





F^upu ^uul





m] m m3
timestamp 16 20 24

portable Pi Pi Pi

timestamp 0 ~g
no. idle ch.

timestamp 20
portable pi pi
destination LPC LP

(a) After m2 is executed.





mm M2 m3
timestamp 20 24

no. idle ch.

event l a
destination LPc 2LPc 2IL

(b) After ms is executed.

Fig. 12. The data structures of LPC, before/after rollback (cont.).


(b) The rollback in progress.

[ IWI I [ N

timestamp 10 16 20 '
portable P P Pi P

no. idle eh.

timestamp 20 .
portable pi Pi Pi
event LP L P L P
destination LPc LPc IPD





10 IU16

Pl Pl

!m cN7

(a) Before the rollback.


no. idle ch.


I nuHll I
L -- J

(c) After the rollback.

Fig. 13. The data structures of LPc, before/after rollback (cont.).

* IuQuu


phone conversation interval ........... residual time interval
real time

t2 l ll-------l l l
(portable p)

ti- -- -- - -***************
(portable p)

10 16 20 24 28 virtual time (timestamp) in LPc

(a) Situation 1. The number of radio channels in LPC, is 1.

real time
t2 .......ii.......vsrr 111 ............i........
(portable p)

t ---- I .---- --- ,,,,,- T.. (portable p)

10 12 14 16 20 23 24 virtual time (timestamp) in LPc

(b) Situation 2. The number of radio channels in LPC is 1.

real time

t2 ,******************"""""" (portablep2)

t2 . . ........ ....... .............. portablee)

10 16 17 20 21 22 24 virtual time (timestamp) in LPc

(c) Situation 3. The number of radio channels in LP(, is 2.

Fig. 14. Situations when lazy cancellation i.i.1i;.. (in these situations, t2 > 1t).


Fig. 15. An PCS example for fossil collection in optimistic PDES.


Pointer pointer

my m m2 M3 M4 I- -
S L -- L I m6 M I m7
timestamp 1 4 7 8 60 es
timestamp 1 \ 15 20
portable P6 Pi P5 P4 Pi
over e lm maove portable P6 P6 P3
event mo e mX n ove move move
- -event "'ve ooe ow

timestamp 1 7 8 60 15 D

move-in coun I 1 1 move-in count i

m-5 m 0i m-4

dtiestination nation P
-Co- timestamp

portable P6 Ps P4 Pi p l P
event m Se rmove vmoe Imove portable p o P6
outevent o e mo e" of i
destination LP ILP r LPcI LP A d L B f -


m8 m9 M10 m M12 11
timestamp ~4 7i7 8 10~ 15
H--h-- -- -- -- tin
portable PI Ps P4 P2 P6
event move mve move move move P
event i out n

timestamp 0 n4 R7
move-in count o o

Fig. 16. The optimistic PDES before fossil collection.




( 0

pointer pointer pointer

M3 m4 "6 M7 m0 M12 mImi
8 60 15 20 8 10 15

P4 P1 P6 P3 P4 P2 P6
on pDa ,n6 in o ? o
move moe move move

I 15 7

[-10 46 11M -,
8 60 15 15 i"T

P4 Pi P6 P6
poRw, p- -TfaT portabb polab
move move move move
LP LPout out in

Fig. 17. The optimistic PDES after fossil collection.



move-in count


[Input Qu


*: the mean cell residence time = 15 minutes
: the mean cell residence time = 15 minutes
o: the mean cell residence time = 75 minutes

5 10 15 20 25 3
Call interarrival time (minutes)
(a) The expected number of portables per cell is 50.

*: the mean cell residence time = 15 minutes
o: the mean cell residence time = 15 minutes
o: the mean cell residence time = 75 minutes

5 10 15 20 25 3
Call interarrival time (minutes)
(b) The expected number of portables per cell is 75.

Fig. 18. Speedup of the Optimistic PDES (The call ,..1i;,,. time is .i..... 1; ii distributed with mean 3 minutes. I i..1~ processors are
used in the I. ,11.1 simulation.)


I Attributes and methods of an LP. .................. ................... .. . 33



Mechanism Attributes Methods
( 'I. : LVT Lv 'Tii'pd.'a. ()
FEL eventList Enqueue()
Synchronization ReceiveMessage (

Application to be elaborated to be elaborated

University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs