Group Title: Department of Computer and Information Science and Engineering Technical Reports
Title: A Simple approach to distributed pools
CITATION PDF VIEWER THUMBNAILS PAGE IMAGE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00095208/00001
 Material Information
Title: A Simple approach to distributed pools
Alternate Title: Department of Computer and Information Science and Engineering Technical Report
Physical Description: Book
Language: English
Creator: Rafanello, Benedict M.
Publisher: Department of Computer and Information Sciences, University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: January, 1994
Copyright Date: 1994
 Record Information
Bibliographic ID: UF00095208
Volume ID: VID00001
Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.

Downloads

This item has the following downloads:

1994122 ( PDF )


Full Text






A SIMPLE APPROACH TO DISTRIBUTED POOLS


BENEDICT M. RAFANELLO & THEODORE JOHNSON


University of Florida, Department of Computer and information Sciences

January 1994




Computer networks hold the potential to coordinate the activities of multiple machines so that their combined com-
putational abilities can be applied to solving a single problem. Several methods have been developed over the years to
harness the power of networked systems for solving certain classes ofproblems. One such class ofproblems is the distrib-
uted producer/consumer problem, in which a set ofproducer processes supply items to a set ofconsumer processes. Each
ofthe processes involved resides on a ti .... 1 'i i,, i ,,,. with the machines being connected by a network and the proc-
esses communicating via message passing. The problem, then, is how to coordinate the activities ofthe producers and con-
sumers so that an acceptable level of throughput can be maintained with a minimal amount ofoverhead. This paper
presents a simple solution to the distributed producer/consumer problem. This solution, which is based upon the notion of
a distributedpool, is described, I.. i. il and itsl'.. '1 .1m. .. analyzed. As the analysis shows, the I, ,il ii... I pools algo-
rithm presented herein is a simple, .. i. .. ot solution to the I, i, ,i -i... Iproducer/consumer problem, and is capable of bet-
ter than 90('. .. o. i iiii. I. common conditions. Its ;,. i failing is that it needs the production rates ofthe producers
to be reasonably si,, i. i

Key words: distributed computing, distributed queue, 1 C'' I "... modeling, producer/consumer simulation.

INTRODUCTION


The distributed producer/consumer problem consists of a set of producer processes that supply items to a set of
consumer processes. Each of the processes involved resides on a different machine, with the machines being connected by
a network. Communication between the producers and consumers is accomplished via message passing.

One example of a distributed producer/consumer problem comes from the High Energy Physics community. High
Energy Physics research typically involves particle collision experiments. These experiments generate huge volumes of
data which are stored on a series of tapes The data itself consists of hundreds of millions of independent events, where
each event describes a collision between two or more particles2. Since the events are independent, they can be processed



1 F. J. Rinaldo and M. R. Fausey, "Event Reconstruction in High-Energy Physics," Computer, June 1993,
p. 68.

2 Rinaldo and Fausey, p. 68.










in parallel. A reasonably cost effective way to do this would be to employ a series of workstations connected by a network.
Several workstations could be reading the data from the tapes and thereby acting as producers, while other workstations
would be used to process the events, thereby acting as consumers. Thus, we have a distributed producer/consumer prob-
lem. As it turns out, the High Energy Physics community has been moving in this very direction for several years now, as
is demonstrated by the work on UFMULTI II3 at the University of Florida and Cooperative Processes Software4at Fermi-
lab. This work is motivated by UFMULTI II.

Existing Solutions


Two existing solutions to the distributed producer/consumer problem are the bag of tasks5 and the concurrent
pool6. The bag of tasks solves a special case and the concurrent pool is designed for a concurrent, shared memory environ-
ment7.

Bag of Tasks

The bag of tasks approach uses one producer and multiple consumers. The producer does not actually produce
anything, but instead manages a bag. While the bag of tasks approach is not a general solution to the distributed pro-
ducer/consumer problem, distributed producer/consumer problems can be altered so that the bag of tasks approach can be
used. This is accomplished by having the actual producers place the items they produce into the bag. The obvious disad-
vantage to this approach is that the bag becomes a bottleneck. A variation on this idea has been successfully employed in
the implementation of the NetQueue mechanism used in UFMULTI II8.

Concurrent Pools

The concurrent pools approach is based on the notion of a pool. A pool is a collection to which items can be
added and removed9. A concurrent pool divides the pool into segments, where each segment resides on the same proces-


3 Paul Avery et al., "A New Approach to Distributed Computing in High Energy Physics," Proceedings of
the 26th International Conference on High Energy Physics, 1992, p. 5.

4 Rinaldo and Fausey, p. 70.

5 G. R. Andrews, "Paradigms for Process Interaction in Distributed Programs," ACM Computing Surveys,
23, No. 1(1991), 85.

6 U. Manber. "On Maintaining Dynamic Information in a Concurrent Environment," SIAM Journal on
Computing, 15, No. 4 (1986), 1130.

7 Manber, p. 1130.

8 Aric F. Zion, "NetQueues: A Distributed Processing System for High-Energy Physics," Thesis,
University of Florida, 1992.

9 D. Kotz and C. S. Ellis, "Evaluation of Concurrent Pools," In Proceedings of the Third Annual Parallel
Processing Symposium, p. 378.










sor as one of the processes which utilizes the concurrent pool. The items in the pool are distributed among the segments
comprising the pool, and the processes using the pool work primarily with the local segment of the pool. When a local
segment of the pool can not meet a request from the processes) it serves, it finds another segment of the concurrent pool
which is not empty, and copies over half of the items.1.

Distributed implementations of the concurrent pool concept have been used successfully in various applications,
such as DIB A Distributed Implementation of Backtracking However, when used in a distributed producer/consumer
situation, concurrent pools have some drawbacks. One such drawback is the bunching phenomenon, which can signifi-
cantly decrease performance12. A second drawback is that, due to the way the concurrent pools algorithm redistributes
items stored in the pool, items may cross the network multiple times. If the items are large, then having the items cross the
network multiple times drives up the load on the network and may result in a performance penalty.

The distributed pool algorithm presented in this paper solves the distributed producer/consumer problem while
avoiding the pitfalls of the existing solutions. The algorithm is presented in detail in Section 2, as are the results of a per-
formance study of the algorithm. Section 3 presents the conclusions drawn from the results of the performance study pre-
sented in Section 2, and presents guidelines for the use of the distributed pool algorithm.

Algorithm Description


The distributed pool algorithm was designed specifically to solve the distributed producer/consumer problem. As
such, all of the assumptions underlying the distributed producer/consumer problem apply to the distributed pool algorithm.
However, in addition to those assumptions, we add one more: the size of an item produced by a producer is large enough
that the cost of transferring an item from one process to another is significant. This assumption is important since it leads
to algorithms designed to minimize the number of times that an item must be transferred. This is in contrast to the concur-
rent pools algorithm, where items may be transferred multiple times before finally being consumed.

Given these assumptions, the distributed pool algorithm operates as follows:

Each producer has a pool associated with it. The pool resides on the same machine as the producer.

Whenever a producer produces an item, it places the item into its associated pool. If the pool is full,
the producer blocks until the pool is no longer full.

Whenever a consumer needs an item to consume, it can make up to X attempts to get one, where X is a
parameter of the algorithm. Each attempt consists of randomly choosing a pool, requesting an item



10 Kotz and Ellis, pp. 378-379.

11 R. Finkel and U. Manber, "DIB A Distributed Implementation of Backtracking," ACM Transactions
on Programming Languages and Systems, 9, No. 2 (1987), 236.

12 Kotz and Ellis, p. 382.

















































ELSE
Send pool empty message
END IF
END IF
END IF
END LOOP
END PROCEDURE


to consumer


Figure 3: Pseudocode for the Consumer


PROCEDURE Producer
LOOP
WHILE NOT PoolFull DO
Produce Item
Send Message Containing Item to Pool
END WHILE
END LOOP
END PROCEDURE


Figure 1: Pseudocode for a Producer

PROCEDURE Pool
LOOP
Get Message
IF message is from producer THEN
IF ConsumerQueue is empty THEN
Add item to pool
IF pool is full THEN
PoolFull = TRUE
END IF
ELSE
Remove message from Consumer Queue
Send item in message from producer to consumer
END IF
ELSE
IF NOT Pool Empty THEN
Remove item from pool
Send item to consumer
PoolFull = FALSE
ELSE
IF message type is "block" THEN
Put message in Consumer Queue


Figure 2: Pseudocode for the Pool

PROCEDURE Consumer
LOOP
X = # of attempts allowed
REPEAT
X = X 1
Choose pool
IF X > 0 THEN
Send "no block" request to pool
ELSE
Send "block" request to pool
END IF
Wait for response from pool
UNTIL Response <> Pool Empty
Consume Item
END LOOP
END PROCEDURE










from that pool, and getting either an item from the pool or a response indicating that the pool is empty.
If the consumer has made X attempts and still has not obtained an item to process, it blocks on the last
pool contacted and waits for that pool to provide it with an item to process.

For purposes of clarification, Figures 1, 2, and 3 present pseudocode which describes the operation of the pro-
ducer, the pool, and the consumer, respectively. The pseudocode assumes that, when a message is sent to a pool, the
sender blocks until it receives a reply. The poolhowever, never blocks after sending a message. All communication is as-
sumed to be reliable.

The pseudocode given in Figures 1 and 2 assume the existence of a flag, PoolFull, which is available to both the
producer and the pool. Since both the producer and the pool reside on the same machine, the PoolFull flag can be imple-
mented using standard IPC mechanisms.

Figures 2 and 3 assume that the consumers can issue two different types of requests. The first type of request is a
"no block" request, which indicates to the pool that, if the pool is empty, it should return a pool empty message. The sec-
ond type of request is a "block" request, which indicates to the pool that the consumer will block until the pool returns an
item. Since the consumer will only block after X unsuccessful attempts to get an item, the consumer will send "no block"
requests for its first (X 1) attempts. For its final attempt, the consumer will use a "block" request, which will cause the
consumer to block until the chosen pool returns an item. Using two types of request is just one method of implementing
the consumers prescribed behavior.

The distributed pool algorithm solves the distributed producer/consumer problem, but the question is how well?
The algorithm is randomized, so it should distribute the request load evenly over the available pools. The pools at the pro-
ducers should handle variations in the request load. In order to determine how well the algorithm works, and under what
circumstances, we made a performance study, which is the subject of the next section.

Performance Analysis


Algorithm Simulation

In order to study the properties of the distributed pool algorithm, a discrete event simulator was built. Each execu-
tion simulated the operation of the algorithm for 6 hours. The inter-production and inter-consumption times were modeled
using exponential distributions. The time consumed by a failed attempt to get an item to process (retry interval) was set to
a constant 0.1 seconds. Each producer had a pool with 10 entries. Confidence Intervals for the resulting data, calculated
with a 95% level of confidence, were within 2%.

Measuring Performance

The performance measures used were actual throughput, achieved throughput, and the average number of requests
per item consumed (ANRIC).










Absolute throughput. Absolute throughput is the total number of items consumed by a (the) consumers) during the
simulation, and is reported directly by the simulator.

Achieved Throughput. Achieved throughput is the absolute throughput expressed as a percentage of the maximum pos-
sible throughput. Achieved throughput was chosen as a performance measure because it is a measure of efficiency, i.e.,
how well the producer/consumer/system being measured is living up to its potential.

Average number of requests per item consumed (ANRIC). ANRIC is the number of messages required for the coor-
dination of the producers and consumers. It is our measure of overhead.

Simulation Experiments

Given the wide array of variables offered by the simulator, examination of each variable's effect on the distributed
pool algorithm was beyond the scope of this paper. Therefore, it was decided to direct the research towards answering
questions about the use of the distributed pool algorithm in practical situations. As such, the examination of variables was
limited to those involved in answering the following questions:

What kind of throughput can be expected from this algorithm?

How much overhead does the algorithm require?

For a given level of consumption/production, how many consumers/producers should be used?

Given an existing producer-consumer situation, what happens if we add another producer or consumer?

How many attempts should consumers have to obtain items?

What happens if the producers produce at different rates?

In order to answer these questions, several series of simulations were set up. Each was designed to provide an-
swers to one or more of the questions being asked. Additionally, several of the series were run multiple times in order to
provide confidence intervals.

Simulation Results

For a given level of consumption, how many consumers should be used?

The first simulation series, which was designed to answer this question, held both consumption and production
constant. This means that the consumption rates of the consumers were altered as consumers were added or removed from
a simulation so that the total level of consumption remained constant from one simulation to the next. Since the level of
consumption was the same in all of the simulations, intuition would indicate that the throughput would remain the same
from one simulation to the next. This, however, was not the case. As it turns out, dividing the level of consumption
among many slow consumers results in better throughput than dividing it among just a few fast consumers. This holds
true regardless of the number of producers or the number of attempts that a consumer has to obtain an item. We can see
this effect in Figures 4 through 8.



























S 65

60
5ss

50






-'e0) 0) Producers
Consumers o L


Figure 4: Achieved Throughput vs. consumers and producers for the case where consumption equals production and
consumers have 1 attempt to obtain items to consume


Figure 5: Achieved Throughput vs. consumers and producers for the case where consumption equals production and
consumers have 5 attempts to obtain items to consume





















. 97

0 96
I-
" 95-
| 94-
4 93


Figure 6: Achieved Throughput vs. consumers and producers for the case where consumption equals production and
consumers have unlimited attempts to obtain items to consume

Figures 4, 5, and 6 demonstrate that using many slow consumers results in better throughput regardless of the num-
ber of producers used. Figure 4 shows the results of the simulations where each consumer had just one attempt to get an
item to process. Since the maximum throughput for these simulations varied by less than one percent, the changes in
achieved throughput seen here are due to increases in actual throughput.

Figures 5 and 6 depict the results of the same simulations as Figure 4, but with the consumers having an increased
number of attempts to get an item to process. Figure 5 represents the case where the consumers had 5 attempts to obtain
an item to process, and Figure 6 depicts the case where the consumers had an unlimited number of attempts to obtain an
item to process. As with Figure 4, achieved throughput increases as the number of consumers used to achieve the level of
consumption is increased. This increase occurs regardless of the number of producers used.

Figures 7 and 8 demonstrate that increasing the number of consumers increases throughput regardless of how
many attempts consumers have to obtain items to process. Figure 7 shows the results from simulations with one producer,
and Figure 8 depicts the results from simulations with 25 producers. What these two figures show is that throughput is al-
ways increased as the number of consumers is increased. They also show that increasing the number of consumers gener-
ates the largest gain when consumers have fewer than three attempts to get items to process. Furthermore, once consumers
have five or more attempts to get items to process, the gains from increasing the number of consumers are minimized.

Last, but not least, we have the issue of overhead. Since the average number of requests per item consumed (AN-
RIC) is a measure of the amount of overhead required by the algorithm, how is the ANRIC affected by the use of many































- 99
25
0

Consumers


Attempts Allowed


Figure 7: Achieved Throughput vs. consumers and attempts for the case where consumption equals production and there
is only 1 producer


Consumers


Attempts Allowed

Figure 8: Achieved Throughput vs. consumers and attempts for the case where consumption equals production and there
are only 25 producers










slow consumers instead ofjust a few fast consumers? Intuitively, one would expect the overhead involved in coordinating
the activities of the consumers to increase in some fashion as the number of consumers is increased, except in the case
where all consumers have only one attempt to get an item to process. Since the case where consumers have one attempt to
obtain an item is a special case with a trivial solution, we will ignore it and address the general case only. As Figure 9
shows, the ANRIC does increase as the number of consumers is increased. Since Figure 9 shows the results of the case
where the consumers had an unlimited number of attempts to obtain an item to process, it represents the worst case, the
limit to which ANRIC can rise for the situation being simulated. For cases where the consumers had a fixed number of at-
tempts in which to obtain an item, the peak of the graph will be below the fixed limit, as in Figure 10, which depicts the
case where consumers had five attempts to obtain an item to process. Thus, while increasing the number of consumers
used to achieve a given level of consumption will result in greater throughput, the extra throughput comes at a price: in-
creased overhead. It is interesting to note, though, that the increase in overhead is also dependent upon the number of pro-
ducers and the number of attempts that consumers have to obtain an item to process. While the number of consumers is
less than the number of producers, increasing the number of consumers has very little effect upon ANRIC. However, once
the number of consumers is equal to or greater than the number of producers, increasing the number of consumers has a
larger impact upon ANRIC.

Now that we have determined that it is better to use many slow consumers to achieve a given level of consumption,
we must consider why this is so. As it turns out, the reason for the better performance obtained by using many slow con-
sumers is that fewer items are being lost to waiting. The number of items lost to waiting by a consumer is the consump-







26.00



21.00



16.00



11.00


1
6.00

9 Producers
1.00 25
99 75 50 25 20 1510 9 8 7 6 5 4 3 2

Consumers
Figure 9: ANRIC vs. consumers and producers for the case where consumption equals production and consumers have
unlimited attempts to obtain items to consume


























2.00


1
1.50
5
9 Producers
1.00- 9 25
99 75 50 25 201510 9 8 7 6 5 4 3 2 1

Consumers

Figure 10: ANRIC vs. consumers and producers for the case where consumption equals production and consumers have 5
attempts to obtain items to consume

tion rate of the consumer multiplied by the total time that consumer spent waiting. The total number of items lost to wait-
ing is then the sum of the items lost to waiting by each consumer. Lets assume that the total number of items lost to wait-
ing is constant. Given this, then, if we were to double the number of consumers in a simulation, each consumer would
have to lose half as many items as before. Since the level of consumption is constant here, when we double the number of
consumers we must cut their consumption rates in half. Cutting the consumption rate in half automatically causes each
consumer to lose half as many items as before, which means that the time spent waiting by each consumer must remain
constant. Since we found that the total number of items lost to waiting decreases, and since the decrease in consumption
rates offsets the increase in the number of consumers in our calculation, then the time spent waiting by each consumer
must decrease as the number of consumers is increased.

For a given level of production, how many producers should be used?

The first simulation series, which was designed to answer this question as well, held consumption and production
constant. This means that, as producers were added or removed from simulations, the production rates of the producers
were altered to maintain a constant level of production. Given this, intuition suggests that throughput would decline as the
number of producers was increased; the producers are slower so blocking times are longer.

The results from the simulations for this case show that our intuition is both right and wrong. For those simula-
tions where the consumers had only one or two attempts to get items to process, the intuitive explanation is right on the
mark. Figure 4 shows, as expected, that throughput decreases as the number of producers increases. But, as the number of










attempts to obtain an item is increased, the intuitive explanation begins to falter. Figure 5 shows that, when attempts to get
an item reaches 5, increasing the number of producers improves throughput until some limit is reached, after which
throughput declines as expected. This behavior was observed until the number of attempts to obtain an item reached 8, at
which point the limit exceeded the range of values tested and therefore no longer appeared in the data.

Turning now to the issue of overhead, Figure 10 shows the average number of requests issued by consumers (AN-
RIC) when attempting to get an item to consume. While Figure 10 shows the case where the consumers had 5 attempts to
get an item to process, the shape of the surface is virtually identical for any case where the number of attempts is greater
than two. The only difference between the cases is in the scaling of the Z axis. The greater the number of attempts that
consumers have, the higher the value associated with the peak in the graph. What Figure 10 shows is that increasing the
number of producers used to achieve a given level of production reduces the ANRIC. It is interesting to note that the re-
duction in ANRIC is greatest when the ratio of consumers to producers is greatest. As this ratio decreases, the reduction
in ANRIC due to increasing the number of producers decreases also. Furthermore, once the number of producers equals
or exceeds the number of consumers, the effect of increasing the number of producers is minimal.

Given an existing producer-consumer situation, what happens if we add another consumer?

The second series of simulations was designed to answer this question. In this simulation series, the level of con-
sumption was allowed to vary with the number of consumers in the simulation while the level of production was held con-
stant. Given this arrangement, we can break existing producer/consumer situations into two groups: those where the level
of consumption is less than the level of production, and those where the level of consumption is greater than or equal to
the level of production. For existing situations where the level of consumption is less than the level of production, intui-
tion says that adding a consumer should increase throughput with little effect upon overhead. For existing situations where
the level of consumption is greater than or equal to the level of production, adding another consumer should have no effect
upon throughput but it should increase overhead as the extra consumer competes with the existing consumers for the lim-
ited number of items available. As it turns out, intuition is indeed correct.

Breaking the existing situations into two cases was done to answer the question as asked. However, the results pre-
dicted for these two cases can be combined into a single set of results which is more general and easier to discuss. In gen-
eral, then, adding a consumer to an existing situation should increase the throughput with only a minimal increase in
overhead until the level of consumption exceeds the level of production, at which point adding another consumer only
serves to drive up overhead, not throughput.

Figure 11 shows the results of simulations in which the consumers had an unlimited number of attempts to obtain
an item to process. This figure demonstrates that actual throughput for the simulation increases as the number of consum-
ers increases, at least until some limit is reached. After that point, throughput is unaffected by further increases in the level
of consumption.

Figure 12 shows the average achieved throughput for the producers in the same simulations used to produce Fig-
ure 11. As we can see, Figure 12 looks almost identical to Figure 11. As consumers are added, the achieved throughput of
the producers increases until it reaches 100%. At this point, the producers have reached their maximum ability to produce






















30000
25000
20000
15000
10000
99
25 5000
10 0
Producers /

1 M Consumers

Figure 11: Actual Throughput vs. consumers and producers for the case where consumption varies with the number of
consumers and consumers have unlimited attempts to obtain items to consume


25

Producers 0

1 2 3 4 5 6 7 8 9 10111213141516
1 Consumers
Figure 12: Average Achieved Throughput for Producers vs. consumers and producers for the case where consumption
varies with the number of consumers and consumers have unlimited attempts
























L I.UU )

16.00

11.00
99
6.00
25
10 1.00

Producers -
4 0)(5)
1 CM Consumers

Figure 13: ANRIC vs. consumers and producers for the case where consumption varies with the number of consumers
and consumers have unlimited attempts to obtain items to consume

and everything produced is being consumed. Thus, actual throughput has reached its limit. Increasing consumption be-
yond this point can not result in any additional items being consumed.


Figure 13 shows the average number of attempts that consumers had to use to obtain an item to consume (AN-
RIC). Figure 13 demonstrates that, while the level of consumption is well below the level of production, there is very little
overhead. But, as the level of consumption begins to approach and then exceed the level of production, the overhead rises.
When the level of consumption exceeds the level of production, the overhead rises almost linearly with the increase in con-
sumption.


Given an existing producer/consumer situation, what happens if we add another producer?


We found this situation to be very similar to that of adding a consumer. As long as the level of production is less
than the level of consumption, adding a producer increases throughput and decreases overhead. But when the level of pro-
duction exceeds the level of consumption, only overhead is affected, and it continues to decrease.


What happens if the producers produce at different rates?


The fourth simulation series was designed to answer this question. In this simulation series, the level of produc-
tion was equal to the level of consumption, and the levels of consumption and production were the same in all simulations.
The number of consumers used to achieve the level of consumption was varied while the number of producers was held
constant at 7. Within a simulation, all of the consumers consumed at the same rate while the production rates of the pro-

















80



60

2 50



30

20

10 25

0 Consumers
0" 0 .n |< '


Standard Deviation in Production Rates
Figure 14: Achieved Throughput vs. consumers and standard deviation in production rates for the case where consumers
had only 1 attempt to obtain items to consume

ducers were allowed to be different. Given this, intuition would suggest that throughput declines as the variance in the pro-
duction rates of the producers increases. Since the load imposed by the consumers is uniformly distributed over the pro-
ducers, the slower producers are not going to be able to keep up with their portion of the load. This results in the pools
associated with these slow producers being empty more often, which causes more waits. The additional waits cause the
consumers to lose more items to waiting, thereby reducing throughput. As far as overhead is concerned, one would expect
that increasing the number of attempts that consumers have to obtain items should reduce the impact of the variation in
production rates by allowing consumers to avoid long waits. Thus, as the variation in production rates increases, the con-
sumers will use a greater percentage of their allowed attempts in order to avoid long waits. This should result in an in-
crease in overhead. As it turns out, the intuitive explanation is generally correct.

Figure 14 represents the results of simulations where the consumers had only 1 attempt to obtain items to process.
Figure 14 shows that, as predicted, the throughput declines as the variation in production rates increases. Figure 15 depicts
the case where consumers had an unlimited number of attempts to obtain items to consume. Figure 15 shows that, even
with an unlimited number of attempts to obtain items, throughput still declines dramatically. However, since the through-
put does not decline as rapidly or to the same extent as in Figure 14, Figure 15 indicates that providing consumers with ex-
tra attempts reduces the decline in throughput. Figure 16 shows how the achieved throughput is affected by both the
variation in production rates and the number of attempts that consumers have to obtain items. This figure, which repre-
sents the case where there are 25 consumers, is representative of all the cases observed. It clearly shows that increasing the
































99
65

60 6 Consumers

0 0 d ." Ti
O ddoL r- O0l 0 a
Standard Deviation in Production Rates
Figure 15: Achieved Throughput vs. consumers and standard deviation in production rates for the case where consumers
had unlimited attempts to obtain items to consume


infinite


I -4 Attempts
o 1 g0 Allowed
0 LO 0 5 O


Standard Deviation in Production Rates -
Figure 16: Achieved Throughput vs. attempts and standard deviation in production rates for the case where there are 25
consumers



















21


16


11


6 25

SConsumers
1 5

0 0 6O M 06 0 O (O

Standard Deviation in Production Rates
Figure 17: ANRIC vs. consumers and standard deviation in production rates for the case where consumers have unlimited
attempts to obtain items to consume

number of attempts that consumers have can reduce the decline in throughput due to increases in the variation of the pro-
duction rates of the producers.

Figure 17 represents the average number of requests per item consumed (ANRIC) for the case where consumers
had an infinite number of attempts to obtain an item to consume. As expected, ANRIC increases as the variation in pro-
duction rates increases.

How many attempts should consumers have to obtain items?

Setting the number of attempts provides a mechanism for balancing the overhead of the distributed pool algorithm
against the throughput achieved. In general, we can improve throughput, but at the cost of higher overhead.

We found that setting the maximum number of attempts to 3 usually provided a good level of throughput, while
sharply limiting overhead. Setting the maximum number of attempts to 6 provides better throughput, and increasing the
number of attempts beyond 6 usually has almost no benefit. Furthermore, ANRIC is usually low even with a maximum
of 6 attempts.

We do not have space to show all of the data that we use to base our conclusions on, but we can show some exam-
ples. Figure 8 shows the typical case when the production rate equals the consumption rate. Figure 18 shows the typical
case when the production and consumption rates are different. Figure 16 shows the typical case when the production rates
of the individual producers is varied. In Figures 8 and 18, the achieved throughput is close to 100% for both 3 and 6 at-






















.- 30060
0 25060
20060
S15060
10060-
16
5060- 13
60- 10
) O 0 L 0 D Consumers


Attempts Allowed

Figure 18: Actual Throughput vs. attempts and consumers for the case where consumption varies with the number of
consumers and there are only 25 producers

tempts for most settings. In Figure 16, 6 attempts are required to achieve a good level of throughput when the variance in
the individual production rates becomes large.

What kind of throughput can be expected from this algorithm?

The answer to this question has already been presented, albeit in bits and pieces, in the answers to the previous
questions. Given this, only a brief summary will be provided here.


Under ideal conditions, the distributed pool algorithm is capable of near 100% achieved throughput performance.
Under less ideal conditions, 90% or greater achieved throughput can easily be obtained if the consumers have 6 or more at-
tempts to obtain items to consume and the variance in production rates of the producers is low. As the variance in the pro-
duction rates becomes high, performance of the algorithm deteriorates rapidly. While performance declines as the number
of attempts consumers have to obtain items drops below 6, it deteriorates markedly when consumers have less than 3 at-
tempts. Overall, then, as long as the consumers have 6 or more attempts to obtain items and the variance in the production
rates of the producers is low, excellent performance can be expected from this algorithm.


How much overhead does the algorithm require?

As with the previous question, the answer to this question has already been presented, in bits and pieces, as part of
the answers to earlier questions. To summarize briefly, the overhead required by the algorithm can be controlled by setting
the number of attempts that consumers have to obtain items to process. The number of attempts that consumers have is an










absolute limit on how high the overhead can be, and the actual overhead will almost always be less. For most cases, giving
consumers 6 attempts to obtain items should yield excellent throughput with a worst case overhead of six requests per item
processed. Should this be too much overhead, the number of attempts that consumers have can be reduced, which will re-
duce the overhead, although there may be a reduction in throughput as well. In any case, though, the overhead of the algo-
rithm can easily be controlled.

Guidelines for the usage of the distributed pools algorithm

Based upon the simulation results presented in this paper, when using the distributed pool algorithm, the following
guidelines should be followed in order to obtain the best results:

In order to avoid wasted capacity, the level of consumption should be approximately equal to the level
of production.

For a given level of consumption, use as many consumers as possible.

For a given level of production, the total number of producers used should be greater than 1 but less
than or equal to the total number of consumers used.

If the variance in the production rates of the producers is small, then giving consumers 6 attempts to ob-
tain items to consume should result in good performance. If the variance in production rates is large,
then the number of attempts that consumers have must be increased.

If the aforementioned guidelines are followed, and if the variance in the production rates of the producers is low,
then the distributed pool algorithm should attain better than 90% achieved throughput.

CONCLUSIONS


The distributed pools algorithm is a simple, efficient solution to the distributed producer/consumer problem. Un-
der ideal conditions, it can achieve near 100% efficiency while keeping overhead low. Under more common conditions, it
can easily achieve more than 90% efficiency with low overhead. Its major failing is that it needs the production rates of
the producers to be reasonably similar. If the variance in the production rates of the producers becomes large, then per-
formance begins to decline quickly.

Directions for future work include:

Evaluation of the effects of pool size, retry interval, and the variance in consumption rates of the con-
sumers on the algorithm's throughput, overhead, and fairness.

Evaluation of assigning producers selection probabilities based upon their production rates.











REFERENCES


Andrews, G.R. "Paradigms for Process Interaction in Distributed Programs." ACM Computing Surveys, 23, No. 1 (1991), 49-90.

Avery, P, C. Chegireddy, J. Brothers, T Johnson, and A. Zion, "A New Approach to Distributed Computing in High Energy
Physics." Proceedings of the 26th International Conference on High Energy Physics, 1992.

Finkel, R. and U. Manber. "DIB A Distributed Implementation of Backtracking." ACM Transactions on Programming
Languages and Systems. 9, No. 2 (1987), pp. 235-256.

Kotz, D. and C. S. Ellis. "Evaluation of Concurrent Pools." Proceedings of the Third Annual Parallel Processing Symposium, pp.
378-385.

Lynch, N. A., and M. J. Fischer. "On Describing the Behavior and Implementation of Distributed Systems." Theoretical
Computer Science, 13 (1981), pp. 17-43.

Manber, U. "On Maintaining Dynamic Information in a Concurrent Environment." SIAM Journal on Computing, 15, No. 4
(1986), pp. 1130-1142.

Rinaldo, F J., and M. R. Fausey. "Event Reconstruction in High-Energy Physics." Computer, June 1993, pp. 68-77.

Zion, A. F "NetQueues: A Distributed Processing System for High-Energy Physics." Thesis, University of Florida, 1992.




University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs