An Analytical Performance Model of Robotic Storage Libraries
Dept. of CISE, University of Florida
June 25, 1996
Large scale scientific projects generate and use huge amounts of data. For example, the NASA
EOSDIS project is expected to archive one petabyte per year of raw satellite data. This data is made
automatically available for processing into higher level data products and for dissemination to the sci-
entific community. Such large volumes of data can only be stored in robotic storage libraries (RSLs)
for near-line access. A characteristic of RSLs is the use of a robot arm that transfers media between a
storage rack and the read/write drives, thus multiplying the capacity of the system.
The performance of the RSLs can be a critical limiting factor of the performance of the archive system.
However, the many interacting components of a RSL make a performance analysis difficult. In addition,
different RSL components can have widely varying performance characteristics. This paper describes
our work to develop performance models of a RSL. We first develop a performance model of a RSL in
isolation. Next, we show how the RSL model can be incorporated into a queuing network model. We
use the models to make some example performance studies of archive systems.
The models described in this paper, developed for the NASA EOSIDS project, are implemented in C
with a well-defined interface. The source code and accompanying documentation are available through
Large scale scientific projects generate and use huge amounts of data. For example, the NASA EOSDIS
project is expected to archive one petabyte per year of raw satellite data . This data is made automatically
available for processing into higher level data products and for dissemination to the scientific community
(see, for example, the reports in ). Automatic management of such large data sets requires the use of
tertiary storage, typically implemented using robotic storage libraries (RSLs). In addition to EOSDIS and
related projects, many organizations and scientific disciplines make use of mass storage archives (for example
high energy physics  and digital libraries ).
The database community has also become interested in the use of RSLs. [13, 6, 36, 35]. This interest is
motivated in part by scientific database problems such as EOSDIS. Another motivation for integrating RSLs
with on-line database systems is to facilitate data warehousing.
Tertiary storage is required when the managed data set becomes too large to store economically with
conventional magnetic disk devices. The point at which tertiary storage becomes necessary is an economic
tradeoff. Currently, it seems that tertiary storage is needed to manage more than a terabyte of data. A RSL
is much slower than magnetic disk storage, and data access latencies can run into minutes even on unloaded
systems. However, RSL-resident data can accessed automatically. Hierarchical storage management systems,
such as Unitree, Filestore, and Amass, provide the illusion that the RSL is an extension of the file system.
Access to archived data incurs a short delay. The storage capacity of a data system can also be increased
by using off-line storage -i.e. tape racks with human operators. Access latencies with off-line storage can
be very large, ranging into hours or days, but the data storage capacity is limited only by the size of the
warehouse that one can afford to rent. Since RSL provides data volumes and access latencies between those
provided by on-line and off-line storage, it is often referred to as near-line storage. A cost analysis of on-line,
near-line, and off-line archives can be found in .
A characteristic of RSLs is the use of removable media and a robot arm. The removable media (e.g.
magnetic tape, optical disk, etc.) are normally located in a storage rack. To service a request for a file,
the robot arm fetches the proper media from the storage rack and delivers it to a read/write drive. The
media is accessed in the normal way to fetch the file. Finally, the media is returned to the storage rack.
The capacity of RSL is the product of the capacity of the media and the size of the storage rack. Recent
magnetic tapes have a data capacity on the order of 10 Gbytes, and storage racks sizes range from 10 to
1000 media (approximately). The time to fetch and mount the media which holds the requested file can be
a large component of the access latency.
The performance of the RSLs can be a critical limiting factor of the performance of the archive system.
Given the high data request rates expected for EOSDIS, attention to handling these requests efficiently is
critical [28, 17]. However, the many interacting components of a RSL make a performance analysis difficult.
In addition, different RSL components can have widely varying performance characteristics.
This paper describes our work to develop performance models of tertiary storage. We first develop a
performance model of a RSL in isolation. Next, we show how the RSL model can be incorporated into a
queuing network model. Finally, we model fork-join jobs to study the tradeoffs of using multiple devices.
We use the models to make some example performance studies of archive systems.
The models described in this paper, developed for the NASA EOSIDS project, are implemented in C with
a well-defined interface. The source code and accompanying documentation are available through WWW
1.1 Previous Work
Considerable work has been done to develop performance models of mass storage. Rahm  presents a
simulation study of a database system with a hierarchy of storage devices. Ramakrishnan and Emer 
present a queuing model of a client/server file system. Drakopoules and Merges  present a closed queuing
model of a client/server storage system with hierarchical storage. Kelly, Haynes, and Ernest  discuss a
benchmark for network storage systems. Hauser, Rivera, and Thoma  discuss the performance of their
networked WORM server.
Some work has been done to characterize the performance of mass storage devices. Waters  presents
a validated model of seek times in hard disk drives. More recently, Ruemmler and Wilkes  present a
detailed model of a modern disk drive, and discuss the difficulties inherent in I/O modeling. Christodoulakis
and Ford  and Christodoulakis  present analytical performance models of optical drives. Chinnaswamy
 presents performance models of a streaming tape drive to investigate the benefit of a cache.
Models of disk arrays resemble the models presented in this paper in several aspects. Burkhard, Claffy,
and Schwarz  present a simulation study of a disk array scheme. Lee and Katz  and Yang, Hu and
Yang  present analytical models of disk arrays. Chen et al.  and Thomasian  present surveys of
research in RAID modeling.
Several authors have modeled a RSL. Butturini  presents the results of a simulation study of an optical
disk jukebox system. Hevner  presents a model of an optical jukebox that is used for a database appli-
cation. Howard  gives a performance model for data duplication from an archive. Finestead and Yeager
 give performance measurements of a Unitree file server at the National Center for Hull and Ranade 
present measurements of tape loading and unloading, and of data throughput, in a tape silo. Supercomputer
Applications. Bedet et al.  discuss the results of a detailed simulation model of the Goddard DAAC.
Pentakalos, Menasce, Halem, and Yesha  develop a queuing network model that incorporates a RSL.
Daigle, Kuehl, and Langford  present a queuing model of an optical disk jukebox. Golubchik, Muntz and
Watson  analyze tape striping on a RSL.
The analyses most closely related to the one in this paper are [31, 12, 19]. The analysis in  gives
a detailed model of access times to data on an optical platter. However, only one drive is permitted and
contention for the robotic arm is not modeled. In , the authors present a detailed model of a data center,
incorporating RAID disk caches and user computation. However, the authors assume that contention for
the drives in the RSL is negligible, and model the RSL as a delay server. Contention for the robotic arm
due to batch arrivals is modeled in , but contention between jobs is not modeled.
The contribution of this work is to present a validated model of a RSL that accounts for batch arrivals,
multiple drives, contention for the robotic arm, and realistic operation. We show how the model can be used
to make a variety of data layout and device comparison studies. Finally, we show how to incorporate the
RSL model into a queuing network model.
2 Model of a Robotic Storage Library
Our model of a RSL is illustrated in Figure 1. Previous studies of mass storage archive log files (see, for
example, [24, 12]) indicate that requests to a mass storage device come in batches. This study has been
corroborated by our studies of access to preliminary versions of the EOSDIS archives (the VO archives)
[1, 16]. As a result, our RSL model model uses batch arrivals.
A user requests that f files be loaded into on-line storage, and these files are distributed over m media
in the RSL. The request is satisfied when every file has been loaded into on-line storage. So, a user request
consists of m jobs, each of which must be completed before the request in finished. A RSL consists of nd
drives, each of which can read or write any of the media in the RSL,1 a storage rack containing the removable
media, and a robot arm for transferring the media between the drives and the storage rack. The model of a
RSL is illustrated in Figure 1.
Poisson batch arrival 0 Drive 1
Figure 1: Architecture of a robotic storage library.
1In some installations, a subset of the drives are designated as read-only or write-only. We will address this complication in
a later version of the model.
The steps taken by a drive in retrieving files from a media is illustrated in Figure 2. When a request
arrives, its jobs are placed in the job queue. If there are jobs in the RSL queue and a drive is idle, the
drive allocates one of the jobs for execution. First, the robot arm fetches the appropriate media from the
storage area and loads it into the drive. If the robot arm is busy serving other drives, the drive must wait
for service. After the media is brought to the drive, it must be mounted. For every file of interest on the
media, the drive must seek to the start of the file, spend a settling time for precise positioning and opening
communications channels, and then transfer the file to on-line storage. After all files have been transferred,
the media is rewound and returned to the storage rack by the robot arm. However, the job is finished once
all of the files have been transferred.
mount File 1 seek File 1 File n seek File n
robot rewind robot
time and transfer time and transfer
fetches time time returns
settling time settling time
the media time time the
Transfer is finished
Figure 2: Steps in loading files from a media.
In the next section, we present an analytical performance model of a RSL system. In this preliminary
model, we make the following assumptions:
1. Requests arrive in a Poisson process.
2. The distribution of the number of media per request and the number of files per request must be
specified. In the model discussion, we assume that the number of media per request and the number
of files per request have geometric distributions. These can be replaced by user-specified distributions
(e.g. empirically determined), but at the cost of requiring the user to specify more parameters.
3. The RSL can contains one robot arm. The robot arm can access every media, and every drive.
4. Every drive can read and write every media.
5. Requests (i.e., jobs) are serviced first-come-first-serve2.
6. The service for a request is completed when the last file of the batch has been read (written).
7. Network or communication channel contention is not significant3.
8. Service times at the drives are independent.
3 Analytical model
A RSL presents many difficulties for performance modeling, including batch arrivals, multiple servers, derived
parameters, and interacting components. The primary component of the RSL model, the M"/G/c queue,
has been studied and solved in the literature . Solving the actual MA /G/c queue is intractable, so the
solution technique is to interpolate between the results for the MA /M/c queue and the MA /D/c queue
using the coefficient of variation of the service time as the interpolation parameter. Because of the potential
complexity of the batch arrival distributions, we do not use explicit (i.e., generating function) formulas.
Instead, we numerically solve the recurrence equation that defines the state occupancy probabilities. If the
occupancy probabilities of the first N states must be computed for an error bound of c, then solving the
MA /M/c queue requires O(N2) time and solving the Mf/D/c queue requires O(N3) time.
Fortunately, we can take advantage of the nature of the problem to speed up the solution times. The
state occupancy distributions eventually converge to a geometric distribution (i.e., PN = tOPN-1). Therefore
the recurrence equations only need to be solved up the first No states, and the remainder can be computed
using the to ratio (or perhaps the performance metrics can be computed directly). No depends primarily on
the distribution of the size of the batch arrival. Fortunately, the batch arrival distribution will have a short
tail one cannot request that more media than exist in the storage rack be mounted, and usually only a few
media are required to satisfy a request. By using these tricks, we implemented batch queue solvers that are
fast enough to be incorporated into a higher level model which calls them many times.
We now define some variables. The following are the inputs to the model.
SA : request arrival rate.
fr : Average number of files per request.
2A simple optimization is to load files for all requests once a media has been mounted. We assume this situation has a
negligible impact on performance in this model
3 Potential model users indicated that communication channel contention is not a problem for their systems. Communication
contention can be incorporated into the seek times or mount times using standard techniques 
m, : Average number of media per request.
b(.) : Distribution of the number of media per request.
nd : number of drives.
Et, Average robot fetch time.
Vt, : Variance of the robot fetch time.
Tit : Media mount time.
Tfs : Time for a full seek across the media.
Tst : Settling time after a seek.
Xb : Transfer rate.
E,,: Average file size.
V, : Variance in the file size.
Erd : Expected time to re-wind the media.
The performance measures are:
Erob : Average delay for the robot fetch.
Prob : Robot arm utilization.
pd : Effective drive utilization utilization.
Pd(.) : Queue length distribution for the drives.
Request : ATL request response time.
We model the RSL as an M"/G/c queue -that is, a queue with Poisson batch arrivals, general service
time distribution, and c servers. The parameters of a Ms/G/c queue are:
A : Arrival rate
Wd : Mean service time
cvd : Co-efficient of variation of service time
b(.) : Batch size distribution
* nd : Number of servers
All but the service time distribution are input parameters, so our analysis is focused on how to compute
Ed and CVd = /V/Ed. To compute queue length distributions and expected waiting times properly, we need
to compute the time that a drive is unable to serve other jobs per media that it serves. This period includes
the time to fetch the media, mount it, seek to each file, transfer each file, rewind and eject the media, and
return it to the storage rack. We will incorporate the time to return the media as part of the media fetch
time, so we have:
drive service = (robot fetch) + (mount time) + (seek time) + (transfer time) + (rewind time)
Analyzing the RSL reduces to analyzing each component of the drive service time. Some of these
components are easy to estimate, others difficult. We will work from easiest to most difficult.
Mount time The mount time Tnt is the time period after the robot arm has placed the media in the
drive and the drive can accept commands to seek to the first file. Typical actions include precise positioning
of the media, drive initialization, and reading a directory sector (to ensure that the proper tape has been
loaded, etc.). These actions are nearly deterministic  and are characteristics of the drive used in the RSL.
Therefore, the mount time is a parameter, and we assume that mounting makes a negligible contribution to
the drive service time variance.
Rewind time The rewind time is time period after the last file has been transferred from the media and
the media can be ejected to the robot arm (it is typical to rewind tapes after use to ensure longevity, hence
the name). The rewind time is device dependent , so we require that the mean and variance of the rewind
time be supplied as parameters.
Transfer time Given that a file of Sz bytes is to be transferred, a commonly accepted model of the
transfer time is a simple linear model:
Txf = a + Sz/Xb
where T1f is the transfer time, Xb is the transfer rate and a is transfer initialization time . In this
model, we will absorb the transfer initialization time into the seek time, so a = 0. Ideally, one could supply
the mean and the variance of the volume of data that is to be transferred from the media. However, this
might be more information than is available. More typically, one can estimate the mean and variance of the
individual files that are transferred. Then, one can use the distribution of the number of files transferred
per media to estimate the mean and variance of the transfer time.
Let bf(.) be the probability density function of the number of files transferred per media. If k files are
transferred, then the mean volume of data transferred is kE,, and the variance is kVz. Therefore, the
second moment of the volume of data transferred is
1 (k) = kV~z + k2Ef
The first and second moments of the volume of data transferred per media is
Evol = YkEskbf(k)
M2, = M,2 (k)bf(k)
To obtain transfer times, we divide by the transfer rate.
Exf = Evoi/Xb
Vxf = (M2, E 2}IXb
If the number of files per request and the number of media per request have a geometric distribution,
and the number of files per media is distributed iid, then the number of files per media has a geometric
distribution also4. If an average of f, files are transferred from each media, then
Exf = fmEsz/Xb
Vxy = fm(V, + EsZ(f 1))/X2
Seek time Unlike the transfer times, there is no commonly accepted model for seek times (magnetic disk
seek times are difficult enough ). The problem is compounded by the random seek distances, disk vs.
single-pass vs. serpentine tape, and fast-seek modes. One simple model is to assume that files are randomly
distributed across a single-pass tape. Suppose that f files are to be transferred. Then :
Eseek Tf + fTtl
M2,k(f) = fTf/(f + 2) + 2f2TtlTf/(f + 1) + f2Tfs
Our solver assumes that user supplies a function which takes the number of files f as parameter and
returns the mean and the second moment of the seek time. The unconditional seek time and variance is
4Note to referee: Here and at other places I suppress some calculations and details in order to save space
Eseek= f=1 Eseek(f)bf (f)
Meek f=l Meek(f)bf(f)
Vseek = -2eek Eseek
Robot arm service time Computing Erob is more complex, because the fetch time includes contention
for a robot arm. If the ATL is empty and a batch request arrives, then a batch request is made to the robot
arm. The queuing model that is appropriate for modeling the robot arms is the Mx/G/1 queue. Then, Erob
is the response time of the average job.
There are some complications. Suppose that nidle of the drives are idle when a batch of size B arrives. If
B > nidle, the nidle idle drives will request will request service immediately. The remaining B -nidle jobs will
request service from the robot arms as the drives finish serving the preceding jobs. Each of these B nidle
requests will be preceded by a matching request to remove the old media from the drive. Therefore, the
B nidle jobs have a doubled service time requirement. The nidle requests also generate matching requests
to unload the drives, but these requests are not correlated with any requests to load a drive.
Let s(nidle) be the probability that the batch is of size nidle or smaller, and let b(nidle) be the average
size of a batch request, given that the batch is for nidle or fewer jobs. That is,
b(nidle) = i b(i)
The arrivals to the robot arm queue consists of the truncated job arrival stream (with rate A), an arrival
stream that corresponds to jobs that must wait for service from a drive (rate Adoub), and an arrival stream
that corresponds to drives returning media without an immediate fetch of new media (rate Asing). Then,
Adoub(nidle) = A(m, b(nidle)) nidle > 0
Asing(nidle) = Ab(nidle) nidle > 0
Arob(nidle) = A + Adoub + Asing idle > 0
Let brob,n,,ed be the distribution of batch arrivals to the robot, given that nidle drives are idle. Then,
brob,nile(l) = (A b(1) + Asing)/Arob idle > 2
(A b(2) + Adoub)/Arob
A (b(nidle) + (1 s(nid1e))/Arob)
nidle > 2
nidi > 2 and 2 < i < nidl
idle > 2
brob,2(l) = (A b(l) + A,in)/Arob
brob,2(2) = (A b(2) + (1 s(2)) + Adoub)/rob
brob,l(l) = (A + Asing)/rob
brob,l(2) = doub/Arob
If all drives are busy, then all jobs are serviced when a busy drive finishes services. Therefore:
Given the drive queue length distribution Pd, let be the probability that all drives are busy (i.e.,
. = 1 (pd(0) + - +Pd(nd 1)). We can compute the parameters to the robot queue by derandomizing
brob(.) and and Arob(.). For example,
Arob = \ (0) + pd(i)Aob(nd -i)
Then, the parameters to the M'/G/1 queue  representing the robot arm are:
Arrival rate Arob
Service time Erob = Etr
Coefficient of variation cvrob = VV/Erob
Batch size distribution brob ()
The queuing model provides for us the robot arm utilization Pob, and the waiting time for robot arm
service Wrob. The M'/G/1 model returns I- ..," waiting times. That is, the increase in waiting time
due to correlated arrivals are not fully accounted for. Recall that if a job is serviced by a drive that was
busy immediately before starting service, then the job must wait for both the return of the old media and
the fetch of the new media. This correlation is captured in the waiting times by the batch arrival stream
Adoub(nidl,). However, the reported service time accounts for only half of the increased increased robot
service time for these jobs. Therefore, we compute the additional waiting time due to correlated arrivals at
the robot arm to be:
Extra(nidle) = Etr A+Adob(ndlie)
Extra = I' P xtra(O) + Zo Pd()E xtra(nd i)
We compute the robot arm response time to be:
Erob = Wrob + Etr + Extra
The M"/G/1 queue solvers compute the state occupancy distributions, so we compute the variance of
the response time Vrob by using this distribution.
The Drives The parameters to the queue representing the RSL are:
Arrival rate A
Service time Ed = Exf + Eseek + Tnt + Erob + Erwd
Coefficient of variation cvSd = /+V Evobd
Batch size distribution b(.)
Number of servers nd
Since we are interested in the response time of the last job in the batch to finish (i.e., instead of the
average job), we need to modify the response time computation. An efficient algorithm for computing the
response time of the last job in the batch is given in .
The modified Mx/G/c queue provides the batch response time Rbatch, the drive utilization Pd, and
pd(O), ., Pd(nd 1), the probability 0, .., nd 1 servers are busy on a request arrival. The effective drive
service time (and Pd(.)) depends on the robot response time, which in turn depends on Pd(.). We use iteration
to converge to the solution.
Finally, the job is finished when the last file has been transferred, there is no need to wait for the tape
Rrequest = Rbatch Erwd
3.1 Validation Study
We wrote a simple RSL simulator. The simulation accepts batch arrivals, requires that a robot unload and
fetch a media before a drive can service a job, handles multiple drives, and accounts for media rewind times.
The components of Ed and Vd, except for Erob and Vrob, are pre-computed and sampled from an Erlang
We used the following values of the parameters in the validation study.
Sf, = 20.
b(.): Geometric distribution.
n = 4.
Et, = 10.0 seconds.
Vt, = 10.0.
Tt =10.0 seconds.
Xb = 1.0 Mbyte/sec.
We ran four sets of experiments to test the model. In the "large files" experiments, Es, = 50 and
V,, = 100, and Tf, = 50, Ttl = 1. In the "small files" experiments, E,, = 5 and V, = 10, and Tf, = 20,
Tti = 2. We tested the model with m, = 2 and m, = 6.
The results of the validation study are shown in Figures 3 through 6. In each case there is close agreement
between the analytical and the simulation models. The most difficult case is when the files are small and
distributed over an average of six media, because the robot fetch times constitute a large portion of the
drive service times (about 22% of the total drive service time when the robotic arm waiting time is added).
However, the analytical model is accurate enough to predict response times and drive utilizations. Charts
comparing analytical and simulation drive utilizations are shown in Figures 7 and 8.
3.2 Performance Study
A performance model is useful for studying implementation alternatives. In this section, we present three
sample performance studies based on the RSL model.
Conventional wisdom holds that striping or declustering is necessary for obtaining high transfer rates from
tertiary storage (by making use of parallel I/O). So, one should spread the files of a typical request around as
many media as possible. Conventional wisdom also holds that swapping media is a source of great inefficiency
in RSL access, so that one should try to ensure that the files of a typical request are placed on as few media
Neither argument is convincing, unless one has a predictive performance model. We ran the "small files"
experiment with m, ranging between 1.2 and 10. In Figure 9, we plot the response time of a request against
the number of media per request for varying arrival rates (A similar chart can be found in an analysis of tape
striping ). For low arrival rates, setting m,. w nd produces the best results. When A = .0001, setting
m, = 3 results in a 22% lower response time than setting m, = 1.2. For high arrival rates, setting m, = 2
gives lower response times than other choices.
In Figure 10, we plot the drive utilization against m, for varying arrival rates. Increasing m, causes a
linear increase in the drive utilization. As the arrival rate increases, it becomes less likely that all nd drives
are available to service the request. So, distributing the files over a smaller number of media reduces queuing
delays. If the demand on the RSL is expected to be close to the device's capacity, then m, should be small
to increase the maximum throughput of the device.
The question of whether to cluster or decluster the files on the media can be summarized as:
If the expected drive utilization is low  and fast response is important, then declustering can be
a good strategy. However, the decrease in transfer times must be larger than the increase in queuing
If high throughput is important, clustering is a good strategy.
3.2.2 Device Selection
A performance model can be used to determine the most appropriate equipment for an application. Consider
the following hypothetical problem. You need to choose between two devices of approximately equal cost.
The workload is the "small files" workload, except that an average of 40 files are loaded per request.
The first device uses optical disks as the storage media. Seek times are negligible, but the storage capacity
per disk is limited, you can afford to purchase only two drives, and the transfer rate is 1 Mbyte/second. The
second device uses tapes. The capacity of the tapes is larger than the capacity of the optical disks, and you
can afford to purchase four drives. However, the seek times are large.
Because of the difference in the capacities of the media, you need to load an average of 6 media per
request if you use the optical disks, and 2 media per request if you use tape. Using these parameters, we
can plot the average response time of the two RSLs under an increasing workload. This chart is shown in
For the parameters of this study, the optical disk system is better than the tape system if the workload
is light, but worse if the workload is heavy. The fast seek times of the optical disks allow you to transfer
the large number of files quickly. However, the tape based system has a higher data capacity. While the
parameters of this study are engineered to give an interesting comparison, the point remains that device
performance depends on the workload offered to the device.
3.2.3 Number of Drives
Many RSLs allow the user to install a ranging number of drives. Adding drives to a RSL can improve the
performance of the device. But after a threshold, adding drives does not significantly improve performance.
We ran a sample study using the "small files" parameters and four media per request. In Figure 12, we
plot the response time versus the arrival rate for a number of drives varying between 2 and 8. Adding a drive
significantly improves performance up to four drives, but gives less benefit after four drives. In Figure 13,
we plot the drive utilization against the arrival rate. Adding a drive to the RSL increases the capacity of
the device. However, the robot arm will start to become a bottleneck. This can be seen in the non-linear
increase in utilization of some of the curves, for example for nd = 8.
3.3 Multiple Robotic Storage Libraries
A large scale data center is likely to have multiple RSLs. The devices might be acquired to handle data
center growth, or multiple small devices might be less expensive than a single large device. In this section,
we discuss an approximation to the request response time when the request is served by multiple RSLs.
If a request is services by two different RSLs, the request is finished when both devices have completed
their part of the request. Since we assume that requests are independent, we need to analyze a fork-join
queue with interfering traffic. Thomasian and Tantawi [39, 38] have found that a good approximation to the
response time of the fork-join job is to take the maximum of the response times of each device.
If we have the response time distribution of the different storage devices, we can compute the expected
value of the maximum response time by using order statistics. However, the exact response time distribution
of an Ms/G/c queue is difficult to compute. However, we can approximate the variance of the response
time by using the state occupancy distributions computed by the M"/M/c and M"/D/c queue solvers, and
approximating the variance of the batch service time using a technique similar to the one used to compute
the mean . We model the response time distribution as an Erlanglk distribution (the tail of the waiting
time distribution of the M'/G/c queue approaches an exponential distribution ). The expected value
of the maximum of two Erlang distributions is computed by integrating each component of the distribution
separately, then summing. The coding is easy, and the routine runs in O(kik2) time to compute the expected
value of an Erlanglkl and an Erlanglk2 distribution.
For an experiment, we applied the "large files" workload to two RSLs, with both receiving the same
arrival rate. We considered a request that required files from six media. In Figure 14, we plot the response
time of this request against the arrival rate, and varied the number of media that must be loaded from each
device. The results show that when the load on the tertiary storage devices is low, it is better to divide
the request evenly between the two devices. But, when the load is high it is better to use one device only.
The reason for this result is that splitting the request between the two devices provides parallel I/O, but
if the request load is high, then the variance in the response times becomes large. Thus, the decision to
allocate files so that most requests use only one device or that most requests use both devices depends on
the expected load placed on the devices.
3.4 Queuing Network Model
A mass storage data system consists of many components in addition to the RSLs. Typical hierarchical
storage management systems use a database to track file to media location mappings, and maintain a
sizeable staging and caching area. The computing centers that use tertiary storage often have large scale
computing tasks. For example, EOSDIS archives must perform product generation to filter, correct, remap,
and fuse satellite images (see the reports in , and also the discussion in ).
To capture the effects of RSLs in computing systems, we need to integrate the RSL model into a queuing
network model. The typical approach for incorporating devices with unusual response time characteristics
into a queuing network model is to use mean value analysis (MVA), and develop a MVA recurrence for the
device in question . However, it is difficult to develop such a recurrence even for multiple server devices.
Therefore, we take the approach of integrating the open RSL queue into a MVA model.
Although the RSL model solver is fast (about 2 seconds of execution time), an exact MVA solver requires
an iteration over every possible population vector. If the population is large and there are many job classes,
solution times become intolerably large. We instead used an approximate MVA solver, making use of
Schweitzer's approximation on queue lengths and Bard's approximation for the load dependent servers .
The approximate MVA solver built using these approximations compute the throughput at each iteration,
which we use as the arrival rate at the RSL (after scaling by the visit ratio).
Incorporating an open queuing model into a closed queuing network requires some care to ensure accurate
solutions. Generally, accuracy is good for large populations but worse for small populations . To improve
the accuracy of the solution, we modified the Ms/G/c solvers to incorporate a heuristic that accounts for
the finite customer population. Suppose that there are i jobs in the RSL, n customers, and an average batch
size of b. Then, the arrival rate in state i is:
To test the accuracy of the approximate MVA model, we simulated a computer system with a RSL and
three other queuing devices. The requests to the RSL used the "large file" workload, and every customer
submits a single request to the RSL per task execution. There are three queuing devices, with per-task
service demands of 250, 400, and 350 units of work, respectively.
We plot the response time of the RSL against the number of customers in the system Figure 15 for a sleep
time of 5000 and 9000. The model is accurate even for a small number of customers. However, the accuracy
declines when the number of customers is large and the sleep time is small. This problem is occurring
because one of the queuing devices is saturated, and the approximate MVA solver becomes inaccurate in
We have developed an analytical model of a robotic storage library and validated the model by comparison
to simulations. The RSL consists of a storage rack for removable media, a set of drives that read and write
the media, and a robotic arm that transfers the media between the storage rack and the drives.
The RSL model can be used for many useful studies. We provide examples of data layout and device
selection studies. A RSL is used as a part of a larger computing system. We incorporated the RSL solver
into an approximate MVA queuing network model, and validated the model by comparison to a simulation.
We have developed this model to support NASA's EOSDIS on-line archiving efforts. Future work will be
directed towards refining the model and providing studies useful to archive sites. This work include:
Refined models of seek times, request sizes, and files per media.
Incorporating scheduling policies into the waiting queue (combining jobs that access the same media,
Heterogeneous drives. Some drives might be faster than others, some might accept a subset of the
media, and some might be read-only.
Incorporating multiple classes of RSL access into the queuing network model. Currently, multiple
customer classes are supported, but all classes must have the same RSL access characteristics.
Incorporating fork-join jobs into the queuing network model.
Developing models of archive data centers, including models of caching, data ingest, and data mainte-
We'd like to thank Ben Kobler and Chris Daly of NASA GSFC, and Bob Howard of Hughes for their
comments, and Alex Thomasian for his advice regarding fork-join jobs.
 Jean-Jacques Bedet. Goddard daac vO log files., 1996. Private communication.
 J.J. Bedet and et al. Simulation of a data archival and distribution system at GSFC. In Goddard
Conference on Mass Storage Systems and Technology, pages 139-160, 1993.
 W.A. Burkhard, K.C. Claffy, and T.J.E. Schwarz. Performance of balanced array schemes. In Mass
Storage Systems Symposium, pages 45-50, 1991.
 R.S. Butturini. Performance simulation of a high capacity optical disk system. In Mass Storage Systems
Symposium, pages 147-153, 1988.
 J.P. Buzen and P.S. Goldberg. Guidelines for the use of infinite source queuing models in the analysis of
computer systems performance. In Proc. AFIPS 1974 National Computer Conf. Vol. 43, pages 471-474,
 M.J. Carey, L.M. Haas, and M. Linvy. Tapes hold data too: Challences of tuples on tertiary store. In
Proc. AC if SIC ifOD, pages 413-418, 1993.
 M.P. Chen, E.K. Lee, G.A. Gibson, R.H. Katz, and D.A. Patterson. RAID: High-performance reliable
secondary memory. Computing Surveys, 26(2):145-185, 1994.
 V. Chinnaswamy. Analysis of cache for streaming tape drive. In Goddard Conference on Mass Storage
Systems and Technology, pages 299-310, 1992.
 S. Christodoulakis. Analysis of retrieval performance for records and objects using optical disk technol-
ogy. AC if Transactions on Database Systems, 12(2):137-169, 1 1 7.
 S. Christodoulakis and D.A. Ford. Performance analysis and fundamental performance tradeoffs for clv
optical disks. In AC if SIC i[OD, pages 286-294, 1988.
 R.A. Coyne and H. Hulen. Toward a digital library strategy for a national information infrastructure.
In Proc. 3rd NASA Godddard Conf. on Mass Storage Systems and Technologies, pages 15-18, 1993.
 J.N. Daigle, R.B. Kuehl, and J.D. Langford. Queuing analysis of an optical disk jukebox based office
system. IEEE Trans. on Computers, 39(6):819-828, 1990.
 J. Dozier, M. Stonebraker, and J. Frew. Sequoia 2000: A next-generation information system for the
study of global change. In Proc. 13th IEEE Mass Storage Systems Symposium, pages 47-53, 1994.
 E. Drakopoulos and M.J. Merges. Perofrmance analysis of client-server storage systems. IEEE Trans-
actions on Computers, 41(11):1442-1452, 1992.
 A. Drapeau and R.H. Katz. Striped tape arrays. In Proc. 12th IEEE Mass Storage Systems Symposium,
pages ,-,7-265, 1993.
 J. Dunham and B. North. Eosdis statistics collection and reporting system, 1996. Available by anony-
mous FTP at eos.nasa.gov:/EosDis/Daacs/Statistics.
 Esdis document catalog. http://spsosun.gsfc.nasa.gov/ESDIS-Docs.html.
 A. Finestead and N. Yeager. Performance of a distributed superscaler storage server. In Goddard
Conference on Mass Storage Systems and Technology, pages 573-580, 1992.
 L. Golubchik, R.R. Muntz, and R.W. Watson. Analysis of striping techniques in robotic storage libraries.
In Proc. 14th IEEE Mass Storage Systems Symposium, pages 225-238, 1995.
 S.E. Hauser, C. Rivera, and G.R. Thoma. Factors affecting the performance of a dos-based WORM file
server. In Mass Storage Systems Symposium, pages 33-37, 1991.
 A.R. Hevner. Evaluation of optical disk systems for very large database applications. In AC if SIG-
METRICS conference, pages 166-172, 1 I-"
 K. Howard. High speed data duplication/data distribution -an adjunct to the mass storage equation.
In Goddard Conference on Mass Storage Systems and Technology, pages 123-133, 1992.
 G. Hull and S. Ranade. Performance measurements and operational characteristics of the Storage Tek
ACS 4400 tape library with the Cray Y-MP EL. In Goddard Conference on Mass Storage Systems and
Technology, pages 111-122, 1993.
 T. Johnson. Analysis of the request patterns to the nssdc on-line archive. In Proc. 4th NASA Goddard
Conf. on Mass Storage Systems and Technologies, 1995.
 K. Kant. Introduction to Computer System Performance Evaluation. McGraw Hill, 1992.
 S.M. Kelly, R.A. Haynes, and M.J. Ernest. Benchmarking a network storage service. In Mass Storage
Systems Symposium, pages 38-44, 1991.
 K.F. Kenk, J.L. Green, and L.A. Treinish. A cost model for NASA data archiving. Technical Report
90-08, National Space Science Data Center, NASA Goddard Space Flight Center, 1990.
 B. Kobler, J. Berbert, P. Caulk, and P.C. Hanrahan. Architecture and design of storage and data
management for the NASA Earth Observing System Data and Information System (EOSDIS). In Proc.
14th IEEE Mass Storage Systems Symposium, pages 65-78, 1995.
 E.K. Lee and R.H. Katz. An analytic performance model of disk arrays. In AC if SIC I./ TRICS, pages
 L. Lueking. Managing and serving a multi-terabyte data set at the Fermilab DO experiment. In Proc.
14th IEEE Mass Storage Systems Symposium, pages 200-208, 1995.
 O.I. Pentakalos, D.A. Menasce, M. Halem, and Y. Yesha. Analytical performance modeling of hierar-
chical mass storage systems. Technical Report TR-CS-96-01, Dept. of Computer Science, University of
Maryland, 1995. A short version appears in the 14th IEEE Mass Storage Symposium proceedings.
 E. Rahm. Performance evaluation of extended storage architectures for transaction processing. In AC if
SIC ifOD, pages 308-317, 1992.
 K.K. Ramakrishnan and J.S. Emer. Performance analysis of mass storage service alternatives for dis-
tributed systems. IEEE Trans. on Software Engineering, 15(2):120-133, 1989.
 C. Ruemmler and J. Wilkes. An introduction to disk drive modeling. IEEE Computer, 27:17-28, 1994.
 S. Sarawagi. Database systems for efficient access to tertiary memory. In Proc. 14th IEEE Mass Storage
Systems Symposium, pages 120-126, 1995.
 Sequoia 2000 home page. http://s2k-ftp.cs.berkeley.edu:8000/.
 A. Thomasian. Surveyor's forum: High performance secondary memory. Computing Surveys, 27(2):292
 A. Thomasian. Approximate analyses for fork/join synchronization in RAID 5. Computer Systems:
Science and Engineering, 1996. To appear.
 A. Thomasian and A.N. Tantawi. Approximate solutions for m/g/! fork/join synchronization. In Proc.
1994 Winter Simulation Conference, 1994.
 H.C. Tijms. Stochastic Models: An Algorithmic Approach. Wiley, 1994.
 S.J. Waters. Estimating disk seeks. The Computer Journal, 18(1):12-17, 1974.
 T. Yang, S. Hu, and Q. Yang. A closed-form formula for queuing delays in disk arrays. In Proc. Intl.
Conf. on Parallel Processing, pages 11:189-192, 1994.
Two media per request, large files
0 I I I I I
0 0.0005 0.001 0.0015 0.002 0.0025
Figure 3: Validation study, response time vs. arrival rate.
Two media per request, small files
0 0.002 0.004 0.006 0.008 0.01
Figure 4: Validation study, response time vs. arrival rate.
Six media per request, large files
0 I I
0 0.0005 0.001 0.0015 0.002
Figure 5: Validation study, response time vs. arrival rate.
Six media per request, small files
0 0.001 0.002 0.003
Figure 6: Validation study, response time vs. arrival rate.
Six media per request, large files
0 0.0005 0.001 0.0015 0.002
Figure 7: Validation study, utilization vs. arrival rate.
Six media per request, small files
0 0.001 0.002 0.003
Figure 8: Validation study, utilization vs. arrival rate.
Varying arrival rate, large
2 3 4 5 6 7
Media per request
Figure 9: Response time vs. media per requ
Varying arrival rate, large files
0 2 4 6 8 10 1
media per request
Figure 10: Drive utilization vs. media per request.
Performance study: Optical vs. Tape drives
0.001 0.002 0.003
Figure 11: Hypothetical robotic storage library comparison.
- - - -- x
Four media per request, small files
0 0.002 0.004 0.006 0.008 0.01
Figure 12: Response time vs. number of drives.
Four media per request, small files
0 0.002 0.004 0.006
Figure 13: Drive utilization vs. number of drives.
Dividing a request between two devices
0 0.001 0.002 0.003
Figure 14: Response time of a fork-join request.
Tertiary storage in a QNM
tertiary storage response time
u I I I I I I I '
0 5 10 15 20 25 30 35 40
Figure 15: Incorporating the RSL model into a queuing network.