Group Title: Department of Computer and Information Science and Engineering Technical Reports
Title: An Analytical performance model of robotic storage libraries
CITATION PDF VIEWER THUMBNAILS PAGE IMAGE ZOOMABLE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00095374/00001
 Material Information
Title: An Analytical performance model of robotic storage libraries
Series Title: Department of Computer and Information Science and Engineering Technical Reports
Physical Description: Book
Language: English
Creator: Johnson, Theodore
Affiliation: University of Florida
Publisher: Department of Computer and Information Science and Engineering, University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: July 25, 1996
Copyright Date: 1996
 Record Information
Bibliographic ID: UF00095374
Volume ID: VID00001
Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.

Downloads

This item has the following downloads:

1996217 ( PDF )


Full Text










An Analytical Performance Model of Robotic Storage Libraries

Theodore Johnson
Dept. of CISE, University of Florida
ted@cis.ufl.edu

June 25, 1996


Abstract
Large scale scientific projects generate and use huge amounts of data. For example, the NASA
EOSDIS project is expected to archive one petabyte per year of raw satellite data. This data is made
automatically available for processing into higher level data products and for dissemination to the sci-
entific community. Such large volumes of data can only be stored in robotic storage libraries (RSLs)
for near-line access. A characteristic of RSLs is the use of a robot arm that transfers media between a
storage rack and the read/write drives, thus multiplying the capacity of the system.
The performance of the RSLs can be a critical limiting factor of the performance of the archive system.
However, the many interacting components of a RSL make a performance analysis difficult. In addition,
different RSL components can have widely varying performance characteristics. This paper describes
our work to develop performance models of a RSL. We first develop a performance model of a RSL in
isolation. Next, we show how the RSL model can be incorporated into a queuing network model. We
use the models to make some example performance studies of archive systems.
The models described in this paper, developed for the NASA EOSIDS project, are implemented in C
with a well-defined interface. The source code and accompanying documentation are available through
WWW at:
http://www.cis.ufl.edu/-ted/


1 Introduction


Large scale scientific projects generate and use huge amounts of data. For example, the NASA EOSDIS

project is expected to archive one petabyte per year of raw satellite data [28]. This data is made automatically

available for processing into higher level data products and for dissemination to the scientific community

(see, for example, the reports in [17]). Automatic management of such large data sets requires the use of

tertiary storage, typically implemented using robotic storage libraries (RSLs). In addition to EOSDIS and

related projects, many organizations and scientific disciplines make use of mass storage archives (for example

high energy physics [30] and digital libraries [11]).

The database community has also become interested in the use of RSLs. [13, 6, 36, 35]. This interest is

motivated in part by scientific database problems such as EOSDIS. Another motivation for integrating RSLs

with on-line database systems is to facilitate data warehousing.

Tertiary storage is required when the managed data set becomes too large to store economically with

conventional magnetic disk devices. The point at which tertiary storage becomes necessary is an economic










tradeoff. Currently, it seems that tertiary storage is needed to manage more than a terabyte of data. A RSL

is much slower than magnetic disk storage, and data access latencies can run into minutes even on unloaded

systems. However, RSL-resident data can accessed automatically. Hierarchical storage management systems,

such as Unitree, Filestore, and Amass, provide the illusion that the RSL is an extension of the file system.

Access to archived data incurs a short delay. The storage capacity of a data system can also be increased

by using off-line storage -i.e. tape racks with human operators. Access latencies with off-line storage can

be very large, ranging into hours or days, but the data storage capacity is limited only by the size of the

warehouse that one can afford to rent. Since RSL provides data volumes and access latencies between those

provided by on-line and off-line storage, it is often referred to as near-line storage. A cost analysis of on-line,

near-line, and off-line archives can be found in [27].

A characteristic of RSLs is the use of removable media and a robot arm. The removable media (e.g.

magnetic tape, optical disk, etc.) are normally located in a storage rack. To service a request for a file,

the robot arm fetches the proper media from the storage rack and delivers it to a read/write drive. The

media is accessed in the normal way to fetch the file. Finally, the media is returned to the storage rack.

The capacity of RSL is the product of the capacity of the media and the size of the storage rack. Recent

magnetic tapes have a data capacity on the order of 10 Gbytes, and storage racks sizes range from 10 to

1000 media (approximately). The time to fetch and mount the media which holds the requested file can be

a large component of the access latency.

The performance of the RSLs can be a critical limiting factor of the performance of the archive system.

Given the high data request rates expected for EOSDIS, attention to handling these requests efficiently is

critical [28, 17]. However, the many interacting components of a RSL make a performance analysis difficult.

In addition, different RSL components can have widely varying performance characteristics.

This paper describes our work to develop performance models of tertiary storage. We first develop a

performance model of a RSL in isolation. Next, we show how the RSL model can be incorporated into a

queuing network model. Finally, we model fork-join jobs to study the tradeoffs of using multiple devices.

We use the models to make some example performance studies of archive systems.

The models described in this paper, developed for the NASA EOSIDS project, are implemented in C with

a well-defined interface. The source code and accompanying documentation are available through WWW

at:


http://www.cis.ufl.edu/~ted/










1.1 Previous Work


Considerable work has been done to develop performance models of mass storage. Rahm [32] presents a

simulation study of a database system with a hierarchy of storage devices. Ramakrishnan and Emer [33]

present a queuing model of a client/server file system. Drakopoules and Merges [14] present a closed queuing

model of a client/server storage system with hierarchical storage. Kelly, Haynes, and Ernest [26] discuss a

benchmark for network storage systems. Hauser, Rivera, and Thoma [20] discuss the performance of their

networked WORM server.

Some work has been done to characterize the performance of mass storage devices. Waters [41] presents

a validated model of seek times in hard disk drives. More recently, Ruemmler and Wilkes [34] present a

detailed model of a modern disk drive, and discuss the difficulties inherent in I/O modeling. Christodoulakis

and Ford [10] and Christodoulakis [9] present analytical performance models of optical drives. Chinnaswamy

[8] presents performance models of a streaming tape drive to investigate the benefit of a cache.

Models of disk arrays resemble the models presented in this paper in several aspects. Burkhard, Claffy,

and Schwarz [3] present a simulation study of a disk array scheme. Lee and Katz [29] and Yang, Hu and

Yang [42] present analytical models of disk arrays. Chen et al. [7] and Thomasian [37] present surveys of

research in RAID modeling.

Several authors have modeled a RSL. Butturini [4] presents the results of a simulation study of an optical

disk jukebox system. Hevner [21] presents a model of an optical jukebox that is used for a database appli-

cation. Howard [22] gives a performance model for data duplication from an archive. Finestead and Yeager

[18] give performance measurements of a Unitree file server at the National Center for Hull and Ranade [23]

present measurements of tape loading and unloading, and of data throughput, in a tape silo. Supercomputer

Applications. Bedet et al. [2] discuss the results of a detailed simulation model of the Goddard DAAC.

Pentakalos, Menasce, Halem, and Yesha [31] develop a queuing network model that incorporates a RSL.

Daigle, Kuehl, and Langford [12] present a queuing model of an optical disk jukebox. Golubchik, Muntz and

Watson [19] analyze tape striping on a RSL.

The analyses most closely related to the one in this paper are [31, 12, 19]. The analysis in [12] gives

a detailed model of access times to data on an optical platter. However, only one drive is permitted and

contention for the robotic arm is not modeled. In [31], the authors present a detailed model of a data center,

incorporating RAID disk caches and user computation. However, the authors assume that contention for

the drives in the RSL is negligible, and model the RSL as a delay server. Contention for the robotic arm

due to batch arrivals is modeled in [19], but contention between jobs is not modeled.










The contribution of this work is to present a validated model of a RSL that accounts for batch arrivals,

multiple drives, contention for the robotic arm, and realistic operation. We show how the model can be used

to make a variety of data layout and device comparison studies. Finally, we show how to incorporate the

RSL model into a queuing network model.


2 Model of a Robotic Storage Library


Our model of a RSL is illustrated in Figure 1. Previous studies of mass storage archive log files (see, for

example, [24, 12]) indicate that requests to a mass storage device come in batches. This study has been

corroborated by our studies of access to preliminary versions of the EOSDIS archives (the VO archives)

[1, 16]. As a result, our RSL model model uses batch arrivals.

A user requests that f files be loaded into on-line storage, and these files are distributed over m media

in the RSL. The request is satisfied when every file has been loaded into on-line storage. So, a user request

consists of m jobs, each of which must be completed before the request in finished. A RSL consists of nd

drives, each of which can read or write any of the media in the RSL,1 a storage rack containing the removable

media, and a robot arm for transferring the media between the drives and the storage rack. The model of a

RSL is illustrated in Figure 1.









Poisson batch arrival 0 Drive 1



0 Drive2


*
Drive 3



Drive 4


Figure 1: Architecture of a robotic storage library.
1In some installations, a subset of the drives are designated as read-only or write-only. We will address this complication in
a later version of the model.










The steps taken by a drive in retrieving files from a media is illustrated in Figure 2. When a request

arrives, its jobs are placed in the job queue. If there are jobs in the RSL queue and a drive is idle, the

drive allocates one of the jobs for execution. First, the robot arm fetches the appropriate media from the

storage area and loads it into the drive. If the robot arm is busy serving other drives, the drive must wait

for service. After the media is brought to the drive, it must be mounted. For every file of interest on the

media, the drive must seek to the start of the file, spend a settling time for precise positioning and opening

communications channels, and then transfer the file to on-line storage. After all files have been transferred,

the media is rewound and returned to the storage rack by the robot arm. However, the job is finished once

all of the files have been transferred.


Media






mount File 1 seek File 1 File n seek File n
robot rewind robot
time and transfer time and transfer
fetches time time returns
settling time settling time
the media time time the
time time
media


Transfer is finished





Figure 2: Steps in loading files from a media.


In the next section, we present an analytical performance model of a RSL system. In this preliminary

model, we make the following assumptions:


1. Requests arrive in a Poisson process.


2. The distribution of the number of media per request and the number of files per request must be

specified. In the model discussion, we assume that the number of media per request and the number

of files per request have geometric distributions. These can be replaced by user-specified distributions

(e.g. empirically determined), but at the cost of requiring the user to specify more parameters.


3. The RSL can contains one robot arm. The robot arm can access every media, and every drive.










4. Every drive can read and write every media.


5. Requests (i.e., jobs) are serviced first-come-first-serve2.


6. The service for a request is completed when the last file of the batch has been read (written).


7. Network or communication channel contention is not significant3.


8. Service times at the drives are independent.


3 Analytical model


A RSL presents many difficulties for performance modeling, including batch arrivals, multiple servers, derived

parameters, and interacting components. The primary component of the RSL model, the M"/G/c queue,

has been studied and solved in the literature [40]. Solving the actual MA /G/c queue is intractable, so the

solution technique is to interpolate between the results for the MA /M/c queue and the MA /D/c queue

using the coefficient of variation of the service time as the interpolation parameter. Because of the potential

complexity of the batch arrival distributions, we do not use explicit (i.e., generating function) formulas.

Instead, we numerically solve the recurrence equation that defines the state occupancy probabilities. If the

occupancy probabilities of the first N states must be computed for an error bound of c, then solving the

MA /M/c queue requires O(N2) time and solving the Mf/D/c queue requires O(N3) time.

Fortunately, we can take advantage of the nature of the problem to speed up the solution times. The

state occupancy distributions eventually converge to a geometric distribution (i.e., PN = tOPN-1). Therefore

the recurrence equations only need to be solved up the first No states, and the remainder can be computed

using the to ratio (or perhaps the performance metrics can be computed directly). No depends primarily on

the distribution of the size of the batch arrival. Fortunately, the batch arrival distribution will have a short

tail one cannot request that more media than exist in the storage rack be mounted, and usually only a few

media are required to satisfy a request. By using these tricks, we implemented batch queue solvers that are

fast enough to be incorporated into a higher level model which calls them many times.

We now define some variables. The following are the inputs to the model.


SA : request arrival rate.


fr : Average number of files per request.
2A simple optimization is to load files for all requests once a media has been mounted. We assume this situation has a
negligible impact on performance in this model
3 Potential model users indicated that communication channel contention is not a problem for their systems. Communication
contention can be incorporated into the seek times or mount times using standard techniques [25]










m, : Average number of media per request.


b(.) : Distribution of the number of media per request.


nd : number of drives.

Et, Average robot fetch time.


Vt, : Variance of the robot fetch time.

Tit : Media mount time.


Tfs : Time for a full seek across the media.


Tst : Settling time after a seek.

Xb : Transfer rate.

E,,: Average file size.


V, : Variance in the file size.


Erd : Expected time to re-wind the media.

The performance measures are:


Erob : Average delay for the robot fetch.


Prob : Robot arm utilization.

pd : Effective drive utilization utilization.


Pd(.) : Queue length distribution for the drives.

Request : ATL request response time.

We model the RSL as an M"/G/c queue -that is, a queue with Poisson batch arrivals, general service

time distribution, and c servers. The parameters of a Ms/G/c queue are:

A : Arrival rate


Wd : Mean service time


cvd : Co-efficient of variation of service time

b(.) : Batch size distribution










* nd : Number of servers


All but the service time distribution are input parameters, so our analysis is focused on how to compute

Ed and CVd = /V/Ed. To compute queue length distributions and expected waiting times properly, we need

to compute the time that a drive is unable to serve other jobs per media that it serves. This period includes

the time to fetch the media, mount it, seek to each file, transfer each file, rewind and eject the media, and

return it to the storage rack. We will incorporate the time to return the media as part of the media fetch

time, so we have:


drive service = (robot fetch) + (mount time) + (seek time) + (transfer time) + (rewind time)


Analyzing the RSL reduces to analyzing each component of the drive service time. Some of these

components are easy to estimate, others difficult. We will work from easiest to most difficult.


Mount time The mount time Tnt is the time period after the robot arm has placed the media in the

drive and the drive can accept commands to seek to the first file. Typical actions include precise positioning

of the media, drive initialization, and reading a directory sector (to ensure that the proper tape has been

loaded, etc.). These actions are nearly deterministic [23] and are characteristics of the drive used in the RSL.

Therefore, the mount time is a parameter, and we assume that mounting makes a negligible contribution to

the drive service time variance.


Rewind time The rewind time is time period after the last file has been transferred from the media and

the media can be ejected to the robot arm (it is typical to rewind tapes after use to ensure longevity, hence

the name). The rewind time is device dependent [15], so we require that the mean and variance of the rewind

time be supplied as parameters.


Transfer time Given that a file of Sz bytes is to be transferred, a commonly accepted model of the

transfer time is a simple linear model:


Txf = a + Sz/Xb


where T1f is the transfer time, Xb is the transfer rate and a is transfer initialization time [8]. In this

model, we will absorb the transfer initialization time into the seek time, so a = 0. Ideally, one could supply

the mean and the variance of the volume of data that is to be transferred from the media. However, this

might be more information than is available. More typically, one can estimate the mean and variance of the










individual files that are transferred. Then, one can use the distribution of the number of files transferred

per media to estimate the mean and variance of the transfer time.

Let bf(.) be the probability density function of the number of files transferred per media. If k files are

transferred, then the mean volume of data transferred is kE,, and the variance is kVz. Therefore, the

second moment of the volume of data transferred is


1 (k) = kV~z + k2Ef


The first and second moments of the volume of data transferred per media is
00
Evol = YkEskbf(k)
k=l

M2, = M,2 (k)bf(k)
k=l
To obtain transfer times, we divide by the transfer rate.


Exf = Evoi/Xb

Vxf = (M2, E 2}IXb
Vl= Eolo)/Xb

If the number of files per request and the number of media per request have a geometric distribution,

and the number of files per media is distributed iid, then the number of files per media has a geometric

distribution also4. If an average of f, files are transferred from each media, then


Exf = fmEsz/Xb

Vxy = fm(V, + EsZ(f 1))/X2


Seek time Unlike the transfer times, there is no commonly accepted model for seek times (magnetic disk

seek times are difficult enough [34]). The problem is compounded by the random seek distances, disk vs.

single-pass vs. serpentine tape, and fast-seek modes. One simple model is to assume that files are randomly

distributed across a single-pass tape. Suppose that f files are to be transferred. Then [15]:


Eseek Tf + fTtl

M2,k(f) = fTf/(f + 2) + 2f2TtlTf/(f + 1) + f2Tfs


Our solver assumes that user supplies a function which takes the number of files f as parameter and

returns the mean and the second moment of the seek time. The unconditional seek time and variance is
4Note to referee: Here and at other places I suppress some calculations and details in order to save space










computed as:


Eseek= f=1 Eseek(f)bf (f)

Meek f=l Meek(f)bf(f)

Vseek = -2eek Eseek


Robot arm service time Computing Erob is more complex, because the fetch time includes contention

for a robot arm. If the ATL is empty and a batch request arrives, then a batch request is made to the robot

arm. The queuing model that is appropriate for modeling the robot arms is the Mx/G/1 queue. Then, Erob

is the response time of the average job.

There are some complications. Suppose that nidle of the drives are idle when a batch of size B arrives. If

B > nidle, the nidle idle drives will request will request service immediately. The remaining B -nidle jobs will

request service from the robot arms as the drives finish serving the preceding jobs. Each of these B nidle

requests will be preceded by a matching request to remove the old media from the drive. Therefore, the

B nidle jobs have a doubled service time requirement. The nidle requests also generate matching requests

to unload the drives, but these requests are not correlated with any requests to load a drive.

Let s(nidle) be the probability that the batch is of size nidle or smaller, and let b(nidle) be the average

size of a batch request, given that the batch is for nidle or fewer jobs. That is,
fdle
s(nidle)= b(i)
i=1
fndle
b(nidle) = i b(i)
i=1

The arrivals to the robot arm queue consists of the truncated job arrival stream (with rate A), an arrival

stream that corresponds to jobs that must wait for service from a drive (rate Adoub), and an arrival stream

that corresponds to drives returning media without an immediate fetch of new media (rate Asing). Then,




Adoub(nidle) = A(m, b(nidle)) nidle > 0

Asing(nidle) = Ab(nidle) nidle > 0

Arob(nidle) = A + Adoub + Asing idle > 0


Let brob,n,,ed be the distribution of batch arrivals to the robot, given that nidle drives are idle. Then,


brob,nile(l) = (A b(1) + Asing)/Arob idle > 2










(A b(2) + Adoub)/Arob

A *b(i)/Arob

A (b(nidle) + (1 s(nid1e))/Arob)


nidle > 2

nidi > 2 and 2 < i < nidl

idle > 2


brob,2(l) = (A b(l) + A,in)/Arob

brob,2(2) = (A b(2) + (1 s(2)) + Adoub)/rob



brob,l(l) = (A + Asing)/rob

brob,l(2) = doub/Arob




If all drives are busy, then all jobs are serviced when a busy drive finishes services. Therefore:


Arob(O)

brob.0(2)


Given the drive queue length distribution Pd, let be the probability that all drives are busy (i.e.,

. = 1 (pd(0) + - +Pd(nd 1)). We can compute the parameters to the robot queue by derandomizing

brob(.) and and Arob(.). For example,
nd-1
Arob = \ (0) + pd(i)Aob(nd -i)
i=0
Then, the parameters to the M'/G/1 queue [40] representing the robot arm are:


Arrival rate Arob

Service time Erob = Etr

Coefficient of variation cvrob = VV/Erob

Batch size distribution brob ()




The queuing model provides for us the robot arm utilization Pob, and the waiting time for robot arm

service Wrob. The M'/G/1 model returns I- ..," waiting times. That is, the increase in waiting time

due to correlated arrivals are not fully accounted for. Recall that if a job is serviced by a drive that was

busy immediately before starting service, then the job must wait for both the return of the old media and


brob,ndie (2)

brob,ndl (i)

brob,n,die (nidle)










the fetch of the new media. This correlation is captured in the waiting times by the batch arrival stream

Adoub(nidl,). However, the reported service time accounts for only half of the increased increased robot

service time for these jobs. Therefore, we compute the additional waiting time due to correlated arrivals at

the robot arm to be:


Extra(nidle) = Etr A+Adob(ndlie)

Extra = I' P xtra(O) + Zo Pd()E xtra(nd i)


We compute the robot arm response time to be:


Erob = Wrob + Etr + Extra


The M"/G/1 queue solvers compute the state occupancy distributions, so we compute the variance of

the response time Vrob by using this distribution.


The Drives The parameters to the queue representing the RSL are:


Arrival rate A

Service time Ed = Exf + Eseek + Tnt + Erob + Erwd
/Vxf+Vseek+V-rob+Vr-d
Coefficient of variation cvSd = /+V Evobd

Batch size distribution b(.)

Number of servers nd


Since we are interested in the response time of the last job in the batch to finish (i.e., instead of the

average job), we need to modify the response time computation. An efficient algorithm for computing the

response time of the last job in the batch is given in [25].

The modified Mx/G/c queue provides the batch response time Rbatch, the drive utilization Pd, and

pd(O), ., Pd(nd 1), the probability 0, .., nd 1 servers are busy on a request arrival. The effective drive

service time (and Pd(.)) depends on the robot response time, which in turn depends on Pd(.). We use iteration

to converge to the solution.

Finally, the job is finished when the last file has been transferred, there is no need to wait for the tape

rewind. Therefore:


Rrequest = Rbatch Erwd










3.1 Validation Study

We wrote a simple RSL simulator. The simulation accepts batch arrivals, requires that a robot unload and

fetch a media before a drive can service a job, handles multiple drives, and accounts for media rewind times.

The components of Ed and Vd, except for Erob and Vrob, are pre-computed and sampled from an Erlang

distribution.

We used the following values of the parameters in the validation study.


Sf, = 20.

b(.): Geometric distribution.


n = 4.

Et, = 10.0 seconds.


Vt, = 10.0.

Tt =10.0 seconds.


Xb = 1.0 Mbyte/sec.

We ran four sets of experiments to test the model. In the "large files" experiments, Es, = 50 and

V,, = 100, and Tf, = 50, Ttl = 1. In the "small files" experiments, E,, = 5 and V, = 10, and Tf, = 20,

Tti = 2. We tested the model with m, = 2 and m, = 6.

The results of the validation study are shown in Figures 3 through 6. In each case there is close agreement

between the analytical and the simulation models. The most difficult case is when the files are small and

distributed over an average of six media, because the robot fetch times constitute a large portion of the

drive service times (about 22% of the total drive service time when the robotic arm waiting time is added).

However, the analytical model is accurate enough to predict response times and drive utilizations. Charts

comparing analytical and simulation drive utilizations are shown in Figures 7 and 8.

3.2 Performance Study

A performance model is useful for studying implementation alternatives. In this section, we present three

sample performance studies based on the RSL model.










3.2.1 Clustering


Conventional wisdom holds that striping or declustering is necessary for obtaining high transfer rates from

tertiary storage (by making use of parallel I/O). So, one should spread the files of a typical request around as

many media as possible. Conventional wisdom also holds that swapping media is a source of great inefficiency

in RSL access, so that one should try to ensure that the files of a typical request are placed on as few media

as possible.

Neither argument is convincing, unless one has a predictive performance model. We ran the "small files"

experiment with m, ranging between 1.2 and 10. In Figure 9, we plot the response time of a request against

the number of media per request for varying arrival rates (A similar chart can be found in an analysis of tape

striping [19]). For low arrival rates, setting m,. w nd produces the best results. When A = .0001, setting

m, = 3 results in a 22% lower response time than setting m, = 1.2. For high arrival rates, setting m, = 2

gives lower response times than other choices.

In Figure 10, we plot the drive utilization against m, for varying arrival rates. Increasing m, causes a

linear increase in the drive utilization. As the arrival rate increases, it becomes less likely that all nd drives

are available to service the request. So, distributing the files over a smaller number of media reduces queuing

delays. If the demand on the RSL is expected to be close to the device's capacity, then m, should be small

to increase the maximum throughput of the device.

The question of whether to cluster or decluster the files on the media can be summarized as:

If the expected drive utilization is low [31] and fast response is important, then declustering can be

a good strategy. However, the decrease in transfer times must be larger than the increase in queuing

delays.

If high throughput is important, clustering is a good strategy.


3.2.2 Device Selection

A performance model can be used to determine the most appropriate equipment for an application. Consider

the following hypothetical problem. You need to choose between two devices of approximately equal cost.

The workload is the "small files" workload, except that an average of 40 files are loaded per request.

The first device uses optical disks as the storage media. Seek times are negligible, but the storage capacity

per disk is limited, you can afford to purchase only two drives, and the transfer rate is 1 Mbyte/second. The

second device uses tapes. The capacity of the tapes is larger than the capacity of the optical disks, and you

can afford to purchase four drives. However, the seek times are large.










Because of the difference in the capacities of the media, you need to load an average of 6 media per

request if you use the optical disks, and 2 media per request if you use tape. Using these parameters, we

can plot the average response time of the two RSLs under an increasing workload. This chart is shown in

Figure 11.

For the parameters of this study, the optical disk system is better than the tape system if the workload

is light, but worse if the workload is heavy. The fast seek times of the optical disks allow you to transfer

the large number of files quickly. However, the tape based system has a higher data capacity. While the

parameters of this study are engineered to give an interesting comparison, the point remains that device

performance depends on the workload offered to the device.


3.2.3 Number of Drives

Many RSLs allow the user to install a ranging number of drives. Adding drives to a RSL can improve the

performance of the device. But after a threshold, adding drives does not significantly improve performance.

We ran a sample study using the "small files" parameters and four media per request. In Figure 12, we

plot the response time versus the arrival rate for a number of drives varying between 2 and 8. Adding a drive

significantly improves performance up to four drives, but gives less benefit after four drives. In Figure 13,

we plot the drive utilization against the arrival rate. Adding a drive to the RSL increases the capacity of

the device. However, the robot arm will start to become a bottleneck. This can be seen in the non-linear

increase in utilization of some of the curves, for example for nd = 8.

3.3 Multiple Robotic Storage Libraries

A large scale data center is likely to have multiple RSLs. The devices might be acquired to handle data

center growth, or multiple small devices might be less expensive than a single large device. In this section,

we discuss an approximation to the request response time when the request is served by multiple RSLs.

If a request is services by two different RSLs, the request is finished when both devices have completed

their part of the request. Since we assume that requests are independent, we need to analyze a fork-join

queue with interfering traffic. Thomasian and Tantawi [39, 38] have found that a good approximation to the

response time of the fork-join job is to take the maximum of the response times of each device.

If we have the response time distribution of the different storage devices, we can compute the expected

value of the maximum response time by using order statistics. However, the exact response time distribution

of an Ms/G/c queue is difficult to compute. However, we can approximate the variance of the response

time by using the state occupancy distributions computed by the M"/M/c and M"/D/c queue solvers, and










approximating the variance of the batch service time using a technique similar to the one used to compute

the mean [25]. We model the response time distribution as an Erlanglk distribution (the tail of the waiting

time distribution of the M'/G/c queue approaches an exponential distribution [40]). The expected value

of the maximum of two Erlang distributions is computed by integrating each component of the distribution

separately, then summing. The coding is easy, and the routine runs in O(kik2) time to compute the expected

value of an Erlanglkl and an Erlanglk2 distribution.

For an experiment, we applied the "large files" workload to two RSLs, with both receiving the same

arrival rate. We considered a request that required files from six media. In Figure 14, we plot the response

time of this request against the arrival rate, and varied the number of media that must be loaded from each

device. The results show that when the load on the tertiary storage devices is low, it is better to divide

the request evenly between the two devices. But, when the load is high it is better to use one device only.

The reason for this result is that splitting the request between the two devices provides parallel I/O, but

if the request load is high, then the variance in the response times becomes large. Thus, the decision to

allocate files so that most requests use only one device or that most requests use both devices depends on

the expected load placed on the devices.

3.4 Queuing Network Model

A mass storage data system consists of many components in addition to the RSLs. Typical hierarchical

storage management systems use a database to track file to media location mappings, and maintain a

sizeable staging and caching area. The computing centers that use tertiary storage often have large scale

computing tasks. For example, EOSDIS archives must perform product generation to filter, correct, remap,

and fuse satellite images (see the reports in [17], and also the discussion in [31]).

To capture the effects of RSLs in computing systems, we need to integrate the RSL model into a queuing

network model. The typical approach for incorporating devices with unusual response time characteristics

into a queuing network model is to use mean value analysis (MVA), and develop a MVA recurrence for the

device in question [25]. However, it is difficult to develop such a recurrence even for multiple server devices.

Therefore, we take the approach of integrating the open RSL queue into a MVA model.

Although the RSL model solver is fast (about 2 seconds of execution time), an exact MVA solver requires

an iteration over every possible population vector. If the population is large and there are many job classes,

solution times become intolerably large. We instead used an approximate MVA solver, making use of

Schweitzer's approximation on queue lengths and Bard's approximation for the load dependent servers [25].

The approximate MVA solver built using these approximations compute the throughput at each iteration,










which we use as the arrival rate at the RSL (after scaling by the visit ratio).

Incorporating an open queuing model into a closed queuing network requires some care to ensure accurate

solutions. Generally, accuracy is good for large populations but worse for small populations [5]. To improve

the accuracy of the solution, we modified the Ms/G/c solvers to incorporate a heuristic that accounts for

the finite customer population. Suppose that there are i jobs in the RSL, n customers, and an average batch

size of b. Then, the arrival rate in state i is:

nb i
nb

To test the accuracy of the approximate MVA model, we simulated a computer system with a RSL and

three other queuing devices. The requests to the RSL used the "large file" workload, and every customer

submits a single request to the RSL per task execution. There are three queuing devices, with per-task

service demands of 250, 400, and 350 units of work, respectively.

We plot the response time of the RSL against the number of customers in the system Figure 15 for a sleep

time of 5000 and 9000. The model is accurate even for a small number of customers. However, the accuracy

declines when the number of customers is large and the sleep time is small. This problem is occurring

because one of the queuing devices is saturated, and the approximate MVA solver becomes inaccurate in

these situations.


4 Conclusions


We have developed an analytical model of a robotic storage library and validated the model by comparison

to simulations. The RSL consists of a storage rack for removable media, a set of drives that read and write

the media, and a robotic arm that transfers the media between the storage rack and the drives.

The RSL model can be used for many useful studies. We provide examples of data layout and device

selection studies. A RSL is used as a part of a larger computing system. We incorporated the RSL solver

into an approximate MVA queuing network model, and validated the model by comparison to a simulation.

We have developed this model to support NASA's EOSDIS on-line archiving efforts. Future work will be

directed towards refining the model and providing studies useful to archive sites. This work include:


Refined models of seek times, request sizes, and files per media.

Incorporating scheduling policies into the waiting queue (combining jobs that access the same media,

for example).










Heterogeneous drives. Some drives might be faster than others, some might accept a subset of the

media, and some might be read-only.

Incorporating multiple classes of RSL access into the queuing network model. Currently, multiple

customer classes are supported, but all classes must have the same RSL access characteristics.

Incorporating fork-join jobs into the queuing network model.

Developing models of archive data centers, including models of caching, data ingest, and data mainte-

nance activities.


5 Acknowledgements


We'd like to thank Ben Kobler and Chris Daly of NASA GSFC, and Bob Howard of Hughes for their

comments, and Alex Thomasian for his advice regarding fork-join jobs.


References


[1] Jean-Jacques Bedet. Goddard daac vO log files., 1996. Private communication.

[2] J.J. Bedet and et al. Simulation of a data archival and distribution system at GSFC. In Goddard

Conference on Mass Storage Systems and Technology, pages 139-160, 1993.

[3] W.A. Burkhard, K.C. Claffy, and T.J.E. Schwarz. Performance of balanced array schemes. In Mass

Storage Systems Symposium, pages 45-50, 1991.

[4] R.S. Butturini. Performance simulation of a high capacity optical disk system. In Mass Storage Systems

Symposium, pages 147-153, 1988.

[5] J.P. Buzen and P.S. Goldberg. Guidelines for the use of infinite source queuing models in the analysis of

computer systems performance. In Proc. AFIPS 1974 National Computer Conf. Vol. 43, pages 471-474,

1974.

[6] M.J. Carey, L.M. Haas, and M. Linvy. Tapes hold data too: Challences of tuples on tertiary store. In

Proc. AC if SIC ifOD, pages 413-418, 1993.

[7] M.P. Chen, E.K. Lee, G.A. Gibson, R.H. Katz, and D.A. Patterson. RAID: High-performance reliable

secondary memory. Computing Surveys, 26(2):145-185, 1994.










[8] V. Chinnaswamy. Analysis of cache for streaming tape drive. In Goddard Conference on Mass Storage

Systems and Technology, pages 299-310, 1992.

[9] S. Christodoulakis. Analysis of retrieval performance for records and objects using optical disk technol-

ogy. AC if Transactions on Database Systems, 12(2):137-169, 1 1 7.

[10] S. Christodoulakis and D.A. Ford. Performance analysis and fundamental performance tradeoffs for clv

optical disks. In AC if SIC i[OD, pages 286-294, 1988.

[11] R.A. Coyne and H. Hulen. Toward a digital library strategy for a national information infrastructure.

In Proc. 3rd NASA Godddard Conf. on Mass Storage Systems and Technologies, pages 15-18, 1993.

[12] J.N. Daigle, R.B. Kuehl, and J.D. Langford. Queuing analysis of an optical disk jukebox based office

system. IEEE Trans. on Computers, 39(6):819-828, 1990.

[13] J. Dozier, M. Stonebraker, and J. Frew. Sequoia 2000: A next-generation information system for the

study of global change. In Proc. 13th IEEE Mass Storage Systems Symposium, pages 47-53, 1994.

[14] E. Drakopoulos and M.J. Merges. Perofrmance analysis of client-server storage systems. IEEE Trans-

actions on Computers, 41(11):1442-1452, 1992.

[15] A. Drapeau and R.H. Katz. Striped tape arrays. In Proc. 12th IEEE Mass Storage Systems Symposium,

pages ,-,7-265, 1993.

[16] J. Dunham and B. North. Eosdis statistics collection and reporting system, 1996. Available by anony-

mous FTP at eos.nasa.gov:/EosDis/Daacs/Statistics.

[17] Esdis document catalog. http://spsosun.gsfc.nasa.gov/ESDIS-Docs.html.

[18] A. Finestead and N. Yeager. Performance of a distributed superscaler storage server. In Goddard

Conference on Mass Storage Systems and Technology, pages 573-580, 1992.

[19] L. Golubchik, R.R. Muntz, and R.W. Watson. Analysis of striping techniques in robotic storage libraries.

In Proc. 14th IEEE Mass Storage Systems Symposium, pages 225-238, 1995.

[20] S.E. Hauser, C. Rivera, and G.R. Thoma. Factors affecting the performance of a dos-based WORM file

server. In Mass Storage Systems Symposium, pages 33-37, 1991.

[21] A.R. Hevner. Evaluation of optical disk systems for very large database applications. In AC if SIG-

METRICS conference, pages 166-172, 1 I-"










[22] K. Howard. High speed data duplication/data distribution -an adjunct to the mass storage equation.

In Goddard Conference on Mass Storage Systems and Technology, pages 123-133, 1992.

[23] G. Hull and S. Ranade. Performance measurements and operational characteristics of the Storage Tek

ACS 4400 tape library with the Cray Y-MP EL. In Goddard Conference on Mass Storage Systems and

Technology, pages 111-122, 1993.

[24] T. Johnson. Analysis of the request patterns to the nssdc on-line archive. In Proc. 4th NASA Goddard

Conf. on Mass Storage Systems and Technologies, 1995.

[25] K. Kant. Introduction to Computer System Performance Evaluation. McGraw Hill, 1992.

[26] S.M. Kelly, R.A. Haynes, and M.J. Ernest. Benchmarking a network storage service. In Mass Storage

Systems Symposium, pages 38-44, 1991.

[27] K.F. Kenk, J.L. Green, and L.A. Treinish. A cost model for NASA data archiving. Technical Report

90-08, National Space Science Data Center, NASA Goddard Space Flight Center, 1990.

[28] B. Kobler, J. Berbert, P. Caulk, and P.C. Hanrahan. Architecture and design of storage and data

management for the NASA Earth Observing System Data and Information System (EOSDIS). In Proc.

14th IEEE Mass Storage Systems Symposium, pages 65-78, 1995.

[29] E.K. Lee and R.H. Katz. An analytic performance model of disk arrays. In AC if SIC I./ TRICS, pages

98-109, 1993.

[30] L. Lueking. Managing and serving a multi-terabyte data set at the Fermilab DO experiment. In Proc.

14th IEEE Mass Storage Systems Symposium, pages 200-208, 1995.

[31] O.I. Pentakalos, D.A. Menasce, M. Halem, and Y. Yesha. Analytical performance modeling of hierar-

chical mass storage systems. Technical Report TR-CS-96-01, Dept. of Computer Science, University of

Maryland, 1995. A short version appears in the 14th IEEE Mass Storage Symposium proceedings.

[32] E. Rahm. Performance evaluation of extended storage architectures for transaction processing. In AC if

SIC ifOD, pages 308-317, 1992.

[33] K.K. Ramakrishnan and J.S. Emer. Performance analysis of mass storage service alternatives for dis-

tributed systems. IEEE Trans. on Software Engineering, 15(2):120-133, 1989.

[34] C. Ruemmler and J. Wilkes. An introduction to disk drive modeling. IEEE Computer, 27:17-28, 1994.









[35] S. Sarawagi. Database systems for efficient access to tertiary memory. In Proc. 14th IEEE Mass Storage

Systems Symposium, pages 120-126, 1995.

[36] Sequoia 2000 home page. http://s2k-ftp.cs.berkeley.edu:8000/.

[37] A. Thomasian. Surveyor's forum: High performance secondary memory. Computing Surveys, 27(2):292
1'- 1995.

[38] A. Thomasian. Approximate analyses for fork/join synchronization in RAID 5. Computer Systems:

Science and Engineering, 1996. To appear.

[39] A. Thomasian and A.N. Tantawi. Approximate solutions for m/g/! fork/join synchronization. In Proc.

1994 Winter Simulation Conference, 1994.

[40] H.C. Tijms. Stochastic Models: An Algorithmic Approach. Wiley, 1994.

[41] S.J. Waters. Estimating disk seeks. The Computer Journal, 18(1):12-17, 1974.

[42] T. Yang, S. Hu, and Q. Yang. A closed-form formula for queuing delays in disk arrays. In Proc. Intl.

Conf. on Parallel Processing, pages 11:189-192, 1994.


Two media per request, large files
Response time
1,400
Analytical
1,200
Simulation
1,000 *

800

600

400

200
20 ----------------------------------------------
0 I I I I I
0 0.0005 0.001 0.0015 0.002 0.0025
Arrival rate
Figure 3: Validation study, response time vs. arrival rate.











Two media per request, small files
Response time
300
Analytical
250 -
Simulation
200 -

150 ---

100

50

0 I
0 0.002 0.004 0.006 0.008 0.01
Arrival rate
Figure 4: Validation study, response time vs. arrival rate.




Six media per request, large files
Response time
2,500
Analytical

2,000 Simulation


1,500


1,000


500
50 -----------------------------------------------
0 I I
0 0.0005 0.001 0.0015 0.002
Arrival rate
Figure 5: Validation study, response time vs. arrival rate.










Six media per request, small files
Response time


600

500

400

300

200

100

0


Analytical

Simulation


0 0.001 0.002 0.003
Arrival rate


0.004 0.005


Figure 6: Validation study, response time vs. arrival rate.




Six media per request, large files
Utilization

Analytical

0.8 Simulation


0 0.0005 0.001 0.0015 0.002
Arrival rate
Figure 7: Validation study, utilization vs. arrival rate.










Six media per request, small files


Utilization
0.8
0.7 ------------------------------
0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 -I
0 0.001 0.002 0.003
Arrival rate


0.004 0.005


Figure 8: Validation study, utilization vs. arrival rate.


Varying arrival rate, large
Response time
4,000 1


1


2 3 4 5 6 7
Media per request
Figure 9: Response time vs. media per requ


files



0.0001


0.00074


0.00138


0.00202


- 0.0025


8

est.


Analytical

Simulation


3,000


2,000


1,000


0










Varying arrival rate, large files
utilization


1


0.8


0.6


0.4


0.2


U


0.0001


0.00074


0.00138


0.00202


0.0025


0 2 4 6 8 10 1
media per request
Figure 10: Drive utilization vs. media per request.


Performance study: Optical vs. Tape drives
Response time
700


0.001 0.002 0.003
Arrival rate


Optical

Tape


0.004


Figure 11: Hypothetical robotic storage library comparison.


-----------
- - - -- x

- -----------------------


600

500

400

300

200

100











Four media per request, small files
Response time
800 ,


2 drives

3 drives

4 drives

5 drives

6 drives

8 drives


0 0.002 0.004 0.006 0.008 0.01
Arrival rate

Figure 12: Response time vs. number of drives.





Four media per request, small files


Utilization


0 0.002 0.004 0.006
Arrival rate


0.008


2 drives

3 drives

4 drives

5 drives

6 drives

8 drives


0.01


Figure 13: Drive utilization vs. number of drives.










Dividing a request between two devices
Six media


Response time
5,000 -


4,000 -


3,000


2,000


1,000


0


0 0.001 0.002 0.003
Arrival rate


Figure 14: Response time of a fork-join request.


Tertiary storage in a QNM
tertiary storage response time
1,400 I


1,200

1,000

800

600

400

200


5000 sim

5000 ana

9000 sim

9000 ana


u I I I I I I I '
0 5 10 15 20 25 30 35 40
customers
Figure 15: Incorporating the RSL model into a queuing network.


3-3


4-2
-4-
5-1


6-0
D-




University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs