Citation
Optimizing service capacity in the drug information service

Material Information

Title:
Optimizing service capacity in the drug information service
Creator:
Halberg, Daniel Lee, 1969-
Publication Date:
Language:
English
Physical Description:
xiv, 243 leaves : ill. ; 29 cm.

Subjects

Subjects / Keywords:
Drug evaluation ( jstor )
Information services ( jstor )
Modeling ( jstor )
Pharmacies ( jstor )
Quality analysis ( jstor )
Questionnaires ( jstor )
Service time ( jstor )
Simulations ( jstor )
Staffing ( jstor )
Telephones ( jstor )
Computer Simulation ( mesh )
Consumer Satisfaction ( mesh )
Department of Pharmacy Health Care Administration thesis Ph.D ( mesh )
Dissertations, Academic -- College of Pharmacy -- Department of Pharmacy Health Care Administration -- UF ( mesh )
Drug Information Services ( mesh )
Quality Assurance, Health Care ( mesh )
Questionnaires ( mesh )
Research ( mesh )
Systems Analysis ( mesh )
Genre:
bibliography ( marcgt )
non-fiction ( marcgt )

Notes

Thesis:
Thesis (Ph.D.)--University of Florida, 1998.
Bibliography:
Bibliography: leaves 234-242.
General Note:
Typescript.
General Note:
Vita.
Statement of Responsibility:
by Daniel Lee Halberg.

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
The University of Florida George A. Smathers Libraries respect the intellectual property rights of others and do not claim any copyright interest in this item. This item may be protected by copyright but is made available here under a claim of fair use (17 U.S.C. §107) for non-profit research and educational purposes. Users of this work have responsibility for determining copyright status prior to reusing, publishing or reproducing this item for purposes other than what is allowed by fair use or other copyright exemptions. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder. The Smathers Libraries would like to learn more about this item and invite individuals or organizations to contact the RDS coordinator (ufdissertations@uflib.ufl.edu) with any additional information they can provide.
Resource Identifier:
029060124 ( ALEPH )
81716234 ( OCLC )

Downloads

This item has the following downloads:


Full Text










OPTIMIZING SERVICE CAPACITY IN THE DRUG INFORMATION SERVICE













By


DANIEL LEE HALBERG











A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY



UNIVERSITY OF FLORIDA 1998





























Copyright 1998 By

Daniel Lee Halberg













This manuscript is dedicated to my two role models, who taught me that there are no limits to the dreams I can achieve. I will forever miss, you both.

To my grandfather, the late James W. Carrig, I would like to thank you for being a father figure and role model at a time in my life when there were few heroes to choose from. You always had a smile on your face and a joke to tell. You taught me that happiness is built on hard work and a loving relationship with family and friends.

To my greatest teacher, the late Kent Harriman, I would like to thank you for being my friend and "coach". Your peerless gift for teaching was a great loss to the world. I am sorry that you can not read this, because I think you would have appreciated this accomplishment. I feel that everything that I am now started with those first steps I took into your classroom. Thank you.














ACKNOWLEDGEMENT S


First, I would like to express my love and appreciation for my wife, Sara A.

Halberg. Without her, I doubt I would have had the courage and discipline necessary to achieve this goal. I will always thank God for making her my wife. I am looking forward to all of our new adventures together.

Second, I would like to extend my sincere and heartfelt gratitude to my major

advisor, Dr. Charles D. Hepler. I thank him for his insight, confidence, and humor. Third, I would like to thank the rest of my committee, Drs. Richard Segal, Earlene Lipowski, Barney Capehart, and Ralph Swain for their interest, direction, and advice. Without their guidance, this project would not have succeeded.

Fourth, I would like to thank the faculty, the staff, and graduate students of the

department of Pharmacy Health Care Administration for their friendship, advice, help and encouragement. However, I would like to extend a special thank you to DeLayne Redding for all her help and concern.

Finally, I would like to thank my entire family, especially my mother, Patricia Halberg, and my two sisters, Darcy and Heather, for their love and support. I bet they thought I would never graduate! Thanks also to my wife's family for their kindness. All of them have supported me in too many ways to mention here, but I wanted them to know that I will always appreciate their love and generosity.











iv













TABLE OF CONTENTS

ACKNOW LEDGEMENT S ............................................................................................ iv

L IS T O F T A B L E S .......................................................................................... ............ v iii

L IS T O F F IG U R E S ....................................... .............................................................. x i

A B S T R A C T ........................................ ..................................................................... x iii

CHAPTERS

1. IN T R O D U C T IO N ..................................................................................................... I

In tro d u c tio n ............. .............. ............ ....... I ............................ .......................... I
B a c k g ro u n d ......................................................................................................... 3
P ro b lem S tate m en t ............................................................................................... 7
R esearch F ram ew o rk ........................................................................................... 8
S ig n ifi c a n c e .................... ..................................................................................... 8
R e se arch Q u e stio n s .................................................................... ...... ............... 9

2. LITERATURE REVIEW ..................... ....... .......................................................... I I

O v e rv ie w ............................................ ................................................... .......... I I
Service Capacity and W ait Times .............................................. ........................ 12
Relationships Among Wait Time, Perceived Service Quality, and Customer
S a tisfa c tio n .......................................................................................... 1 9
Perceived Service Quality ........................................................... ....................... 23
Summary of the Literature ................................................... .............................. 30

3. RESEARCH FRAMEW ORK AND HYPOTHESES .... ........ ....... ......................... 32

Research Framework ................................ ......................................................... 32
Research Hypothesis and Specific Research Questions .............. .......... ............. 35

4 M E T H O D S ............................................................................................................. 4 1

O v e rv ie w ....................... .......................... ........................................................ 4 1
S tu d y L o c atio n ................................................................................................... 4 1
Data Sources, Sample Selection, and Data Collection Procedures ....................... 43
Sample Size Calculations .................................................................................... 48
S tu d y V a riab le s .................................................................................................. 5 1


v








Questionnaire D evelopm ent and V alidation ........................................................ 55
Simulation Development, Verification, and Validation ........................................ 67
D ata A n a ly sis ..................................................................................................... 7 0

5. PR E L IM IN A R Y R E SU L T S .................................................................................... 73

O v e rv ie w ........................................................................................................... 7 3
Part O ne: H istorical and Concurrent D ata ........................................................... 74
Part Tw o: Student and D irector Interview s ........................................................ 94

6 M A IN R E S U L T S .................................................................................................. 10 6

O v e rv ie w ..................................................... ................................................... 1 0 6
Part Three: Relationships Among PSQ, OSQ, Behavioral Intention,
Perceived Service Time, Actual Service Time and Service Delays ...... 106
P art F our: Sim ulation R esults ................................. ...... ............................ ..... 128

7. DISCU SSION AN D CON CLU SION S .................................................................. 154

O v e rv ie w .......................................... .............................................................. 1 5 4
Sum m ary and D iscussion of R esults ................................................................. 155
C o n c lu sio n s .................................................................................. ................... 1 6 3
L im itatio n s o f S tu dy ......................................................................................... 16 4
R ecom m endations for Future Studies ............................................................... 166

APPENDICES

A. TEXT OF PRE-TEST COVER LETTER ................................................ 169

B. PRE-TEST QUESTIONAIRE, VERSION I ......................................... 170

C. PRE-TEST QUESTIONATRE VERSION 2 .......................................... 175

D. PRE-TEST FOLLOWUP POSTCARD ................................................... 180

E. RESPONSES TO PRETEST QUESTIONAIRE VERSION ONE ........... 181

F. RESPONSES TO PRETEST QUESTIONNAIRE VERSION TWO. ........ 182 G. PRETEST QUESTIONNAIRE WRITTEN COMMENTS ...................... 183

H H ISTORICAL D A TA SHE ET ................................................................ 192

1. D A TA COLLECTION FORM .......................................... ..................... 194




vi









J. SEMI-STRUCTURED OUTLINE FOR STUDENT INTERVIEWS .... 196

K. SEMI-STRUCTURED OUTLINE FOR CO-DIRECTOR
INTERVIEWS.......................... .................................. 198

L. TEXT OF COVER LETTER FOR MAIN QUESTIONNAIRE ......... 200 M. M AIN QUESTIONAIRE ..................................... 201

N. FOLLOWUP POST CARD FOR MAIN QUESTIONNAIRE ......... 206 0. RESPONSES TO MAIN QUESTIONAIRE ....................... 207

P. MAIN QUESTIONNAIRE WRITTEN COMMENTS.................... 208

Q. SIMULATION BLOCK DIAGRAMS ...........................219

R. SIMIUILATION PROGRAM CODE ......................................... 225

LIST OF REFERENCES .................. _............ ......................... 234

BIOGRAPHICAL SKETCH................................................................ 243





























vii













LIST OF TABLES

Table Page

4-1. Required Sample Sizes for Selected System Parameters ................................ 49

4-2. Responses to Pre-test Questions 13 and 21 ...................................... ............ 62

4-3. Pre-test Item-total Statistics for SERVPERF Subscale .................................. 64

4-4. Pre-test Item-total Statistics for Perceived Service Time ............................... 65

4-5. P re-test Item C om m qualities ......... ................................................ .............. 66

4-6. Rotated Component Matrix of Pre-test Responses ........................................ 66

5-1. Percentage of Questions by Profession ......... .................. ........................ 75

5-2. Percentage of Questions by Subscription Status ............................................ 75

5-3. Percentage of Q uestions by Type ............................................ ..................... 78

5-4. Service Times in Minutes by Question Type ................................................. 78

5-5. Service Times in Minutes Using Three Combined Question Types ................ 79

5-6. Percentage of Questions by Response Type Requested ........................ ........ 81

5-7. Service Time in Minutes by Response Type Requested ................................. 82

5-8. Frequency and Percentage of Response Types by Question Type .................. 82

5-9. Percentage of Questions by D elay Status ...................................................... 83

5-10. Percentage of Delays in Service by Time Needed .......................................... 84

5-11. Descriptive Statistics for the Average Number of Questions Answered
by M onth for the Past Ten Y ears ............. ............................................. 85

5-12. Descriptive Statistics for Question Interarrival Times .................................... 86

5-13. Significant P-Values for Interarrival Times by Hour of Day .......................... 89




viii








5-14. Descriptive Statistics for Question Interarrival Times in Minutes by
Flour of Day................................................................... 90

5-15. Summary Statistics for Input Distributions ...................................... 91

5-16. Kolmogorov-Smvirnov Tests for Exponentially Distributed Variables......... 91

6-1, Questionnaire Sample Description.............................................. 107

6-2. Reasons for Not Reponding..................................................... 109

6-3: Results of One-Way ANOVA Procedures Measuring Response Bias....... 109

6-4. Initial Rotated Component Matrix of Main Questionnaire Responses
for the SERVPERF Sub-Scale.............................................. 110

6-5. Main Questionnaire Communalities for the SERVPERF Sub-Scale ........ 111

6-6. Final Rotated Component Matrix of Main Questionnaire Responses for
the SERVPERF Sub-Scale ................................................. 113

6-7. Item-total Statistics for SERVPERF Sub-Scale................................ 116

6-8. Item-Total Statistics for Service Time Perceptions........................... 117

6-9. Final Sub-Scale Reliabilities ..................................................... 117

6-10. Descriptive Statistics for Questionnaire Measures............................ 117

6-11, Descriptive Statistics for PSQ Items ........................................... 118

6-12. Correlations Between Study Variables......................................... 119

6-13. PSQ by Level of Behavioral Intention .......................................... 121

6-14. OSQ by Level of Behavioral Intention ......................................... 122

6-15. PSQ by Level Perceived Service Time ......................................... 123

6-16. Perceived Service Quality by Delay in Service ..................... ........... 125

6-17, Percentage Below/Above the Mean Actual Service Time by Q27
Expected Time............................................................... 128

6-18. Student Utilization, Total Service Time, and Expected Number in
System by Arrival Modifier at 20 Simulated Days........................ 136





ix








6-19. Descriptive Statistics for Selected Comparisons Between Observed and
S im u late d D ata .................................................................. .................. 13 7

6-20. Comparison of Simulated Queue Statistics Versus Exact Solution .............. 137

6-21. Descriptive for Queue Statistics by Number of Students ............................ 148

6-22. Descriptive Statistics for the Total Service Time, Time in Queue,
Number in System, and Queue Length by Percentage Change in
R esearch and A pproval Tim e ............................................................... 149

6-23. Descriptive Statistics for Number Completed, Utilization Percentage,
and Delay Percentage by Percent Change in Research and Approval
T im e ................................................................................................... 1 5 0

6-24. Effectiveness of Service Capacity Improvements Under Normal Arrival
R a t e s .................................................................................................... 1 5 1

6-25. Sensitivity of Optimal Solution to Changes in the Arrival Rate .................... 153
































x













LIST OF FIGURES

FVle

H ypothesized R elationship s ............................................................................ 8

3-1. H ypothesized Fram ew ork .............. ...... ..................................................... 40

4-1, Scree P lot of P re-test D ata ........................................................................... 66

5-1. 95% Confidence Intervals of Service Time by Question Type (In
M in u te s) ................................................................................................. 7 7

5-2. 95% Confidence Intervals of Service Time Using Three Combined
Q uestion T ypes (in M inutes) ...................................................... .............. ... 80

5-3. 95% Confidence Intervals of Service Time by Response Type (in
M in u te s) ..... ........................................................................................... 8 2

5-4. 95% Confidence Intervals of Average Daily Arrivals by Month .............. ...... 86

5-5. 95% Confidence Intervals of Interarrival Times in Minutes by Hour of
D a y .................. ..................................................................................... 9 0

5-6. Frequency Histogram of Historical Interarrival Times ........ ............... .... 92

5-7. Frequency Histogram of Historical Total Service Times ........ ............ .......... 92

5-8. Frequency Histogram of Service Times for Question Group One .................. 93

5-9. Frequency Histogram of Service Times for Question Group Two ................. 93

5-10 Frequency Histogram of Service Times for Question Group Three ................ 94

6-1. Scree Plot of M ain Questionnaire Responses .............................................. 113

6-2. Regression Equation Plot of PSQ and Predicted PSQ by Actual Service
Time where PSQ=34.443+0.0013 *(Total Service Time) ....................... 126

6-3. Residual Plot of PSQ on Total Service Time ............................................... 126

6-4. Residual Plot of PSQ on Total Service Times Occurring within One
D a y ...................................... ........ ...................................................... 1 2 7


xi








6-5. Expected Number in System for Six Replications of 20 Days ................ ..... 136

6-6. Observed Versus Simulated Probability Density Functions (PDF) ............... 138

6-7. Observed Versus Simulated Cumulative Density Functions (CDF). I ............ 138

6-8. 95% Confidence Intervals for Delay Percentage by Service Rate
M odifier and N um ber of Servers ....................................................... ... 152

6-9, 95% Confidence Intervals for Total Service Time (in Minutes) by
Service Rate Modifier and Number of Servers ...................................... 152








































xii













Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

OPTIMIZING SERVICE CAPACITY IN THE DRUG INFORMATION SERVICE By

Daniel Lee Halberg

May 1998

Chairman: Professor Charles D. Hepler
Major Department: Pharmacy Health Care Administration


Health services are often developed without appropriately analyzing the system's ability to meet the needs of the consumer, and attempts to improve quality and efficiency often do not succeed because of the complexity and dynamic nature of services. However, some organizations are using sophisticated techniques such as simulation to analyze service systems. This study had three primary objectives-. (1) to develop a computer simulation model for a drug information service; (2) to investigate the associations among actual service time, service delays, and perceived service quality; and

(3) to recommend system improvements based on the simulation.

This study used both experimental and non-experimental methods. It was

conducted at the Drug Information and Pharmacy Resource Center (DIPRC) at Shands at the University of Florida. Overall, seven hypotheses and three specific research questions were used to explore relationships among the study variables. Six data sources were used:

(1) historical data sheets, (2) historical workload data, (3) data collection forms, (4) personal interviews, (5) service quality questionnaires, and (6) simulation runs.



xiii









The first three hypotheses tested the relationships among perceived service quality (PSQ), overall service quality (OSQ), and two measures of behavioral intention. A strong, positive relationship was found between PSQ and OSQ. In addition, relationships were found between PSQ and behavioral intention and between OSQ and behavioral intention. The remaining four hypotheses tested the relationships among PSQ, actual service time, service delays, and perceived service time. It was found that only service delays and perceived service time were significantly related to PSQ. However, perceived service time seemed more important than service delays with regard to PSQ. No relationship was found between actual service time and PSQ Surprisingly, no practically significant relationships were found among actual service time, service delays, and perceived service time.

A simulation model was constructed using GPSS/H (General Purpose Simulation System). The simulation was validated and found to be a credible model for analyzing the service system at the DIPRC. Exploration of the three specific research questions indicated that improving service times was more efficient than staffing increases for the purpose of reducing the percentage of service delays.






















xiv













CHAPTER 1

INTRODUCTION


Introduction


Health services are often developed without careful consideration of the actual needs of the consumer, or without appropriately analyzing the service system's ability to meet these needs (Shostack, 1984). Services that fall short of meeting consumer needs must be modified or redesigned in order to improve the quality of the service. Two popular transformation paradigms that are often used to examine the quality problems related to service processes are total quality improvement (TQM) and business process reengineering (BPR).

The TQM paradigm has gained considerable support from the healthcare

community as a transformation philosophy (Boerstler et al., 1996). TQM was pioneered by W. Edwards Deming in the 1 950s, and focuses on the concept of "kaizen ", a Japanese word meaning the continuous incremental improvement of an existing process (Hammer and Champy, 1993). In essence, TQM is an organization wide commitment to steadily and continuously improve quality of the system (Schmele, 1993).

BPR is currently of considerable interest in many service environments including health care. Many individuals confuse the concepts of BPR with TQM. Re-engineering has been defined as "the fundamental rethinking and radical redesign of business processes to achieve dramatic improvements in critical, contemporary measures of performance, such as cost, quality, service, and speed" (Hammer and Champy, 1993, p.32). Thus, reengineering is not fundamentally about process improvement, but process re-invention (Hammer and Champy, 1993).



1





2


However different the approaches, these two paradigms share a common, primary focus on consumer needs and the outcomes of a process. Often, the goal is to achieve a desired outcome (i.e., production of a product or service) while meeting several objectives, including- improved productivity, improved quality, reduced total process time, increased throughput, and reduced waiting times (Hammer and Champy, 1993 Tumay, 1995).

Unfortunately, it is clear that a large percentage of BPR and TQM efforts fail to deliver any of the promised benefits (Boerstler et al., 1996; Hammer and Champy, 1993 Geisler, 1996, Kiely, 1995, Kotter, 1995, Rust et al., 1995). Although there have been many reasons given for these failures, part of the trouble inherently rests in the difficulty of understanding the complex and dynamic interdependencies of service systems. Because of this, some recent transformation projects have used sophisticated techniques, such as computer simulation, in order to analyze the relationships among the various components of service systems and the effects of implementing changes in the system Jumay, 1995).

This study had three primary objectives- (1) to develop a computer simulation

model for a drug information service and to validate the model against the existing system',

(2) to investigate the associations among actual service time, service delays, and evaluations of perceived service quality in a drug information service setting; and (3) to recommend system improvements based on the simulation model; in particular those improvements that reduce the time required to respond to consumer questions and information requests.

The following sections of this chapter will provide background information

describing the underlying concepts used in the research and will introduce the problem statement for the proposed study. The background will specifically address a) how queuing theory can be used as a basis for establishing optimal levels of service capacity; b) how service capacity can affect costs, waiting times, and perceived service quality; and c)





3


the measurement of perceived service quality. Each of these issues, however, will be discussed more critically in the next chapter.


Background



Queuing Theory and the Simulation of a Service SygLem


The goal of most queuing theory and simulation based models is to understand the behavior of a particular system and to make decisions regarding the system based on the behavior of the model (Seila, 1992). Many real-world systems that involve random arrival and service rates can be examined by structuring them as queuing problems. Essentially, these problems can be evaluated in two ways, through either a closed form solution or an open form solution.

Queuing theory uses closed form mathematical relationships to achieve exact

answers to waiting-time and waiting-line problems. Simulations of queuing systems are open form, computationally dynamic models that describe the behavior of a system with respect to time. Open form solutions are used when there are no known equations for the operating characteristics, such as waiting time in the queue, for the system of interest. When available, closed form solutions are usually preferred to open form solutions because of their exactness and theoretical power. However, simulations are used to study waiting situations when closed form solutions are too complex or are not available. For example, simulations are used when the properties of the system to be modeled violate the underlying assumptions of queuing theory, or when the researcher desires more information than the queuing theory approximation provides (Krajewski and Ritzman, 1990; Maryanski, 1980).





4


Service Capacity and its Relationship to Perceived Service Quality


Service capacity can be defined using the queuing theory framework as a function of the staffing level (s) and the service rate (nt). Staffing level refers to the number of employees available to serve consumers. The service rate refers to a variable amount of time required by an employee to complete a service request given their personal ability, their equipment, and the organization of the work (Krajewski and Ritzman, 1990;1 Lovelock, 1987; Winston, 199 1). When there is a shortage of capacity relative to demand

(k), queues form, and total time in the system increases (Lovelock, 1987). It has been demonstrated in the literature that as waiting time increases, evaluations of perceived service quality and customer satisfaction are negatively affected (Bolton and Drew, 1994;Clemnmer and Schneider, 1993; Davis and Vollmann, 1990; Davis, 1991; Dube'-Rioux et al., 1989; Hui and Tse, 1996; Katz, Larson, and Larson, 1991; Taylor, 1994a; Tom and Lucey, 1995).

Two approaches have been used to reduce the adverse effects of waiting on customer satisfaction and perceived service quality in the literature: perceptions management and operations management. Perceptions management is an approach that attempts to reduce the perceived wait times of the consumers of a system through the creative use of distractions, apologies, queue information (e.g., place in line and estimated wait times), and the manipulation of perceived pre-service and post-service waits. According to reports in the literature, perceptions management has met with limited success; however, it is still unclear as to how successful these techniques are across a variety of service settings (Clemmer and Schneider, 1989a, 1989b, and 1993; Hui and Tse, 1996). Operations management is an approach that attempts to reduce the actual wait times of the consumers of a service system using scheduling techniques, queue management, and work flow changes. At the heart of the capacity issue is the development of appropriate queuing systems that utilize capacity to its best advantage





5


(Lovelock, 1987). This research focused primarily on the use of operations management techniques, specifically queuing theory and simulation, to manage service capacity.


Measurement of Perceived Service Quality


Four basic characteristics apply to most services: (1) benefits received from services are largely intangible; (2) services are activity focused rather than product focused; (3) services are simultaneously produced and consumed; and (4) the consumer participates in the production process (Gronroos, 1990). Because of these characteristics it is generally recognized that service quality is harder to evaluate than product quality (Heskett, 1987; Parasuraman et al., 1985).

The concept of quality, however, is difficult to define under any circumstance. Nevertheless, we think of quality in terms of the superiority, excellence, or value of a product or service (Christensen and Penna, 1995; Westgard and Barry, 1986). Two basic methods have been used to assess quality in organizations. The first method uses objective indicators (e.g., age of equipment, number of defects, or consumer time in the system) as measures of quality. The second method uses subjective indicators (e.g., perceived service quality, customer satisfaction, or employee satisfaction) as measures of quality.

Service quality has been defined as the relative superiority of an organization and the services it provides (Parasuraman et al., 1988). Perceived service quality is concerned with the measurement of consumer attitudes regarding an organization's service quality. There has been significant debate in the literature concerning the dimensions and measurement of perceived service quality (Cronin and Taylor, 1992, 1994; Gronroos, 1993; McAlexander, 1994; Parasuraman et al., 1985, 1988; Peter et al., 1993). A perceptions-only scale, derived from the SERVPERF scale (Cronin and Taylor, 1992) will be used as a measure of perceived service quality in this research primarily because it more





6



practical than many of the available instruments and because it avoids some of the measurement issues related to the use of difference scores (Cronin and Taylor 1992, 1994;1 Peter et al., 1993; Zeithaml et al., 1996).


Drug Information Services


The first formally recognized drug information center (DIC) was established at the University of Kentucky in 1962 and by 1995 more than 175 organizations maintained DICs (Parker, 1965; Rosenberg et al., 1995; Vanscoy et al., 1996). The initial role of these centers was to evaluate and compare drugs and to promote rational drug therapy;, however, the role of many centers has evolved to include educational activities, medication policy development, and outcomes research (Beaird et al., 1992; Vanscoy et al., 1996). Health care professionals are currently challenged by the necessity to keep up with the latest developments in new drugs and advances in therapies. Many of the DICs have come into existence because of the recognition by management that it is not efficient to have practitioners review the literature and identify solutions to all of the drug therapy problems they encounter. As such, DICs were developed as a central, organized approach to meeting these needs and to help disseminate drug information to the medical and nursing staff (Skoutakis, 1987; Smith, 1988).

In the current healthcare environment, however, the outlook for DICs is uncertain. Several factors are currently placing increasing pressure on DICs to provide increased levels of service. First, advancing information technologies and managed care influences are forcing DICs to provide the highest quality service with near instantaneous access to information. Second, not only are DICs responsible for dispensing information regarding clinical decisions, they are also being asked to document their impact on patient care using outcomes measurement tools derived from disciplines such as pharmaco epidemiology and





7



pharmacoeconomics. Third, many DICs are also required to participate in scholarly research and educational activities (Skoutakis, 1987; Vanscoy et al., 1996).

While there is a recognized need for drug information services, the current era of cost containment and outcomes management presents a dilemma for health organizations, hospitals, and universities who are now required to justify support for non-profitable programs. Even though DICs are being asked to provide more services, cutbacks in staffing are not uncommon when funds available for such programs are reduced (Mailhot and Giacona-Dahl, 1987; Skoutakis, 1987; Vanscoy et al., 1996).

Thus, the ability of DICs to evaluate their service in terms of effectiveness or

outcomes is critical in maintaining a DIC in the current health care environment. Although there have been a number of articles documenting the activities of drug information and toxicology resources, very few of these articles have addressed drug information quality or effectiveness of DICs under resource constraints (Lilja, 1985; Rosenberg et al., 1995, Skoutakis, 1987). As John Lilja states, "it is safe to say that we know astonishingly little on how to optimize resources for drug information programs" (1985, p.4 12).


Problem Statement


Determining sufficient service capacity in a drug information service is a

challenging issue for managers trying to maintain acceptable levels of service quality. However, questions regarding capacity often involve decisions related to the acceptable amount of time required to deliver the service (e.g., answer a drug-therapy question). Unfortunately, the behavior of queuing systems is deceptively complex and often nonintuitive. If staffed according to "common sense" approaches, many systems are unable to handle the workload.

The mathematics of queuing theory show that inadequate service capacity can

greatly increase the waiting time before service is completed. Consequently, a consumer's





8


overall time in the system will often be longer than intended. Consumers who wait long periods of time for service may be more likely to downgrade the quality of the service, even though other aspects of service performance may have been delivered competently (Taylor, 1994a; Taylor and Claxton, 1994). Unfortunately, however, service processes are often the most complex systems to understand because they frequently depend on the random nature of arrival processes and service times and the dynamic interdependencies of system behavior (Tumay 1995).


Research Framework


This research used a queuing paradigm to evaluate the relationship between service capacity in a drug information service and the length of time that a consumer must wait to obtain a response to a question or information request. In addition, the relationship between waiting time and consumer evaluations of perceived service quality was determined. It was proposed that a relationship existed between service capacity and service time or delays in service, and between service time or delays in service and evaluations of perceived service quality (Figure 1-1). This framework is developed in more detail in chapter three.




Service Service Time Perceived
Capacity and Delays P- Service quality



Figure 1-1. Hypothesized Relationships


Significant


Although simulation research involving staffing patterns has been conducted in a variety of health related systems, including nursing, psychiatry, and emergency





9


departments, no research has been published describing the system behavior of drug information services using a queuing paradigm. In addition, some research has been conducted examining the relationship between perceived service quality and service capacity, however, this area is still in the rudimentary stages of development. This research would help to expand existing knowledge in these areas.

From a more practical standpoint, various researchers have proposed a positive relationship between improved service quality and increased revenues or improved productivity (Gronroos, 1990). However, this benefit may be less useful to "free" service programs that are funded under fixed or shrinking budgets. In this environment, it is more valuable to suggest two possible impacts that service capacity decisions can have on intentions and utilization. First, if service quality is perceived as inferior by consumers then they may be less likely to rely on the service. Thus, inadequate service capacity may reduce the perceived value of the service. If utilization decreases then it becomes more difficult to justify the service's existence. The service may be suspended because consumers feel the quality of a service is poor and have decided not to use it, regardless of their actual need.

Second, budgetary constraints often make it difficult to justify increases in staffing or other improvements in service capacity without significant evidence of need. By using the information obtained from simulation models and perceived service quality surveys, it may be easier to demonstrate current shortfalls in quality, current and projected demand levels, and the positive and negative consequences of changes in service capacity.


Research Question


There are four questions that this research will attempt to address: I Can information regarding staffing levels, service rates, call arrival patterns, and

system structure be used to build a simulation model that is a reliable and valid





10


substitute for the actual drug information service for the purpose of capacity

planning?

2. How do staffing levels, arrival and service rates, and system structure affect the

important performance characteristics of the system? (Adapted from Krajewski and

Ritzman (1990)):

A. Queue Length: This is the expected number of consumer information requests

and questions in the system at a given point in time.

B. Service time: This is the expected total service time required to deliver a

response to a question or information request. This is measured from the

arrival of a question or request for information into the information service

until the delivery of a response.

C. Utilization Rate: This is the collective utilization of the service facilities reflects

the percentage of time the service personnel are busy (as opposed to the time

they are idle). This is described as a ratio of the amount of time the server was

busy over the total time measured.

3. How does total service time relate to consumers' perceived service quality? How

do delays in service relate to consumers' perceived service quality? Could these relationships be used in the simulation model to reflect the impact of changes in

service capacity on consumers' perceived service quality?

4. What are the critical variables that affect the simulation model? Based on these

variables, what management rules can be recommended to improve service capacity

and reduce the response time of the system?














CHAPTER 2
LITERATURE REVIEW


Overview


Chapter one introduced four important concepts and their relationship to this research. First, the capacity of a service system can be described in terms of a queuing paradigm. Second, complex queuing systems can be modeled using computer simulation. Third, waiting times can be influenced by changing the service capacity. Fourth, it was proposed that there is an inverse relationship between waiting time and evaluations of service quality.

This chapter critically reviews the previous research related to the concepts mentioned above and their application to this project. It will begin by presenting an overview of queuing theory. Second, methods for improving the performance (i.e., reducing waiting times) of queuing systems will be described. Third, evidence demonstrating the relationship between wait time and consumers' evaluations of services (in terms of service quality and satisfaction) will be reviewed. It will then discuss how managers often underestimate acceptable wait times and how consumers' overestimate the time waited. Fourth, the chapter will examine the conceptual basis of perceived service quality and describe why this study will use a perceptions-only instrument to measure perceived service quality. The chapter will end with a description of the practical relationship between service quality and behavioral intention. In chapter three, the research framework will be presented and the research hypothesis for this project will be developed.





12


Service Capacity and Wait Times



Overview of Queuing Theor


There are three primary components of a queuing problem. The first is the input source, defined as the population of potential entrants into the service system. We can describe this population in terms of its size, the nature or urgency of need, and the arrival distribution. The size of the input source may be infinite or finite, depending on whether the number of customers in the system significantly affects the arrival rate. The nature or urgency of need influences the relationship between waiting time or queue length and reneging or balking (i.e., leaving without service) (Krajewski and Ritzman, 1990; Winston, 1991).

The arrival distribution is a probability distribution that describes either the number of arrivals per unit time or the time between arrivals (i.e., the interarrival time) (Krajewski and Ritzman, 1990; Winston, 1991). In effect, many managers responsible for assessing service capacity inappropriately assume some constant or narrowly defined arrival pattern. There are, however, other probability distributions that can describe customer arrival streams better than a constant. For instance, arrivals per unit time are sometimes assumed to be Poisson distributed and interarrival intervals are often approximated by some form of the gamma distribution, usually the exponential (Krajewski and Ritzman, 1990; Winston, 1991).

The second component of a queuing system is the service process. We describe the service process in terms of the service arrangement and service rate distribution. The service arrangement comprises the organization of the service (i.e., workflow), the number of servers available to handle the arrivals, and the number of lines leading to those servers. The service rate distribution is a probability distribution governing the amount of the time a server takes to service a customer. Often, the service rate is described by the





13


exponential, Weibull or Erlang distributions (Krajewski and Ritzman, 1990; Law and Kelton, 1991, Winston, 1991).

The third component of a queuing problem involves the queue discipline, also

sometimes called the priority discipline. The queue discipline refers to the order in which customers are processed through the queue. There are several common queue disciplines, including HFO or FCFS (first come first served), LTFO or LCFS (last come first served), SPTF (shortest processing time first), and LPTF (longest processing time first). The latter two are more specifically termed priority queue disciplines because customers are categorized based on their expected length of service. These categories are given a priority level, in which those customers allocated to higher priority levels go before those customers with lower priority. Within each category, however, customers are serviced in a standard queue discipline such as FCFS (Krajewski and Ritzman, 1990; Winston, 1991).

Two other queue disciplines exist which are more difficult to model. The first is "Shortest Lead Time", in which the arrival with the shortest time between the current time and the promised time has a highest priority regardless of when they entered the system. The second is "Arbitrary Priority", where the service order and time are dependent on the servers' preferences or some form of undetermined triaging mechanism. These disciplines are often modeled as a SIRO (service in random order) discipline (Larson, 1987, Schwartz, 1975).

Most queuing models depend on a steady state system for estimating queue statistics for a varying number of servers. A system is in steady state if


(1) the number of servers, the average arrival rate, and the average service
rate are not changing, (2) the average arrival rate is less than the average
service rate times the number of servers, and (3) these conditions have existed for a substantial period of time ... The opposite of steady state is transience, which refers to the behavior of the system during the period
following some change (McClain and Thomas, 1985, p.550).





14


It is not hard to imagine that if the service rate is less than the arrival rate, then the queue will grow without bound because the system is not physically capable of handling the volume of arrivals. However, it is also mathematically true that the queue length will approach infinity when the service rate equals the arrival rate. This can be illustrated using two queuing theory results based on the NMI (single server, single queue) model. First, the utilization rate (p) equals the arrival rate (k) divided by the service rate ( t) (Winston, 199 1). Second, the expected number of customers in a line (L) is a function the utilization rate (p) such that (Winston, 199 1)


(Equation 1-1)




L (Equation 1-2)


Notice that as the arrival rate approaches the service rate, the utilization rate approaches one. Therefore, as the utilization rate approaches one then L approaches infinity (i.e., I divided by 0 --> oo ). This is not an intuitive result, and this is the primary pitfall of naive staffing models (i.e. models that do not account for the effect of random variation on queue behavior). When managers try to match the service capacity exactly to the demand, long waiting lines will occur.

There are six options typically available for improving the performance of systems under a queuing paradigm: (1) add servers, (2) increase the service rate, (3) increase queue size, (4) change the distribution of arrivals, (5) reduce the variance in service times or interarrival times, (6) change the queue discipline. These options are discussed in more detail below.





15


Methods for Improving System Performance


First, adding servers (i.e., increasing staffing) is usually the most frequently

considered method for improving service capacity; however, it can also be costly and often is the least efficient method. As you add more servers, the marginal impact that each new server has on the system decreases. There is also the tradeoff of balancing the utilization rates with excess capacity. Management is interested in maintaining high utilization, but this objective may have an adverse impact on the other operating characteristics. For instance, when utilization is too high, workers may have trouble adapting to changes in the service demand. However, when utilization is too low workers will have too much idle time (Krajewski and Ritzman, 1990). There may also be regulatory or accreditation restrictions that limit the minimum number of servers in a particular setting (Duraiswamy et al., 1981). There have been three different approaches used in the literature to optimize staffing levelsI Adjust staff levels in terms of the actual number of service personnel

(Duraiswamy et al., 198 1 Hammond and Mahesh, 1995 Ishimoto et al.,

1990; Lamy et al., 1970; Saunders et al., 1989; Sumner and Hsieh, 1972).

2. Adjust staff levels based on the number qffiull time equivalents (FTE s)

(Hashimoto et al., 1987). In many settings, a more appropriate method of optimizing staffing is to consider the number of FTEs rather than the actual

number of persons. In this way it becomes easier to consider part-time

employees, full-time employees that devote parts of their work day to

different tasks, and employees that have many different simultaneous tasks

to complete.

3, Adjust staff levels based on a percentage of maximum (rather than

expected) workload (McHugh, 1989). In instances when there are extreme





16


demand shifts, it is sometimes necessary to anticipate staffing levels for the maximum rather than the expected workload. Models using this approach

usually discuss staffing levels in terms of a percentage of the maximum

workload.


Second, queue sizes and wait times can be improved by increasing the service rate rather than the staffing level. Service rates can be improved through new technologies, training, and workflow redesign (Krajewski and Ritzman, 1990). For instance, Carruthers (1970) examined the work turnover rate in a laboratory setting. It was found that the purchase of new equipment was more cost-effective than increasing staffing. Also, Kumar and Kapur (1989) found that increasing hospital nurses' shift length from eight to twelve hours was more beneficial than the addition of staff. Chin and Sprecher (1990) and Ozeki and Ikeuchi (1992) both found that workflow changes were at least as important as staffing increases in improving system wait times. However, increasing service rates do not always improve system wait times, especially when service times are already relatively short. For example, Lamy et al. (1970) found that only a fraction of the total waiting time was related to the actual service time in a pharmacy setting. In this case, the staffing level and variations in arrival times were more significant predictors of queue waits than the service time.

Third, increasing the queue size may be an option if customers are being turned

away because they cannot even enter the system (e.g., busy telephone line). Since queues are stochastic systems, there may be times when a queue exceeds its limitations, even when p is less than one (i.e., arrival rate is less than the service rate). If this happens too often, then queues will have high rejection rates. Costs of increasing the queue size will vary dramatically depending on the type of queue. For instance, the addition of several telephone lines may be inexpensive when compared to the construction of a larger waiting area (Krajewski and Ritzman, 1990).





17


Fourth, changing the arrival rate is usually one of the most subtle and overlooked areas of improvement. Some examples of how arrival rates can be influenced include (1) informing consumers of typically idle or slow periods so that you might attract them to use services during these times instead of peak times, (2) get customers to use alternate routes for obtaining the same information such as a fax-back service or web-site, and (3) schedule appointments with some or all consumers so that arrivals are less random and are reduced during peak times (Krajewski and Ritzman, 1990). Few studies have discussed techniques for modifying the arrival rate into the system, other than appointment systems. One study conducted by Reilly et al. (1978) discussed the impact of optimizing staffing in conjunction with a patient delay- scheduling model, where patients are given a delay time before being admitted to the system. This delay manifested itself either as an appointment or an anticipated waiting time. This had two potential benefits. First, since patients had better knowledge of the length of the wait, they were not necessarily bound to the clinic and could spend time elsewhere; hence, the patients could improve the quality of their waits. Second, although not directly reported by Reilly et al., the realized interarrival variance should have decreased, allowing for a more accurate prediction of staffing requirements.

Fifth, reducing the variance in service times or interarrival times can also reduce queue lengths. By examining the steady state equations for the MJG/k queue characteristics, it can be shown that reducing variance can have a substantial effect on reducing the effective waiting times and queue lengths, Methods of reducing variance in service times have been discussed in the contexts of work design, facility design, total quality management (TQM), and statistical process control (SPC). For example, redesigning or standardizing work flow in order to eliminate errors, backtracking, and rework would help reduce the variance in service times by eliminating inconsistent work patterns. It should be noted that these techniques are often the same ones used to reduce





18


service times; hence, when making changes in the system that are designed to reduce service times, the variance in service times is also often reduced.

Reducing the variance in interarrivals may achieve similar benefits. As mentioned previously, reducing the variance on arrivals might take the form of appointments or blocking types of arrivals into specific time slots (Kleinrock, 1975; Konz, 1990; Lamy et al., 1970; Law and Kelton, 1991; Reilly et al., 1979; Westgard and Barry, 1986; Winston, 1991).

Sixth, changing the queue discipline has also been shown to affect system

performance in terms of waiting times and line lengths. For example, it has been shown that using the shortest-processing-time-first (SPTF) discipline will decrease the variance of the wait times in a system, thereby decreasing the number of long service times due to random variation. However, in many health care services, urgency (i.e., priority) plays a significant role on queue behavior due to prioritization and preemption of service requests based on need (e.g., emergency care), so it would often be impossible to strictly adhere to a SPTF discipline. However, it may still be possible to service non-urgent arrivals using a SPTF discipline (Krajewski and Ritzman, 1990; Law and Kelton, 199 1; Schriber, 1991; Winston, 1991).

In the previous section, an overview of queuing theory was given in order to describe the important variables and concepts that are often used in queuing and simulation models to model waiting times, service times, and delays. In addition, methods for improving system performance were reviewed. In the next section, the relationship among consumer waiting time, perceived service quality, and customer satisfaction are discussed.





19


Relationships Among Wait Time, Perceived Service Quality, and Customer Satisfaction



Waiting Time and Perceived Service Quality


Only three studies were found that examined the effect of wait time on service

quality, and none of the studies used the SERVQUAL or SERVPERF scales to measure perceived service quality. Furthermore, these studies considered service delays rather than queue waits. However, the results of these studies do indicate that wait times can adversely affect evaluations of service quality.

Taylor (1994a) proposed a framework called "The Wait Experience Model" for describing the relationship between the wait experience and the overall service evaluation. This model was evaluated using a sample of airline passengers who experienced preboarding delays ir their flight plans. Taylor found that customers' overall evaluations of service quality were primarily related to their level of anger and its associated feelings of annoyance, irritation and frustration. Anger could be caused by (1) the customer's uncertainty about the length of the delay, (2) the actual length of the delay, (3) the customer's perception of the service provider's control over the delay, and (4) the degree of filled time.

In further research related to this model, Taylor and Claxton (1994) found that individuals who encountered pre-boarding delays were less likely to be satisfied with the quality of the other services offered during the flight. Similarly, Dube'-Rioux et al. (1989) found in a restaurant setting that the timing of the delay (i.e., pre-service, in-service, or post-service) and level of customer need were significant in evaluations of service quality.


Waiting Time and Customer Satisfaction


A much more thorough examination has been conducted in the literature

concerning the relationship between wait time and customer satisfaction. Customer





20


satisfaction is a consumer'Is post-service reflection regarding how well the service compared with their expectations. Customer satisfaction occurs when the perceived performance of the service exceeds expectations, and dissatisfaction occurs when performance is lower than expectations (Bolton and Drew, 1991;- Oliver, 1993).

Evaluations of customer satisfaction and perceived service quality usually rely, either implicitly or explicitly, on the confirmation-disconfirmation paradigm, in which consumer evaluations are based on a confirmation of expectations. In addition, the service quality literature generally conceptualizes customer satisfaction as antecedent to perceived service quality. (Boulding et al., 1993; Bitner, 1990; Cronin and Taylor, 1992; Parasuraman et al., 1985; Parasuraman et al., 1988; Taylor, 1994b; Taylor and Cronin, 1994).

Since the constructs of customer satisfaction and perceived service quality share some of the same dimensions, often a common theoretical basis, and perhaps a causal relationship, it is probable that waiting time and service delays affect both constructs. Therefore, an examination of the literature involving waiting time, service delays, and customer satisfaction is important, especially since many of these studies do not adequately describe their satisfaction instruments or their definitions of satisfaction. This makes it difficult to ascertain whether the authors are measuring customer satisfaction or service quality or both.

Hui and Tse (1996) tested a service evaluation model for a computerized course registration service at a university. Katz, Larson, and Larson (1991) conducted a study with bank customers. Studies done by Davis and Vollmann (1990) and Davis (1991) concerned the length of waits in a fast food restaurant and the percentages of satisfied customers. In all of these studies, the results indicated an inverse relationship between perceived wait times and customer satisfaction. They all provide evidence that as wait times increase customer satisfaction decreases. However, Davis (199 1) suggested that a non-linear relationship exists between waiting time and satisfaction. The proposition of a





21


non-linear relationship between waiting and consumer preference was also suggested by Richard Larson (1987) in a discussion regarding the perceived utility of waiting.

In addition, Tom and Lucey (1995) found two important results concerning

expected waiting times and customer satisfaction in a supermarket setting. First, their results found that customers were more satisfied in situations where the wait was shorter than expected compared with situations where the wait was longer than expected. More importantly, however, the researchers found that it was the reason for the wait that most affected the levels of satisfaction. If the customers blamed the store for the unexpected wait, then satisfaction with the store tended to decrease. However, if customers attributed the wait to something outside of the store's control then no changes in the satisfaction levels were evident.


Manager and Consumer Estimates of Waiting Tim


As discussed above, the literature seems to establish a relationship between the amount of time consumers wait for service and their evaluations of service quality. However, there is evidence to suggest that managers and consumers perceive the wait experience differently. Davis and Vollmann (1990) and Davis (1991) found that managers tended to overestimate the duration of what a customer would consider an acceptable delay. This result is consistent with the gap analysis research conducted by Parasuraman et al. and others (Brown and Swartz, 1989; Parasuraman et al., 1985; Swartz and Brown, 198 9). Furthermore, Katz, Larson, and Larson (199 1) conducted a study with bank customers that found that individuals tend to overestimate their waits. In addition, the researchers asked customers to define what they would consider an acceptable wait. Customers with longer definitions of acceptable wait times tended to be more satisfied than customers with shorter definitions. Thus, if managers overestimate what consumers





22


consider an acceptable delay and consumers overestimate the time they have waited, there is potential for unintended magnification of the actual wait experience.


Linking Service Evaluations to Service Capacity


Two methods have been used to predict how changes in service capacity will affect consumer evaluations of the service. The first method is to operationalize the service quality dimensions as measurable variables. For instance, Ozeki and Ikeuchi (1992) studied service evaluation in a telephone service setting using a workflow simulator and measures of service quality (MO SQs) for different components of the work process. The authors defined an MOSQ as some quantifiable operationalization of service quality, such as response time. Using simulation, the authors were able to see the effects of system changes on these MOSQs, where a change in the desired direction implicitly represented an improvement in quality.

The second method is to describe the relationship between changes in system

performance and service evaluations in terms of a cumulative probability distribution. For instance, Buxton and Gatland (1995) conducted an extensive simulation model that used a customer satisfaction index to model the effects of work-in-process (WIP) and delivery time on levels of customer satisfaction. This customer satisfaction index was expressed as a probability distribution, where a delivery time (e.g., delivery within seven days) was equated to an expected level of customer satisfaction. This approach does not produce exact results; however, it can allow managers to examine relationships between service capacity and perceived service quality.

The previous section evaluated the literature describing how consumer wait times might influence evaluations perceived service performance, such as perceived service quality and customer satisfaction. Furthermore, it was shown how managers' and consumers' perceptions of the wait experience could potentially magnify this relationship.





23


In addition, two methods of building this relationship into a simulation model were summarized. The next section will identify the current state of development in the measurement of perceived service quality and present research indicating that perceived service quality is associated with intended future behavior.


Perceived Service Quality



Overview of the Conceptual Basis of Perceived Service Qualit


Understanding how consumers of a service evaluate service quality is an issue of importance to managers. It is clear that if a service provider understands how consumers evaluate a particular service, managers can use these evaluations to focus on ways to improve. However, developing a model to effectively evaluate service quality has been an evolving and highly debated research issue (Gronroos, 1990).

Four basic characteristics apply to most services. First, services are essentially intangible. Second, services usually focus on activities or information rather than products. Third, services are produced and consumed simultaneously (i.e., they cannot be inventoried). Fourth, the consumer is a participant in the production process (Gronroos, 1990, Lovelock, 1980). As Shostack (1984) describes them, servicess are unusual in that they have impact, but no form" (p. 134). Because of the intangibility of service performance and the aspect of simultaneous production and consumption, it is generally more difficult to develop quality indicators for services than for products (Heskett, 1987; Parasuraman et al., 1985).

Quality, however, is an abstract concept and hard to adequately define. At its most basic level it can be generally thought of from the point of view of Philip Crosby's 4C conformance to requirements" or J.M, Juran's "fitness for use" definitions (Westgard and Barry, 1986, p.5). However, it may be more useful to consider quality as the inherent or





24


implicit degree of excellence, value, or worth of a product or service measured by its ability to satisfy a given need (Christensen and Penna, 1995, Westgard and Barry, 1986). We can usually describe quality in terms of one or more of three dimensions: (1) a structural dimension (i.e., the attributes of the facility, equipment, human resources, and organizational structure that are the components of process); (2) a process related dimension (i.e., the activities that make up the process); and (3) a technical or outcome related dimension (i.e., the end result or effect of a process) (Angaran, 1993; Gronroos, 1990).

Much of the early research concerning service quality focused primarily on

identifying measurable dimensions of service quality. Two early developments of service quality were Lehtinen and Lehtinen's Interaction Quality and Gronroos's Perceived Service Quality (PSQ) models.

The basis of interaction quality was founded on the premise that service quality is formed through the consumer's interaction with the elements of a service organization. This model suggested there were three elements of interaction quality: physical quality (i.e., the tangible aspects of the service such as the equipment or facility); corporate quality (iLe., the image of the service provider); and interactive quality, (i.e., the consumer's interactions with the service provider and other consumers) (Lebtinen and Lehitinen, 1982 as cited in Gronroos, 1993; Parasuraman, Zeithaml, and Berry, 1985; and Swartz and Brown 1989).

The PSQ model developed by Gronroos (1988, 1990) used the confirmationdisconfirmation paradigm to define total perceived service quality as the gap between expected service quality and experienced service quality. Expected service quality is the level of quality that the consumer expects to receive. Experienced service quality is made up of three basic dimensions: (1) technical quality (i.e., quality of the outcome of service);(2) functional quality (i~e, quality of the service process);- and (3) perceived image of the organization (Gronroos 1988, 1990, 1992, 1993).





25


Parasuraman, Zeithaml, and Berry (1985, 1988, 1991) built upon the conceptual basis formed by interaction quality and PSQ in their development of a gap analysis model and the SERVQUAL instrument. Based on extensive focus group interviews, their initial work described five potential gaps in the provision of services: (1) consumer expectation management perception gap, (2) management perception service quality specification gap, (3) service quality specifications service delivery gap, (4) service delivery external communications gap, and (5) expected service perceived service gap. Two fundamental conclusions were developed from the use of this model. First, perceived service quality is a multidimensional construct; however, interaction with the service provider is the most important variable in the assessment service quality. Second, there are often significant perception gaps between the consumers and providers of a service, indicating the service providers do not always understand the expectations of consumers (Brown and Swartz, 1989; Parasuraman et al., 1985; Swartz and Brown, 1989).

Building upon the gap analysis model, Parasuraman, Zeithaml and Berry (1985, 1988, 1991) continued to develop and validate an instrument called SERVQUAL. SERVQUAL is the most widely known measurement of perceived service quality and its development has had considerable impact on the systematic advancement of research concerning perceived service quality of consumer services (Gronroos, 1993). Like the PSQ model before it, SERVQUAL is based on the disconfirmation of expectations paradigm (Gronroos, 1990; Parasuraman et al., 1988).

The SERVQUAL scale consists of 22 item pairs measuring five dimensions of service quality: (1) tangibles, (2) reliability, (3) responsiveness, (4) assurance, and (5) empathy. Factor analysis and reliability testing on data from four service industries were used to develop the final SERVQUAL scale (Parasuraman et al., 1988).





26


Measurement of Perceived Service Oualitv


Although SERVQUAL is perhaps the most widely used instrument to measure service quality, it has received criticism from other researchers who have begun to examine the application of SERVQUAL in various settings. There have been three general areas of concern regarding SERVQUNL: (1) use of difference scores, (2) dimensionality of SERVQUAL, and (3) external validity.

One of the most debated issues that have surfaced concerning the SERVQIJAL instrument is the use of difference scores as prescribed by the confirmationdisconfirmation framework. The first problem with the use of these scores is the timing of the expectation measurement. Expectations may be altered during and after the service experience, suggesting that expectations measured during or after service delivery are not accurate representations of the expectations of the consumer at the point in time when service commenced (Carman, 1990; Gronroos, 1993).

The second problem concerns the implicit nature of the perception measure. Since the perception measure is already a comparison between what the consumer expected and what they perceived as the actual service event, the expectation is already implied in the perception measure. If expectations and perceptions are both measured then expectations are, in effect, measured twice (Gronroos, 1993; Oliver 1993).

The third problem lies in the questionable reliability of difference scores (Brown et al., 1993; Oliver, 1993; Peter et al., 1993). As Peter et al, (1993) state, "[d]ifference scores (1) are typically less reliable than other measures, (2) may appear to demonstrate discriminant validity when this conclusion is not warranted, (3) may be only spuriously correlated to other measures since they typically do not discriminate from at least one of their components, and (4) may exhibit variance restriction." Therefore, the use of difference scores may not be reliable, even when the reliability statistics suggest that the instrument is reliable.





27


Concerns about SERVQUAL's dimensionality have also surfaced in the literature. As mentioned previously, Parasuraman, Zeithaml, and Berry's development of SERVQUAL resulted in a 22-item scale measuring five dimensions. However, several authors have reported results that demonstrate that SERVQUAL's five dimensions do not always generalize across service settings. Studies conducted by Babakus and Mangold (1992), Babakus and Boiler (1992), Brown et al. (1993), Carman (1990), Cronin and Taylor (1992), and Headley and Miller (1993) and all fail in some degree to replicate the original dimensions.

The external validity (i.e., generalizability) of SERVQUAL has been questioned because of the evidence that the level of usefulness of the instrument "as is" may vary depending on the service. It has been shown that the 22-items do not necessarily load on the same factors (Babakus and Boller, 1992; Brown et al., 1993; Carman, 1990, Headley and Miller, 1993; Taylor and Cronin, 1994). In addition, some researchers have suggested that significant wording changes are necessary so that the items are useful in a particular service setting (Babakus and Mangold, 1992; Carman, 1990). Furthermore, it may actually be necessary to modify the length the scale depending on the setting (Babakus and Boller, 1992; Babakus and Mangold, 1992; Carman, 1990).

Because of these problems, other approaches in measuring service quality have been suggested. One of these approaches is to use a perceptions- only scale for the measurement of service quality, such as the SERVPERF instrument tested by Cronin and Taylor (1992). Perceptions-only scales avoid the concerns about the use and reliability of difference scores, without sacrificing scale performance. Perception-only scales also have the advantage of being easier to administer, primarily because the subject does not have to answer both the expectation and performance question subsets. This advantage greatly enhances the practicality of the scale (Babakus and Boller, 1992; Brown et al., 1993; and Cronin and Taylor, 1992, 1994; Headley and Miller, 1993; McAlexander, 1994). Zeithaml et al. (1996) recently recognized the value of perceptions-only scales such as SERVPERF.





28


They state, "The perceptions-only operationalization is appropriate if the primary purpose of measuring service quality is to attempt to explain the variance in some dependent construct..." (p.40).

SERVPERF essentially eliminates the expectation portion of the SERVQUAL scale and focuses entirely on service performance. Cronin and Taylor (1992) tested SERVPERF (i.e., perceptions-only) versus SERVQUAL (i.e., perceptions-minusexpectations). Four service industries were analyzed: banking, pest control, dry cleaning, and fast food. The results of the LISREL and oblique factor analysis procedures did not indicate that the dimensionality conformed to the five-factors proposed by Parasuraman et al., 1988. However, strong reliability scores were exhibited for all for industries (coefficient alphas greater than 0.800). Based on these results, and the failure of other studies to exactly replicate the five dimensions, Cronin and Taylor (1992) suggest that items in SERVQUAL (and hence SERVPERF) should be considered a uni-dimensional measure of service quality rather than a multi-dimensional measure. In addition, SERVPERF explained slightly more of the variation in perceived overall service quality, satisfaction, and purchase intention than SERVQUAL.

Significant debate has occurred in the literature regarding Cronin and Taylor's (1992) performance only approach to measuring perceived service quality (Cronin and Taylor, 1994; Parasuraman et al., 1994). While, Parasuraman et al. (1994) concede that performance only measures such as SERVPERF tend to offer greater predictive power, they do not have as much diagnostic value as disconfirmation measures such as SERVQUAL. In response, Cronin and Taylor (1994) suggest that the SERVPERF scale could be used as a summed or averaged service quality score that might be plotted over time. Therefore, a performance-only measure, such as SERVPERF, should be used when the objective is to obtain an overall measure of service quality that can be used as a dependent variable and analyzed over time.





29


A modified SERVPERF scale will be used in this research because it eliminates the disadvantages of using the difference score approach. In addition, the elimination of the expectation portion of the scale reduces the number of questions that the respondent is required to answer, which enhances ease of administration and should improve the response rate. Furthermore, one of the goals of the proposed research is to study the relationship between wait times and service quality. Since SERVPERF can be used as a summed interval score, this measure is more useful than SERVQUAL for predictive purposes.


Perceived Service Quality and Behavioral Intention


Presumably, there are two reasons for measuring perceived service quality. First, so we can understand and improve the shortcomings of service delivery. Second, to understand the impact that service quality has on future behavior. As mentioned in the previous chapter, behavioral intentions, such as return intention and recommendation, are significant factors in maintaining an effective drug information service. Authors from the health care and other fields have studied the behavioral consequences of service quality.

Babakus and Mangold (1989, 1992), Boulding et al. (1993), Cronin and Taylor (1992), Headley and Miller (1993), Parasuraman et al. (199 1), and Zeithaml et al. (1996) all used modified versions of the SERVQUAL scale to measure service quality and its influence on future intentions in a wide variety of service settings. These studies indicated that perceived service quality was related to loyalty, switching intention, complaining, compliment and recommendation intention, and return intention. Dube'-Rioux et al. (1989) and Bitner (1990) used alternative instruments to look at the issue of service quality and future intent. These two studies used role-playing methodologies involving restaurant and air travel delays, respectively. The results of these studies were consistent





30


with those using the modified SERYQUAL instruments, where service quality was related to intended future behavior.


Summary of the Literature


Queuing theory describes service capacity as a function of the arrival rate (k), the service rate (ri), the number of servers (s). There are typically six options available for reducing waiting times and delays in a system: (1) add servers, (2) improve the service rate, (3) increase queue size, (4) change the arrival rate, (5) reduce the variance in service times or interarrival times, and (6) change the queue discipline. There are at least two methods of building the relationship between waiting time and perceived service quality into a simulation model: (1) operationalizing service quality as a measurable variable, and

(2) estimating the relationship as a probability distribution.

Perceived service quality is a concept that is still evolving. Currently,

SERVQUAL is perhaps the most widely used instrument to measure perceived service quality. The SERVQUAL scale consists of 22 item pairs measuring five dimensions of service quality: (1) tangibles, (2) reliability, (3) responsiveness, (4) assurance, and (5) empathy. Recently, however, it has received criticism from other researchers involving three general areas of concern: (1) SERVQUAL's use of difference scores, (2) the dimensionality of SERVQUAL, and (3) SERVQUAL's external validity.

SERVPERF is a scale that avoids many of the concerns listed above by

ascertaining only consumer perceptions (as opposed to expectations and perceptions) regarding the five dimensions of service quality listed above. Arguments have also been made that SERVPERF exhibits stronger reliability and validity than SERVQUAL SERVPERF also has the advantage of being easier to administer, thus enhancing the practicality of the scale.





31


The literature also suggests that perceived service quality is related to future

consumer behavior, such as return intention and intent to recommend. In addition, there is evidence to hypothesize that evaluations of service quality are affected by waiting time and delays in service. Furthermore, it appears that management tends to overestimate acceptable waits and customers tend to overestimate actual waits.

Chapter three will use the information presented in this literature review to

construct a framework for the variables to be studied. Hypotheses and specific research questions will then be developed based on this framework. Chapter four will present the methods used to test these hypotheses and specific research questions.














CHAPTER 3
RESEARCH FRAMEWORK AND HYPOTHESES


Research Framework

The previous two chapters have described the potential relationships among

service capacity, wait times, perceived service quality, and behavioral intention. Possible changes for improving the performance of service systems, from a queuing theory standpoint were also suggested. This chapter builds on the concepts presented in the introduction and the literature review by presenting a research framework for the variables used in this study.

There are four primary relationships necessary to understanding the research

framework for this study. First, service times can be described using a queuing paradigm as a function of the arrival rate, the service rate, the number of servers, and the priority discipline. important output variables of the system would include the expected service times and queue waits, the percentage of service delays, information regarding number in the system and queue lengths, and the utilization rates of the servers. Second, waiting times and service delays are related to consumer perceptions of service quality, and these perceptions are related to future behavioral intention. However, consumers may not accurately estimate actual waiting times, therefore, perceived waiting time may be a more important variable. Third, by creating and manipulating a valid simulation of the service system (i.e., a computer model of the actual system), we can propose changes in the system that will decrease the amount of time a consumer waits for service to be completed. Fourth, service capacity might be optimized by defining a mathematical relationship between perceived service quality and service times or service delays.



32





33


Definitions of the concepts used in the theoretical framework for this study (Figure 3-1) are discussed below. Where applicable, the first definition refers to the concept as it applies to the empirical measurement and the second definition refers to the concept as it applies to the simulation.


Arrival Rate (k): (1) The empirically observed interarrival distribution of consumer information requests and questions in the Drug Information Service (DIS), (2) The probability distribution input into the simulation model to predict the interarrival distribution of consumer information requests and questions into the DIS.


Actual Service Time: The total time required to research an answer to the question, obtain an approval, and return the answer to the caller that was empirically observed for service process in the DIS. Service time is also referred to as waiting time or response time.


Behavioral Intention: A subject's assessment regarding their future intentions regarding the service. More specifically, it relates to whether or not the subject intends to use the service again or recommend the service to a colleague.


Expected Number in System (L): A simulation output variable that indicates the average number of uncompleted information requests and questions in the DIS.


Expected Queue Length (Lq): A simulation output variable that indicates the average number of information requests and questions in the DIS that have not yet started the research process.





34


Expected Time in Queue (Wq): A simulation output variable that indicates the average amount of time that questions must wait in the queue before starting the research process.


Expected Time in System (W): A simulation output variable that indicates the average total amount of time that a question or information request spends in the system.


Expected Utilization Rate (p): A simulation output variable indicating the average percentage of time that servers were busy.



Overall Service Quality (OSQ): A subject's overall perception of the service quality of the drug information service.


Perceived Service Quality (PSQ): A subject's evaluation of the service quality of the drug information service based on the items in the SERVPERF instrument.


Perceived Service Time: A subject's perceptions regarding the response time of the drug information service. More specifically, it relates to perceptions regarding (1) the acceptability of the response time, (2) the usefulness of the answer once the response was received, (3) the subject's desire for quicker responses from the DIS, and (4) whether the response time was shorter, equal, or longer than expected.


Queue Discipline: (1) The method currently used in the drug information service to prioritize consumers for service, as described by the service providers during personal interviews. (2) A simulation input used to describe the way in which servers decide the order in which information requests and questions are handled by the drug information service.





35


Service Delay: A state indicating whether the actual service time was longer than the response time needed by the caller



Service Rate ([t): (1) The empirically observed service time distributions of consumer information requests and questions in the Drug Information Service (DIS). (2) The probability distribution input into the simulation model to predict the service times for the steps in the service process in the DIS.



Staffing Level (s): (1) The observed number of individuals available to serve consumers and their roles in handling consumer information requests and questions. (2) A simulation input describing the number of individuals available to handle service requests during the various steps of the service process.


Research Hypothesis and Specific Research Questions

The literature has proposed that significant positive relationships should exist among measures of PSQ, OSQ, and intended future behavior (Boulding et al., 1993; Cronin and Taylor, 1992; Parasuraman et al., 1988). Hypothesis one (Hi1) assesses the relationship between PSQ and OSQ. Hypotheses two (H2) and three (H13) are aimed at ascertaining the strength of the relationship between PSQ and behavioral intention (i.e., intent to call again and intent to recommend service), and between OSQ and behavioral intention.

HI: There is a positive relationship between evaluations of perceived service quality
(PS Q) and evaluations of overall service quality (OSQJ.

H2a.: Intention to use the service in the future is positively associated with evaluations
of perceived service quality (PS Q).





36

H2b: Intention to recommend the service to a colleague is positively associated with
evaluations of perceived service quality (PSQ).

H3a: Intention to use the service in the future is positively associated with evaluations
of overall service quality (OSQ).

H3b: Intention to recommend the service to a colleague is positively associated with
evaluations of overall service quality (OSQ).


Previous research has indicated the powerful role of customer perceptions on

evaluations of perceived service quality, including perceptions regarding service time. It has been shown that consumers often cannot accurately ascertain the amount of time within which the service was completed (Katz, Larson, and Larson, 1991). If consumers cannot accurately evaluate the actual service time, then perceived time may be a more important predictor of perceived service quality than actual time. For instance, Tom and Lucey (1995) found that customers were more satisfied with the service when the wait was shorter than expected than when the wait was longer than expected. Hypothesis four

(H4) is aimed at gaining more information regarding how callers' perceptions regarding the response time of the service are related to their attitudes regarding PSQ.


H4a: Acceptability of the response time of the service is positively associated with
evaluations of perceived service quality (PSQ).

H4b: Perceived usefulness of the information once the response was received is
positively associated with evaluations ofperceived service quality (PSQ).

H4c: Perceived quickness of response is positively associated with evaluations of
perceived service quality (PSQ).

H4d Deviations from expected response times are positively associated with
evaluations of perceived service quality (PSQ).





37


As reported in chapter 2, the results of Bolton and Drew (1994), Cleminer and Schneider (I1989a), Davis (199 1), Hui and Tse (1996), and Katz, Larson, and Larson (1991) have all reported evidence to support that an inverse relationship exists between evaluations of the service (i.e., perceived service quality or satisfaction) and waiting time. Additionally, it has been suggested that the relationship between waiting times and perceived service quality may be non-linear (Davis, 1991;, Larson, 1987). Hypothesis five

(H5) evaluates the relationship between service times and evaluations of perceived service quality for generalizability to the drug information setting.


H~a: There is a signifi cant inverse relationship between evaluations of perceived
service quality ('PS Q) and actual service time.

H5b: There is a non-linear relationship between actual service time and perceived
service quality (PS Q).


The literature often does not make a distinction between waiting times and delays in service; however, both have been shown to effect perceived service quality. A delay occurs whenever the actual response time is longer than the promised time. Taylor (1 994a) and Taylor and Claxton (1994) reported that subj ects who experienced boarding delays in an airport setting had lower evaluations of overall service than non-delayed subjects. However, this effect may be moderated by the degree of filled time and consumers' perceptions regarding how much control the service provider had over the delay. Similarly, Dube'-Rioux (1989), using role-playing scenarios for a restaurant setting, also found that delays could affect customer satisfaction; however, the research suggested that perceived need and timing of the delays were also important. Based on this research,





38


hypothesis six (H6) assesses the association between delays in service and evaluations of PSQ.


H6: Delays in service are negatively related to evaluations of perceived service
quality (PS Q).


Additionally, there is little information in the service quality or satisfaction

literature concerning the relationship between actual and perceived service time;, therefore, it is unclear to what degree they are actually associated. However, several authors do suggest that perceived service time may be more important that actual service time in relation to perceived service quality (Hornik, 1982, 1984; Katz, Larson, and Larson, 1991; Taylor, 1994a). Hypothesis seven (H7) explores the associations among the variables measuring perceived service time, actual service time, and service delays.


H7a: There is a positive relationship between actual service time and perceived service
time.

H7b: There is a positive relationship between service delays and perceived service time.


The previous chapter discussed ways in which the performance of a system could be improved. It is believed that by changing service capacity in ways that decrease the overall service times and the number of service delays, consumer evaluations of perceived service quality may be improved. Using a queuing theory framework to optimize service capacity suggests that we should examine the effects of changes in staffing levels (s) and service rates (p}) Specific research questions one through three (RI, R2, and R3) assess the relative impact and sensitivity of these methods in terms of improvements in service times and service delays.





39


RI: How do changes in staffing levels and service rates impact simulated service
times in the drug information service?

R2: What combination of changes in staffing levels, service rates optimizes the system
for delays in service when compared against service quality and cost in the drug
information service?

R3: How sensitive is this solution to random variation in the system variables (e.g.,
arrival rate) ?







40






INPUTS


Arvl Service staffing DQu
Rate,, Rate La Level (s) Dicipl)Ine







OBSERVED Actual Simulated PREDICTED
System System


OUTPUTS
,/ \\'
// R1, R2,
&R3
/ eriActual / 7\ "".

eay / ime
Tme Expected Expected Number in Service Time Uxtileid
n Q Servce Delays
t L W) and Queue Rate D a
NNumber in Wait (Wq) R


Perceived
Perceived Service .
Service H4-
Time Quality
\ i-\ (PSQ)

-T



Overall
Service oBehaviral
Quality H Intention




1. Use in Future
1. Acceptability of Service Time 2. Recommend to
2. Response Still Useful Colleague
3. Quicker Response
4. Expected Service Time

NOTES:

1. Arrows relating to study hypotheses and specific research questions are labelled with the
number of the hypothesis (H) or research question (R).

2. Arrow direction does not necessarily imply causality.


Figure 3-1. Hypothesized Framework













CHAPTER 4
METHODS


Overview

This chapter describes the methods used to test the research hypotheses and

explore the specific research questions presented in the previous chapter. It includes a description of the study location, the sources of data, the sample selection procedures used in the study, and methods of data collection. In addition, it describes the techniques used to develop and validate the service quality questionnaire and simulation program. It concludes with a description of the data analysis procedures used to test the hypotheses and research questions.


Study Location

This study was conducted at the Drug Information and Pharmacy Resource Center (DIPRC) at Shands at the University of Florida in Gainesville, Florida. The DIPRC accepts drug information questions from practitioners from all over the North Florida region. The DIPRC accepts calls only from practitioners (e.g. pharmacists, physicians, nurses, law enforcement, etc.). Calls from the public are redirected to other resources. The DIPRC categorizes callers into three categories: (1) subscribers, (2) non-subscribers, and (3) University of Florida Health System employees, The service is provided free of charge to all callers; however, subscribers pay a voluntary membership fee to help support the DIPRC.

Questions are presented to the DIPRC from various sources, including telephone, facsimile (fax), electronic mail, and through in-person visits to the center. However, the



41





42


vast majority of calls are presented to the center via telephone. The DIPRC classifies questions into 14 general categories- (1) drug availability, (2) drug dosage and administration, (3) drug identification, (4) drug interactions, (5) drug therapy and efficacy,

(6) drug use in pregnancy or lactation, (7) investigational drugs, (8) IV compatibility or stability, (9) legal, (10) other, (11) pharmacokinetics, (12) side effects or adverse effects,

(13) toxicology, and (14) veterinary drugs.

Besides the nature of the question, subscription status, and profession, callers are asked to provide various demographic information such as name, address, phone number and/or fax number. Callers are also asked for the amount time that they can allow the center to research the question. The DIPRC categorizes these times into four categories,

(1) within 15 minutes (stat), (2) within the day (today), (3) by a specific date (date), and

(4) no rush. Callers may request an oral response to their questions, a written response, or both.

The DIPRC is usually staffed by three Pharm.D. students during their clerkship rotations. Sometimes internship students, visiting international students, and hospital pharmacy residents will assist the DIPRC in answering questions. However, the DIPRC does not usually know when additional students or residents will become available. In addition, the level of contribution that they provide is often limited and unpredictable.

A drug information resident is also usually present in the DIPRC; however, this resident has varying duties and does not usually spend their time directly answering questions. Two co-directors manage the service's operation and approve student responses to information requests. The drug information resident can also approve student responses once he or she is qualified and experienced, as determined by the codirectors.





43


Data Sources, Sami)le Selection. and Data Collection Procedures


The data for the empirical parts of this study were obtained by non-experimental methods. The parts of this study involving computer simulation were experimental in design. Data for this study was collected from six sources. The first two were taken from historical data sources (i.e., historical data sheets and a database documenting the past monthly workload). The third, fourth, and fifth data sources were collected concurrently during the data collection period from June, 1997 through July, 1997 (i.e., specially designed data collection forms, personal interviews, and service quality questionnaires). The last source of data was obtained from computer runs of the simulation program,


Historical Data Sheets


The historical data sheets are the standard forms that the DIPRC uses to document responses to caller questions and information requests (Appendix H), Every question answered by the D1PRC is recorded onto one of these data sheets. In addition, all information regarding a particular call is written on or attached to these data sheets. All of the archived data sheets were retrieved for September, 1996 through May, 1997 (approximately nine months) resulting in a total historical data sheet sample size of 2,3 8 5.

Information taken from the historical data sheets included the file number and the requestor's name, profession, and subscription status. In addition, the question type, when the response was needed, the response type requested, the date and time received, and the date and time completed were recorded, From the data sheets, it was also possible to determine whether or not the service was delayed past the time needed, and whether or not the same person who answered the call also completed the answer, All data were entered into a Microsoft Excel database.





44


Historical Database


A historical database maintained by one of the co-directors of the DIPRC

documenting the monthly workload since 1987 was used to analyze monthly arrivals for seasonal trends. The data consisted of a total of 126 data points (i.e., I I each for January through June and 10 each for July through December). This database contained the total number of questions answered each month as well as the average daily number of questions answered for the month. The average daily number of questions was produced by dividing the total number of questions answered during the month by the number of days the DIPRC was open to take calls. This data was made available to the principal investigator in spreadsheet format.


Data Collecton Forms


A special form was designed to facilitate the specific data collection needs of this study. This form was very similar to historical data sheets described above-, however, modifications were made to incorporate space for additional information, such as the recording of dates and times for specific service activities (Appendix 1). These data collection forms were collected several times each week from June, 1997 through July, 1997, resulting in a total sample of 526 forms. By comparing the file numbers for the data collection forms obtained against a separate entry log, it was determined that 16 data collection forms were missing and could not be located. Therefore, over 97% of the data collection forms filled out during this period were located and entered.

Information taken from the data collection forms included1. The file number.

2. The requestor's name and contact information., the requestor's profession, and

the requestor's subscription status.

3. The question type.





45


4. The time the response was needed and the response type requested.

5. The date and times for each of the four work activities (i.e., take call, research

answer, approve answer, and return answer to caller).

6. Whether or not the service was delayed past the time needed.

7. The persons approving and completing the answer.

8. The number of persons working on the question.


In addition to the information mentioned above available from analyzing the

historical data sheets, the activity based service times on the data collection forms allowed for a more detailed analysis of the service processes of the DIPRC. This provided information necessary to the development of a more robust simulation. All data were entered into a special Microsoft Access database specifically designed for this project.


Personal Interviews


Personal interviews with DIPRC co-directors and externship students were

conducted in order to obtain a more thorough understanding of the various aspects of the center's operations, including the process for answering questions and the priority discipline used by the students to organize work. Twelve personal interviews were conducted from May, 1997 to July, 1997. The interviewees consisted of ten Pharm.D. students and the two co-directors for the DIPRC. The students were all interviewed during their third rotation week. The interviews were semi- structured, using interview outlines to maintain consistency (Appendices J and K). The interviews were audio recorded for accuracy of recall. The interview outlines were developed based on suggestions made by Stewart and Cash (1988).





46

Service Quality Questionnaires

The fourth data source used was a service quality questionnaire sent to

practitioners using the service from June, 1997 through July, 1997. This questionnaire was administered to callers after service completion (Appendix M). This questionnaire assessed perceived service quality (PSQ) using a modified SERVPERF scale, perceived service time, perceived overall service quality (OSQ), and behavioral intention. The inclusion criterion for receiving a questionnaire was any practitioner who submitted a drug information question to the DIPRC during the study period. Callers were excluded from this portion of the study if they had already been sent a questionnaire (i.e., callers were not surveyed more than once). Out of the available 526 samples, 332 questionnaires were sent out to practitioners, 183 samples were repeat callers who had already been sent a questionnaire, and contact information was not available for 11 of the callers.

A data collection procedure based on the "Total Design Method" developed by

Dillman (1978, 1994) was used to maximize the response rate of the questionnaire. There were three phases to this procedure. First, eligible subjects identified from the data collection forms were sent a questionnaire (Appendix M) along with a cover letter from the co-director and the principal investigator explaining the purpose of the research and asking for the subjects' participation (Appendix Q. A self addressed, stamped envelope was also enclosed for the subject to return the survey. Second, approximately one week after the questionnaires were mailed, a reminder post card (Appendix N) was sent to nonresponders asking subjects to fill-out and send in the questionnaire or to contact the principal investigator if they did not receive a questionnaire. Subjects who replied to the post card saying that they never received a questionnaire were promptly sent another. Third, approximately two weeks after the first questionnaire was sent, an attempt was made to contact each non-responding subject by telephone. If the subject was reached, the interviewer attempted to ascertain the reason for the non-response, and, if still





47


applicable, the subject was reminded to send in the questionnaire. If the subject was not reached, an attempt was made to leave a message or reminder through a receptionist or co-worker.

All of the pre-test and main questionnaires were developed using the Survey Pro for Windows software program. This program also provides a facility for data entry and export which was used to record the responses to the questionnaires for future analysis. Responses to the final question (i.e., Q3 5) asking for additional comments were recorded into a word processing document along with all other comments written next to the individual items.


Simulation Runs


The sixth data source used was the output generated by the simulation program. The simulation model was constructed using the GPSS/H (General Purpose Simulation System) simulation language produced by Wolverine Software for use on an MS-DOS based personal computer. Results from the simulation runs were entered into a database for the purposes of summarization and statistical analysis. Five separate groups of simulation runs were made. The first three runs were used to verify and validate the simulation program. The fourth run was used to test specific research questions one and two. The final run was used to determine the sensitivity of the optimal solution as described by the third specific research question.


Procedures for Protecting Privacy and Confidentialit


Each record of information gathered from the historical data sheets or concurrent data collection forms was coded with an identification number. For tracking purposes only, surveys sent out using information collected from the data collection forms were also coded with an identification number. Once all mail questionnaires were returned, all post





48


card and telephone follow-ups completed, and all data entered and verified, the portion of the database containing the callers' contact information was deleted. Furthermore, written comments transcribed from the questionnaires were edited to exclude any personal references. All historical data sheets were returned to the DIPRC once data entry was completed. Hard copies kept of the data collection forms were coded with the identification number and personal information contained on these hard copies was masked using permanent marker. In addition, any comments or quotes used from the personal interviews were edited so that they could not be traced to the speaker. This project was reviewed by the Health Center Institutional Review Board at the University of Florida and approved on July 29, 1997.


.Sample Size Calculations


Required Number of Data Sheets

In order to estimate the sample sizes necessary to provide accurate estimates of the arrival and service time distributions required to construct the simulation of the DIPRC, the data sheets from approximately the first week of data collection were compiled and analyzed. The necessary sample sizes were calculated using the method proposed by Mendenhall, Wackerly, and Scheaffer (1990) for establishing a large sample 95% confidence interval for a given standard deviation and error of estimation- The sample size calculations are summarized in Table 4-1.

The largest of these estimates is the 335 samples required for a 95% confidence

interval using a standard deviation of 45.6 minutes and an error of estimation of 5 minutes. Since the DIPRC receives between 250 to 300 calls per month, two months of data were collected for this study in order to satisfy this sample size requirement. This time period resulted in an actual sample of 526 data sheets, which provided a conservatively large





49


sample from which to estimate the required parameters and allow considerations for missing data.


Table 4-1. Required Sample Sizes for Selected SIstem Parameters
Required
Error of Sample Size
Parameter Mean St. Dev. Estimation (OL=0.05)
Interarrivals 45 min. 4 3 min. 5 min. 296
Reception of Call 4.1 min. 3.7 min. I min. 55
Service Time 42.5 min. 45.6 min. 5 min. 335*
Approval 1.9 min. 3.7 nuin. 1 min. 54
Return Answer 5.9 mi. 8.8 min. 1 min. 310
Largest sample size required


Required Number of Questionnaires


The primary statistical methods used in testing relationships involving the items in the service quality instrument were correlation (HIthrough H4, H6, and H7) and linear regression (H5). As such, there were two considerations driving the sample size determination for the service quality questionnaires. Primary consideration was given to the hypotheses to be tested. Secondary consideration was given to the required number of data points necessary to conduct principal components factor analysis on the SERVPERF portion of the survey. As recommended by Sawyer and Ball (1981), the type I error rate for sample size calculations was set at a=0.05, and the type 11 error rate was set at P=0.20 (where power equals 0.80). The rationale for these error rates is based on the assumption that, for this study, committing a type I error was more critical than committing a type 11 error. Therefore, a and 0 were selected so that only a small chance existed that the null hypothesis would be rejected when no true differences exist and a reasonably high probability of rejecting the null hypothesis when differences do exist.

First, HI through H4 and H6 through H7 used correlation as the primary statistic to detect significant associations among the study variables. The software program "PCSIZE" (Dallal, 1986) was used to estimate the required sample sizes for the correlation.





50


All algorithms used in the program are based on published statistical literature (see Dallal (1986) for references). The results produced from the program itself were verified by Dallal (1986) against a selection of entries from tables presented in Cohen (1977), Fleiss (1981), and Odeh and Fox (1975). The results from the pre-test and reports of similar correlations in the literature suggested that correlations as small as 0.25 may be significant. 1n order to detect a statistically significant correlation of at least 0.25 with a power of 0.80 and ca level of 0.05, a sample size of at least 123 surveys is necessary.

Second, rules-of-thumb for conducting factor analysis generally suggest sample

sizes of at least 100 or between 5 and 10 samples per variable to be analyzed, whichever is greater. The actual number of samples needed depends of the amount of variability explained by the factors, the strength of the factor loadings, and the communalities of the individual variables (Crocker and Algina 1986; Stevens 1996). The factor analysis for the pre-test was conducted with an average of 4.65 samples per variable. Six rotated factors explained 71 percent of the variance, and nearly all of the items loaded strongly on one of the six factors, Furthermore, all but two of the items had communalities greater than 0.6. This evidence suggests that five samples per variable is adequate to factor analyze the SERVPERF portion of the questionnaire. Since there are 20 variables to be analyzed, then 20 times 5 equals a minimum sample size of 100.

Third, H5 used regression analysis to examine the relationship between service time (in minutes) and PSQ. Determining the number of samples required for regression analysis is complex since statistical power for this method is a function of both the number of predictors used in explaining the variance in the dependent variable and the effect size (as measured by R 2) that the researcher wants to detect (Green 199 1). However, S .B. Green (199 1) has developed a two-step methodology for estimating sample sizes for regression purposes that compares favorably with Cohen's (1988) more complicated procedures for multiple regression power analysis. For a statistical power of at least 0.80, the minimum sample size necessary is L divided by/ (i.e., N= L //). Where L =6.4 +





51


1.65m 0.05 M2 and/ = R 2/(1 R 2), where miis the number of predictors to be used in the model (Green 1991, p. 504). To detect a significant relationship between service time and PSQ with an R 2 of at least 0. 10 (iLe., small effect) using simple linear regression (i.e., one predictor), results in an L = 8 and an/ = 11. Using these numbers in the equation for N presented above results in a required minimum sample size of 72.

Therefore, the minimum number of questionnaires needed to conduct the

hypothesis tests with sufficient power was 123. The pre-test achieved a response rate of approximately 67% for a sample of 20 1. Based on the pre-test response rate and discussions with the DIPRC co-directors, a minimum response rate of 50% for the main questionnaire was reasonably anticipated. Thus, a sample of at least 246 callers was calculated as the necessary sample size for the questionnaire portion of this project. During two months of data collection a sample of 332 callers was identified and sent questionnaires, which was considered sufficient for the purposes of this study.


Study Variables


The previous chapter introduced definitions for the variables used in this study as presented in the hypothesized research framework. This section discusses how these variables were measured using the data sources described above.



Arrival Rate (k): Interarrival times were estimated by arranging the arrival times from the historical data sheets (Appendix H) and data collection forms (Appendix I) in ascending order by date and time of arrival. The interarrival time was obtained by subtracting the arrival time for the previous record from the time of the current arrival. For example, if the current record has an arrival time of 12:00 p.m. and the previous arrival occurred at 11:30 am,, then the interarrival time was 30 minutes. Only interarrivals within each day were estimated.





52


Actual Service Time: The actual service time refers to the amount of time required to respond to a question. This was obtained by subtracting the "End Time" for receiving a call from the "Start Time" for returning an answer to a caller. These data were taken from entries made on the data collection forms (Appendix 1).


Behavioral Intention: Behavioral intention was measured by two items (items 33 and 34 in Appendix M). The first item measured was worded, "I intend to use this service in the future." The second item measured was worded, "I would recommend this service to a colleague." Each of these perceptions was measured on a 7-point scale with a one representing "Strongly Agree" and a seven representing "Strongly Disagree". Thus, lower scores indicate greater intention.


Expected Number in System (L): A simulation output variable that indicates the average number of uncompleted information requests in the system. This value was obtained from the queue reports generated by GPSS/H for the queue labeled "TOTALQ", which collects queuing and service time information relating to the entire service process.


Expected Queue Length (Lq): A simulation output variable that indicates the average number of questions in the system that have not yet started the research process. This value was obtained from the queue reports generated by GPSS/H for the queue labeled "BOARDW, which collects queuing and service time information relating just to the period in which questions spend in the queue before starting the research process.


Expected Time in Queue (Wq): A simulation output variable that indicates the average amount of time that an information requests or questions must wait in the queue before starting the research process. This value was obtained from the queue reports generated by GPSS/H for the queue labeled "BOARDQ".





53


Expected Time in System (W): A simulation output variable that indicates the average total amount of time that a question or information request spends in the system. This value was obtained from the queue reports generated by GPSS/H for the queue labeled
if
"TOTALQ.


Expected Utilization Rate (p): A simulation output variable indicating the average percentage of time that servers are busy. This was obtained from the facility reports generated by GPSS/H, which reports utilization as the percentage of the total time that the facilities (i.e., students) were captured (i.e., utilized). Utilization rates for each of the simulated students were averaged together to obtain an expected overall utilization rate.


Overall Service Quality (OSQ): A subject's overall evaluation of the service quality of the drug information service. This perception was evaluated using a single-item measured on a 6-point scale (item 28 in Appendix M). The item was worded "The overall quality of the services provided by the DIPRC is best described as." An "Excellent" rating was scored as a one and an "Unacceptable" rating was scored as a six. Thus, lower scores indicated higher perceived OSQ.


Perceived Service Quality (PSQ): A subject's evaluation of the service quality of the drug information service was measured by summing the individual items of the SERVPERF instrument to obtain an overall perceived service quality score. During the main phase of the study, 20 items initially composed the PSQ scale (numbers four through twenty-three in Appendix M). These items were measured on a seven-point scale anchored with "Strongly Agree" (receiving a value of one) and "Strongly Disagree" (receiving a value of seven). Negatively worded items were reversed scored. One item was dropped from the scale, resulting in a final measure composed of 19 items. Therefore, PSQ had a possible range of 19 to 133 points, with lower scores indicating higher perceived quality,





54




Perceived Service Time: A subject's perceptions regarding the response time of the drug information service. This variable was measured by four questionnaire items. The first was question 24 "Acceptable Time". This item was worded, "The amount of time that it took the DIPRC to respond to my most recent question was acceptable." The second was question 25 "No Longer Useful". This item was worded, "By the time I received a response from the DIPRC, the information was no longer useful to me. The third was question 26 "Quicker Response". This item was worded, "I wish the DIPRC could provide a quicker response to my questions." All three of these items were scored on a seven-point scale anchored with "Strongly Agree" (receiving a value of one) and "Strongly Disagree" (receiving a value of seven). The fourth item was worded "The amount of time that it took the DIPRC to respond to my most recent question was. This item was also scaled on a seven-point scale; however, it was anchored by "Much Shorter than Expected" (receiving a value of one) and "Much Longer than Expected" (receiving a value of seven).


Queue Discipline: The queue discipline is the method currently used in the DIPRC to prioritize questions as they arrive, This discipline was described by the students and codirectors during personal interviews. The queue discipline used by the simulation model was developed using these descriptions.


Service Delay: This is a dichotomous variable measured by comparing the observed service time with the response time needed as reported on the data collection forms and historical data sheets. If the service time was longer than the time needed then the variable was coded with a value of one indicating "Delayed". If the service time was shorter than the time needed then the variable was coded with a value of zero indicating "Not delayed".





55


Service Rate ( i): The probability distribution that is input into the simulation model to describe the service times for the steps in the service process in the DIS. These probability distributions were estimated from the time information obtained from the historical data sheets and concurrent data collection forms discussed above.


Staffing Level (s): A simulation input describing the number of individuals available to handle service requests. Under "Normal Operation" this variable had a value of three, indicating that three students were responsible for staffing the DIPRC. This value was varied from one to five in the simulation model to assess the impact of changes in staffing levels on the results of the simulation model.


Questionnaire Development and Validation

As described in the literature review, service quality instruments based on the items in the SERVQUAL scale are in wide use. Therefore, the consensus among those using the scale appears to be, generally, that the items composing the scale are an adequate representation of perceived service quality, barring modifications necessary for applicability. Furthermore, the procedures used by Parasuraman, Zeithaml, and Berry (1985, 1988) to develop the individual items appear to be well-supported (Cronin and Taylor, 1992; Oliver 1993). Therefore, the question of whether the items in the SERVPERF instrument actually measure the construct of perceived service quality is not at issue in this research. However, because the SERVPERF instrument was modified for use in the drug information setting, it was necessary to validate the content of the questionnaire, explore some general aspects of construct validity, establish internal consistency, and report the potential for non-response bias.





56

Content Validit


Content validity is an assessment of whether the items in a scale adequately

measure the construct of interest (Crocker and Algina, 1986). Three methods were used to establish the content validity of the instrument. First, as suggested by Crocker and Algina (1986) and DeVellis (1991), an expert panel was asked to evaluate the service quality instrument used in this study for clarity and readability. In addition, the panel was also asked to assess whether or not the instrument covered all of the domains that they believed were important in measuring the quality of a drug information service. This panel consisted of


1. Two co-directors of the Drug Information and Pharmacy Resource Center at Shands

and the University of Florida.

2. The director and employees of the Arkansas Poison and Drug Information Center at

the University of Arkansas for Medical Sciences.

3. Three recent callers to the DIPRC identified from the data sheets (2 pharmacists and I

nurse).

4. Three senior graduate students with pharmacy degrees as well as experience and

educational background in survey design and methodology.

5. Two professors experienced in survey research and familiar with the literature

involving perceived service quality.


The recent callers and the graduate students were asked to complete the

instrument and report the length of time spent on the questionnaire. From these six completed questionnaires, it was determined that the typical time for completion of the questionnaire is between 5-10 minutes, depending on the number of written comments.





57


Changes were made to the initial instrument based on the recommendations of the panel. These recommendations were primarily related to clarity and item order issues, however, five items were added to the initial instrument based on the panel's comments:

(1) a question concerning calling frequency (Item 3), (2) a question related to the need for written supporting documents (Item 31), (3) a question involving the service's relation to patient outcomes (Item 32), and (4) and two questions concerning the usefulness of the service (Items 29 and 30).

Second, the original version of the SERVPERF scale did not include a response option allowing subjects to differentiate between items for which they had no opinion or no experience versus items for which they actually had neutral feelings. Survey research conducted by Kippen, Strasser, and Joshi (1997) reported differences in response patterns for subjects that had "No Opinion" and "No Experience" response categories versus subjects that were forced to choose a response. The pre-test was conducted using two versions of the questionnaire. The first version forced the subjects to choose a response from a seven-point scale (Appendix E). The second version included an eighth scale option allowing subjects to select "Don't Know" for questions that they did not have an opinion or they felt were not applicable (Appendix F). The response patterns between these two versions were used to select out items that were not applicable to the drug information setting.

Third, comments were often written next to individual items and in response to the final question of the questionnaire (Appendices G and P). These comments were used in combination with the other validation techniques to help decide if individual items were applicable to the drug information setting.





58


Construct Validity


Construct validity is concerned with the theoretical relationships between variables to the extent that the measures used to represent variables behave as expected in relation to other measures (Crocker and Algina, 1986). Two issues relevant to the construct validity of the questionnaire were examined.

First, the dimensionality of the SERVPERE portion of the questionnaire was

explored. Parasuraman et al. (1985, 1988) have described perceived service quality as a multidimensional construct covering five separate dimensions: tangibles, reliability, responsiveness, empathy, and assurance. However, as described in the literature review, other researchers have had trouble duplicating the original five dimensions. Furthermore, researchers have found that items do not always load on the same dimension. Cronin and Taylor (1992) suggested that the perceived service quality should not be considered as a multi-dimensional construct, but instead a uni-dimensional construct. If SERVPERF can be considered as a multi-dimensional construct then it may be important to examine the differential effects of service time and delays on these dimensions as well as the overall instrument.

This research explored the dimensionality of the SERVPERF portion of the service quality questionnaire using principal components factor analysis with a Varimax (i.e., orthogonal) rotation. The number of components to retain was decided using the Kaiser criterion, such that components with eigenvalues of 1 .00 or higher were retained, and components with less than 1.00 were excluded (Stevens, 1996). Items were assigned according to their factor loadings. Items with loadings less than 0.40 were rejected (DeVellis, 1991; Stephens, 1996).

Second, the correlations among PSQ, OSQ, and behavioral intention obtained from this questionnaire were compared with similar correlations obtained by Cronin and Taylor (1992). In other words, the question was asked, 'Do the variables behave as





59


expected?" This step is important due to the changes made to SERVPERF for use in the drug information setting. Using these comparisons, it was possible to see if the variables used in this research performed consistently with what has been reported in the previous research. Hypotheses HI, H2, and H3 were used to compare these relationships.


Reliability Assessment

Internal consistency measures reliability for a single time period and essentially measures item homogeneity and quality (Crocker and Algina, 1986; DeVellis, 1991). Internal consistency was measured using coefficient alpha (i.e., Cronbach's alpha). Coefficient alpha describes the proportion of the scale score variance attributable to the true score, and is affected by many item quality problems (e.g., non-central mean, poor variability, negative or weak inter-item correlation, and poor item-scale correlation) (DeVellis, 1991). As an absolute measure, an alpha ranging from 0.60 to 0.70 was considered acceptable, 0.70 to 0.80 was considered good, and an alpha ranging from 0.80 to 0.90 was considered very good (DeVellis, 1991).


Assessment of Non-Response Bias

The potential for non-response bias was evaluated in two ways. First, the reasons that subjects gave for not responding to the survey when asked during the reminder telephone call were collated and summarized. Second, one-way ANOVA procedures were conducted for subscriber status, profession, and response interval to check for significant differences in PSQ and OSQ among the respective groups. These dependent variables were chosen because of their importance in the hypothesis tests.

Each questionnaire response was received within one of three response intervals:

(1) response was received before the reminder post card was mailed, (2) response was received after the reminder post card, but before the follow up phone call, and (3)





60


response was received after the follow up phone call. The rationale for analyzing using the response interval to assess non-response bias is based on the assumption that late responders were more likely to be like non-responders in their responses to the questionnaire than subjects who returned the questionnaire without needing a reminder. If significant differences existed between the early and late responders, it was likely that some degree of non-response bias also existed.


Pre-test of Questionnaire


The pre-test for this study was conducted at the Arkansas Poison and Drug Information Center (APDIC) in Little Rock, Arkansas. The APDIC takes drug information and poison intervention calls from practitioners and lay consumers from the state of Arkansas, and has contractual relationships with other organizations and corporations requiring drug information services. The APDIC is staffed primarily by registered pharmacists-, however, Pharm.D. interns do occasionally provide additional assistance.

Only health care practitioners calling in with drug information questions (i.e., nonpoison related) were identified as potential participants. During the study period, 244 callers fit these criteria, however, 41 (16.8%) of these callers were identified as repeat callers, and as such were only sent one questionnaire. Contact information was unavailable for two of the callers. This resulted in a usable pre-test sample of 201 subjects.

Questionnaires were mailed to these 201 subjects on June 5, 1997 and June 6, 1997. A cover letter (Appendix A) from the director of the APDIC was printed on letterhead and sent along with one of two versions of the pre-test questionnaire (Appendices B and C). One of the versions was randomly assigned to each subject. Approximately three weeks after the initial mailing, a postcard was sent to non-responders urging them to mail in the questionnaire (Appendix D). This postcard also thanked





61


respondents if they had already sent in the questionnaire, and asked those who had not received the questionnaire to contact the researcher (see Appendix C). In total, 134 subjects responded in time to be included in the analysis. Three envelopes were returned as undeliverable, two subjects called to say that they had received the postcard but not the questionnaire, and one subject returned the survey completely unanswered. Two subjects returned completed surveys too late to be included in the analysis. This equaled a raw response rate of 66.7% (134 divided by 201) and an adjusted response rate of 69.1% (134 divided by 194).

Of these 134 respondents, 119 (88.8%) were pharmacists, 2 (1.5%) were

physicians, 3 (2.2 %) and 7 (5.2%) were categorized as "Other". One of surveys sent to a physician was returned as undeliverable, and there was no response received from two nurses who were mailed a questionnaire.

For analysis purposes, negatively worded questions (pre-test question numbers 4, 11, 12, 14, 15, 16, 18, 22, 23, 24, 26, and 27) were reverse scored. Furthermore, the first version of the questionnaire included a "Don't Know" checkbox, while the second version did not include this as an option. By comparing the patterns of response for the two versions, it was possible to gain some insight regarding how subjects responded to items that they did not know how to assess or that did not apply to them. In addition, these comparisons made it easier to separate items that were not applicable to the setting from those items about which respondents actually had neutral feelings.

Of the 100 questionnaires sent out without a "Don't Know" response category (i.e., version one), 64 (64.0%) were returned. Of the 101 questionnaires sent out including a "Don't Know" response category (i.e., version two), 70 (69.3%) were returned. The response patterns for the two versions indicated that items 13 ("The APDIC keeps it records accurately.") and 21 ("Employees get adequate support from the APDIC to do their jobs well.") were difficult to answer. Table 4-2 below shows the responses for questions 13 and 21 based on the version. Question 13 on version 2 of the





62


questionnaire drew 34 (34.2%) "Don't Know" responses, and question 21 drew 20 (28.6%) "Don't Know" responses. When comparing versions I and 2 for both of these items, a clear shift is evident from the "Neutral" and no response categories to the "Don't Know" category. This suggests that when subjects were not able to put "Don't Know" as a response, they often responded with a neutral response instead. These data also suggests that these items were not applicable to the drug information setting. Appendices E and F illustrate the responses for all items.

This lack of applicability is further supported by the comments written next to the items (Appendix G). For question 13, six of the respondents wrote in "don't know" next to the question and another wrote in that they did not understand the statement. Similarly, for question 2 1, six respondents indicated that they did not exactly know how to answer the question, one respondent stated that the question did not apply, and another stated that they did not understand the question. As one respondent wrote, "[t]he only hint I have is how well the employees answer my questions," suggesting that subjects find these items difficult to evaluate because they have no direct experience with these issues. Because of this data, it was decided to exclude items 13 and 21 from the rest of the analysis and eliminate them from the final version of the questionnaire,


Table 4-2. Responses to Pre-test Questions 13 and 21
Question 13 Question 13 Question 21 Question 21 Response Version 1 Version 2 Version I Version 2
Category Freg. (%) Freq. (%) Freg. (%) Freg. (%)
Strongly Agree 11(16.9) 9(12.9) 16(25.0) 16(22.9)
Agree 16(24.6) 14(20) 24(37,5) 26(37.1)
Somewhat Agree 0(0.0) 2(2.7) 3 (4.7) 3 (4.3)
Neutral 27(41.5) 9(12.9) 17(26.6) 3 (4.3)
Somewhat Disagree 2(0.03) 0(0.0) 0(0.0) 0(0.0)
Disagree 0(0.0) 0(0.0) 0(0.0) 0(0.0)
Strongly Disagree 0(0.0) 1(1.5) 0(0.0) 1(1.4)
"Don't Know n/a 34(34.2) n/a 20(28.6)
No Response -9(13.9) 1(1.5) 4(6.3) 1(1.4)





63


Reliabilities of Pre-test Measures

The service quality questionnaire used in this study contained four measures: (1) the SERVPERF scale measuring perceived service quality (i.e., items 3 through 24), (2) four items measuring service time perceptions (i.e., items 25 through 28), and (3) overall service quality, and (4) intended future behavior (items 34 and 35). Chronbach's alpha was used to assess the reliabilities of these measures. Perceived service quality as measured by SERVPERF had an alpha of 0.8873 (n=93). The alpha for the items evaluating service time perceptions was 0.6560 (n=13 1). The alpha for intended future behavior was 0.6292 (n=126). Since OSQ was a single item measure, reliability could not be assessed; however, the reliabilities for the other measures were in an acceptable range. Tables 4-3 and 4-4 show the item-to-total statistics for perceived service quality and perceived service time measures. Only question 4 had a corrected item-to-total correlation of less than 0.30; however, since the deletion of question 4 resulted in only a small improvement in alpha (from 0.8873 to 0.8886) the item was retained. Factor Analysis of SERVPERF Scale

A principal components factors analysis using a Varimax rotation was used to

explore the factor structure of the modified SERVPERF scale in this setting. The analysis revealed six factors with eigenvalues over 1.00 explaining approximately 71.4% of the variance. The eigenvalue for the seventh factor was 0.872; therefore, it was unlikely that seven factors would have produced a better separation of the variables. The Scree Plot presented in Figure 4-1 shows the eigenvalues for each component, where components numbered seven and below have eigenvalues of less than one. As expected from the reliability testing, the communalities of each of the variables (Table 4-5) were all satisfactory, with only three of variables having communalities below 0.6 (i.e., Q-5, Q-22, and Q-23).

Table 4-6 below shows the rotated component matrix for the six factor solution. Unfortunately, replication of the factors demonstrated by Parasuraman et. al (1988) was





64



not achieved. However, at least one item from each of the hypothesized dimensions (i.e.,

tangibles, reliability, responsiveness, assurance, and empathy) defined each of the

respective factors. The differences in the factor structure may have resulted from a

number of sources. First, two of the items were reworded to improve clarity. Second,

eight of the items were reworded from second person perspective to first person

perspective. Third, the order of some of the items was rearranged to reduce the sense of

redundancy. Fourth, the items related to the tangibles dimension were replaced by new

items, so it was not determined where these new items would load, or how their

correlations with the other items would affect the factor structure.




Table 4-3. Pre-test Item-total Statistics for SERVPERF Subscale (n=93) Corrected Item- Alpha if Item Item Total Correlation Deleted
Q3 0.3555 0.8865
Q4R 0.2762 0.8886
Q5 0.3189 0.8878
Q6 0.6317 0.8806
Q7 0.6494 0.8791
Q8 0.6609 0.8764
Q9 0.6800 0.8790
Q10 0.6249 0.8795
Q11R 0.6019 0.8788
Q12R 0.5296 0.8832
Q14R 0.6328 0.8786
Q15R 0.6123 0.8802
Q1 6R 0.6572 0.8767
Q17 0.3328 0.8910
Q18R 0.7585 0,8764
Q20 0.6836 0.8790
Q22R 0,5498 0.8807
Q23R 0.3354 0,8875
Q24R 0.3238 0.8890
"R next to the question number indicates that the question was reverse codedfor analysis purposes.





65



Table 4-4. Pre-test Item-total Statistics for Perceived Service Time (n=131) t Corrected Item- Alpha if Item Item Total Correlation Deleted
Q25 0.5235 0.5545
Q26R 0.5074 0.5803
Q27R 0.6125 0.5389
Q28 0.3462 0.6465
t "R" next to the question number indicates that the question was reverse coded for analysis purposes.




Table 4-5. Pre-test Item Communalitiest Item Communality
Q3-Necessary Resources 0.843
Q4R-Background Noise 0.849
Q5-Written Materials 0.555
Q6-Speak Clearly 0.652
Q7-Promised Time 0.831
Q8-Sympathetic & Reassuring 0.611
Q9-Dependable 0.719
Q1O- Provides in Time 0.884
Q11 JR-Individual Attention 0.698
Q12R-When Performed 0.687
Q14R-Prompt Service 0.792
Q15R-Willingness to Help 0.662
Q16R-Too Busy 0.827
Q17-Trust Employees 0.728
Q18R-Personal Attention 0.747
Q19-Polite Employees 0.772
Q20-Safe Interactions 0.738
Q22R-Know Needs 0.518
Q23R-Best Interests 0.469
Q24R-Operating Hours 0.691
t"R" next to the question number indicates that the question was reverse coded for analysis purposes.






66


Table 4-6. Rotated Component Matrix of Pre-test Responses (n=93) t*
Item 1 2 3 4 5 6
Q10O-Provides in Time 0.895 (RE) Q7-Promised Time 0.847 (RE)
Q9-Dependable 0.715 (RE)
Q14R-Prompt Service 0.660 (RS) 0.380 0.391
Q16R-Too Busy 0.853 (RS)
QI 5R-Willingness to Help 0.749 (RS)
Q18R-Personal Attention 0.667 (EM) 0.359
Q19-Polite Employees 0.398 0.600 (AS) 0.501
Q20-Safe Interactions 0.537 (AS) 0.505
Q5-Written Materials 0.719 (TA)
Q1IR-Individual 0.648 (EM)
Attention
Q12R-When Performed 0.432 0.633 (RS)
Q24R-Operating Hours 0.814 (EM)
Q8-Sympathetic & 0.358 0.515 (TA)
Reassuring
Q22R-Know Needs 0.382 0.514 (EM)
Qi 7-Trust Employees 0.432 0.717 (AS)
Q23R-Best Interests 0.536 (EM)
Q6-Speak Clearly 0.452 0.356 0.455 (TA)
Q4R-Background Noise 0.896 (TA)
Q3-Necessary Resources 0.858 (TA)
t "R" next to the question number indicates that the question was reverse coded for analysis purposes.
The letters in parentheses indicate the dimension on which the item loaded in the original SERVQUAL research conducted by Parasuraman et al. (1988). Where EM=Empathy, RS=Responsiveness, RE=Reliability, AS=Assurance, and TA=Tangibles.

10



8



6.



4.



= 2


0)
il]i o
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Component Number

Figure 4-1. Scree Plot of Pre-test Data (Variables = 20; N=93)





67

Simulation Development, Verification, and Validation

Simulation was chosen as the appropriate modeling tool to answer the three

specific research questions posed by this study for three reasons. First, simulations make it easier to clarify thinking about systems problems, because the focus is placed on defining the components of the system rather than the complex interrelationships. Second, simulation enables the consideration of the impact of changes in all the factors influencing the system, such as staffing levels, service rates, and arrival rates. Third, since these changes are made on the computer, effects of systems changes can be evaluated and tested without subjecting the real system to unnecessary stress (Reilly et al., 1978). This section describes the techniques used to construct, verify, and validate the simulation model.


Model Construction

The simulation model was constructed using the GPSS/H (General Purpose

Simulation System) simulation language on an MS-DOS based personal computer. Four steps were necessary to the construction of the simulation, First, approximately 12 hours were spent by the principal investigator observing the system in order to gain a firstperson perspective of the actual work processes in the DIPRC. Different days and times were selected so that a broad perspective was achieved. Second, flow charts were developed documenting the steps necessary to complete a service transaction. From these flow charts, it was be possible to identify the necessary events, facilities, variables, decisions, inputs, and outputs necessary to model the system (Hoover and Perry, 1989). Third, these flow charts were translated into block diagrams that directly represented program code (Appendix G). Fourth, after the historical data sheets, the data collection forms, and the personal interviews were analyzed, the simulation code was written using the data derived from these sources as inputs (Appendix H).





68

Verification of the Model

Once the computer model was constructed, it underwent a verification process. Verification is the process by which the model is tested to make sure that it is performing as it should. In other words, the verification process analyzes whether the program code Correctly represents the model assumptions and system data" (Carson 1989, p.552). Three techniques were used to verify the model. First, tracing the simulation is a common process used to make sure that entities move through the simulation as expected (Hoover and Perry, 1989). The GPSS/H tool called the "interactive debugger", which allows for interactive traces of the simulation, was used for this step of the verification process (Schriber, 1991). Second, logical relationships imposed in the model were verified. This confirmed that facilities and queues were not exceeding their capacity (Hoover and Perry, 1989). Third, simulation results were compared to those expected by an analytical model. This was accomplished by changing the parameters of the model (e.g. changing empirical distribution to exponential distributions and using constants instead of random variables) so that the simulation results could be compared to the results mathematically derived from known equations (Hoover and Perry, 1989).


Model Validation

Throughout the process of the building the simulation, the simulation model

underwent a series of validation steps. Validation is the process by which the researcher determines if the model is "sufficiently accurate for the purpose at hand and which can be used as a substitute for the real system" (Carson, 1989, p.552). In other words, a valid model can be used in place of the real system for purposes of asking questions and making comparisons (Carson, 1989; Hoover and Perry, 1989). Three techniques were used to validate the model- (1) face validation, (2) extreme-conditions tests, and (3) comparison of simulation output to data from the real system (Balci, 1989; Hoover and Perry, 1989).





69


First, face validation was necessary to judge whether or not the model seemed

reasonable to those knowledgeable about the system being studied (Balci, 1989; Carson, 1989,- Hoover and Perry, 1989). In two separate meetings, the co-directors of the DIPRC were given a structured walk-through of the simulation using the block-diagrams. The codirectors were both asked four general questions: (1) "Do you understand how the model will operate?", (2) "Does the model conform to your knowledge of the service processes of the DIPRCT', (3) "Is there anything present in the model that seems incorrect?", and

(4) "Is there some part of the service process that is missing from the model?" Changes to the simulation were made based on the co-directors responses to these questions.

Second, the computer model was validated under extreme conditions. To some

extent, the behavior of a system under extreme conditions should be plausible and conform in the expected direction (Hoover and Perry, 1989; Sargent, 1992). For example, if the arrival rate increases dramatically, we would expect that that line length, time in the queue, and utilization percentage should also increase dramatically.

Third, one of the most powerful techniques for validation is the comparison of the model to the original system. Chi-square goodness-of-fit tests will be used to answer questions concerning the equality of the underlying distributions. It was recognized a priori, however, that the simulation results would probably not exactly match those of the real system. This is because the model is a simplified version of the real system and did not reflect many elements intentionally excluded from the model (Balci, 1989, Hoover and Perry, 1989; Sargent, 1992). Therefore, regression analysis was also conducted to detect how much variation in the real system the simulation actually predicts.


Variance Reduction


Variance reduction in simulation models is important because it helps improve the power for detecting significant statistical differences in the simulation experiments.





70



Variance reduction techniques allow simulation experiments to obtain greater precision (e.g. smaller confidence intervals) with less simulation (Law and Kelton, 1991). One method of variance reduction commonly used is antithetic variation, which uses strong negative covariances to reduce the variation among experimental runs (Law and Kelton 1991; Neelamkavil 1987). Simulation runs were constructed used the antithetic variate capabilities available in GPSS/H through the RMIJLT statements.


Data Analysis


To facilitate the reporting of the results, the data analysis was broken down into four separate parts. The first two parts of this study were essential to develop an understanding of the work processes in the DIPRC necessary to construct the simulation. The results of these first two parts are reported in chapter five as preliminary data. The third and fourth parts tested the research hypotheses and specific research questions developed in previous chapters. The results of these analyses are presented in chapter six as main study results.


Analysis of Preliminary Data


Three data sources were used to evaluate the calling population characteristics as well as the arrival and service rate trends in the DIPRC: (1) archived historical data sheets fr-om September, 1996 through May, 1997 (Appendix H), (2) data collection forms collected concurrently during the data collection period from June 1, 1997 until August 1, 1997 (Appendix I), and (3) an internal database containing the number of questions answered by month for the past ten years. Three analyses were conduced using this data. First, the consistency between the historical data and the concurrent data was evaluated. The two samples were compared by profession, subscription status, question type, response type requested, and percentage of service delays. Proportional differences





71


between the historical and concurrent groups were detected using a two-tailed z-test with (= 0.05. Second, temporal trends were examined by month of year, day of week, and time of day. One-way analysis of variance (ANO VA) was used to detect overall differences in means, and Scheffe' multiple comparisons procedures were used to analyze for significant mean differences between groups. Alpha for the ANOVA and post-hoc procedures was set at the 0.05 level. Third, the empirically observed interarrival and service time distributions were compared with distributions known to be useful in modeling system behavior (i.e., exponential and Weibull). This comparison was completed using Kolmogorov-Smirnov goodness-of-fit tests and regression analysis.

The second part describes the results from personal interviews conducted with the students and co-directors working in the Drug Information and Pharmacy Resource Center (DJPRC). These data were used primarily to obtain information concerning the work process and the prioritization system used to organize the work. In addition, perceptions about caller preferences and recommendations for improvement were also obtained. The data from each of the twelve audio-recorded personal interviews were collated and content analyzed based on the responses to the semi-structured interview outline.


Analysis of Data Related to Main Study


The third part reports the results of the validation and reliability testing of the service quality questionnaire and the hypothesis tests used to explore the relationships among service time, service delays, perceived service time, evaluations of perceived service quality, and behavioral intention in the drug information service setting. Chapter three presented eight hypotheses related to these variables. Hypothesis 5a (H15a) was tested using simple linear regression, and hypothesis 5b (H5b) was evaluated through an examination of residual plots obtained from the regression analysis. The remaining





72


hypotheses were tested using correlation, t-tests, and one-way analysis of variance (ANOVA). Pearson correlation coefficients were used to detect significant relationships between variables. Hypotheses that did not demonstrate statistically significant correlations between the study variables were rejected. Additional analysis of using ANOVA was conducted for hypotheses with statistically significant correlations. Scheffe' multiple comparison procedures were conducted when an ANOVA resulted in a significant F-value indicating overall differences among level means. All analyses were conducted at the 0.05 level.

The fourth part used information from the first three parts to develop and validate the simulation model used to optimize service capacity in the DIPRC based on total service time, percentage of service delays and percent utilization. Validation and verification of the simulation was conducted using regression, X', linear regression, and comparisons of relevant 95% percent confidence intervals. Descriptive statistics, confidence intervals, and ratio analysis was used to explore the three research questions outlined in chapter three.














CHAPTER 5
PRELIMINARY RESULT S


Overview

As discussed in the previous chapter, the data analysis was broken into four related parts. This chapter presents the preliminary results generated from parts one and two of the data. Part one has three subsections. First, the data gathered from the historical data sheets were compared with the concurrent data gathered from the data collection forms. This was done to establish the degree of homogeneity between the historical and concurrent samples and to explore potential sources of bias in the results. The two samples were compared by profession, subscription status, question type, response type, and the occurrence of delays in service. Second, the historical data were analyzed for temporal trends by month, day of week, and time of day. This was done to explore for trends in arrivals that could be reasonably reproduced by the simulation program, and to identify periods for which the simulation results may not be valid. Third, the historical interarrival and service time distributions were tested for goodness-of-fit with known probability distributions (i.e., exponential and Weibull) to determine if the simulation could use approximated known distributions to model empirically observed interarrivals and service times.

Part two has two subsections. First, the interviews conducted with the codirectors of the DJPRC are summarized. Second, a summary of the interviews conducted with the externship students is presented.







73





74

Part One: Historical and Concurrent Data


Analysis by Profession

Four types of profession categories were tracked on the historical and concurrent data sheets- (1) pharmacist/Pharm.D., (2) physician, (3) nurse/nurse practitioner, and (4) a miscellaneous category called "other" (e.g., physician assistants, dentists, nutritionists, law enforcement, etc.) Table 5-1 presents the frequency of occurrence of the four profession categories along with the individual and cumulative percentages that the frequencies represented out of the total number of usable samples. The number of usable data points out of the total number of samples is displayed near the bottom of the table along with the number of missing data points.

Pharmacists represented the largest professional group that the DIPRC serviced. Pharmacists posed 66. 8% of the questions in the historical group and 67.8% in the concurrent group. Although the proportions of pharmacists and nurses were not significantly different between the historical and concurrent samples, it was determined that the historical data had a larger sample of physicians than the concurrent sample (i.e., 11.4% historically versus 8.2% in the concurrent sample (p=0.036)) and a smaller number of other professional types (Le., 13.9% historically versus 17. 1 % in the concurrent sample (p=0,068)). However, the ANOVA did not reveal any significant differences in total service times related to the profession of the caller (F=2.23 1, p=O. 084), reinforcing the homogeneity of the two samples,


Analysis by Subscription Status

Subscription status is typically broken into three groups by the DIPRC: (1) nonsubscribers, (2) subscribers (i.e., those who have donated a subscription fee to the DIPRQ, and (3) University of Florida Health System (UFHS) employees. Analysis using





75


z-tests revealed differences between the historical and concurrent samples with regard to

their subscriber and UFHS percentages (p<0.001 in both cases); however, no differences

were detected for non-subscribers (Table 5-2). Historically, about 39.5% of the callers

were subscribers, 30.0% were non-subscribers, and 30.5 percent were employees of the

UFHS. In the concurrent sample, 26.6% were subscribers, 32.5% were non-subscribers,

and 40.9 percent were employees of the UFHS. Since the ANOVA did not reveal any

overall differences in total service times among the three subscription status groups

(F=2.015, p=0. 135), there was not much concern that differences in subscription status

between the historical and concurrent samples would bias the simulation results.

Table 5-1. Percentage of Questions by Professiont
Historical Data Concurrent Data
Cumul. Cumul.
Profession Freg. Percent Percent Freg. Percent Percent
Pharmacist 1554 66.8 66.8 354 67.8 67.8
Physician* 265 11.4 78.2 43 8.2 76.0
Nurse/N.P. 182 7.9 86.1 36 6.9 82.9
Other 324 13.9 100.0 89 17.1 100.0
Total Questions
Counted 2325 97.5 522 99.2
Missing Data 60 2.5 4 0.8
Total 2385 100.0 526 100.0
t Significance denotes differences between the historical and concurrent groups.
*p<.05 **p<.O1 ***p<. 001


Table 5-2. Percentage of Questions by Subscription Statust
Historical Data Concurrent Data
Cumul. Cumul.
Subscription Status Freq. Percent Percent Freq. Percent Percent
Subscriber*** 932 39.5 39.5 136 26.6 26.6
Non-Subscriber 708 30.0 69.5 166 32.5 59.1
UFHealth System*** 719 30.5 100.0 209 40.9 100.0
Total Questions
Counted 2359 98.9 511 97.2
Missing Data 26 1.1 15 2.8
Total 2385 100.0 526 100.0
t Significance denotes differences between the historical and concurrent groups.
*p<.05 **p<. O ***p<. 001





76

Analysis by Question Typ


As discussed in the previous chapter, the DIPRC answers a variety of drug related questions covering fourteen separate categories. Of these fourteen categories, the five largest in both samples were (1) drug availability, (2) drug dosage and administration, (3) drug identification, (4) drug therapy and efficacy, and (5) a miscellaneous category called "Other". These five categories made up 74.8% and 71.8% of the historical and concurrent questions answered, respectively (Table 5-3).

Only three question categories demonstrated statistically significant proportional differences between the historical and concurrent samples. First, questions concerning drug therapy and efficacy represented about 19.5% in the historical sample; however, they only represented 13. 1% of the total questions in the concurrent sample (p<0.001). This was the only question out of the top five to display a significant difference. Second, questions involving drug use in pregnancy and lactation represented only 2. 1% of the historical sample; however, it represented 7.8% of the concurrent sample (p
Although the proportional differences in question types between the historical and concurrent samples was not great, it was necessary to account for differences in service times among question types. For analysis purposes, question types representing less than five percent of the sample in both the historical and concurrent samples were grouped into one category called "combined". As such, six question types were classified as "combined": investigational drugs, IV compatibility and stability, legal, pharmacokinetics, toxicology, and veterinary drugs. Although questions falling into the drug use in pregnancy and lactation category only represented 2. 1 % of the questions in the historical sample, this category was kept separate since it represented 7.8% of the questions in the concurrent sample.





77


The ANOVA comparing the service times (i.e., time to complete a question) in minutes for each of the nine question groups revealed that the mean service time for at least one question type was significantly different (F=5.127, p<0.001). Post hoc calculations revealed that the drug identification questions were significantly different from questions regarding drug therapy and efficacy (p=O. 00 1) and questions regarding drug use in pregnancy and lactation (p=0.028). No other significant differences were detected. Table 5-4 presents the descriptive statistics for each of these categories and Figure 5-1 graphically illustrates the 95% confidence intervals for service time for each of the question categories. In addition to the two significant differences in service times above, the confidence intervals also suggested that questions regarding side effects and adverse drug effects tended to have longer service times than drug identification questions. Also, it appeared that questions from the "Other" category also tended to have shorter service times with the exception of drug identification questions.


100, 80,

60.
E
4020

0
N 65 60 ;9 3"6 ;7 ; 5 27 40
11 %
01, 6741 41
q, 0&
CP 01
C?
L-J,
Question Type (Combined Categories 7,8,9,11,13,14)

Figure 5-1. 95% Confidence Intervals of Service Time by Question Type (in Minutes)





78


Table 5-3. Percentage of Questions by Type' Historical Data Concurrent Data
Cumul. Cumul.
Question Type Freg. Percent Percent Freq. Percent Percent
Drug Availability 236 10.8 10.8 70 13.3 13.3
Drug Dosage &Admin. 271 12.3 23,1 66 12.5 25.8
Drug Identifi cation 432 19.7 42.8 100 19.0 44.8
Drug Interactions 151 6.9 49.7 36 6.8 51.6
Drug Ther. &Efficacy*** 429 19.5 69.2 69 13.1 64.7
Drug Use in Pregnancy** 46 2.1 71.3 41 7.8 72.6
Investigational Drugs 4 0.2 71.5 3 0.6 73.2
IV Compat. or Stability 51 2.3 73.8 9 1.7 74.9
Legal 33 1.5 75.3 12 2.3 77.2
Other 274 12.5 87.8 73 13.9 91.1
Pharmacokinetics 49 2.2 90.0 17 3.2 94.3
Side Effe cts or ADEs* 199 9.1 99.1 28 5.3 99.6
Toxicology 18 0.8 100.0 1 0.2 99.8
Veterinary Drugs 1 <0. 1 100.0 1 0.2 100.0
Total Questions Counted 2194 92.0 526 100.0
Missing Data 191 8.0 0 0.0
Total 2385 100.0 526 100.0
t Significance denotes differences between the historical and concurrent groups.
* p<.o5 **p<.o] ***p<.o001




Table 5-4. Service Times in Minutes by Question Type
95% 95%
Confidence Confidence
Question Type Median Mean St.Dev. Lower Bound Upper Bound
Drug Availability 21.00 44.20 56.42 30.22 58.18
Drug Dosage &Admin. 28.50 47.50 54.51 33.42 61.58
Drug Identification 15.00 23.18 27.76 17.65 28.72
Drug Interactions 32.50 51.61 45.85 36.10 67.12
Drug Ther. & Efficacy 55.00 65.48 50.98 53.04 77.91
Drug Use in Pregnancy 34.00 63.05 59.28 40.59 85.51
Other 23.00 38.00 44.99 26.85 49.15
Side Effects or ADEs 50.00 63.41 61.31 39.15 87.66
Combined 36.00 53.80 59.19 34.87 72.73
Overall 30.00 46.30 52.13 41.71 50.89





79


Based on the service time results presented above, question categories were

further collapsed into three basic groups for purposes of the simulation. Group one had the shortest service time profile and represented the drug identification and "other" categories equaling approximately 32.2% of the questions, based on the historical data. Group two had a moderate service time profile and represented questions related to drug availability, drug dosage and administration, drug interactions, and the "combined" categories equaling approximately 37. 1% of the questions. Group three had the longest service time profile and represented the drug therapy and efficacy, drug use in pregnancy and lactation, and side effects and adverse drug effects question categories equaling approximately 30.7% of the questions.

The ANOVA comparing the mean service times of the three combined question types was significant (F=18.30, p<0.001) indicating overall differences in means among the groups. Post hoc analysis indicated that the mean service times for all three combined question types were significantly different from one another. Group one had a mean service time of 29.05 minutes (s=26.21 minutes), which was significantly different from both groups two and three (p
Table 5-5. Service Times in Minutes
Using Three Combined Questions Types (in Minutes)
95% 95%
Confidence Confidence
Question Type Median Mean St.Dev. Lower Bound Upper Bound
Group One 15.50 29.05 36.21 23.47 34.64
Group Two 30.00 48.42 54.41 40.86 55.99
Group Three 30,00 64.35 58.51 54.31 74.38





80


80, 70, 60, 50,

4030

cr) 20
N 164 201 133
Group 1 Group 2 Group 3

Three Question Type Categories

Figure 5-2. 95% Confidence Intervals of Service Time Using Three Combined Question Types (in Minutes)


Analysis by Response Type Requested


When students accepted calls from practitioners, the caller was asked how they would like to receive the response to their question. Four response types were available-.

(1) oral; (2) written, (3) both (i.e., oral and written), and (4) either (i.e., oral or written). The most common type of response requested by the callers was an oral response, representing 59.0% in the historical sample and 59.2% in the concurrent sample. The second most requested response type was written, representing 16.9% in the historical sample and 21.2% in the concurrent sample (Table 5-6). The z-tests revealed significant differences regarding the proportion of responses from the "both" and "either" response types observed in the historical and concurrent groups (p<0.001). However, since a significant difference in total service time was not detected between these two groups (p=0.994), then little bias is likely to be introduced.





81


The ANOVA revealed significant service time differences (F=I 1.228, p<0.001) among the response types. The post hoe procedures indicated differences between the oral and both response types (p=0.044) and between the oral and either response types (p<0.001). The 95% confidence intervals for the service times by response type are presented in Table 5-7, and illustrated in Figure 5-3. These data suggested that responses involving a written component tended to take longer than responses requiring just an oral response. These differences may be partially explained by examining the distribution of question types by response type requested. Recalling that drug identification questions tended to have the lowest service times of the eight question categories analyzed, it is evident from Table 5-8 that this category also had the highest percentage of oral responses (83.7%). Furthermore, questions regarding drug therapy, drug use in pregnancy, and side effects tended to have the highest service times and the lowest percentages of oral responses of 37.9%, 43.6%, and 48. 1%, respectively. Since Figure 5-3 shows that oral responses tend to have the lowest service times, it is understandable that drug identification questions would tend to have lower service times than the other question categories. Therefore, for simulation purposes, it was assumed that deviations in service times as a function of response type are explained by the type of question asked.


Table 5-6. Percentage of Questions by Response Type Requeste& Historical Data Concurrent Data
CUMUL CUMUL
Response Type Freq. Percent Percent Freg. Percent Percent
Oral 1273 59.0 59.0 299 59,2 59,2
Written 364 16,9 75.9 107 21.2 80.4
Both 320 14.8 90.7 25 4.9 85.3
Either 201 9.3 100.0 74 14.7 100.0
Total Questions
Counted 2343 98.2 499 94.9
Missing Data 42 1.8 27 5.1
Total 2385 100.0 526 100.0
t Significance denotes differences between the historical and concurrent groups.
*p<.05 **P<.Ol ***P<. 001





82


Table 5-7. Service Time in Minutes by Response Type Requested
95% 95%
Confidence Confidence
Response Type Median Mean St.Dev. Lower Bound Upper Bound
Oral 24.50 35.77 42.44 31.78 41.76
Written 40.00 52.41 48.61 42.91 61.91
Both 36.00 68.43 76.32 35.43 101.44
Either 40.00 71.89 71.79 55.14 88.64
Overall 30.00 46.30 52.13 41.71 50.89


Table 5-8. Frequency and Percentage of Response Types by Question Type Question Type Oral Written Both Either
Drug Availability 42(61.8%) 10(14.7%) 3(4.4%) 13 (19.1%)
Drug Dosage &Admin. 37(57.8%) 15(23.4%) 3(4.7%) 9(14.1%) Drug Identification 82(83.7%) 3 (3.1%) 2(2.0%) 11(11.2%)
Drug Interactions 22(64.7%) 4(11.8%) 3 (8.8%) 5(14.7%)
Drug Ther. & Efficacy 25 (37.9%) 27 (40.9%) 5 (7.6%) 9 (13.6%) Drug Use in Pregnancy 17(43.6%) 9(23.1%) 4(10.3%) 9(23.1%)
Other 40 (57.1%) 23 (32.9%) 0(0.0%) 7(10.0%)
Side Effects or ADEs 13 (48.1%) 9(33.3%) 2(7.4%) 3 (11.1%) Combined 21 (53.8%) 7(17.9%) 3 (7.7%) 8(20.5%)
Overall 299 (59.2%) 107 (21.2%) 25 (5.051) 74 (14. 7%Y)

120


100


80


E 60V)



280 201N. 23Q 73
Oral Written Both Ether

Response Type
Figure 5-3. 95% Confidence Intervals of Service Time by Response Type (in Minutes)





83


Occurance of Service Delays


Each of the samples was evaluated to determine if there had been a delay in service, where the response time was longer than the time requested. The z-test did not reveal any significant differences between the historical and the concurrent data groups in terms of the percentage of delays. The historical data indicated that, overall, 16.3% of questions result in service delays (Table 5-9). The concurrent data was very similar indicating that 18.6% of the questions evaluated during the study period resulted in service delays.

In order to evaluate when delays occur, the distribution of service delays was examined by the time requested for the four categories available on the data collection form (i.e., "Stat", "Today", "Date", and "No Rush"). The results using the concurrent data indicated that those questions requesting Stat" (< 15 minutes) attention resulted in the highest percentage of delays at 58.6% (Table 5-10). Questions requesting an answer to the question within the day (i.e., "Today") were delayed 19.5% of the time, and those requesting an answer by a specific date were delayed 23.2% of the time. "No rush" questions were arbitrarily marked as delayed when total service times were greater than two weeks (2. 1%). Given that the majority of calls requesting "Stat" attention are delayed, it is clear that under the current system in the DJPRC it is very difficult to provide a fifteen minute turnaround time.


Table 5-9. Percentage of Questions by Delay Statust
Historical Data Concurrent Data
Cumul Cumul
Delay Status Freq. Percent Percent Freq. Percent Percent
No 1936 83.7 83.7 428 81.4 81.4
Yes 378 16.3 100 98 18.6 100.0
Total Questions
Counted 2314 97.0 526 100.0
Missing Data 71 3.0 0 0.0
Total 2385 100.0 526 100.0
SSignificance denotes differences between the historical and concurrent groups.
p<.05 **p<.O]1 ***p<.Qo]





84


Table 5-10. Percentage of Delays in Service by Time Needed Delay in Service
No Yes Yes
Time Needed Freg. Freg. Percent
Stat (<15 min) 24 34 58.6
Today 182 44 19.5
By Spec. Date 53 16 23.2
No Rush 143 3 2.1


Arrivals by Mont


A database containing the number of questions per month from January 1987 to June 1997 was analyzed to determine if there were any significant monthly trends in the data. In order to the reduce the error introduced by the differing number of days the DIPRC was available to answer questions each month, the number of questions answered during each month was divided by the number of days the center was actually open. This resulted in an average number of questions answered per day during a particular month. Table 5 -11 shows the descriptive statistics from January to December based on the average number of question arrivals per day.

The ANOVA revealed that at least one month was significantly different (F=3.432, p<0.001) from the other months. Post-hoc calculations using the Scheffe' procedure revealed that the month of December was significantly different from November (p=0.023). No other month-pairs revealed statistically significant differences in average arrivals; however, January (p=O. 074) and August (p=O, 13 9) also tended to have higher average daily arrivals than December. Figure 5-4 graphically illustrates the 95% confidence intervals for the average daily number of arrivals for each of the twelve months.

Overall, the analysis revealed that the average daily number of questions from January 1987 to June 1997 was 12.62 (n=126, s=1.38) and average total number of questions per month was 263.13 (n=126, s=30.32). Limiting this to only the past five years resulted in only slightly higher numbers. From July 1992 to June 1997 the average





85


daily number of questions was 12.75 (n=66, s=1 .25) and the average total number of

questions per month was 265.98 (n=66, s=25.26). In general, however, there are no

practically significant differences among the month that would necessitate special

consideration in the simulation program. However, December represents a special case

for which the simulation program may not be valid.


Table 5-11. Descriptive Statistics for the Average Number of Questions Answered by Month for the Past Ten Years.
95% Lower 95% Upper
Month Median Mean Std. Dev. Bound Bound
Jan. 13.43 13.50 1.59 12.44 14.57
Feb. 12.90 13.00 1.38 12.08 13.93
Mar. 12.39 12.29 1.07 11.57 13.01
A4pr. 12.10 12.27 0.91 11.66 12.89
May. 12.32 12.12 1.27 11.27 12.98
Jun. 12.05 11.85 1.00 11.18 12.52
Jul. 12.69 12.74 1.17 11.91 13.59
Aug. 13.77 13,39 1.22 12.53 14.26
Sep. 12.85 12.65 1.11 11.85 13.45
Oct. 12.67 12.63 0.74 12.10 13.16
Nov. 13.62 13.82 1.72 12.59 15.05
Dec. 11.32 11.15 1.27 10.24 10.24
Overall 12.61 12.62 1.38 12.37 12.86




Interarrivals by Day of Week


The interarrival times (i.e., the time between arrivals) were evaluated by day of

week using the historical data. The ANOVA did not reveal any significant differences in

interarrivals by day of week (F=0.348, p=0.846). however Monday did have the shortest

mean interarrival time of 37.08 minutes (s=40.77), and Friday had the longest at 40.28

(s=39.95) (Table 5-12). However, these differences were not practically significant and

do not warrant special consideration in the simulation.





86


16

15

14,

13,

12,


0~ 10.

Vt)
N= 11 11 11 11 1 11 10 1;0 10 10 10 1;0
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Month

Figure 5-4. 95% Confidence Intervals of Average Daily Arrivals by Month



Table 5-12. Descriptive Statistics for Question Interarrival Times in Minutes by Day of Week 95% Lower 95% Upper Day Median Mean Std. Dev. Bound Bound
Mon. 25.00 37.08 40.77 33.09 41.07
Tue. 27.00 39.12 40.42 34.98 43.26
Wed. 25.00 38.31 42.20 34.14 42.49
Thu. 25.00 39.65 42.52 35.31 44.00
Fri. 30.00 40.28 39.95 36.09 44.47
Overall 25.00 38.84 41.18 38.98 40.70




Full Text
xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E7LGPARKO_NFA1G9 INGEST_TIME 2014-08-25T18:06:04Z PACKAGE AA00024954_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES



PAGE 1

237,0,=,1* 6(59,&( &$3$&,7< ,1 7+( '58* ,1)250$7,21 6(59,&( %\ '$1,(/ /(( +$/%(5* $ ',66(57$7,21 35(6(17(' 72 7+( *5$'8$7( 6&+22/ 2) 7+( 81,9(56,7< 2) )/25,'$ ,1 3$57,$/ )8/),//0(17 2) 7+( 5(48,5(0(176 )25 7+( '(*5(( 2) '2&725 2) 3+,/2623+< 81,9(56,7< 2) )/25,'$

PAGE 2

&RS\ULJKW %\ 'DQLHO /HH +DOEHUJ

PAGE 3

7KLV PDQXVFULSW LV GHGLFDWHG WR P\ WZR UROH PRGHOV ZKR WDXJKW PH WKDW WKHUH DUH QR OLPLWV WR WKH GUHDPV FDQ DFKLHYH ZLOO IRUHYHU PLVV \RX ERWK 7R P\ JUDQGIDWKHU WKH ODWH -DPHV : &DUULJ ZRXOG OLNH WR WKDQN \RX IRU EHLQJ D IDWKHU ILJXUH DQG UROH PRGHO DW D WLPH LQ P\ OLIH ZKHQ WKHUH ZHUH IHZ KHURHV WR FKRRVH IURP
PAGE 4

$&.12:/('*(0(176 )LUVW ZRXOG OLNH WR H[SUHVV P\ ORYH DQG DSSUHFLDWLRQ IRU P\ ZLIH 6DUD $ +DOEHUJ :LWKRXW KHU GRXEW ZRXOG KDYH KDG WKH FRXUDJH DQG GLVFLSOLQH QHFHVVDU\ WR DFKLHYH WKLV JRDO ZLOO DOZD\V WKDQN *RG IRU PDNLQJ KHU P\ ZLIH DP ORRNLQJ IRUZDUG WR DOO RI RXU QHZ DGYHQWXUHV WRJHWKHU 6HFRQG ZRXOG OLNH WR H[WHQG P\ VLQFHUH DQG KHDUWIHOW JUDWLWXGH WR P\ PDMRU DGYLVRU 'U &KDUOHV +HSOHU WKDQN KLP IRU KLV LQVLJKW FRQILGHQFH DQG KXPRU 7KLUG ZRXOG OLNH WR WKDQN WKH UHVW RI P\ FRPPLWWHH 'UV 5LFKDUG 6HJDO (DUOHQH /LSRZVNL %DUQH\ &DSHKDUW DQG 5DOSK 6ZDLQ IRU WKHLU LQWHUHVW GLUHFWLRQ DQG DGYLFH :LWKRXW WKHLU JXLGDQFH WKLV SURMHFW ZRXOG QRW KDYH VXFFHHGHG )RXUWK ZRXOG OLNH WR WKDQN WKH IDFXOW\ WKH VWDII DQG JUDGXDWH VWXGHQWV RI WKH GHSDUWPHQW RI 3KDUPDF\ +HDOWK &DUH $GPLQLVWUDWLRQ IRU WKHLU IULHQGVKLS DGYLFH KHOS DQG HQFRXUDJHPHQW +RZHYHU ZRXOG OLNH WR H[WHQG D VSHFLDO WKDQN \RX WR 'H/D\QH 5HGGLQJ IRU DOO KHU KHOS DQG FRQFHUQ )LQDOO\ ZRXOG OLNH WR WKDQN P\ HQWLUH IDPLO\ HVSHFLDOO\ P\ PRWKHU 3DWULFLD +DOEHUJ DQG P\ WZR VLVWHUV 'DUF\ DQG +HDWKHU IRU WKHLU ORYH DQG VXSSRUW EHW WKH\ WKRXJKW ZRXOG QHYHU JUDGXDWH 7KDQNV DOVR WR P\ ZLIHnV IDPLO\ IRU WKHLU NLQGQHVV $OO RI WKHP KDYH VXSSRUWHG PH LQ WRR PDQ\ ZD\V WR PHQWLRQ KHUH EXW ZDQWHG WKHP WR NQRZ WKDW ZLOO DOZD\V DSSUHFLDWH WKHLU ORYH DQG JHQHURVLW\ ,9

PAGE 5

7$%/( 2) &217(176 $&.12:/('*(0(176 LY /,67 2) 7$%/(6 YLLL /,67 2) ),*85(6 [L $%675$&7 [LLL &+$37(56 ,1752'8&7,21 ,QWURGXFWLRQ %DFNJURXQG 3UREOHP 6WDWHPHQW 5HVHDUFK )UDPHZRUN 6LJQLILFDQFH 5HVHDUFK 4XHVWLRQV /,7(5$785( 5(9,(: 2YHUYLHZ 6HUYLFH &DSDFLW\ DQG :DLW 7LPHV 5HODWLRQVKLSV $PRQJ :DLW 7LPH 3HUFHLYHG 6HUYLFH 4XDOLW\ DQG &XVWRPHU 6DWLVIDFWLRQ 3HUFHLYHG 6HUYLFH 4XDOLW\ 6XPPDU\ RI WKH /LWHUDWXUH 5(6($5&+ )5$0(:25. $1' +<327+(6(6 5HVHDUFK )UDPHZRUN 5HVHDUFK +\SRWKHVLV DQG 6SHFLILF 5HVHDUFK 4XHVWLRQV 0(7+2'6 2YHUYLHZ 6WXG\ /RFDWLRQ 'DWD 6RXUFHV 6DPSOH 6HOHFWLRQ DQG 'DWD &ROOHFWLRQ 3URFHGXUHV 6DPSOH 6L]H &DOFXODWLRQV 6WXG\ 9DULDEOHV Y

PAGE 6

4XHVWLRQQDLUH 'HYHORSPHQW DQG 9DOLGDWLRQ 6LPXODWLRQ 'HYHORSPHQW 9HULILFDWLRQ DQG 9DOLGDWLRQ 'DWD $QDO\VLV 35(/,0,1$5< 5(68/76 2YHUYLHZ 3DUW 2QH +LVWRULFDO DQG &RQFXUUHQW 'DWD 3DUW 7ZR 6WXGHQW DQG 'LUHFWRU ,QWHUYLHZV 0$,1 5(68/76 2YHUYLHZ 3DUW 7KUHH 5HODWLRQVKLSV $PRQJ 364 264 %HKDYLRUDO ,QWHQWLRQ 3HUFHLYHG 6HUYLFH 7LPH $FWXDO 6HUYLFH 7LPH DQG 6HUYLFH 'HOD\V 3DUW )RXU 6LPXODWLRQ 5HVXOWV ',6&866,21 $1' &21&/86,216 2YHUYLHZ 6XPPDU\ DQG 'LVFXVVLRQ RI 5HVXOWV &RQFOXVLRQV /LPLWDWLRQV RI 6WXG\ 5HFRPPHQGDWLRQV IRU )XWXUH 6WXGLHV $33(1',&(6 $ 7(;7 2) 35(7(67 &29(5 /(77(5 % 35(7(67 48(67,21$,5( 9(56,21 & 35(7(67 48(67,21$,5( 9(56,21 35(7(67 )2//2:83 3267&$5' ( 5(63216(6 72 35(7(67 48(67,21$,5( 9(56,21 21( ) 5(63216(6 72 35(7(67 48(67,21$,5( 9(56,21 7:2 35(7(67 48(67,211$,5( :5,77(1 &200(176 + +,6725,&$/ '$7$ 6+((7 '$7$ &2//(&7,21 )250 YL

PAGE 7

6(0,6758&785(' 287/,1( )25 678'(17 ,17(59,(:6 6(0,6758&785(' 287/,1( )25 &2',5(&725 ,17(59,(:6 / 7(;7 2) &29(5 /(77(5 )25 0$,1 48(67,211$,5( 0 0$,1 48(67,21 $,5( 1 )2//2:83 3267 &$5' )25 0$,1 48(67,211$,5( 2 5(63216(6 72 0$,1 48(67,21 $,5( 3 0$,1 48(67,211$,5( :5,77(1 &200(176 4 6,08/$7,21 %/2&. ',$*5$06 5 6,08/$7,21 352*5$0 &2'( /,67 2) 5()(5(1&(6 %,2*5$3+,&$/ 6.(7&+ YLL

PAGE 8

/,67 2) 7$%/(6 7DEOH 3DJH 5HTXLUHG 6DPSOH 6L]HV IRU 6HOHFWHG 6\VWHP 3DUDPHWHUV 5HVSRQVHV WR 3UHWHVW 4XHVWLRQV DQG 3UHWHVW ,WHPWRWDO 6WDWLVWLFV IRU 6(593(5) 6XEVFDOH 3UHWHVW ,WHPWRWDO 6WDWLVWLFV IRU 3HUFHLYHG 6HUYLFH 7LPH 3UHWHVW ,WHP &RPPXQDOLWLHV 5RWDWHG &RPSRQHQW 0DWUL[ RI 3UHWHVW 5HVSRQVHV 3HUFHQWDJH RI 4XHVWLRQV E\ 3URIHVVLRQ 3HUFHQWDJH RI 4XHVWLRQV E\ 6XEVFULSWLRQ 6WDWXV 3HUFHQWDJH RI 4XHVWLRQV E\ 7\SH 6HUYLFH 7LPHV LQ 0LQXWHV E\ 4XHVWLRQ 7\SH 6HUYLFH 7LPHV LQ 0LQXWHV 8VLQJ 7KUHH &RPELQHG 4XHVWLRQ 7\SHV 3HUFHQWDJH RI 4XHVWLRQV E\ 5HVSRQVH 7\SH 5HTXHVWHG 6HUYLFH 7LPH LQ 0LQXWHV E\ 5HVSRQVH 7\SH 5HTXHVWHG )UHTXHQF\ DQG 3HUFHQWDJH RI 5HVSRQVH 7\SHV E\ 4XHVWLRQ 7\SH 3HUFHQWDJH RI 4XHVWLRQV E\ 'HOD\ 6WDWXV 3HUFHQWDJH RI 'HOD\V LQ 6HUYLFH E\ 7LPH 1HHGHG 'HVFULSWLYH 6WDWLVWLFV IRU WKH $YHUDJH 1XPEHU RI 4XHVWLRQV $QVZHUHG E\ 0RQWK IRU WKH 3DVW 7HQ
PAGE 9

'HVFULSWLYH 6WDWLVWLFV IRU 4XHVWLRQ ,QWHUDUULYDO 7LPHV LQ 0LQXWHV E\ +RXU RI 'D\ 6XPPDU\ 6WDWLVWLFV IRU ,QSXW 'L VWULEXWLRQV .ROPRJRURY6PLUQRY 7HVWV IRU ([SRQHQWLDOO\ 'LVWULEXWHG 9DULDEOHV 4XHVWLRQQDLUH 6DPSOH 'HVFULSWLRQ 5HDVRQV IRU 1RW 5HSRQGLQJ 5HVXOWV RI 2QH:D\ $129$ 3URFHGXUHV 0HDVXULQJ 5HVSRQVH %LDV ,QLWLDO 5RWDWHG &RPSRQHQW 0DWUL[ RI 0DLQ 4XHVWLRQQDLUH 5HVSRQVHV IRU WKH 6(593(5) 6XE6FDOH 0DLQ 4XHVWLRQQDLUH &RPPXQDOLWLHV IRU WKH 6(593(5) 6XE6FDOH ,OO )LQDO 5RWDWHG &RPSRQHQW 0DWUL[ RI 0DLQ 4XHVWLRQQDLUH 5HVSRQVHV IRU WKH 6(593(5) 6XE6FDOH ,WHPWRWDO 6WDWLVWLFV IRU 6(593(5) 6XE6FDOH ,WHP7RWDO 6WDWLVWLFV IRU 6HUYLFH 7LPH 3HUFHSWLRQV )LQDO 6XE6FDOH 5HOLDELOLWLHV 'HVFULSWLYH 6WDWLVWLFV IRU 4XHVWLRQQDLUH 0HDVXUHV 'HVFULSWLYH 6WDWLVWLFV IRU 364 ,WHPV &RUUHODWLRQV %HWZHHQ 6WXG\ 9DULDEOHV 364 E\ /HYHO RI %HKDYLRUDO ,QWHQWLRQ 264 E\ /HYHO RI %HKDYLRUDO ,QWHQWLRQ 364 E\ /HYHO 3HUFHLYHG 6HUYLFH 7LPH 3HUFHLYHG 6HUYLFH 4XDOLW\ E\ 'HOD\ LQ 6HUYLFH 3HUFHQWDJH %HORZ$ERYH WKH 0HDQ $FWXDO 6HUYLFH 7LPH E\ 4 ([SHFWHG 7LPH 6WXGHQW 8WLOL]DWLRQ 7RWDO 6HUYLFH 7LPH DQG ([SHFWHG 1XPEHU LQ 6\VWHP E\ $UULYDO 0RGLILHU DW 6LPXODWHG 'D\V L[

PAGE 10

'HVFULSWLYH 6WDWLVWLFV IRU 6HOHFWHG &RPSDULVRQV %HWZHHQ 2EVHUYHG DQG 6LPXODWHG 'DWD &RPSDULVRQ RI 6LPXODWHG 4XHXH 6WDWLVWLFV 9HUVXV ([DFW 6ROXWLRQ 'HVFULSWLYHV IRU 4XHXH 6WDWLVWLFV E\ 1XPEHU RI 6WXGHQWV 'HVFULSWLYH 6WDWLVWLFV IRU WKH 7RWDO 6HUYLFH 7LPH 7LPH LQ 4XHXH 1XPEHU LQ 6\VWHP DQG 4XHXH /HQJWK E\ 3HUFHQWDJH &KDQJH LQ 5HVHDUFK DQG $SSURYDO 7LPH 'HVFULSWLYH 6WDWLVWLFV IRU 1XPEHU &RPSOHWHG 8WLOL]DWLRQ 3HUFHQWDJH DQG 'HOD\ 3HUFHQWDJH E\ 3HUFHQW &KDQJH LQ 5HVHDUFK DQG $SSURYDO 7LPH (IIHFWLYHQHVV RI 6HUYLFH &DSDFLW\ ,PSURYHPHQWV 8QGHU 1RUPDO $UULYDO 5DWHV 6HQVLWLYLW\ RI 2SWLPDO 6ROXWLRQ WR &KDQJHV LQ WKH $UULYDO 5DWH [

PAGE 11

/,67 2) ),*85(6 )LJXUH 3DJH +\SRWKHVL]HG 5HODWLRQVKLSV +\SRWKHVL]HG )UDPHZRUN 6FUHH 3ORW RI 3UHWHVW 'DWD b &RQILGHQFH ,QWHUYDOV RI 6HUYLFH 7LPH E\ 4XHVWLRQ 7\SH ,Q 0LQXWHVf b &RQILGHQFH ,QWHUYDOV RI 6HUYLFH 7LPH 8VLQJ 7KUHH &RPELQHG 4XHVWLRQ 7\SHV LQ 0LQXWHVf b &RQILGHQFH ,QWHUYDOV RI 6HUYLFH 7LPH E\ 5HVSRQVH 7\SH LQ 0LQXWHVf b &RQILGHQFH ,QWHUYDOV RI $YHUDJH 'DLO\ $UULYDOV E\ 0RQWK b &RQILGHQFH ,QWHUYDOV RI ,QWHUDUULYDO 7LPHV LQ 0LQXWHV E\ +RXU RI 'D\ )UHTXHQF\ +LVWRJUDP RI +LVWRULFDO ,QWHUDUULYDO 7LPHV )UHTXHQF\ +LVWRJUDP RI +LVWRULFDO 7RWDO 6HUYLFH 7LPHV )UHTXHQF\ +LVWRJUDP RI 6HUYLFH 7LPHV IRU 4XHVWLRQ *URXS 2QH )UHTXHQF\ +LVWRJUDP RI 6HUYLFH 7LPHV IRU 4XHVWLRQ *URXS 7ZR )UHTXHQF\ +LVWRJUDP RI 6HUYLFH 7LPHV IRU 4XHVWLRQ *URXS 7KUHH 6FUHH 3ORW RI 0DLQ 4XHVWLRQQDLUH 5HVSRQVHV 5HJUHVVLRQ (TXDWLRQ 3ORW RI 364 DQG 3UHGLFWHG 364 E\ $FWXDO 6HUYLFH 7LPH ZKHUH 364 r7RWDO 6HUYLFH 7LPHf 5HVLGXDO 3ORW RI 364 RQ 7RWDO 6HUYLFH 7LPH 5HVLGXDO 3ORW RI 364 RQ 7RWDO 6HUYLFH 7LPHV 2FFXUULQJ ZLWKLQ 2QH 'D\ [L

PAGE 12

([SHFWHG 1XPEHU LQ 6\VWHP IRU 6L[ 5HSOLFDWLRQV RI 'D\V 2EVHUYHG 9HUVXV 6LPXODWHG 3UREDELOLW\ 'HQVLW\ )XQFWLRQV 3')f 2EVHUYHG 9HUVXV 6LPXODWHG &XPXODWLYH 'HQVLW\ )XQFWLRQV &')f b &RQILGHQFH ,QWHUYDOV IRU 'HOD\ 3HUFHQWDJH E\ 6HUYLFH 5DWH 0RGLILHU DQG 1XPEHU RI 6HUYHUV b &RQILGHQFH ,QWHUYDOV IRU 7RWDO 6HUYLFH 7LPH LQ 0LQXWHVf E\ 6HUYLFH 5DWH 0RGLILHU DQG 1XPEHU RI 6HUYHUV [LL

PAGE 13

$EVWUDFW RI 'LVVHUWDWLRQ 3UHVHQWHG WR WKH *UDGXDWH 6FKRRO RI WKH 8QLYHUVLW\ RI )ORULGD LQ 3DUWLDO )XOILOOPHQW RI WKH 5HTXLUHPHQWV IRU WKH 'HJUHH RI 'RFWRU RI 3KLORVRSK\ 237,0,=,1* 6(59,&( &$3$&,7< ,1 7+( '58* ,1)250$7,21 6(59,&( %\ 'DQLHO /HH +DOEHUJ 0D\ &KDLUPDQ 3URIHVVRU &KDUOHV +HSOHU 0DMRU 'HSDUWPHQW 3KDUPDF\ +HDOWK &DUH $GPLQLVWUDWLRQ +HDOWK VHUYLFHV DUH RIWHQ GHYHORSHG ZLWKRXW DSSURSULDWHO\ DQDO\]LQJ WKH V\VWHPfV DELOLW\ WR PHHW WKH QHHGV RI WKH FRQVXPHU DQG DWWHPSWV WR LPSURYH TXDOLW\ DQG HIILFLHQF\ RIWHQ GR QRW VXFFHHG EHFDXVH RI WKH FRPSOH[LW\ DQG G\QDPLF QDWXUH RI VHUYLFHV +RZHYHU VRPH RUJDQL]DWLRQV DUH XVLQJ VRSKLVWLFDWHG WHFKQLTXHV VXFK DV VLPXODWLRQ WR DQDO\]H VHUYLFH V\VWHPV 7KLV VWXG\ KDG WKUHH SULPDU\ REMHFWLYHV f WR GHYHORS D FRPSXWHU VLPXODWLRQ PRGHO IRU D GUXJ LQIRUPDWLRQ VHUYLFH f WR LQYHVWLJDWH WKH DVVRFLDWLRQV DPRQJ DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH TXDOLW\ DQG f WR UHFRPPHQG V\VWHP LPSURYHPHQWV EDVHG RQ WKH VLPXODWLRQ 7KLV VWXG\ XVHG ERWK H[SHULPHQWDO DQG QRQH[SHULPHQWDO PHWKRGV ,W ZDV FRQGXFWHG DW WKH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVRXUFH &HQWHU ',35&f DW 6KDQGV DW WKH 8QLYHUVLW\ RI )ORULGD 2YHUDOO VHYHQ K\SRWKHVHV DQG WKUHH VSHFLILF UHVHDUFK TXHVWLRQV ZHUH XVHG WR H[SORUH UHODWLRQVKLSV DPRQJ WKH VWXG\ YDULDEOHV 6L[ GDWD VRXUFHV ZHUH XVHG f KLVWRULFDO GDWD VKHHWV f KLVWRULFDO ZRUNORDG GDWD f GDWD FROOHFWLRQ IRUPV f SHUVRQDO LQWHUYLHZV f VHUYLFH TXDOLW\ TXHVWLRQQDLUHV DQG f VLPXODWLRQ UXQV

PAGE 14

7KH ILUVW WKUHH K\SRWKHVHV WHVWHG WKH UHODWLRQVKLSV DPRQJ SHUFHLYHG VHUYLFH TXDOLW\ 364f RYHUDOO VHUYLFH TXDOLW\ 264f DQG WZR PHDVXUHV RI EHKDYLRUDO LQWHQWLRQ $ VWURQJ SRVLWLYH UHODWLRQVKLS ZDV IRXQG EHWZHHQ 364 DQG 264 ,Q DGGLWLRQ UHODWLRQVKLSV ZHUH IRXQG EHWZHHQ 364 DQG EHKDYLRUDO LQWHQWLRQ DQG EHWZHHQ 264 DQG EHKDYLRUDO LQWHQWLRQ 7KH UHPDLQLQJ IRXU K\SRWKHVHV WHVWHG WKH UHODWLRQVKLSV DPRQJ 364 DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH WLPH ,W ZDV IRXQG WKDW RQO\ VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH WLPH ZHUH VLJQLILFDQWO\ UHODWHG WR 364 +RZHYHU SHUFHLYHG VHUYLFH WLPH VHHPHG PRUH LPSRUWDQW WKDQ VHUYLFH GHOD\V ZLWK UHJDUG WR 364 1R UHODWLRQVKLS ZDV IRXQG EHWZHHQ DFWXDO VHUYLFH WLPH DQG 364 6XUSULVLQJO\ QR SUDFWLFDOO\ VLJQLILFDQW UHODWLRQVKLSV ZHUH IRXQG DPRQJ DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH WLPH $ VLPXODWLRQ PRGHO ZDV FRQVWUXFWHG XVLQJ *366+ *HQHUDO 3XUSRVH 6LPXODWLRQ 6\VWHPf 7KH VLPXODWLRQ ZDV YDOLGDWHG DQG IRXQG WR EH D FUHGLEOH PRGHO IRU DQDO\]LQJ WKH VHUYLFH V\VWHP DW WKH ',35& ([SORUDWLRQ RI WKH WKUHH VSHFLILF UHVHDUFK TXHVWLRQV LQGLFDWHG WKDW LPSURYLQJ VHUYLFH WLPHV ZDV PRUH HIILFLHQW WKDQ VWDIILQJ LQFUHDVHV IRU WKH SXUSRVH RI UHGXFLQJ WKH SHUFHQWDJH RI VHUYLFH GHOD\V [LY

PAGE 15

&+$37(5 ,1752'8&7,21 ,QWURGXFWLRQ +HDOWK VHUYLFHV DUH RIWHQ GHYHORSHG ZLWKRXW FDUHIXO FRQVLGHUDWLRQ RI WKH DFWXDO QHHGV RI WKH FRQVXPHU RU ZLWKRXW DSSURSULDWHO\ DQDO\]LQJ WKH VHUYLFH V\VWHPfV DELOLW\ WR PHHW WKHVH QHHGV 6KRVWDFN f 6HUYLFHV WKDW IDOO VKRUW RI PHHWLQJ FRQVXPHU QHHGV PXVW EH PRGLILHG RU UHGHVLJQHG LQ RUGHU WR LPSURYH WKH TXDOLW\ RI WKH VHUYLFH 7ZR SRSXODU WUDQVIRUPDWLRQ SDUDGLJPV WKDW DUH RIWHQ XVHG WR H[DPLQH WKH TXDOLW\ SUREOHPV UHODWHG WR VHUYLFH SURFHVVHV DUH WRWDO TXDOLW\ LPSURYHPHQW 740f DQG EXVLQHVV SURFHVV UHn HQJLQHHULQJ %35f 7KH 740 SDUDGLJP KDV JDLQHG FRQVLGHUDEOH VXSSRUW IURP WKH KHDOWKFDUH FRPPXQLW\ DV D WUDQVIRUPDWLRQ SKLORVRSK\ %RHUVWOHU HW DK f 740 ZDV SLRQHHUHG E\ : (GZDUGV 'HPLQJ LQ WKH V DQG IRFXVHV RQ WKH FRQFHSW RI fNDL]HQ f D -DSDQHVH ZRUG PHDQLQJ WKH FRQWLQXRXV LQFUHPHQWDO LPSURYHPHQW RI DQ H[LVWLQJ SURFHVV +DPPHU DQG &KDPS\ f ,Q HVVHQFH 740 LV DQ RUJDQL]DWLRQ ZLGH FRPPLWPHQW WR VWHDGLO\ DQG FRQWLQXRXVO\ LPSURYH TXDOLW\ RI WKH V\VWHP 6FKPHOH f %35 LV FXUUHQWO\ RI FRQVLGHUDEOH LQWHUHVW LQ PDQ\ VHUYLFH HQYLURQPHQWV LQFOXGLQJ KHDOWK FDUH 0DQ\ LQGLYLGXDOV FRQIXVH WKH FRQFHSWV RI %35 ZLWK 740 5HHQJLQHHULQJ KDV EHHQ GHILQHG DV fWKH IXQGDPHQWDO UHWKLQNLQJ DQG UDGLFDO UHGHVLJQ RI EXVLQHVV SURFHVVHV WR DFKLHYH GUDPDWLF LPSURYHPHQWV LQ FULWLFDO FRQWHPSRUDU\ PHDVXUHV RI SHUIRUPDQFH VXFK DV FRVW TXDOLW\ VHUYLFH DQG VSHHGf +DPPHU DQG &KDPS\ Sf 7KXV UHHQJLQHHULQJ LV QRW IXQGDPHQWDOO\ DERXW SURFHVV LPSURYHPHQW EXW SURFHVV UHLQYHQWLRQ +DPPHU DQG &KDPS\ f

PAGE 16

+RZHYHU GLIIHUHQW WKH DSSURDFKHV WKHVH WZR SDUDGLJPV VKDUH D FRPPRQ SULPDU\ IRFXV RQ FRQVXPHU QHHGV DQG WKH RXWFRPHV RI D SURFHVV 2IWHQ WKH JRDO LV WR DFKLHYH D GHVLUHG RXWFRPH L H SURGXFWLRQ RI D SURGXFW RU VHUYLFHf ZKLOH PHHWLQJ VHYHUDO REMHFWLYHV LQFOXGLQJ LPSURYHG SURGXFWLYLW\ LPSURYHG TXDOLW\ UHGXFHG WRWDO SURFHVV WLPH LQFUHDVHG WKURXJKSXW DQG UHGXFHG ZDLWLQJ WLPHV +DPPHU DQG &KDPS\ 7XPD\ f 8QIRUWXQDWHO\ LW LV FOHDU WKDW D ODUJH SHUFHQWDJH RI %35 DQG 740 HIIRUWV IDLO WR GHOLYHU DQ\ RI WKH SURPLVHG EHQHILWV %RHUVWOHU HW DO +DPPHU DQG &KDPS\ *HLVOHU .LHO\ .RWWHU 5XVW HW DO f $OWKRXJK WKHUH KDYH EHHQ PDQ\ UHDVRQV JLYHQ IRU WKHVH IDLOXUHV SDUW RI WKH WURXEOH LQKHUHQWO\ UHVWV LQ WKH GLIILFXOW\ RI XQGHUVWDQGLQJ WKH FRPSOH[ DQG G\QDPLF LQWHUGHSHQGHQFLHV RI VHUYLFH V\VWHPV %HFDXVH RI WKLV VRPH UHFHQW WUDQVIRUPDWLRQ SURMHFWV KDYH XVHG VRSKLVWLFDWHG WHFKQLTXHV VXFK DV FRPSXWHU VLPXODWLRQ LQ RUGHU WR DQDO\]H WKH UHODWLRQVKLSV DPRQJ WKH YDULRXV FRPSRQHQWV RI VHUYLFH V\VWHPV DQG WKH HIIHFWV RI LPSOHPHQWLQJ FKDQJHV LQ WKH V\VWHP 7XPD\ f 7KLV VWXG\ KDG WKUHH SULPDU\ REMHFWLYHV f WR GHYHORS D FRPSXWHU VLPXODWLRQ PRGHO IRU D GUXJ LQIRUPDWLRQ VHUYLFH DQG WR YDOLGDWH WKH PRGHO DJDLQVW WKH H[LVWLQJ V\VWHP f WR LQYHVWLJDWH WKH DVVRFLDWLRQV DPRQJ DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V DQG HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ LQ D GUXJ LQIRUPDWLRQ VHUYLFH VHWWLQJ DQG f WR UHFRPPHQG V\VWHP LPSURYHPHQWV EDVHG RQ WKH VLPXODWLRQ PRGHO LQ SDUWLFXODU WKRVH LPSURYHPHQWV WKDW UHGXFH WKH WLPH UHTXLUHG WR UHVSRQG WR FRQVXPHU TXHVWLRQV DQG LQIRUPDWLRQ UHTXHVWV 7KH IROORZLQJ VHFWLRQV RI WKLV FKDSWHU ZLOO SURYLGH EDFNJURXQG LQIRUPDWLRQ GHVFULELQJ WKH XQGHUO\LQJ FRQFHSWV XVHG LQ WKH UHVHDUFK DQG ZLOO LQWURGXFH WKH SUREOHP VWDWHPHQW IRU WKH SURSRVHG VWXG\ 7KH EDFNJURXQG ZLOO VSHFLILFDOO\ DGGUHVV Df KRZ TXHXLQJ WKHRU\ FDQ EH XVHG DV D EDVLV IRU HVWDEOLVKLQJ RSWLPDO OHYHOV RI VHUYLFH FDSDFLW\ Ef KRZ VHUYLFH FDSDFLW\ FDQ DIIHFW FRVWV ZDLWLQJ WLPHV DQG SHUFHLYHG VHUYLFH TXDOLW\ DQG Ff

PAGE 17

WKH PHDVXUHPHQW RI SHUFHLYHG VHUYLFH TXDOLW\ (DFK RI WKHVH LVVXHV KRZHYHU ZLOO EH GLVFXVVHG PRUH FULWLFDOO\ LQ WKH QH[W FKDSWHU %DFNJURXQG 4XHXLQJ 7KHRU\ DQG WKH 6LPXODWLRQ RI D 6HUYLFH 6\VWHP 7KH JRDO RI PRVW TXHXLQJ WKHRU\ DQG VLPXODWLRQ EDVHG PRGHOV LV WR XQGHUVWDQG WKH EHKDYLRU RI D SDUWLFXODU V\VWHP DQG WR PDNH GHFLVLRQV UHJDUGLQJ WKH V\VWHP EDVHG RQ WKH EHKDYLRU RI WKH PRGHO 6HLOD f 0DQ\ UHDOZRUOG V\VWHPV WKDW LQYROYH UDQGRP DUULYDO DQG VHUYLFH UDWHV FDQ EH H[DPLQHG E\ VWUXFWXULQJ WKHP DV TXHXLQJ SUREOHPV (VVHQWLDOO\ WKHVH SUREOHPV FDQ EH HYDOXDWHG LQ WZR ZD\V WKURXJK HLWKHU D FORVHG IRUP VROXWLRQ RU DQ RSHQ IRUP VROXWLRQ 4XHXLQJ WKHRU\ XVHV FORVHG IRUP PDWKHPDWLFDO UHODWLRQVKLSV WR DFKLHYH H[DFW DQVZHUV WR ZDLWLQJWLPH DQG ZDLWLQJOLQH SUREOHPV 6LPXODWLRQV RI TXHXLQJ V\VWHPV DUH RSHQ IRUP FRPSXWDWLRQDOO\ G\QDPLF PRGHOV WKDW GHVFULEH WKH EHKDYLRU RI D V\VWHP ZLWK UHVSHFW WR WLPH 2SHQ IRUP VROXWLRQV DUH XVHG ZKHQ WKHUH DUH QR NQRZQ HTXDWLRQV IRU WKH RSHUDWLQJ FKDUDFWHULVWLFV VXFK DV ZDLWLQJ WLPH LQ WKH TXHXH IRU WKH V\VWHP RI LQWHUHVW :KHQ DYDLODEOH FORVHG IRUP VROXWLRQV DUH XVXDOO\ SUHIHUUHG WR RSHQ IRUP VROXWLRQV EHFDXVH RI WKHLU H[DFWQHVV DQG WKHRUHWLFDO SRZHU +RZHYHU VLPXODWLRQV DUH XVHG WR VWXG\ ZDLWLQJ VLWXDWLRQV ZKHQ FORVHG IRUP VROXWLRQV DUH WRR FRPSOH[ RU DUH QRW DYDLODEOH )RU H[DPSOH VLPXODWLRQV DUH XVHG ZKHQ WKH SURSHUWLHV RI WKH V\VWHP WR EH PRGHOHG YLRODWH WKH XQGHUO\LQJ DVVXPSWLRQV RI TXHXLQJ WKHRU\ RU ZKHQ WKH UHVHDUFKHU GHVLUHV PRUH LQIRUPDWLRQ WKDQ WKH TXHXLQJ WKHRU\ DSSUR[LPDWLRQ SURYLGHV .UDMHZVNL DQG 5LW]PDQ 0DU\DQVNL f

PAGE 18

6HUYLFH &DSDFLW\ DQG LWV 5HODWLRQVKLS WR 3HUFHLYHG 6HUYLFH 4XDOLW\ 6HUYLFH FDSDFLW\ FDQ EH GHILQHG XVLQJ WKH TXHXLQJ WKHRU\ IUDPHZRUN DV D IXQFWLRQ RI WKH VWDIILQJ OHYHO f DQG WKH VHUYLFH UDWH Sf 6WDIILQJ OHYHO UHIHUV WR WKH QXPEHU RI HPSOR\HHV DYDLODEOH WR VHUYH FRQVXPHUV 7KH VHUYLFH UDWH UHIHUV WR D YDULDEOH DPRXQW RI WLPH UHTXLUHG E\ DQ HPSOR\HH WR FRPSOHWH D VHUYLFH UHTXHVW JLYHQ WKHLU SHUVRQDO DELOLW\ WKHLU HTXLSPHQW DQG WKH RUJDQL]DWLRQ RI WKH ZRUN .UDMHZVNL DQG 5LW]PDQ /RYHORFN :LQVWRQ f :KHQ WKHUH LV D VKRUWDJH RI FDSDFLW\ UHODWLYH WR GHPDQG /f TXHXHV IRUP DQG WRWDO WLPH LQ WKH V\VWHP LQFUHDVHV /RYHORFN f ,W KDV EHHQ GHPRQVWUDWHG LQ WKH OLWHUDWXUH WKDW DV ZDLWLQJ WLPH LQFUHDVHV HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ DQG FXVWRPHU VDWLVIDFWLRQ DUH QHJDWLYHO\ DIIHFWHG %ROWRQ DQG 'UHZ &OHPPHU DQG 6FKQHLGHU 'DYLV DQG 9ROOPDQQ 'DYLV 'XEHf5LRX[ HW DO +XL DQG 7VH .DW] /DUVRQ DQG /DUVRQ 7D\ORU D 7RP DQG /XFH\ f 7ZR DSSURDFKHV KDYH EHHQ XVHG WR UHGXFH WKH DGYHUVH HIIHFWV RI ZDLWLQJ RQ FXVWRPHU VDWLVIDFWLRQ DQG SHUFHLYHG VHUYLFH TXDOLW\ LQ WKH OLWHUDWXUH SHUFHSWLRQV PDQDJHPHQW DQG RSHUDWLRQV PDQDJHPHQW 3HUFHSWLRQV PDQDJHPHQW LV DQ DSSURDFK WKDW DWWHPSWV WR UHGXFH WKH SHUFHLYHG ZDLW WLPHV RI WKH FRQVXPHUV RI D V\VWHP WKURXJK WKH FUHDWLYH XVH RI GLVWUDFWLRQV DSRORJLHV TXHXH LQIRUPDWLRQ HJ SODFH LQ OLQH DQG HVWLPDWHG ZDLW WLPHVf DQG WKH PDQLSXODWLRQ RI SHUFHLYHG SUHVHUYLFH DQG SRVWVHUYLFH ZDLWV $FFRUGLQJ WR UHSRUWV LQ WKH OLWHUDWXUH SHUFHSWLRQV PDQDJHPHQW KDV PHW ZLWK OLPLWHG VXFFHVV KRZHYHU LW LV VWLOO XQFOHDU DV WR KRZ VXFFHVVIXO WKHVH WHFKQLTXHV DUH DFURVV D YDULHW\ RI VHUYLFH VHWWLQJV &OHPPHU DQG 6FKQHLGHU D E DQG +XL DQG 7VH f 2SHUDWLRQV PDQDJHPHQW LV DQ DSSURDFK WKDW DWWHPSWV WR UHGXFH WKH DFWXDO ZDLW WLPHV RI WKH FRQVXPHUV RI D VHUYLFH V\VWHP XVLQJ VFKHGXOLQJ WHFKQLTXHV TXHXH PDQDJHPHQW DQG ZRUN IORZ FKDQJHV $W WKH KHDUW RI WKH FDSDFLW\ LVVXH LV WKH GHYHORSPHQW RI DSSURSULDWH TXHXLQJ V\VWHPV WKDW XWLOL]H FDSDFLW\ WR LWV EHVW DGYDQWDJH

PAGE 19

/RYHORFN f 7KLV UHVHDUFK IRFXVHG SULPDULO\ RQ WKH XVH RI RSHUDWLRQV PDQDJHPHQW WHFKQLTXHV VSHFLILFDOO\ TXHXLQJ WKHRU\ DQG VLPXODWLRQ WR PDQDJH VHUYLFH FDSDFLW\ 0HDVXUHPHQW RI 3HUFHLYHG 6HUYLFH 4XDOLW\ )RXU EDVLF FKDUDFWHULVWLFV DSSO\ WR PRVW VHUYLFHV f EHQHILWV UHFHLYHG IURP VHUYLFHV DUH ODUJHO\ LQWDQJLEOH f VHUYLFHV DUH DFWLYLW\ IRFXVHG UDWKHU WKDQ SURGXFW IRFXVHG f VHUYLFHV DUH VLPXOWDQHRXVO\ SURGXFHG DQG FRQVXPHG DQG f WKH FRQVXPHU SDUWLFLSDWHV LQ WKH SURGXFWLRQ SURFHVV *URQURRV f %HFDXVH RI WKHVH FKDUDFWHULVWLFV LW LV JHQHUDOO\ UHFRJQL]HG WKDW VHUYLFH TXDOLW\ LV KDUGHU WR HYDOXDWH WKDQ SURGXFW TXDOLW\ +HVNHWW 3DUDVXUDPDQ HW DO f 7KH FRQFHSW RI TXDOLW\ KRZHYHU LV GLIILFXOW WR GHILQH XQGHU DQ\ FLUFXPVWDQFH 1HYHUWKHOHVV ZH WKLQN RI TXDOLW\ LQ WHUPV RI WKH VXSHULRULW\ H[FHOOHQFH RU YDOXH RI D SURGXFW RU VHUYLFH &KULVWHQVHQ DQG 3HQQD :HVWJDUG DQG %DUU\ f 7ZR EDVLF PHWKRGV KDYH EHHQ XVHG WR DVVHVV TXDOLW\ LQ RUJDQL]DWLRQV 7KH ILUVW PHWKRG XVHV REMHFWLYH LQGLFDWRUV HJ DJH RI HTXLSPHQW QXPEHU RI GHIHFWV RU FRQVXPHU WLPH LQ WKH V\VWHPf DV PHDVXUHV RI TXDOLW\ 7KH VHFRQG PHWKRG XVHV VXEMHFWLYH LQGLFDWRUV HJ SHUFHLYHG VHUYLFH TXDOLW\ FXVWRPHU VDWLVIDFWLRQ RU HPSOR\HH VDWLVIDFWLRQf DV PHDVXUHV RI TXDOLW\ 6HUYLFH TXDOLW\ KDV EHHQ GHILQHG DV WKH UHODWLYH VXSHULRULW\ RI DQ RUJDQL]DWLRQ DQG WKH VHUYLFHV LW SURYLGHV 3DUDVXUDPDQ HW DO f 3HUFHLYHG VHUYLFH TXDOLW\ LV FRQFHUQHG ZLWK WKH PHDVXUHPHQW RI FRQVXPHU DWWLWXGHV UHJDUGLQJ DQ RUJDQL]DWLRQfV VHUYLFH TXDOLW\ 7KHUH KDV EHHQ VLJQLILFDQW GHEDWH LQ WKH OLWHUDWXUH FRQFHUQLQJ WKH GLPHQVLRQV DQG PHDVXUHPHQW RI SHUFHLYHG VHUYLFH TXDOLW\ &URQLQ DQG 7D\ORU *URQURRV 0F$OH[DQGHU 3DUDVXUDPDQ HW DO 3HWHU HW DO f $ SHUFHSWLRQVRQO\ VFDOH GHULYHG IURP WKH 6(593(5) VFDOH &URQLQ DQG 7D\ORU f ZLOO EH XVHG DV D PHDVXUH RI SHUFHLYHG VHUYLFH TXDOLW\ LQ WKLV UHVHDUFK SULPDULO\ EHFDXVH LW PRUH

PAGE 20

SUDFWLFDO WKDQ PDQ\ RI WKH DYDLODEOH LQVWUXPHQWV DQG EHFDXVH LW DYRLGV VRPH RI WKH PHDVXUHPHQW LVVXHV UHODWHG WR WKH XVH RI GLIIHUHQFH VFRUHV &URQLQ DQG 7D\ORU 3HWHU HW DO =HLWKDPO HW DO f 'UXJ ,QIRUPDWLRQ 6HUYLFHV 7KH ILUVW IRUPDOO\ UHFRJQL]HG GUXJ LQIRUPDWLRQ FHQWHU ',&f ZDV HVWDEOLVKHG DW WKH 8QLYHUVLW\ RI .HQWXFN\ LQ DQG E\ PRUH WKDQ RUJDQL]DWLRQV PDLQWDLQHG ',&V 3DUNHU 5RVHQEHUJ HW DO 9DQVFR\ HW DO f 7KH LQLWLDO UROH RI WKHVH FHQWHUV ZDV WR HYDOXDWH DQG FRPSDUH GUXJV DQG WR SURPRWH UDWLRQDO GUXJ WKHUDS\ KRZHYHU WKH UROH RI PDQ\ FHQWHUV KDV HYROYHG WR LQFOXGH HGXFDWLRQDO DFWLYLWLHV PHGLFDWLRQ SROLF\ GHYHORSPHQW DQG RXWFRPHV UHVHDUFK %HDLUG HW DO 9DQVFR\ HW DO f +HDOWK FDUH SURIHVVLRQDOV DUH FXUUHQWO\ FKDOOHQJHG E\ WKH QHFHVVLW\ WR NHHS XS ZLWK WKH ODWHVW GHYHORSPHQWV LQ QHZ GUXJV DQG DGYDQFHV LQ WKHUDSLHV 0DQ\ RI WKH ',&V KDYH FRPH LQWR H[LVWHQFH EHFDXVH RI WKH UHFRJQLWLRQ E\ PDQDJHPHQW WKDW LW LV QRW HIILFLHQW WR KDYH SUDFWLWLRQHUV UHYLHZ WKH OLWHUDWXUH DQG LGHQWLI\ VROXWLRQV WR DOO RI WKH GUXJ WKHUDS\ SUREOHPV WKH\ HQFRXQWHU $V VXFK ',&V ZHUH GHYHORSHG DV D FHQWUDO RUJDQL]HG DSSURDFK WR PHHWLQJ WKHVH QHHGV DQG WR KHOS GLVVHPLQDWH GUXJ LQIRUPDWLRQ WR WKH PHGLFDO DQG QXUVLQJ VWDII 6NRXWDNLV 6PLWK f ,Q WKH FXUUHQW KHDOWKFDUH HQYLURQPHQW KRZHYHU WKH RXWORRN IRU ',&V LV XQFHUWDLQ 6HYHUDO IDFWRUV DUH FXUUHQWO\ SODFLQJ LQFUHDVLQJ SUHVVXUH RQ ',&V WR SURYLGH LQFUHDVHG OHYHOV RI VHUYLFH )LUVW DGYDQFLQJ LQIRUPDWLRQ WHFKQRORJLHV DQG PDQDJHG FDUH LQIOXHQFHV DUH IRUFLQJ ',&V WR SURYLGH WKH KLJKHVW TXDOLW\ VHUYLFH ZLWK QHDU LQVWDQWDQHRXV DFFHVV WR LQIRUPDWLRQ 6HFRQG QRW RQO\ DUH ',&V UHVSRQVLEOH IRU GLVSHQVLQJ LQIRUPDWLRQ UHJDUGLQJ FOLQLFDO GHFLVLRQV WKH\ DUH DOVR EHLQJ DVNHG WR GRFXPHQW WKHLU LPSDFW RQ SDWLHQW FDUH XVLQJ RXWFRPHV PHDVXUHPHQW WRROV GHULYHG IURP GLVFLSOLQHV VXFK DV SKDUPDFRHSLGHPLRORJ\ DQG

PAGE 21

SKDUPDFRHFRQRPLFV 7KLUG PDQ\ ',&V DUH DOVR UHTXLUHG WR SDUWLFLSDWH LQ VFKRODUO\ UHVHDUFK DQG HGXFDWLRQDO DFWLYLWLHV 6NRXWDNLV 9DQVFR\ HW DO f :KLOH WKHUH LV D UHFRJQL]HG QHHG IRU GUXJ LQIRUPDWLRQ VHUYLFHV WKH FXUUHQW HUD RI FRVW FRQWDLQPHQW DQG RXWFRPHV PDQDJHPHQW SUHVHQWV D GLOHPPD IRU KHDOWK RUJDQL]DWLRQV KRVSLWDOV DQG XQLYHUVLWLHV ZKR DUH QRZ UHTXLUHG WR MXVWLI\ VXSSRUW IRU QRQSURILWDEOH SURJUDPV (YHQ WKRXJK ',&V DUH EHLQJ DVNHG WR SURYLGH PRUH VHUYLFHV FXWEDFNV LQ VWDIILQJ DUH QRW XQFRPPRQ ZKHQ IXQGV DYDLODEOH IRU VXFK SURJUDPV DUH UHGXFHG 0DLOKRW DQG *LDFRQD'DKO 6NRXWDNLV 9DQVFR\ HW DK f 7KXV WKH DELOLW\ RI ',&V WR HYDOXDWH WKHLU VHUYLFH LQ WHUPV RI HIIHFWLYHQHVV RU RXWFRPHV LV FULWLFDO LQ PDLQWDLQLQJ D ',& LQ WKH FXUUHQW KHDOWK FDUH HQYLURQPHQW $OWKRXJK WKHUH KDYH EHHQ D QXPEHU RI DUWLFOHV GRFXPHQWLQJ WKH DFWLYLWLHV RI GUXJ LQIRUPDWLRQ DQG WR[LFRORJ\ UHVRXUFHV YHU\ IHZ RI WKHVH DUWLFOHV KDYH DGGUHVVHG GUXJ LQIRUPDWLRQ TXDOLW\ RU HIIHFWLYHQHVV RI ',&V XQGHU UHVRXUFH FRQVWUDLQWV /LOMD 5RVHQEHUJ HW DK 6NRXWDNLV f $V -RKQ /LOMD VWDWHV fLW LV VDIH WR VD\ WKDW ZH NQRZ DVWRQLVKLQJO\ OLWWOH RQ KRZ WR RSWLPL]H UHVRXUFHV IRU GUXJ LQIRUPDWLRQ SURJUDPVf Sf 3UREOHP 6WDWHPHQW 'HWHUPLQLQJ VXIILFLHQW VHUYLFH FDSDFLW\ LQ D GUXJ LQIRUPDWLRQ VHUYLFH LV D FKDOOHQJLQJ LVVXH IRU PDQDJHUV WU\LQJ WR PDLQWDLQ DFFHSWDEOH OHYHOV RI VHUYLFH TXDOLW\ +RZHYHU TXHVWLRQV UHJDUGLQJ FDSDFLW\ RIWHQ LQYROYH GHFLVLRQV UHODWHG WR WKH DFFHSWDEOH DPRXQW RI WLPH UHTXLUHG WR GHOLYHU WKH VHUYLFH H J DQVZHU D GUXJWKHUDS\ TXHVWLRQf 8QIRUWXQDWHO\ WKH EHKDYLRU RI TXHXLQJ V\VWHPV LV GHFHSWLYHO\ FRPSOH[ DQG RIWHQ QRQ LQWXLWLYH ,I VWDIIHG DFFRUGLQJ WR fFRPPRQ VHQVHf DSSURDFKHV PDQ\ V\VWHPV DUH XQDEOH WR KDQGOH WKH ZRUNORDG 7KH PDWKHPDWLFV RI TXHXLQJ WKHRU\ VKRZ WKDW LQDGHTXDWH VHUYLFH FDSDFLW\ FDQ JUHDWO\ LQFUHDVH WKH ZDLWLQJ WLPH EHIRUH VHUYLFH LV FRPSOHWHG &RQVHTXHQWO\ D FRQVXPHUfV

PAGE 22

RYHUDOO WLPH LQ WKH V\VWHP ZLOO RIWHQ EH ORQJHU WKDQ LQWHQGHG &RQVXPHUV ZKR ZDLW ORQJ SHULRGV RI WLPH IRU VHUYLFH PD\ EH PRUH OLNHO\ WR GRZQJUDGH WKH TXDOLW\ RI WKH VHUYLFH HYHQ WKRXJK RWKHU DVSHFWV RI VHUYLFH SHUIRUPDQFH PD\ KDYH EHHQ GHOLYHUHG FRPSHWHQWO\ 7D\ORU D 7D\ORU DQG &OD[WRQ f 8QIRUWXQDWHO\ KRZHYHU VHUYLFH SURFHVVHV DUH RIWHQ WKH PRVW FRPSOH[ V\VWHPV WR XQGHUVWDQG EHFDXVH WKH\ IUHTXHQWO\ GHSHQG RQ WKH UDQGRP QDWXUH RI DUULYDO SURFHVVHV DQG VHUYLFH WLPHV DQG WKH G\QDPLF LQWHUGHSHQGHQFLHV RI V\VWHP EHKDYLRU 7XPD\ f 5HVHDUFK )UDPHZRUN 7KLV UHVHDUFK XVHG D TXHXLQJ SDUDGLJP WR HYDOXDWH WKH UHODWLRQVKLS EHWZHHQ VHUYLFH FDSDFLW\ LQ D GUXJ LQIRUPDWLRQ VHUYLFH DQG WKH OHQJWK RI WLPH WKDW D FRQVXPHU PXVW ZDLW WR REWDLQ D UHVSRQVH WR D TXHVWLRQ RU LQIRUPDWLRQ UHTXHVW ,Q DGGLWLRQ WKH UHODWLRQVKLS EHWZHHQ ZDLWLQJ WLPH DQG FRQVXPHU HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ ZDV GHWHUPLQHG ,W ZDV SURSRVHG WKDW D UHODWLRQVKLS H[LVWHG EHWZHHQ VHUYLFH FDSDFLW\ DQG VHUYLFH WLPH RU GHOD\V LQ VHUYLFH DQG EHWZHHQ VHUYLFH WLPH RU GHOD\V LQ VHUYLFH DQG HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ )LJXUH f 7KLV IUDPHZRUN LV GHYHORSHG LQ PRUH GHWDLO LQ FKDSWHU WKUHH 6HUYLFH 6HUYLFH 7LPH 3HUFHLYHG &DSDFLW\ DQG 'HOD\V 6HUYLFH TXDOLW\ )LJXUH +\SRWKHVL]HG 5HODWLRQVKLSV 6LJQLILFDQFH $OWKRXJK VLPXODWLRQ UHVHDUFK LQYROYLQJ VWDIILQJ SDWWHUQV KDV EHHQ FRQGXFWHG LQ D YDULHW\ RI KHDOWK UHODWHG V\VWHPV LQFOXGLQJ QXUVLQJ SV\FKLDWU\ DQG HPHUJHQF\

PAGE 23

GHSDUWPHQWV QR UHVHDUFK KDV EHHQ SXEOLVKHG GHVFULELQJ WKH V\VWHP EHKDYLRU RI GUXJ LQIRUPDWLRQ VHUYLFHV XVLQJ D TXHXLQJ SDUDGLJP ,Q DGGLWLRQ VRPH UHVHDUFK KDV EHHQ FRQGXFWHG H[DPLQLQJ WKH UHODWLRQVKLS EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ DQG VHUYLFH FDSDFLW\ KRZHYHU WKLV DUHD LV VWLOO LQ WKH UXGLPHQWDU\ VWDJHV RI GHYHORSPHQW 7KLV UHVHDUFK ZRXOG KHOS WR H[SDQG H[LVWLQJ NQRZOHGJH LQ WKHVH DUHDV )URP D PRUH SUDFWLFDO VWDQGSRLQW YDULRXV UHVHDUFKHUV KDYH SURSRVHG D SRVLWLYH UHODWLRQVKLS EHWZHHQ LPSURYHG VHUYLFH TXDOLW\ DQG LQFUHDVHG UHYHQXHV RU LPSURYHG SURGXFWLYLW\ *URQURRV f +RZHYHU WKLV EHQHILW PD\ EH OHVV XVHIXO WR fIUHHf VHUYLFH SURJUDPV WKDW DUH IXQGHG XQGHU IL[HG RU VKULQNLQJ EXGJHWV ,Q WKLV HQYLURQPHQW LW LV PRUH YDOXDEOH WR VXJJHVW WZR SRVVLEOH LPSDFWV WKDW VHUYLFH FDSDFLW\ GHFLVLRQV FDQ KDYH RQ LQWHQWLRQV DQG XWLOL]DWLRQ )LUVW LI VHUYLFH TXDOLW\ LV SHUFHLYHG DV LQIHULRU E\ FRQVXPHUV WKHQ WKH\ PD\ EH OHVV OLNHO\ WR UHO\ RQ WKH VHUYLFH 7KXV LQDGHTXDWH VHUYLFH FDSDFLW\ PD\ UHGXFH WKH SHUFHLYHG YDOXH RI WKH VHUYLFH ,I XWLOL]DWLRQ GHFUHDVHV WKHQ LW EHFRPHV PRUH GLIILFXOW WR MXVWLI\ WKH VHUYLFHfV H[LVWHQFH 7KH VHUYLFH PD\ EH VXVSHQGHG EHFDXVH FRQVXPHUV IHHO WKH TXDOLW\ RI D VHUYLFH LV SRRU DQG KDYH GHFLGHG QRW WR XVH LW UHJDUGOHVV RI WKHLU DFWXDO QHHG 6HFRQG EXGJHWDU\ FRQVWUDLQWV RIWHQ PDNH LW GLIILFXOW WR MXVWLI\ LQFUHDVHV LQ VWDIILQJ RU RWKHU LPSURYHPHQWV LQ VHUYLFH FDSDFLW\ ZLWKRXW VLJQLILFDQW HYLGHQFH RI QHHG %\ XVLQJ WKH LQIRUPDWLRQ REWDLQHG IURP VLPXODWLRQ PRGHOV DQG SHUFHLYHG VHUYLFH TXDOLW\ VXUYH\V LW PD\ EH HDVLHU WR GHPRQVWUDWH FXUUHQW VKRUWIDOOV LQ TXDOLW\ FXUUHQW DQG SURMHFWHG GHPDQG OHYHOV DQG WKH SRVLWLYH DQG QHJDWLYH FRQVHTXHQFHV RI FKDQJHV LQ VHUYLFH FDSDFLW\ 5HVHDUFK 4XHVWLRQV 7KHUH DUH IRXU TXHVWLRQV WKDW WKLV UHVHDUFK ZLOO DWWHPSW WR DGGUHVV &DQ LQIRUPDWLRQ UHJDUGLQJ VWDIILQJ OHYHOV VHUYLFH UDWHV FDOO DUULYDO SDWWHUQV DQG V\VWHP VWUXFWXUH EH XVHG WR EXLOG D VLPXODWLRQ PRGHO WKDW LV D UHOLDEOH DQG YDOLG

PAGE 24

VXEVWLWXWH IRU WKH DFWXDO GUXJ LQIRUPDWLRQ VHUYLFH IRU WKH SXUSRVH RI FDSDFLW\ SODQQLQJ" +RZ GR VWDIILQJ OHYHOV DUULYDO DQG VHUYLFH UDWHV DQG V\VWHP VWUXFWXUH DIIHFW WKH LPSRUWDQW SHUIRUPDQFH FKDUDFWHULVWLFV RI WKH V\VWHP" $GDSWHG IURP .UDMHZVNL DQG 5LW]PDQ ff $ 4XHXH /HQJWK 7KLV LV WKH H[SHFWHG QXPEHU RI FRQVXPHU LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV LQ WKH V\VWHP DW D JLYHQ SRLQW LQ WLPH % 6HUYLFH WLPH 7KLV LV WKH H[SHFWHG WRWDO VHUYLFH WLPH UHTXLUHG WR GHOLYHU D UHVSRQVH WR D TXHVWLRQ RU LQIRUPDWLRQ UHTXHVW 7KLV LV PHDVXUHG IURP WKH DUULYDO RI D TXHVWLRQ RU UHTXHVW IRU LQIRUPDWLRQ LQWR WKH LQIRUPDWLRQ VHUYLFH XQWLO WKH GHOLYHU\ RI D UHVSRQVH & 8WLOL]DWLRQ 5DWH 7KLV LV WKH FROOHFWLYH XWLOL]DWLRQ RI WKH VHUYLFH IDFLOLWLHV UHIOHFWV WKH SHUFHQWDJH RI WLPH WKH VHUYLFH SHUVRQQHO DUH EXV\ DV RSSRVHG WR WKH WLPH WKH\ DUH LGOHf 7KLV LV GHVFULEHG DV D UDWLR RI WKH DPRXQW RI WLPH WKH VHUYHU ZDV EXV\ RYHU WKH WRWDO WLPH PHDVXUHG +RZ GRHV WRWDO VHUYLFH WLPH UHODWH WR FRQVXPHUVf SHUFHLYHG VHUYLFH TXDOLW\" +RZ GR GHOD\V LQ VHUYLFH UHODWH WR FRQVXPHUVf SHUFHLYHG VHUYLFH TXDOLW\" &RXOG WKHVH UHODWLRQVKLSV EH XVHG LQ WKH VLPXODWLRQ PRGHO WR UHIOHFW WKH LPSDFW RI FKDQJHV LQ VHUYLFH FDSDFLW\ RQ FRQVXPHUVf SHUFHLYHG VHUYLFH TXDOLW\" :KDW DUH WKH FULWLFDO YDULDEOHV WKDW DIIHFW WKH VLPXODWLRQ PRGHO" %DVHG RQ WKHVH YDULDEOHV ZKDW PDQDJHPHQW UXOHV FDQ EH UHFRPPHQGHG WR LPSURYH VHUYLFH FDSDFLW\ DQG UHGXFH WKH UHVSRQVH WLPH RI WKH V\VWHP"

PAGE 25

&+$37(5 /,7(5$785( 5(9,(: 2YHUYLHZ &KDSWHU RQH LQWURGXFHG IRXU LPSRUWDQW FRQFHSWV DQG WKHLU UHODWLRQVKLS WR WKLV UHVHDUFK )LUVW WKH FDSDFLW\ RI D VHUYLFH V\VWHP FDQ EH GHVFULEHG LQ WHUPV RI D TXHXLQJ SDUDGLJP 6HFRQG FRPSOH[ TXHXLQJ V\VWHPV FDQ EH PRGHOHG XVLQJ FRPSXWHU VLPXODWLRQ 7KLUG ZDLWLQJ WLPHV FDQ EH LQIOXHQFHG E\ FKDQJLQJ WKH VHUYLFH FDSDFLW\ )RXUWK LW ZDV SURSRVHG WKDW WKHUH LV DQ LQYHUVH UHODWLRQVKLS EHWZHHQ ZDLWLQJ WLPH DQG HYDOXDWLRQV RI VHUYLFH TXDOLW\ 7KLV FKDSWHU FULWLFDOO\ UHYLHZV WKH SUHYLRXV UHVHDUFK UHODWHG WR WKH FRQFHSWV PHQWLRQHG DERYH DQG WKHLU DSSOLFDWLRQ WR WKLV SURMHFW ,W ZLOO EHJLQ E\ SUHVHQWLQJ DQ RYHUYLHZ RI TXHXLQJ WKHRU\ 6HFRQG PHWKRGV IRU LPSURYLQJ WKH SHUIRUPDQFH LH UHGXFLQJ ZDLWLQJ WLPHVf RI TXHXLQJ V\VWHPV ZLOO EH GHVFULEHG 7KLUG HYLGHQFH GHPRQVWUDWLQJ WKH UHODWLRQVKLS EHWZHHQ ZDLW WLPH DQG FRQVXPHUVn HYDOXDWLRQV RI VHUYLFHV LQ WHUPV RI VHUYLFH TXDOLW\ DQG VDWLVIDFWLRQf ZLOO EH UHYLHZHG ,W ZLOO WKHQ GLVFXVV KRZ PDQDJHUV RIWHQ XQGHUHVWLPDWH DFFHSWDEOH ZDLW WLPHV DQG KRZ FRQVXPHUVn RYHUHVWLPDWH WKH WLPH ZDLWHG )RXUWK WKH FKDSWHU ZLOO H[DPLQH WKH FRQFHSWXDO EDVLV RI SHUFHLYHG VHUYLFH TXDOLW\ DQG GHVFULEH ZK\ WKLV VWXG\ ZLOO XVH D SHUFHSWLRQVRQO\ LQVWUXPHQW WR PHDVXUH SHUFHLYHG VHUYLFH TXDOLW\ 7KH FKDSWHU ZLOO HQG ZLWK D GHVFULSWLRQ RI WKH SUDFWLFDO UHODWLRQVKLS EHWZHHQ VHUYLFH TXDOLW\ DQG EHKDYLRUDO LQWHQWLRQ ,Q FKDSWHU WKUHH WKH UHVHDUFK IUDPHZRUN ZLOO EH SUHVHQWHG DQG WKH UHVHDUFK K\SRWKHVLV IRU WKLV SURMHFW ZLOO EH GHYHORSHG

PAGE 26

6HUYLFH &DSDFLW\ DQG :DLW 7LPHV 2YHUYLHZ RI 4XHXLQJ 7KHRU\ 7KHUH DUH WKUHH SULPDU\ FRPSRQHQWV RI D TXHXLQJ SUREOHP 7KH ILUVW LV WKH LQSXW VRXUFH GHILQHG DV WKH SRSXODWLRQ RI SRWHQWLDO HQWUDQWV LQWR WKH VHUYLFH V\VWHP :H FDQ GHVFULEH WKLV SRSXODWLRQ LQ WHUPV RI LWV VL]H WKH QDWXUH RU XUJHQF\ RI QHHG DQG WKH DUULYDO GLVWULEXWLRQ 7KH VL]H RI WKH LQSXW VRXUFH PD\ EH LQILQLWH RU ILQLWH GHSHQGLQJ RQ ZKHWKHU WKH QXPEHU RI FXVWRPHUV LQ WKH V\VWHP VLJQLILFDQWO\ DIIHFWV WKH DUULYDO UDWH 7KH QDWXUH RU XUJHQF\ RI QHHG LQIOXHQFHV WKH UHODWLRQVKLS EHWZHHQ ZDLWLQJ WLPH RU TXHXH OHQJWK DQG UHQHJLQJ RU EDONLQJ LH OHDYLQJ ZLWKRXW VHUYLFHf .UDMHZVNL DQG 5LW]PDQ :LQVWRQ f 7KH DUULYDO GLVWULEXWLRQ LV D SUREDELOLW\ GLVWULEXWLRQ WKDW GHVFULEHV HLWKHU WKH QXPEHU RI DUULYDOV SHU XQLW WLPH RU WKH WLPH EHWZHHQ DUULYDOV LH WKH LQWHUDUULYDO WLPHf .UDMHZVNL DQG 5LW]PDQ :LQVWRQ f ,Q HIIHFW PDQ\ PDQDJHUV UHVSRQVLEOH IRU DVVHVVLQJ VHUYLFH FDSDFLW\ LQDSSURSULDWHO\ DVVXPH VRPH FRQVWDQW RU QDUURZO\ GHILQHG DUULYDO SDWWHUQ 7KHUH DUH KRZHYHU RWKHU SUREDELOLW\ GLVWULEXWLRQV WKDW FDQ GHVFULEH FXVWRPHU DUULYDO VWUHDPV EHWWHU WKDQ D FRQVWDQW )RU LQVWDQFH DUULYDOV SHU XQLW WLPH DUH VRPHWLPHV DVVXPHG WR EH 3RLVVRQ GLVWULEXWHG DQG LQWHUDUULYDO LQWHUYDOV DUH RIWHQ DSSUR[LPDWHG E\ VRPH IRUP RI WKH JDPPD GLVWULEXWLRQ XVXDOO\ WKH H[SRQHQWLDO .UDMHZVNL DQG 5LW]PDQ :LQVWRQ f 7KH VHFRQG FRPSRQHQW RI D TXHXLQJ V\VWHP LV WKH VHUYLFH SURFHVV :H GHVFULEH WKH VHUYLFH SURFHVV LQ WHUPV RI WKH VHUYLFH DUUDQJHPHQW DQG VHUYLFH UDWH GLVWULEXWLRQ 7KH VHUYLFH DUUDQJHPHQW FRPSULVHV WKH RUJDQL]DWLRQ RI WKH VHUYLFH LH ZRUNIORZf WKH QXPEHU RI VHUYHUV DYDLODEOH WR KDQGOH WKH DUULYDOV DQG WKH QXPEHU RI OLQHV OHDGLQJ WR WKRVH VHUYHUV 7KH VHUYLFH UDWH GLVWULEXWLRQ LV D SUREDELOLW\ GLVWULEXWLRQ JRYHUQLQJ WKH DPRXQW RI WKH WLPH D VHUYHU WDNHV WR VHUYLFH D FXVWRPHU 2IWHQ WKH VHUYLFH UDWH LV GHVFULEHG E\ WKH

PAGE 27

H[SRQHQWLDO :HLEXOO RU (UODQJ GLVWULEXWLRQV .UDMHZVNL DQG 5LW]PDQ /DZ DQG .HOWRQ :LQVWRQ f 7KH WKLUG FRPSRQHQW RI D TXHXLQJ SUREOHP LQYROYHV WKH TXHXH GLVFLSOLQH DOVR VRPHWLPHV FDOOHG WKH SULRULW\ GLVFLSOLQH 7KH TXHXH GLVFLSOLQH UHIHUV WR WKH RUGHU LQ ZKLFK FXVWRPHUV DUH SURFHVVHG WKURXJK WKH TXHXH 7KHUH DUH VHYHUDO FRPPRQ TXHXH GLVFLSOLQHV LQFOXGLQJ ),)2 RU )&)6 ILUVW FRPH ILUVW VHUYHGf /,)2 RU /&)6 ODVW FRPH ILUVW VHUYHGf 637) VKRUWHVW SURFHVVLQJ WLPH ILUVWf DQG /37) ORQJHVW SURFHVVLQJ WLPH ILUVWf 7KH ODWWHU WZR DUH PRUH VSHFLILFDOO\ WHUPHG SULRULW\ TXHXH GLVFLSOLQHV EHFDXVH FXVWRPHUV DUH FDWHJRUL]HG EDVHG RQ WKHLU H[SHFWHG OHQJWK RI VHUYLFH 7KHVH FDWHJRULHV DUH JLYHQ D SULRULW\ OHYHO LQ ZKLFK WKRVH FXVWRPHUV DOORFDWHG WR KLJKHU SULRULW\ OHYHOV JR EHIRUH WKRVH FXVWRPHUV ZLWK ORZHU SULRULW\ :LWKLQ HDFK FDWHJRU\ KRZHYHU FXVWRPHUV DUH VHUYLFHG LQ D VWDQGDUG TXHXH GLVFLSOLQH VXFK DV )&)6 .UDMHZVNL DQG 5LW]PDQ :LQVWRQ f 7ZR RWKHU TXHXH GLVFLSOLQHV H[LVW ZKLFK DUH PRUH GLIILFXOW WR PRGHO 7KH ILUVW LV 6KRUWHVW /HDG 7LPH LQ ZKLFK WKH DUULYDO ZLWK WKH VKRUWHVW WLPH EHWZHHQ WKH FXUUHQW WLPH DQG WKH SURPLVHG WLPH KDV D KLJKHVW SULRULW\ UHJDUGOHVV RI ZKHQ WKH\ HQWHUHG WKH V\VWHP 7KH VHFRQG LV $UELWUDU\ 3ULRULW\ ZKHUH WKH VHUYLFH RUGHU DQG WLPH DUH GHSHQGHQW RQ WKH VHUYHUVf SUHIHUHQFHV RU VRPH IRUP RI XQGHWHUPLQHG WULDJLQJ PHFKDQLVP 7KHVH GLVFLSOLQHV DUH RIWHQ PRGHOHG DV D 6,52 VHUYLFH LQ UDQGRP RUGHUf GLVFLSOLQH /DUVRQ 6FKZDUW] f 0RVW TXHXLQJ PRGHOV GHSHQG RQ D VWHDG\ VWDWH V\VWHP IRU HVWLPDWLQJ TXHXH VWDWLVWLFV IRU D YDU\LQJ QXPEHU RI VHUYHUV $ V\VWHP LV LQ VWHDG\ VWDWH LI f WKH QXPEHU RI VHUYHUV WKH DYHUDJH DUULYDO UDWH DQG WKH DYHUDJH VHUYLFH UDWH DUH QRW FKDQJLQJ f WKH DYHUDJH DUULYDO UDWH LV OHVV WKDQ WKH DYHUDJH VHUYLFH UDWH WLPHV WKH QXPEHU RI VHUYHUV DQG f WKHVH FRQGLWLRQV KDYH H[LVWHG IRU D VXEVWDQWLDO SHULRG RI WLPH7KH RSSRVLWH RI VWHDG\ VWDWH LV WUDQVLHQFH ZKLFK UHIHUV WR WKH EHKDYLRU RI WKH V\VWHP GXULQJ WKH SHULRG IROORZLQJ VRPH FKDQJH 0F&ODLQ DQG 7KRPDV Sf

PAGE 28

,W LV QRW KDUG WR LPDJLQH WKDW LI WKH VHUYLFH UDWH LV OHVV WKDQ WKH DUULYDO UDWH WKHQ WKH TXHXH ZLOO JURZ ZLWKRXW ERXQG EHFDXVH WKH V\VWHP LV QRW SK\VLFDOO\ FDSDEOH RI KDQGOLQJ WKH YROXPH RI DUULYDOV +RZHYHU LW LV DOVR PDWKHPDWLFDOO\ WUXH WKDW WKH TXHXH OHQJWK ZLOO DSSURDFK LQILQLW\ ZKHQ WKH VHUYLFH UDWH HTXDOV WKH DUULYDO UDWH 7KLV FDQ EH LOOXVWUDWHG XVLQJ WZR TXHXLQJ WKHRU\ UHVXOWV EDVHG RQ WKH 00 VLQJOH VHUYHU VLQJOH TXHXHf PRGHO )LUVW WKH XWLOL]DWLRQ UDWH Sf HTXDOV WKH DUULYDO UDWH ;f GLYLGHG E\ WKH VHUYLFH UDWH Sf :LQVWRQ f 6HFRQG WKH H[SHFWHG QXPEHU RI FXVWRPHUV LQ D OLQH /f LV D IXQFWLRQ WKH XWLOL]DWLRQ UDWH Sf VXFK WKDW :LQVWRQ f UW f§ $ 3 f§ (TXDWLRQ f f§ f§f§ (TXDWLRQ f 3 1RWLFH WKDW DV WKH DUULYDO UDWH DSSURDFKHV WKH VHUYLFH UDWH WKH XWLOL]DWLRQ UDWH DSSURDFKHV RQH 7KHUHIRUH DV WKH XWLOL]DWLRQ UDWH DSSURDFKHV RQH WKHQ / DSSURDFKHV LQILQLW\ LH GLYLGHG E\ JRf 7KLV LV QRW DQ LQWXLWLYH UHVXOW DQG WKLV LV WKH SULPDU\ SLWIDOO RI QDLYH VWDIILQJ PRGHOV LH PRGHOV WKDW GR QRW DFFRXQW IRU WKH HIIHFW RI UDQGRP YDULDWLRQ RQ TXHXH EHKDYLRUf :KHQ PDQDJHUV WU\ WR PDWFK WKH VHUYLFH FDSDFLW\ H[DFWO\ WR WKH GHPDQG ORQJ ZDLWLQJ OLQHV ZLOO RFFXU 7KHUH DUH VL[ RSWLRQV W\SLFDOO\ DYDLODEOH IRU LPSURYLQJ WKH SHUIRUPDQFH RI V\VWHPV XQGHU D TXHXLQJ SDUDGLJP f DGG VHUYHUV f LQFUHDVH WKH VHUYLFH UDWH f LQFUHDVH TXHXH VL]H f FKDQJH WKH GLVWULEXWLRQ RI DUULYDOV f UHGXFH WKH YDULDQFH LQ VHUYLFH WLPHV RU LQWHUDUULYDO WLPHV f FKDQJH WKH TXHXH GLVFLSOLQH 7KHVH RSWLRQV DUH GLVFXVVHG LQ PRUH GHWDLO EHORZ

PAGE 29

0HWKRGV IRU ,PSURYLQJ 6\VWHP 3HUIRUPDQFH )LUVW DGGLQJ VHUYHUV LH LQFUHDVLQJ VWDIILQJf LV XVXDOO\ WKH PRVW IUHTXHQWO\ FRQVLGHUHG PHWKRG IRU LPSURYLQJ VHUYLFH FDSDFLW\ KRZHYHU LW FDQ DOVR EH FRVWO\ DQG RIWHQ LV WKH OHDVW HIILFLHQW PHWKRG $V \RX DGG PRUH VHUYHUV WKH PDUJLQDO LPSDFW WKDW HDFK QHZ VHUYHU KDV RQ WKH V\VWHP GHFUHDVHV 7KHUH LV DOVR WKH WUDGHRII RI EDODQFLQJ WKH XWLOL]DWLRQ UDWHV ZLWK H[FHVV FDSDFLW\ 0DQDJHPHQW LV LQWHUHVWHG LQ PDLQWDLQLQJ KLJK XWLOL]DWLRQ EXW WKLV REMHFWLYH PD\ KDYH DQ DGYHUVH LPSDFW RQ WKH RWKHU RSHUDWLQJ FKDUDFWHULVWLFV )RU LQVWDQFH ZKHQ XWLOL]DWLRQ LV WRR KLJK ZRUNHUV PD\ KDYH WURXEOH DGDSWLQJ WR FKDQJHV LQ WKH VHUYLFH GHPDQG +RZHYHU ZKHQ XWLOL]DWLRQ LV WRR ORZ ZRUNHUV ZLOO KDYH WRR PXFK LGOH WLPH .UDMHZVNL DQG 5LW]PDQ f 7KHUH PD\ DOVR EH UHJXODWRU\ RU DFFUHGLWDWLRQ UHVWULFWLRQV WKDW OLPLW WKH PLQLPXP QXPEHU RI VHUYHUV LQ D SDUWLFXODU VHWWLQJ 'XUDLVZDP\ HW DO f 7KHUH KDYH EHHQ WKUHH GLIIHUHQW DSSURDFKHV XVHG LQ WKH OLWHUDWXUH WR RSWLPL]H VWDIILQJ OHYHOV $GMXVW VWDII OHYHOV LQ WHUPV RI WKH DFWXDO QXPEHU RI VHUYLFH SHUVRQQHO 'XUDLVZDP\ HW DO +DPPRQG DQG 0DKHVK ,VKLPRWR HW DO /DP\ HW DO 6DXQGHUV HW DO 6XPQHU DQG +VLHK f $GMXVW VWDII OHYHOV EDVHG RQ WKH QXPEHU RI IXOO WLPH HTXLYDOHQWV )7(Vf +DVKLPRWR HW DO f ,Q PDQ\ VHWWLQJV D PRUH DSSURSULDWH PHWKRG RI RSWLPL]LQJ VWDIILQJ LV WR FRQVLGHU WKH QXPEHU RI )7(V UDWKHU WKDQ WKH DFWXDO QXPEHU RI SHUVRQV ,Q WKLV ZD\ LW EHFRPHV HDVLHU WR FRQVLGHU SDUWWLPH HPSOR\HHV IXOOWLPH HPSOR\HHV WKDW GHYRWH SDUWV RI WKHLU ZRUN GD\ WR GLIIHUHQW WDVNV DQG HPSOR\HHV WKDW KDYH PDQ\ GLIIHUHQW VLPXOWDQHRXV WDVNV WR FRPSOHWH $FOMXVW VWDII OHYHOV EDVHG RQ D SHUFHQWDJH RI PD[LPXP UDWKHU WKDQ H[SHFWHGf ZRUNORDG 0F+XJK f ,Q LQVWDQFHV ZKHQ WKHUH DUH H[WUHPH

PAGE 30

GHPDQG VKLIWV LW LV VRPHWLPHV QHFHVVDU\ WR DQWLFLSDWH VWDIILQJ OHYHOV IRU WKH PD[LPXP UDWKHU WKDQ WKH H[SHFWHG ZRUNORDG 0RGHOV XVLQJ WKLV DSSURDFK XVXDOO\ GLVFXVV VWDIILQJ OHYHOV LQ WHUPV RI D SHUFHQWDJH RI WKH PD[LPXP ZRUNORDG 6HFRQG TXHXH VL]HV DQG ZDLW WLPHV FDQ EH LPSURYHG E\ LQFUHDVLQJ WKH VHUYLFH UDWH UDWKHU WKDQ WKH VWDIILQJ OHYHO 6HUYLFH UDWHV FDQ EH LPSURYHG WKURXJK QHZ WHFKQRORJLHV WUDLQLQJ DQG ZRUNIORZ UHGHVLJQ .UDMHZVNL DQG 5LW]PDQ f )RU LQVWDQFH &DUUXWKHUV f H[DPLQHG WKH ZRUN WXUQRYHU UDWH LQ D ODERUDWRU\ VHWWLQJ ,W ZDV IRXQG WKDW WKH SXUFKDVH RI QHZ HTXLSPHQW ZDV PRUH FRVWHIIHFWLYH WKDQ LQFUHDVLQJ VWDIILQJ $OVR .XPDU DQG .DSXU f IRXQG WKDW LQFUHDVLQJ KRVSLWDO QXUVHVf VKLIW OHQJWK IURP HLJKW WR WZHOYH KRXUV ZDV PRUH EHQHILFLDO WKDQ WKH DGGLWLRQ RI VWDII &KLQ DQG 6SUHFKHU f DQG 2]HNL DQG ,NHXFKL f ERWK IRXQG WKDW ZRUNIORZ FKDQJHV ZHUH DW OHDVW DV LPSRUWDQW DV VWDIILQJ LQFUHDVHV LQ LPSURYLQJ V\VWHP ZDLW WLPHV +RZHYHU LQFUHDVLQJ VHUYLFH UDWHV GR QRW DOZD\V LPSURYH V\VWHP ZDLW WLPHV HVSHFLDOO\ ZKHQ VHUYLFH WLPHV DUH DOUHDG\ UHODWLYHO\ VKRUW )RU H[DPSOH /DP\ HW DO f IRXQG WKDW RQO\ D IUDFWLRQ RI WKH WRWDO ZDLWLQJ WLPH ZDV UHODWHG WR WKH DFWXDO VHUYLFH WLPH LQ D SKDUPDF\ VHWWLQJ ,Q WKLV FDVH WKH VWDIILQJ OHYHO DQG YDULDWLRQV LQ DUULYDO WLPHV ZHUH PRUH VLJQLILFDQW SUHGLFWRUV RI TXHXH ZDLWV WKDQ WKH VHUYLFH WLPH 7KLUG LQFUHDVLQJ WKH TXHXH VL]H PD\ EH DQ RSWLRQ LI FXVWRPHUV DUH EHLQJ WXUQHG DZD\ EHFDXVH WKH\ FDQQRW HYHQ HQWHU WKH V\VWHP HJ EXV\ WHOHSKRQH OLQHf 6LQFH TXHXHV DUH VWRFKDVWLF V\VWHPV WKHUH PD\ EH WLPHV ZKHQ D TXHXH H[FHHGV LWV OLPLWDWLRQV HYHQ ZKHQ S LV OHVV WKDQ RQH LH DUULYDO UDWH LV OHVV WKDQ WKH VHUYLFH UDWHf ,I WKLV KDSSHQV WRR RIWHQ WKHQ TXHXHV ZLOO KDYH KLJK UHMHFWLRQ UDWHV &RVWV RI LQFUHDVLQJ WKH TXHXH VL]H ZLOO YDU\ GUDPDWLFDOO\ GHSHQGLQJ RQ WKH W\SH RI TXHXH )RU LQVWDQFH WKH DGGLWLRQ RI VHYHUDO WHOHSKRQH OLQHV PD\ EH LQH[SHQVLYH ZKHQ FRPSDUHG WR WKH FRQVWUXFWLRQ RI D ODUJHU ZDLWLQJ DUHD .UDMHZVNL DQG 5LW]PDQ f

PAGE 31

)RXUWK FKDQJLQJ WKH DUULYDO UDWH LV XVXDOO\ RQH RI WKH PRVW VXEWOH DQG RYHUORRNHG DUHDV RI LPSURYHPHQW 6RPH H[DPSOHV RI KRZ DUULYDO UDWHV FDQ EH LQIOXHQFHG LQFOXGH f LQIRUPLQJ FRQVXPHUV RI W\SLFDOO\ LGOH RU VORZ SHULRGV VR WKDW \RX PLJKW DWWUDFW WKHP WR XVH VHUYLFHV GXULQJ WKHVH WLPHV LQVWHDG RI SHDN WLPHV f JHW FXVWRPHUV WR XVH DOWHUQDWH URXWHV IRU REWDLQLQJ WKH VDPH LQIRUPDWLRQ VXFK DV D ID[EDFN VHUYLFH RU ZHEVLWH DQG f VFKHGXOH DSSRLQWPHQWV ZLWK VRPH RU DOO FRQVXPHUV VR WKDW DUULYDOV DUH OHVV UDQGRP DQG DUH UHGXFHG GXULQJ SHDN WLPHV .UDMHZVNL DQG 5LW]PDQ f )HZ VWXGLHV KDYH GLVFXVVHG WHFKQLTXHV IRU PRGLI\LQJ WKH DUULYDO UDWH LQWR WKH V\VWHP RWKHU WKDQ DSSRLQWPHQW V\VWHPV 2QH VWXG\ FRQGXFWHG E\ 5HLOO\ HW DO f GLVFXVVHG WKH LPSDFW RI RSWLPL]LQJ VWDIILQJ LQ FRQMXQFWLRQ ZLWK D SDWLHQW GHOD\VFKHGXOLQJ PRGHO ZKHUH SDWLHQWV DUH JLYHQ D GHOD\ WLPH EHIRUH EHLQJ DGPLWWHG WR WKH V\VWHP 7KLV GHOD\ PDQLIHVWHG LWVHOI HLWKHU DV DQ DSSRLQWPHQW RU DQ DQWLFLSDWHG ZDLWLQJ WLPH 7KLV KDG WZR SRWHQWLDO EHQHILWV )LUVW VLQFH SDWLHQWV KDG EHWWHU NQRZOHGJH RI WKH OHQJWK RI WKH ZDLW WKH\ ZHUH QRW QHFHVVDULO\ ERXQG WR WKH FOLQLF DQG FRXOG VSHQG WLPH HOVHZKHUH KHQFH WKH SDWLHQWV FRXOG LPSURYH WKH TXDOLW\ RI WKHLU ZDLWV 6HFRQG DOWKRXJK QRW GLUHFWO\ UHSRUWHG E\ 5HLOO\ HW DO WKH UHDOL]HG LQWHUDUULYDO YDULDQFH VKRXOG KDYH GHFUHDVHG DOORZLQJ IRU D PRUH DFFXUDWH SUHGLFWLRQ RI VWDIILQJ UHTXLUHPHQWV )LIWK UHGXFLQJ WKH YDULDQFH LQ VHUYLFH WLPHV RU LQWHUDUULYDO WLPHV FDQ DOVR UHGXFH TXHXH OHQJWKV %\ H[DPLQLQJ WKH VWHDG\ VWDWH HTXDWLRQV IRU WKH 0*N TXHXH FKDUDFWHULVWLFV LW FDQ EH VKRZQ WKDW UHGXFLQJ YDULDQFH FDQ KDYH D VXEVWDQWLDO HIIHFW RQ UHGXFLQJ WKH HIIHFWLYH ZDLWLQJ WLPHV DQG TXHXH OHQJWKV 0HWKRGV RI UHGXFLQJ YDULDQFH LQ VHUYLFH WLPHV KDYH EHHQ GLVFXVVHG LQ WKH FRQWH[WV RI ZRUN GHVLJQ IDFLOLW\ GHVLJQ WRWDO TXDOLW\ PDQDJHPHQW 740f DQG VWDWLVWLFDO SURFHVV FRQWURO 63&f )RU H[DPSOH UHGHVLJQLQJ RU VWDQGDUGL]LQJ ZRUN IORZ LQ RUGHU WR HOLPLQDWH HUURUV EDFNWUDFNLQJ DQG UHZRUN ZRXOG KHOS UHGXFH WKH YDULDQFH LQ VHUYLFH WLPHV E\ HOLPLQDWLQJ LQFRQVLVWHQW ZRUN SDWWHUQV ,W VKRXOG EH QRWHG WKDW WKHVH WHFKQLTXHV DUH RIWHQ WKH VDPH RQHV XVHG WR UHGXFH

PAGE 32

6HUYLFH WLPHV KHQFH ZKHQ PDNLQJ FKDQJHV LQ WKH V\VWHP WKDW DUH GHVLJQHG WR UHGXFH VHUYLFH WLPHV WKH YDULDQFH LQ VHUYLFH WLPHV LV DOVR RIWHQ UHGXFHG 5HGXFLQJ WKH YDULDQFH LQ LQWHUDUULYDOV PD\ DFKLHYH VLPLODU EHQHILWV $V PHQWLRQHG SUHYLRXVO\ UHGXFLQJ WKH YDULDQFH RQ DUULYDOV PLJKW WDNH WKH IRUP RI DSSRLQWPHQWV RU EORFNLQJ W\SHV RI DUULYDOV LQWR VSHFLILF WLPH VORWV .OHLQURFN .RQ] /DP\ HW DO /DZ DQG .HOWRQ 5HLOO\ HW DO :HVWJDUG DQG %DUU\ :LQVWRQ f 6L[WK FKDQJLQJ WKH TXHXH GLVFLSOLQH KDV DOVR EHHQ VKRZQ WR DIIHFW V\VWHP SHUIRUPDQFH LQ WHUPV RI ZDLWLQJ WLPHV DQG OLQH OHQJWKV )RU H[DPSOH LW KDV EHHQ VKRZQ WKDW XVLQJ WKH VKRUWHVWSURFHVVLQJWLPHILUVW 637)f GLVFLSOLQH ZLOO GHFUHDVH WKH YDULDQFH RI WKH ZDLW WLPHV LQ D V\VWHP WKHUHE\ GHFUHDVLQJ WKH QXPEHU RI ORQJ VHUYLFH WLPHV GXH WR UDQGRP YDULDWLRQ +RZHYHU LQ PDQ\ KHDOWK FDUH VHUYLFHV XUJHQF\ LH SULRULW\f SOD\V D VLJQLILFDQW UROH RQ TXHXH EHKDYLRU GXH WR SULRULWL]DWLRQ DQG SUHHPSWLRQ RI VHUYLFH UHTXHVWV EDVHG RQ QHHG H J HPHUJHQF\ FDUHf VR LW ZRXOG RIWHQ EH LPSRVVLEOH WR VWULFWO\ DGKHUH WR D 637) GLVFLSOLQH +RZHYHU LW PD\ VWLOO EH SRVVLEOH WR VHUYLFH QRQXUJHQW DUULYDOV XVLQJ D 637) GLVFLSOLQH .UDMHZVNL DQG 5LW]PDQ /DZ DQG .HOWRQ 6FKULEHU :LQVWRQ f ,Q WKH SUHYLRXV VHFWLRQ DQ RYHUYLHZ RI TXHXLQJ WKHRU\ ZDV JLYHQ LQ RUGHU WR GHVFULEH WKH LPSRUWDQW YDULDEOHV DQG FRQFHSWV WKDW DUH RIWHQ XVHG LQ TXHXLQJ DQG VLPXODWLRQ PRGHOV WR PRGHO ZDLWLQJ WLPHV VHUYLFH WLPHV DQG GHOD\V ,Q DGGLWLRQ PHWKRGV IRU LPSURYLQJ V\VWHP SHUIRUPDQFH ZHUH UHYLHZHG ,Q WKH QH[W VHFWLRQ WKH UHODWLRQVKLS DPRQJ FRQVXPHU ZDLWLQJ WLPH SHUFHLYHG VHUYLFH TXDOLW\ DQG FXVWRPHU VDWLVIDFWLRQ DUH GLVFXVVHG

PAGE 33

5HODWLRQVKLSV $PRQJ :DLW 7LPH 3HUFHLYHG 6HUYLFH 4XDOLW\ DQG &XVWRPHU 6DWLVIDFWLRQ :DLWLQJ 7LPH DQG 3HUFHLYHG 6HUYLFH 4XDOLW\ 2QO\ WKUHH VWXGLHV ZHUH IRXQG WKDW H[DPLQHG WKH HIIHFW RI ZDLW WLPH RQ VHUYLFH TXDOLW\ DQG QRQH RI WKH VWXGLHV XVHG WKH 6(5948$/ RU 6(593(5) VFDOHV WR PHDVXUH SHUFHLYHG VHUYLFH TXDOLW\ )XUWKHUPRUH WKHVH VWXGLHV FRQVLGHUHG VHUYLFH GHOD\V UDWKHU WKDQ TXHXH ZDLWV +RZHYHU WKH UHVXOWV RI WKHVH VWXGLHV GR LQGLFDWH WKDW ZDLW WLPHV FDQ DGYHUVHO\ DIIHFW HYDOXDWLRQV RI VHUYLFH TXDOLW\ 7D\ORU Df SURSRVHG D IUDPHZRUN FDOOHG 7KH :DLW ([SHULHQFH 0RGHO IRU GHVFULELQJ WKH UHODWLRQVKLS EHWZHHQ WKH ZDLW H[SHULHQFH DQG WKH RYHUDOO VHUYLFH HYDOXDWLRQ 7KLV PRGHO ZDV HYDOXDWHG XVLQJ D VDPSOH RI DLUOLQH SDVVHQJHUV ZKR H[SHULHQFHG SUHn ERDUGLQJ GHOD\V LQ WKHLU IOLJKW SODQV 7D\ORU IRXQG WKDW FXVWRPHUVf RYHUDOO HYDOXDWLRQV RI VHUYLFH TXDOLW\ ZHUH SULPDULO\ UHODWHG WR WKHLU OHYHO RI DQJHU DQG LWV DVVRFLDWHG IHHOLQJV RI DQQR\DQFH LUULWDWLRQ DQG IUXVWUDWLRQ $QJHU FRXOG EH FDXVHG E\ f WKH FXVWRPHUnV XQFHUWDLQW\ DERXW WKH OHQJWK RI WKH GHOD\ f WKH DFWXDO OHQJWK RI WKH GHOD\ f WKH FXVWRPHUnV SHUFHSWLRQ RI WKH VHUYLFH SURYLGHUnV FRQWURO RYHU WKH GHOD\ DQG f WKH GHJUHH RI ILOOHG WLPH ,Q IXUWKHU UHVHDUFK UHODWHG WR WKLV PRGHO 7D\ORU DQG &OD[WRQ f IRXQG WKDW LQGLYLGXDOV ZKR HQFRXQWHUHG SUHERDUGLQJ GHOD\V ZHUH OHVV OLNHO\ WR EH VDWLVILHG ZLWK WKH TXDOLW\ RI WKH RWKHU VHUYLFHV RIIHUHG GXULQJ WKH IOLJKW 6LPLODUO\ 'XEHf5LRX[ HW DO f IRXQG LQ D UHVWDXUDQW VHWWLQJ WKDW WKH WLPLQJ RI WKH GHOD\ LH SUHVHUYLFH LQVHUYLFH RU SRVWVHUYLFHf DQG OHYHO RI FXVWRPHU QHHG ZHUH VLJQLILFDQW LQ HYDOXDWLRQV RI VHUYLFH TXDOLW\ :DLWLQJ 7LPH DQG &XVWRPHU 6DWLVIDFWLRQ $ PXFK PRUH WKRURXJK H[DPLQDWLRQ KDV EHHQ FRQGXFWHG LQ WKH OLWHUDWXUH FRQFHUQLQJ WKH UHODWLRQVKLS EHWZHHQ ZDLW WLPH DQG FXVWRPHU VDWLVIDFWLRQ &XVWRPHU

PAGE 34

VDWLVIDFWLRQ LV D FRQVXPHUfV SRVWVHUYLFH UHIOHFWLRQ UHJDUGLQJ KRZ ZHOO WKH VHUYLFH FRPSDUHG ZLWK WKHLU H[SHFWDWLRQV &XVWRPHU VDWLVIDFWLRQ RFFXUV ZKHQ WKH SHUFHLYHG SHUIRUPDQFH RI WKH VHUYLFH H[FHHGV H[SHFWDWLRQV DQG GLVVDWLVIDFWLRQ RFFXUV ZKHQ SHUIRUPDQFH LV ORZHU WKDQ H[SHFWDWLRQV %ROWRQ DQG 'UHZ 2OLYHU f (YDOXDWLRQV RI FXVWRPHU VDWLVIDFWLRQ DQG SHUFHLYHG VHUYLFH TXDOLW\ XVXDOO\ UHO\ HLWKHU LPSOLFLWO\ RU H[SOLFLWO\ RQ WKH FRQILUPDWLRQGLVFRQILUPDWLRQ SDUDGLJP LQ ZKLFK FRQVXPHU HYDOXDWLRQV DUH EDVHG RQ D FRQILUPDWLRQ RI H[SHFWDWLRQV ,Q DGGLWLRQ WKH VHUYLFH TXDOLW\ OLWHUDWXUH JHQHUDOO\ FRQFHSWXDOL]HV FXVWRPHU VDWLVIDFWLRQ DV DQWHFHGHQW WR SHUFHLYHG VHUYLFH TXDOLW\ %RXOGLQJ HW DO %LWQHU &URQLQ DQG 7D\ORU 3DUDVXUDPDQ HW DO 3DUDVXUDPDQ HW DO 7D\ORU E 7D\ORU DQG &URQLQ f 6LQFH WKH FRQVWUXFWV RI FXVWRPHU VDWLVIDFWLRQ DQG SHUFHLYHG VHUYLFH TXDOLW\ VKDUH VRPH RI WKH VDPH GLPHQVLRQV RIWHQ D FRPPRQ WKHRUHWLFDO EDVLV DQG SHUKDSV D FDXVDO UHODWLRQVKLS LW LV SUREDEOH WKDW ZDLWLQJ WLPH DQG VHUYLFH GHOD\V DIIHFW ERWK FRQVWUXFWV 7KHUHIRUH DQ H[DPLQDWLRQ RI WKH OLWHUDWXUH LQYROYLQJ ZDLWLQJ WLPH VHUYLFH GHOD\V DQG FXVWRPHU VDWLVIDFWLRQ LV LPSRUWDQW HVSHFLDOO\ VLQFH PDQ\ RI WKHVH VWXGLHV GR QRW DGHTXDWHO\ GHVFULEH WKHLU VDWLVIDFWLRQ LQVWUXPHQWV RU WKHLU GHILQLWLRQV RI VDWLVIDFWLRQ 7KLV PDNHV LW GLIILFXOW WR DVFHUWDLQ ZKHWKHU WKH DXWKRUV DUH PHDVXULQJ FXVWRPHU VDWLVIDFWLRQ RU VHUYLFH TXDOLW\ RU ERWK +XL DQG 7VH f WHVWHG D VHUYLFH HYDOXDWLRQ PRGHO IRU D FRPSXWHUL]HG FRXUVH UHJLVWUDWLRQ VHUYLFH DW D XQLYHUVLW\ .DW] /DUVRQ DQG /DUVRQ f FRQGXFWHG D VWXG\ ZLWK EDQN FXVWRPHUV 6WXGLHV GRQH E\ 'DYLV DQG 9ROOPDQQ f DQG 'DYLV f FRQFHUQHG WKH OHQJWK RI ZDLWV LQ D IDVW IRRG UHVWDXUDQW DQG WKH SHUFHQWDJHV RI VDWLVILHG FXVWRPHUV ,Q DOO RI WKHVH VWXGLHV WKH UHVXOWV LQGLFDWHG DQ LQYHUVH UHODWLRQVKLS EHWZHHQ SHUFHLYHG ZDLW WLPHV DQG FXVWRPHU VDWLVIDFWLRQ 7KH\ DOO SURYLGH HYLGHQFH WKDW DV ZDLW WLPHV LQFUHDVH FXVWRPHU VDWLVIDFWLRQ GHFUHDVHV +RZHYHU 'DYLV f VXJJHVWHG WKDW D QRQOLQHDU UHODWLRQVKLS H[LVWV EHWZHHQ ZDLWLQJ WLPH DQG VDWLVIDFWLRQ 7KH SURSRVLWLRQ RI D

PAGE 35

QRQOLQHDU UHODWLRQVKLS EHWZHHQ ZDLWLQJ DQG FRQVXPHU SUHIHUHQFH ZDV DOVR VXJJHVWHG E\ 5LFKDUG /DUVRQ f LQ D GLVFXVVLRQ UHJDUGLQJ WKH SHUFHLYHG XWLOLW\ RI ZDLWLQJ ,Q DGGLWLRQ 7RP DQG /XFH\ f IRXQG WZR LPSRUWDQW UHVXOWV FRQFHUQLQJ H[SHFWHG ZDLWLQJ WLPHV DQG FXVWRPHU VDWLVIDFWLRQ LQ D VXSHUPDUNHW VHWWLQJ )LUVW WKHLU UHVXOWV IRXQG WKDW FXVWRPHUV ZHUH PRUH VDWLVILHG LQ VLWXDWLRQV ZKHUH WKH ZDLW ZDV VKRUWHU WKDQ H[SHFWHG FRPSDUHG ZLWK VLWXDWLRQV ZKHUH WKH ZDLW ZDV ORQJHU WKDQ H[SHFWHG 0RUH LPSRUWDQWO\ KRZHYHU WKH UHVHDUFKHUV IRXQG WKDW LW ZDV WKH UHDVRQ IRU WKH ZDLW WKDW PRVW DIIHFWHG WKH OHYHOV RI VDWLVIDFWLRQ ,I WKH FXVWRPHUV EODPHG WKH VWRUH IRU WKH XQH[SHFWHG ZDLW WKHQ VDWLVIDFWLRQ ZLWK WKH VWRUH WHQGHG WR GHFUHDVH +RZHYHU LI FXVWRPHUV DWWULEXWHG WKH ZDLW WR VRPHWKLQJ RXWVLGH RI WKH VWRUHnV FRQWURO WKHQ QR FKDQJHV LQ WKH VDWLVIDFWLRQ OHYHOV ZHUH HYLGHQW 0DQDJHU DQG &RQVXPHU (VWLPDWHV RI :DLWLQJ 7LPH $V GLVFXVVHG DERYH WKH OLWHUDWXUH VHHPV WR HVWDEOLVK D UHODWLRQVKLS EHWZHHQ WKH DPRXQW RI WLPH FRQVXPHUV ZDLW IRU VHUYLFH DQG WKHLU HYDOXDWLRQV RI VHUYLFH TXDOLW\ +RZHYHU WKHUH LV HYLGHQFH WR VXJJHVW WKDW PDQDJHUV DQG FRQVXPHUV SHUFHLYH WKH ZDLW H[SHULHQFH GLIIHUHQWO\ 'DYLV DQG 9ROOPDQQ f DQG 'DYLV f IRXQG WKDW PDQDJHUV WHQGHG WR RYHUHVWLPDWH WKH GXUDWLRQ RI ZKDW D FXVWRPHU ZRXOG FRQVLGHU DQ DFFHSWDEOH GHOD\ 7KLV UHVXOW LV FRQVLVWHQW ZLWK WKH JDS DQDO\VLV UHVHDUFK FRQGXFWHG E\ 3DUDVXUDPDQ HW DO DQG RWKHUV %URZQ DQG 6ZDUW] 3DUDVXUDPDQ HW DO 6ZDUW] DQG %URZQ f )XUWKHUPRUH .DW] /DUVRQ DQG /DUVRQ f FRQGXFWHG D VWXG\ ZLWK EDQN FXVWRPHUV WKDW IRXQG WKDW LQGLYLGXDOV WHQG WR RYHUHVWLPDWH WKHLU ZDLWV ,Q DGGLWLRQ WKH UHVHDUFKHUV DVNHG FXVWRPHUV WR GHILQH ZKDW WKH\ ZRXOG FRQVLGHU DQ DFFHSWDEOH ZDLW &XVWRPHUV ZLWK ORQJHU GHILQLWLRQV RI DFFHSWDEOH ZDLW WLPHV WHQGHG WR EH PRUH VDWLVILHG WKDQ FXVWRPHUV ZLWK VKRUWHU GHILQLWLRQV 7KXV LI PDQDJHUV RYHUHVWLPDWH ZKDW FRQVXPHUV

PAGE 36

FRQVLGHU DQ DFFHSWDEOH GHOD\ DQG FRQVXPHUV RYHUHVWLPDWH WKH WLPH WKH\ KDYH ZDLWHG WKHUH LV SRWHQWLDO IRU XQLQWHQGHG PDJQLILFDWLRQ RI WKH DFWXDO ZDLW H[SHULHQFH /LQNLQJ 6HUYLFH (YDOXDWLRQV WR 6HUYLFH &DSDFLW\ 7ZR PHWKRGV KDYH EHHQ XVHG WR SUHGLFW KRZ FKDQJHV LQ VHUYLFH FDSDFLW\ ZLOO DIIHFW FRQVXPHU HYDOXDWLRQV RI WKH VHUYLFH 7KH ILUVW PHWKRG LV WR RSHUDWLRQDOL]H WKH VHUYLFH TXDOLW\ GLPHQVLRQV DV PHDVXUDEOH YDULDEOHV )RU LQVWDQFH 2]HNL DQG ,NHXFKL f VWXGLHG VHUYLFH HYDOXDWLRQ LQ D WHOHSKRQH VHUYLFH VHWWLQJ XVLQJ D ZRUNIORZ VLPXODWRU DQG PHDVXUHV RI VHUYLFH TXDOLW\ 0264Vf IRU GLIIHUHQW FRPSRQHQWV RI WKH ZRUN SURFHVV 7KH DXWKRUV GHILQHG DQ 0264 DV VRPH TXDQWLILDEOH RSHUDWLRQDOL]DWLRQ RI VHUYLFH TXDOLW\ VXFK DV UHVSRQVH WLPH 8VLQJ VLPXODWLRQ WKH DXWKRUV ZHUH DEOH WR VHH WKH HIIHFWV RI V\VWHP FKDQJHV RQ WKHVH 0264V ZKHUH D FKDQJH LQ WKH GHVLUHG GLUHFWLRQ LPSOLFLWO\ UHSUHVHQWHG DQ LPSURYHPHQW LQ TXDOLW\ 7KH VHFRQG PHWKRG LV WR GHVFULEH WKH UHODWLRQVKLS EHWZHHQ FKDQJHV LQ V\VWHP SHUIRUPDQFH DQG VHUYLFH HYDOXDWLRQV LQ WHUPV RI D FXPXODWLYH SUREDELOLW\ GLVWULEXWLRQ )RU LQVWDQFH %X[WRQ DQG *DWODQG f FRQGXFWHG DQ H[WHQVLYH VLPXODWLRQ PRGHO WKDW XVHG D FXVWRPHU VDWLVIDFWLRQ LQGH[ WR PRGHO WKH HIIHFWV RI ZRUNLQSURFHVV :,3f DQG GHOLYHU\ WLPH RQ OHYHOV RI FXVWRPHU VDWLVIDFWLRQ 7KLV FXVWRPHU VDWLVIDFWLRQ LQGH[ ZDV H[SUHVVHG DV D SUREDELOLW\ GLVWULEXWLRQ ZKHUH D GHOLYHU\ WLPH HJ GHOLYHU\ ZLWKLQ VHYHQ GD\Vf ZDV HTXDWHG WR DQ H[SHFWHG OHYHO RI FXVWRPHU VDWLVIDFWLRQ 7KLV DSSURDFK GRHV QRW SURGXFH H[DFW UHVXOWV KRZHYHU LW FDQ DOORZ PDQDJHUV WR H[DPLQH UHODWLRQVKLSV EHWZHHQ VHUYLFH FDSDFLW\ DQG SHUFHLYHG VHUYLFH TXDOLW\ 7KH SUHYLRXV VHFWLRQ HYDOXDWHG WKH OLWHUDWXUH GHVFULELQJ KRZ FRQVXPHU ZDLW WLPHV PLJKW LQIOXHQFH HYDOXDWLRQV SHUFHLYHG VHUYLFH SHUIRUPDQFH VXFK DV SHUFHLYHG VHUYLFH TXDOLW\ DQG FXVWRPHU VDWLVIDFWLRQ )XUWKHUPRUH LW ZDV VKRZQ KRZ PDQDJHUVf DQG FRQVXPHUVf SHUFHSWLRQV RI WKH ZDLW H[SHULHQFH FRXOG SRWHQWLDOO\ PDJQLI\ WKLV UHODWLRQVKLS

PAGE 37

,Q DGGLWLRQ WZR PHWKRGV RI EXLOGLQJ WKLV UHODWLRQVKLS LQWR D VLPXODWLRQ PRGHO ZHUH VXPPDUL]HG 7KH QH[W VHFWLRQ ZLOO LGHQWLI\ WKH FXUUHQW VWDWH RI GHYHORSPHQW LQ WKH PHDVXUHPHQW RI SHUFHLYHG VHUYLFH TXDOLW\ DQG SUHVHQW UHVHDUFK LQGLFDWLQJ WKDW SHUFHLYHG VHUYLFH TXDOLW\ LV DVVRFLDWHG ZLWK LQWHQGHG IXWXUH EHKDYLRU 3HUFHLYHG 6HUYLFH 4XDOLW\ 2YHUYLHZ RI WKH &RQFHSWXDO %DVLV RI 3HUFHLYHG 6HUYLFH 4XDOLW\ 8QGHUVWDQGLQJ KRZ FRQVXPHUV RI D VHUYLFH HYDOXDWH VHUYLFH TXDOLW\ LV DQ LVVXH RI LPSRUWDQFH WR PDQDJHUV ,W LV FOHDU WKDW LI D VHUYLFH SURYLGHU XQGHUVWDQGV KRZ FRQVXPHUV HYDOXDWH D SDUWLFXODU VHUYLFH PDQDJHUV FDQ XVH WKHVH HYDOXDWLRQV WR IRFXV RQ ZD\V WR LPSURYH +RZHYHU GHYHORSLQJ D PRGHO WR HIIHFWLYHO\ HYDOXDWH VHUYLFH TXDOLW\ KDV EHHQ DQ HYROYLQJ DQG KLJKO\ GHEDWHG UHVHDUFK LVVXH *URQURRV f )RXU EDVLF FKDUDFWHULVWLFV DSSO\ WR PRVW VHUYLFHV )LUVW VHUYLFHV DUH HVVHQWLDOO\ LQWDQJLEOH 6HFRQG VHUYLFHV XVXDOO\ IRFXV RQ DFWLYLWLHV RU LQIRUPDWLRQ UDWKHU WKDQ SURGXFWV 7KLUG VHUYLFHV DUH SURGXFHG DQG FRQVXPHG VLPXOWDQHRXVO\ LH WKH\ FDQQRW EH LQYHQWRULHGf )RXUWK WKH FRQVXPHU LV D SDUWLFLSDQW LQ WKH SURGXFWLRQ SURFHVV *URQURRV /RYHORFN f $V 6KRVWDFN f GHVFULEHV WKHP f>VMHUYLFHV DUH XQXVXDO LQ WKDW WKH\ KDYH LPSDFW EXW QR IRUPf S f %HFDXVH RI WKH LQWDQJLELOLW\ RI VHUYLFH SHUIRUPDQFH DQG WKH DVSHFW RI VLPXOWDQHRXV SURGXFWLRQ DQG FRQVXPSWLRQ LW LV JHQHUDOO\ PRUH GLIILFXOW WR GHYHORS TXDOLW\ LQGLFDWRUV IRU VHUYLFHV WKDQ IRU SURGXFWV +HVNHWW 3DUDVXUDPDQ HW DO f 4XDOLW\ KRZHYHU LV DQ DEVWUDFW FRQFHSW DQG KDUG WR DGHTXDWHO\ GHILQH $W LWV PRVW EDVLF OHYHO LW FDQ EH JHQHUDOO\ WKRXJKW RI IURP WKH SRLQW RI YLHZ RI 3KLOLS &URVE\fV fFRQIRUPDQFH WR UHTXLUHPHQWVf RU -0 -XUDQfV fILWQHVV IRU XVHf GHILQLWLRQV :HVWJDUG DQG %DUU\ Sf +RZHYHU LW PD\ EH PRUH XVHIXO WR FRQVLGHU TXDOLW\ DV WKH LQKHUHQW RU

PAGE 38

LPSOLFLW GHJUHH RI H[FHOOHQFH YDOXH RU ZRUWK RI D SURGXFW RU VHUYLFH PHDVXUHG E\ LWV DELOLW\ WR VDWLVI\ D JLYHQ QHHG &KULVWHQVHQ DQG 3HQQD :HVWJDUG DQG %DUU\ f :H FDQ XVXDOO\ GHVFULEH TXDOLW\ LQ WHUPV RI RQH RU PRUH RI WKUHH GLPHQVLRQV f D VWUXFWXUDO GLPHQVLRQ LH WKH DWWULEXWHV RI WKH IDFLOLW\ HTXLSPHQW KXPDQ UHVRXUFHV DQG RUJDQL]DWLRQDO VWUXFWXUH WKDW DUH WKH FRPSRQHQWV RI SURFHVVf f D SURFHVV UHODWHG GLPHQVLRQ LH WKH DFWLYLWLHV WKDW PDNH XS WKH SURFHVVf DQG f D WHFKQLFDO RU RXWFRPH UHODWHG GLPHQVLRQ LH WKH HQG UHVXOW RU HIIHFW RI D SURFHVVf $QJDUDQ *URQURRV f 0XFK RI WKH HDUO\ UHVHDUFK FRQFHUQLQJ VHUYLFH TXDOLW\ IRFXVHG SULPDULO\ RQ LGHQWLI\LQJ PHDVXUDEOH GLPHQVLRQV RI VHUYLFH TXDOLW\ 7ZR HDUO\ GHYHORSPHQWV RI VHUYLFH TXDOLW\ ZHUH /HKWLQHQ DQG /HKWLQHQfV ,QWHUDFWLRQ 4XDOLW\ DQG *URQURRVfV 3HUFHLYHG 6HUYLFH 4XDOLW\ 364f PRGHOV 7KH EDVLV RI LQWHUDFWLRQ TXDOLW\ ZDV IRXQGHG RQ WKH SUHPLVH WKDW VHUYLFH TXDOLW\ LV IRUPHG WKURXJK WKH FRQVXPHUfV LQWHUDFWLRQ ZLWK WKH HOHPHQWV RI D VHUYLFH RUJDQL]DWLRQ 7KLV PRGHO VXJJHVWHG WKHUH ZHUH WKUHH HOHPHQWV RI LQWHUDFWLRQ TXDOLW\ SK\VLFDO TXDOLW\ LH WKH WDQJLEOH DVSHFWV RI WKH VHUYLFH VXFK DV WKH HTXLSPHQW RU IDFLOLW\f FRUSRUDWH TXDOLW\ LH WKH LPDJH RI WKH VHUYLFH SURYLGHUf DQG LQWHUDFWLYH TXDOLW\ LH WKH FRQVXPHUfV LQWHUDFWLRQV ZLWK WKH VHUYLFH SURYLGHU DQG RWKHU FRQVXPHUVf /HKWLQHQ DQG /HKWLQHQ DV FLWHG LQ *URQURRV 3DUDVXUDPDQ =HLWKDPO DQG %HUU\ DQG 6ZDUW] DQG %URZQ f 7KH 364 PRGHO GHYHORSHG E\ *URQURRV f XVHG WKH FRQILUPDWLRQ GLVFRQILUPDWLRQ SDUDGLJP WR GHILQH WRWDO SHUFHLYHG VHUYLFH TXDOLW\ DV WKH JDS EHWZHHQ H[SHFWHG VHUYLFH TXDOLW\ DQG H[SHULHQFHG VHUYLFH TXDOLW\ ([SHFWHG VHUYLFH TXDOLW\ LV WKH OHYHO RI TXDOLW\ WKDW WKH FRQVXPHU H[SHFWV WR UHFHLYH ([SHULHQFHG VHUYLFH TXDOLW\ LV PDGH XS RI WKUHH EDVLF GLPHQVLRQV f WHFKQLFDO TXDOLW\ LH TXDOLW\ RI WKH RXWFRPH RI VHUYLFHf f IXQFWLRQDO TXDOLW\ LH TXDOLW\ RI WKH VHUYLFH SURFHVVf DQG f SHUFHLYHG LPDJH RI WKH RUJDQL]DWLRQ *URQURRV f

PAGE 39

3DUDVXUDPDQ =HLWKDPO DQG %HUU\ f EXLOW XSRQ WKH FRQFHSWXDO EDVLV IRUPHG E\ LQWHUDFWLRQ TXDOLW\ DQG 364 LQ WKHLU GHYHORSPHQW RI D JDS DQDO\VLV PRGHO DQG WKH 6(5948$/ LQVWUXPHQW %DVHG RQ H[WHQVLYH IRFXV JURXS LQWHUYLHZV WKHLU LQLWLDO ZRUN GHVFULEHG ILYH SRWHQWLDO JDSV LQ WKH SURYLVLRQ RI VHUYLFHV f FRQVXPHU H[SHFWDWLRQ PDQDJHPHQW SHUFHSWLRQ JDS f PDQDJHPHQW SHUFHSWLRQ VHUYLFH TXDOLW\ VSHFLILFDWLRQ JDS f VHUYLFH TXDOLW\ VSHFLILFDWLRQV VHUYLFH GHOLYHU\ JDS f VHUYLFH GHOLYHU\ H[WHUQDO FRPPXQLFDWLRQV JDS DQG f H[SHFWHG VHUYLFH SHUFHLYHG VHUYLFH JDS 7ZR IXQGDPHQWDO FRQFOXVLRQV ZHUH GHYHORSHG IURP WKH XVH RI WKLV PRGHO )LUVW SHUFHLYHG VHUYLFH TXDOLW\ LV D PXOWLGLPHQVLRQDO FRQVWUXFW KRZHYHU LQWHUDFWLRQ ZLWK WKH VHUYLFH SURYLGHU LV WKH PRVW LPSRUWDQW YDULDEOH LQ WKH DVVHVVPHQW VHUYLFH TXDOLW\ 6HFRQG WKHUH DUH RIWHQ VLJQLILFDQW SHUFHSWLRQ JDSV EHWZHHQ WKH FRQVXPHUV DQG SURYLGHUV RI D VHUYLFH LQGLFDWLQJ WKH VHUYLFH SURYLGHUV GR QRW DOZD\V XQGHUVWDQG WKH H[SHFWDWLRQV RI FRQVXPHUV %URZQ DQG 6ZDUW] 3DUDVXUDPDQ HW DO 6ZDUW] DQG %URZQ f %XLOGLQJ XSRQ WKH JDS DQDO\VLV PRGHO 3DUDVXUDPDQ =HLWKDPO DQG %HUU\ f FRQWLQXHG WR GHYHORS DQG YDOLGDWH DQ LQVWUXPHQW FDOOHG 6(5948$/ 6(5948$/ LV WKH PRVW ZLGHO\ NQRZQ PHDVXUHPHQW RI SHUFHLYHG VHUYLFH TXDOLW\ DQG LWV GHYHORSPHQW KDV KDG FRQVLGHUDEOH LPSDFW RQ WKH V\VWHPDWLF DGYDQFHPHQW RI UHVHDUFK FRQFHUQLQJ SHUFHLYHG VHUYLFH TXDOLW\ RI FRQVXPHU VHUYLFHV *URQURRV f /LNH WKH 364 PRGHO EHIRUH LW 6(5948$/ LV EDVHG RQ WKH GLVFRQILUPDWLRQ RI H[SHFWDWLRQV SDUDGLJP *URQURRV 3DUDVXUDPDQ HW DO f 7KH 6(5948$/ VFDOH FRQVLVWV RI LWHP SDLUV PHDVXULQJ ILYH GLPHQVLRQV RI VHUYLFH TXDOLW\ f WDQJLEOHV f UHOLDELOLW\ f UHVSRQVLYHQHVV f DVVXUDQFH DQG f HPSDWK\ )DFWRU DQDO\VLV DQG UHOLDELOLW\ WHVWLQJ RQ GDWD IURP IRXU VHUYLFH LQGXVWULHV ZHUH XVHG WR GHYHORS WKH ILQDO 6(5948$/ VFDOH 3DUDVXUDPDQ HW DO f

PAGE 40

0HDVXUHPHQW RI 3HUFHLYHG 6HUYLFH 4XDOLW\ $OWKRXJK 6(5948$/ LV SHUKDSV WKH PRVW ZLGHO\ XVHG LQVWUXPHQW WR PHDVXUH VHUYLFH TXDOLW\ LW KDV UHFHLYHG FULWLFLVP IURP RWKHU UHVHDUFKHUV ZKR KDYH EHJXQ WR H[DPLQH WKH DSSOLFDWLRQ RI 6(5948$/ LQ YDULRXV VHWWLQJV 7KHUH KDYH EHHQ WKUHH JHQHUDO DUHDV RI FRQFHUQ UHJDUGLQJ 6(5948$/ f XVH RI GLIIHUHQFH VFRUHV f GLPHQVLRQDOLW\ RI 6(5948$/ DQG f H[WHUQDO YDOLGLW\ 2QH RI WKH PRVW GHEDWHG LVVXHV WKDW KDYH VXUIDFHG FRQFHUQLQJ WKH 6(5948 $/ LQVWUXPHQW LV WKH XVH RI GLIIHUHQFH VFRUHV DV SUHVFULEHG E\ WKH FRQILUPDWLRQ GLVFRQILUPDWLRQ IUDPHZRUN 7KH ILUVW SUREOHP ZLWK WKH XVH RI WKHVH VFRUHV LV WKH WLPLQJ RI WKH H[SHFWDWLRQ PHDVXUHPHQW ([SHFWDWLRQV PD\ EH DOWHUHG GXULQJ DQG DIWHU WKH VHUYLFH H[SHULHQFH VXJJHVWLQJ WKDW H[SHFWDWLRQV PHDVXUHG GXULQJ RU DIWHU VHUYLFH GHOLYHU\ DUH QRW DFFXUDWH UHSUHVHQWDWLRQV RI WKH H[SHFWDWLRQV RI WKH FRQVXPHU DW WKH SRLQW LQ WLPH ZKHQ VHUYLFH FRPPHQFHG &DUPDQ *URQURRV f 7KH VHFRQG SUREOHP FRQFHUQV WKH LPSOLFLW QDWXUH RI WKH SHUFHSWLRQ PHDVXUH 6LQFH WKH SHUFHSWLRQ PHDVXUH LV DOUHDG\ D FRPSDULVRQ EHWZHHQ ZKDW WKH FRQVXPHU H[SHFWHG DQG ZKDW WKH\ SHUFHLYHG DV WKH DFWXDO VHUYLFH HYHQW WKH H[SHFWDWLRQ LV DOUHDG\ LPSOLHG LQ WKH SHUFHSWLRQ PHDVXUH ,I H[SHFWDWLRQV DQG SHUFHSWLRQV DUH ERWK PHDVXUHG WKHQ H[SHFWDWLRQV DUH LQ HIIHFW PHDVXUHG WZLFH *URQURRV 2OLYHU f 7KH WKLUG SUREOHP OLHV LQ WKH TXHVWLRQDEOH UHOLDELOLW\ RI GLIIHUHQFH VFRUHV %URZQ HW DO 2OLYHU 3HWHU HW DO f $V 3HWHU HW DO f VWDWH >GLIIHUHQFH VFRUHV f DUH W\SLFDOO\ OHVV UHOLDEOH WKDQ RWKHU PHDVXUHV f PD\ DSSHDU WR GHPRQVWUDWH GLVFULPLQDQW YDOLGLW\ ZKHQ WKLV FRQFOXVLRQ LV QRW ZDUUDQWHG f PD\ EH RQO\ VSXULRXVO\ FRUUHODWHG WR RWKHU PHDVXUHV VLQFH WKH\ W\SLFDOO\ GR QRW GLVFULPLQDWH IURP DW OHDVW RQH RI WKHLU FRPSRQHQWV DQG f PD\ H[KLELW YDULDQFH UHVWULFWLRQ 7KHUHIRUH WKH XVH RI GLIIHUHQFH VFRUHV PD\ QRW EH UHOLDEOH HYHQ ZKHQ WKH UHOLDELOLW\ VWDWLVWLFV VXJJHVW WKDW WKH LQVWUXPHQW LV UHOLDEOH

PAGE 41

&RQFHUQV DERXW 6(5948$/fV GLPHQVLRQDOLW\ KDYH DOVR VXUIDFHG LQ WKH OLWHUDWXUH $V PHQWLRQHG SUHYLRXVO\ 3DUDVXUDPDQ =HLWKDPO DQG %HUU\fV GHYHORSPHQW RI 6(5948$/ UHVXOWHG LQ D LWHP VFDOH PHDVXULQJ ILYH GLPHQVLRQV +RZHYHU VHYHUDO DXWKRUV KDYH UHSRUWHG UHVXOWV WKDW GHPRQVWUDWH WKDW 6(5948$/fV ILYH GLPHQVLRQV GR QRW DOZD\V JHQHUDOL]H DFURVV VHUYLFH VHWWLQJV 6WXGLHV FRQGXFWHG E\ %DEDNXV DQG 0DQJROG f %DEDNXV DQG %RLOHU f %URZQ HW DO f &DUPDQ f &URQLQ DQG 7D\ORU f DQG +HDGOH\ DQG 0LOOHU f DQG DOO IDLO LQ VRPH GHJUHH WR UHSOLFDWH WKH RULJLQDO GLPHQVLRQV 7KH H[WHUQDO YDOLGLW\ LH JHQHUDOL]DELOLW\f RI 6(5948$/ KDV EHHQ TXHVWLRQHG EHFDXVH RI WKH HYLGHQFH WKDW WKH OHYHO RI XVHIXOQHVV RI WKH LQVWUXPHQW fDV LVf PD\ YDU\ GHSHQGLQJ RQ WKH VHUYLFH ,W KDV EHHQ VKRZQ WKDW WKH LWHPV GR QRW QHFHVVDULO\ ORDG RQ WKH VDPH IDFWRUV %DEDNXV DQG %RLOHU %URZQ HW DO &DUPDQ +HDGOH\ DQG 0LOOHU 7D\ORU DQG &URQLQ f ,Q DGGLWLRQ VRPH UHVHDUFKHUV KDYH VXJJHVWHG WKDW VLJQLILFDQW ZRUGLQJ FKDQJHV DUH QHFHVVDU\ VR WKDW WKH LWHPV DUH XVHIXO LQ D SDUWLFXODU VHUYLFH VHWWLQJ %DEDNXV DQG 0DQJROG &DUPDQ f )XUWKHUPRUH LW PD\ DFWXDOO\ EH QHFHVVDU\ WR PRGLI\ WKH OHQJWK WKH VFDOH GHSHQGLQJ RQ WKH VHWWLQJ %DEDNXV DQG %RLOHU %DEDNXV DQG 0DQJROG &DUPDQ f %HFDXVH RI WKHVH SUREOHPV RWKHU DSSURDFKHV LQ PHDVXULQJ VHUYLFH TXDOLW\ KDYH EHHQ VXJJHVWHG 2QH RI WKHVH DSSURDFKHV LV WR XVH D SHUFHSWLRQVRQO\ VFDOH IRU WKH PHDVXUHPHQW RI VHUYLFH TXDOLW\ VXFK DV WKH 6(593(5) LQVWUXPHQW WHVWHG E\ &URQLQ DQG 7D\ORU f 3HUFHSWLRQVRQO\ VFDOHV DYRLG WKH FRQFHUQV DERXW WKH XVH DQG UHOLDELOLW\ RI GLIIHUHQFH VFRUHV ZLWKRXW VDFULILFLQJ VFDOH SHUIRUPDQFH 3HUFHSWLRQRQO\ VFDOHV DOVR KDYH WKH DGYDQWDJH RI EHLQJ HDVLHU WR DGPLQLVWHU SULPDULO\ EHFDXVH WKH VXEMHFW GRHV QRW KDYH WR DQVZHU ERWK WKH H[SHFWDWLRQ DQG SHUIRUPDQFH TXHVWLRQ VXEVHWV 7KLV DGYDQWDJH JUHDWO\ HQKDQFHV WKH SUDFWLFDOLW\ RI WKH VFDOH %DEDNXV DQG %RLOHU %URZQ HW DO DQG &URQLQ DQG 7D\ORU +HDGOH\ DQG 0LOOHU 0F$OH[DQGHU f =HLWKDPO HW DO f UHFHQWO\ UHFRJQL]HG WKH YDOXH RI SHUFHSWLRQVRQO\ VFDOHV VXFK DV 6(593(5)

PAGE 42

7KH\ VWDWH f7KH SHUFHSWLRQVRQO\ RSHUDWLRQDOL]DWLRQ LV DSSURSULDWH LI WKH SULPDU\ SXUSRVH RI PHDVXULQJ VHUYLFH TXDOLW\ LV WR DWWHPSW WR H[SODLQ WKH YDULDQFH LQ VRPH GHSHQGHQW FRQVWUXFWSf 6(593(5) HVVHQWLDOO\ HOLPLQDWHV WKH H[SHFWDWLRQ SRUWLRQ RI WKH 6(5948$/ VFDOH DQG IRFXVHV HQWLUHO\ RQ VHUYLFH SHUIRUPDQFH &URQLQ DQG 7D\ORU f WHVWHG 6(593(5) LH SHUFHSWLRQVRQO\f YHUVXV 6(5948$/ LH SHUFHSWLRQVPLQXV H[SHFWDWLRQVf )RXU VHUYLFH LQGXVWULHV ZHUH DQDO\]HG EDQNLQJ SHVW FRQWURO GU\ FOHDQLQJ DQG IDVW IRRG 7KH UHVXOWV RI WKH /,65(/ DQG REOLTXH IDFWRU DQDO\VLV SURFHGXUHV GLG QRW LQGLFDWH WKDW WKH GLPHQVLRQDOLW\ FRQIRUPHG WR WKH ILYHIDFWRUV SURSRVHG E\ 3DUDVXUDPDQ HW DO )ORZHYHU VWURQJ UHOLDELOLW\ VFRUHV ZHUH H[KLELWHG IRU DOO IRU LQGXVWULHV FRHIILFLHQW DOSKDV JUHDWHU WKDQ f %DVHG RQ WKHVH UHVXOWV DQG WKH IDLOXUH RI RWKHU VWXGLHV WR H[DFWO\ UHSOLFDWH WKH ILYH GLPHQVLRQV &URQLQ DQG 7D\ORU f VXJJHVW WKDW LWHPV LQ 6(5948$/ DQG KHQFH 6(593(5)f VKRXOG EH FRQVLGHUHG D XQLGLPHQVLRQDO PHDVXUH RI VHUYLFH TXDOLW\ UDWKHU WKDQ D PXOWLGLPHQVLRQDO PHDVXUH ,Q DGGLWLRQ 6(593(5) H[SODLQHG VOLJKWO\ PRUH RI WKH YDULDWLRQ LQ SHUFHLYHG RYHUDOO VHUYLFH TXDOLW\ VDWLVIDFWLRQ DQG SXUFKDVH LQWHQWLRQ WKDQ 6(5948$/ 6LJQLILFDQW GHEDWH KDV RFFXUUHG LQ WKH OLWHUDWXUH UHJDUGLQJ &URQLQ DQG 7D\ORUnV f SHUIRUPDQFH RQO\ DSSURDFK WR PHDVXULQJ SHUFHLYHG VHUYLFH TXDOLW\ &URQLQ DQG 7D\ORU 3DUDVXUDPDQ HW DO f :KLOH 3DUDVXUDPDQ HW DO f FRQFHGH WKDW SHUIRUPDQFH RQO\ PHDVXUHV VXFK DV 6(593(5) WHQG WR RIIHU JUHDWHU SUHGLFWLYH SRZHU WKH\ GR QRW KDYH DV PXFK GLDJQRVWLF YDOXH DV GLVFRQILUPDWLRQ PHDVXUHV VXFK DV 6(5948$/ ,Q UHVSRQVH &URQLQ DQG 7D\ORU f VXJJHVW WKDW WKH 6(593(5) VFDOH FRXOG EH XVHG DV D VXPPHG RU DYHUDJHG VHUYLFH TXDOLW\ VFRUH WKDW PLJKW EH SORWWHG RYHU WLPH 7KHUHIRUH D SHUIRUPDQFHRQO\ PHDVXUH VXFK DV 6(593(5) VKRXOG EH XVHG ZKHQ WKH REMHFWLYH LV WR REWDLQ DQ RYHUDOO PHDVXUH RI VHUYLFH TXDOLW\ WKDW FDQ EH XVHG DV D GHSHQGHQW YDULDEOH DQG DQDO\]HG RYHU WLPH

PAGE 43

$ PRGLILHG 6(593(5) VFDOH ZLOO EH XVHG LQ WKLV UHVHDUFK EHFDXVH LW HOLPLQDWHV WKH GLVDGYDQWDJHV RI XVLQJ WKH GLIIHUHQFH VFRUH DSSURDFK ,Q DGGLWLRQ WKH HOLPLQDWLRQ RI WKH H[SHFWDWLRQ SRUWLRQ RI WKH VFDOH UHGXFHV WKH QXPEHU RI TXHVWLRQV WKDW WKH UHVSRQGHQW LV UHTXLUHG WR DQVZHU ZKLFK HQKDQFHV HDVH RI DGPLQLVWUDWLRQ DQG VKRXOG LPSURYH WKH UHVSRQVH UDWH )XUWKHUPRUH RQH RI WKH JRDOV RI WKH SURSRVHG UHVHDUFK LV WR VWXG\ WKH UHODWLRQVKLS EHWZHHQ ZDLW WLPHV DQG VHUYLFH TXDOLW\ 6LQFH 6(593(5) FDQ EH XVHG DV D VXPPHG LQWHUYDO VFRUH WKLV PHDVXUH LV PRUH XVHIXO WKDQ 6(5948$/ IRU SUHGLFWLYH SXUSRVHV 3HUFHLYHG 6HUYLFH 4XDOLW\ DQG %HKDYLRUDO ,QWHQWLRQ 3UHVXPDEO\ WKHUH DUH WZR UHDVRQV IRU PHDVXULQJ SHUFHLYHG VHUYLFH TXDOLW\ )LUVW VR ZH FDQ XQGHUVWDQG DQG LPSURYH WKH VKRUWFRPLQJV RI VHUYLFH GHOLYHU\ 6HFRQG WR XQGHUVWDQG WKH LPSDFW WKDW VHUYLFH TXDOLW\ KDV RQ IXWXUH EHKDYLRU $V PHQWLRQHG LQ WKH SUHYLRXV FKDSWHU EHKDYLRUDO LQWHQWLRQV VXFK DV UHWXUQ LQWHQWLRQ DQG UHFRPPHQGDWLRQ DUH VLJQLILFDQW IDFWRUV LQ PDLQWDLQLQJ DQ HIIHFWLYH GUXJ LQIRUPDWLRQ VHUYLFH $XWKRUV IURP WKH KHDOWK FDUH DQG RWKHU ILHOGV KDYH VWXGLHG WKH EHKDYLRUDO FRQVHTXHQFHV RI VHUYLFH TXDOLW\ %DEDNXV DQG 0DQJROG f %RXOGLQJ HW DO f &URQLQ DQG 7D\ORU f +HDGOH\ DQG 0LOOHU f 3DUDVXUDPDQ HW DO f DQG =HLWKDPO HW DO f DOO XVHG PRGLILHG YHUVLRQV RI WKH 6(5948$/ VFDOH WR PHDVXUH VHUYLFH TXDOLW\ DQG LWV LQIOXHQFH RQ IXWXUH LQWHQWLRQV LQ D ZLGH YDULHW\ RI VHUYLFH VHWWLQJV 7KHVH VWXGLHV LQGLFDWHG WKDW SHUFHLYHG VHUYLFH TXDOLW\ ZDV UHODWHG WR OR\DOW\ VZLWFKLQJ LQWHQWLRQ FRPSODLQLQJ FRPSOLPHQW DQG UHFRPPHQGDWLRQ LQWHQWLRQ DQG UHWXUQ LQWHQWLRQ 'XEHf5LRX[ HW DO f DQG %LWQHU f XVHG DOWHUQDWLYH LQVWUXPHQWV WR ORRN DW WKH LVVXH RI VHUYLFH TXDOLW\ DQG IXWXUH LQWHQW 7KHVH WZR VWXGLHV XVHG UROHSOD\LQJ PHWKRGRORJLHV LQYROYLQJ UHVWDXUDQW DQG DLU WUDYHO GHOD\V UHVSHFWLYHO\ 7KH UHVXOWV RI WKHVH VWXGLHV ZHUH FRQVLVWHQW

PAGE 44

ZLWK WKRVH XVLQJ WKH PRGLILHG 6(5948$/ LQVWUXPHQWV ZKHUH VHUYLFH TXDOLW\ ZDV UHODWHG WR LQWHQGHG IXWXUH EHKDYLRU 6XPPDU\ RI WKH /LWHUDWXUH 4XHXLQJ WKHRU\ GHVFULEHV VHUYLFH FDSDFLW\ DV D IXQFWLRQ RI WKH DUULYDO UDWH $f WKH VHUYLFH UDWH Sf WKH QXPEHU RI VHUYHUV Vf 7KHUH DUH W\SLFDOO\ VL[ RSWLRQV DYDLODEOH IRU UHGXFLQJ ZDLWLQJ WLPHV DQG GHOD\V LQ D V\VWHP f DGG VHUYHUV f LPSURYH WKH VHUYLFH UDWH f LQFUHDVH TXHXH VL]H f FKDQJH WKH DUULYDO UDWH f UHGXFH WKH YDULDQFH LQ VHUYLFH WLPHV RU LQWHUDUULYDO WLPHV DQG f FKDQJH WKH TXHXH GLVFLSOLQH 7KHUH DUH DW OHDVW WZR PHWKRGV RI EXLOGLQJ WKH UHODWLRQVKLS EHWZHHQ ZDLWLQJ WLPH DQG SHUFHLYHG VHUYLFH TXDOLW\ LQWR D VLPXODWLRQ PRGHO f RSHUDWLRQDOL]LQJ VHUYLFH TXDOLW\ DV D PHDVXUDEOH YDULDEOH DQG f HVWLPDWLQJ WKH UHODWLRQVKLS DV D SUREDELOLW\ GLVWULEXWLRQ 3HUFHLYHG VHUYLFH TXDOLW\ LV D FRQFHSW WKDW LV VWLOO HYROYLQJ &XUUHQWO\ 6(5948$/ LV SHUKDSV WKH PRVW ZLGHO\ XVHG LQVWUXPHQW WR PHDVXUH SHUFHLYHG VHUYLFH TXDOLW\ 7KH 6(5948$/ VFDOH FRQVLVWV RI LWHP SDLUV PHDVXULQJ ILYH GLPHQVLRQV RI VHUYLFH TXDOLW\ f WDQJLEOHV f UHOLDELOLW\ f UHVSRQVLYHQHVV f DVVXUDQFH DQG f HPSDWK\ 5HFHQWO\ KRZHYHU LW KDV UHFHLYHG FULWLFLVP IURP RWKHU UHVHDUFKHUV LQYROYLQJ WKUHH JHQHUDO DUHDV RI FRQFHUQ If 6(5948$/fV XVH RI GLIIHUHQFH VFRUHV f WKH GLPHQVLRQDOLW\ RI 6(5948$/ DQG f 6(5948$/fV H[WHUQDO YDOLGLW\ 6(593(5) LV D VFDOH WKDW DYRLGV PDQ\ RI WKH FRQFHUQV OLVWHG DERYH E\ DVFHUWDLQLQJ RQO\ FRQVXPHU SHUFHSWLRQV DV RSSRVHG WR H[SHFWDWLRQV DQG SHUFHSWLRQVf UHJDUGLQJ WKH ILYH GLPHQVLRQV RI VHUYLFH TXDOLW\ OLVWHG DERYH $UJXPHQWV KDYH DOVR EHHQ PDGH WKDW 6(593(5) H[KLELWV VWURQJHU UHOLDELOLW\ DQG YDOLGLW\ WKDQ 6(5948$/ 6(593(5) DOVR KDV WKH DGYDQWDJH RI EHLQJ HDVLHU WR DGPLQLVWHU WKXV HQKDQFLQJ WKH SUDFWLFDOLW\ RI WKH VFDOH

PAGE 45

7KH OLWHUDWXUH DOVR VXJJHVWV WKDW SHUFHLYHG VHUYLFH TXDOLW\ LV UHODWHG WR IXWXUH FRQVXPHU EHKDYLRU VXFK DV UHWXUQ LQWHQWLRQ DQG LQWHQW WR UHFRPPHQG ,Q DGGLWLRQ WKHUH LV HYLGHQFH WR K\SRWKHVL]H WKDW HYDOXDWLRQV RI VHUYLFH TXDOLW\ DUH DIIHFWHG E\ ZDLWLQJ WLPH DQG GHOD\V LQ VHUYLFH )XUWKHUPRUH LW DSSHDUV WKDW PDQDJHPHQW WHQGV WR RYHUHVWLPDWH DFFHSWDEOH ZDLWV DQG FXVWRPHUV WHQG WR RYHUHVWLPDWH DFWXDO ZDLWV &KDSWHU WKUHH ZLOO XVH WKH LQIRUPDWLRQ SUHVHQWHG LQ WKLV OLWHUDWXUH UHYLHZ WR FRQVWUXFW D IUDPHZRUN IRU WKH YDULDEOHV WR EH VWXGLHG +\SRWKHVHV DQG VSHFLILF UHVHDUFK TXHVWLRQV ZLOO WKHQ EH GHYHORSHG EDVHG RQ WKLV IUDPHZRUN &KDSWHU IRXU ZLOO SUHVHQW WKH PHWKRGV XVHG WR WHVW WKHVH K\SRWKHVHV DQG VSHFLILF UHVHDUFK TXHVWLRQV

PAGE 46

&+$37(5 5(6($5&+ )5$0(:25. $1' +<327+(6(6 5HVHDUFK )UDPHZRUN 7KH SUHYLRXV WZR FKDSWHUV KDYH GHVFULEHG WKH SRWHQWLDO UHODWLRQVKLSV DPRQJ VHUYLFH FDSDFLW\ ZDLW WLPHV SHUFHLYHG VHUYLFH TXDOLW\ DQG EHKDYLRUDO LQWHQWLRQ 3RVVLEOH FKDQJHV IRU LPSURYLQJ WKH SHUIRUPDQFH RI VHUYLFH V\VWHPV IURP D TXHXLQJ WKHRU\ VWDQGSRLQW ZHUH DOVR VXJJHVWHG 7KLV FKDSWHU EXLOGV RQ WKH FRQFHSWV SUHVHQWHG LQ WKH LQWURGXFWLRQ DQG WKH OLWHUDWXUH UHYLHZ E\ SUHVHQWLQJ D UHVHDUFK IUDPHZRUN IRU WKH YDULDEOHV XVHG LQ WKLV VWXG\ 7KHUH DUH IRXU SULPDU\ UHODWLRQVKLSV QHFHVVDU\ WR XQGHUVWDQGLQJ WKH UHVHDUFK IUDPHZRUN IRU WKLV VWXG\ )LUVW VHUYLFH WLPHV FDQ EH GHVFULEHG XVLQJ D TXHXLQJ SDUDGLJP DV D IXQFWLRQ RI WKH DUULYDO UDWH WKH VHUYLFH UDWH WKH QXPEHU RI VHUYHUV DQG WKH SULRULW\ GLVFLSOLQH ,PSRUWDQW RXWSXW YDULDEOHV RI WKH V\VWHP ZRXOG LQFOXGH WKH H[SHFWHG VHUYLFH WLPHV DQG TXHXH ZDLWV WKH SHUFHQWDJH RI VHUYLFH GHOD\V LQIRUPDWLRQ UHJDUGLQJ QXPEHU LQ WKH V\VWHP DQG TXHXH OHQJWKV DQG WKH XWLOL]DWLRQ UDWHV RI WKH VHUYHUV 6HFRQG ZDLWLQJ WLPHV DQG VHUYLFH GHOD\V DUH UHODWHG WR FRQVXPHU SHUFHSWLRQV RI VHUYLFH TXDOLW\ DQG WKHVH SHUFHSWLRQV DUH UHODWHG WR IXWXUH EHKDYLRUDO LQWHQWLRQ +RZHYHU FRQVXPHUV PD\ QRW DFFXUDWHO\ HVWLPDWH DFWXDO ZDLWLQJ WLPHV WKHUHIRUH SHUFHLYHG ZDLWLQJ WLPH PD\ EH D PRUH LPSRUWDQW YDULDEOH 7KLUG E\ FUHDWLQJ DQG PDQLSXODWLQJ D YDOLG VLPXODWLRQ RI WKH VHUYLFH V\VWHP L H D FRPSXWHU PRGHO RI WKH DFWXDO V\VWHPf ZH FDQ SURSRVH FKDQJHV LQ WKH V\VWHP WKDW ZLOO GHFUHDVH WKH DPRXQW RI WLPH D FRQVXPHU ZDLWV IRU VHUYLFH WR EH FRPSOHWHG )RXUWK VHUYLFH FDSDFLW\ PLJKW EH RSWLPL]HG E\ GHILQLQJ D PDWKHPDWLFDO UHODWLRQVKLS EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ DQG VHUYLFH WLPHV RU VHUYLFH GHOD\V

PAGE 47

'HILQLWLRQV RI WKH FRQFHSWV XVHG LQ WKH WKHRUHWLFDO IUDPHZRUN IRU WKLV VWXG\ )LJXUH f DUH GLVFXVVHG EHORZ :KHUH DSSOLFDEOH WKH ILUVW GHILQLWLRQ UHIHUV WR WKH FRQFHSW DV LW DSSOLHV WR WKH HPSLULFDO PHDVXUHPHQW DQG WKH VHFRQG GHILQLWLRQ UHIHUV WR WKH FRQFHSW DV LW DSSOLHV WR WKH VLPXODWLRQ $UULYDO 5DWH $f f 7KH HPSLULFDOO\ REVHUYHG LQWHUDUULYDO GLVWULEXWLRQ RI FRQVXPHU LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV LQ WKH 'UXJ ,QIRUPDWLRQ 6HUYLFH ',6f f 7KH SUREDELOLW\ GLVWULEXWLRQ LQSXW LQWR WKH VLPXODWLRQ PRGHO WR SUHGLFW WKH LQWHUDUULYDO GLVWULEXWLRQ RI FRQVXPHU LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV LQWR WKH ',6 $FWXDO 6HUYLFH 7LPH 7KH WRWDO WLPH UHTXLUHG WR UHVHDUFK DQ DQVZHU WR WKH TXHVWLRQ REWDLQ DQ DSSURYDO DQG UHWXUQ WKH DQVZHU WR WKH FDOOHU WKDW ZDV HPSLULFDOO\ REVHUYHG IRU VHUYLFH SURFHVV LQ WKH ',6 6HUYLFH WLPH LV DOVR UHIHUUHG WR DV ZDLWLQJ WLPH RU UHVSRQVH WLPH %HKDYLRUDO ,QWHQWLRQ $ VXEMHFWnV DVVHVVPHQW UHJDUGLQJ WKHLU IXWXUH LQWHQWLRQV UHJDUGLQJ WKH VHUYLFH 0RUH VSHFLILFDOO\ LW UHODWHV WR ZKHWKHU RU QRW WKH VXEMHFW LQWHQGV WR XVH WKH VHUYLFH DJDLQ RU UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH ([SHFWHG 1XPEHU LQ 6\VWHP /f $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH QXPEHU RI XQFRPSOHWHG LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV LQ WKH ',6 ([SHFWHG 4XHXH /HQJWK /Tf $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH QXPEHU RI LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV LQ WKH ',6 WKDW KDYH QRW \HW VWDUWHG WKH UHVHDUFK SURFHVV

PAGE 48

([SHFWHG 7LPH LQ 4XHXH :Tf $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH DPRXQW RI WLPH WKDW TXHVWLRQV PXVW ZDLW LQ WKH TXHXH EHIRUH VWDUWLQJ WKH UHVHDUFK SURFHVV ([SHFWHG 7LPH LQ 6\VWHP :f $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH WRWDO DPRXQW RI WLPH WKDW D TXHVWLRQ RU LQIRUPDWLRQ UHTXHVW VSHQGV LQ WKH V\VWHP ([SHFWHG 8WLOL]DWLRQ 5DWH Sf $ VLPXODWLRQ RXWSXW YDULDEOH LQGLFDWLQJ WKH DYHUDJH SHUFHQWDJH RI WLPH WKDW VHUYHUV ZHUH EXV\ 2YHUDOO 6HUYLFH 4XDOLW\ 264f $ VXEMHFWnV RYHUDOO SHUFHSWLRQ RI WKH VHUYLFH TXDOLW\ RI WKH GUXJ LQIRUPDWLRQ VHUYLFH 3HUFHLYHG 6HUYLFH 4XDOLW\ 364f $ VXEMHFWnV HYDOXDWLRQ RI WKH VHUYLFH TXDOLW\ RI WKH GUXJ LQIRUPDWLRQ VHUYLFH EDVHG RQ WKH LWHPV LQ WKH 6(593(5) LQVWUXPHQW 3HUFHLYHG 6HUYLFH 7LPH $ VXEMHFWnV SHUFHSWLRQV UHJDUGLQJ WKH UHVSRQVH WLPH RI WKH GUXJ LQIRUPDWLRQ VHUYLFH 0RUH VSHFLILFDOO\ LW UHODWHV WR SHUFHSWLRQV UHJDUGLQJ f WKH DFFHSWDELOLW\ RI WKH UHVSRQVH WLPH f WKH XVHIXOQHVV RI WKH DQVZHU RQFH WKH UHVSRQVH ZDV UHFHLYHG f WKH VXEMHFWnV GHVLUH IRU TXLFNHU UHVSRQVHV IURP WKH ',6 DQG f ZKHWKHU WKH UHVSRQVH WLPH ZDV VKRUWHU HTXDO RU ORQJHU WKDQ H[SHFWHG 4XHXH 'LVFLSOLQH f 7KH PHWKRG FXUUHQWO\ XVHG LQ WKH GUXJ LQIRUPDWLRQ VHUYLFH WR SULRULWL]H FRQVXPHUV IRU VHUYLFH DV GHVFULEHG E\ WKH VHUYLFH SURYLGHUV GXULQJ SHUVRQDO LQWHUYLHZV f $ VLPXODWLRQ LQSXW XVHG WR GHVFULEH WKH ZD\ LQ ZKLFK VHUYHUV GHFLGH WKH RUGHU LQ ZKLFK LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV DUH KDQGOHG E\ WKH GUXJ LQIRUPDWLRQ VHUYLFH

PAGE 49

6HUYLFH 'HOD\ $ VWDWH LQGLFDWLQJ ZKHWKHU WKH DFWXDO VHUYLFH WLPH ZDV ORQJHU WKDQ WKH UHVSRQVH WLPH QHHGHG E\ WKH FDOOHU 6HUYLFH 5DWH Sf f 7KH HPSLULFDOO\ REVHUYHG VHUYLFH WLPH GLVWULEXWLRQV RI FRQVXPHU LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV LQ WKH 'UXJ ,QIRUPDWLRQ 6HUYLFH ',6f f 7KH SUREDELOLW\ GLVWULEXWLRQ LQSXW LQWR WKH VLPXODWLRQ PRGHO WR SUHGLFW WKH VHUYLFH WLPHV IRU WKH VWHSV LQ WKH VHUYLFH SURFHVV LQ WKH ',6 6WDIILQJ /HYHO Vf f 7KH REVHUYHG QXPEHU RI LQGLYLGXDOV DYDLODEOH WR VHUYH FRQVXPHUV DQG WKHLU UROHV LQ KDQGOLQJ FRQVXPHU LQIRUPDWLRQ UHTXHVWV DQG TXHVWLRQV f $ VLPXODWLRQ LQSXW GHVFULELQJ WKH QXPEHU RI LQGLYLGXDOV DYDLODEOH WR KDQGOH VHUYLFH UHTXHVWV GXULQJ WKH YDULRXV VWHSV RI WKH VHUYLFH SURFHVV 5HVHDUFK +\SRWKHVLV DQG 6SHFLILF 5HVHDUFK 4XHVWLRQV 7KH OLWHUDWXUH KDV SURSRVHG WKDW VLJQLILFDQW SRVLWLYH UHODWLRQVKLSV VKRXOG H[LVW DPRQJ PHDVXUHV RI 364 264 DQG LQWHQGHG IXWXUH EHKDYLRU %RXOGLQJ HW DK &URQLQ DQG 7D\ORU 3DUDVXUDPDQ HW DK f +\SRWKHVLV RQH +,f DVVHVVHV WKH UHODWLRQVKLS EHWZHHQ 364 DQG 264 +\SRWKHVHV WZR +f DQG WKUHH +f DUH DLPHG DW DVFHUWDLQLQJ WKH VWUHQJWK RI WKH UHODWLRQVKLS EHWZHHQ 364 DQG EHKDYLRUDO LQWHQWLRQ LH LQWHQW WR FDOO DJDLQ DQG LQWHQW WR UHFRPPHQG VHUYLFHf DQG EHWZHHQ 264 DQG EHKDYLRUDO LQWHQWLRQ +, 7KHUH LV D SRVLWLYH UHODWLRQVKLS EHWZHHQ HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f DQG HYDOXDWLRQV RI RYHUDOO VHUYLFH TXDOLW\ 264f +D ,QWHQWLRQ WR XVH WKH VHUYLFH LQ WKH IXWXUH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f

PAGE 50

+E ,QWHQWLRQ WR UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 362f +D ,QWHQWLRQ WR XVH WKH VHUYLFH LQ WKH IXWXUH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI RYHUDOO VHUYLFH TXDOLW\ 262f +E ,QWHQWLRQ WR UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI RYHUDOO VHUYLFH TXDOLW\ 262f 3UHYLRXV UHVHDUFK KDV LQGLFDWHG WKH SRZHUIXO UROH RI FXVWRPHU SHUFHSWLRQV RQ HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ LQFOXGLQJ SHUFHSWLRQV UHJDUGLQJ VHUYLFH WLPH ,W KDV EHHQ VKRZQ WKDW FRQVXPHUV RIWHQ FDQQRW DFFXUDWHO\ DVFHUWDLQ WKH DPRXQW RI WLPH ZLWKLQ ZKLFK WKH VHUYLFH ZDV FRPSOHWHG .DW] /DUVRQ DQG /DUVRQ f ,I FRQVXPHUV FDQQRW DFFXUDWHO\ HYDOXDWH WKH DFWXDO VHUYLFH WLPH WKHQ SHUFHLYHG WLPH PD\ EH D PRUH LPSRUWDQW SUHGLFWRU RI SHUFHLYHG VHUYLFH TXDOLW\ WKDQ DFWXDO WLPH )RU LQVWDQFH 7RP DQG /XFH\ f IRXQG WKDW FXVWRPHUV ZHUH PRUH VDWLVILHG ZLWK WKH VHUYLFH ZKHQ WKH ZDLW ZDV VKRUWHU WKDQ H[SHFWHG WKDQ ZKHQ WKH ZDLW ZDV ORQJHU WKDQ H[SHFWHG +\SRWKHVLV IRXU +f LV DLPHG DW JDLQLQJ PRUH LQIRUPDWLRQ UHJDUGLQJ KRZ FDOOHUVn SHUFHSWLRQV UHJDUGLQJ WKH UHVSRQVH WLPH RI WKH VHUYLFH DUH UHODWHG WR WKHLU DWWLWXGHV UHJDUGLQJ 364 +D $FFHSWDELOLW\ RI WKH UHVSRQVH WLPH RI WKH VHUYLFH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 362f +E 3HUFHLYHG XVHIXOQHVV RI WKH LQIRUPDWLRQ RQFH WKH UHVSRQVH ZDV UHFHLYHG LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 362f +F 3HUFHLYHG TXLFNQHVV RI UHVSRQVH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f +G 'HYLDWLRQV IURP H[SHFWHG UHVSRQVH WLPHV DUH SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 362f

PAGE 51

$V UHSRUWHG LQ FKDSWHU WKH UHVXOWV RI %ROWRQ DQG 'UHZ f &OHPPHU DQG 6FKQHLGHU Df 'DYLV f +XL DQG 7VH f DQG .DW] /DUVRQ DQG /DUVRQ f KDYH DOO UHSRUWHG HYLGHQFH WR VXSSRUW WKDW DQ LQYHUVH UHODWLRQVKLS H[LVWV EHWZHHQ HYDOXDWLRQV RI WKH VHUYLFH LH SHUFHLYHG VHUYLFH TXDOLW\ RU VDWLVIDFWLRQf DQG ZDLWLQJ WLPH $GGLWLRQDOO\ LW KDV EHHQ VXJJHVWHG WKDW WKH UHODWLRQVKLS EHWZHHQ ZDLWLQJ WLPHV DQG SHUFHLYHG VHUYLFH TXDOLW\ PD\ EH QRQOLQHDU 'DYLV /DUVRQ f +\SRWKHVLV ILYH +f HYDOXDWHV WKH UHODWLRQVKLS EHWZHHQ VHUYLFH WLPHV DQG HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ IRU JHQHUDOL]DELOLW\ WR WKH GUXJ LQIRUPDWLRQ VHWWLQJ +D 7KHUH LV D VLJQLILFDQW LQYHUVH UHODWLRQVKLS EHWZHHQ HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 362f DQG DFWXDO VHUYLFH WLPH +E 7KHUH LV D QRQOLQHDU UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DQG SHUFHLYHG VHUYLFH TXDOLW\ 362f 7KH OLWHUDWXUH RIWHQ GRHV QRW PDNH D GLVWLQFWLRQ EHWZHHQ ZDLWLQJ WLPHV DQG GHOD\V LQ VHUYLFH KRZHYHU ERWK KDYH EHHQ VKRZQ WR HIIHFW SHUFHLYHG VHUYLFH TXDOLW\ $ GHOD\ RFFXUV ZKHQHYHU WKH DFWXDO UHVSRQVH WLPH LV ORQJHU WKDQ WKH SURPLVHG WLPH 7D\ORU Df DQG 7D\ORU DQG &OD[WRQ f UHSRUWHG WKDW VXEMHFWV ZKR H[SHULHQFHG ERDUGLQJ GHOD\V LQ DQ DLUSRUW VHWWLQJ KDG ORZHU HYDOXDWLRQV RI RYHUDOO VHUYLFH WKDQ QRQGHOD\HG VXEMHFWV +RZHYHU WKLV HIIHFW PD\ EH PRGHUDWHG E\ WKH GHJUHH RI ILOOHG WLPH DQG FRQVXPHUVn SHUFHSWLRQV UHJDUGLQJ KRZ PXFK FRQWURO WKH VHUYLFH SURYLGHU KDG RYHU WKH GHOD\ 6LPLODUO\ 'XEHn5LRX[ f XVLQJ UROHSOD\LQJ VFHQDULRV IRU D UHVWDXUDQW VHWWLQJ DOVR IRXQG WKDW GHOD\V FRXOG DIIHFW FXVWRPHU VDWLVIDFWLRQ KRZHYHU WKH UHVHDUFK VXJJHVWHG WKDW SHUFHLYHG QHHG DQG WLPLQJ RI WKH GHOD\V ZHUH DOVR LPSRUWDQW %DVHG RQ WKLV UHVHDUFK

PAGE 52

K\SRWKHVLV VL[ +f DVVHVVHV WKH DVVRFLDWLRQ EHWZHHQ GHOD\V LQ VHUYLFH DQG HYDOXDWLRQV RI 364 + 'HOD\V LQ VHUYLFH DUH QHJDWLYHO\ UHODWHG WR HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f $GGLWLRQDOO\ WKHUH LV OLWWOH LQIRUPDWLRQ LQ WKH VHUYLFH TXDOLW\ RU VDWLVIDFWLRQ OLWHUDWXUH FRQFHUQLQJ WKH UHODWLRQVKLS EHWZHHQ DFWXDO DQG SHUFHLYHG VHUYLFH WLPH WKHUHIRUH LW LV XQFOHDU WR ZKDW GHJUHH WKH\ DUH DFWXDOO\ DVVRFLDWHG +RZHYHU VHYHUDO DXWKRUV GR VXJJHVW WKDW SHUFHLYHG VHUYLFH WLPH PD\ EH PRUH LPSRUWDQW WKDW DFWXDO VHUYLFH WLPH LQ UHODWLRQ WR SHUFHLYHG VHUYLFH TXDOLW\ +RUQLN .DW] /DUVRQ DQG /DUVRQ 7D\ORU Df +\SRWKHVLV VHYHQ +f H[SORUHV WKH DVVRFLDWLRQV DPRQJ WKH YDULDEOHV PHDVXULQJ SHUFHLYHG VHUYLFH WLPH DFWXDO VHUYLFH WLPH DQG VHUYLFH GHOD\V +D 7KHUH LV D SRVLWLYH UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DQG SHUFHLYHG VHUYLFH WLPH +E 7KHUH LV D SRVLWLYH UHODWLRQVKLS EHWZHHQ VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH WLPH 7KH SUHYLRXV FKDSWHU GLVFXVVHG ZD\V LQ ZKLFK WKH SHUIRUPDQFH RI D V\VWHP FRXOG EH LPSURYHG ,W LV EHOLHYHG WKDW E\ FKDQJLQJ VHUYLFH FDSDFLW\ LQ ZD\V WKDW GHFUHDVH WKH RYHUDOO VHUYLFH WLPHV DQG WKH QXPEHU RI VHUYLFH GHOD\V FRQVXPHU HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ PD\ EH LPSURYHG 8VLQJ D TXHXLQJ WKHRU\ IUDPHZRUN WR RSWLPL]H VHUYLFH FDSDFLW\ VXJJHVWV WKDW ZH VKRXOG H[DPLQH WKH HIIHFWV RI FKDQJHV LQ VWDIILQJ OHYHOV Vf DQG VHUYLFH UDWHV Sf 6SHFLILF UHVHDUFK TXHVWLRQV RQH WKURXJK WKUHH 5O 5 DQG 5f DVVHVV WKH UHODWLYH LPSDFW DQG VHQVLWLYLW\ RI WKHVH PHWKRGV LQ WHUPV RI LPSURYHPHQWV LQ VHUYLFH WLPHV DQG VHUYLFH GHOD\V

PAGE 53

5O +RZ GR FKDQJHV LQ VWDIILQJ OHYHOV DQG VHUYLFH UDWHV LPSDFW VLPXODWHG VHUYLFH WLPHV LQ WKH GUXJ LQIRUPDWLRQ VHUYLFH" 5 :KDW FRPELQDWLRQ RI FKDQJHV LQ VWDIILQJ OHYHOV VHUYLFH UDWHV RSWLPL]HV WKH V\VWHP IRU GHOD\V LQ VHUYLFH ZKHQ FRPSDUHG DJDLQVW VHUYLFH TXDOLW\ DQG FRVW LQ WKH GUXJ LQIRUPDWLRQ VHUYLFH" 5 +RZ VHQVLWLYH LV WKLV VROXWLRQ WR UDQGRP YDULDWLRQ LQ WKH V\VWHP YDULDEOHV HJ DUULYDO UDWHf"

PAGE 54

,13876 $UURZV UHODWLQJ WR VWXG\ K\SRWKHVHV DQG VSHFLILF UHVHDUFK TXHVWLRQV DUH ODEHOOHG ZLWK WKH QXPEHU RI WKH K\SRWKHVLV +f RU UHVHDUFK TXHVWLRQ 5f $UURZ GLUHFWLRQ GRHV QRW QHFHVVDULO\ ,PSO\ FDXVDOLW\ )LJXUH +\SRWKHVL]HG )UDPHZRUN

PAGE 55

&+$37(5 0(7+2'6 2YHUYLHZ 7KLV FKDSWHU GHVFULEHV WKH PHWKRGV XVHG WR WHVW WKH UHVHDUFK K\SRWKHVHV DQG H[SORUH WKH VSHFLILF UHVHDUFK TXHVWLRQV SUHVHQWHG LQ WKH SUHYLRXV FKDSWHU ,W LQFOXGHV D GHVFULSWLRQ RI WKH VWXG\ ORFDWLRQ WKH VRXUFHV RI GDWD WKH VDPSOH VHOHFWLRQ SURFHGXUHV XVHG LQ WKH VWXG\ DQG PHWKRGV RI GDWD FROOHFWLRQ ,Q DGGLWLRQ LW GHVFULEHV WKH WHFKQLTXHV XVHG WR GHYHORS DQG YDOLGDWH WKH VHUYLFH TXDOLW\ TXHVWLRQQDLUH DQG VLPXODWLRQ SURJUDP ,W FRQFOXGHV ZLWK D GHVFULSWLRQ RI WKH GDWD DQDO\VLV SURFHGXUHV XVHG WR WHVW WKH K\SRWKHVHV DQG UHVHDUFK TXHVWLRQV 6WXG\ /RFDWLRQ 7KLV VWXG\ ZDV FRQGXFWHG DW WKH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVRXUFH &HQWHU ',35&f DW 6KDQGV DW WKH 8QLYHUVLW\ RI )ORULGD LQ *DLQHVYLOOH )ORULGD 7KH ',35& DFFHSWV GUXJ LQIRUPDWLRQ TXHVWLRQV IURP SUDFWLWLRQHUV IURP DOO RYHU WKH 1RUWK )ORULGD UHJLRQ 7KH ',35& DFFHSWV FDOOV RQO\ IURP SUDFWLWLRQHUV H J SKDUPDFLVWV SK\VLFLDQV QXUVHV ODZ HQIRUFHPHQW HWFf &DOOV IURP WKH SXEOLF DUH UHGLUHFWHG WR RWKHU UHVRXUFHV 7KH ',35& FDWHJRUL]HV FDOOHUV LQWR WKUHH FDWHJRULHV f VXEVFULEHUV f QRQVXEVFULEHUV DQG f 8QLYHUVLW\ RI )ORULGD +HDOWK 6\VWHP HPSOR\HHV 7KH VHUYLFH LV SURYLGHG IUHH RI FKDUJH WR DOO FDOOHUV KRZHYHU VXEVFULEHUV SD\ D YROXQWDU\ PHPEHUVKLS IHH WR KHOS VXSSRUW WKH ',35& 4XHVWLRQV DUH SUHVHQWHG WR WKH ',35& IURP YDULRXV VRXUFHV LQFOXGLQJ WHOHSKRQH IDFVLPLOH ID[f HOHFWURQLF PDLO DQG WKURXJK LQSHUVRQ YLVLWV WR WKH FHQWHU +RZHYHU WKH

PAGE 56

YDVW PDMRULW\ RI FDOOV DUH SUHVHQWHG WR WKH FHQWHU YLD WHOHSKRQH 7KH ',35& FODVVLILHV TXHVWLRQV LQWR JHQHUDO FDWHJRULHV f GUXJ DYDLODELOLW\ f GUXJ GRVDJH DQG DGPLQLVWUDWLRQ f GUXJ LGHQWLILFDWLRQ f GUXJ LQWHUDFWLRQV f GUXJ WKHUDS\ DQG HIILFDF\ f GUXJ XVH LQ SUHJQDQF\ RU ODFWDWLRQ f LQYHVWLJDWLRQDO GUXJV f ,9 FRPSDWLELOLW\ RU VWDELOLW\ f OHJDO f RWKHU f SKDUPDFRNLQHWLFV f VLGH HIIHFWV RU DGYHUVH HIIHFWV f WR[LFRORJ\ DQG f YHWHULQDU\ GUXJV %HVLGHV WKH QDWXUH RI WKH TXHVWLRQ VXEVFULSWLRQ VWDWXV DQG SURIHVVLRQ FDOOHUV DUH DVNHG WR SURYLGH YDULRXV GHPRJUDSKLF LQIRUPDWLRQ VXFK DV QDPH DGGUHVV SKRQH QXPEHU DQGRU ID[ QXPEHU &DOOHUV DUH DOVR DVNHG IRU WKH DPRXQW WLPH WKDW WKH\ FDQ DOORZ WKH FHQWHU WR UHVHDUFK WKH TXHVWLRQ 7KH ',35& FDWHJRUL]HV WKHVH WLPHV LQWR IRXU FDWHJRULHV f ZLWKLQ PLQXWHV VWDWf f ZLWKLQ WKH GD\ WRGD\f f E\ D VSHFLILF GDWH GDWHf DQG f QR UXVK &DOOHUV PD\ UHTXHVW DQ RUDO UHVSRQVH WR WKHLU TXHVWLRQV D ZULWWHQ UHVSRQVH RU ERWK 7KH ',35& LV XVXDOO\ VWDIIHG E\ WKUHH 3KDUP' VWXGHQWV GXULQJ WKHLU FOHUNVKLS URWDWLRQV 6RPHWLPHV LQWHUQVKLS VWXGHQWV YLVLWLQJ LQWHUQDWLRQDO VWXGHQWV DQG KRVSLWDO SKDUPDF\ UHVLGHQWV ZLOO DVVLVW WKH ',35& LQ DQVZHULQJ TXHVWLRQV +RZHYHU WKH ',35& GRHV QRW XVXDOO\ NQRZ ZKHQ DGGLWLRQDO VWXGHQWV RU UHVLGHQWV ZLOO EHFRPH DYDLODEOH ,Q DGGLWLRQ WKH OHYHO RI FRQWULEXWLRQ WKDW WKH\ SURYLGH LV RIWHQ OLPLWHG DQG XQSUHGLFWDEOH $ GUXJ LQIRUPDWLRQ UHVLGHQW LV DOVR XVXDOO\ SUHVHQW LQ WKH ',35& KRZHYHU WKLV UHVLGHQW KDV YDU\LQJ GXWLHV DQG GRHV QRW XVXDOO\ VSHQG WKHLU WLPH GLUHFWO\ DQVZHULQJ TXHVWLRQV 7ZR FRGLUHFWRUV PDQDJH WKH VHUYLFHfV RSHUDWLRQ DQG DSSURYH VWXGHQW UHVSRQVHV WR LQIRUPDWLRQ UHTXHVWV 7KH GUXJ LQIRUPDWLRQ UHVLGHQW FDQ DOVR DSSURYH VWXGHQW UHVSRQVHV RQFH KH RU VKH LV TXDOLILHG DQG H[SHULHQFHG DV GHWHUPLQHG E\ WKH FRn GLUHFWRUV

PAGE 57

'DWD 6RXUFHV 6DPSOH 6HOHFWLRQ DQG 'DWD &ROOHFWLRQ 3URFHGXUHV 7KH GDWD IRU WKH HPSLULFDO SDUWV RI WKLV VWXG\ ZHUH REWDLQHG E\ QRQH[SHULPHQWDO PHWKRGV 7KH SDUWV RI WKLV VWXG\ LQYROYLQJ FRPSXWHU VLPXODWLRQ ZHUH H[SHULPHQWDO LQ GHVLJQ 'DWD IRU WKLV VWXG\ ZDV FROOHFWHG IURP VL[ VRXUFHV 7KH ILUVW WZR ZHUH WDNHQ IURP KLVWRULFDO GDWD VRXUFHV LH KLVWRULFDO GDWD VKHHWV DQG D GDWDEDVH GRFXPHQWLQJ WKH SDVW PRQWKO\ ZRUNORDGf 7KH WKLUG IRXUWK DQG ILIWK GDWD VRXUFHV ZHUH FROOHFWHG FRQFXUUHQWO\ GXULQJ WKH GDWD FROOHFWLRQ SHULRG IURP -XQH WKURXJK -XO\ LH VSHFLDOO\ GHVLJQHG GDWD FROOHFWLRQ IRUPV SHUVRQDO LQWHUYLHZV DQG VHUYLFH TXDOLW\ TXHVWLRQQDLUHVf 7KH ODVW VRXUFH RI GDWD ZDV REWDLQHG IURP FRPSXWHU UXQV RI WKH VLPXODWLRQ SURJUDP +LVWRULFDO 'DWD 6KHHWV 7KH KLVWRULFDO GDWD VKHHWV DUH WKH VWDQGDUG IRUPV WKDW WKH ',35& XVHV WR GRFXPHQW UHVSRQVHV WR FDOOHU TXHVWLRQV DQG LQIRUPDWLRQ UHTXHVWV $SSHQGL[ +f (YHU\ TXHVWLRQ DQVZHUHG E\ WKH ',35& LV UHFRUGHG RQWR RQH RI WKHVH GDWD VKHHWV ,Q DGGLWLRQ DOO LQIRUPDWLRQ UHJDUGLQJ D SDUWLFXODU FDOO LV ZULWWHQ RQ RU DWWDFKHG WR WKHVH GDWD VKHHWV $OO RI WKH DUFKLYHG GDWD VKHHWV ZHUH UHWULHYHG IRU 6HSWHPEHU WKURXJK 0D\ DSSUR[LPDWHO\ QLQH PRQWKVf UHVXOWLQJ LQ D WRWDO KLVWRULFDO GDWD VKHHW VDPSOH VL]H RI ,QIRUPDWLRQ WDNHQ IURP WKH KLVWRULFDO GDWD VKHHWV LQFOXGHG WKH ILOH QXPEHU DQG WKH UHTXHVWRUnV QDPH SURIHVVLRQ DQG VXEVFULSWLRQ VWDWXV ,Q DGGLWLRQ WKH TXHVWLRQ W\SH ZKHQ WKH UHVSRQVH ZDV QHHGHG WKH UHVSRQVH W\SH UHTXHVWHG WKH GDWH DQG WLPH UHFHLYHG DQG WKH GDWH DQG WLPH FRPSOHWHG ZHUH UHFRUGHG )URP WKH GDWD VKHHWV LW ZDV DOVR SRVVLEOH WR GHWHUPLQH ZKHWKHU RU QRW WKH VHUYLFH ZDV GHOD\HG SDVW WKH WLPH QHHGHG DQG ZKHWKHU RU QRW WKH VDPH SHUVRQ ZKR DQVZHUHG WKH FDOO DOVR FRPSOHWHG WKH DQVZHU $OO GDWD ZHUH HQWHUHG LQWR D 0LFURVRIW ([FHO GDWDEDVH

PAGE 58

+LVWRULFDO 'DWDEDVH $ KLVWRULFDO GDWDEDVH PDLQWDLQHG E\ RQH RI WKH FRGLUHFWRUV RI WKH ',35& GRFXPHQWLQJ WKH PRQWKO\ ZRUNORDG VLQFH ZDV XVHG WR DQDO\]H PRQWKO\ DUULYDOV IRU VHDVRQDO WUHQGV 7KH GDWD FRQVLVWHG RI D WRWDO RI GDWD SRLQWV LH HDFK IRU -DQXDU\ WKURXJK -XQH DQG HDFK IRU -XO\ WKURXJK 'HFHPEHUf 7KLV GDWDEDVH FRQWDLQHG WKH WRWDO QXPEHU RI TXHVWLRQV DQVZHUHG HDFK PRQWK DV ZHOO DV WKH DYHUDJH GDLO\ QXPEHU RI TXHVWLRQV DQVZHUHG IRU WKH PRQWK 7KH DYHUDJH GDLO\ QXPEHU RI TXHVWLRQV ZDV SURGXFHG E\ GLYLGLQJ WKH WRWDO QXPEHU RI TXHVWLRQV DQVZHUHG GXULQJ WKH PRQWK E\ WKH QXPEHU RI GD\V WKH ',35& ZDV RSHQ WR WDNH FDOOV 7KLV GDWD ZDV PDGH DYDLODEOH WR WKH SULQFLSDO LQYHVWLJDWRU LQ VSUHDGVKHHW IRUPDW 'DWD &ROOHFWRQ )RUPV $ VSHFLDO IRUP ZDV GHVLJQHG WR IDFLOLWDWH WKH VSHFLILF GDWD FROOHFWLRQ QHHGV RI WKLV VWXG\ 7KLV IRUP ZDV YHU\ VLPLODU WR KLVWRULFDO GDWD VKHHWV GHVFULEHG DERYH KRZHYHU PRGLILFDWLRQV ZHUH PDGH WR LQFRUSRUDWH VSDFH IRU DGGLWLRQDO LQIRUPDWLRQ VXFK DV WKH UHFRUGLQJ RI GDWHV DQG WLPHV IRU VSHFLILF VHUYLFH DFWLYLWLHV $SSHQGL[ ,f 7KHVH GDWD FROOHFWLRQ IRUPV ZHUH FROOHFWHG VHYHUDO WLPHV HDFK ZHHN IURP -XQH WKURXJK -XO\ UHVXOWLQJ LQ D WRWDO VDPSOH RI IRUPV %\ FRPSDULQJ WKH ILOH QXPEHUV IRU WKH GDWD FROOHFWLRQ IRUPV REWDLQHG DJDLQVW D VHSDUDWH HQWU\ ORJ LW ZDV GHWHUPLQHG WKDW GDWD FROOHFWLRQ IRUPV ZHUH PLVVLQJ DQG FRXOG QRW EH ORFDWHG 7KHUHIRUH RYHU b RI WKH GDWD FROOHFWLRQ IRUPV ILOOHG RXW GXULQJ WKLV SHULRG ZHUH ORFDWHG DQG HQWHUHG ,QIRUPDWLRQ WDNHQ IURP WKH GDWD FROOHFWLRQ IRUPV LQFOXGHG 7KH ILOH QXPEHU 7KH UHTXHVWRUnV QDPH DQG FRQWDFW LQIRUPDWLRQ WKH UHTXHVWRUnV SURIHVVLRQ DQG WKH UHTXHVWRUnV VXEVFULSWLRQ VWDWXV 7KH TXHVWLRQ W\SH

PAGE 59

7KH WLPH WKH UHVSRQVH ZDV QHHGHG DQG WKH UHVSRQVH W\SH UHTXHVWHG 7KH GDWH DQG WLPHV IRU HDFK RI WKH IRXU ZRUN DFWLYLWLHV LH WDNH FDOO UHVHDUFK DQVZHU DSSURYH DQVZHU DQG UHWXUQ DQVZHU WR FDOOHUf :KHWKHU RU QRW WKH VHUYLFH ZDV GHOD\HG SDVW WKH WLPH QHHGHG 7KH SHUVRQV DSSURYLQJ DQG FRPSOHWLQJ WKH DQVZHU 7KH QXPEHU RI SHUVRQV ZRUNLQJ RQ WKH TXHVWLRQ ,Q DGGLWLRQ WR WKH LQIRUPDWLRQ PHQWLRQHG DERYH DYDLODEOH IURP DQDO\]LQJ WKH KLVWRULFDO GDWD VKHHWV WKH DFWLYLW\ EDVHG VHUYLFH WLPHV RQ WKH GDWD FROOHFWLRQ IRUPV DOORZHG IRU D PRUH GHWDLOHG DQDO\VLV RI WKH VHUYLFH SURFHVVHV RI WKH ',35& 7KLV SURYLGHG LQIRUPDWLRQ QHFHVVDU\ WR WKH GHYHORSPHQW RI D PRUH UREXVW VLPXODWLRQ $OO GDWD ZHUH HQWHUHG LQWR D VSHFLDO 0LFURVRIW $FFHVV GDWDEDVH VSHFLILFDOO\ GHVLJQHG IRU WKLV SURMHFW 3HUVRQDO ,QWHUYLHZV 3HUVRQDO LQWHUYLHZV ZLWK ',35& FRGLUHFWRUV DQG H[WHUQVKLS VWXGHQWV ZHUH FRQGXFWHG LQ RUGHU WR REWDLQ D PRUH WKRURXJK XQGHUVWDQGLQJ RI WKH YDULRXV DVSHFWV RI WKH FHQWHUfV RSHUDWLRQV LQFOXGLQJ WKH SURFHVV IRU DQVZHULQJ TXHVWLRQV DQG WKH SULRULW\ GLVFLSOLQH XVHG E\ WKH VWXGHQWV WR RUJDQL]H ZRUN 7ZHOYH SHUVRQDO LQWHUYLHZV ZHUH FRQGXFWHG IURP 0D\ WR -XO\ 7KH LQWHUYLHZHHV FRQVLVWHG RI WHQ 3KDUP VWXGHQWV DQG WKH WZR FRGLUHFWRUV IRU WKH ',35& 7KH VWXGHQWV ZHUH DOO LQWHUYLHZHG GXULQJ WKHLU WKLUG URWDWLRQ ZHHN 7KH LQWHUYLHZV ZHUH VHPLVWUXFWXUHG XVLQJ LQWHUYLHZ RXWOLQHV WR PDLQWDLQ FRQVLVWHQF\ $SSHQGLFHV DQG .f 7KH LQWHUYLHZV ZHUH DXGLR UHFRUGHG IRU DFFXUDF\ RI UHFDOO 7KH LQWHUYLHZ RXWOLQHV ZHUH GHYHORSHG EDVHG RQ VXJJHVWLRQV PDGH E\ 6WHZDUW DQG &DVK f

PAGE 60

6HUYLFH 4XDOLW\ 4XHVWLRQQDLUHV 7KH IRXUWK GDWD VRXUFH XVHG ZDV D VHUYLFH TXDOLW\ TXHVWLRQQDLUH VHQW WR SUDFWLWLRQHUV XVLQJ WKH VHUYLFH IURP -XQH WKURXJK -XO\ 7KLV TXHVWLRQQDLUH ZDV DGPLQLVWHUHG WR FDOOHUV DIWHU VHUYLFH FRPSOHWLRQ $SSHQGL[ 0f 7KLV TXHVWLRQQDLUH DVVHVVHG SHUFHLYHG VHUYLFH TXDOLW\ 364f XVLQJ D PRGLILHG 6(593(5) VFDOH SHUFHLYHG VHUYLFH WLPH SHUFHLYHG RYHUDOO VHUYLFH TXDOLW\ 264f DQG EHKDYLRUDO LQWHQWLRQ 7KH LQFOXVLRQ FULWHULRQ IRU UHFHLYLQJ D TXHVWLRQQDLUH ZDV DQ\ SUDFWLWLRQHU ZKR VXEPLWWHG D GUXJ LQIRUPDWLRQ TXHVWLRQ WR WKH ',35& GXULQJ WKH VWXG\ SHULRG &DOOHUV ZHUH H[FOXGHG IURP WKLV SRUWLRQ RI WKH VWXG\ LI WKH\ KDG DOUHDG\ EHHQ VHQW D TXHVWLRQQDLUH LH FDOOHUV ZHUH QRW VXUYH\HG PRUH WKDQ RQFHf 2XW RI WKH DYDLODEOH VDPSOHV TXHVWLRQQDLUHV ZHUH VHQW RXW WR SUDFWLWLRQHUV VDPSOHV ZHUH UHSHDW FDOOHUV ZKR KDG DOUHDG\ EHHQ VHQW D TXHVWLRQQDLUH DQG FRQWDFW LQIRUPDWLRQ ZDV QRW DYDLODEOH IRU RI WKH FDOOHUV $ GDWD FROOHFWLRQ SURFHGXUH EDVHG RQ WKH 7RWDO 'HVLJQ 0HWKRG GHYHORSHG E\ 'LOOPDQ f ZDV XVHG WR PD[LPL]H WKH UHVSRQVH UDWH RI WKH TXHVWLRQQDLUH 7KHUH ZHUH WKUHH SKDVHV WR WKLV SURFHGXUH )LUVW HOLJLEOH VXEMHFWV LGHQWLILHG IURP WKH GDWD FROOHFWLRQ IRUPV ZHUH VHQW D TXHVWLRQQDLUH $SSHQGL[ 0f DORQJ ZLWK D FRYHU OHWWHU IURP WKH FRGLUHFWRU DQG WKH SULQFLSDO LQYHVWLJDWRU H[SODLQLQJ WKH SXUSRVH RI WKH UHVHDUFK DQG DVNLQJ IRU WKH VXEMHFWVn SDUWLFLSDWLRQ $SSHQGL[ /f $ VHOI DGGUHVVHG VWDPSHG HQYHORSH ZDV DOVR HQFORVHG IRU WKH VXEMHFW WR UHWXUQ WKH VXUYH\ 6HFRQG DSSUR[LPDWHO\ RQH ZHHN DIWHU WKH TXHVWLRQQDLUHV ZHUH PDLOHG D UHPLQGHU SRVW FDUG $SSHQGL[ 1f ZDV VHQW WR QRQn UHVSRQGHUV DVNLQJ VXEMHFWV WR ILOORXW DQG VHQG LQ WKH TXHVWLRQQDLUH RU WR FRQWDFW WKH SULQFLSDO LQYHVWLJDWRU LI WKH\ GLG QRW UHFHLYH D TXHVWLRQQDLUH 6XEMHFWV ZKR UHSOLHG WR WKH SRVW FDUG VD\LQJ WKDW WKH\ QHYHU UHFHLYHG D TXHVWLRQQDLUH ZHUH SURPSWO\ VHQW DQRWKHU 7KLUG DSSUR[LPDWHO\ WZR ZHHNV DIWHU WKH ILUVW TXHVWLRQQDLUH ZDV VHQW DQ DWWHPSW ZDV PDGH WR FRQWDFW HDFK QRQUHVSRQGLQJ VXEMHFW E\ WHOHSKRQH ,I WKH VXEMHFW ZDV UHDFKHG WKH LQWHUYLHZHU DWWHPSWHG WR DVFHUWDLQ WKH UHDVRQ IRU WKH QRQUHVSRQVH DQG LI VWLOO

PAGE 61

DSSOLFDEOH WKH VXEMHFW ZDV UHPLQGHG WR VHQG LQ WKH TXHVWLRQQDLUH ,I WKH VXEMHFW ZDV QRW UHDFKHG DQ DWWHPSW ZDV PDGH WR OHDYH D PHVVDJH RU UHPLQGHU WKURXJK D UHFHSWLRQLVW RU FRZRUNHU $OO RI WKH SUHWHVW DQG PDLQ TXHVWLRQQDLUHV ZHUH GHYHORSHG XVLQJ WKH 6XUYH\ 3UR IRU :LQGRZV VRIWZDUH SURJUDP 7KLV SURJUDP DOVR SURYLGHV D IDFLOLW\ IRU GDWD HQWU\ DQG H[SRUW ZKLFK ZDV XVHG WR UHFRUG WKH UHVSRQVHV WR WKH TXHVWLRQQDLUHV IRU IXWXUH DQDO\VLV 5HVSRQVHV WR WKH ILQDO TXHVWLRQ LH 4f DVNLQJ IRU DGGLWLRQDO FRPPHQWV ZHUH UHFRUGHG LQWR D ZRUG SURFHVVLQJ GRFXPHQW DORQJ ZLWK DOO RWKHU FRPPHQWV ZULWWHQ QH[W WR WKH LQGLYLGXDO LWHPV 6LPXODWLRQ 5XQV 7KH VL[WK GDWD VRXUFH XVHG ZDV WKH RXWSXW JHQHUDWHG E\ WKH VLPXODWLRQ SURJUDP 7KH VLPXODWLRQ PRGHO ZDV FRQVWUXFWHG XVLQJ WKH *366+ *HQHUDO 3XUSRVH 6LPXODWLRQ 6\VWHPf VLPXODWLRQ ODQJXDJH SURGXFHG E\ :ROYHULQH 6RIWZDUH IRU XVH RQ DQ 06'26 EDVHG SHUVRQDO FRPSXWHU 5HVXOWV IURP WKH VLPXODWLRQ UXQV ZHUH HQWHUHG LQWR D GDWDEDVH IRU WKH SXUSRVHV RI VXPPDUL]DWLRQ DQG VWDWLVWLFDO DQDO\VLV )LYH VHSDUDWH JURXSV RI VLPXODWLRQ UXQV ZHUH PDGH 7KH ILUVW WKUHH UXQV ZHUH XVHG WR YHULI\ DQG YDOLGDWH WKH VLPXODWLRQ SURJUDP 7KH IRXUWK UXQ ZDV XVHG WR WHVW VSHFLILF UHVHDUFK TXHVWLRQV RQH DQG WZR 7KH ILQDO UXQ ZDV XVHG WR GHWHUPLQH WKH VHQVLWLYLW\ RI WKH RSWLPDO VROXWLRQ DV GHVFULEHG E\ WKH WKLUG VSHFLILF UHVHDUFK TXHVWLRQ 3URFHGXUHV IRU 3URWHFWLQJ 3ULYDF\ DQG &RQILGHQWLDOLW\ (DFK UHFRUG RI LQIRUPDWLRQ JDWKHUHG IURP WKH KLVWRULFDO GDWD VKHHWV RU FRQFXUUHQW GDWD FROOHFWLRQ IRUPV ZDV FRGHG ZLWK DQ LGHQWLILFDWLRQ QXPEHU )RU WUDFNLQJ SXUSRVHV RQO\ VXUYH\V VHQW RXW XVLQJ LQIRUPDWLRQ FROOHFWHG IURP WKH GDWD FROOHFWLRQ IRUPV ZHUH DOVR FRGHG ZLWK DQ LGHQWLILFDWLRQ QXPEHU 2QFH DOO PDLO TXHVWLRQQDLUHV ZHUH UHWXUQHG DOO SRVW

PAGE 62

FDUG DQG WHOHSKRQH IROORZXSV FRPSOHWHG DQG DOO GDWD HQWHUHG DQG YHULILHG WKH SRUWLRQ RI WKH GDWDEDVH FRQWDLQLQJ WKH FDOOHUVn FRQWDFW LQIRUPDWLRQ ZDV GHOHWHG )XUWKHUPRUH ZULWWHQ FRPPHQWV WUDQVFULEHG IURP WKH TXHVWLRQQDLUHV ZHUH HGLWHG WR H[FOXGH DQ\ SHUVRQDO UHIHUHQFHV $OO KLVWRULFDO GDWD VKHHWV ZHUH UHWXUQHG WR WKH ',35& RQFH GDWD HQWU\ ZDV FRPSOHWHG +DUG FRSLHV NHSW RI WKH GDWD FROOHFWLRQ IRUPV ZHUH FRGHG ZLWK WKH LGHQWLILFDWLRQ QXPEHU DQG SHUVRQDO LQIRUPDWLRQ FRQWDLQHG RQ WKHVH KDUG FRSLHV ZDV PDVNHG XVLQJ SHUPDQHQW PDUNHU ,Q DGGLWLRQ DQ\ FRPPHQWV RU TXRWHV XVHG IURP WKH SHUVRQDO LQWHUYLHZV ZHUH HGLWHG VR WKDW WKH\ FRXOG QRW EH WUDFHG WR WKH VSHDNHU 7KLV SURMHFW ZDV UHYLHZHG E\ WKH +HDOWK &HQWHU ,QVWLWXWLRQDO 5HYLHZ %RDUG DW WKH 8QLYHUVLW\ RI )ORULGD DQG DSSURYHG RQ -XO\ 6DPSOH 6L]H &DOFXODWLRQV 5HTXLUHG 1XPEHU RI 'DWD 6KHHWV ,Q RUGHU WR HVWLPDWH WKH VDPSOH VL]HV QHFHVVDU\ WR SURYLGH DFFXUDWH HVWLPDWHV RI WKH DUULYDO DQG VHUYLFH WLPH GLVWULEXWLRQV UHTXLUHG WR FRQVWUXFW WKH VLPXODWLRQ RI WKH ',35& WKH GDWD VKHHWV IURP DSSUR[LPDWHO\ WKH ILUVW ZHHN RI GDWD FROOHFWLRQ ZHUH FRPSLOHG DQG DQDO\]HG 7KH QHFHVVDU\ VDPSOH VL]HV ZHUH FDOFXODWHG XVLQJ WKH PHWKRG SURSRVHG E\ 0HQGHQKDOO :DFNHUO\ DQG 6FKHDIIHU f IRU HVWDEOLVKLQJ D ODUJH VDPSOH b FRQILGHQFH LQWHUYDO IRU D JLYHQ VWDQGDUG GHYLDWLRQ DQG HUURU RI HVWLPDWLRQ 7KH VDPSOH VL]H FDOFXODWLRQV DUH VXPPDUL]HG LQ 7DEOH 7KH ODUJHVW RI WKHVH HVWLPDWHV LV WKH VDPSOHV UHTXLUHG IRU D b FRQILGHQFH LQWHUYDO XVLQJ D VWDQGDUG GHYLDWLRQ RI PLQXWHV DQG DQ HUURU RI HVWLPDWLRQ RI PLQXWHV 6LQFH WKH ',35& UHFHLYHV EHWZHHQ WR FDOOV SHU PRQWK WZR PRQWKV RI GDWD ZHUH FROOHFWHG IRU WKLV VWXG\ LQ RUGHU WR VDWLVI\ WKLV VDPSOH VL]H UHTXLUHPHQW 7KLV WLPH SHULRG UHVXOWHG LQ DQ DFWXDO VDPSOH RI GDWD VKHHWV ZKLFK SURYLGHG D FRQVHUYDWLYHO\ ODUJH

PAGE 63

VDPSOH IURP ZKLFK WR HVWLPDWH WKH UHTXLUHG SDUDPHWHUV DQG DOORZ FRQVLGHUDWLRQV IRU PLVVLQJ GDWD 7DEOH 5HTXLUHG 6DPSOH 6L]HV IRU 6HOHFWHG 6\VWHP 3DUDPHWHUV 3DUDPHWHU 0HDQ 6W 'HY (UURU RI (VWLPDWLRQ 5HTXLUHG 6DPSOH 6L]H D f ,QWHUDUULYDOV PLQ PLQ PLQ 5HFHSWLRQ RI &DOO PLQ PLQ PP 6HUYLFH 7LPH PLQ PLQ PLQ r $SSURYDO PLQ PP PP 5HWXUQ $QVZHU PLQ PLQ PLQ r /DUJHVW VDPSOH VL]H UHTXLUHG 5HTXLUHG 1XPEHU RI 4XHVWLRQQDLUHV 7KH SULPDU\ VWDWLVWLFDO PHWKRGV XVHG LQ WHVWLQJ UHODWLRQVKLSV LQYROYLQJ WKH LWHPV LQ WKH VHUYLFH TXDOLW\ LQVWUXPHQW ZHUH FRUUHODWLRQ +,WKURXJK + + DQG +f DQG OLQHDU UHJUHVVLRQ +f $V VXFK WKHUH ZHUH WZR FRQVLGHUDWLRQV GULYLQJ WKH VDPSOH VL]H GHWHUPLQDWLRQ IRU WKH VHUYLFH TXDOLW\ TXHVWLRQQDLUHV 3ULPDU\ FRQVLGHUDWLRQ ZDV JLYHQ WR WKH K\SRWKHVHV WR EH WHVWHG 6HFRQGDU\ FRQVLGHUDWLRQ ZDV JLYHQ WR WKH UHTXLUHG QXPEHU RI GDWD SRLQWV QHFHVVDU\ WR FRQGXFW SULQFLSDO FRPSRQHQWV IDFWRU DQDO\VLV RQ WKH 6(593(5-) SRUWLRQ RI WKH VXUYH\ $V UHFRPPHQGHG E\ 6DZ\HU DQG %DOO f WKH W\SH HUURU UDWH IRU VDPSOH VL]H FDOFXODWLRQV ZDV VHW DW D DQG WKH W\SH ,, HUURU UDWH ZDV VHW DW ZKHUH SRZHU HTXDOV f 7KH UDWLRQDOH IRU WKHVH HUURU UDWHV LV EDVHG RQ WKH DVVXPSWLRQ WKDW IRU WKLV VWXG\ FRPPLWWLQJ D W\SH HUURU ZDV PRUH FULWLFDO WKDQ FRPPLWWLQJ D W\SH ,, HUURU 7KHUHIRUH D DQG 3 ZHUH VHOHFWHG VR WKDW RQO\ D VPDOO FKDQFH H[LVWHG WKDW WKH QXOO K\SRWKHVLV ZRXOG EH UHMHFWHG ZKHQ QR WUXH GLIIHUHQFHV H[LVW DQG D UHDVRQDEO\ KLJK SUREDELOLW\ RI UHMHFWLQJ WKH QXOO K\SRWKHVLV ZKHQ GLIIHUHQFHV GR H[LVW )LUVW +, WKURXJK + DQG + WKURXJK + XVHG FRUUHODWLRQ DV WKH SULPDU\ VWDWLVWLF WR GHWHFW VLJQLILFDQW DVVRFLDWLRQV DPRQJ WKH VWXG\ YDULDEOHV 7KH VRIWZDUH SURJUDP 3& 6,=( 'DOODO f ZDV XVHG WR HVWLPDWH WKH UHTXLUHG VDPSOH VL]HV IRU WKH FRUUHODWLRQ

PAGE 64

$OO DOJRULWKPV XVHG LQ WKH SURJUDP DUH EDVHG RQ SXEOLVKHG VWDWLVWLFDO OLWHUDWXUH VHH 'DOODO f IRU UHIHUHQFHVf 7KH UHVXOWV SURGXFHG IURP WKH SURJUDP LWVHOI ZHUH YHULILHG E\ 'DOODO f DJDLQVW D VHOHFWLRQ RI HQWULHV IURP WDEOHV SUHVHQWHG LQ &RKHQ f )OHLVV f DQG 2GHK DQG )R[ f 7KH UHVXOWV IURP WKH SUHWHVW DQG UHSRUWV RI VLPLODU FRUUHODWLRQV LQ WKH OLWHUDWXUH VXJJHVWHG WKDW FRUUHODWLRQV DV VPDOO DV PD\ EH VLJQLILFDQW ,Q RUGHU WR GHWHFW D VWDWLVWLFDOO\ VLJQLILFDQW FRUUHODWLRQ RI DW OHDVW ZLWK D SRZHU RI DQG D OHYHO RI D VDPSOH VL]H RI DW OHDVW VXUYH\V LV QHFHVVDU\ 6HFRQG UXOHVRIWKXPE IRU FRQGXFWLQJ IDFWRU DQDO\VLV JHQHUDOO\ VXJJHVW VDPSOH VL]HV RI DW OHDVW RU EHWZHHQ DQG VDPSOHV SHU YDULDEOH WR EH DQDO\]HG ZKLFKHYHU LV JUHDWHU 7KH DFWXDO QXPEHU RI VDPSOHV QHHGHG GHSHQGV RI WKH DPRXQW RI YDULDELOLW\ H[SODLQHG E\ WKH IDFWRUV WKH VWUHQJWK RI WKH IDFWRU ORDGLQJV DQG WKH FRPPXQDOLWLHV RI WKH LQGLYLGXDO YDULDEOHV &URFNHU DQG $OJLQD 6WHYHQV f 7KH IDFWRU DQDO\VLV IRU WKH SUHWHVW ZDV FRQGXFWHG ZLWK DQ DYHUDJH RI VDPSOHV SHU YDULDEOH 6L[ URWDWHG IDFWRUV H[SODLQHG SHUFHQW RI WKH YDULDQFH DQG QHDUO\ DOO RI WKH LWHPV ORDGHG VWURQJO\ RQ RQH RI WKH VL[ IDFWRUV )XUWKHUPRUH DOO EXW WZR RI WKH LWHPV KDG FRPPXQDOLWLHV JUHDWHU WKDQ 7KLV HYLGHQFH VXJJHVWV WKDW ILYH VDPSOHV SHU YDULDEOH LV DGHTXDWH WR IDFWRU DQDO\]H WKH 6(593(5) SRUWLRQ RI WKH TXHVWLRQQDLUH 6LQFH WKHUH DUH YDULDEOHV WR EH DQDO\]HG WKHQ WLPHV HTXDOV D PLQLPXP VDPSOH VL]H RI 7KLUG + XVHG UHJUHVVLRQ DQDO\VLV WR H[DPLQH WKH UHODWLRQVKLS EHWZHHQ VHUYLFH WLPH LQ PLQXWHVf DQG 364 'HWHUPLQLQJ WKH QXPEHU RI VDPSOHV UHTXLUHG IRU UHJUHVVLRQ DQDO\VLV LV FRPSOH[ VLQFH VWDWLVWLFDO SRZHU IRU WKLV PHWKRG LV D IXQFWLRQ RI ERWK WKH QXPEHU RI SUHGLFWRUV XVHG LQ H[SODLQLQJ WKH YDULDQFH LQ WKH GHSHQGHQW YDULDEOH DQG WKH HIIHFW VL]H DV PHDVXUHG E\ 5f WKDW WKH UHVHDUFKHU ZDQWV WR GHWHFW *UHHQ f +RZHYHU 6 % *UHHQ f KDV GHYHORSHG D WZRVWHS PHWKRGRORJ\ IRU HVWLPDWLQJ VDPSOH VL]HV IRU UHJUHVVLRQ SXUSRVHV WKDW FRPSDUHV IDYRUDEO\ ZLWK &RKHQnV f PRUH FRPSOLFDWHG SURFHGXUHV IRU PXOWLSOH UHJUHVVLRQ SRZHU DQDO\VLV )RU D VWDWLVWLFDO SRZHU RI DW OHDVW WKH PLQLPXP VDPSOH VL]H QHFHVVDU\ LV / GLYLGHG E\ I LH 1 / LIf :KHUH /

PAGE 65

P P DQG I 5O 5f ZKHUH P LV WKH QXPEHU RI SUHGLFWRUV WR EH XVHG LQ WKH PRGHO *UHHQ S f 7R GHWHFW D VLJQLILFDQW UHODWLRQVKLS EHWZHHQ VHUYLFH WLPH DQG 364 ZLWK DQ 5 RI DW OHDVW LH VPDOO HIIHFWf XVLQJ VLPSOH OLQHDU UHJUHVVLRQ LH RQH SUHGLFWRUf UHVXOWV LQ DQ / DQG DQ I 8VLQJ WKHVH QXPEHUV LQ WKH HTXDWLRQ IRU 1 SUHVHQWHG DERYH UHVXOWV LQ D UHTXLUHG PLQLPXP VDPSOH VL]H RI 7KHUHIRUH WKH PLQLPXP QXPEHU RI TXHVWLRQQDLUHV QHHGHG WR FRQGXFW WKH K\SRWKHVLV WHVWV ZLWK VXIILFLHQW SRZHU ZDV 7KH SUHWHVW DFKLHYHG D UHVSRQVH UDWH RI DSSUR[LPDWHO\ b IRU D VDPSOH RI %DVHG RQ WKH SUHWHVW UHVSRQVH UDWH DQG GLVFXVVLRQV ZLWK WKH ',35& FRGLUHFWRUV D PLQLPXP UHVSRQVH UDWH RI b IRU WKH PDLQ TXHVWLRQQDLUH ZDV UHDVRQDEO\ DQWLFLSDWHG 7KXV D VDPSOH RI DW OHDVW FDOOHUV ZDV FDOFXODWHG DV WKH QHFHVVDU\ VDPSOH VL]H IRU WKH TXHVWLRQQDLUH SRUWLRQ RI WKLV SURMHFW 'XULQJ WZR PRQWKV RI GDWD FROOHFWLRQ D VDPSOH RI FDOOHUV ZDV LGHQWLILHG DQG VHQW TXHVWLRQQDLUHV ZKLFK ZDV FRQVLGHUHG VXIILFLHQW IRU WKH SXUSRVHV RI WKLV VWXG\ 6WXG\ 9DULDEOHV 7KH SUHYLRXV FKDSWHU LQWURGXFHG GHILQLWLRQV IRU WKH YDULDEOHV XVHG LQ WKLV VWXG\ DV SUHVHQWHG LQ WKH K\SRWKHVL]HG UHVHDUFK IUDPHZRUN 7KLV VHFWLRQ GLVFXVVHV KRZ WKHVH YDULDEOHV ZHUH PHDVXUHG XVLQJ WKH GDWD VRXUFHV GHVFULEHG DERYH $UULYDO 5DWH $f ,QWHUDUULYDO WLPHV ZHUH HVWLPDWHG E\ DUUDQJLQJ WKH DUULYDO WLPHV IURP WKH KLVWRULFDO GDWD VKHHWV $SSHQGL[ +f DQG GDWD FROOHFWLRQ IRUPV $SSHQGL[ ,f LQ DVFHQGLQJ RUGHU E\ GDWH DQG WLPH RI DUULYDO 7KH LQWHUDUULYDO WLPH ZDV REWDLQHG E\ VXEWUDFWLQJ WKH DUULYDO WLPH IRU WKH SUHYLRXV UHFRUG IURP WKH WLPH RI WKH FXUUHQW DUULYDO )RU H[DPSOH LI WKH FXUUHQW UHFRUG KDV DQ DUULYDO WLPH RI SP DQG WKH SUHYLRXV DUULYDO RFFXUUHG DW D P WKHQ WKH LQWHUDUULYDO WLPH ZDV PLQXWHV 2QO\ LQWHUDUULYDOV ZLWKLQ HDFK GD\ ZHUH HVWLPDWHG

PAGE 66

$FWXDO 6HUYLFH 7LPH 7KH DFWXDO VHUYLFH WLPH UHIHUV WR WKH DPRXQW RI WLPH UHTXLUHG WR UHVSRQG WR D TXHVWLRQ 7KLV ZDV REWDLQHG E\ VXEWUDFWLQJ WKH (QG 7LPH IRU UHFHLYLQJ D FDOO IURP WKH 6WDUW 7LPH IRU UHWXUQLQJ DQ DQVZHU WR D FDOOHU 7KHVH GDWD ZHUH WDNHQ IURP HQWULHV PDGH RQ WKH GDWD FROOHFWLRQ IRUPV $SSHQGL[ ,f %HKDYLRUDO ,QWHQWLRQ %HKDYLRUDO LQWHQWLRQ ZDV PHDVXUHG E\ WZR LWHPV LWHPV DQG LQ $SSHQGL[ 0f 7KH ILUVW LWHP PHDVXUHG ZDV ZRUGHG LQWHQG WR XVH WKLV VHUYLFH LQ WKH IXWXUH 7KH VHFRQG LWHP PHDVXUHG ZDV ZRUGHG ZRXOG UHFRPPHQG WKLV VHUYLFH WR D FROOHDJXH (DFK RI WKHVH SHUFHSWLRQV ZDV PHDVXUHG RQ D SRLQW VFDOH ZLWK D RQH UHSUHVHQWLQJ 6WURQJO\ $JUHH DQG D VHYHQ UHSUHVHQWLQJ 6WURQJO\ 'LVDJUHH 7KXV ORZHU VFRUHV LQGLFDWH JUHDWHU LQWHQWLRQ ([SHFWHG 1XPEHU LQ 6\VWHP /f $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH QXPEHU RI XQFRPSOHWHG LQIRUPDWLRQ UHTXHVWV LQ WKH V\VWHP 7KLV YDOXH ZDV REWDLQHG IURP WKH TXHXH UHSRUWV JHQHUDWHG E\ *366+ IRU WKH TXHXH ODEHOHG 727$/4 ZKLFK FROOHFWV TXHXLQJ DQG VHUYLFH WLPH LQIRUPDWLRQ UHODWLQJ WR WKH HQWLUH VHUYLFH SURFHVV ([SHFWHG 4XHXH /HQJWK /Tf $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH QXPEHU RI TXHVWLRQV LQ WKH V\VWHP WKDW KDYH QRW \HW VWDUWHG WKH UHVHDUFK SURFHVV 7KLV YDOXH ZDV REWDLQHG IURP WKH TXHXH UHSRUWV JHQHUDWHG E\ *366+ IRU WKH TXHXH ODEHOHG %2$5'4 ZKLFK FROOHFWV TXHXLQJ DQG VHUYLFH WLPH LQIRUPDWLRQ UHODWLQJ MXVW WR WKH SHULRG LQ ZKLFK TXHVWLRQV VSHQG LQ WKH TXHXH EHIRUH VWDUWLQJ WKH UHVHDUFK SURFHVV ([SHFWHG 7LPH LQ 4XHXH :Tf $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH DPRXQW RI WLPH WKDW DQ LQIRUPDWLRQ UHTXHVWV RU TXHVWLRQV PXVW ZDLW LQ WKH TXHXH EHIRUH VWDUWLQJ WKH UHVHDUFK SURFHVV 7KLV YDOXH ZDV REWDLQHG IURP WKH TXHXH UHSRUWV JHQHUDWHG E\ *366+ IRU WKH TXHXH ODEHOHG %2$5'4

PAGE 67

([SHFWHG 7LPH LQ 6\VWHP :f $ VLPXODWLRQ RXWSXW YDULDEOH WKDW LQGLFDWHV WKH DYHUDJH WRWDO DPRXQW RI WLPH WKDW D TXHVWLRQ RU LQIRUPDWLRQ UHTXHVW VSHQGV LQ WKH V\VWHP 7KLV YDOXH ZDV REWDLQHG IURP WKH TXHXH UHSRUWV JHQHUDWHG E\ *366+ IRU WKH TXHXH ODEHOHG 727$/4 ([SHFWHG 8WLOL]DWLRQ 5DWH Sf $ VLPXODWLRQ RXWSXW YDULDEOH LQGLFDWLQJ WKH DYHUDJH SHUFHQWDJH RI WLPH WKDW VHUYHUV DUH EXV\ 7KLV ZDV REWDLQHG IURP WKH IDFLOLW\ UHSRUWV JHQHUDWHG E\ *366+ ZKLFK UHSRUWV XWLOL]DWLRQ DV WKH SHUFHQWDJH RI WKH WRWDO WLPH WKDW WKH IDFLOLWLHV LH VWXGHQWVf ZHUH FDSWXUHG LH XWLOL]HGf 8WLOL]DWLRQ UDWHV IRU HDFK RI WKH VLPXODWHG VWXGHQWV ZHUH DYHUDJHG WRJHWKHU WR REWDLQ DQ H[SHFWHG RYHUDOO XWLOL]DWLRQ UDWH 2YHUDOO 6HUYLFH 4XDOLW\ 264f $ VXEMHFWnV RYHUDOO HYDOXDWLRQ RI WKH VHUYLFH TXDOLW\ RI WKH GUXJ LQIRUPDWLRQ VHUYLFH 7KLV SHUFHSWLRQ ZDV HYDOXDWHG XVLQJ D VLQJOHLWHP PHDVXUHG RQ D SRLQW VFDOH LWHP LQ $SSHQGL[ 0f 7KH LWHP ZDV ZRUGHG 7KH RYHUDOO TXDOLW\ RI WKH VHUYLFHV SURYLGHG E\ WKH ',35& LV EHVW GHVFULEHG DV $Q ([FHOOHQW UDWLQJ ZDV VFRUHG DV D RQH DQG DQ 8QDFFHSWDEOH UDWLQJ ZDV VFRUHG DV D VL[ 7KXV ORZHU VFRUHV LQGLFDWHG KLJKHU SHUFHLYHG 264 3HUFHLYHG 6HUYLFH 4XDOLW\ 364f $ VXEMHFWnV HYDOXDWLRQ RI WKH VHUYLFH TXDOLW\ RI WKH GUXJ LQIRUPDWLRQ VHUYLFH ZDV PHDVXUHG E\ VXPPLQJ WKH LQGLYLGXDO LWHPV RI WKH 6(593(5) LQVWUXPHQW WR REWDLQ DQ RYHUDOO SHUFHLYHG VHUYLFH TXDOLW\ VFRUH 'XULQJ WKH PDLQ SKDVH RI WKH VWXG\ LWHPV LQLWLDOO\ FRPSRVHG WKH 364 VFDOH QXPEHUV IRXU WKURXJK WZHQW\WKUHH LQ $SSHQGL[ 0f 7KHVH LWHPV ZHUH PHDVXUHG RQ D VHYHQSRLQW VFDOH DQFKRUHG ZLWK 6WURQJO\ $JUHH UHFHLYLQJ D YDOXH RI RQHf DQG 6WURQJO\ 'LVDJUHH UHFHLYLQJ D YDOXH RI VHYHQf 1HJDWLYHO\ ZRUGHG LWHPV ZHUH UHYHUVHG VFRUHG 2QH LWHP ZDV GURSSHG IURP WKH VFDOH UHVXOWLQJ LQ D ILQDO PHDVXUH FRPSRVHG RI LWHPV 7KHUHIRUH 364 KDG D SRVVLEOH UDQJH RI WR SRLQWV ZLWK ORZHU VFRUHV LQGLFDWLQJ KLJKHU SHUFHLYHG TXDOLW\

PAGE 68

3HUFHLYHG 6HUYLFH 7LPH $ VXEMHFWnV SHUFHSWLRQV UHJDUGLQJ WKH UHVSRQVH WLPH RI WKH GUXJ LQIRUPDWLRQ VHUYLFH 7KLV YDULDEOH ZDV PHDVXUHG E\ IRXU TXHVWLRQQDLUH LWHPV 7KH ILUVW ZDV TXHVWLRQ f$FFHSWDEOH 7LPHf 7KLV LWHP ZDV ZRUGHG 7KH DPRXQW RI WLPH WKDW LW WRRN WKH ',35& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV DFFHSWDEOH 7KH VHFRQG ZDV TXHVWLRQ f1R /RQJHU 8VHIXOf 7KLV LWHP ZDV ZRUGHG %\ WKH WLPH UHFHLYHG D UHVSRQVH IURP WKH ',35& WKH LQIRUPDWLRQ ZDV QR ORQJHU XVHIXO WR PH 7KH WKLUG ZDV TXHVWLRQ f4XLFNHU 5HVSRQVHf 7KLV LWHP ZDV ZRUGHG ZLVK WKH ',35& FRXOG SURYLGH D TXLFNHU UHVSRQVH WR P\ TXHVWLRQV $OO WKUHH RI WKHVH LWHPV ZHUH VFRUHG RQ D VHYHQSRLQW VFDOH DQFKRUHG ZLWK 6WURQJO\ $JUHH UHFHLYLQJ D YDOXH RI RQHf DQG 6WURQJO\ 'LVDJUHH UHFHLYLQJ D YDOXH RI VHYHQf 7KH IRXUWK LWHP ZDV ZRUGHG 7KH DPRXQW RI WLPH WKDW LW WRRN WKH ',35& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV 7KLV LWHP ZDV DOVR VFDOHG RQ D VHYHQSRLQW VFDOH KRZHYHU LW ZDV DQFKRUHG E\ 0XFK 6KRUWHU WKDQ ([SHFWHG UHFHLYLQJ D YDOXH RI RQHf DQG 0XFK /RQJHU WKDQ ([SHFWHG UHFHLYLQJ D YDOXH RI VHYHQf 4XHXH 'LVFLSOLQH 7KH TXHXH GLVFLSOLQH LV WKH PHWKRG FXUUHQWO\ XVHG LQ WKH ',35& WR SULRULWL]H TXHVWLRQV DV WKH\ DUULYH 7KLV GLVFLSOLQH ZDV GHVFULEHG E\ WKH VWXGHQWV DQG FRn GLUHFWRUV GXULQJ SHUVRQDO LQWHUYLHZV 7KH TXHXH GLVFLSOLQH XVHG E\ WKH VLPXODWLRQ PRGHO ZDV GHYHORSHG XVLQJ WKHVH GHVFULSWLRQV 6HUYLFH 'HOD\ 7KLV LV D GLFKRWRPRXV YDULDEOH PHDVXUHG E\ FRPSDULQJ WKH REVHUYHG VHUYLFH WLPH ZLWK WKH UHVSRQVH WLPH QHHGHG DV UHSRUWHG RQ WKH GDWD FROOHFWLRQ IRUPV DQG KLVWRULFDO GDWD VKHHWV ,I WKH VHUYLFH WLPH ZDV ORQJHU WKDQ WKH WLPH QHHGHG WKHQ WKH YDULDEOH ZDV FRGHG ZLWK D YDOXH RI RQH LQGLFDWLQJ 'HOD\HG ,I WKH VHUYLFH WLPH ZDV VKRUWHU WKDQ WKH WLPH QHHGHG WKHQ WKH YDULDEOH ZDV FRGHG ZLWK D YDOXH RI ]HUR LQGLFDWLQJ 1RW GHOD\HG

PAGE 69

6HUYLFH 5DWH Sf 7KH SUREDELOLW\ GLVWULEXWLRQ WKDW LV LQSXW LQWR WKH VLPXODWLRQ PRGHO WR GHVFULEH WKH VHUYLFH WLPHV IRU WKH VWHSV LQ WKH VHUYLFH SURFHVV LQ WKH ',6 7KHVH SUREDELOLW\ GLVWULEXWLRQV ZHUH HVWLPDWHG IURP WKH WLPH LQIRUPDWLRQ REWDLQHG IURP WKH KLVWRULFDO GDWD VKHHWV DQG FRQFXUUHQW GDWD FROOHFWLRQ IRUPV GLVFXVVHG DERYH 6WDIILQJ /HYHO Vf $ VLPXODWLRQ LQSXW GHVFULELQJ WKH QXPEHU RI LQGLYLGXDOV DYDLODEOH WR KDQGOH VHUYLFH UHTXHVWV 8QGHU 1RUPDO 2SHUDWLRQ WKLV YDULDEOH KDG D YDOXH RI WKUHH LQGLFDWLQJ WKDW WKUHH VWXGHQWV ZHUH UHVSRQVLEOH IRU VWDIILQJ WKH ',35& 7KLV YDOXH ZDV YDULHG IURP RQH WR ILYH LQ WKH VLPXODWLRQ PRGHO WR DVVHVV WKH LPSDFW RI FKDQJHV LQ VWDIILQJ OHYHOV RQ WKH UHVXOWV RI WKH VLPXODWLRQ PRGHO 4XHVWLRQQDLUH 'HYHORSPHQW DQG 9DOLGDWLRQ $V GHVFULEHG LQ WKH OLWHUDWXUH UHYLHZ VHUYLFH TXDOLW\ LQVWUXPHQWV EDVHG RQ WKH LWHPV LQ WKH 6(5948$/ VFDOH DUH LQ ZLGH XVH 7KHUHIRUH WKH FRQVHQVXV DPRQJ WKRVH XVLQJ WKH VFDOH DSSHDUV WR EH JHQHUDOO\ WKDW WKH LWHPV FRPSRVLQJ WKH VFDOH DUH DQ DGHTXDWH UHSUHVHQWDWLRQ RI SHUFHLYHG VHUYLFH TXDOLW\ EDUULQJ PRGLILFDWLRQV QHFHVVDU\ IRU DSSOLFDELOLW\ )XUWKHUPRUH WKH SURFHGXUHV XVHG E\ 3DUDVXUDPDQ =HLWKDPO DQG %HUU\ f WR GHYHORS WKH LQGLYLGXDO LWHPV DSSHDU WR EH ZHOOVXSSRUWHG &URQLQ DQG 7D\ORU 2OLYHU f 7KHUHIRUH WKH TXHVWLRQ RI ZKHWKHU WKH LWHPV LQ WKH 6(593(5) LQVWUXPHQW DFWXDOO\ PHDVXUH WKH FRQVWUXFW RI SHUFHLYHG VHUYLFH TXDOLW\ LV QRW DW LVVXH LQ WKLV UHVHDUFK +RZHYHU EHFDXVH WKH 6(593(5) LQVWUXPHQW ZDV PRGLILHG IRU XVH LQ WKH GUXJ LQIRUPDWLRQ VHWWLQJ LW ZDV QHFHVVDU\ WR YDOLGDWH WKH FRQWHQW RI WKH TXHVWLRQQDLUH H[SORUH VRPH JHQHUDO DVSHFWV RI FRQVWUXFW YDOLGLW\ HVWDEOLVK LQWHUQDO FRQVLVWHQF\ DQG UHSRUW WKH SRWHQWLDO IRU QRQUHVSRQVH ELDV

PAGE 70

&RQWHQW 9DOLGLW\ &RQWHQW YDOLGLW\ LV DQ DVVHVVPHQW RI ZKHWKHU WKH LWHPV LQ D VFDOH DGHTXDWHO\ PHDVXUH WKH FRQVWUXFW RI LQWHUHVW &URFNHU DQG $OJLQD f 7KUHH PHWKRGV ZHUH XVHG WR HVWDEOLVK WKH FRQWHQW YDOLGLW\ RI WKH LQVWUXPHQW )LUVW DV VXJJHVWHG E\ &URFNHU DQG $OJLQD f DQG 'H9HOOLV f DQ H[SHUW SDQHO ZDV DVNHG WR HYDOXDWH WKH VHUYLFH TXDOLW\ LQVWUXPHQW XVHG LQ WKLV VWXG\ IRU FODULW\ DQG UHDGDELOLW\ ,Q DGGLWLRQ WKH SDQHO ZDV DOVR DVNHG WR DVVHVV ZKHWKHU RU QRW WKH LQVWUXPHQW FRYHUHG DOO RI WKH GRPDLQV WKDW WKH\ EHOLHYHG ZHUH LPSRUWDQW LQ PHDVXULQJ WKH TXDOLW\ RI D GUXJ LQIRUPDWLRQ VHUYLFH 7KLV SDQHO FRQVLVWHG RI 7ZR FRGLUHFWRUV RI WKH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVRXUFH &HQWHU DW 6KDQGV DQG WKH 8QLYHUVLW\ RI )ORULGD 7KH GLUHFWRU DQG HPSOR\HHV RI WKH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU DW WKH 8QLYHUVLW\ RI $UNDQVDV IRU 0HGLFDO 6FLHQFHV 7KUHH UHFHQW FDOOHUV WR WKH ',35& LGHQWLILHG IURP WKH GDWD VKHHWV SKDUPDFLVWV DQG QXUVHf 7KUHH VHQLRU JUDGXDWH VWXGHQWV ZLWK SKDUPDF\ GHJUHHV DV ZHOO DV H[SHULHQFH DQG HGXFDWLRQDO EDFNJURXQG LQ VXUYH\ GHVLJQ DQG PHWKRGRORJ\ 7ZR SURIHVVRUV H[SHULHQFHG LQ VXUYH\ UHVHDUFK DQG IDPLOLDU ZLWK WKH OLWHUDWXUH LQYROYLQJ SHUFHLYHG VHUYLFH TXDOLW\ 7KH UHFHQW FDOOHUV DQG WKH JUDGXDWH VWXGHQWV ZHUH DVNHG WR FRPSOHWH WKH LQVWUXPHQW DQG UHSRUW WKH OHQJWK RI WLPH VSHQW RQ WKH TXHVWLRQQDLUH )URP WKHVH VL[ FRPSOHWHG TXHVWLRQQDLUHV LW ZDV GHWHUPLQHG WKDW WKH W\SLFDO WLPH IRU FRPSOHWLRQ RI WKH TXHVWLRQQDLUH LV EHWZHHQ PLQXWHV GHSHQGLQJ RQ WKH QXPEHU RI ZULWWHQ FRPPHQWV

PAGE 71

&KDQJHV ZHUH PDGH WR WKH LQLWLDO LQVWUXPHQW EDVHG RQ WKH UHFRPPHQGDWLRQV RI WKH SDQHO 7KHVH UHFRPPHQGDWLRQV ZHUH SULPDULO\ UHODWHG WR FODULW\ DQG LWHP RUGHU LVVXHV KRZHYHU ILYH LWHPV ZHUH DGGHG WR WKH LQLWLDO LQVWUXPHQW EDVHG RQ WKH SDQHOnV FRPPHQWV f D TXHVWLRQ FRQFHUQLQJ FDOOLQJ IUHTXHQF\ ,WHP f f D TXHVWLRQ UHODWHG WR WKH QHHG IRU ZULWWHQ VXSSRUWLQJ GRFXPHQWV ,WHP f f D TXHVWLRQ LQYROYLQJ WKH VHUYLFHnV UHODWLRQ WR SDWLHQW RXWFRPHV ,WHP f DQG f DQG WZR TXHVWLRQV FRQFHUQLQJ WKH XVHIXOQHVV RI WKH VHUYLFH ,WHPV DQG f 6HFRQG WKH RULJLQDO YHUVLRQ RI WKH 6(593(5) VFDOH GLG QRW LQFOXGH D UHVSRQVH RSWLRQ DOORZLQJ VXEMHFWV WR GLIIHUHQWLDWH EHWZHHQ LWHPV IRU ZKLFK WKH\ KDG QR RSLQLRQ RU QR H[SHULHQFH YHUVXV LWHPV IRU ZKLFK WKH\ DFWXDOO\ KDG QHXWUDO IHHOLQJV 6XUYH\ UHVHDUFK FRQGXFWHG E\ .LSSHQ 6WUDVVHU DQG -RVKL f UHSRUWHG GLIIHUHQFHV LQ UHVSRQVH SDWWHUQV IRU VXEMHFWV WKDW KDG 1R 2SLQLRQ DQG 1R ([SHULHQFH UHVSRQVH FDWHJRULHV YHUVXV VXEMHFWV WKDW ZHUH IRUFHG WR FKRRVH D UHVSRQVH 7KH SUHWHVW ZDV FRQGXFWHG XVLQJ WZR YHUVLRQV RI WKH TXHVWLRQQDLUH 7KH ILUVW YHUVLRQ IRUFHG WKH VXEMHFWV WR FKRRVH D UHVSRQVH IURP D VHYHQSRLQW VFDOH $SSHQGL[ (f 7KH VHFRQG YHUVLRQ LQFOXGHG DQ HLJKWK VFDOH RSWLRQ DOORZLQJ VXEMHFWV WR VHOHFW 'RQnW .QRZ IRU TXHVWLRQV WKDW WKH\ GLG QRW KDYH DQ RSLQLRQ RU WKH\ IHOW ZHUH QRW DSSOLFDEOH $SSHQGL[ )f 7KH UHVSRQVH SDWWHUQV EHWZHHQ WKHVH WZR YHUVLRQV ZHUH XVHG WR VHOHFW RXW LWHPV WKDW ZHUH QRW DSSOLFDEOH WR WKH GUXJ LQIRUPDWLRQ VHWWLQJ 7KLUG FRPPHQWV ZHUH RIWHQ ZULWWHQ QH[W WR LQGLYLGXDO LWHPV DQG LQ UHVSRQVH WR WKH ILQDO TXHVWLRQ RI WKH TXHVWLRQQDLUH $SSHQGLFHV DQG 3f 7KHVH FRPPHQWV ZHUH XVHG LQ FRPELQDWLRQ ZLWK WKH RWKHU YDOLGDWLRQ WHFKQLTXHV WR KHOS GHFLGH LI LQGLYLGXDO LWHPV ZHUH DSSOLFDEOH WR WKH GUXJ LQIRUPDWLRQ VHWWLQJ

PAGE 72

&RQVWUXFW 9DOLGLW\ &RQVWUXFW YDOLGLW\ LV FRQFHUQHG ZLWK WKH WKHRUHWLFDO UHODWLRQVKLSV EHWZHHQ YDULDEOHV WR WKH H[WHQW WKDW WKH PHDVXUHV XVHG WR UHSUHVHQW YDULDEOHV EHKDYH DV H[SHFWHG LQ UHODWLRQ WR RWKHU PHDVXUHV &URFNHU DQG $OJLQD f 7ZR LVVXHV UHOHYDQW WR WKH FRQVWUXFW YDOLGLW\ RI WKH TXHVWLRQQDLUH ZHUH H[DPLQHG )LUVW WKH GLPHQVLRQDOLW\ RI WKH 6(593(5) SRUWLRQ RI WKH TXHVWLRQQDLUH ZDV H[SORUHG 3DUDVXUDPDQ HW DO f KDYH GHVFULEHG SHUFHLYHG VHUYLFH TXDOLW\ DV D PXOWLGLPHQVLRQDO FRQVWUXFW FRYHULQJ ILYH VHSDUDWH GLPHQVLRQV WDQJLEOHV UHOLDELOLW\ UHVSRQVLYHQHVV HPSDWK\ DQG DVVXUDQFH +RZHYHU DV GHVFULEHG LQ WKH OLWHUDWXUH UHYLHZ RWKHU UHVHDUFKHUV KDYH KDG WURXEOH GXSOLFDWLQJ WKH RULJLQDO ILYH GLPHQVLRQV )XUWKHUPRUH UHVHDUFKHUV KDYH IRXQG WKDW LWHPV GR QRW DOZD\V ORDG RQ WKH VDPH GLPHQVLRQ &URQLQ DQG 7D\ORU f VXJJHVWHG WKDW WKH SHUFHLYHG VHUYLFH TXDOLW\ VKRXOG QRW EH FRQVLGHUHG DV D PXOWLGLPHQVLRQDO FRQVWUXFW EXW LQVWHDG D XQLGLPHQVLRQDO FRQVWUXFW ,I 6(593(5) FDQ EH FRQVLGHUHG DV D PXOWLGLPHQVLRQDO FRQVWUXFW WKHQ LW PD\ EH LPSRUWDQW WR H[DPLQH WKH GLIIHUHQWLDO HIIHFWV RI VHUYLFH WLPH DQG GHOD\V RQ WKHVH GLPHQVLRQV DV ZHOO DV WKH RYHUDOO LQVWUXPHQW 7KLV UHVHDUFK H[SORUHG WKH GLPHQVLRQDOLW\ RI WKH 6(593(5) SRUWLRQ RI WKH VHUYLFH TXDOLW\ TXHVWLRQQDLUH XVLQJ SULQFLSDO FRPSRQHQWV IDFWRU DQDO\VLV ZLWK D 9DULPD[ LH RUWKRJRQDOf URWDWLRQ 7KH QXPEHU RI FRPSRQHQWV WR UHWDLQ ZDV GHFLGHG XVLQJ WKH .DLVHU FULWHULRQ VXFK WKDW FRPSRQHQWV ZLWK HLJHQYDOXHV RI RU KLJKHU ZHUH UHWDLQHG DQG FRPSRQHQWV ZLWK OHVV WKDQ ZHUH H[FOXGHG 6WHYHQV f ,WHPV ZHUH DVVLJQHG DFFRUGLQJ WR WKHLU IDFWRU ORDGLQJV ,WHPV ZLWK ORDGLQJV OHVV WKDQ ZHUH UHMHFWHG 'H9HOOLV 6WHSKHQV f 6HFRQG WKH FRUUHODWLRQV DPRQJ 364 264 DQG EHKDYLRUDO LQWHQWLRQ REWDLQHG IURP WKLV TXHVWLRQQDLUH ZHUH FRPSDUHG ZLWK VLPLODU FRUUHODWLRQV REWDLQHG E\ &URQLQ DQG 7D\ORU f ,Q RWKHU ZRUGV WKH TXHVWLRQ ZDV DVNHG 'R WKH YDULDEOHV EHKDYH DV

PAGE 73

H[SHFWHG" 7KLV VWHS LV LPSRUWDQW GXH WR WKH FKDQJHV PDGH WR 6(593(5) IRU XVH LQ WKH GUXJ LQIRUPDWLRQ VHWWLQJ 8VLQJ WKHVH FRPSDULVRQV LW ZDV SRVVLEOH WR VHH LI WKH YDULDEOHV XVHG LQ WKLV UHVHDUFK SHUIRUPHG FRQVLVWHQWO\ ZLWK ZKDW KDV EHHQ UHSRUWHG LQ WKH SUHYLRXV UHVHDUFK (O\SRWKHVHV +, + DQG + ZHUH XVHG WR FRPSDUH WKHVH UHODWLRQVKLSV 5HOLDELOLW\ $VVHVVPHQW ,QWHUQDO FRQVLVWHQF\ PHDVXUHV UHOLDELOLW\ IRU D VLQJOH WLPH SHULRG DQG HVVHQWLDOO\ PHDVXUHV LWHP KRPRJHQHLW\ DQG TXDOLW\ &URFNHU DQG $OJLQD 'H9HOOLV f ,QWHUQDO FRQVLVWHQF\ ZDV PHDVXUHG XVLQJ FRHIILFLHQW DOSKD LH &URQEDFKfV DOSKDf &RHIILFLHQW DOSKD GHVFULEHV WKH SURSRUWLRQ RI WKH VFDOH VFRUH YDULDQFH DWWULEXWDEOH WR WKH WUXH VFRUH DQG LV DIIHFWHG E\ PDQ\ LWHP TXDOLW\ SUREOHPV HJ QRQFHQWUDO PHDQ SRRU YDULDELOLW\ QHJDWLYH RU ZHDN LQWHULWHP FRUUHODWLRQ DQG SRRU LWHPVFDOH FRUUHODWLRQf 'H9HOOLV f $V DQ DEVROXWH PHDVXUH DQ DOSKD UDQJLQJ IURP WR ZDV FRQVLGHUHG DFFHSWDEOH WR ZDV FRQVLGHUHG JRRG DQG DQ DOSKD UDQJLQJ IURP WR ZDV FRQVLGHUHG YHU\ JRRG 'H9HOOLV f $VVHVVPHQW RI ,1 RQ5HVSRQVH %LDV 7KH SRWHQWLDO IRU QRQUHVSRQVH ELDV ZDV HYDOXDWHG LQ WZR ZD\V )LUVW WKH UHDVRQV WKDW VXEMHFWV JDYH IRU QRW UHVSRQGLQJ WR WKH VXUYH\ ZKHQ DVNHG GXULQJ WKH UHPLQGHU WHOHSKRQH FDOO ZHUH FROODWHG DQG VXPPDUL]HG 6HFRQG RQHZD\ $129$ SURFHGXUHV ZHUH FRQGXFWHG IRU VXEVFULEHU VWDWXV SURIHVVLRQ DQG UHVSRQVH LQWHUYDO WR FKHFN IRU VLJQLILFDQW GLIIHUHQFHV LQ 364 DQG 264 DPRQJ WKH UHVSHFWLYH JURXSV 7KHVH GHSHQGHQW YDULDEOHV ZHUH FKRVHQ EHFDXVH RI WKHLU LPSRUWDQFH LQ WKH K\SRWKHVLV WHVWV (DFK TXHVWLRQQDLUH UHVSRQVH ZDV UHFHLYHG ZLWKLQ RQH RI WKUHH UHVSRQVH LQWHUYDOV f UHVSRQVH ZDV UHFHLYHG EHIRUH WKH UHPLQGHU SRVW FDUG ZDV PDLOHG f UHVSRQVH ZDV UHFHLYHG DIWHU WKH UHPLQGHU SRVW FDUG EXW EHIRUH WKH IROORZ XS SKRQH FDOO DQG f

PAGE 74

UHVSRQVH ZDV UHFHLYHG DIWHU WKH IROORZ XS SKRQH FDOO 7KH UDWLRQDOH IRU DQDO\]LQJ XVLQJ WKH UHVSRQVH LQWHUYDO WR DVVHVV QRQUHVSRQVH ELDV LV EDVHG RQ WKH DVVXPSWLRQ WKDW ODWH UHVSRQGHUV ZHUH PRUH OLNHO\ WR EH OLNH QRQUHVSRQGHUV LQ WKHLU UHVSRQVHV WR WKH TXHVWLRQQDLUH WKDQ VXEMHFWV ZKR UHWXUQHG WKH TXHVWLRQQDLUH ZLWKRXW QHHGLQJ D UHPLQGHU ,I VLJQLILFDQW GLIIHUHQFHV H[LVWHG EHWZHHQ WKH HDUO\ DQG ODWH UHVSRQGHUV LW ZDV OLNHO\ WKDW VRPH GHJUHH RI QRQUHVSRQVH ELDV DOVR H[LVWHG 3UHWHVW RI 4XHVWLRQQDLUH 7KH SUHWHVW IRU WKLV VWXG\ ZDV FRQGXFWHG DW WKH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU $3',&f LQ /LWWOH 5RFN $UNDQVDV 7KH $3',& WDNHV GUXJ LQIRUPDWLRQ DQG SRLVRQ LQWHUYHQWLRQ FDOOV IURP SUDFWLWLRQHUV DQG OD\ FRQVXPHUV IURP WKH VWDWH RI $UNDQVDV DQG KDV FRQWUDFWXDO UHODWLRQVKLSV ZLWK RWKHU RUJDQL]DWLRQV DQG FRUSRUDWLRQV UHTXLULQJ GUXJ LQIRUPDWLRQ VHUYLFHV 7KH $3',& LV VWDIIHG SULPDULO\ E\ UHJLVWHUHG SKDUPDFLVWV KRZHYHU 3KDUP' LQWHUQV GR RFFDVLRQDOO\ SURYLGH DGGLWLRQDO DVVLVWDQFH 2QO\ KHDOWK FDUH SUDFWLWLRQHUV FDOOLQJ LQ ZLWK GUXJ LQIRUPDWLRQ TXHVWLRQV LH QRQn SRLVRQ UHODWHGf ZHUH LGHQWLILHG DV SRWHQWLDO SDUWLFLSDQWV 'XULQJ WKH VWXG\ SHULRG FDOOHUV ILW WKHVH FULWHULD KRZHYHU bf RI WKHVH FDOOHUV ZHUH LGHQWLILHG DV UHSHDW FDOOHUV DQG DV VXFK ZHUH RQO\ VHQW RQH TXHVWLRQQDLUH &RQWDFW LQIRUPDWLRQ ZDV XQDYDLODEOH IRU WZR RI WKH FDOOHUV 7KLV UHVXOWHG LQ D XVDEOH SUHWHVW VDPSOH RI VXEMHFWV 4XHVWLRQQDLUHV ZHUH PDLOHG WR WKHVH VXEMHFWV RQ -XQH DQG -XQH $ FRYHU OHWWHU $SSHQGL[ $f IURP WKH GLUHFWRU RI WKH $3',& ZDV SULQWHG RQ OHWWHUKHDG DQG VHQW DORQJ ZLWK RQH RI WZR YHUVLRQV RI WKH SUHWHVW TXHVWLRQQDLUH $SSHQGLFHV % DQG &f 2QH RI WKH YHUVLRQV ZDV UDQGRPO\ DVVLJQHG WR HDFK VXEMHFW $SSUR[LPDWHO\ WKUHH ZHHNV DIWHU WKH LQLWLDO PDLOLQJ D SRVWFDUG ZDV VHQW WR QRQUHVSRQGHUV XUJLQJ WKHP WR PDLO LQ WKH TXHVWLRQQDLUH $SSHQGL[ 'f 7KLV SRVWFDUG DOVR WKDQNHG

PAGE 75

UHVSRQGHQWV LI WKH\ KDG DOUHDG\ VHQW LQ WKH TXHVWLRQQDLUH DQG DVNHG WKRVH ZKR KDG QRW UHFHLYHG WKH TXHVWLRQQDLUH WR FRQWDFW WKH UHVHDUFKHU VHH $SSHQGL[ &f ,Q WRWDO VXEMHFWV UHVSRQGHG LQ WLPH WR EH LQFOXGHG LQ WKH DQDO\VLV 7KUHH HQYHORSHV ZHUH UHWXUQHG DV XQGHOLYHUDEOH WZR VXEMHFWV FDOOHG WR VD\ WKDW WKH\ KDG UHFHLYHG WKH SRVWFDUG EXW QRW WKH TXHVWLRQQDLUH DQG RQH VXEMHFW UHWXUQHG WKH VXUYH\ FRPSOHWHO\ XQDQVZHUHG 7ZR VXEMHFWV UHWXUQHG FRPSOHWHG VXUYH\V WRR ODWH WR EH LQFOXGHG LQ WKH DQDO\VLV 7KLV HTXDOHG D UDZ UHVSRQVH UDWH RI b GLYLGHG E\ f DQG DQ DGMXVWHG UHVSRQVH UDWH RI b GLYLGHG E\ f 2I WKHVH UHVSRQGHQWV bf ZHUH SKDUPDFLVWV bf ZHUH SK\VLFLDQV bf DQG bf ZHUH FDWHJRUL]HG DV 2WKHU 2QH RI VXUYH\V VHQW WR D SK\VLFLDQ ZDV UHWXUQHG DV XQGHOLYHUDEOH DQG WKHUH ZDV QR UHVSRQVH UHFHLYHG IURP WZR QXUVHV ZKR ZHUH PDLOHG D TXHVWLRQQDLUH )RU DQDO\VLV SXUSRVHV QHJDWLYHO\ ZRUGHG TXHVWLRQV SUHWHVW TXHVWLRQ QXPEHUV DQG f ZHUH UHYHUVH VFRUHG )XUWKHUPRUH WKH ILUVW YHUVLRQ RI WKH TXHVWLRQQDLUH LQFOXGHG D 'RQnW .QRZ FKHFNER[ ZKLOH WKH VHFRQG YHUVLRQ GLG QRW LQFOXGH WKLV DV DQ RSWLRQ %\ FRPSDULQJ WKH SDWWHUQV RI UHVSRQVH IRU WKH WZR YHUVLRQV LW ZDV SRVVLEOH WR JDLQ VRPH LQVLJKW UHJDUGLQJ KRZ VXEMHFWV UHVSRQGHG WR LWHPV WKDW WKH\ GLG QRW NQRZ KRZ WR DVVHVV RU WKDW GLG QRW DSSO\ WR WKHP ,Q DGGLWLRQ WKHVH FRPSDULVRQV PDGH LW HDVLHU WR VHSDUDWH LWHPV WKDW ZHUH QRW DSSOLFDEOH WR WKH VHWWLQJ IURP WKRVH LWHPV DERXW ZKLFK UHVSRQGHQWV DFWXDOO\ KDG QHXWUDO IHHOLQJV 2I WKH TXHVWLRQQDLUHV VHQW RXW ZLWKRXW D 'RQnW .QRZ UHVSRQVH FDWHJRU\ LH YHUVLRQ RQHf bf ZHUH UHWXUQHG 2I WKH TXHVWLRQQDLUHV VHQW RXW LQFOXGLQJ D 'RQnW .QRZ UHVSRQVH FDWHJRU\ LH YHUVLRQ WZRf bf ZHUH UHWXUQHG 7KH UHVSRQVH SDWWHUQV IRU WKH WZR YHUVLRQV LQGLFDWHG WKDW LWHPV 7KH $3',& NHHSV LW UHFRUGV DFFXUDWHO\ f DQG (PSOR\HHV JHW DGHTXDWH VXSSRUW IURP WKH $3',& WR GR WKHLU MREV ZHOO f ZHUH GLIILFXOW WR DQVZHU 7DEOH EHORZ VKRZV WKH UHVSRQVHV IRU TXHVWLRQV DQG EDVHG RQ WKH YHUVLRQ 4XHVWLRQ RQ YHUVLRQ RI WKH

PAGE 76

TXHVWLRQQDLUH GUHZ bf 'RQnW .QRZ UHVSRQVHV DQG TXHVWLRQ GUHZ bf 'RQnW .QRZ UHVSRQVHV :KHQ FRPSDULQJ YHUVLRQV DQG IRU ERWK RI WKHVH LWHPV D FOHDU VKLIW LV HYLGHQW IURP WKH 1HXWUDO DQG QR UHVSRQVH FDWHJRULHV WR WKH 'RQnW .QRZ FDWHJRU\ 7KLV VXJJHVWV WKDW ZKHQ VXEMHFWV ZHUH QRW DEOH WR SXW 'RQnW .QRZ DV D UHVSRQVH WKH\ RIWHQ UHVSRQGHG ZLWK D QHXWUDO UHVSRQVH LQVWHDG 7KHVH GDWD DOVR VXJJHVWV WKDW WKHVH LWHPV ZHUH QRW DSSOLFDEOH WR WKH GUXJ LQIRUPDWLRQ VHWWLQJ $SSHQGLFHV ( DQG ) LOOXVWUDWH WKH UHVSRQVHV IRU DOO LWHPV 7KLV ODFN RI DSSOLFDELOLW\ LV IXUWKHU VXSSRUWHG E\ WKH FRPPHQWV ZULWWHQ QH[W WR WKH LWHPV $SSHQGL[ *f )RU TXHVWLRQ VL[ RI WKH UHVSRQGHQWV ZURWH LQ GRQnW NQRZ QH[W WR WKH TXHVWLRQ DQG DQRWKHU ZURWH LQ WKDW WKH\ GLG QRW XQGHUVWDQG WKH VWDWHPHQW 6LPLODUO\ IRU TXHVWLRQ VL[ UHVSRQGHQWV LQGLFDWHG WKDW WKH\ GLG QRW H[DFWO\ NQRZ KRZ WR DQVZHU WKH TXHVWLRQ RQH UHVSRQGHQW VWDWHG WKDW WKH TXHVWLRQ GLG QRW DSSO\ DQG DQRWKHU VWDWHG WKDW WKH\ GLG QRW XQGHUVWDQG WKH TXHVWLRQ $V RQH UHVSRQGHQW ZURWH f>W@KH RQO\ KLQW KDYH LV KRZ ZHOO WKH HPSOR\HHV DQVZHU P\ TXHVWLRQVf VXJJHVWLQJ WKDW VXEMHFWV ILQG WKHVH LWHPV GLIILFXOW WR HYDOXDWH EHFDXVH WKH\ KDYH QR GLUHFW H[SHULHQFH ZLWK WKHVH LVVXHV %HFDXVH RI WKLV GDWD LW ZDV GHFLGHG WR H[FOXGH LWHPV DQG IURP WKH UHVW RI WKH DQDO\VLV DQG HOLPLQDWH WKHP IURP WKH ILQDO YHUVLRQ RI WKH TXHVWLRQQDLUH 7DEOH 5HVSRQVHV WR 3UHWHVW 4XHVWLRQV DQG 5HVSRQVH &DWHJRU\ 4XHVWLRQ 9HUVLRQ )UHT bf 4XHVWLRQ 9HUVLRQ )UHT bf 4XHVWLRQ 9HUVLRQ )UHT bf 4XHVWLRQ 9HUVLRQ )UHT bf 6WURQJO\ $JUHH f f f f $JUHH f f f f 6RPHZKDW $JUHH f f f f 1HXWUDO f f f f 6RPHZKDW 'LVDJUHH f f f f 'LVDJUHH f f f f 6WURQJO\ 'LVDJUHH f f f f 'RQnW .QRZf QD f QD f 1R 5HVSRQVH f f f f

PAGE 77

5HOLDELOLWLHV RI 3UHWHVW 0HDVXUHV 7KH VHUYLFH TXDOLW\ TXHVWLRQQDLUH XVHG LQ WKLV VWXG\ FRQWDLQHG IRXU PHDVXUHV f WKH 6(593(5) VFDOH PHDVXULQJ SHUFHLYHG VHUYLFH TXDOLW\ LH LWHPV WKURXJK f f IRXU LWHPV PHDVXULQJ VHUYLFH WLPH SHUFHSWLRQV LH LWHPV WKURXJK f DQG f RYHUDOO VHUYLFH TXDOLW\ DQG f LQWHQGHG IXWXUH EHKDYLRU LWHPV DQG f &KURQEDFKnV DOSKD ZDV XVHG WR DVVHVV WKH UHOLDELOLWLHV RI WKHVH PHDVXUHV 3HUFHLYHG VHUYLFH TXDOLW\ DV PHDVXUHG E\ 6(593(5) KDG DQ DOSKD RI Q f 7KH DOSKD IRU WKH LWHPV HYDOXDWLQJ VHUYLFH WLPH SHUFHSWLRQV ZDV Q f 7KH DOSKD IRU LQWHQGHG IXWXUH EHKDYLRU ZDV Q f 6LQFH 264 ZDV D VLQJOH LWHP PHDVXUH UHOLDELOLW\ FRXOG QRW EH DVVHVVHG KRZHYHU WKH UHOLDELOLWLHV IRU WKH RWKHU PHDVXUHV ZHUH LQ DQ DFFHSWDEOH UDQJH 7DEOHV DQG VKRZ WKH LWHPWRWRWDO VWDWLVWLFV IRU SHUFHLYHG VHUYLFH TXDOLW\ DQG SHUFHLYHG VHUYLFH WLPH PHDVXUHV 2QO\ TXHVWLRQ KDG D FRUUHFWHG LWHPWRWRWDO FRUUHODWLRQ RI OHVV WKDQ KRZHYHU VLQFH WKH GHOHWLRQ RI TXHVWLRQ UHVXOWHG LQ RQO\ D VPDOO LPSURYHPHQW LQ DOSKD IURP WR f WKH LWHP ZDV UHWDLQHG )DFWRU $QDO\VLV RI 6(593(5) 6FDOH $ SULQFLSDO FRPSRQHQWV IDFWRUV DQDO\VLV XVLQJ D 9DULPD[ URWDWLRQ ZDV XVHG WR H[SORUH WKH IDFWRU VWUXFWXUH RI WKH PRGLILHG 6(593(5) VFDOH LQ WKLV VHWWLQJ 7KH DQDO\VLV UHYHDOHG VL[ IDFWRUV ZLWK HLJHQYDOXHV RYHU 722 H[SODLQLQJ DSSUR[LPDWHO\ b RI WKH YDULDQFH 7KH HLJHQYDOXH IRU WKH VHYHQWK IDFWRU ZDV WKHUHIRUH LW ZDV XQOLNHO\ WKDW VHYHQ IDFWRUV ZRXOG KDYH SURGXFHG D EHWWHU VHSDUDWLRQ RI WKH YDULDEOHV 7KH 6FUHH 3ORW SUHVHQWHG LQ )LJXUH VKRZV WKH HLJHQYDOXHV IRU HDFK FRPSRQHQW ZKHUH FRPSRQHQWV QXPEHUHG VHYHQ DQG EHORZ KDYH HLJHQYDOXHV RI OHVV WKDQ RQH $V H[SHFWHG IURP WKH UHOLDELOLW\ WHVWLQJ WKH FRPPXQDOLWLHV RI HDFK RI WKH YDULDEOHV 7DEOH f ZHUH DOO VDWLVIDFWRU\ ZLWK RQO\ WKUHH RI YDULDEOHV KDYLQJ FRPPXQDOLWLHV EHORZ LH 4 4 DQG 4f 7DEOH EHORZ VKRZV WKH URWDWHG FRPSRQHQW PDWUL[ IRU WKH VL[ IDFWRU VROXWLRQ 8QIRUWXQDWHO\ UHSOLFDWLRQ RI WKH IDFWRUV GHPRQVWUDWHG E\ 3DUDVXUDPDQ HW DO f ZDV

PAGE 78

QRW DFKLHYHG +RZHYHU DW OHDVW RQH LWHP IURP HDFK RI WKH K\SRWKHVL]HG GLPHQVLRQV LH WDQJLEOHV UHOLDELOLW\ UHVSRQVLYHQHVV DVVXUDQFH DQG HPSDWK\f GHILQHG HDFK RI WKH UHVSHFWLYH IDFWRUV 7KH GLIIHUHQFHV LQ WKH IDFWRU VWUXFWXUH PD\ KDYH UHVXOWHG IURP D QXPEHU RI VRXUFHV )LUVW WZR RI WKH LWHPV ZHUH UHZRUGHG WR LPSURYH FODULW\ 6HFRQG HLJKW RI WKH LWHPV ZHUH UHZRUGHG IURP VHFRQG SHUVRQ SHUVSHFWLYH WR ILUVW SHUVRQ SHUVSHFWLYH 7KLUG WKH RUGHU RI VRPH RI WKH LWHPV ZDV UHDUUDQJHG WR UHGXFH WKH VHQVH RI UHGXQGDQF\ )RXUWK WKH LWHPV UHODWHG WR WKH WDQJLEOHV GLPHQVLRQ ZHUH UHSODFHG E\ QHZ LWHPV VR LW ZDV QRW GHWHUPLQHG ZKHUH WKHVH QHZ LWHPV ZRXOG ORDG RU KRZ WKHLU FRUUHODWLRQV ZLWK WKH RWKHU LWHPV ZRXOG DIIHFW WKH IDFWRU VWUXFWXUH 7DEOH 3UHWHVW ,WHPWRWDO 6WDWLVWLFV IRU 6(593(5) 6XEVFDOH Q fW ,WHP &RUUHFWHG ,WHP 7RWDO &RUUHODWLRQ $OSKD LI ,WHP 'HOHWHG 45 4 4 4 4 4+5 5 5 5 45 5 5 45 5 A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV

PAGE 79

7DEOH 3UHWHVW ,WHPWRWDO 6WDWLVWLFV IRU 3HUFHLYHG 6HUYLFH 7LPH Q fI ,WHP &RUUHFWHG ,WHP 7RWDO &RUUHODWLRQ $OSKD LI ,WHP 'HOHWHG 5 45 A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV 7DEOH 3UHWHVW ,WHP &RPPXQDOLWLHV ,WHP &RPPXQDOLW\ 1HFHVVDU\ 5HVRXUFHV 45%DFNJURXQG 1RLVH 4:ULWWHQ 0DWHULDOV 46SHDN &OHDUO\ 43URPLVHG 7LPH 46\PSDWKHWLF t 5HDVVXULQJ 4'HSHQGDEOH 4 3URYLGHV LQ 7LPH 4 O5,QGLYLGXDO $WWHQWLRQ 45:KHQ 3HUIRUPHG 453URPSW 6HUYLFH 2O5:LOOLQJQHVV WR +HOS 2,57 %XV\ 47UXVW (PSOR\HHV 453HUVRQDO $WWHQWLRQ 43ROLWH (PSOR\HHV 46DIH ,QWHUDFWLRQV 5.QRZ 1HHGV 45%HVW ,QWHUHVWV 452SHUDWLQJ +RXUV A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV

PAGE 80

RA? Wr ,WHP 43URYLGHV LQ 7LPH 5(f 3URPLVHG 7LPH 5(f 4'HSHQGDEOH 5(f 453URPSW 6HUYLFH 56f 4,57 %XV\ 56f 45:LOOLQJQHVV WR +HOS 56f 453HUVRQDO $WWHQWLRQ (0f 43ROLWH (PSOR\HHV $6f 46DIH ,QWHUDFWLRQV $6f 4:ULWWHQ 0DWHULDOV 7$f O5,QGLYLGXDO (0f $WWHQWLRQ 45:KHQ 3HUIRUPHG 56f 452SHUDOLQJ +RXUV (0f 46\PSDWKHWLF t 7$f 5HDVVXULQJ 45.QRZ 1HHGV (0f 7UXVW (PSOR\HHV $6f 5%HVW ,QWHUHVWV (0f 46SHDN &OHDUO\ 7$f 5%DFNJURXQG 1RLVH 7$f 1HFHVVDU\ 5HVRXUFHV 7$f 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV r 7KH OHWWHUV LQ SDUHQWKHVHV LQGLFDWH WKH GLPHQVLRQ RQ ZKLFK WKH LWHP ORDGHG LQ WKH RULJLQDO 6(5948$/ UHVHDUFK FRQGXFWHG E\ 3DUDVXUDPDQ HW DO f :KHUH (0 (PSDWK\ 56 5HVSRQVLYHQHVV 5( 5HOLDELOLW\ $6 U$VVXUDQFH DQG 7$ 7DQJLEOHV &' 72 & D! JL /X &RPSRQHQW 1XPEHU )LJXUH 6FUHH 3ORW RI 3UHWHVW 'DWD 9DULDEOHV 1 f

PAGE 81

6LPXODWLRQ 'HYHORSPHQW 9HULILFDWLRQ DQG 9DOLGDWLRQ 6LPXODWLRQ ZDV FKRVHQ DV WKH DSSURSULDWH PRGHOLQJ WRRO WR DQVZHU WKH WKUHH VSHFLILF UHVHDUFK TXHVWLRQV SRVHG E\ WKLV VWXG\ IRU WKUHH UHDVRQV )LUVW VLPXODWLRQV PDNH LW HDVLHU WR FODULI\ WKLQNLQJ DERXW V\VWHPV SUREOHPV EHFDXVH WKH IRFXV LV SODFHG RQ GHILQLQJ WKH FRPSRQHQWV RI WKH V\VWHP UDWKHU WKDQ WKH FRPSOH[ LQWHUUHODWLRQVKLSV 6HFRQG VLPXODWLRQ HQDEOHV WKH FRQVLGHUDWLRQ RI WKH LPSDFW RI FKDQJHV LQ DOO WKH IDFWRUV LQIOXHQFLQJ WKH V\VWHP VXFK DV VWDIILQJ OHYHOV VHUYLFH UDWHV DQG DUULYDO UDWHV 7KLUG VLQFH WKHVH FKDQJHV DUH PDGH RQ WKH FRPSXWHU HIIHFWV RI V\VWHPV FKDQJHV FDQ EH HYDOXDWHG DQG WHVWHG ZLWKRXW VXEMHFWLQJ WKH UHDO V\VWHP WR XQQHFHVVDU\ VWUHVV 5HLOO\ HW DO f 7KLV VHFWLRQ GHVFULEHV WKH WHFKQLTXHV XVHG WR FRQVWUXFW YHULI\ DQG YDOLGDWH WKH VLPXODWLRQ PRGHO 0RGHO &RQVWUXFWLRQ 7KH VLPXODWLRQ PRGHO ZDV FRQVWUXFWHG XVLQJ WKH *366+ *HQHUDO 3XUSRVH 6LPXODWLRQ 6\VWHPf VLPXODWLRQ ODQJXDJH RQ DQ 06'26 EDVHG SHUVRQDO FRPSXWHU )RXU VWHSV ZHUH QHFHVVDU\ WR WKH FRQVWUXFWLRQ RI WKH VLPXODWLRQ )LUVW DSSUR[LPDWHO\ KRXUV ZHUH VSHQW E\ WKH SULQFLSDO LQYHVWLJDWRU REVHUYLQJ WKH V\VWHP LQ RUGHU WR JDLQ D ILUVW SHUVRQ SHUVSHFWLYH RI WKH DFWXDO ZRUN SURFHVVHV LQ WKH ',35& 'LIIHUHQW GD\V DQG WLPHV ZHUH VHOHFWHG VR WKDW D EURDG SHUVSHFWLYH ZDV DFKLHYHG 6HFRQG IORZ FKDUWV ZHUH GHYHORSHG GRFXPHQWLQJ WKH VWHSV QHFHVVDU\ WR FRPSOHWH D VHUYLFH WUDQVDFWLRQ )URP WKHVH IORZ FKDUWV LW ZDV EH SRVVLEOH WR LGHQWLI\ WKH QHFHVVDU\ HYHQWV IDFLOLWLHV YDULDEOHV GHFLVLRQV LQSXWV DQG RXWSXWV QHFHVVDU\ WR PRGHO WKH V\VWHP +RRYHU DQG 3HUU\ f 7KLUG WKHVH IORZ FKDUWV ZHUH WUDQVODWHG LQWR EORFN GLDJUDPV WKDW GLUHFWO\ UHSUHVHQWHG SURJUDP FRGH $SSHQGL[ *f )RXUWK DIWHU WKH KLVWRULFDO GDWD VKHHWV WKH GDWD FROOHFWLRQ IRUPV DQG WKH SHUVRQDO LQWHUYLHZV ZHUH DQDO\]HG WKH VLPXODWLRQ FRGH ZDV ZULWWHQ XVLQJ WKH GDWD GHULYHG IURP WKHVH VRXUFHV DV LQSXWV $SSHQGL[ +f

PAGE 82

9HULILFDWLRQ RI WKH 0RGHO 2QFH WKH FRPSXWHU PRGHO ZDV FRQVWUXFWHG LW XQGHUZHQW D YHULILFDWLRQ SURFHVV 9HULILFDWLRQ LV WKH SURFHVV E\ ZKLFK WKH PRGHO LV WHVWHG WR PDNH VXUH WKDW LW LV SHUIRUPLQJ DV LW VKRXOG ,Q RWKHU ZRUGV WKH YHULILFDWLRQ SURFHVV DQDO\]HV ZKHWKHU WKH SURJUDP FRGH fFRUUHFWO\ UHSUHVHQWV WKH PRGHO DVVXPSWLRQV DQG V\VWHP GDWDf &DUVRQ S f 7KUHH WHFKQLTXHV ZHUH XVHG WR YHULI\ WKH PRGHO )LUVW WUDFLQJ WKH VLPXODWLRQ LV D FRPPRQ SURFHVV XVHG WR PDNH VXUH WKDW HQWLWLHV PRYH WKURXJK WKH VLPXODWLRQ DV H[SHFWHG +RRYHU DQG 3HUU\ f 7KH *366+ WRRO FDOOHG WKH LQWHUDFWLYH GHEXJJHU ZKLFK DOORZV IRU LQWHUDFWLYH WUDFHV RI WKH VLPXODWLRQ ZDV XVHG IRU WKLV VWHS RI WKH YHULILFDWLRQ SURFHVV 6FKULEHU f 6HFRQG ORJLFDO UHODWLRQVKLSV LPSRVHG LQ WKH PRGHO ZHUH YHULILHG 7KLV FRQILUPHG WKDW IDFLOLWLHV DQG TXHXHV ZHUH QRW H[FHHGLQJ WKHLU FDSDFLW\ +RRYHU DQG 3HUU\ f 7KLUG VLPXODWLRQ UHVXOWV ZHUH FRPSDUHG WR WKRVH H[SHFWHG E\ DQ DQDO\WLFDO PRGHO 7KLV ZDV DFFRPSOLVKHG E\ FKDQJLQJ WKH SDUDPHWHUV RI WKH PRGHO H J FKDQJLQJ HPSLULFDO GLVWULEXWLRQ WR H[SRQHQWLDO GLVWULEXWLRQV DQG XVLQJ FRQVWDQWV LQVWHDG RI UDQGRP YDULDEOHVf VR WKDW WKH VLPXODWLRQ UHVXOWV FRXOG EH FRPSDUHG WR WKH UHVXOWV PDWKHPDWLFDOO\ GHULYHG IURP NQRZQ HTXDWLRQV +RRYHU DQG 3HUU\ f 0RGHO 9DOLGDWLRQ 7KURXJKRXW WKH SURFHVV RI WKH EXLOGLQJ WKH VLPXODWLRQ WKH VLPXODWLRQ PRGHO XQGHUZHQW D VHULHV RI YDOLGDWLRQ VWHSV 9DOLGDWLRQ LV WKH SURFHVV E\ ZKLFK WKH UHVHDUFKHU GHWHUPLQHV LI WKH PRGHO LV fVXIILFLHQWO\ DFFXUDWH IRU WKH SXUSRVH DW KDQG DQG ZKLFK FDQ EH XVHG DV D VXEVWLWXWH IRU WKH UHDO V\VWHPf &DUVRQ Sf ,Q RWKHU ZRUGV D YDOLG PRGHO FDQ EH XVHG LQ SODFH RI WKH UHDO V\VWHP IRU SXUSRVHV RI DVNLQJ TXHVWLRQV DQG PDNLQJ FRPSDULVRQV &DUVRQ +RRYHU DQG 3HUU\ f 7KUHH WHFKQLTXHV ZHUH XVHG WR YDOLGDWH WKH PRGHO f IDFH YDOLGDWLRQ f H[WUHPHFRQGLWLRQV WHVWV DQG f FRPSDULVRQ RI VLPXODWLRQ RXWSXW WR GDWD IURP WKH UHDO V\VWHP %DOFL +RRYHU DQG 3HUU\ f

PAGE 83

)LUVW IDFH YDOLGDWLRQ ZDV QHFHVVDU\ WR MXGJH ZKHWKHU RU QRW WKH PRGHO VHHPHG UHDVRQDEOH WR WKRVH NQRZOHGJHDEOH DERXW WKH V\VWHP EHLQJ VWXGLHG %DOFL &DUVRQ +RRYHU DQG 3HUU\ f ,Q WZR VHSDUDWH PHHWLQJV WKH FRGLUHFWRUV RI WKH ',35& ZHUH JLYHQ D VWUXFWXUHG ZDONWKURXJK RI WKH VLPXODWLRQ XVLQJ WKH EORFNGLDJUDPV 7KH FRn GLUHFWRUV ZHUH ERWK DVNHG IRXU JHQHUDO TXHVWLRQV f 'R \RX XQGHUVWDQG KRZ WKH PRGHO ZLOO RSHUDWH" f 'RHV WKH PRGHO FRQIRUP WR \RXU NQRZOHGJH RI WKH VHUYLFH SURFHVVHV RI WKH ',35&" f ,V WKHUH DQ\WKLQJ SUHVHQW LQ WKH PRGHO WKDW VHHPV LQFRUUHFW" DQG f ,V WKHUH VRPH SDUW RI WKH VHUYLFH SURFHVV WKDW LV PLVVLQJ IURP WKH PRGHO" &KDQJHV WR WKH VLPXODWLRQ ZHUH PDGH EDVHG RQ WKH FRGLUHFWRUV UHVSRQVHV WR WKHVH TXHVWLRQV 6HFRQG WKH FRPSXWHU PRGHO ZDV YDOLGDWHG XQGHU H[WUHPH FRQGLWLRQV 7R VRPH H[WHQW WKH EHKDYLRU RI D V\VWHP XQGHU H[WUHPH FRQGLWLRQV VKRXOG EH SODXVLEOH DQG FRQIRUP LQ WKH H[SHFWHG GLUHFWLRQ +RRYHU DQG 3HUU\ 6DUJHQW f )RU H[DPSOH LI WKH DUULYDO UDWH LQFUHDVHV GUDPDWLFDOO\ ZH ZRXOG H[SHFW WKDW WKDW OLQH OHQJWK WLPH LQ WKH TXHXH DQG XWLOL]DWLRQ SHUFHQWDJH VKRXOG DOVR LQFUHDVH GUDPDWLFDOO\ 7KLUG RQH RI WKH PRVW SRZHUIXO WHFKQLTXHV IRU YDOLGDWLRQ LV WKH FRPSDULVRQ RI WKH PRGHO WR WKH RULJLQDO V\VWHP &KLVTXDUH JRRGQHVVRIILW WHVWV ZLOO EH XVHG WR DQVZHU TXHVWLRQV FRQFHUQLQJ WKH HTXDOLW\ RI WKH XQGHUO\LQJ GLVWULEXWLRQV ,W ZDV UHFRJQL]HG D SULRUL KRZHYHU WKDW WKH VLPXODWLRQ UHVXOWV ZRXOG SUREDEO\ QRW H[DFWO\ PDWFK WKRVH RI WKH UHDO V\VWHP 7KLV LV EHFDXVH WKH PRGHO LV D VLPSOLILHG YHUVLRQ RI WKH UHDO V\VWHP DQG GLG QRW UHIOHFW PDQ\ HOHPHQWV LQWHQWLRQDOO\ H[FOXGHG IURP WKH PRGHO %DOFL +RRYHU DQG 3HUU\ 6DUJHQW f 7KHUHIRUH UHJUHVVLRQ DQDO\VLV ZDV DOVR FRQGXFWHG WR GHWHFW KRZ PXFK YDULDWLRQ LQ WKH UHDO V\VWHP WKH VLPXODWLRQ DFWXDOO\ SUHGLFWV 9DULDQFH 5HGXFWLRQ 9DULDQFH UHGXFWLRQ LQ VLPXODWLRQ PRGHOV LV LPSRUWDQW EHFDXVH LW KHOSV LPSURYH WKH SRZHU IRU GHWHFWLQJ VLJQLILFDQW VWDWLVWLFDO GLIIHUHQFHV LQ WKH VLPXODWLRQ H[SHULPHQWV

PAGE 84

9DULDQFH UHGXFWLRQ WHFKQLTXHV DOORZ VLPXODWLRQ H[SHULPHQWV WR REWDLQ JUHDWHU SUHFLVLRQ HJ VPDOOHU FRQILGHQFH LQWHUYDOVf ZLWK OHVV VLPXODWLRQ /DZ DQG .HOWRQ f 2QH PHWKRG RI YDULDQFH UHGXFWLRQ FRPPRQO\ XVHG LV DQWLWKHWLF YDULDWLRQ ZKLFK XVHV VWURQJ QHJDWLYH FRYDULDQFHV WR UHGXFH WKH YDULDWLRQ DPRQJ H[SHULPHQWDO UXQV /DZ DQG .HOWRQ 1HHODPNDYLO f 6LPXODWLRQ UXQV ZHUH FRQVWUXFWHG XVHG WKH DQWLWKHWLF YDULDWH FDSDELOLWLHV DYDLODEOH LQ *366+ WKURXJK WKH 508/7 VWDWHPHQWV 'DWD $QDO\VLV 7R IDFLOLWDWH WKH UHSRUWLQJ RI WKH UHVXOWV WKH GDWD DQDO\VLV ZDV EURNHQ GRZQ LQWR IRXU VHSDUDWH SDUWV 7KH ILUVW WZR SDUWV RI WKLV VWXG\ ZHUH HVVHQWLDO WR GHYHORS DQ XQGHUVWDQGLQJ RI WKH ZRUN SURFHVVHV LQ WKH '35& QHFHVVDU\ WR FRQVWUXFW WKH VLPXODWLRQ 7KH UHVXOWV RI WKHVH ILUVW WZR SDUWV DUH UHSRUWHG LQ FKDSWHU ILYH DV SUHOLPLQDU\ GDWD 7KH WKLUG DQG IRXUWK SDUWV WHVWHG WKH UHVHDUFK K\SRWKHVHV DQG VSHFLILF UHVHDUFK TXHVWLRQV GHYHORSHG LQ SUHYLRXV FKDSWHUV 7KH UHVXOWV RI WKHVH DQDO\VHV DUH SUHVHQWHG LQ FKDSWHU VL[ DV PDLQ VWXG\ UHVXOWV $QDO\VLV RI 3UHOLPLQDU\ 'DWD 7KUHH GDWD VRXUFHV ZHUH XVHG WR HYDOXDWH WKH FDOOLQJ SRSXODWLRQ FKDUDFWHULVWLFV DV ZHOO DV WKH DUULYDO DQG VHUYLFH UDWH WUHQGV LQ WKH ',35& f DUFKLYHG KLVWRULFDO GDWD VKHHWV IURP 6HSWHPEHU WKURXJK 0D\ $SSHQGL[ +f f GDWD FROOHFWLRQ IRUPV FROOHFWHG FRQFXUUHQWO\ GXULQJ WKH GDWD FROOHFWLRQ SHULRG IURP -XQH XQWLO $XJXVW $SSHQGL[ ,f DQG f DQ LQWHUQDO GDWDEDVH FRQWDLQLQJ WKH QXPEHU RI TXHVWLRQV DQVZHUHG E\ PRQWK IRU WKH SDVW WHQ \HDUV 7KUHH DQDO\VHV ZHUH FRQGXFHG XVLQJ WKLV GDWD )LUVW WKH FRQVLVWHQF\ EHWZHHQ WKH KLVWRULFDO GDWD DQG WKH FRQFXUUHQW GDWD ZDV HYDOXDWHG 7KH WZR VDPSOHV ZHUH FRPSDUHG E\ SURIHVVLRQ VXEVFULSWLRQ VWDWXV TXHVWLRQ W\SH UHVSRQVH W\SH UHTXHVWHG DQG SHUFHQWDJH RI VHUYLFH GHOD\V 3URSRUWLRQDO GLIIHUHQFHV

PAGE 85

EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW JURXSV ZHUH GHWHFWHG XVLQJ D WZRWDLOHG ]WHVW ZLWK D 6HFRQG WHPSRUDO WUHQGV ZHUH H[DPLQHG E\ PRQWK RI \HDU GD\ RI ZHHN DQG WLPH RI GD\ 2QHZD\ DQDO\VLV RI YDULDQFH $129$f ZDV XVHG WR GHWHFW RYHUDOO GLIIHUHQFHV LQ PHDQV DQG 6FKHIIHf PXOWLSOH FRPSDULVRQV SURFHGXUHV ZHUH XVHG WR DQDO\]H IRU VLJQLILFDQW PHDQ GLIIHUHQFHV EHWZHHQ JURXSV $OSKD IRU WKH $129$ DQG SRVWKRF SURFHGXUHV ZDV VHW DW WKH OHYHO 7KLUG WKH HPSLULFDOO\ REVHUYHG LQWHUDUULYDO DQG VHUYLFH WLPH GLVWULEXWLRQV ZHUH FRPSDUHG ZLWK GLVWULEXWLRQV NQRZQ WR EH XVHIXO LQ PRGHOLQJ V\VWHP EHKDYLRU LH H[SRQHQWLDO DQG :HLEXOOf 7KLV FRPSDULVRQ ZDV FRPSOHWHG XVLQJ .ROPRJRURY6PLUQRY JRRGQHVVRIILW WHVWV DQG UHJUHVVLRQ DQDO\VLV 7KH VHFRQG SDUW GHVFULEHV WKH UHVXOWV IURP SHUVRQDO LQWHUYLHZV FRQGXFWHG ZLWK WKH VWXGHQWV DQG FRGLUHFWRUV ZRUNLQJ LQ WKH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVRXUFH &HQWHU ',35&f 7KHVH GDWD ZHUH XVHG SULPDULO\ WR REWDLQ LQIRUPDWLRQ FRQFHUQLQJ WKH ZRUN SURFHVV DQG WKH SULRULWL]DWLRQ V\VWHP XVHG WR RUJDQL]H WKH ZRUN ,Q DGGLWLRQ SHUFHSWLRQV DERXW FDOOHU SUHIHUHQFHV DQG UHFRPPHQGDWLRQV IRU LPSURYHPHQW ZHUH DOVR REWDLQHG 7KH GDWD IURP HDFK RI WKH WZHOYH DXGLRUHFRUGHG SHUVRQDO LQWHUYLHZV ZHUH FROODWHG DQG FRQWHQW DQDO\]HG EDVHG RQ WKH UHVSRQVHV WR WKH VHPLVWUXFWXUHG LQWHUYLHZ RXWOLQH $QDO\VLV RI 'DWD 5HODWHG WR 0DLQ 6WXG\ 7KH WKLUG SDUW UHSRUWV WKH UHVXOWV RI WKH YDOLGDWLRQ DQG UHOLDELOLW\ WHVWLQJ RI WKH VHUYLFH TXDOLW\ TXHVWLRQQDLUH DQG WKH K\SRWKHVLV WHVWV XVHG WR H[SORUH WKH UHODWLRQVKLSV DPRQJ VHUYLFH WLPH VHUYLFH GHOD\V SHUFHLYHG VHUYLFH WLPH HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ DQG EHKDYLRUDO LQWHQWLRQ LQ WKH GUXJ LQIRUPDWLRQ VHUYLFH VHWWLQJ &KDSWHU WKUHH SUHVHQWHG HLJKW K\SRWKHVHV UHODWHG WR WKHVH YDULDEOHV +\SRWKHVLV D +Df ZDV WHVWHG XVLQJ VLPSOH OLQHDU UHJUHVVLRQ DQG K\SRWKHVLV E +Ef ZDV HYDOXDWHG WKURXJK DQ H[DPLQDWLRQ RI UHVLGXDO SORWV REWDLQHG IURP WKH UHJUHVVLRQ DQDO\VLV 7KH UHPDLQLQJ

PAGE 86

K\SRWKHVHV ZHUH WHVWHG XVLQJ FRUUHODWLRQ WWHVWV DQG RQHZD\ DQDO\VLV RI YDULDQFH $129$f 3HDUVRQ FRUUHODWLRQ FRHIILFLHQWV ZHUH XVHG WR GHWHFW VLJQLILFDQW UHODWLRQVKLSV EHWZHHQ YDULDEOHV +\SRWKHVHV WKDW GLG QRW GHPRQVWUDWH VWDWLVWLFDOO\ VLJQLILFDQW FRUUHODWLRQV EHWZHHQ WKH VWXG\ YDULDEOHV ZHUH UHMHFWHG $GGLWLRQDO DQDO\VLV RI XVLQJ $129$ ZDV FRQGXFWHG IRU K\SRWKHVHV ZLWK VWDWLVWLFDOO\ VLJQLILFDQW FRUUHODWLRQV 6FKHIIHn PXOWLSOH FRPSDULVRQ SURFHGXUHV ZHUH FRQGXFWHG ZKHQ DQ $129$ UHVXOWHG LQ D VLJQLILFDQW )YDOXH LQGLFDWLQJ RYHUDOO GLIIHUHQFHV DPRQJ OHYHO PHDQV $OO DQDO\VHV ZHUH FRQGXFWHG DW WKH OHYHO 7KH IRXUWK SDUW XVHG LQIRUPDWLRQ IURP WKH ILUVW WKUHH SDUWV WR GHYHORS DQG YDOLGDWH WKH VLPXODWLRQ PRGHO XVHG WR RSWLPL]H VHUYLFH FDSDFLW\ LQ WKH ',35& EDVHG RQ WRWDO VHUYLFH WLPH SHUFHQWDJH RI VHUYLFH GHOD\V DQG SHUFHQW XWLOL]DWLRQ 9DOLGDWLRQ DQG YHULILFDWLRQ RI WKH VLPXODWLRQ ZDV FRQGXFWHG XVLQJ UHJUHVVLRQ OLQHDU UHJUHVVLRQ DQG FRPSDULVRQV RI UHOHYDQW b SHUFHQW FRQILGHQFH LQWHUYDOV 'HVFULSWLYH VWDWLVWLFV FRQILGHQFH LQWHUYDOV DQG UDWLR DQDO\VLV ZDV XVHG WR H[SORUH WKH WKUHH UHVHDUFK TXHVWLRQV RXWOLQHG LQ FKDSWHU WKUHH

PAGE 87

&+$37(5 35(/,0,1$5< 5(68/76 2YHUYLHZ $V GLVFXVVHG LQ WKH SUHYLRXV FKDSWHU WKH GDWD DQDO\VLV ZDV EURNHQ LQWR IRXU UHODWHG SDUWV 7KLV FKDSWHU SUHVHQWV WKH SUHOLPLQDU\ UHVXOWV JHQHUDWHG IURP SDUWV RQH DQG WZR RI WKH GDWD 3DUW RQH KDV WKUHH VXEVHFWLRQV )LUVW WKH GDWD JDWKHUHG IURP WKH KLVWRULFDO GDWD VKHHWV ZHUH FRPSDUHG ZLWK WKH FRQFXUUHQW GDWD JDWKHUHG IURP WKH GDWD FROOHFWLRQ IRUPV 7KLV ZDV GRQH WR HVWDEOLVK WKH GHJUHH RI KRPRJHQHLW\ EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW VDPSOHV DQG WR H[SORUH SRWHQWLDO VRXUFHV RI ELDV LQ WKH UHVXOWV 7KH WZR VDPSOHV ZHUH FRPSDUHG E\ SURIHVVLRQ VXEVFULSWLRQ VWDWXV TXHVWLRQ W\SH UHVSRQVH W\SH DQG WKH RFFXUUHQFH RI GHOD\V LQ VHUYLFH 6HFRQG WKH KLVWRULFDO GDWD ZHUH DQDO\]HG IRU WHPSRUDO WUHQGV E\ PRQWK GD\ RI ZHHN DQG WLPH RI GD\ 7KLV ZDV GRQH WR H[SORUH IRU WUHQGV LQ DUULYDOV WKDW FRXOG EH UHDVRQDEO\ UHSURGXFHG E\ WKH VLPXODWLRQ SURJUDP DQG WR LGHQWLI\ SHULRGV IRU ZKLFK WKH VLPXODWLRQ UHVXOWV PD\ QRW EH YDOLG 7KLUG WKH KLVWRULFDO LQWHUDUULYDO DQG VHUYLFH WLPH GLVWULEXWLRQV ZHUH WHVWHG IRU JRRGQHVVRIILW ZLWK NQRZQ SUREDELOLW\ GLVWULEXWLRQV LH H[SRQHQWLDO DQG :HLEXOOf WR GHWHUPLQH LI WKH VLPXODWLRQ FRXOG XVH DSSUR[LPDWHG NQRZQ GLVWULEXWLRQV WR PRGHO HPSLULFDOO\ REVHUYHG LQWHUDUULYDOV DQG VHUYLFH WLPHV 3DUW WZR KDV WZR VXEVHFWLRQV )LUVW WKH LQWHUYLHZV FRQGXFWHG ZLWK WKH FRn GLUHFWRUV RI WKH ',35& DUH VXPPDUL]HG 6HFRQG D VXPPDU\ RI WKH LQWHUYLHZV FRQGXFWHG ZLWK WKH H[WHUQVKLS VWXGHQWV LV SUHVHQWHG

PAGE 88

3DUW 2QH +LVWRULFDO DQG &RQFXUUHQW 'DWD $QDO\VLV E\ 3URIHVVLRQ )RXU W\SHV RI SURIHVVLRQ FDWHJRULHV ZHUH WUDFNHG RQ WKH KLVWRULFDO DQG FRQFXUUHQW GDWD VKHHWV f SKDUPDFLVW3KDUP' f SK\VLFLDQ f QXUVHQXUVH SUDFWLWLRQHU DQG f D PLVFHOODQHRXV FDWHJRU\ FDOOHG RWKHU HJ SK\VLFLDQ DVVLVWDQWV GHQWLVWV QXWULWLRQLVWV ODZ HQIRUFHPHQW HWFf 7DEOH SUHVHQWV WKH IUHTXHQF\ RI RFFXUUHQFH RI WKH IRXU SURIHVVLRQ FDWHJRULHV DORQJ ZLWK WKH LQGLYLGXDO DQG FXPXODWLYH SHUFHQWDJHV WKDW WKH IUHTXHQFLHV UHSUHVHQWHG RXW RI WKH WRWDO QXPEHU RI XVDEOH VDPSOHV 7KH QXPEHU RI XVDEOH GDWD SRLQWV RXW RI WKH WRWDO QXPEHU RI VDPSOHV LV GLVSOD\HG QHDU WKH ERWWRP RI WKH WDEOH DORQJ ZLWK WKH QXPEHU RI PLVVLQJ GDWD SRLQWV 3KDUPDFLVWV UHSUHVHQWHG WKH ODUJHVW SURIHVVLRQDO JURXS WKDW WKH ',35& VHUYLFHG 3KDUPDFLVWV SRVHG b RI WKH TXHVWLRQV LQ WKH KLVWRULFDO JURXS DQG b LQ WKH FRQFXUUHQW JURXS $OWKRXJK WKH SURSRUWLRQV RI SKDUPDFLVWV DQG QXUVHV ZHUH QRW VLJQLILFDQWO\ GLIIHUHQW EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW VDPSOHV LW ZDV GHWHUPLQHG WKDW WKH KLVWRULFDO GDWD KDG D ODUJHU VDPSOH RI SK\VLFLDQV WKDQ WKH FRQFXUUHQW VDPSOH LH b KLVWRULFDOO\ YHUVXV b LQ WKH FRQFXUUHQW VDPSOH S ff DQG D VPDOOHU QXPEHU RI RWKHU SURIHVVLRQDO W\SHV LH b KLVWRULFDOO\ YHUVXV b LQ WKH FRQFXUUHQW VDPSOH S ff +RZHYHU WKH $129$ GLG QRW UHYHDO DQ\ VLJQLILFDQW GLIIHUHQFHV LQ WRWDO VHUYLFH WLPHV UHODWHG WR WKH SURIHVVLRQ RI WKH FDOOHU ) S f UHLQIRUFLQJ WKH KRPRJHQHLW\ RI WKH WZR VDPSOHV $QDO\VLV E\ 6XEVFULSWLRQ 6WDWXV 6XEVFULSWLRQ VWDWXV LV W\SLFDOO\ EURNHQ LQWR WKUHH JURXSV E\ WKH ',35& f QRQn VXEVFULEHUV f VXEVFULEHUV LH WKRVH ZKR KDYH GRQDWHG D VXEVFULSWLRQ IHH WR WKH ',35&f DQG f 8QLYHUVLW\ RI )ORULGD +HDOWK 6\VWHP 8)+6f HPSOR\HHV $QDO\VLV XVLQJ

PAGE 89

]WHVWV UHYHDOHG GLIIHUHQFHV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW VDPSOHV ZLWK UHJDUG WR WKHLU VXEVFULEHU DQG 8)+6 SHUFHQWDJHV S LQ ERWK FDVHVf KRZHYHU QR GLIIHUHQFHV ZHUH GHWHFWHG IRU QRQVXEVFULEHUV 7DEOH f +LVWRULFDOO\ DERXW b RI WKH FDOOHUV ZHUH VXEVFULEHUV b ZHUH QRQVXEVFULEHUV DQG SHUFHQW ZHUH HPSOR\HHV RI WKH 8)+6 ,Q WKH FRQFXUUHQW VDPSOH b ZHUH VXEVFULEHUV b ZHUH QRQVXEVFULEHUV DQG SHUFHQW ZHUH HPSOR\HHV RI WKH 8)+6 6LQFH WKH $129$ GLG QRW UHYHDO DQ\ RYHUDOO GLIIHUHQFHV LQ WRWDO VHUYLFH WLPHV DPRQJ WKH WKUHH VXEVFULSWLRQ VWDWXV JURXSV ) S f WKHUH ZDV QRW PXFK FRQFHUQ WKDW GLIIHUHQFHV LQ VXEVFULSWLRQ VWDWXV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW VDPSOHV ZRXOG ELDV WKH VLPXODWLRQ UHVXOWV 7DEOH 3HUFHQWDJH RI 4XHVWLRQV E\ 3URIHVVLRQ 3URIHVVLRQ +LVWRULFDO 'DWD &RQFXUUHQW 'DWD )UHT 3HUFHQW &XPXO 3HUFHQW )UHT 3HUFHQW &XPXO 3HUFHQW 3KDUPDFLVW 3K\VLFLDQ r 1XUVH13 2WKHU 7RWDO 4XHVWLRQV &RXQWHG 0LVVLQJ 'DWD 7RWDO A 6LJQLILFDQFH GHQRWHV GLIIHUHQFHV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW JURXSV rS rrS rrrS 7DEOH 3HUFHQWDJH RI 4XHVWLRQV E\ 6XEVFULSWLRQ 6WDWXV 6XEVFULSWLRQ 6WDWXV +LVWRULFDO 'DWD &RQFXUUHQW 'DWD )UHT 3HUFHQW &XPXO 3HUFHQW )UHT 3HUFHQW &XPXO 3HUFHQW 6XEVFULEHUrrr 1RQ6XEVFULEHU 8) +HDOWK 6\VWHP r r r 7RWDO 4XHVWLRQV &RXQWHG 0LVVLQJ 'DWD 7RWDO A 6LJQLILFDQFH GHQRWHV GLIIHUHQFHV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW JURXSV rS rrS rrrS

PAGE 90

$QDO\VLV E\ 4XHVWLRQ 7\SH $V GLVFXVVHG LQ WKH SUHYLRXV FKDSWHU WKH ',35& DQVZHUV D YDULHW\ RI GUXJ UHODWHG TXHVWLRQV FRYHULQJ IRXUWHHQ VHSDUDWH FDWHJRULHV 2I WKHVH IRXUWHHQ FDWHJRULHV WKH ILYH ODUJHVW LQ ERWK VDPSOHV ZHUH f GUXJ DYDLODELOLW\ f GUXJ GRVDJH DQG DGPLQLVWUDWLRQ f GUXJ LGHQWLILFDWLRQ f GUXJ WKHUDS\ DQG HIILFDF\ DQG f D PLVFHOODQHRXV FDWHJRU\ FDOOHG 2WKHU 7KHVH ILYH FDWHJRULHV PDGH XS b DQG b RI WKH KLVWRULFDO DQG FRQFXUUHQW TXHVWLRQV DQVZHUHG UHVSHFWLYHO\ 7DEOH f 2QO\ WKUHH TXHVWLRQ FDWHJRULHV GHPRQVWUDWHG VWDWLVWLFDOO\ VLJQLILFDQW SURSRUWLRQDO GLIIHUHQFHV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW VDPSOHV )LUVW TXHVWLRQV FRQFHUQLQJ GUXJ WKHUDS\ DQG HIILFDF\ UHSUHVHQWHG DERXW b LQ WKH KLVWRULFDO VDPSOH KRZHYHU WKH\ RQO\ UHSUHVHQWHG b RI WKH WRWDO TXHVWLRQV LQ WKH FRQFXUUHQW VDPSOH Sf 7KLV ZDV WKH RQO\ TXHVWLRQ RXW RI WKH WRS ILYH WR GLVSOD\ D VLJQLILFDQW GLIIHUHQFH 6HFRQG TXHVWLRQV LQYROYLQJ GUXJ XVH LQ SUHJQDQF\ DQG ODFWDWLRQ UHSUHVHQWHG RQO\ b RI WKH KLVWRULFDO VDPSOH KRZHYHU LW UHSUHVHQWHG b RI WKH FRQFXUUHQW VDPSOH Sf )LQDOO\ TXHVWLRQV FRQFHUQLQJ VLGH HIIHFWV RU DGYHUVH GUXJ HIIHFWV UHSUHVHQWHG b RI WKH KLVWRULFDO VDPSOH EXW RQO\ b RI WKH FRQFXUUHQW VDPSOH S f $OWKRXJK WKH SURSRUWLRQDO GLIIHUHQFHV LQ TXHVWLRQ W\SHV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW VDPSOHV ZDV QRW JUHDW LW ZDV QHFHVVDU\ WR DFFRXQW IRU GLIIHUHQFHV LQ VHUYLFH WLPHV DPRQJ TXHVWLRQ W\SHV )RU DQDO\VLV SXUSRVHV TXHVWLRQ W\SHV UHSUHVHQWLQJ OHVV WKDQ ILYH SHUFHQW RI WKH VDPSOH LQ ERWK WKH KLVWRULFDO DQG FRQFXUUHQW VDPSOHV ZHUH JURXSHG LQWR RQH FDWHJRU\ FDOOHG FRPELQHG $V VXFK VL[ TXHVWLRQ W\SHV ZHUH FODVVLILHG DV FRPELQHG LQYHVWLJDWLRQDO GUXJV ,9 FRPSDWLELOLW\ DQG VWDELOLW\ OHJDO SKDUPDFRNLQHWLFV WR[LFRORJ\ DQG YHWHULQDU\ GUXJV $OWKRXJK TXHVWLRQV IDOOLQJ LQWR WKH GUXJ XVH LQ SUHJQDQF\ DQG ODFWDWLRQ FDWHJRU\ RQO\ UHSUHVHQWHG b RI WKH TXHVWLRQV LQ WKH KLVWRULFDO VDPSOH WKLV FDWHJRU\ ZDV NHSW VHSDUDWH VLQFH LW UHSUHVHQWHG b RI WKH TXHVWLRQV LQ WKH FRQFXUUHQW VDPSOH

PAGE 91

7KH $129$ FRPSDULQJ WKH VHUYLFH WLPHV LH WLPH WR FRPSOHWH D TXHVWLRQf LQ PLQXWHV IRU HDFK RI WKH QLQH TXHVWLRQ JURXSV UHYHDOHG WKDW WKH PHDQ VHUYLFH WLPH IRU DW OHDVW RQH TXHVWLRQ W\SH ZDV VLJQLILFDQWO\ GLIIHUHQW ) Sf 3RVW KRF FDOFXODWLRQV UHYHDOHG WKDW WKH GUXJ LGHQWLILFDWLRQ TXHVWLRQV ZHUH VLJQLILFDQWO\ GLIIHUHQW IURP TXHVWLRQV UHJDUGLQJ GUXJ WKHUDS\ DQG HIILFDF\ S f DQG TXHVWLRQV UHJDUGLQJ GUXJ XVH LQ SUHJQDQF\ DQG ODFWDWLRQ S f 1R RWKHU VLJQLILFDQW GLIIHUHQFHV ZHUH GHWHFWHG 7DEOH SUHVHQWV WKH GHVFULSWLYH VWDWLVWLFV IRU HDFK RI WKHVH FDWHJRULHV DQG )LJXUH JUDSKLFDOO\ LOOXVWUDWHV WKH b FRQILGHQFH LQWHUYDOV IRU VHUYLFH WLPH IRU HDFK RI WKH TXHVWLRQ FDWHJRULHV ,Q DGGLWLRQ WR WKH WZR VLJQLILFDQW GLIIHUHQFHV LQ VHUYLFH WLPHV DERYH WKH FRQILGHQFH LQWHUYDOV DOVR VXJJHVWHG WKDW TXHVWLRQV UHJDUGLQJ VLGH HIIHFWV DQG DGYHUVH GUXJ HIIHFWV WHQGHG WR KDYH ORQJHU VHUYLFH WLPHV WKDQ GUXJ LGHQWLILFDWLRQ TXHVWLRQV $OVR LW DSSHDUHG WKDW TXHVWLRQV IURP WKH 2WKHU FDWHJRU\ DOVR WHQGHG WR KDYH VKRUWHU VHUYLFH WLPHV ZLWK WKH H[FHSWLRQ RI GUXJ LGHQWLILFDWLRQ TXHVWLRQV Q &' ??6?: Z b f2 b 4XHVWLRQ 7\SH &RPELQHG &DWHJRULHV f )LJXUH b &RQILGHQFH ,QWHUYDOV RI 6HUYLFH 7LPH E\ 4XHVWLRQ 7\SH LQ 0LQXWHVf

PAGE 92

7DEOH 3HUFHQWDJH RI 4XHVWLRQV E\ 7\SH 4XHVWLRQ 7\SH +LVWRULFDO 'DWD &RQFXUUHQW 'DWD )UHT 3HUFHQW &XPXO 3HUFHQW )UHT 3HUFHQW &XPXO 3HUFHQW 'UXJ $ YDLODELOLW\ 'UXJ 'RVDJH t $GPLQ 'UXJ ,GHQWLILFDWLRQ 'UXJ ,QWHUDFWLRQV 'UXJ 7KHU t (IILFDF\rrr 'UXJ 8VH LQ 3UHJQDQF\rrr ,QYHVWLJDWLRQDO 'UXJV ,9 &RPSDW RU 6WDELOLW\ /HJDO 2WKHU 3KDUPDFRNLQHWLFV 6LGH (IIHFWV RU $ ,'(Vrr 7R[LFRORJ\ 9HWHULQDU\ 'UXJV 7RWDO 4XHVWLRQV &RXQWHG 0LVVLQJ 'DWD 7RWDO A 6LJQLILFDQFH GHQRWHV GLIIHUHQFHV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW JURXSV rS rrS rrrS 7DEOH 6HUYLFH 7LPHV LQ 0LQXWHV E\ 4XHVWLRQ 7\S H b b &RQILGHQFH &RQILGHQFH 4XHVWLRQ 7\SH 0HGLDQ 0HDQ 6W'HY /RZHU %RXQG 8SSHU %RXQG 'UXJ $ YDLODELOLW\ 'UXJ 'RVDJH t $GPLQ 'UXJ ,GHQWLILFDWLRQ 'UXJ ,QWHUDFWLRQV 'UXJ 7KHU t (IILFDF\ 'UXJ 8VH LQ 3UHJQDQF\ 2WKHU 6LGH (IIHFWV RU $'(V &RPELQHG 2YHUDOO

PAGE 93

%DVHG RQ WKH VHUYLFH WLPH UHVXOWV SUHVHQWHG DERYH TXHVWLRQ FDWHJRULHV ZHUH IXUWKHU FROODSVHG LQWR WKUHH EDVLF JURXSV IRU SXUSRVHV RI WKH VLPXODWLRQ *URXS RQH KDG WKH VKRUWHVW VHUYLFH WLPH SURILOH DQG UHSUHVHQWHG WKH GUXJ LGHQWLILFDWLRQ DQG RWKHU FDWHJRULHV HTXDOLQJ DSSUR[LPDWHO\ b RI WKH TXHVWLRQV EDVHG RQ WKH KLVWRULFDO GDWD *URXS WZR KDG D PRGHUDWH VHUYLFH WLPH SURILOH DQG UHSUHVHQWHG TXHVWLRQV UHODWHG WR GUXJ DYDLODELOLW\ GUXJ GRVDJH DQG DGPLQLVWUDWLRQ GUXJ LQWHUDFWLRQV DQG WKH FRPELQHG FDWHJRULHV HTXDOLQJ DSSUR[LPDWHO\ b RI WKH TXHVWLRQV *URXS WKUHH KDG WKH ORQJHVW VHUYLFH WLPH SURILOH DQG UHSUHVHQWHG WKH GUXJ WKHUDS\ DQG HIILFDF\ GUXJ XVH LQ SUHJQDQF\ DQG ODFWDWLRQ DQG VLGH HIIHFWV DQG DGYHUVH GUXJ HIIHFWV TXHVWLRQ FDWHJRULHV HTXDOLQJ DSSUR[LPDWHO\ b RI WKH TXHVWLRQV 7KH $129$ FRPSDULQJ WKH PHDQ VHUYLFH WLPHV RI WKH WKUHH FRPELQHG TXHVWLRQ W\SHV ZDV VLJQLILFDQW ) Sf LQGLFDWLQJ RYHUDOO GLIIHUHQFHV LQ PHDQV DPRQJ WKH JURXSV 3RVW KRF DQDO\VLV LQGLFDWHG WKDW WKH PHDQ VHUYLFH WLPHV IRU DOO WKUHH FRPELQHG TXHVWLRQ W\SHV ZHUH VLJQLILFDQWO\ GLIIHUHQW IURP RQH DQRWKHU *URXS RQH KDG D PHDQ VHUYLFH WLPH RI PLQXWHV V PLQXWHVf ZKLFK ZDV VLJQLILFDQWO\ GLIIHUHQW IURP ERWK JURXSV WZR DQG WKUHH Sf ZKLFK KDG UHVSHFWLYH PHDQ VHUYLFH WLPHV RI PLQXWHV V PLQXWHVf DQG PLQXWHV V PLQXWHVf $OVR WKH PHDQ VHUYLFH WLPH IRU JURXS WZR ZDV VLJQLILFDQWO\ GLIIHUHQW IURP WKH PHDQ VHUYLFH WLPH IRU JURXS WKUHH S f 7DEOH VKRZV WKH GHVFULSWLYH VWDWLVWLFV IRU WKH WKUHH JURXSV DQG )LJXUH LOOXVWUDWHV WKH QHZ FRQILGHQFH LQWHUYDOV REWDLQHG IURP JURXSLQJ WKH TXHVWLRQV LQ WKLV PDQQHU 7DEOH 6HUYLFH 7LPHV LQ 0LQXWHV 8VLQJ 7KUHH &RPELQHG 4XHVWLRQV 7\SHV LQ 0LQXWHVf 4XHVWLRQ 7\SH 0HGLDQ 0HDQ 6W'HY b &RQILGHQFH /RZHU %RXQG b &RQILGHQFH 8SSHU %RXQG *URXS 2QH *URXS 7ZR *URXS 7KUHH

PAGE 94

*URXS *URXS 7KUHH 4XHVWLRQ 7\SH &DWHJRULHV )LJXUH b &RQILGHQFH ,QWHUYDOV RI 6HUYLFH 7LPH 8VLQJ 7KUHH &RPELQHG 4XHVWLRQ 7\SHV LQ 0LQXWHVf $QDO\VLV E\ 5HVSRQVH 7\SH 5HTXHVWHG :KHQ VWXGHQWV DFFHSWHG FDOOV IURP SUDFWLWLRQHUV WKH FDOOHU ZDV DVNHG KRZ WKH\ ZRXOG OLNH WR UHFHLYH WKH UHVSRQVH WR WKHLU TXHVWLRQ )RXU UHVSRQVH W\SHV ZHUH DYDLODEOH f RUDO f ZULWWHQ f ERWK LH RUDO DQG ZULWWHQf DQG f HLWKHU LH RUDO RU ZULWWHQf 7KH PRVW FRPPRQ W\SH RI UHVSRQVH UHTXHVWHG E\ WKH FDOOHUV ZDV DQ RUDO UHVSRQVH UHSUHVHQWLQJ b LQ WKH KLVWRULFDO VDPSOH DQG b LQ WKH FRQFXUUHQW VDPSOH 7KH VHFRQG PRVW UHTXHVWHG UHVSRQVH W\SH ZDV ZULWWHQ UHSUHVHQWLQJ b LQ WKH KLVWRULFDO VDPSOH DQG b LQ WKH FRQFXUUHQW VDPSOH 7DEOH f 7KH ]WHVWV UHYHDOHG VLJQLILFDQW GLIIHUHQFHV UHJDUGLQJ WKH SURSRUWLRQ RI UHVSRQVHV IURP WKH ERWK DQG HLWKHU UHVSRQVH W\SHV REVHUYHG LQ WKH KLVWRULFDO DQG FRQFXUUHQW JURXSV Sf +RZHYHU VLQFH D VLJQLILFDQW GLIIHUHQFH LQ WRWDO VHUYLFH WLPH ZDV QRW GHWHFWHG EHWZHHQ WKHVH WZR JURXSV S f WKHQ OLWWOH ELDV LV OLNHO\ WR EH LQWURGXFHG

PAGE 95

7KH $129$ UHYHDOHG VLJQLILFDQW VHUYLFH WLPH GLIIHUHQFHV ) O Sf DPRQJ WKH UHVSRQVH W\SHV 7KH SRVW KRF SURFHGXUHV LQGLFDWHG GLIIHUHQFHV EHWZHHQ WKH RUDO DQG ERWK UHVSRQVH W\SHV S f DQG EHWZHHQ WKH RUDO DQG HLWKHU UHVSRQVH W\SHV Sf 7KH b FRQILGHQFH LQWHUYDOV IRU WKH VHUYLFH WLPHV E\ UHVSRQVH W\SH DUH SUHVHQWHG LQ 7DEOH DQG LOOXVWUDWHG LQ )LJXUH 7KHVH GDWD VXJJHVWHG WKDW UHVSRQVHV LQYROYLQJ D ZULWWHQ FRPSRQHQW WHQGHG WR WDNH ORQJHU WKDQ UHVSRQVHV UHTXLULQJ MXVW DQ RUDO UHVSRQVH 7KHVH GLIIHUHQFHV PD\ EH SDUWLDOO\ H[SODLQHG E\ H[DPLQLQJ WKH GLVWULEXWLRQ RI TXHVWLRQ W\SHV E\ UHVSRQVH W\SH UHTXHVWHG 5HFDOOLQJ WKDW GUXJ LGHQWLILFDWLRQ TXHVWLRQV WHQGHG WR KDYH WKH ORZHVW VHUYLFH WLPHV RI WKH HLJKW TXHVWLRQ FDWHJRULHV DQDO\]HG LW LV HYLGHQW IURP 7DEOH WKDW WKLV FDWHJRU\ DOVR KDG WKH KLJKHVW SHUFHQWDJH RI RUDO UHVSRQVHV bf )XUWKHUPRUH TXHVWLRQV UHJDUGLQJ GUXJ WKHUDS\ GUXJ XVH LQ SUHJQDQF\ DQG VLGH HIIHFWV WHQGHG WR KDYH WKH KLJKHVW VHUYLFH WLPHV DQG WKH ORZHVW SHUFHQWDJHV RI RUDO UHVSRQVHV RI b b DQG b UHVSHFWLYHO\ 6LQFH )LJXUH VKRZV WKDW RUDO UHVSRQVHV WHQG WR KDYH WKH ORZHVW VHUYLFH WLPHV LW LV XQGHUVWDQGDEOH WKDW GUXJ LGHQWLILFDWLRQ TXHVWLRQV ZRXOG WHQG WR KDYH ORZHU VHUYLFH WLPHV WKDQ WKH RWKHU TXHVWLRQ FDWHJRULHV 7KHUHIRUH IRU VLPXODWLRQ SXUSRVHV LW ZDV DVVXPHG WKDW GHYLDWLRQV LQ VHUYLFH WLPHV DV D IXQFWLRQ RI UHVSRQVH W\SH DUH H[SODLQHG E\ WKH W\SH RI TXHVWLRQ DVNHG 7DEOH 3HUFHQWDJH RI 4XHVWLRQV E\ 5HVSRQVH 7\SH 5HTXHVWHGA 5HVSRQVH 7\SH +LVWRULFDO 'DWD &RQFXUUHQW 'DWD )UHT 3HUFHQW &XPXO 3HUFHQW )UHT 3HUFHQW &XPXO 3HUFHQW 2UDO :ULWWHQ %RWK r r r (LWKHUrrr 7RWDO 4XHVWLRQV &RXQWHG 0LVVLQJ 'DWD 7RWDO 6LJQLILFDQFH GHQRWHV GLIIHUHQFHV EHWZHHQ WKH KLVWRULFDO DQG FRQFXUUHQW JURXSV r S rrS rrrS

PAGE 96

7DIOH 6HUYLFH 7LPH LQ 0LQXWHV E\ 5HVSRQVH 7\SH 5HTXHVWHG b b &RQILGHQFH &RQILGHQFH 5HVSRQVH 7\SH 0HGLDQ 0HDQ 6W'HY /RZHU %RXQG 8SSHU %RXQG 2UDO :ULWWHQ %RWK (LWKHU 2YHUDOO 7DEOH )UHTXHQF\ DQG 3HUFHQWDJH RI 5HVSRQVH 7\SHV E\ 4XHVWLRQ 7\SH 4XHVWLRQ 7\SH 2UDO :ULWWHQ %RWK (LWKHU 'UXJ $YDLODELOLW\ bf bf bf bf 'UXJ 'RVDJH t $GPLQ bf bf bf bf 'UXJ ,GHQWLILFDWLRQ bf bf bf bf 'UXJ ,QWHUDFWLRQV bf bf bf bf 'UXJ 7KHU t (IILFDF\ bf bf bf bf 'UXJ 8VH LQ 3UHJQDQF\ bf bf bf bf 2WKHU bf bf bf bf 6LGH (IIHFWV RU $'(V bf bf bf bf &RPELQHG bf bf bf bf 2YHUDOO bf bf bf bf 5HVSRQVH 7\SH )LJXUH b &RQILGHQFH ,QWHUYDOV RI 6HUYLFH 7LPH E\ 5HVSRQVH 7\SH LQ 0LQXWHVf

PAGE 97

2FFXUDQFH RI 6HUYLFH 'HOD\V (DFK RI WKH VDPSOHV ZDV HYDOXDWHG WR GHWHUPLQH LI WKHUH KDG EHHQ D GHOD\ LQ VHUYLFH ZKHUH WKH UHVSRQVH WLPH ZDV ORQJHU WKDQ WKH WLPH UHTXHVWHG 7KH ]WHVW GLG QRW UHYHDO DQ\ VLJQLILFDQW GLIIHUHQFHV EHWZHHQ WKH KLVWRULFDO DQG WKH FRQFXUUHQW GDWD JURXSV LQ WHUPV RI WKH SHUFHQWDJH RI GHOD\V 7KH KLVWRULFDO GDWD LQGLFDWHG WKDW RYHUDOO b RI TXHVWLRQV UHVXOW LQ VHUYLFH GHOD\V 7DEOH f 7KH FRQFXUUHQW GDWD ZDV YHU\ VLPLODU LQGLFDWLQJ WKDW b RI WKH TXHVWLRQV HYDOXDWHG GXULQJ WKH VWXG\ SHULRG UHVXOWHG LQ VHUYLFH GHOD\V ,Q RUGHU WR HYDOXDWH ZKHQ GHOD\V RFFXU WKH GLVWULEXWLRQ RI VHUYLFH GHOD\V ZDV H[DPLQHG E\ WKH WLPH UHTXHVWHG IRU WKH IRXU FDWHJRULHV DYDLODEOH RQ WKH GDWD FROOHFWLRQ IRUP LH 6WDW f7RGD\f f'DWHf DQG f1R 5XVKff 7KH UHVXOWV XVLQJ WKH FRQFXUUHQW GDWD LQGLFDWHG WKDW WKRVH TXHVWLRQV UHTXHVWLQJ f6WDWf PLQXWHVf DWWHQWLRQ UHVXOWHG LQ WKH KLJKHVW SHUFHQWDJH RI GHOD\V DW b 7DEOH f 4XHVWLRQV UHTXHVWLQJ DQ DQVZHU WR WKH TXHVWLRQ ZLWKLQ WKH GD\ LH f7RGD\ff ZHUH GHOD\HG b RI WKH WLPH DQG WKRVH UHTXHVWLQJ DQ DQVZHU E\ D VSHFLILF GDWH ZHUH GHOD\HG b RI WKH WLPH f1R UXVKf TXHVWLRQV ZHUH DUELWUDULO\ PDUNHG DV GHOD\HG ZKHQ WRWDO VHUYLFH WLPHV ZHUH JUHDWHU WKDQ WZR ZHHNV bf *LYHQ WKDW WKH PDMRULW\ RI FDOOV UHTXHVWLQJ f6WDWf DWWHQWLRQ DUH GHOD\HG LW LV FOHDU WKDW XQGHU WKH FXUUHQW V\VWHP LQ WKH ',35& LW LV YHU\ GLIILFXOW WR SURYLGH D ILIWHHQ PLQXWH WXUQDURXQG WLPH 7DEOH 3HUFHQWDJH RI 4XHVWLRQV E\ 'HOD\ 6WDWXVI 'HOD\ 6WDWXV +LVWRULFDO 'DWD &RQFXUUHQW 'DWD )UHT 3HUFHQW &XPXO 3HUFHQW )UHT 3HUFHQW &XPXO 3HUFHQW 1R
PAGE 98

7DEOH 3HUFHQWDJH RI 'HOD\V LQ 6HUYLFH E\ 7LPH 1HHGHG 7LPH 1HHGHG 'HOD\ LQ 6HUYLFH 1R )UHT
PAGE 99

GDLO\ QXPEHU RI TXHVWLRQV ZDV Q V f DQG WKH DYHUDJH WRWDO QXPEHU RI TXHVWLRQV SHU PRQWK ZDV Q V f ,Q JHQHUDO KRZHYHU WKHUH DUH QR SUDFWLFDOO\ VLJQLILFDQW GLIIHUHQFHV DPRQJ WKH PRQWK WKDW ZRXOG QHFHVVLWDWH VSHFLDO FRQVLGHUDWLRQ LQ WKH VLPXODWLRQ SURJUDP +RZHYHU 'HFHPEHU UHSUHVHQWV D VSHFLDO FDVH IRU ZKLFK WKH VLPXODWLRQ SURJUDP PD\ QRW EH YDOLG 7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU WKH $YHUDJH 1XPEHU RI 4XHVWLRQV $QVZHUHG E\ 0RQWK IRU WKH 3DVW 7HQ
PAGE 100

2 & /2 2 1 -$1 )(% 0$5 $35 0$< -81 -8/ $8* 6(3 2&7 129 '(& 0RQWK )LJXUH b &RQILGHQFH ,QWHUYDOV RI $YHUDJH 'DLO\ $UULYDOV E\ 0RQWK 7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU 4XHVWLRQ ,QWHUDUULYDO 7LPHV LQ 0LQXWHV E\ 'D\ RI :HHN 'D\ 0HGLDQ 0HDQ 6WG 'HY b /RZHU %RXQG b 8SSHU %RXQG 0RQ 7XH :HG 7KX )UL 2YHUDOO

PAGE 101

,QWHUDUULYDOV E\ 7LPH RI 'D\ :KHQ WKH LQWHUDUULYDOV ZHUH FRPSDUHG E\ WLPH RI GD\ XVLQJ WKH KLVWRULFDO GDWD WKH $129$ UHYHDOHG VLJQLILFDQW GLIIHUHQFHV DPRQJ WKH WLPH LQWHUYDOV H[DPLQHG ) Sf 7KH SRVW KRF DQDO\VHV LQGLFDWHG WKDW WKHUH ZHUH VLJQLILFDQW GLIIHUHQFHV EHWZHHQ WKH PRUQLQJ LQWHUDUULYDOV REVHUYHG GXULQJ WKH DP WR DP WLPH LQWHUYDOV DQG WKH DIWHUQRRQ LQWHUDUULYDOV REVHUYHG GXULQJ WKH WLPH LQWHUYDOV IURP SP WR SP 7DEOH f 7DEOH VKRZV WKH GHVFULSWLYH VWDWLVWLFV IRU HDFK RI WKH REVHUYHG WLPH LQWHUYDOV )LJXUH JUDSKLFDOO\ LOOXVWUDWHV WKH LQWHUDUULYDO WUHQG E\ WLPH RI GD\ 7KH VKRUWHVW LQWHUDUULYDOV RFFXUHG LQ WKH PRUQLQJ VWDUWLQJ DW DP ZLWK D PHDQ LQWHUUDUULYDO WLPH RI PLQXWHV RU DSSUR[LPDWHO\ IRXU SHU KRXU ,QWHUDUULYDO WLPHV GHFUHDVHG WKURXJKRXW WKH PRUQLQJ XQWLO SP GXULQJ ZKLFK WLPH WKH LQWHUDUULYDO WLPH ZDV DSSUR[LPDWHO\ PLQXWHV RU WZR SHU KRXU $IWHU WKH LQWHUDUULYDO WLPHV OHYHOHG RII DW D PHDQ RI DSSUR[LPDWHO\ PLQXWHV RU D OLWWOH EHWWHU WKDQ RQH SHU KRXU 6LQFH WKHVH GLIIHUHQFHV ZHUH SUDFWLFDOO\ VLJQLILFDQW DV ZHOO DV VWDWLVWLFDOO\ VLJQLILFDQW WKH VLPXODWLRQ ZV SURJUDPPHG WR JHQHUDWH VLPXODWHG FDOOV WDNLQJ LQWR FRQVLGHUDWLRQ WKH KRXU RI WKH GD\ ,QWHUDUULYDO DQG 6HUYLFH 7LPH 'LVWULEXWLRQV 7KH FRHIILFLHQW RI YDULDWLRQ &9f LV DOWHUQDWLYH PHDVXUH RI YDULDELOLW\ WKDW RIWHQ JLYHV LQIRUPDWLRQ DERXW WKH IRUP RI D FRQWLQXRXV GLVWULEXWLRQ ,W FDOFXODWHG E\ GLYLGLQJ WKH VWDQGDUG GHYLDWLRQ E\ WKH PHDQ )RU WKH H[SRQHQWLDO GLVWULEXWLRQ WKH &9 VKRXOG EH UHJDUGOHVV RI WKH VFDOH SDUDPHWHU 3f ZKHUH 3 LV DSSUR[LPDWHG E\ WKH PHDQ 6NHZQHVV LV D PHDVXUH RI WKH V\PPHWU\ RI WKH GLVWULEXWLRQV DQG VKRXOG EH HTXDO WR IRU DQ H[SRQHQWLDO GLVWULEXWLRQ /DZ DQG .HOWRQ f 7KH LQWHUDUULYDO GLVWULEXWLRQ ZDV WKH PRVW SRVLWLYH FDQGLGDWH IRU D JRRG ILW ZLWK WKH H[SRQHQWLDO GLVWULEXWLRQ VLQFH LW KDG D &9 RI DQG D VNHZQHVV PHDVXUH RI 7DEOH f +RZHYHU WRWDO VHUYLFH WLPH DQG VHUYLFH WLPHV E\ TXHVWLRQ JURXS LH FRUUHVSRQGLQJ WR WKH WKUHH FRPELQHG JURXSV

PAGE 102

SUHVHQWHG DERYHf ZHUH DOVR JRRG SRVVLELOLWLHV IRU D ILW ZLWK WKH H[SRQHQWLDOO\ GLVWULEXWLRQ VLQFH WKHLU FRHIILFLHQWV RI YDULDWLRQ DQG VNHZQHVV PHDVXUHV ZHUH DOO IDLUO\ FORVH WR WKH GHVLUHG YDOXHV ,W ZDV XQOLNHO\ WKDW D JRRGQHVV RI ILW WHVW ZRXOG LQGLFDWH WKDW WKH XQGHUO\LQJ GLVWULEXWLRQV IRU WKH 7LPH WR D 7DNH &DOO WKH $SSURYDO 7LPH DQG WKH 7LPH WR D 5HWXUQ D &DOO ZHUH H[SRQHQWLDO JLYHQ WKDW WKH VNHZQHVV PHDVXUHV ZHUH DW OHDVW WZR WLPHV WKH GHVLUHG LH 7LPH WR 7DNH &DOO KDG D VNHZQHVV RI $SSURYDO 7LPH KDG D VNHZQHVV HTXDO WR DQG 7LPH WR 5HWXUQ D &DOO KDG D VNHZQHVV HTXDO f 7KH .ROPRJRURY6PLUQRY JRRGQHVVRIILW WHVW .6 WHVWf ZDV XVHG WR FDOFXODWH ZKHWKHU RU QRW WKH LQWHUDUULYDO DQG VHUYLFH WLPH GLVWULEXWLRQV ZHUH H[SRQHQWLDOO\ GLVWULEXWHG 7KH FRPSXWDWLRQDO SURFHGXUHV RXWOLQHG E\ /DZ DQG .HOWRQ f ZHUH XVHG WR FRPSXWH WKH YDOXH RI 'f ZKLFK LQ WXUQ ZDV XVHG WR FDOFXODWH WKH .6 WHVW VWDWLVWLF 7KLV WHVW VWDWLVWLF ZDV WKHQ FRPSDUHG WR D FULWLFDO YDOXH LH F
PAGE 103

7KH :HLEXOO GLVWULEXWLRQ LV DQRWKHU FRQWLQXRXV GLVWULEXWLRQ XVHG H[WHQVLYHO\ WR PRGHO LQWHUDUULYDO DQG VHUYLFH WLPHV DQG KDV D VKDSH VLPLODU WR WKH JDPPD H[SRQHQWLDO DQG (UODQJ GLVWULEXWLRQV GHSHQGLQJ RQ LWV SDUDPHWHU YDOXHV /DZ DQG .HOWRQ f 7KH .6 WHVW ZDV FRPSXWHG IRU WKH ,QWHUDUULYDO 7LPH DQG 7RWDO 6HUYLFH 7LPH GLVWULEXWLRQV LQ RUGHU WR GLVFRYHU LI WKH :HLEXOO GLVWULEXWLRQ RU DQRWKHU GLVWULEXWLRQ RI VLPLODU VKDSH ZRXOG SURYLGH D EHWWHU ILW WKDQ WKH H[SRQHQWLDO GLVWULEXWLRQ 7KH FULWLFDO YDOXH IRU D ODUJH VDPSOH .6 WHVW DW D HTXDO WR IRU WKH :HLEXOO GLVWULEXWLRQ LV DSSUR[LPDWHO\ KRZHYHU WKH WHVW VWDWLVWLFV IRU WKH LQWHUDUULYDO DQG WRWDO VHUYLFH WLPH GLVWULEXWLRQV ZHUH DQG UHVSHFWLYHO\ 7KLV LQGLFDWHG WKDW WKH :HLEXOO GLVWULEXWLRQ LV DFWXDOO\ D SRRUHU ILW WKDQ WKH H[SRQHQWLDO GLVWULEXWLRQ ZLWK UHVSHFW WR LQWHUDUULYDOV DQG WRWDO VHUYLFH WLPH 7KH H[SRQHQWLDO GLVWULEXWLRQ FRXOG VWLOO SUREDEO\ EH XVHG LQ WKH VLPXODWLRQ ZLWKRXW JUHDWO\ MHRSDUGL]LQJ WKH YDOLGLW\ RI WKH PRGHO /LQHDU UHJUHVVLRQ DQDO\VLV EHWZHHQ WKH REVHUYHG DQG H[SHFWHG SUREDELOLW\ GHQVLW\ IXQFWLRQV LQGLFDWHG WKDW WKH H[SRQHQWLDO GLVWULEXWLRQ H[SODLQV DSSUR[LPDWHO\ b RI WKH YDULDQFH LQ LQWHUDUULYDOV f DQG b RI WKH YDULDQFH LQ RYHUDOO VHUYLFH WLPH f 7KLV LPSOLHG WKDW WKH H[SRQHQWLDO GLVWULEXWLRQ ZRXOG SUREDEO\ SURYLGH D UHDVRQDEOH HVWLPDWH RI WKH REVHUYHG GLVWULEXWLRQV HYHQ WKRXJK WKH REVHUYHG GDWD DUH QRW H[SRQHQWLDOO\ GLVWULEXWHG +RZHYHU VLQFH *366+ DOORZV XVHU GHILQHG IXQFWLRQV LW ZDV GHFLGHG WR XVH WKH HPSLULFDO GLVWULEXWLRQV WR SUHVHUYH DFFXUDF\ )UHTXHQF\ KLVWRJUDPV IRU WKH LQWHUDUULYDO WLPHV WRWDO VHUYLFH WLPH DQG VHUYLFH WLPHV IRU WKH WKUHH JHQHUDO TXHVWLRQ FDWHJRULHV DUH SUHVHQWHG LQ )LJXUHV WKURXJK 7DEOH 6LJQLILFDQW 39DOXHV IRU ,QWHUDUULYDO 7LPHV E\ +RXU RI 'D\ +RXU

PAGE 104

7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU 4XHVWLRQ ,QWHUDUULYDO 7LPHV LQ 0LQXWHV E\ +RXU RI 'D\ +RXU 0HGLDQ 0HDQ 6WG 'HY b /RZHU %RXQG b 8SSHU %RXQG 2YHUDOO ,;( f§ X 1 +RXU 5HFHLYHG )LJXUH b &RQILGHQFH ,QWHUYDOV RI ,QWHUDUULYDO 7LPHV LQ 0LQXWHV E\ +RXU RI 'D\

PAGE 105

7DEOH 6XPPDU\ 6WDWLVWLFV IRU ,QSXW 'LVWULEXWLRQV 7LPH 6WG &RHIILFLHQW 'DWD 'LVWULEXWLRQ 0HGLDQ 0HDQ 'HY RI 9DULDWLRQ 6NHZQHVV 6RXUFH ,QWHUDUULYDO 7LPH +LVWRULFDO 7LPH WR 7DNH &DOO &RQFXUUHQW 7RWDO 6HUYLFH 7LPH +LVWRULFDO 6HUYLFH 7LPH *URXS &RQFXUUHQW 6HUYLFH 7LPH *URXS &RQFXUUHQW 6HUYLFH 7LPH *URXS &RQFXUUHQW $SSURYDO 7LPH &RQFXUUHQW 7LPH WR 5HWXUQ &DOO &RQFXUUHQW 7DEOH .ROPRJRURY 6PLUQRY 7HVWV IRU ([SRQHQWLDOO\ 'LVWULEXWHG 9DULDEOHV 7LPH 7HVW 'LVWULEXWLRQ 1 'f 'Q 'f 6WDWLVWLF ,QWHUDUULYDO 7LPH 7RWDO 6HUYLFH 7LPH 6HUYLFH 7LPH *URXS 6HUYLFH 7LPH *URXS 6HUYLFH 7LPH *URXS

PAGE 106

)UHTXHQF\ r6 )UHTXHQF\ ,QWHUDUULYDO 7LPH 0LQXWHVf UH )UHTXHQF\ +LVWRJUDP RI +LVWRULFDO ,QWHUDUULYDO 7LPHV 7 6HUYLFH 7LPH 0LQXWHVf )LJXUH )UHTXHQF\ +LVWRJUDP RI +LVWRULFDO 7RWDO 6HUYLFH 7LPHV

PAGE 107

6HUYLFH 7LPH 0LQXWHVf )LJXUH )UHTXHQF\ +LVWRJUDP RI 6HUYLFH 7LPHV IRU 4XHVWLRQ *URXS 2QH 6HUYLFH 7LPH 0LQXWHVf )LJXUH )UHTXHQF\ +LVWRJUDP RI 6HUYLFH 7LPHV IRU 4XHVWLRQ *URXS 7ZR

PAGE 108

7 6HUYLFH 7LPH 0LQXWHVf )LJXUH )UHTXHQF\ +LVWRJUDP RI 6HUYLFH 7LPHV IRU 4XHVWLRQ *URXS 7KUHH 3DUW 7ZR 6WXGHQW DQG 'LUHFWRU ,QWHUYLHZV 7ZHOYH SHUVRQDO LQWHUYLHZV ZHUH FRQGXFWHG IURP 0D\ WR -XO\ 7KH LQWHUYLHZHHV FRQVLVWHG RI WKH WZR FRGLUHFWRUV IRU WKH ',35& DQG WHQ 3KDUP' VWXGHQWV LQ WKHLU WKLUG URWDWLRQ ZHHN 7KH LQWHUYLHZV ZHUH VHPLVWUXFWXUHG DQG ZHUH FRQGXFWHG XVLQJ LQWHUYLHZ RXWOLQHV $SSHQGLFHV DQG .f 'LUHFWRU ,QWHUYLHZV 7KHUH ZDV D FRQVLGHUDEOH DPRXQW RI FRQVLVWHQF\ EHWZHHQ WKH 6WXGHQW DQG FRn GLUHFWRU UHVSRQVHV WR WKH TXHVWLRQV DVNHG GXULQJ WKH LQWHUYLHZ SURFHVV HVSHFLDOO\ FRQFHUQLQJ WKH KRXUV RI RSHUDWLRQ OXQFK WLPHV DQG WKH SURFHVV IRU SULRULWL]LQJ DQG DQVZHULQJ TXHVWLRQV +RZHYHU WKH FRGLUHFWRUV ZHUH DVNHG VRPH DGGLWLRQDO TXHVWLRQV

PAGE 109

WKDW WKH VWXGHQWV ZHUH QRW DVNHG ,Q DGGLWLRQ VRPH TXHVWLRQV ZHUH DVNHG RI WKH FRn GLUHFWRUV LQ D ZD\ WKDW GHSHQGHG ODUJHO\ RQ H[WHQGHG H[SHULHQFH ZRUNLQJ LQ WKH ',35& 7KH ILUVW TXHVWLRQ WKDW WKH FRGLUHFWRUV ZHUH DVNHG ZDV :KR ZRUNV LQ WKH ',35&" 7KH\ LQGLFDWHG WKDW W\SLFDOO\ WKHUH ZHUH WZR FRGLUHFWRUV D GUXJ LQIRUPDWLRQ UHVLGHQW DQG WKUHH 3KDUP' H[WHUQVKLS VWXGHQWV ZRUNLQJ LQ WKH ',35& 2QH RI WKH FRn GLUHFWRUV LV SULPDULO\ UHVSRQVLEOH IRU WKH RSHUDWLRQDO DVSHFWV RI WKH ',35& DQG WKH RWKHU LV UHVSRQVLEOH IRU WKH HGXFDWLRQDO FRPSRQHQW RI WKH GUXJ LQIRUPDWLRQ URWDWLRQ +RZHYHU WKH\ ERWK VWUHVVHG WKDW WKHLU GXWLHV RYHUODSSHG FRQVLGHUDEO\ HVSHFLDOO\ UHJDUGLQJ WKH HGXFDWLRQDO DVSHFWV RI WKH URWDWLRQ )RU DERXW WZRWKLUGV RI WKH \HDU WKH ',35& KDV D GUXJ LQIRUPDWLRQ UHVLGHQW 7KH GUXJ LQIRUPDWLRQ UHVLGHQW GRHV VSHQG VRPH WLPH DV D IURQW OLQH SHUVRQ DQVZHULQJ SKRQH FDOOV KRZHYHU WKH\ HYHQWXDOO\ PRYH RQ WR IXOILOO D PRUH VXSHUYLVRU\ UROH ZLWK UHJDUG WR WKH H[WHUQVKLS VWXGHQWV (DFK PRQWK WKUHH QHZ 3KDUP' H[WHUQVKLS VWXGHQWV ZRUN D IRXU ZHHN URWDWLRQ 7KH ',35& LV VWDIIHG ZHHNV D \HDU E\ WKHVH 3KDUP' VWXGHQWV KRZHYHU WKHUH PD\ EH RQO\ WZR VWXGHQWV GXULQJ WKH &KULVWPDV EUHDN $GGLWLRQDOO\ WZR PRQWKV RXW RI WKH \HDU D SKDUPDF\ SUDFWLFH UHVLGHQW VSHQGV VRPH WLPH LQ WKH ',35& KRZHYHU WKHLU UROH YDULHV FRQVLGHUDEO\ 8VXDOO\ WKH SKDUPDF\ SUDFWLFH UHVLGHQWV DUH ZRUNLQJ RQ RWKHU SURMHFWV VR RQO\ IURP DERXW D TXDUWHU WR RQH KDOI RI WKHLU WLPH LV VSHQW RQ WKH IURQW OLQH DQVZHULQJ TXHVWLRQV 2FFDVLRQDOO\ WKH ',35& KDV LQWHUQ VWXGHQWV DQG IRUHLJQ YLVLWRUV 7KHVH VWXGHQWV XVXDOO\ MXVW REVHUYH DQG KHOS ZKHUH SRVVLEOH EXW VRPH DUH DGYDQFHG HQRXJK WR WDNH FDOOV DQG DQVZHU TXHVWLRQV 7KH VWXGHQWV DUH HQFRXUDJHG WR ZRUN RQ PRUH WKDQ RQH TXHVWLRQ DW D WLPH PHDQLQJ WKDW WKH VWXGHQWV DUH UHTXLUHG WR UHSULRULWL]H WKHLU ZRUN DV QHFHVVDU\ DQG EH HIILFLHQW ZLWK WKHLU WLPH $GGLWLRQDOO\ WKH FRGLUHFWRUV VWDWHG WKDW VWXGHQWV ZHUH UHTXLUHG WR ILOO RXW D GDWD VKHHW IRU HYHU\ TXHVWLRQ DQG WR KDYH HYHU\ TXHVWLRQ DSSURYHG )XUWKHUPRUH WKH FRGLUHFWRUV VWDWHG WKDW WKH VWXGHQWV ZHUH LQGHHG UHTXLUHG WR LQWHUUXSW WKHLU ZRUN DQG DQVZHU LQFRPLQJ WHOHSKRQH FDOOV DQG DGGHG WKDW WKH\ KDG D WKUHH ULQJ UXOH ZKHUH WKH WHOHSKRQH VKRXOG EH DQVZHUHG ZLWKLQ WKUHH ULQJV 2QH RI WKH FR

PAGE 110

GLUHFWRUV FDOOHG WKLV WKH JUHDWHVW LUULWDQW RI DOO IRU WKH VWXGHQWV ZRUNLQJ LQ WKH FHQWHU EHFDXVH LW UHTXLUHV WKHP WR UHGLUHFW DQG UHIRFXV WKHLU DWWHQWLRQ RQ VRPHWKLQJ QHZ 7KH FRGLUHFWRUV DJUHHG ZLWK WKH VWXGHQWV FRQFHUQLQJ WKH HDVLHVW W\SHV RI TXHVWLRQV WR DQVZHU FLWLQJ GUXJ LGHQWLILFDWLRQ DQG DYDLODELOLW\ TXHVWLRQV DV UHTXLULQJ WKH OHDVW DPRXQW RI ZRUN :KHQ DVNHG DERXW WKH PRVW GLIILFXOW W\SHV RI TXHVWLRQV WKDW VWXGHQWV W\SLFDOO\ KDYH WURXEOH DQVZHULQJ WKH\ LQGLFDWHG WKUHH NLQGV RI TXHVWLRQV )LUVW YDJXH TXHVWLRQV DUH IUXVWUDWLQJ IRU WKH VWXGHQWV EHFDXVH WKHUH LV QR ZD\ WR IRUP D FRQFLVH DQVZHU 6HFRQG TXHVWLRQV WKDW DFWXDOO\ KDYH QR DQVZHU JLYH VWXGHQWV WURXEOH EHFDXVH WKH\ GR QRW NQRZ ZKHQ WR TXLW VHDUFKLQJ 7KLUG TXHVWLRQV LQ ZKLFK WKH VWXGHQW KDV WR V\QWKHVL]H LQIRUPDWLRQ IURP PXOWLSOH VRXUFHV ZKHUH QR RQH KDV DOUHDG\ ZULWWHQ WKH DQVZHU GRZQ QLFH DQG QHDW &RQFHUQLQJ KRZ TXLFNO\ VWXGHQWV EHFRPH VNLOOIXO DW DQVZHULQJ TXHVWLRQV WKH FRn GLUHFWRUV LQGLFDWHG WKDW WKH OHDUQLQJ FXUYH ZDV VWHHS 7KHUH DUH WUHPHQGRXV LPSURYHPHQWV LQ WKH ILUVW IHZ GD\V RI WKH URWDWLRQ DQG PRVW VWXGHQWV XQGHUVWDQG WKH SURFHVV DW WKH EHJLQQLQJ RI WKHLU VHFRQG ZHHN 7KH FRGLUHFWRUV IHOW WKDW WKH PRVW YDOXHG DVSHFW RI WKH VHUYLFH WKDW WKH\ SURYLGH LV D KLJK TXDOLW\ DQVZHU XVLQJ UHVRXUFHV WKDW WKH FDOOHU GRHV QRW KDYH DFFHVV WR RU WLPH WR UHWULHYH DQG WKDW LV GHOLYHUHG LQ D WLPHO\ IDVKLRQ :KHQ DVNHG DERXW WKH OHDVW YDOXHG DVSHFW RI WKHLU VHUYLFH WKH FRGLUHFWRUV LQGLFDWHG WKDW PDQ\ RI WKH FDOOHUV RIWHQ GR QRW ZDQW WR JLYH DOO RI WKH QHFHVVDU\ EDFNJURXQG LQIRUPDWLRQ &DOOHUV GR QRW DOZD\V XQGHUVWDQG WKDW WKH EDFNJURXQG LQIRUPDWLRQ LV QHFHVVDU\ WR GHOLYHU D XVHIXO DQVZHU 2QH RI WKH FRGLUHFWRUV LQGLFDWHG WKDW WKH EDFNJURXQG LQIRUPDWLRQ ZDV LPSRUWDQW LQ GHOLYHULQJ D JRRG UHVSRQVH 7KH PRUH VSHFLILF WKH TXHVWLRQ WKH PRUH VSHFLILF WKH DQVZHU 7KH PRUH JHQHUDO WKH TXHVWLRQ WKH PRUH JHQHUDO WKH DQVZHU DQG ZKDW \RX UHDOO\ ZDQW DV D SUDFWLWLRQHU LV D VSHFLILF DQVZHU

PAGE 111

:KHQ DVNHG DERXW WKH IDFWRUV WKDW WKH\ IHOW LQIOXHQFHG WKH FDOOHUVn SHUFHSWLRQV RI WKH TXDOLW\ RI WKH VHUYLFH WKH FRGLUHFWRUV LQGLFDWHG WKDW WKH ZD\ WKH VWXGHQW LQWHUDFWV ZLWK WKH FDOOHU DQG WKH FDOOHUV H[SHFWDWLRQV DUH WKH WZR PRVW LPSRUWDQW LVVXHV 2QH RI P\ WKHRULHV LV WKDW >ILQGLQJ@ LQIRUPDWLRQ LV UHDOO\ RQO\ RU SHUFHQW RI ZKDW >VWXGHQWV@ GR 7KH SDFNDJLQJ WKH ULEERQV DQG ERZV WKH FRPPXQLFDWLRQ G\QDPLF LV UHDOO\ WKH RWKHU WR SHUFHQW ,I >WKH VWXGHQW@ WKDW >WKH FDOOHU LQWHUDFWV@ ZLWK UHDOO\ WDNHV DQ LQWHUHVW LQ WKH TXHVWLRQ VHHPV WR XQGHUVWDQG ZKDW WKH\ DUH DVNLQJ SURYLGHV D ZHOO GRFXPHQWHG DQG WKRURXJKO\ UHVHDUFKHG >DQVZHU@ DQG FRPPXQLFDWHV WKDW >DQVZHU@ EDFN LQ D FOHDU DQG XQGHUVWDQGDEOH ZD\ WKDW LV WKH SHUIHFW VFHQDULR WKLQN >FDOOHUV SHUFHLYHG VHUYLFH TXDOLW\@ LV PXOWLIDFWRULDO WKLQN D ORW RI LW LV WKHLU H[SHFWDWLRQV ,nP ELJ EHOLHYHU LQ H[SHFWDWLRQV $ ILUVW WLPH FDOOHU ZKR KDV QR H[SHFWDWLRQV WKLQNV ZH KDYH D JUHDW VHUYLFH >LI WKH\ JHW DQ\ NLQG RI LQIRUPDWLRQ@ $ IUHTXHQW FDOOHU ZKR >XVHV@ RXU VHUYLFH D ORW DQG >KDV@ KLJK H[SHFWDWLRQV WR EHJLQ ZLWK WKLQN WKDW WKH\ FRXOG JHW SUHWW\ JRRG LQIRUPDWLRQ DQG WKLQN LW LV PHGLRFUH +RZHYHU DV RQH RI WKH GLUHFWRUV SRLQWHG RXW LI WKH FDOOHUV DUH QRW KDSS\ ZLWK WKH TXDOLW\ RI WKH VHUYLFH WKH\ WHQG QRW WR FDOO EDFN :KHQ DVNHG DERXW ZKHWKHU GHOD\V LQ VHUYLFH LQIOXHQFH WKH FDOOHUVn LPSUHVVLRQV RI TXDOLW\ WKH FRPPHQWV IURP WKH FRGLUHFWRUV ZHUH PL[HG 2QH FRGLUHFWRU VWDWHG WKDW KH GLG QRW EHOLHYH D GHOD\ ZRXOG FDXVH D GHFUHDVH LQ WKH SHUFHLYHG TXDOLW\ EHFDXVH WKH\ WU\ WR VDIHJXDUG DQ\ PLVXQGHUVWDQGLQJV DERXW ZKHQ WKH TXHVWLRQ ZLOO EH DQVZHUHG 6WXGHQWV DUH LQVWUXFWHG WR FRQWDFW FDOOHUV ZKHQ QHFHVVDU\ DQG XSGDWH WKHP RQ SURJUHVV RI WKH TXHVWLRQ +RZHYHU WKH RWKHU GLUHFWRU IHOW WKDW FDOOHUV ZHUH YHU\ VHQVLWLYH WR GHOD\V VWDWLQJ >&DOOHUV DUH@ >Y@HU\ VHQVLWLYH WR >GHOD\V@ PHDQ ZKHWKHU WKH\ UHDOO\ QHHG LW RU QRW LI WKH\ VD\ WKH\ QHHG LW WRGD\ DQG ZH GRQnW JHW LW WR WKHP WRGD\ WKDWnV D PDMRU IDLOXUH RQ RXU SDUW :LWK UHJDUG WR WKH IXWXUH RI WKH ',35& ERWK FRGLUHFWRUV IHOW VRPH SUHVVXUH IURP WKH IDFW WKDW IXQGLQJ IURP WKH KRVSLWDO IRU WKH VHUYLFH KDV EHHQ GURSSLQJ RII IRU WKH ODVW IHZ \HDUV (YHQWXDOO\ WKH\ IHOW WKDW VRPH GHFLVLRQV ZRXOG KDYH WR EH PDGH UHJDUGLQJ

PAGE 112

ZKHWKHU WR OLPLW WKH VHUYLFH WR D VHOHFW JURXS RI FDOOHUV RU ZKHWKHU RU QRW WKH VHUYLFH VKRXOG EH PDUNHWLQJ RQ D IHHIRUVHUYLFH RU FRQWUDFWXDO EDVLV ,I WKH ',35& QHHGHG WR DGG VWDII WR VXSSRUW LQFUHDVHV LQ TXHVWLRQ YROXPH WKH FRn GLUHFWRUV IHOW LW ZRXOG EH SRVVLEOH WR LQFUHDVH WKH VWDIILQJ OHYHO IURP WKUHH VWXGHQWV WR DV PDQ\ DV ILYH VWXGHQWV D PRQWK 0RUH WKDQ ILYH VWXGHQWV PD\ EH SRVVLEOH LQ WKH IXWXUH JLYHQ WKH IDFW WKDW PRUH VWXGHQWV DUH FRPLQJ WKURXJK WKH SKDUPDF\ SURJUDP DQG WKHUH DUH IHZHU URWDWLRQ VLWHV IURP ZKLFK WR FKRRVH 7KH FRGLUHFWRUV DGGHG WKDW WKH SULPDU\ OLPLWDWLRQ ZLWK XVLQJ PRUH VWXGHQWV LV WKH OLPLWHG VXSHUYLVRU\ UHVRXUFHV 7KH\ IHOW WKDW PRUH WKDQ ILYH VWXGHQWV ZRXOG EH GLIILFXOW WR VXSSRUW ERWK RSHUDWLRQDOO\ DQG HGXFDWLRQDOO\ ZLWKRXW DW OHDVW RQH PRUH GUXJ LQIRUPDWLRQ UHVLGHQW 2QH RI WKH FRn GLUHFWRUV IHOW WKH LGHDO VROXWLRQ ZRXOG EH WR KLUH D IXOO WLPH VWDII SKDUPDFLVW WR KHOS DQVZHU SKRQHV VXSHUYLVH VWXGHQWV DQG DSSURYH UHVSRQVHV KRZHYHU WKLV ZRXOG UHTXLUH FRQVLGHUDEOH ILQDQFLDO UHVRXUFHV QRW FXUUHQWO\ DYDLODEOH ,Q DGGLWLRQ FXUUHQW VSDFH UHVWULFWLRQV SUHYHQW PRUH WKDQ ILYH VWXGHQWV IURP ZRUNLQJ SURGXFWLYHO\ LQ WKH ',35& 6WXGHQW ,QWHUYLHZV 7KH ILUVW TXHVWLRQ WKDW WKH VWXGHQWV ZHUH DVNHG ZDV WR GHVFULEH WKHLU UROH LQ WKH ',35& $OO RI WKH VWXGHQWV DQVZHUHG VLPLODUO\ LQGLFDWLQJ WKDW WKH\ ZHUH UHVSRQVLEOH IRU WDNLQJ FDOOV IURP SUDFWLWLRQHUV UHVHDUFKLQJ TXHVWLRQV DQG SURYLGLQJ DQVZHUV WR WKRVH TXHVWLRQV EDFN WR WKH FDOOHUV 7KHUH ZDV QR LQGLFDWLRQ WKDW WKHUH DUH GLIIHUHQFHV EHWZHHQ WKH VWXGHQWV LQ WHUPV RI WKHLU UHVSRQVLELOLWLHV RU WKH MREV WKDW WKH\ ZHUH DVNHG WR FRPSOHWH $OO RI WKH VWXGHQWV XQLIRUPO\ VWDWHG WKDW WKH\ DOO KDG HTXDO UHVSRQVLELOLW\ IRU FRPSOHWLQJ WKH VDPH ZRUN $V RQH VWXGHQW VWDWHG (YHU\ERG\ GRZQ WKHUH LV WUHDWHG WKH VDPH :H DUH DOO H[SHFWHG WR GR WKH VDPH WKLQJV ,W LV D OHDUQLQJ H[SHULHQFH IRU XV EXW DW WKH VDPH WLPH ZH DUH SURYLGLQJ D VHUYLFH

PAGE 113

6WXGHQWV DUH UHTXLUHG WR ZRUN LQ WKH ',35& IURP D P XQWLO SP HDFK GD\ 7KH ',35& LV W\SLFDOO\ RSHQ WR WDNH FDOOV IURP D P XQWLO SP 0RQGD\ WKURXJK )ULGD\ ZLWK WKH H[FHSWLRQ RI FHUWDLQ KROLGD\V VXFK DV 7KDQNVJLYLQJ DQG &KULVWPDV 7KH VWXGHQWV UDUHO\ VWD\ SDVW SP $W FORVLQJ WLPH WKH VWXGHQWV UHDFK D VWRSSLQJ SRLQW SRVW ZKDWHYHU WKH\ DUH ZRUNLQJ RQ RQWR D EXOOHWLQ ERDUG GHVLJQDWHG IRU QHZ DQG XQILQLVKHG TXHVWLRQV DQG WKHQ GHSDUW WKH FHQWHU 7KH ODWHVW WLPH WKDW D VWXGHQW KDV VWD\HG EH\RQG SP ZDV SP KRZHYHU PRVW RI WKH VWXGHQWV VWDWHG WKDW WKH\ UDUHO\ VWD\ SDVW SP 7KH VWXGHQWV DUH LQYROYHG LQ VRPH PRUQLQJ DFWLYLWLHV WKDW SRWHQWLDOO\ LQWHUIHUH ZLWK KRZ VRRQ WKH\ FDQ DFWXDOO\ EHJLQ WR ZRUN RQ TXHVWLRQV UHFHLYHG IURP FDOOHUV 2Q 7XHVGD\ 7KXUVGD\ DQG )ULGD\ WKH VWXGHQWV PXVW SDUWLFLSDWH LQ D MRXUQDO FOXE ZKHUH WKH\ DUH DVNHG WR UHDG DQDO\]H DQG SUHVHQW DUWLFOHV IURP WKH OLWHUDWXUH 2Q :HGQHVGD\ D 4XDOLW\ $VVHVVPHQW SURFHGXUH LV FRQGXFWHG ZKHUH D SHUFHQW UDQGRP VDPSOH RI WKH SUHYLRXV ZHHNnV TXHVWLRQV DUH UHYLHZHG IRU FODULW\ DFFXUDF\ DQG UHVSRQVH TXDOLW\ 7KHVH DFWLYLWLHV EHJLQ DW DP DQG XVXDOO\ FRQFOXGH EHWZHHQ DP DQG DP ZLWK WKH PRVW FRPPRQ FRQFOXGLQJ WLPH RI DERXW DP 7KH VWXGHQWV ZHUH DVNHG ZKDW KDSSHQV GXULQJ WKLV RYHUODSSLQJ WLPH IURP DP XQWLO DSSUR[LPDWHO\ DP ZKHQ WKH\ DUH RFFXSLHG LQ RQH RI WKHVH PHHWLQJV 7KH\ VWDWHG WKDW WKH WHOHSKRQHV ZHUH UHOLJLRXVO\ RSHQHG DW DP ,I WKH WHOHSKRQH ULQJV DQ\ VWXGHQW QRW SUHVHQWLQJ PD\ JHW XS DQG DQVZHU WKH WHOHSKRQH ,I WKH TXHVWLRQ LV DQ XUJHQW TXHVWLRQ WKHQ WKH DQVZHULQJ VWXGHQW ZLOO LPPHGLDWHO\ EHJLQ ZRUN RQ WKH TXHVWLRQ ,I WKH TXHVWLRQ LV QRW XUJHQW LH QRW QHHGHG ZLWKLQ WKH KRXUf WKHQ WKH TXHVWLRQ LV SRVWHG RQ WKH EXOOHWLQ ERDUG IRU ODWHU UHWULHYDO $OO RI WKH VWXGHQWV WDNH D OXQFK KRZHYHU WKH DPRXQW RI WLPH WDNHQ VHHPHG WR YDU\ FRQVLGHUDEO\ IURP PLQXWHV WR WKH IXOO KRXU DOORZHG 7ZR VWXGHQWV VWDWHG WKDW WKH\ XVXDOO\ WDNH OHVV WKDQ DQ KRXU DQG RIWHQ VSHQG WKHLU OXQFK LQ WKH ',35& )LYH RI WKH VWXGHQWV VWDWHG WKDW WKH\ XVXDOO\ WDNH WKHLU OXQFK RXWVLGH WKH ',53& DQG WKDW OXQFK IRU

PAGE 114

WKHP ODVWV IURP WR PLQXWHV 2QO\ RQH VWXGHQW WRRN WKH IXOO KRXU IRU OXQFK HDFK GD\ :KHQ SURPSWHG DOO RI WKH VWXGHQWV LQGLFDWHG WKDW WKH\ ZRXOG KHOS DQVZHU WKH WHOHSKRQHV DQG ZRUN RQ XUJHQW TXHVWLRQV LI WKH\ GHFLGH WR WDNH WKHLU OXQFK LQ WKH ',35& /XQFKHV DUH XVXDOO\ VWDJJHUHG IURP D P XQWLO SP 7KH VWXGHQWV DOVR WU\ WR VFKHGXOH WKHLU OXQFKHV VR WKDW QR RQH LV OHIW E\ KLPVHOI RU KHUVHOI WR DQVZHU WKH WHOHSKRQHV $OO RI WKH VWXGHQWV LQGLFDWHG WKDW WKHLU SULPDU\ UROH LQ WKH ',35& LV WR WDNH FDOOV IURP KHDOWKFDUH SUDFWLWLRQHUV DQG ZULWH GRZQ WKHLU TXHVWLRQV $IWHU WDNLQJ GRZQ WKH TXHVWLRQ WKH\ DUH LQVWUXFWHG WR DVN VRPH SURELQJ TXHVWLRQV LQ RUGHU WR ILQG RXW PRUH VSHFLILFDOO\ ZKHUH WKH TXHVWLRQ RULJLQDWHG DQG ZKDW UHVRXUFHV WKH SUDFWLWLRQHU KDG DOUHDG\ FRQVXOWHG $OVR WKH\ REWDLQ WKH YDULRXV GHPRJUDSKLF LQIRUPDWLRQ QHFHVVDU\ WR FRPSOHWH WKH GDWD VKHHW LH GDWD FROOHFWLRQ IRUPf VXFK DV VXEVFULEHU VWDWXV SURIHVVLRQ DQG FRQWDFW LQIRUPDWLRQ VHH $SSHQGLFHV + DQG ,f 0RVW RI WKLV LQIRUPDWLRQ LV WDNHQ RQ D VHSDUDWH VKHHW RI SDSHU DQG WKHQ WUDQVFULEHG RQWR WKH GDWD FROOHFWLRQ IRUP DIWHU WKH FDOO LV FRPSOHWHG 7KH FDOO LV WKHQ ORJJHG LQWR D ORJERRN IRU WUDFNLQJ SXUSRVHV $W WKLV SRLQW WKH VWXGHQWV GHVFULEHG WKUHH SRVVLEOH URXWHV RI DFWLRQ )LUVW LI WKH TXHVWLRQ LV QRW XUJHQW WKH\ ZLOO WDFN WKH TXHVWLRQ RQWR WKH EXOOHWLQ ERDUG ZLWK WKH RWKHU TXHVWLRQV WKDW KDYH QRW \HW EHHQ VWDUWHG 6HFRQG LI WKH TXHVWLRQ LV XUJHQW DQG WKH TXHVWLRQ WKDW WKH\ DUH FXUUHQWO\ ZRUNLQJ RQ LV QRW XUJHQW WKHQ WKH\ ZLOO VWRS ZRUNLQJ RQ WKH FXUUHQW TXHVWLRQ DQG EHJLQ RQ WKH QHZ TXHVWLRQ 7KLUG LI WKH\ ZHUH DOUHDG\ ZRUNLQJ RQ DQ XUJHQW TXHVWLRQ WKH\ ZRXOG DVN RQH RI WKH RWKHU VWXGHQWV WR WDNH WKH TXHVWLRQ 7KH QH[W VWHS LV WR UHVHDUFK DQG ZULWH D UHVSRQVH WR WKH TXHVWLRQ EDVHG RQ WKH LQIRUPDWLRQ WKH\ KDYH IRXQG )ROORZLQJ WKLV WKH\ EULQJ WKLV UHVSRQVH WR RQH RI WKH FRn GLUHFWRUV RU D TXDOLILHG UHVLGHQW IRU DSSURYDO ,I WKH FRGLUHFWRUV LQGLFDWH WKDW WKH UHVSRQVH QHHGV PRUH ZRUN WKHQ WKH\ FRQWLQXH WR UHVHDUFK DQG UHZRUN WKH DQVZHU 2QFH WKH DQVZHU LV DSSURYHG WKH\ UHVSRQG WR WKH FDOOHU XVXDOO\ E\ WHOHSKRQH EXW VRPHWLPHV E\ ID[ RU OHWWHU )LQDOO\ WKH FDOO LV ORJJHG RXW RI WKH ORJERRN DQG WKH GDWD VKHHW LV SODFHG LQ WKH 4XDOLW\ $VVXUDQFH ELQ

PAGE 115

$OO RI WKH VWXGHQWV VWDWHG WKDW GDWD VKHHWV DUH ILOOHG RXW DQG DSSURYDO LV REWDLQHG IRU HDFK TXHVWLRQ DVNHG UHJDUGOHVV RI WKH TXHVWLRQnV DSSDUHQW VLPSOLFLW\ VXFK DV FDOOHUV DVNLQJ IRU SKRQH QXPEHUV WR SKDUPDFHXWLFDO FRPSDQLHV 6RPHWLPHV FRPSOH[ TXHVWLRQV DUH EURNHQ GRZQ LQWR VHYHUDO VPDOOHU TXHVWLRQV )XUWKHUPRUH LI D FDOO FRPHV LQ WKH VWXGHQWV DUH LQVWUXFWHG WR LQWHUUXSW WKHLU ZRUN DQG DQVZHU WKH WHOHSKRQH $V RQH VWXGHQWV VWDWHG >,WnV@ SUHWW\ PXFK RXU GXW\ LI WKH SKRQH ULQJV WR DQVZHU LW UHJDUGOHVV RI ZKDW ZH DUH GRLQJ XQOHVV ZH KDSSHQ WR EH RQ WKH OLQH ZLWK VRPHRQH HOVH 2QFH ZH UHFHLYH WKH FDOO ZKHWKHU ZHnG JR RQ WR ZRUN RQ LW ULJKW DW WKDW PRPHQW LV D VXEMHFWLYH WKLQJ 7KH VWXGHQWV DUH DOVR LQ FKDUJH RI SULRULWL]LQJ FDOOV LQ WHUPV RI ZKDW JHWV ZRUNHG RQ ZKHQ 7KH VWXGHQWV ZHUH FRQVLVWHQW LQ WKHLU DQVZHUV ZKHQ DVNHG KRZ PDQ\ TXHVWLRQV DW D WLPH WKH\ W\SLFDOO\ KDQGOH $OO RI WKH VWXGHQWV VDLG WKDW WKH\ QHYHU ZRUN RQ PRUH WKDQ RQH TXHVWLRQ DW WKH VDPH WLPH EXW PD\ ZRUN RQ RWKHU TXHVWLRQV ZKLOH WKH\ DUH ZDLWLQJ IRU UHWXUQ SKRQH FDOOV RU ZKHQ KLJKHU SULRULW\ TXHVWLRQV DUULYH $V RQH VWXGHQW GHVFULEHG ,WnV QRW OLNH \RX KDYH WZR SLHFHV RI SDSHU EHVLGH \RX DQG \RX KDYH ERRNV RQ HDFK VXEMHFW WKDWnV QRW KRZ ZH ZRUN 2QH >TXHVWLRQ@ WDNHV SULRULW\ RYHU WKH RWKHU ,WnV QHYHU OLNH DQ\RQH KDV IRXU RU ILYH TXHVWLRQV UXQQLQJ DURXQG >:H DOZD\V@ SXW WKHP EDFN RQ WKH >EXOOHWLQ@ ERDUG WR NHHS WKHP VWUDLJKW :KHQ DVNHG KRZ WKH\ GHFLGH ZKLFK TXHVWLRQ WR ZRUN RQ QH[W DOO RI WKH VWXGHQWV VWDWHG WKDW WKH ILUVW FRQVLGHUDWLRQ LV JLYHQ WR WKH WLPH IUDPH WKDW WKH UHTXHVWRU SUHVHQWHG WR WKHP 3ULPDULO\ WKH VWXGHQWV WHQG WR SD\ SULPDU\ DWWHQWLRQ WR WKH f6WDWf f7RGD\f f'DWHf DQG f1R 5XVKf FDWHJRULHV IRXQG RQ WKH GDWD VKHHWV )URP WKH VWDWHPHQWV PDGH UHJDUGLQJ SULRULWLHV GXULQJ WKH LQWHUYLHZV WKHUH VHHPHG WR EH WKUHH SULRULW\ OHYHOV )LUVW f6WDWf TXHVWLRQV WHQG WR UHFHLYH WKH KLJKHVW SULRULW\ DORQJ ZLWK f7RGD\f TXHVWLRQV QHHGHG ZLWKLQ RQH KRXU 6HFRQG f7RGD\f TXHVWLRQV ZLWKRXW D SDUWLFXODU WLPH DQG f'DWHf TXHVWLRQV QHHGHG RQ WKDW SDUWLFXODU GD\ DUH JLYHQ WKH QH[W KLJKHVW FRQVLGHUDWLRQ 7KH

PAGE 116

ORZHVW SULRULW\ LV JLYHQ WR f'DWHf TXHVWLRQV QRW QHHGHG RQ WKDW SDUWLFXODU GD\ DQG f1R 5XVKf TXHVWLRQV :LWKLQ WKHVH SDUWLFXODU FDWHJRULHV WKRVH TXHVWLRQV WKDW DUH SDWLHQW UHODWHG DV RSSRVHG WR JHQHUDO LQIRUPDWLRQ DUH JLYHQ SULRULW\ 7KHUH ZDV QR HYLGHQFH VXJJHVWLQJ WKDW VWXGHQWV JLYH VSHFLDO SULRULW\ WR TXHVWLRQV EDVHG RQ WKH FDOOHUVn SURIHVVLRQ RU VXEVFULEHU VWDWXV LH VXEVFULEHU QRQVXEVFULEHU RU KHDOWK FHQWHU HPSOR\HHf 7KH VWXGHQWV LQGLFDWHG WKDW RQFH WKH\ EHJLQ WR DQVZHU D TXHVWLRQ WKH\ XVXDOO\ DOVR FRPSOHWH WKH TXHVWLRQ 7KH VWXGHQW ZKR WDNHV D FDOO RIWHQ IHHOV REOLJDWHG WR DQVZHU WKDW TXHVWLRQ 7ZR VWXGHQWV HVWLPDWHG WKDW IRU WR SHUFHQW RI WKH TXHVWLRQV WKDW WKH\ SHUVRQDOO\ UHFRUGHG WKH\ DOVR FRPSOHWHG WKH DQVZHU $V RQH VWXGHQW H[SODLQHG ,WnV DOPRVW HDVLHU MXVW WR ILQLVK XS DQVZHULQJ \RXU RZQ TXHVWLRQV WKDQ WU\ WR JHW LQYROYHG LQ DQVZHULQJ VRPHRQH HOVHnV TXHVWLRQV >7KH\@ PD\ KDYH GRQH D ORW RI UHVHDUFK >RQ WKH TXHVWLRQ@ DQG ORRNHG LQ DUHDV WKDW >DUH@ QRW ZULWWHQ GRZQ >8VXDOO\@ ZKHQ SHRSOH DUH GRLQJ D TXHVWLRQ WKHUH LV D WKRXJKW SURFHVV JRLQJ RQ 7KH VWXGHQWV XQLIRUPO\ VWDWHG WKDW WHOHSKRQH QXPEHUV WR SKDUPDFHXWLFDO FRPSDQLHV DQG GUXJ LGHQWLILFDWLRQ TXHVWLRQV ZHUH WKH HDVLHVW TXHVWLRQV WR DQVZHU +RZHYHU WKH TXHVWLRQ W\SHV WKDW VWXGHQWV VHOHFWHG DV WKH PRVW GLIILFXOW W\SHV WR DQVZHU WHQGHG WR YDU\ )RXU VWXGHQWV PHQWLRQHG WKDW TXHVWLRQV UHJDUGLQJ QHZ DQG IRUHLJQ PHGLFDWLRQV ZHUH WKH PRVW GLIILFXOW 7KUHH VWXGHQWV VWDWHG WKDW WKH PRVW GLIILFXOW TXHVWLRQV LQYROYHG WUHDWPHQWV IRU XQXVXDO GLVHDVH VWDWHV RU ZKHQ OLWWOH LV NQRZQ DERXW WKH WKHUDS\ 7ZR VWXGHQWV VDLG WKDW SKDUPDFRNLQHWLF DQG GRVLQJ TXHVWLRQV ZHUH WKH PRVW GLIILFXOW $GGLWLRQDOO\ WZR VWXGHQWV VDLG WKDW EURDG QRQVSHFLILF TXHVWLRQV ZHUH DOVR YHU\ GLIILFXOW EHFDXVH LW LV KDUG WR JXHVV ZKDW WKH FDOOHU DFWXDOO\ ZDQWV WR NQRZ DERXW WKH WRSLF 6L[ RI WKH VWXGHQWV VDLG WKDW WKH\ ZHUH FRPIRUWDEOH DQVZHULQJ TXHVWLRQV DIWHU WKHLU ILUVW GD\ LQ WKH ',35& 7KUHH RI VWXGHQWV VDLG WKDW DIWHU WKH ILUVW ZHHN WKH\ ZHUH FRPIRUWDEOH 2QH VWXGHQW ZDV VWLOO QRW FRPIRUWDEOH DQVZHULQJ TXHVWLRQV HYHQ DIWHU WKHLU WKLUG ZHHN 6RPH RI WKH VWXGHQWV LQGLFDWHG WKDW WKH\ DUH VWLOO OHDUQLQJ KRZ WR DQVZHU

PAGE 117

FHUWDLQ W\SHV RI TXHVWLRQV HVSHFLDOO\ WKRVH ZKHUH WKH FDOOHU LV DVNLQJ WKHLU SURIHVVLRQDO RSLQLRQ WKDW PD\ RQO\ EH ORRVHO\ VXSSRUWHG E\ WKH OLWHUDWXUH 7KH VWXGHQWV IHOW WKDW DFFXUDWH LQIRUPDWLRQ DQG FRUUHFW DQVZHUV ZHUH WKH PRVW YDOXHG DVSHFW RI WKH VHUYLFH WKDW WKH\ SURYLGH $V RQH VWXGHQW VWDWHG :H SURYLGH D VHQVH RI PHQWDO VHFXULW\ IRU >WKH FDOOHUV@ 7KH VWXGHQWV IHOW WKDW WKH FDOOHUV GLG QRW DOZD\V OLNH WR JLYH WKHLU GHPRJUDSKLF LQIRUPDWLRQ RU KDYLQJ WR ZDLW IRU DQ DQVZHU 7KH\ IHOW WKDW VRPH RI WKH FDOOHUV PLVXQGHUVWRRG WKH NLQGV RI LQIRUPDWLRQ DYDLODEOH LQ WKH ',35& DQG KRZ TXLFNO\ GUXJ LQIRUPDWLRQ FRXOG EH UHWULHYHG $OVR WKH\ IHOW WKDW FDOOHUV ZDQWHG FRQFLVH DQVZHUV DQG GR QRW DSSUHFLDWH DGGLWLRQDO LQIRUPDWLRQ QRW GLUHFWO\ UHOHYDQW WR WKH TXHVWLRQ DVNHG :KHQ DVNHG VSHFLILFDOO\ DERXW IDFWRUV WKDW PRVW LQIOXHQFH WKH FDOOHUVn SHUFHSWLRQV RI WKH TXDOLW\ RI WKH VHUYLFH WKH IROORZLQJ ILYH IDFWRUV ZHUH PHQWLRQHG )LUVW KRZ WKH TXHVWLRQ ZDV DQVZHUHG ZDV PHQWLRQHG VL[ WLPHV 7KLV UHODWHV WR KRZ V\VWHPDWLFDOO\ WKH TXHVWLRQ ZDV DQVZHUHG DQG KRZ WKH UHVSRQVH ZDV SUHVHQWHG 6HFRQG WLPHOLQHVV RU SURPSWQHVV RI WKH DQVZHU ZDV PHQWLRQHG IRXU WLPHV 7KLUG WKH QXPEHU DQG TXDOLW\ RI WKH UHIHUHQFHV XVHG ZDV DOVR PHQWLRQ IRXU WLPHV )RXUWK WKH DWWLWXGH RI WKH VWXGHQW ZKHQ WKH FDOOHU ILUVW FDOOV LQ WR WKH FHQWHU ZDV PHQWLRQHG WZR WLPHV )LIWK WKH DFFXUDF\ LH FRUUHFWQHVVf RI WKH LQIRUPDWLRQ ZDV PHQWLRQHG RQFH 7KXV WKH PRVW LPSRUWDQW IDFWRU IURP WKH VWXGHQWVn SHUVSHFWLYH ZDV WKH VWUHQJWK DQG VW\OH RI WKH JLYHQ UHVSRQVH $V RQH VWXGHQW GHVFULEHG LV@ RQH RI WKHP VD\V GRQfW FDUH >DQG@ WKH RWKHU RQH VD\V WULHG EXW FRXOGQfW ILQG DQ\WKLQJ IRU \RX ,Q RWKHU ZRUGV KRZ \RX ZRUG WKH DQVZHU LV LPSRUWDQW DQG ZKDW \RX FDQ XVH WR FLWH DV \RXU DQVZHU LV LPSRUWDQW 7KH VWXGHQWV DOVR IHOW WKDW WLPHOLQHVV DQG SURPSWQHVV ZDV LPSRUWDQW WR FDOOHUVn SHUFHSWLRQV RI VHUYLFH TXDOLW\ KRZHYHU ZKHQ DVNHG PRUH VSHFLILFDOO\ DERXW GHOD\V LQ

PAGE 118

VHUYLFH WKHUH ZHUH VRPH PL[HG UHVXOWV )LYH VWXGHQWV LQGLFDWHG WKDW FDOOHUV SUREDEO\ ZHUH QRW YHU\ VHQVLWLYH WR GHOD\V LQ VHUYLFH )RXU RI WKH VWXGHQWV VDLG WKDW WKH FDOOHUV UDQJHG IURP VHQVLWLYH WR YHU\ VHQVLWLYH WR GHOD\V LQ VHUYLFH 2QH VWXGHQW VWDWHG WKDW WKH VHQVLWLYLW\ GHSHQGHG RQ WKH SHUVRQ ,I WKH FDOOHU KDV ZRUNHG LQ GUXJ LQIRUPDWLRQ EHIRUH WKHQ WKH\ RIWHQ WHQGHG WR EH D OLWWOH PRUH XQGHUVWDQGLQJ 6RPH RI WKH VWXGHQWV IHOW WKDW PDQDJLQJ WKH FDOOHUV H[SHFWDWLRQV UHJDUGLQJ ZKHQ WKH\ ZHUH JRLQJ WR UHFHLYH D UHVSRQVH ZDV LPSRUWDQW LQ PDLQWDLQLQJ WKH FDOOHUV VDWLVIDFWLRQ DV WZR VWXGHQWV VWDWHG 3HRSOH ZKR DUH VHQVLWLYH DUH WKRVH >ZKR@ FDOO ZDQWLQJ >DQ@ RQ WKH VSRW GRVLQJ UHFRPPHQGDWLRQ 7KH\ DUH D OLWWOH IUXVWUDWHG WKDW \RX FDQnW MXVW RSHQ \RXU ERRN DQG WHOO WKHP WKH DQVZHU ULJKW WKHUH 7KRVH DUH WKH RQHV ZKHUH \RX FDQ VHQVH WKH IUXVWUDWLRQ 7LPH LV YHU\ LPSRUWDQW WR FDOOHUV 7KDW LV KRZ IHHO DERXW LW /LNH ZKHQ >WKH FDOOHU KDV@ D SDWLHQW >LQ WKHLU VWRUH RU RIILFH@ ZKR KDV D TXHVWLRQ IRU WKHP WKH\ JLYH XV D FDOO KRSLQJ WR JHW DQ DQVZHU EHIRUH WKH SDWLHQW OHDYHV 7KDW FRQVWUDLQV WKHP WR FHUWDLQ WLPH OLPLW :KLFK PD\ PDNH WKHP IHHO OLNH WKH\ DUH PRUH GHSHQGHQW RQ XV :KHQ WKH\ DUH PRUH GHSHQGHQW WKH\ DUH PRUH OLNHO\ WR EH GLVDSSRLQWHG LI ZH GRQnW SURYLGH /DVWO\ ZKHQ DVNHG DERXW ZKDW FRXOG EH GRQH WR LPSURYH WKH SURFHVV DW WKH ',35& QRQH RI WKH VWXGHQWV IHOW WKDW WKH\ ZHUH EHLQJ KLQGHUHG LQ DQ\ ZD\ IURP GRLQJ WKHLU MRE DQG ZHUH JHQHUDOO\ YHU\ SOHDVHG ZLWK WKHLU H[SHULHQFHV GXULQJ WKH URWDWLRQ +RZHYHU VL[ JHQHUDO VXJJHVWLRQV ZHUH PDGH )LUVW DOO UHIHUHQFHV VKRXOG EH XS WR GDWH 6HFRQG FDOOHUV VKRXOG EH LQIRUPHG WKDW WKH ',35& LV VWDIIHG E\ VWXGHQWV ZKR FDQQRW DQVZHU TXHVWLRQV ULJKW DZD\ 7KLUG FRQVLGHU VKRUWHQLQJ WKH WLPH DOORZHG IRU OXQFK 6HYHUDO RI WKH VWXGHQWV IHOW WKDW DQ KRXU IRU OXQFK ZDV H[FHVVLYH )RXUWK DW OHDVW WZR PRUH FRPSXWHUV ZLWK XSJUDGHG VRIWZDUH VKRXOG EH PDGH DYDLODEOH ,QWHUQHW DFFHVV VKRXOG EH SURYLGHG RQ DOO FRPSXWHUV VLQFH WKDW LV RIWHQ WKH RQO\ ZD\ WR ILQG LQIRUPDWLRQ RQ KHUEDO DQG IRON UHPHGLHV )LIWK D EHWWHU RULHQWDWLRQ LQ WKH EHJLQQLQJ UHJDUGLQJ ZKHUH WR ILQG

PAGE 119

LQIRUPDWLRQ RQ IRUHLJQ GUXJV ZRXOG EH KHOSIXO LQ UHGXFLQJ WKH DPRXQW RI WLPH WKDW LW LQLWLDOO\ WDNHV LQ ILQGLQJ LQIRUPDWLRQ RQ WKHVH GUXJV

PAGE 120

&+$37(5 0$,1 5(68/76 2YHUYLHZ 7KLV FKDSWHU SUHVHQWV WKH WKLUG DQG IRXUWK SDUWV RI WKH GDWD DQDO\VLV 7KH WKLUG SDUW UHSRUWV WKH UHVXOWV RI WKH GDWD REWDLQHG IURP WKH VHUYLFH TXDOLW\ TXHVWLRQQDLUH 7KH YDOLGLW\ DQG UHOLDELOLW\ RI WKH VHUYLFH TXDOLW\ TXHVWLRQQDLUH LV GLVFXVVHG ILUVW IROORZHG E\ WKH UHVXOWV RI WKH K\SRWKHVLV WHVWV UHODWHG WR WKH YDULDEOHV GHULYHG IURP WKH TXHVWLRQQDLUHV DQG GDWD FROOHFWLRQ IRUPV 7KH IRXUWK SDUW GHVFULEHV WKH GHYHORSPHQW YHULILFDWLRQ DQG YDOLGDWLRQ RI WKH VLPXODWLRQ PRGHO ,W DOVR SUHVHQWV UHSRUWV WKH UHVXOWV RI WKH VLPXODWHG H[SHULPHQWV XVHG WR H[SORUH WKH WKUHH VSHFLILF UHVHDUFK TXHVWLRQV 3DUW 7KUHH 5HODWLRQVKLSV $PRQJ 364 262 %HKDYLRUDO ,QWHQWLRQ 3HUFHLYHG 6HUYLFH 7LPH $FWXDO 6HUYLFH 7LPH DQG 6HUYLFH 'HOD\V 5HVSRQVH WR 4XHVWLRQQDLUH 7KUHH KXQGUHG WZHQW\ QLQH TXHVWLRQQDLUHV ZHUH PDLOHG RXW GXULQJ WKH LQLWLDO PDLOLQJ $SSUR[LPDWHO\ RQH ZHHN DIWHU WKH LQLWLDO PDLOLQJ D UHPLQGHU SRVWFDUG ZDV VHQW RXW WR QRQUHVSRQGHUV XUJLQJ WKHP WR PDLO LQ WKH FRPSOHWHG TXHVWLRQQDLUH 7KH SRVW FDUG DOVR DVNHG VXEMHFWV WR FRQWDFW WKH UHVHDUFKHU LI WKH\ KDG QRW UHFHLYHG D VXUYH\ 7ZR ZHHNV DIWHU WKH SRVW FDUG ZDV PDLOHG QRQUHVSRQGHUV ZHUH FRQWDFWHG E\ WHOHSKRQH WR UHPLQG WKHP WR VHQG LQ WKHLU TXHVWLRQQDLUH RU WR DVFHUWDLQ WKHLU UHDVRQV IRU QRW UHVSRQGLQJ 2I WKH TXHVWLRQQDLUHV WKDW ZHUH VHQW RXW ZHUH FRPSOHWHG DQG UHWXUQHG UHVXOWLQJ LQ D UDZ UHVSRQVH UDWH RI b (LJKW PDLOHG TXHVWLRQQDLUHV DQG RQH SRVWFDUG ZHUH UHWXUQHG DV XQGHOLYHUDEOH :KHQ DWWHPSWV WR FRQWDFW QRQUHVSRQGHUV E\ WHOHSKRQH

PAGE 121

ZHUH PDGH LW ZDV OHDUQHG WKDW WZR VXEMHFWV ZHUH RQ YDFDWLRQ DQG WHQ VXEMHFWV QR ORQJHU ZRUNHG DW WKH ORFDWLRQ ZKHUH WKH TXHVWLRQQDLUH ZDV PDLOHG WKHUHIRUH WKHVH VXEMHFWV QHYHU UHFHLYHG WKH VXUYH\ $GMXVWLQJ WKH UHVSRQVH UDWH IRU WKHVH VXEMHFWV XVLQJ WKH SURFHGXUH SURSRVHG E\ 'LOOPDQ f UHVXOWHG LQ D UHYLVHG UHVSRQVH UDWH RI b IRU WKH PDLQ TXHVWLRQQDLUH 7DEOH 4XHVWLRQQDLUH 6DPSOH 'HVFULSWLRQ &KDUDFWHULVWLF )UHTXHQF\ bf b 5HVSRQGLQJ 6XEVFULEHU 6WDWXV +HDOWK 6\VWHP f 6XEVFULEHU f 1RQ6XEVFULEHU f 3URIHVVLRQ 53K3KDUP' f 3K\VLFLDQ f 511XUVH 3UDFW f 2WKHU f )UHTXHQF\ RI 8VH )LUVW 7LPH 8VHU f 7LPHV
PAGE 122

UHVSRQVHV FDPH IURP SKDUPDFLVWV bf 3KDUPDFLVWV DOVR KDG WKH KLJKHVW UDZ UHVSRQVH UDWH bf ZKLOH SK\VLFLDQV KDG WKH ORZHVW UHVSRQVH UDWH bf 4XHVWLRQV DUULYHG ZLWKLQ RQH RI WKUHH UHVSRQVH LQWHUYDOV )LUVW UHVSRQVHV bf ZHUH UHFHLYHG EHIRUH WKH UHPLQGHU SRVWFDUG ZDV VHQW RXW 6HFRQG bf RI WKH UHVSRQVHV ZHUH UHFHLYHG DIWHU WKH SRVWFDUG ZDV VHQW EXW EHIRUH WKH UHPLQGHU WHOHSKRQH FDOO ZDV PDGH 7KLUG WKH UHPDLQLQJ bf TXHVWLRQQDLUHV ZHUH UHFHLYHG DIWHU WKH WHOHSKRQH UHPLQGHU $VVHVVPHQW RI 1RQ5HVSRQVH %LDV 7KH PRVW FRPPRQ UHDVRQ JLYHQ IRU QRW UHVSRQGLQJ WR WKH VXUYH\ ZDV ODFN RI WLPH )RXUWHHQ VXEMHFWV VWDWHG WKDW WKH\ KDG QRW KDG WKH WLPH EXW ZRXOG ILOO RXW DQG PDLO WKH TXHVWLRQQDLUH VRRQ 7DEOH f 2I WKHVH IRXUWHHQ VHYHQ bf DFWXDOO\ UHWXUQHG WKH TXHVWLRQQDLUH DV SURPLVHG $QRWKHU VXEMHFWV VWDWHG WKDW WKH\ KDG QRW \HW UHFHLYHG D TXHVWLRQQDLUH LQ WKH PDLO DQG DVNHG WR KDYH DQRWKHU TXHVWLRQQDLUH VHQW WR WKHP 4XHVWLRQQDLUHV ZHUH UHPDLOHG WR WKHVH LQGLYLGXDOV DQG HLJKW bf ZHUH UHWXUQHG 7ZR VXEMHFWV VWDWHG WKDW WKH\ QHYHU XVHG WKH VHUYLFH HYHQ WKRXJK WKH GDWD VKHHW OLVWHG WKHLU QDPH DQG DGGUHVV 7KH RQO\ DVFHUWDLQDEOH UHDVRQ IRU WKLV ZDV WKDW VXEVFULEHUV VRPHWLPHV DOORZ RWKHU FRZRUNHUV WR ERUURZ WKHLU QDPH DQG VXEVFULEHU QXPEHU WR FDOO WKH ',35& 7ZR VXEMHFWV GLG QRW ILOO RXW WKH TXHVWLRQQDLUH EHFDXVH WKH\ GLG QRW IHHO LW DSSOLHG WR WKHP VLQFH WKH\ ZHUH QRW KHDOWK SURIHVVLRQDOV 2QO\ WZR VXEMHFWV FRQWDFWHG JDYH DQ\ GLUHFW LQGLFDWLRQ WKDW WKH\ DEVROXWHO\ GLG QRW ZDQW WR ILOO RXW WKH TXHVWLRQQDLUH DQG WKH\ GLG QRW ZDQW WR JLYH DQ\ VSHFLILF UHDVRQV IRU WKHLU GHFLVLRQ $129$ SURFHGXUHV ZHUH FRQGXFWHG IRU VXEVFULEHU VWDWXV SURIHVVLRQ DQG UHVSRQVH LQWHUYDO XVLQJ SHUFHLYHG VHUYLFH TXDOLW\ 364f DQG RYHUDOO VHUYLFH TXDOLW\ 264f DV WKH GHSHQGHQW YDULDEOHV 1RQH RI WKH DQDO\VHV SHUIRUPHG UHVXOWHG LQ VLJQLILFDQW RYHUDOO

PAGE 123

) YDOXHV 7DEOH f 7KHUHIRUH WKHUH ZDV QR UHDVRQ WR VXVSHFW UHVSRQVH ELDV VLJQLILFDQW HQRXJK WR GDPDJH WKH H[WHUQDO YDOLGLW\ RI WKH UHVXOWV GHULYHG IURP WKH TXHVWLRQQDLUH GDWD 7DEOH 5HDVRQV IRU 1RW 5HSRQGLQJ 5HDVRQ )UHTXHQF\ b 5HWXUQHGfr +DYHQnW KDG WLPH EXW ZLOO GR LW VRRQ bf 'LG QRW UHFHLYH LQ PDLO PDLO DQRWKHUf bf 3HUVRQ QR ORQJHU ZRUNV WKHUH 0DLOHG WKH TXHVWLRQQDLUH DOUHDG\ 3HUVRQ LV D SDUWWLPH ZRUNHU 4XHVWLRQQDLUH GLG QRW DSSO\ WR PH 1RW LQWHUHVWHG LQ ILOOLQJ RXW TXHVWLRQQDLUH 'LG QRW XVH \RXU VHUYLFH 2Q YDFDWLRQ RU PDWHUQLW\ OHDYH r1RWH :KHUH SUHVHQW SHUFHQW UHWXUQHG HTXDOV WKH UHVSRQVH UDWH DIWHU WKH WHOHSKRQH IROORZ XS DWWHPSW ZDV PDGH WR UHDFK WKH VXEMHFW 7DEOH 5HVXOWV RI 2QH:D\ $129$ 3URFHGXUHV 0HDVXULQJ 5HVSRQVH %LDV 6XEVFULSWLRQ 6WDWXV 3URIHVVLRQ 5HVSRQVH ,QWHUYDO 0HDVXUH ) SYDOXHf ) SYDOXHf ) SYDOXHf 364 f f f 264 f f f 'LPHQVLRQDOLW\ RI WKH 6(593(5) 6XE6FDOH 7KH GLPHQVLRQDOLW\ RI WKH 6(593(5) VXEVFDOH PHDVXULQJ 364 ZDV DVVHVVHG ZLWK SULQFLSDO FRPSRQHQWV IDFWRU DQDO\VLV XVLQJ D 9DULPD[ URWDWLRQ 7KH DQDO\VLV UHVXOWHG LQ IRXU IDFWRUV ZLWK HLJHQYDOXHV RI JUHDWHU WKDQ RQH H[SODLQLQJ b RI WKH YDULDQFH 7KH URWDWHG FRPSRQHQW PDWUL[ LV SUHVHQWHG LQ 7DEOH $ IDFWRU ORDGLQJ LV HVVHQWLDOO\ WKH FRUUHODWLRQ EHWZHHQ WKH LWHP DQG WKH IDFWRU &URFNHU DQG $OJLQD f 7KH PDMRULW\ RI WKH IDFWRU ORDGLQJV IRU HDFK IDFWRU ZHUH FORVH WR RU DERYH ,WHPV ILYH %DFNJURXQG 1RLVH VL[ :ULWWHQ 0DWHULDOV VHYHQ 6SHDN &OHDUO\ DQG WZHQW\RQH .QRZ 1HHGV KDG WKH ORZHVW IDFWRU ORDGLQJV RI

PAGE 124

DQG UHVSHFWLYHO\ 2I WKHVH KRZHYHU RQO\ LWHP ILYHnV IDFWRU ORDGLQJ ZDV EHORZ WKH GHOHWLRQ WKUHVKROG RI ,WHP ILYH DOVR KDG D UHODWLYHO\ ORZ FRPPXQDOLW\ 7DEOH f RI LQGLFDWLQJ WKDW WKH FRPPRQ IDFWRUV GLG QRW H[SODLQ WKH PDMRULW\ RI WKH YDULDQFH IRU WKLV LWHP 7KH ORZ IDFWRU ORDGLQJ FRPELQHG ZLWK WKH ORZ FRPPXQDOLW\ VXJJHVWHG WKDW LWHP ILYH ZDV QRW D YDOLG LWHP )RUFLQJ WKH GDWD WR ILW ILYH IDFWRUV FRQILUPHG WKLV FRQFOXVLRQ VLQFH LWHP ILYH ZDV WKH RQO\ LWHP WR ORDG RQ WKH ILIWK IDFWRU $GGLWLRQDOO\ WKH SRVLWLRQV RI WKH RWKHU LWHP ORDGLQJV UHPDLQHG XQFKDQJHG ,W ZDV WKHUHIRUH GHFLGHG WR UHPRYHG LWHP ILYH IURP WKH PRGHO 7DEOH ,QLWLDO 5RWDWHG &RPSRQHQW 0DWUL[ RI 0DLQ 4XHVWLRQQDLUH 5HVSRQVHV IRU WKH 6(593(5) 6XE6FDOH 1 fI ,WHP 45OQGLYLGXDO $WWHQWLRQ 453HUVRQDO $WWHQWLRQ 5:KHQ 3HUIRUPHG 453URPSW 6HUYLFH 2,57 %XV\ 5:LOOLQJQHVV WR +HOS 45%HVW ,QWHUHVWV 46SHDN &OHDUO\ 43URPLVHG 7LPH 3URYLGHV LQ 7LPH 4'HSHQGDEOH 46\PSDWKHWLF t 5HDVVXULQJ :ULWWHQ 0DWHULDOV 46DIH ,QWHUDFWLRQV 43ROLOH (PSOR\HHV 47UXVW (PSOR\HHV 5SHUDWLQJ +RXUV 1HFHVVDU\ 5HVRXUFHV 5.QRZ 1HHGV 45%DFNJURXQG 1RLVH A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV

PAGE 125

,OO 7DEOH 0DLQ 4XHVWLRQQDLUH &RPPXQDOLWLHV IRU WKH 6(593(5) 6XE6FDOH 1 fI ,WHP &RPPXQDOLW\ 41HFHVVDU\ 5HVRXUFHV 45%DFNJURXQG 1RLVH 4:ULWWHQ 0DWHULDOV 6SHDN &OHDUO\ 3URPLVHG 7LPH 46\PSDWKHWLF t 5HDVVXULQJ 4'HSHQGDEOH 4 3URYLGHV LQ 7LPH 5,QGLYLGXDO $WWHQWLRQ 5:KHQ 3HUIRUPHG 4O 53URPSW 6HUYLFH 4O5:LOOLQJQHVV WR +HOS 4,57 %XV\ 4O 7UXVW (PSOR\HHV 453HUVRQDO $WWHQWLRQ 43ROLWH (PSOR\HHV 46DIH ,QWHUDFWLRQV 5.QRZ 1HHGV 45%HVW ,QWHUHVWV 5SHUDWLQJ +RXUV A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV 7DEOH VKRZV WKH ILQDO URWDWHG FRPSRQHQW PDWUL[ DIWHU UHPRYLQJ LWHP ILYH IURP WKH DQDO\VLV 7KLV PRGHO DOVR UHYHDOHG IRXU IDFWRUV ZLWK WKH H[SODLQHG YDULDQFH LPSURYLQJ VOLJKWO\ WR b $ VFUHH SORW VKRZV WKDW LQGHHG WKH HLJHQYDOXHV IRU WKH FRPSRQHQWV EHJLQ WR OHYHO RII DIWHU IRXU FRPSRQHQWV )LJXUH f 6LQFH WKH HLJHQYDOXH IRU WKH ILIWK FRPSRQHQW ZDV DQG QRW FORVH WR WKH IRXUIDFWRU VROXWLRQ ZDV VHOHFWHG DV WKH RSWLPXP IDFWRU VWUXFWXUH )LYH GLPHQVLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ FRPSRVHG WKH RULJLQDO 6(5948$/ VFDOH 3DUDVXUDPDQ HW DO f WDQJLEOHV 7$f UHVSRQVLYHQHVV 56f UHOLDELOLW\ 5(f DVVXUDQFH $6f DQG HPSDWK\ (0f 1H[W WR WKH IDFWRU ORDGLQJ IRU HDFK LWHP OLVWHG LQ 7DEOH LV D WZR OHWWHU FRGH FRUUHVSRQGLQJ WR WKH IDFWRU RQ ZKLFK WKH LWHP VKRXOG ORDG DV K\SRWKHVL]HG E\ WKH RULJLQDO 6(5948$/ GLPHQVLRQV

PAGE 126

,QWHUHVWLQJO\ WKHUH LV VRPH FRUUHVSRQGHQFH EHWZHHQ WKH RULJLQDO ILYH GLPHQVLRQV UHSRUWHG E\ 3DUDVXUDPDQ HW DO f DQG WKH ILYH IDFWRUV SUHVHQWHG LQ 7DEOH $OO RI WKH LWHPV UHSUHVHQWLQJ DVVXUDQFH DQG UHOLDELOLW\ ORDGHG RQ VHSDUDWH GLPHQVLRQV +RZHYHU LWHPV RULJLQDOO\ EHORQJLQJ WR WKH UHVSRQVLYHQHVV GLPHQVLRQ ORDGHG ZLWK WKUHH RI WKH LWHPV IURP WKH HPSDWK\ GLPHQVLRQ LWHP ,QGLYLGXDO $WWHQWLRQ LWHP 3HUVRQDO $WWHQWLRQ DQG LWHP %HVW ,QWHUHVWVf SHUKDSV LQGLFDWLQJ WKDW FDOOHUV GR QRW VHH WKHVH GRPDLQV DV VHSDUDWH FRQVWUXFWV 7KH UHPDLQLQJ WZR HPSDWK\ LWHPV LWHP 2SHUDWLQJ +RXUV DQG LWHP .QRZ 1HHGVf ORDGHG RQ IDFWRU IRXU ZLWK LWHP IRXU 1HFHVVDU\ 5HVRXUFHV $GGLWLRQDOO\ WKH QHZ LWHPV UHSODFLQJ WKH WDQJLEOHV GLPHQVLRQ GLG QRW ORDG WRJHWKHU DV H[SHFWHG ,WHP VHYHQ 6SHDN &OHDUO\ ORDGHG RQ IDFWRU RQH ZLWK WKH HPSDWK\ DQG UHVSRQVLYHQHVV LWHPV $V PHQWLRQHG DERYH LWHP IRXU 1HFHVVDU\ 5HVRXUFHV ORDGHG RQ IDFWRU IRXU ,WHP VL[ :ULWWHQ 0DWHULDOV ORDGHG RQ IDFWRU WZR ZKLFK ZDV GHILQHG E\ WKH UHOLDELOLW\ LWHPV 7KXV WKHUH LV VRPH HYLGHQFH VXJJHVWLQJ WKDW 364 LV D PXOWLGLPHQVLRQDO FRQVWUXFW DQG WKDW WKH ILYH GLPHQVLRQV VXJJHVWHG E\ 3DUDVXUDPDQ HW DO f PLJKW KDYH PHULW LQ WKH GUXJ LQIRUPDWLRQ VHWWLQJ +RZHYHU LW PXVW DOVR EH UHFRJQL]HG WKDW WKH SURSRVHG ILYH GLPHQVLRQV ZHUH QRW IXOO\ UHSOLFDWHG ,Q DGGLWLRQ RWKHU UHVHDUFKHUV KDYH DOVR KDG GLIILFXOW\ GXSOLFDWLQJ WKHVH GLPHQVLRQV %DEDNXV DQG %RLOHU f %DEDNXV DQG 0DQJROG f %URZQ HW DO f &DUPDQ f &URQLQ DQG 7D\ORU f +HDGOH\ DQG 0LOOHU ff )XUWKHUPRUH WKH FRPSRQHQW VWUXFWXUH REWDLQHG IURP WKH IDFWRU DQDO\VLV FRQGXFWHG RQ WKH PDLQ TXHVWLRQQDLUH UHVSRQVHV GLG QRW FRUUHVSRQG WR WKH FRPSRQHQW VWUXFWXUH DFKLHYHG IURP DQDO\]LQJ WKH UHVSRQVHV WR WKH SUHWHVW 7KHUHIRUH WKHUH LV QRW HQRXJK HYLGHQFH WR GUDZ UHOLDEOH FRQFOXVLRQV UHJDUGLQJ WKH GLPHQVLRQDOLW\ RI 364 DV PHDVXUHG E\ WKH LWHPV LQ WKH 6(593(5) VFDOH 7KHUHIRUH IRU WKH SXUSRVHV RI WKLV VWXG\ WKH LWHPV LQ WKH 6(593(5) VFDOH ZLOO EH HYDOXDWHG DV D XQLGLPHQVLRQDO PHDVXUH RI 364

PAGE 127

7DEOH )LQDO 5RWDWHG &RPSRQHQW 0DWUL[ RI 0DLQ 4XHVWLRQQDLUH 5HVSRQVHV IRU WKH 6(593(5) 6XE6FDOH 1 fIr ,WHP 45,QGLYLGXDO $WWHQWLRQ (0f 453HUVRQDO $WWHQWLRQ (0f 5:KHQ 3HUIRUPHG 56f 453URPSW 6HUYLFH 56f 57 %XV\ 56f 45:LOOLQJQHVV WR +HOS 56f 5%HVW ,QWHUHVWV (0f 46SHDN &OHDUO\ 7$f 43URPLVHG 7LPH 5(f 3URYLGHV LQ 7LPH 5(f 4'HSHQGDEOH 5(f 46\PSDWKHWLF t 5HDVVXULQJ 5(f 4:ULWWHQ 0DWHULDOV 7$f 46DIH ,QWHUDFWLRQV $6f 43ROLWH (PSOR\HHV $6f 47UXVW (PSOR\HHV $6f 452SHUDWLQJ +RXUV (0f 41HFHVVDU\ 5HVRXUFHV 7$f 45.QRZ 1HHGV (0f A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV r 7KH OHWWHUV LQ SDUHQWKHVHV LQGLFDWH WKH GLPHQVLRQ RQ ZKLFK WKH LWHP ORDGHG LQ WKH RULJLQDO 6(5948$/ UHVHDUFK FRQGXFWHG E\ 3DUDVXUDPDQ HW DO f :KHUH (0 (PSDWK\ 56 5HVSRQVLYHQHVV 5( 5HOLDELOLW\ $6 $VVXUDQFH DQG 7$ 7DQJLEOHV &RPSRQHQW 1XPEHU )LJXUH 6FUHH 3ORW RI 0DLQ 4XHVWLRQQDLUH 5HVSRQVHV 9DULDEOHV 1 f

PAGE 128

5HOLDELOLW\ RI 6XEVFDOHV 5HOLDELOLW\ HVWLPDWHV ZHUH REWDLQHG XVLQJ &URQEDFKnV DOSKD IRU WKUHH VXEVFDOHV PHDVXUHG E\ WKH TXHVWLRQQDLUH f 364 LWHPV WKURXJK f f VHUYLFH WLPH SHUFHSWLRQV LWHPV WKURXJK f DQG f EHKDYLRUDO LQWHQWLRQ LWHPV DQG f $OSKD ZDV QRW DSSOLFDEOH WR 264 VLQFH LW LV RQO\ D VLQJOHLWHP PHDVXUH 7KH LWHP 6(593(5) VXEVFDOH KDG DQ H[FHOOHQW LQLWLDO DOSKD RI 4XHVWLRQ ILYH %DFNJURXQG 1RLVH ZDV WKH RQO\ LWHP ZLWK DQ LWHPWRWDO FRUUHODWLRQ RI OHVV WKDQ 4XHVWLRQ ILYHnV LWHPWRWDO FRUUHODWLRQ ZDV LQGLFDWLQJ WKDW WKLV TXHVWLRQ GLG QRW FRQWULEXWH ZHOO WR WKH VFDOH 7DEOH f 7KLV LV HYLGHQFHG E\ D VLJQLILFDQW LPSURYHPHQW LQ DOSKD LI WKH LWHP ZHUH WR EH GHOHWHG IURP WR f ,WHP ILYH ZDV WKHUHIRUH UHPRYHG IURP WKH 6(593(5) VXEVFDOH UHVXOWLQJ LQ D ILQDO VFDOH FRPSRVHG RI WKH UHPDLQLQJ LWHPV 7KH ILQDO 364 VXEVFDOH DOSKD FRPSDUHG ZHOO WR ERWK WKH SUHn WHVW DOSKD f 7DEOH f DQG WKH DOSKDV IRXQG E\ &URQLQ DQG 7D\ORU f WR f LQGLFDWLQJ WKDW WKH UHOLDELOLW\ RI WKH 6(593(5) LWHPV DUH UHODWLYHO\ FRQVLVWHQW DFURVV VHUYLFH VHWWLQJV 7KH DOSKD IRU WKH VHUYLFH WLPH SHUFHSWLRQV VXEVFDOH ZDV DQG WKH DOSKD IRU WKH VXEVFDOH PHDVXULQJ EHKDYLRUDO LQWHQWLRQ ZDV 7DEOHV f 7KHUHIRUH WKH LQWHUQDO FRQVLVWHQF\ RI WKHVH WZR VFDOHV ZHUH DOVR FRQVLGHUHG WR EH YHU\ JRRG )XUWKHUPRUH H[DPLQLQJ WKH LWHPWRWDO FRUUHODWLRQV RI WKH SHUFHLYHG VHUYLFH WLPH VFDOH GLG QRW UHYHDO DQ\ LWHPV RI TXHVWLRQDEOH YDOXH 7DEOH f 'HVFULSWLYH 6WDWLVWLFV RI 4XHVWLRQQDLUH 0HDVXUHV 7DEOH OLVWV WKH GHVFULSWLYH VWDWLVWLFV LH QXPEHU PLQLPXP PD[LPXP PHGLDQ PHDQ VWDQGDUG HUURU DQG VWDQGDUG GHYLDWLRQf IRU WKH VWXG\ YDULDEOHV WKDW ZHUH GHULYHG IURP WKH TXHVWLRQQDLUH )LUVW WKH PHDQ YDOXH RI WKH 364 VFDOH ZDV SRLQWV Q V O f 7DEOH f 7KLV PHDQW WKDW WKH DYHUDJH VFRUH RQ HDFK RI WKH YDULDEOHV

PAGE 129

PDNLQJ XS WKH 364 VFDOH ZDV DSSUR[LPDWHO\ f DV HYLGHQFHG E\ 7DEOH $OWKRXJK WKH SRVVLEOH UDQJH IRU WKH 364 VFRUH ZDV EHWZHHQ DQG WKH ORZHVW 364 VFRUH ZDV D DQG KLJKHVW REVHUYHG 364 VFRUH ZDV D ,QGLYLGXDOO\ WKH WZR LWHPV WKDW VFRUHG EHVW RQ WKH 364 VFDOH ZHUH LWHPV IRXU 1HFHVVDU\ 5HVRXUFHV DQG ILIWHHQ :LOOLQJQHVV WR +HOS 7DEOH f 7KLV VXJJHVWHG WKDW RYHUDOO UHVSRQGHQWV IHOW WKDW ',53& KDV DFFHVV WR WKH UHVRXUFHV QHFHVVDU\ WR DQVZHU WKHLU TXHVWLRQV PHDQ V f DQG WKDW WKH FHQWHU ZDV YHU\ ZLOOLQJ WR KHOS WKHP ILQG DQVZHUV WR WKHLU TXHVWLRQV PHDQ V f ,WHP WZHQW\WKUHH 2SHUDWLQJ +RXUV VFRUHG WKH ZRUVW RQ WKH 364 PHDQ V f VXJJHVWLQJ WKDW UHVSRQGHQWV ZHUH QRW DV FRQWHQW ZLWK WKH RSHUDWLQJ KRXUV RI WKH VHUYLFH DV ZLWK WKH DVSHFWV RI SHUFHLYHG VHUYLFH TXDOLW\ PHDVXUHG E\ WKH RWKHU LWHPV 6HFRQG WKH DYHUDJH 264 VFRUH ZDV Q V f LQGLFDWLQJ WKDW UHVSRQGHQWV JHQHUDOO\ IHOW WKDW WKH RYHUDOO VHUYLFH TXDOLW\ RI WKH ',35& ZDV 9HU\ *RRG 7KH EHVW VFRUH JLYHQ ZDV D RQH LH ([FHOOHQWf DQG WKH ORZHVW VFRUH JLYHQ ZDV D ILYH LH 3RRUf 7KLUG SHUFHLYHG VHUYLFH WLPH ZDV PHDVXUHG E\ IRXU LWHPV RQ WKH TXHVWLRQQDLUH 2Q DYHUDJH UHVSRQGHQWV DJUHHG WKDW WKH UHVSRQVH WLPH IRU WKHLU PRVW UHFHQW TXHVWLRQ ZDV DFFHSWDEOH Q PHDQ V f 6LPLODUO\ UHVSRQGHQWV WHQGHG WR GLVDJUHH ZLWK WKH VWDWHPHQW WKDW E\ WKH WLPH WKH\ UHFHLYHG WKH LQIRUPDWLRQ LW ZDV QR ORQJHU XVHIXO Q PHDQ V Of +RZHYHU UHVSRQGHQWV ZHUH QHXWUDO ZKHQ DVNHG LI WKH\ ZLVKHG WKH ',35& ZRXOG SURYLGH TXLFNHU UHVSRQVHV WR TXHVWLRQV Q PHDQ V f *HQHUDOO\ WKH UHVSRQGHQWV IHOW WKDW WKH UHVSRQVH WLPH ZDV D OLWWOH VKRUWHU RU HTXDO WR ZKDW WKH\ H[SHFWHG Q PHDQ V f 7KLUG WKH UHVXOWV UHJDUGLQJ WKH UHVSRQGHQWVn LQWHQWLRQV WR XVH WKH ',35& LQ WKH IXWXUH LWHP f DQG UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH LWHP f ZHUH YHU\ SRVLWLYH 2YHUDOO WKH UHVSRQGHQWV LQGLFDWHG WKDW QRW RQO\ GLG WKH\ LQWHQG WR XVH WKH VHUYLFH LQ WKH IXWXUH PHDQ V f WKH\ ZRXOG DOVR UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH PHDQ V f

PAGE 130

6LQFH SHUFHLYHG VHUYLFH TXDOLW\ LV D VXEMHFWLYH PHDVXUH LW LV LPSRUWDQW WR GHVFULEH UHVXOWV LQ WHUPV RI SUDFWLFDO DV ZHOO DV VWDWLVWLFDO VLJQLILFDQFH 7KH VWDQGDUG HUURU IRU 364 ZDV SRLQWV ZKLFK WUDQVODWHG LQWR D b FRQILGHQFH LQWHUYDO DURXQG WKH 364 PHDQ RI SOXV RU PLQXV DSSUR[LPDWHO\ SRLQWV 6LQFH WKH ZLGWK RI WKLV FRQILGHQFH LQWHUYDO LV DERXW SRLQWV LW ZDV GHFLGHG WKDW PHDQ GLIIHUHQFHV RI ILYH SRLQWV RU PRUH RQ WKH LWHP 364 VFDOH ZRXOG EH SUDFWLFDOO\ VLJQLILFDQW )RU RYHUDOO VHUYLFH TXDOLW\ WKH VWDQGDUG HUURU ZDV SRLQWV WUDQVODWLQJ LQWR D FRQILGHQFH LQWHUYDO ZLGWK RI DERXW SRLQWV 7KHUHIRUH LW ZDV GHFLGHG WKDW PHDQ GLIIHUHQFHV RI DW OHDVW D SRLQW RQ WKH 264 PHDVXUH ZRXOG EH D FRQVHUYDWLYH HVWLPDWH RI SUDFWLFDO VLJQLILFDQFH 7DEOH ,WHPWRWDO 6WDWLVWLFV IRU 6(593(5) 6XE6FDOH Q fI ,WHP &RUUHFWHG ,WHP7RWDO &RUUHODWLRQ $OSKD LI ,WHP 'HOHWHG 4 45 4 4 4 4 4 4OO 45 45 45 45 45 4 45 4 4 45 45 45 A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV

PAGE 131

7DEOH WHP7RWDO 6WDWLVWLFV IRU 6HUYLFH 7LPH 3HUFHSWLRQV Q f ,WHP &RUUHFWHG ,WHP 7RWDO &RUUHODWLRQ $OSKD LI ,WHP 'HOHWHG 4 45 45 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV 7DEOH )LQDO 6XE6FDOH 5HOLDELOLWLHV ,WHP 1 ,WHPV LQ 6FDOH $OSKD 3UHWHVW $OSKD 6(593(5) 362f 7LPH 3HUFHSWLRQV %HKDYLRUDO ,QWHQWLRQ 7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU 4XHVWLRQQDLUH 0HDVXUHV 0HDVXUH 1 0LQ 0D[ 0HGLDQ 0HDQ 6WG 'HY 6WG (UURU 364 264 3HUFHLYHG 6HUYLFH 7LPH 4$FFHSWDKOH 7LPH 41R /RQJHU 8VHIXO 42XLFNHU 5HVSRQVH 4([SHFWHG 7LPH %HKDYLRUDO ,QWHQWLRQ 48VH LQ )XWXUH 45HFRPPHQG WR &ROOHDJXH

PAGE 132

7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU 364 ,WHPV Q fW 0HDVXUH 0LQ 0D[ 0HGLDQ 0HDQ 6WG 'HY 6WG (UURU 41HFHVVDU\ 5HVRXUFHV 4:ULWWHQ 0DWHULDOV 46SHDN &OHDUO\ 43URPLVHG 7LPH 46\PSDWKHWLF t 5HDVVXULQJ 4'HSHQGDE OH 4@ 3URYLGHV LQ 7LPH 45,QGLYLGXDO $WWHQWLRQ 45:KHQ 3HUIRUPHG 53URPSW 6HUYLFH 45:LOOLQJQHVV WR +HOS 2,57 %XV\ 47UXVW (PSOR\HHV 453HUVRQDO $WWHQWLRQ 43ROLWH (PSOR\HHV 6DIH ,QWHUDFWLRQV 5.QRZ 1HHGV 5%HVW ,QWHUHVWV 452SHUDWLQJ +RXUV A 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV 5HVXOWV RI +\SRWKHVLV 7HVWV 7KLV VHFWLRQ UHSRUWV WKH UHVXOWV RI WKH VHYHQ JHQHUDO K\SRWKHVHV WHVWHG LQ WKLV VWXG\ )LUVW K\SRWKHVHV RQH WKURXJK WKUHH +, + DQG +f ZHUH XVHG WR WHVW WKH UHODWLRQVKLSV DPRQJ 364 264 DQG EHKDYLRUDO LQWHQWLRQ 7KH UHVXOWV IURP WKH GDWD REWDLQHG LQ WKLV VWXG\ DUH UHSRUWHG DQG WKHQ FRPSDUHG ZLWK WKH UHVXOWV REWDLQHG E\ &URQLQ DQG 7D\ORU f WR HVWDEOLVK FRQVWUXFW YDOLGLW\ 1H[W K\SRWKHVLV IRXU +f WHVWV WKH UHODWLRQVKLS EHWZHHQ 364 DQG SHUFHLYHG VHUYLFH WLPH )ROORZLQJ WKLV K\SRWKHVHV ILYH DQG VL[ + DQG +f WHVW WKH UHODWLRQVKLSV EHWZHHQ DFWXDO VHUYLFH WLPH DQG 364 DQG EHWZHHQ VHUYLFH GHOD\V DQG 364 /DVW K\SRWKHVLV VHYHQ +f WHVWV WKH UHODWLRQVKLSV DPRQJ DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH WLPH &RUUHODWLRQV EHWZHHQ DOO RI WKH YDULDEOHV XVHG LQ WKH K\SRWKHVLV WHVWLQJ DUH SUHVHQWHG LQ 7DEOH EHORZ

PAGE 133

7DEOH &RUUHODWLRQV %HWZHHQ 6WXG\ 9DULDEOHV 0HDVXUH 362 262 'HOD\ $67 362 a a a a a 262 rrr 3HUFHLYHG 6HUYLFH 7LPH 4$FFHSWDEOH 7LPH rrr rrr 51R /RQJHU 8VHIXO rrr rrr rrr 5XLFNHU 5HVSRQVH rrr rrr rrr rrr ([SHFWHG 7LPH rrr rrr rrr rrr rrr %HKDYLRUDO ,QWHQWLRQ 48VH LQ )XWXUH rrr rrr r rr r 45HFRPPHQG 7R &ROOHDJXH rrr rrr rrr rrr rrr rr rrr a 6HUYLFH 'HOD\ 'HOD\f rr r $FWXDO 6HUYLFH 7LPH $67f r r r r rS rrS rrrS n 5 QH[W WR WKH TXHVWLRQ QXPEHU LQGLFDWHV WKDW WKH TXHVWLRQ ZDV UHYHUVH FRGHG IRU DQDO\VLV SXUSRVHV

PAGE 134

+, 7KHUH LV D SRVLWLYH UHODWLRQVKLS EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ 364f DQG RYHUDOO VHUYLFH TXDOLW\ 264f 7KH FRUUHODWLRQ EHWZHHQ 364 DQG 264 ZDV Sf 7DEOH f LQGLFDWLQJ WKDW HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ WHQG WR FRUUHVSRQG ZLWK HYDOXDWLRQV RI RYHUDOO VHUYLFH TXDOLW\ 7KLV UHODWLRQVKLS LV ERWK VWURQJ DQG SRVLWLYH FRQILUPLQJ WKH UHODWLRQVKLS GHVFULEHG LQ +, 7KLV FRUUHODWLRQ ZDV DOVR FRQVLVWHQW ZLWK WKH FRUUHODWLRQ UHSRUWHG E\ &URQLQ DQG 7D\ORU f ZKR UHSRUWHG D FRUUHODWLRQ EHWZHHQ 6(593(5) DQG RYHUDOO TXDOLW\ RI +D ,QWHQWLRQ WR XVH WKH VHUYLFH LQ WKH IXWXUH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f +E ,QWHQWLRQ WR UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f 7KH FRUUHODWLRQ EHWZHHQ LWHP 8VH LQ )XWXUH DQG 364 ZDV Sf DQG WKH FRUUHODWLRQ EHWZHHQ LWHP 5HFRPPHQG WR &ROOHDJXH DQG 364 ZDV Sf 7DEOH f ,QWHUHVWLQJO\ WKH FRUUHODWLRQV REVHUYHG LQ WKLV VWXG\ ZHUH VWURQJHU WKDQ WKRVH REVHUYHG E\ &URQLQ DQG 7D\ORU f ZKR UHSRUWHG D FRUUHODWLRQ EHWZHHQ 6(593(5) DQG SXUFKDVH LQWHQWLRQ RI RQO\ )XUWKHU DQDO\VLV ZDV FRQGXFWHG E\ FDWHJRUL]LQJ WKH LQGLYLGXDO UHVSRQVHV LQWR WZR OHYHOV f6WURQJO\ $JUHHf DQG f$OO RWKHU UHVSRQVHVf 7WHVWV UHYHDOHG VWDWLVWLFDOO\ VLJQLILFDQW GLIIHUHQFHV LQ 364 OHYHO PHDQV IRU ERWK TXHVWLRQV Sf )RU LWHP f8VH LQ )XWXUHf WKH PHDQ GLIIHUHQFH LQ 364 EHWZHHQ f6WURQJO\ $JUHHf DQG f$OO RWKHU UHVSRQVHVf ZDV SRLQWV )RU LWHP f5HFRPPHQG WR &ROOHDJXHf WKH PHDQ GLIIHUHQFH LQ 364 EHWZHHQ f6WURQJO\ $JUHHf DQG DOO RWKHU UHVSRQVHV ZDV SRLQWV 7KHUHIRUH WKH UHODWLRQVKLS EHWZHHQ 364 DQG EHKDYLRUDO LQWHQWLRQ LV DOVR SUDFWLFDOO\ VLJQLILFDQW 7KXV K\SRWKHVHV +D DQG +E DUH VXSSRUWHG

PAGE 135

7DEOH 364 E\ /HYHO RI %HKDYLRUDO ,QWHQWLRQ 0HDVXUH 1 0HDQ 364 6WG 'HY b &, IRU 0HDQ /RZHU 8SSHU %RXQG %RXQG 8VH LQ )XWXUH 6WURQJO\ $JUHH $OO 2WKHU 5HVSRQVHV 5HFRPPHQG WR &ROOHDJXH 6WURQJO\ $JUHH $OO 2WKHU 5HVSRQVHV +D ,QWHQWLRQ WR XVH WKH VHUYLFH LQ WKH IXWXUH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI RYHUDOO VHUYLFH TXDOLW\ 264f +K ,QWHQWLRQ WR UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI RYHUDOO VHUYLFH TXDOLW\ 264f 7KH FRUUHODWLRQ EHWZHHQ LWHP 8VH LQ )XWXUH DQG 264 ZDV Sf DQG WKH FRUUHODWLRQ EHWZHHQ LWHP 5HFRPPHQG WR &ROOHDJXH DQG 264 ZDV Sf 7DEOH f 7KHVH FRUUHODWLRQV ZHUH FRQVLVWHQW ZLWK WKH FRUUHODWLRQ EHWZHHQ 264 DQG SXUFKDVH LQWHQWLRQ UHSRUWHG E\ &URQLQ DQG 7D\ORU f ZKR UHSRUWHG D FRUUHODWLRQ EHWZHHQ RYHUDOO TXDOLW\ DQG SXUFKDVH LQWHQWLRQ RI ,W LV LQWHUHVWLQJ WR QRWH WKDW WKH UHODWLRQVKLS UHJDUGLQJ IXWXUH XVH ZDV QRW DV VWURQJ DV WKH UHODWLRQVKLS UHJDUGLQJ IXWXUH UHFRPPHQGDWLRQ SHUKDSV VXJJHVWLQJ WKDW SHUFHLYHG VHUYLFH TXDOLW\ LV PRUH LPSRUWDQW WR LQWHQGHG IXWXUH UHFRPPHQGDWLRQ EHKDYLRU WKDQ LQWHQGHG IXWXUH XVH $V ZLWK +D DQG +E DGGLWLRQDO DQDO\VLV ZDV FRQGXFWHG E\ FDWHJRUL]LQJ WKH LQGLYLGXDO UHVSRQVHV LQWR WZR OHYHOV f6WURQJO\ $JUHHf DQG f$OO RWKHU UHVSRQVHVf 7WHVWV UHYHDOHG VWDWLVWLFDOO\ VLJQLILFDQW GLIIHUHQFHV LQ 264 EHWZHHQ WKH UHVSRQVH OHYHOV IRU ERWK LWHPV DQG Sf 7KH PHDQ GLIIHUHQFH EHWZHHQ f6WURQJO\ $JUHHf DQG DOO RWKHU UHVSRQVHV LQ 264 IRU LWHP ZDV SRLQWV 7KH PHDQ GLIIHUHQFH EHWZHHQ f6WURQJO\ $JUHHf DQG DOO RWKHU UHVSRQVHV LQ 264 IRU LWHP ZDV SRLQWV 7KHUHIRUH WKH

PAGE 136

UHODWLRQVKLS EHWZHHQ 364 DQG EHKDYLRUDO LQWHQWLRQ LV DOVR SUDFWLFDOO\ VLJQLILFDQW $V VXFK K\SRWKHVHV +D DQG +E DUH DFFHSWHG 7DEOH 264 E\ /HYHO RI %HKDYLRUDO ,QWHQWLRQ 0HDVXUH 1 0HDQ 264 6WG 'HY b &, IRU 0HDQ /RZHU 8SSHU %RXQG %RXQG 8VH LQ )XWXUH 6WURQJO\ $JUHH $OO 2WKHU 5HVSRQVHV 5HFRPPHQG WR &ROOHDJXH 6WURQJO\ $JUHH $OO 2WKHU 5HVSRQVHV +D $FFHSWDELOLW\ RI WKH UHVSRQVH WLPH RI WKH VHUYLFH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f +E 3HUFHLYHG XVHIXOQHVV RI WKH LQIRUPDWLRQ RQFH WKH UHVSRQVH ZDV UHFHLYHG LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f +F 3HUFHLYHG TXLFNQHVV RI UHVSRQVH LV SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f +G 'HYLDWLRQV IURP H[SHFWHG UHVSRQVH WLPHV DUH SRVLWLYHO\ DVVRFLDWHG ZLWK HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f 7KH FRUUHODWLRQV EHWZHHQ 364 DQG WKH PHDVXUHV RI SHUFHLYHG VHUYLFH WLPH ZHUH DOO VWDWLVWLFDOO\ VLJQLILFDQW 7DEOH f )LUVW WKH FRUUHODWLRQ EHWZHHQ 364 DQG LWHP f$FFHSWDEOH 7LPHf ZDV LQGLFDWLQJ WKDW DV WKH DFFHSWDELOLW\ ZLWK WKH VHUYLFH WLPH LPSURYHG VR GLG SHUFHLYHG VHUYLFH TXDOLW\ 6HFRQG WKH FRUUHODWLRQ EHWZHHQ 364 DQG LWHP f1R /RQJHU 8VHIXOf ZDV LQGLFDWLQJ WKDW UHVSRQGHQWV ZKR IHOW WKDW WKH LQIRUPDWLRQ ZDV OHVV XVHIXO EHFDXVH RI SURORQJHG UHVSRQVH WLPHV DOVR KDG ORZHU

PAGE 137

SHUFHSWLRQV RI VHUYLFH TXDOLW\ 7KLUG WKH FRUUHODWLRQ EHWZHHQ 364 DQG LWHP f4XLFNHU 5HVSRQVHf ZDV LQGLFDWLQJ WKDW WKH GHVLUH IRU TXLFNHU UHVSRQVH WLPHV ZDV VLJQLILFDQWO\ UHODWHG WR HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ )RXUWK WKH FRUUHODWLRQ EHWZHHQ 364 DQG LWHP f([SHFWHG 7LPHf ZDV LQGLFDWLQJ WKDW ORQJHU WKDQ H[SHFWHG UHVSRQVH WLPHV ZHUH DVVRFLDWHG ZLWK ORZHU 364 VFRUHV 7DEOH 364 E\ /HYHO 3HUFHLYHG 6HUYLFH 7LPH 0HDVXUH 1 0HDQ 364 6WG 'HY b &, IRU 0HDQ /RZHU 8SSHU %RXQG %RXQG 6FKHIIHf 'LIIHUHQFHV $FFHSWDEOH 7LPH f 6WURQJO\ $JUHH f $JUHH f 6RPHZKDW $JUHH f 1HXWUDO'LVDJUHH $129$ ) S2 1R /RQVHU 8VHIXO f $JUHH1HXWUDO f 6RPHZKDW 'LVDJUHH f 'LVDJUHH f 6WURQJO\ 'LVDJUHH $129$ ) S&2 2XLFNHU 5HVSRQVH f $JUHH1HXWUDO f 6RPHZKDW 'LVDJUHH 1RQH f 'LVDJUHH f 6WURQJO\ 'LVDJUHH $129$ ) S2 ([SHFWHG 7LPH f 6KRUWHU WKDQ ([SHFWHG f (TXDO WR ([SHFWHG f /RQJHU WKDQ ([SHFWHG $129$ ) S2 ,Q WHUPV RI SUDFWLFDO GLIIHUHQFHV WKH $129$ SURFHGXUHV UHYHDOHG VLJQLILFDQW RYHUDOO GLIIHUHQFHV DPRQJ WKH OHYHOV IRU DOO YDULDEOHV WHVWHG DOO Sf 7DEOH f 7KH SRVW KRF DQDO\VHV UHYHDOHG VWDWLVWLFDOO\ VLJQLILFDQW GLIIHUHQFHV IRU DOO YDULDEOHV EHWZHHQ WKH ILUVW DQG ODVW OHYHOV RI WKH DQDO\VHV DOO Sf +RZHYHU LWHP f$FFHSWDEOH 7LPHf VHHPHG WR KDYH WKH EHVW GLVFULPLQDWLRQ RI DOO WKH LWHPV XVHG 7KHUH ZDV D ODUJH

PAGE 138

GLIIHUHQFH REVHUYHG EHWZHHQ WKH 364 PHDQV RI WKH ILUVW OHYHO f6WURQJO\ $JUHHf DQG WKH ODVW OHYHO f1HXWUDO'LVDJUHHf SRLQWVf 0RUH LQWHUHVWLQJ KRZHYHU LV WKDW WKH SUDFWLFDOO\ VLJQLILFDQW GLIIHUHQFH WKDW HPHUJHG HYHQ EHWZHHQ OHYHOV RQH DQG WZR 7KH PHDQ GLIIHUHQFH EHWZHHQ f6WURQJO\ $JUHHf DQG f$JUHHf IRU LWHP f$FFHSWDEOH 7LPHf ZDV SRLQWV ZKLFK LV DOPRVW WZLFH WKH WKUHVKROG VHW IRU SUDFWLFDO VLJQLILFDQFH 2YHUDOO WKHVH UHVXOWV LQGLFDWH WKDW D VWURQJ UHODWLRQVKLS H[LVWV EHWZHHQ WKH SHUFHLYHG DFFHSWDELOLW\ RI WKH UHVSRQVH WLPH DQG SHUFHLYHG VHUYLFH TXDOLW\ %DVHG RQ WKHVH UHVXOWV DOO IRXU K\SRWKHVHV UHODWLQJ WR SHUFHLYHG VHUYLFH WLPH +D WKURXJK +Ef ZHUH DFFHSWHG +D 7KHUH LV D VLJQLILFDQW LQYHUVH UHODWLRQVKLS EHWZHHQ HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f DQG DFWXDO VHUYLFH WLPH +E 7KHUH LV D QRQOLQHDU UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DULG SHUFHLYHG VHUYLFH TXDOLW\ 364f 7KH OLQHDU UHJUHVVLRQ GLG QRW UHYHDO D VWDWLVWLFDOO\ VLJQLILFDQW UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DQG 364 5 $GMXVWHG 5 f $OWKRXJK WKH UHJUHVVLRQ DQDO\VLV ZDV QRQVLJQLILFDQW S f WKH GLUHFWLRQ DQG VWUHQJWK RI WKH UHODWLRQVKLS LV ZRUWK DGGLWLRQDO FRPPHQW $Q H[DPLQDWLRQ RI )LJXUH GRHV VKRZ D GHILQLWH LI VOLJKW XSZDUG OLQHDU WUHQG LQ SUHGLFWHG 364 LQGLFDWLQJ WKDW DV DFWXDO VHUYLFH WLPH LH WKH REVHUYHG UHVSRQVH WLPHf LQFUHDVHG WKH 364 VFRUH DOVR WHQGHG WR LQFUHDVH LH SHUFHLYHG VHUYLFH TXDOLW\ GHFUHDVHGf 7KH SUHGLFWLRQ HTXDWLRQ RI WKH RULJLQDO PRGHO ZDV 364 r $FWXDO 6HUYLFH 7LPHf 7KHUHIRUH RQ DYHUDJH LW WRRN URXJKO\ PLQXWHV EHIRUH DQ LQFUHDVH LQ 364 RI RQH SRLQW RU URXJKO\ RQH IXOO GD\ )XUWKHUPRUH WR GD\V SDVVHG EHIRUH 364 LQFUHDVHG E\ ILYH SRLQWV 7KHVH UHVXOWV VXJJHVW WKDW DFWXDO VHUYLFH WLPHV KDYH QHJOLJLEOH HIIHFW RQ 364 $OWKRXJK WKHUH PD\ EH D VOLJKW UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DQG 364 DV VXJJHVWHG E\ K\SRWKHVLV +D

PAGE 139

WKLV UHODWLRQVKLS LV QRW SUDFWLFDOO\ VLJQLILFDQW 7KHUHIRUH K\SRWKHVLV +D LV UHMHFWHG RQ ERWK D SUDFWLFDO DQG VWDWLVWLFDO EDVLV )XUWKHUPRUH DQ H[DPLQDWLRQ RI WKH UHVLGXDO SORWV LQ )LJXUHV DQG GLG QRW UHYHDO DQ\ GLVWLQFW SDWWHUQV VXJJHVWLQJ WKDW D QRQOLQHDU WUDQVIRUPDWLRQ ZRXOG H[SODLQ WKH UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DQG 364 EHWWHU WKDQ VLPSOH OLQHDU UHJUHVVLRQ )LJXUH VKRZV WKH UHVLGXDOV IRU DFWXDO VHUYLFH WLPHV RQ D VFDOH RI WR PLQXWHV 6LQFH D ODUJH QXPEHU RI WKH UHVLGXDOV DUH FOXVWHUHG LQ WKH RQH GD\ UDQJH IURP WR PLQXWHV )LJXUH ZDV DOVR SURGXFHG $V PHQWLRQHG QHLWKHU RI WKHVH ILJXUHV UHYHDOHG D UHFRJQL]DEOH SDWWHUQ WKDW FRXOG EH UHGXFHG WKURXJK QRQOLQHDU UHJUHVVLRQ PHWKRGV 7KHUHIRUH +E ZDV DOVR UHMHFWHG + 'HOD\V LQ VHUYLFH DUH QHJDWLYHO\ UHODWHG WR ORZHU HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 364f 7KHUH ZDV D VLJQLILFDQW FRUUHODWLRQ EHWZHHQ GHOD\V LQ VHUYLFH DQG 364 U S f 7DEOH f 'HOD\HG UHVSRQVHV KDG D PHDQ 364 RI V f ZKHUHDV UHVSRQVHV WKDW ZHUH QRW GHOD\HG KDG D PHDQ 364 RI V O f 7DEOH f 7KLV LV D SUDFWLFDOO\ VLJQLILFDQW PHDQ GLIIHUHQFH RI SRLQWV 7KHUHIRUH WKH K\SRWKHVLV LV DFFHSWHG WKDW GHOD\V LQ VHUYLFH DUH UHODWHG WR ORZHU HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 7DEOH 3HUFHLYHG 6HUYLFH 4XDOLW\ E\ 'HOD\ LQ 6HUYLFH b &, IRU 0HDQ 'HOD\ LQ 0HDQ 6WG /RZHU 8SSHU 6HUYLFH 1 0HGLDQ 364 'HY %RXQG %RXQG 1R
PAGE 140

)LJXUH 5HJUHVVLRQ (TXDWLRQ 3ORW RI 364 DQG 3UHGLFWHG 364 E\ $FWXDO 6HUYLFH 7LPH ZKHUH 364 r$FWXDO 6HUYLFH 7LPHf )LJXUH 5HVLGXDO 3ORW RI 364 RQ $FWXDO 6HUYLFH 7LPH

PAGE 141

A r 5HVLGXDOV r‘ 1! R R r f f r f m r f6 R RR A r r 2 r n R A r r R R A A r 2 2 m $FWXDO 6HUYLFH 7LPH ,Q 0LQXWHVf )LJXUH 5HVLGXDO 3ORW RI 364 RQ $FWXDO 6HUYLFH 7LPHV 2FFXUULQJ ZLWKLQ 2QH 'D\ +D 7KHUH LV D SRVLWLYH UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DQG SHUFHLYHG VHUYLFH WLPH +E 7KHUH LV D SRVLWLYH UHODWLRQVKLS EHWZHHQ VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH WLPH 7KHVH WZR K\SRWKHVHV ZHUH XVHG WR HYDOXDWH WKH UHODWLRQVKLS DPRQJ PHDVXUHV RI WKH DFWXDO VHUYLFH WLPH VHUYLFH GHOD\ DQG SHUFHLYHG VHUYLFH WLPH 2QO\ WZR FRUUHODWLRQV RI WKH HLJKW DQDO\]HG ZHUH VLJQLILFDQW 7DEOH f )LUVW DFWXDO VHUYLFH WLPH YHUVXV LWHP 4XLFNHU 5HVSRQVH KDG D FRUUHODWLRQ RI S f 6HFRQG DFWXDO VHUYLFH WLPH YHUVXV LWHP ([SHFWHG 7LPH KDG D FRUUHODWLRQ RI S f 6XUSULVLQJO\ WKHUH ZDV QR UHODWLRQVKLS GHWHFWHG EHWZHHQ WKH SHUFHLYHG VHUYLFH WLPH PHDVXUHV DQG GHOD\V LQ VHUYLFH )XUWKHUPRUH VLQFH WKH WZR VLJQLILFDQW FRUUHODWLRQV ZHUH VPDOO DQG YHU\ OLWWOH RI

PAGE 142

WKH DVVRFLDWHG YDULDQFH EHWZHHQ DFWXDO VHUYLFH WLPH DQG SHUFHLYHG VHUYLFH WLPH ZDV H[SODLQHG LW ZDV GLIILFXOW WR PDNH DQ\ SUDFWLFDO GHWHUPLQDWLRQ RI VLJQLILFDQFH ,W LV LQWHUHVWLQJ WR QRWH KRZHYHU WKDW WKHUH GRHV DSSHDU WR EH DW OHDVW D WUHQG LQ WKH GDWD FRQFHUQLQJ DFWXDO VHUYLFH WLPH 7DEOH VKRZV WKH SHUFHQWDJH RI VDPSOHV DERYH WKH PHDQ DFWXDO VHUYLFH WLPH IRU WKH UHVSRQVH FDWHJRULHV FRUUHVSRQGLQJ WR LWHP ([SHFWHG 7LPH 2XW RI WKH UHVSRQVHV LQGLFDWLQJ WKDW WKH UHVSRQVH WLPH ZDV VKRUWHU WKDQ H[SHFWHG RQO\ bf KDG DFWXDO VHUYLFH WLPHV KLJKHU WKDQ WKH PHDQ +RZHYHU RI WKH UHVSRQVHV LQGLFDWLQJ WKDW WKH UHVSRQVH WLPH ZDV ORQJHU WKDQ H[SHFWHG bf KDG DFWXDO VHUYLFH WLPHV KLJKHU WKDQ WKH PHDQ $ M DQDO\VLV UXQ RQ WKH FHOO FRXQWV RQO\ UHVXOWHG LQ D ERUGHUOLQH YDOXH RI S f 7KHUHIRUH DOWKRXJK WKHUH LV VRPH HYLGHQFH VXJJHVWLQJ WKDW D UHODWLRQVKLS PD\ H[LVW EHWZHHQ DFWXDO VHUYLFH WLPH DQG SHUFHLYHG VHUYLFH WLPH WKLV HYLGHQFH LV QRW VWDWLVWLFDOO\ FRQFOXVLYH 7KHUHIRUH K\SRWKHVHV +D DQG +E DUH UHMHFWHG 7DEOH 3HUFHQWDJH %HORZ$ERYH WKH 0HDQ $FWXDO 6HUYLFH 7LPH E\ 4([SHFWHG 7LPH 4([SHFWHG 7LPH 1 3HUFHQWDJH %HORZ 0HDQ 3HUFHQWDJH $ERYH 0HDQ 6KRUWHU WKDQ ([SHFWHG b b (TXDO WR ([SHFWHG b b /RQJHU WKDQ ([SHFWHG b b 3DUW )RXU 6LPXODWLRQ 5HVXOWV &RQVWUXFWLRQ RI 6LPXODWLRQ 7KH VLPXODWLRQ ZDV FRQVWUXFWHG LQ *366+ SURIHVVLRQDO DQG UXQ RQ DQ 06'26 IRUPDW FRPSXWHU 7KH GDWD QHHGHG WR FRQVWUXFW WKH VLPXODWLRQ ZHUH REWDLQHG IURP WKH

PAGE 143

DQDO\VHV RI WKH KLVWRULFDO DQG FRQFXUUHQW GDWD DV ZHOO DV WKH FRGLUHFWRU DQG VWXGHQW LQWHUYLHZV GHVFULEHG HDUOLHU LQ WKLV FKDSWHU (PSLULFDO LQWHUDUULYDO GLVWULEXWLRQV ZHUH REWDLQHG IURP WKH KLVWRULFDO GDWD DQG VHSDUDWHG E\ WLPH RI GD\ (PSLULFDO VHUYLFH WLPH GLVWULEXWLRQV IRU WKH WKUHH JHQHUDO FDWHJRULHV RI TXHVWLRQV DV ZHOO DV WKH WLPHV WR WDNH FDOOV DQG UHWXUQ DQVZHUV ZHUH REWDLQHG IURP WKH FRQFXUUHQW GDWD 7KH SULRULW\ GLVFLSOLQH IRU VWDW WRGD\ GDWH DQG QR UXVK TXHVWLRQV ZDV EXLOW LQWR WKH PRGHO DV GHVFULEHG E\ WKH VWXGHQWV DQG FRGLUHFWRUV GXULQJ WKH SHUVRQDO LQWHUYLHZV 7KH GLVWULEXWLRQV RI WKH GHVLUHG UHVSRQVH WLPHV E\ WLPH RI GD\ ZHUH REWDLQHG IURP WKH KLVWRULFDO GDWD ,Q RUGHU WR DVVXUH VWDWLVWLFDO LQGHSHQGHQFH VHSDUDWH SVHXGRUDQGRP QXPEHU VWUHDPV DOWRJHWKHUf ZHUH XVHG IRU HDFK LQWHUDUULYDO DQG VHUYLFH WLPH IXQFWLRQ DV ZHOO DV IRU VWDWLVWLFDO WUDQVIHUV DQG VHOHFWLRQ YDULDEOHV )XUWKHUPRUH GLIIHUHQW UDQGRP QXPEHU VHHGV ZHUH XVHG IRU HDFK UHSOLFDWLRQ RI WKH VLPXODWLRQ )RXU SULPDU\ DVVXPSWLRQV ZHUH PDGH GXULQJ WKH FRQVWUXFWLRQ RI WKH VLPXODWLRQ )LUVW DOO VWXGHQWV KDYH WKH VDPH VHUYLFH WLPH GLVWULEXWLRQV :KLOH WKHUH FHUWDLQO\ DUH GLIIHUHQFHV LQ VWXGHQWV ZLWK UHJDUG WR WKHLU LQGLYLGXDO VHUYLFH WLPHV LW LV GLIILFXOW WR SUHGLFW HDFK PRQWK KRZ ZHOO WKH LQGLYLGXDO VWXGHQWV DUH JRLQJ WR SHUIRUP ,W ZDV GHFLGHG WR DVVXPH WKDW DOO VWXGHQWV DUH HTXDO VLQFH GHFLVLRQV PXVW EH PDGH LQ WKH ',35& UHJDUGOHVV RI UDQGRP GLIIHUHQFHV LQ VWXGHQW SURGXFWLYLW\ IURP PRQWK WR PRQWK 6HFRQG QR DUULYDOV RFFXU EHIRUH D P RU DIWHU SP 7KLV DVVXPSWLRQ GRHV QRW YLRODWH UHDOLW\ ZLWK DQ\ JUHDW VLJQLILFDQFH VLQFH WKH WHOHSKRQHV DUH QRW WXUQHG RQ XQWLO DP LQ WKH PRUQLQJ DQG DUH WXUQHG RII FRQVLVWHQWO\ DW SP 4XHVWLRQV DUH VRPHWLPHV OHIW RQ WKH YRLFH PDLO GXULQJ QRQZRUNLQJ KRXUV KRZHYHU LW ZDV GHWHUPLQHG IURP WKH LQWHUYLHZV WKDW WKLV LV D UDUH RFFXUUHQFH 7KLUG DOO DUULYDOV RFFXU E\ WHOHSKRQH DQG DOO UHVSRQVHV DUH UHWXUQHG E\ WHOHSKRQH $OWKRXJK WKH ODUJH PDMRULW\ RI DUULYDOV GR RFFXU E\ WHOHSKRQH VRPHWLPHV D TXHVWLRQ ZLOO DUULYH E\ ID[ HOHFWURQLF PDLO RU SHUVRQDO YLVLW 7KLV DVVXPSWLRQ PDNHV OLWWOH GLIIHUHQFH LQ WKLV VHWWLQJ DW WKH FXUUHQW TXHVWLRQ YROXPH VLQFH WKH WHOHSKRQH FDSDFLW\ RI WKH ',53& LV

PAGE 144

H[WUHPHO\ ODUJH LQ UHODWLRQ WR WKH YROXPH RI FDOOV )XUWKHUPRUH WKH VHUYLFH WLPH GLVWULEXWLRQV GR QRW SUHFOXGH WKDW D ZULWWHQ UHVSRQVH ZDV GHOLYHUHG LQ DGGLWLRQ WR D YHUEDO UHVSRQVH ,I WKH YROXPH RI FDOOV ZHUH WR LQFUHDVH WR D SRLQW ZKHUH EDONLQJ FRQVLVWHQWO\ RFFXUV LW PD\ EH QHFHVVDU\ PDNH FKDQJHV LQ WKH VLPXODWLRQ GLVWLQJXLVKLQJ WKHVH DUULYDO VRXUFHV ,Q WKH FXUUHQW VLPXODWLRQ KRZHYHU LW ZDV DVVXPHG WKDW LI D VLPXODWHG FDOOHU HQFRXQWHUV D EXV\ RU DQ XQDQVZHUHG WHOHSKRQH WKH FDOOHU ZRXOG WU\ EDFN RQFH LPPHGLDWHO\ ,I WKH FDOOHU WKHQ HQFRXQWHUHG D EXV\ VLJQDO RU XQDQVZHUHG WHOHSKRQH WKH FDOOHU ZRXOG ZDLW PLQXWHV XQLIRUPO\ GLVWULEXWHGf DQG WKHQ WU\ DJDLQ ,I WKH FDOOHU HQFRXQWHUHG D EXV\ RU XQDQVZHUHG WHOHSKRQH D WKLUG WLPH WKH FDOOHU EDONV 7KLV DVVXPSWLRQ ZDV EDVHG RQ GLVFXVVLRQV ZLWK WKH FRGLUHFWRUV UHJDUGLQJ WKH EHKDYLRU WKH\ ZRXOG H[SHFW RI FDOOHUV HQFRXQWHULQJ WKLV VLWXDWLRQ LQ WKH UHDO V\VWHP )RXUWK TXHVWLRQV UHTXHVWLQJ VWDW DWWHQWLRQ KDYH WKH KLJKHVW SULRULW\ IROORZHG LQ SULRULW\ E\ fWRGD\f fGDWHf DQG fQR UXVKf TXHVWLRQV ZLWK fQR UXVKf TXHVWLRQV KDYLQJ WKH ORZHVW SULRULW\ 4XHVWLRQV DUH DQVZHUHG DQG UHWXUQHG EDVHG RQ WKLV SULRULW\ V\VWHP )XUWKHUPRUH TXHVWLRQV ZLWK KLJKHU SULRULW\ SUHHPSW TXHVWLRQV RI ORZHU SULRULW\ 4XHVWLRQV ZLWKLQ WKH VDPH SULRULW\ FODVV DUH VHUYHG RQ D ILUVWFRPHILUVWVHUYHG EDVLV 7KLV LV WKH SULRULW\ V\VWHP GHVFULEHG E\ WKH FRGLUHFWRUV DQG VWXGHQWV GXULQJ WKH SHUVRQDO LQWHUYLHZV 9HULILFDWLRQ DQG 9DOLGDWLRQ RI WKH 6LPXODWLRQ ,QWHUYLHZV ZHUH FRQGXFWHG ZLWK HDFK RI WKH FRGLUHFWRUV RI WKH FHQWHU LQ RUGHU WR H[SODLQ WKH DVVXPSWLRQV PDGH LQ WKH VLPXODWLRQ DQG HVWDEOLVK WKH IDFH YDOLGLW\ RI WKH VLPXODWLRQ SURJUDP ,Q WKHVH LQWHUYLHZV WKH UHVHDUFKHU V\VWHPDWLFDOO\ GHVFULEHG WKH VLPXODWLRQ SURJUDP WR HDFK RI WKH FRGLUHFWRUV XVLQJ WKH EORFN GLDJUDPV $SSHQGL[ 4f %RWK FRGLUHFWRUV DJUHHG ZLWK WKDW WKH VLPXODWLRQ DFFXUDWHO\ GHVFULEHG WKH ZRUN SURFHVV RI WKH ',35& DQG RQO\ PLQRU FKDQJHV ZHUH VXJJHVWHG

PAGE 145

)RXU SKDVHV RI SLORW UXQV ZHUH XVHG WR YHULI\ WKH EHKDYLRU RI WKH VLPXODWLRQ DQG YDOLGDWH WKH SUHGLFWLYH DELOLW\ RI WKH VLPXODWLRQ 7KH ILUVW SKDVH RI VLPXODWLRQV ZDV XVHG WR WUDFH WKH VLPXODWLRQ IRU HUURUV 7KLV ZDV FRQGXFWHG XVLQJ WKH LQWHUDFWLYH GHEXJJHU IHDWXUH SURYLGHG E\ *366+ DQG WKH EORFN FRXQW RXWSXW 7KH EORFN FRXQWV SURYLGH LQIRUPDWLRQ UHJDUGLQJ KRZ PDQ\ WUDQVDFWLRQV SDVV WKURXJK D SDUWLFXODU OLQH RI FRGH GXULQJ D VLPXODWLRQ UXQ ,Q WKLV SKDVH LW ZDV YHULILHG WKDW JHQHUDWHG WUDQVDFWLRQV ZHUH EHLQJ URXWHG WKURXJK WKH VLPXODWLRQ DV H[SHFWHG $OVR DOO RI WKH EORFN FRXQWV VXPPHG DV H[SHFWHG HJ LI WUDQVDFWLRQV HQWHUHG WKH V\VWHP WKH SUHVHQFH RU WHUPLQDWLRQ RI DOO ZHUH DFFRXQWHGf 7KH VHFRQG SKDVH RI VLPXODWLRQV ZDV XVHG WR YHULI\ WKDW WKH VLPXODWLRQ EHKDYHG DV H[SHFWHG ZKHQ LQWHUDUULYDO WLPHV LQFUHDVH RU GHFUHDVH *LYHQ WKH XQGHUO\LQJ IRXQGDWLRQ RI TXHXLQJ WKHRU\ DV LQWHUDUULYDO WLPHV GHFUHDVH LH DUULYDOV EHFRPH PRUH IUHTXHQWf WKH VWXGHQW XWLOL]DWLRQ SHUFHQWDJH WKH WRWDO VHUYLFH WLPH DQG WKH H[SHFWHG QXPEHU RI WUDQVDFWLRQV LQ WKH V\VWHP VKRXOG LQFUHDVH DV\PSWRWLFDOO\ /LNHZLVH DV WKH LQWHUDUULYDO WLPHV LQFUHDVH LH DUULYDOV EHFRPH OHVV IUHTXHQWf XWLOL]DWLRQ WRWDO VHUYLFH WLPH DQG H[SHFWHG QXPEHU LQ WKH V\VWHP VKRXOG GHFUHDVH 7R YHULI\ WKDW WKH VLPXODWHG V\VWHP H[KLELWHG WKLV K\SRWKHVL]HG EHKDYLRU V\VWHP GD\V LH RQH PRQWKf ZHUH VLPXODWHG LQ WHQ VHSDUDWH UXQV RI RQH UHSOLFDWLRQ HDFK ZKHUH HDFK UHSOLFDWLRQ LQFOXGHG DQ DQWLWKHWLF SDLU RI UXQVf $ PRGLILHU ZDV XVHG LQ WKH *(1(5$7( VWDWHPHQWV RI WKH *366+ FRGH VHH $SSHQGL[ 5 OLQHV WR f IRU HDFK RI WKH WHQ UXQV WR LQFUHDVH RU GHFUHDVH WKH LQWHUDUULYDO UDWH E\ D JLYHQ SHUFHQWDJH 7KH V\VWHP WR EH YHULILHG LH WKH VLPXODWLRQ UHIOHFWLQJ WKH REVHUYHG GDWDf KDG D PRGLILHU RI RQH LH t$0 f 7KH RWKHU QLQH UXQV XVHG UHVSHFWLYH PRGLILHUV RI DQG 7DEOH f 7KLV FRUUHVSRQGHG WR VLPXODWHG LQFUHDVHV LQ LQWHUDUULYDO WLPHV RI b WR b LH IHZHU DUULYDOVf DQG GHFUHDVHV LQ LQWHUDUULYDO WLPHV RI b WR b LH PRUH DUULYDOVf 7KH VLPXODWHG V\VWHP EHKDYHG DV H[SHFWHG $V WKH DUULYDO PRGLILHU LQFUHDVHG IURP WR WKH XWLOL]DWLRQ SHUFHQWDJH GHFUHDVHG IURP LH IXOO\ XWLOL]HGf WR

PAGE 146

LH OHVV WKDQ KDOI XWLOL]HGf ,Q DGGLWLRQ WKH WRWDO VHUYLFH WLPH LQ PLQXWHV DQG H[SHFWHG QXPEHU LQ WKH V\VWHP GHFUHDVHG IURP WR PLQXWHV DQG WR TXHVWLRQV UHVSHFWLYHO\ 7KH WKLUG SKDVH RI SLORW UXQV ZDV XVHG WR HVWDEOLVK DQ DSSURSULDWH ZDUPXS SHULRG IRU WKH VLPXODWLRQ WR YHULI\ WKDW WKH VLPXODWLRQ SURGXFHG WKH H[SHFWHG QXPEHU RI DUULYDOV DQG GHOD\V DQG WR YDOLGDWH WKH WRWDO VHUYLFH WLPH GLVWULEXWLRQ 7R REWDLQ GDWD IRU WKHVH FRPSDULVRQV VL[ UHSOLFDWLRQV RI VLPXODWHG GD\V HDFK ZHUH UXQ LH VL[ LQGHSHQGHQW VLPXODWHG PRQWKVf 6LQFH WKH VLPXODWLRQ EHJLQV ZLWK ]HUR WUDQVDFWLRQV LQ WKH V\VWHP LW LV LPSRUWDQW WR GHWHFW WKH DPRXQW RI WLPH QHFHVVDU\ IRU WKH VLPXODWLRQ WR ZDUP XS LH DFKLHYH VWHDG\ VWDWHf 7KH JUDSKLFDO PHWKRG VXJJHVWHG E\ +RRYHU DQG 3HUU\ f ZDV XVHG WR VHOHFW WKH ZDUP XS SHULRG XVHG LQ WKH VLPXODWLRQ 7KH VLPXODWLRQ SURJUDP ZDV SURJUDPPHG WR YHULI\ DQG RXWSXW WKH WRWDO QXPEHU RI TXHVWLRQV LQ WKH V\VWHP DW D UDQGRP SRLQW RQFH SHU VLPXODWHG KRXU (DFK GD\ UXQ UHVXOWHG LQ DQWLWKHWLF SDLUV WRWDO VDPSOHVf 7KH WZHOYH VDPSOHV FRUUHVSRQGLQJ WR HDFK KRXU ZHUH DYHUDJHG WR REWDLQ DQ H[SHFWHG QXPEHU LQ WKH V\VWHP IRU WKDW SDUWLFXODU KRXU DFURVV DOO UHSOLFDWLRQV $W HDFK KRXUO\ FKDQJH WKH UHVXOW ZDV DYHUDJHG ZLWK WKH SUHYLRXV KRXUO\ UHVXOWV WR REWDLQ WKH H[SHFWDWLRQ IXQFWLRQ SUHVHQWHG LQ )LJXUH LH ZKHUH (1XPEHU LQ 6\VWHPf 1L 1fLf ZKHUH L WR Qf $V )LJXUH LOOXVWUDWHV WKH DSSURSULDWH ZDUP XS SHULRG IRU WKH VLPXODWLRQ DSSHDUV WR EH MXVW XQGHU ILYH GD\V LQGLFDWLQJ WKDW WKH V\VWHP PRUH RU OHVV UHDFKHV VWHDG\ VWDWH GXULQJ WKH IRXUWK VLPXODWHG GD\ 7R DOORZ IRU VRPH PDUJLQ RI HUURU ILYH GD\V LH VLPXODWHG PLQXWHVf ZDV VHOHFWHG DV WKH ZDUP XS SHULRG 7KH VLPXODWLRQ FRPSDUHG IDYRUDEO\ DJDLQVW WKH KLVWRULFDO GDWD RQ WKUHH YDULDEOHV RI LQWHUHVW f WRWDO TXHVWLRQ DUULYDOV SHU PRQWK f DYHUDJH TXHVWLRQ DUULYDOV SHU PRQWK DQG f VHUYLFH GHOD\ SHUFHQWDJH $ VLJQLILFDQW GLIIHUHQFH LQ WKH DYHUDJH QXPEHU RI DUULYDOV SHU GD\ DQG SHU PRQWK EHWZHHQ WKH KLVWRULFDO DQG VLPXODWHG V\VWHPV ZRXOG LQGLFDWH WKDW WKH LQWHUDUULYDO GLVWULEXWLRQV LQSXW LQWR WKH PRGHO ZHUH QRW EHKDYLQJ DV H[SHFWHG $V

PAGE 147

PHQWLRQHG HDUOLHU WKH KLVWRULFDO GDWD IRU WKH SDVW ILYH \HDUV UHYHDOHG DQ DYHUDJH WRWDO QXPEHU RI DUULYDOV HDFK PRQWK RI V f DQG DQ DYHUDJH GDLO\ QXPEHU RI TXHVWLRQV RI V O f 7DEOH f 7KH VLPXODWLRQ SURGXFHG UHVXOWV YHU\ FORVH WR WKHVH ZLWK DQ DYHUDJH WRWDO QXPEHU RI TXHVWLRQV SHU PRQWK RI V O f DQG DYHUDJH GDLO\ QXPEHU RI TXHVWLRQV RI V f 7KH VPDOO GLIIHUHQFHV REVHUYHG ZHUH QRW VWDWLVWLFDOO\ VLJQLILFDQW ZLWK WKH PHDQ GLIIHUHQFH LQ WRWDO PRQWKO\ DUULYDOV KDYLQJ D 7 VWDWLVWLF HTXDO WR S f DQG WKH PHDQ GLIIHUHQFH LQ DYHUDJH GDLO\ DUULYDOV KDYLQJ D 7 VWDWLVWLF HTXDO WR S f 6LQFH GHOD\V LQ VHUYLFH DSSHDUHG WR KDYH D VLJQLILFDQW UHODWLRQVKLS WR SHUFHLYHG VHUYLFH TXDOLW\ LW ZDV LPSRUWDQW WR DVVHVV ZKHWKHU WKH VLPXODWLRQ FRXOG DGHTXDWHO\ SUHGLFW WKH SHUFHQWDJH RI VHUYLFH GHOD\V 7KH UHVXOWV LQGLFDWHG WKDW WKH SHUFHQWDJH RI VHUYLFH GHOD\V IRU ERWK WKH VLPXODWHG DQG KLVWRULFDO GDWD ZHUH WKH VDPH DW b 7 S ZKHUH WKH VLPXODWLRQ V DQG KLVWRULFDO V f 7DEOH f ,Q DGGLWLRQ WR LQVLJQLILFDQW GLIIHUHQFHV LQ WKH PHDQ YDOXHV WKH FRQILGHQFH LQWHUYDOV IRU WKH WKUHH FRPSDULVRQV DUH DOVR YHU\ FRQVLVWHQW KDYLQJ VLPLODU ORZHU DQG XSSHU ERXQGV 7KHVH UHVXOWV LQGLFDWH WKDW WKH VLPXODWLRQ SURJUDP PRGHOV ERWK WKH QXPEHU RI TXHVWLRQ DUULYDOV LQWR WKH V\VWHP DQG WKH SHUFHQWDJH RI VHUYLFH GHOD\V ZLWK JRRG SUHFLVLRQ 7KH GDWD IURP WKLV WKLUG SKDVH RI VLPXODWLRQ UXQV ZDV DOVR XVHG WR FRPSDUH WKH FRQFXUUHQW DQG VLPXODWHG WRWDO VHUYLFH WLPH GLVWULEXWLRQV 7KH VLPXODWLRQ ZDV SURJUDPPHG WR SURGXFH D WDEOH RI WRWDO VHUYLFH WLPHV LQ PLQXWH LQWHUYDOV /LNHZLVH WKH FRQFXUUHQW GDWD ZDV DOVR VRUWHG DQG FRXQWHG LQWR LQ PLQXWH WLPH LQWHUYDOV 7KH FRQFXUUHQW DQG VLPXODWHG GLVWULEXWLRQV ZHUH FRPSDUHG IRU WRWDO VHUYLFH WLPHV OHVV WKDQ PLQXWHV LH FRPSOHWHG ZLWKLQ RQH GD\f 7KLV FXWRII ZDV VHOHFWHG EHFDXVH RI WKH GLVWRUWLRQ EHWZHHQ UHDO WLPH DQG VLPXODWHG WLPH DIWHU PLQXWHV )RU H[DPSOH GD\ WZR RI WKH VLPXODWLRQ VWDUWV DW PLQXWHV RQ WKH VLPXODWLRQ FORFN KRZHYHU GD\ WZR LQ UHDO WLPH VWDUWV DW PLQXWHV

PAGE 148

,W ZDV QRW VXUSULVLQJ WKDW WKH FKLVTXDUH JRRGQHVV RI ILW WHVW EHWZHHQ WKH FRQFXUUHQW DQG VLPXODWHG WRWDO VHUYLFH WLPH GLVWULEXWLRQV UHVXOWHG LQ D UHMHFWLRQ RI WKH K\SRWKHVLV WKDW WKH WZR GLVWULEXWLRQV DUH HTXDO GI [ Sf $OWKRXJK )LJXUH JUDSKLFDOO\ LOOXVWUDWHV D UHDVRQDEO\ JRRG ILW EHWZHHQ WKH WZR SUREDELOLW\ GHQVLW\ IXQFWLRQV 3')f WKHUH DUH VHYHUDO VSLNHV LQ WKH FRQFXUUHQW GDWD LH DW DQG PLQXWHVf ZKLFK FUHDWH ODUJH GLIIHUHQFHV LQ WKH FHOO SHUFHQWDJHV &RPSRXQGLQJ WKLV SUREOHP LV D ODUJH VLPXODWHG VDPSOH VL]H Q f ZKLFK WHQGHG WR PDJQLI\ WKHVH FHOO GLIIHUHQFHV 6LPSOH OLQHDU UHJUHVVLRQ RI WKH FHOO SHUFHQWDJHV ZDV XVHG LQVWHDG WR REWDLQ D EHWWHU IHHO IRU KRZ ZHOO WKH VLPXODWHG WRWDO VHUYLFH WLPHV SUHGLFW WKH GLVWULEXWLRQ RI WRWDO VHUYLFH WLPHV REVHUYHG LQ WKH FRQFXUUHQW GDWD 7KH UHJUHVVLRQ \LHOGHG DQ DGMXVWHG 5 RI LQGLFDWLQJ WKDW WKH VLPXODWHG GHQVLW\ IXQFWLRQ H[SODLQV DSSUR[LPDWHO\ b RI WKH YDULDQFH LQ WKH FRQFXUUHQW GHQVLW\ IXQFWLRQ ,GHDOO\ WKH FRHIILFLHQWV IRU WKH LQWHUFHSW f DQG WKH VLPXODWHG VHUYLFH WLPHV f ZRXOG EH DQG UHVSHFWLYHO\ LQGLFDWLQJ D SHUIHFW ILW 7KH UHDOL]HG FRHIILFLHQWV ZHUH FORVH ZLWK DQ LQWHUFHSW FRHIILFLHQW LI DQG D VHUYLFH WLPH FRHIILFLHQW RI ,QWHUHVWLQJO\ WKH FXPXODWLYH GHQVLW\ IXQFWLRQ &')f SURGXFHG DQ HYHQ EHWWHU ILW ZLWK DQ DGMXVWHG 5 RI DQ LQWHUFHSW FRHIILFLHQW RI DQG D VHUYLFH WLPH FRHIILFLHQW RI 7KLV LPSURYHPHQW FDQ ODUJHO\ EH DWWULEXWHG WR WKH VPRRWKLQJ RI WKH VSLNHV LQ WKH GHQVLWLHV PHQWLRQHG HDUOLHU 3ORWV RI WKH &')V DUH SUHVHQWHG LQ )LJXUH 7KH IRXUWK SKDVH RI SLORW UXQV ZDV XVHG WR YDOLGDWH WKH PRGHO XQGHU UHVWULFWHG FRQGLWLRQV ZKHUH DQ DQDO\WLFDO VROXWLRQ EDVHG RQ TXHXLQJ WKHRU\ FRXOG EH FRPSDUHG WR WKH RXWSXW JHQHUDWHG E\ WKH VLPXODWLRQ 7KH VLPXODWLRQ ZDV PRGLILHG VR WKDW LW FORVHO\ UHVHPEOHG D 00 TXHXLQJ V\VWHP ,Q RUGHU WR DFFRPSOLVK WKLV ILYH FKDQJHV ZHUH PDGH WR WKH PRGHO )LUVW WKH HPSLULFDO LQWHUDUULYDO GLVWULEXWLRQV E\ WLPH RI GD\ ZHUH UHSODFHG ZLWK RQH H[SRQHQWLDO GLVWULEXWLRQ GHVFULELQJ WKH H[SHFWHG LQWHUDUULYDO WLPH IRU WKH HQWLUH GD\ 6HFRQG WKH VHUYLFH WLPH GLVWULEXWLRQV IRU WKH VHSDUDWH TXHVWLRQ FDWHJRULHV ZHUH

PAGE 149

UHSODFHG ZLWK RQH H[SRQHQWLDOO\ GLVWULEXWHG IXQFWLRQ GHVFULELQJ WKH RYHUDOO H[SHFWHG VHUYLFH WLPHV 7KLUG WKH WLPHV WR WDNH D FDOO DQG UHWXUQ DQ DQVZHU ZHUH VHW HTXDO WR ]HUR VR WKDW PXOWLSOH SKDVHV ZRXOG QRW GLVWRUW WKH UHVXOWV )RXUWK WKH SULRULWLHV IRU DOO WUDQVDFWLRQV ZHUH PDGH HTXDO LH fVWDWf fWRGD\f fGDWHf DQG fQR UXVKf TXHVWLRQV ZHUH DOO WUHDWHG HTXDOO\f )LIWK DOO LQVWDQFHV RI SUHHPSWLRQ ZHUH UHSODFHG ZLWK D ILUVWFRPH ILUVWVHUYHG SULRULW\ VWDWXV 7KH UHVXOWV RI WKH VLPXODWHG UXQV ZHUH WKHQ FRPSDUHG WR WKH DQDO\WLFDO UHVXOWV 7ZHOYH UXQV RI VLPXODWHG GD\V HDFK ZHUH JHQHUDWHG XVLQJ WKLV PRGLILHG VLPXODWLRQ SURJUDP 7KH SURJUDP ZDV DOVR FRGHG VR WKDW LW LQFOXGHG WKH ILYHGD\ ZDUP XS SHULRG DVVHVVHG HDUOLHU 7KH TXHXH VWDWLVWLFV DW WKH HQG RI GD\V ZHUH FRPSLOHG UHVXOWLQJ LQ VDPSOH RI GDWD SRLQWV LH WZHOYH DQWLWKHWLF SDLUVf 7KHUH ZDV QR VWDWLVWLFDOO\ VLJQLILFDQW HYLGHQFH WR VXJJHVW WKDW WKH VLPXODWHG PRGHO GLIIHUHG IURP WKH H[DFW DQDO\WLFDO VROXWLRQ XVLQJ WKH IRUPXODV IRU D 00V TXHXLQJ V\VWHP 7KH YDOXHV IRU S /T / :T DQG : REWDLQHG IURP WKH UHVWULFWHG VLPXODWLRQ DQG FORVHG IRUP PHWKRGV ZHUH DOO FRPSDUDEOH DQG QRQH RI WKH SYDOXHV IRU WKH FRPSXWHG 7VWDWLVWLFV ZHUH HYHQ FORVH WR VWDWLVWLFDO VLJQLILFDQFH 7DEOH f 6LQFH WKH UHVWULFWHG PRGHO SHUIRUPHG DV H[SHFWHG ZKHQ FRPSDUHG WR D NQRZQ FORVHG IRUP PHWKRG WKLV VXJJHVWHG WKDW WKH XQUHVWULFWHG PRGHO VKRXOG DOVR EHKDYH DV H[SHFWHG LI DQ DQDO\WLFDO PRGHO H[LVWHG IRU FRPSDULVRQ 7KH UHVXOWV IURP WKH DERYH YHULILFDWLRQ DQG YDOLGDWLRQ PHWKRGV LQGLFDWHG WKDW WKH VLPXODWLRQ ZDV D FUHGLEOH UHSUHVHQWDWLRQ RI WKH DFWXDO V\VWHP :KLOH LW LV QRW H[SHFWHG WKDW WKH PRGHO ZLOO SHUIHFWO\ HPXODWH WKH UHDO V\VWHP WKH PRGHO GRHV VHHP WR SHUIRUP ZLWK D UHDVRQDEO\ FORVH ILW WKDW LV VXIILFLHQW IRU GHFLVLRQ PDNLQJ 7KHUHIRUH LW ZDV FRQFOXGHG WKDW WKLV PRGHO ZDV YDOLG IRU XVH DV D GHFLVLRQ PDNLQJ WRRO LQ WKH ',35&

PAGE 150

7DEOH 6WXGHQW 8WLOL]DWLRQ 7RWDO 6HUYLFH 7LPH DQG ([SHFWHG 1XPEHU LQ 6\VWHP E\ $UULYDO 0RGLILHU DW 6LPXODWHG 'D\V 7 $UULYDO 0RGLILHU t$0f 6WXGHQW 8WLOL]DWLRQ 3HUFHQWDJH 7RWDO 6HUYLFH 7LPH PLQf ([SHFWHG 1XPEHU LQ 6\VWHP r /RZHU $UULYDO 0RGLILHUV HTXDO VKRUWHU LQWHUDUULYDOV ( &' W! &2 F f 6, ( /8 ; ; ; ; 2 $ 'B ; 2 2 R A 6n A 6n A 6n ‘" R! 9 ‘t t ‘! e! "R L" +RXU RI 'D\ )LJXUH ([SHFWHG 1XPEHU LQ 6\VWHP IRU 6L[ 5HSOLFDWLRQV RI 'D\V

PAGE 151

7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU 6HOHFWHG &RPSDULVRQV %HWZHHQ 2EVHUYHG DQG 6LPXODWHG 'DWDr 1 0HDQ 6WG 'HYLDWLRQ b &, IRU 0HDQ /RZHU 8SSHU %RXQG %RXQG 2EVHUYHG 7RWDO 4XHVWLRQV 'DLO\ 4XHVWLRQV 'HOD\HG 3HUFHQW 6LPXODWHG 7RWDO 4XHVWLRQV 'DLO\ 4XHVWLRQV 'HOD\HG 3HUFHQW r1RWH &RQILGHQFH ,QWHUYDOV 3URGXFHG 8VLQJ 76WDWLVWLF 7DEOH &RPSDULVRQ RI 6LPXODWHG 4XHXH 6WDWLVWLFV 9HUVXV ([DFW 6ROXWLRQ n ([DFW 6ROXWLRQ 6LPXO 0HDQ 6LP 6WG 'HYLDWLRQ b &, IRU 6LPXODWHG 0HDQ SYDOXH /RZHU %RXQG 8SSHU %RXQG 8WLOL]DWLRQ Sf 1R :DLWLQJ /Tf 1R LQ 6\VWHP /f 4XHXH :DLW :Tf 7RWDO 6HUYLFH 7LPH :f A &RQILGHQFH ,QWHUYDOV 3URGXFHG 8VLQJ 6WDWLVWLF

PAGE 152

A4 [LD r r rt 2 YR tt r A 7RWDO 6HUYLFH 7LPHV REVHUYHG SGI 0LQXWHVf 6LPXODWHG 3') )LJXUH 2EVHUYHG 9HUVXV 6LPXODWHG 3UREDELOLW\ 'HQVLW\ )XQFWLRQV 3')f 7RWDO 6HUYLFH 7LPHV 0LQXWHVf 2EVHUYHG &') B 6LPXODWHG &') )LJXUH 2EVHUYHG 9HUVXV 6LPXODWHG &XPXODWLYH 'HQVLW\ )XQFWLRQV &')f

PAGE 153

5HODWLRQVKLS RI 6WDIILQJ /HYHOV DQG 6HUYLFH 7LPHV WR WKH 3HUFHQWDJH RI 6HUYLFH 'HOD\V 7KH YDOLGDWHG VLPXODWLRQ SURJUDP ZDV XVHG WR WHVW WKUHH VSHFLILF UHVHDUFK TXHVWLRQV )LUVW WKH VLPXODWLRQ ZDV XVHG WR H[SORUH KRZ FKDQJHV LQ WKH VWDIILQJ OHYHOV DQG VHUYLFH UDWHV ZRXOG DIIHFW WRWDO VHUYLFH WLPHV DQG VHUYLFH GHOD\V 6HFRQG WKH VLPXODWLRQ GDWD ZDV DQDO\]HG WR GHWHUPLQH WKH RSWLPDO FRPELQDWLRQ RI VWDIILQJ OHYHOV DQG SRWHQWLDO LPSURYHPHQWV LQ VHUYLFH UDWHV DQG VHUYLFH GHOD\V 7KLUG WKH RSWLPDO VROXWLRQ ZDV WHVWHG IRU VHQVLWLYLW\ WR FKDQJHV LQ FDOO DUULYDO UDWH 5O +RZ GR FKDQJHV LQ VWDIILQJ OHYHOV DQG VHUYLFH UDWHV LPSDFW VLPXODWHG WRWDO VHUYLFH WLPHV DQG WKH SHUFHQWDJH RI VHUYLFH GHOD\V LQ WKH GUXJ LQIRUPDWLRQ VHUYLFH" 6L[W\ VLPXODWHG PRQWKV DQWLWKHWLF SDLUVf ZHUH UXQ IRU ILYH VWDIILQJ OHYHOV DQG QLQH OHYHOV RI VHUYLFH UDWHV 7KH VWDIILQJ OHYHOV ZHUH YDULHG IURP RQH WR ILYH VWXGHQWV 7KH VHUYLFH UDWH FRUUHVSRQGLQJ WR WKH WLPH UHTXLUHG WR UHVHDUFK D TXHVWLRQ DQG REWDLQ DQ DSSURYDO LH UHVHDUFK WLPHf ZDV YDULHG E\ GHJUHHV RI b b b DQG b DERYH DQG EHORZ fQRUPDOf RSHUDWLRQ f1RUPDOf RSHUDWLRQ ZDV GHILQHG DV WKUHH VWXGHQWV VWDIILQJ WKH FHQWHU DQG ZLWK DUULYDO DQG VHUYLFH UDWHV GHULYHG IURP HPSLULFDOO\ REVHUYHG GLVWULEXWLRQV 7R HYDOXDWH WKLV UHVHDUFK TXHVWLRQ VWDIILQJ OHYHOV DQG YDULDWLRQ RI VHUYLFH UDWHV ZHUH HYDOXDWHG LQGHSHQGHQWO\ FRPSDUHG WR WKH fQRUPDOf V\VWHP 7DEOHV DQG SUHVHQW WKH GHVFULSWLYH VWDWLVWLFV IRU WKH YDULRXV SDUDPHWHUV PHDVXUHG E\ WKH VLPXODWLRQ LQFOXGLQJ WRWDO VHUYLFH WLPH :f ZDLW LQ TXHXH :Tf QXPEHU LQ V\VWHP /f TXHXH OHQJWK /Tf QXPEHU FRPSOHWHG XWLOL]DWLRQ Sf DQG GHOD\ SHUFHQWDJH (IIHFW RI &KDQJHV LQ 6WDIILQJ /HYHOV 7KHUH ZHUH VKDUS GHFUHDVHV LQ : DV WKH QXPEHU RI VLPXODWHG VWXGHQWV VWDIILQJ WKH VHUYLFH ZDV LQFUHDVHG IURP RQH VWXGHQW PLQXWHV V f WR WZR VWXGHQWV PLQXWHV V f DQG IURP WZR VWXGHQWV WR WKUHH VWXGHQWV PLQXWHV V f 7DEOH

PAGE 154

f $ ODUJH GHFUHDVH LQ : ZDV DOVR REVHUYHG DV WKH QXPEHU RI VWXGHQWV ZDV LQFUHDVHG IURP WKUHH WR IRXU VWXGHQWV PLQXWHV V f DOWKRXJK WKH WUHQG ZDV OHVV GUDPDWLF 'HFUHDVHV LQ VHUYLFH WLPHV DSSHDUHG WR OHYHO RII DIWHU IRXU VWXGHQWV 7KHVH GHFUHDVHV LQ : FDQ ODUJHO\ EH H[SODLQHG E\ H[DPLQLQJ WKH UHVXOWV IRU :T )RU D VWDIILQJ OHYHO RI RQO\ RQH VWXGHQW PLQXWHV V f RI WKH WRWDO VHUYLFH WLPH bf ZHUH VSHQW MXVW ZDLWLQJ LQ WKH TXHXH 7DEOH f ,Q FRQWUDVW RQO\ V f PLQXWHV bf RI WKH WRWDO VHUYLFH WLPH ZDV VSHQW LQ WKH TXHXH ZKHQ WKUHH VWXGHQWV VWDIIHG WKH FHQWHU :T GHFUHDVHG HYHQ PRUH ZKHQ IRXU DQG ILYH VWXGHQWV VWDIIHG WKH FHQWHU 7KXV DV PRUH VWXGHQWV DUH HPSOR\HG TXHVWLRQV HQWHU VHUYLFH TXLFNHU DQG VSHQG OHVV WLPH LQ WKH TXHXH KRZHYHU DW WKH FXUUHQW TXHVWLRQ YROXPH VWDIILQJ OHYHOV RI PRUH WKDQ ILYH VWXGHQWV ZRXOG KDYH OLPLWHG EHQHILW RQ :T 7KH UHVXOWV IRU / DQG /T ZHUH FRQVLVWHQW ZLWK WKHVH UHVXOWV )RU RQH VWXGHQW /T ZDV b RI/ KRZHYHU WKLV GURSSHG WR b ZKHQ WKUHH VWXGHQWV ZHUH HPSOR\HG +RZHYHU ORRNLQJ DW WKH QXPEHU RI TXHVWLRQV FRPSOHWHG UHYHDOV DQ REYLRXV SODWHDX DIWHU WKUHH VWXGHQWV RI DSSUR[LPDWHO\ 7DEOH f 7KLV SODWHDX RFFXUUHG EHFDXVH WKH VLPXODWHG VWXGHQWV ILQLVKHG QHDUO\ DOO RI WKH TXHVWLRQV HQWHULQJ WKH V\VWHP ZLWKLQ WKH JLYHQ PRQWK &KDQJHV LQ VWDIILQJ OHYHOV ZHUH DOVR HYDOXDWHG IRU WKHLU HIIHFW RQ VLPXODWHG GHOD\ DQG XWLOL]DWLRQ SHUFHQWDJHV :KHQ OHVV WKDQ WKUHH VWXGHQWV ZHUH HPSOR\HG LQ WKH VLPXODWLRQ GUDPDWLF LQFUHDVHV LQ WKH SHUFHQWDJH RI VHUYLFH GHOD\V RFFXUUHG 8VLQJ RQO\ RQH VLPXODWHG VWXGHQW b V bf RI WKH TXHVWLRQV ZHUH GHOD\HG SDVW WKH QHHGHG WLPH DQG ZLWK WZR VWXGHQWV b V bf RI WKH TXHVWLRQV ZHUH GHOD\HG 7DEOH f :KHQ WKUHH VWXGHQWV ZHUH XVHG WKH SHUFHQWDJH RI GHOD\V ZDV VXEVWDQWLDOO\ ORZHU DW b GHOD\HG V bf +RZHYHU WKH JDLQV DIWHU WKUHH VWXGHQWV OHYHOHG RII FRQVLGHUDEO\ )RXU VWXGHQWV GLG LPSURYH WKH GHOD\ SHUFHQWDJH E\ DOPRVW b KRZHYHU WKHUH ZDV QRW D VLJQLILFDQW LPSURYHPHQW EH\RQG WKH DGGLWLRQ RI RQH VWXGHQW LH ILYH RU PRUH VWXGHQWVf 7KH GLIIHUHQFH LQ GHOD\ SHUFHQWDJH EHWZHHQ IRXU DQG ILYH VWXGHQWV ZDV RQO\ b

PAGE 155

)XUWKHUPRUH ZKHQ VWDIIHG E\ WKUHH VWXGHQWV WKH DYHUDJH XWLOL]DWLRQ SHUFHQW Sf ZDV b 7DEOH f :KHQ RQO\ RQH RU WZR VWXGHQWV ZDV VLPXODWHG S URVH WR DOPRVW b LQ ERWK FDVHV 7KLV LQGLFDWHG WKDW WKH VLPXODWHG FHQWHU ZDV XQGHUVWDIIHG ZKHQ OHVV WKDQ WKUHH VWXGHQWV ZRUNHG $V HYLGHQFHG E\ WKH LQFUHDVHV LQ : :T DQG GHOD\HG SHUFHQWDJH RQH RU WZR VWXGHQWV VLPSO\ FDQQRW PDQDJH WKH FXUUHQW YROXPH RI ZRUN :KHQ IRXU DQG ILYH VWXGHQWV ZHUH XVHG S GURSSHG WR b DQG b UHVSHFWLYHO\ 7KHVH UHVXOWV SURYLGH HDUO\ HYLGHQFH VXJJHVWLQJ WKDW IRXU VWXGHQWV PD\ UHSUHVHQW D ZRUWKZKLOH LPSURYHPHQW JLYHQ WKH VLJQLILFDQW LPSURYHPHQWV LQ WRWDO VHUYLFH WLPH DQG GHOD\HG SHUFHQWDJH $OVR WKH b WR b XWLOL]DWLRQ UHDOL]HG ZKHQ IRXU VWXGHQWV ZHUH XVHG LV SUREDEO\ LQ DQ DFFHSWDEOH UDQJH JLYHQ WKH VWXGHQWV RWKHU DFWLYLWLHV DQG UHVSRQVLELOLWLHV QRW PHDVXUHG E\ WKH VLPXODWLRQ +RZHYHU WKH XVH RI ILYH VWXGHQWV DW WKLV TXHVWLRQ YROXPH LV TXHVWLRQDEOH VLQFH WKH DGGLWLRQ RI WKH ILIWK VWXGHQW GRHV QRW VHHP WR RIIHU VLJQLILFDQW LPSURYHPHQWV LQ : RU WKH GHOD\HG SHUFHQWDJH (IIHFW RI &KDQJHV LQ 6HUYLFH 5DWHV 8QGHU QRUPDO VLPXODWLRQ FRQGLWLRQV WKH PHDQ WRWDO VHUYLFH WLPH :f ZDV DSSUR[LPDWHO\ PLQXWHV 7DEOH f 7KH VLPXODWHG V\VWHP ZDV HYDOXDWHG IRU ERWK LQFUHDVHV DQG GHFUHDVHV LQ UHVHDUFK DQG DSSURYDO WLPHV RI b b b DQG b ,PSURYHPHQWV RI b b b DQG b UHVXOWHG LQ GHFUHDVHV LQ H[SHFWHG WRWDO VHUYLFH WLPHV :f RI DSSUR[LPDWHO\ PLQXWHV PLQXWHV PLQXWHV DQG PLQXWHV 6LPLODUO\ : LQFUHDVHG DV WKH WLPHV UHTXLUHG WR UHVHDUFK DQG DSSURYH TXHVWLRQV LQFUHDVHG $ b LQFUHDVH LQ UHVHDUFK WLPH UHVXOWHG LQ DQ LQFUHDVH LQ : IURP DSSUR[LPDWHO\ PLQXWHV V f WR PLQXWHV V f DQG D b LQFUHDVH UHVXOWHG LQ DQ LQFUHDVH LQ : WR V f PLQXWHV ,Q DGGLWLRQ D b LQFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV UHVXOWHG LQ DQ LQFUHDVH LQ : WR PLQXWHV V f DQG D b LQFUHDVH UHVXOWHG LQ DQ LQFUHDVH LQ : WR PLQXWHV V f

PAGE 156

7KH EHKDYLRU RI / ZDV FRQVLVWHQW ZLWK WKH UHVXOWV IRU : GHVFULEHG DERYH 8QGHU QRUPDO FRQGLWLRQV WKH H[SHFWHG QXPEHU LQ WKH VLPXODWHG V\VWHP ZDV TXHVWLRQV 7DEOH f ,QFUHDVHV RI b DQG b LQ UHVHDUFK DQG DSSURYDO WLPHV UHVXOWHG LQ LQFUHDVHV LQ / WR DSSUR[LPDWHO\ V f DQG V f TXHVWLRQV UHVSHFWLYHO\ $ b LQFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV LQFUHDVHG / WR DERXW V f DQG D b LQFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV LQFUHDVHG WKH /T WR DERXW V f $V H[SHFWHG ZKHQ UHVHDUFK DQG DSSURYDO WLPHV LQFUHDVHG VLJQLILFDQWO\ WKH SHUFHQWDJH RI VHUYLFH GHOD\V DQG WKH VWXGHQW XWLOL]DWLRQ SHUFHQWDJH DOVR LQFUHDVHG $ b LQFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV UHVXOWHG LQ DQ RFFXUUHQFH RI VHUYLFH GHOD\V b V bf RI WKH WLPH ZKLOH XWLOL]DWLRQ LQFUHDVHG WR b V bf $Q LQFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV RI b LQFUHDVHG WKH SHUFHQWDJH RI GHOD\V WR b V bf DQG XWLOL]DWLRQ WR b V bf $ b LQFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV LQFUHDVHG WKH SHUFHQWDJH RI GHOD\V WR b V bf DQG WKH XWLOL]DWLRQ SHUFHQWDJH WR b V bf 'HFUHDVHV LQ WKH WLPH WR UHVHDUFK DQG DSSURYH TXHVWLRQV OHDG WR LPSURYHPHQWV LQ WKH SHUFHQWDJH RI VHUYLFH GHOD\V DQG GHFUHDVHV LQ WKH OHYHO RI VWXGHQW XWLOL]DWLRQ $ b GHFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV LPSURYHG WKH GHOD\ SHUFHQWDJH IURP b V bf WR b V bf DQG XWLOL]DWLRQ GHFUHDVHG IURP b V bf WR b V bf 7DEOH f $ b GHFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV LPSURYHG WKH GHOD\ SHUFHQWDJH WR b V bf EXW RQO\ UHGXFHG WKH XWLOL]DWLRQ WR b V bf ,Q DGGLWLRQ D b GHFUHDVH LQ UHVHDUFK DQG DSSURYDO WLPHV LPSURYHG WKH GHOD\ SHUFHQWDJH WR b V bf DQG UHGXFHG XWLOL]DWLRQ WR b V bf 7KHVH UHVXOWV LPSOLHG WKDW DQ RYHUDOO b UHGXFWLRQ LQ UHVHDUFK WLPHV ZRXOG GHFUHDVH WKH GHOD\ SHUFHQWDJH E\ PRUH WKDQ WKH DGGLWLRQ RI DQRWKHU VWXGHQW DQG VWLOO PDNH PRUH HIILFLHQW XVH RI WKH H[LVWLQJ SHUVRQQHO 7KXV FRQVLGHULQJ WKDW DGGLQJ RQH VWXGHQW LPSURYHG WKH VLPXODWHG GHOD\ SHUFHQWDJH IURP b WR b EXW UHGXFHG XWLOL]DWLRQ WR

PAGE 157

b LW DSSHDUV WKDW LPSURYLQJ VHUYLFH UDWHV RIIHUV D PRUH HIILFLHQW DOWHUQDWLYH ZKHQ SRVVLEOH 5 :KDW FRPELQDWLRQ RI FKDQJHV LQ VWDIILQJ OHYHOV DQG VHUYLFH UDWHV RSWLPL]HV WKH V\VWHP IRU GHOD\V LQ VHUYLFH DQG WRWDO VHUYLFH WLPH" 7KH VHUYLFH TXDOLW\ UHVXOWV SUHVHQWHG HDUOLHU SURYLGHG HYLGHQFH LQGLFDWLQJ WKDW ERWK SHUFHLYHG VHUYLFH WLPH +D WKURXJK +Ef DQG VHUYLFH GHOD\V +f ZHUH UHODWHG WR HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ 6LQFH SHUFHLYHG VHUYLFH WLPH ZDV QHFHVVDULO\ PHDVXUHG E\ VXEMHFWLYH PHDQV WKH UHODWLRQVKLSV REVHUYHG FRXOG QRW REMHFWLYHO\ EH TXDQWLILHG LQ D ZD\ XVHIXO IRU WKH SXUSRVHV RI VLPXODWLRQ ,Q DGGLWLRQ SHUFHLYHG VHUYLFH WLPH ZDV QRW VLJQLILFDQWO\ UHODWHG WR DFWXDO VHUYLFH WLPH RU VHUYLFH GHOD\V +D DQG +Ef VR WKH WUDQVIRUPDWLRQ RI SHUFHLYHG VHUYLFH WLPH LQWR D XVHIXO YDULDEOH ZDV QRW SRVVLEOH 7KHUHIRUH VHUYLFH GHOD\ ZDV WKH RQO\ UHPDLQLQJ VWDWLVWLFDOO\ VLJQLILFDQW OLQN EHWZHHQ HYDOXDWLRQV RI VHUYLFH TXDOLW\ DQG FRQWUROODEOH DVSHFWV RI WKH V\VWHP VWXGLHG &RQVHTXHQWO\ WKH H[SHFWHG SHUFHQWDJH RI VHUYLFH GHOD\V ZDV XVHG DV WKH SULPDU\ YDULDEOH WR EH RSWLPL]HG LQ WKH VLPXODWLRQ PRGHO 6HFRQGDU\ FRQVLGHUDWLRQ ZDV JLYHQ WR OHYHOV RI SHUIRUPDQFH UHVXOWLQJ LQ HIILFLHQW WRWDO VHUYLFH WLPHV :f 6LQFH WKH DGGLWLRQ RI VWXGHQWV GRHV QRW UHVXOW LQ SURSRUWLRQDO LQFUHDVHV GLUHFW FRVWV LH KRXUO\ ZDJHVf FRVW ZDV DQ LQDSSURSULDWH PHFKDQLVP IRU RSWLPL]DWLRQ +RZHYHU WKH FRGLUHFWRUV GHVFULEHG DW OHDVW ILYH QHJDWLYH FRQVHTXHQFHV RI DGGLQJ DGGLWLRQDO VWXGHQWV )LUVW HDFK DGGLWLRQDO VWXGHQW UHTXLUHV DGGLWLRQDO WUDLQLQJ WLPH VXSSRUW DQG VXSHUYLVLRQ DQG VLQFH QHZ VWXGHQWV DUH EURXJKW LQ DQG WUDLQHG HDFK PRQWK WKLV FRVW GRHV QRW W\SLFDOO\ GLPLQLVK RYHU WLPH 6HFRQG ODFN RI VSDFH DQG FRPSXWHU HTXLSPHQW PDNHV WKH XVH RI PRUH WKDQ IRXU VWXGHQWV XQFRPIRUWDEOH 7KLUG HDFK VWXGHQW DGGV WR WKH WLPH UHTXLUHG WR FRPSOHWH WKH JURXS HGXFDWLRQDO DFWLYLWLHV HDFK ZHHN VXFK DV MRXUQDO FOXE SUHVHQWDWLRQV )RXUWK HDFK VWXGHQW UHTXLUHV PRUH LQGLYLGXDO HGXFDWLRQ WLPH LQ WHUPV RI JXLGDQFH DQG JUDGLQJ )LIWK LQFUHDVLQJ WKH QXPEHU RI VWXGHQWV GHFUHDVHV WKH QXPEHU RI

PAGE 158

TXHVWLRQV WKDW WKH VWXGHQWV DQVZHU RYHUDOO GXULQJ WKHLU URWDWLRQ SHUKDSV GLPLQLVKLQJ WKHLU H[SHULHQFH 7KXV WKH RYHUULGLQJ DWWLWXGH EHWZHHQ WKH FRGLUHFWRUV VHHPHG WR UHIOHFW D GHVLUH WR SURYLGH WKH EHVW SRVVLEOH VHUYLFH ZLWKRXW DGGLQJ XQQHFHVVDU\ SHUVRQQHO 7KHUHIRUH VWXGHQW XWLOL]DWLRQ SHUFHQWDJH ZDV FKRVHQ DV WKH YDULDEOH DJDLQVW ZKLFK GHOD\V DQG WRWDO VHUYLFH WLPH ZHUH RSWLPL]HG 8WLOL]DWLRQ SHUFHQWDJH DV DSSOLHG IURP WKH GHVFULSWLRQ LQ FKDSWHU RQH LV WKH SHUFHQWDJH RI WKH WRWDO WLPH WKDW WKH VWXGHQWV DUH EXV\ 7KHUHIRUH RQH PLQXV WKH XWLOL]DWLRQ SHUFHQWDJH LH 8f LV WKH LGOH WLPH SHUFHQWDJH 7KHUH LV DQ LPSRUWDQW WUDGHRII EHWZHHQ LGOH WLPH DQG GHFUHDVHV LQ VHUYLFH GHOD\V DQG WRWDO VHUYLFH WLPHV $V 0F&ODLQ DQG 7KRPDV f GHVFULEH f>ZMDLWLQJ WLPHV FDQ EH UHGXFHG E\ LQFUHDVLQJ WKH UHODWLYH VHUYLFH FDSDFLW\ 7KLV PD\ EH DFFRPSOLVKHG E\ UHGXFLQJ WKH DUULYDO UDWH RU LQFUHDVLQJ WKH QXPEHU RI VHUYHUV RU WKHLU ZRUN SDFH VHUYLFH UDWHf $OO WKHVH DFWLRQV ZLOO LQFUHDVH WKH DYHUDJH LGOH WLPH RI WKH VHUYHUVf S f 7KHUHIRUH WKH XVH RI XWLOL]DWLRQ SHUFHQWDJH DV WKH EDODQFLQJ YDULDEOH SURYLGHG D PHDQV RI HVWDEOLVKLQJ ZKLFK FKDQJHV KDG WKH ODUJHVW HIIHFW RQ VHUYLFH GHOD\V ZKLOH PDNLQJ WKH PRVW HIILFLHQW XVH RI H[LVWLQJ SHUVRQQHO 7KLV PHWKRG RI HYDOXDWLQJ HIILFLHQF\ LV DOVR FRQVLVWHQW ZLWK PHWKRGV SUHVHQWHG E\ 7KRPSVRQ f DQG :HVWJDUG DQG %HUU\ f 7KH RSWLPDO FRPELQDWLRQ RI LPSURYHPHQWV ZDV HYDOXDWHG XVLQJ D UDWLR DQDO\VLV FRPSDULQJ WKH HIIHFW RI VLPXODWHG LPSURYHPHQWV LQ VHUYLFH FDSDFLW\ RQ GHOD\ SHUFHQWDJH DQG WRWDO VHUYLFH WLPHV YHUVXV FKDQJHV LQ VWXGHQW XWLOL]DWLRQ SHUFHQWDJH $V GHVFULEHG DERYH SRWHQWLDO LPSURYHPHQWV LQFOXGHG WKH DGGLWLRQ RI RQH RU WZR VWXGHQWV LH IRXU RU ILYH WRWDOf DQGRU b b b RU b LPSURYHPHQWV LQ UHVHDUFK DQG DSSURYDO WLPHV (DFK SRWHQWLDO FRPELQDWLRQ RI LPSURYHPHQWV ZDV FRPSDUHG WR QRUPDO RSHUDWLQJ FRQGLWLRQV LH WKUHH VWXGHQWV DQG HPSLULFDOO\ REVHUYHG VHUYLFH DQG DUULYDO UDWHVf 7ZR YDOXHV PHDVXULQJ HIILFLHQF\ ZHUH FDOFXODWHG )LUVW D YDOXH PHDVXULQJ WKH HIILFLHQF\ RI WKH VLPXODWHG H[SHFWHG GHOD\ SHUFHQWDJH YHUVXV WKH VLPXODWHG H[SHFWHG XWLOL]DWLRQ SHUFHQWDJH ZDV FDOFXODWHG E\ GLYLGLQJ WKH XWLOL]DWLRQ SHUFHQWDJH E\ WKH GHOD\ SHUFHQWDJH LH 8'f 6LQFH WKH JRDO ZDV WR KDYH ORZHU SHUFHQWDJHV RI GHOD\V SHU

PAGE 159

SHUFHQWDJH RI XWLOL]DWLRQ ODUJHU YDOXHV LQGLFDWHG PRUH HIILFLHQW XVH RI SHUVRQQHO 6HFRQG D YDOXH PHDVXULQJ WKH HIILFLHQF\ RI WKH VLPXODWHG H[SHFWHG WRWDO VHUYLFH WLPH YHUVXV WKH VLPXODWHG H[SHFWHG XWLOL]DWLRQ SHUFHQWDJH ZDV FDOFXODWHG E\ GLYLGLQJ WKH XWLOL]DWLRQ SHUFHQWDJH E\ WKH H[SHFWHG WRWDO VHUYLFH WLPH LQ PLQXWHV WLPHV LH 8: [ f 6LPLODUO\ ODUJHU YDOXHV IRU WKLV PHDVXUH LQGLFDWHG JUHDWHU HIILFLHQF\ VLQFH WKH JRDO ZDV WR DFKLHYH ORZHU WRWDO VHUYLFH WLPHV ZLWKRXW VXEVWDQWLDOO\ GHFUHDVLQJ XWLOL]DWLRQ 7DEOH UHSRUWV WKH UHVXOWV RI WKLV DQDO\VLV $ b GHFUHDVH LQ VHUYLFH UDWHV ZKLOH PDLQWDLQLQJ D VWDIILQJ OHYHO RI WKUHH VWXGHQWV ZDV WKH PRVW HIILFLHQW PHDQV RI UHGXFLQJ GHOD\V LQ VHUYLFH 7KLV OHYHO RI LPSURYHPHQW UHVXOWHG LQ HIILFLHQF\ UDWLR YDOXH 8'f HTXDO WR 8 b bf LQGLFDWLQJ WKDW HDFK SHUFHQWDJH SRLQW LQ GHOD\ LV HTXDO WR DSSUR[LPDWHO\ SHUFHQWDJH SRLQWV LQ XWLOL]DWLRQ (YHQ WKRXJK WKH DGGLWLRQ RI WZR VWXGHQWV LH ILYH VWXGHQWVf WHQGV WR PLQLPL]H WKH SHUFHQWDJH RI GHOD\V LW GRHV QRW SURGXFH RSWLPXP UHVXOWV VLQFH VXEVWDQWLDO GHFUHDVHV LQ XWLOL]DWLRQ LH LQFUHDVHV LQ LGOH WLPHf DUH DOVR UHDOL]HG 7KH DGGLWLRQ RI RQH VWXGHQW UHVXOWHG LQ DQ RYHUDOO HIILFLHQF\ UDWLR RI 7DEOH LQGLFDWHV WKDW ZKLOH WKH DGGLWLRQ RI D VWXGHQW GRHV VLJQLILFDQWO\ GHFUHDVH WKH SHUFHQWDJH RI GHOD\V WKH V\VWHP ZRXOG DFWXDOO\ EH OHVV HIILFLHQW VLQFH WKH RYHUDOO HIILFLHQF\ UDWLR XQGHU QRUPDO FRQGLWLRQV ZDV $OWKRXJK WKH ORZHVW GHOD\ SHUFHQWDJHV ZHUH REVHUYHG ZKHQ DGGLWLRQDO VWXGHQWV ZHUH DGGHG LW GRHV QRW DSSHDU WKDW WKH FXUUHQW TXHVWLRQ YROXPH QHFHVVLWDWHV DGGLWLRQDO VWXGHQWV 7KLV SRLQW LV LOOXVWUDWHG E\ )LJXUH ZKLFK VKRZV WKDW ILYH VWXGHQWV DOZD\V SURGXFHV IHZHU GHOD\V DW D JLYHQ VHUYLFH UDWH +RZHYHU DV VHUYLFH UDWHV LPSURYHG WKH DGYDQWDJH RI DGGLQJ DGGLWLRQDO VWXGHQWV GHFUHDVHG $ b GHFUHDVH LQ VHUYLFH UDWHV SURGXFHG QHDUO\ WKH VDPH EHQHILW XVLQJ WKUHH VWXGHQWV bf DV LW ZRXOG XVLQJ IRXU VWXGHQWV bf )XUWKHUPRUH D b GHFUHDVH LQ VHUYLFH UDWHV XQGHU IRXU DQG ILYH VWXGHQWV SURGXFHG VLPLODU GHFUHDVHV LQ GHOD\V LH GLIIHUHQFH RI RQO\ bf KRZHYHU WKH XWLOL]DWLRQ SHUFHQWDJH ZDV PXFK ORZHU ZLWK ILYH VWXGHQWV LH b ZLWK ILYH VWXGHQWV b ZLWK IRXU VWXGHQWVf 7KH UHVXOWV LQGLFDWH WKDW LPSURYLQJ WKH VHUYLFH UDWH

PAGE 160

ZDV JHQHUDOO\ D PRUH LPSRUWDQW IDFWRU LQ UHGXFLQJ WKH GHOD\ SHUFHQWDJH WKDQ DGGLWLRQDO VWDIILQJ ,QWHUHVWLQJO\ KRZHYHU WKH PRVW HIILFLHQW VHUYLFH FDSDFLW\ LQ WHUPV RI WRWDO VHUYLFH WLPHV GLG UHTXLUH DQ DGGLWLRQDO VWXGHQW LH IRXU VWXGHQWVf $ b GHFUHDVH LQ VHUYLFH UDWHV ZKLOH PDLQWDLQLQJ D VWDIILQJ OHYHO RI IRXU VWXGHQWV SURGXFHG WKH PRVW HIIHFWLYH PHDQV RI UHGXFLQJ WKH WRWDO VHUYLFH WLPHV 7KLV OHYHO RI LPSURYHPHQW UHVXOWHG LQ DQ HIILFLHQF\ UDWLR 8$9f HTXDO WR 8 b : PLQXWHVf LQGLFDWLQJ WKDW HDFK PLQXWH RI VHUYLFH WLPH ZDV HTXDO WR DSSUR[LPDWHO\ SHUFHQWDJH SRLQWV LQ XWLOL]DWLRQ $JDLQ DOWKRXJK WKH ORZHVW WRWDO VHUYLFH WLPHV ZHUH UHDOL]HG ZKHQ ILYH VWXGHQWV ZHUH VLPXODWHG WKHVH EHQHILWV DUH RIIVHW E\ GHFUHDVHV LQ XWLOL]DWLRQ )XUWKHUPRUH LQ WHUPV RI WRWDO VHUYLFH WLPH WKH fQRUPDOf VHUYLFH FDSDFLW\ WXUQHG RXW WR EH WKH OHDVW HIILFLHQW RI WKH OHYHOV H[DPLQHG 7KHVH UHVXOWV LQGLFDWHG WKDW ZKLOH VHUYLFH UDWHV DUH LPSRUWDQW LQ HIILFLHQWO\ PDQDJLQJ WKH WRWDO VHUYLFH WLPH WKH VWDIILQJ OHYHO ZDV DW OHDVW HTXDOO\ LPSRUWDQW 2YHUDOO GHOD\V LQ VHUYLFH WHQGHG WR EH PRUH VHQVLWLYH LQ ZLWK UHJDUG WR FKDQJHV LQ WKH VHUYLFH UDWHV WKDQ WRWDO VHUYLFH WLPHV %\ FRPSDULQJ )LJXUHV DQG LW LV DSSDUHQW WKDW D VWDIILQJ OHYHO RI WKUHH VWXGHQWV ZDV IDLUO\ VHQVLWLYH WR FKDQJHV LQ WKH UHVHDUFK DQG DSSURYDO WLPHV ZLWK UHVSHFW WR WRWDO VHUYLFH WLPH +RZHYHU VWDIILQJ OHYHOV RI IRXU DQG ILYH VWXGHQWV DUH QRW VHQVLWLYH WR FKDQJHV LQ WKH VHUYLFH UDWH 7KHUHIRUH WKUHH VWXGHQWV FRPELQHG ZLWK UHGXFWLRQV LQ UHVHDUFK DQG DSSURYDO WLPHV RI DW OHDVW b DFKLHYHG WKH RSWLPXP VHUYLFH FDSDFLW\ ZKHQ FRQVLGHULQJ VHUYLFH GHOD\V +RZHYHU LI WRWDO VHUYLFH WLPH ZHUH WR EH FRQVLGHUHG PRUH LPSRUWDQW WKDQ UHGXFLQJ WKH GHOD\ SHUFHQWDJH WKHQ WKH RSWLPDO VROXWLRQ ZRXOG EH WR VWDII WKH ',35& ZLWK IRXU VWXGHQWV DQG UHGXFH UHVHDUFK DQG DSSURYDO WLPHV E\ DW OHDVW b 5 +RZ VHQVLWLYH LV WKH RSWLPXP VROXWLRQ WR UDQGRP YDULDWLRQ LQ WKH DUULYDO UDWH" 6LQFH TXHXLQJ V\VWHPV DUH RIWHQ JUHDWO\ DIIHFWHG E\ UDQGRP YDULDWLRQ LQ WKH DUULYDO UDWH 0F&ODLQ DQG 7KRPDV f WKH RSWLPL]DWLRQ PHWKRGV XVHG DERYH ZHUH UHFDOFXODWHG

PAGE 161

XQGHU FRQGLWLRQV RI YDU\LQJ DUULYDO UDWHV 7KH VLPXODWLRQ XVHG D PRGLILHU WR LQFUHDVH DQG GHFUHDVH WKH LQWHUDUULYDO UDWHV LQ b LQFUHPHQWV ,QFUHDVHV DQG GHFUHDVHV ZHUH PDGH LQ WKH DUULYDO UDWHV XQWLO WKH WKUHVKROG DW ZKLFK ERWK WKH HIILFLHQF\ UDWLRV 8' DQG 8:f VKLIWHG IURP WKHLU RSWLPDO VROXWLRQV SUHVHQWHG DERYH $V GHVFULEHG DERYH DW WKH FXUUHQW FDOO YROXPH WKH PRVW HIIHFWLYH SRLQW IRU PDQDJLQJ GHOD\V RFFXUUHG ZLWK WKUHH VWXGHQWV ZLWK DQ LPSURYHPHQW LQ UHVHDUFK DQG DSSURYDO WLPHV RI b )XUWKHUPRUH WKH PRVW HIIHFWLYH SRLQW IRU PDQDJLQJ WRWDO VHUYLFH WLPH RFFXUV ZLWK IRXU VWXGHQWV ZLWK DQ LPSURYHPHQW LQ WRWDO VHUYLFH WLPHV RI b 7DEOH SUHVHQWV WKH UHVXOWV RI WKH VHQVLWLYLW\ DQDO\VLV 7KH RSWLPXP VROXWLRQV WHQG WR EH IDLUO\ VWDEOH 7KH WKUHVKROG IRU LQFUHDVHV LQ LQWHUDUULYDOV RFFXUUHG DW b HVVHQWLDOO\ LQGLFDWLQJ WKDW LI WKH FXUUHQW YROXPH ZHUH WR LQFUHDVH VRPHZKHUH EHWZHHQ b WR b WKHQ DQ DGGLWLRQDO VWXGHQW ZRXOG EH ZDUUDQWHG 7KH WKUHVKROG IRU GHFUHDVHV LQ LQWHUDUULYDOV ZDV VOLJKWO\ PRUH VHQVLWLYH $ GHFUHDVH LQ LQWHUDUULYDO WLPHV RI b WR b ZDV WKH WKUHVKROG IRU WKH FKDQJH LQ WKH HIILFLHQF\ UDWLR IRU PDQDJLQJ GHOD\V LQ VHUYLFH 8'f $ GHFUHDVH LQ LQWHUDUULYDO WLPHV RI b WR b ZDV WKH WKUHVKROG IRU WKH FKDQJH LQ WKH HIILFLHQF\ UDWLR IRU PDQDJLQJ WRWDO VHUYLFH WLPH 8:f

PAGE 162

7DEOH 'HVFULSWLYHV IRU 4XHXH 6WDWLVWLFV E\ 1XPEHU RI 6WXGHQWV 1 f 1XPEHU RI 6WXGHQWV 0HDQ 6WG 'HYLDWLRQ b &, IRU 0HDQ /RZHU 8SSHU %RXQG %RXQG 7RWDO 6HUYLFH 7LPH ,9f 6WXGHQW 6WXGHQWV 6WXGHQWV 6WXGHQWV 6WXGHQWV :DLW LQ 4XHXH :Tf 6WXGHQW 6WXGHQWV 6WXGHQWV 6WXGHQWV 6WXGHQWV 1XPEHU LQ 6\VWHP /f 6WXGHQW 6WXGHQWV 6WXGHQWV 6WXGHQWV 6WXGHQWV 4XHXH /HQJWK /Tf 6WXGHQW 6WXGHQWV 6WXGHQWV 6WXGHQWV 6WXGHQWV 1XPEHU &RPSOHWHG 6WXGHQW 6WXGHQWV 6WXGHQWV 6WXGHQWV 6WXGHQWV 8WLOL]DWLRQ 3HUFHQWDJH Sf 6WXGHQW 6WXGHQWV 6WXGHQWV 6WXGHQWV 6WXGHQWV 'HOD\ 3HUFHQWDJH 6WXGHQW 6WXGHQWV 6WXGHQWV 6WXGHQWV 6WXGHQWV

PAGE 163

7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU WKH 7RWDO 6HUYLFH 7LPH 7LPH LQ 4XHXH 1XPEHU LQ 6\VWHP DQG 4XHXH /HQJWK E\ 3HUFHQWDJH &KDQJH LQ 5HVHDUFK DQG b &KDQJH LQ 5HVHDUFK 7LPH 0HDQ 6WG 'HYLDWLRQ b &, /RZHU %RXQG IRU 0HDQ 8SSHU %RXQG 7RWDO 6HUYLFH 7LPH :f b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH 1RUPDO b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH 7LPH LQ 4XHXH :Tf b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH 1RUPDO b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH 1XPEHU LQ 6\VWHP /f b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH b! 'HFUHDVH 1RUPDO b ,QFUHDVH b ,QFUHDVH b! ,QFUHDVH b ,QFUHDVH 4XHXH /HQJWK /Tf b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH 1RUPDO b ,QFUHDVH bR ,QFUHDVH b! ,QFUHDVH b ,QFUHDVH

PAGE 164

7DEOH 'HVFULSWLYH 6WDWLVWLFV IRU 1XPEHU &RPSOHWHG 8WLOL]DWLRQ 3HUFHQWDJH DQG 'HOD\ 3HUFHQWDJH E\ 3HUFHQW &KDQJH LQ 5HVHDUFK DQG $SSURYDO 7LPH 1 f b &/ IRU 0HDQ b &KDQJH LQ 6HUYLFH 7LPH 0HDQ 6WG 'HYLDWLRQ /RZHU %RXQG 8SSHU %RXQG 1XPEHU &RPSOHWHG b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH 1RUPDO b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH 8WLOL]DWLRQ 3HUFHQWDJH Sf b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH 1RUPDO b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH b ,QFUHDVH 'HOD\ 3HUFHQWDJH b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH b 'HFUHDVH 1RUPDO b ,QFUHDVH b ,QFUHDVH b! ,QFUHDVH b ,QFUHDVH

PAGE 165

7DEOH (IIHFWLYHQHVV RI 6HUYLFH &DSDFLW\ ,PSURYHPHQWV 8QGHU 1RUPDO $UULYDO 5DWHV Q f /HYHA ([SHFWHG 7RWDO 6HUYLFH 7LPH :fr ([SHFWHG b 'HOD\HG 'f ([SHFWHG b 8WLOL]DWLRQ 8f 8' 8: [O222f 1RUPDO b b b b b b b b b b b b b b 1RUPDO b b b b b b b b b b b b b b 1RUPDO b b b b b b b b b b b b b b $OO r 1XPEHU RI 6WXGHQWV3HUFHQW 'HFUHDVH LQ 5HVHDUFK DQG $SSURYDO 7LPH r ,Q 0LQXWHV

PAGE 166

6(59(56 a ,) 6HUYLFH 5DWH 0RGLILHU )LJXUH b &RQILGHQFH ,QWHUYDOV IRU 'HOD\ 3HUFHQWDJH E\ 6HUYLFH 5DWH 0RGLILHU DQG 1XPEHU RI 6HUYHUV 6(59(56 D f ‘ 6HUYLFH 5DWH 0RGLILHU )LJXUH b &RQILGHQFH ,QWHUYDOV IRU 7RWDO 6HUYLFH 7LPH LQ 0LQXWHVf E\ 6HUYLFH 5DWH 0RGLILHU DQG 1XPEHU RI 6HUYHUV

PAGE 167

7DEOH 6HQVLWLYLW\ RI 2SWLPDO 6ROXWLRQ WR &KDQJHV LQ WKH $UULYDO 5DWH Q f ,QWHUDUULYDO 0RGLILFDWLRQ /HYFI ([SHFWHG 7RWDO 6HUYLFH 7LPH :fr ([SHFWHG b 'HOD\HG 'f ([SHFWHG 8WLOL]DWLRQ 8f 8: 8' [O222f b ,QFUHDVH b 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b b 'HFUHDVH b b b 'HFUHDVH b b b ,QFUHDVH bc 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b b 'HFUHDVH b b b 'HFUHDVH b b b ,QFUHDVH b! 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b b! 'HFUHDVH b b b 'HFUHDVH b b b ,QFUHDVH b! 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b bR 'HFUHDVH b b b 'HFUHDVH b b b ,QFUHDVH bc 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b b! 'HFUHDVH b b b 'HFUHDVH b b b 'HFUHDVH b! 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b b! 'HFUHDVH b b b 'HFUHDVH b b b 'HFUHDVH bf 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b bR 'HFUHDVH b b b 'HFUHDVH b b b 'HFUHDVH b 'HFUHDVH b b 1RUPDO b b b 'HFUHDVH b b bR 'HFUHDVH b b b 'HFUHDVH b b b 'HFUHDVH bR 'HFUHDVH b b 1RUPDO b b b! 'HFUHDVH b b b! 'HFUHDVH b b b 'HFUHDVH b b 1XPEHU RI 6WXGHQWV3HUFHQW 'HFUHDVH LQ 5HVHDUFK DQG $SSURYDO 7LPH r ,Q 0LQXWHV

PAGE 168

&+$37(5 ',6&866,21 $1' &21&/86,216 2YHUYLHZ 7KLV VWXG\ KDG WKUHH SULPDU\ REMHFWLYHV )LUVW WR GHYHORS D FRPSXWHU VLPXODWLRQ IRU D GUXJ LQIRUPDWLRQ VHUYLFH DQG YDOLGDWH WKH PRGHO DJDLQVW WKH H[LVWLQJ V\VWHP 6HFRQG WR LQYHVWLJDWH WKH DVVRFLDWLRQV DPRQJ DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V DQG HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ LQ D GUXJ LQIRUPDWLRQ VHUYLFH VHWWLQJ 7KLUG WR UHFRPPHQG V\VWHP LPSURYHPHQWV EDVHG RQ WKH VLPXODWLRQ PRGHO LQ SDUWLFXODU WKRVH LPSURYHPHQWV WKDW PDLQWDLQ TXDOLW\ ZKLOH UHGXFLQJ GHOD\V DQG UHVSRQVH WLPHV 7R DFKLHYH WKHVH REMHFWLYHV WKH GDWD DQDO\VLV ZDV EURNHQ XS LQWR IRXU UHODWHG SDUWV 7KH ILUVW DQG VHFRQG SDUWV ZHUH SUHVHQWHG LQ FKDSWHU ILYH DV SUHOLPLQDU\ UHVXOWV ZKLFK GLVFXVVHG WKH UHVXOWV RI WKH DQDO\VLV RI WKH KLVWRULFDO GDWD DQG SHUVRQDO LQWHUYLHZV 7KH WKLUG DQG IRXUWK SDUWV ZHUH GLUHFWO\ UHODWHG WR WKH K\SRWKHVHV DQG VSHFLILF UHVHDUFK TXHVWLRQV SRVHG LQ FKDSWHU WKUHH DQG ZHUH SUHVHQWHG LQ FKDSWHU VL[ DV PDLQ UHVXOWV 7KLV FKDSWHU EHJLQV E\ VXPPDUL]LQJ DQG GLVFXVVLQJ WKH UHVXOWV IURP FKDSWHU VL[ DV WKH\ UHODWH WR WKH K\SRWKHVLV WHVWV DQG VSHFLILF UHVHDUFK TXHVWLRQV :KHUH DSSOLFDEOH WKHVH UHVXOWV DUH FRPSDUHG ZLWK WKH H[LVWLQJ OLWHUDWXUH 1H[W WKH OLPLWDWLRQV RI WKH VWXG\ DUH GHVFULEHG 7KLV FKDSWHU FRQFOXGHV ZLWK D SUHVHQWDWLRQ RI WKH VWXG\ FRQFOXVLRQV DQG UHFRPPHQGDWLRQV IRU IXWXUH UHVHDUFK

PAGE 169

6XPPDU\ DQG 'LVFXVVLRQ RI 5HVXOWV 362 262 DQG %HKDYLRUDO ,QWHQWLRQ 7KH ILUVW WKUHH VWXG\ K\SRWKHVHV +, + DQG +f WHVWHG WKH UHODWLRQVKLSV DPRQJ 364 264 DQG EHKDYLRUDO LQWHQWLRQ $ LWHP VFDOH PHDVXUHG SHUFHLYHG VHUYLFH TXDOLW\ ZKHUH WKH LWHPV ZHUH VXPPHG WR SURGXFH D 364 VFRUH 264 ZDV D VLQJOH LWHP PHDVXUH DVNLQJ FDOOHUV WR UDQN WKH RYHUDOO VHUYLFH RQ D VFDOH IURP f([FHOOHQWf WR f8QDFFHSWDEOHf %HKDYLRUDO LQWHQWLRQ ZDV D WZRLWHP PHDVXUH UHIOHFWLQJ WKH FDOOHUVf SHUFHSWLRQV UHJDUGLQJ WKHLU LQWHQWLRQ WR XVH WKH VHUYLFH DJDLQ DQG UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH 7KLV UHVHDUFK FRQILUPHG WKH DQWLFLSDWHG UHODWLRQVKLSV DPRQJ WKHVH YDULDEOHV 7KH VWURQJHVW FRUUHODWLRQ EHWZHHQ WKHVH YDULDEOHV ZDV REVHUYHG EHWZHHQ 364 DQG 264 U f +RZHYHU VLJQLILFDQW UHODWLRQVKLSV ZHUH DOVR REVHUYHG EHWZHHQ 364 DQG EHKDYLRUDO LQWHQWLRQ U DQG f DQG EHWZHHQ 264 DQG EHKDYLRUDO LQWHQWLRQ U DQG f $V GLVFXVVHG LQ FKDSWHU VL[ WKHVH UHODWLRQVKLSV ZHUH FRQVLVWHQW ZLWK VLPLODU FRUUHODWLRQV UHSRUWHG LQ &URQLQ DQG 7D\ORU f ,Q DGGLWLRQ +HDGOH\ DQG 0LOOHU f UHSRUWHG VWURQJ VLJQLILFDQW UHODWLRQVKLSV EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ DQG RYHUDOO TXDOLW\ ZLWK FRPSDUDEOH GHJUHHV RI H[SODLQHG YDULDQFH 7KH GLUHFWLRQ RI WKLV UHODWLRQVKLS ZDV DOVR FRQVLVWHQW ZLWK UHSRUWV IURP %RXOGLQJ HW DO f 0F$OH[DQGHU HW DO f 3DUDVXUDPDQ f DQG =HLWKDPO HW DO f 7KLV KLJK GHJUHH RI FRQYHUJHQFH VXSSRUWV WKH FRQVWUXFW YDOLGLW\ RI WKHVH PHDVXUHV $OWKRXJK WKH FRUUHODWLRQ EHWZHHQ 364 DQG 264 ZDV UHODWLYHO\ KLJK WKH UHDOLW\ RI WKLV VWDWLVWLF LQGLFDWHV WKDW 364 RQO\ H[SODLQV DSSUR[LPDWHO\ b RI WKH YDULDQFH LQ 264 7KLV VHHPLQJO\ ORZ SUHGLFWLYH DELOLW\ PLJKW EH SDUWLDOO\ H[SODLQHG LI ZH FRQVLGHU WKDW 364 DV GHULYHG IURP 6(593(5)f LV ODUJHO\ D PHDVXUH RI IXQFWLRQDO TXDOLW\ L H SURFHVV UHODWHG TXDOLW\f %DEDNXV DQG 0DQJROG *URQURRV f ,Q DGGLWLRQ WR IXQFWLRQDO TXDOLW\ YDULRXV UHVHDUFKHUV KDYH VXJJHVWHG WKDW SHUFHSWLRQV RI RYHUDOO TXDOLW\ PD\ EH

PAGE 170

LQIOXHQFHG E\ WHFKQLFDO RU RXWFRPH TXDOLW\ WKH LPDJH RI WKH VHUYLFH SUHYLRXV VDWLVIDFWLRQ H[SHULHQFHV DQG FRQVXPHUVf SHUVRQDO DQG VLWXDWLRQ IDFWRUV %ROWRQ DQG 'UHZ *URQURRV 6WHHQNDPS =HLWKDPO f ,W LV FOHDU WKDW PRUH UHVHDUFK LV QHHGHG LQ RUGHU WR FUHDWH D PXOWLGLPHQVLRQDO PHDVXUHPHQW WRRO WKDW SUHGLFWV RYHUDOO SHUFHSWLRQV RI TXDOLW\ EHWWHU WKDQ 6(5948$/ RU 6(593(5) )XUWKHUPRUH JLYHQ WKH PDQ\ W\SHV RI VHUYLFHV DQG WKHLU LQKHUHQW LGLRV\QFUDVLHV LW PD\ QRW EH SRVVLEOH WR FUHDWH D VLQJXODUO\ YDOLG LQVWUXPHQW IRU HYDOXDWLQJ SHUFHLYHG VHUYLFH TXDOLW\ 1HYHUWKHOHVV WKH VHUYLFH TXDOLW\ OLWHUDWXUH FRQWHQGV WKDW WKHUH LV D VWURQJ UHODWLRQVKLS EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ DQG EHKDYLRUDO LQWHQWLRQ 7KH UHVXOWV RI WKLV VWXG\ FRPSOLPHQW WKHVH DVVHUWLRQV 7KXV LW DSSHDUV WKDW WKH PRUH SRVLWLYH WKDW FDOOHUV IHHO DERXW WKH VHUYLFH TXDOLW\ RI WKH ',35& WKH PRUH OLNHO\ WKH\ DUH WR KDYH IDYRUDEOH LQWHQWLRQV RI FDOOLQJ WKH VHUYLFH DJDLQ DQG UHFRPPHQGLQJ WKH VHUYLFH WR D FROOHDJXH ,W LV LQWHUHVWLQJ WR QRWH KRZHYHU WKDW LQ ERWK WKLV VWXG\ DQG WKH VWXG\ FRQGXFWHG E\ &URQLQ DQG 7D\ORU f WKH UHODWLRQVKLSV EHWZHHQ PHDVXUHV RI RYHUDOO TXDOLW\ DQG EHKDYLRUDO LQWHQWLRQ ZHUH VWURQJHU WKDQ WKH UHODWLRQVKLSV EHWZHHQ 364 DQG EHKDYLRUDO LQWHQWLRQ 7KLV VXJJHVWV SHUKDSV WKDW SHUFHSWLRQV RI RYHUDOO TXDOLW\ L H RYHUDOO LPSUHVVLRQVf DUH PRUH LPSRUWDQW LQ GULYLQJ IXWXUH EHKDYLRU WKDQ HYDOXDWLRQV RI TXDOLW\ EDVHG RQ VSHFLILF DVSHFWV RI WKH VHUYLFH $JDLQ KRZHYHU IXUWKHU UHVHDUFK LV QHFHVVDU\ WR YDOLGDWH WKLV DVVXPSWLRQ $FWXDO 6HUYLFH 7LPH 6HUYLFH 'HOD\V 3HUFHLYHG 6HUYLFH 7LPH DQG 364 +\SRWKHVHV + + + DQG + ZHUH GHYHORSHG WR WHVW WKH UHODWLRQVKLSV DPRQJ DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V SHUFHLYHG VHUYLFH WLPH DQG 364 $FWXDO VHUYLFH WLPH UHIHUUHG WR WKH DFWXDO DPRXQW RI WLPH LQ PLQXWHVf UHTXLUHG WR UHVSRQG WR D TXHVWLRQ DQG ZDV PHDVXUHG E\ VXEWUDFWLQJ WKH fHQG WLPHf IRU UHFHLYLQJ D FDOO IURP WKH fVWDUW WLPHf IRU UHWXUQLQJ D DQVZHU 6HUYLFH GHOD\ ZDV D GLFKRWRPRXV YDULDEOH UHIHUULQJ WR ZKHWKHU RU QRW WKH UHVSRQVH WLPH ZDV ORQJHU WKDQ WKH WLPH QHHGHG LH fVWDWf fWRGD\f fGDWHf DQG fQR

PAGE 171

UXVKff 3HUFHLYHG VHUYLFH WLPH ZDV PHDVXUHG E\ IRXU LWHPV LQ WKH TXHVWLRQQDLUH 4 f$FFHSWDEOH 7LPHf 4 f 1R /RQJHU 8VHIXOf 4 f4XLFNHU 5HVSRQVH DQG 4 f([SHFWHG 7LPHff 2QH RI WKH UHODWLRQVKLSV SRVHG E\ WKH OLWHUDWXUH ZDV WKDW DFWXDO ZDLWLQJ WLPH ZDV UHODWHG WR VHUYLFH HYDOXDWLRQV %X[WRQ DQG *DWODQG +XL DQG 7VH 'DYLV 'DYLV DQG 9ROOPDQQ .DW] /DUVRQ DQG /DUVRQ /DUVRQ 7RP DQG /XFH\ f 7KLV VWXG\ GLG QRW ILQG DQ\ VWDWLVWLFDOO\ RU SUDFWLFDOO\ VLJQLILFDQW UHODWLRQVKLS EHWZHHQ DFWXDO VHUYLFH WLPH DQG SHUFHLYHG VHUYLFH TXDOLW\ (YHQ LI ZH ZHUH WR DVVXPH VWDWLVWLFDO VLJQLILFDQFH EDVHG RQ D ERUGHUOLQH SYDOXH S f WKH UHJUHVVLRQ HTXDWLRQ VXJJHVWHG WKDW DW EHVW 364 ZDV DIIHFWHG E\ DFWXDO VHUYLFH WLPH RQO\ ZKHQ RQH IXOO GD\ SDVVHG ZLWKRXW D UHVSRQVH 3UDFWLFDOO\ VLJQLILFDQW VKLIWV LQ 364 GLG QRW RFFXU XQWLO WKH HTXLYDOHQW RI IRXU RU ILYH GD\V KDG ODSVHG LQ DFWXDO VHUYLFH WLPH $OWKRXJK VXUSULVLQJ DW ILUVW JODQFH JLYHQ WKH OLWHUDWXUH VXSSRUW WKLV ODFN RI UHODWLRQVKLS PDNHV VHQVH LQ WKLV VHWWLQJ LI ZH FRQVLGHU WKH HLJKW SULQFLSOHV RI ZDLWLQJ DV SURSRVHG E\ 'DYLG 0DLVWHU f 3URSRVLWLRQ 3URSRVLWLRQ 3URSRVLWLRQ 3URSRVLWLRQ 3URSRVLWLRQ 3URSRVLWLRQ 3URSRVLWLRQ 3URSRVLWLRQ 8QRFFXSLHG 7LPH )HHOV /RQJHU WKDQ 2FFXSLHG 7LPH 3UHSURFHVV :DLWV )HHO /RQJHU WKDQ ,Q3URFHVV :DLWV $Q[LHW\ 0DNHV :DLWV 6HHP /RQJHU 8QFHUWDLQ :DLWV $UH /RQJHU WKDQ .QRZQ )LQLWH :DLWV 8QH[SODLQHG :DLWV $UH /RQJHU WKDQ ([SODLQHG :DLWV 8QIDLU :DLWV $UH /RQJHU WKDQ (TXLWDEOH :DLWV 7KH 0RUH 9DOXDEOH WKH 6HUYLFH WKH /RQJHU WKH &XVWRPHU :LOO :DLW 6ROR :DLWV )HHO /RQJHU WKDQ *URXS :DLWV $V GHWHUPLQHG LQ FKDSWHU WKH SUHSURFHVV WLPH UHTXLUHG WR WDNH D FDOO ZDV DSSUR[LPDWHO\ IRXU PLQXWHV DQG RQO\ DERXW b RI WKH FDOOV DUH fVWDWf TXHVWLRQV QHHGHG ZLWKLQ ILIWHHQ PLQXWHV 7DEOHV DQG f 7KHUHIRUH VLQFH WKH SUHSURFHVV ZDLWV DUH VKRUW DQG WKHUH LV XVXDOO\ QR SDUWLFXODU LPPHGLDWH UXVK WR FRPSOHWH TXHVWLRQV WKH HIIHFWV RI SURSRVLWLRQV RQH DQG WZR DERYH DUH OLPLWHG )XUWKHUPRUH FDOOHUV GR QRW DFWXDOO\ ZDLW LQ WKH ',35& IRU WKHLU TXHVWLRQV WR EH DQVZHUHG WKHUHIRUH WKH\ GR QRW ZDLW LQ DQ fDFWXDOf

PAGE 172

TXHXH 6LQFH WKHUH LV QR WLPH VSHQW ZDLWLQJ RQ KROG VROR ZDLWV SURSRVLWLRQ HLJKWf DQG XQIDLU ZDLWV SURSRVLWLRQ VL[f DUH QRW UHDOO\ DSSOLFDEOH WR WKLV VHWWLQJ ,Q DGGLWLRQ ZKLOH WKH ',35& LV ZRUNLQJ RQ WKH TXHVWLRQ WKH FDOOHU LV SUREDEO\ RIWHQ RFFXSLHG ZLWK RWKHU GXWLHV DQG WDVNV ZKLFK OLPLWV WKH HIIHFW RI SURSRVLWLRQ RQH )LQDOO\ WKH FDOOHUV WHQG WR SHUFHLYH WKH VHUYLFH SURYLGHG E\ WKH ',35& DV KLJKO\ YDOXDEOH SURSRVLWLRQ VHYHQf 7KLV LV HYLGHQFHG E\ JHQHUDOO\ KLJK 364 DQG 264 VFRUHV DV ZHOO DV WKH GLVWULEXWLRQ RI UHVSRQVHV WR TXHVWLRQV DQG LQ WKH TXHVWLRQQDLUH ZKLFK DVNHG FDOOHUV WR UDWH WKH KRZ XVHIXO DQG HVVHQWLDO WKH VHUYLFH ZDV WR WKHP $SSUR[LPDWHO\ b RI WKH UHVSRQGHQWV DJUHHG WR VRPH H[WHQW WKDW WKH VHUYLFH ZDV XVHIXO WR WKHP DQG DERXW b RI WKH UHVSRQGHQWV DJUHHG WR VRPH H[WHQW WKDW WKH VHUYLFH ZDV HVVHQWLDO 7KHUHIRUH EDVHG RQ 0DLVWHUfV SURSRVLWLRQV WKHUH DUH FOHDU UHDVRQV QRW WR H[SHFW D VLJQLILFDQW UHODWLRQVKLS EHWZHHQ 364 DQG DFWXDO VHUYLFH WLPH .HHSLQJ ZLWK 0DLVWHUfV SURSRVLWLRQV LW LV LQWHUHVWLQJ WR QRWH WKDW VHUYLFH GHOD\V GLG KDYH D VLJQLILFDQW HIIHFW RQ SHUFHLYHG VHUYLFH TXDOLW\ ZKLFK LV FRQVLVWHQW ZLWK SURSRVLWLRQ ILYH )XUWKHUPRUH WKLV UHVXOW ZDV FRQVLVWHQW ZLWK UHSRUWV IURP 7D\ORU Df 7D\ORU DQG &OD[WRQ f DQG 'XEHf5LRX[ HW DO f 7KH UHVXOWV RI WKLV VWXG\ VXJJHVW WKDW FDOOHUV ZKR H[SHULHQFH GHOD\V LQ VHUYLFH KDYH ORZHU SHUFHSWLRQV RI VHUYLFH TXDOLW\ WKDQ FDOOHUV ZKR GR QRW H[SHULHQFH GHOD\V LQ VHUYLFH ,W VKRXOG EH QRWHG KRZHYHU WKDW WKH UHODWLRQVKLS EHWZHHQ GHOD\V DQG 364 ZDV QRW YHU\ VWURQJ U f 7KH VWUHQJWK RI WKLV UHODWLRQVKLS KRZHYHU LV FRQVLVWHQW ZLWK HIIHFW VL]HV UHSRUWHG E\ 7D\ORU Df ZKR VXJJHVWV WKDW WKHUH DUH YDULDEOHV VXFK DV DQJHU XQFHUWDLQW\ DQG SHUFHLYHG SXQFWXDOLW\f WKDW PHGLDWH WKH UHODWLRQVKLS EHWZHHQ GHOD\V DQG VHUYLFH HYDOXDWLRQV ,Q DGGLWLRQ 'XEHf5LRX[ HW DO f VXJJHVWHG WKDW SHUFHLYHG QHFHVVLW\ PD\ DOVR PHGLDWH WKLV UHODWLRQVKLS $PRQJ WKH WKUHH fWLPH UHODWHGf PHDVXUHV DFWXDO VHUYLFH WLPH VHUYLFH GHOD\V DQG SHUFHLYHG VHUYLFH WLPHf SHUFHLYHG VHUYLFH WLPH KDG WKH VWURQJHVW UHODWLRQVKLS ZLWK 364 VXJJHVWLQJ WKDW FDOOHUVf SHUFHLYHG VHUYLFH WLPHV DUH PRUH LPSRUWDQW LQ GHWHUPLQLQJ WKHLU

PAGE 173

HYDOXDWLRQ RI 364 DQG IRU WKDW PDWWHU 264f WKDQ DFWXDO VHUYLFH WLPH DQG VHUYLFH GHOD\V 7KLV FRQWHQWLRQ LV VXSSRUWHG LQ WKH OLWHUDWXUH E\ 7D\ORU Df .DW] /DUVRQ DQG /DUVRQ f DQG +RUQLN f 7KXV ZKLOH WKLV VWXG\ ZDV RULJLQDOO\ IRFXVHG RQ RSHUDWLRQDO UHODWLRQVKLSV EHWZHHQ DFWXDO VHUYLFH WLPHV DQG VHUYLFH GHOD\V LW PD\ EH WKDW PRUH HPSKDVLV VKRXOG EH SODFHG RQ WKH PDQDJHPHQW RI SHUFHLYHG VHUYLFH WLPHV )URP WKH OLWHUDWXUH ZH FDQ GHULYH VHYHQ UHFRPPHQGDWLRQV WR PDQDJHUV RI GUXJ LQIRUPDWLRQ VHUYLFHV DWWHPSWLQJ WR LPSURYH SHUFHLYHG VHUYLFH WLPHV )LUVW ZKHQ QRWLI\LQJ FDOOHUV RI WKH H[SHFWHG ZDLW GXUDWLRQ LW LV EHWWHU WR RYHUHVWLPDWH WKH ZDLW WKDQ WR XQGHUHVWLPDWH WKH ZDLW KRZHYHU GR QRW SURYLGH HVWLPDWLRQV WKDW DUH ORQJHU WKDQ WKH FRQVXPHU LV ZLOOLQJ WR ZDLW RWKHUZLVH WKH\ PD\ EDON IURP WKH V\VWHP 7KHUHIRUH LW LV LPSRUWDQW WR NQRZ ZKDW FDOOHUV FRQVLGHU DQ DFFHSWDEOH ZDLW 7D\ORU D .DW] /DUVRQ DQG /DUVRQ +RUQLN f 6HFRQG VKRUWHQ SUHSURFHVV ZDLWV LH WKH DPRXQW RI WLPH UHTXLUHG WR WDNH LQIRUPDWLRQf DV PXFK DV SRVVLEOH .DW] /DUVRQ DQG /DUVRQ 'XEHf5LRX[ HW DO f 7KLUG GLVWUDFW DWWHQWLRQ IURP WKH GXUDWLRQ RI WKH ZDLW .DW] /DUVRQ DQG /DUVRQ f 7KLV FDQ EH GRQH IRU TXHVWLRQV WKDW DUH WDNLQJ DQ LQRUGLQDWH DPRXQW RI WLPH WR UHVHDUFK E\ XSGDWLQJ WKH FDOOHU ZLWK WKH FXUUHQW SURJUHVV )RXUWK ZKHQ VHUYLFH LV GHOD\HG SDVW WKH HVWLPDWHG WLPH LW LV LPSRUWDQW WR DSRORJL]H IRU WKH IDLOXUH EXW LW LV HTXDOO\ LPSRUWDQW WR VRXQG VLQFHUH VLQFH LQVLQFHUH DSRORJLHV FDQ DFWXDOO\ ORZHU VDWLVIDFWLRQ &OHPPHU DQG 6FKQLHGHU f )LIWK ZKHQ VHUYLFH LV GHOD\HG SDVW WKH HVWLPDWHG WLPH LW LV RIWHQ LPSRUWDQW WR H[SODLQ WR WKH FDOOHU ZK\ WKH GHOD\ RFFXUUHG RU WKH\ PLJKW LQIHU WKHLU RZQ UHDVRQV IRU WKH GHOD\ 7D\ORU Df 6L[WK DVVHVV FDOOHU DWWLWXGHV HYDOXDWH WKHLU WLPH SUHVVXUHV DQG KDQGOH ILUVW WKRVH FDOOHUV ZLWK WKH JUHDWHVW SHUFHLYHG QHHG .DW] /DUVRQ DQG /DUVRQ 'XEHf5LRX[ HW DO f 6HYHQWK SURYLGH FDOOHUV LQIRUPDWLRQ UHJDUGLQJ SHDN GHPDQG WLPHV VR WKDW WKH\ NQRZ ZKHQ WR H[SHFW ORQJHU ZDLWV .DW] /DUVRQ DQG /DUVRQ &OHPPHU DQG 6FKHLGHU f

PAGE 174

6HUYLFH 6\VWHP 6LPXODWLRQ 7KH '35& DW WKH 8QLYHUVLW\ RI )ORULGD ZDV VLPXODWHG XVLQJ WKH *366+ VLPXODWLRQ SURJUDP 7KH VLPXODWLRQ ZDV YHULILHG XVLQJ WKUHH PHWKRGV f WUDFHV RI WKH VLPXODWLRQ SURJUDP FRGH XVLQJ WKH LQWHUDFWLYH GHEXJJHU f WHVWV RI ORJLF UHODWLRQVKLSV LH PRGHO EHKDYLRUf DQG f D OLPLWHG IRUP RI WKH VLPXODWLRQ ZDV FRPSDUHG WR DQDO\WLFDO UHVXOWV EDVHG RQ DQ 00 TXHXH ,Q DGGLWLRQ WKH VLPXODWLRQ ZDV YDOLGDWHG XVLQJ WKUHH PHWKRGV f IDFH YDOLGLW\ ZDV HVWDEOLVKHG E\ VLPXODWLRQ ZDONWKURXJKV ZLWK ',35& FRn GLUHFWRUV f H[SHFWHG VLPXODWLRQ EHKDYLRU ZDV HYDOXDWHG XVLQJ H[WUHPH FRQGLWLRQ WHVWV DQG f WKH UHVXOWV RI WKH VLPXODWLRQ ZHUH FRPSDUHG WR GDWD FROOHFWHG IURP WKH DFWXDO V\VWHP 7KH VLPXODWLRQ SDVVHG DOO VWHSV RI WKH YHULILFDWLRQ DQG YDOLGDWLRQ SURFHVV ,W ZDV WKHUHIRUH GHWHUPLQHG WKDW WKH VLPXODWLRQ ZDV DQ DFFHSWDEOH GHFLVLRQ PDNLQJ WRRO IRU XVH LQ WKH ',53& 7KH VLPXODWLRQ ZDV XVHG WR HYDOXDWH WKUHH VSHFLILF UHVHDUFK TXHVWLRQV 7KH ILUVW TXHVWLRQ DVNHG KRZ FKDQJHV LQ VWDIILQJ OHYHOV DQG VHUYLFH UDWHV DIIHFW WRWDO VHUYLFH WLPHV DQG GHOD\V LQ VHUYLFH 7KHVH FKDQJHV ZHUH FRPSDUHG WR WKH fQRUPDOf OHYHO RI RSHUDWLRQ LH VLPXODWLRQ UXQV XVLQJ REVHUYHG DUULYDO DQG VHUYLFH UDWHV DQG D VWDIILQJ OHYHO RI WKUHH VWXGHQWVf ,Q WHUPV RI VWDIILQJ OHYHOV WKH VLPXODWLRQ UHVXOWV VXJJHVW WKDW D VWDIILQJ OHYHO RI OHVV WKDQ WKUHH VWXGHQWV LV VLPSO\ LQDGHTXDWH WR KDQGOH WKH FXUUHQW TXHVWLRQ YROXPH LQ WKH ',35& +RZHYHU WKHUH VHHPHG WR EH VLJQLILFDQW LPSURYHPHQWV LQ VLPXODWHG WRWDO VHUYLFH WLPHV DQG VHUYLFH GHOD\V ZKHQ WKH VWDIILQJ OHYHO ZDV LQFUHDVHG IURP WKUHH WR IRXU VWXGHQWV &RQYHUVHO\ WKH GLIIHUHQFH LQ VHUYLFH GHOD\V DQG WRWDO VHUYLFH WLPHV EHWZHHQ IRXU DQG ILYH VWXGHQWV ZHUH QRW GUDPDWLF LQGLFDWLQJ WKDW WKUHH RU IRXU VWXGHQWV LV SUREDEO\ DQ DGHTXDWH VWDIILQJ OHYHO IRU WKH ',35& &KDQJHV LQ VHUYLFH UDWHV ZHUH FRQGXFWHG E\ PDQLSXODWLQJ WKH UHVHDUFK DQG DSSURYDO WLPHV SURJUDPPHG LQWR WKH VLPXODWLRQ FRGH EDVHG RQ WKH REVHUYHG GDWD 7KH VLPXODWLRQ UHVXOWV LQGLFDWHG WKDW DV H[SHFWHG LQFUHDVHV LQ UHVHDUFK DQG DSSURYDO WLPHV

PAGE 175

LQFUHDVHG ERWK WRWDO VHUYLFH WLPHV DQG VHUYLFH GHOD\V ZKLOH GHFUHDVHV LQ UHVHDUFK DQG DSSURYDO WLPHV GHFUHDVHG ERWK WRWDO VHUYLFH WLPHV DQG VHUYLFH GHOD\V +RZHYHU ZKHQ FRPSDULQJ UHGXFWLRQV LQ WRWDO VHUYLFH WLPH DQG GHOD\ SHUFHQWDJH WR UHGXFWLRQV LQ SHUFHQW XWLOL]DWLRQ LW DSSHDUHG WKDW VHUYLFH UDWH LPSURYHPHQWV ZHUH PRUH HIILFLHQW WKDQ DGGLQJ VHUYHUV IRU UHGXFLQJ WRWDO VHUYLFH WLPHV DQG VHUYLFH GHOD\V 7KH VHFRQG TXHVWLRQ DVNHG ZKDW FRPELQDWLRQ RI FKDQJHV LQ VWDIILQJ OHYHOV DQG VHUYLFH UDWHV RSWLPL]HV WKH V\VWHP IRU GHOD\V LQ VHUYLFH DQG WRWDO VHUYLFH WLPH /DUJHO\ WKH UHVXOWV RI WKH HIIHFWLYHQHVV UDWLRV PLUURUHG ZKDW ZDV DQWLFLSDWHG IURP WKH UHVXOWV RI WKH ILUVW UHVHDUFK TXHVWLRQ 7KH UHVXOWV RI WKH VLPXODWLRQ LQGLFDWHG WKDW ZKHQ WU\LQJ WR UHGXFH WKH SHUFHQWDJH RI VHUYLFH GHOD\V UHGXFWLRQV LQ VHUYLFH UDWHV ZHUH PRUH LPSRUWDQW WKDQ WKH DGGLWLRQ RI VHUYHUV 7KLV UHVXOW ZDV FRQVLVWHQW ZLWK UHVHDUFK IURP &DUUXWKHUV f &KLQ DQG 6SUHFKHU f .XPDU DQG .DSXU f DQG 2]HNL DQG ,NHXFKL f ,QWHUHVWLQJO\ KRZHYHU IRU UHGXFWLRQV LQ WRWDO VHUYLFH WLPH DSSURSULDWH VWDIILQJ OHYHOV ZHUH DW OHDVW DV LPSRUWDQW DV UHGXFWLRQV LQ WKH WLPH WR UHVHDUFK DQG DSSURYH TXHVWLRQV ZKLFK ZDV FRQVLVWHQW ZLWK UHVHDUFK GRQH E\ /DP\ HW DO f )RU UHGXFLQJ VHUYLFH GHOD\V WKH RSWLPDO VWDIILQJ OHYHO ZDV WKUHH VWXGHQWV DQG IRU UHGXFLQJ WRWDO VHUYLFH WLPHV WKH RSWLPDO VWDIILQJ OHYHO ZDV IRXU VWXGHQWV +RZHYHU LQ ERWK FDVHV LPSURYLQJ VHUYLFH UDWHV DOZD\V LPSURYHG HIILFLHQF\ 7KH WKLUG TXHVWLRQ DVNHG KRZ VHQVLWLYH WKH RSWLPDO VROXWLRQ ZDV WR FKDQJHV LQ WKH DUULYDO UDWHV 6LQFH IOXFWXDWLRQV LQ WKH UDWH RI DUULYDOV FDQ GUDPDWLFDOO\ DIIHFW PDQ\ TXHXLQJ V\VWHPV 0F&ODLQ DQG 7KRPDV f LW ZDV LPSRUWDQW WR LGHQWLI\ WKUHVKROGV DW ZKLFK WKH RSWLPDO VHUYLFH FDSDFLW\ GHWHUPLQHG E\ TXHVWLRQ WZR DERYH ZHUH DIIHFWHG 7KH RSWLPXP VROXWLRQV WHQGHG WR EH UHODWLYHO\ VWDEOH &DOO YROXPH ZRXOG KDYH WR LQFUHDVH E\ b WR b RU GHFUHDVH E\ b WR b IRU WKH VROXWLRQ WR FKDQJH ,Q SUDFWLFDO WHUPV WKHVH UHVXOWV LQGLFDWHG WKDW DQ LQFUHDVH LQ YROXPH RI DSSUR[LPDWHO\ WR FDOOV SHU PRQWK LH DYHUDJH RI FDOOV SHU PRQWK r f ZRXOG UHIOHFW D QHHG WR DGG DQ DGGLWLRQDO

PAGE 176

VWXGHQW $ GHFUHDVH LQ FDOO YROXPH RI DSSUR[LPDWHO\ WR FDOOV SHU PRQWK LH r f ZRXOG DOORZ IRU D GHFUHDVH LQ WKH QXPEHU RI VWXGHQWV VWDIILQJ WKH FHQWHU ,Q D GHEULHILQJ VHVVLRQ ZLWK WKH FRGLUHFWRUV DQG WKH GUXJ LQIRUPDWLRQ UHVLGHQW IRXU SRWHQWLDO PHWKRGV IRU UHGXFLQJ VHUYLFH WLPHV ZHUH VXJJHVWHG 7KH ILUVW PHWKRG LQYROYHG WKH XVH RI DQ RQOLQH GDWDEDVH WR VWRUH FDOOHU FRQWDFW LQIRUPDWLRQ DORQJ ZLWK DQVZHUV WR TXHVWLRQV SRVHG 7KLV LPSURYHPHQW ZDV GHHPHG WR KDYH WKH PRVW SRWHQWLDO IRU LPSURYLQJ WKH UHVSRQVH WLPH RI WKH ',35& )XUWKHUPRUH WKLV GDWDEDVH KDG DOUHDG\ EHHQ VODWHG IRU LPSOHPHQWDWLRQ DQG ZDV EHJLQQLQJ WKH LQLWLDO VWDJHV RI WKH GHVLJQ SURFHVV DW WKH WLPH RI WKLV SURMHFW 7KLV GDWDEDVH LV H[SHFWHG WR UHGXFH WKH DPRXQW RI WLPH UHTXLUHG WDNH FDOOV VLQFH UHSHDW FDOOHUV ZLOO SUHVXPDEO\ DOUHDG\ KDYH WKHLU LQIRUPDWLRQ HQWHUHG LQWR WKH GDWDEDVH $OVR VLQFH PDQ\ TXHVWLRQV WHQG WR EH UHSHDWHG RYHU WKH FRXUVH RI WLPH SUHYLRXV UHVSRQVHV FDQ EH UHWULHYHG DQG UHXVHG ZKLFK FRXOG VXEVWDQWLDOO\ GHFUHDVH UHVHDUFK WLPHV IRU PDQ\ TXHVWLRQV 7KH VHFRQG PHWKRG LQYROYHG WKH HQFRXUDJHPHQW RI VWXGHQWV WR XVH WKH SDJHU V\VWHP WR FRQWDFW FRGLUHFWRUV WR DSSURYH UHVSRQVHV ZKHQ WKH\ DUH RXW RI WKH RIILFH $OWKRXJK WKHUH LV XVXDOO\ VRPHRQH SUHVHQW LQ WKH ',35& ZKR FDQ DSSURYH UHVSRQVHV RFFDVLRQV DULVH ZKHQ DOO SHUVRQV HOLJLEOH WR DSSURYH DQVZHUV DUH RXW 0RUH HIIHFWLYH XVH RI WKH SDJHU V\VWHP ZLOO SUHYHQW KLJK SULRULW\ TXHVWLRQV HJ fVWDWf DQG fWRGD\ff IURP EHLQJ GHOD\HG EHFDXVH WKH VWXGHQW LV ZDLWLQJ IRU DSSURYDO 7KH WKLUG PHWKRG SURSRVHG GXULQJ WKLV GHEULHILQJ ZRXOG EH WR GHYHORS D KLHUDUFK\ RI DSSURYDOV VR WKDW IRU FHUWDLQ W\SHV RI TXHVWLRQV VXFK DV SKRQH QXPEHUV IRU GUXJ FRPSDQLHV DQG GUXJ LGHQWLILFDWLRQ TXHVWLRQVf VWXGHQWV PD\ JLYH D UHVSRQVH ZLWKRXW SULRU DSSURYDO 7KH IRXUWK PHWKRG VXJJHVWHG FRQFHUQHG WKH DGGLWLRQ RI VHYHUDO FRPSXWHUV WKDW ZRXOG DOORZ VWXGHQWV PRUH LPPHGLDWH DFFHVV WR ZRUGSURFHVVLQJ DQG RQOLQH LQIRUPDWLRQ UHVRXUFHV ZLWKRXW KDYLQJ WR ZDLW IRU RWKHU VWXGHQWV WR ILQLVK ,W ZDV DOVR QRWHG WKDW RQFH WKH RQOLQH GDWDEDVH KDV EHHQ LPSOHPHQWHG LW ZLOO EH FUXFLDO IRU HYHU\ VWXGHQW WR KDYH DFFHVV WR D FRPSXWHU 7KH RYHUULGLQJ TXHVWLRQ WKDW UHPDLQV LV WKH LVVXH RI ZKHWKHU RU QRW VLPXODWLRQ LV XVHIXO DV D WRRO IRU XQGHUVWDQGLQJ FRPSOH[ VHUYLFH V\VWHPV VXFK DV GUXJ LQIRUPDWLRQ DQG

PAGE 177

RWKHU SKDUPDF\ V\VWHPV 7KH DQVZHU WR WKLV TXHVWLRQ VHHPV WR EH D TXDOLILHG \HV 2QFH WKH VLPXODWLRQ IRU WKLV VWXG\ ZDV FRQVWUXFWHG YHULILHG DQG YDOLGDWHG WKH VLPXODWLRQ SURYLGHG D UHODWLYHO\ HDV\ PHDQV RI H[SORULQJ WKH HIIHFW RI VHUYLFH FDSDFLW\ FKDQJHV RQ YDULRXV TXHXH VWDWLVWLFV LH : :T / /T S DQG GHOD\Vf &OHDUO\ WKLV DWWHPSW DW VLPXODWLQJ D GUXJ LQIRUPDWLRQ VHUYLFH KDV GHPRQVWUDWHG WKH XWLOLW\ RI VLPXODWLRQ DQG ZLWK WKH FXUUHQW KHDOWK FDUH HQYLURQPHQW SXVKLQJ WRZDUGV FRVW FRQWDLQPHQW UHVRXUFH HIILFLHQF\ DQG WRWDO TXDOLW\ PDQDJHPHQW WKHUH VHHPV WR EH D QHHG IRU WRROV WKDW FDQ KHOS GHFLVLRQ PDNHUV HYDOXDWH DQG GHVLJQ RU UHGHVLJQf FRPSOH[ VHUYLFH V\VWHPV ,W PXVW EH FRQVLGHUHG KRZHYHU WKDW PRQWKV RI EDFNJURXQG UHVHDUFK DQG GHYHORSPHQW ZHQW LQWR WKH FRQVWUXFWLRQ YHULILFDWLRQ DQG YDOLGDWLRQ RI WKH VLPXODWLRQ XVHG LQ WKLV SURMHFW $V 6FKULEHU f VWDWHV f$ VLPXODWLRQ VKRXOG EH VWDUWHG ZHOO EHIRUH WKH UHVXOWV DUH QHHGHG ,Q SUDFWLFH XQIRUWXQDWHO\ WKH UHVXOWV RI VLPXODWLRQ DUH XVXDOO\ QHHGHG f\HVWHUGD\ff Sf 8QIRUWXQDWHO\ WKH OHQJWK RI WLPH QHFHVVDU\ WR GHYHORS JRRG VLPXODWLRQ PRGHOV PD\ QRW DOZD\V EH DSSURSULDWH WKHUHIRUH VLPXODWLRQ PD\ QRW DOZD\V EH WKH PRVW DSSURSULDWH WRRO IRU V\VWHPV DQDO\VLV &RQFOXVLRQV )RXU FRQFOXVLRQV ZHUH GHULYHG IURP WKH UHVXOWV RI WKLV VWXG\ )LUVW EDVHG RQ UHVXOWV IURP WKLV UHVHDUFK DQG UHVHDUFK GRQH E\ RWKHUV WKHUH DSSHDUV WR EH D FOHDU UHODWLRQVKLS EHWZHHQ HYDOXDWLRQV RI SHUFHLYHG VHUYLFH TXDOLW\ DQG HYDOXDWLRQV RI RYHUDOO VHUYLFH TXDOLW\ KRZHYHU VFDOHV EDVHG RQ GHULYDWLYHV RI WKH 6(5948$/ VFDOH VXFK DV 6(593(5) DQG WKH 364 LQVWUXPHQW XVHG LQ WKLV VWXG\f VHHP OLPLWHG LQ WKHLU DELOLW\ WR FDSWXUH SHUFHSWLRQV RI TXDOLW\ EH\RQG IXQFWLRQDO TXDOLW\ 6HFRQG SHUFHSWLRQV RI GUXJ LQIRUPDWLRQ VHUYLFH TXDOLW\ DUH UHODWHG WR WKH FDOOHUVf EHKDYLRUDO LQWHQWLRQV VXJJHVWLQJ WKDW ZKHQ FDOOHUV IHHO WKDW WKH VHUYLFH GHOLYHUV KLJK

PAGE 178

TXDOLW\ VHUYLFH WKH\ ZLOO QRW RQO\ XVH WKH VHUYLFH DJDLQ EXW ZLOO DOVR UHFRPPHQG WKH VHUYLFH WR D FROOHDJXH ZKHQ JLYHQ WKH RSSRUWXQLW\ 7KLUG GUXJ LQIRUPDWLRQ VHUYLFH SURYLGHUV VKRXOG EH FRQFHUQHG ZLWK ERWK GHOD\V LQ VHUYLFH DQG SHUFHLYHG ZDLW WLPHV VLQFH ERWK DSSHDU WR KDYH DQ DIIHFW RQ 364 $FWXDO ZDLW WLPHV GR QRW VHHP WR EH UHODWHG WR 364 7KHUHIRUH PDQDJHUV VKRXOG XVH RSHUDWLRQV PDQDJHPHQW DSSURDFKHV WR UHGXFH WKH IUHTXHQF\ RI VHUYLFH GHOD\V LH LQFUHDVH WKH VHUYLFH WLPH DQGRU DGG VHUYHUVf DQG WKH\ VKRXOG XVH SHUFHSWLRQV PDQDJHPHQW DSSURDFKHV HJ H[SHFWHG ZDLW LQIRUPDWLRQ DSRORJLHV HWFf WR LPSURYH WKH H[SHULHQFH RI ZDLWLQJ )RXUWK EDVHG RQ WKH VLPXODWLRQ UHVXOWV LW DSSHDUV WKDW IRU WKLV VHWWLQJ UHGXFLQJ WKH VHUYLFH UDWHV LV D PRUH HIILFLHQW PHWKRG RI LPSURYLQJ WKH SHUFHQWDJH RI GHOD\V WKDQ LQFUHDVLQJ WKH QXPEHU RI VHUYHUV +RZHYHU LI WRWDO VHUYLFH WLPH ZHUH WR EH FRQVLGHUHG DQ LPSRUWDQW LQGLFDWRU WKHQ LQFUHDVHV LQ VWDIILQJ EHFRPH DW OHDVW DV LPSRUWDQW DV UHGXFWLRQV LQ UHVHDUFK DQG DSSURYDO WLPHV /LPLWDWLRQV RI 6WXG\ 7KHUH ZHUH IRXU VWXG\ OLPLWDWLRQV WKDW GHVHUYH FRQVLGHUDWLRQ )LUVW WKH GHVLJQ RI WKH TXHVWLRQQDLUH SRUWLRQ RI WKLV SURMHFW ZDV H[SORUDWRU\ DQG QRQH[SHULPHQWDO ZKLFK PDNHV WKH LQWHUQDO YDOLGLW\ RI WKH UHVXOWV GLIILFXOW WR GHWHUPLQH ,Q DGGLWLRQ RQO\ RQH PHDVXUHPHQW RI SHUFHLYHG VHUYLFH TXDOLW\ RYHUDOO VHUYLFH TXDOLW\ SHUFHLYHG VHUYLFH WLPH DQG EHKDYLRUDO LQWHQWLRQ ZDV PDGH IRU HDFK VXEMHFW 7KHUHIRUH VLQFH XQPHDVXUHG H[WHUQDO YDULDEOHV FRXOG KDYH PHGLDWHG RU LQWHUYHQHG LQ WKH DFWXDO UHODWLRQVKLSV WKH UHVXOWV UHSRUWHG LQ WKLV VWXG\ FRXOG EH VSXULRXV +RZHYHU WKH IDFW WKDW WKH UHODWLRQVKLSV REVHUYHG ZHUH FRQVLVWHQW ZLWK WKRVH UHSRUWHG E\ &URQLQ DQG 7D\ORU f DQG RWKHU UHVHDUFKHUV KHOSV WR FRQILUP WKH GLUHFWLRQ DQG VWUHQJWK RI WKH REVHUYHG UHODWLRQVKLSV ,Q DGGLWLRQ VLQFH WKH UHVHDUFK GHVLJQ ZDV QRW H[SHULPHQWDO RU TXDVLH[SHULPHQWDOf WKH H[LVWHQFH DQG GLUHFWLRQ RI FDXVDOLW\ FRXOG QRW EH DVVXPHG

PAGE 179

6HFRQG EHFDXVH RI WLPH DQG FRVW FRQVWUDLQWV WKH VWXG\ GDWD ZHUH QRW GUDZQ IURP D UDQGRP VDPSOH RI FDOOHUV 7KHUHIRUH LW LV SRVVLEOH WKDW VRPH RI WKH UHVXOWV PD\ KDYH RFFXUUHG GXH WR VDPSOH LGLRV\QFUDVLHV +RZHYHU WKH FKDUDFWHULVWLFV RI WKH FRQFXUUHQW VDPSOH ZHUH VLPLODU WR WKH FKDUDFWHULVWLFV RI REVHUYHG IURP WKH KLVWRULFDO VDPSOH $OVR WKHUH ZDV QR UHDVRQ WR VXVSHFW WKDW WKH UHVSRQVHV JLYHQ RQ WKH TXHVWLRQQDLUHV ZHUH GHSHQGHQW RQ RQH DQRWKHU 7KHUHIRUH WKHUH ZDV QR UHDVRQ WR VXVSHFW WKDW GLIIHUHQW UHVXOWV ZRXOG KDYH EHHQ UHDFKHG XVLQJ D WUXO\ UDQGRP VDPSOH 7KLUG LW ZDV QRW SRVVLEOH WR XVH WKLV VLPXODWLRQ WR DQDO\WLFDOO\ VROYH IRU WKH fWUXHf RSWLPXP VWDWH DV LV SRVVLEOH ZLWK RWKHU DQDO\WLFDO WRROV XVHG LQ RSHUDWLRQV UHVHDUFK VXFK DV OLQHDU SURJUDPPLQJf 7KHUHIRUH WKH RSWLPXP VROXWLRQ SUHVHQWHG LQ WKLV VWXG\ LV RQO\ WKH EHVW FKRLFH VHOHFWHG IURP WKH VLPXODWHG H[SHULPHQWV ,I RWKHU YDULDEOHV RI LQWHUHVW ZHUH WR EH VWXGLHG DQGRU GLIIHUHQW LQSXWV ZHUH JLYHQ WR WKH PRGHO WKHQ WKH UHVXOWV PD\ GLIIHU )XUWKHUPRUH VLQFH DFWXDO PDQLSXODWLRQV RI WKH V\VWHP ZHUH QRW FRQGXFWHG LW LV QRW SRVVLEOH WR NQRZ IRU VXUH ZKHWKHU RU QRW WKH UHVXOWV RI WKH VLPXODWLRQ DUH WUXO\ DFFXUDWH ZKHQ DFWXDO FKDQJHV DUH PDGH WR V\VWHP LQSXWV HJ VWDIILQJ OHYHO VHUYLFH UDWHV DQG DUULYDO UDWHVf )RXUW K WKH VLPXODWLRQ SURJUDP LWVHOI PD\ QRW EH JHQHUDOL]HDEOH WR RWKHU GUXJ LQIRUPDWLRQ VHWWLQJV JLYHQ WKDW RWKHU FHQWHUV PD\ KDYH GLIIHUHQW ZRUN SDWWHUQV GLIIHUHQW FDOOLQJ SRSXODWLRQV DQG YDU\LQJ UHVSRQVLELOLWLHV &RQVHTXHQWO\ WKH VSHFLILF UHVXOWV LH TXHXH VWDWLVWLFVf JHQHUDWHG E\ WKH VLPXODWLRQ PRGHO FUHDWHG LQ WKLV SURMHFW DUH QRW QHFHVVDULO\ JHQHUDOL]DEOH WR RWKHU GUXJ LQIRUPDWLRQ VHUYLFHV $OVR VLQFH WKLV VWXG\ LV WKH ILUVW WR XVH VLPXODWLRQ WR H[DPLQH WKH HIIHFWLYHQHVV RI GUXJ LQIRUPDWLRQ VHUYLFHV WKHUH DUH QR VWXGLHV DYDLODEOH WR FRPSDUH WKH VLPXODWHG UHVXOWV

PAGE 180

5HFRPPHQGDWLRQV IRU )XWXUH 6WXGLHV 8VH RI 6LPXODWLRQ LQ 2WKHU 3KDUPDF\ 6HWWLQJV 2QH RI WKH SULPDU\ JRDOV RI WKLV VWXG\ ZDV WR GHWHUPLQH LI VLPXODWLRQ FRXOG HIIHFWLYHO\ EH XVHG DV D GHFLVLRQ PDNLQJ WRRO LQ SKDUPDF\ V\VWHPV :KLOH WKLV VWXG\ KDV PDGH D VLJQLILFDQW VWHS LQ WKDW GLUHFWLRQ WKHUH DUH VWLOO TXHVWLRQV WR EH DQVZHUHG 2QH RI WKH LQLWLDO UHDVRQV WKDW WKH GUXJ LQIRUPDWLRQ VHUYLFH VHWWLQJ ZDV VHOHFWHG ZDV EHFDXVH RI WKH UHODWLYH VLPSOLFLW\ RI WKLV V\VWHP YHUVXV RWKHU SKDUPDF\ V\VWHPV 7KLV VWXG\ LQGLFDWHG WKDW VLPXODWLRQ ZDV D SRZHUIXO GHFLVLRQ PDNLQJ WRRO IRU DQDO\]LQJ VHUYLFH V\VWHPV KRZHYHU LW WDNHV FRQVLGHUDEOH WLPH DQG HIIRUW WR DGHTXDWHO\ FRQVWUXFW YHULI\ DQG YDOLGDWH D VLPXODWLRQ 7KHUHIRUH LWV XVHIXOQHVV LQ PRUH FRPSOH[ V\VWHPV LV VWLOO DW LVVXH ,Q WKH UHVHDUFKHUfV RSLQLRQ WKHUH DUH DW OHDVW WKUHH SKDUPDF\ VHWWLQJV ZKHUH WKLV VLPXODWLRQ PHWKRGRORJ\ FRXOG EH LPSOHPHQWHG QH[W ZLWK LPPHGLDWH LPSDFW LQ WKH OLWHUDWXUH )LUVW DQ LQWHUHVWLQJ XVH RI VLPXODWLRQ ZRXOG EH WR VWXG\ WKH HIIHFWV RI ZRUNORDG RQ PHGLFDWLRQ HUURU UDWHV LQ D KRVSLWDO RU FRPPXQLW\ SKDUPDF\ 7KLV UHVHDUFK PLJKW DQVZHU VHYHUDO TXHVWLRQV :KDW LV WKH ZRUNORDGVWDIILQJ EDODQFH LQ RUGHU WR PLQLPL]H HUURU UDWHV" +RZ GR WKH FRQWULEXWLQJ IDFWRUV RI HUURUV HJ ZRUNORDG ZRUNIORZ IDWLJXH DQG KXPDQ IDFWRUVf LQWHUUHODWH" &DQ VWDQGDUGV EH VHW IRU ZRUNORDG OLPLWV" 6HFRQG RQH RI WKH PRVW IUHTXHQWO\ PHQWLRQHG EDUULHUV WR WKH LPSOHPHQWDWLRQ RI SKDUPDFHXWLFDO FDUH LQ FRPPXQLW\ SKDUPDFLHV LV ODFN RI VXIILFLHQW VWDII DQGRU WLPH WR SURYLGH VHUYLFHV 6LPXODWLRQ FRXOG KHOS SURYLGH SUDFWLWLRQHUV ZLWK UHDOLVWLF H[SHFWDWLRQV DV WR KRZ PDQ\ SKDUPDFLVWVWHFKQLFLDQV DUH QHHGHG WR SURYLGH SKDUPDFHXWLFDO FDUH VHUYLFHV KRZZKHQ SDWLHQWV FRXOG EH VFKHGXOHG DQG KRZ FKDQJHV LQ ZRUNIORZ FRXOG LPSURYH HIILFLHQF\ RI WKH FXUUHQW VWDII 7KLUG LQ SUHYLRXVO\ SXEOLVKHG UHVHDUFK E\ WKH UHVHDUFKHU +DOEHUJ HW DO f LQYROYLQJ WKH XVH RI DQHVWKHWLFV LQ DQ DPEXODWRU\ FDUH VHWWLQJ WKHUH ZHUH GLIIHUHQFHV

PAGE 181

GLVFRYHUHG DPRQJ DQHVWKHWLFV LQ WHUPV RI ERWK WLPH DQG FRVW IDFWRUV +RZHYHU WZR OLPLWDWLRQV RI WKLV VWXG\ ZHUH f LW GLG QRW DFFRXQW IRU ODERU FRVWV RI QXUVLQJ VWDII DQG DQHVWKHVLRORJLVW WLPH EHFDXVH WKH GLIILFXOW\ RI WUDFNLQJ WKLV LQIRUPDWLRQ f VPDOO VDPSOH VL]HV UHVWULFWHG WKH SRZHU RI WKH VWDWLVWLFDO WHVWV XVHG WR GHWHFW GLIIHUHQFHV LQ DQHVWKHWLFV ,I WKH V\VWHP RI VXUJHU\ FRXOG EH VLPXODWHG ZLWK VRPH DFFXUDF\ WKHQ PRUH SUHFLVH SKDUPDFRHFRQRPLF MXGJHPHQWV FRXOG EH PDGH FRQFHUQLQJ WKH XVH RI DQHVWKHWLFV 7KLV PHWKRG FRXOG DOVR H[WHQG LWVHOI HYHQWXDOO\ WR RWKHU SKDUPDFRHFRQRPLF FKRLFHV ZKHUH VDPSOH VL]HV DUH ORZ RU ZKHQ FRVWV RI FROOHFWLQJ GDWD DUH UHVWULFWLYH LQ WHUPV RI GROODUV RU WLPH $VVHVVPHQW RI WKH 6HUYLFH 4XDOLW\ RI 'UXJ ,QIRUPDWLRQ 6HUYLFHV $QRWKHU JRDO RI WKLV VWXG\ ZDV WR HVWDEOLVK WKH UHODWLRQVKLSV EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ DQG DFWXDO VHUYLFH WLPH RU GHOD\V LQ VHUYLFH 8QIRUWXQDWHO\ WKH UHODWLRQVKLSV REVHUYHG ZHUH ZHDNHU WKDQ H[SHFWHG ,QWHUHVWLQJO\ KRZHYHU SHUFHLYHG VHUYLFH WLPH ZDV UHODWHG WR SHUFHLYHG VHUYLFH TXDOLW\ ZLWK VRPH VLJQLILFDQFH $OWKRXJK VWURQJ UHODWLRQVKLSV PD\ QRW DFWXDOO\ H[LVW EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ DQG DFWXDO VHUYLFH WLPH RU GHOD\V LQ VHUYLFH LQ WKH GUXJ LQIRUPDWLRQ VHUYLFH VHWWLQJ WKHUH LV VWLOO FRQVLGHUDEOH URRP IRU UHVHDUFK LQ WKLV DUHD HVSHFLDOO\ LQ SKDUPDF\ V\VWHPV VXFK DV FRPPXQLW\ SKDUPDF\f ZKHUH OD\ SHUVRQV UDWKHU WKDQ SURIHVVLRQDOV DUH WKH SULPDU\ FRQVXPHUV )LUVW LW PD\ EH WKDW WKH 6(5948$/ DQG 6(593(5) LQVWUXPHQWV GHYHORSHG E\ 3DUDVXUDPDQ HW DO f DQG &URQLQ DQG 7D\ORU f DUH QRW DGHTXDWH IRU PHDVXULQJ WKH SHUFHLYHG VHUYLFH TXDOLW\ IRU SURIHVVLRQDO VHUYLFHV VXFK DV D GUXJ LQIRUPDWLRQ VHUYLFH 7KH GLVWLQFWLRQ EHWZHHQ FRQVXPHUV DQG SURIHVVLRQDOV ZLWK UHJDUG WR SHUFHLYHG VHUYLFH TXDOLW\ DQG WKH GHYHORSPHQW RI D PRUH XVHIXO VFDOH IRU XVH LQ DVVHVVLQJ WKH SHUFHLYHG

PAGE 182

VHUYLFH TXDOLW\ RI SURIHVVLRQDO VHUYLFHV ZRXOG ILOO D JDS LQ WKH HYROYLQJ VHUYLFH TXDOLW\ OLWHUDWXUH 6HFRQG RQH RI WKH WKHRUHWLFDO GLVWLQFWLRQV EHWZHHQ SHUFHLYHG VHUYLFH TXDOLW\ DQG FRQVXPHU VDWLVIDFWLRQ LV WKDW FXVWRPHU VDWLVIDFWLRQ LV W\SLFDOO\ FRQFHUQHG ZLWK RQO\ D VLQJOH VHUYLFH HQFRXQWHU DQG SHUFHLYHG VHUYLFH TXDOLW\ LV UHIOHFWLYH RI DOO H[SHULHQFHV DQG LPSUHVVLRQV D FRQVXPHU PD\ KROG ,W PD\ EH WKDW FRQVXPHU VDWLVIDFWLRQGLVVDWLVIDFWLRQ &6'f PHDVXUHV EHKDYH GLIIHUHQWO\ WKDQ SHUFHLYHG VHUYLFH TXDOLW\ PHDVXUHV ZLWK UHVSHFW WR DFWXDO VHUYLFH WLPH DQG VHUYLFH GHOD\V 7KHUHIRUH LW PD\ EH PRUH DSSURSULDWH WR UHODWH DFWXDO VHUYLFH WLPH WR VRPH PHDVXUH RI FXVWRPHU VDWLVIDFWLRQ UDWKHU WKDQ SHUFHLYHG VHUYLFH TXDOLW\ %H\RQG &6' RWKHU IUDPHZRUNV VXFK DV SHUFHLYHG YDOXH PD\ REWDLQ PRUH XVHIXO UHVXOWV IRU FRPELQLQJ DVSHFWV RI RSHUDWLRQV UHVHDUFK ZLWK SHUFHSWLRQV PDQDJHPHQW 7KLUG EH\RQG WKH ZRUN GRQH E\ +RUQLN DQG f DQG .DW] /DUVRQ DQG /DUVRQ f LW VHHPV WKDW YHU\ OLWWOH UHVHDUFK KDV EHHQ GRQH WR DVFHUWDLQ WKH UHODWLRQVKLSV DPRQJ SHUFHLYHG VHUYLFH WLPH DQG DFWXDO VHUYLFH WLPHV DV LW DSSOLHV WR VHUYLFH VHWWLQJV ,I WKLV UHODWLRQVKLS FDQ EH PRUH DFFXUDWHO\ GHVFULEHG WKHQ LW PD\ FUHDWH DGGLWLRQDO DYHQXHV IRU PDQDJHUV ZKR GHVLUH WR LPSURYH TXDOLW\ XVLQJ RSHUDWLRQV PDQDJHPHQW WHFKQLTXHV

PAGE 183

$33(1',; $ 7(;7 2) 35(7(67 &29(5 /(77(5 -XQH 'HDU &ROOHDJXH 7KH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU $3',&f DW 8$06 &ROOHJH RI 3KDUPDF\ LV FXUUHQWO\ ZRUNLQJ WRZDUG LPSURYLQJ WKH TXDOLW\ RI WKH VHUYLFHV ZH SURYLGH ,Q RUGHU WR DFKLHYH WKLV JRDO ZH KDYH GHFLGHG WR DVN RXU PRVW UHFHQW FDOOHUV VRPH VSHFLILF TXHVWLRQV UHJDUGLQJ RXU VHUYLFH
PAGE 184

$33(1',; % 35(7(67 48(67,21 $,5( 9(56,21 21(

PAGE 185

',5(&7,216 3OHDVH DQVZHU WKH TXHVWLRQV WR WKH EHVW RI \RXU DELOLW\ E\ SODFLQJ D FKHFNPDUN LQ 21( RI WKH ER[HV QH[W WR HDFK TXHVWLRQ 7KHUH DUH IRXU VHWV RI TXHVWLRQV f VRPH EDVLF LQIRUPDWLRQ DERXW \RX f \RXU IHHOLQJV DERXW WKH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU $3',&f DW WKH 8QLYHUVLW\ RI $UNDQVDV IRU 0HGLFDO 6FLHQFHV f \RXU SHUFHSWLRQV UHJDUGLQJ WKH DPRXQW RI WLPH WKH $3',& WRRN WR IXOILOO \RXU UHTXHVW DQG f \RXU RYHUDOO IHHOLQJV UHJDUGLQJ WKH $3',& DQG \RXU LPSUHVVLRQV DERXW IXWXUH EHKDYLRUV UHJDUGLQJ WKH $3',& 3$57 7KH IROORZLQJ WZR TXHVWLRQV JDWKHU VRPH LQIRUPDWLRQ DERXW \RX 7KLV LQIRUPDWLRQ ZLOO EH XVHG LQ FRQMXQFWLRQ ZLWK WKH LQIRUPDWLRQ JLYHQ LQ WKH UHVW RI WKH TXHVWLRQQDLUH WR DVVHVV KRZ QHHGV DQG SHUFHSWLRQV GLIIHU DPRQJ RXU FDOOHUV f :KDW LV \RXU SURIHVVLRQ" ’ 53K3KDUP' /L 3K\VLFLDQ 8 1XUVH1XUVH 3UDFWLWLRQHU ’ 2WKHU f +RZ RIWHQ GR \RX XVH WKH $3',&" ’ )LUVW WLPH XVHU ’ WLPHV SHU \HDU ’ WLPHV SHU \HDU ’ WLPHV SHU \HDU WLPHV SHU \HDU BPRUH WKDQ WLPHV SHU \HDU 3$57 ,, 7KH IROORZLQJ VHW RI VWDWHPHQWV UHODWH WR \RXU IHHOLQJV DERXW WKH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU $3',&f DW WKH 8QLYHUVLW\ RI $UNDQVDV IRU 0HGLFDO 6FLHQFHV )RU HDFK VWDWHPHQW SOHDVH FKHFN WKH ER[ WKDW EHVW GHVFULEHV WKH H[WHQW WR ZKLFK \RX EHOLHYH WKH $3',& KDV WKDW FKDUDFWHULVWLF 7KH UDQJH RI VHOHFWLRQ YDULHV IURP 6WURQJO\ $JUHH WR 6WURQJO\ 'LVDJUHH KRZHYHU \RX PD\ FKHFN DQ\ RI WKH ER[HV SURYLGHG ,I \RX IHHO WKDW \RX FDQQRW DQVZHU D TXHVWLRQ RU WKDW WKH TXHVWLRQ GRHV QRW DSSO\ WR \RX \RX PD\ FKHFN WKH ER[ ODEHOOHG 'RQnW .QRZ f 7KH $3',& KDV WKH HTXLSPHQW DQG LQIRUPDWLRQ UHVRXUFHV QHFHVVDU\ WR $ -7 D -7 Hp U DQVZHU P\ TXHVWLRQV ’ ’ ’ ’ ’ ’ ’ f :KHQ FDOO WKH $3',& EDFNJURXQG QRLVH RQ WKHLU HQG LQWHUIHUHV ZLWK P\ DELOLW\ WR FRPPXQLFDWH RYHU WKH WHOHSKRQH M 8 X , X f :KHQ UHFHLYH ZULWWHQ PDWHULDOV IURP WKH $3',& WKH\ DUH FOHDU DQG HDV\ WR UHDG ’ ’ ’ ’ ’ ’ f (PSOR\HHV RI WKH $3',& VSHDN LQ D PDQQHU WKDW LV HDV\ WR XQGHUVWDQG BO BO ’ X X f :KHQ WKH $3',& SURPLVHV WR GR VRPHWKLQJ E\ D FHUWDLQ WLPH LW GRHV VR ’ ’ ’ ’ ’ ’ ’ 29(5f

PAGE 186

f :KHQ KDYH D SUREOHP WKH $3',& LV V\PSDWKHWLF DQG UHDVVXULQJ f 7KH $3',& LV GHSHQGDEOH -V -S D A 7 M6 F I I I FI -" I Rn A I I A -r R L? 9 Y R9 Y $ Â’ Â’Â’Â’Â’Â’Â’ Â’ Â’Â’Â’Â’Â’Â’ f7KH $3',& SURYLGHV LWV VHUYLFHV LQ WKH WLPH LW SURPLVHV ,Â’Â’Â’BfÂ’Â’ f7KH $3',& GRHV QRW JLYH PH LQGLYLGXDO DWWHQWLRQ Â’ Â’Â’Â’Â’Â’Â’ f7KH $3',& GRHV QRW WHOO PH H[DFWO\ ZKHQ VHUYLFHV ZLOO EH SHUIRUPHG Â’ Â’Â’Â’Â’Â’Â’ f 7KH $3',& NHHSV LWV UHFRUGV DFFXUDWHO\ Â’ Â’ Â’ Â’ Â’ f GR QRW UHFHLYH SURPSW VHUYLFH IURP $3',& HPSOR\HHV Â’ Â’ Â’ Â’ Â’ Â’ Â’ f (PSOR\HHV RI WKH $3',& DUH QRW DOZD\V ZLOOLQJ WR KHOS PH Â’ Â’ X Â’ Â’ Â’ Â’ f (PSOR\HHV RI WKH $3',& DUH WRR EXV\ WR UHVSRQG WR FDOOHU UHTXHVWV SURPSWO\ Â’ Â’ Â’ Â’ Â’ f FDQ WUXVW HPSOR\HHV RI WKH $3',& Â’ Â’ Â’ Â’ Â’ =O Â’ f (PSOR\HHV RI WKH $3',& GR QRW JLYH PH SHUVRQDO DWWHQWLRQ Â’ Â’ X Â’ Â’ Â’ f (PSOR\HHV RI WKH $3',& DUH SROLWH Â’ Â’ Â’ Â’ Â’ Â’ Â’ f IHHO VDIH LQ P\ LQWHUDFWLRQV ZLWK WKH $3',& HPSOR\HHV 8 Â’ Â’ X Â’ Â’ Â’ f (PSOR\HHV JHW DGHTXDWH VXSSRUW IURP WKH $3',& WR GR WKHLU MREV ZHOO Â’ Â’ Â’ X X Â’ f (PSOR\HHV RI WKH $3',& GR QRW NQRZ ZKDW P\ QHHGV DUH Â’ Â’ Â’ Â’ Â’ ,X

PAGE 187

’ ’ f 7KH $3',& GRHV QRW KDYH P\ EHVW LQWHUHVWV DW KHDUW ’ ’ /, ’ ’ f 7KH $3',& GRHV QRW KDYH RSHUDWLQJ KRXUV FRQYHQLHQW WR PH ’ ’’’’’’ 3$57 ,,, 7KH IROORZLQJ TXHVWLRQV UHJDUG \RXU SHUFHSWLRQV DERXW WKH OHQJWK RI WLPH LQ ZKLFK WKH VHUYLFH ZDV UHQGHUHG 3OHDVH WKLQN DERXW WKH QH[W IRXU LWHPV LQ WHUPV RI WKH ODVW TXHVWLRQ \RX SUHVHQWHG WR WKH $3',& DQG UHDFW WR WKH VWDWHPHQWV EHORZ XVLQJ WKH VFDOH SURYLGHG $JDLQ \RX PD\ FKHFN DQ\ RI WKH ER[HV RQ WKH VFDOH WR VKRZ KRZ VWURQJ \RXU IHHOLQJV DUH f 7KH DPRXQW RI WLPH WKDW LW WRRN WKH $3',& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV DFFHSWDEOH 9 Hp !$ !e A t Y LY R< IY f f f A r !rY t ’ ’ ’ ’ ’ ’ f %\ WKH WLPH UHFHLYHG D UHVSRQVH IURP WKH $3',& WKH LQIRUPDWLRQ ZDV QR ORQJHU XVHIXO WR PH ’ ’ T T T ’ T f ZLVK WKH $3',& FRXOG SURYLGH D TXLFNHU UHVSRQVH WR P\ TXHVWLRQV M M ’ LM ’ ’ ’ f 7KH DPRXQW RI WLPH WKDW LW WRRN WKH $3',& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV ’ 08&+ 6+257(5 7+$1 (;3(&7(' ’ 6+257(5 7+$1 (;3(&7(' ’ (48$/ 72 :+$7 (;3(&7(' ’ /21*(5 7+$1 (;3(&7(' ’ 08&+ /21*(5 7+$1 (;3(&7(' 29(5f

PAGE 188

3$57 ,9 7KH IROORZLQJ VHYHQ VWDWHPHQWV UHODWH WR \RXU RYHUDOO IHHOLQJV DERXW WKH $3',& 3OHDVH UHVSRQG E\ FKHFNLQJ WKH ER[ ZKLFK EHVW UHIOHFWV \RXU RZQ SHUFHSWLRQV f 7KH RYHUDOO TXDOLW\ RI WKH VHUYLFHV SURYLGHG E\ WKH $3',& LV EHVW GHVFULEHG DV ’ ([FHOOHQW ’ 9HU\ *RRG ’*RRG ’)DLU ’ 3RRU ’8QDFFHSWDEOH f 7KH UHVSRQVHV UHFHLYH IURP WKH $3',& DUH XVHIXO WR PH LQ P\ SUDFWLFH t LV \} 2` U U U U U I LV r RU V G" ’ ’ U ’ ’ f 7KH UHVSRQVHV UHFHLYH IURP WKH $3',& DUH HVVHQWLDO WR PH LQ P\ SUDFWLFH ’ ’’’’’’ f ,W LV LPSRUWDQW WKDW WKH $3',& ID[ PH WKH VXSSRUWLQJ GRFXPHQWV HJ UHFHQW OLWHUDWXUHf IRU WKHLU DQVZHUV WR P\ TXHVWLRQV \ \ \ \ \ \ \ f 7KH $3',&nV DQVZHUV WR P\ TXHVWLRQV DUH XVHG WR LPSURYH SDWLHQW RXWFRPHV ’ ’’’’’’ f LQWHQG WR XVH WKLV VHUYLFH LQ WKH IXWXUH ’ ’’’’’ ’ f ZRXOG UHFRPPHQG WKLV VHUYLFH WR D FROOHDJXH ’ ’’’’’ 8 f $'',7,21$/ &200(176 ,V WKHUH DQ\WKLQJ HOVH WKDW \RX ZRXOG OLNH WR WHOO XV DERXW \RXU H[SHULHQFHVf ZLWK WKH $3',&" $OVR DQ\ FRPPHQWV \RX ZLVK WR PDNH UHJDUGLQJ KRZ ZH FRXOG LPSURYH RXU VHUYLFH ZLOO EH DSSUHFLDWHG HLWKHU KHUH RU LQ D VHSDUDWH OHWWHU 7KDQN \RX YHU\ PXFK IRU \RXU KHOS

PAGE 189

$33(1',; & 35(7(67 48(67,21$,5( 9(56,21 7:2

PAGE 190

',5(&7,216 3OHDVH DQVZHU WKH TXHVWLRQV WR WKH EHVW RI \RXU DELOLW\ E\ SODFLQJ D FKHFNPDUN LQ 21( RI WKH ER[HV QH[W WR HDFK TXHVWLRQ 7KHUH DUH IRXU VHWV RI TXHVWLRQV f VRPH EDVLF LQIRUPDWLRQ DERXW \RX f \RXU IHHOLQJV DERXW WKH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU $3',&f DW WKH 8QLYHUVLW\ RI $UNDQVDV IRU 0HGLFDO 6FLHQFHV f \RXU SHUFHSWLRQV UHJDUGLQJ WKH DPRXQW RI WLPH WKH $3',& WRRN WR IXOILOO \RXU UHTXHVW DQG f \RXU RYHUDOO IHHOLQJV UHJDUGLQJ WKH $3',& DQG \RXU LPSUHVVLRQV DERXW IXWXUH EHKDYLRUV UHJDUGLQJ WKH $3',& 3$57 7KH IROORZLQJ WZR TXHVWLRQV JDWKHU VRPH LQIRUPDWLRQ DERXW \RX 7KLV LQIRUPDWLRQ ZLOO EH XVHG LQ FRQMXQFWLRQ ZLWK WKH LQIRUPDWLRQ JLYHQ LQ WKH UHVW RI WKH TXHVWLRQQDLUH WR DVVHVV KRZ QHHGV DQG SHUFHSWLRQV GLIIHU DPRQJ RXU FDOOHUV f:KDW LV \RXU SURIHVVLRQ" Â’ 53K3KDUP' Â’3K\VLFLDQ Â’ 1XUVH1XUVH 3UDFWLWLRQHU Â’2WKHU f+RZ RIWHQ GR \RX XVH WKH $3',&" Â’ )LUVW WLPH XVHU Â’ WLPHV SHU \HDU Â’ WLPHV SHU \HDU 4 WLPHV SHU \HDU Â’ WLPHV SHU \HDU Â’ PRUH WKDQ WLPHV SHU \HDU 3$57 ,, 7KH IROORZLQJ VHW RI VWDWHPHQWV UHODWH WR \RXU IHHOLQJV DERXW WKH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU $3',&f DW WKH 8QLYHUVLW\ RI $UNDQVDV IRU 0HGLFDO 6FLHQFHV )RU HDFK VWDWHPHQW SOHDVH FKHFN WKH ER[ WKDW EHVW GHVFULEHV WKH H[WHQW WR ZKLFK \RX EHOLHYH WKH $3',& KDV WKDW FKDUDFWHULVWLF 7KH UDQJH RI VHOHFWLRQ YDULHV IURP 6WURQJO\ $JUHH WR 6WURQJO\ 'LVDJUHH KRZHYHU \RX PD\ FKHFN DQ\ RI WKH ER[HV SURYLGHG ,I \RX IHHO WKDW \RX FDQQRW DQVZHU D TXHVWLRQ RU WKDW WKH TXHVWLRQ GRHV QRW DSSO\ WR \RX \RX PD\ FKHFN WKH ER[ ODEHOOHG 'RQnW .QRZ f 7KH $3',& KDV WKH HTXLSPHQW DQG LQIRUPDWLRQ UHVRXUFHV QHFHVVDU\ WR DQVZHU P\ TXHVWLRQV f :KHQ FDOO WKH $3',& EDFNJURXQG QRLVH RQ WKHLU HQG LQWHUIHUHV ZLWK P\ DELOLW\ WR FRPPXQLFDWH RYHU WKH WHOHSKRQH f :KHQ UHFHLYH ZULWWHQ PDWHULDOV IURP WKH $3',& WKH\ DUH FOHDU DQG HDV\ WR UHDG f (PSOR\HHV RI WKH $3',& VSHDN LQ D PDQQHU WKDW LV HDV\ WR XQGHUVWDQG f :KHQ WKH $3',& SURPLVHV WR GR VRPHWKLQJ E\ D FHUWDLQ WLPH LW GRHV VR U: A FM! U U 6 X Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ X Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Q Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ Q Â’ 29(5f

PAGE 191

f :KHQ KDYH D SUREOHP WKH $3',& LV V\PSDWKHWLF DQG UHDVVXULQJ f 7KH $3',& LV GHSHQGDEOH f 7KH $3',& SURYLGHV LWV VHUYLFHV LQ WKH WLPH LW SURPLVHV f 7KH $3',& GRHV QRW JLYH PH LQGLYLGXDO DWWHQWLRQ f 7KH $3',& GRHV QRW WHOO PH H[DFWO\ ZKHQ VHUYLFHV ZLOO EH SHUIRUPHG 2 7I} mr Fr r 44444484 Â’ Â’Â’Â’Â’Â’Â’Â’ Â’ X Â’ Â’ X Â’ Â’ X Â’ Â’Â’Â’Â’Â’Â’Â’ Â’ Â’Â’Â’Â’Â’Â’Â’ f 7KH $3',& NHHSV LWV UHFRUGV DFFXUDWHO\ Â’ Â’ Â’ Â’ Â’ ,M f GR QRW UHFHLYH SURPSW VHUYLFH IURP $3',& HPSOR\HHV Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ f (PSOR\HHV RI WKH $3',& DUH QRW DOZD\V ZLOOLQJ WR KHOS PH Â’ Â’ 8 X Â’ X Â’ Â’ f (PSOR\HHV RI WKH $3',& DUH WRR EXV\ WR UHVSRQG WR FDOOHU UHTXHVWV SURPSWO\ Â’ Â’ Â’ Â’ Â’ Â’ Â’ ,f FDQ WUXVW HPSOR\HHV RI WKH $3',& Â’ Â’ Â’ Â’ Â’ f (PSOR\HHV RI WKH $3',& GR QRW JLYH PH SHUVRQDO DWWHQWLRQ Â’ Â’ X X Â’ Â’ Â’ f (PSOR\HHV RI WKH $3',& DUH SROLWH Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ f IHHO VDIH LQ P\ LQWHUDFWLRQV ZLWK WKH $3',& HPSOR\HHV Â’ Â’ X Â’ X Â’ Â’ Â’ f (PSOR\HHV JHW DGHTXDWH VXSSRUW IURP WKH $3',& WR GR WKHLU MREV ZHOO Â’ Â’ Â’ Â’ Â’ Â’ Â’ f (PSOR\HHV RI WKH $3',& GR QRW NQRZ ZKDW P\ QHHGV DUH Â’ Â’ Â’ Â’ Â’ Â’ Â’ -

PAGE 192

f 7KH $3',& GRHV QRW KDYH P\ EHVW LQWHUHVWV DW KHDUW Â’ Â’Â’Â’Â’Â’Â’Â’ f 7KH $3',& GRHV QRW KDYH RSHUDWLQJ KRXUV FRQYHQLHQW WR PH Â’ Â’Â’ Â’Â’Â’Â’Â’ 3$57 ,,, 7KH IROORZLQJ TXHVWLRQV UHJDUG \RXU SHUFHSWLRQV DERXW WKH OHQJWK RI WLPH LQ ZKLFK WKH VHUYLFH ZDV UHQGHUHG 3OHDVH WKLQN DERXW WKH QH[W IRXU LWHPV LQ WHUPV RI WKH ODVW TXHVWLRQ \RX SUHVHQWHG WR WKH $3',& DQG UHDFW WR WKH VWDWHPHQWV EHORZ XVLQJ WKH VFDOH SURYLGHG $JDLQ \RX PD\ FKHFN DQ\ RI WKH ER[HV RQ WKH VFDOH WR VKRZ KRZ VWURQJ \RXU IHHOLQJV DUH f 7KH DPRXQW RI WLPH WKDW LW WRRN WKH $3',& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV DFFHSWDEOH Â’ Â’Â’Â’Â’Â’Â’Â’ f %\ WKH WLPH UHFHLYHG D UHVSRQVH IURP WKH $3',& WKH LQIRUPDWLRQ ZDV QR ORQJHU XVHIXO WR PH Â’ Â’Â’Â’Â’Â’Â’Â’ f ZLVK WKH $3',& FRXOG SURYLGH D TXLFNHU UHVSRQVH WR P\ TXHVWLRQV Â’ Â’Â’Â’Â’Â’Â’Â’ f 7KH DPRXQW RI WLPH WKDW LW WRRN WKH $3',& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV Â’ 08&+ 6+257(5 7+$1 (;3(&7(' 8 6+257(5 7+$1 (;3(&7(' Â’ (48$/ 72 :+$7 (;3(&7(' Â’ /21*(5 7+$1 (;3(&7(' Â’ 08&+ /21*(5 7+$1 (;3(&7(' 29(5f

PAGE 193

3$57 ,9 7KH IROORZLQJ VHYHQ VWDWHPHQWV UHODWH WR \RXU RYHUDOO IHHOLQJV DERXW WKH $3',& 3OHDVH UHVSRQG E\ FKHFNLQJ WKH ER[ ZKLFK EHVW UHIOHFWV \RXU RZQ SHUFHSWLRQV f 7KH RYHUDOO TXDOLW\ RI WKH VHUYLFHV SURYLGHG E\ WKH $3',& LV EHVW GHVFULEHG DV ’ ([FHOOHQW 4 9HU\ *RRG ’ *RRG ’ )DLU ’ 3RRU ’ 8QDFFHSWDEOH f 7KH UHVSRQVHV UHFHLYH IURP WKH $3',& DUH XVHIXO WR PH LQ P\ SUDFWLFH $ D M6n [IF -r M\ I I I I I r t r t ’ ’’’’’ f 7KH UHVSRQVHV UHFHLYH IURP WKH $3',& DUH HVVHQWLDO WR PH LQ P\ SUDFWLFH ’ ’ ’ ’ ’ ’ ’ ’ f ,W LV LPSRUWDQW WKDW WKH $3',& ID[ PH WKH VXSSRUWLQJ GRFXPHQWV HJ UHFHQW OLWHUDWXUHf IRU WKHLU DQVZHUV WR P\ TXHVWLRQV ’ 8 ’ ’ X BL f 7KH $3',&V DQVZHUV WR P\ TXHVWLRQV DUH XVHG WR LPSURYH SDWLHQW RXWFRPHV ’ ’ ’ ’ ’ ’ ’ ’ f LQWHQG WR XVH WKLV VHUYLFH LQ WKH IXWXUH ’ ’ ’ ’ ’ M f ZRXOG UHFRPPHQG WKLV VHUYLFH WR D FROOHDJXH ’ ’ ’ ’ ’ ’ X M f $'',7,21$/ &200(176 ,V WKHUH DQ\WKLQJ HOVH WKDW \RX ZRXOG OLNH WR WHOO XV DERXW \RXU H[SHULHQFHVf ZLWK WKH $3',&" $OVR DQ\ FRPPHQWV \RX ZLVK WR PDNH UHJDUGLQJ KRZ ZH FRXOG LPSURYH RXU VHUYLFH ZLOO EH DSSUHFLDWHG HLWKHU KHUH RU LQ D VHSDUDWH OHWWHU 7KDQN \RX YHU\ PXFK IRU \RXU KHOS

PAGE 194

$33(1',; 35(7(67 )2//2:83 3267&$5' Â’ -$06 OOQnLYUWUVL\ R $U8YYVDV IRU 0HGLU D, 6FLHQFHV &2//(*( 2) 3+$50$&< :HVW 0DUNKDP 6W 6ORW /LWWOH 5RFN $UNDQVDV 6LGH 2QHf 'HDU &ROOHDJXH $ERXW IRXU ZHHNV DJR D TXHVWLRQQDLUH VHHNLQJ \RXU RSLQLRQV DERXW WKH VHUYLFH TXDOLW\ RI WKH $UNDQVDV 3RLVRQ DQG 'UXJ ,QIRUPDWLRQ &HQWHU $3',&f ZDV PDLOHG WR \RX ,I \RX KDYH DOUHDG\ FRPSOHWHG DQG UHWXUQHG WKH TXHVWLRQQDLUH WR XV SOHDVH DFFHSW RXU VLQFHUH WKDQNV ,I QRW SOHDVH GR VR WRGD\ %HFDXVH LW ZDV VHQW WR RQO\ D VPDOO VDPSOH RI RXU UHFHQW FDOOHUV LW LV H[WUHPHO\ LPSRUWDQW WKDW \RXUV DOVR EH LQFOXGHG LQ WKH VWXG\ LI WKH UHVXOWV DUH WR DFFXUDWHO\ UHSUHVHQW WKH IHHOLQJV RI WKH SURIHVVLRQDOV ZH VHUYH ,I E\ VRPH FKDQFH \RX GLG QRW UHFHLYH WKH TXHVWLRQQDLUH RU LW JRW PLVSODFHG SOHDVH FDOO PH DW f & DQG ZLOO JHW DQRWKHU RQH LQ WKH PDLO WR \RX
PAGE 195

$33(1',; ( 5(63216(6 72 35(7(67 48(67,21$,5( 9(56,21 21( 9HUVLRQ 'RQnW .QRZ 5HVSRQVH 9DOXH ([FOXGHG 5HVSRQVH 6WURQJO\ $JUHH $JUHH 6RPHZKDW $JUHH 1HXWUDO 6RPHZKDW 'LVDJUHH 'LVDJUHH 6WURQJO\ 'LVDJUHH 1R 5HVSRQVH 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

PAGE 196

$33(1',; ) 5(63216(6 72 35(7(67 48(67,21$,5( 9(56,21 7:2 9HUVLRQ 'RQnW .QRZ 5HVSRQVH 9DOXH ,QFOXGHG 6WURQJO\ 6RPHZKDW 6RPHZKDW 6WURQJO\ 'RQnW 1R 5HVSRQVH $JUHH $JUHH $JUHH 1HXWUDO 'LVDJUHH 'LVDJUHH 'LVDJUHH .QRZ 5HVSRQVH 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

PAGE 197

$33(1',; 35(7(67 48(67,211$,5( :5,77(1 &200(176 4XHVWLRQ 7KH $3',& KDV WKH HTXLSPHQW DQG LQIRUPDWLRQ UHVRXUFHV QHFHVVDU\ WR DQVZHU P\ TXHVWLRQV f+HUEDO SURGXFWV DUH DQ DUHD RI FRQVWDQW TXHVWLRQV WKDW JRRG UHIHUHQFHV DUH QHHGHG QRQH DUH HYHQ DYDLODEOH EHVLGHV /DZUHQFH DQG 0LFURPHGH[ff 4XHVWLRQ :KHQ FDOO WKH $3',& EDFNJURXQG QRLVH RQ WKHLU HQG LQWHUIHUHV ZLWK P\ DELOLW\ WR FRPPXQLFDWH RYHU WKH WHOHSKRQH f1R EDFNJURXQG QRLVHf 4XHVWLRQ :KHQ UHFHLYH PDWHULDOV IURP WKH $3',& WKH\ DUH FOHDU DQG HDV\ WR UHDG f1$ 0RVW RI P\ LQIR LV RYHU WKH SKRQH KDYH QRW UHFHLYHG ZULWWHQ PDWHULDOf m f;HUR[HV VRPHWLPHV RII FHQWHU DQG FXW RII SDUW RI WKH SDJHf f'RQfW .QRZf f:H KDYH QRW UHFHLYHG DQ\ ZULWWHQ PDWHULDO \HWf f1$f f f-XVW WKH ID[HV DUH KDUG WR UHDGf f)D[HV VRPHWLPHV KDUG WR UHDG 7KLV LV WR EH H[SHFWHGf f$3',& KDV RIIHUHG ,fYH QHYHU QHHGHGf

PAGE 198

4XHVWLRQV :KHQ WKH $3',& SURPLVHV WR GR VRPHWKLQJ E\ D FHUWDLQ WLPH LW GRHV VR f+DYH RQO\ DVNHG IRU LQIRUPDWLRQ YHU\ TXLFN RQFHf 4XHVWLRQ :KHQ KDYH D SUREOHP WKH $3',& LV V\PSDWKHWLF DQG UHDVVXULQJ f3URIHVVLRQDOf 1RWH :DV D ZULWWHQ RQ WKH TXHVWLRQQDLUH DV LI WR FODULI\ WKH UHVSRQGHQWV LQWHUSUHWDWLRQ RI WKH TXHVWLRQ f7KLV FRXOG EH DVNHG LQ D GLIIHUHQW ZD\f 4XHVWLRQ 7KH $3',& SURYLGHV LWV VHUYLFHV LQ WKH WLPH LW SURPLVHV f8VXDOO\ 6RPHWLPHV D VPDOO GHOD\ EHFDXVH RI DQRWKHU HPHUJHQF\ RQ D GLIILFXOW TXHVWLRQ 7KH\ DOZD\V H[SODLQ WKH UHDVRQ IRU WKH GHOD\f 4XHVWLRQ 7KH $3',& GRHV QRW WHOO H[DFWO\ ZKHQ VHUYLFHV ZLOO EH SHUIRUPHG f,I DVN ZKHQ WKH\ DOZD\V WHOO PHf f%XW KDYH RQO\ DVNHG WKHP IRU VSHFLILF WLPHVf 4XHVWLRQ 7KH $3',& NHHSV LWV UHFRUGV DFFXUDWHO\ f'RQfW .QRZf f f'RQfW .QRZ ZRXOG DVVXPH WKDW WKH\ GRf f, GRQfW NQRZ ,fYH QHYHU QRWLFHG DQ\ FRPPXQLFDWLRQUHOD\ SUREOHPVf f'RQfW UHDOO\ XQGHUVWDQG WKH VWDWHPHQWf f7R WKH EHVW RI P\ NQRZOHGJHf 4XHVWLRQ (PSOR\HHV RI WKH $3',& DUH WRR EXV\ WR UHVSRQG WR FDOOHU UHTXHVWV SURPSWO\ f6HUYLFH LVQfW DV IDVW DV LW XVHG WR EH EXW FHUWDLQO\ LV VWLOO YHU\ WLPHO\ DQG SULRULWL]HG EDVHG RQ WKH UHTXHVWf

PAGE 199

4XHVWLRQ FDQ WUXVW HPSOR\HHV RI WKH $3',& f'RQfW .QRZ f 4XHVWLRQ IHHO VDIH LQ P\ LQWHUDFWLRQV ZLWK WKH $3',& HPSOR\HHV f"f &LUFOHG WKH ZRUG VDIH! f+XK"f 4XHVWLRQ (PSOR\HHV JHW DGHTXDWH VXSSRUW IURP WKH $3',& WR GR WKHLU MREV ZHOO &&A f'RQfW .QRZf f f+RZ ZRXOG NQRZ"f f:KR" 0\ HPSOR\HH RU $3',& HPSOR\HH"f f7KH RQO\ KLQW KDYH LV KRZ ZHOO WKH HPSOR\HHV DQVZHU P\ TXHVWLRQVf f1$f f'RQfW XQGHUVWDQGf 4XHVWLRQ (PSOR\HHV RI WKH $3',& GR QRW NQRZ ZKDW P\ QHHGV DUH LF\} f7KH\ DOZD\V UHDGGUHVV TXHVWLRQ LI WKH\ GLGQfW XQGHUVWDQG WKH IRFXVf f'RQfW XQGHUVWDQG f 4XHVWLRQ ZLVK WKH $3',& FRXOG SURYLGH D TXLFNHU UHVSRQVH WR P\ TXHVWLRQV f7KH\ FDQfW JHW DQ\ IDVWHUf f, WKLQN WKDW LV DQ XQIDLU TXHVWLRQ ,W DOZD\V GHSHQGV RQ ZKDW RWKHU TXHVWLRQV WKH\ KDYH SHQGLQJ &DOOHUV QHHG WR XQGHUVWDQG DQG UHVSHFW WKDW KDYH QHYHU EHHQ XQVDWLVILHG ZLWK D UHVSRQVH WLPH RI DQ XUJHQW TXHVWLRQf

PAGE 200

f'XH WR SULRU H[SHULHQFH ZLWK WKHPf ,Q H[SODQDWLRQ RI WKHLU DQVZHU! f,Q JHQHUDO RU MXVW WKH ODVW RQH"f 4XHVWLRQ 7KH DPRXQW RI WLPH WKDW LW WRRN WKH $3',& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV f, DOZD\V JHW JUHDW VHUYLFHf f%XW $3',& ZDV XQDEOH WR )$; RU PDLO WKH LQIRUPDWLRQ UHTXHVWHG IRXQG WKLV D OLWWOH RGG DQG LW GLG QRW IXOO\ PHHW P\ QHHGV IRU GUXJ LQIRUPDWLRQf 4XHVWLRQ ,W LV LPSRUWDQW WKDW WKH $3',& ID[ PH WKH VXSSRUWLQJ GRFXPHQWV HJ UHFHQW OLWHUDWXUHf IRU WKHLU DQVZHUV WR P\ TXHVWLRQV f7KLV WHQGV WR YDU\ IURP VLWXDWLRQ WR VLWXDWLRQf f8QGRFXPHQWHG DQVZHUV DUH XVHOHVVf f,I QHFHVVDU\ \HVf f,Q PRVW FDVHV EXW QRW DOZD\Vf f)$; RU PDLOf f1$f f9HU\ LPSRUWDQWf 4XHVWLRQ ZRXOG UHFRPPHQG WKLV VHUYLFH WR D FROOHDJXH 7KUHH FKHFNV RQ VWURQJO\ DJUHHf f$QG KDYH GRQHf

PAGE 201

4XHVWLRQ $'',7,21$/ &200(176 ,V WKHUH DQ\WKLQJ HOVH WKDW \RX ZRXOG OLNH WR WHOO XV DERXW \RXU H[SHULHQFHVf ZLWK WKH $3',&" $OVR DQ\ FRPPHQWV \RX ZLVK WR PDNH UHJDUGLQJ KRZ ZH FRXOG LPSURYH RXU VHUYLFH ZLOO KH DSSUHFLDWHG HLWKHU KHUH RU LQ D VHSDUDWH OHWWHU f7KH\ QHHG WR XSGDWH *HQHULF /LVWLQJV IRU ERWK 5[ DQG 27& PHGLFDWLRQV LGHQWLILFDWLRQ DQG PDQXIDFWXUHUf f, EHOLHYH WKH $3',& LV RQH RI WKH EHVW WKLQJV WKH 3KDUPDF\ 6FKRRO GRHV IRU WKH SXEOLF KDYH QHYHU KDG DQ\ WKLQJ EXW JRRG H[SHULHQFHV ZKHQ FDOO IRU GUXJ RU SRLVRQ LQIRUPDWLRQ ,Q IDFW FDOOHG WRGD\ DQG ZDV JLYHQ DQ DQVZHU WR D SRLVRQ TXHVWLRQ LQ PLQXWHV .HHS XS WKH JRRG ZRUN 1$0(6 20,77('!f f,PSURYH WKH TXDOLW\ RI \RXU )$; HTXLSPHQW 7KDQN \RXf f1R WKHUH LV QRWKLQJ HOVH ZRXOG OLNH WR VD\ H[FHSW WKDW WKH VHUYLFH ZDV JUHDWf f7KH $3',& KDV FRPH WR RXU UHVFXH PDQ\ WLPHV FDQfW VD\ HQRXJK DERXW WKH VHUYLFH 1$0(6 20,77('! DUH WKH JUHDWHVW 7KDQNVf f
PAGE 202

f$ YHU\ ZHOO QDQ RSHUDWLRQ NQRZ FDQ fKDQJ P\ KDWf RQ $3',& *UHDW EXQFK RI GHSHQGDEOH SHUVRQDEOH NQRZOHGJHDEOH SURIHVVLRQDOV LPSURYLQJ WKH RYHUDOO KHDOWK FDUH IRU $UNDQVDQVf f7KH\ DUH JUHDWf f:LVK \RX FRXOG VHDUFK DEVWUDFWLQJ GDWDEDVHV VXFK DV (0%$6(f f, FDQfW WKLQN RI D WKLQJ WR LPSURYH (YHU\RQH KDYH WDONHG WR LV YHU\ NQRZOHGJHDEOH FRXUWHRXV DQG KHOSIXO 7KH\ GR DQ RXWVWDQGLQJ MRE RI SULRULWL]LQJ TXHVWLRQV DQG VROLFLWLQJ WKH LQIRUPDWLRQ LQ RUGHU WR IRFXV WKH TXHVWLRQ FRUUHFWO\ 7KDQN \RX IRU PDNLQJ P\ MRE HDVLHUf f([FHOOHQWf f2QH PLQRU WKLQJ OLNH WR NQRZ ZKR DP WDONLQJ WR XVXDOO\ H[SHFW VWDII DW DQ RUJDQL]DWLRQ WR LGHQWLI\ WKHPVHOYHV ZKHQ DQVZHULQJf f7KH $3',& (PSOR\HH WULHG WR DQVZHU P\ TXHVWLRQ DERXW D VSHFLILF JHQHULF HTXLYDOHQF\ EXW FRXOG QRW ILQG DQ DQVZHUf f, EHJDQ XVLQJ $3',& ZKHQ ZH EHFDPH DQ 2ZHQ DFFRXQW ODWH f 6R IDU KDYH XVHG $3',& >DSSUR[LPDWHO\ WLPHV D PRQWK@ 1$0( 20,77('! 8VXDOO\ WDNHV P\ FDOOV +H GRHV D JUHDW MRE LQ REWDLQLQJ D UHVSRQVH DQG JHWWLQJ EDFN WR PH WKDW VDPH GD\ DQG LV JRRG DERXW OHWWLQJ PH NQRZ KRZ ORQJ LW ZLOO WDNH GHSHQGLQJ RQ WKH ZRUNORDG 2WKHUV KDYH LQWHUDFWHG ZLWK KDYH DOVR SURYLGHG JUHDW FXVWRPHU VHUYLFHf f, KDYH XVHG WKH $3',& VHYHUDO WLPHV DQG KDYH DOZD\V IRXQG WKH VWDII WR EH FRXUWHRXV DQG ZHOO WUDLQHG ,fYH UHFHLYHG LQWHOOLJHQW DQVZHUV WR P\ TXHVWLRQV DV ZHOO DV VXSSRUWLQJ GRFXPHQWV ZKHQ QHHGHGf f*RRG MRE 7KDQN \RXf f, KDYH EHHQ H[WUHPHO\ KDSS\ ZLWK P\ VHUYLFH 7KHUH DUH PDQ\ WLPHV FDOO MXVW IRU LQIRUPDWLRQ QRW DQ HPHUJHQF\f WR TXHVWLRQV JHW RQ YDULRXV GUXJV HWF )RU H[DPSOH WRGD\ D QXUVH FDOOHG DQG ZDQWHG WR NQRZ LI 06 &RQWHQ PJ FRXOG EH LQVHUWHG UHFWDOO\ 6KH KDG KHDUG WKDW LW FRXOG EXW WKH SKDUPDFLVW VKH FDOOHG GLGQfW NQRZ FDOOHG $3',& DQG JRW WKH LQIRUPDWLRQ RUDOO\ DQG E\ ID[ LQ D PDWWHU RI PLQXWHV ,W LV JUHDW WR NQRZ WKDW TXHVWLRQV OLNH WKLV FDQ EH DQVZHUHG .HHS XS WKH JRRG ZRUN 7KDQN \RXf f2Q WKH PDLQ SRLQWV YHU\ JUDWHIXO IRU D XVHIXO VHUYLFH 0\ RQO\ FRQFHUQ LV KDYH QR LGHD ZKDW UHVRXUFHV ZHUH XVHG WR KDQGOH P\ TXHULHVf f, FDOOHG UHFHQWO\ DQG QHHGHG GRVLQJ LQIRUPDWLRQ RQ VHYHUDO GUXJV KLJKHU GRVHV WKDQ FRXOG ILQG LQ DQ\ UHIHUHQFHVf QHHGHG WKLV LQIR VWDW ZLWKLQ PLQXWHVf 0DUN FDOOHG PH

PAGE 203

EDFN ZLWKLQ WKH PLQXWHV ZLWK WKH LQIR QHHGHG UHDOO\ DSSUHFLDWH WKH SURPSWQHVV RI \RXU VHUYLFH $3',& LV DOZD\V YHU\ KHOSIXO WR PHf f:H DUH YHU\ KDSS\ ZLWK WKLV VHUYLFHf fTXHVWLRQV@ ZRXOG EH JLYHQ SULRULW\ WKRXJK KDYH QHYHU FDOOHG DERXW D SRLVRQ TXHVWLRQ DV ZH KDYH RXU RZQ SRLVRQ FRQWURO FHQWHU DW 2.&f f, KDYH ZRUNHG LQ D SRLVRQ FRQWURO VHUYLFH DQG \RX FOHDUO\ KDYH WKH UHVRXUFHV QHHGHG DQG WKH SHUVRQQHO WR GR DQ H[FHOOHQW MREf f:H WKLQN WKH HQWLUH VWDII GRHV D JUHDW MRE JHWWLQJ WKH LQIRUPDWLRQ ZH QHHG WR XV LQ D SURPSW PDQQHU 7KH\ DUH D JUHDW EHQHILW WR XVf f, IHHO WKDW WKH $3',& RIIHUV WKH PRVW LPSRUWDQW LQIRUPDWLRQ VHUYLFH D KHDOWK SURIHVVLRQDO PHPEHU KDV DFFHVV WR ,I WKLV VHUYLFH ZHUH QRW DYDLODEOH P\ DELOLW\ WR SUDFWLFH P\ SURIHVVLRQ ZRXOG EH GLPLQLVKHGf f
PAGE 204

f, ZRUN ZLWK D QXPEHU RI GLUHFWRUV DQG FOLQLFDO PDQDJHUV 0\ LPSUHVVLRQ LV WKDW \RX GRQfW DOZD\V DVN WKH ULJKW TXHVWLRQV WR GHWHUPLQH ZKDW WKH UHTXHVWRU UHDOO\ QHHGV DW WKH $3',&@ DUH H[FHOOHQW FOLQLFLDQV f0\ LQWHUDFWLRQV ZLWK $3',& KDYH DOO EHHQ RYHU WKH SKRQH (YHU\ WLPH WKH VWDII KDYH EHHQ YHU\ KHOSIXO DQG SROLWH 0RVW RI P\ TXHVWLRQV KDYHQfW KDG DQ HPHUJHQW UHVSRQVH WLPH EXW ZKHQ QHHGHG WKH TXLFN UHVSRQVH WKH\ >KDYH@ EHHQ JRRG ,Q P\ SUHYLRXV SRVLWLRQ ZDV SK\VLFDOO\ DVNHG WR JR WR D ',& ZKLFK ZDV QLFH :H KDYH D FRQWUDFW ZLWK \RXU GHSDUWPHQW DQG ZKDW PLJKW EH KDQG\ LV D OLVWLQJ RI WKH YDULRXV VHUYLFHV \RX RIIHU DQG PD\EH D OLVW RI GDWDEDVHV \RX XVH %RWWRQ OLQH \HV \RXU GHSDUWPHQW LV YHU\ XVHIXO DQG VKRXOG EH IXQGHG IXOO\ ZLWK UHVRXUFHV DQG VWDIIf

PAGE 205

f,fYH XVHG DQG KDYH LQWHUDFWHG ZLWK GUXJ LQIRUPDWLRQ VHUYLFHV DW 8$% 6DQIRUG 8QLYHUVLW\ DV ZHOO 1R RQH FRPSDUHV WR 8$%fV TXDOLW\ DQG H[SHUWLVH LQ SURYLGLQJ GUXJ LQIRUPDWLRQ WKH HPSOR\HHV DW WKH $3',& DUH@ YHU\ FDSDEOH DQG HDJHU WR KHOS SHRSOH DOZD\V NQRZ WKH\ DUH ULJKW RQ LW DQG QRW GRLQJ D KDSKD]DUG MRE WKH\fUH YHU\ SROLWH ZDQW WR KHOS DQG MXVW SODLQ JHW LW GRQH UHFRPPHQG WKHP DOO RI WKH WLPHf f,WfV JRRG WR NQRZ \RXfUH WKHUH EHKLQG PH ZKHQ QHHG \RX 7KDQNVf f*RRG VHUYLFHf

PAGE 206

$33(1',; + +,6725,&$/ '$7$ 6+((7

PAGE 207

),/( '58* ,1)250$7,21 6(59,&( '$7$ 6+((7 +RZ GLG WKLV TXHVWLRQ FRPH DERXW" 5HTXHVWRU 5SK 0' 51 2WKHU 6XEVFULEHU 1RQ6XEVFULEHU 3KRQH f ([WHQWLRQ $GGUHVV =LS &RGH ),/(
PAGE 208

$33(1',; '$7$ &2//(&7,21 )250

PAGE 209

),/( '58* ,1)250$7,21 6(59,&( '$7$ 6+((7 ,1)250$7,21 5(48(67(' 7\SH RI 4XHVWLRQ &ODVVLILFDWLRQf +RZ GLG WKLV TXHVWLRQ FRPH DERXW" 5HTXHVWRU 3KRQH f ([WHQWLRQ 3DJHU )D[ f $GGUHVV =LS &RGH 7\SH RI 5HVSRQVH 5HTXHVWHG Â’ :ULWWHQ Â’ 2UDO Â’ %RWK Â’ (LWKHU 5HVSRQVH 1HHGHG E\ Â’ 6WDW PLQXWHVf Â’ 7RGD\ 7LPH Â’ 'DWH Â’ 1R 5XVK 6XEVFULEHU Â’ 1RQ6XEVFULEHU Â’ +HDOWK &HQWHU Â’ 53K3KDUP' Â’ 0' (+ 5113 (+ 2WKHU '$7( $1' 7,0( 75$&.,1* $FWLYLW\ &RGHV 5HFHSWLRQ RI &DOO RU )D[ 2EWDLQLQJ ,QIRUPDWLRQ DQG :ULWLQJ $QVZHU $SSURYDO 5HWXUQLQJ $QVZHU WR &DOOHU $FWLYLW\ 3HUVRQ 6WDUW 'DWH 6WDUW 7LPH (QG 'DWH (QG 7LPH 5HVSRQVH &RPSOHWHG %\ 5HVSRQVH $SSURYHG %\

PAGE 210

$33(1',; 6(0,6758&785(' 287/,1( )25 678'(17 ,17(59,(:6 ,QWURGXFWLRQ ,QWURGXFH VHOI ([SODLQ WKH SXUSRVH RI WKH LQWHUYLHZ DQG GHVFULEH WKH LVVXHV WKDW ZLOO EH GLVFXVVHG ([SODLQ WKDW WKHLU QDPHV ZLOO QHYHU EH DWWDFKHG WR DQ\WKLQJ VDLG GXULQJ WKH LQWHUYLHZ $VN SHUPLVVLRQ WR DXGLR UHFRUG WKH LQWHUYLHZ IRU UHFROOHFWLRQ SXUSRVHV ,, 8QGHUVWDQGLQJ WKH 6HUYLFH 6\VWHP 3OHDVH GHVFULEH \RXU MRE KHUH DW WKH ',35& $GGLWLRQDO IROORZXS TXHVWLRQV D :KDW MREV DUH DOO RI WKH VWXGHQWV UHVSRQVLEOH IRU FRPSOHWLQJ" E $UH WKHUH DQ\ MREV WKDW RQO\ \RX DUH UHVSRQVLEOH IRU FRPSOHWLQJ" F %HVLGHV ZKDW ZH KDYH WDONHG DERXW DERYH ZKDW RWKHU DFWLYLWLHV RFFXS\ \RXU WLPH GXULQJ WKH ZHHN H J OXQFK PHHWLQJVf" G +RZ ORQJ GR \RX XVXDOO\ WDNH IRU OXQFK" $UH \RX DYDLODEOH WR WDNH FDOOV GXULQJ \RXU OXQFKWLPH" H :KDW KRXUV LV WKH ',35& RSHQ WR WDNH FDOOV" +RZ RIWHQ GR \RXU PRUQLQJ PHHWLQJV GLVUXSW WKHVH WLPHV" ZRXOG OLNH \RX GR GHVFULEH WKH ZRUN SURFHVV KHUH DW WKH ',35& ([SODLQ WR PH ZKDW KDSSHQV IURP WKH DUULYDO RI D TXHVWLRQ XQWLO DQ DQVZHU LV UHWXUQHG WR WKH FDOOHU $GGLWLRQDO IROORZXS TXHVWLRQV D 'R \RX ILOO RXW D VKHHW IRU HDFK TXHVWLRQ DVNHG" E 'R \RX DQVZHU WKH WHOHSKRQH LI \RX DUH DOUHDG\ ZRUNLQJ RQ D TXHVWLRQ DQG DUH LQ WKH RIILFH" F +RZ RIWHQ GR \RX XVXDOO\ ZRUN RQ PRUH WKDQ RQH TXHVWLRQ" :KDW LV WKH PRVW \RX HYHU ZRUNHG RQ DW RQH WLPH" G 2Q VRPH RI WKH GDWD VKHHWV DQ DQVZHU ZRXOG EH UHWXUQHG EHIRUH LW ZDV DSSURYHG :KHQ GRHV WKLV RFFXU" H 'R \RX REWDLQ DSSURYDO IRU DOO TXHVWLRQV EHIRUH \RX UHWXUQ DQ DQVZHU WR WKH FDOOHU" I :KDW TXHVWLRQV GR \RX IHHO DUH WKH HDVLHVW WR DQVZHU" J :KDW TXHVWLRQV GR \RX IHHO DUH WKH PRVW GLIILFXOW WR DQVZHU"

PAGE 211

K :KDW GR \RX GR DIWHU WKH TXHVWLRQ KDV EHHQ DQVZHUHG" L +RZ ORQJ GRHV LW WDNH IRU \RX WR fFORVHRXWf D TXHVWLRQ DIWHU \RX KDYH UHWXUQHG DQ DQVZHU" M +RZ ORQJ GLG LW WDNH \RX WR EHFRPH FRPIRUWDEOH DQVZHULQJ TXHVWLRQV SRVHG E\ FDOOHUV" N 'HVFULEH IRU PH WKH XVH RI WKH EXOOHWLQ ERDUG" :KHQ GR \RX XVH LW DQG ZKHQ GR \RX KROG RQ WR WKH GDWDFDOO VKHHW" 2QFH \RX KDYH FRPSOHWHG D TXHVWLRQ KRZ GR \RX GHFLGH ZKLFK TXHVWLRQ WR ZRUN RQ QH[W" $ GGLWLRQDO IROORZXS TXHVWLRQV D 'R VRPH W\SHV RI TXHVWLRQV UHFHLYH D KLJKHU SULRULW\ WKDQ RWKHU W\SHV" E $UH DQVZHUHG ILUVW EHFDXVH WKH\ DUH HDVLHU RU PRUH LPSRUWDQW" F 'R \RX VRPHWLPHV VHOHFW TXHVWLRQV EHFDXVH RI LQWHUHVW UDWKHU WKDQ LPSRUWDQFH" G 'R TXHVWLRQV JDLQ KLJKHU LPSRUWDQFH EHFDXVH RI WKH UHTXHVWHG UHVSRQVH WLPH" H 'R TXHVWLRQV JDLQ KLJKHU LPSRUWDQFH EHFDXVH WKH\ DUH D SKDUPDFLVW SK\VLFLDQ HWF" I 'R TXHVWLRQV JDLQ KLJKHU LPSRUWDQFH EHFDXVH WKH\ DUH D KHDOWK FHQWHU HPSOR\HH RU D VXEVFULEHU" ,,, &RQVXPHU ([SHFWDWLRQV DQG 3HUFHSWLRQV :KDW GR \RX WKLQN LV WKH PRVW YDOXHG DVSHFW RI WKH VHUYLFH \RX SURYLGH" :KDW GR \RX WKLQN LV WKH OHDVW YDOXHG DVSHFW RI WKH VHUYLFH \RX SURYLGH" +DYH DQ\ FDOOHUV VHHPHG GLVVDWLVILHG ZLWK D UHVSRQVH \RX KDYH JLYHQ" :K\" :KDW IDFWRUV GR \RX WKLQN PRVW LQIOXHQFH D FDOOHUfV SHUFHSWLRQ RI WKH TXDOLW\ RI WKH VHUYLFH \RX SURYLGH DW WKH ',35&" +RZ VHQVLWLYH GR \RX IHHO WKDW WKH FDOOHUV DUH WR WKH DPRXQW RI WLPH LW WDNHV WR DQVZHU D TXHVWLRQ 2Q \RXU VKHHWV \RX DVN WKH FRQVXPHU WR VSHFLI\ E\ ZKHQ WKH\ QHHG WKH TXHVWLRQ DQVZHUHG +RZ VHQVLWLYH GR \RX IHHO FDOOHUV DUH WR GHOD\V LQ WKLV WLPH IUDPH" ,9 6XJJHVWLRQV IRU ,PSURYHPHQW :KDW GLG \RX OLNH PRVW DERXW ZRUNLQJ DW WKH ',35&" :KDW GLG \RX OLNH OHDVW DERXW ZRUNLQJ DW WKH ',35&" ,V WKHUH DQ\WKLQJ WKDW \RX QHHG RU WKDW ZRXOG PDNH \RXU MRE HDVLHU WKDW LV QRW SURYLGHG E\ WKH ',35&" $UH WKHUH DQ\ VXJJHVWLRQV \RX KDYH IRU LPSURYLQJ WKH SURFHVV DW WKH ',35&"

PAGE 212

$33(1',; 6(0,6758&785(' 287/,1( )25 &2',5(&725 ,17(59,(:6 ,QWURGXFWLRQ ([SODLQ WKH SXUSRVH RI WKH LQWHUYLHZ DQG GHVFULEH WKH LVVXHV WKDW ZLOO EH GLVFXVVHG ([SODLQ WKDW WKHLU QDPHV ZLOO QHYHU EH DWWDFKHG WR DQ\WKLQJ VDLG GXULQJ WKH LQWHUYLHZ $VN SHUPLVVLRQ WR DXGLR UHFRUG WKH LQWHUYLHZ IRU UHFROOHFWLRQ SXUSRVHV ,, 8QGHUVWDQGLQJ WKH 6HUYLFH 6\VWHP 3OHDVH GHVFULEH \RXU MRE DV LW UHODWHV WR WKH ',35& $GGLWLRQDO IROORZXS TXHVWLRQV D %HVLGHV ZKDW ZH KDYH WDONHG DERXW DERYH ZKDW RWKHU DFWLYLWLHV RFFXS\ \RXU WLPH GXULQJ WKH ZHHN HJ OXQFK PHHWLQJVf" E :KDW KRXUV LV WKH ',35& RSHQ WR WDNH FDOOV" ZRXOG OLNH \RX GR GHVFULEH WKH ZRUN SURFHVV KHUH DW WKH ',35& ([SODLQ WR PH ZKDW KDSSHQV IURP WKH DUULYDO RI D TXHVWLRQ XQWLO DQ DQVZHU LV UHWXUQHG WR WKH FDOOHU $GGLWLRQDO IROORZXS TXHVWLRQV D :KR ZRUNV LQ WKH ',53&" :KDW DUH WKHLU UROHVUHVSRQVLELOLWLHV" E $UH WKH VWXGHQWV VXSSRVHG WR ILOO RXW D VKHHW IRU HDFK TXHVWLRQ DVNHG" F $UH WKH VWXGHQWV VXSSRVHG WR DQVZHU WKH SKRQH LI WKH\ DUH DOUHDG\ ZRUNLQJ RQ D TXHVWLRQ DQG DUH LQ WKH RIILFH" G :KDW LV WKH VXJJHVWHG ZRUN SURFHVV" +RZ ZHOO GR WKH VWXGHQWV IROORZ WKLV SURFHGXUH" H 'R \RX HQFRXUDJH RU GLVFRXUDJH WKH VWXGHQWV WR ZRUN RQ PRUH WKDQ RQH TXHVWLRQ DW D WLPH" I 2Q VRPH RI WKH GDWD VKHHWV DQ DQVZHU ZRXOG EH UHWXUQHG EHIRUH LW ZDV DSSURYHG :KHQ GRHV WKLV RFFXU" J ,V WKH VWXGHQW VXSSRVHG WR REWDLQ DSSURYDO IRU DOO TXHVWLRQV EHIRUH \RX UHWXUQ DQ DQVZHU WR WKH FDOOHU" K 'HVFULEH KRZ D TXHVWLRQ LV DSSURYHG SURFHVV L :KDW TXHVWLRQV GR \RX WKLQN WKH VWXGHQWV ILQG DUH WKH HDVLHVW WR DQVZHU" M :KDW TXHVWLRQV GR \RX WKLQN WKH VWXGHQWV ILQG DUH WKH PRVW GLIILFXOW WR DQVZHU"

PAGE 213

N +RZ ORQJ GRHV LW WDNH IRU WKH VWXGHQWV WR EHFRPH FRPIRUWDEOH DQVZHULQJ TXHVWLRQV SRVHG E\ FDOOHUV" O 'HVFULEH WKH XVH RI WKH EXOOHWLQ ERDUG" :KHQ GR \RX XVH LW DQG ZKHQ GR \RX KROG RQ WR WKH GDWDFDOO VKHHW" 2QFH \RX KDYH FRPSOHWHG D TXHVWLRQ KRZ GRHV WKH VWXGHQW GHFLGH ZKLFK TXHVWLRQ WR ZRUN RQ QH[W" $GGLWLRQDO IROORZXS TXHVWLRQV D 'R VRPH W\SHV RI TXHVWLRQV UHFHLYH D KLJKHU SULRULW\ WKDQ RWKHU W\SHV" E $UH VRPH TXHVWLRQV DQVZHUHG ILUVW EHFDXVH WKH\ DUH HDVLHU RU PRUH LPSRUWDQW" F 'R VWXGHQWV VRPHWLPHV VHOHFW TXHVWLRQV EHFDXVH RI LQWHUHVW UDWKHU WKDQ LPSRUWDQFH" G 'R TXHVWLRQV JDLQ KLJKHU LPSRUWDQFH EHFDXVH RI WKH UHTXHVWHG UHVSRQVH WLPH" H 'R TXHVWLRQV JDLQ KLJKHU LPSRUWDQFH EHFDXVH WKH\ DUH D SKDUPDFLVW SK\VLFLDQ HWF" I 'R TXHVWLRQV JDLQ KLJKHU LPSRUWDQFH EHFDXVH WKH\ DUH D KHDOWK FHQWHU HPSOR\HH RU D VXEVFULEHU" ,,, &RQVXPHU ([SHFWDWLRQV DQG 3HUFHSWLRQV :KDW GR \RX WKLQN LV WKH PRVW YDOXHG DVSHFW RI WKH VHUYLFH \RX SURYLGH" :KDW GR \RX WKLQN LV WKH OHDVW YDOXHG DVSHFW RI WKH VHUYLFH \RX SURYLGH" :KDW IDFWRUV GR \RX WKLQN PRVW LQIOXHQFH D FDOOHUfV SHUFHSWLRQ RI WKH TXDOLW\ RI WKH VHUYLFH \RX SURYLGH DW WKH ',35&" +RZ VHQVLWLYH GR \RX IHHO WKDW WKH FDOOHUV DUH WR WKH DPRXQW RI WLPH LW WDNHV WR DQVZHU D TXHVWLRQ 2Q \RXU VKHHWV \RX DVN WKH FRQVXPHU WR VSHFLI\ E\ ZKHQ WKH\ QHHG WKH TXHVWLRQ DQVZHUHG +RZ VHQVLWLYH GR \RX IHHO FDOOHUV DUH WR GHOD\V LQ WKLV WLPH IUDPH" ,9 )XWXUH RI WKH ',35& :KHUH GR \RX WKLQN WKH ',35& LV KHDGHG IRU WKH QH[W \HDU" 7KH QH[W ILYH \HDUV" :KDW LV WKH YLDELOLW\ RI JHWWLQJ PRUH URWDWLRQ VWXGHQWV LI \RX QHHGHG DQ LQFUHDVH LQ VHUYLFH FDSDFLW\" :RXOG \RX QHHG WR KLUH SKDUPDFLVWUHVLGHQWV WR DQVZHU TXHVWLRQV" +RZ IDVW DUH WKH SKDUPDFLVWVUHVLGHQWV ZKHQ FRPSDUHG WR WKH 3KDUP' VWXGHQWV"

PAGE 214

$33(1',; / 7(;7 2) &29(5 /(77(5 )25 0$,1 48(67,211$,5( -XO\ 'HDU &ROOHDJXH 7KH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVHDUFK &HQWHU ',35&f DW 6KDQGV DW WKH 8QLYHUVLW\ RI )ORULGD LV FXUUHQWO\ ZRUNLQJ WRZDUG LPSURYLQJ WKH TXDOLW\ RI VHUYLFHV ZH SURYLGH ,Q RUGHU WR DFKLHYH WKLV JRDO ZH KDYH GHFLGHG WR DVN RXU PRVW UHFHQW FDOOHUV VRPH TXHVWLRQV UHJDUGLQJ VSHFLILF DVSHFWV RI RXU VHUYLFH
PAGE 215

$33(1',; 0 0$,1 48(67,21$,5(

PAGE 216

3DJH ',5(&7,216 3OHDVH DQVZHU WKH TXHVWLRQV WR WKH EHVW RI \RXU DELOLW\ E\ SODFLQJ D FKHFNPDUN LQ 21( RI WKH ER[HV QH[W WR HDFK TXHVWLRQ 7KHUH DUH IRXU VHWV RI TXHVWLRQV f VRPH EDVLF LQIRUPDWLRQ DERXW \RX f \RXU IHHOLQJV DERXW WKH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVRXUFH &HQWHU ',35&f DW 6KDQGV DW WKH 8QLYHUVLW\ RI )ORULGD f \RXU SHUFHSWLRQV UHJDUGLQJ WKH DPRXQW RI WLPH WKH ',35& WRRN WR IXOILOO \RXU UHTXHVW DQG f \RXU RYHUDOO IHHOLQJV UHJDUGLQJ WKH ',35& DQG \RXU LPSUHVVLRQV DERXW IXWXUH EHKDYLRUV UHJDUGLQJ WKH ',35& 3$57 7KH IROORZLQJ WZR TXHVWLRQV JDWKHU VRPH LQIRUPDWLRQ DERXW \RX 7KLV LQIRUPDWLRQ ZLOO EH XVHG LQ FRQMXQFWLRQ ZLWK WKH LQIRUPDWLRQ JLYHQ LQ WKH UHVW RI WKH TXHVWLRQQDLUH WR DVVHVV KRZ QHHGV DQG SHUFHSWLRQV GLIIHU DPRQJ RXU FDOOHUV f :KDW LV \RXU SURIHVVLRQ" Â’ 53KL3KDUP' 8 3K\VLFLDQ Â’ 1XUVH1XUVH 3UDFWLWLRQHU Â’ 2WKHU f $UH \RX D VXEVFULEHU D QRQVXEVFULEHU RU D 6KDQGV +HDOWK &HQWHU HPSOR\HH" Â’ 6XEVFULEHU 8 1RQ6XEVFULEHU 2 +HDOWK &HQWHU (PSOR\HH f +RZ RIWHQ GR \RX XVH WKH ',35&" Â’ )LUVW WLPH XVHU Â’ WLPHV SHU \HDU 8 WLPHV SHU \HDU Â’ WLPHV SHU \HDU Â’ WLPHV SHU \HDU Â’ PRUH WKDQ WLPHV SHU \HDU 3$57 ,, 7KH IROORZLQJ VHW RI VWDWHPHQWV UHODWH WR \RXU IHHOLQJV DERXW WKH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVRXUFH &HQWHU ',35&f DW 6KDQGV DW WKH 8QLYHUVLW\ RI )ORULGD )RU HDFK VWDWHPHQW SOHDVH FKHFN WKH ER[ WKDW EHVW GHVFULEHV WKH H[WHQW WR ZKLFK \RX EHOLHYH WKH ',35& KDV WKDW FKDUDFWHULVWLF 7KH UDQJH RI VHOHFWLRQ YDULHV IURP 6WURQJO\ $JUHH WR 6WURQJO\ 'LVDJUHH KRZHYHU \RX PD\ FKHFN DQ\ RI WKH ER[HV SURYLGHG ,I \RX IHHO WKDW \RX FDQQRW DQVZHU D TXHVWLRQ RU WKDW WKH TXHVWLRQ GRHV QRW DSSO\ WR \RX \RX PD\ FKHFN WKH ER[ ODEHOOHG 'RQnW .QRZ f 7KH ',35& KDV WKH HTXLSPHQW DQG LQIRUPDWLRQ UHVRXUFHV QHFHVVDU\ WR DQVZHU P\ TXHVWLRQV Â’ Â’ Â’ Â’ Â’ BO Â’ Â’ f :KHQ FDOO WKH ',35& EDFNJURXQG QRLVH RQ WKHLU HQG LQWHUIHUHV ZLWK P\ DELOLW\ WR FRPPXQLFDWH RYHU WKH WHOHSKRQH Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ f :KHQ UHFHLYH ZULWWHQ PDWHULDOV IURP WKH ',35& WKH\ DUH FOHDU DQG HDV\ WR UHDG Â’ Â’ Â’ Â’ Â’ Â’ Â’ f (PSOR\HHV RI WKH ',35& VSHDN LQ D PDQQHU WKDW LV HDV\ WR XQGHUVWDQG Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ 29(5f

PAGE 217

3DJH f :KHQ WKH ',35& SURPLVHV WR GR VRPHWKLQJ E\ D FHUWDLQ WLPH LW GRHV VR f :KHQ KDYH D SUREOHP WKH ',35& LV V\PSDWKHWLF DQG UHDVVXULQJ f 7KH ',35& LV GHSHQGDEOH f 7KH ',35& SURYLGHV LWV VHUYLFHV LQ WKH WLPH LW SURPLVHV f 7KH ',35& GRHV QRW JLYH PH LQGLYLGXDO DWWHQWLRQ f 7KH ',35& GRHV QRW WHOO PH H[DFWO\ ZKHQ VHUYLFHV ZLOO EH SHUIRUPHG f GR QRW UHFHLYH SURPSW VHUYLFH IURP ',35& HPSOR\HHV f (PSOR\HHV RI WKH ',35& DUH QRW DOZD\V ZLOOLQJ WR KHOS PH f (PSOR\HHV RI WKH ',35& DUH WRR EXV\ WR UHVSRQG WR FDOOHU UHTXHVWV SURPSWO\ f FDQ WUXVW HPSOR\HHV RI WKH ',35& f (PSOR\HHV RI WKH ',35& GR QRW JLYH PH SHUVRQDO DWWHQWLRQ f (PSOR\HHV RI WKH ',35& DUH SROLWH f IHHO VDIH LQ P\ LQWHUDFWLRQV ZLWK WKH ',35& HPSOR\HHV f (PSOR\HHV RI WKH ',35& GR QRW NQRZ ZKDW P\ QHHGV DUH 9 t t ; A Dp 3 r F Hp I U I I U U U ’ ’ ’ ’ ’ ’ ’ 8 X ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ 8 X ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ X BL X ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ X ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’

PAGE 218

3DJH f 7KH ',35& GRHV QRW KDYH P\ EHVW LQWHUHVWV DW KHDUW $ A br e r mU U U U I I U U ’ ’’’’’’ t S ‘ Rr ’ f 7KH ',35& GRHV QRW KDYH RSHUDWLQJ KRXUV FRQYHQLHQW WR PH ’ ’ ’ ’ ’ ’ ’ ’ 3$57 ,,, 7KH IROORZLQJ TXHVWLRQV UHJDUG \RXU SHUFHSWLRQV DERXW WKH OHQJWK RI WLPH LQ ZKLFK WKH VHUYLFH ZDV UHQGHUHG 3OHDVH WKLQN DERXW WKH QH[W IRXU LWHPV LQ WHUPV RI WKH ODVW TXHVWLRQ \RX SUHVHQWHG WR WKH ',35& DQG UHDFW WR WKH VWDWHPHQWV EHORZ XVLQJ WKH VFDOH SURYLGHG $JDLQ \RX PD\ FKHFN DQ\ RI WKH ER[HV RQ WKH VFDOH WR VKRZ KRZ VWURQJ \RXU IHHOLQJV DUH f 7KH DPRXQW RI WLPH WKDW LW WRRN WKH ',35& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV DFFHSWDEOH r U U ’ ’ A ‘r \ U U U U U ’’’’’’ f %\ WKH WLPH UHFHLYHG D UHVSRQVH IURP WKH ',35& WKH LQIRUPDWLRQ ZDV QR ORQJHU XVHIXO WR PH 2 ’ T T ’ ’ ’ M f ZLVK WKH ',35& FRXOG SURYLGH D TXLFNHU UHVSRQVH WR P\ TXHVWLRQV 8 8 /, 8 f 7KH DPRXQW RI WLPH WKDW LW WRRN WKH ',35& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV ’ 08&+ 6+257(5 7+$1 (;3(&7(' ’ 6+257(5 7+$1 (;3(&7(' ’ $ /,77/( 6+257(5 7+$1 (;3(&7(' ’ (48$/ 72 :+$7 (;3(&7(' ’ $ /,77/( /21*(5 7+$1 (;3(&7(' ’ /21*(5 7+$1 (;3(&7(' ’ 08&+ /21*(5 7+$1 (;3(&7(' 29(5f

PAGE 219

3DJH 3$57 ,9 7KH IROORZLQJ VHYHQ VWDWHPHQWV UHODWH WR \RXU RYHUDOO IHHOLQJV DERXW WKH ',35& 3OHDVH UHVSRQG E\ FKHFNLQJ WKH ER[ ZKLFK EHVW UHIOHFWV \RXU RZQ SHUFHSWLRQV f7KH RYHUDOO TXDOLW\ RI WKH VHUYLFHV SURYLGHG E\ WKH ',35& LV EHVW GHVFULEHG DV BO ([FHOOHQW 4 9HU\ *RRG 4 *RRG Â’ )DLU Â’ 3RRU 8 8QDFFHSWDEOH f7KH UHVSRQVHV UHFHLYH IURP WKH ',35& DUH XVHIXO WR PH LQ P\ SUDFWLFH 2n Rn -r mV Rn \} F" F" fn F U U U U U U 4 Rnf n666 Â’ 8 f7KH UHVSRQVHV UHFHLYH IURP WKH ',35& DUH HVVHQWLDO WR PH LQ P\ SUDFWLFH Â’ Â’Â’Â’Â’Â’Â’Â’ f ,W LV LPSRUWDQW WKDW WKH ',35& PDLO DQGRU ID[ PH WKH VXSSRUWLQJ GRFXPHQWV HJ UHFHQW OLWHUDWXUHf IRU WKHLU DQVZHUV WR P\ TXHVWLRQV Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ f 7KH ',35&nV DQVZHUV WR P\ TXHVWLRQV DUH XVHG WR LPSURYH SDWLHQW RXWFRPHV Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ f LQWHQG WR XVH WKLV VHUYLFH LQ WKH IXWXUH Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ f ZRXOG UHFRPPHQG WKLV VHUYLFH WR D FROOHDJXH Â’ Â’ Â’ Â’ Â’ Â’ Â’ Â’ f $'',7,21$/ &200(176 ,V WKHUH DQ\WKLQJ HOVH WKDW \RX ZRXOG OLNH WR WHOO XV DERXW \RXU H[SHULHQFHVf ZLWK WKH ',35&" $OVR DQ\ FRPPHQWV \RX ZLVK WR PDNH UHJDUGLQJ KRZ ZH FRXOG LPSURYH RXU VHUYLFH ZLOO EH DSSUHFLDWHG HLWKHU KHUH RU LQ D VHSDUDWH OHWWHU 7KDQN \RX YHU\ PXFK IRU \RXU KHOS

PAGE 220

$33(1',; 1 )2//2:83 3267 &$5' )25 0$,1 48(67,211$,5( 6+$1'6 DW WKH 8QLYHUVLW\ RI )ORULGD W 81,9(56,7< 2) )/25,'$ 'UXJ ,QIRUPDWLRQ t 3KDUPDF\ 5HVRXUFH &HQWHU 32 %R[ r *DLQHVYLOOH 3/ 6LGH 2QHf 'HDU &ROOHDJXH $ERXW RQH ZHHN DJR D TXHVWLRQQDLUH VHHNLQJ \RXU RSLQLRQV DERXW WKH VHUYLFH TXDOLW\ RI WKH 'UXJ ,QIRUPDWLRQ DQG 3KDUPDF\ 5HVRXUFH &HQWHU ',35&f ZDV PDLOHG WR \RX ,I \RX KDYH DOUHDG\ FRPSOHWHG DQG UHWXUQHG WKH TXHVWLRQQDLUH WR XV SOHDVH DFFHSW RXU VLQFHUH WKDQNV ,I QRW SOHDVH GR VR WRGD\ %HFDXVH WKH TXHVWLRQQDLUH ZDV VHQW WR RQO\ D VPDOO VDPSOH RI RXU UHFHQW FDOOHUV LW LV H[WUHPHO\ LPSRUWDQW WKDW \RXUV DOVR EH LQFOXGHG LQ WKH VWXG\ LI WKH UHVXOWV DUH WR DFFXUDWHO\ UHSUHVHQW WKH IHHOLQJV RI WKH SURIHVVLRQDOV ZH VHUYH ,I E\ VRPH FKDQFH \RX GLG QRW UHFHLYH WKH TXHVWLRQQDLUH RU LW JRW PLVSODFHG SOHDVH FDOO P\ RIILFH DW f RU HLQDLO PH DW +$7%(5*#&23+($,7+8)/('8 DQG ZLOO JHW DQRWKHU RQH LQ WKH PDLO WR \RX
PAGE 221

$33(1',; 2 5(63216(6 72 0$,1 48(67,21$,5( 0DLQ 4XHVWLRQQDLUH 5HVSRQVHV 5HVSRQVH 6WURQJO\ $JUHH $JUHH 6RPHZKDW $JUHH 1HXWUDO 6RPHZKDW 'LVDJUHH 'LVDJUHH 6WURQJO\ 'LVDJUHH 'RQnW .QRZ 1R 5HVSRQVH 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

PAGE 222

$33(1',; 3 0$,1 48(67,211$,5( :5,77(1 &200(176 4XHVWLRQ 7KH ',35& KDV WKH HTXLSPHQW DQG LQIRUPDWLRQ UHVRXUFHV QHFHVVDU\ WR DQVZHU P\ TXHVWLRQV f>1@HYHU EHHQ WKHUH WR VHHf +DYH QHLWKHU SHUVRQDOO\ VHHP QRU XWLOL]HG HTXLSPHQWLQIR 5HVRXUFHV EXW KDG TXHVWLRQV DQVZHUHG VDWLVIDFWRULO\ 4XHVWLRQ :KHQ FDOO WKH ',35& EDFNJURXQG QRLVH RQ WKHLU HQG LQWHUIHUHV ZLWK P\ DELOLW\ WR FRPPXQLFDWH RYHU WKH WHOHSKRQH f, QHYHU FDOOf )URP D SK\VLFLDQ SUHVXPDEO\ WKH\ KDYH WKHLU RIILFH DVVLVWDQW FDOO IRU WKHPf 4XHVWLRQ :KHQ UHFHLYH ZULWWHQ PDWHULDOV IURP WKH ',35& WKH\ DUH FOHDU DQG HDV\ WR UHDG f1R H[SHULHQFHf 1$ f%XW WKDWfV 2 f ,Q UHVSRQVH WR DQ DQVZHU RIf6RPHZKDW $JUHHff 4XHVWLRQ (PSOR\HHV RI WKH ',35& VSHDN LQ D PDQQHU WKDW LV HDV\ WR XQGHUVWDQG f6RPH HPSOR\HHV IURP $VLD RU 0LGGOH (DVWf KDYH PRUH RI D SUREOHPf f9DULDEOH GHSHQGLQJ RQ VWXGHQWf f, QHYHU FDOOf )URP D SK\VLFLDQ SUHVXPDEO\ WKH\ KDYH WKHLU RIILFH DVVLVWDQW FDOO IRU WKHPf f2Q RQH RFFDVLRQ WKH SHUVRQ DQVZHULQJ SKRQH ZDV YHU\ KDUG WR XQGHUVWDQG GXH WR DFFHQWf

PAGE 223

f7KH SHUVRQ VSRNH ZLWK ZDV GLIILFXOW WR XQGHUVWDQG EXW WKDW GRHV QRW PHDQ HYHU\RQH LV OLNH WKDWf f7KH RFFDVVLRQDO VWXGHQW ZLWK HWKQLF DFFHQWV FDQ EH GLIILFXOW $VLDQ HWFff f)RUHLJQ VWXGHQWV VWUXJJOH D OLWWOH PRUH XQGHUVWDQGDEO\f 4XHVWLRQ :KHQ KDYH D SUREOHP WKH ',35& LV V\PSDWKHWLF DQG UHDVVXULQJ f>1MHYHU KDG RQHf f, DP QRW ORRNLQJ IRU V\PSDWK\ RU UHDVVXUDQFH"f 1$ 4XHVWLRQ 7KH ',35& LV GHSHQGDEOH 2XWVWDQGLQJ :ULWWHQ DERYH DQ n([FHOOHQWn JUDGH RQ WKH TXHVWLRQf 4XHVWLRQ FDQ WUXVW HPSOR\HHV RI WKH ',35& f>&-RPH QRZf $SSDUHQWO\ UHODWLQJ WR WKH UHOHYDQFH RI WKH LWHPf 4XHVWLRQ (PSOR\HH RI WKH ',35& DUH SROLWH f9HU\ kf 4XHVWLRQ IHHO VDIH LQ P\ LQWHUDFWLRQV ZLWK WKH ',35& HPSOR\HHV f>'HSHQGLQJ RQ WKH VWXGHQWV WKHUH WKDW PRQWKf f:KHQ FDQ XQGHUVWDQG ZKDW WKH\ DUH VD\LQJf f1$f 4XHVWLRQ (PSOR\HHV RI WKH ',35& GR QRW NQRZ ZKDW P\ QHHGV DUH f+RZ FRXOG WKH\ LI GRQfW WHOO WKHP"f

PAGE 224

f5HJDUGLQJ TXHVWLRQ 7KH VWDII PHPEHU ZKR WRRN WKH FDOO DQG LQWHUSUHWHG P\ TXHVWLRQ GLG QRW KDYH DQ\ LGHD RI WKH VXEMHFW RI P\ LQTXLU\ 33' DQG DQHUJ\ WHVWLQJ VWHS YV VWHS PHWKRGV IRU 33' QHHGHG WR H[SODLQ DQG VSHOO >RXW@ WKH LQIR IRU KHU ZLVK \RX FRXOG KDYH H[SHULHQFHG SKDUPDFLVWV SURFHVVLQJ WKH LQFRPLQJ TXHVWLRQVf 4XHVWLRQ QHHG FODULI\LQJ 'R \RX PHDQ n8QGHUVWDQG P\ UHTXHVWV IRU VSHFLILF LQIRn RU QHHG UHODWLQJ WR D WLPHO\ UHVSRQVH" $UH \RX WU\LQJ WR WULFN PH" 2I FRXUVH WKH\ GRQnW XQWLO WHOO WKHP 4XHVWLRQ 7KH ',35& GRHV QRW KDYLQJ RSHUDWLQJ KRXUV FRQYHQLHQW WR PH f'R QRW NQRZ \RXU H[DFW KRXUV %XW >WKH\ KDYH@ EHHQ DYDLODEOH ZKHQ ,fYH QHHGHG WKHPf 'R QRW NQRZ KRXUV RI RSHUDWLRQ ,nP RSHQ ZHHN HQGV DQG FHQWUDO VWDQGDUG WLPH >XQWLO@ 30 ,Q FRPPHQW WR D n6RPHZKDW $JUHHn JUDGH RQ WKH TXHVWLRQ 4XHVWLRQ 7KH DPRXQW RI WLPH WKDW LW WRRN WKH ',35& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV DFFHSWDEOH f$V ZDV WKH UHVSRQVH WLPH RQ RXW RI >WKH@ ODVW UHTXHVWV PXFK PRUH WKDQ DGHTXDWH DPD]LQJ HYHQff 4XHVWLRQ ZLVK WKH ',35& FRXOG SURYLGH D TXLFNHU UHVSRQVH WR P\ TXHVWLRQV f>@QO\ LQ RQH RI WHQ FDVHVf f6RPHWLPHV \RX QHHG LQIR TXLFN RWKHU WLPHV LWfV QRW LPSRUWDQWf f, KDYH QRW KDG D SUREOHP LQ WKLV UHJDUGf 6RPHWLPHV DV LW ZDV LQ P\ FDVH WKDW LV LPSRVVLEOH

PAGE 225

4XHVWLRQ 7KH DPRXQW RI WLPH WKDW LW WRRN WKH ',35& WR UHVSRQG WR P\ PRVW UHFHQW TXHVWLRQ ZDV f, FDOOHG EHFDXVH LQIRUPDWLRQ ZDV QRW DYDLODEOH LQ P\ FXUUHQW OLEUDU\f &RPPHQWLQJ RQ D fORQJHU WKDQ H[SHFWHGf UHVSRQVHf f, GRQfW NQRZ KRZ PXFK UHVHDUFK ZDV LQYROYHGf >$@QG H[DFWO\ ZKHQ WKH\ WROG PH 3 6 DOVR FRQWDFWHG WKH /LIH ([WHQVLRQ )RXQGDWLRQ DQG UHFHLYHG DQ DQVZHU ZLWKLQ PLQXWHV EHFDXVH RQH RI WKHLU VWDII NQHZ WKH DQVZHU )ORZHYHU ZDV QRW GLVVDWLVILHG ZLWK WKH ',35& UHVSRQVH VLQFH LW LQYROYHG PHGLFDWLRQ QRW DYDLODEOH LQ WKH 8 6 4XHVWLRQ 7KH RYHUDOO TXDOLW\ RI WKH VHUYLFHV SURYLGHG E\ WKH ',35& LV EHVW GHVFULEHG DV 7KH f([FHOOHQWf UHVSRQVH ZDV FKHFNHG DQG GRXEOH XQGHUOLQHGf f 2XWVWDQGLQJ :ULWWHQ DERYH DQ n([FHOOHQWn JUDGH RQ WKH TXHVWLRQf 4XHVWLRQV 7KH UHVSRQVHV UHFHLYH IURP WKH ',35& DUH XVHIXO WR PH LQ P\ SUDFWLFH 1$ 4XHVWLRQ 7KH UHVSRQVHV UHFHLYH IURP WKH ',35& DUH HVVHQWLDO WR PH LQ P\ SUDFWLFH 1$ 4XHVWLRQ ,W LV LPSRUWDQW WKDW WKH ',35& DQGRU ID[ PH WKH VXSSRUWLQJ GRFXPHQW HJ UHFHQW OLWHUDWXUHf IRU WKHLU DQVZHUV WR P\ TXHVWLRQV f)D[f f6RPHWLPHV EXW QRW DOZD\Vf 'HSHQGV RQ WKH TXHVWLRQ QRW WKH VDPH IRU DOO f>7@KH\ GLG SURYLGH JUHDW >LQIRUPDWLRQ@ RQ RXU SKDUP GUXJ IRU D VHGDWLRQ ZRUNVKRSf

PAGE 226

1$ 4XHVWLRQ 7KH ',35&fV DQVZHUV WR P\ TXHVWLRQV DUH XVHG WR LPSURYH SDWLHQW RXWFRPHV f1$f f f8QOHVV UHTXHVW QRW WR ERWKHU ZKLFK ZLOO EH DW OHDVW 9L WKH WLPHf f>DQG@ QXUVHV UHVSRQVHV WR PHG TXHVWLRQVf 4XHVWLRQ ZRXOG UHFRPPHQG WKLV VHUYLFH WR D FROOHDJXH 5HVSRQGHQW PDUNHG fKLJKO\f DERYH UHFRPPHQG LQGLFDWHG D fKLJKO\ UHFRPPHQGff f, KDYH GRQH VR RQ QXPHURXV RFFDVLRQVf f0RVW GHILQLWHO\ f 4XHVWLRQ $GGLWLRQDO &RPPHQWV ,V WKHUH DQ\WKLQJ HOVH WKDW \RX ZRXOG OLNH WR WHOO XV DERXW \RXU H[SHULHQFHVf ZLWK WKH ',35&" $OVR DQ\ FRPPHQWV \RX ZLVK WR PDNH UHJDUGLQJ KRZ ZH FRXOG LPSURYH RXU VHUYLFH ZLOO EH DSSUHFLDWHG HLWKHU KHUH RU LQ D VHSDUDWH OHWWHU f, DP JUDWHIXO WKDW WKLV VHUYLFH LV DYDLODEOH DV LW VDYHV PH WLPH DQG KHOSV P\ SDWLHQWV 7KDQN \RXf f, DOZD\V ILQG WKH VWDII UHFHSWLYH DQG H[WUHPHO\ KHOSIXO 5HVXOWV DUH DFFXUDWH DQG DVVLVW PH D JUHDW GHDO LQ P\ IRUHQVLF LGHQWLILFDWLRQV JUHDWO\ DSSUHFLDWH WKH IDVW UHVSRQVH WLPH QHFHVVDU\ IRU WKH W\SH RI ZRUN SHUIRUP 7KDQN \RXf f$UH \RX JX\V RQ WKH ZRUOG ZLGH ZHE" ,I VR PD\EH PDNH \RXU HPDLO DGGUHVV DYDLODEOHf f)RU D ILUVW WLPH FDOOHU WR \RXU 'UXJ ,QIRUPDWLRQ 6HUYLFH DP YHU\ SOHDVHG DQG DSSUHFLDWH \RXU VHUYLFHV 0\ FXVWRPHU ZDV IURP RXW RI WKH FRXQWU\ DQG ZDV TXLWH LPSUHVVHG ZLWK WKH LQIR ZH VKDUHG ZLWK KLP .HHS XS WKH JRRG ZRUNf f
PAGE 227

f$W WLPHV WKLQN WKH VWXGHQW DQVZHULQJ WKH >WHOHSKRQH@ GRHV QRW XQGHUVWDQG WKH TXHVWLRQ RU FDQQRW XQGHUVWDQG (QJOLVKf 7KH\ FRXOG LPSURYH WKHLU FRPPXQLFDWLRQ VNLOOV LH OLVWHQ EHWWHU UHVWDWH WKH TXHVWLRQ HWFff f'(35& KDV DOZD\V EHHQ YHU\ KHOSIXO ZLWK DOO RI P\ TXHVWLRQV 7KH TXHVWLRQV DVN DUH XVXDOO\ YHU\ GLIILFXOW DQG REVFXUH DQG DUH UHIHUUHG WR ',35& ZKHQ FDQfW ILQG DQ\WKLQJ ZLWK D EDVLF 0HGOLQH VHDUFK DQG UHVRXUFHV KDYH DYDLODEOH WR PHf f7KH ODVW WLPH XVHG ',35& ZDV IRU LQIRUPDWLRQ DERXW 5HGX[ DQG LWV SRVVLEOH DIIHFW RQ D SDWLHQWfV KDLU ORVV GLG UHFHLYH D SKRQH FDOO EDFN EXW WKH\ ZHUH DOVR WR PDLO VRPH >NLQG@ RI VXUYH\ WKDW ZDVQfW UHFHLYHGf f, IHHO \RXU VHUYLFHV KDYH DOZD\V EHHQ LQYDOXDEOH WR P\ SUDFWLFH ZLWK LPSURYHG SDWLHQW RXWFRPHVf f.HHS XS WKH JRRG ZRUNf f7KH ODVW LQWHUDFWLRQ DQG UHTXHVW ZDV KDQGOHG H[WUHPHO\ ZHOO DQG LQIRUPDWLRQ UHFHLYHG ZDV YHU\ QHFHVVDU\ DQG KHOSHG XV LPPHQVHO\ :H KDYH KRZHYHU QRW UHFHLYHG WKLV NLQG RI VHUYLFH LQ WKH SDVW 6R PDQ\ RI P\ UHVSRQVHV ZHUH WHPSHUHG E\ SDVW LQWHUDFWLRQV 7KH ODVW UHTXHVW FRXOG QRW KDYH EHHQ KDQGOHG PRUH DSSURSULDWHO\ RU SURIHVVLRQDOO\f f+H\ 'U +DWWRQ DQG 3URIHVVRU 'RHULQJ
PAGE 228

f7KDQN \RX IRU KHOSLQJ PH WR fORRN PRUH NQRZOHGJHDEOHf WR RWKHU KHDOWK FDUH SURIHVVLRQDOV DQG SDWLHQWV WR@ LGHQWLI\ KLPKHUVHOI ZKHQHYHU KHVKH UHVSRQGV WR D FDOO 6RPHWLPHV FDOOHU ILQGV KLPVHOI YHU\ DZNZDUG LQ DVNLQJ WKH QHZ UHVSRQGHU ZKR WKH ILUVW RQH ZDV DQG DOO WKH JRRG LQWHQWLRQ ZDV ORVWf f*R *DWRUVf f7KH\ DUH QRW D UHVRXUFH LQ SHQWLQHDO GLDO\VLV GUXJV DQG WKLV ZRXOG EH KHOSIXOf f, WKLQN LW ZRXOG EH KHOSIXO LI WKH ',53& VWDII DVNHG ZKDW VRUW RI GRFXPHQWDWLRQ LV QHHGHG KDYH KDG PDQ\ SRVLWLYH LQWHUDFWLRQV ZLWK WKH ',35& EXW KDG RQH LQVWDQFH LQ ZKLFK ZDV SURYLGHG ZLWK QR VXSSRUWLQJ GRFXPHQWV ZKHQ H[SHFWHG VRPHf f8QIRUWXQDWHO\ WKH ',35& ZDV XQDEOH WR SURYLGH PH WKH YHU\ VSHFLILF LQIRUPDWLRQ UHTXHVWHG DQG ZDV IRUFHG WR UHTXHVW LW GLUHFWO\ IURP WKH GUXJ FRPSDQ\ $GPLWWHGO\ WKH LQIR UHTXHVWHG ZDV GHWDLOHG DQG QHZ UHJDUGLQJ ODWH[ FRQWHQW RI PDQ\ GUXJV 7KDQNVf f$V D IRUHQVLF FKHPLVW ZRUNLQJ IRU WKH 6WDWH RI )ORULGD )'/(f WKH\ VXUYH\ GRHV QRW UHDOO\ DSSO\ EXW DP JUDWHIXO IRU WKH VHUYLFH ZRXOG EH ZLOOLQJ WR VXSSO\ LQIRUPDWLRQ QRW LQ \RXU FRPSXWHUV DV WR WKH LGHQWLW\ RI WDEOHWV HWFf f7KH RQO\ SUREOHP KDYH HYHU KDG ZDV WKH LQDELOLW\ WR XQGHUVWDQG WKH SHUVRQ DQVZHULQJ WKH SKRQH DQG JHWWLQJ KLP WR XQGHUVWDQG PH 7KLV RQO\ RFFXUUHG RQFH 7KLV ZDV GXH WR ODQJXDJH EDUULHUf f, KDYH KDG YHU\ SRVLWLYH H[SHULHQFHVf f, DP D GHSXW\ VKHULII DQG XVHG \RXU VHUYLFH WR LGHQWLI\ VRPH SUHVFULSWLRQ GUXJV FRQILVFDWHG RII D>Q@ DUUHVWHH
PAGE 229

f, UHDOO\ OLNH WKH IHDWXUH RI KDYLQJ ',35& )D[ WKH DQVZHUV WR WKH 'UfV5QfV$513fV HWF HVSHFLDOO\ ZKHQ WLPH LV RI WKH HVVHQFH 2FFDVLRQDOO\ ZRXOG OLNH D V\QRSVLV RI WKH DQVZHU WR WKH TXHU\ IRU P\ RZQ LQIR %XW SUREDEO\ OHVV WKDQ 9 WKH WLPHf f ZRQGHU LI LW ZRXOG EH SRVVLEOH WR JHW H[WHPSRUDQHRXV FRPSRXQGLQJ KHOS >WKURXJK@ WKH ',53& RQ D IDLUO\ UDSLG WLPHWDEOH" JRW XVHG WR KDYLQJ DFFHVV WR WKH KHOS GHVN DW 3&&$ RXW LQ 7H[DV ZKLOH SXUFKDVLQJ HQRXJK IURP WKHP WR ZDUUDQW WKH VHUYLFH IRU \HDUV UHFHQWO\f f
PAGE 230

f0RVW IUHTXHQWO\ P\ QHHGV DUH UHODWHG WR FRVW DQG HQRXJK LQIRUPDWLRQ DERXW D GUXJ WR MXVWLI\ LWfV XVH WR DQ LQVXUDQFH FRPSDQ\ $ QHJDWLYH RXWFRPH FDQ PHDQ WKDW D SDWLHQWfV WUHDWPHQW SODQ FRXOGZRXOG EH FKDQJHG WR D OHVV RSWLPDO FRXUVH RI WUHDWPHQWf f, DSSUHFLDWHG WKH LQIRUPDWLRQ UHFHLYHG EXW LW ZDV PXFK PRUH WHFKQLFDO WKDQ ZDQWHG ,W ZRXOG EH KHOSIXO WR KDYH D EURDGHU UDQJH RI UHIHUHQFHV 7KH SHUVRQ VSRNH ZLWK ZDV YHU\ ZLOOLQJ WR KHOSf f4XHVWLRQV5HVSRQVHV YLD HOHFWURQLF PDLO RU D :HE VLWH VHWXS ZRXOG EH D QLFH DGGLWLRQf f1HYHU XVHG 8)f f, KDYH EHHQ LPSUHVVHG ZLWK DOO DVSHFWV RI DVVLVWDQFH FRXUWHV\ DQG WLPHOLQHVV >1DPH RPLWWHG@ ZDV YHU\ KHOSIXOf f*UHDW -RE 3OHDVH FRQWLQXH 7KDQNVf f.HHS XS WKH JRRG ZRUNf f6RPHWLPHV LW LV GLIILFXOW WR XQGHUVWDQG WKH ',35& VWXGHQWV ZKR DQVZHU WKH SKRQH DQG WKH\ UHTXLUH UHSHDWLQJ WKH UHTXHVW VHYHUDO WLPHV EHIRUH WKH\ XQGHUVWDQG LWf f,I FDOOHU FRXOG EH DVVLVWHG LQ SULRULWL]LQJ TXHVWLRQV DSSURSULDWHO\ VRPHKRZ ZRUNORDG FRXOG EH HYHQHG RXW LH LV SDWLHQW EOHHGLQJ" RU LQ DQ H[DP URRP" RU ZRQfW EH XQWLO QH[W ZHHNfV DSSRLQWPHQWf f,fP D FDOOHU IURP DQRWKHU ',& LQTXLULQJ DERXW IRUPXODU\ VWDWXV RI IRVSKHQ\WRLQ ,QIRUPDWLRQ SUHVHQWHG ZDV FRQFLVH DFFXUDWH DQG WLPHO\f f1R SUREOHPV ZLWK ',53& GR P\ RZQ LQLWLDO UHVHDUFK LQWR SUREOHPV ZKHQ H[KDXVWHG XVH ',35& DV fODVW UHVRUWf WR VROYH SUREOHP *LYHQ WKDW VFHQDULR DQ\ DQVZHU WR D SUREOHP LV ZHOFRPHf f7KLV LV D ZRQGHUIXO VHUYLFH \RX SURYLGH ,V WKH VHUYLFH OLPLWHG WR MXVW 8) RU LV LW DYDLODEOH FRXQW\ ZLGH" QHYHU NQRZ \RX H[LVWHG DV D UHVRXUFH 7KDQNV VR PXFKf f5HVSRQVH WLPH ZDV IDQWDVWLF DQG PRVW EHQHILFLDOf f7KH HPSOR\HH ZDV YHU\ KHOSIXO DQG ZLOOLQJ WR ORRN IRU DQVZHUVf f7KDQNV IRU WKH ZRQGHUIXOFOHDUFRQFLVH LQIR RQ WKH GUXJV ZH UHTXHVWHG >1DPH RPLWWHG@ ZDV YHU\ KHOSIXOf f$OORZ IRU VXEPLVVLRQ RI TXHVWLRQV YLD HPDLOf

PAGE 231

f'RQfW PDNH VXEVFULEHUV JLYH \RX RXU GHPRJUDSKLF GDWD HDFK WLPH ZH FDOO 7KH@ SHUVRQ ZKR KHOSHG PH ZDV YHU\ SROLWH DQG KHOSIXO k 7KH SUREOHP ZLWK WKH VORZ WXUQ DURXQG WLPH RI P\ ODVW TXHVWLRQ ZDV WKH UHVXOW RI RQH QHZ VWXGHQWnV ODFN RI H[SHULHQFH 7KH TXHVWLRQ ZDV PRUH RI D FRQILUPDWLRQ RI LQIRUPDWLRQ DQG WKH WXUQDURXQG WLPH ZDV QRW FULWLFDO

PAGE 232

LQIRUPDWLRQ@ SUHSDUHG E\ D VWXGHQW RU D SUDFWLWLRQHU &RXUWHRXV SURPSW LQWHUHVWHG LQ UHTXHVWV 6HUYLFH LV YHU\ PXFK DSSUHFLDWHG DQG KDYH UHFRPPHQGHG LW WR RWKHUV 9HU\ PXFK DSSUHFLDWHG $QRWKHU GUXJ LQIRUPDWLRQ VHUYLFH GLG D nEDVK WKH SKDUPDFLVWn UHVSRQVH DQG QHYHU DQVZHUHG WKH TXHVWLRQ
PAGE 233

$33(1',; 4 6,08/$7,21 %/2&. ',$*5$06

PAGE 234

(17(5 f§ /($9( /,1(6? 6(/(&7 t6P5,1*287 9678'f $66,*1

PAGE 239

$33(1',; 5 6,08/$7,21 352*5$0 &2'( *366+ 352)(66,21$/ 5(/($6( Q& 8*f 2FW ),/( GLVJSV /,1( ,) '' %/2&. r/&& 23(5$7,21 $ % & ( ) &200(176 r 5($//2&$7( &20 6,08/$7( 6WDUW *366+ -RE I fNNNNNNNNNLHNNNNNNNNnLUNN LUN NNNNN ‘NLFNLFNnNnNnNLLLWLWnNnN rrrrrrrrr rrrrrrrnrrrrrrrnrr r ,1,7,$/,=( 1(&(66$5< 6725$*(6 9$5,$%/(6 $1' )81&7,216 r 4 fN f"frn fr"rrfrf fNLUnNnNLFLILINnNnLINnLILILnNnNnNNnLLUNLIMFNnNnNnNnNNnNNNNNnNNNNNN,FNNNNNnNnNnNnNnNnNnNnNnNnNnNnNnN ,17(*(5 t, t50 t69 6' t67' 6&/ ,QLWLDOL]H 9DULDEOHV ,17(*(5 669& t$0& t670& 6/7 5($/ t&/.0,1 t$9$5 t'$ 67$ t+&85 5($/ L$9,6 t'/&7 t'/3& $0 t6,0 5($/ t693) t$987 t74t747 6W%47 72,$/4 (44 ,QLWLDOL]H 4XHXH 1XPEHUV $16:(54 (44 %&$5(4 (44 6(59,&(4 (48 4 5(78514 (24 :$,7694 (44 /,1(6 6725$*( 6HW 1XQEHU RI 3KRQH /LQHV 678' 9$5,$%/( 51rt69f 9DULDEOH 6WXGHQW 6HOHFWLRQ &(/$< %9$5,$%/( &3 f!f t'f§3f %RROHDQ 9DULDEOH 67$7 'HOD\ '(/$< %9$5/$%/( t'3f! %RROHDQ 9DULDEOH 7RGD\ 'HOD\ '(/$< %9$5,$%/( ^t'3f! %RROHDQ 9DULDEOH 'DWH 'HOD\ 6(77<3 )81&7,21 3' 5RXWLQJ )XQFWLRQ 4XHVWLRQ 7\SH 47<3(47<3( 47<3( 47<3( %86< )81&7,21 51& )XQFWLRQ :DLW 0LQXWHV 5(02+. )81&7,21 51& )XQFWLRQ &KHFN RQFH SHU KRXU (1769& )81&7,21 3' 5RXWLQJ )XQFWLRQ %HJLQ $QVZHULQJ 7(15 7(15 7&15 67$7 47<3( (81&7,&1 51 47\SH *HQHUDWRU O *URXS O *URXS *URXS IRU 6WDW 4XHVWLRQV 47<3( )81&7,21 51 47\SH *HQHUDWRU O *URXS O *URXS *URXS n IRU 7RGD\ 4XHVWLRQV 47<3( )81&7,21 51' 47\SH *HQHUDWRU O *URXS O *URXS *URXS IRU 'DWHG 4XHVWLRQV 47<3( )81&7,21 51' 47\SH *HQHUDWRU O *URXS O *URXS *URXS IRU 1R 5XVK 4XHVWLRQV 48(67 )81&7,21 51' )XQFWLRQ 3ULRULWLHV IRU n 48(67 (81&7,&1 51' )XQFWLRQ 3ULRULWLHV IRU

PAGE 240

&8(67 (81&7,&1 51' (8QFWLRQ 3ULRULWLHV IRU 4-(67 )81&7,21 51 )XQFWLRQ 3ULRULWLHV IRU 4-(67 )81&7,21 51 (8QFWLRQ 3ULRULWLHV IRU 4-(67 (81&7,&1 51' )XQFWLRQ 3ULRULWLHV IRU 4-(67 (81&7,&1 51 (8QFWLRQ 3ULRULWLHV IRU 4-(67 (81&7,&1 51' (8QFWLRQ 3ULRULWLHV IRU ,$7 (81&7,&1 51& (8QFWLRQ ,$7V IRU ,$7 )81&7,21 51& (8QFWLRQ ,$7V IRU ,$7 (81&7,&1 51& (8QFWLRQ ,$7V IRU ,$7 )81&7,21 51& )XQFWLRQ ,$7V IRU ,$7$(7 (81&7,&1 51& (8QFWLRQ ,$7V IRU $IWHUQRRQ *5&83 (81&7,&1 51& (8QFWLRQ 6HUYLFH 7LPHV IRU 4XHVWLRQ 7\SH *5&83 (81&7,&1 51& 4 (8QFWLRQ 6HUYLFH 7LPHV IRU 4XHVWLRQ 7\SH *5&83 (81&7,&1 51& (8QFWLRQ 6HUYLFH 7LPHV IRU 4XHVWLRQ 7\SH 67$7,0 (81&7,&1 51&6 (8QFWLRQ 6HUYLFH 7LPHV IRU 6WDW &DOOV 7&0,1 (81&7,&1 51& (8QFWLRQ 6HUYLFH 7LPHV IRU 7DNLQJ &DOOV 5,0,1, (81&7,&1 51& (8QFWLRQ 5HWXUQ &DOO 7LPH IRU *URXS

PAGE 241

570,1 (81&7,&1 51& 5,0,1 (81&7,&1 51& 5767$7 (81&7,&1 51& )XQFWLRQ 5HWXUQ &DOO 7LPH IRU *URXS )XQFWLRQ 5HWXUQ &DOO 7LPH IRU *URXS )XQFWLRQ 5HWXUQ &DOO 7LPH IRU 6WDW 4XHVWLRQV rrr r rrrrrrrrrrrrrrrrrrrrrrr r rr r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r r r r r *(1(5$7( &$//6 r rrrrrrrrrrr rrrr rr rrrrr r rrr -U rrrrrrrrrrr r r r r r rrrr r r r r r r r r r r rr r r r r fLI fNLUN r *(1(5$7( (1,$7fr6$0f 6&/rf 3+ *HQHUDWH &DOOHUV )URP 7(67 &O t&/rf3$67&/. 7(67 /( &Ot&/rf3$67&/. $66,*1 (1 4-(67f 75$1((5 &$//(5 *(1(5$7( (1,$7frt$0f t&/rf 3+ *HQHUDWH &DOOHUV )URP 7(67 *( &O t&/rf 3$67&/. 7(67 /( &O t&/r8f 3$67&/. $66,*1 (1 48(67f 75$16)(5 &$//(5 *(1(5$7( (1,$7fr6$0f t&/rf 3+ *HQHUDWH &DOOHUV )URP 7(67 *( &O t&/0f3$67&/. 7(67 /( &O t&/rf3$67&/. $66,*1 (14-(67f 75$16)(5 &$//(5 *(1(5$7( (1,$7frt30f t&/rf 3+ *HQHUDWH &DOOHUV )URP 7(67 *( &O t&/rf3$67&/. 7(67 /( &O t&/rf3$67&/. $66,*1 )1 4-(67f 75$16((5 &$//(5 *(1(5$7( )1 ,$7$(7f rt$0f 6&/rf 3+ *HQHUDWH $IWHUQRRQ &DOOHUV 7(67 *( &O t&/0f 3$67&/. t 6KXW 'DZQ 3KRQHV 7(67 /( &O t&/rf 3$67&/. 7,0( 7(67 / &O t&/rf7,0( $66,*1 )1 4-(67f 75$16)(5 &$//(5 7,0( 7(67 / &Ot&/rf7,0( $66,*1 (14-(67f 75$16((5 &$//(5 7,0( 7(67 / &Ot&/rf7,0( $66,*1 )1 48(67f 75$16)(5 &$//(5 7,0( $66,*1 )14-(67f 75$16((5 &$//(5 3$67&/. 7(50,1$7( 7HUPLQDWH H[WUD DUULYDOV IURP RWKHU *(1 EORFNV r r r r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r &$// $55,9(6 r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr &$//(5 $66,*1 $VVLJQ 1R RI 7ULHV 35,25,7< ,QFRPLQJ &DOOV +DYH +LJKHVW 3ULRULW\ 75<3+1 75$16)(5 %27+3+1/1(%86< 7HVW 3KRQH /LQHV 3+1/1( (17(5 /,1(6 3KRQH FDOO DUULYHV 75$16)(5 6(/678' %86< %/(7 3+ 3+ 7(67 35(3($7 7(67 35(3($7 7U\ WZR WLPHV %$/. 7(50,1$7( %DON 3KRQH LV %XV\ rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr

PAGE 242

r 6(/(&7 678'(17 72 $16:(5 &$// r rrrrrr rrr rrfrrfrr‘ rrrrrrrrr r UN r rrrrrrrrrrrrrrrrrrrrrr rrrrr rrrrrrrrrrrr r‘ 6(/678' 6(/(&7 /5 t69 5,1*&87 &KHFN LI D 6WXGHQW $YDLO WR 7DNH &DOO 6(/678' $66,*1 9678'f 5DUHGRQW\ $VVLJQ D 6WXGHQW *$7( /5 36(/6,8' &KHFN WR VHH LI 6WXGHQW LV RQ SKRQH 75$16)(5 7$.(&$// ,I QRW RQ SKRQH VWXGHQW WDNHV FDOO 5,1*&87 /($9( /,1(6 (YHU\RQH LV %XV\ %/(7 3+ 3+ 7(67 35(3($7 7(67 35(3($7 7U\ WZR WLPHV %$/. 7(50,1$7( %DON 1R DYDLODEOH WR DQVZHU SKRQH rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r 3+&1( :$6 %86< 25 12 &1( $16:(5(' r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr 5(3(.7 $'9$1&( :DLW RQH PLQXWH WKHQ FDOO DJDLQ 75$16((5 75<3+1 5(3($7 $'9$1&( (1 %86
PAGE 243

rrrrrrrrrr(71' :+2n6 $9$,/$%/( 72 :25. &1 48(67,21rrrrrrrrrrrrrrrrr $96,8' 7(67 ( 43f$9678' &KHFN LI 6WXGHQW ZKR DQVZHUHG LV DYDLODEOH 75$1)(5 (1769& $96,8' 6(/(&7 ( t694$9678' &KHFN LI D 6WXGHQW $YDLO WR 7DNH &DOO 75$16((5 (1769& $9678' 7(67 3$9678' *HW )LUVW $YDLODEOH 6WXGHQW ,I 6WDW $9678' 75$16)(5 %27+ )677+67 )67 35((037 3, 35 7U\ $QVZHULQJ 6WXGHQW )LUVW $'9$1&( 5(7851 3, 75$16((5 (1769& &7+67 $66,*1 t69 7+-7 75$16)(5 %27+ /&23 7KHQ 7U\ $OO 2WKHUV 35((037 3 35 $'9$1&( 5(7851 3 $66,*1 3 75$16((5 (1769& /&3 /223 27+67 $'9$1&( /RRS 8QWLO 6WXGHQW %HFRPHV $YDLODEOH 75$16((5 $9678' $9678' 75$16((5 (1769& b RI 7LPH 6WXGHQW $QVZHULQJ :RUNV 4XHVWLRQ %/(7 t67' 9 678'f 2WKHUZLVH 5DQGRPO\ $VVLJQ D 6WXGHQW 7(67 1( 3,t67'$9678' %/(7 3+ t67' (1769& 75$16((5 (1 (1769&f rrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r rrr r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r ZRUN RQ TXHVWLRQ r rrrrrrrrrr rrr r rrrrrrrrrrr rrrrrrrrrrrrrrrrr r rr rrrrrr rrrrrrrrrrrrrrrr r 67$7 $66,*1 (167$770f 7&15 35((037 3, 35 48(8( 3, '(3$57 %&$5%4 48(8( 6(.9,&(4 $'9$1&( 3rt670 :RUN RQ 4XHVWLRQ 5(7851 3, '(3$57 6(59,&(4 'EQH $QVZHULQJ 4XHVWLRQ 75$16((5 :$,7571 rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r 5(7851 $16:(5 r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr :$,7571 48(8( 5(78514 7UDFN 7LPH WR 5HWXUQ &DOO 574 7(67 ( 3574 5HWXUQ 6WDW &DOOV ,UUPHGLDWHO\ 35,25,7< 75$16((5 &13+1 574 7(67 ( 3574 35,25,7< )LQLVK :RUNLQJ RQ D 6WDW 4XHVWLRQV )LUVW 75$16)(5 &13+1 574 35,25,7< 5HWXUQ (ODWH DQG 1R 5XVK $QVZHUV $V 7LPH $OORZV &13+1 *$7( /5 3, 0DNH 6XUH 6WXGHQW LV 1RW $QVZHULQJ &DOO 35((037 3, 35 5HWXUQ $QVZHU (17(5 /,1(6 /2*,& 6 3, 2Q 3KRQH '&1( 7(67 ( 3'&1( $'9$1&( (15767$7f 75$16((5 5(7$16 '&1( 7(67 ( 3 (&1( 7DON WR &DOOHU $'9$1&( (1 5,0,1,f 75$16((5 5(7$16 '&1( 7(67 ( 3 '&1( $'9$1&( (15,0,1f 75$16)(5 5(7$16 '&1( $'9$1&( (1 570,1f

PAGE 244

5(7$16 /($9( /,1(6 /2*,& 5 3, 5(7851 3, '(3$57 5(78514 '(3$57 3, '(3$57 727$/4 +DQJ 8S 3KRQH rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r ),1,6+(' &RXQW 'HOD\V DQG /RJRXW r fNnNnNnNnNnNnNnNnNnNnNnNnNnLUnLWnNnNnNnOILUNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnIFnNnNnNnNnNnNnWHnOLnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnNnN '/67$7 7(67 ( 3 '/72'$< 7UDFN 'HOD\V $66,*1 %9'(,$
PAGE 245

/(7 '$ 67$57 13 (1''2 5(6(7 /(7 7$ &OHDU 9DULDEOHV IRU 1H[W 5XQ /(7 t&/ /(7 '/&3 /(7 6'/322 508/7 r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 6HW ,WK QRQDQWLWKHWLF 51 6HHG r 6HW OWK QRQDQWLWKHWLF 51 6HHG r 3873,& /,1(6 (,,e 6<635,17 69 670 f N r1&1317,7+(7,& 9$5,$7(6 6(59(56rr 67 02',),(5 rrrr 5(3/,&$7,21 rr fN '2 /(7 &/ /(7 'IW 67$57 13 /(7 $9,6 4$727$/4f r )873,& /,1(6 ),/( 6<635,17 U t' t&$ t7$ t$9,6f r1rr 'D\ rr $UULYDOV rr 7WO$UULYDOV rrrr $YJ&RQWHQWV f NaN NN (1''2 /(7 t&/t' /(7 t'$ 67$57 13 /(7 $987 (5)5)5(5(5f 69 /(7 '/3& '/&71 /&*&87f /(7 &/.0,1& /(7 $9$5I 7$' /(7 7 4577$,-f /(7 74,.7727$/4f /(7 %&1 4$%4$5'4f /(7 %4/ 47%&$5'4f /(7 ,&*,A 1 /2*287f )873,& 19 rrr r N r1rr 'D\ N r1R&EQS r r&/&&. NNNNNNNNNNn r 5XQ IRU 3-73,& )873,& )873,& /,1(6 ),/( 6<635,17 ,''$7$$9,6f /,1(6 (/( 6<635,17 1 /&*&87f '/&7 '/%& $9$5f D\HG rrr 3HW'HOD\HG rrrr $YJ'D\ rrrr 81(6 ),/( 6<635,17 &/.0,147 6(59,&(4f 693)f &/($5 &OHDU IRU $QWLWKHWLF 5HSOLFDWLRQ ‘N N NNN NNNN N N N N N N N N N N N NN NN N N frf‘rr N N N N N N N N N NNNNN N N NN F N NNNN NN NN N N N NNN N N NN N N N N N N N N N NNN N N N N N N N N N N N N N N N NNN NNNN NNNN N NNNN N N N NN N N NN N N N N

PAGE 246

'2 650 :DUP 8S IRU DQWLWKHWLF UXQ 508/7 r650 (1''2 '2 650 508/7 r 650f (1'&2 '2 6' /(7 6&' t' /(7 6'$ 67$57 13 (1''2 5(6(7 /(7 67IW &OHDU 9DULDEOHV IRU 1H[W 5XQ /(7 6&' /(7 6'/&7 /(7 6'/.! 508/7 r6, 6HW t,WK QRQDQWLWKHWLF 51 6HHG r6, 6HW t,WK QRQDQWLWKHWLF 51 6HHG r6, 6HW t,WK QRQDQWLWKHWLF 51 6HHG r6, 6HW t,WK QRQDQWLWKHWLF 51 6HHG r6, 6HW t,WK QRQDQWLWKHWLF 51 6HHG r6, 6HW t,WK QRQDQWLWKHWLF 51 6HHG f§ r6Of 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r 6,f 6HW t,WK DQWLWKHWLF 51 6HHG 4r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r6,f 6HW t,WK DQWLWKHWLF 51 6HHG r 6,f 6HW t,WK DQWLWKHWLF 51 6HHG r (873,& /,1(6 ),/( 6<635,17 t69t670U t,f N r1&1$17,7+(7,& 9$5,$7(6 6(59(56rr 67 02',),(5 rrrr 5(3/,&$7,21 rr r '2 62, /(7 t&/ t' /(7 t($I 67$57 13 /(7 t$9,6 4$ 727$/4f r (873,& /,1(6 OI),/( 6<635,17 t, I t' t'$ t7$ 6$9(6f r1rr 'D\ rr $UULYDOV rr 7WO$UULYDOV rrrr $YJ&RQWHQWV rrrr (1''2 /(7 6&26'O /(7 VFD R 67$57 13 /(7 6$987 )5(5(5(5(5f 669 /(7 6 '/.2t'/&71 ,&*2-7f /(7 6&/.0,1 & /(7 t$9$5 t7$t' /(7 t72A 4$ 727$/4f /(7 t747 47 7&7$/4f /(7 t(&0 4$(&$5'4f /(7 t%47 47%&$5'4f /(7 t/&*7 1/&*&87f (873,& /,1(6 (,/26<6 35,17 t6,0 669 t, t$987 67p t74, 6%p t(47 67$ 6'&*7 t''&7 t'/3& t693)f $9 fN NN N NN NNNN NNN NNN NN NNNN NN NNN NN NNNN NNNN r1rr &t\ N r1R&RUUS )873,& /,1(6 (7/( 6<635,17 6, 6't'$ 67$ 6$9,6f rr $UULYDOV rr 7WO$UULYDOV rrrr $YJ&RQWHQWV rrrr )873,& /,1(6 ),/% 6<635,17 1/&;&87f6'/&,6'/3&6$9$5f rrr 1R'DOD\HG rrr 3HW 'HOD\HG rrrr $YJ'D\ rrrr )873,& /,1(6 (7/26<635,17 6&/.0,1 47 6(59,&(4f 6693)f NNN NNN

PAGE 247

r &/2&. rrrrr &/($5 (1'(' (1''' (1''2 (1' $YJ6HUYLFH ‘NNN NN $YJ6(593(5) rrrrr &OHDU IRU QH[W LWK LWHUDWLRQ

PAGE 248

/,67 2) 5()(5(1&(6 $QJDUDQ 0 f 7KH 7ULDG LQ 4XDOLW\ $VVHVVPHQW 6WUXFWXUH 3URFHVV DQG 2XWFRPH $GPLQLVWUDWLYH t 3UDFWLFH 7UHQGV LQ 3KDUPDFHXWLFDO &DUH f %DEDNXV ( t %RLOHU : f $Q (PSLULFDO $VVHVVPHQW RI WKH 6(5948$/ 6FDOH -RXUQDO RI %XVLQHVV 5HVHDUFK 0DYf %DEDNXV ( t 0DQJROG : f $GDSWLQJ WKH 6(5948$/ 6FDOH WR +HDOWK &DUH (QYLURQPHQW $Q (PSLULFDO $VVHVVPHQW ,Q 3 %ORRP 5 :LQHU + + .DVVDUMLDQ t / 6FDPPRQ (GVf (QKDQFLQJ .QRZOHGJH 'HYHORSPHQW LQ 0DUNHWLQJ SS f &KLFDJR ,/ $PHULFDQ 0DUNHWLQJ $VVRFLDWLRQ %DEDNXV ( t 0DQJROG : f $GDSWLQJ WKH 6(5948$/ 6FDOH WR +RVSLWDO 6HUYLFHV $Q (PSLULFDO ,QYHVWLJDWLRQ +65 +HDOWK 6HUYLFHV 5HVHDUFK f %DOG 2 f +RZ WR $VVHVV WKH $FFHSWDEOLOLW\ DQG &UHGLELOLW\ RI 6LPXODWLRQ 5HVXOWV ,Q ( $ 0DF1DLU 0XVVHOPDQ t 3 +HLGHOEHUJHU (GV f 3URFHHGLQJV RI WKH :LQWHU 6LPXODWLRQ &RQIHUHQFH SS f :DVKLQJWRQ '& %HDLUG 6 / &ROH\ 0 5 t &UHD $ f &XUUHQW 6WDWXV RI 'UXJ ,QIRUPDWLRQ &HQWHUV $PHULFDQ -RXUQDO RI +RVSLWDO 3KDUPDF\ -DQf %LWQHU 0 f (YDOXDWLQJ 6HUYLFH (QFRXQWHUV 7KH (IIHFWV RI 3K\VLFDO 6XUURXQGLQJV DQG (PSOR\HH 5HVSRQVHV -RXUQDO RI 0DUNHWLQJ $SULOf %RHUVWOHU + )RVWHU 5 : 2n&RQQRU ( 2n%ULHQ / 6KRUWHOO 6 0 &DUPDQ 0 t +XJKHV ( ) ; f ,PSOHPHQWDWLRQ RI 7RWDO 4XDOLW\ 0DQDJHPHQW &RQYHQWLRQDO :LVGRP YHUVXV 5HDOLW\ +RVSLWDO DQG +HDOWK 6HUYLFHV $GPLQLVWUDWLRQ f %ROWRQ 5 1 t 'UHZ + f $ /RQJLWXGLQDO $QDO\VLV RI WKH ,PSDFW RI 6HUYLFH &KDQJHV RQ &XVWRPHU $WWLWXGHV -RXUQDO RI 0DUNHWLQJ -DQXDU\f %ROWRQ 5 1 t 'UHZ + f /LQNLQJ &XVWRPHU 6DWLVIDFWLRQ WR 6HUYLFH 2SHUDWLRQV DQG 2XWFRPHV ,Q 5 7 5XVW t 5 / 2OLYHU (GV f 6HUYLFH 4XDOLW\ 1HZ 'LUHFWLRQV LQ 7KHRU\ DQG 3UDFWLFH SS f 7KRXVDQG 2DNV &$ 6DJH 3XEOLFDWLRQV

PAGE 249

%RXOGLQJ : .DOUD $ 6WDHOLQ 5 t =HLWKDPO 9 $ f $ '\QDPLF 3URFHVV 0RGHO RI 6HUYLFH 4XDOLW\ )URP ([SHFWDWLRQV WR %HKDYLRUDO ,QWHQWLRQV -RXUQDO RI 0DUNHWLQJ 5HVHDUFK ;;;)HEUXDU\f %URZQ 6 : t 6ZDUW] 7 $ f $ *DS $QDO\VLV RI 3URIHVVLRQDO 6HUYLFH 4XDOLW\ -RXUQDO RI 0DUNHWLQJ $SULOf %URZQ 7 &KXUFKLOO $ -U t 3HWHU 3 f ,PSURYLQJ WKH 0HDVXUHPHQW RI 6HUYLFH 4XDOLW\ -RXUQDO RI 5HWDLOLQJ %X[WRQ 9 t *DWODQG 5 f 6LPXODWLQJ WKH (IIHFWV RI :RUN,Q3URFHVV RQ &XVWRPHU 6DWLVIDFWLRQ LQ D 0DQXIDFWXULQJ (QYLURQPHQW ,Q & $OH[RSRXORV .DQJ : 5 /LOHJGRQ t *ROGVPDQ (GVf 3URFHHGLQJV RI WKH :LQWHU 6LPXODWLRQ &RQIHUHQFH SS f $UOLQJWRQ 9$ &DUPDQ 0 f &RQVXPHU 3HUFHSWLRQV RI WKH 6HUYLFH 4XDOLW\ $Q $VVHVVPHQW RI WKH 6(5948$/ 'LPHQVLRQV -RXUQDO RI 5HWDLOLQJ 6SULQJf &DUUXWKHUV 0 ( f &RPSXWHU $QDO\VLV RI 5RXWLQH 3DWKRORJ\ :RUN 6FKHGXOHV 8VLQJ D 6LPXODWLRQ 3URJUDPPH -RXUQDO RI &OLQLFDO 3DWKRORJ\ &DUVRQ 6 ,, f 9HULILFDWLRQ DQG 9DOLGDWLRQ $ &RQVXOWDQWnV 3HUVSHFWLYH ,Q ( $ 0DF1DLU 0XVVHOPDQ t 3 +HLGHOEHUJHU (GVf 3URFHHGLQJV RI WKH :LQWHU 6LPXODWLRQ &RQIHUHQFH SS f :DVKLQJWRQ & &KLQ 9 t 6SUHFKHU 6 & f 8VLQJ D 0DQXIDFWXULQJ %DVHG 6LPXODWLRQ 3DFNDJH WR 0RGHO D &XVWRPHU 6HUYLFH &HQWHU ,Q 2 %DOFL 5 3 6DGRZVNL t 5 ( 1DQFH (GVf 3URFHHGLQJV RI WKH :LQWHU 6LPXODWLRQ &RQIHUHQFH 3URFHHGLQJV SS f 1HZ 2UOHDQV /$ &KULVWHQVHQ % t 3HQQD 3 0 f 4XDOLW\ $VVHVVPHQW DQG 4XDOLW\ $VVXUDQFH RI 3KDUPDF\ 6HUYLFHV -RXUQDO RI 0DQDJHG &DUH 3KDUPDF\ f &OHPPHU ( & t 6FKQHLGHU % Df 7RZDUG 8QGHUVWDQGLQJ DQG &RQWUROOLQJ &XVWRPHU 'LVVDWLVIDFWLRQ ZLWK :DLWLQJ 'XULQJ 3HDN 'HPDQG 7LPHV ,Q 0 %LWQHU t / $ &URVE\ (GVf 'HVLJQLQJ D :LQQLQJ 6HUYLFH 6WUDWHJ\ SS f &KLFDJR ,/ $PHULFDQ 0DUNHWLQJ $VVRFLDWLRQ &OHPPHU ( & t 6FKQHLGHU % f 0DQDJLQJ &XVWRPHU 'LVVDWLVIDFWLRQ ZLWK :DLWLQJ $SSO\LQJ 6RFLDO3V\FKRORJLFDO 7KHRU\ LQ D 6HUYLFH 6HWWLQJ $GYDQFHV LQ 6HUYLFHV 0DUNHWLQJ DQG 0DQDJHPHQW &OHPPHU (& t 6FKQHLGHU % Ef 7RZDUG 8QGHUVWDQGLQJ DQG &RQWUROOLQJ &XVWRPHU 'LVVDWLVIDFWLRQ :LWK :DLWLQJ f &DPEULGJH 0$ 0DUNHWLQJ 6FLHQFH ,QVWLWXWH

PAGE 250

&RKHQ f 6WDWLVWLFDO 3RZHU $QDO\VLV IRU WKH %HKDYLRUDO 6FLHQFHV 1HZ
PAGE 251

*UHHQ 6% f +RZ 0DQ\ 6XEMHFWV 'RHV ,W 7DNH 7R 'R $ 5HJUHVVLRQ $QDO\VLV" 0XOWLYDULDWH %HKDYLRUDO 5HVHDUFK f *URQURRV & f 6HUYLFH 4XDOLW\ 7KH 6L[ &ULWHULD RI *RRG 6HUYLFH 4XDOLW\ 5HYLHZ RI %XVLQHVV *URQURRV & f 6HUYLFH 0DQDJHPHQW DQG 0DUNHWLQJ /H[LQJWRQ 0$ /H[LQJWRQ %RRNV *URQURRV & f 4XDOLW\ &RPHV WR 6HUYLFH ,Q ( &KHXLQJ ( t : ) &KULVWRSKHU (GV f 7KH 6HUYLFH 4XDOLW\ +DQGERRN SS f 1HZ
PAGE 252

+RUQLN f 6LWXDWLRQDO (IIHFWV RQ WKH &RQVXPSWLRQ RI 7LPH -RXUQDO RI 0DUNHWLQJ )DOOf +RUQLN f 6XEMHFWLYH YV 2EMHFWLYH 7LPH 0HDVXUHV $ 1RWH RQ WKH 3HUFHSWLRQ RI 7LPH LQ &RQVXPHU %HKDYLRU -RXUQDO RI &RQVXPHU 5HVHDUFK .-XQHf +XL 0 t 7VH f :KDW WR 7HOO &RQVXPHUV LQ :DLWV RI 'LIIHUHQW /HQJWKV $Q ,QWHJUDWLYH 0RGHO RI 6HUYLFH (YDOXDWLRQ -RXUQDO RI 0DUNHWLQJ $SULOf ,VKLPRWR ,VKLPLWVX 7 .RVKLUR $ t +LURVH 6 f &RPSXWHU 6LPXODWLRQ RI 2SWLPXP 3HUVRQQHO $VVLJQPHQW LQ +RVSLWDO 3KDUPDF\ XVLQJ D :RUN6DPSOLQJ 0HWKRG 0HGLFDO ,QIRUPDWLFV f .DW] / /DUVRQ % 0 t /DUVRQ 5 & f 3UHVFULSWLRQ IRU WKH :DLWLQJLQ/LQH %OXHV (QWHUWDLQ (QOLJKWHQ DQG (QJDJH 6ORDQ 0DQDJPHQW 5HYLHZ :LQWHUf .LHO\ 7 f 0DQDJLQJ &KDQJH :K\ 5HHQJLQHHULQJ 3URMHFWV )DLO +DUYDUG %XVLQHVV 5HYLHZ 0DUFK$SULOf .LSSHQ /6 6WUDVVHU 6 -RVKL 0 f ,PSURYLQJ WKH 4XDOLW\ RI WKH 1&4$ $QQXDO 0HPEHU +HDOWK &DUH 6XUYH\ 9HUVLRQ 7KH $PHULFDQ -RXUQDO RI 0DQDJHG &DUH f .OHLQURFN / f 9ROXPH 4XHXHLQJ 6\VWHPV 1HZ
PAGE 253

/DZ $ 0 t .HOWRQ : f 6LPXODWLRQ 0RGHOOLQJ DQG $QDO\VLV 1HZ
PAGE 254

2GHK 5 ( t )R[ 0 f 6DPSOH 6L]H &KRLFH &KDUWV IRU ([SHULPHQV ZLWK /LQHDU 0RGHOV 1HZ
PAGE 255

6DXQGHUV & ( 0DNHQV 3 t /HEODQF / f 0RGHOLQJ (PHUJHQF\ 'HSDUWPHQW 2SHUDWLRQV 8VLQJ $GYDQFHG &RPSXWHU 6LPXODWLRQ 6\VWHPV $QQDOV RI (PHUJHQF\ 0HGLFLQH f 6DZ\HU $ t %DOO $ f 6WDWLVWLFDO 3RZHU DQG (IIHFW 6L]H LQ 0DUNHWLQJ 5HVHDUFK -RXUQDO RI 0DUNHWLQJ 5HVHDUFK ;9,,,I $XJXVWf 6FKPHOH $ f 5HVHDUFK DQG 7RWDO 4XDOLW\ ,Q $ ) $O$VVDI t $ 6FKPHOH (GV f 7KH 7H[WERRN RI 7RWDO 4XDOLW\ LQ +HDOWKFDUH 'HOUD\ %HDFK )/ 6W /XFLH 3UHVV 6FKULEHU 7 f $Q ,QWURGXFWLRQ WR 6LPXODWLRQ 8VLQJ *366+ 1HZ
PAGE 256

7D\ORU 6 Df :DLWLQJ IRU 6HUYLFH 7KH 5HODWLRQVKLS %HWZHHQ 'HOD\V DQG (YDOXDWLRQV RI 6HUYLFH -RXUQDO RI 0DUNHWLQJ f 7D\ORU 6 $ Ef 'LVWLQJXLVKLQJ 6HUYLFH 4XDOLW\ IURP 3DWLHQW 6DWLVIDFWLRQ LQ 'HYHORSLQJ +HDOWK &DUH 0DUNHWLQJ 6WUDWHJLHV +RVSLWDO DQG +HDOWK 6HUYLFHV $GPLQLVWUDWLRQ f 7D\ORU 6 $ t &URQLQ f 0RGHOLQJ 3DWLHQW 6DWLVIDFWLRQ DQG 6HUYLFH 4XDOLW\ -RXUQDO RI +HDOWK &DUH 0DUNHWLQJ f 7D\ORU 6 t &OD[WRQ f 'HOD\V DQG WKH '\QDPLFV RI 6HUYLFH (YDOXDWLRQV -RXUQDO RI WKH $FDGHP\ RI 0DUNHWLQJ 6FLHQFH f 7KRPSVRQ 0 f ,PSURYLQJ WKH 8WLOL]DWLRQ RI )URQW /LQH 6HUYLFH 'HOLYHU\ 6\VWHP 3HUVRQQHO 'HFLVLRQ 6FLHQFHV 7RP t /XFH\ 6 f :DLWLQJ 7LPH 'HOD\V DQG &XVWRPHU 6DWLVIDFWLRQ LQ 6XSHUPDUNHWV -RXUQDO RI 6HUYLFHV 0DUNHWLQJ f 7XPD\ f %XVLQHVV 3URFHVV 6LPXODWLRQ ,Q & $OH[RSRXORXV .DQJ : 5 /LOHJGRQ t *ROGVPDQ (GVf 3URFHHGLQJV RI WKH 3URFHHGLQJV RI WKH :LQWHU 6LPXODWLRQ &RQIHUHQFH SS f $UOLQJWRQ 9$ 9DQVFR\ *DMHZVNL / 7\OHU / 6 *RUD+DUSHU 0 / *UDQW / t 0D\ 5 f 7KH )XWXUH RI 0HGLFDWLRQ ,QIRUPDWLRQ 3UDFWLFH $ &RQVHQVXV 7KH $QQDOV RI 3KDUPDFRWKHUDS\ -XO\$XJXVWf :HVWJDUG t %DUU\ 3 / f &RVW(IIHFWLYH 4XDOLW\ &RQWURO 0DQDJLQJ WKH 4XDOLW\ DQG 3URGXFWLYLW\ RI $QDO\WLFDO 3URFHVVHV :DVKLQJWRQ '& $$&& 3UHVV :LQVWRQ : / f 2SHUDWLRQV 5HVHDUFK $SSOLFDWLRQV DQG $OJRULWKPV QG HG f %HOPRQW &$ 'X[EXU\ 3UHVV =HLWKDPO 9$ %HUU\ / / t 3DUDVXUDPDQ $ f &RPPXQLFDWLRQ DQG &RQWURO 3URFHVVHV LQ WKH 'HOLYHU\ RI 6HUYLFH 4XDOLW\ -RXUQDO RI 0DUNHWLQJ $SULOf =HLWKDPO 9 $ %HUU\ / / t 3DUDVXUDPDQ $ f 7KH %HKDYLRUDO &RQVHTXHQFHV RI 6HUYLFH 4XDOLW\ -RXUQDO RI 0DUNHWLQJ $SULOf

PAGE 257

%,2*5$3+,&$/ 6.(7&+ 'DQLHO /HH +DOEHUJ ZDV ERUQ RQ 2FWREHU LQ 1HZ /RQGRQ &RQQHFWLFXW +H UHFHLYHG D %DFKHORU RI 6FLHQFH GHJUHH LQ EXVLQHVV DGPLQLVWUDWLRQ ZLWK D PDMRU LQ ILQDQFH IURP WKH 8QLYHUVLW\ RI )ORULGD LQ 'HFHPEHU RI +H HQWHUHG WKH 3K SURJUDP LQ WKH GHSDUWPHQW RI 3KDUPDF\ +HDOWK &DUH $GPLQLVWUDWLRQ DW WKH 8QLYHUVLW\ RI )ORULGD LQ WKH VSULQJ RI %HVLGHV VHUYLQJ DV D JUDGXDWH DVVLVWDQW IRU WKH GHSDUWPHQW 'DQLHO DOVR VHUYHG DV DQ DGMXQFW IDFXOW\ PHPEHU IRU 6DQWD )H &RPPXQLW\ &ROOHJH 6)&&f LQ WKH GHSDUWPHQW RI &RPSXWHU DQG ,QIRUPDWLRQ 6FLHQFHV ,Q DGGLWLRQ KH ZRUNHG DV DQ LQGHSHQGHQW FRQVXOWDQW IRU WKH &HQWHU IRU 3URIHVVLRQDO 'HYHORSPHQW DW 6)&& VSHFLDOL]LQJ LQ FRPSXWHU DSSOLFDWLRQV WUDLQLQJ '/ +DOEHUJfV FXUUHQW UHVHDUFK LQWHUHVWV LQYROYH HIILFLHQW KHDOWK FDUH UHVRXUFH DOORFDWLRQ GHFLVLRQV XVLQJ WHFKQLTXHV RI SKDUPDFRHFRQRPLFV DQG RSHUDWLRQV UHVHDUFK ZLWK SDUWLFXODU HPSKDVLV RQ GUXJ XVH V\VWHPV DQG FRPSXWHU VLPXODWLRQ

PAGE 258

, FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ &KDUOHV +HSOHI 3URIHVVRU RI 3KDUPDF\+HDOWK &DUH $GPLQLVWUDWLRQ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV GLVVHUWDWLRQISU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ 5LFKDUG 6HJDO 3URIHVVRU RI 3KDUPDF\ +HDOWK &DUH $GPLQLVWUDWLRQ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ A m (DUOHQH /LSRZVNL $VVRFLDWH 3URIHVVRU RI 3KDUPDF\ +HDOWK &DUH $GPLQLVWUDWLRQ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ 2J %DUQH\ / &DSKDUW 3URIHVVRU RI ,QGXVWULDO DQG 6\VWHPV (QJLQHHULQJ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ 5DOSK 6MZDLQ 3URIHVVRU RI ,QGXVWULDO DQG 6\VWHPV (QJLQHHULQJ 7KLV GLVVHUWDWLRQ ZDV VXEPLWWHG WR WKH *UDGXDWH )DFXOW\ RI WKH &ROOHJH RI 3KDUPDF\ DQG WR WKH *UDGXDWH 6FKRRO DQG ZDV DFFHSWHG DV SDUWLDO IXOILOOPHQW RI WKH UHTXLUHPHQWV IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ 0D\ 'HDQ &ROOHJH RI 3KDUPDF 'HDQ *UDGXDWH 6FKRRO


OPTIMIZING SERVICE CAPACITY IN THE DRUG INFORMATION SERVICE
By
DANIEL LEE HALBERG
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1998

Copyright 1998
By
Daniel Lee Halberg

This manuscript is dedicated to my two role models, who taught me that there are
no limits to the dreams I can achieve. I will forever miss you both.
To my grandfather, the late James W. Carrig, I would like to thank you for being a
father figure and role model at a time in my life when there were few heroes to choose
from. You always had a smile on your face and a joke to tell. You taught me that
happiness is built on hard work and a loving relationship with family and friends.
To my greatest teacher, the late Kent Harriman, I would like to thank you for
being my friend and "coach". Your peerless gift for teaching was a great loss to the
world. I am sorry that you can not read this, because I think you would have appreciated
this accomplishment. I feel that everything that I am now started with those first steps I
took into your classroom. Thank you.

ACKNOWLEDGEMENTS
First, I would like to express my love and appreciation for my wife, Sara A.
Halberg. Without her, I doubt I would have had the courage and discipline necessary to
achieve this goal. I will always thank God for making her my wife. I am looking forward
to all of our new adventures together.
Second, I would like to extend my sincere and heartfelt gratitude to my major
advisor, Dr. Charles D. Hepler. I thank him for his insight, confidence, and humor. Third,
I would like to thank the rest of my committee, Drs. Richard Segal, Earlene Lipowski,
Barney Capehart, and Ralph Swain for their interest, direction, and advice. Without their
guidance, this project would not have succeeded.
Fourth, I would like to thank the faculty, the staff, and graduate students of the
department of Pharmacy Health Care Administration for their friendship, advice, help and
encouragement. However, I would like to extend a special thank you to DeLayne
Redding for all her help and concern.
Finally, I would like to thank my entire family, especially my mother, Patricia
Halberg, and my two sisters, Darcy and Heather, for their love and support. I bet they
thought I would never graduate! Thanks also to my wife's family for their kindness. All
of them have supported me in too many ways to mention here, but I wanted them to know
that I will always appreciate their love and generosity.
IV

TABLE OF CONTENTS
ACKNOWLEDGEMENTS iv
LIST OF TABLES viii
LIST OF FIGURES xi
ABSTRACT xiii
CHAPTERS
1. INTRODUCTION 1
Introduction 1
Background 3
Problem Statement 7
Research Framework 8
Significance 8
Research Questions 9
2. LITERATURE REVIEW 11
Overview 11
Service Capacity and Wait Times 12
Relationships Among Wait Time, Perceived Service Quality, and Customer
Satisfaction 19
Perceived Service Quality 23
Summary of the Literature 30
3. RESEARCH FRAMEWORK AND HYPOTHESES 32
Research Framework 32
Research Hypothesis and Specific Research Questions 35
4. METHODS 41
Overview 41
Study Location 41
Data Sources, Sample Selection, and Data Collection Procedures 43
Sample Size Calculations 48
Study Variables 51
v

Questionnaire Development and Validation 55
Simulation Development, Verification, and Validation 67
Data Analysis 70
5. PRELIMINARY RESULTS 73
Overview 73
Part One: Historical and Concurrent Data 74
Part Two: Student and Director Interviews 94
6. MAIN RESULTS 106
Overview 106
Part Three: Relationships Among PSQ, OSQ, Behavioral Intention,
Perceived Service Time, Actual Service Time and Service Delays 106
Part Four: Simulation Results 128
7. DISCUSSION AND CONCLUSIONS 154
Overview 154
Summary and Discussion of Results 155
Conclusions 163
Limitations of Study 164
Recommendations for Future Studies 166
APPENDICES
A. TEXT OF PRE-TEST COVER LETTER 169
B. PRE-TEST QUESTIONAIRE - VERSION 1 170
C. PRE-TEST QUESTIONAIRE - VERSION 2 175
D. PRE-TEST FOLLOWUP POSTCARD 180
E. RESPONSES TO PRETEST QUESTIONAIRE VERSION ONE 181
F. RESPONSES TO PRETEST QUESTIONAIRE VERSION TWO 182
G. PRETEST QUESTIONNAIRE WRITTEN COMMENTS 183
H. HISTORICAL DATA SHEET 192
I. DATA COLLECTION FORM 194
vi

J. SEMI-STRUCTURED OUTLINE FOR STUDENT INTERVIEWS 196
K. SEMI-STRUCTURED OUTLINE FOR CO-DIRECTOR
INTERVIEWS 198
L. TEXT OF COVER LETTER FOR MAIN QUESTIONNAIRE 200
M. MAIN QUESTION AIRE 201
N. FOLLOWUP POST CARD FOR MAIN QUESTIONNAIRE 206
O. RESPONSES TO MAIN QUESTION AIRE 207
P. MAIN QUESTIONNAIRE WRITTEN COMMENTS 208
Q. SIMULATION BLOCK DIAGRAMS 219
R. SIMULATION PROGRAM CODE 225
LIST OF REFERENCES 234
BIOGRAPHICAL SKETCH 243
vii

LIST OF TABLES
Table Page
4-1. Required Sample Sizes for Selected System Parameters 49
4-2. Responses to Pre-test Questions 13 and 21 62
4-3. Pre-test Item-total Statistics for SERVPERF Subscale 64
4-4. Pre-test Item-total Statistics for Perceived Service Time 65
4-5. Pre-test Item Communalities 66
4-6. Rotated Component Matrix of Pre-test Responses 66
5-1. Percentage of Questions by Profession 75
5-2. Percentage of Questions by Subscription Status 75
5-3. Percentage of Questions by Type 78
5-4. Service Times in Minutes by Question Type 78
5-5. Service Times in Minutes Using Three Combined Question Types 79
5-6. Percentage of Questions by Response Type Requested 81
5-7. Service Time in Minutes by Response Type Requested 82
5-8. Frequency and Percentage of Response Types by Question Type 82
5-9. Percentage of Questions by Delay Status 83
5-10. Percentage of Delays in Service by Time Needed 84
5-11. Descriptive Statistics for the Average Number of Questions Answered
by Month for the Past Ten Years 85
5-12. Descriptive Statistics for Question Interarrival Times 86
5-13. Significant P-Values for Interarrival Times by Hour of Day 89
viii

5-14. Descriptive Statistics for Question Interarrival Times in Minutes by
Hour of Day 90
5-15. Summary Statistics for Input Di stributions 91
5-16. Kolmogorov-Smirnov Tests for Exponentially Distributed Variables 91
6-1. Questionnaire Sample Description 107
6-2. Reasons for Not Reponding 109
6-3: Results of One-Way ANOVA Procedures Measuring Response Bias 109
6-4. Initial Rotated Component Matrix of Main Questionnaire Responses
for the SERVPERF Sub-Scale 110
6-5. Main Questionnaire Communalities for the SERVPERF Sub-Scale Ill
6-6. Final Rotated Component Matrix of Main Questionnaire Responses for
the SERVPERF Sub-Scale 113
6-7. Item-total Statistics for SERVPERF Sub-Scale 116
6-8. Item-Total Statistics for Service Time Perceptions 117
6-9. Final Sub-Scale Reliabilities 117
6-10. Descriptive Statistics for Questionnaire Measures 117
6-11. Descriptive Statistics for PSQ Items 118
6-12. Correlations Between Study Variables 119
6-13. PSQ by Level of Behavioral Intention 121
6-14. OSQ by Level of Behavioral Intention 122
6-15. PSQ by Level Perceived Service Time 123
6-16. Perceived Service Quality by Delay in Service 125
6-17. Percentage Below/Above the Mean Actual Service Time by Q27
Expected Time 128
6-18. Student Utilization, Total Service Time, and Expected Number in
System by Arrival Modifier at 20 Simulated Days 136
ix

6-19. Descriptive Statistics for Selected Comparisons Between Observed and
Simulated Data 137
6-20. Comparison of Simulated Queue Statistics Versus Exact Solution 137
6-21. Descriptives for Queue Statistics by Number of Students 148
6-22. Descriptive Statistics for the Total Service Time, Time in Queue,
Number in System, and Queue Length by Percentage Change in
Research and Approval Time 149
6-23. Descriptive Statistics for Number Completed, Utilization Percentage,
and Delay Percentage by Percent Change in Research and Approval
Time 150
6-24. Effectiveness of Service Capacity Improvements Under Normal Arrival
Rates 151
6-25. Sensitivity of Optimal Solution to Changes in the Arrival Rate 153
x

LIST OF FIGURES
Figure Page
1-1. Hypothesized Relationships 8
3-1. Hypothesized Framework 40
4-1. Scree Plot of Pre-test Data 66
5-1. 95% Confidence Intervals of Service Time by Question Type (In
Minutes) 77
5-2. 95% Confidence Intervals of Service Time Using Three Combined
Question Types (in Minutes) 80
5-3. 95% Confidence Intervals of Service Time by Response Type (in
Minutes) 82
5-4. 95% Confidence Intervals of Average Daily Arrivals by Month 86
5-5. 95% Confidence Intervals of Interarrival Times in Minutes by Hour of
Day 90
5-6. Frequency Histogram of Historical Interarrival Times 92
5-7. Frequency Histogram of Historical Total Service Times 92
5-8. Frequency Histogram of Service Times for Question Group One 93
5-9. Frequency Histogram of Service Times for Question Group Two 93
5-10. Frequency Histogram of Service Times for Question Group Three 94
6-1. Scree Plot of Main Questionnaire Responses 113
6-2. Regression Equation Plot of PSQ and Predicted PSQ by Actual Service
Time where PSQ=34.443+0.0013*(Total Service Time) 126
6-3. Residual Plot of PSQ on Total Service Time 126
6-4. Residual Plot of PSQ on Total Service Times Occurring within One
Day 127
xi

6-5. Expected Number in System for Six Replications of 20 Days 136
6-6. Observed Versus Simulated Probability Density Functions (PDF) 138
6-7. Observed Versus Simulated Cumulative Density Functions (CDF) 138
6-8. 95% Confidence Intervals for Delay Percentage by Service Rate
Modifier and Number of Servers 152
6-9. 95% Confidence Intervals for Total Service Time (in Minutes) by
Service Rate Modifier and Number of Servers 152
xii

Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy
OPTIMIZING SERVICE CAPACITY IN THE DRUG INFORMATION SERVICE
By
Daniel Lee Halberg
May 1998
Chairman: Professor Charles D. Hepler
Major Department: Pharmacy Health Care Administration
Health services are often developed without appropriately analyzing the system’s
ability to meet the needs of the consumer, and attempts to improve quality and efficiency
often do not succeed because of the complexity and dynamic nature of services.
However, some organizations are using sophisticated techniques such as simulation to
analyze service systems. This study had three primary objectives: (1) to develop a
computer simulation model for a drug information service; (2) to investigate the
associations among actual service time, service delays, and perceived service quality; and
(3) to recommend system improvements based on the simulation.
This study used both experimental and non-experimental methods. It was
conducted at the Drug Information and Pharmacy Resource Center (DIPRC) at Shands at
the University of Florida. Overall, seven hypotheses and three specific research questions
were used to explore relationships among the study variables. Six data sources were used
(1) historical data sheets, (2) historical workload data, (3) data collection forms, (4)
personal interviews, (5) service quality questionnaires, and (6) simulation runs.

The first three hypotheses tested the relationships among perceived service quality
(PSQ), overall service quality (OSQ), and two measures of behavioral intention. A strong,
positive relationship was found between PSQ and OSQ. In addition, relationships were
found between PSQ and behavioral intention and between OSQ and behavioral intention.
The remaining four hypotheses tested the relationships among PSQ, actual service time,
service delays, and perceived service time. It was found that only service delays and
perceived service time were significantly related to PSQ. However, perceived service time
seemed more important than service delays with regard to PSQ. No relationship was
found between actual service time and PSQ. Surprisingly, no practically significant
relationships were found among actual service time, service delays, and perceived service
time.
A simulation model was constructed using GPSS/H (General Purpose Simulation
System). The simulation was validated and found to be a credible model for analyzing the
service system at the DIPRC. Exploration of the three specific research questions
indicated that improving service times was more efficient than staffing increases for the
purpose of reducing the percentage of service delays.
xiv

CHAPTER 1
INTRODUCTION
Introduction
Health services are often developed without careful consideration of the actual
needs of the consumer, or without appropriately analyzing the service system’s ability to
meet these needs (Shostack, 1984). Services that fall short of meeting consumer needs
must be modified or redesigned in order to improve the quality of the service. Two
popular transformation paradigms that are often used to examine the quality problems
related to service processes are total quality improvement (TQM) and business process re¬
engineering (BPR).
The TQM paradigm has gained considerable support from the healthcare
community as a transformation philosophy (Boerstler et al., 1996). TQM was pioneered
by W. Edwards Deming in the 1950s, and focuses on the concept of “kaizen ”, a Japanese
word meaning the continuous incremental improvement of an existing process (Hammer
and Champy, 1993). In essence, TQM is an organization wide commitment to steadily
and continuously improve quality of the system (Schmele, 1993).
BPR is currently of considerable interest in many service environments including
health care. Many individuals confuse the concepts of BPR with TQM. Re-engineering
has been defined as “the fundamental rethinking and radical redesign of business processes
to achieve dramatic improvements in critical, contemporary measures of performance,
such as cost, quality, service, and speed” (Hammer and Champy, 1993, p.32). Thus,
reengineering is not fundamentally about process improvement, but process re-invention
(Hammer and Champy, 1993).

2
However different the approaches, these two paradigms share a common, primary
focus on consumer needs and the outcomes of a process. Often, the goal is to achieve a
desired outcome (i. e., production of a product or service) while meeting several
objectives, including: improved productivity, improved quality, reduced total process time,
increased throughput, and reduced waiting times (Hammer and Champy, 1993; Tumay,
1995).
Unfortunately, it is clear that a large percentage of BPR and TQM efforts fail to
deliver any of the promised benefits (Boerstler et al., 1996; Hammer and Champy, 1993;
Geisler, 1996; Kiely, 1995; Kotter, 1995, Rust et al., 1995). Although there have been
many reasons given for these failures, part of the trouble inherently rests in the difficulty of
understanding the complex and dynamic interdependencies of service systems. Because of
this, some recent transformation projects have used sophisticated techniques, such as
computer simulation, in order to analyze the relationships among the various components
of service systems and the effects of implementing changes in the system (Tumay, 1995).
This study had three primary objectives: (1) to develop a computer simulation
model for a drug information service and to validate the model against the existing system;
(2) to investigate the associations among actual service time, service delays, and
evaluations of perceived service quality in a drug information service setting; and (3) to
recommend system improvements based on the simulation model; in particular those
improvements that reduce the time required to respond to consumer questions and
information requests.
The following sections of this chapter will provide background information
describing the underlying concepts used in the research and will introduce the problem
statement for the proposed study. The background will specifically address a) how
queuing theory can be used as a basis for establishing optimal levels of service capacity; b)
how service capacity can affect costs, waiting times, and perceived service quality; and c)

3
the measurement of perceived service quality. Each of these issues, however, will be
discussed more critically in the next chapter.
Background
Queuing Theory and the Simulation of a Service System
The goal of most queuing theory and simulation based models is to understand the
behavior of a particular system and to make decisions regarding the system based on the
behavior of the model (Seila, 1992). Many real-world systems that involve random arrival
and service rates can be examined by structuring them as queuing problems. Essentially,
these problems can be evaluated in two ways, through either a closed form solution or an
open form solution.
Queuing theory uses closed form mathematical relationships to achieve exact
answers to waiting-time and waiting-line problems. Simulations of queuing systems are
open form, computationally dynamic models that describe the behavior of a system with
respect to time. Open form solutions are used when there are no known equations for the
operating characteristics, such as waiting time in the queue, for the system of interest.
When available, closed form solutions are usually preferred to open form solutions
because of their exactness and theoretical power. However, simulations are used to study
waiting situations when closed form solutions are too complex or are not available. For
example, simulations are used when the properties of the system to be modeled violate the
underlying assumptions of queuing theory, or when the researcher desires more
information than the queuing theory approximation provides (Krajewski and Ritzman,
1990; Maryanski, 1980).

4
Service Capacity and its Relationship to Perceived Service Quality
Service capacity can be defined using the queuing theory framework as a function
of the staffing level (5) and the service rate (p.) Staffing level refers to the number of
employees available to serve consumers. The service rate refers to a variable amount of
time required by an employee to complete a service request given their personal ability,
their equipment, and the organization of the work (Krajewski and Ritzman, 1990;
Lovelock, 1987; Winston, 1991). When there is a shortage of capacity relative to demand
(L), queues form, and total time in the system increases (Lovelock, 1987). It has been
demonstrated in the literature that as waiting time increases, evaluations of perceived
service quality and customer satisfaction are negatively affected (Bolton and Drew, 1994;
Clemmer and Schneider, 1993; Davis and Vollmann, 1990; Davis, 1991; Dube’-Rioux et
al., 1989; Hui and Tse, 1996, Katz, Larson, and Larson, 1991, Taylor, 1994a, Tom and
Lucey, 1995).
Two approaches have been used to reduce the adverse effects of waiting on
customer satisfaction and perceived service quality in the literature: perceptions
management and operations management. Perceptions management is an approach that
attempts to reduce the perceived wait times of the consumers of a system through the
creative use of distractions, apologies, queue information (e.g., place in line and estimated
wait times), and the manipulation of perceived pre-service and post-service waits.
According to reports in the literature, perceptions management has met with limited
success; however, it is still unclear as to how successful these techniques are across a
variety of service settings (Clemmer and Schneider, 1989a, 1989b, and 1993; Hui and Tse,
1996). Operations management is an approach that attempts to reduce the actual wait
times of the consumers of a service system using scheduling techniques, queue
management, and work flow changes. At the heart of the capacity issue is the
development of appropriate queuing systems that utilize capacity to its best advantage

5
(Lovelock, 1987). This research focused primarily on the use of operations management
techniques, specifically queuing theory and simulation, to manage service capacity.
Measurement of Perceived Service Quality
Four basic characteristics apply to most services: (1) benefits received from
services are largely intangible; (2) services are activity focused rather than product
focused; (3) services are simultaneously produced and consumed; and (4) the consumer
participates in the production process (Gronroos, 1990). Because of these characteristics
it is generally recognized that service quality is harder to evaluate than product quality
(Heskett, 1987; Parasuraman et al., 1985).
The concept of quality, however, is difficult to define under any circumstance.
Nevertheless, we think of quality in terms of the superiority, excellence, or value of a
product or service (Christensen and Penna, 1995; Westgard and Barry, 1986). Two basic
methods have been used to assess quality in organizations. The first method uses
objective indicators (e.g., age of equipment, number of defects, or consumer time in the
system) as measures of quality. The second method uses subjective indicators (e.g.,
perceived service quality, customer satisfaction, or employee satisfaction) as measures of
quality.
Service quality has been defined as the relative superiority of an organization and
the services it provides (Parasuraman et al., 1988). Perceived service quality is concerned
with the measurement of consumer attitudes regarding an organization’s service quality.
There has been significant debate in the literature concerning the dimensions and
measurement of perceived service quality (Cronin and Taylor, 1992, 1994; Gronroos,
1993; McAlexander, 1994; Parasuraman et al., 1985, 1988; Peter et al., 1993). A
perceptions-only scale, derived from the SERVPERF scale (Cronin and Taylor, 1992) will
be used as a measure of perceived service quality in this research primarily because it more

6
practical than many of the available instruments and because it avoids some of the
measurement issues related to the use of difference scores (Cronin and Taylor 1992, 1994;
Peter et al., 1993; Zeithaml et al., 1996).
Drug Information Services
The first formally recognized drug information center (DIC) was established at the
University of Kentucky in 1962 and by 1995 more than 175 organizations maintained
DICs (Parker, 1965; Rosenberg et al., 1995; Vanscoy et al., 1996). The initial role of
these centers was to evaluate and compare drugs and to promote rational drug therapy;
however, the role of many centers has evolved to include educational activities,
medication policy development, and outcomes research (Beaird et al., 1992, Vanscoy et
al., 1996). Health care professionals are currently challenged by the necessity to keep up
with the latest developments in new drugs and advances in therapies. Many of the DICs
have come into existence because of the recognition by management that it is not efficient
to have practitioners review the literature and identify solutions to all of the drug therapy
problems they encounter. As such, DICs were developed as a central, organized approach
to meeting these needs and to help disseminate drug information to the medical and
nursing staff (Skoutakis, 1987; Smith, 1988).
In the current healthcare environment, however, the outlook for DICs is uncertain.
Several factors are currently placing increasing pressure on DICs to provide increased
levels of service. First, advancing information technologies and managed care influences
are forcing DICs to provide the highest quality service with near instantaneous access to
information. Second, not only are DICs responsible for dispensing information regarding
clinical decisions, they are also being asked to document their impact on patient care using
outcomes measurement tools derived from disciplines such as pharmacoepidemiology and

7
pharmacoeconomics. Third, many DICs are also required to participate in scholarly
research and educational activities (Skoutakis, 1987; Vanscoy et al., 1996).
While there is a recognized need for drug information services, the current era of
cost containment and outcomes management presents a dilemma for health organizations,
hospitals, and universities who are now required to justify support for non-profitable
programs. Even though DICs are being asked to provide more services, cutbacks in
staffing are not uncommon when funds available for such programs are reduced (Mailhot
and Giacona-Dahl, 1987; Skoutakis, 1987; Vanscoy et ah, 1996).
Thus, the ability of DICs to evaluate their service in terms of effectiveness or
outcomes is critical in maintaining a DIC in the current health care environment. Although
there have been a number of articles documenting the activities of drug information and
toxicology resources, very few of these articles have addressed drug information quality or
effectiveness of DICs under resource constraints (Lilja, 1985; Rosenberg et ah, 1995,
Skoutakis, 1987). As John Lilja states, “it is safe to say that we know astonishingly little
on how to optimize resources for drug information programs” (1985, p.412).
Problem Statement
Determining sufficient service capacity in a drug information service is a
challenging issue for managers trying to maintain acceptable levels of service quality.
However, questions regarding capacity often involve decisions related to the acceptable
amount of time required to deliver the service (e g., answer a drug-therapy question).
Unfortunately, the behavior of queuing systems is deceptively complex and often non-
intuitive. If staffed according to “common sense” approaches, many systems are unable to
handle the workload.
The mathematics of queuing theory show that inadequate service capacity can
greatly increase the waiting time before service is completed. Consequently, a consumer’s

8
overall time in the system will often be longer than intended. Consumers who wait long
periods of time for service may be more likely to downgrade the quality of the service,
even though other aspects of service performance may have been delivered competently
(Taylor, 1994a; Taylor and Claxton, 1994). Unfortunately, however, service processes
are often the most complex systems to understand because they frequently depend on the
random nature of arrival processes and service times and the dynamic interdependencies of
system behavior (Tumay 1995).
Research Framework
This research used a queuing paradigm to evaluate the relationship between service
capacity in a drug information service and the length of time that a consumer must wait to
obtain a response to a question or information request. In addition, the relationship
between waiting time and consumer evaluations of perceived service quality was
determined. It was proposed that a relationship existed between service capacity and
service time or delays in service, and between service time or delays in service and
evaluations of perceived service quality (Figure 1-1). This framework is developed in
more detail in chapter three.
Service
Service Time
Perceived
Capacity
and Delays
P>-
Service quality
Figure 1-1. Hypothesized Relationships
Significance
Although simulation research involving staffing patterns has been conducted in a
variety of health related systems, including nursing, psychiatry, and emergency

9
departments, no research has been published describing the system behavior of drug
information services using a queuing paradigm. In addition, some research has been
conducted examining the relationship between perceived service quality and service
capacity; however, this area is still in the rudimentary stages of development. This
research would help to expand existing knowledge in these areas.
From a more practical standpoint, various researchers have proposed a positive
relationship between improved service quality and increased revenues or improved
productivity (Gronroos, 1990). However, this benefit may be less useful to “free” service
programs that are funded under fixed or shrinking budgets. In this environment, it is more
valuable to suggest two possible impacts that service capacity decisions can have on
intentions and utilization. First, if service quality is perceived as inferior by consumers
then they may be less likely to rely on the service. Thus, inadequate service capacity may
reduce the perceived value of the service. If utilization decreases then it becomes more
difficult to justify the service’s existence. The service may be suspended because
consumers feel the quality of a service is poor and have decided not to use it, regardless of
their actual need.
Second, budgetary constraints often make it difficult to justify increases in staffing
or other improvements in service capacity without significant evidence of need. By using
the information obtained from simulation models and perceived service quality surveys, it
may be easier to demonstrate current shortfalls in quality, current and projected demand
levels, and the positive and negative consequences of changes in service capacity.
Research Questions
There are four questions that this research will attempt to address:
1. Can information regarding staffing levels, service rates, call arrival patterns, and
system structure be used to build a simulation model that is a reliable and valid

10
substitute for the actual drug information service for the purpose of capacity
planning?
2. How do staffing levels, arrival and service rates, and system structure affect the
important performance characteristics of the system? (Adapted from Krajewski and
Ritzman (1990)):
A. Queue Length: This is the expected number of consumer information requests
and questions in the system at a given point in time.
B. Service time: This is the expected total service time required to deliver a
response to a question or information request. This is measured from the
arrival of a question or request for information into the information service
until the delivery of a response.
C. Utilization Rate: This is the collective utilization of the service facilities reflects
the percentage of time the service personnel are busy (as opposed to the time
they are idle). This is described as a ratio of the amount of time the server was
busy over the total time measured.
3. How does total service time relate to consumers’ perceived service quality? How
do delays in service relate to consumers’ perceived service quality? Could these
relationships be used in the simulation model to reflect the impact of changes in
service capacity on consumers’ perceived service quality?
4. What are the critical variables that affect the simulation model? Based on these
variables, what management rules can be recommended to improve service capacity
and reduce the response time of the system?

CHAPTER 2
LITERATURE REVIEW
Overview
Chapter one introduced four important concepts and their relationship to this
research. First, the capacity of a service system can be described in terms of a queuing
paradigm. Second, complex queuing systems can be modeled using computer simulation.
Third, waiting times can be influenced by changing the service capacity. Fourth, it was
proposed that there is an inverse relationship between waiting time and evaluations of
service quality.
This chapter critically reviews the previous research related to the concepts
mentioned above and their application to this project. It will begin by presenting an
overview of queuing theory. Second, methods for improving the performance (i.e.,
reducing waiting times) of queuing systems will be described. Third, evidence
demonstrating the relationship between wait time and consumers' evaluations of services
(in terms of service quality and satisfaction) will be reviewed. It will then discuss how
managers often underestimate acceptable wait times and how consumers' overestimate the
time waited. Fourth, the chapter will examine the conceptual basis of perceived service
quality and describe why this study will use a perceptions-only instrument to measure
perceived service quality. The chapter will end with a description of the practical
relationship between service quality and behavioral intention. In chapter three, the
research framework will be presented and the research hypothesis for this project will be
developed.
11

12
Service Capacity and Wait Times
Overview of Queuing Theory
There are three primary components of a queuing problem. The first is the input
source, defined as the population of potential entrants into the service system. We can
describe this population in terms of its size, the nature or urgency of need, and the arrival
distribution. The size of the input source may be infinite or finite, depending on whether
the number of customers in the system significantly affects the arrival rate. The nature or
urgency of need influences the relationship between waiting time or queue length and
reneging or balking (i.e., leaving without service) (Krajewski and Ritzman, 1990; Winston,
1991).
The arrival distribution is a probability distribution that describes either the number
of arrivals per unit time or the time between arrivals (i.e., the interarrival time) (Krajewski
and Ritzman, 1990; Winston, 1991). In effect, many managers responsible for assessing
service capacity inappropriately assume some constant or narrowly defined arrival pattern.
There are, however, other probability distributions that can describe customer arrival
streams better than a constant. For instance, arrivals per unit time are sometimes assumed
to be Poisson distributed and interarrival intervals are often approximated by some form of
the gamma distribution, usually the exponential (Krajewski and Ritzman, 1990; Winston,
1991).
The second component of a queuing system is the service process. We describe
the service process in terms of the service arrangement and service rate distribution. The
service arrangement comprises the organization of the service (i.e., workflow), the number
of servers available to handle the arrivals, and the number of lines leading to those servers.
The service rate distribution is a probability distribution governing the amount of the time
a server takes to service a customer. Often, the service rate is described by the

13
exponential, Weibull or Erlang distributions (Krajewski and Ritzman, 1990; Law and
Kelton, 1991; Winston, 1991).
The third component of a queuing problem involves the queue discipline, also
sometimes called the priority discipline. The queue discipline refers to the order in which
customers are processed through the queue. There are several common queue disciplines,
including FIFO or FCFS (first come first served), LIFO or LCFS (last come first served),
SPTF (shortest processing time first), and LPTF (longest processing time first). The latter
two are more specifically termed priority queue disciplines because customers are
categorized based on their expected length of service. These categories are given a
priority level, in which those customers allocated to higher priority levels go before those
customers with lower priority. Within each category, however, customers are serviced in
a standard queue discipline such as FCFS (Krajewski and Ritzman, 1990; Winston, 1991).
Two other queue disciplines exist which are more difficult to model. The first is
"Shortest Lead Time", in which the arrival with the shortest time between the current time
and the promised time has a highest priority regardless of when they entered the system.
The second is "Arbitrary Priority", where the service order and time are dependent on the
servers’ preferences or some form of undetermined triaging mechanism. These disciplines
are often modeled as a SIRO (service in random order) discipline (Larson, 1987,
Schwartz, 1975).
Most queuing models depend on a steady state system for estimating queue
statistics for a varying number of servers. A system is in steady state if
(1) the number of servers, the average arrival rate, and the average service
rate are not changing, (2) the average arrival rate is less than the average
service rate times the number of servers, and (3) these conditions have
existed for a substantial period of time...The opposite of steady state is
transience, which refers to the behavior of the system during the period
following some change (McClain and Thomas, 1985, p.550).

14
It is not hard to imagine that if the service rate is less than the arrival rate, then the
queue will grow without bound because the system is not physically capable of handling
the volume of arrivals. However, it is also mathematically true that the queue length will
approach infinity when the service rate equals the arrival rate. This can be illustrated using
two queuing theory results based on the M/M/1 (single server, single queue) model.
First, the utilization rate (p) equals the arrival rate (X) divided by the service rate (p)
(Winston, 1991). Second, the expected number of customers in a line (L) is a function the
utilization rate (p) such that (Winston, 1991)
P — (Equation 1-1)
P
— —-— (Equation 1-2)
1 -P
Notice that as the arrival rate approaches the service rate, the utilization rate
approaches one. Therefore, as the utilization rate approaches one then L approaches
infinity (i.e., 1 divided by 0 -> go). This is not an intuitive result, and this is the primary
pitfall of naive staffing models (i.e. models that do not account for the effect of random
variation on queue behavior). When managers try to match the service capacity exactly to
the demand, long waiting lines will occur.
There are six options typically available for improving the performance of systems
under a queuing paradigm: (1) add servers, (2) increase the service rate, (3) increase
queue size, (4) change the distribution of arrivals, (5) reduce the variance in service times
or interarrival times, (6) change the queue discipline. These options are discussed in more
detail below.

15
Methods for Improving System Performance
First, adding servers (i.e., increasing staffing) is usually the most frequently
considered method for improving service capacity, however, it can also be costly and often
is the least efficient method. As you add more servers, the marginal impact that each new
server has on the system decreases. There is also the tradeoff of balancing the utilization
rates with excess capacity. Management is interested in maintaining high utilization, but
this objective may have an adverse impact on the other operating characteristics. For
instance, when utilization is too high, workers may have trouble adapting to changes in the
service demand. However, when utilization is too low workers will have too much idle
time (Krajewski and Ritzman, 1990). There may also be regulatory or accreditation
restrictions that limit the minimum number of servers in a particular setting (Duraiswamy
et al., 1981). There have been three different approaches used in the literature to optimize
staffing levels:
1. Adjust staff levels in terms of the actual number of service personnel
(Duraiswamy et al., 1981; Hammond and Mahesh, 1995; Ishimoto et al.,
1990; Lamy et al., 1970; Saunders et al., 1989; Sumner and Hsieh, 1972).
2. Adjust staff levels based on the number of full time equivalents (FTEs)
(Hashimoto et al., 1987). In many settings, a more appropriate method of
optimizing staffing is to consider the number of FTEs rather than the actual
number of persons. In this way it becomes easier to consider part-time
employees, full-time employees that devote parts of their work day to
different tasks, and employees that have many different simultaneous tasks
to complete.
3. Adjust staff levels based on a percentage of maximum (rather than
expected) workload (McHugh, 1989). In instances when there are extreme

16
demand shifts, it is sometimes necessary to anticipate staffing levels for the
maximum rather than the expected workload. Models using this approach
usually discuss staffing levels in terms of a percentage of the maximum
workload.
Second, queue sizes and wait times can be improved by increasing the service rate
rather than the staffing level Service rates can be improved through new technologies,
training, and workflow redesign (Krajewski and Ritzman, 1990). For instance, Carruthers
(1970) examined the work turnover rate in a laboratory setting. It was found that the
purchase of new equipment was more cost-effective than increasing staffing. Also, Kumar
and Kapur (1989) found that increasing hospital nurses’ shift length from eight to twelve
hours was more beneficial than the addition of staff. Chin and Sprecher (1990) and Ozeki
and Ikeuchi (1992) both found that workflow changes were at least as important as
staffing increases in improving system wait times. However, increasing service rates do
not always improve system wait times, especially when service times are already relatively
short. For example, Lamy et al. (1970) found that only a fraction of the total waiting time
was related to the actual service time in a pharmacy setting. In this case, the staffing level
and variations in arrival times were more significant predictors of queue waits than the
service time.
Third, increasing the queue size may be an option if customers are being turned
away because they cannot even enter the system (e.g., busy telephone line). Since queues
are stochastic systems, there may be times when a queue exceeds its limitations, even
when p is less than one (i.e., arrival rate is less than the service rate). If this happens too
often, then queues will have high rejection rates. Costs of increasing the queue size will
vary dramatically depending on the type of queue. For instance, the addition of several
telephone lines may be inexpensive when compared to the construction of a larger waiting
area (Krajewski and Ritzman, 1990).

17
Fourth, changing the arrival rate is usually one of the most subtle and overlooked
areas of improvement. Some examples of how arrival rates can be influenced include (1)
informing consumers of typically idle or slow periods so that you might attract them to use
services during these times instead of peak times, (2) get customers to use alternate routes
for obtaining the same information such as a fax-back service or web-site, and (3)
schedule appointments with some or all consumers so that arrivals are less random and are
reduced during peak times (Krajewski and Ritzman, 1990). Few studies have discussed
techniques for modifying the arrival rate into the system, other than appointment systems.
One study conducted by Reilly et al. (1978) discussed the impact of optimizing staffing in
conjunction with a patient delay-scheduling model, where patients are given a delay time
before being admitted to the system. This delay manifested itself either as an appointment
or an anticipated waiting time. This had two potential benefits. First, since patients had
better knowledge of the length of the wait, they were not necessarily bound to the clinic
and could spend time elsewhere; hence, the patients could improve the quality of their
waits. Second, although not directly reported by Reilly et al., the realized interarrival
variance should have decreased, allowing for a more accurate prediction of staffing
requirements.
Fifth, reducing the variance in service times or interarrival times can also reduce
queue lengths. By examining the steady state equations for the M/G/k queue
characteristics, it can be shown that reducing variance can have a substantial effect on
reducing the effective waiting times and queue lengths. Methods of reducing variance in
service times have been discussed in the contexts of work design, facility design, total
quality management (TQM), and statistical process control (SPC). For example,
redesigning or standardizing work flow in order to eliminate errors, backtracking, and
rework would help reduce the variance in service times by eliminating inconsistent work
patterns. It should be noted that these techniques are often the same ones used to reduce

18
Service times; hence, when making changes in the system that are designed to reduce
service times, the variance in service times is also often reduced.
Reducing the variance in interarrivals may achieve similar benefits. As mentioned
previously, reducing the variance on arrivals might take the form of appointments or
blocking types of arrivals into specific time slots (Kleinrock, 1975; Konz, 1990; Lamy et
al., 1970, Law and Kelton, 1991; Reilly et al., 1979; Westgard and Barry, 1986; Winston,
1991).
Sixth, changing the queue discipline has also been shown to affect system
performance in terms of waiting times and line lengths. For example, it has been shown
that using the shortest-processing-time-first (SPTF) discipline will decrease the variance of
the wait times in a system, thereby decreasing the number of long service times due to
random variation. However, in many health care services, urgency (i.e., priority) plays a
significant role on queue behavior due to prioritization and preemption of service requests
based on need (e g., emergency care), so it would often be impossible to strictly adhere to
a SPTF discipline. However, it may still be possible to service non-urgent arrivals using a
SPTF discipline (Krajewski and Ritzman, 1990; Law and Kelton, 1991; Schriber, 1991;
Winston, 1991).
In the previous section, an overview of queuing theory was given in order to
describe the important variables and concepts that are often used in queuing and
simulation models to model waiting times, service times, and delays. In addition, methods
for improving system performance were reviewed. In the next section, the relationship
among consumer waiting time, perceived service quality, and customer satisfaction are
discussed.

19
Relationships Among Wait Time, Perceived Service Quality, and Customer Satisfaction
Waiting Time and Perceived Service Quality
Only three studies were found that examined the effect of wait time on service
quality, and none of the studies used the SERVQUAL or SERVPERF scales to measure
perceived service quality. Furthermore, these studies considered service delays rather than
queue waits. However, the results of these studies do indicate that wait times can
adversely affect evaluations of service quality.
Taylor (1994a) proposed a framework called "The Wait Experience Model" for
describing the relationship between the wait experience and the overall service evaluation.
This model was evaluated using a sample of airline passengers who experienced pre¬
boarding delays in their flight plans. Taylor found that customers’ overall evaluations of
service quality were primarily related to their level of anger and its associated feelings of
annoyance, irritation and frustration. Anger could be caused by (1) the customer's
uncertainty about the length of the delay, (2) the actual length of the delay, (3) the
customer's perception of the service provider's control over the delay, and (4) the degree
of filled time.
In further research related to this model, Taylor and Claxton (1994) found that
individuals who encountered pre-boarding delays were less likely to be satisfied with the
quality of the other services offered during the flight. Similarly, Dube’-Rioux et al. (1989)
found in a restaurant setting that the timing of the delay (i.e., pre-service, in-service, or
post-service) and level of customer need were significant in evaluations of service quality.
Waiting Time and Customer Satisfaction
A much more thorough examination has been conducted in the literature
concerning the relationship between wait time and customer satisfaction. Customer

20
satisfaction is a consumer’s post-service reflection regarding how well the service
compared with their expectations. Customer satisfaction occurs when the perceived
performance of the service exceeds expectations, and dissatisfaction occurs when
performance is lower than expectations (Bolton and Drew, 1991; Oliver, 1993).
Evaluations of customer satisfaction and perceived service quality usually rely,
either implicitly or explicitly, on the confirmation-disconfirmation paradigm, in which
consumer evaluations are based on a confirmation of expectations. In addition, the service
quality literature generally conceptualizes customer satisfaction as antecedent to perceived
service quality. (Boulding et al., 1993; Bitner, 1990; Cronin and Taylor, 1992;
Parasuraman et al., 1985; Parasuraman et al., 1988; Taylor, 1994b; Taylor and Cronin,
1994).
Since the constructs of customer satisfaction and perceived service quality share
some of the same dimensions, often a common theoretical basis, and perhaps a causal
relationship, it is probable that waiting time and service delays affect both constructs.
Therefore, an examination of the literature involving waiting time, service delays, and
customer satisfaction is important, especially since many of these studies do not
adequately describe their satisfaction instruments or their definitions of satisfaction. This
makes it difficult to ascertain whether the authors are measuring customer satisfaction or
service quality or both.
Hui and Tse (1996) tested a service evaluation model for a computerized course
registration service at a university. Katz, Larson, and Larson (1991) conducted a study
with bank customers. Studies done by Davis and Vollmann (1990) and Davis (1991)
concerned the length of waits in a fast food restaurant and the percentages of satisfied
customers. In all of these studies, the results indicated an inverse relationship between
perceived wait times and customer satisfaction. They all provide evidence that as wait
times increase customer satisfaction decreases. However, Davis (1991) suggested that a
non-linear relationship exists between waiting time and satisfaction. The proposition of a

21
non-linear relationship between waiting and consumer preference was also suggested by
Richard Larson (1987) in a discussion regarding the perceived utility of waiting.
In addition, Tom and Lucey (1995) found two important results concerning
expected waiting times and customer satisfaction in a supermarket setting. First, their
results found that customers were more satisfied in situations where the wait was shorter
than expected compared with situations where the wait was longer than expected. More
importantly, however, the researchers found that it was the reason for the wait that most
affected the levels of satisfaction. If the customers blamed the store for the unexpected
wait, then satisfaction with the store tended to decrease. However, if customers attributed
the wait to something outside of the store's control then no changes in the satisfaction
levels were evident.
Manager and Consumer Estimates of Waiting Time
As discussed above, the literature seems to establish a relationship between the
amount of time consumers wait for service and their evaluations of service quality.
However, there is evidence to suggest that managers and consumers perceive the wait
experience differently. Davis and Vollmann (1990) and Davis (1991) found that managers
tended to overestimate the duration of what a customer would consider an acceptable
delay. This result is consistent with the gap analysis research conducted by Parasuraman
et al. and others (Brown and Swartz, 1989; Parasuraman et al., 1985; Swartz and Brown,
1989). Furthermore, Katz, Larson, and Larson (1991) conducted a study with bank
customers that found that individuals tend to overestimate their waits. In addition, the
researchers asked customers to define what they would consider an acceptable wait.
Customers with longer definitions of acceptable wait times tended to be more satisfied
than customers with shorter definitions. Thus, if managers overestimate what consumers

22
consider an acceptable delay and consumers overestimate the time they have waited, there
is potential for unintended magnification of the actual wait experience.
Linking Service Evaluations to Service Capacity
Two methods have been used to predict how changes in service capacity will affect
consumer evaluations of the service. The first method is to operationalize the service
quality dimensions as measurable variables. For instance, Ozeki and Ikeuchi (1992)
studied service evaluation in a telephone service setting using a workflow simulator and
measures of service quality (MOSQs) for different components of the work process. The
authors defined an MOSQ as some quantifiable operationalization of service quality, such
as response time. Using simulation, the authors were able to see the effects of system
changes on these MOSQs, where a change in the desired direction implicitly represented
an improvement in quality.
The second method is to describe the relationship between changes in system
performance and service evaluations in terms of a cumulative probability distribution For
instance, Buxton and Gatland (1995) conducted an extensive simulation model that used a
customer satisfaction index to model the effects of work-in-process (WIP) and delivery
time on levels of customer satisfaction. This customer satisfaction index was expressed as
a probability distribution, where a delivery time (e.g., delivery within seven days) was
equated to an expected level of customer satisfaction This approach does not produce
exact results, however, it can allow managers to examine relationships between service
capacity and perceived service quality.
The previous section evaluated the literature describing how consumer wait times
might influence evaluations perceived service performance, such as perceived service
quality and customer satisfaction. Furthermore, it was shown how managers’ and
consumers’ perceptions of the wait experience could potentially magnify this relationship.

23
In addition, two methods of building this relationship into a simulation model were
summarized. The next section will identify the current state of development in the
measurement of perceived service quality and present research indicating that perceived
service quality is associated with intended future behavior.
Perceived Service Quality
Overview of the Conceptual Basis of Perceived Service Quality
Understanding how consumers of a service evaluate service quality is an issue of
importance to managers. It is clear that if a service provider understands how consumers
evaluate a particular service, managers can use these evaluations to focus on ways to
improve. However, developing a model to effectively evaluate service quality has been an
evolving and highly debated research issue (Gronroos, 1990).
Four basic characteristics apply to most services. First, services are essentially
intangible. Second, services usually focus on activities or information rather than
products. Third, services are produced and consumed simultaneously (i.e., they cannot be
inventoried). Fourth, the consumer is a participant in the production process (Gronroos,
1990, Lovelock, 1980). As Shostack (1984) describes them, “[sjervices are unusual in
that they have impact, but no form” (p. 134). Because of the intangibility of service
performance and the aspect of simultaneous production and consumption, it is generally
more difficult to develop quality indicators for services than for products (Heskett, 1987,
Parasuraman et al., 1985).
Quality, however, is an abstract concept and hard to adequately define. At its
most basic level it can be generally thought of from the point of view of Philip Crosby’s
“conformance to requirements” or J.M. Juran’s “fitness for use” definitions (Westgard and
Barry, 1986, p.5). However, it may be more useful to consider quality as the inherent or

24
implicit degree of excellence, value, or worth of a product or service measured by its
ability to satisfy a given need (Christensen and Penna, 1995, Westgard and Barry, 1986).
We can usually describe quality in terms of one or more of three dimensions: (1) a
structural dimension (i.e., the attributes of the facility, equipment, human resources, and
organizational structure that are the components of process); (2) a process related
dimension (i.e., the activities that make up the process); and (3) a technical or outcome
related dimension (i.e., the end result or effect of a process) (Angaran, 1993; Gronroos,
1990).
Much of the early research concerning service quality focused primarily on
identifying measurable dimensions of service quality. Two early developments of service
quality were Lehtinen and Lehtinen’s Interaction Quality and Gronroos’s Perceived
Service Quality (PSQ) models.
The basis of interaction quality was founded on the premise that service quality is
formed through the consumer’s interaction with the elements of a service organization.
This model suggested there were three elements of interaction quality: physical quality
(i.e., the tangible aspects of the service such as the equipment or facility); corporate
quality (i.e., the image of the service provider); and interactive quality, (i.e., the
consumer’s interactions with the service provider and other consumers) (Lehtinen and
Lehtinen, 1982 as cited in Gronroos, 1993; Parasuraman, Zeithaml, and Berry, 1985; and
Swartz and Brown 1989).
The PSQ model developed by Gronroos (1988, 1990) used the confirmation-
disconfirmation paradigm to define total perceived service quality as the gap between
expected service quality and experienced service quality. Expected service quality is the
level of quality that the consumer expects to receive. Experienced service quality is made
up of three basic dimensions: (1) technical quality (i.e , quality of the outcome of service);
(2) functional quality (i.e, quality of the service process); and (3) perceived image of the
organization (Gronroos 1988, 1990, 1992, 1993).

25
Parasuraman, Zeithaml, and Berry (1985, 1988, 1991) built upon the conceptual
basis formed by interaction quality and PSQ in their development of a gap analysis model
and the SERVQUAL instrument. Based on extensive focus group interviews, their initial
work described five potential gaps in the provision of services: (1) consumer expectation -
management perception gap, (2) management perception - service quality specification
gap, (3) service quality specifications - service delivery gap, (4) service delivery - external
communications gap, and (5) expected service - perceived service gap. Two fundamental
conclusions were developed from the use of this model First, perceived service quality is
a multidimensional construct; however, interaction with the service provider is the most
important variable in the assessment service quality. Second, there are often significant
perception gaps between the consumers and providers of a service, indicating the service
providers do not always understand the expectations of consumers (Brown and Swartz,
1989; Parasuraman et al., 1985; Swartz and Brown, 1989).
Building upon the gap analysis model, Parasuraman, Zeithaml and Berry (1985,
1988, 1991) continued to develop and validate an instrument called SERVQUAL.
SERVQUAL is the most widely known measurement of perceived service quality and its
development has had considerable impact on the systematic advancement of research
concerning perceived service quality of consumer services (Gronroos, 1993). Like the
PSQ model before it, SERVQUAL is based on the disconfirmation of expectations
paradigm (Gronroos, 1990; Parasuraman et al., 1988).
The SERVQUAL scale consists of 22 item pairs measuring five dimensions of
service quality: (1) tangibles, (2) reliability, (3) responsiveness, (4) assurance, and (5)
empathy. Factor analysis and reliability testing on data from four service industries were
used to develop the final SERVQUAL scale (Parasuraman et al., 1988).

26
Measurement of Perceived Service Quality
Although SERVQUAL is perhaps the most widely used instrument to measure
service quality, it has received criticism from other researchers who have begun to
examine the application of SERVQUAL in various settings. There have been three
general areas of concern regarding SERVQUAL: (1) use of difference scores, (2)
dimensionality of SERVQUAL, and (3) external validity.
One of the most debated issues that have surfaced concerning the SERVQU AL
instrument is the use of difference scores as prescribed by the confirmation-
disconfirmation framework. The first problem with the use of these scores is the timing of
the expectation measurement. Expectations may be altered during and after the service
experience, suggesting that expectations measured during or after service delivery are not
accurate representations of the expectations of the consumer at the point in time when
service commenced (Carman, 1990; Gronroos, 1993).
The second problem concerns the implicit nature of the perception measure. Since
the perception measure is already a comparison between what the consumer expected and
what they perceived as the actual service event, the expectation is already implied in the
perception measure. If expectations and perceptions are both measured then expectations
are, in effect, measured twice (Gronroos, 1993, Oliver 1993).
The third problem lies in the questionable reliability of difference scores (Brown et
al., 1993; Oliver, 1993; Peter et al., 1993). As Peter et al. (1993) state, "[difference
scores (1) are typically less reliable than other measures, (2) may appear to demonstrate
discriminant validity when this conclusion is not warranted, (3) may be only spuriously
correlated to other measures since they typically do not discriminate from at least one of
their components, and (4) may exhibit variance restriction." Therefore, the use of
difference scores may not be reliable, even when the reliability statistics suggest that the
instrument is reliable.

27
Concerns about SERVQUAL’s dimensionality have also surfaced in the literature.
As mentioned previously, Parasuraman, Zeithaml, and Berry’s development of
SERVQUAL resulted in a 22-item scale measuring five dimensions. However, several
authors have reported results that demonstrate that SERVQUAL’s five dimensions do not
always generalize across service settings. Studies conducted by Babakus and Mangold
(1992), Babakus and Boiler (1992), Brown et al. (1993), Carman (1990), Cronin and
Taylor (1992), and Headley and Miller (1993) and all fail in some degree to replicate the
original dimensions.
The external validity (i.e., generalizability) of SERVQUAL has been questioned
because of the evidence that the level of usefulness of the instrument “as is” may vary
depending on the service. It has been shown that the 22-items do not necessarily load on
the same factors (Babakus and Boiler, 1992; Brown et al., 1993; Carman, 1990, Headley
and Miller, 1993; Taylor and Cronin, 1994). In addition, some researchers have suggested
that significant wording changes are necessary so that the items are useful in a particular
service setting (Babakus and Mangold, 1992, Carman, 1990). Furthermore, it may
actually be necessary to modify the length the scale depending on the setting (Babakus and
Boiler, 1992; Babakus and Mangold, 1992; Carman, 1990).
Because of these problems, other approaches in measuring service quality have
been suggested. One of these approaches is to use a perceptions-only scale for the
measurement of service quality, such as the SERVPERF instrument tested by Cronin and
Taylor (1992) Perceptions-only scales avoid the concerns about the use and reliability of
difference scores, without sacrificing scale performance. Perception-only scales also have
the advantage of being easier to administer, primarily because the subject does not have to
answer both the expectation and performance question subsets. This advantage greatly
enhances the practicality of the scale (Babakus and Boiler, 1992; Brown et al., 1993; and
Cronin and Taylor, 1992, 1994; Headley and Miller, 1993; McAlexander, 1994). Zeithaml
et al. (1996) recently recognized the value of perceptions-only scales such as SERVPERF.

28
They state, “The perceptions-only operationalization is appropriate if the primary purpose
of measuring service quality is to attempt to explain the variance in some dependent
construct..(p.40).
SERVPERF essentially eliminates the expectation portion of the SERVQUAL
scale and focuses entirely on service performance. Cronin and Taylor (1992) tested
SERVPERF (i.e., perceptions-only) versus SERVQUAL (i.e., perceptions-minus-
expectations). Four service industries were analyzed: banking, pest control, dry cleaning,
and fast food. The results of the LISREL and oblique factor analysis procedures did not
indicate that the dimensionality conformed to the five-factors proposed by Parasuraman et
al., 1988. Flowever, strong reliability scores were exhibited for all for industries
(coefficient alphas greater than 0.800). Based on these results, and the failure of other
studies to exactly replicate the five dimensions, Cronin and Taylor (1992) suggest that
items in SERVQUAL (and hence SERVPERF) should be considered a uni-dimensional
measure of service quality rather than a multi-dimensional measure. In addition,
SERVPERF explained slightly more of the variation in perceived overall service quality,
satisfaction, and purchase intention than SERVQUAL.
Significant debate has occurred in the literature regarding Cronin and Taylor's
(1992) performance only approach to measuring perceived service quality (Cronin and
Taylor, 1994; Parasuraman et al., 1994). While, Parasuraman et al. (1994) concede that
performance only measures such as SERVPERF tend to offer greater predictive power,
they do not have as much diagnostic value as disconfirmation measures such as
SERVQUAL. In response, Cronin and Taylor (1994) suggest that the SERVPERF scale
could be used as a summed or averaged service quality score that might be plotted over
time. Therefore, a performance-only measure, such as SERVPERF, should be used when
the objective is to obtain an overall measure of service quality that can be used as a
dependent variable and analyzed over time.

29
A modified SERVPERF scale will be used in this research because it eliminates the
disadvantages of using the difference score approach. In addition, the elimination of the
expectation portion of the scale reduces the number of questions that the respondent is
required to answer, which enhances ease of administration and should improve the
response rate. Furthermore, one of the goals of the proposed research is to study the
relationship between wait times and service quality. Since SERVPERF can be used as a
summed interval score, this measure is more useful than SERVQUAL for predictive
purposes.
Perceived Service Quality and Behavioral Intention
Presumably, there are two reasons for measuring perceived service quality. First,
so we can understand and improve the shortcomings of service delivery. Second, to
understand the impact that service quality has on future behavior. As mentioned in the
previous chapter, behavioral intentions, such as return intention and recommendation, are
significant factors in maintaining an effective drug information service. Authors from the
health care and other fields have studied the behavioral consequences of service quality.
Babakus and Mangold (1989, 1992), Boulding et al. (1993), Cronin and Taylor
(1992), Headley and Miller (1993), Parasuraman et al. (1991), and Zeithaml et al. (1996)
all used modified versions of the SERVQUAL scale to measure service quality and its
influence on future intentions in a wide variety of service settings. These studies indicated
that perceived service quality was related to loyalty, switching intention, complaining,
compliment and recommendation intention, and return intention. Dube’-Rioux et al.
(1989) and Bitner (1990) used alternative instruments to look at the issue of service
quality and future intent. These two studies used role-playing methodologies involving
restaurant and air travel delays, respectively. The results of these studies were consistent

30
with those using the modified SERVQUAL instruments, where service quality was related
to intended future behavior.
Summary of the Literature
Queuing theory describes service capacity as a function of the arrival rate (A,), the
service rate (p), the number of servers (s). There are typically six options available for
reducing waiting times and delays in a system: (1) add servers, (2) improve the service
rate, (3) increase queue size, (4) change the arrival rate, (5) reduce the variance in service
times or interarrival times, and (6) change the queue discipline. There are at least two
methods of building the relationship between waiting time and perceived service quality
into a simulation model: (1) operationalizing service quality as a measurable variable, and
(2) estimating the relationship as a probability distribution.
Perceived service quality is a concept that is still evolving. Currently,
SERVQUAL is perhaps the most widely used instrument to measure perceived service
quality. The SERVQUAL scale consists of 22 item pairs measuring five dimensions of
service quality: (1) tangibles, (2) reliability, (3) responsiveness, (4) assurance, and (5)
empathy. Recently, however, it has received criticism from other researchers involving
three general areas of concern: (f) SERVQUAL’s use of difference scores, (2) the
dimensionality of SERVQUAL, and (3) SERVQUAL’s external validity.
SERVPERF is a scale that avoids many of the concerns listed above by
ascertaining only consumer perceptions (as opposed to expectations and perceptions)
regarding the five dimensions of service quality listed above. Arguments have also been
made that SERVPERF exhibits stronger reliability and validity than SERVQUAL.
SERVPERF also has the advantage of being easier to administer, thus enhancing the
practicality of the scale.

31
The literature also suggests that perceived service quality is related to future
consumer behavior, such as return intention and intent to recommend. In addition, there is
evidence to hypothesize that evaluations of service quality are affected by waiting time and
delays in service. Furthermore, it appears that management tends to overestimate
acceptable waits and customers tend to overestimate actual waits.
Chapter three will use the information presented in this literature review to
construct a framework for the variables to be studied Hypotheses and specific research
questions will then be developed based on this framework. Chapter four will present the
methods used to test these hypotheses and specific research questions.

CHAPTER 3
RESEARCH FRAMEWORK AND HYPOTHESES
Research Framework
The previous two chapters have described the potential relationships among
service capacity, wait times, perceived service quality, and behavioral intention. Possible
changes for improving the performance of service systems, from a queuing theory
standpoint were also suggested. This chapter builds on the concepts presented in the
introduction and the literature review by presenting a research framework for the variables
used in this study.
There are four primary relationships necessary to understanding the research
framework for this study. First, service times can be described using a queuing paradigm
as a function of the arrival rate, the service rate, the number of servers, and the priority
discipline. Important output variables of the system would include the expected service
times and queue waits, the percentage of service delays, information regarding number in
the system and queue lengths, and the utilization rates of the servers. Second, waiting
times and service delays are related to consumer perceptions of service quality, and these
perceptions are related to future behavioral intention. However, consumers may not
accurately estimate actual waiting times; therefore, perceived waiting time may be a more
important variable. Third, by creating and manipulating a valid simulation of the service
system (i .e., a computer model of the actual system), we can propose changes in the
system that will decrease the amount of time a consumer waits for service to be
completed Fourth, service capacity might be optimized by defining a mathematical
relationship between perceived service quality and service times or service delays.
32

33
Definitions of the concepts used in the theoretical framework for this study (Figure
3-1) are discussed below. Where applicable, the first definition refers to the concept as it
applies to the empirical measurement and the second definition refers to the concept as it
applies to the simulation.
Arrival Rate (A,): (1) The empirically observed interarrival distribution of consumer
information requests and questions in the Drug Information Service (DIS). (2) The
probability distribution input into the simulation model to predict the interarrival
distribution of consumer information requests and questions into the DIS.
Actual Service Time: The total time required to research an answer to the question,
obtain an approval, and return the answer to the caller that was empirically observed for
service process in the DIS. Service time is also referred to as waiting time or response
time.
Behavioral Intention: A subject's assessment regarding their future intentions regarding
the service. More specifically, it relates to whether or not the subject intends to use the
service again or recommend the service to a colleague.
Expected Number in System (L): A simulation output variable that indicates the average
number of uncompleted information requests and questions in the DIS.
Expected Queue Length (Lq): A simulation output variable that indicates the average
number of information requests and questions in the DIS that have not yet started the
research process.

34
Expected Time in Queue (Wq): A simulation output variable that indicates the average
amount of time that questions must wait in the queue before starting the research process.
Expected Time in System (W): A simulation output variable that indicates the average
total amount of time that a question or information request spends in the system.
Expected Utilization Rate (p): A simulation output variable indicating the average
percentage of time that servers were busy.
Overall Service Quality (OSQ): A subject's overall perception of the service quality of
the drug information service.
Perceived Service Quality (PSQ): A subject's evaluation of the service quality of the
drug information service based on the items in the SERVPERF instrument.
Perceived Service Time: A subject's perceptions regarding the response time of the drug
information service. More specifically, it relates to perceptions regarding (1) the
acceptability of the response time, (2) the usefulness of the answer once the response was
received, (3) the subject's desire for quicker responses from the DIS, and (4) whether the
response time was shorter, equal, or longer than expected.
Queue Discipline: (1) The method currently used in the drug information service to
prioritize consumers for service, as described by the service providers during personal
interviews. (2) A simulation input used to describe the way in which servers decide the
order in which information requests and questions are handled by the drug information
service.

35
Service Delay: A state indicating whether the actual service time was longer than the
response time needed by the caller
Service Rate (p): (1) The empirically observed service time distributions of consumer
information requests and questions in the Drug Information Service (DIS). (2) The
probability distribution input into the simulation model to predict the service times for the
steps in the service process in the DIS.
Staffing Level (s): (1) The observed number of individuals available to serve consumers
and their roles in handling consumer information requests and questions. (2) A simulation
input describing the number of individuals available to handle service requests during the
various steps of the service process.
Research Hypothesis and Specific Research Questions
The literature has proposed that significant positive relationships should exist
among measures of PSQ, OSQ, and intended future behavior (Boulding et ah, 1993;
Cronin and Taylor, 1992; Parasuraman et ah, 1988). Hypothesis one (HI) assesses the
relationship between PSQ and OSQ. Hypotheses two (H2) and three (H3) are aimed at
ascertaining the strength of the relationship between PSQ and behavioral intention (i.e.,
intent to call again and intent to recommend service), and between OSQ and behavioral
intention.
HI: There is a positive relationship between evaluations of perceived service quality
(PSQ) and evaluations of overall service quality (OSQ).
H2a: Intention to use the service in the future is positively associated with evaluations
of perceived service quality (PSQ).

36
H2b: Intention to recommend the service to a colleague is positively associated with
evaluations of perceived service quality (PSO).
H3a: Intention to use the service in the future is positively associated with evaluations
of overall service quality (OSO).
H3h: Intention to recommend the service to a colleague is positively associated with
evaluations of overall service quality (OSO).
Previous research has indicated the powerful role of customer perceptions on
evaluations of perceived service quality, including perceptions regarding service time. It
has been shown that consumers often cannot accurately ascertain the amount of time
within which the service was completed (Katz, Larson, and Larson, 1991). If consumers
cannot accurately evaluate the actual service time, then perceived time may be a more
important predictor of perceived service quality than actual time. For instance, Tom and
Lucey (1995) found that customers were more satisfied with the service when the wait
was shorter than expected than when the wait was longer than expected. Hypothesis four
(H4) is aimed at gaining more information regarding how callers' perceptions regarding
the response time of the service are related to their attitudes regarding PSQ.
H4a: Acceptability of the response time of the service is positively associated with
evaluations of perceived service quality (PSO).
H4b: Perceived usefidness of the information once the response was received is
positively associated with evaluations of perceived service quality (PSQ).
H4c: Perceived quickness of response is positively associated with evaluations of
perceived service quality (PSQ).
H4d: Deviations from expected response times are positively associated with
evaluations of perceived service quality (PSO).

37
As reported in chapter 2, the results of Bolton and Drew (1994), Clemmer and
Schneider (1989a), Davis (1991), Hui and Tse (1996), and Katz, Larson, and Larson
(1991) have all reported evidence to support that an inverse relationship exists between
evaluations of the service (i.e., perceived service quality or satisfaction) and waiting time.
Additionally, it has been suggested that the relationship between waiting times and
perceived service quality may be non-linear (Davis, 1991; Larson, 1987). Hypothesis five
(H5) evaluates the relationship between service times and evaluations of perceived service
quality for generalizability to the drug information setting.
H5a: There is a significant inverse relationship between evaluations of perceived
service quality (PSO) and actual service time.
H5b: There is a non-linear relationship between actual service time and perceived
service quality (PSO).
The literature often does not make a distinction between waiting times and delays
in service; however, both have been shown to effect perceived service quality. A delay
occurs whenever the actual response time is longer than the promised time. Taylor
(1994a) and Taylor and Claxton (1994) reported that subjects who experienced boarding
delays in an airport setting had lower evaluations of overall service than non-delayed
subjects. However, this effect may be moderated by the degree of filled time and
consumers' perceptions regarding how much control the service provider had over the
delay. Similarly, Dube'-Rioux (1989), using role-playing scenarios for a restaurant setting,
also found that delays could affect customer satisfaction; however, the research suggested
that perceived need and timing of the delays were also important. Based on this research,

38
hypothesis six (H6) assesses the association between delays in service and evaluations of
PSQ
H6: Delays in service are negatively related to evaluations of perceived service
quality (PSQ).
Additionally, there is little information in the service quality or satisfaction
literature concerning the relationship between actual and perceived service time, therefore,
it is unclear to what degree they are actually associated. However, several authors do
suggest that perceived service time may be more important that actual service time in
relation to perceived service quality (Hornik, 1982, 1984; Katz, Larson, and Larson, 1991;
Taylor, 1994a). Hypothesis seven (H7) explores the associations among the variables
measuring perceived service time, actual service time, and service delays.
H7a: There is a positive relationship between actual service time and perceived service
time.
H7b: There is a positive relationship between service delays and perceived service time.
The previous chapter discussed ways in which the performance of a system could
be improved. It is believed that by changing service capacity in ways that decrease the
overall service times and the number of service delays, consumer evaluations of perceived
service quality may be improved. Using a queuing theory framework to optimize service
capacity suggests that we should examine the effects of changes in staffing levels (s) and
service rates (p). Specific research questions one through three (Rl, R2, and R3) assess
the relative impact and sensitivity of these methods in terms of improvements in service
times and service delays.

39
Rl: How do changes in staffing levels and service rates impact simulated service
times in the drug information service?
R2: What combination of changes in staffing levels, service rates optimizes the system
for delays in service when compared against service quality and cost in the drug
information service?
R3: How sensitive is this solution to random variation in the system variables (e.g.,
arrival rate)?

40
INPUTS
1. Arrows relating to study hypotheses and specific research questions are labelled with the
number of the hypothesis (H) or research question (R).
2. Arrow direction does not necessarily imply causality.
Figure 3-1. Hypothesized Framework

CHAPTER 4
METHODS
Overview
This chapter describes the methods used to test the research hypotheses and
explore the specific research questions presented in the previous chapter It includes a
description of the study location, the sources of data, the sample selection procedures
used in the study, and methods of data collection. In addition, it describes the techniques
used to develop and validate the service quality questionnaire and simulation program. It
concludes with a description of the data analysis procedures used to test the hypotheses
and research questions.
Study Location
This study was conducted at the Drug Information and Pharmacy Resource Center
(DIPRC) at Shands at the University of Florida in Gainesville, Florida. The DIPRC
accepts drug information questions from practitioners from all over the North Florida
region. The DIPRC accepts calls only from practitioners (e g. pharmacists, physicians,
nurses, law enforcement, etc.). Calls from the public are redirected to other resources.
The DIPRC categorizes callers into three categories: (1) subscribers, (2) non-subscribers,
and (3) University of Florida Health System employees. The service is provided free of
charge to all callers; however, subscribers pay a voluntary membership fee to help support
the DIPRC
Questions are presented to the DIPRC from various sources, including telephone,
facsimile (fax), electronic mail, and through in-person visits to the center. However, the
41

42
vast majority of calls are presented to the center via telephone. The DIPRC classifies
questions into 14 general categories: (1) drug availability, (2) drug dosage and
administration, (3) drug identification, (4) drug interactions, (5) drug therapy and efficacy,
(6) drug use in pregnancy or lactation, (7) investigational drugs, (8) IV compatibility or
stability, (9) legal, (10) other, (11) pharmacokinetics, (12) side effects or adverse effects,
(13) toxicology, and (14) veterinary drugs.
Besides the nature of the question, subscription status, and profession, callers are
asked to provide various demographic information such as name, address, phone number
and/or fax number. Callers are also asked for the amount time that they can allow the
center to research the question The DIPRC categorizes these times into four categories:
(1) within 15 minutes (stat), (2) within the day (today), (3) by a specific date (date), and
(4) no rush. Callers may request an oral response to their questions, a written response, or
both.
The DIPRC is usually staffed by three Pharm.D. students during their clerkship
rotations. Sometimes internship students, visiting international students, and hospital
pharmacy residents will assist the DIPRC in answering questions. However, the DIPRC
does not usually know when additional students or residents will become available. In
addition, the level of contribution that they provide is often limited and unpredictable.
A drug information resident is also usually present in the DIPRC, however, this
resident has varying duties and does not usually spend their time directly answering
questions. Two co-directors manage the service’s operation and approve student
responses to information requests. The drug information resident can also approve
student responses once he or she is qualified and experienced, as determined by the co¬
directors.

43
Data Sources, Sample Selection, and Data Collection Procedures
The data for the empirical parts of this study were obtained by non-experimental
methods. The parts of this study involving computer simulation were experimental in
design. Data for this study was collected from six sources. The first two were taken from
historical data sources (i.e., historical data sheets and a database documenting the past
monthly workload). The third, fourth, and fifth data sources were collected concurrently
during the data collection period from June, 1997 through July, 1997 (i.e., specially
designed data collection forms, personal interviews, and service quality questionnaires).
The last source of data was obtained from computer runs of the simulation program.
Historical Data Sheets
The historical data sheets are the standard forms that the DIPRC uses to document
responses to caller questions and information requests (Appendix H). Every question
answered by the DIPRC is recorded onto one of these data sheets. In addition, all
information regarding a particular call is written on or attached to these data sheets. All of
the archived data sheets were retrieved for September, 1996 through May, 1997
(approximately nine months) resulting in a total historical data sheet sample size of 2,385.
Information taken from the historical data sheets included the file number and the
requestor's name, profession, and subscription status. In addition, the question type, when
the response was needed, the response type requested, the date and time received, and the
date and time completed were recorded. From the data sheets, it was also possible to
determine whether or not the service was delayed past the time needed, and whether or
not the same person who answered the call also completed the answer. All data were
entered into a Microsoft Excel database.

44
Historical Database
A historical database maintained by one of the co-directors of the DIPRC
documenting the monthly workload since 1987 was used to analyze monthly arrivals for
seasonal trends. The data consisted of a total of 126 data points (i.e., 11 each for January
through June and 10 each for July through December). This database contained the total
number of questions answered each month as well as the average daily number of
questions answered for the month. The average daily number of questions was produced
by dividing the total number of questions answered during the month by the number of
days the DIPRC was open to take calls. This data was made available to the principal
investigator in spreadsheet format.
Data Collecton Forms
A special form was designed to facilitate the specific data collection needs of this
study. This form was very similar to historical data sheets described above; however,
modifications were made to incorporate space for additional information, such as the
recording of dates and times for specific service activities (Appendix I). These data
collection forms were collected several times each week from June, 1997 through July,
1997, resulting in a total sample of 526 forms. By comparing the file numbers for the data
collection forms obtained against a separate entry log, it was determined that 16 data
collection forms were missing and could not be located. Therefore, over 97% of the data
collection forms filled out during this period were located and entered.
Information taken from the data collection forms included:
1. The file number.
2. The requestor's name and contact information., the requestor's profession, and
the requestor's subscription status.
3. The question type.

45
4. The time the response was needed and the response type requested.
5. The date and times for each of the four work activities (i.e., take call, research
answer, approve answer, and return answer to caller).
6. Whether or not the service was delayed past the time needed.
7. The persons approving and completing the answer.
8. The number of persons working on the question.
In addition to the information mentioned above available from analyzing the
historical data sheets, the activity based service times on the data collection forms allowed
for a more detailed analysis of the service processes of the DIPRC. This provided
information necessary to the development of a more robust simulation. All data were
entered into a special Microsoft Access database specifically designed for this project.
Personal Interviews
Personal interviews with DIPRC co-directors and externship students were
conducted in order to obtain a more thorough understanding of the various aspects of the
center’s operations, including the process for answering questions and the priority
discipline used by the students to organize work. Twelve personal interviews were
conducted from May, 1997 to July, 1997. The interviewees consisted of ten Pharm D.
students and the two co-directors for the DIPRC. The students were all interviewed
during their third rotation week. The interviews were semi-structured, using interview
outlines to maintain consistency (Appendices J and K). The interviews were audio
recorded for accuracy of recall. The interview outlines were developed based on
suggestions made by Stewart and Cash (1988).

46
Service Quality Questionnaires
The fourth data source used was a service quality questionnaire sent to
practitioners using the service from June, 1997 through July, 1997. This questionnaire
was administered to callers after service completion (Appendix M). This questionnaire
assessed perceived service quality (PSQ) using a modified SERVPERF scale, perceived
service time, perceived overall service quality (OSQ), and behavioral intention. The
inclusion criterion for receiving a questionnaire was any practitioner who submitted a drug
information question to the DIPRC during the study period. Callers were excluded from
this portion of the study if they had already been sent a questionnaire (i.e., callers were not
surveyed more than once). Out of the available 526 samples, 332 questionnaires were sent
out to practitioners, 183 samples were repeat callers who had already been sent a
questionnaire, and contact information was not available for 11 of the callers.
A data collection procedure based on the "Total Design Method" developed by
Dillman (1978, 1994) was used to maximize the response rate of the questionnaire. There
were three phases to this procedure. First, eligible subjects identified from the data
collection forms were sent a questionnaire (Appendix M) along with a cover letter from
the co-director and the principal investigator explaining the purpose of the research and
asking for the subjects' participation (Appendix L). A self addressed, stamped envelope
was also enclosed for the subject to return the survey. Second, approximately one week
after the questionnaires were mailed, a reminder post card (Appendix N) was sent to non¬
responders asking subjects to fill-out and send in the questionnaire or to contact the
principal investigator if they did not receive a questionnaire. Subjects who replied to the
post card saying that they never received a questionnaire were promptly sent another.
Third, approximately two weeks after the first questionnaire was sent, an attempt was
made to contact each non-responding subject by telephone. If the subject was reached,
the interviewer attempted to ascertain the reason for the non-response, and, if still

47
applicable, the subject was reminded to send in the questionnaire. If the subject was not
reached, an attempt was made to leave a message or reminder through a receptionist or
co-worker.
All of the pre-test and main questionnaires were developed using the Survey Pro
for Windows software program. This program also provides a facility for data entry and
export which was used to record the responses to the questionnaires for future analysis.
Responses to the final question (i.e., Q35) asking for additional comments were recorded
into a word processing document along with all other comments written next to the
individual items.
Simulation Runs
The sixth data source used was the output generated by the simulation program.
The simulation model was constructed using the GPSS/H (General Purpose Simulation
System) simulation language produced by Wolverine Software for use on an MS-DOS
based personal computer. Results from the simulation runs were entered into a database
for the purposes of summarization and statistical analysis. Five separate groups of
simulation runs were made. The first three runs were used to verify and validate the
simulation program. The fourth run was used to test specific research questions one and
two. The final run was used to determine the sensitivity of the optimal solution as
described by the third specific research question.
Procedures for Protecting Privacy and Confidentiality
Each record of information gathered from the historical data sheets or concurrent
data collection forms was coded with an identification number. For tracking purposes
only, surveys sent out using information collected from the data collection forms were also
coded with an identification number. Once all mail questionnaires were returned, all post

48
card and telephone follow-ups completed, and all data entered and verified, the portion of
the database containing the callers' contact information was deleted Furthermore, written
comments transcribed from the questionnaires were edited to exclude any personal
references. All historical data sheets were returned to the DIPRC once data entry was
completed. Hard copies kept of the data collection forms were coded with the
identification number and personal information contained on these hard copies was
masked using permanent marker. In addition, any comments or quotes used from the
personal interviews were edited so that they could not be traced to the speaker This
project was reviewed by the Health Center Institutional Review Board at the University of
Florida and approved on July 29, 1997.
Sample Size Calculations
Required Number of Data Sheets
In order to estimate the sample sizes necessary to provide accurate estimates of the
arrival and service time distributions required to construct the simulation of the DIPRC,
the data sheets from approximately the first week of data collection were compiled and
analyzed. The necessary sample sizes were calculated using the method proposed by
Mendenhall, Wackerly, and Scheaffer (1990) for establishing a large sample 95%
confidence interval for a given standard deviation and error of estimation The sample size
calculations are summarized in Table 4-1.
The largest of these estimates is the 335 samples required for a 95% confidence
interval using a standard deviation of 45.6 minutes and an error of estimation of 5 minutes.
Since the DIPRC receives between 250 to 300 calls per month, two months of data were
collected for this study in order to satisfy this sample size requirement. This time period
resulted in an actual sample of 526 data sheets, which provided a conservatively large

49
sample from which to estimate the required parameters and allow considerations for
missing data.
Table 4-1. Required Sample Sizes for Selected System Parameters
Parameter
Mean
St. Dev.
Error of
Estimation
Required
Sample Size
(a=0.05)
Interarrivals
45 min.
43 min.
5 min.
296
Reception of Call
4.1 min.
3.7 min.
1 mm.
55
Service Time
42.5 min.
45.6 min.
5 min.
335*
Approval
1.9 min.
3.7 mm.
1 mm.
54
Return Answer
5.9 min.
8 8 min.
1 min.
310
* Largest, sample size required.
Required Number of Questionnaires
The primary statistical methods used in testing relationships involving the items in
the service quality instrument were correlation (HIthrough H4, H6, and H7) and linear
regression (H5). As such, there were two considerations driving the sample size
determination for the service quality questionnaires. Primary consideration was given to
the hypotheses to be tested. Secondary consideration was given to the required number of
data points necessary to conduct principal components factor analysis on the SERVPERJF
portion of the survey. As recommended by Sawyer and Ball (1981), the type I error rate
for sample size calculations was set at a=0.05, and the type II error rate was set at (3=0.20
(where power equals 0.80). The rationale for these error rates is based on the assumption
that, for this study, committing a type I error was more critical than committing a type II
error. Therefore, a and P were selected so that only a small chance existed that the null
hypothesis would be rejected when no true differences exist and a reasonably high
probability of rejecting the null hypothesis when differences do exist.
First, HI through H4 and H6 through H7 used correlation as the primary statistic
to detect significant associations among the study variables. The software program "PC-
SIZE" (Dallal, 1986) was used to estimate the required sample sizes for the correlation

50
All algorithms used in the program are based on published statistical literature (see Dallal
(1986) for references). The results produced from the program itself were verified by
Dallal (1986) against a selection of entries from tables presented in Cohen (1977), Fleiss
(1981), and Odeh and Fox (1975). The results from the pre-test and reports of similar
correlations in the literature suggested that correlations as small as 0.25 may be
significant. In order to detect a statistically significant correlation of at least 0.25 with a
power of 0.80 and a level of 0.05, a sample size of at least 123 surveys is necessary.
Second, rules-of-thumb for conducting factor analysis generally suggest sample
sizes of at least 100 or between 5 and 10 samples per variable to be analyzed, whichever is
greater. The actual number of samples needed depends of the amount of variability
explained by the factors, the strength of the factor loadings, and the communalities of the
individual variables (Crocker and Algina 1986; Stevens 1996). The factor analysis for the
pre-test was conducted with an average of 4.65 samples per variable. Six rotated factors
explained 71 percent of the variance, and nearly all of the items loaded strongly on one of
the six factors. Furthermore, all but two of the items had communalities greater than 0.6.
This evidence suggests that five samples per variable is adequate to factor analyze the
SERVPERF portion of the questionnaire. Since there are 20 variables to be analyzed,
then 20 times 5 equals a minimum sample size of 100.
Third, H5 used regression analysis to examine the relationship between service
time (in minutes) and PSQ. Determining the number of samples required for regression
analysis is complex since statistical power for this method is a function of both the number
of predictors used in explaining the variance in the dependent variable and the effect size
(as measured by R2) that the researcher wants to detect (Green 1991). However, S B.
Green (1991) has developed a two-step methodology for estimating sample sizes for
regression purposes that compares favorably with Cohen's (1988) more complicated
procedures for multiple regression power analysis. For a statistical power of at least 0.80,
the minimum sample size necessary is L divided by f (i.e., N= L if). Where L = 6.4 +

51
1 65m - 0.05m2 and f - R2/(l - R2), where m is the number of predictors to be used in the
model (Green 1991, p. 504). To detect a significant relationship between service time and
PSQ with an R2 of at least 0.10 (i.e., small effect) using simple linear regression (i.e., one
predictor), results in an L = 8 and an f = .11. Using these numbers in the equation for N
presented above results in a required minimum sample size of 72.
Therefore, the minimum number of questionnaires needed to conduct the
hypothesis tests with sufficient power was 123. The pre-test achieved a response rate of
approximately 67% for a sample of 201. Based on the pre-test response rate and
discussions with the DIPRC co-directors, a minimum response rate of 50% for the main
questionnaire was reasonably anticipated. Thus, a sample of at least 246 callers was
calculated as the necessary sample size for the questionnaire portion of this project.
During two months of data collection a sample of 332 callers was identified and sent
questionnaires, which was considered sufficient for the purposes of this study.
Study Variables
The previous chapter introduced definitions for the variables used in this study as
presented in the hypothesized research framework. This section discusses how these
variables were measured using the data sources described above.
Arrival Rate (A,): Interarrival times were estimated by arranging the arrival times from the
historical data sheets (Appendix H) and data collection forms (Appendix I) in ascending
order by date and time of arrival. The interarrival time was obtained by subtracting the
arrival time for the previous record from the time of the current arrival. For example, if
the current record has an arrival time of 12:00 p.m. and the previous arrival occurred at
11:30 a m., then the interarrival time was 30 minutes. Only interarrivals within each day
were estimated.

52
Actual Service Time: The actual service time refers to the amount of time required to
respond to a question. This was obtained by subtracting the "End Time" for receiving a
call from the "Start Time" for returning an answer to a caller. These data were taken from
entries made on the data collection forms (Appendix I).
Behavioral Intention: Behavioral intention was measured by two items (items 33 and 34
in Appendix M). The first item measured was worded, "I intend to use this service in the
future " The second item measured was worded, "I would recommend this service to a
colleague." Each of these perceptions was measured on a 7-point scale with a one
representing "Strongly Agree" and a seven representing "Strongly Disagree". Thus,
lower scores indicate greater intention.
Expected Number in System (L): A simulation output variable that indicates the average
number of uncompleted information requests in the system. This value was obtained from
the queue reports generated by GPSS/H for the queue labeled "TOTALQ", which collects
queuing and service time information relating to the entire service process
Expected Queue Length (Lq): A simulation output variable that indicates the average
number of questions in the system that have not yet started the research process. This
value was obtained from the queue reports generated by GPSS/H for the queue labeled
"BOARDQ", which collects queuing and service time information relating just to the
period in which questions spend in the queue before starting the research process.
Expected Time in Queue (Wq): A simulation output variable that indicates the average
amount of time that an information requests or questions must wait in the queue before
starting the research process. This value was obtained from the queue reports generated
by GPSS/H for the queue labeled "BOARDQ".

53
Expected Time in System (W): A simulation output variable that indicates the average
total amount of time that a question or information request spends in the system. This
value was obtained from the queue reports generated by GPSS/H for the queue labeled
"TOTALQ".
Expected Utilization Rate (p): A simulation output variable indicating the average
percentage of time that servers are busy. This was obtained from the facility reports
generated by GPSS/H, which reports utilization as the percentage of the total time that the
facilities (i.e., students) were captured (i.e., utilized). Utilization rates for each of the
simulated students were averaged together to obtain an expected overall utilization rate.
Overall Service Quality (OSQ): A subject's overall evaluation of the service quality of
the drug information service. This perception was evaluated using a single-item measured
on a 6-point scale (item 28 in Appendix M). The item was worded "The overall quality of
the services provided by the DIPRC is best described as." An "Excellent" rating was
scored as a one and an "Unacceptable" rating was scored as a six. Thus, lower scores
indicated higher perceived OSQ.
Perceived Service Quality (PSQ): A subject's evaluation of the service quality of the
drug information service was measured by summing the individual items of the
SERVPERF instrument to obtain an overall perceived service quality score. During the
main phase of the study, 20 items initially composed the PSQ scale (numbers four through
twenty-three in Appendix M). These items were measured on a seven-point scale anchored
with "Strongly Agree" (receiving a value of one) and "Strongly Disagree" (receiving a
value of seven). Negatively worded items were reversed scored. One item was dropped
from the scale, resulting in a final measure composed of 19 items. Therefore, PSQ had a
possible range of 19 to 133 points, with lower scores indicating higher perceived quality.

54
Perceived Service Time: A subject's perceptions regarding the response time of the drug
information service. This variable was measured by four questionnaire items. The first
was question 24 “Acceptable Time”. This item was worded, "The amount of time that it
took the DIPRC to respond to my most recent question was acceptable." The second was
question 25 “No Longer Useful”. This item was worded, "By the time I received a
response from the DIPRC, the information was no longer useful to me." The third was
question 26 “Quicker Response”. This item was worded, "I wish the DIPRC could
provide a quicker response to my questions." All three of these items were scored on a
seven-point scale anchored with "Strongly Agree" (receiving a value of one) and "Strongly
Disagree" (receiving a value of seven). The fourth item was worded "The amount of time
that it took the DIPRC to respond to my most recent question was." This item was also
scaled on a seven-point scale; however, it was anchored by "Much Shorter than Expected"
(receiving a value of one) and "Much Longer than Expected" (receiving a value of seven).
Queue Discipline: The queue discipline is the method currently used in the DIPRC to
prioritize questions as they arrive. This discipline was described by the students and co¬
directors during personal interviews. The queue discipline used by the simulation model
was developed using these descriptions.
Service Delay: This is a dichotomous variable measured by comparing the observed
service time with the response time needed as reported on the data collection forms and
historical data sheets. If the service time was longer than the time needed then the variable
was coded with a value of one indicating "Delayed". If the service time was shorter than
the time needed then the variable was coded with a value of zero indicating "Not delayed".

55
Service Rate (p.): The probability distribution that is input into the simulation model to
describe the service times for the steps in the service process in the DIS. These probability
distributions were estimated from the time information obtained from the historical data
sheets and concurrent data collection forms discussed above.
Staffing Level (s): A simulation input describing the number of individuals available to
handle service requests. Under "Normal Operation" this variable had a value of three,
indicating that three students were responsible for staffing the DIPRC. This value was
varied from one to five in the simulation model to assess the impact of changes in staffing
levels on the results of the simulation model.
Questionnaire Development and Validation
As described in the literature review, service quality instruments based on the items
in the SERVQUAL scale are in wide use. Therefore, the consensus among those using
the scale appears to be, generally, that the items composing the scale are an adequate
representation of perceived service quality, barring modifications necessary for
applicability. Furthermore, the procedures used by Parasuraman, Zeithaml, and Berry
(1985, 1988) to develop the individual items appear to be well-supported (Cronin and
Taylor, 1992; Oliver 1993). Therefore, the question of whether the items in the
SERVPERF instrument actually measure the construct of perceived service quality is not
at issue in this research. However, because the SERVPERF instrument was modified for
use in the drug information setting, it was necessary to validate the content of the
questionnaire, explore some general aspects of construct validity, establish internal
consistency, and report the potential for non-response bias.

56
Content Validity
Content validity is an assessment of whether the items in a scale adequately
measure the construct of interest (Crocker and Algina, 1986). Three methods were used
to establish the content validity of the instrument. First, as suggested by Crocker and
Algina (1986) and DeVellis (1991), an expert panel was asked to evaluate the service
quality instrument used in this study for clarity and readability. In addition, the panel was
also asked to assess whether or not the instrument covered all of the domains that they
believed were important in measuring the quality of a drug information service. This panel
consisted of.
1. Two co-directors of the Drug Information and Pharmacy Resource Center at Shands
and the University of Florida.
2. The director and employees of the Arkansas Poison and Drug Information Center at
the University of Arkansas for Medical Sciences.
3. Three recent callers to the DIPRC identified from the data sheets (2 pharmacists and 1
nurse).
4. Three senior graduate students with pharmacy degrees as well as experience and
educational background in survey design and methodology.
5. Two professors experienced in survey research and familiar with the literature
involving perceived service quality.
The recent callers and the graduate students were asked to complete the
instrument and report the length of time spent on the questionnaire. From these six
completed questionnaires, it was determined that the typical time for completion of the
questionnaire is between 5-10 minutes, depending on the number of written comments.

57
Changes were made to the initial instrument based on the recommendations of the
panel. These recommendations were primarily related to clarity and item order issues,
however, five items were added to the initial instrument based on the panel's comments:
(1) a question concerning calling frequency (Item 3), (2) a question related to the need for
written supporting documents (Item 31), (3) a question involving the service's relation to
patient outcomes (Item 32), and (4) and two questions concerning the usefulness of the
service (Items 29 and 30).
Second, the original version of the SERVPERF scale did not include a response
option allowing subjects to differentiate between items for which they had no opinion or
no experience versus items for which they actually had neutral feelings. Survey research
conducted by Kippen, Strasser, and Joshi (1997) reported differences in response patterns
for subjects that had "No Opinion" and "No Experience" response categories versus
subjects that were forced to choose a response. The pre-test was conducted using two
versions of the questionnaire. The first version forced the subjects to choose a response
from a seven-point scale (Appendix E). The second version included an eighth scale
option allowing subjects to select "Don't Know" for questions that they did not have an
opinion or they felt were not applicable (Appendix F). The response patterns between
these two versions were used to select out items that were not applicable to the drug
information setting.
Third, comments were often written next to individual items and in response to the
final question of the questionnaire (Appendices G and P). These comments were used in
combination with the other validation techniques to help decide if individual items were
applicable to the drug information setting.

58
Construct Validity
Construct validity is concerned with the theoretical relationships between variables
to the extent that the measures used to represent variables behave as expected in relation
to other measures (Crocker and Algina, 1986). Two issues relevant to the construct
validity of the questionnaire were examined.
First, the dimensionality of the SERVPERF portion of the questionnaire was
explored. Parasuraman et al. (1985, 1988) have described perceived service quality as a
multidimensional construct covering five separate dimensions: tangibles, reliability,
responsiveness, empathy, and assurance. However, as described in the literature review,
other researchers have had trouble duplicating the original five dimensions. Furthermore,
researchers have found that items do not always load on the same dimension. Cronin and
Taylor (1992) suggested that the perceived service quality should not be considered as a
multi-dimensional construct, but instead a uni-dimensional construct. If SERVPERF can
be considered as a multi-dimensional construct then it may be important to examine the
differential effects of service time and delays on these dimensions as well as the overall
instrument.
This research explored the dimensionality of the SERVPERF portion of the service
quality questionnaire using principal components factor analysis with a Varimax (i.e.,
orthogonal) rotation. The number of components to retain was decided using the Kaiser
criterion, such that components with eigenvalues of 1.00 or higher were retained, and
components with less than 1.00 were excluded (Stevens, 1996). Items were assigned
according to their factor loadings. Items with loadings less than 0.40 were rejected
(DeVellis, 1991; Stephens, 1996).
Second, the correlations among PSQ, OSQ, and behavioral intention obtained
from this questionnaire were compared with similar correlations obtained by Cronin and
Taylor (1992). In other words, the question was asked, "Do the variables behave as

59
expected?" This step is important due to the changes made to SERVPERF for use in the
drug information setting. Using these comparisons, it was possible to see if the variables
used in this research performed consistently with what has been reported in the previous
research. Elypotheses HI, H2, and H3 were used to compare these relationships.
Reliability Assessment
Internal consistency measures reliability for a single time period and essentially
measures item homogeneity and quality (Crocker and Algina, 1986, DeVellis, 1991).
Internal consistency was measured using coefficient alpha (i.e., Cronbach’s alpha).
Coefficient alpha describes the proportion of the scale score variance attributable to the
true score, and is affected by many item quality problems (e.g., non-central mean, poor
variability, negative or weak inter-item correlation, and poor item-scale correlation)
(DeVellis, 1991). As an absolute measure, an alpha ranging from 0.60 to 0.70 was
considered acceptable, 0.70 to 0.80 was considered good, and an alpha ranging from 0.80
to 0.90 was considered very good (DeVellis, 1991).
Assessment of IN on-Response Bias
The potential for non-response bias was evaluated in two ways. First, the reasons
that subjects gave for not responding to the survey when asked during the reminder
telephone call were collated and summarized. Second, one-way ANOVA procedures
were conducted for subscriber status, profession, and response interval to check for
significant differences in PSQ and OSQ among the respective groups. These dependent
variables were chosen because of their importance in the hypothesis tests.
Each questionnaire response was received within one of three response intervals:
(1) response was received before the reminder post card was mailed, (2) response was
received after the reminder post card, but before the follow up phone call, and (3)

60
response was received after the follow up phone call. The rationale for analyzing using
the response interval to assess non-response bias is based on the assumption that late
responders were more likely to be like non-responders in their responses to the
questionnaire than subjects who returned the questionnaire without needing a reminder. If
significant differences existed between the early and late responders, it was likely that
some degree of non-response bias also existed.
Pre-test of Questionnaire
The pre-test for this study was conducted at the Arkansas Poison and Drug
Information Center (APDIC) in Little Rock, Arkansas. The APDIC takes drug
information and poison intervention calls from practitioners and lay consumers from the
state of Arkansas, and has contractual relationships with other organizations and
corporations requiring drug information services. The APDIC is staffed primarily by
registered pharmacists; however, Pharm.D. interns do occasionally provide additional
assistance.
Only health care practitioners calling in with drug information questions (i.e., non¬
poison related) were identified as potential participants. During the study period, 244
callers fit these criteria; however, 41 (16.8%) of these callers were identified as repeat
callers, and as such were only sent one questionnaire. Contact information was
unavailable for two of the callers. This resulted in a usable pre-test sample of 201 subjects.
Questionnaires were mailed to these 201 subjects on June 5, 1997 and June 6,
1997. A cover letter (Appendix A) from the director of the APDIC was printed on
letterhead and sent along with one of two versions of the pre-test questionnaire
(Appendices B and C). One of the versions was randomly assigned to each subject.
Approximately three weeks after the initial mailing, a postcard was sent to non-responders
urging them to mail in the questionnaire (Appendix D). This postcard also thanked

61
respondents if they had already sent in the questionnaire, and asked those who had not
received the questionnaire to contact the researcher (see Appendix C). In total, 134
subjects responded in time to be included in the analysis. Three envelopes were returned
as undeliverable, two subjects called to say that they had received the postcard but not the
questionnaire, and one subject returned the survey completely unanswered. Two subjects
returned completed surveys too late to be included in the analysis. This equaled a raw
response rate of 66.7% (134 divided by 201) and an adjusted response rate of 69.1% (134
divided by 194).
Of these 134 respondents, 119 (88.8%) were pharmacists, 2 (1.5%) were
physicians, 3 (2.2 %) and 7 (5.2%) were categorized as "Other". One of surveys sent to a
physician was returned as undeliverable, and there was no response received from two
nurses who were mailed a questionnaire.
For analysis purposes, negatively worded questions (pre-test question numbers 4,
11, 12, 14, 15, 16, 18, 22, 23, 24, 26, and 27) were reverse scored. Furthermore, the first
version of the questionnaire included a "Don't Know" checkbox, while the second version
did not include this as an option. By comparing the patterns of response for the two
versions, it was possible to gain some insight regarding how subjects responded to items
that they did not know how to assess or that did not apply to them. In addition, these
comparisons made it easier to separate items that were not applicable to the setting from
those items about which respondents actually had neutral feelings.
Of the 100 questionnaires sent out without a "Don't Know" response category
(i.e., version one), 64 (64.0%) were returned Of the 101 questionnaires sent out
including a "Don't Know" response category (i.e., version two), 70 (69.3%) were
returned. The response patterns for the two versions indicated that items 13 ("The
APDIC keeps it records accurately ") and 21 ("Employees get adequate support from the
APDIC to do their jobs well ") were difficult to answer. Table 4-2 below shows the
responses for questions 13 and 21 based on the version. Question 13 on version 2 of the

62
questionnaire drew 34 (34.2%) "Don't Know" responses, and question 21 drew 20
(28.6%) "Don't Know" responses. When comparing versions 1 and 2 for both of these
items, a clear shift is evident from the "Neutral" and no response categories to the "Don't
Know" category. This suggests that when subjects were not able to put "Don't Know" as
a response, they often responded with a neutral response instead. These data also
suggests that these items were not applicable to the drug information setting. Appendices
E and F illustrate the responses for all items.
This lack of applicability is further supported by the comments written next to the
items (Appendix G). For question 13, six of the respondents wrote in "don't know" next
to the question and another wrote in that they did not understand the statement. Similarly,
for question 21, six respondents indicated that they did not exactly know how to answer
the question, one respondent stated that the question did not apply, and another stated that
they did not understand the question. As one respondent wrote, “[t]he only hint I have is
how well the employees answer my questions,” suggesting that subjects find these items
difficult to evaluate because they have no direct experience with these issues. Because of
this data, it was decided to exclude items 13 and 21 from the rest of the analysis and
eliminate them from the final version of the questionnaire.
Table 4-2. Responses to Pre-test Questions 13 and 21
Response
Category
Question 13
Version 1
Freq. (%)
Question 13
Version 2
Freq. (%)
Question 21
Version 1
Freq. (%)
Question 21
Version 2
Freq. (%)
Strongly Agree
11 (16.9)
9 (12.9)
16 (25.0)
16 (22.9)
Agree
16 (24.6)
14 (20)
24 (37.5)
26 (37.1)
Somewhat Agree
0 (0.0)
2 (2.7)
3 (4.7)
3 (4.3)
Neutral
27 (41.5)
9(12.9)
17(26.6)
3 (4.3)
Somewhat Disagree
2 (0.03)
0(0.0)
0 (0.0)
0 (0.0)
Disagree
0 (0.0)
0 (0.0)
0 (0.0)
0 (0.0)
Strongly Disagree
0 (0.0)
1(1.5)
0 (0.0)
1(1.4)
"Don't Know”
n/a
34 (34.2)
n/a
20 (28.6)
No Response
9 (13.9)
1(1.5)
4(6.3)
1(14)

63
Reliabilities of Pre-test Measures
The service quality questionnaire used in this study contained four measures: (1)
the SERVPERF scale measuring perceived service quality (i.e,, items 3 through 24), (2)
four items measuring service time perceptions (i.e., items 25 through 28), and (3) overall
service quality, and (4) intended future behavior (items 34 and 35). Chronbach's alpha
was used to assess the reliabilities of these measures. Perceived service quality as
measured by SERVPERF had an alpha of 0.8873 (n=93). The alpha for the items
evaluating service time perceptions was 0.6560 (n=T31). The alpha for intended future
behavior was 0.6292 (n=126). Since OSQ was a single item measure, reliability could not
be assessed; however, the reliabilities for the other measures were in an acceptable range.
Tables 4-3 and 4-4 show the item-to-total statistics for perceived service quality and
perceived service time measures. Only question 4 had a corrected item-to-total
correlation of less than 0.30; however, since the deletion of question 4 resulted in only a
small improvement in alpha (from 0.8873 to 0.8886) the item was retained.
Factor Analysis of SERVPERF Scale
A principal components factors analysis using a Varimax rotation was used to
explore the factor structure of the modified SERVPERF scale in this setting. The analysis
revealed six factors with eigenvalues over TOO explaining approximately 71.4% of the
variance. The eigenvalue for the seventh factor was 0.872; therefore, it was unlikely that
seven factors would have produced a better separation of the variables. The Scree Plot
presented in Figure 4-1 shows the eigenvalues for each component, where components
numbered seven and below have eigenvalues of less than one. As expected from the
reliability testing, the communalities of each of the variables (Table 4-5) were all
satisfactory, with only three of variables having communalities below 0.6 (i.e., Q-5, Q-22,
and Q-23).
Table 4-6 below shows the rotated component matrix for the six factor solution.
Unfortunately, replication of the factors demonstrated by Parasuraman et. al (1988) was

64
not achieved. However, at least one item from each of the hypothesized dimensions (i.e.,
tangibles, reliability, responsiveness, assurance, and empathy) defined each of the
respective factors. The differences in the factor structure may have resulted from a
number of sources. First, two of the items were reworded to improve clarity. Second,
eight of the items were reworded from second person perspective to first person
perspective. Third, the order of some of the items was rearranged to reduce the sense of
redundancy. Fourth, the items related to the tangibles dimension were replaced by new
items, so it was not determined where these new items would load, or how their
correlations with the other items would affect the factor structure.
Table 4-3, Pre-test Item-total Statistics for SERVPERF Subscale (n=93)t
Item
Corrected Item-
Total Correlation
Alpha if Item
Deleted
03
0.3555
0.8865
Q4R
0.2762
0.8886
05
0.3189
0.8878
06
0.6317
0.8806
Q7
0.6494
0.8791
Q8
0.6609
0.8764
Q9
0.6800
0.8790
Q10
0.6249
0.8795
Q11R
0.6019
0.8788
Q12R
0.5296
0.8832
014R
0.6328
0.8786
015R
0.6123
0.8802
Q16R
0.6572
0.8767
017
0.3328
0.8910
018R
0.7585
0.8764
020
0.6836
0.8790
022R
0.5498
0.8807
Q23R
0.3354
0.8875
0.24R
0.3238
0.8890
^ "R " next to the question number indicates that the question was reverse coded for analysis purposes.

65
Table 4-4. Pre-test Item-total Statistics for Perceived Service Time (n=131)f
Item
Corrected Item-
Total Correlation
Alpha if Item
Deleted
025
0.5235
0.5545
026R
0.5074
0.5803
Q27R
0.6125
0.5389
028
0.3462
0.6465
^ "R" next to the question number indicates that the question was reverse coded for analysis purposes.
Table 4-5. Pre-test Item Communalities1
Item
Communality
03-Necessary Resources
0.843
Q4R-Background Noise
0.849
Q5-Written Materials
0.555
Q6-Speak Clearly
0.652
Q7-Promised Time
0.831
Q8-Sympathetic & Reassuring
0.611
Q9-Dependable
0.719
Q10- Provides in Time
0.884
Q1 lR-Individual Attention
0.698
Q12R-When Performed
0.687
Q14R-Prompt Service
0.792
015R-Willingness to Help
0.662
OI6R-T00 Busy
0.827
Q17-Trust Employees
0.728
Q18R-Personal Attention
0.747
Q19-Polite Employees
0.772
Q20-Safe Interactions
0.738
022R-Know Needs
0.518
Q23R-Best Interests
0.469
Q24R-Operaling Hours
0.691
^ "R " next to the question number indicates that the question was reverse coded for analysis purposes.

66
Item
1
2
3
4
5
6
Q10-Provides in Time
0.895 (RE)
Q7-Promised Time
0.847 (RE)
Q9-Dependable
0.715 (RE)
Q14R-Prompt. Service
0.660 (RS)
0.380
0.391
QI6R-T00 Busy
0.853 (RS)
Q15R-Willingness to Help
0.749 (RS)
Q18R-Personal Attention
0.667 (EM)
0.359
Q19-Polite Employees
0.398
0.600 (AS)
0.501
Q20-Safe Interactions
0.537 (AS)
0.505
Q5-Written Materials
0.719 (TA)
011R-Individual
0.648 (EM)
Attention
Q12R-When Performed
0.432
0.633 (RS)
Q24R-Operaling Hours
0.814 (EM)
Q8-Sympathetic &
0.358
0.515 (TA)
Reassuring
Q22R-Know Needs
0.382
0.514 (EM)
017-Trust Employees
0.432
0.717 (AS)
023R-Best Interests
0.536 (EM)
Q6-Speak Clearly
0.452
0.356
0.455 (TA)
04R-Background Noise
0.896 (TA)
0.3-Necessary Resources
0.858 (TA)
"R " next to the question number indicates that the question -was reverse coded for analysis purposes.
* The letters in parentheses indicate the dimension on which the item loaded in the original
SERVQUAL research conducted by Parasuraman et al. (1988). Where EM=Empathy,
RS=Responsiveness, RE Reliability, AS ^Assurance, and TA=Tangibles.
CD
3
TO
>
C
a>
gi
Lu
Component Number
Figure 4-1. Scree Plot of Pre-test Data
(Variables = 20; N=93)

67
Simulation Development. Verification, and Validation
Simulation was chosen as the appropriate modeling tool to answer the three
specific research questions posed by this study for three reasons. First, simulations make
it easier to clarify thinking about systems problems, because the focus is placed on
defining the components of the system rather than the complex interrelationships. Second,
simulation enables the consideration of the impact of changes in all the factors influencing
the system, such as staffing levels, service rates, and arrival rates. Third, since these
changes are made on the computer, effects of systems changes can be evaluated and tested
without subjecting the real system to unnecessary stress (Reilly et al., 1978). This section
describes the techniques used to construct, verify, and validate the simulation model.
Model Construction
The simulation model was constructed using the GPSS/H (General Purpose
Simulation System) simulation language on an MS-DOS based personal computer. Four
steps were necessary to the construction of the simulation. First, approximately 12 hours
were spent by the principal investigator observing the system in order to gain a first-
person perspective of the actual work processes in the DIPRC. Different days and times
were selected so that a broad perspective was achieved. Second, flow charts were
developed documenting the steps necessary to complete a service transaction. From these
flow charts, it was be possible to identify the necessary events, facilities, variables,
decisions, inputs, and outputs necessary to model the system (Hoover and Perry, 1989).
Third, these flow charts were translated into block diagrams that directly represented
program code (Appendix G). Fourth, after the historical data sheets, the data collection
forms, and the personal interviews were analyzed, the simulation code was written using
the data derived from these sources as inputs (Appendix H).

68
Verification of the Model
Once the computer model was constructed, it underwent a verification process.
Verification is the process by which the model is tested to make sure that it is performing
as it should. In other words, the verification process analyzes whether the program code
“correctly represents the model assumptions and system data” (Carson 1989, p 552).
Three techniques were used to verify the model First, tracing the simulation is a
common process used to make sure that entities move through the simulation as expected
(Hoover and Perry, 1989). The GPSS/H tool called the "interactive debugger", which
allows for interactive traces of the simulation, was used for this step of the verification
process (Schriber, 1991). Second, logical relationships imposed in the model were
verified. This confirmed that facilities and queues were not exceeding their capacity
(Hoover and Perry, 1989). Third, simulation results were compared to those expected by
an analytical model. This was accomplished by changing the parameters of the model (e g.
changing empirical distribution to exponential distributions and using constants instead of
random variables) so that the simulation results could be compared to the results
mathematically derived from known equations (Hoover and Perry, 1989).
Model Validation
Throughout the process of the building the simulation, the simulation model
underwent a series of validation steps. Validation is the process by which the researcher
determines if the model is “sufficiently accurate for the purpose at hand and which can be
used as a substitute for the real system” (Carson, 1989, p.552). In other words, a valid
model can be used in place of the real system for purposes of asking questions and making
comparisons (Carson, 1989; Hoover and Perry, 1989) Three techniques were used to
validate the model: (1) face validation, (2) extreme-conditions tests, and (3) comparison of
simulation output to data from the real system (Balci, 1989; Hoover and Perry, 1989)

69
First, face validation was necessary to judge whether or not the model seemed
reasonable to those knowledgeable about the system being studied (Balci, 1989; Carson,
1989; Hoover and Perry, 1989). In two separate meetings, the co-directors of the DIPRC
were given a structured walk-through of the simulation using the block-diagrams. The co¬
directors were both asked four general questions: (1) "Do you understand how the model
will operate?", (2) "Does the model conform to your knowledge of the service processes
of the DIPRC?", (3) "Is there anything present in the model that seems incorrect?", and
(4) "Is there some part of the service process that is missing from the model?" Changes to
the simulation were made based on the co-directors responses to these questions.
Second, the computer model was validated under extreme conditions. To some
extent, the behavior of a system under extreme conditions should be plausible and conform
in the expected direction (Hoover and Perry, 1989; Sargent, 1992). For example, if the
arrival rate increases dramatically, we would expect that that line length, time in the
queue, and utilization percentage should also increase dramatically.
Third, one of the most powerful techniques for validation is the comparison of the
model to the original system. Chi-square goodness-of-fit tests will be used to answer
questions concerning the equality of the underlying distributions. It was recognized a
priori, however, that the simulation results would probably not exactly match those of the
real system. This is because the model is a simplified version of the real system and did
not reflect many elements intentionally excluded from the model (Balci, 1989, Hoover and
Perry, 1989; Sargent, 1992). Therefore, regression analysis was also conducted to detect
how much variation in the real system the simulation actually predicts.
Variance Reduction
Variance reduction in simulation models is important because it helps improve the
power for detecting significant statistical differences in the simulation experiments.

70
Variance reduction techniques allow simulation experiments to obtain greater precision
(e.g. smaller confidence intervals) with less simulation (Law and Kelton, 1991). One
method of variance reduction commonly used is antithetic variation, which uses strong
negative covariances to reduce the variation among experimental runs (Law and Kelton
1991; Neelamkavil 1987). Simulation runs were constructed used the antithetic variate
capabilities available in GPSS/H through the RMULT statements.
Data Analysis
To facilitate the reporting of the results, the data analysis was broken down into
four separate parts. The first two parts of this study were essential to develop an
understanding of the work processes in the D1PRC necessary to construct the simulation.
The results of these first two parts are reported in chapter five as preliminary data. The
third and fourth parts tested the research hypotheses and specific research questions
developed in previous chapters. The results of these analyses are presented in chapter six
as main study results.
Analysis of Preliminary Data
Three data sources were used to evaluate the calling population characteristics as
well as the arrival and service rate trends in the DIPRC: (1) archived historical data sheets
from September, 1996 through May, 1997 (Appendix H), (2) data collection forms
collected concurrently during the data collection period from June 1, 1997 until August 1,
1997 (Appendix I), and (3) an internal database containing the number of questions
answered by month for the past ten years. Three analyses were conduced using this data.
First, the consistency between the historical data and the concurrent data was evaluated.
The two samples were compared by profession, subscription status, question type,
response type requested, and percentage of service delays. Proportional differences

71
between the historical and concurrent groups were detected using a two-tailed z-test with
a = 0.05. Second, temporal trends were examined by month of year, day of week, and
time of day. One-way analysis of variance (ANOVA) was used to detect overall
differences in means, and Scheffe’ multiple comparisons procedures were used to analyze
for significant mean differences between groups. Alpha for the ANOVA and post-hoc
procedures was set at the 0.05 level Third, the empirically observed interarrival and
service time distributions were compared with distributions known to be useful in
modeling system behavior (i.e., exponential and Weibull). This comparison was
completed using Kolmogorov-Smirnov goodness-of-fit tests and regression analysis.
The second part describes the results from personal interviews conducted with the
students and co-directors working in the Drug Information and Pharmacy Resource
Center (DIPRC). These data were used primarily to obtain information concerning the
work process and the prioritization system used to organize the work. In addition,
perceptions about caller preferences and recommendations for improvement were also
obtained. The data from each of the twelve audio-recorded personal interviews were
collated and content analyzed based on the responses to the semi-structured interview
outline.
Analysis of Data Related to Main Study
The third part reports the results of the validation and reliability testing of the
service quality questionnaire and the hypothesis tests used to explore the relationships
among service time, service delays, perceived service time, evaluations of perceived
service quality, and behavioral intention in the drug information service setting. Chapter
three presented eight hypotheses related to these variables. Hypothesis 5a (H5a) was
tested using simple linear regression, and hypothesis 5b (H5b) was evaluated through an
examination of residual plots obtained from the regression analysis. The remaining

72
hypotheses were tested using correlation, t-tests, and one-way analysis of variance
(ANOVA). Pearson correlation coefficients were used to detect significant relationships
between variables Hypotheses that did not demonstrate statistically significant
correlations between the study variables were rejected. Additional analysis of using
ANOVA was conducted for hypotheses with statistically significant correlations. Scheffe'
multiple comparison procedures were conducted when an ANOVA resulted in a
significant F-value indicating overall differences among level means. All analyses were
conducted at the 0.05 level.
The fourth part used information from the first three parts to develop and validate
the simulation model used to optimize service capacity in the DIPRC based on total
service time, percentage of service delays and percent utilization. Validation and
verification of the simulation was conducted using regression, j2, linear regression, and
comparisons of relevant 95% percent confidence intervals. Descriptive statistics,
confidence intervals, and ratio analysis was used to explore the three research questions
outlined in chapter three.

CHAPTER 5
PRELIMINARY RESULTS
Overview
As discussed in the previous chapter, the data analysis was broken into four related
parts. This chapter presents the preliminary results generated from parts one and two of
the data. Part one has three subsections. First, the data gathered from the historical data
sheets were compared with the concurrent data gathered from the data collection forms.
This was done to establish the degree of homogeneity between the historical and
concurrent samples and to explore potential sources of bias in the results. The two
samples were compared by profession, subscription status, question type, response type,
and the occurrence of delays in service. Second, the historical data were analyzed for
temporal trends by month, day of week, and time of day. This was done to explore for
trends in arrivals that could be reasonably reproduced by the simulation program, and to
identify periods for which the simulation results may not be valid. Third, the historical
interarrival and service time distributions were tested for goodness-of-fit with known
probability distributions (i.e., exponential and Weibull) to determine if the simulation could
use approximated known distributions to model empirically observed interarrivals and
service times.
Part two has two subsections. First, the interviews conducted with the co¬
directors of the DIPRC are summarized. Second, a summary of the interviews conducted
with the externship students is presented.
73

74
Part One: Historical and Concurrent Data
Analysis by Profession
Four types of profession categories were tracked on the historical and concurrent
data sheets: (1) pharmacist/Pharm.D., (2) physician, (3) nurse/nurse practitioner, and (4) a
miscellaneous category called "other" (e.g., physician assistants, dentists, nutritionists, law
enforcement, etc.) Table 5-1 presents the frequency of occurrence of the four profession
categories along with the individual and cumulative percentages that the frequencies
represented out of the total number of usable samples. The number of usable data points
out of the total number of samples is displayed near the bottom of the table along with the
number of missing data points.
Pharmacists represented the largest professional group that the DIPRC serviced.
Pharmacists posed 66.8% of the questions in the historical group and 67.8% in the
concurrent group. Although the proportions of pharmacists and nurses were not
significantly different between the historical and concurrent samples, it was determined
that the historical data had a larger sample of physicians than the concurrent sample (i.e.,
11.4% historically versus 8.2% in the concurrent sample (p=0.036)) and a smaller number
of other professional types (i.e., 13.9% historically versus 17.1% in the concurrent sample
(p=0.068)). However, the ANOVA did not reveal any significant differences in total
service times related to the profession of the caller (F=2.231, p=0.084), reinforcing the
homogeneity of the two samples.
Analysis by Subscription Status
Subscription status is typically broken into three groups by the DIPRC: (1) non¬
subscribers, (2) subscribers (i.e., those who have donated a subscription fee to the
DIPRC), and (3) University of Florida Health System (UFHS) employees. Analysis using

75
z-tests revealed differences between the historical and concurrent samples with regard to
their subscriber and UFHS percentages (p<0.001 in both cases); however, no differences
were detected for non-subscribers (Table 5-2). Historically, about 39.5% of the callers
were subscribers, 30.0% were non-subscribers, and 30.5 percent were employees of the
UFHS. In the concurrent sample, 26.6% were subscribers, 32.5% were non-subscribers,
and 40.9 percent were employees of the UFHS. Since the ANOVA did not reveal any
overall differences in total service times among the three subscription status groups
(F=2.015, p=0.135), there was not much concern that differences in subscription status
between the historical and concurrent samples would bias the simulation results.
Table 5-1. Percentage of Questions by Profession1
Profession
Historical Data
Concurrent Data
Freq.
Percent
Cumul.
Percent
Freq.
Percent
Cumul.
Percent
Pharmacist
1554
66.8
66.8
354
67.8
67.8
Physician *
265
11.4
78.2
43
8.2
76.0
Nurse/N.P.
182
7.9
86.1
36
6.9
82.9
Other
324
13.9
100.0
89
17.1
100.0
Total Questions
Counted
2325
97.5
522
99.2
Missing Data
60
2.5
4
0.8
Total
2385
100.0
526
100.0
^ Significance denotes differences between the historical and concurrent groups.
*p<.05 **p<.01 ***p<.001
Table 5-2. Percentage of Questions by Subscription Status
Subscription Status
Historical Data
Concurrent Data
Freq.
Percent
Cumul.
Percent
Freq.
Percent
Cumul.
Percent
Subscriber***
932
39.5
39.5
136
26.6
26.6
Non-Subscriber
708
30.0
69.5
166
32.5
59.1
UF Health System * * *
719
30.5
100.0
209
40.9
100.0
Total Questions
Counted
2359
98.9
511
97.2
Missing Data
26
1.1
15
2.8
Total
2385
100.0
526
100.0
^ Significance denotes differences between the historical and concurrent groups.
*p<.05 **p<.01 ***p<.001

76
Analysis by Question Type
As discussed in the previous chapter, the DIPRC answers a variety of drug related
questions covering fourteen separate categories. Of these fourteen categories, the five
largest in both samples were (1) drug availability, (2) drug dosage and administration, (3)
drug identification, (4) drug therapy and efficacy, and (5) a miscellaneous category called
"Other" These five categories made up 74.8% and 71.8% of the historical and concurrent
questions answered, respectively (Table 5-3).
Only three question categories demonstrated statistically significant proportional
differences between the historical and concurrent samples. First, questions concerning
drug therapy and efficacy represented about 19.5% in the historical sample; however, they
only represented 13.1 % of the total questions in the concurrent sample (p<0.001). This
was the only question out of the top five to display a significant difference. Second,
questions involving drug use in pregnancy and lactation represented only 2 .1% of the
historical sample; however, it represented 7.8% of the concurrent sample (p<0.001).
Finally, questions concerning side effects or adverse drug effects represented 9.1% of the
historical sample, but only 5.1% of the concurrent sample (p=0.005).
Although the proportional differences in question types between the historical and
concurrent samples was not great, it was necessary to account for differences in service
times among question types. For analysis purposes, question types representing less than
five percent of the sample in both the historical and concurrent samples were grouped into
one category called "combined". As such, six question types were classified as
"combined": investigational drugs, IV compatibility and stability, legal, pharmacokinetics,
toxicology, and veterinary drugs. Although questions falling into the drug use in
pregnancy and lactation category only represented 2.1% of the questions in the historical
sample, this category was kept separate since it represented 7.8% of the questions in the
concurrent sample.

77
The ANOVA comparing the service times (i.e., time to complete a question) in
minutes for each of the nine question groups revealed that the mean service time for at
least one question type was significantly different (F=5.127, p<0.001). Post hoc
calculations revealed that the drug identification questions were significantly different from
questions regarding drug therapy and efficacy (p=0.001) and questions regarding drug use
in pregnancy and lactation (p=0.028). No other significant differences were detected.
Table 5-4 presents the descriptive statistics for each of these categories and Figure 5-1
graphically illustrates the 95% confidence intervals for service time for each of the
question categories. In addition to the two significant differences in service times above,
the confidence intervals also suggested that questions regarding side effects and adverse
drug effects tended to have longer service times than drug identification questions. Also,
it appeared that questions from the "Other" category also tended to have shorter service
times with the exception of drug identification questions.
100
10 n
CD 0
N = 65 60 99 36 67 39 65 27 40
\A\\\\> \\
v % \ *â–  v x \ % \
% 4,
O.
4
Question Type (Combined Categories 7,8,9,11,13,14)
Figure 5-1. 95% Confidence Intervals of Service Time
by Question Type (in Minutes)

78
Table 5-3. Percentage of Questions by Type
Question Type
Historical Data
Concurrent Data
Freq.
Percent
Cumul
Percent
Freq. Percent
Cumul.
Percent
Drug A vailability
236
10.8
10.8
70
13.3
13.3
Drug Dosage & Admin.
271
12.3
23.1
66
12.5
25.8
Drug Identification
432
19.7
42.8
100
19.0
44.8
Drug Interactions
151
6.9
49.7
36
6.8
51.6
Drug Ther. & Efficacy***
429
19.5
69.2
69
13.1
64.7
Drug Use in Pregnancy***
46
2.1
71.3
41
7.8
72.6
Investigational Drugs
4
0.2
71.5
3
0.6
73.2
IV Compat. or Stability
51
2.3
73.8
9
1.7
74.9
Legal
33
1.5
75.3
12
2.3
77.2
Other
274
12.5
87.8
73
13.9
91.
1
Pharmacokinetics
49
2.2
90.0
17
3.2
94.3
Side Effects or A IDEs**
199
9.1
99.1
28
5.3
99.6
Toxicology
18
0.8
100.0
1
0.2
99.8
Veterinary Drugs
1
<0.1
100.0
1
0.2
100.0
Total Questions Counted
2194
92.0
526
100.0
Missing Data
191
8.0
0
0.0
Total
2385
100.0
526
100.0
^ Significance denotes differences between the historical and concurrent groups.
* p<.05 **p<.01
***p< 001
Table 5-4. Service Times in Minutes by Question Typ
e
95%
95%
Confidence
Confidence
Question Type Median
Mean St.Dev. Lower Bound
Upper Bound
Drug A vailability
21.00
44.20
56.42
30.22
58.18
Drug Dosage & Admin.
28.50
47.50
54.51
33.42
61.58
Drug Identification
15.00
23.18
27.76
17.65
28.72
Drug Interactions
32.50
51.61
45.85
36.10
67.12
Drug Ther. & Efficacy
55.00
65.48
50.98
53.04
77.91
Drug Use in Pregnancy
34.00
63.05
59.28
40.59
85.51
Other
23.00
38.00
44.99
26.85
49.15
Side Effects or ADEs
50.00
63.41
61.31
39.15
87.66
Combined
36.00
53.80
59.19
34.87
72.73
Overall
30.00
46.30
52.13
41.71
50.89

79
Based on the service time results presented above, question categories were
further collapsed into three basic groups for purposes of the simulation. Group one had
the shortest service time profile and represented the drug identification and "other"
categories equaling approximately 32.2% of the questions, based on the historical data.
Group two had a moderate service time profile and represented questions related to drug
availability, drug dosage and administration, drug interactions, and the "combined"
categories equaling approximately 37 .1% of the questions. Group three had the longest
service time profile and represented the drug therapy and efficacy, drug use in pregnancy
and lactation, and side effects and adverse drug effects question categories equaling
approximately 30.7% of the questions.
The ANOVA comparing the mean service times of the three combined question
types was significant (F=18.30, p<0.001) indicating overall differences in means among
the groups. Post hoc analysis indicated that the mean service times for all three combined
question types were significantly different from one another. Group one had a mean
service time of 29.05 minutes (s=26.21 minutes), which was significantly different from
both groups two and three (p<0.001) which had respective mean service times of 48.42
minutes (s=54.41 minutes) and 64.35 minutes (s=58.51 minutes). Also, the mean service
time for group two was significantly different from the mean service time for group three
(p=0.019). Table 5-5 shows the descriptive statistics for the three groups, and Figure 5-2
illustrates the new confidence intervals obtained from grouping the questions in this
manner.
Table 5-5. Service Times in Minutes
Using Three Combined Questions Types (in Minutes)
Question Type
Median
Mean
St.Dev.
95%
Confidence
Lower Bound
95%
Confidence
Upper Bound
Group One
15.50
29.05
36.21
23.47
34.64
Group Two
30.00
48.42
54.41
40.86
55.99
Group Three
30.00
64.35
58.51
54.31
74.38

80
201
Group 2
133
Group 3
Three Question Type Categories
Figure 5-2. 95% Confidence Intervals of Service Time
Using Three Combined Question Types (in Minutes)
Analysis by Response Type Requested
When students accepted calls from practitioners, the caller was asked how they
would like to receive the response to their question Four response types were available:
(1) oral; (2) written; (3) both (i.e., oral and written); and (4) either (i.e., oral or written).
The most common type of response requested by the callers was an oral response,
representing 59.0% in the historical sample and 59.2% in the concurrent sample. The
second most requested response type was written, representing 16.9% in the historical
sample and 21.2% in the concurrent sample (Table 5-6). The z-tests revealed significant
differences regarding the proportion of responses from the "both" and "either" response
types observed in the historical and concurrent groups (p<0.001). However, since a
significant difference in total service time was not detected between these two groups
(p=0.994), then little bias is likely to be introduced.

81
The ANOVA revealed significant service time differences (F=l 1 228, p<0.001)
among the response types. The post hoc procedures indicated differences between the
oral and both response types (p=0.044) and between the oral and either response types
(p<0.001). The 95% confidence intervals for the service times by response type are
presented in Table 5-7, and illustrated in Figure 5-3. These data suggested that responses
involving a written component tended to take longer than responses requiring just an oral
response. These differences may be partially explained by examining the distribution of
question types by response type requested. Recalling that drug identification questions
tended to have the lowest service times of the eight question categories analyzed, it is
evident from Table 5-8 that this category also had the highest percentage of oral responses
(83.7%). Furthermore, questions regarding drug therapy, drug use in pregnancy, and side
effects tended to have the highest service times and the lowest percentages of oral
responses of 37.9%, 43.6%, and 48.1%, respectively. Since Figure 5-3 shows that oral
responses tend to have the lowest service times, it is understandable that drug
identification questions would tend to have lower service times than the other question
categories. Therefore, for simulation purposes, it was assumed that deviations in service
times as a function of response type are explained by the type of question asked.
Table 5-6. Percentage of Questions by Response Type Requestedt
Response Type
Historical Data
Concurrent Data
Freq.
Percent
Cumul.
Percent
Freq.
Percent
Cumul.
Percent
Oral
1273
59.0
59.0
299
59.2
59.2
Written
364
16.9
75.9
107
21.2
80.4
Both * * *
320
14.8
90.7
25
4.9
85.3
Either***
201
9.3
100.0
74
14.7
100.0
Total Questions
Counted
2343
98.2
499
94.9
Missing Data
42
1.8
27
5.1
Total
2385
100.0
526
100.0
Significance denotes differences between the historical and concurrent groups.
* p<.05 **p<.01 ***p<.001

82
Table 5-7. Service Time in Minutes by Response Type Requested
95%
95%
Confidence
Confidence
Response Type Median Mean
St.Dev. Lower Bound Upper Bound
Oral
24.50 35.77
42.44
31.78
41.76
Written
40.00 52.41
48.61
42.91
61.91
Both
36.00 68.43
76.32
35.43
101.44
Either
40.00 71.89
71.79
55.14
88.64
Overall
30.00 46.30
52.13
41.71
50.89
Table 5-8. Frequency and Percentage of Response
Types by Question Type
Question Type
Oral
Written
Both
Either
Drug Availability
42 (61.8%)
10(14.7%)
3 (4.4%)
13 (19.1%)
Drug Dosage & Admin.
37 (57.8%)
15 (23.4%)
3 (4.7%)
9(14.1%)
Drug Identification
82 (83.7%)
3 (3.1%)
2 (2.0%)
11 (11.2%)
Drug Interactions
22 (64.7%)
4 (11.8%)
3 (8.8%)
5 (14.7%)
Drug Ther. & Efficacy
25 (37.9%)
27 (40.9%)
5 (7.6%)
9(13.6%)
Drug Use in Pregnancy
17 (43.6%)
9(23.1%)
4(10.3%)
9 (23.1%)
Other
40 (57.1%)
23 (32.9%)
0 (0.0%)
7 (10.0%)
Side Effects or ADEs
13 (48.1%)
9 (33.3%)
2 (7.4%)
3 (11.1%)
Combined.
21 (53.8%)
7 (17.9%)
3 (7.7%)
8 (20.5%)
Overall
299 (59.2%)
107(21.2%)
25(5.0%)
74 (14. 7%)
120
100
0)
E
o
o
CO
o
in
O)
Response Type
Figure 5-3. 95% Confidence Intervals of Service Time
by Response Type (in Minutes)

83
Occurance of Service Delays
Each of the samples was evaluated to determine if there had been a delay in service, where
the response time was longer than the time requested. The z-test did not reveal any
significant differences between the historical and the concurrent data groups in terms of
the percentage of delays. The historical data indicated that, overall, 16.3% of questions
result in service delays (Table 5-9). The concurrent data was very similar indicating that
18.6% of the questions evaluated during the study period resulted in service delays.
In order to evaluate when delays occur, the distribution of service delays was
examined by the time requested for the four categories available on the data collection
form (i.e., "Stat", “Today”, “Date”, and “No Rush”). The results using the concurrent
data indicated that those questions requesting “Stat” (<15 minutes) attention resulted in
the highest percentage of delays at 58.6% (Table 5-10). Questions requesting an answer
to the question within the day (i.e., “Today”) were delayed 19.5% of the time, and those
requesting an answer by a specific date were delayed 23.2% of the time. “No rush”
questions were arbitrarily marked as delayed when total service times were greater than
two weeks (2 1%). Given that the majority of calls requesting “Stat” attention are
delayed, it is clear that under the current system in the DIPRC it is very difficult to provide
a fifteen minute turnaround time.
Table 5-9. Percentage of Questions by Delay Statusf
Delay Status
Historical Data
Concurrent Data
Freq.
Percent
Cumul.
Percent
Freq.
Percent
Cumul.
Percent
No
1936
83.7
83.7
428
81.4
81.4
Yes
378
16.3
100
98
18.6
100.0
Total Questions
Counted
2314
97.0
526
100.0
Missing Data
71
3.0
0
0.0
Total
2385
100.0
526
100.0
^ Significance denotes differences between the historical and concurrent groups.
* p<. 05 **p<.01 ***p<.001

84
Table 5-10. Percentage of Delays in Service by Time Needed
Time Needed
Delay in Service
No
Freq.
Yes
Freq.
Yes
Percent
Stai (<15 min)
24
34
58.6
Today
182
44
19.5
By Spec. Dale
53
16
23.2
No Rush
143
3
2.1
Arrivals by Month
A database containing the number of questions per month from January 1987 to
June 1997 was analyzed to determine if there were any significant monthly trends in the
data. In order to the reduce the error introduced by the differing number of days the
DEPRC was available to answer questions each month, the number of questions answered
during each month was divided by the number of days the center was actually open. This
resulted in an average number of questions answered per day during a particular month.
Table 5-11 shows the descriptive statistics from January to December based on the
average number of question arrivals per day.
The ANOVA revealed that at least one month was significantly different (F=3.432,
p<0.001) from the other months. Post-hoc calculations using the SchefTe' procedure
revealed that the month of December was significantly different from November
(p=0.023). No other month-pairs revealed statistically significant differences in average
arrivals; however, January (p=0.074) and August (p=0.139) also tended to have higher
average daily arrivals than December. Figure 5-4 graphically illustrates the 95%
confidence intervals for the average daily number of arrivals for each of the twelve
months.
Overall, the analysis revealed that the average daily number of questions from
January 1987 to June 1997 was 12.62 (n=126, s=l .38) and average total number of
questions per month was 263.13 (n=126, s=30.32). Limiting this to only the past five
years resulted in only slightly higher numbers. From July 1992 to June 1997 the average

85
daily number of questions was 12.75 (n=66, s=1.25) and the average total number of
questions per month was 265.98 (n=66, s=25.26). In general, however, there are no
practically significant differences among the month that would necessitate special
consideration in the simulation program. However, December represents a special case
for which the simulation program may not be valid
Table 5-11. Descriptive Statistics for the Average Number of Questions Answered
by Month for the Past Ten Years.
Month
Median
Mean
Std. Dev.
95% Lower
Bound
95% Upper
Bound
Jan.
13.43
13.50
1.59
12.44
14.57
Feb.
12.90
13.00
1.38
12.08
13.93
Mar.
12.39
12.29
1.07
11.57
13.01
Apr.
12.10
12.27
0.91
11.66
12.89
May.
12.32
12.12
1.27
11.27
12.98
Jun.
12.05
11.85
1.00
11.18
12.52
Jul.
12.69
12.74
1.17
11.91
13.59
Aug.
13.77
13.39
1.22
12.53
14.26
Sep.
12.85
12.65
1.11
11.85
13.45
Oct.
12.67
12.63
0.74
12.10
13.16
Nov.
13.62
13.82
1.72
12.59
15.05
Dec.
11.32
11.15
1.27
10.24
10.24
Overall
12.61
12.62
1.38
12.37
12.86
Interarrivals by Day of Week
The interarrival times (i.e., the time between arrivals) were evaluated by day of
week using the historical data. The ANOVA did not reveal any significant differences in
interarrivals by day of week (F=0.348; p=0.846); however Monday did have the shortest
mean interarrival time of 37.08 minutes (s=40.77), and Friday had the longest at 40.28
(s=39.95) (Table 5-12). However, these differences were not practically significant and
do not warrant special consideration in the simulation.

86
16
15
14
13
12
O
C3 11
10
LO
O) 9
N = 11 11 11 11 11 11 10 10 10 10 10 10
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
Month
Figure 5-4. 95% Confidence Intervals of Average Daily Arrivals by Month
Table 5-12. Descriptive Statistics for Question Interarrival Times
in Minutes by Day of Week
Day
Median
Mean
Std. Dev.
95% Lower
Bound
95% Upper
Bound
Mon.
25.00
37.08
40.77
33.09
41.07
Tue.
27.00
39.12
40.42
34.98
43.26
Wed.
25.00
38.31
42.20
34.14
42.49
Thu.
25.00
39.65
42.52
35.31
44.00
Fri.
30.00
40.28
39.95
36.09
44.47
Overall
25.00
38.84
41.18
38.98
40. 70

87
Interarrivals by Time of Day
When the interarrivals were compared by time of day using the historical data, the
ANOVA revealed significant differences among the time intervals examined (F=19.63,
p<0.001). The post hoc analyses indicated that there were significant differences between
the morning interarrivals observed during the 9:00 a.m. to 11:59 a.m. time intervals and
the afternoon interarrivals observed during the time intervals from 12:00 p.m. to 4:59 p.m.
(Table 5-13). Table 5-14 shows the descriptive statistics for each of the observed time
intervals. Figure 5-5 graphically illustrates the interarrival trend by time of day. The
shortest interarrivals occured in the morning starting at 9:00 a.m. with a mean interrarrival
time of 17.97 minutes, or approximately four per hour. Interarrival times decreased
throughout the morning until 1:00 p.m. during which time the interarrival time was
approximately 30 minutes, or two per hour. After 13:00, the interarrival times leveled off
at a mean of approximately 50 minutes, or a little better than one per hour. Since these
differences were practically significant as well as statistically significant, the simulation ws
programmed to generate simulated calls taking into consideration the hour of the day.
Interarrival and Service Time Distributions
The coefficient of variation (CV) is alternative measure of variability that often
gives information about the form of a continuous distribution. It calculated by dividing the
standard deviation by the mean. For the exponential distribution, the CV should be 1.00
regardless of the scale parameter (P), where P is approximated by the mean. Skewness is
a measure of the symmetry of the distributions, and should be equal to 2.00 for an
exponential distribution (Law and Kelton, 1991). The interarrival distribution was the
most positive candidate for a good fit with the exponential distribution since it had a CV
of 1.06 and a skewness measure of 2.01 (Table 5-15). However, total service time and
service times by question group (i.e., corresponding to the three combined groups

88
presented above) were also good possibilities for a fit with the exponentially distribution
since their coefficients of variation and skewness measures were all fairly close to the
desired values. It was unlikely that a goodness of fit test would indicate that the
underlying distributions for the Time to a Take Call, the Approval Time, and the Time to
a Return a Call were exponential given that the skewness measures were at least two times
the desired 2.00 (i.e., Time to Take Call had a skewness of 4.64, Approval Time had a
skewness equal to 9.20, and Time to Return a Call had a skewness equal 4 30.) The
Kolmogorov-Smirnov goodness-of-fit test (K-S test) was used to calculate whether or not
the interarrival and service time distributions were exponentially distributed. The
computational procedures outlined by Law and Kelton (1991) were used to compute the
value of D„, which in turn was used to calculate the K-S test statistic. This test statistic
was then compared to a critical value (i.e., cYa) where:
n = number of samples and i = i-th sample
D+= max{i/n-F*(xi))
D'= max{F*(xi)-(i-l)/«}
D„ = max{D,D"}
r
n A
/— 0 5
•vn + 0.26 H—j=
> c i-
(Formula 5-1)
(Formula 5-2)
(Formula 5-3)
(Formula 5-4)
in j
Formulas 5-1 through 5-4.
Kolmogorov-Smirnov Test for the Exponential Distribution
The critical value of the K-S test at an a level equal to 0.05 for an exponential
distribution is 1.094 (Law and Kelton, 1991). If the test statistic was larger than the
critical value then the hypothesis that the distribution is exponentially distributed was
rejected. Therefore, large values of the test statistic indicated a poor fit. None of the test
statistics resulted in values indicating that the distributions were exponentially distributed
(Table 5-16); however Total Service Time (i.e., the time from the end of the initial request
until a response was given) was the closest with a test statistic of 1.701.

89
The Weibull distribution is another continuous distribution used extensively to
model interarrival and service times, and has a shape similar to the gamma, exponential
and Erlang distributions depending on its parameter values (Law and Kelton, 1991). The
K-S test was computed for the Interarrival Time and Total Service Time distributions in
order to discover if the Weibull distribution, or another distribution of similar shape,
would provide a better fit than the exponential distribution. The critical value for a large
sample K-S test at a equal to 0.05 for the Weibull distribution is approximately 0.874;
however, the test statistics for the interarrival and total service time distributions were
11.51 and 15.30, respectively. This indicated that the Weibull distribution is actually a
poorer fit than the exponential distribution with respect to interarrivals and total service
time.
The exponential distribution could still probably be used in the simulation without
greatly jeopardizing the validity of the model. Linear regression analysis between the
observed and expected probability density functions indicated that the exponential
distribution explains approximately 84.7% of the variance in interarrivals ((30=0.008,
(31=0.684) and 92.3% of the variance in overall service time ((30=0.008, (31=1.291). This
implied that the exponential distribution would probably provide a reasonable estimate of
the observed distributions, even though the observed data are not exponentially
distributed. However, since GPSS/H allows user defined functions, it was decided to use
the empirical distributions to preserve accuracy. Frequency histograms for the interarrival
times, total service time, and service times for the three general question categories are
presented in Figures 5-6 through 5-10.
Table 5-13. Significant P-Values for Interarrival Times by Hour of Day
Hour
9:00 -
9:59
10:00-
10:59
11:00-
11:59
12:00-
12:59
13:00-
13:59
14:00-
14:59
15:00-
15:59
16:00-
16:59
9:00-9:59
0.039
<0.001
<0.001
<0.001
<0.001
10:00-10:59
<0.001
<0.001
<0.001
<0 001
11:00-11:59
<0.001
<0.001
<0.001
<0.001

90
Table 5-14. Descriptive Statistics for Question Interarrival Times
in Minutes by Hour of Day
Hour
Median
Mean
Std. Dev.
95% Lower
Bound
95% Upper
Bound
9:00-9:59
15.00
17.97
23.81
13.82
22.12
10:00-10:59
20.00
24.12
21.39
21.77
26.47
11:00-11:59
25.00
31.46
27.17
28.27
34.66
12:00-12:59
30.00
36.51
33.88
31.92
41.10
13:00-13:59
35.00
51.82
50.82
45.14
58.49
14:00-14:59
30.00
48.92
50.46
42.45
55.39
15:00-15:59
30.00
46.48
51.16
39.97
52.98
16:00-16:59
30.00
47.58
42.64
42.02
53.35
Overall
25.00
38.84
41.18
38.98
40.70
70
60
50
IXJ
30
E
<
— 20
u
N = 129 321 280 212 225 236 240 220
9:00-9:59 11:00-11:59 13:00-13:59 15:00-15:59
10:00-10:59 12:00-12:59 14:00-14:59 16:00-16:59
Hour Received
Figure 5-5. 95% Confidence Intervals of Interarrival Times in Minutes
by Hour of Day

91
Table 5-15. Summary Statistics for Input Distributions
Time
Std.
Coefficient
Data
Distribution
Median
Mean
Dev.
of Variation
Skewness
Source
Interarrival
Time
25.00
38.84
41.18
1.06
2.01
Historical
Time to Take
Call
3.00
3.97
3.61
0.91
4.64
Concurrent
Total Service
Time
75.00
106.40
98.92
0.93
1.23
Historical
Service Time
Group 1
15.50
29.05
36.21
1.24
2.26
Concurrent
Service Time
Group 2
30.00
48.42
54.41
1.12
2.25
Concurrent
Service Time
Group 3
30.00
64.35
58.51
0.91
1.94
Concurrent
Approval
Time
1.00
2.47
7.63
3.08
9.20
Concurrent
Time to
Return Call
3.00
4.25
5.36
1.26
4.30
Concurrent
Table 5-16.
Kolmogorov
-Smirnov Tests for Exponentially Distributed Variables
Time
Test
Distribution
N
D„+
Dn
D„
Statistic
Interarrival Time
1889
0.096
0.038
0.096
4.209
Total Service Time
1477
0.044
0.038
0.044
1.701
Service Time Group 1
164
0.143
0.044
0.143
1.854
Service Time Group 2
164
0.232
0.129
0.232
3.021
Service Time Group 3
133
-0.092
0.424
0.424
5.005

Frequency <*S Frequency
92
Interarrival Time (Minutes)
re 5-6.
Frequency Histogram of Historical Interarrival Times
200 T
Service Time (Minutes)
Figure 5-7. Frequency Histogram of Historical Total Service Times

93
Service Time (Minutes)
Figure 5-8. Frequency Histogram of Service Times for Question Group One
Service Time (Minutes)
Figure 5-9. Frequency Histogram of Service Times for Question Group Two

94
30 T
Service Time (Minutes)
Figure 5-10. Frequency Histogram of Service Times for Question Group Three
Part Two: Student and Director Interviews
Twelve personal interviews were conducted from May, 1997 to July, 1997. The
interviewees consisted of the two co-directors for the DIPRC and ten Pharm.D. students
in their third rotation week. The interviews were semi-structured and were conducted
using interview outlines (Appendices J and K).
Director Interviews
There was a considerable amount of consistency between the Student and co¬
director responses to the questions asked during the interview process, especially
concerning the hours of operation, lunch times, and the process for prioritizing and
answering questions. However, the co-directors were asked some additional questions

95
that the students were not asked. In addition, some questions were asked of the co¬
directors in a way that depended largely on extended experience working in the DIPRC.
The first question that the co-directors were asked was "Who works in the
DIPRC?" They indicated that typically there were two co-directors, a drug information
resident, and three Pharm.D. externship students working in the DIPRC. One of the co¬
directors is primarily responsible for the operational aspects of the DIPRC and the other is
responsible for the educational component of the drug information rotation. However,
they both stressed that their duties overlapped considerably, especially regarding the
educational aspects of the rotation. For about two-thirds of the year, the DIPRC has a
drug information resident. The drug information resident does spend some time as a front
line person answering phone calls; however, they eventually move on to fulfill a more
supervisory role with regard to the externship students. Each month, three new Pharm.D.
externship students work a four week rotation. The DIPRC is staffed 52 weeks a year by
these Pharm.D. students; however, there may be only two students during the Christmas
break. Additionally, two months out of the year a pharmacy practice resident spends some
time in the DIPRC; however, their role varies considerably. Usually, the pharmacy
practice residents are working on other projects, so only from about a quarter to one half
of their time is spent on the front line answering questions. Occasionally, the DIPRC has
intern students and foreign visitors. These students usually just observe and help where
possible, but some are advanced enough to take calls and answer questions.
The students are encouraged to work on more than one question at a time,
meaning that the students are required to reprioritize their work as necessary and be
efficient with their time. Additionally, the co-directors stated that students were required
to fill out a data sheet for every question, and to have every question approved.
Furthermore, the co-directors stated that the students were indeed required to
interrupt their work and answer incoming telephone calls, and added that they had a "three
ring rule" where the telephone should be answered within three rings. One of the co-

96
directors called this "the greatest irritant of all" for the students working in the center
because it requires them to redirect and refocus their attention on something new.
The co-directors agreed with the students concerning the easiest types of questions
to answer, citing drug identification and availability questions as requiring the least amount
of work When asked about the most difficult types of questions that students typically
have trouble answering, they indicated three kinds of questions. First, vague questions are
frustrating for the students because there is no way to form a concise answer. Second,
questions that actually have no answer give students trouble because they do not know
when to quit searching. Third, questions in which the student has to synthesize
information from multiple sources, where no one has already written the answer down
"nice and neat".
Concerning how quickly students become skillful at answering questions, the co¬
directors indicated that the learning curve was steep. There are tremendous improvements
in the first few days of the rotation, and most students understand the process at the
beginning of their second week.
The co-directors felt that the most valued aspect of the service that they provide is
a high quality answer using resources that the caller does not have access to or time to
retrieve, and that is delivered in a timely fashion. When asked about the least valued
aspect of their service, the co-directors indicated that many of the callers often do not
want to give all of the necessary background information. Callers do not always
understand that the background information is necessary to deliver a useful answer. One
of the co-directors indicated that the background information was important in delivering
a good response: "The more specific the question the more specific the answer. The more
general the question the more general the answer, and what you really want as a
practitioner is a specific answer."

97
When asked about the factors that they felt influenced the callers' perceptions of
the quality of the service, the co-directors indicated that the way the student interacts with
the caller and the callers expectations are the two most important issues:
One of my theories.. .is that [finding] information is really only 10 or 20
percent of what [students] do. The packaging, the ribbons and bows, the
communication dynamic is really the other 80 to 90 percent. If [the
student] that [the caller interacts] with really takes an interest in the
question, seems to understand what they are asking..., provides a well
documented and thoroughly researched [answer], and communicates that
[answer] back in a clear and understandable way, that is the perfect
scenario.
I think [callers1 perceived service quality] is multifactorial. I think a lot of it
is their expectations. I'm big believer in expectations. A first time caller
who has no expectations.. thinks we have a great service [if they get any
kind of information], A frequent caller who [uses] our service a lot and
[has] high expectations to begin with.. .1 think that they could get pretty
good information and think it is mediocre.
However, as one of the directors pointed out, if the callers are not happy with the
quality of the service, they tend not to call back
When asked about whether delays in service influence the callers' impressions of
quality, the comments from the co-directors were mixed. One co-director stated that he
did not believe a delay would cause a decrease in the perceived quality, because they try to
safeguard any misunderstandings about when the question will be answered. Students are
instructed to contact callers when necessary and update them on progress of the question.
However, the other director felt that callers were very sensitive to delays, stating:
[Callers are] [v]ery sensitive to [delays], I mean, whether they really need
it or not, if they say they need it today, and we don't get it to them today,
that's a major failure on our part.
With regard to the future of the DIPRC, both co-directors felt some pressure from
the fact that funding from the hospital for the service has been dropping off for the last
few years. Eventually, they felt that some decisions would have to be made regarding

98
whether to limit the service to a select group of callers or whether or not the service
should be marketing on a fee-for-service or contractual basis.
If the DIPRC needed to add staff to support increases in question volume, the co¬
directors felt it would be possible to increase the staffing level from three students to as
many as five students a month. More than five students may be possible in the future
given the fact that more students are coming through the pharmacy program and there are
fewer rotation sites from which to choose. The co-directors added that the primary
limitation with using more students is the limited supervisory resources. They felt that
more than five students would be difficult to support, both operationally and
educationally, without at least one more drug information resident. One of the co¬
directors felt the ideal solution would be to hire a full time staff pharmacist to help answer
phones, supervise students, and approve responses; however, this would require
considerable financial resources not currently available. In addition, current space
restrictions prevent more than five students from working productively in the DIPRC.
Student Interviews
The first question that the students were asked was to describe their role in the
DIPRC. All of the students answered similarly, indicating that they were responsible for
taking calls from practitioners, researching questions, and providing answers to those
questions back to the callers. There was no indication that there are differences between
the students in terms of their responsibilities or the jobs that they were asked to complete.
All of the students uniformly stated that they all had equal responsibility for completing the
same work. As one student stated, "Everybody down there is treated the same. We are
all expected to do the same things. It is a learning experience for us, but at the same time
we are providing a service."

99
Students are required to work in the DIPRC from 8:00 a m. until 5:00 p.m. each
day. The DIPRC is typically open to take calls from 9:00 a m. until 5:00 p.m., Monday
through Friday, with the exception of certain holidays such as Thanksgiving and
Christmas. The students rarely stay past 5:00 p.m. At closing time, the students reach a
stopping point, post whatever they are working on onto a bulletin board designated for
new and unfinished questions, and then depart the center. The latest time that a student
has stayed beyond 5:00 p.m. was 5:25 p.m.; however, most of the students stated that they
rarely stay past 5:05 p.m.
The students are involved in some morning activities that potentially interfere with
how soon they can actually begin to work on questions received from callers. On
Tuesday, Thursday, and Friday the students must participate in a journal club where they
are asked to read, analyze, and present articles from the literature. On Wednesday, a
Quality Assessment procedure is conducted where a 10 percent random sample of the
previous week's questions are reviewed for clarity, accuracy, and response quality. These
activities begin at 8:00 a.m. and usually conclude between 9:00 a.m. and 10:00 a.m., with
the most common concluding time of about 9:30 a.m.
The students were asked what happens during this overlapping time from 9:00
a.m. until approximately 9:30 a.m. when they are occupied in one of these meetings. They
stated that the telephones were "religiously" opened at 9:00 a.m. If the telephone rings,
any student not presenting may get up and answer the telephone. If the question is an
urgent question then the answering student will immediately begin work on the question.
If the question is not urgent (i.e., not needed within the hour), then the question is posted
on the bulletin board for later retrieval.
All of the students take a lunch; however, the amount of time taken seemed to vary
considerably from 10 minutes to the full hour allowed. Two students stated that they
usually take less than an hour and often spend their lunch in the DIPRC. Five of the
students stated that they usually take their lunch outside the DIRPC and that lunch, for

100
them, lasts from 20 to 45 minutes. Only one student took the full hour for lunch each day.
When prompted, all of the students indicated that they would help answer the telephones
and work on urgent questions if they decide to take their lunch in the DIPRC. Lunches
are usually staggered from 11:00 a m. until 1:30 p.m. The students also try to schedule
their lunches so that no one is left by himself or herself to answer the telephones.
All of the students indicated that their primary role in the DIPRC is to take calls
from healthcare practitioners and write down their questions. After taking down the
question they are instructed to ask some "probing" questions in order to find out more
specifically where the question originated and what resources the practitioner had already
consulted. Also, they obtain the various demographic information necessary to complete
the data sheet (i.e., data collection form) such as subscriber status, profession, and contact
information (see Appendices H and I). Most of this information is taken on a separate
sheet of paper and then transcribed onto the data collection form after the call is
completed. The call is then logged into a logbook for tracking purposes.
At this point, the students described three possible routes of action. First, if the
question is not urgent they will tack the question onto the bulletin board with the other
questions that have not yet been started. Second, if the question is urgent and the
question that they are currently working on is not urgent, then they will stop working on
the current question and begin on the new question. Third, if they were already working
on an urgent question, they would ask one of the other students to take the question.
The next step is to research and write a response to the question based on the
information they have found. Following this, they bring this response to one of the co¬
directors or a qualified resident for approval. If the co-directors indicate that the response
needs more work then they continue to research and re-work the answer. Once the
answer is approved, they respond to the caller, usually by telephone, but sometimes by fax
or letter. Finally, the call is logged out of the logbook and the data sheet is placed in the
Quality Assurance bin.

101
All of the students stated that data sheets are filled out and approval is obtained for
each question asked, regardless of the question's apparent simplicity, such as callers asking
for phone numbers to pharmaceutical companies. Sometimes, complex questions are
broken down into several smaller questions. Furthermore, if a call comes in, the students
are instructed to interrupt their work and answer the telephone. As one students stated:
[It's] pretty much our duty if the phone rings to answer it regardless of
what we are doing unless we happen to be on the line with someone
else... Once we receive the call, whether we'd go on to work on it right at
that moment is.. a subjective thing. The students are also in charge of
prioritizing calls, in terms of what gets worked on when.
The students were consistent in their answers when asked how many questions at a
time they typically handle. All of the students said that they never work on more than one
question at the same time, but may work on other questions while they are waiting for
return phone calls or when higher priority questions arrive. As one student described:
It's not like you have two pieces of paper beside you and you have books
on each subject, that's not how we work... One [question] takes priority
over the other. It's never like anyone has four or five questions running
around... [We always] put them back on the [bulletin] board to keep them
straight.
When asked how they decide which question to work on next, all of the students
stated that the first consideration is given to the "time frame" that the requestor presented
to them. Primarily, the students tend to pay primary attention to the “Stat”, “Today”,
“Date”, and “No Rush” categories found on the data sheets. From the statements made
regarding priorities during the interviews, there seemed to be three priority levels. First,
“Stat” questions tend to receive the highest priority along with “Today” questions needed
within one hour. Second, “Today” questions without a particular time and “Date”
questions needed on that particular day are given the next highest consideration. The

102
lowest priority is given to “Date” questions not needed on that particular day and “No
Rush” questions. Within these particular categories, those questions that are patient
related as opposed to general information are given priority. There was no evidence
suggesting that students give special priority to questions based on the callers' profession
or subscriber status (i.e., subscriber, non-subscriber, or health center employee).
The students indicated that once they begin to answer a question, they usually also
complete the question. The student who takes a call often feels obligated to answer that
question. Two students estimated that for 60 to 90 percent of the questions that they
personally recorded, they also completed the answer. As one student explained:
It's almost easier just to finish up answering your own questions than try to
get involved in answering someone else's questions... [They] may have
done a lot of research [on the question]... and looked in areas that [are] not
written down... [Usually] when people are doing a question, there is... a
thought process going on.
The students uniformly stated that telephone numbers to pharmaceutical
companies and drug identification questions were the easiest questions to answer.
However, the question types that students selected as the most difficult types to answer
tended to vary. Four students mentioned that questions regarding new and foreign
medications were the most difficult. Three students stated that the most difficult questions
involved treatments for unusual disease states or when little is known about the therapy.
Two students said that pharmacokinetic and dosing questions were the most difficult.
Additionally, two students said that broad, non-specific questions were also very difficult
because it is hard to guess what the caller actually wants to know about the topic.
Six of the students said that they were comfortable answering questions after their
first day in the DIPRC. Three of students said that after the first week they were
comfortable. One student was still not comfortable answering questions even after their
third week. Some of the students indicated that they are still learning how to answer

103
certain types of questions, especially those where the caller is asking their professional
opinion that may only be loosely supported by the literature.
The students felt that "accurate information" and "correct answers" were the most
valued aspect of the service that they provide. As one student stated, "We provide a sense
of mental security for [the callers]." The students felt that the callers did not always like
to give their demographic information or having to wait for an answer. They felt that
some of the callers misunderstood the kinds of information available in the DIPRC and
how quickly drug information could be retrieved. Also, they felt that callers wanted
concise answers and do not appreciate "additional information" not directly relevant to the
question asked. When asked specifically about factors that most influence the callers'
perceptions of the quality of the service the following five factors were mentioned. First,
how the question was answered was mentioned six times. This relates to how
systematically the question was answered and how the response was presented. Second,
timeliness or promptness of the answer was mentioned four times. Third, the number and
quality of the references used was also mention four times. Fourth, the attitude of the
student when the caller first calls in to the center was mentioned two times. Fifth, the
accuracy (i.e., correctness) of the information was mentioned once. Thus, the most
important factor from the students' perspective was the strength and style of the given
response. As one student described:
You can tell them, ‘Well I don’t know the answer to your question’, or you
can say T searched the literature and after an exhaustive search, I’m unable
to find anything supporting the use of this drug for this disease. ’ The
difference between those answers [is] one of them says I don’t care [and]
the other one says I tried but.. .1 couldn’t find anything for you. In other
words, how you word the answer is important, and what you can use to
cite as your answer is important
The students also felt that timeliness and promptness was important to callers'
perceptions of service quality; however, when asked more specifically about delays in

104
service, there were some mixed results. Five students indicated that callers probably were
not very sensitive to delays in service. Four of the students said that the callers ranged
from sensitive to very sensitive to delays in service. One student stated that the sensitivity
depended on the person. If the caller has worked in drug information before, then they
often tended to be a little more understanding. Some of the students felt that managing
the callers expectations regarding when they were going to receive a response was
important in maintaining the callers satisfaction, as two students stated:
People who are sensitive are those [who] call wanting [an] on the spot
dosing recommendation. They are a little frustrated .. that you can't just
open your book and tell them the answer right there. Those are the ones
where you can sense the frustration.
Time is very important to callers. That is how I feel about it. Like when
[the caller has] a patient [in their store or office] who has a question for
them. . .they give us a call hoping to get an answer before the patient leaves.
That constrains them to certain time limit. Which may make them feel like
they are more dependent on us. When they are more dependent they are
more likely to be disappointed if we don't provide.
Lastly, when asked about what could be done to improve the process at the
DIPRC, none of the students felt that they were being hindered in any way from doing
their job, and were generally very pleased with their experiences during the rotation.
However, six general suggestions were made. First, all references should be up to date.
Second, callers should be informed that the DIPRC is staffed by students who cannot
answer questions right away. Third, consider shortening the time allowed for lunch.
Several of the students felt that an hour for lunch was excessive. Fourth, at least two
more computers with upgraded software should be made available. Internet access should
be provided on all computers since that is often the only way to find information on herbal
and folk remedies. Fifth, a better orientation in the beginning regarding where to find

105
information on foreign drugs would be helpful in reducing the amount of time that it
initially takes in finding information on these drugs.

CHAPTER 6
MAIN RESULTS
Overview
This chapter presents the third and fourth parts of the data analysis. The third part
reports the results of the data obtained from the service quality questionnaire. The validity
and reliability of the service quality questionnaire is discussed first, followed by the results
of the hypothesis tests related to the variables derived from the questionnaires and data
collection forms. The fourth part describes the development, verification, and validation
of the simulation model. It also presents reports the results of the simulated experiments
used to explore the three specific research questions.
Part Three: Relationships Among PSQ. OSO, Behavioral Intention. Perceived Service
Time. Actual Service Time and Service Delays
Response to Questionnaire
Three hundred twenty nine questionnaires were mailed out during the initial
mailing. Approximately one week after the initial mailing a reminder postcard was sent
out to non-responders urging them to mail in the completed questionnaire. The post card
also asked subjects to contact the researcher if they had not received a survey. Two
weeks after the post card was mailed, non-responders were contacted by telephone to
remind them to send in their questionnaire or to ascertain their reasons for not responding.
Of the 332 questionnaires that were sent out, 203 were completed and returned,
resulting in a raw response rate of 61.1%. Eight mailed questionnaires and one postcard
were returned as undeliverable. When attempts to contact non-responders by telephone
106

107
were made, it was learned that two subjects were on vacation and ten subjects no longer
worked at the location where the questionnaire was mailed; therefore, these subjects never
received the survey. Adjusting the response rate for these 21 subjects using the procedure
proposed by Dillman (1978) resulted in a revised response rate of 65.3% for the main
questionnaire.
Table 6-1. Questionnaire Sample Description
Characteristic
Frequency (%)
% Responding
Subscriber Status:
Health System
55 (27.1)
59.8
Subscriber
68 (33.5)
72.1
Non-Subscriber
80 (39.4)
54.0
Profession:
R.Ph./Pharm.D.
141 (69.5)
68.4
Physician
13 (6.4)
46.4
R.N./Nurse Pract.
16 (7.9)
57.1
Other
33 (16.2)
47.8
Frequency of Use:
First Time User
49 (24.4)
1-2 Times/Year
26 (12.9)
3-5 Times/Year
46 (22.9)
5-10 Times/Ycar
40 (19.9)
10-15 Times/Year
17(8.5)
>15 Times/Year
23 (114)
Received After :
First Mailout
119(58.6)
Post Card
61 (30.1)
Telephone Call
23 (11.3)
Of the total number of responses, 55 (27 .1%) were employees of the UF Health
System, 68 (33.5%) subscribed to the DIPRC, and 80 (39.4%) were non-subscribers
(Table 6-1). Of these, subscribers had the highest response rate of 72.1% and non¬
subscribers had the lowest response rate of 54.0%. As expected, the majority of the

108
responses came from pharmacists (69.5%). Pharmacists also had the highest raw response
rate (68.4%), while physicians had the lowest response rate (46.4%).
Questions arrived within one of three response intervals. First, 119 responses
(58.6%) were received before the reminder postcard was sent out. Second, 61 (30.1%) of
the responses were received after the postcard was sent but before the reminder telephone
call was made. Third, the remaining 23 (11.3%) questionnaires were received after the
telephone reminder.
Assessment of Non-Response Bias
The most common reason given for not responding to the survey was lack of time.
Fourteen subjects stated that they had not had the time, but would fill out and mail the
questionnaire soon (Table 6-2). Of these fourteen, seven (50.0%) actually returned the
questionnaire as promised. Another 12 subjects stated that they had not yet received a
questionnaire in the mail, and asked to have another questionnaire sent to them.
Questionnaires were re-mailed to these individuals, and eight (66.7%) were returned
Two subjects stated that they never used the service, even though the data sheet
listed their name and address. The only ascertainable reason for this was that subscribers
sometimes allow other co-workers to borrow their name and subscriber number to call the
DIPRC. Two subjects did not fill out the questionnaire because they did not feel it applied
to them since they were not health professionals. Only two subjects contacted gave any
direct indication that they absolutely did not want to fill out the questionnaire, and they did
not want to give any specific reasons for their decision
ANOVA procedures were conducted for subscriber status, profession, and
response interval using perceived service quality (PSQ) and overall service quality (OSQ)
as the dependent variables. None of the analyses performed resulted in significant overall

109
F values (Table 6-3). Therefore, there was no reason to suspect response bias significant
enough to damage the external validity of the results derived from the questionnaire data.
Table 6-2. Reasons for Not Reponding
Reason
Frequency
(% Returned)*
Haven't had time, but will do it soon.
14 (50.0%)
Did not receive in mail (mail another).
12 (66.7%)
Person no longer works there.
10
Mailed the questionnaire already.
7
Person is a part-time worker
5
Questionnaire did not apply to me.
2
Not interested in filling out questionnaire.
2
Did not use your service.
2
On vacation or maternity leave
2
*Note: Where present, percent returned equals the response rate after the telephone
follow up attempt was made to reach the subject.
Table 6-3. Results of One-Way ANOVA Procedures Measuring Response Bias
Subscription
Status
Profession
Response
Interval
Measure
F (p-value)
F (p-value)
F (p-value)
PSQ
1.288 (0.28)
0.918 (0.43)
0.107 (0.90)
OSQ
0.505 (0.60)
0.617(0.61)
0.054 (0.95)
Dimensionality of the SERVPERF Sub-Scale
The dimensionality of the SERVPERF sub-scale measuring PSQ was assessed with
principal components factor analysis using a Varimax rotation. The analysis resulted in
four factors with eigenvalues of greater than one, explaining 62.0% of the variance. The
rotated component matrix is presented in Table 6-4.
A factor loading is essentially the correlation between the item and the factor
(Crocker and Algina, 1986). The majority of the factor loadings for each factor were
close to or above 0.600. Items five "Background Noise", six "Written Materials", seven
"Speak Clearly", and twenty-one "Know Needs" had the lowest factor loadings of-0.390,

110
0.506, 0.478, and 0.478, respectively. Of these, however, only item five's factor loading
was below the deletion threshold of 0.400.
Item five also had a relatively low communality (Table 6-5) of 0.288, indicating
that the common factors did not explain the majority of the variance for this item. The
low factor loading combined with the low communality suggested that item five was not a
valid item. Forcing the data to fit five factors confirmed this conclusion, since item five
was the only item to load on the fifth factor. Additionally, the positions of the other item
loadings remained unchanged. It was therefore decided to removed item five from the
model.
Table 6-4. Initial Rotated Component Matrix of Main Questionnaire Responses
for the SERVPERF Sub-Scale (N=118)f
Item
1
2
3
4
Q12R-lndividual Attention
0.892
Q18R-Personal Attention
0.861
013R-When Performed
0.711
QJ 4R-Prompt Service
0.708
0.318
QI6R-T00 Busy
0.626
0.455
015R-Willingness to Help
0.624
Q22R-Best Interests
0.506
0.411
Q7-Speak Clearly
0.478
0.301
Q8-Promised Time
0.827
011-Provides in Time
0.312
0.820
Q10-Dependable
0.375
0.738
Q9-Sympathetic & Reassuring
0.714
06-Written Materials
0.582
Q20-Safe Interactions
0.901
Q19-Polile Employees
0.894
Q17-Trust Employees
0.634
023R-0perating Hours
0.617
04-Necessary Resources
0.465
0.581
021R-Know Needs
0.460
0.308
0.478
Q5R-Background Noise
-0.390
^ "R " next to the question number indicates that the question was reverse coded for analysis purposes.

Ill
Table 6-5. Main Questionnaire Communalities for the SERVPERF Sub-Scale
(N=118)f
Item
Communality
Q4-Necessary Resources
0.571
Q5R-Background Noise
0.288
Q6-Written Materials
0.385
07-Speak Clearly
0.360
OS-Promised Time
0.762
Q9-Sympathetic & Reassuring
0.572
Q10-Dependable
0.774
Q11 -Provides in Time
0.846
012R-Individual Attention
0.822
013R-When Performed
0.572
Ql 4R-Prompt Service
0.710
Q15R-Willingness to Help
0.576
QI6R-T00 Busy
0.615
Ql 7-Trust Employees
0.500
Q18R-Personal Attention
0.812
019-Polite Employees
0.828
Q20-Safe Interactions
0.881
021R-Know Needs
0.548
Q22R-Best Interests
0.510
023R-0perating Hours
0.471
"R" next to the question number indicates that the question was reverse coded for analysis purposes.
Table 6-5 shows the final rotated component matrix after removing item five from
the analysis. This model also revealed four factors with the explained variance improving
slightly to 64.6%. A scree plot shows that, indeed, the eigenvalues for the components
begin to level off after four components (Figure 6-1). Since the eigenvalue for the fifth
component was 0.906 and not close to 1.00, the four-factor solution was selected as the
optimum factor structure.
Five dimensions of perceived service quality composed the original SERVQUAL
scale (Parasuraman et al., 1988): tangibles (TA), responsiveness (RS), reliability (RE),
assurance (AS); and empathy (EM). Next to the factor loading for each item listed in
Table 6-6 is a two letter code corresponding to the factor on which the item should load
as hypothesized by the original SERVQUAL dimensions.

112
Interestingly, there is some correspondence between the original five dimensions
reported by Parasuraman et al. (1988) and the five factors presented in Table 6-6. All of
the items representing assurance and reliability loaded on separate dimensions. However,
items originally belonging to the responsiveness dimension loaded with three of the items
from the empathy dimension (item 12 "Individual Attention", item 18 "Personal
Attention", and item 22 "Best Interests"), perhaps indicating that callers do not see these
domains as separate constructs. The remaining two empathy items (item 23 "Operating
Hours" and item 21 "Know Needs") loaded on factor four with item four "Necessary
Resources." Additionally, the new items replacing the tangibles dimension did not load
together as expected. Item seven "Speak Clearly" loaded on factor one with the empathy
and responsiveness items. As mentioned above, item four "Necessary Resources" loaded
on factor four. Item six "Written Materials" loaded on factor two, which was defined by
the reliability items.
Thus, there is some evidence suggesting that PSQ is a multi-dimensional construct,
and that the five dimensions suggested by Parasuraman et al. (1985; 1988) might have
merit in the drug information setting. However, it must also be recognized that the
proposed five dimensions were not fully replicated. In addition, other researchers have
also had difficulty duplicating these dimensions (Babakus and Boiler (1992); Babakus and
Mangold (1992); Brown et al. (1993); Carman (1990); Cronin and Taylor (1992); Headley
and Miller (1993)). Furthermore, the component structure obtained from the factor
analysis conducted on the main questionnaire responses did not correspond to the
component structure achieved from analyzing the responses to the pre-test. Therefore,
there is not enough evidence to draw reliable conclusions regarding the dimensionality of
PSQ as measured by the items in the SERVPERF scale. Therefore, for the purposes of
this study, the items in the SERVPERF scale will be evaluated as a uni-dimensional
measure of PSQ.

113
Table 6-6. Final Rotated Component Matrix of Main Questionnaire Responses
for the SERVPERF Sub-Scale (N=120)f*
Item
1
2
3
4
012R-lndividual. Attention
0.884 (EM)
Q18R-Personal Attention
0.853 (EM)
013R-When Performed
0.712 (RS)
Q14R-Prompt Service
0.703 (RS)
0.331
OI6R-T00 Busy
0.619 (RS)
0.465
Q15R-Willingness to Help
0.616 (RS)
0.301
022R-Best Interests
0.493 (EM)
0.445
Q7-Speak Clearly
0.479 (TA)
0.304
Q8-Promised Time
0.838 (RE)
011-Provides in Time
0.306
0.812 (RE)
Q10-Dependable
0.369
0.733 (RE)
Q9-Sympathetic & Reassuring
0.699 (RE)
Q6-Written Materials
0.522 (TA)
Q20-Safe Interactions
0.904 (AS)
Q19-Polite Employees
0.902 (AS)
Q17-Trust Employees
0.623 (AS)
Q23R-Operating Hours
0.748 (EM)
Q4-Necessary Resources
.410
0.657 (TA)
Q21R-Know Needs
0.454
0.498 (EM)
^ "R" next to the question number indicates that the question was reverse coded for analysis purposes.
* The letters in parentheses indicate the dimension on which the item loaded in the original
SERVQUAL research conducted by Parasuraman et al. (1988). Where EM=Empathy,
RS=Responsiveness, RE =Reliability, AS=Assurance, and TA=Tangibles.
Component Number
Figure 6-1. Scree Plot of Main Questionnaire Responses
(Variables = 20; N=141)

114
Reliability of Sub-scales
Reliability estimates were obtained using Cronbach's alpha for three sub-scales
measured by the questionnaire: (1) PSQ (items 4 through 23), (2) service time perceptions
(items 24 through 27), and (3) behavioral intention (items 33 and 34). Alpha was not
applicable to OSQ since it is only a single-item measure.
The 20-item SERVPERF sub-scale had an excellent initial alpha of 0.9053.
Question five "Background Noise" was the only item with an item-total correlation of less
than 0.30. Question five's item-total correlation was 0.1601, indicating that this question
did not contribute well to the scale (Table 6-8). This is evidenced by a significant
improvement in alpha if the item were to be deleted (from 0.9053 to 0.9137). Item five
was therefore removed from the SERVPERF sub-scale, resulting in a final scale composed
of the 19 remaining items. The final PSQ sub-scale alpha compared well to both the pre¬
test alpha (0.8873) (Table 6-9) and the alphas found by Cronin and Taylor (1992) (0.884
to 0.964), indicating that the reliability of the SERVPERF items are relatively consistent
across service settings.
The alpha for the service time perceptions sub-scale was 0.8322, and the alpha for
the sub-scale measuring behavioral intention was 0.8997 (Tables 6-9). Therefore, the
internal consistency of these two scales were also considered to be very good.
Furthermore, examining the item-total correlations of the perceived service time scale did
not reveal any items of questionable value (Table 6-8).
Descriptive Statistics of Questionnaire Measures
Table 6-10 lists the descriptive statistics (i.e., number, minimum, maximum,
median, mean, standard error, and standard deviation) for the study variables that were
derived from the questionnaire. First, the mean value of the PSQ scale was 35.70 points
(n=120, s=l 1.76) (Table 6-10). This meant that the average score on each of the variables

115
making up the PSQ scale was approximately 2.00 (35.70 / 19=1.92), as evidenced by
Table 6-11. Although the possible range for the PSQ score was between 19 and 133, the
lowest PSQ score was a 19 and highest observed PSQ score was a 93. Individually, the
two items that scored best on the PSQ scale were items four "Necessary Resources" and
fifteen "Willingness to Help" (Table 6-11). This suggested that, overall, respondents felt
that DIRPC has access to the resources necessary to answer their questions (mean=1.60,
s=0.74) and that the center was very willing to help them find answers to their questions
(mean=1.63, s=0.89). Item twenty-three "Operating Hours" scored the worst on the PSQ
(mean=2.29, s=1.21), suggesting that respondents were not as content with the operating
hours of the service as with the aspects of perceived service quality measured by the other
items.
Second, the average OSQ score was 1.74 (n=193, s=0.73), indicating that
respondents generally felt that the overall service quality of the D1PRC was "Very Good".
The best score given was a one (i.e., "Excellent") and the lowest score given was a five
(i.e., "Poor"). Third, perceived service time was measured by four items on the
questionnaire. On average, respondents agreed that the response time for their most
recent question was acceptable (n=201, mean=1.96, s= 1.12). Similarly, respondents
tended to disagree with the statement that by the time they received the information it was
no longer useful (n=201, mean=5.85, s=l.36). However, respondents were neutral when
asked if they wished the DIPRC would provide quicker responses to questions (n=197,
mean=4.31, s=1.77). Generally, the respondents felt that the response time was a little
shorter or equal to what they expected (n=200, mean=3.58, s=1.34). Third, the results
regarding the respondents' intentions to use the DIPRC in the future (item 33) and
recommend the service to a colleague (item 34) were very positive. Overall, the
respondents indicated that not only did they intend to use the service in the future (mean
1.46, s=0.66), they would also recommend the service to a colleague (mean-1.3 8,
s=0.55).

116
Since perceived service quality is a subjective measure, it is important to describe
results in terms of practical as well as statistical significance. The standard error for PSQ
was 1.07 points, which translated into a 95% confidence interval around the PSQ mean of
plus or minus approximately 2.10 points. Since the width of this confidence interval is
about 4.40 points, it was decided that mean differences of five points or more on the 19-
item PSQ scale would be practically significant. For overall service quality, the standard
error was 0.05 points, translating into a confidence interval width of about 0.20 points.
Therefore, it was decided that mean differences of at least 0.25 a point on the OSQ
measure would be a conservative estimate of practical significance.
Table 6-7. Item-total Statistics for SERVPERF Sub-Scale (n=118)f
Item
Corrected
Item-Total
Correlation
Alpha if
Item
Deleted
Q4
0.2759
0.8475
Q5R
0.1655
0.8521
Q6
0.2630
0.8563
Q7
0.4715
0.8386
Q8
0.5702
0.8380
Q9
0.2627
0.8561
Q10
0.6993
0.8347
Qll
0.6759
0.8343
Q12R
0.5866
0.8348
Q13R
0.4901
0.8378
Q14R
0.6695
0.8321
Q15R
0.6351
0.8347
Q16R
0.5473
0.8357
Q17
0.3480
0.8447
Q18R
0.6877
0.8316
Q19
0.4034
0.8419
Q20
0.3967
0.8419
Q21R
0.5806
0.8338
Q22R
0.5696
0.8369
Q23R
0.3272
0.8447
* "R" next to the question number indicates that the question was reverse coded for analysis purposes.

117
Table 6-8.
tem-Total Statistics for Service Time Perceptions (n=196)
Item
Corrected Item-
Total Correlation
Alpha if Item
Deleted
Q24
0.7508
0.7659
Q25R
0.6540
0.7913
Q26R
0.6547
0.8104
027
0.6574
0.7901
"R" next to the question number indicates that the question was reverse coded for analysis purposes.
Table 6-9. Final Sub-Scale Reliabilities
Item
N
Items in
Scale
Alpha
Pretest
Alpha
SERVPERF (PSO)
145
19
0.9137
0.8873
Time Perceptions
196
4
0.8322
0.6560
Behavioral Intention
191
2
0.8997
0.6292
Table 6-10. Descriptive Statistics for Questionnaire Measures
Measure
N
Min
Max
Median
Mean
Std.
Dev.
Std.
Error
PSQ
120
19
93
36.5
35.70
11.76
1.07
OSQ
193
1
5
2
1.74
0.73
0.05
Perceived Service Time
Q24-Acceptahle Time
201
1
7
2
1.96
1.12
0.08
Q25-No Longer Useful
201
1
7
6
5.85
1.36
0.10
Q26-Ouicker Response
197
1
7
4
4.31
1.77
0.13
Q27-Expected Time
200
1
7
4
3.58
1.34
0.09
Behavioral Intention
Q33-Use in Future
199
1
6
1
1.46
0.66
0.05
Q34-Recommend to
197
1
4
1
1.38
0.55
0.04
Colleague

118
Table 6-11. Descriptive Statistics for PSQ Items (n=120)t
Measure Min
Max
Median
Mean
Std.
Dev.
Std.
Error
Q4-Necessary Resources
1
4
1
1.60
0.74
0.07
Q6-Written Materials
1
7
2
2.00
1.00
0.09
Q7-Speak Clearly
1
6
2
2.02
1.14
0.10
Q8-Promised Time
1
5
2
1.84
0.77
0.07
Q9-Sympathetic & Reassuring
1
4
2
2.28
1.05
0.10
Q10-Dependab le
1
6
2
1.70
0.75
0.07
Q] 1 -Provides in Time
1
6
2
1.87
0.83
0.08
Q12R-Individual Attention
1
6
2
1.79
1.04
0.09
Q13R-When Performed
1
6
2
2.16
1.24
0.11
014R-Prompt Service
1
7
2
1.83
1.02
0.09
Q15R-Willingness to Help
1
6
1
1.63
0.89
0.08
OI6R-T00 Busy
1
6
2
1.96
1.13
0.10
Q17-Trust Employees
1
7
2
1.84
1.06
0.10
Q18R-Personal Attention
1
6
2
1.71
1.00
0.09
Q19-Polite Employees
1
7
2
1.65
0.85
0.08
020-Safe Interactions
1
7
2
1.77
0.91
0.08
021R-Know Needs
1
6
2
2.06
1.09
0.10
022R-Best Interests
1
6
2
1.71
0.89
0.08
Q23R-Operating Hours
1
6
2
2.29
1.21
0.11
^ "R " next to the question number indicates that the question was reverse coded for analysis purposes.
Results of Hypothesis Tests
This section reports the results of the seven general hypotheses tested in this study.
First, hypotheses one through three (HI, H2, and H3) were used to test the relationships
among PSQ, OSQ, and behavioral intention. The results from the data obtained in this
study are reported and then compared with the results obtained by Cronin and Taylor
(1992) to establish construct validity. Next, hypothesis four (H4) tests the relationship
between PSQ and perceived service time. Following this, hypotheses five and six (H5 and
H6) test the relationships between actual service time and PSQ and between service delays
and PSQ. Last, hypothesis seven (H7) tests the relationships among actual service time,
service delays, and perceived service time. Correlations between all of the variables used
in the hypothesis testing are presented in Table 6-12 below.

Table 6-12. Correlations Between Study Variables
Measure
PSO
OSO
024
025
026
027
033
034
Delay
AST
PSO
1.000
-
~
~
~
-
~
~
OSO
0.633
***
1.000
-
-
-
-
-
-
-
-
Perceived Service Time:
Q24-Acceptable Time
0.652
***
0.548
***
1.000
-
-
-
-
-
-
-
025R-No Longer Useful 0.525
***
0.407
***
0.641
***
1.000
-
-
-
-
-
-
026R-0uicker Response 0.520
***
0.394
***
0.605
***
0.533
***
1.000
-
-
-
-
-
027-Expected Time
0.475
***
0.404
***
0.629
***
0.518
***
0546
***
1.000
-
-
-
-
Behavioral Intention:
Q33-Use in Future
0.449
* * *
0.471
***
0.180
*
0.189
**
0.169
0.220
*
1.000
-
-
-
Q34-Recommend
To Colleague
0.482
***
0.612
***
0.316
***
0.245
***
0.201
***
0.294
**
0.832
***
1.000
~
Service Delay
(Delay)
0.235
**
0.148
*
0.062
0.028
0.096
0.028
0.108
0.077
1.000
-
Actual Service Time
(AST)
0.179
0.175
*
0.141
0.052
0.164
*
0.153
*
0.058
0.151
*
0.100
1.000
*p<0.05 **p<0.01 ***p<0.001
" "R " next to the question number indicates that the question
way reverse coded for analysis purposes.

120
HI: There is a positive relationship between perceived service quality (PSQ) and
overall service quality (OSQ).
The correlation between PSQ and OSQ was 0.633 (p<0.001) (Table 6-12),
indicating that evaluations of perceived service quality tend to correspond with evaluations
of overall service quality. This relationship is both strong and positive, confirming the
relationship described in HI. This correlation was also consistent with the correlation
reported by Cronin and Taylor (1992), who reported a correlation between SERVPERF
and overall quality of 0.601.
H2a: Intention to use the service in the future is positively associated with evaluations
of perceived service quality (PSQ).
H2b: Intention to recommend the service to a colleague is positively associated with
evaluations of perceived service quality (PSQ).
The correlation between item 33 "Use in Future" and PSQ was 0.449 (p<0.001),
and the correlation between item 34 "Recommend to Colleague" and PSQ was 0.482
(p<0.001) (Table 6-12). Interestingly, the correlations observed in this study were
stronger than those observed by Cronin and Taylor (1992), who reported a correlation
between SERVPERF and purchase intention of only 0.365.
Further analysis was conducted by categorizing the individual responses into two
levels, “Strongly Agree” and “All other responses”. T-tests revealed statistically
significant differences in PSQ level means for both questions (p<0.001). For item 33
“Use in Future”, the mean difference in PSQ between “Strongly Agree” and “All other
responses” was 11.75 points. For item 34 “Recommend to Colleague”, the mean
difference in PSQ between “Strongly Agree” and all other responses was 11.5 points.
Therefore, the relationship between PSQ and behavioral intention is also practically
significant. Thus, hypotheses H2a and H2b are supported.

121
Table 6-13. PSQ by Level of Behavioral Intention
Measure
N
Mean
PSQ
Std.
Dev.
95% C.I. for Mean
Lower Upper
Bound Bound
033-Use in Future
Strongly Agree
76
31.39
8.94
29.35
33.44
All Other Responses
44
43.14
12.40
39.37
46.91
034-Recommend to
Colleague
Strongly Agree
80
31.99
9.61
29.85
34.13
All Other Responses
39
43.49
12.19
39.54
47.44
H3a: Intention to use the service in the future is positively associated with evaluations
of overall service quality (OSQ).
H3h: Intention to recommend the service to a colleague is positively associated with
evaluations of overall service quality (OSQ).
The correlation between item 33 "Use in Future" and OSQ was 0.471 (p<0.001),
and the correlation between item 34 "Recommend to Colleague" and OSQ was 0.612
(p<0.001) (Table 6-12). These correlations were consistent with the correlation between
OSQ and purchase intention reported by Cronin and Taylor (1992), who reported a
correlation between overall quality and purchase intention of 0.527. It is interesting to
note that the relationship regarding future use was not as strong as the relationship
regarding future recommendation, perhaps suggesting that perceived service quality is
more important to intended future recommendation behavior than intended future use.
As with H2a and H2b, additional analysis was conducted by categorizing the
individual responses into two levels, “Strongly Agree” and “All other responses”. T-tests
revealed statistically significant differences in OSQ between the response levels for both
items 33 and 34 (p<0.001). The mean difference between “Strongly Agree” and all other
responses in OSQ for item 33 was 0.80 points. The mean difference between “Strongly
Agree” and all other responses in OSQ for item 34 was 0.96 points. Therefore, the

122
relationship between PSQ and behavioral intention is also practically significant. As such,
hypotheses H3a and H3b are accepted.
Table 6-14. OSQ by Level of Behavioral Intention
Measure
N
Mean
OSQ
Std.
Dev.
95% C.I. for Mean
Lower Upper
Bound Bound
033-Use in Future
Strongly Agree
114
1.41
0.55
1.31
1.51
All Other Responses
78
2.21
0.71
2.05
2.37
034-Recommend to
Colleague
Strongly Agree
123
1.40
0.52
1.30
1.49
All Other Responses
67
2.36
0.64
2.20
2.52
H4a: Acceptability of the response time of the service is positively associated with
evaluations of perceived service quality (PSQ).
H4b: Perceived usefulness of the information once the response was received is
positively associated with evaluations of perceived service quality (PSQ).
H4c: Perceived quickness of response is positively associated with evaluations of
perceived service quality (PSQ).
H4d: Deviations from expected response times are positively associated with
evaluations of perceived service quality (PSQ).
The correlations between PSQ and the measures of perceived service time were all
statistically significant (Table 6-12). First, the correlation between PSQ and item 24
“Acceptable Time” was 0.652, indicating that as the acceptability with the service time
improved, so did perceived service quality. Second, the correlation between PSQ and
item 25 “No Longer Useful” was 0.525, indicating that respondents who felt that the
information was less useful because of prolonged response times also had lower

123
perceptions of service quality. Third, the correlation between PSQ and item 26 “Quicker
Response” was 0.520, indicating that the desire for quicker response times was
significantly related to evaluations of perceived service quality. Fourth, the correlation
between PSQ and item 27 “Expected Time” was 0.475, indicating that longer than
expected response times were associated with lower PSQ scores.
Table 6-15. PSQ by Level Perceived Service Time
Measure
N
Mean
PSQ
Std.
Dev.
95% C.I. for Mean
Lower Upper
Bound Bound
Scheffe’
Differences
024-Acceptable Time
1) Strongly Agree
44
27.82
6.74
25.77
29.87
2, 3,4
2) Agree
59
37.56
9.14
35.18
39.94
1,4
3) Somewhat Agree
9
44.33
6.30
39.49
49.18
1
4) Neutral/Disagree
8
55.63
19.11
39.65
71.60
ANOVA
1, 2
F=27.1, pO.001
025-No Lonser Useful
1) Agree/Neutral
19
47.84
18.75
38.80
56.88
3, 4
2) Somewhat Disagree
7
40.29
9.60
31.40
49.17
4
3) Disagree
55
36.47
47.84
18.75
38.80
1, 4
4) Strongly Disagree
39
27.87
40.29
9.60
31.40
ANOVA
1,2,3
F=18.7, pCO.001
026-Ouicker Response
1) Agree/Neutral
69
40.00
12.57
36.98
43.02
3,4
2) Somewhat Disagree
8
32.00
7.45
25.78
38.22
None
3) Disagree
30
32.10
7.01
29.48
34.72
1
4) Strongly Disagree
13
23.46
3.80
21.17
25.76
ANOVA
1
F= 11.3, pO.001
027-Expected Time
1) Shorter than Expected
44
29.68
9.64
26.75
32.61
2, 3
2) Equal to Expected
50
36.26
9.02
33.70
38.82
1, 3
3) Longer than Expected
26
44.81
13.72
39.27
50.35
ANOVA
1, 2
F=17.4, pO.001
In terms of practical differences, the ANOVA procedures revealed significant
overall differences among the levels for all variables tested (all p<0.001) (Table 6-15).
The post hoc analyses revealed statistically significant differences for all variables between
the first and last levels of the analyses (all p<0.001). However, item 24 “Acceptable
Time” seemed to have the best discrimination of all the items used. There was a large

124
difference observed between the PSQ means of the first level “Strongly Agree” and the
last level “Neutral/Disagree” (27.81 points). More interesting, however, is that the
practically significant difference that emerged even between levels one and two. The mean
difference between “Strongly Agree” and “Agree” for item 24 “Acceptable Time” was
9.74 points, which is almost twice the threshold set for practical significance.
Overall, these results indicate that a strong relationship exists between the
perceived acceptability of the response time and perceived service quality. Based on these
results, all four hypotheses relating to perceived service time (H4a through H4b) were
accepted.
H5a: There is a significant inverse relationship between evaluations of perceived
service quality (PSQ) and actual service time.
H5b: There is a non-linear relationship between actual service time arid perceived
service quality (PSQ).
The linear regression did not reveal a statistically significant relationship between
actual service time and PSQ (R=0.179, Adjusted R2=0.022, (30=34.443, (31=0.0013).
Although the regression analysis was non-significant (p=0.07), the direction and strength
of the relationship is worth additional comment. An examination of Figure 6-2 does show
a definite, if slight, upward linear trend in predicted PSQ, indicating that as actual service
time (i.e., the observed response time) increased the PSQ score also tended to increase
(i.e., perceived service quality decreased). The prediction equation of the original model
was PSQ = 34.443 + 0.0013 * (Actual Service Time). Therefore, on average, it took
roughly 1,300 minutes before an increase in PSQ of one point, or roughly one full day.
Furthermore, 4 to 5 days passed before PSQ increased by five points. These results
suggest that actual service times have negligible effect on PSQ. Although there may be a
slight relationship between actual service time and PSQ as suggested by hypothesis H5a ,

125
this relationship is not practically significant. Therefore, hypothesis H5a is rejected on
both a practical and statistical basis.
Furthermore, an examination of the residual plots in Figures 6-3 and 6-4 did not
reveal any distinct patterns suggesting that a non-linear transformation would explain the
relationship between actual service time and PSQ better than simple linear regression.
Figure 6-3 shows the residuals for actual service times on a scale of 0 to 9000 minutes.
Since a large number of the residuals are clustered in the one day range from 0 to 500
minutes, Figure 6-4 was also produced. As mentioned, neither of these figures revealed a
recognizable pattern that could be reduced through non-linear regression methods.
Therefore, H5b was also rejected.
H6: Delays in service are negatively related to lower evaluations of perceived service
quality (PSQ)
There was a significant correlation between delays in service and PSQ (r=0.235,
p=0.01) (Table 6-12). Delayed responses had a mean PSQ of 34.37 (s=10.55), whereas,
responses that were not delayed had a mean PSQ of 41.50 (s=l5.22) (Table 6-16). This is
a practically significant mean difference of 7.13 points. Therefore, the hypothesis is
accepted that delays in service are related to lower evaluations of perceived service
quality.
Table 6-16. Perceived Service Quality by Delay in Service
95% C.I. for Mean
Delay in
Mean
Std.
Lower
Upper
Service
N
Median
PSQ
Dev.
Bound
Bound
No
97
35
34.37
10.55
32.25
36.50
Yes
22
40
41.50
15.22
34.75
48.25

126
Figure 6-2. Regression Equation Plot of PSQ and Predicted PSQ by Actual Service
Time where PSQ=34.443+0.0013*(Actual Service Time)
Figure 6-3. Residual Plot of PSQ on Actual Service Time

127
50
40
4>
30
©
©
Residuals
-4 N>
o o
©
..
• :• 4* ♦
©
*
©
©
©
©0
© ©
4 4 *
©
50 * 100
©
150 200 ^ 250 3004 350 400
450
500
-10
* 4 ^
© ©
♦ 4
4>
4 ♦ ©
©
© O © O
© ©
-20
Actual Service Time (In Minutes)
Figure 6-4. Residual Plot of PSQ
on Actual Service Times Occurring within One Day
H7a: There is a positive relationship between actual service time and perceived service
time.
H7b: There is a positive relationship between service delays and perceived service time.
These two hypotheses were used to evaluate the relationship among measures of
the actual service time, service delay and perceived service time. Only two correlations of
the eight analyzed were significant (Table 6-12). First, actual service time versus item 26
"Quicker Response" had a correlation of 0.164 (p=0.033). Second, actual service time
versus item 27 "Expected Time" had a correlation of 0.153 (p=0.045). Surprisingly, there
was no relationship detected between the perceived service time measures and delays in
service. Furthermore, since the two significant correlations were small and very little of

128
the associated variance between actual service time and perceived service time was
explained, it was difficult to make any practical determination of significance.
It is interesting to note, however, that there does appear to be at least a trend in
the data concerning actual service time. Table 6-17 shows the percentage of samples
above the mean actual service time for the response categories corresponding to item 27
"Expected Time". Out of the 58 responses indicating that the response time was shorter
than expected, only 16 (27.6%) had actual service times higher than the mean. However,
of the 37 responses indicating that the response time was longer than expected, 19
(51.4%) had actual service times higher than the mean. A j2 analysis run on the cell
counts only resulted in a borderline value of 5.64 (p=0.06). Therefore, although there is
some evidence suggesting that a relationship may exist between actual service time and
perceived service time, this evidence is not statistically conclusive. Therefore, hypotheses
H7a and H7b are rejected.
Table 6-17. Percentage Below/Above the Mean Actual Service Time
by Q27-Expected Time
Q27-Expected Time
N
Percentage
Below Mean
Percentage
Above Mean
Shorter than Expected
58
72.4%
27.6%
Equal to Expected
78
64.4%
35.6%
Longer than Expected
37
48.6%
51.4%
Part Four; Simulation Results
Construction of Simulation
The simulation was constructed in GPSS/H professional and run on an MS-DOS
format computer. The data needed to construct the simulation were obtained from the

129
analyses of the historical and concurrent data as well as the co-director and student
interviews described earlier in this chapter. Empirical interarrival distributions were
obtained from the historical data and separated by time of day. Empirical service time
distributions for the three general categories of questions, as well as the times to take calls
and return answers were obtained from the concurrent data. The priority discipline for
"stat", "today", "date", and "no rush" questions was built into the model as described by
the students and co-directors during the personal interviews. The distributions of the
desired response times by time of day were obtained from the historical data.
In order to assure statistical independence, separate pseudo-random number
streams (20 altogether) were used for each interarrival and service time function, as well
as for statistical transfers and selection variables. Furthermore, different random number
seeds were used for each replication of the simulation.
Four primary assumptions were made during the construction of the simulation.
First, all students have the same service time distributions. While there certainly are
differences in students with regard to their individual service times, it is difficult to predict
each month how well the individual students are going to perform. It was decided to
assume that all students are equal since decisions must be made in the DIPRC regardless
of random differences in student productivity from month to month.
Second, no arrivals occur before 9:00 a m. or after 5:00 p.m. This assumption
does not violate reality with any great significance since the telephones are not turned on
until 9:00 a.m. in the morning and are turned off consistently at 5:00 p.m. Questions are
sometimes left on the voice mail during non-working hours; however, it was determined
from the interviews that this is a rare occurrence.
Third, all arrivals occur by telephone and all responses are returned by telephone.
Although the large majority of arrivals do occur by telephone, sometimes a question will
arrive by fax, electronic mail, or personal visit. This assumption makes little difference in
this setting at the current question volume since the telephone capacity of the DIRPC is

130
extremely large in relation to the volume of calls. Furthermore, the service time
distributions do not preclude that a written response was delivered in addition to a verbal
response. If the volume of calls were to increase to a point where balking consistently
occurs, it may be necessary make changes in the simulation distinguishing these arrival
sources. In the current simulation, however, it was assumed that if a simulated caller
encounters a busy or an unanswered telephone, the caller would try back once
immediately. If the caller then encountered a busy signal or unanswered telephone, the
caller would wait 5-10 minutes (uniformly distributed) and then try again. If the caller
encountered a busy or unanswered telephone a third time, the caller balks. This
assumption was based on discussions with the co-directors regarding the behavior they
would expect of callers encountering this situation in the real system.
Fourth, questions requesting "stat" attention have the highest priority, followed in
priority by “today”, “date”, and “no rush” questions, with “no rush” questions having the
lowest priority. Questions are answered and returned based on this priority system.
Furthermore, questions with higher priority preempt questions of lower priority.
Questions within the same priority class are served on a first-come-first-served basis. This
is the priority system described by the co-directors and students during the personal
interviews.
Verification and Validation of the Simulation
Interviews were conducted with each of the co-directors of the center in order to
explain the assumptions made in the simulation and establish the face validity of the
simulation program. In these interviews, the researcher systematically described the
simulation program to each of the co-directors using the block diagrams (Appendix Q)
Both co-directors agreed with that the simulation accurately described the work process
of the DIPRC, and only minor changes were suggested.

131
Four phases of pilot runs were used to verify the behavior of the simulation and
validate the predictive ability of the simulation. The first phase of simulations was used to
trace the simulation for errors. This was conducted using the interactive debugger feature
provided by GPSS/H and the block count output. The block counts provide information
regarding how many transactions pass through a particular line of code during a simulation
run. In this phase, it was verified that generated transactions were being routed through
the simulation as expected. Also, all of the block counts summed as expected (eg., if 100
transactions entered the system, the presence or termination of all 100 were accounted).
The second phase of simulations was used to verify that the simulation behaved as
expected when interarrival times increase or decrease. Given the underlying foundation of
queuing theory, as interarrival times decrease (i.e., arrivals become more frequent) the
student utilization percentage, the total service time, and the expected number of
transactions in the system should increase asymptotically. Likewise, as the interarrival
times increase (i.e., arrivals become less frequent), utilization, total service time, and
expected number in the system should decrease.
To verify that the simulated system exhibited this hypothesized behavior, 20
system days (i.e., one month) were simulated in ten separate runs of one replication each
(where each replication included an antithetic pair of runs). A modifier was used in the
GENERATE statements of the GPSS/H code (see Appendix R, lines 151 to 171) for each
of the ten runs to increase or decrease the interarrival rate by a given percentage. The
system to be verified (i.e., the simulation reflecting the observed data) had a modifier of
one (i.e., &AM = 1). The other nine runs used respective modifiers of 0.20, 0.40, 0.60,
0.80, 1.20, 1.40, 1.60, 1.80, and 2.00 (Table 6-18). This corresponded to simulated
increases in interarrival times of 20% to 100% (i.e., fewer arrivals), and decreases in
interarrival times of 20% to 80% (i.e., more arrivals).
The simulated system behaved as expected. As the arrival modifier increased from
0.2 to 2.0, the utilization percentage decreased from 0.999 (i.e., fully utilized) to 0.460

132
(i.e., less than half utilized). In addition, the total service time in minutes and expected
number in the system decreased from 3519 to 107 minutes and 499.5 to 1.7 questions,
respectively.
The third phase of pilot runs was used to establish an appropriate warm-up period
for the simulation, to verify that the simulation produced the expected number of arrivals
and delays, and to validate the total service time distribution. To obtain data for these
comparisons, six replications of 20 simulated days each were run (i.e., six independent
simulated months).
Since the simulation begins with zero transactions in the system, it is important to
detect the amount of time necessary for the simulation to warm up (i.e., achieve steady
state). The graphical method suggested by Hoover and Perry (1989) was used to select
the warm up period used in the simulation. The simulation program was programmed to
verify and output the total number of questions in the system at a random point once per
simulated hour. Each 20-day run resulted in 160 antithetic pairs (960 total samples). The
twelve samples corresponding to each hour were averaged to obtain an expected number
in the system for that particular hour across all replications. At each hourly change, the
result was averaged with the previous hourly results to obtain the expectation function
presented in Figure 6-5 (i.e., where E(Number in System)=((Ni+... +Nl)/i), where i = 1 to
n). As Figure 6-5 illustrates, the appropriate warm up period for the simulation appears to
be just under five days, indicating that the system more or less reaches steady state during
the fourth simulated day. To allow for some margin of error, five days (i.e., 2400
simulated minutes) was selected as the warm up period.
The simulation compared favorably against the historical data on three variables of
interest: (1) total question arrivals per month, (2) average question arrivals per month, and
(3) service delay percentage. A significant difference in the average number of arrivals per
day and per month between the historical and simulated systems would indicate that the
interarrival distributions input into the model were not behaving as expected. As

133
mentioned earlier, the historical data for the past five years revealed an average total
number of arrivals each month of 266.0 (s=25.26) and an average daily number of
questions of 12.8 (s=l .25) (Table 6-19). The simulation produced results very close to
these with an average total number of questions per month of 259.4 (s=l 4.93) and
average daily number of questions of 13.08 (s=0.75). The small differences observed were
not statistically significant with the mean difference in total monthly arrivals having a T
statistic equal to -1.24 (p=0.23) and the mean difference in average daily arrivals having a
T statistic equal to 0.82 (p=0.42).
Since delays in service appeared to have a significant relationship to perceived
service quality, it was important to assess whether the simulation could adequately predict
the percentage of service delays. The results indicated that the percentage of service
delays for both the simulated and historical data were the same at 16% (T = -0.39, p =
0.71, where the simulation s=0.03 and historical s=0.37) (Table 6-19).
In addition to insignificant differences in the mean values, the confidence intervals
for the three comparisons are also very consistent, having similar lower and upper bounds.
These results indicate that the simulation program models both the number of question
arrivals into the system and the percentage of service delays with good precision.
The data from this third phase of simulation runs was also used to compare the
concurrent and simulated total service time distributions. The simulation was programmed
to produce a table of total service times in 15-minute intervals. Likewise, the concurrent
data was also sorted and counted into in 15-minute time intervals. The concurrent and
simulated distributions were compared for total service times less than 480 minutes (i.e.,
completed within one day). This cutoff was selected because of the distortion between
real time and simulated time after 480 minutes. For example, day two of the simulation
starts at 481 minutes on the simulation clock; however, day two in real time starts at 1,441
minutes.

134
It was not surprising that the chi-square goodness of fit test between the
concurrent and simulated total service time distributions resulted in a rejection of the
hypothesis that the two distributions are equal (d.f.=31, x2=585.96, p<0.005). Although
Figure 6-6 graphically illustrates a reasonably good fit between the two probability density
functions (PDF), there are several spikes in the concurrent data (i.e., at 135, 180, 255,
285, and 300 minutes) which create large differences in the cell percentages.
Compounding this problem is a large simulated sample size (n=2814) which tended to
magnify these cell differences.
Simple linear regression of the cell percentages was used instead to obtain a better
feel for how well the simulated total service times predict the distribution of total service
times observed in the concurrent data. The regression yielded an adjusted R2 of 0.86,
indicating that the simulated density function explains approximately 86% of the variance
in the concurrent density function. Ideally, the coefficients for the intercept ((30) and the
simulated service times ((31) would be 0.00 and 1.00, respectively, indicating a perfect fit.
The realized coefficients were close, with an intercept coefficient if 0.002 and a service
time coefficient of 0.93. Interestingly, the cumulative density function (CDF) produced an
even better fit, with an adjusted R2 of 0.996, an intercept coefficient of -0.042, and a
service time coefficient of 1.07. This improvement can largely be attributed to the
smoothing of the spikes in the densities mentioned earlier. Plots of the CDFs are
presented in Figure 6-7.
The fourth phase of pilot runs was used to validate the model under restricted
conditions where an analytical solution based on queuing theory could be compared to the
output generated by the simulation. The simulation was modified so that it closely
resembled a M/M/3 queuing system. In order to accomplish this, five changes were made
to the model. First, the empirical interarrival distributions by time of day were replaced
with one exponential distribution describing the expected interarrival time for the entire
day. Second, the service time distributions for the separate question categories were

135
replaced with one exponentially distributed function describing the overall expected
service times. Third, the times to take a call and return an answer were set equal to zero
so that multiple phases would not distort the results. Fourth, the priorities for all
transactions were made equal (i.e., “stat”, “today”, “date”, and “no rush” questions were
all treated equally). Fifth, all instances of preemption were replaced with a first-come
first-served priority status. The results of the simulated runs were then compared to the
analytical results.
Twelve runs of 20 simulated days each were generated using this modified
simulation program. The program was also coded so that it included the five-day warm up
period assessed earlier. The queue statistics at the end of 20 days were compiled,
resulting in sample of 24 data points (i.e., twelve antithetic pairs).
There was no statistically significant evidence to suggest that the simulated model
differed from the exact analytical solution using the formulas for a M/M/s queuing system.
The values for p, Lq, L, Wq, and W obtained from the restricted simulation and closed
form methods were all comparable, and none of the p-values for the computed T-statistics
were even close to statistical significance (Table 6-20). Since the restricted model
performed as expected when compared to a known closed form method, this suggested
that the unrestricted model should also behave as expected if an analytical model existed
for comparison.
The results from the above verification and validation methods indicated that the
simulation was a credible representation of the actual system. While it is not expected that
the model will perfectly emulate the real system, the model does seem to perform with a
reasonably close fit that is sufficient for decision making. Therefore, it was concluded that
this model was valid for use as a decision making tool in the DIPRC.

136
Table 6-18. Student Utilization, Total Service Time, and Expected Number in
System by Arrival Modifier at 20 Simulated Days
Arrival
Modifier
(&AM)
Student
Utilization
Percentage
Total
Service Time
(min.)
Expected
Number in
System
0.2
0.999
3519
499.5
0.4
0.996
2454
173.0
0.6
0.984
1329
61.5
0.8
0.967
682
25.0
1.0
0.758
221
6.4
1.2
0.698
162
3.8
1.4
0.564
143
2.8
1.6
0.510
110
1.9
1.8
0.468
108
1.8
2.0
0.460
107
1.7
* Lower Arrival Modifiers equal shorter interarrivals
E
CD
(/)
>.
CO
c
CD
SI
E
3
LU
<1 <1 o. 'S' S' ^ ,'S- .'S' ."S- .‘S- .'S' .'S' -
V “V "'T? "V > “cs> <9
70 77 7 <9
Hour of Day
Figure 6-5. Expected Number in System for Six Replications of 20 Days

137
Table 6-19. Descriptive Statistics for Selected Comparisons
Between Observed and Simulated Data*
N
Mean
Std.
Deviation
95% C.I. for Mean
Lower Upper
Bound Bound
Observed:
Total Questions
2395
266.0
25.26
259.8
272.2
Daily Questions
66
12.8
1.25
12.4
13.1
Delayed Percent
2395
0.16
0.37
0.15
0.18
Simulated:
Total Questions
12
259.4
14.93
249.9
268.9
Daily Questions
12
13.0
0.75
12.5
13.4
Delayed Percent
12
0.16
0.03
0.14
0.18
*Note: Confidence Intervals Produced Using T-Statistic
Table 6-20. Comparison of Simulated Queue Statistics Versus Exact Solution '
Exact
Solution
Simul.
Mean
Sim. Std.
Deviation
95% C.I. for
Simulated Mean
p-value
Lower
Bound
Upper
Bound
Utilization (p)
0.78
0.78
0.07
0.82
0.75
0.81
No. Waiting (Lq)
2.25
2.39
1.88
0.71
1.60
3.19
No. in System (L)
4.61
4.74
2.05
0.76
3.87
5.60
Queue Wait (Wq)
80.28
82.60
58.90
0.85
57.70
107.50
Total Service Time (W)
164.03
164.10
61.0
0.99
138.40
189.90
^ Confidence Intervals Produced Using 7-Statistic

138
/T<' kI? xia ^?r> '&p
0 ° °& óo vo && °0 'O' V? ®&
Total Service Times observed pdf
(Minutes) - - - Simulated PDF
Figure 6-6. Observed Versus Simulated Probability Density Functions (PDF)
Total Service Times
(Minutes)
Observed CDF
. _ . . Simulated CDF
Figure 6-7. Observed Versus Simulated Cumulative Density Functions (CDF)

139
Relationship of Staffing Levels and Service Times to the Percentage of Service Delays
The validated simulation program was used to test three specific research
questions. First, the simulation was used to explore how changes in the staffing levels and
service rates would affect total service times and service delays. Second, the simulation
data was analyzed to determine the optimal combination of staffing levels and potential
improvements in service rates and service delays. Third, the optimal solution was tested
for sensitivity to changes in call arrival rate.
Rl: How do changes in staffing levels and service rates impact simulated total service
times and the percentage of service delays in the drug information service?
Sixty simulated months (30 antithetic pairs) were run for five staffing levels and
nine levels of service rates. The staffing levels were varied from one to five students. The
service rate corresponding to the time required to research a question and obtain an
approval (i.e., research time) was varied by degrees of 5%, 10%, 15%, and 20% above
and below “normal” operation. “Normal” operation was defined as three students staffing
the center and with arrival and service rates derived from empirically observed
distributions To evaluate this research question, staffing levels and variation of service
rates were evaluated independently compared to the “normal” system. Tables 6-21, 6-22,
and 6-23 present the descriptive statistics for the various parameters measured by the
simulation, including: total service time (W), wait in queue (Wq), number in system (L),
queue length (Lq), number completed, utilization (p), and delay percentage.
Effect of Changes in Staffing Levels
There were sharp decreases in W as the number of simulated students staffing the
service was increased from one student (3521 minutes, s=331.5) to two students (1385.0
minutes, s=437.4), and from two students to three students (239 minutes, s=97.6) (Table

140
6-21). A large decrease in W was also observed as the number of students was increased
from three to four students (122.0 minutes, s=17.7), although the trend was less dramatic.
Decreases in service times appeared to level off after four students.
These decreases in W can largely be explained by examining the results for Wq
For a staffing level of only one student, 3478.6 minutes (s=339.5) of the total service time
(98.7%) were spent just waiting in the queue (Table 6-21). In contrast, only 121 (s=86.1)
minutes (50.6%) of the total service time was spent in the queue when three students
staffed the center. Wq decreased even more when four and five students staffed the
center. Thus, as more students are employed, questions enter service quicker and spend
less time in the queue, however, at the current question volume staffing levels of more
than five students would have limited benefit on Wq.
The results for L and Lq were consistent with these results. For one student, Lq
was 98.1% ofL; however, this dropped to 50.5% when three students were employed.
However, looking at the number of questions completed reveals an obvious plateau after
three students of approximately 268 (Table 6-21). This plateau occurred because the
simulated students finished nearly all of the questions entering the system within the given
month.
Changes in staffing levels were also evaluated for their effect on simulated delay
and utilization percentages. When less than three students were employed in the
simulation, dramatic increases in the percentage of service delays occurred. Using only
one simulated student, 79.8% (s=4.9%) of the questions were delayed past the needed
time, and with two students 42.8% (s=12.0%) of the questions were delayed (Table 6-21).
When three students were used, the percentage of delays was substantially lower at 17.0%
delayed (s=4.3%). However, the gains after three students leveled off considerably. Four
students did improve the delay percentage by almost 4%; however, there was not a
significant improvement beyond the addition of one student (i.e., five or more students).
The difference in delay percentage between four and five students was only 0.4%.

141
Furthermore, when staffed by three students, the average utilization percent (p)
was 80.9% (Table 6-21). When only one or two students was simulated, p rose to almost
100% in both cases. This indicated that the simulated center was understaffed when less
than three students worked. As evidenced by the increases in W, Wq, and delayed
percentage, one or two students simply cannot manage the current volume of work.
When four and five students were used, p dropped to 60.9% and 48.8%, respectively.
These results provide early evidence suggesting that four students may represent a
worthwhile improvement given the significant improvements in total service time and
delayed percentage. Also, the 60% to 62% utilization realized when four students were
used is probably in an acceptable range given the students other activities and
responsibilities not measured by the simulation. However, the use of five students at this
question volume is questionable since the addition of the fifth student does not seem to
offer significant improvements in W or the delayed percentage.
Effect of Changes in Service Rates
Under normal simulation conditions, the mean total service time (W) was
approximately 239 minutes (Table 6-22). The simulated system was evaluated for both
increases and decreases in research and approval times of 5%, 10%, 15%, and 20%.
Improvements of 5%, 10%, 15%, and 20% resulted in decreases in expected total service
times (W) of approximately 45 minutes, 72 minutes, 95 minutes, and 113 minutes.
Similarly, W increased as the times required to research and approve questions increased.
A 5% increase in research time resulted in an increase in W from approximately 239.1
minutes (s=97.6) to 282.5 minutes (s=128.4), and a 10% increase resulted in an increase in
W to 348.1 (s=169.7) minutes. In addition, a 15% increase in research and approval times
resulted in an increase in W to 423.0 minutes (s= 169.7), and a 20% increase resulted in an
increase in W to 507.45 minutes (s=250.53).

142
The behavior of L was consistent with the results for W described above. Under
normal conditions the expected number in the simulated system was 6.89 questions (Table
6-22). Increases of 5% and 10% in research and approval times resulted in increases in L
to approximately 8.2 (s=4.2) and 10.1 (s=5.6) questions, respectively. A 15% increase in
research and approval times increased L to about 12.4 (s=7.0), and a 20% increase in
research and approval times increased the Lq to about 14.9 (s=8.1).
As expected, when research and approval times increased significantly, the
percentage of service delays and the student utilization percentage also increased. A 10%
increase in research and approval times resulted in an occurrence of service delays 21.0%
(s=6.1%) of the time, while utilization increased to 87.3% (s=5.8%). An increase in
research and approval times of 15% increased the percentage of delays to 23.5% (s=7.1%)
and utilization to 89.9% (s=5.3%). A 20% increase in research and approval times
increased the percentage of delays to 26.7% (s=7.3%) and the utilization percentage to
92.4% (s=4.7%).
Decreases in the time to research and approve questions lead to improvements in
the percentage of service delays and decreases in the level of student utilization. A 10%
decrease in research and approval times improved the delay percentage from 17.0%
(s=4.3%) to 13.5% (s=3.1%) and utilization decreased from 80.9% (s=6.2%) to 73.7%
(s=5.8%) (Table 6-23). A 15% decrease in research and approval times improved the
delay percentage to 12.3% (s=2.8%) but only reduced the utilization to 70.2% (s=5.8%).
In addition, a 20% decrease in research and approval times improved the delay percentage
to 11.0% (s=2.3%) and reduced utilization .to 66.6% (s=5.6%).
These results implied that an overall 15% reduction in research times would
decrease the delay percentage by more than the addition of another student, and still make
more efficient use of the existing personnel. Thus, considering that adding one student
improved the simulated delay percentage from 17.0% to 13.0%, but reduced utilization to

143
60.9%, it appears that improving service rates offers a more efficient alternative when
possible.
R2: What combination of changes in staffing levels and service rates optimizes the
system for delays in service and total service time?
The service quality results presented earlier provided evidence indicating that both
perceived service time (H4a through H4b) and service delays (H6) were related to
evaluations of perceived service quality. Since perceived service time was necessarily
measured by subjective means, the relationships observed could not objectively be
quantified in a way useful for the purposes of simulation. In addition, perceived service
time was not significantly related to actual service time or service delays (H7a and H7b),
so the transformation of perceived service time into a useful variable was not possible.
Therefore, service delay was the only remaining statistically significant link between
evaluations of service quality and controllable aspects of the system studied.
Consequently, the expected percentage of service delays was used as the primary variable
to be optimized in the simulation model. Secondary consideration was given to levels of
performance resulting in efficient total service times (W).
Since the addition of students does not result in proportional increases direct costs
(i.e., hourly wages), cost was an inappropriate mechanism for optimization. However,
the co-directors described at least five negative consequences of adding additional
students. First, each additional student requires additional training time, support, and
supervision, and since new students are brought in and trained each month, this cost does
not typically diminish over time. Second, lack of space and computer equipment makes the
use of more than four students uncomfortable. Third, each student adds to the time
required to complete the group educational activities each week, such as journal club
presentations. Fourth, each student requires more individual education time in terms of
guidance and grading Fifth, increasing the number of students decreases the number of

144
questions that the students answer overall during their rotation, perhaps diminishing their
experience. Thus, the overriding attitude between the co-directors seemed to reflect a
desire to provide the best possible service without adding unnecessary personnel.
Therefore, student utilization percentage was chosen as the variable against which
delays and total service time were optimized. Utilization percentage, as applied from the
description in chapter one, is the percentage of the total time that the students are busy.
Therefore, one minus the utilization percentage (i.e., 1-U) is the idle time percentage.
There is an important tradeoff between idle time and decreases in service delays and total
service times. As McClain and Thomas (1985) describe, “[wjaiting times can be reduced
by increasing the relative service capacity. This may be accomplished by reducing the
arrival rate, or increasing the number of servers or their work pace (service rate). All
these actions will increase the average idle time of the servers” (p.562). Therefore, the
use of utilization percentage as the balancing variable provided a means of establishing
which changes had the largest effect on service delays while making the most efficient use
of existing personnel. This method of evaluating efficiency is also consistent with methods
presented by Thompson (1992) and Westgard and Berry (1986).
The optimal combination of improvements was evaluated using a ratio analysis
comparing the effect of simulated improvements in service capacity on delay percentage
and total service times versus changes in student utilization percentage. As described
above, potential improvements included the addition of one or two students (i.e., four or
five total) and/or 5%, 10%, 15%, or 20% improvements in research and approval times.
Each potential combination of improvements was compared to normal operating
conditions (i.e., three students and empirically observed service and arrival rates).
Two values measuring efficiency were calculated. First, a value measuring the
efficiency of the simulated expected delay percentage versus the simulated expected
utilization percentage was calculated by dividing the utilization percentage by the delay
percentage (i.e., U/D). Since the goal was to have lower percentages of delays per

145
percentage of utilization, larger values indicated more efficient use of personnel. Second,
a value measuring the efficiency of the simulated expected total service time versus the
simulated expected utilization percentage was calculated by dividing the utilization
percentage by the expected total service time in minutes times 1000 (i.e., U/W x 1000).
Similarly, larger values for this measure indicated greater efficiency since the goal was to
achieve lower total service times without substantially decreasing utilization.
Table 6-24 reports the results of this analysis. A 20% decrease in service rates
while maintaining a staffing level of three students was the most efficient means of
reducing delays in service. This level of improvement resulted in efficiency ratio value
(U/D) equal to 6.04 (U=66.55%, D=11.03%), indicating that each percentage point in
delay is equal to approximately 6 percentage points in utilization. Even though the
addition of two students (i.e., five students) tends to minimize the percentage of delays, it
does not produce optimum results since substantial decreases in utilization (i.e., increases
in idle time) are also realized. The addition of one student resulted in an overall efficiency
ratio of 4.69. Table 6-24 indicates that while the addition of a student does significantly
decrease the percentage of delays, the system would actually be less efficient since the
overall efficiency ratio under normal conditions was 4.77.
Although the lowest delay percentages were observed when additional students
were added it does not appear that the current question volume necessitates additional
students. This point is illustrated by Figure 6-8, which shows that five students always
produces fewer delays at a given service rate. However, as service rates improved, the
advantage of adding additional students decreased. A 20% decrease in service rates
produced nearly the same benefit using three students (11.03%) as it would using four
students (9.79%). Furthermore, a 20% decrease in service rates under four and five
students produced similar decreases in delays (i.e., difference of only 0.42%), however,
the utilization percentage was much lower with five students (i.e., 39.99% with five
students, 50.01% with four students). The results indicate that improving the service rate

146
was generally a more important factor in reducing the delay percentage than additional
staffing.
Interestingly, however, the most efficient service capacity in terms of total service
times did require an additional student (i.e., four students). A 20% decrease in service
rates while maintaining a staffing level of four students produced the most effective means
of reducing the total service times. This level of improvement resulted in an efficiency
ratio (UAV) equal to 5.71 (U=50.01%, W=87.6 minutes), indicating that each minute of
service time was equal to approximately 0.571 percentage points in utilization. Again,
although the lowest total service times were realized when five students were simulated,
these benefits are offset by decreases in utilization. Furthermore, in terms of total service
time the “normal” service capacity turned out to be the least efficient of the levels
examined. These results indicated that while service rates are important in efficiently
managing the total service time, the staffing level was at least equally important.
Overall, delays in service tended to be more sensitive in with regard to changes in
the service rates than total service times. By comparing Figures 6-8 and 6-9, it is apparent
that a staffing level of three students was fairly sensitive to changes in the research and
approval times with respect to total service time. However, staffing levels of four and five
students are not sensitive to changes in the service rate.
Therefore, three students combined with reductions in research and approval times
of at least 10% achieved the optimum service capacity when considering service delays.
However, if total service time were to be considered more important than reducing the
delay percentage then the optimal solution would be to staff the DIPRC with four students
and reduce research and approval times by at least 10%.
R3: How sensitive is the optimum solution to random variation in the arrival rate?
Since queuing systems are often greatly affected by random variation in the arrival
rate (McClain and Thomas, 1985) the optimization methods used above were recalculated

147
under conditions of varying arrival rates. The simulation used a modifier to increase and
decrease the interarrival rates in 5% increments. Increases and decreases were made in the
arrival rates until the threshold at which both the efficiency ratios (U/D and U/W) shifted
from their optimal solutions presented above. As described above, at the current call
volume, the most effective point for managing delays occurred with three students with an
improvement in research and approval times of 20%. Furthermore, the most effective
point for managing total service time occurs with four students with an improvement in
total service times of 20%.
Table 6-25 presents the results of the sensitivity analysis. The optimum solutions
tend to be fairly stable. The threshold for increases in interarrivals occurred at 25%,
essentially indicating that if the current volume were to increase somewhere between 20%
to 25%, then an additional student would be warranted. The threshold for decreases in
interarrivals was slightly more sensitive. A decrease in interarrival times of 15% to 20%
was the threshold for the change in the efficiency ratio for managing delays in service
(U/D). A decrease in interarrival times of 10% to 15% was the threshold for the change in
the efficiency ratio for managing total service time (U/W).

148
Table 6-21. Descriptives for Queue Statistics
by Number of Students (N=60)
Number of Students
Mean
Std.
Deviation
95% C.I. for Mean
Lower Upper
Bound Bound
Total Service Time (IV)
1 Student
3521.9
331.5
3436.3
3607.6
2 Students
1385.0
437.4
1272.0
1498.0
3 Students
239.1
97.6
213.9
264.4
4 Students
122.0
17.7
117.4
126.6
5 Students
99.6
9.3
97.2
102.0
Wait in Queue (Wq)
1 Student
3477.6
339.5
3389.9
3565.3
2 Students
1250.4
444.3
1135.6
1365.2
3 Students
121.1
86.1
98.9
143.4
4 Students
24.5
11.5
21.6
27.5
5 Students
7.7
4.0
6.7
8.7
Number in System (L)
1 Student
108.2
13.6
104.7
111.8
2 Students
41.3
14.9
37.4
45.2
3 Students
6.9
3.2
6.1
7.7
4 Students
3.5
0.6
3.3
3.6
5 Students
2.8
0.3
2.7
2.9
Queue Length (Lq)
1 Student
106.2
13.8
102.6
109.8
2 Students
36.9
14.8
33.0
40.7
3 Students
3.5
2.7
2.8
4.2
4 Students
0.7
0.4
0.6
0.8
5 Students
0.2
0.1
0.2
0.3
Number Completed
1 Student
117.1
10.0
114.5
120.0
2 Students
220.2
14.8
216.3
224.0
3 Students
267.1
14.9
263.3
271.0
4 Students
268.1
15.6
264.1
272.1
5 Students
268.2
15.3
264.3
272.2
Utilization Percentage (p)
1 Student
1.000
0.000
1.000
1.00
2 Students
0.994
0.001
0.991
1.00
3 Students
0.809
0.006
0.793
0.825
4 Students
0.609
0.005
0.596
0.622
5 Students
0.488
0.004
0.477
0.499
Delay Percentage
1 Student
0.798
0.049
0.785
0.810
2 Students
0.428
0.120
0.397
0.459
3 Students
0.170
0.043
0.158
0.181
4 Students
0.130
0.025
0.123
0.136
5 Students
0.122
0.022
0.116
0.128

149
Table 6-22. Descriptive Statistics for the Total Service Time, Time in Queue,
Number in System, and Queue Length by Percentage Change in Research and
% Change in Research Time
Mean
Std.
Deviation
95% C.I.
Lower
Bound
for Mean
Upper
Bound
Total Service Time (W)
20% Decrease
125.9
24.9
119.5
132.4
15% Decrease
144.0
35.6
134.7
153.2
10% Decrease
167.9
46.6
155.8
179.9
5% Decrease
194.4
63.0
178.1
210.6
Normal
239.1
97.6
213.9
264.4
5% Increase
282.5
128.4
249.4
315.7
10% Increase
348.1
169.7
304.3
391.9
15% Increase
423.0
216.2
367.1
478.8
20% Increase
507.5
250.5
442.7
572.2
Time in Queue (Wq)
20% Decrease
39.4
18.9
34.5
44.3
15% Decrease
51.6
28.1
44.4
58.9
10% Decrease
67.3
36.7
57.8
76.8
5% Decrease
86.3
51.9
72.8
99.7
Normal
121.1
86.1
98.9
143.4
5% Increase
157.0
116.0
127.1
187.0
10% Increase
211.7
156.4
171.3
252.1
15% Increase
279.6
203.6
227.0
332.1
20% Increase
353.4
237.0
292.2
414.6
Number in System (L)
20% Decrease
3.6
0.9
3.4
3.8
15% Decrease
4.1
1.2
3.8
4.4
10% Decrease
4.8
1.5
4.4
5.2
5% Decrease
5.6
2.1
5.0
6.1
Normal
6.9
3.2
6.1
7.7
5% Increase
8.26
4.2
7.1
9.2
10% Increase
10.1
5.6
8.7
11.6
15%) Increase
12.4
7.0
10.6
14.2
20% Increase
14.9
8.4
12.8
17.0
Queue Length (Lq)
20% Decrease
1.1
0.6
1.0
1.3
15% Decrease
1.5
0.9
1.2
1.7
10% Decrease
1.9
1.1
1.6
2.2
5% Decrease
2.5
1.6
2.1
2.9
Normal
3.5
2.7
2.8
4.2
5% Increase
4.5
3.6
3.6
5.5
10% Increase
6.1
5.0
4.9
7.4
15% Increase
8.1
6.4
6.5
9.8
20% Increase
10.3
7.5
8.3
12.2

150
Table 6-23. Descriptive Statistics for Number Completed, Utilization Percentage,
and Delay Percentage by Percent Change in Research and Approval Time (N=60)
95% C.I. for Mean
% Change in Service Time
Mean
Std.
Deviation
Lower
Bound
Upper
Bound
Number Completed
20% Decrease
268.3
15.7
264.2
272.3
15% Decrease
268.0
15.5
264.0
271.9
10% Decrease
267.1
15.5
263.1
271.1
5% Decrease
267.4
15.5
263.3
271.4
Normal
267.1
14.9
263.3
271.0
5% Increase
266.3
14.6
262.5
270.1
10% Increase
264.9
14.0
261.3
268.6
15% Increase
262.8
13.5
259.4
266.3
20% Increase
259.6
13.3
256.2
263.1
Utilization Percentage (p)
20%> Decrease
0.666
0.056
0.651
0.680
15% Decrease
0.702
0.058
0.687
0.717
10% Decrease
0.737
0.061
0.722
0.753
5% Decrease
0.775
0.062
0.758
0.791
Normal
0.809
0.062
0.793
0.825
5% Increase
0.842
0.062
0.826
0.858
10% Increase
0.874
0.058
0.858
0.889
15% Increase
0.899
0.053
0.885
0.913
20% Increase
0.924
0.047
0.912
0.937
Delay Percentage
20% Decrease
0.1103
0.023
0.104
0.116
15%> Decrease
0.1226
0.028
0.115
0.130
10% Decrease
0.1352
0.031
0.127
0.143
5% Decrease
0.1504
0.036
0.141
0.160
Normal
0.1695
0.043
0.158
0.181
5% Increase
0.1866
0.050
0.174
0.200
10% Increase
0.2101
0.061
0.194
0.226
15% Increase
0.2352
0.071
0.217
0.254
20% Increase
0.2667
0.073
0.248
0.286

151
Table 6-24. Effectiveness of Service Capacity Improvements Under
Normal Arrival Rates (n=60)
Level1
Expected
Total Service
Time (W)*
Expected
% Delayed
(D)
Expected
% Utilization
(U)
U/D
U/W
(xlOOO)
3/Normal
239.1
16.95%
80.90%
4.77
3.38
3/5%
194.4
15.04%
77.46%
5.15
3.99
3/10%
167.9
13.52%
73.72%
5.45
4.39
3/15%
143.9
12.26%
70.16%
5.72
4.87
3/20%
125.9
11.03%
66.55%
6.04
5.29
4/Normal
122.0
12.98%
60.89%
4.69
4.99
4/5%
112.5
12.19%
58.19%
4.78
5.17
4/10%
103.3
11.21%
55.46%
4.95
5.37
4/15%
95.5
10.51%
52.74%
5.02
5.52
4/20%
87.6
9.79%
50.01%
5.11
5.71
5/Normal
99.6
12.18%
48.77%
4.00
4.90
5/5%
94.2
11.19%
46.58%
4.16
4.94
5/10%
88.2
10.64%
44.38%
4.17
5.03
5/15%
83.0
9.98%
42.19%
4.23
5.09
5/20%
77.4
9.37%
39.99%
All
5.17
* Number of Students/Percent Decrease in Research and Approval Time
* In Minutes

152
SERVERS
I
~ 3
I
IF 4
I
5
Service Rate Modifier
Figure 6-8. 95% Confidence Intervals for Delay Percentage by Service Rate
Modifier and Number of Servers
SERVERS
I
a 3
I
• 4
I
â–  5
Service Rate Modifier
Figure 6-9. 95% Confidence Intervals for Total Service Time (in Minutes) by
Service Rate Modifier and Number of Servers

153
Table 6-25. Sensitivity of Optimal Solution to Changes in the Arrival Rate (n=60)
Interarrival
Modification
Levcf
Expected
Total Service
Time (W)*
Expected
% Delayed
(D)
Expected
Utilization
(U)
U/W
U/D (xlOOO)
25% Increase
2/20% Decrease
1664.9
46.79%
99.94%
2.14
0.60
3/Normal
861.9
30. 76%
98.10%
3.19
1.14
3/20% Decrease
270.2
14.93%
88.16%
5.91
3.26
4/20% Decrease
109.7
10.39%
66.60%
6.41
6.07
5/20% Decrease
85.0
9.57%
53.27%
5.56
6.26
20% Increase
2/20% Decrease
1437.3
39.31%
99.43%
2.53
0.69
3/Normal
610.0
26.11%
95.34%
3.65
1.56
3/20% Decrease
204.5
13.66%
82.24%
6.02
4.02
4/20% Decrease
101.3
10.31%
61.86%
6.00
6.11
5/20% Decrease
82.7
9.52%
49.51%
5.20
5.99
15% Increase
2/20%> Decrease
1180.0
35.05%
99.25%
2.83
0.84
3/Normal
444.4
21.48%
92.70%
4.31
2.09
3/20% Decrease
163.5
12.05%
77.60%
6.44
4.75
4/20%> Decrease
95.9
10.07%
58.25%
5.79
6.07
5/20% Decrease
80.4
9.52%
46.59%
4.90
5.80
10% Increase
2/20%> Decrease
939.3
29.18%
98.74%
3.38
1.05
3/Normal
347.1
19.65%
89.17%
4.54
2.57
3/20% Decrease
150.3
11.84%
73.92%
6.24
4.92
4/20%o Decrease
93.8
9.74%
55.52%
5.70
5.92
5/20% Decrease
80.4
9.45%
44.44%
4.70
5.53
5% Increase
2/20%¡ Decrease
756.3
26.29%
96.81%
3.68
1.28
3/Normal
281.7
18.32%
84.94%
4.64
3.02
3/20% Decrease
134.4
11.74%
70.38%
6.00
5.24
4/20%> Decrease
90.2
10.07%
52.89%
5.25
5.86
5/20% Decrease
78.5
9.66%
42.34%
4.38
5.39
5% Decrease
2/20%> Decrease
492.5
18.92%
92.99%
4.91
1.89
3/Normal
205.0
15.64%
77.76%
4.97
3.79
3/20% Decrease
115.9
10.70%
63.67%
5.95
5.49
4/20%> Decrease
85.0
9.48%
47.76%
5.04
5.62
5/20% Decrease
76.7
9.22%
38.20%
4.14
4.98
10% Decrease
2/20%o Decrease
417.9
17.42%
90.37%
5.19
2.16
3/Normal
192.6
15.31%
74.78%
4.88
3.88
3/20% Decrease
110.5
10.62%
61.11%
5.76
5.53
4/20%o Decrease
83.2
9.65%
45.75%
4.74
5.50
5/20% Decrease
75.9
9.39%
36.59%
3.90
4.82
15% Decrease
2/20% Decrease
327.2
16.26%
86.96%
5.35
2.66
3/Normal
170.6
14.64%
71.01%
4.85
4.16
3/20% Decrease
105.7
10.48%
58.17%
5.55
5.50
4/20%o Decrease
81.7
9.57%
43.59%
4.55
5.34
5/20% Decrease
75.4
9.34%
34.86%
3.73
4.62
20% Decrease
2/20%o Decrease
264.1
14.10%
83.72%
5.94
3.17
3/Normal
152.1
13.98%
68.60%
4.91
4.51
3/20%> Decrease
98.9
10.33%
56.13%
5.44
5.67
4/20%> Decrease
79.8
9.59%
42.06%
4.38
5.27
5/20% Decrease
74.6
9.46%
33.64%
3.56
4.51
Number of Students/Percent Decrease in Research and Approval Time * In Minutes

CHAPTER 7
DISCUSSION AND CONCLUSIONS
Overview
This study had three primary objectives. First, to develop a computer simulation
for a drug information service and validate the model against the existing system. Second,
to investigate the associations among actual service time, service delays, and evaluations
of perceived service quality in a drug information service setting. Third, to recommend
system improvements based on the simulation model, in particular those improvements
that maintain quality while reducing delays and response times. To achieve these
objectives, the data analysis was broken up into four related parts. The first and second
parts were presented in chapter five as preliminary results, which discussed the results of
the analysis of the historical data and personal interviews. The third and fourth parts were
directly related to the hypotheses and specific research questions posed in chapter three,
and were presented in chapter six as main results.
This chapter begins by summarizing and discussing the results from chapter six as
they relate to the hypothesis tests and specific research questions. Where applicable, these
results are compared with the existing literature. Next, the limitations of the study are
described. This chapter concludes with a presentation of the study conclusions and
recommendations for future research.
154

155
Summary and Discussion of Results
PSO. OSO, and Behavioral Intention
The first three study hypotheses (HI, H2, and H3) tested the relationships among
PSQ, OSQ, and behavioral intention. A 19-item scale measured perceived service quality,
where the items were summed to produce a PSQ score. OSQ was a single item measure
asking callers to rank the overall service on a scale from “Excellent” to “Unacceptable”.
Behavioral intention was a two-item measure reflecting the callers’ perceptions regarding
their intention to use the service again and recommend the service to a colleague.
This research confirmed the anticipated relationships among these variables. The
strongest correlation between these variables was observed between PSQ and OSQ (r =
0.633). However, significant relationships were also observed between PSQ and
behavioral intention (r = 0.449 and 0.482) and between OSQ and behavioral intention (r =
0.471 and 0.612). As discussed in chapter six, these relationships were consistent with
similar correlations reported in Cronin and Taylor (1992). In addition, Headley and Miller
(1993) reported strong, significant relationships between perceived service quality and
overall quality with comparable degrees of explained variance. The direction of this
relationship was also consistent with reports from Boulding et al., (1993), McAlexander et
al. (1994), Parasuraman (1991), and Zeithaml et al. (1996). This high degree of
convergence supports the construct validity of these measures.
Although the correlation between PSQ and OSQ was relatively high, the reality of
this statistic indicates that PSQ only explains approximately 36% of the variance in OSQ.
This seemingly low predictive ability might be partially explained if we consider that PSQ
(as derived from SERVPERF) is largely a measure of functional quality (i .e., process
related quality) (Babakus and Mangold 1992; Gronroos, 1990). In addition to functional
quality, various researchers have suggested that perceptions of overall quality may be

156
influenced by technical or outcome quality, the image of the service, previous satisfaction
experiences, and consumers’ personal and situation factors (Bolton and Drew, 1994;
Gronroos 1990, Steenkamp 1990, Zeithaml, 1988). It is clear that more research is
needed in order to create a multi-dimensional measurement tool that predicts overall
perceptions of quality better than SERVQUAL or SERVPERF. Furthermore, given the
many types of services and their inherent idiosyncrasies, it may not be possible to create a
singularly valid instrument for evaluating perceived service quality.
Nevertheless, the service quality literature contends that there is a strong
relationship between perceived service quality and behavioral intention. The results of this
study compliment these assertions. Thus, it appears that the more positive that callers feel
about the service quality of the DIPRC, the more likely they are to have favorable
intentions of calling the service again and recommending the service to a colleague. It is
interesting to note, however, that in both this study and the study conducted by Cronin
and Taylor (1992), the relationships between measures of overall quality and behavioral
intention were stronger than the relationships between PSQ and behavioral intention. This
suggests, perhaps, that perceptions of overall quality (i. e., overall impressions) are more
important in driving future behavior than evaluations of quality based on specific aspects
of the service. Again, however, further research is necessary to validate this assumption.
Actual Service Time. Service Delays. Perceived Service Time and PSQ
Hypotheses H4, H5, H6, and H7 were developed to test the relationships among
actual service time, service delays, perceived service time, and PSQ. Actual service time
referred to the actual amount of time (in minutes) required to respond to a question, and
was measured by subtracting the “end time” for receiving a call from the “start time” for
returning a answer. Service delay was a dichotomous variable referring to whether or not
the response time was longer than the time needed (i.e., “stat”, “today”, “date”, and “no

157
rush”). Perceived service time was measured by four items in the questionnaire (Q24
“Acceptable Time”, Q25 “ No Longer Useful”, Q26 “Quicker Response, and Q27
“Expected Time”).
One of the relationships posed by the literature was that actual waiting time was
related to service evaluations (Buxton and Gatland, 1995; Hui and Tse, 1996; Davis 1991,
Davis and Vollmann, 1990; Katz, Larson, and Larson, 1991; Larson, 1987; Tom and
Lucey, 1995). This study did not find any statistically or practically significant relationship
between actual service time and perceived service quality. Even if we were to assume
statistical significance based on a borderline p-value (p=0.07), the regression equation
suggested that, at best, PSQ was affected by actual service time only when one full day
passed without a response. Practically significant shifts in PSQ did not occur until the
equivalent of four or five days had lapsed in actual service time. Although surprising at
first glance given the literature support, this lack of relationship makes sense in this setting
if we consider the eight principles of waiting as proposed by David Maister (1985):
Proposition 1:
Proposition 2:
Proposition 3:
Proposition 4:
Proposition 5:
Proposition 6:
Proposition 7:
Proposition 8:
Unoccupied Time Feels Longer than Occupied Time
Pre-process Waits Feel Longer than In-Process Waits
Anxiety Makes Waits Seem Longer
Uncertain Waits Are Longer than Known, Finite Waits
Unexplained Waits Are Longer than Explained Waits
Unfair Waits Are Longer than Equitable Waits
The More Valuable the Service, the Longer the Customer Will Wait
Solo Waits Feel Longer than Group Waits
As determined in chapter 5, the pre-process time required to take a call was
approximately four minutes and only about 12% of the calls are “stat” questions needed
within fifteen minutes (Tables 5-10 and 5-15). Therefore, since the pre-process waits are
short and there is usually no particular immediate rush to complete questions, the effects
of propositions one and two above are limited. Furthermore, callers do not actually wait
in the DIPRC for their questions to be answered, therefore they do not wait in an “actual”

158
queue. Since there is no time spent waiting on hold, solo waits (proposition eight) and
unfair waits (proposition six) are not really applicable to this setting. In addition, while the
DIPRC is working on the question, the caller is probably often occupied with other duties
and tasks, which limits the effect of proposition one. Finally, the callers tend to perceive
the service provided by the DIPRC as highly valuable (proposition seven). This is
evidenced by generally high PSQ and OSQ scores, as well as the distribution of responses
to questions 29 and 30 in the questionnaire which asked callers to rate the how useful and
essential the service was to them. Approximately 97% of the respondents agreed to some
extent that the service was useful to them, and about 81% of the respondents agreed to
some extent that the service was essential. Therefore, based on Maister’s propositions,
there are clear reasons not to expect a significant relationship between PSQ and actual
service time.
Keeping with Maister’s propositions, it is interesting to note that service delays did
have a significant effect on perceived service quality, which is consistent with proposition
five. Furthermore, this result was consistent with reports from Taylor (1994a), Taylor and
Claxton (1994), and Dube’-Rioux et al. (1989). The results of this study suggest that
callers who experience delays in service have lower perceptions of service quality than
callers who do not experience delays in service. It should be noted, however, that the
relationship between delays and PSQ was not very strong (r = 0.235). The strength of this
relationship, however, is consistent with effect sizes reported by Taylor (1994a), who
suggests that there are variables (such as anger, uncertainty, and perceived punctuality)
that mediate the relationship between delays and service evaluations. In addition,
Dube’Rioux et al. (1989) suggested that perceived necessity may also mediate this
relationship.
Among the three “time related” measures (actual service time, service delays, and
perceived service time) perceived service time had the strongest relationship with PSQ,
suggesting that callers’ perceived service times are more important in determining their

159
evaluation of PSQ (and for that matter OSQ) than actual service time and service delays.
This contention is supported in the literature by Taylor (1994a), Katz, Larson, and Larson
(1991) and Hornik (1982, 1984). Thus, while this study was originally focused on
operational relationships between actual service times and service delays, it may be that
more emphasis should be placed on the management of perceived service times.
From the literature we can derive seven recommendations to managers of drug
information services attempting to improve perceived service times. First, when notifying
callers of the expected wait duration, it is better to overestimate the wait than to
underestimate the wait; however, do not provide estimations that are longer than the
consumer is willing to wait otherwise they may balk from the system. Therefore, it is
important to know what callers consider an acceptable wait (Taylor 1994a; Katz, Larson
and Larson, 1991; Hornik, 1994). Second, shorten pre-process waits (i.e., the amount of
time required to take information) as much as possible (Katz, Larson, and Larson, 1991;
Dube’-Rioux et al. 1989). Third, distract attention from the duration of the wait (Katz,
Larson, and Larson, 1991). This can be done for questions that are taking an inordinate
amount of time to research by updating the caller with the current progress.
Fourth, when service is delayed past the estimated time, it is important to
apologize for the failure, but it is equally important to sound sincere since insincere
apologies can actually lower satisfaction (Clemmer and Schnieder, 1993). Fifth, when
service is delayed past the estimated time, it is often important to explain to the caller why
the delay occurred or they might infer their own reasons for the delay (Taylor, 1994a).
Sixth, assess caller attitudes, evaluate their time pressures, and handle first those
callers with the greatest perceived need (Katz, Larson, and Larson, 1991; Dube’-Rioux et
al. 1989). Seventh, provide callers information regarding peak demand times, so that they
know when to expect longer waits (Katz, Larson, and Larson, 1991; Clemmer and
Scheider, 1993).

160
Service System Simulation
The D1PRC at the University of Florida was simulated using the GPSS/H
simulation program. The simulation was verified using three methods: (1) traces of the
simulation program code using the interactive debugger, (2) tests of logic relationships
(i.e., model behavior), and (3) a limited form of the simulation was compared to analytical
results based on an M/M/3 queue. In addition, the simulation was validated using three
methods: (1) face validity was established by simulation walk-throughs with DIPRC co¬
directors, (2) expected simulation behavior was evaluated using extreme condition tests,
and (3) the results of the simulation were compared to data collected from the actual
system. The simulation passed all steps of the verification and validation process. It was
therefore determined that the simulation was an acceptable decision making tool for use in
the DIRPC.
The simulation was used to evaluate three specific research questions. The first
question asked how changes in staffing levels and service rates affect total service times
and delays in service. These changes were compared to the “normal” level of operation,
(i.e., simulation runs using observed arrival and service rates and a staffing level of three
students). In terms of staffing levels, the simulation results suggest that a staffing level of
less than three students is simply inadequate to handle the current question volume in the
DIPRC. However, there seemed to be significant improvements in simulated total service
times and service delays, when the staffing level was increased from three to four students.
Conversely, the difference in service delays and total service times between four and five
students were not dramatic, indicating that three or four students is probably an adequate
staffing level for the DIPRC.
Changes in service rates were conducted by manipulating the research and
approval times programmed into the simulation code based on the observed data. The
simulation results indicated that, as expected, increases in research and approval times

161
increased both total service times and service delays while decreases in research and
approval times decreased both total service times and service delays. However, when
comparing reductions in total service time and delay percentage to reductions in percent
utilization, it appeared that service rate improvements were more efficient than adding
servers for reducing total service times and service delays.
The second question asked what combination of changes in staffing levels and
service rates optimizes the system for delays in service and total service time. Largely, the
results of the effectiveness ratios mirrored what was anticipated from the results of the
first research question. The results of the simulation indicated that when trying to reduce
the percentage of service delays, reductions in service rates were more important than the
addition of servers This result was consistent with research from Carruthers (1970), Chin
and Sprecher (1990), Kumar and Kapur (1989), and Ozeki and Ikeuchi (1992).
Interestingly, however, for reductions in total service time, appropriate staffing levels were
at least as important as reductions in the time to research and approve questions, which
was consistent with research done by Lamy et al. (1970). For reducing service delays, the
optimal staffing level was three students, and for reducing total service times, the optimal
staffing level was four students. However, in both cases improving service rates always
improved efficiency.
The third question asked how sensitive the optimal solution was to changes in the
arrival rates. Since fluctuations in the rate of arrivals can dramatically affect many queuing
systems (McClain and Thomas, 1985), it was important to identify thresholds at which the
optimal service capacity determined by question two above were affected. The optimum
solutions tended to be relatively stable. Call volume would have to increase by 20% to
25% or decrease by 10% to 20% for the solution to change. In practical terms, these
results indicated that an increase in volume of approximately 53 to 67 calls per month (i.e.,
average of 266 calls per month * 0.25 = 66.5), would reflect a need to add an additional

162
student. A decrease in call volume of approximately 27 to 53 calls per month (i.e., 266 *
0.20 = 53.2), would allow for a decrease in the number of students staffing the center.
In a debriefing session with the co-directors and the drug information resident,
four potential methods for reducing service times were suggested. The first method
involved the use of an online database to store caller contact information along with
answers to questions posed. This improvement was deemed to have the most potential for
improving the response time of the DIPRC. Furthermore, this database had already been
slated for implementation and was beginning the initial stages of the design process at the
time of this project. This database is expected to reduce the amount of time required take
calls since repeat callers will presumably already have their information entered into the
database. Also, since many questions tend to be repeated over the course of time,
previous responses can be retrieved and reused, which could substantially decrease
research times for many questions. The second method involved the encouragement of
students to use the pager system to contact co-directors to approve responses when they
are out of the office. Although there is usually someone present in the DIPRC who can
approve responses, occasions arise when all persons eligible to approve answers are out.
More effective use of the pager system will prevent high priority questions (eg., “stat”
and “today”) from being delayed because the student is waiting for approval. The third
method proposed during this debriefing would be to develop a hierarchy of approvals, so
that for certain types of questions (such as phone numbers for drug companies and drug
identification questions) students may give a response without prior approval. The fourth
method suggested concerned the addition of several computers that would allow students
more immediate access to word-processing and on-line information resources without
having to wait for other students to finish. It was also noted that once the online database
has been implemented, it will be crucial for every student to have access to a computer.
The overriding question that remains is the issue of whether or not simulation is
useful as a tool for understanding complex service systems, such as drug information and

163
other pharmacy systems. The answer to this question seems to be a qualified "yes". Once
the simulation for this study was constructed, verified, and validated, the simulation
provided a relatively easy means of exploring the effect of service capacity changes on
various queue statistics (i.e., W, Wq, L, Lq, p, and delays) Clearly this attempt at
simulating a drug information service has demonstrated the utility of simulation, and with
the current health care environment pushing towards cost containment, resource
efficiency, and total quality management, there seems to be a need for tools that can help
decision makers evaluate and design (or redesign) complex service systems. It must be
considered, however, that months of background research and development went into the
construction, verification, and validation of the simulation used in this project. As Schriber
(1991) states, “A simulation should be started well before the results are needed In
practice, unfortunately, the results of simulation are usually needed ‘yesterday’” (p.7).
Unfortunately, the length of time necessary to develop good simulation models may not
always be appropriate; therefore, simulation may not always be the most appropriate tool
for systems analysis.
Conclusions
Four conclusions were derived from the results of this study. First, based on
results from this research and research done by others, there appears to be a clear
relationship between evaluations of perceived service quality and evaluations of overall
service quality, however, scales based on derivatives of the SERVQUAL scale (such as
SERVPERF and the PSQ instrument used in this study) seem limited in their ability to
capture perceptions of quality beyond functional quality.
Second, perceptions of drug information service quality are related to the callers’
behavioral intentions, suggesting that when callers feel that the service delivers high

164
quality service, they will not only use the service again, but will also recommend the
service to a colleague when given the opportunity.
Third, drug information service providers should be concerned with both delays in
service and perceived wait times, since both appear to have an affect on PSQ. Actual wait
times do not seem to be related to PSQ. Therefore, managers should use operations
management approaches to reduce the frequency of service delays (i.e., increase the
service time and/or add servers) and they should use perceptions management approaches
(eg., expected wait information, apologies, etc.) to improve the experience of waiting.
Fourth, based on the simulation results, it appears that for this setting, reducing the
service rates is a more efficient method of improving the percentage of delays than
increasing the number of servers. However, if total service time were to be considered an
important indicator, then increases in staffing become at least as important as reductions in
research and approval times.
Limitations of Study
There were four study limitations that deserve consideration. First, the design of
the questionnaire portion of this project was exploratory and non-experimental, which
makes the internal validity of the results difficult to determine. In addition, only one
measurement of perceived service quality, overall service quality, perceived service time,
and behavioral intention was made for each subject. Therefore, since unmeasured external
variables could have mediated or intervened in the actual relationships, the results reported
in this study could be spurious. However, the fact that the relationships observed were
consistent with those reported by Cronin and Taylor (1992) and other researchers helps to
confirm the direction and strength of the observed relationships. In addition, since the
research design was not experimental (or quasi-experimental), the existence and direction
of causality could not be assumed.

165
Second, because of time and cost constraints, the study data were not drawn from
a random sample of callers. Therefore, it is possible that some of the results may have
occurred due to sample idiosyncrasies. However, the characteristics of the concurrent
sample were similar to the characteristics of observed from the historical sample. Also,
there was no reason to suspect that the responses given on the questionnaires were
dependent on one another. Therefore, there was no reason to suspect that different results
would have been reached using a truly random sample.
Third, it was not possible to use this simulation to analytically solve for the
“true” optimum state (as is possible with other analytical tools used in operations research,
such as linear programming). Therefore, the optimum solution presented in this study is
only the best choice selected from the simulated experiments. If other variables of interest
were to be studied and/or different inputs were given to the model, then the results may
differ. Furthermore, since actual manipulations of the system were not conducted, it is not
possible to know for sure whether or not the results of the simulation are truly accurate
when actual changes are made to system inputs (e.g., staffing level, service rates, and
arrival rates)
Fourt h, the simulation program itself may not be generalizeable to other drug
information settings, given that other centers may have different work patterns, different
calling populations, and varying responsibilities. Consequently, the specific results (i.e.,
queue statistics) generated by the simulation model created in this project are not
necessarily generalizable to other drug information services. Also, since this study is the
first to use simulation to examine the effectiveness of drug information services, there are
no studies available to compare the simulated results.

166
Recommendations for Future Studies
Use of Simulation in Other Pharmacy Settings
One of the primary goals of this study was to determine if simulation could
effectively be used as a decision making tool in pharmacy systems. While this study has
made a significant step in that direction, there are still questions to be answered. One of
the initial reasons that the drug information service setting was selected was because of
the relative simplicity of this system versus other pharmacy systems. This study indicated
that simulation was a powerful decision making tool for analyzing service systems;
however, it takes considerable time and effort to adequately construct, verify, and validate
a simulation. Therefore, its usefulness in more complex systems is still at issue. In the
researcher’s opinion, there are at least three pharmacy settings where this simulation
methodology could be implemented next with immediate impact in the literature.
First, an interesting use of simulation would be to study the effects of workload on
medication error rates in a hospital or community pharmacy. This research might answer
several questions. What is the workload/staffing balance in order to minimize error rates?
How do the contributing factors of errors (e.g. workload, workflow, fatigue, and human
factors) interrelate? Can standards be set for workload limits?
Second, one of the most frequently mentioned barriers to the implementation of
pharmaceutical care in community pharmacies is lack of sufficient staff and/or time to
provide services. Simulation could help provide practitioners with realistic expectations as
to how many pharmacists/technicians are needed to provide pharmaceutical care services,
how/when patients could be scheduled, and how changes in workflow could improve
efficiency of the current staff.
Third, in previously published research by the researcher (Halberg et al., 1996)
involving the use of anesthetics in an ambulatory care setting there were differences

167
discovered among anesthetics in terms of both time and cost factors. However, two
limitations of this study were: (1) it did not account for labor costs of nursing staff and
anesthesiologist time because the difficulty of tracking this information; (2) small sample
sizes restricted the power of the statistical tests used to detect differences in anesthetics.
If the system of surgery could be simulated with some accuracy then more precise
pharmacoeconomic judgements could be made concerning the use of anesthetics. This
method could also extend itself eventually to other pharmacoeconomic choices where
sample sizes are low or when costs of collecting data are restrictive in terms of dollars or
time.
Assessment of the Service Quality of Drug Information Services
Another goal of this study was to establish the relationships between perceived
service quality and actual service time or delays in service. Unfortunately, the
relationships observed were weaker than expected. Interestingly, however, perceived
service time was related to perceived service quality with some significance. Although
strong relationships may not actually exist between perceived service quality and actual
service time or delays in service in the drug information service setting, there is still
considerable room for research in this area, especially in pharmacy systems (such as
community pharmacy) where lay persons rather than professionals are the primary
consumers.
First, it may be that the SERVQUAL and SERVPERF instruments developed by
Parasuraman et al. (1988) and Cronin and Taylor (1992) are not adequate for measuring
the perceived service quality for professional services, such as a drug information service.
The distinction between consumers and professionals with regard to perceived service
quality and the development of a more useful scale for use in assessing the perceived

168
service quality of professional services would fill a gap in the evolving service quality
literature.
Second, one of the theoretical distinctions between perceived service quality and
consumer satisfaction is that customer satisfaction is typically concerned with only a single
service encounter, and perceived service quality is reflective of all experiences and
impressions a consumer may hold It may be that consumer satisfaction/dissatisfaction
(CS/D) measures behave differently than perceived service quality measures with respect
to actual service time and service delays. Therefore, it may be more appropriate to relate
actual service time to some measure of customer satisfaction rather than perceived service
quality. Beyond CS/D, other frameworks such as perceived value may obtain more useful
results for combining aspects of operations research with perceptions management.
Third, beyond the work done by Hornik (1982 and 1984) and Katz, Larson, and
Larson (1992), it seems that very little research has been done to ascertain the
relationships among perceived service time and actual service times as it applies to service
settings. If this relationship can be more accurately described, then it may create
additional avenues for managers who desire to improve quality using operations
management techniques.

APPENDIX A
TEXT OF PRE-TEST COVER LETTER
June 6, 1997
Dear Colleague,
The Arkansas Poison and Drug Information Center (APDIC) at UAMS
College of Pharmacy is currently working toward improving the quality of
the services we provide. In order to achieve this goal, we have decided to
ask our most recent callers some specific questions regarding our service.
You can help us by participating in a brief survey concerning your recent
experience(s) with the APDIC. Your prompt response is very important to
us. You are one of only a small number of practitioners who are being
asked to give their opinions about our service, so it is critical that each
questionnaire is completed and returned.
Please answer all of the questions in this questionnaire (it should only take
between 5-10 minutes to fill out) When complete, refold the questionnaire
and tape it closed. A stamp has been placed on the questionnaire for your
convenience. You may be assured of complete confidentiality. The
questionnaire has an identification number for mailing purposes only. This
is so we may check your name off the mailing list when your questionnaire
is returned. Your name will never be placed on the questionnaire itself, nor
will your responses be linked to you personally during the analysis.
Thank you for your participation.
Sincerely,
Charles S. Campbell, P.D.
Director, Arkansas Poison and Drug Information Center
Daniel L. Halberg
Doctoral Candidate, University of Florida
169

APPENDIX B
PRE-TEST QUESTION AIRE - VERSION ONE

DIRECTIONS: Please answer the questions to the best of your ability by placing a
checkmark in ONE of the boxes next to each question. There are four sets of
questions: (1) some basic information about you; (2) your feelings about the
Arkansas Poison and Drug Information Center (APDIC) at the University of
Arkansas for Medical Sciences; (3) your perceptions regarding the amount of
time the APDIC took to fulfill your request; and, (4) your overall feelings regarding
the APDIC and your impressions about future behaviors regarding the APDIC.
PART I: The following two questions gather some information about you. This
information will be used in conjunction with the information given in the rest of
the questionnaire to assess how needs and perceptions differ among our callers.
(1) What is your profession?
â–¡ RPh/Pharm.D. Li Physician U Nurse/Nurse Practitioner â–¡ Other:
(2) How often do you use the APDIC?
â–¡ First time user â–¡ 3-5 times per year â–¡ 10-15 times per year
□ 1-2 times per year Ü 5-10 times per year _l more than 15 times per year
PART II: The following set of statements relate to your feelings about the
Arkansas Poison and Drug Information Center (APDIC) at the University of
Arkansas for Medical Sciences. For each statement, please check the box that
best describes the extent to which you believe the APDIC has that characteristic.
The range of selection varies from "Strongly Agree" to "Strongly Disagree";
however, you may check any of the boxes provided. If you feel that you cannot
answer a question, or that the question does not apply to you, you may check the
box labelled "Don't Know".
(3) The APDIC has the equipment and
information resources necessary to
i(& Jr « dp ojP ^ cjP’ (((((((
answer my questions
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(4)
When 1 call the APDIC, background noise
on their end interferes with my ability to
communicate over the telephone
j
J
U
â–¡
U
Ü
u
(5)
When 1 receive written materials from the
APDIC, they are clear and easy to read ..
3
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(6)
Employees of the APDIC speak in a
manner that is easy to understand
_l
_l
â–¡
Li
Ü
u
u
(7)
When the APDIC promises to do
something by a certain time, it does so .
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(OVER)
171

172
(8) When I have a problem, the APDIC is
sympathetic and reassuring
(9) The APDIC is dependable
Jp a ^ J*
**vVVV
f f f f f f f
JT
□ □ □ □ □ Ü J
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡
(10)The APDIC provides its services in the
time it promises
IJ IJ â–¡ â–¡ IJ Q â–¡
(11)The APDIC does not give me individual
attention
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡
(12) The APDIC does not tell me exactly when
services will be performed
â–¡
u
â–¡
â–¡
â–¡
â–¡
â–¡
(13) The APDIC keeps its records accurately .
â–¡
â–¡
â–¡
J
J
â–¡
â–¡
(14) I do not receive prompt service from
APDIC employees
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(15) Employees of the APDIC are not always
willing to help me
â–¡
u
IJ
â–¡
â–¡
J
â–¡
(16) Employees of the APDIC are too busy to
respond to caller requests promptly ....
â–¡
â–¡
â–¡
â–¡
IJ
IJ
J
(17) I can trust employees of the APDIC
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(18) Employees of the APDIC do not give me
personal attention
â–¡
â–¡
u
â–¡
â–¡
â–¡
(19) Employees of the APDIC are polite
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(20) I feel safe in my interactions with the
APDIC employees
U
â–¡
â–¡
u
â–¡
â–¡
â–¡
(21) Employees get adequate support from the
APDIC to do their jobs well
â–¡
â–¡
â–¡
u
IJ
â–¡
J
(22) Employees of the APDIC do not know
what my needs are
â–¡
â–¡
â–¡
â–¡
â–¡
IJ
u

â–¡ â–¡
(23)The APDIC does not have my best
interests at heart
â–¡ â–¡ U â–¡ â–¡
(24)The APDIC does not have operating
hours convenient to me
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡
PART III: The following questions regard your perceptions about the length of
time in which the service was rendered. Please think about the next four items in
terms of the last question you presented to the APDIC, and react to the
statements below using the scale provided. Again, you may check any of the
boxes on the scale to show how strong your feelings are.
(25)The amount of time that it took the APDIC
to respond to my most recent question
was acceptable
¿V .a® >A
/v & 4? ^ C? ^ c?
(((((((
â–¡ â–¡ â–¡ â–¡
â–¡ â–¡
(26)By the time I received a response from
the APDIC, the information was no longer
useful to me â–¡ â–¡ q q q â–¡ q
(27) I wish the APDIC could provide a quicker
response to my questions j j â–¡ ij â–¡ â–¡ â–¡
(28) The amount of time that it took the APDIC to respond to my most recent question
was
â–¡ MUCH SHORTER THAN I EXPECTED
â–¡ SHORTER THAN I EXPECTED
â–¡ EQUAL TO WHAT I EXPECTED
â–¡ LONGER THAN I EXPECTED
â–¡ MUCH LONGER THAN I EXPECTED
(OVER)

174
PART IV: The following seven statements relate to your overall feelings about the
APDIC. Please respond by checking the box which best reflects your own
perceptions.
(29) The overall quality of the services provided by the APDIC is best described as
â–¡ Excellent â–¡ Very Good â–¡Good â–¡Fair â–¡ Poor â–¡Unacceptable
(30) The responses I receive from the APDIC
are useful to me in my practice
& -is
y» c? ^ r r r r r
fs
¡y
* #
or if cf c&
â–¡ â–¡
r
â–¡ â–¡
(31) The responses I receive from the APDIC
are essential to me in my practice
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡
(32) It is important that the APDIC fax me the
supporting documents (e.g., recent
literature) for their answers to my
questions y y y y y y y
(33) The APDIC's answers to my questions are
used to improve patient outcomes
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡
(34) I intend to use this service in the future. .
â–¡ â–¡â–¡â–¡â–¡â–¡ â–¡
(35) I would recommend this service to a
colleague
â–¡ â–¡â–¡â–¡â–¡â–¡ U
(36) ADDITIONAL COMMENTS: Is there anything else that you would like to tell us about
your experience(s) with the APDIC? Also, any comments you wish to make
regarding how we could improve our service will be appreciated, either here or in a
separate letter.
Thank you very much for your help.

APPENDIX C
PRE-TEST QUESTIONAIRE - VERSION TWO

DIRECTIONS: Please answer the questions to the best of your ability by placing a
checkmark in ONE of the boxes next to each question. There are four sets of
questions: (1) some basic information about you; (2) your feelings about the
Arkansas Poison and Drug Information Center (APDIC) at the University of
Arkansas for Medical Sciences; (3) your perceptions regarding the amount of
time the APDIC took to fulfill your request; and, (4) your overall feelings regarding
the APDIC and your impressions about future behaviors regarding the APDIC.
PART I: The following two questions gather some information about you. This
information will be used in conjunction with the information given in the rest of
the questionnaire to assess how needs and perceptions differ among our callers.
(1) What is your profession?
â–¡ RPh/Pharm.D. â–¡Physician â–¡ Nurse/Nurse Practitioner â–¡Other:
(2) How often do you use the APDIC?
â–¡ First time user â–¡ 3-5 times per year â–¡ 10-15 times per year
â–¡ 1-2 times per year â–¡ 5-10 times per year U more than 15 times per year
PART II: The following set of statements relate to your feelings about the
Arkansas Poison and Drug Information Center (APDIC) at the University of
Arkansas for Medical Sciences. For each statement, please check the box that
best describes the extent to which you believe the APDIC has that characteristic.
The range of selection varies from "Strongly Agree" to "Strongly Disagree";
however, you may check any of the boxes provided. If you feel that you cannot
answer a question, or that the question does not apply to you, you may check the
box labelled "Don't Know".
(3)The APDIC has the equipment and
information resources necessary to
answer my questions
A
JF
,
-P X»'
r
â–¡
r
â–¡
r
â–¡
r r
â–¡ â–¡
J?' e <$*** A .a® oP
Jo dv 'V
<£• Jy ¿Or ° r
â–¡
r
u
(4)When I call the APDIC, background noise
on their end interferes with my ability to
communicate over the telephone
â–¡ â–¡ 'â–¡â–¡â–¡â–¡â–¡
â–¡
(5)When I receive written materials from the
APDIC, they are clear and easy to read .
â–¡ â–¡â–¡â–¡â–¡â–¡
â–¡ â–¡
(6) Employees of the APDIC speak in a
manner that is easy to understand .
(7) When the APDIC promises to do
something by a certain time, it does
â–¡ â–¡
¡J
Ü
U
â–¡
â–¡
â–¡
so .
U â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(OVER)
176

177
(8) When I have a problem, the APDIC is
sympathetic and reassuring
(9) The APDIC is dependable
(10) The APDIC provides its services in the
time it promises
(11) The APDIC does not give me individual
attention
(12) The APDIC does not tell me exactly when
services will be performed
O,
# T?» «jP f í f f f f f f
QQQQQQUQ
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
â–¡ u â–¡ â–¡ u â–¡ â–¡ u
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
(13) The APDIC keeps its records accurately
â–¡
â–¡
â–¡
â–¡
â–¡
IJ
J
j
(14) I do not receive prompt service from
APDIC employees
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(15) Employees of the APDIC are not always
willing to help me
â–¡
â–¡
U
u
â–¡
u
â–¡
â–¡
(16) Employees of the APDIC are too busy to
respond to caller requests promptly . . .
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
IJ
(17) I can trust employees of the APDIC ....
J
J
â–¡
â–¡
â–¡
J
â–¡
â–¡
(18) Employees of the APDIC do not give me
personal attention
â–¡
â–¡
J
[J
u
â–¡
â–¡
â–¡
(19) Employees of the APDIC are polite ....
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(20) I feel safe in my interactions with the
APDIC employees
â–¡
â–¡
u
â–¡
u
â–¡
â–¡
â–¡
(21) Employees get adequate support from
the APDIC to do their jobs well
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
J
(22) Employees of the APDIC do not know
what my needs are
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
J

178
(23) The APDIC does not have my best
interests at heart
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
(24) The APDIC does not have operating
hours convenient to me
â–¡ â–¡â–¡ â–¡â–¡â–¡â–¡â–¡
PART III: The following questions regard your perceptions about the length of
time in which the service was rendered. Please think about the next four items in
terms of the last question you presented to the APDIC, and react to the
statements below using the scale provided. Again, you may check any of the
boxes on the scale to show how strong your feelings are.
(25) The amount of time that it took the
APDIC to respond to my most recent
question was acceptable
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
(26) By the time I received a response from
the APDIC, the information was no longer
useful to me
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
(27) I wish the APDIC could provide a quicker
response to my questions
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
(28) The amount of time that it took the APDIC to respond to my most recent question
was
â–¡ MUCH SHORTER THAN I EXPECTED
U SHORTER THAN I EXPECTED
â–¡ EQUAL TO WHAT I EXPECTED
â–¡ LONGER THAN I EXPECTED
â–¡ MUCH LONGER THAN I EXPECTED
(OVER)

179
PART IV: The following seven statements relate to your overall feelings about the
APDIC. Please respond by checking the box which best reflects your own
perceptions.
(29)The overall quality of the services provided by the APDIC is best described as
â–¡ Excellent â–¡ Very Good â–¡ Good â–¡ Fair â–¡ Poor â–¡Unacceptable
(30) The responses I receive from the APDIC
are useful to me in my practice
(31) The responses I receive from the APDIC
are essential to me in my practice
(32) It is important that the APDIC fax me the
supporting documents (e.g., recent
literature) for their answers to my
questions
(33) The APDIC's answers to my questions
are used to improve patient outcomes ..
(34) I intend to use this service in the future.
(35) I would recommend this service to a
colleague
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡â–¡
â–¡
u
â–¡
â–¡
â–¡
â–¡
â–¡
-I
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
J
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
u
â–¡
(36)ADDITIONAL COMMENTS: Is there anything else that you would like to tell us about
your experience(s) with the APDIC? Also, any comments you wish to make
regarding how we could improve our service will be appreciated, either here or in a
separate letter.
Thank you very much for your help.

APPENDIX D
PRE-TEST FOLLOWUP POSTCARD
â–¡
(JAMS
University of Arkansas for Medical Sciences
COLLEGE OF PHARMACY
4301 West Markham St., Slot 522-
Little Rock, Arkansas 72205-7122
(Side One)
Dear Colleague:
About four weeks ago, a questionnaire seeking your opinions about the service quality of the Arkansas
Poison and Drug Information Center (APDIC) was mailed to you,
If you have already completed and returned the questionnaire to us please accept our sincere thanks.
If not, please do so today. Because it was sent to only a small sample of our recent callers it is
extremely important that yours also be included in the study if the results are to accurately represent
the feelings of the professionals we serve.
If by some chance you did not receive the questionnaire, or it got misplaced, please call me at (352)
392-9035 and I will get another one in the mail to you. Your contribution to the success of this study
will be greatly appreciated.
Sincerely,
Daniel L. Halberg
Doctoral Candidate
(Side Two)
180

APPENDIX E
RESPONSES TO PRETEST QUESTIONAIRE VERSION ONE
Version 1 - "Don't Know" Response Value Excluded
Response
Strongly
Agree
Agree
Somewhat
Agree
Neutral
Somewhat
Disagree
Disagree
Strongly
Disagree
No Response
Q-1
57
1
2
3
0
0
0
1
Q-2
6
1
7
8
13
28
0
1
Q-3
25
33
1
2
0
1
1
1
Q-4
1
0
0
1
0
26
36
0
Q-5
20
30
3
3
1
1
0
6
Q-6
34
28
1
1
0
0
0
0
Q-7
28
30
4
1
0
0
0
1
Q-8
20
22
6
14
0
0
0
2
Q-9
32
29
2
1
0
0
0
0
Q-10
29
29
4
2
0
0
0
0
Q-11
0
1
0
2
2
26
33
0
Q-12
1
3
4
4
4
25
23
0
Q-13
11
16
0
27
1
0
0
9
Q-14
0
1
0
1
4
24
33
1
Q-15
0
0
0
2
3
21
38
0
Q-16
1
0
3
4
1
23
32
0
Q-17
27
25
2
4
0
2
2
2
Q-18
0
0
0
1
4
26
33
0
Q-19
39
23
2
0
0
0
0
0
Q-20
31
30
1
2
0
0
0
0
Q-21
16
24
3
17
0
0
0
4
Q-22
0
0
2
13
3
26
19
1
Q-23
1
0
0
2
3
29
29
0
Q-24
0
0
1
1
1
28
33
0
Q-25
30
30
4
0
0
0
0
0
Q-26
0
0
0
1
3
29
31
0
Q-27
0
4
7
16
2
17
18
0
Q-28
8
10
45
1
0
0
0
0
Q-29
31
26
1
1
0
0
0
5
Q-30
35
23
1
0
1
0
0
4
Q-31
31
20
6
1
0
2
0
4
Q-32
19
23
8
8
1
0
1
4
Q-33
35
22
1
1
1
0
0
4
Q-34
44
16
0
0
0
0
0
4
Q-35
45
14
1
0
0
0
0
4
181

APPENDIX F
RESPONSES TO PRETEST QUESTIONAIRE VERSION TWO
Version 2 - "Don't Know" Response Value Included
Strongly
Somewhat
Somewhat
Strongly Don't
No
Response
Agree
Agree
Agree
Neutral
Disagree
Disagree Disagree Know
Response
Q-1
62
1
1
4
0
0
0 0
2
Q-2
5
4
6
15
10
29
0 0
1
Q-3
26
41
0
0
0
0
1 1
1
Q-4
0
0
0
2
3
24
39 1
1
Q-5
17
36
6
2
2
0
0 6
1
Q-6
36
33
0
0
0
0
0 0
1
Q-7
29
32
2
2
1
0
0 3
1
Q-8
21
31
4
5
0
0
0 8
1
Q-9
31
34
4
0
0
0
0 0
1
Q-10
27
35
2
2
1
0
1 2
0
Q-11
0
1
0
0
3
28
37 1
0
Q-12
1
3
1
4
1
28
27 5
0
Q-13
9
14
2
9
0
0
1 34
1
Q-14
0
0
0
1
3
16
50 0
0
Q-15
0
0
0
0
1
16
53 0
0
Q-16
1
0
0
1
5
15
42 5
1
Q-17
27
28
5
4
0
0
2 4
0
Q-18
1
1
0
1
3
18
44 2
0
Q-19
37
32
1
0
0
0
0 0
0
Q-20
26
38
1
1
0
0
0 4
0
Q-21
16
26
3
3
0
0
1 20
1
Q-22
0
1
2
5
4
23
27 7
1
Q-23
0
1
0
2
2
25
36 4
0
Q-24
4
0
0
4
0
22
33 7
0
Q-25
34
34
0
0
1
0
1 0
0
Q-26
0
0
1
0
2
24
42 0
1
Q-27
0
4
7
11
6
21
18 1
2
Q-28
10
20
38
2
0
0
0 0
0
Q-29
40
23
3
0
0
0
0 0
4
Q-30
42
25
0
0
0
0
0 0
3
Q-31
30
30
6
1
0
0
0 0
3
Q-32
25
18
10
9
0
2
0 2
4
Q-33
40
24
2
0
0
0
0 1
3
Q-34
52
15
0
0
0
0
0 0
3
Q-35
48
17
0
1
0
1
0 0
3
182

APPENDIX G
PRETEST QUESTIONNAIRE WRITTEN COMMENTS
Question 3: The APDIC has the equipment and information resources necessary to
answer my questions.
“Herbal products are an area of constant questions that good references are needed (none
are even available besides Lawrence and Micromedex.)”
Question 4: When I call the APDIC, background noise on their end interferes with my
ability to communicate over the telephone.
“No background noise.”
Question 5: When I receive materials from the APDIC, they are clear and easy to
read,
“N/A. Most of my info is over the phone. I have not received written material.”
«055
“Xeroxes sometimes off center and cut off part of the page.”
“Don’t Know.”
“We have not received any written material yet.”
“N/A” (2)
“Just the faxes are hard to read.”
“Faxes sometimes hard to read. This is to be expected.”
“APDIC has offered. I’ve never needed.”
183

184
Questions 7: When the APDICpromises to do something by a certain time, it does so.
“Have only asked for information very quick once.”
Question 8: When I have a problem, the APDIC is sympathetic and reassuring.
“Professional.” interpretation of the question. >
“This could be asked in a different way.”
Question 10: The APDIC provides its services in the time it promises.
“Usually. Sometimes a small delay because of another emergency on a difficult question.
They always explain the reason for the delay.”
Question 12: The APDIC does not tell exactly when services will be performed
“If I ask when, they always tell me.”
“But I have only asked them for specific times.”
Question 13: The APDIC keeps its records accurately.
“Don’t Know.” (4)
“Don’t Know. 1 would assume that they do.”
“I don’t know. I’ve never noticed any communication/relay problems.”
“Don’t really understand the statement.”
“To the best of my knowledge.”
Question 16: Employees of the APDIC are too busy to respond to caller requests
promptly.
“Service isn’t as fast as it used to be but certainly is still very timely and prioritized based
on the request.”

185
Question 17: I can trust employees of the APDIC.
“Don’t Know ”
Question 20: I feel safe in my interactions with the APDIC employees.
“?”
“Huh?”
Question 21: Employees get adequate support from the APDIC to do their jobs well.
CC<^53
“Don’t Know.” (2)
“How would I know?”
“Who? My employee or APDIC employee?”
“The only hint I have is how well the employees answer my questions.”
“N/A”
“Don’t understand.”
Question 22: Employees of the APDIC do not know what my needs are.
íicy»
“They always readdress question if they didn’t understand the focus.”
“Don’t understand ”
Question 27: I wish the APDIC could provide a quicker response to my questions.
“They can’t get any faster.”
“I think that is an unfair question. It always depends on what other questions they have
pending. Callers need to understand and respect that. I have never been unsatisfied with a
response time of an urgent question.”

186
“Due to prior experience with them.”
“In general, or just the last one?”
Question 28: The amount of time that it took the APDIC to respond to my most recent
question was
“I always get great service!”
“But APDIC was unable to FAX or mail the information I requested. I found this a little
odd and it did not fully meet my needs for drug information!”
Question 32: It is important that the APDIC fax me the supporting documents (e.g.
recent literature) for their answers to my questions.
“This tends to vary from situation to situation.”
“Undocumented answers are useless!”
“If necessary, yes.”
“In most cases but not always.”
“FAX or mail.”
“N/A”
“Very important.”
Question 35: I would recommend this service to a colleague.
(Three checks on strongly agree)
“And have done.”

187
Question 36: ADDITIONAL COMMENTS: Is there anything else that you would like
to tell us about your experience(s) with the APDIC? Also, any comments you wish to
make regarding how we could improve our service will he appreciated, either here or in
a separate letter.
“They need to update Generic Listings for both Rx and OTC medications - identification
and manufacturer.”
“I believe the APDIC is one of the best things the Pharmacy School does for the public. I
have never had any thing but good experiences when I call for drug or poison information.
In fact, I called today and was given an answer to a poison question in 2 minutes. Keep
up the good work. .”
“Improve the quality of your FAX equipment. Thank you!”
“No, there is nothing else I would like to say, except that the service was great!”
“The APDIC has come to our rescue many times - I can’t say enough about the service!
are the greatest! Thanks!”
“You do an outstanding job! 1 could not provide the level of care to my patients currently
available without your support.”
“Thanks for being there for so many health care professionals.”
“I have been very satisfied with the type of service and the professionalism of the
APDIC.”
“The information I was given was very useful and was given in a prompt manner. I would
definitely recommend this service to a colleague.”
“Although budgetary restraints have impacted on all of us, I would encourage you and
your library to expand your journal holdings as much as possible.”
“All APDIC people I have spoken with have been pharmacists - have the understanding of
the situations line workers, clinical staff (pharmacy) and pharmacy administration staff are
dealing with at the time of the call. Thanks and keep up the terrific service!”
“Eve been very satisfied with the service I’ve received and always have been dealt with
respectfully and promptly.”
“Keep up the good work!”

188
“A very well nan operation. I know I can ‘hang my hat’ on APDIC. Great bunch of
dependable, personable, knowledgeable professionals improving the overall health care for
Arkansans.”
“They are great!”
“Wish you could search abstracting databases such as EMBASE.”
“I can’t think of a thing to improve. Everyone I have talked to is very knowledgeable,
courteous and helpful. They do an outstanding job of prioritizing questions and soliciting
the information in order to focus the question correctly. Thank you for making my job
easier!”
“Excellent.”
“One minor thing -1 like to know who I am talking to -1 usually expect staff at an
organization to identify themselves when answering.”
“The APDIC Employee tried to answer my question about a specific generic equivalency,
but could not find an answer.”
“I began using APDIC when we became an Owen account late ’96. So far I have used
APDIC [approximately 2 times a month] Usually takes my calls.
He does a great job in obtaining a response and getting back to me that same day and is
good about letting me know how long it will take depending on the workload. Others I
have interacted with have also provided great customer service.”
“I have used the APDIC several times and I have always found the staff to be courteous
and well trained. I’ve received intelligent answers to my questions as well as supporting
documents when needed.”
“Good job. Thank you.”
“I have been extremely happy with my service. There are many times I call just for
information (not an emergency) to questions I get on various drugs, etc. For example,
today a nurse called and wanted to know if MS Conten 30 mg. could be inserted rectally.
She had heard that it could but the pharmacist she called didn’t know. I called APDIC
and got the information orally and by fax in a matter of minutes. It is great to know that
questions like this can be answered. Keep up the good work. Thank you.”
“On the main points, very grateful for a useful service. My only concern is I have no idea
what resources were used to handle my queries.”
“I called recently and needed dosing information on several drugs (higher doses than I
could find in any references). I needed this info stat (within 15 minutes) Mark called me

189
back within the 15 minutes with the info I needed. I really appreciate the promptness of
your service. APDIC is always very helpful to me!”
“We are very happy with this service.”
“Your staff is very courteous and professional and appear to enjoy learning from our
questions and situations that we present.”
“The center is underutilized by Arkansas pharmacists.”
“I am very grateful for the competent staff and the service I have received in the past.”
“I find the staff of APDIC to be very knowledgeable, supporting, and confident in their
work. Keep up the great work.”
“Overall the service is very good. I have submitted questions though that were never
answered. If no answer could be found, it would be helpful if someone could call me back
to let me know. Thank you.”
“They didn’t tell me how long it would take to get info so I didn’t know if it would be 5
minutes or 2 days. Also, the faxed response was actually a little bit of overkill - 26 pages
was quite a bit more than what I really needed.”
“I think the APDIC is a great resource. I don’t know what I would do without it.”
“With the Owen Healthcare, Inc. contract it may be beneficial to get on MS E-mail with
the pharmacy directors if you are not already on it.”
“Maybe ask when info is needed. I assume that since this is a poison control center that
these [questions] would be given priority, though I have never called about a poison
question as we have our own poison control center at OKC.”
“I have worked in a poison control service and you clearly have the resources needed and
the personnel to do an excellent job.”
“We think the entire staff does a great job getting the information we need to us in a
prompt manner. They are a great benefit to us.”
“I feel that the APDIC offers the most important information service a health professional
member has access to. If this service were not available my ability to practice my
profession would be diminished.”
“Your staff is always very courteous and extremely valuable.”

190
“I work with a number of directors and clinical managers. My impression is that you
don’t always ask the right questions to determine what the requestor really needs. You
end up providing the right answer to the wrong question. Some people need help
expressing what they really need.”
“I have always been satisfied with the help I received from poison control and the people
have been easy to work with. We really appreciate what you do. Thanks for the good
work.”
“My last request was concerning an OTC, I was surprised by the response and 10 faxed
documentation. Thanks. Keep up the good work.”
“Great!”
“You guys make me look good!”
“Very courteous and helpful.”
“1 use this service for tablet identification primarily! Love the cooperation we get. My
customers think I’m very smart!”
“I was very pleased with the response 1 received from my last call to the APDIC. The
information was very helpful and was given in a timely manner.”
“Need more availability to literature - journals, etc. Often items have been ‘checked out’
so a copy cannot be obtained. More comparative information.”
“Thank you for all the help.”
“This organization is one that I use consistently and am very happy to have this service
available to me.”
“Great group of professionals.”
“They are an excellent source of both information from the literature and practice
standards since most individuals interact with physicians in the medical facility.
Pharmacists [at the APDIC] are excellent clinicians.
“My interactions with APDIC have all been over the phone. Every time the staff have
been very helpful and polite. Most of my questions haven’t had an emergent response
time but when I needed the quick response they [have] been good. In my previous
position I was physically asked to go to a DIC which was nice. We have a contract with
your department and what might be handy is a listing of the various services you offer and
maybe a list of databases you use. Botton line, yes your department is very useful and
should be funded fully with resources and staff.”

191
“I’ve used and have interacted with drug information services at UAB, Sanford University
as well. No one compares to UAB’s quality and expertise in providing drug information.
You should try to model ALL aspects of your center after UAB’s drug information
center.”
“With the exception of one occurrence about a year ago, I’ve had nothing but fantastic
results and also [the employees at the APDIC are] very capable and eager to help people.
I always know they are right on it and not doing a haphazard job - they’re very polite,
want to help, and just plain get it done. I recommend them all of the time.”
“It’s good to know you’re there behind me when I need you. Thanks.”
“Good service.”

APPENDIX H
HISTORICAL DATA SHEET

FILE #
DRUG INFORMATION SERVICE DATA SHEET
How did this question come about?
Requestor
Rph MD RN Other
Subscriber #: Non-Subscriber:
Phone #: ( ) Extention:
Address:
Zip Code:
FILE: Yes No
FILENAME:
Information Reviewed By:
Health Center:
Pager #:
Response Needed by:
Stat (<15 min)
Today
Date:
No Rush!
Date Received:
Time Received:
Received By:
Date Completed:
Time Completed:
Returned By:
193

APPENDIX I
DATA COLLECTION FORM

FILE#
DRUG INFORMATION SERVICE DATA SHEET
INFORMATION REQUESTED
Type of Question (Classification):
How did this question come about?
Requestor:
Phone #: (
)
Extention:
Pager #:
Fax#: (
)
Address:
Zip Code:
Type of Response Requested:
â–¡
Written
â–¡
Oral
â–¡
Both
â–¡
Either
Response Needed by:
â–¡ Stat (<15 minutes)
â–¡ Today Time:
â–¡ Date / /
EH No Rush!
Subscriber #:
â–¡ Non-Subscriber
â–¡ Health Center
â–¡ RPh/Pharm.D.
â–¡ MD
D RN/NP
CH Other;
DATE AND TIME TRACKING
Activity Codes: 1 = Reception of Call or Fax
2 = Obtaining Information and Writing Answer
3 = Approval
4 = Returning Answer to Caller
Activity
Person
Start Date
Start Time
End Date
End Time
Response Completed By:
Response Approved By:
195

APPENDIX J
SEMI-STRUCTURED OUTLINE FOR STUDENT INTERVIEWS
I. Introduction
1. Introduce self.
2. Explain the purpose of the interview and describe the issues that will be discussed.
3. Explain that their names will never be attached to anything said during the
interview.
4. Ask permission to audio record the interview for recollection purposes.
II. Understanding the Service System
Please describe your job here at the DIPRC.
Additional follow-up questions:
a. What jobs are all of the students responsible for completing?
b. Are there any jobs that only you are responsible for completing?
c. Besides what we have talked about above, what other activities occupy your
time during the week (e g. lunch, meetings)?
d. How long do you usually take for lunch? Are you available to take calls during
your lunchtime?
e. What hours is the DIPRC open to take calls? How often do your morning
meetings disrupt these times?
I would like you do describe the work process here at the DIPRC. Explain to me what
happens from the arrival of a question until an answer is returned to the caller.
Additional follow-up questions:
a. Do you fill out a sheet for each question asked?
b. Do you answer the telephone if you are already working on a question and are
in the office?
c. How often do you usually work on more than one question? What is the most
you ever worked on at one time?
d. On some of the data sheets, an answer would be returned before it was
approved. When does this occur?
e. Do you obtain approval for all questions before you return an answer to the
caller?
f. What questions do you feel are the easiest to answer?
g. What questions do you feel are the most difficult to answer?
196

197
h. What do you do after the question has been answered?
i. How long does it take for you to “close-out” a question after you have
returned an answer?
j. How long did it take you to become comfortable answering questions posed by
callers?
k. Describe for me the use of the bulletin board? When do you use it and when
do you hold on to the data/call sheet?
Once you have completed a question, how do you decide which question to work on next?
A dditional follow-up questions:
a. Do some types of questions receive a higher priority than other types?
b. Are answered first because they are easier or more important?
c. Do you sometimes select questions because of interest rather than importance?
d. Do questions gain higher importance because of the requested response time?
e. Do questions gain higher importance because they are a pharmacist, physician,
etc.?
f. Do questions gain higher importance because they are a health center employee
or a subscriber?
III. Consumer Expectations and Perceptions
1. What do you think is the most valued aspect of the service you provide?
2. What do you think is the least valued aspect of the service you provide?
3. Have any callers seemed dissatisfied with a response you have given? Why?
4. What factors do you think most influence a caller’s perception of the quality of the
service you provide at the DIPRC?
5. How sensitive do you feel that the callers are to the amount of time it takes to answer
a question.
6. On your sheets, you ask the consumer to specify by when they need the question
answered. How sensitive do you feel callers are to delays in this time frame?
IV. Suggestions for Improvement
1. What did you like most about working at the DIPRC?
2. What did you like least about working at the DIPRC?
3. Is there anything that you need or that would make your job easier that is not provided
by the DIPRC?
4. Are there any suggestions you have for improving the process at the DIPRC?

APPENDIX K
SEMI-STRUCTURED OUTLINE FOR CO-DIRECTOR INTERVIEWS
Introduction
1. Explain the purpose of the interview and describe the issues that will be discussed.
2. Explain that their names will never be attached to anything said during the
interview.
3. Ask permission to audio record the interview for recollection purposes.
II. Understanding the Service System
Please describe your job as it relates to the DIPRC.
Additional follow-up questions:
a. Besides what we have talked about above, what other activities occupy your
time during the week (e.g. lunch, meetings)?
b. What hours is the DIPRC open to take calls?
I would like you do describe the work process here at the DIPRC. Explain to me what
happens from the arrival of a question until an answer is returned to the caller.
Additional follow-up questions:
a. Who works in the DIRPC? What are their roles/responsibilities?
b. Are the students supposed to fill out a sheet for each question asked?
c. Are the students supposed to answer the phone if they are already working on
a question and are in the office?
d. What is the suggested work process? How well do the students follow this
procedure?
e. Do you encourage or discourage the students to work on more than one
question at a time?
f. On some of the data sheets, an answer would be returned before it was
approved. When does this occur?
g. Is the student supposed to obtain approval for all questions before you return
an answer to the caller?
h. Describe how a question is approved process.
i. What questions do you think the students find are the easiest to answer?
j. What questions do you think the students find are the most difficult to answer?
198

199
k. How long does it take for the students to become comfortable answering
questions posed by callers?
l. Describe the use of the bulletin board? When do you use it and when do you
hold on to the data/call sheet?
Once you have completed a question, how does the student decide which question to
work on next?
Additional follow-up questions:
a. Do some types of questions receive a higher priority than other types?
b. Are some questions answered first because they are easier or more
important?
c. Do students sometimes select questions because of interest rather than
importance?
d. Do questions gain higher importance because of the requested response
time?
e. Do questions gain higher importance because they are a pharmacist,
physician, etc.?
f. Do questions gain higher importance because they are a health center
employee or a subscriber?
III. Consumer Expectations and Perceptions
1. What do you think is the most valued aspect of the service you provide?
2. What do you think is the least valued aspect of the service you provide?
3. What factors do you think most influence a caller’s perception of the quality of the
service you provide at the D1PRC?
4. How sensitive do you feel that the callers are to the amount of time it takes to answer
a question.
5. On your sheets, you ask the consumer to specify by when they need the question
answered. How sensitive do you feel callers are to delays in this time frame?
IV. Future of the DIPRC
1. Where do you think the DIPRC is headed for the next year? The next five years?
2. What is the viability of getting more rotation students if you needed an increase in
service capacity? Would you need to hire pharmacist/residents to answer questions?
3. How fast are the pharmacists/residents when compared to the Pharm.D. students?

APPENDIX L
TEXT OF COVER LETTER FOR MAIN QUESTIONNAIRE
July 30, 1997
Dear Colleague,
The Drug Information and Pharmacy Research Center (DIPRC) at Shands at the
University of Florida is currently working toward improving the quality of services we
provide. In order to achieve this goal, we have decided to ask our most recent callers
some questions regarding specific aspects of our service.
You can help us by participating in a brief survey concerning your recent experience(s)
with the DIPRC. Your prompt response is very important to us. You are one of only a
small number of practitioners who are being asked to give their opinions about our
service, so it is critical that each questionnaire is completed and returned.
Please answer all of the questions in this questionnaire (it should only take between 5-10
minutes to complete) and place it in the preaddressed, postage paid envelope provided.
You may be assured of complete confidentiality. The questionnaire has an identification
number for mailing purposes only. This is so we may check your name off the mailing list
when your questionnaire is returned. Your name will never be placed on the questionnaire
itself, nor will your responses be linked to you personally during the analyses.
Thank you for your participation.
Sincerely,
Randy C. Hatton, Pharm.D.,BCPS
Co-Director, DIPRC
Clinical Professor
Daniel L. Halberg
Doctoral Candidate, University of Florida
200

APPENDIX M
MAIN QUESTIONAIRE

Page 1
DIRECTIONS: Please answer the questions to the best of your ability by placing a
checkmark in ONE of the boxes next to each question. There are four sets of
questions: (1) some basic information about you; (2) your feelings about the Drug
Information and Pharmacy Resource Center (DIPRC) at Shands at the University
of Florida; (3) your perceptions regarding the amount of time the DIPRC took to
fulfill your request; and, (4) your overall feelings regarding the DIPRC and your
impressions about future behaviors regarding the DIPRC.
PART I: The following two questions gather some information about you. This
information will be used in conjunction with the information given in the rest of
the questionnaire to assess how needs and perceptions differ among our callers.
(1) What is your profession?
â–¡ RPhi/Pharm.D. U Physician O Nurse/Nurse Practitioner â–¡ Other:
(2) Are you a subscriber, a non-subscriber, or a Shands Health Center employee?
â–¡ Subscriber U Non-Subscriber O Health Center Employee
(3) How often do you use the DIPRC?
â–¡ First time user â–¡ 3-5 times per year U 10-15 times per year
â–¡ 1-2 times per year â–¡ 5-10 times per year â–¡ more than 15 times per year
PART II: The following set of statements relate to your feelings about the Drug
Information and Pharmacy Resource Center (DIPRC) at Shands at the University
of Florida. For each statement, please check the box that best describes the
extent to which you believe the DIPRC has that characteristic. The range of
selection varies from "Strongly Agree" to "Strongly Disagree"; however, you may
check any of the boxes provided. If you feel that you cannot answer a question,
or that the question does not apply to you, you may check the box labelled "Don't
Know".
(4) The DIPRC has the equipment and
information resources necessary to
answer my questions
â–¡
â–¡
â–¡
â–¡
â–¡
_l
â–¡
â–¡
(5)
When I call the DIPRC, background noise
on their end interferes with my ability to
communicate over the telephone
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(6)
When I receive written materials from the
DIPRC, they are clear and easy to read .
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
J
â–¡
(7)
Employees of the DIPRC speak in a
manner that is easy to understand
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(OVER)
202

203
Page 2
(8) When the DIPRC promises to do
something by a certain time, it does so .
(9) When I have a problem, the DIPRC is
sympathetic and reassuring
(10) The DIPRC is dependable
(11) The DIPRC provides its services in the
time it promises
(12) The DIPRC does not give me individual
attention
(13) The DIPRC does not tell me exactly when
services will be performed
(14) I do not receive prompt service from
DIPRC employees
(15) Employees of the DIPRC are not always
willing to help me
(16) Employees of the DIPRC are too busy to
respond to caller requests promptly . . .
(17) I can trust employees of the DIPRC ....
(18) Employees of the DIPRC do not give me
personal attention
(19) Employees of the DIPRC are polite ....
(20) I feel safe in my interactions with the
DIPRC employees
(21) Employees of the DIPRC do not know
what my needs are
/V
& & 4
x ^ a®
^ ¿P .
°
e.®
/
f
r
f
f
r
r
r
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
U
ü
u
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
U
u
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
u
_i
ü
u
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
u
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡

204
Page 3
(22) The DIPRC does not have my best
interests at heart
^vv
r r í'fí r r
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡
é «>
& o°
â–¡
(23) The DIPRC does not have operating
hours convenient to me
â–¡ â–¡ â–¡
â–¡ â–¡ â–¡ â–¡ â–¡
PART III: The following questions regard your perceptions about the length of
time in which the service was rendered. Please think about the next four items in
terms of the last question you presented to the DIPRC, and react to the
statements below using the scale provided. Again, you may check any of the
boxes on the scale to show how strong your feelings are.
(24) The amount of time that it took the
DIPRC to respond to my most recent
question was acceptable
*3
&
r r
â–¡ â–¡
A?
^ A^ ¿
, r r r r r
â–¡â–¡â–¡â–¡â–¡â–¡
(25)By the time I received a response from
the DIPRC, the information was no longer
useful to me O â–¡ q q â–¡ â–¡ â–¡ j
(26)I wish the DIPRC could provide a quicker
response to my questions
U U Ü LI ü ü U
J
(27)The amount of time that it took the DIPRC to respond to my most recent question
was
â–¡ MUCH SHORTER THAN I EXPECTED
â–¡ SHORTER THAN I EXPECTED
â–¡ A LITTLE SHORTER THAN I EXPECTED
â–¡ EQUAL TO WHAT I EXPECTED
â–¡ A LITTLE LONGER THAN I EXPECTED
â–¡ LONGER THAN I EXPECTED
Jl MUCH LONGER THAN I EXPECTED
(OVER)

205
Page 4
PART IV: The following seven statements relate to your overall feelings about the
DIPRC. Please respond by checking the box which best reflects your own
perceptions.
(28) The overall quality of the services provided by the DIPRC is best described as:
_l Excellent Q Very Good Q Good â–¡ Fair â–¡ Poor (_) Unacceptable
(29) The responses I receive from the DIPRC
are useful to me in my practice
A*
r r r
â–¡ u
U LI U U Li
(30) The responses I receive from the DIPRC
are essential to me in my practice
â–¡ â–¡â–¡â–¡â–¡â–¡â–¡
â–¡
(31) It is important that the DIPRC mail and/or
fax me the supporting documents (e.g.,
recent literature) for their answers to my
questions
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(32)
The DIPRC's answers to my questions
are used to improve patient outcomes ..
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(33)
I intend to use this service in the future.
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(34)
I would recommend this service to a
colleague
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
â–¡
(35) ADDITIONAL COMMENTS: Is there anything else that you would like to tell us about
your experience(s) with the DIPRC? Also, any comments you wish to make
regarding how we could improve our service will be appreciated, either here or in a
separate letter.
Thank you very much for your help.

APPENDIX N
FOLLOWUP POST CARD FOR MAIN QUESTIONNAIRE
SHANDS
at the University of Florida
t UNIVERSITY OF
# FLORIDA
Drug Information & Pharmacy Resource Center
PO Box 100316* Gainesville, FL 32610-0356
(Side One)
Dear Colleague:
About one week ago, a questionnaire seeking your opinions about the service quality of the Drug
Information and Pharmacy Resource Center (DIPRC) was mailed to you.
If you have already completed and returned the questionnaire to 11s please accept our sincere
thanks. If not, please do so today. Because the questionnaire was sent to only a small sample of
our recent callers it is extremely important that yours also be included in the study if the results
are to accurately represent the feelings of the professionals we serve.
If by some chance you did not receive the questionnaire, or it got misplaced, please call my office at
(352) 392-9035 or e-inail me at HAT.BERG@COP3.HEAITH.UFL.EDU and I will get another one
in the mail to you. Your contribution to the success of this study is greatly appreciated.
Sincerely,
Daniel L. Ilalherg
Doctoral Candidate (Side Two)
206

APPENDIX O
RESPONSES TO MAIN QUESTIONAIRE
Main Questionnaire Responses
Response
Strongly
Agree
Agree
Somewhat
Agree
Neutral
Somewhat
Disagree
Disagree
Strongly
Disagree
Don't
Know
No
Response
Q-1
131
13
21
37
0
0
0
0
1
Q-2
79
67
49
0
0
0
0
0
8
Q-3
49
26
46
40
17
23
0
0
2
Q-4
90
84
5
6
2
0
0
13
3
Q-5
4
3
2
16
9
99
64
4
2
Q-6
49
88
11
16
0
0
2
35
2
Q-7
71
92
22
2
4
5
2
2
3
Q-8
68
98
27
4
2
0
0
3
1
Q-9
38
70
18
37
2
0
0
36
2
Q-10
74
110
10
2
1
1
0
4
1
Q-11
67
100
22
4
2
1
0
6
1
Q-12
0
5
0
9
5
91
85
7
1
Q-13
0
5
10
16
6
91
63
11
1
Q-14
4
1
3
7
10
90
84
1
3
Q-15
0
2
1
2
6
84
103
3
2
Q-16
0
3
8
10
6
87
78
9
2
Q-17
69
90
12
9
0
4
5
13
1
Q-18
0
6
0
3
3
90
95
5
1
Q-19
88
103
5
0
1
1
2
2
1
Q-20
69
107
8
6
0
1
2
7
3
Q-21
2
5
7
18
12
89
58
11
1
Q-22
0
1
2
6
4
93
84
11
2
Q-23
0
4
8
22
12
92
43
21
1
Q-24
73
94
19
6
4
3
2
1
1
Q-25
1
8
11
8
16
85
72
1
1
Q-26
10
24
37
44
12
46
24
3
3
Q-27
17
35
17
88
32
10
1
0
3
Q-28
80
86
26
0
1
0
0
0
10
Q-29
92
92
8
5
1
0
0
3
2
Q-30
50
72
38
30
6
2
0
3
2
Q-31
69
64
28
23
3
6
2
6
2
Q-32
85
76
15
13
0
1
2
8
3
Q-33
120
71
6
1
0
1
0
2
2
Q-34
129
63
4
1
0
0
0
2
4
207

APPENDIX P
MAIN QUESTIONNAIRE WRITTEN COMMENTS
Question 4: The DIPRC has the equipment and information resources necessary to
answer my questions.
“[N]ever been there to see.”
"Have neither personally seem nor utilized equipment/info. Resources but had questions
answered satisfactorily."
Question 5: When I call the DIPRC, background noise on their end interferes with my
ability to communicate over the telephone.
“I never call” (From a physician - presumably they have their office assistant call for
them.)
Question 6: When I receive written materials from the DIPRC, they are clear and easy
to read
“No experience.”
"N/A"
“But that’s O K ” (In response to an answer of‘Somewhat Agree’)
Question 7: Employees of the DIPRC speak in a manner that is easy to understand
“Some employees (from Asia or Middle East) have more of a problem.”
“Variable depending on student.”
“I never call” (From a physician - presumably they have their office assistant call for
them.)
“On one occasion the person answering phone was very hard to understand due to
accent.”
208

209
“The person I spoke with was difficult to understand, but that does not mean everyone is
like that.”
“The occassional student with ethnic accents can be difficult. (Asian, etc...)”
“Foreign students struggle a little more, understandably.”
Question 9: When I have a problem, the DIPRC is sympathetic and reassuring.
“[Njever had one.”
“I am not looking for sympathy or reassurance!?”
"N/A"
Question 10: The DIPRC is dependable.
"Outstanding." (Written above an 'Excellent' grade on the question.)
Question 17: I can trust employees of the DIPRC.
“[CJome, now!” (Apparently relating to the relevance of the item)
Question 19: Employee of the DIPRC are polite.
“Very. ©”
Question 20: I feel safe in my interactions with the DIPRC employees.
“[Depending on the students there that month.”
“When I can understand what they are saying.”
“N/A”
Question 21: Employees of the DIPRC do not know what my needs are.
“How could they if I don’t tell them?”

210
“Regarding question #21 - The staff member who took the call and interpreted my
question did not have any idea of the subject of my inquiry - PPD and anergy testing - 1
step vs. 2 step methods for PPD; I needed to explain and spell [out] the info, for her -1
wish you could have experienced pharmacists processing the incoming questions.”
"Question need clarifying. Do you mean 'Understand my requests for specific info' or
need relating to a timely response?"
"Are you trying to trick me?"
"Of course they don't until I tell them."
Question 23: The DIPRC does not having operating hours convenient to me.
“Do not know your exact hours. But [they have] been available when I’ve needed them.”
"Do not know hours of operation."
"I'm open week ends and central standard time [until] 6 PM." (In comment to a
'Somewhat Agree' grade on the question."
Question 24: The amount of time that it took the DIPRC to respond to my most recent
question was acceptable.
“As was the response time on 9 out of [the] last 10 requests much more than adequate
(amazing even).”
Question 26: I wish the DIPRC could provide a quicker response to my questions.
“[0]nly in one of ten cases.”
“Sometimes you need info, quick, other times it’s not important.”
“I have not had a problem in this regard.”
"Sometimes, as it was in my case, that is impossible."

211
Question 27: The amount of time that it took the DIPRC to respond to my most recent
question was.
“I called because information was not available in my current library.” (Commenting on a
‘longer than expected’ response.)
“I don’t know how much research was involved.”
"[A]nd exactly when they told me."
"P S. I also contacted the Life Extension Foundation and received an answer within 5
minutes, because one of their staff knew the answer. Flowever, I was not dissatisfied with
the DIPRC response, since it involved medication not available in the U S."
Question 28: The overall quality of the services provided by the DIPRC is best
described as:
(The “Excellent” response was checked and double underlined) (2)
"Outstanding." (Written above an 'Excellent' grade on the question.)
Questions 29: The responses I receive from the DIPRC are useful to me in my
practice.
"N/A"
Question 30: The responses I receive from the DIPRC are essential to me in my
practice.
"N/A"
Question 31: It is important that the DIPRC and/or fax me the supporting document
(e.g., recent literature) for their answers to my questions.
“Fax.”
“Sometimes, but not always.”
"Depends on the question not the same for all."
“[T]hey did provide great [information] on our pharm. drug for a sedation workshop.”

212
"N/A"
Question 32: The DIPRC’s answers to my questions are used to improve patient
outcomes.
“N/A” (2)
“Unless I request not to bother which will be at least Vi the time.”
“[and] nurses responses to med questions.”
Question 34: I would recommend this service to a colleague.
(Respondent marked ‘highly’ above recommend, indicated a ‘highly recommend’)
“I have done so on numerous occasions.”
“Most definitely ”
Question 35: Additional Comments: Is there anything else that you would like to tell
us about your experience(s) with the DIPRC? Also, any comments you wish to make
regarding how we could improve our service will be appreciated, either here or in a
separate letter.
“I am grateful that this service is available as it saves me time and helps my patients.
Thank you.”
“I always find the staff receptive and extremely helpful. Results are accurate and assist me
a great deal in my forensic identifications. I greatly appreciate the fast response time
necessary for the type of work I perform. Thank you.”
“Are you guys on the world wide web? If so, maybe make your e-mail address available.”
“For a first time caller to your Drug Information Service I am very pleased and appreciate
your services. My customer was from out of the country and was quite impressed with
the info, we shared with him. Keep up the good work!”
“Your service is provided with efficiency and effectiveness that is without competition
from any other resource. Congratulations and thank you for your effort.”

213
“At times I think the student answering the [telephone] does not understand the question
(or cannot understand English) They could improve their communication skills (i.e., listen
better, restate the question, etc.)”
“DEPRC has always been very helpful with all of my questions. The questions I ask are
usually very difficult and obscure and are referred to DIPRC when I can’t find anything
with a basic Medline search and resources I have available to me.”
“The last time I used DIPRC was for information about Redux and its possible affect on a
patient’s hair loss. I did receive a phone call back but they were also to mail some [kind]
of survey that wasn’t received.”
“I feel your services have always been invaluable to my practice with improved patient
outcomes.”
“Keep up the good work!”
“The last interaction and request was handled extremely well and information received was
very necessary and helped us immensely. We have however not received this kind of
service in the past. So many of my responses were tempered by past interactions. The
last request could not have been handled more appropriately or professionally.”
“Hey Dr. Hatton and Professor Doering! You guys are awesome.”
“Most of these questions do not apply to my situation/question. Therefore, my answers
may not help.”
“I do not believe that I have used DIPRC in the past.” (In response to a blank
questionnaire.)
“Long response time to medical urgencies is not useful to me - need to rank urgency of
calls and establish priorities for meeting requests. Also, staff need to be familiar with
names and spellings of drugs in order to assist callers.”
“Although this is probably impossible.. .it would be great if the DIS staffed students who
were nearing the end of their clinical rotations rather than the beginning. Sometimes
students don’t understand even the most basic questions due to clinical inexperience.
Although they do eventually come through after being helped by Paul and/or Randy I’m
sure.”
“Great job - We need you guys.”
“Good job, keep it up.”

214
“Thank you for helping me to ‘look more knowledgeable’ to other health care
professionals and patients. Your behind the scenes work is greatly appreciated.”
“DIRPC is a SUPER important resource for my pharmaceutical questions.”
“I am a very satisfied first time user, my coworkers have always been very satisfied with
the service.”
“Keep up the great work! Your accessibility and willingness to help is most appreciated!”
“The drug information service personnel have always been polite. They have always
contacted me in a timely fashion. If the available resources don’t have the information I
need, the personnel go ‘the extra mile’ and contact drug manufacturers etc., to help me
get the information 1 need. My colleagues and I appreciate all the hard work!”
“Please allow the responder or ask responder of DIPRC [to] identify him/herself whenever
he/she responds to a call. Sometimes caller finds himself very awkward in asking the new
responder who the first one was and all the good intention was lost.”
“Go Gators.”
“They are not a resource in pentineal dialysis drugs and this would be helpful.”
“I think it would be helpful if the DIRPC staff asked what sort of documentation is needed
-1 have had many positive interactions with the DIPRC, but had one instance in which I
was provided with no supporting documents when I expected some.”
“Unfortunately the DIPRC was unable to provide me the very specific information I
requested and I was forced to request it directly from the drug company. Admittedly, the
info. I requested was detailed and new regarding latex content of many drugs. Thanks.”
“As a forensic chemist, working for the State of Florida (FDLE) they survey does not
really apply - but I am grateful for the service. I would be willing to supply information
not in your computers as to the identity of tablets, etc.”
“The only problem I have ever had was the inability to understand the person answering
the phone and getting him to understand me. This only occurred once. This was due to
language barrier.”
“I have had very positive experiences.”
“I am a deputy sheriff and used your service to identify some prescription drugs I
confiscated off a[n] arrestee. Your service was polite, prompt and very helpful.”

215
“I really like the feature of having DIPRC Fax the answers to the Dr’s/Rn’s/ARNP’s, etc.
especially when time is of the essence. Occasionally I would like a synopsis of the answer
to the query for my own info. But probably less than V2 the time.”
“1 wonder if it would be possible to get extemporaneous compounding help [through] the
DIRPC on a fairly rapid timetable? I got used to having access to the help desk at
P.C.C.A. out in Texas while purchasing enough from them to warrant the service for 2
years recently.”
“You have been great - You are great - and I expect you shall continue being great - and
we thank you.”
“I appreciate that, while I’m not a subscriber, I receive the same courteous, considerate,
and prompt attention to my questions as a subscriber.”
“Being that this was the first time I used your service, I do not know if what I have to say
is a fair assessment. With that out of the way, it took 24 hours to get a response to my
question That really was not a problem for me in this particular situation; however, I can
see how that much time would be frustrating in other situations, especially if that is a
typical response time. But I think that is a great service you oifer. Thanks! ©”
“Found the service useful. Follow up info, on PRO-FIBE was sent out to me rather
quickly, and I appreciate it.”
“The DIPRC offers extremely reliable and prompt service! Keep up the good work!”
“DIPRC provides an excellent experience for clinical students. Randy and Paul know just
about everything or know where to look it up!”
“Need to make shorter questionnaire!”
“I enjoy this service. Thank you.”
“DIRPC is a wonderful resource and I really appreciate it’s availability to health care
practitioners.”
“It would be great to have this type of service of other than nutrient drug related research
But your folks are doing great, thank you!”
“The two ladies that helped me were considerate, knowledgeable, interested, and
efficient.”
“Outstanding response time.”

216
“Most frequently my needs are related to cost and enough information about a drug to
justify it’s use to an insurance company. A negative outcome can mean that a patient’s
treatment plan could/would be changed to a less optimal course of treatment.”
“I appreciated the information received but it was much more technical than I wanted. It
would be helpful to have a broader range of references. The person I spoke with was very
willing to help.”
“Questions/Responses via electronic mail, or a Web site set-up would be a nice addition!”
“Never used U.F”
“I have been impressed with all aspects of assistance, courtesy and timeliness. [Name
omitted] was very helpful.”
“Great Job!! Please continue. Thanks.”
“Keep up the good work.”
“Sometimes it is difficult to understand the DIPRC students who answer the phone and
they require repeating the request several times before they understand it.”
“If caller could be assisted in prioritizing questions appropriately somehow, workload
could be evened out. i.e., is patient bleeding? , or in an exam room? or won’t be until
next week’s appointment.”
“I’m a caller from another DIC inquiring about formulary status of fosphenytoin.
Information presented was concise, accurate, and timely.”
“No problems with DIRPC. I do my own initial research into problems; when exhausted,
I use DIPRC as ‘last resort’ to solve problem. Given that scenario, any answer to a
problem is welcome.”
“This is a wonderful service you provide. Is the service limited to just UF or is it available
county wide? I never know you existed as a resource! Thanks so much!”
“Response time was fantastic and most beneficial.”
“The employee was very helpful and willing to look for answers.”
“Thanks for the wonderful-clear-concise info, on the drugs we requested! [Name omitted]
was very helpful!”
“Allow for submission of questions via e-mail.”

217
“Don’t make subscribers give you our demographic data each time we call. You already
have it and should be able to access it.”
“Your services are extremely useful for questions which do not require an immediate
answer. However, for questions in which a physician is waiting on an answer A.S A P.,
the turnaround time is too long for the service to be useful. As a community hospital with
no attached health center, finding literature can be difficult. It would be helpful it there
was a system in place to answer more urgent questions quickly.”
"Upon the advice of a colleague, I called the DIPRC. I have only used the service once,
therefore it is difficult to judge the overall service. I will say that when I did call the
service was helpful and I found out what I needed to know in a short period of time. I
would definitely use the service again if the need arises."
"Have more experienced people available if at all possible. From initial conversations, I
sometimes get the idea the person is very uninformed about clinically relevant issues - at
other times just the opposite."
"The DIPRC is an invaluable resource for practicing pharmacists. Thank you for the
assistance you provide!"
"This was my first experience with using DIPRC - was very helpful -
problem was researched and answered efficiently/promptly - Thank you."
"I have had trouble understanding some of the personnel on various occasions. They try
hard, but it is a little frustrating sometimes."
"Internet access to e-mail requests would be useful."
"I have never had a bad experience with the DIPRC. All questions have been answered in
a very timely fashion. Some questions are difficult and have no answers. Receiving that
info is also good and reassuring to professionals who reached their own dead ends. Thank
you for all your help."
"The U of F DIPRC was initiated by M. Peter Prevonka when a question arose about a
decent and factual information given in a timely manner. I have not been disappointed.
All health care professionals had a need for current, practical answers to questions. More
information has come our way in the last 8 years than in the previous 200 years. Ask
Oscar Araujo as a senior advisor and precious resource."
"[The] person who helped me was very polite and helpful. ©"
"The problem with the slow turn around time of my last question was the result of one
new student's lack of experience. The question was more of a confirmation of information
and the turnaround time was not critical."

218
"Your help was great in coding the durg. 'New' street drugs are not always in our encoder
or ICD-9-CM coding books. Thank for you help."
"I have had an experience that could have resulted in a more positive outcome.
Apparently, a pharmacy intern researched a question I had and provided what I found to
be a superficial treatment. Many questions only need such treatment some need more.
Consider giving the caller the choice in having [information] prepared by a student or a
practitioner."
"Courteous, prompt, interested in requests."
"Service is very much appreciated and have recommended it to others.
"Very much appreciated."
"Another drug information service did a 'bash the pharmacist' response and never
answered the question."
"You might consider a membership in the Life Extension Foundation which monitors
research done in other countries, as well as leading edge research in the U S. 1 -800-841 -
5433."
"Shortened turn-around time would be great."
"I personally find the DIPRC to be extremely useful especially working as a retail
pharmacist whereby our resources are very limited and it is hard to answer patients'
questions and concerns in a timely fashion But thanks to the DIRPC, their assistance in
the past have proven very valuable."

APPENDIX Q
SIMULATION BLOCK DIAGRAMS
219

220
ENTER
—►
LEAVE
/ LINES\
SELECT
1,1 ,&S,mRINGOUT
7
J 1 ,V(STUD)
ASSIGN

221
2, FN(QTYPE3) I
ASSIGN
P5
PRIORITY
2, FN(QTYPE4)
ASSIGN
PRIORITY

222

223

224

APPENDIX R
SIMULATION PROGRAM CODE
GPSS/H PROFESSIONAL RELEASE 3.0n-C10 (UG207)
9 Oct 1997 23:28:54 FILE: dis.gps
LINE# IF DD BLOCK# *LCC OPERATION A, B, C, D, E, F, G COMMENTS
1 *234567890+2345678901+
2
3 REALLOCATE COM, 300000
4 SIMULATE Start GPSS/H Job
5
f. -k-k-k'k-k-k-k-k-k-k-k-k-k-k-k-k-k-k'k-k-k-k-k-k-k k -k-kkck-k kick k * * kkkkkkkkkkkkkkkkkkkkkkkkk •A'**'
7 * INITIALIZE NECESSARY STORAGES, VARIABLES, AND FUNCTIONS *
Q kkkkkkkk'k'i:kkkkkkkkkk'k'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
9
10
INTEGER
&I, &RM, &SV, S,D, &STD, SCL
Initialize Variables
11
INTEGER
SSVC, &AMC, &STMC, SL03T
12
REAL
&CLKMIN, &AVAR, &DA, STA, &HCUR
13
REAL
iAVIS, &DLCT, &DLPC, 5AM, &SIM
14
REAL
&SVPF, &AVUT, &TQ4,&TQT, &BQ4, StBQT
15
16
TOTALQ
EQJ
10,Q
Initialize Queue Numbers
17
ANSWERQ
EQJ
11,Q
18
BCAREQ
EQJ
12, Q
19
SERVICEQ EQU
13,Q
20
RETURNQ
EOJ
14,Q
21
WAITSVQ
EQJ
15, Q
22
23
LINES
STORAGE
4
Set Number of Phone Lines
24
25
STUD
VARIABLE
(RN3*&SV/1000)+1
Variable: Student Selection
26
27
EELAY1
BVARIABLE
((C1-F7 )>15)+(&D-P8)>=1
Boolean Variable: STAT Delay-
28
DELAY2
BVARIABLE
(&D-P8) >=1
Boolean Variable: Today Delay
29
DEIAY3
BVARIABLE
{&D-P8)>=3
Boolean Variable: Date Delay
30
31
SETTYP
FUNCTION
P2,D4
Routing Function: Question Type
32
1, QTYPE4/3,0TYPE3/4, QTYPE2/8, QTYPEl
33
34
BUSY
FUNCTION
RN2,C2
Function: Wait 5-10 Minutes
35
0,5/1,10
36
37
REM3HK
FUNCTION
RN6,C2
Function: Check once per hour
38
0,0/1,59
39
40
ENTSVC
FUNCTION
P5,D4
Routing Function: Begin Answering
41
2,TENR/4
, TENR/ 6, TCNR/8, STAT
42
43
QTYPE1
EUNCTICN
RN4, D3
QType Generator l=Group l/2=Group 2/3=Group 3
44
.483,1/.
330,2/1,3
for Stat Questions
45
46
QTYPE2
EUNCTICN
RN4, D3
QType Generator l=Group l/2=Group 2/3=Group 3
47
.360,1/.'
743,2/1,3
for Today Questions
48
49
QTYPE3
EUNCTICN
RN4, D3
QType Generator l=Group l/2=Group 2/3=Group 3
50
.215,1/.:
572,2/1,3
for Dated Questions
51
52
QTYPE4
FUNCTION
RN4,D3
QType Generator l=Group l/2=Group 2/3=Group 3
53
.278,1/J
658,2/1,3
for No Rush Questions
54
55
QUEST9
FUNCTION
RN5,D4
Function: Priorities for 9:00-10:00
56
.120,8/.'
681,6/.847,<
1/1,2
57
58
QUEST10
EUNCTICN
RN5,D4
Function: Priorities for 10:00-11:00
225

59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
226
110,8/.690,6/.855,4/1,2
CUEST11 EUNCTICN RN5,D4
.100,8/.662,6/.863,4/1,2
EUnction: Priorities for 11:00-12:00
OEST12 FUNCTION RN5, D4
.100,8/.685,6/.842,4/1,2
Function: Priorities for 12:00-1:00
QJEST13 FUNCTION RN5, D4
.110,8/.508,6/.841,4/1,2
EUnction: Priorities for 1:00-2:00
QUEST14 EUNCTICN RN5,D4
.110,8/.572,6/.863,4/1,2
Function: Priorities for 2:00-3:00
QJEST15 FUNCTION RN5, D4
.140,8/.498,6/.817,4/.998,2
EUnction: Priorities for 3:00-4:00
QJEST16 EUNCTICN RN5,D4
.100,8/.320,6/.822,4/1,2
EUnction: Priorities for 4:00-5:00
IAT9 EUNCTICN RN7,C6
0,1/.403,10/.701,20/.830,30/.959,40/1,65
EUnction: IATs for 9-10
IAT10 FUNCTION RN8,C1Q
0,1/.251,10/.4 95,20/.641,30/.760,40
.844,50/.903,60/.947,70/.981,80
1,110
EUnction: IATs for 10-11
IAT11 EUNCTICN RN9,C11
0,1/.167,10/.375,20/.561,30/.682,40
.750,50/.854,60/.893,70/.957, 90
.979,110/1,170
EUnction: IATs for 11-12
IAT12 FUNCTION RN10,C14
0,1/.188,10/.387,20/.4 91,30/.642,40
.703,50/.768,60/.816,70/.868,80
.892,90/.925,100/.957,110/.976,120
1,190
Function: IATs for 12-1
IATAET EUNCTICN RN11,C15
0,1/.149,10/.325,20/.426,30/.542,40
.619,50/.675,60/.734,70/.786,80
.823,90/.861,100/.919,130/.968,180
.986,200/1,290
EUnction: IATs for Afternoon
GRCUP1 EUNCTICN RN12,C13
0,0/.213,10/.394,20/.504,30/.567,4 0
.661,50/.709,60/.764,90/.811,120
.866,150/.906,180/.945,250/1,310
EUnction: Service Times for Question Type 1
GRCUP2 EUNCTICN RN13,C16
0,0/.Q79,10/.191,20/.322,30/.428,40
.480,50/.533,60/.605,70/.651,80
.697,100/.776,130/.829,160/.888,210
.954,260/.980,300/1,530
EUnction: Service Times for Question Type 2
GRCUP3 EUNCTICN RN14,C20
0,0/.01,10/.12,20/.19,30/.25,40
.30,50/.38,60/.41,70/.49,80
.56,90/.63,100/.68,110/.72,120
.76,140/.80,160/.86,190/.90,210
.93,290/.95,320/1,470
EUnction: Service Times for Question Type 3
STATIM EUNCTICN RN15,C1S
0,0/.12,5/.255,10/.490,15/.627,20
.706,25/.725,30/.784,35/.804,40
.862,45/.901,50/.921,60/.941,65
.961,75/1,80
EUnction: Service Times for Stat Calls
TCMIN EUNCTICN RN16,C6
0,0/.683,5/.930,10/.97 9,15/.990,20/1,45
EUnction: Service Times for Taking Calls
RIMINI EUNCTICN RN17,C8
0,0/. 564,2/.820,4/.916,6/.944,8
.980,10/.994,16/1,28
EUnction: Return Call Time for Group 1

134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
227
RTMIN2 EUNCTICN RN18,C9
0,0/.438,2/.700,4/.877,6/.894,8
.95,10/.961,12/.983,16/1,48
RIMIN3 EUNCTICN RN19,C12
0,0/.358,2/.538,4/.769,6/.829,8
.880,10/.897,12/.923,16/. 940,22
.966,26/.982,36/1,46
RTSTAT FUNCTION RN20,C6
0,0/.610,2/.837,4/.938,6/.940,8/1,10
Function: Return Call Time for Group 2
Function: Return Call Time for Group 3
Function: Return Call Time for Stat Questions
*** * ********************* **** * ** * ****** A-*************************** * * * * *
* GENERATE CALLS *
***************************** * ************* * **** ******** * *** *** *** **
1
GENERATE
(EN(IAT9)*SAM),, (SCL*480),, ,10PH
Generate Callers From 9-10
2
TEST G
Cl, (&CL*480),PASTCLK
3
TEST LE
Cl,(&CL*480+59),PASTCLK
4
ASSIGN
5, EN (QJEST9)
5
TRANEER
, CALLER
6
GENERATE
(EN(IAT10)*&AM),, (&CL*480+59),,,
10PH
Generate Callers From 10-11
7
TEST GE
Cl, (&CL*480+60), PASTCLK
8
TEST LE
Cl, (&CL*480+119), PASTCLK
9
ASSIGN
5, EN (QUEST10)
10
TRANSFER
,CALLER
11
GENERATE
(EN(IAT11)*SAM),, (&CL*480+119),,
,10PH
Generate Callers From 11-12
12
TEST GE
Cl, (&CLM80+120),PASTCLK
13
TEST LE
Cl, (&CL*480+179),PASTCLK
14
ASSIGN
5, EN (QJEST11)
15
TRANSFER
, CALLER
16
GENERATE
(EN(IAT12)*&PM),, (&CL*480+179),,
, 10PH
Generate Callers From 12-1
17
TEST GE
Cl, (&CL*480+180),PASTCLK
18
T