UFDC Home  UF Theses & Dissertations   Help 
Material Information
Thesis/Dissertation Information
Subjects
Notes
Record Information

Full Text 
PAGE 1 1 OPTIMAL ALLOCATION OF BLOOD INVENTORIES By YAN QIN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2011 PAGE 2 2 2011 Yan Qin PAGE 3 3 To my parents, Xianglin Qin and Zhengxian Mao, for a lifetime of love and support PAGE 4 4 ACKNOWLEDGMENTS First and foremost, I would like to express my sincerest gratitude to my dissertation advisor, Dr. Anand Paul, for his guidance and encouragement through the completion of this dissertation. This work would not have been possible without his help I am also grate ful for his v aluable life advice that help s mold a better me and lead to a more fulfilling life. S pecial thank s also go to this exciting research area. I am deeply thankful for Dr. Asoo Vakharia for patiently guiding me thr ough my first ever paper in the PhD program and all the constructive academic suggestions he gave me. I would like to thank Dr. Janice Carrillo and Dr. Haldun Aytug for their career guidance and continued support. I also appreciate the enlightening feedback from Dr. Jos eph Geunes on this dissertation My grateful thanks also go to Ms. Patricia Brawner, Ms. Shawn Lee, and Ms. Jennifer Shockley for their assistance and support during the program. Last but not least, I would like to thank all my fellow PhD students and fri ends, especially Ms. Wei Dai, for their e ncouragement, friendship, and pleasant companion all the way along. PAGE 5 5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ ............... 4 LIST OF TABLES ................................ ................................ ................................ ........................... 7 LIST OF FIGURE S ................................ ................................ ................................ ......................... 8 ABSTRACT ................................ ................................ ................................ ................................ ..... 9 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .................. 11 Overview of Blood Banking ................................ ................................ ................................ ... 11 Characteristics of Blood Banking Problems ................................ ................................ ........... 12 Current Industrial Practices ................................ ................................ ................................ .... 13 2 L ITERATURE REVIEW ................................ ................................ ................................ ....... 22 3 A NEW CRITERION FOR MYOPIC OPTIMALITY IN MARKOV DECISION PROCESSES ................................ ................................ ................................ .......................... 27 Preliminaries ................................ ................................ ................................ ........................... 27 Methodological Development ................................ ................................ ................................ 29 Main Result ................................ ................................ ................................ ..................... 29 Proof of the Main Theorem ................................ ................................ ............................. 31 Application to Classical Inventory Models ................................ ................................ ............ 37 4 THE SINGLE PERIOD BLOOD INVENTORY ALLOCATION PROBLEM .................... 42 Problem Setup ................................ ................................ ................................ ......................... 42 Rotation Mod el ................................ ................................ ................................ ....................... 43 Retention Model ................................ ................................ ................................ ..................... 47 Hybrid Model ................................ ................................ ................................ .......................... 49 5 THE MULTI PERI OD BLOOD INVENTORY ALLOCATION PROBLEM ..................... 52 Application to the Rotation Model ................................ ................................ ......................... 52 Proof of the Existence of a Myopic Optimal Policy ................................ ........................ 52 Proof of the Myopic Optimality of a Particular Policy ................................ ................... 53 Application to the Retention Model ................................ ................................ ....................... 55 6 SIMULATION STUDY ................................ ................................ ................................ ......... 57 Simulation Model Setup ................................ ................................ ................................ ......... 57 Analysis of Empirical Data ................................ ................................ ................................ ..... 58 PAGE 6 6 Descriptive Statistics ................................ ................................ ................................ ....... 59 Input Estimation ................................ ................................ ................................ .............. 60 Simulation Results ................................ ................................ ................................ .................. 62 7 CONCLUSIONS ................................ ................................ ................................ .................... 74 Key Contributions ................................ ................................ ................................ ................... 74 Future Research ................................ ................................ ................................ ...................... 76 APPENDIX A FIGURES SUMMARIZING EMPIRICAL DATA ................................ ............................... 78 B TEST OF ANALYSIS OF VARIANCE ASSUMPTIONS ................................ ................... 80 LIST OF REFERENCES ................................ ................................ ................................ ............... 83 BIOGRAPHIC AL SKETCH ................................ ................................ ................................ ......... 87 PAGE 7 7 LIST OF TABLES Table page 6 1 Summary of the simulation results when Q/D = 0.76. (A) Average transshipment rates; (B) Average shortage Rates; (C) Average outdate rates ................................ .......... 65 6 2 Summary of the simulation results when Q/D = 1.0. A) Average transshipment rate; B) Average shortage rates; C) Average outdate rates. ................................ ....................... 66 6 3 Analysis of va riance table for the total number of units transshipped. .............................. 67 6 4 Analysis of variance table for the total number of units sent by e mergency shipment. .... 67 6 5 Analysis of variance table for the total number of units outdated. ................................ .... 67 PAGE 8 8 LIST OF FIGURES Figure page 1 1 Regional blood banking System. ................................ ................................ ....................... 19 1 2 Life cycle of whole blood products ................................ ................................ ................... 20 1 3 Blood importing. ................................ ................................ ................................ ................ 21 6 1 Mean daily demands. ................................ ................................ ................................ ......... 68 6 2 Comparison of mean aggregated demand to mean supply ................................ ................ 68 6 3 Impact of the extent of inventory rotation and the choice of allocation policy on the total nu mber o f transshipped units. A) m yopi c policy; B) i ntuitive policy. ...................... 69 6 4 Impact of the ratio of supply to demand on the number of units sent by emergency shipment under the myopic policy. ................................ ................................ .................... 70 6 5 Impact of the ratio L/T on the total number of outdated units under myopic policy. A) L/T = 0.1; B) L/T = 0.3. ................................ ................................ ................................ 71 6 6 Advantage of implementing the myopic policy in a less flexible system. A) in a rotation system, B) in a hybrid system, C) in a retention system. ................................ ..... 72 A 1 Daily fresh supply. ................................ ................................ ................................ ............. 78 A 2 Daily demand at depot 1. ................................ ................................ ................................ ... 78 A 3 Daily demand at depot 2. ................................ ................................ ................................ ... 79 A 4 Daily Demand at depot 3. ................................ ................................ ................................ .. 79 B 1 Test of assumptions of normality and homogeneity in variance for the total number of transshipped units. A) test of normality, B) test of homogeneity in v ariance. .............. 80 B 2 Test of assumptions of normality and homogeneity in variance for the total number of units sent by emergency shipment. A) test of normality, B) test of homogeneity in variance. ................................ ................................ ................................ ............................. 81 B 3 Test of assumptions of normality and homogeneity in va riance for the total number of outdated units. A) test of normality, B) test of homogeneity in variance. ..................... 82 PAGE 9 9 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy OPTIMAL ALLOCATION OF BLOOD INVENTORIES By Yan Qin August 2011 Chair: Anand Paul Major: Business Administration Blood banking problems studied in the operations management literature are usually the efficient allocation of blood products to several demand locations in a region. Owing to the limited shelf lives o f blood products, and the randomness in both supply and demand, multi period blood banking problems have remained relatively untouched in the past two decades, leaving some practical questions unanswered. In this work, we construct a multi period model of blood bank storage and distribution based on discussions with managers from a local non profit blood bank. Assuming stationary demand and supply distributions, we show th e existence of a myopic optimal policy over arbitrary finite horizons using some new M arkov Decision Processes machinery developed in this work. A Markov decision process with finite state and action sets is proved to have a myopic optimal policy over an arbitrary finite horizon if (1) the Markov decision process is ergodic, that is, if the Markov chain induced by each stationary policy is irreducible; or (2) the Markov decision process is strongly multi chain with no transient states. A simulation study is carried out to explore the impact of the extent of inventory rotation performed in t he system, the ratio of supply to demand, and the length of shelf life on system performance. Performance measures include : the total number of units transshipped during the PAGE 10 10 whole planning horizon, the total number of units sent by emergency shipment, a nd the total number of units outdated during the horizon Inputs such as the supply and demand distributions were estimated based on the empirical data obtained from the blood bank we studied at. It is observed that the ext ent of inventory rotation performed in the syste m may not be a crucial factor in minimizing the number of units to be transshipp ed under the myopic optimal policy identified in the work. The myopic optimal policy exhibits profound advantage for a product with a high supply to demand ratio an d short shelf life that is distributed in a less flexible system. Possible future research directions are pointed out in the last chapter of this work. PAGE 11 11 CHAPTER 1 INTRODUCTION Overview of Blood Banking A blood ce nter, which collects, tests, manufactures and transports a number of blood products, is a crucial link between blood donations and transfusions. According to the 2007 National Blood Collection and Utilization Survey (NBCUS) r eport, a total of 16,174,000 un its of Whole Blood/ Red Blood Cells were collected in the United States in 2006. Over 90% of the blood supply came from various local or regional blood centers. A typical regional blood center oversees a system consisting of a hub, several local depots, a nd a number of participating medical transfusion facilities as shown in Figure 1 1. The hub receives blood donations, produces blood component products, stocks and distributes the products to the local depots in the region. Each depot serves a bunch of med ical transfusion facilities, most of which are hospitals, in a way specified by the contract signed with each participating transfusion facility. The regional blood center typically replenishes the depot and hospital inventories to ensure the effective us e of the limited blood resources under certain realistic operational constraints. OM researchers have long recognized blood banking as a rich source of all kinds of forecasting, inventory management, and distribution problems. The AABB (American Associati on of Blood Banks) established three performance indicators for blood centers and medical facilities: (1) the outdate rate, measured as the ratio of the number of units outdated to the total number of units processed /produced in the same period ; (2) the fr equency of emergency blood shipments; and (3) the number of delays in scheduled elective surgeries. Blood components of medical interest include Red Blood Cells, plasma, and platelets. In 2006, 2.4% of Whole Blood/Red Blood Cells units and 2.0% of plasma u nits outdated. The outdate rates of whole blood derived platelets and apheresis platelets were 22.2% PAGE 12 12 and 10.9%, respectively, due to the short shelf life. The 2007 NB CUS r eport indicated a total of 6 .89% of hospitals surveyed postponed elective surgeries b ecause of blood shortage and a total of 13.5% of hospitals experienced inadequate nonsurgical blood supply. Blood shortage and outdating are chronic issues in blood banking practices Characteristics of Blood Banking Problems Great effort has been made in academia to help address the problems in managing blood inventories. Most blood banking problems are hard to model analytically because of the following characteristics of blood products. The supply of blood is random as it is driven by voluntary donati on. Blood centers usually schedule blood drives with organizations, develop promotions, and/or contact repeat donors to get more donations. However, the exact number of donations varies from period to period in spite of these efforts There are a number of blood products with various shelf lives in each product category. Blood component products can be divided into three general categories: Red Blood Cell products, plasma products and platelet products. Within each category, the products can be further di stinguished based on the anticoagulan t/preservative solution added, if any, the additi ve solution, and the processing procedure For example, a Red Blood Cell unit collected by apheresis can be leukoreduced with CP2D as the anticoagulant and AS 3 as the ad ditive. The anticoagulant/preservative and additive solutions are used to extend the shelf lives of the blood products. Red Blood Cell products with ACD/CPD/CP2D as anticoagulant have a shelf life of 21 days and additive solutions can extend the shelf life to 42 days. The production, inventory and distribution decisions need to take into account the various shelf lives of the products in the same category. The demand for each product category is usually random. Elective surgeries can be placed and canceled on a short notice at a hospital. And a natural or man made disaster can cause immediate and vast medical needs for blood. Physicians may have detailed requests regarding the freshness of the blood units they need as well as the type of preservative or addi tive solutions used Requests of this sort can further complicate the operations conducted at the contracted blood center. There may be hospitals of various sizes in a region. Some hospitals may have no more than 100 inpatient surgeries a year, while other s may perform over 2000 inpatient surgeries. Cross matching policies may differ from hospital to hospital. Cross matching is the process of testing and locating an appropriate blood unit in inventory against a blood request. The re can be two types of inven tories at a hospital, reserved and free. A unit PAGE 13 13 reserved through cross matching will either be u if unused after a certain length of time. The length of time after which an unused reserved is called the cross match release period. Each hospital may have its own cross match release period, which distribution decisions through demand forecasts. The ABO/Rh compatibility among blood products of different blood types varies in categories. For Red Blood Cell products, patients of blood type AB positive can receive transfusions of any blood types. Patients of blood type O negative, on the contrary, can only receive transfusions of the same blood type. For pl asma products, patients of blood type O pos itive or O negative are now universal recipients and can receive transfusions of any blood type. Plasma products of blood type O positive or O negative are not supposed to be transfused to patients of other blood types. The ABO/Rh compatibility is not always required when platelet products are to be transfused. Most of the existing literature on blood inventory management addresses at most three or four of the characteristics listed above. Interested in making our own contribution to the field of perishable inventory management, which includes blood inventory management as a branch, we studied with a regional non profit blood bank in the hope of formulating a problem that is relevant to practitioners and theoretical ly interesting as well. Current Industrial Practices The blood bank we studied with operates several regional blood centers located in different states. In each region, the hub collects, produces, and transports blood products to the local depots in that region. The depots in turn service the hospi tals At a hub, designated areas a re established for stocking unprocessed whole blood units freshly added to inventory, labeled blood products that a re ready to be transported, quarantined blood products, blood c omponents obtained from apheresis, and the disposed units. Whole blood units are tested rigorously before they are used to produce various blood component products. Production and inventory decisions are made based on the average weekly demands derived fro m historical data. Processes performed at a typical blood bank include donor recruiting, donor registration, blood collection, preparing, testing, labeling, and distribution. Figure 1 2 shows the complete life PAGE 14 14 cycl e of whole blood derived products at the blood bank we studied at Solid arrows in the figure are material flows. Dashed arrows are information flows. Donor recruiting : The staffs from Donor Recruitment d epartment are responsible for getting people to donate blood with them. They give speeches a t businesses to encourage groups to sponsor blood drives, schedule blood drives and contact repeat donors by phone to encourage them to donate again. Donor registration and collection: The blood bank collects blood at its center locations and at blood driv es in the community. The first step in the donation process is to obtain identification from the donor and then enter all of the donor related information into a computer system. Once the donor has registered, the collection process begins. It starts with an interview in which questions are asked regarding specialist then performs a brief. The results are recorded on a form called the Single Donation Record (SDR). This part of process takes about 30 to 45 minutes. Then blood is collected. The time of the actual donation depends on the type of the donation. There are two t ypes of donation, whole blood and apheresis. A whole blood donation is a standard donation, the actual donation of which only last s 4 to 8 minutes. An apheresis donation is the process of removing a specific component of the blood and returning the remaining components during the donation. It takes up to 2 ho urs to finish the donation. T he quantity of platelets collected through this process is equivalent to the quantity colle c ted from 8 to 10 whole blood donations. Preparing: epartment. CMP units based on the type of donation Whole blood units are separated into different components. U nits collected by PAGE 15 15 apheresis are ready for storage at this time (If a unit is over 6 hours past its collection time, Station 1 gives a warning, since some com ponents must be processed or refrigerated within 8 hours.) S taffs then enter all component products stock them in the unprocessed/non inventory storage area for labeling. The operations support specialists at the h ub scan Single Donation Records i nto the system for review by corporate operations s taff. Sample tubes from each unit are on their way to the Donor Testing Lab at the same time For apheresis platelet products, a product sample a nd an extra donor tube are sent to the Reference and Quality Control Lab (RQL). Testing : CMP staffs at regional centers also test for bacteria on apheresis platelet products drawn over 24 hours ago. Upon receiving the sample tubes, staffs in the Donor Testing Lab (DTL) enter the sa mples into a computer program, which helps track if all tubes are sent / received. Once the tubes are entered and prepped, DTL staffs begin testing. Tests conducted at DTL include ABO/Rh, antibody screen, Hepatitis, HIV West Nile Virus, and etc. DTL staff s notify the regional CMP staff to begin the labeling process after they complete the lab run and post the results. The test of the apheresis product samples sent to the Reference and Quality Control Lab for platelet count is conducted simultaneously with the test s in the Donor Testing Lab. RQL staffs also enter the test results into a computer program to share with the regional CMP staffs. CMP staffs split the apheresis platelet products if RQL platelet count results indicate that these products are qualif ied to do so. Labeling: CMP staffs begin to quarantine initial reactive units and discard repeat re actives after they receive the test results from the Donor Testing Lab Products are then labeled in the following order: (1) Platelets ; (2) Liquid Product s (RBCs and plasma); and ( 3) Frozen PAGE 16 16 Plasma Products. The labeled products are moved to the processed/inventory storage area Products that fail the tests are placed in s pecial t the processed/inventory storage area are now ready to be transported to a customer. Distribution: The Resource Management (RM) d epartment oversees the storage and distribution of blood products. RM staffs gather order and inventory i nformation from hospitals, fill orders and evaluate hospital inventory levels to determine if short date d units need to be rotated The blood center expect s daily updated inventory information from the hospitals by fax or phone. RM staffs also make daily production decisions early in the morning based on the current inventory status at the hub and their supply forecasts for today and tomorrow. Efficient communication between different regions and blood centers is required to see if products need to be tra nsferred or imported and from where Typical blood importing processes are given in Figure 1 3. Hospitals serviced by the blood bank are supposed to report their inventory levels every day to help the regional center make inventory and allocation decisions Daily demands are satisfied on consignment. The FIFO (First In First Out) issuing policy is exerted whenever possible in the distribution system Orders for Red Blood Cell units from hospitals are filled with short dates available when possible. Red Bloo d Cell units that have less than 21 days of shelf life left are considered as short dates. Some hospitals do request units no older than a ce rtain age category though. O rders of plasma units are also satisfied with the oldest units available The productio n and inventory decisions for platelet products are usually more challenging as they are good for only five days. PAGE 17 17 The blood bank we studied at rotates short dated units among hospitals if necessary R egional center s collect inventory information from h ospi tals it services every day and decide whether to replace short dated units in the hospital inventories with fresh ones. If a region is not collecti ng enough units, they allow short dates to stay at hospitals. Otherwise, short dates are sent back to the cor responding regional center first and shipped to high traffic hospitals afterward to reduce the risk of outdating. This particular practice indicates the conditional application of the rotation /retention policy. When the rotation policy is implemented, all the at the end of a period. Under the retention policy, units stay in the depot inventory until us ed or outdated. The two policies were first described in Prastacos (1978). A local depot seeks help from other depots in the same region if out of stock Transshipment between local depots in the same region is performed if necessary. If a region is short of supply, it turns to other regions operated by the same blood bank or other blood cent ers. Importing blood from another blood center can be very costly. Prices are much higher than the production costs. T he blood bank sometimes has to buy unwanted products in order to get what it needs. For example, a blood bank may end up purchasing three boxes of A positive products as well as the two boxes of O negative products it needs. Through the extensive meetings we had with the managers at the blood bank, current challenges in the blood banking industry include: No serious statistical work is incorporated in making production and inventory decisions units on their own shelves. As the blood bank offers full credit for most out dated units and commits to a 100% service level, over ordering has b ecome a serious problem considering the number of outdates it creates. The managers are interested in learning the efficient inventory allocation strategy. PAGE 18 18 We decide to address the second challenge by first formul ating a multi period inventory storage and distribution problem. Product perishability is considered and t ransshipment is enabled under the rotation/ retention /hybrid policy in this work. In the hybrid case, only units younger than a certain age are rotated Th e objective is to minimize the total expected costs incurred by transshipment, emergency shipment, and outdating over the whole planning horizon Decisions are evaluated from the perspective of a regional blood center. We hope this work can serve as a nice starting point for a series of meaningful work on modeling realistic blood banking problems The rest of the work is organized as follows. We review the relevant literature in Chapter 2 and derive a new criterion for myopic optimality in Markov decis ion p rocesses in Chapter 3. Three single period models are formulated in Chapter 4 featuring different extents of inventory rotation Optimal policies for each of the three single period models are identified. We call those policies myopically optimal as they may not be optimal for the multi period versions of the models. The application of our new myopic optimality criterion to the multi period problems is discussed in Chapter 5. A particular myopically optimal policy is shown to optimal for the multi per iod allocation problems we formulated over any finite horizon. Simulation results are then analyzed in Chapter 6 to provide meaning ful managerial insights regarding the impact of the extent of inventory rotation, the ratio of supply to demand, the length of shelf life, and the choice of allocation policies on system performance Key contributions and possible future research directions c an be found in Chapter 7. PAGE 19 19 Figure 1 1. Regional blood banking System. PAGE 20 20 Figure 1 2. Life cycle of whole blood products PAGE 21 21 Figure 1 3. Blood importing. PAGE 22 22 CHAPTER 2 LITERATURE REVIEW There has been extensive literature on perishable inventory management which includes blood inventory management as a special sub area, since the early 60s in the last century. Topics that have attracted most attention in this field include the derivation approximation, and evaluation of vari ous inventory ordering and issuing policies at a single location level and the allocation policies at a regional level. Analytical models established so far in the field of perishable inventory management c an be cate gorized based on researcher assumptions regarding demand, supply, shelf life the perishability pattern of the product(s) under consideration, review frequency and the length of planning horizon. Demand, supply, and the length of shelf life of a product c an be assumed deterministic or stochastic. Units can be assumed to perish on a step function basis or continuously according to some expon ential function. Inventory status is reviewed either periodically or continuously in a model. Most problems addressed so far on perishable inventory management seek to optimize a certain objective over a single period due to modeling complexities resulted from perishability. Papers focusing on the derivation and approximation of inventory ordering policies for perishable products usually assume a fixed product shelf life and instantaneous replenishments. Inventory levels are sometimes modeled as Markov chains. Relevant literature includes Jennings (1973), Fries (1975), Nahmias (1975), Brodheim, Derman and Prastacos (1975) Chazan and Gal (1977), Weiss (1980), Schmidt and Nahmias (1985), Brodheim, Hirsch and Prastacos (1976), Cohen and Pierskalla (1978), and etc. Jennings (1973) evaluates the performance of several inventory ordering policies for whole blood units at both a single hospital level and a regional level using simulation. The degree of PAGE 23 23 interaction among hospitals is measured in terms of the timings and sizes of transshipment. Given identical inventory control policies at the hospitals in a regional system, variou s degrees of interaction are considered in the paper. Brodheim, Derman and Prastacos (1975) model on hand inventory levels at the end of each period as a Markov chain and evaluate the performance of a group of inventory policies from the perspective of a single depot. The Markovian modeling approach is justified by the assumptions of a fixed order quantity and a known age distribution for any on hand inventories. Chazan and Gal (1977) derive the upper and lower bounds of the expected daily outdating rate i n an inventory system operated under a fixed critical number ordering policy and a FIFO issuing policy. Both independent and identically distributed demands and periodic demands are considered. Weiss (1980) considers a continuous review model. The demand f or the perishable product is modeled as a Poisson process. A FIFO issuing policy is assumed in both the lost sales and backlogging cases. Liu and Lian (1999) study a similar problem by modeling demand as a general renewal process. Cohen and Pierskalla (19 78) extend the analysis in Brodheim, Hirsch and Prastacos (1976). The authors came up with a simple inventory decision rule through numerical analysis to minimize the averages of the daily shortage and outdating costs at a hospital blood bank. The practice of cross matching is taken into account in this model. It is recommended that hospitals with lower daily demands receive fresher units for better cost performanc e. The literature on blood inventory management usually assume s random demand and supply, fixed shelf life, and a step wise perishability pattern. Both continuous and periodic review models can be applied. T opics on efficient blood inventory management attracted PAGE 24 24 significant interest in the OR/MS field in the 70s and 80s and have remained relatively untouched ever since. According to Pierskalla (2005), OR researchers left this area in the 80s because of the lack of federal funding and the increasing complexity of the remaining problems. The multi period blood inventory allocation problem formulated in this dissertation work is of significant practical interest as blood centers nowadays are faced with great cost reduction pressures. Prastacos (1978, 1981) are the most relevant papers to this work. Prastacos (1978) consid ers a distribution system consisting of a single warehouse center and a number of local depots for a product that perishes on a step function basis, such as blood. The warehouse center receives a random quantity of fresh units at the beginning of a period and decides the size and age composition of each allocation Myopically optimal policies are derived under the objective of minimizing the expected shortage and outdating costs in both the rotation and retention situations It is shown in Prastacos (1981) that the myopically optimal policy should not be very different from an optimal policy over an infinite horizon in the rotation case. Transshipment between depots is not allowed in Prastacos's setup, while it is an important structural feature of our model As Prastacos's work is limited to the single period and infinite horizon problems, one of our major contributio ns in this paper is that we prove the optimality of some myop ically optimal policy over arbitrary finite horizon s under some general assumption s on demand and supply. Those assumptions, to be stated later, can be easily satisfied in practice. Brodhe im and Prastacos (1979) present the details of the design and implementation of a decision support system, a.k.a Programmed Blood Distribution System (PBDS), aimed to improve the efficiency of regional blood management in Long Island, New York. As in Prastacos (1978, 1981), the paper considers a regional blood center that services a number of Hospital PAGE 25 25 Blood Banks (HBBs) in the region. Unlike what is as sumed in Prastacos (1978 1981 ) where all blood units are either rotation or retention units, a fixed number of rotation units together with a fixed amount of retention units are delivered to each HBB on each delivery day The performance o f the system is measured by the product availability and utilization rates at each HBB. The availability rate is defined as the percentage of days in which no emergency shipment is performed and the utilization rate r efers to the ratio of the number of units transfused to the total number of units available at a HBB. The pool of desirable distribution policies is limited to the ones that equalize the availability and utilization rates across the HBBs as the myopically optimal policy derived in Prastacos (1978). Simulation results indicate that a policy that minimizes the number of rotation units required to achieve desirable availability and utilization rates optimizes the two performance measures in this particular design. The mathematical programming model that serves as the basis of PBDS is given in Prastacos and Brodheim (1980). The model seeks to minimize the total number of rotation units subject to several operational constraints. Nose, Ishii and Nishida (1983) extends the single period rotation model constructed in Prastacos (1978) by adding a linear cost term for transportation in the objective function. O ptimal allocations are shown to at least partly depend on the location wise cost parameters The same model with fuz zy unit shortage cost is studied in Katagiri and Ishii (2000). Federgruen, Prastaco s and Zipkin (1986) investigate an integrated single period inventory allocation and distribution problem for a perishable product. Two delivery patterns are considered. The model resembles the one in Nose, Ishii and Nishida (1983) when individual deliveries are made When units are delivered by a fleet of vehicles making multiple stops, the PAGE 26 26 model involves solving a complica ted Vehicle Routing problem. C omputational algorithm s are provided in both cases The last stre am of literature on perishable inventory management we would like to discuss focuses on the derivation of op timality conditions for issuing policies. The two most common issuing policies are FIFO (First In, First Out) and LIFO (Last In, First Out). A FIFO issuing policy is assumed in our models, that is, we always use the oldest units in stock to satisfy incoming orders. Most of the early work in this field assumes known demand and supply. Pierskalla and Roach (1972) is probably one of the earliest papers in this field that considers product perishability together with the randomness in supply and demand Given independent stochastic supply and demand processes, the paper shows the optimality of the FIFO policy under two objectives: the objective of minimiz ing the total stock outs and the objective of minimizing the total outdates. We will not dev ote too much space to this topic since a FIFO issuing policy is assumed in our models, that is, a depot always uses the oldest unit available in stock to satisfy a demand unit. This is a pretty common assumption in the perishable inventory literature. For a detailed review of the blood banking literature published before the 80s, please refer to Nahmias (1982) and Prastacos (1984) Baafat (1991) is a survey paper fo cusing on continuously deteriorating inventory models. Goyal and Giri (2001) review the perishable inventory management literature on ordering and issuing policies published since the 90s. PAGE 27 27 CHAPTER 3 A NEW CRITERION FOR MYOPIC OPTIMALITY IN MARKOV DECISION PROCESSES A detailed description of the set up of the multi period allocation problem will be given in Chapter 4. Three single period models will be constructed, featuring different extents of inventory ro tation. Optimal policies for the single period models are called myopically optimal as those policies may not be optimal over a multi period finite horizon. To identify/construct an optimal policy for the multi period allocation problems, one conjecture is that some myopically optimal policy is optimal over finite horizons. However, with product perishability and the randomness in supply and demand, it is not straightforward to see the existence of such a policy. We therefore would like to apply some myopic optimality criterion to the allocation problems and identify/construct such a myopically optimal policy if exists. We end up deriving a new criterion for myopic optimality in stationary Markov decision processes in this chapter. Preliminaries A Markov de cision process is said to have a myopic optimal solution over a finite or infinite horizon if there is an optimal solution that consists of a stationary policy that simply repeats an optimal one period solution to the problem. In the parlance of the theory of algorithms, a problem with a myopic optimal solution is one that can be solved by a greedy procedure. Myopic optimality is a very useful property to uncover in a multi period problem. It greatly eases the complexity of computing the optimal solution, a nd it helps to uncover structural properties of the problem. Generally multi period finite horizon problems do not have myopic optimal solutions, so it is of considerable interest to find classes of problems that do have this property. PAGE 28 28 We provide a brief review of past research on myopic optimality of finite horizon Markov decision processes (MDPs). Early work in the mathematical inventory theory literature established the myopic optimality of order up to policies in the classical Arrow Harris Marschak mod el, such as Karlin (1960), Veinott (1965), Ignall and Veinott (1969), Johnson and Thompson (1975), Lovejoy (1990), Cetinkaya and Parlar (1998), Khang and Fujiwara (2000), Wang (2001). Rosenfield (1992) and Prastacos (1981) investigate myopic optimality in specific perishable inventory models. We provi de a supply chain application following the proof of the new myopic optimality criterion. Some recent work in multi period stochastic inventory models that is relevant to our modeling application includes Huh a nd Janakiraman (2008a, 2008b), and Huh et al. (2009). Nerlove and Arrow (1982) and Dhrymes (1962) study myopic optimality in the context of an advertising model, and Lamond and Sobel (1995) in the context of reservoir operations. Sobel (1981), to our best knowledge, presents the first set of sufficient myopic optimality conditions for a general dynamic decision process. A finite discrete time Markov decision process is shown to have a myopic solution if (1) state and action can be decomposed additively in the single period reward function; (2) transition probabilities are independent of the starting state and depend only on the action taken and the destination state; (3) if a myopic optimal action is taken at some stage, it is feasible at the next stage, at every stage of the planning horizon. An amendment to the corollary in this paper is provided in Kochel (1985). New sufficient conditions are provided in Sobel (1990a) for a Markov decision process but with the limitation that the reward function can depen d only linearly on the current state. Results in Sobel (1990a) are extended in Sobel (1990b) to affine models in which rewards and state dynamics depend linearly on previous states and actions as well as the current state. Sobel and PAGE 29 29 Wei (2010) derive an al ternative set of sufficient myopic optimality conditions for a finite Markov decision process in which the expected state dynamics can be decomposed additively or multiplicatively from the reward. A restriction on the structure of the transition probabilit ies is required to ensure the availability of a myopically optimal action in a subsequent period if a myopically optimal action is taken in the current period. In this chapter we present a new criterion for myopic optimality in stationary Markov decision processes with finite state and action sets, one that is intuitive and, as we demonstrate, quite easy to apply in some supply chain models. We state our main result, prove it, and present a modeling application. The application is to a prototypical multi p eriod stochastic inventory management problem. Under some mild restrictions on the demand distribution, we show that there is a myopic optimal policy in the space of order up to policies. Methodological Development Main Result We consider a stationary Mark ov decision process (MDP) with finite state and action sets, with a non negative and bounded return from every state action pair. The objective is to maximize the total expected return (or equivalently, minimize total expected cost) over an arbitrary finit e planning horizon. It is a standard result that when the state and action sets are finite and the return function is bounded above, there is a stationary optimal policy over the infinite horizon. However, a stationary policy is not generally optimal over a finite horizon. In this paper, we single out a class of problems for which a stationary policy is optimal over any finite horizon. It is clear that such a stationary policy is in principle relatively easy to compute, since it must maximize the expected return in a single period. PAGE 30 30 A finite MDP is said to be ergodic if the Markov chain induced by each stationary policy is irreducible. An MDP is said to be multi chain if the Markov chain induced by some stationary policy consists of recurrent classes, possib ly with one or more transient states (See Puterman (2005), page 348). Our main theorem holds for a class of finite MDPs that is larger than the class of ergodic MDPs but smaller than the class of multi chain MDPs. To state our result concisely, we define a n MDP to be strongly multi chain if the Markov chain induced by each stationary policy over the infinite horizon consists of recurrent classes, possibly with one or more transient states. Theorem 3 1: Consider a stationary MDP with finite state and action sets, in which the objective is to maximize the total expected return or minimize total expected cost over a finite horizon. Then the MDP has a myopic optimal policy under either of the following conditions: The MDP is ergodic; The MDP is strongly multi chain with no transient states. Note that Theorem 3 1 applies only to a Markov decision process in which end of horizon effects are either absent, or may be accounted for in a fashion that transforms a nominally non homogeneous process into an equivalent h omogeneous one. For instance, in the supply chain application discussed later, we assume that product left over at the end of the horizon may be salvaged at cost. The non zero salvage value makes the model non homogeneous but a transformation to an equival ent homogeneous model can easily be effected; we refer to Heyman and Sobel (1984), page 79, for details of the transformation. In the pioneering paper by Sobel (1981) and in subsequent works (Sobel (1990a, 1990b), Sobel and Wei (2010)) conditions for myopic optimality are found that apply to both stationary and non stationary MDPs, with both discounted and undiscounted returns. These results therefore apply to a much wider test bed of MDPs than ours. On the other hand, within the PAGE 31 31 narrower and more tractable class of stationary MDPs, our main theorem gives a parsimonious criterion for myopic optimality, one that perhaps has some theoretical interest in its own right, and may be useful in models where the conditions derive d in the published literature may not hold, or may be hard to verify. Proof of the Main Theorem We now describe the proof of Theorem 3 1. Before we plunge into the details, let us give a thumbnail sketch of the strategy of the proof. We use the fact that in a finite MDP with discounted returns, there is an optimal stationary policy over the infinite horizon, and therefore a nearly optimal stationary policy over a sufficiently large finite horizon. We then make prove that if a stationary policy in an ergodi c finite MDP is nearly optimal over an n period horizon, it is nearly optimal in a manner that we precisely define over an ( n 1) period horizon. A finite sequence of such backward induction moves allows us to extend this result to a K period horizon, w here K is an arbitrary positive integer, provided we control the discount factor appropriately. We now give the technical details of the proof. We focus on the proof of the first part of Theorem 3 1; the proof of the second part of the main theorem proceed s in an almost identical fashion. Let K be an arbitrary positive integer, which we shall take to be the number of periods in the finite horizon. We embed the finite horizon problem in an infinite horizon and introduce a discount factor This, This, together with the assumption of uniformly bounded returns in every period, ensures that the expected discounted return over the infinite horizon is finite, regardless of the policy that is followed. We denote the total expected discounted return from a p olicy by V ( ). A policy is called optimal if given an arbitrary > 0, the total expected discounted return V ( ) obtained PAGE 32 32 by following satisfies the inequality for all possible It is well known that a finite state, finit e action discounted MDPs has a stationary optimal policy over the infinite horizon. We shall use the notation to denote a stationary policy that is optimal over an infinite horizon. It is a standard fact that for every > 0 and an initial state there exists an integer such that is optimal over every m period horizon where (Shapiro (1968), Corollary 3). Define is well defined due to the finiteness of the state set S Lemma 3 1 is an adaptation of the afore mentioned fact. Lemma 3 1: Let > 0 be given, howsoever small. Then there exists a positive integer such that is optimal over every m period horizon, where regardless of the initial state. A Markov chai n has a stationary probability distribution if there is a probability vector such that where is the transition probability matrix of the Markov chain and the sum of the elements of is 1. An irreducible finite Markov chain has a strictly p ositive stationary probability distribution (Levin et al. (2008), Proposition 1 14). This fact, together with the following simple lemma, plays a key role in our proof. Lemma 3 2 : Let be any probability distribution assigning a strictly positive probabi lity to every state in the state set S. Suppose a policy is optimal when the start of the first period is characterized by . Then is optimal when the initial state is s, for Proof: The expected return from is a weighted average of the expected returns starting from each state If is not optimal starting from some initial state, then the weighted average of its expected returns over all the states will be strictly smaller than that of an optim al policy, since an optimal policy is optimal regardless of the initial state and a suboptimal PAGE 33 33 policy yields a strictly smaller return than an optimal policy starting from at least one state. Q.E.D Remark: The requirement that assign a strictly positive probability to every state is vital. The lemma does not hold if the probability assigned to some state is zero. We note that a finite Markov chain with transient states has a stationary probability distribution that attaches zero probability to each transi ent state. Next, we state and prove a backward induction result. Lemma 3 3: Suppose we are given a finite ergodic MDP. Let be an arbitrary positive number and let a discount factor be given. Suppose the initial state is the stationary distribution induced by Then if is optimal over a horizon consisting of periods through it is optimal over periods through Proof: It follows from the assump tion that the transition probability matrix induced by is irreducible that the associated Markov chain has a unique stationary probability distribution attaching a strictly positive probability to every state. Suppose the start of the first peri od is characterized by precisely the probability distribution over all the states Then the state at the beginning of the second period is also characterized by the probability distribution over all the states in S The same can be said about the state at the start of any period. Let > 0 be given. By Lemma 3 1, there exists a positive integer n such that the policy is optimal over an n period horizon for all initial states, and therefore for all probability distributions over initial states. Let the n period horizon be Let the expected return of following in followed by NOP over be and l et the expected return of following over be Let the optimal expected return over with the probability distribution as the initial state distribution be Let E be the expected return in the first period PAGE 34 34 in following either policy, and let and be the expected return of following and NOP respectively, over as evaluated at the beginning of Then we have (3 1) (3 2) Supp ose that is not optimal over the horizon Then which implies that since But this implies that optimal over a contradiction. Q.E.D B y repeating the argument in the proof of Lemma 3 3 finitely many times, we infer that if optimal over a horizon consisting of periods through it is optimal over the K period horizon through provided the init ial state is the stationary probability distribution. Let denote the optimal return over periods through and let denote the return of over periods through given . We want to show that for so me positive integer we have (3 3 ) from which it will follow that (3 4 ) which will prove the optimality of the stationary policy over a K period horizon, since we We note that and both depend on the underlying discount rate, but to minimize clutter we do not show this dependence using notat ion. Observe that Inequality 3 3 does not f ollow directly from Lemma 3 1 because both sides of the inequality depend on n e PAGE 35 35 shall show that Inequality 3 3 does hold for some n close to 1. MDPs with discount factors in a neighborhood of 1 were analyzed first by Blackwell (1962), and subsequently by Shapiro (1968). We shall use a theorem of Blackwell (1962) in the last step of the proof. (3 5 ) Then Inequ ality 3 4 reduces to (3 6 ) Consider the inequality (3 7 ) It follows from Lemma 3 there is a positive integer such that Inequality 3 7 holds for all n are linked through Equati on 3 5 ; so it is possible that an integer induce s through Equation 3 5 that is different from from below, as we shall explain. It is clear from Equation 3 5 that as appr oaches 1 from below, n induced by Equation 3 5 is larger than m for any given positive integer m In particular, we can that the value of n it determine s is greater than However, the left hand side of Inequality 3 7 is based on rather than so we cannot claim that Inequality 3 7 holds. But we shall exploit the idea that when and are clos e enough, then if Inequality 3 7 holds for I nequality 3 6 holds for since is, for each value of n PAGE 36 36 We construct a sequence such that as we have For each we record the smallest integer such that Inequality 3 7 holds with discount rate We also note which is obtained by setting in Equation 3 5 and taking the integer part of the resulting value. As it is clear from Equation 3 5 that We generate a new sequence by amending as follows: for each i if we write ; otherwise we set Since for each i Inequality 3 7 holds for all with a discount factor of it also holds for all Also, since we have that We then generate another sequence of discount factors by defining to be the solution of Equation 3 5 for each value of Now given any there is an integer so large that we have We choose a sufficiently small value of such that when Inequality 3 7 holds for some sufficiently large and an underlying discount factor of we have that Inequality 3 6 holds for the same value and an underlying discount factor of since n This is the case for for all ; that is, the foregoing argument applies to all discount factors sufficiently close to 1. We remark that policy over the infinite horizon (See Blackwell (1962), Theorem 5). This fact is necessary for the continuity argument in the last step to work, since it assures us that we are dealing with the that for all discount factors sufficiently clos e to 1, Inequality 3 3 holds. Hence, there is a myopic optimal policy ( ) over a finite horizon provided the total expected return is discounted by a discount factor sufficiently close to 1 It follows that is PAGE 37 37 optimal over a finite horizon wh en the objective is to maximize total expected return. To see this, we note that the objective of maximizing the expected return corresponds to a discount factor equal to 1 over a finite horizon. Now any policy with an expected return over the finite horiz on that is strictly greater than that of at must, by the continuity of the expected return of any policy as a function of have an expected return over the finite horizon that is strictly greater than that of in some neighborhood of a contradiction. We have assumed throughout the analysis that the initial state is the stationary distribution associated with the Markov chain induced by a stationary optimal policy over the infinite horizon, but by Lemma 3 2 there is no loss o f generality in assuming this. This completes the proof of the first part of Theorem 3 1. Note that in the last period of a finite horizon, every optimal policy must follow the myopic (one period) optimal policy. Hence an optimal stationary policy must nec essarily follow some myopic optimal policy in each period (otherwise it would be non stationary). The proof of the second part of Theorem 3 1 follows by applying the foregoing argument to each irreducible class of the induced Markov chain separately. Sup pose the Markov chain induced by has closed communicating classes of recurrent states and no transient states. Given that the initial state is chosen from class there is a unique stationary distribution with strictly positiv e probabilities for each state in for To prove the second part of Theorem 3 1, therefore, we need only repeat the proof for the first part of Theorem 3 1 separately to each of the closed classes. Application to Classical Inventory Mod els In this subsection, we discuss the applicability of our m yopic optimality conditions to a classical multi period stochastic inventory model. The setup is as follows. We have a discrete PAGE 38 38 finite horizon of length N with independent and identically distributed demand for a single product. At the start of each period, a decision is made to order a certain number of units of the product at a cost of c per unit. The replenishment lead time is a positive integral number of periods, At the end of each period, we account for the cost of inventory holding ( h per unit per unit time) and shortage ( b per unit) during the period. Any inventory remaining at the end of the horizon may be disposed of at cost. The problem is to d etermine the optimal ordering policy at the start of each period so as to minimize the total expected cost over the horizon. We first assume that unmet demand is lost (that is, backorders are not permitted), an assumption that holds in the case of retail sales of many products. In this case, the optimal infinite horizon policy cannot be easily characterized, although recent work by Huh et al. (2009) shows that order up to policies are average case optimal over an infinite horizon provided shortage costs do minate inventory holding costs, in a precisely defined sense. However, we shall show, by applying Theorem 3 1, that if we consider only the space of order up to policies, then the problem has a myopic optimal order up to policy over any finite horizon prov ided the initial inventory is zero. Let us consider a decision process in the subspace of order up to policies. Assume that demand is independent and identically distributed over time, which follows a discrete distribution over all the non negative integer values with the probability mass function for each possible realization. Let denote an exogenous upper limit on the order up to level in any period (we can allow to be as large as we please when we construct the model, but we must commi t to a finite number once the model is set up). The actions available in each feasible state are restricted to allow only the order up to decision rules in a period. That is, each period t is associated with a fixed critical number such that if the on hand inventory level is and the PAGE 39 39 number of units in delivery is at the start of period t a replenishment order is placed for units. This rule limits the set of actions that may be taken at a given state in a given period. The on hand inventory level is counted after a previous order, if any, arrives at the start of period t The state of the system is characterized by the vector We set for non positive int egral values of t to have a well defined model. The non positive indices are considered to achieve symmetric state vectors over time and have no impact on the performance of the system. We assume that A state is feasible as long as or as The objective is to minimize the total expected discounted cost. Under these conditions, it is clear that the decision process under consideration is a Markov decision process and that there exists a stationary optimal policy over the infinite horizon, for any discount factor. To apply Theorem 3 1, we wish to show that the MDP defined above is ergodic. Consider the finite Markov chain associated with a stationary order up to policy with the critical value A state is feasible in this Markov chain if As an order of subsequent period, the number of units to order in period t is equal to the number of units consumed in period ( t 1), that is, for as and So no order is placed in period 2 through no matter how demand realizes ll show next that any feasible state in this Markov chain is accessible from the state and vice versa. PAGE 40 40 As any feasible state accessed from the initial state P is strictly positive as for any possible demand realization. Hence, any feasible state in this Markov chain is accessible from the state To show the converse, consider the situa tion where no demand occurs in period through period 1) As the initial state in period must be We thus have shown that any feasible state has a positive probability of transiting to the state in a finite number of periods. It then follows that the finite Markov chain induced by the stationary order up to policy with critical value y is irreducible. As this property holds for any stationary order up to policy, the Markov decision process under consideration is ergodic. We may now invoke Theorem 3 1 and infer that there is a myopic order up to policy that minimizes the total expected cost over any finite horizon. The same result extends to the backlogging case, provided we set an exogenous upper bound on the total number of backlogged units permitted to ensure that the model remains a finite MDP. The perishable case: We conclude with an interesting variation of the above model in which our main theorem does not apply; this is the case when the product has a lifetime of L periods. Previous work (Nahmias (1975), Fries (1975)) has studied this problem in the setup of a f inite horizon with discounted back logging or lost sales costs in the sub space of order up to policies. Fries (1975) considers a discounted lost sales situation where remaining stock at the end of the horizon is ignored in the cost function. The order or not decision in each period is shown to be determined by a unique critical number. When an order needs to be placed, the optimal order up to level is jointly determined by the initial on hand inventory level and its age PAGE 41 41 distribution except in the last peri od of a finite horizon. Nahmias (1975) obtained similar results in the discounted model with complete backlogging where remaining stock can be salvaged at $c$ per unit at the end of the planning horizon. Readers are referred to Nahmias (1982) for a review. As in the non perishable case, we would like to consider only order up to policies. Assume that demand is again independent and identically distributed over time. We know that there exists a stationary optimal order up to policy over the infinite h orizon. But since the order up to level in the perishable case may well be jointly determined by the quantity of the initial stock in a period and its age distribution, even a stationary order up to policy may have varying order up to levels over time This implies that the Markov chain induced by a stationary order up to policy may have transient states and Theorem 3 1 does not apply in this case. PAGE 42 42 CHAPTER 4 THE SINGLE PERIOD BLOOD INVENTORY ALLO CATION PROBLEM Problem Setup We consider a dist ribution network consisting of one central warehouse and n local depots for a product with a fixed shelf life of L periods. Assume that the product perishes on a step function basis. This assumption restricts the number of possible age categories of the pr oduct to L ranging from 0, the freshest, to ( L 1), the oldest. Outdated units have no salvage value and must be disposed at a cost. These above mentioned assumptions do apply to most blood products. The sequence of events can be described as follows: At the beginning of each period, the central warehouse receives a random supply of fresh units and allocates the fresh units together with its initial inventory, if any, to the n depots. Demand then occurs. Each depot starts to fulfill the incoming orders using its local stock, which includes a depot's initial inventory in the period, if any, and the units just received. Transshipment is performed if necessary among the n depots during the period. When there is unmet demand in the system after transshipment the central warehouse imports units from expensive outside sources. At the end of the period, all the units left in the distribution system advance in age by one. Outdated unit s are disposed. All the units left at the depots that are no older than p eriods are transported back to the central warehouse for reallocation in the next period. The remaining units stay at the current location. The objective is to minimize the total expected transshipment cost plus the emergency shipment and outdating costs d uring the whole planning h orizon. We assume that the unit transshipment cost is lower than the unit emergency shipment cost. A First In First Out (FIFO) issuing policy is adopted at each of the n depots. As discussed in the literature review section, a FIF O issuing policy can be a satisfactory c hoice for perishable products. Random s upply and demand are both independent and identically distributed over time. Let denote the random supply in period t and denote the stochastic demand at depot k in period t We assume that PAGE 43 43 is defined over the integer values within with the probability mass function for all possible realizations, where is the maximum amount of supply that can arrive in a period. Demand follows some discr ete distribution defined over the integer values within with the probability mass function for each demand realization. Demand is independent across depots. As there are more than two depots in the model, a transshipment rule needs to be spec ified. We shall use the following transshipment rule in the current setting. A depot experiencing unsatisfied demand in a period seeks transshipment from other depots one by one in the ascending order of the depots' indices until enough units are received or there are no more units left in the system. A depot with units left after satisfying its own demand always fulfills the transshipment requests from a lower indexed depot before those from a higher indexed depot. This transshipment rule applies in vario us transshipment situations since it imposes no restriction on the way the depots be indexed. The depots can be indexed based on the mean demands, the distances from the central warehouse, o r some other priorities. If depots are replaced with hospitals, th e priority list can also be specified by the contracts signed with the blood center. Assumi ng given depot indices, we follow this particular transshipmen t rule in both the rotation, retention and hybrid models t o be formulated in the following subsections Rotation Model Set Let denote the number of units of age j allocated to depot k in period t where in the rotation case and in the retention case. Let denote the number of units to be transshipped from d epot l to depot m in period t for and We will give the exact expression of later in this subsection. For now, we would like to use the notation to address the relationship that the total number of units to be PAGE 44 44 transshipped in a period equals the difference in the total number of units left in the system before and after transshipment, that is We denote c as the unit transshipment cost, b as the unit emergency shipment cost, and w as the unit outdating cost. Then when a single period, say period t is considered, the objective of minimizing the expected sum of the transshipment, emergency shipment and out dating costs can be formulated as follows. The not ation refers to the number of units of age j initially available at the warehouse in period t Note that the first constraint in the model is always binding at optimality, that is, a myopically optimal policy for the model must empty the stock at the central warehouse after allocation. Let denote the total number of units allocated to depot k in period t Let denote the demand realization at depot k in period t Given a feasible allocation the second term in the expression of disappears when An additional unit at depot l that is about to outdate in period t improves the objective value by ( b w ) if and ( c b w ) if where as by assumption. Similarly, given a feasible allocation such that PAGE 45 45 the term representing emergency shipment cost disappears from the objective function. An additional unit at depot l that is about to outdate in period t reduces the single period objective value by ( c +w ) if and w if it is used at another depot. It induces no change in the objective value if outdated Combining the two observations, we can conclude that i t is always favorable to allocate as many units as possible from the central warehouse. The single period rotation model can then be expressed as With the new formulation, it is clear that the number of units to be imported in period t is fixed for a given A policy is therefore, myopically optimal if it minimizes the number of units to be transshipped plus the number of units to be outdated in a period. Moreover, as the objective of mini mizing the total number of units to be transshipped is equivalent to the objective of maximizing the expected number of demand units to be satisfied locally before transshipment. With a little calculus, we can show that a policy minimizes the expected numb er of units to be transshipped in a period as long as it maximizes the probability of each allocated unit being used myopic policy, derived in Prastacos (1978), tha t allocates units one by one from the oldest to the PAGE 46 46 unit during the period. It is worth noting that there are a number of myopically optimal policies for the rotation model as the model imposes no restriction on the age composition of the allocations in a period. Prastacos's myopic policy is just one of a many. We rem ark that when m periods are considered, the allocation decisions regarding the units in age categories in period t affect the system performance in the subsequent periods. For a policy needs to specify the allocation criteria for each age category in period t We would like to show in Chapter 5 that some myopically optimal policy that instructs the allocation decisions regarding each age category, such as Prastacos's myopic policy, is optimal over an arbitrary finite horizon. But let us first address the state dynamics involved in the current model. The initial state of the whole system in period t denote as can be represented by the vector A state is feasible as long as for each Let denote the total number of units available for transshipment at the first depots in period t where Let denote the to tal number of unsatisfied demand units at the first ( m 1) depots in period t before transshipment, where The number of units of age j to be transshipped from depot l to depot m can then be expressed as PAGE 47 47 The first term within the Minimum operator calculates the number of units of age j available at depot l for transshipping after sat isfying its own demand and the transshipment requests from the first ( m 1) depots. The second term gives the number of unsatisfied demand units at depot m right before units of age j at depot l if any, are transshipped to depot m The total number of unit s to be transshipped from depot l to depot m can be expressed as The first term within the Minimum operator computes the number of units available at depot l to be transshipped to depots with indices no less than m And the second term gives the number of unsatisfied demand units at depot m after receiving the transshipments from the first ( l 1) depots. Let denote the number of units of age j left at depot k in period t before age is updated. Then the number of units of age j initially available at the warehouse in period ( t +1), for , while Retention Model Set to ensure that units stay at the depots until used or outdated. The problem can then be formulated as PAGE 48 48 Note that only fresh units are allocated in this case. The expression of in the retention case is different from that in the rotation case in that be replaced by the number of units of age j initially available at depot k in period t for in the expression. Simil depot is also optimal for this single period retention model. Let denote the optimal quantity to allocate to depot k in the rotation case when the total number of units available at the warehouse equals If then is the final inventory level to be observed at depot k right after allocation in period t in the retention case. Now consider the transition between the initial states in period t and period ( t +1). The initial state of the system in period t in the retentio n case can be represented by the vector where for A state is feasible as long as the total number of units of age j initially available in the system is no more than for As in the previous subsection, we use to denote the total number of units available for transshipment at the first ( l 1) depots and to denote the total number of unsatisfied demand units at the first ( m 1) depots. With replaced by for in the PAGE 49 49 expression of and given in the rotation case, the number of units of age j left at depot k in period t before age is updated can be expressed as follows. Then at the start of period ( t +1), we have We will show in Chapter 5 that both the rotation and retention model have a myopic optimal solution over an arbitrary finite horizon. Hybrid Model For we have a single period hybrid model that can be formulated as In this model, the total number of transshipped units PAGE 50 50 The hybrid model combines the rotation and retention policies by only rotating units no greater than periods at the end of each period. We can see from th e objective function that a policy is myopically optimal for the hybrid model as long as it minimizes the expected number of units to be transshipped in a period. The expected emergency shipment and outdating costs are fixed for a given initial state for o ne period. Prastacos's myopic solution, which allocates units one by one from the oldest to the youngest to the depot with the highest probability of using the unit, is again myopically optimal for this hybrid model. Suppose the model rotates older units only, say, units no less than a certain age. A policy is then myopically optimal if it maximizes the total expected number of demand units that can be satisfied locally and minimizes the expected number of outdated units. Prastacos's myopic solution may no t be optimal for this particular single period hybrid model Take the oldest age category for exa mple. O ne unit deviation from Prastacos's myopic solution may increase the expected transshipment cost but reduce the expected outdating cost. We currently lea ve the situation for future research. The initial state of the whole system in period t denote as can be represented by the vector where A state i s feasible as long as for each and for each Let denote the total number of units available for transshipment at the first depots in period t where Let denote the total number of unsatisfied demand units at the first depots in period t before transshipment, where PAGE 51 51 The total number of units to be transshipped from depot l to depot m can be expressed as The first term within the Minimum operator computes the number of units available at depot l to be transshipped to depots with indices no less than m And the second term gives the number of unsatisfied demand units at depot m after receiving the transshipments from the first ( l 1 ) depots. Let denote the number of units of age j left at depot k in period t before age is updated. Then, for For The initial system state in period can be represented by where PAGE 52 52 CHAPTER 5 THE MULTI PERIOD BLOOD INVENTO RY ALLOCATION PROBLE M In this chapter, we would like apply the new myopic optimality criterion derived in Chapter 3 and prove the existence of some myopic policies that are optimal over arbitrary finite horizons for the rotation/retention model described above. The proof of myo pic optimality for the hybrid model is quite similar to that for the rotation model and is therefore omitted. Application to the Rotation Model Proof of the Existence of a Myopic Optimal Policy As stated in Theorem 3 1, a myopic optimal solution exists for an MDP over a finite horizon if (1) the MDP is finite; and (2) the Markov chain induced by any stationary policy for the MDP is irreducible. We will show in the rest of the section that (1) both conditions are satisfied in the rotation case given that the process starts from a feasible state; and (2) there exists some myopic optimal solution in the retention case if the MDP has no units available at the start of the horizon. We first focus on a decision process that adopts the rotation model every perio d over an infinite horizon. As demand and supply are both independent and identically distributed over time, the decision process is Markovian when the initial state of the system in period t is represented by the vector where A state is feasible as long as for The state set of the MDP is well defined due to the fixed limited shelf life of the product. The MDP is finite since demand and supply are assumed to be integer valued in this pap er and supply is finitely upper bounded. The first condition is therefore satisfied that the decision process under consideration is a finite MDP. We shall show next the irreducibility of the Markov chain associated with each stationary policy for the MDP. PAGE 53 53 Co nsider the situation where no demand occurs at any depot in the first L period s Let denote the zero demand probability in a period. It is clear that as for each k by assumption. An arbitrary state in period ( L +1) can then be accessed from any initial state in the first period of the planning horizon with probability Note that transshipment is not invoked in the zero demand situation. The transshipment rule therefore has no impact on the strictly positive lower bounds of the L step transition probabilities. By definition, the feasible states of the induced Markov chain form a single communicating class. As this property holds for any stationary policy, the finite MDP under consideration is thus ergodic. Referring to the theorem stated at the beginning of the section, we conclude that there exists a myopic optimal solution over arbitrary finite horizons for the rotation model with transshipment enabled. Proof of the Myopic Optimality of a Particular Policy We remark that the exis tence of some myopic optimal solution for the rotation model over a finite horizon gives little information of the structure of that myopic policy. There are a number of myopic policies for the rotation model as no restriction is imposed on the age composi tion of the units allocated to a depot in a period. The performance of the myopic policies over finite horizons, however, may depend on the age composition of the allocations through outdating. Although it is possible to enumerate the myopic optimal soluti ons given a specific initial state and a particular horizon length, we would like to show, in general, the myopic optimality of Prastacos's myopic solution for our rotation model. To do so, we consider an augmented version of the current model, the objecti ve of which is to minimize the expected sum of the transshipment cost, the emergency shipment cost, the outdating cost, and the aging cost incurred by the units left in each period. PAGE 54 54 Suppose that each unit left in a period after transshipment incurs some ag ing cost, which can be interpreted as the deterioration in the perceived value of a unit. Let denote the unit aging cost incurred by each unit aged j left after transshipment but before age is updated. Assume that for The single period augmented model can then be formulated as The term refers to the number of units to be transshipped from depot l to depot m in period t denotes the number of units of age j left in period t after transshipment but before age is updated. The expre ssions of both are given in Chapter 4 and contain for and Specifically, the total number of units to be tr ansshipped in period t can be expressed as It is then clear that the augmented model has a unique myopic policy that minimizes the expected number of units left in each age category in a period, which is exactly what Prastacos's myopic solution achieves. We can invoke Theorem 3 1 and conclude that Prastacos's myopic solution in the rotation case is optimal for the augmented model over arbitrary finite horizons. PAGE 55 55 The argument of myopic optimality is omitted due to its high resemblance to that for the original rotation model. Now consider a vector sequence , that converges to 0 Let denote a sequence of objective values of the T period augmented problems under lement of As each element of is non negative, bounded and continuous in the aging costs, the sequence of optimal objective values of the T period augmented problems, converges to the optimal objective value of the original model as The optimal objective value of the original model over the T period horizon can be augmented model in t optimal for the original rotation model. Application to the Retention Model We now want to show the existence of a myopic optimal solution for the retention model over any finite horizon given that there are zero units available at the start of the horizon. Consider a decision process that optimizes the retention model over an infinite horizon. Represent the initial state of the system in period t by the vector where and A state is feasible for the decision for the decision process as long as and The decision process is again a finite MDP due to the assumpti ons imposed on demand and supply. It is worth noting that the state set of the Markov chain induced by a stationary policy is a subset of that of the MDP in the retention case. The induced Markov chains in the rotation case share the state set of the MDP. PAGE 56 56 Let the initial state in period 1 be (0, 0 0 0 ). The feasible states of each induced Markov chain form a closed communicating class as any feasible state that is accessible from the zero state has a strictly positive probability of transiting to the z ero state. It then follows that the finite MDP in the retention case is ergodic given zero initial inventory in period 1. A myopic optimal solution therefore exists over any finite horizon in this case. Different from the rotation case, Prastacos's myopic solution is the unique myopic policy for the retention model and is optimal over arbitrary finite horizons. PAGE 57 57 CHAPTER 6 SIMULATION STUDY In this chapter, we first present the simulation model constructed to compare the performance of the myopic optim al policy that allocates in an intuitive policy that allocates based on the relative magnitude of mean demands in three different experiments. This is followed by a detailed description of the empirical data we obtained to e stimate the input parameters of the model. Simulation results are summarized in tables and depicted in figures afterward We also provide useful managerial insights based on the simulation results in the last subsection. Simulation Model Setup The simulati on model employed can be conceived as a centralized regional blood banking system in which the regional center observes the inventory levels at the three depots it services and allocates to achieve the best possible performance of the whole system. The mo del runs day by day. Two allocation policies are considered, the myopic optimal policy described earlier that allocates a unit to depots in increasing order of Type 1 service level s and an intuitive policy that allocates based on the relative magnitude of mean demands. To illustrate what happens at the regional center under the intuitive policy, suppose that Depot 1 has a mean demand of 7 units per day, that Depot 2 has a mean demand of 5 units per day, and that Depot 3 has a mean demand of 10 units per day. Then 7/(7+5+10) of the total units available at the regional center go to Depot 1. Depot 2 receives 5/(7+5+10) of the units. Depot 3 receives 10/(7+5+10) of the units under the in tuitive policy. The performance criteria include the total number of units transshipped during the whole planning horizon, the total number of units imported, that is, the total number of units sent by emergency shipment, and the total number of units out dated during the horizon. PAGE 58 58 Three types of experiments are carried out under each policy. Experiment 1 is designed to examine the impact of the extent of inventory rotation performed in the system on the system performance. Under complete inventory rotation all the units left at the depots at the end of a period will be transported back to the regional center for reallocation. Units allocated to a depot in a retention system are never returned to the regional center We also consider a hybrid system which i s a combination of the rotation and retention systems. We assume that units in the youngest one third age categories at the end of a period are rotation units and will be sent back to the regional center. All the other units are retention units. The PBDS s ystem studied in Brodheim and Prastacos (1979) and Prastacos and Brodheim (1980) involves the adoption of a similar combined system. Experiment 2 investigates the impact of the ratio of mean supply to the sum of the three mean depot demands on system perf ormance. The empirical data we obtained from the blood bank we studied at revealed a mean supply slightly lower than the sum of the mean demands. Two ratios are considered in this experiment, 0.76 and 1.0. We denote the ratio by Q/D. T he ratio of the lengt denoted as L/T is adjusted in Experiment 3 to accommodate different perishable products. It is hypothesized that this ratio affects system performance through outdating and thus the total number of units available for allocation during the planning horizon when everything else remains the same. Again, three ratios are experimented with 0.1, 0.2, and 0.3. In each of the three experiment s, we performed 6 00 runs of a 30 day simulation u nder both policies Analysis of Empirical Data Two types of empirical data were obtained from the blood bank for a component product produced at a hub, which also serves as a regional center, over a 12 month period: the number of fresh units available per day and the aggregated daily demands from three sources, the major PAGE 59 59 depot, hospitals, and a group of regional centers outside of the region. We treat each demand source as a depot in the simulation. Depot 1 in the simulation thus refers to the local depot serviced by the hub. Depot 2 refers to a number of nearby hospitals. And Depot 3 refers to the group of outside regional centers requesting units from the hub. Descriptive Statistics A total of 35,728 units were produced at the hub during the 12 month peri od, averaging at 97.9 units per day. Depot 1 had a total demand of 4,770 units with a daily average of 13.1 units. Depot 2, the hospitals, requested a total of 35,424 units in those 12 months with a daily average of 97.1 units. Depot 3 ordered 7,008 units in total, averaging at 19.2 units per day. The annual fresh supply was about 24% lower than the total annual demand based o n the empirical data. Figure 6 1 presents the mean daily demands from the three sources for e ach of the 12 months. Figure 6 2 compare s the mean daily aggregated demand with fresh supply on a monthly basis. Figures featuring daily supply and demand can be found in the appendix. As we can see from Figure 6 1 the mean daily demand from Depot 1, which is averaged on a monthly basis, does n ot exhibit too much fluctuation over the 12 month period. The mean daily demand from Depot 2, which refers to a number of nearby hospitals, is relatively high in January and February and falls significantly in June. As the hub is located in a college town the rise in hospital demand in the first two months can probably be explained by the influx of college students and local residents after the Christmas break The drop in June might be caused by students taking vacations in the first half of the summer s emester. It is interesting to observe that the curve representing the monthly mean daily demand from Depot 3, which refers to a group of outside regional centers, looks like a vertical reflection of the curve for hospital demand. The resemblance might be a result of centralized coordinati on among the regions. Figure 6 2 demonstrates the relationship between the mean daily fresh supply and the aggregated daily PAGE 60 60 demand. Fresh supply is a little behind the aggregated demand most of the. The decline in fresh sup ply in June and July is likely due to the summer exodus of college students Input Estimation Inputs for the 30 day simulation include distributions for daily fresh supply and daily demand from Depot 1, Depot 2, and Depot3, as well as the length of shelf l ife of the product. A Generalized Extreme Value distribution truncated at value 0 is fitted to the empirical supply data with shape parameter k = 82.054. The probability mass over negative v alues in the original distrib ution is transferred to that at the point 0. The cumulative probability at the point 0 is less than 0.0027. The fit of the original GEV distribution passes Kolmogorov Smirnov and Anderson Darling goodness of fit tests at signif icance levels of 0.01, 0.02, 0.05, 0.1, and 0.2 and Chi Squared test at the above mentioned significance levels except 0.2. Another Generalized Extreme Value distribution with shape parameter k = 0.10754, scale er = 9.0307 is truncated at 0 in the same manner to be fitted to the non negative daily demand from Depot 1. The cumulative probability at the point 0 is about 0.063. The fit passes the Kolmogorov Smirnov and Anderson Darling tests for significance level s no higher than 0.05 but fails the Chi Squared test. A three parameter Dagum distribution is fitted to the hospital demand data with shape goodness of fit hypothesis i s not rejected in the Kolmogorov Smirnov and Anderson Darling tests at significance levels of 0.01, 0.02, 0.05, 0.1, and 0.2. The Chi Squared test rejects the hypothesis for significance levels no less than 0.1. The daily demand from Depot 3 fits a Genera lized Pareto distribution best with shape 3.3302. The PAGE 61 61 Generalized Pareto distribution is again truncated at 0 to accommodate the non negative demand values. The cumulative prob ability at the point 0 is about 0.19. The fit does not pass the three common goodness of fit tests. Although not all the fits in the simulation model give satisfactory test results, the model is acceptable as the aim of the simulation study is to stress te st the myopic and intuitive allocation policies in the presence of randomness in both demand and supply. Due to the software limit of a maximum of 150 entities in the system at the same time, the empirical supply and demand data are scaled down by 40% to fit distributions in the same manner for the baseline situation. The distribution fitted to supply is trun cated at value 0 and 90. The ratio of mean supply to the sum of mean demands is about 0.76 in the baseline case. To perform experiments where the ratio of mean supply to the sum of mean demands is 1.0, the demand data in the baseline case are scaled down a ccordingly and new distributions, truncated if necessary, are fitted. 3,000 random numbers are then generated from each of the fitted distributions. Numbers with decimals are rounded to the nearest integers. The detailed setup of the simulations to be perf ormed in order to complete the three experiments is the following. Length of simulation: 30 days Length of shelf life: 3 days, 6 days, 9 days Extent of inventory rotation: full rotation, hybrid, full retention Allocation policies: myopic policy, intuitive policy (10%, 75%, 15%) Supply: Truncated GEV ( 0.19679, 23.112, 49.238) Demand Set 1: Demand 1, Truncated GEV ( 0.10992, 5.0514, 5.4305); Demand 2, Dagum (0.10079, 11.615, 105.04); Demand 3, Truncated GPD (0.32836, 9.0607, 1.9671). Demand Set 2: Demand 1 Truncated GEV ( 0.10989, 3.7869, 4.1722); Demand 2, Dagum (0.10277, 11.472, 79.101); Demand 3, Truncated GPD (0.32209, 6.914, 1.4346). PAGE 62 62 Simulation Results We define transshipment rate as the ratio of the number of units transshipped per day to the sum of daily demands, shortage rate as the ratio of the number of units imported per day to the sum of daily demands, and outdate rate as the percentage of the number of units that outdate in the system each day. In this subsection, simulation results are first summarized in tables using the averages of the three rates and then compared in figures. Table 6 1(A), 6 1 (B), and 6 1(C) present the simulation results for the situation when demands are generated from distributions in Data Set 1, that is, when the ratio of mean supply to the sum of mean demands is 0.76. The other set of tables, Table 6 2(A), 6 2 (B), and 6 2 (C) display the averages for the situation when random demands are drawn from distributions in Data Set 2, that is, when the ratio of mean supply to the sum of mean demands is 1.0. The Analysis of Variance table for the number of units transshipped, Table 6 3, indicates the impact of the extent of inventory rotation, the ratio of supply to demand, the length of shelf life, and the choice of allocation policy on this performance measure. The ratio of the length of shelf life to the length of planning horizon alone has no significant influence on the total number of units transshipped during the whole planning horizon. There tend to be more transshipped units in the case where Q/D = 0.76 under both the myopic policy and the intuitive policy. The increase is minimal and almost constant for varying extent s of inventory rotation under the myopic optimal policy. That is, when the myopic optimal policy is adopted, the extent of inventory rotation to be performed in the syste m may not be a crucial factor in minimizing the number of units to be transshipped. See Figure 6 3 (A) for an example with L/T = 0.1. The difference reaches its maximum when the intuitive policy is implemented in a hybrid model with L/T = 0.1, as shown in Figure 6 3 (B). PAGE 63 63 Table 6 4 is the Analysis of Variance table for the performance measure: the total number of units imported. Only the ratio of mean supply to the sum of mean demands is shown to have a significant impact on the shortag e. As we can see from Figure 6 4 there is a non negligible jump in the number of imported units when the ratio Q/D is adjusted from 0.76 to 1.0 under the myopic policy. The insignificance of othe r factors in this case, such as the choice of allocation policies, is a result of the performance of transshipment among the demand locations. According to Table 6 5, the total number of units outdated during the whole planning horizon is under significant impact of the extent of inventory rotation performed in the system, the ratio of supply to demand, the ratio of shelf life to the length of planning horizon, and the choice of allocation policies. Few units outdate when Q/D = 0.76. The number of units out dated decrease dramatically under both policies when the ratio L/T is lifted from 0.1 to 0.3. See Figure 6 5 (A) and (B) for a compa rison under the myopic policy. A s imilar relationship exists under the intuitive policy. The observation is intuitive that p roducts with a longer shelf life usually have fewer units outdated within a certain amount of time when everything else is identical. The myopic policy outperforms the intuitive policy most of the time regardless of the extent of inventory rotation perform ed in the system. The advantage of implementing the myopic allocation policy becomes more profound when the extent of inventory rotation decreases Readers can refer to Figure 6 6 (A), (B), and (C) for a comparison. The optimal allocation policy is seen to make a significant difference for a product with a high supply to demand ratio and short shelf life that is distributed in a less flexible system. The major assumptions of Analysis of Variance include: independence, normality, and homogeneity in variance The assumption of independent observations is satisfied in this study as supply and demand values are generated randomly from corresponding distributions. Moderate PAGE 64 64 deviations from normality were observed in simulation results for the total number of unit s transshipped and the total number of units outdated during the 30 day simulation. It may not be a concern since the F statistics is generally quite robust against non normal data especially in fixed effects experiments. There is no evidence that the resu lts for the total number of units sent by emergency shipment violate the normality assumption. As for the assumption of homogeneity in variance, only the results for the total number of transshipped units exhibit a moderate deviation. Not much is known abo ut the response of the test statistics used in Analysis of Variance to the violati on of the homogeneity variance when there are multiple response variables. In a univariate ANOVA, the F statistic is quite robust against the heterogeneity in variance. Figur es display the testing results of the assumptions of normality and homogeneity in variance can be found in Appendix B. PAGE 65 65 Table 6 1. Summary of the simulation results when Q/D = 0.76. A) Average transshipment rate s, B) Average shortage Rates, C) Average outdate r ates A Rotation Hybrid Retention Shelf Life/30 days Myopic Intuitive Myopic Intuitive Myopic Intuitive 0.3 0.0033 0.0264 0.0036 0.0261 0.0038 0.0359 0.2 0.0033 0.0264 0.0033 0.0266 0.0033 0.0363 0.1 0.0033 0.0264 0.0033 0.0409 0.0033 0.0349 B Rotation Hybrid Retention Shelf Life/30 days Myopic Intuitive Myopic Intuitive Myopic Intuitive 0.3 0.2734 0.2734 0.2734 0.2734 0.2734 0.2734 0.2 0.2734 0.2734 0.2734 0.2734 0.2734 0.2734 0.1 0.2734 0.2734 0.2736 0.2744 0.2738 0.2756 C Rotation Hybrid Retention Shelf Life/30 days Myopic Intuitive Myopic Intuitive Myopic Intuitive 0.3 0 .0000 0 .0000 0 .0000 0 .0000 0 .0000 0 .0000 0.2 0 .0000 0 .0000 0 .0000 0 .0000 0 .0000 0 .0000 0.1 0 .0000 0 .0000 0.0002 0.0015 0.0006 0.0033 PAGE 66 66 Table 6 2. Summary of the simu lation results when Q/D = 1.0 A) Average transshipment rate, B) Average shortage rates, C) Average outdate rates. A Rotation Hybrid Retention Shelf Life/30 days Myopic Intuitive Myopic Intuitive Myopic Intuitive 0.3 0.0001 0.0212 0.0001 0.0209 0.0002 0.0284 0.2 0.0001 0.0212 0.0001 0.0201 0.0002 0.0263 0.1 0.0001 0.0212 0.0001 0.0174 0.0002 0.0214 B Rotation Hybrid Retention Shelf Life/30 days Myopic Intuitive Myopic Intuitive Myopic Intuitive 0.3 0.0572 0.0572 0.0572 0.0572 0.0572 0.0579 0.2 0.0572 0.0572 0.0572 0.0574 0.0586 0.0624 0.1 0.0572 0.0572 0.0621 0.0678 0.0697 0.0767 C Rotation Hybrid Retention Shelf Life/30 days Myopic Intuitive Myopic Intuitive Myopic Intuitive 0.3 0 0 0 0 0.0004 0.0023 0.2 0 0 0.0001 0.0006 0.0029 0.0083 0.1 0 0 0.0126 0.0154 0.0183 0.0279 PAGE 67 67 Table 6 3. Analysis of variance table for the total number of units transshipped. Source Sum of squares df Mean square F value p value Model 2.945E+006 35 84145.59 154.18 <0.0001 A Rotation extent 29675.86 2 14837.93 27.19 <0.0001 B Supply/Demand 3.548E+005 1 3.548E+005 650.08 <0.0001 C Shelf life/Horizon 1324.91 2 662.46 1.21 0.2972 D Policy 2.301E+006 1 2.301E+006 4215.58 <0.0001 Table 6 4. Analysis of variance table for the total number of units sent by emergency shipment. Source Sum of squares df Mean square F value p value Model 2.527E+008 1 2.527E+008 16876.00 <0.0001 B Supply/Demand 2.527E+008 1 2.527E+008 16876.00 <0.0001 Residual 5.386E+007 3597 14973.68 Lack of Fit 1.482E+005 34 4358.83 0.29 1.0000 Pure Error 5.371E+007 3563 3563 Cor Total 3.066E+008 3598 3598 Table 6 5. Analysis of variance table for the total number of units outdated. Source Sum of squares df Mean square F value p value Model 20801.22 35 594.32 169.39 <0.0001 A Rotation extent 4815.38 2 2407.69 686.23 <0.0001 B Supply/Demand 2163.43 1 2163.43 616.61 <0.0001 C Shelf life/Horizon 6576.63 2 3288.31 937.22 <0.0001 D Policy 306.21 1 306.21 87.27 <0.0001 PAGE 68 68 Figure 6 1. Mean daily demands. Figure 6 2. Comparison of mean aggregated demand to mean supply PAGE 69 69 A B Figure 6 3. Impact of the extent of inventory rotation and the choice of allocation policy on the total number of transshipped u nits. A) Myopic policy; B) Intuitive policy. PAGE 70 70 F igure 6 4. Impact of the ratio of supply to demand on the number of units sent by emergency shipment under the myopic policy. PAGE 71 71 A B Figure 6 5. Impact of the ratio L/T on the total number of outdated units under myopic policy. A) L/T = 0.1; B) L/T = 0.3. PAGE 72 72 A B Figure 6 6. Advantage of implementing the myopic policy in a less flexible system. A) in a rotation system, B) in a hybrid system, C) in a retention system. PAGE 73 73 C Figure 6 6. Continued. PAGE 74 74 CHAPTER 7 CONCLUSIONS Key Contributions In this dissertation work we studied a blood logistics problem targeted at the efficient allocation of a blood product with a limited shelf life from a central warehouse to a number of local depots in a region The central warehouse receives a random quantity of fresh units at the beginning of each period, observes the inventory status at the demand locations and then allocates the fresh supply received together with its initial inventories, if any, to the n demand locations. Transshipment is performed among the depots if necessary, that is, if some depots have excess inventories after using their local stock to satisfy the random demand and others h ave unmet demand The region turns to an outside source for emergency supply if there is unsatisfied demand in the system after transshipment. Fresh units outdate in L period s and must be disposed at a cost once expired. The central warehouse determines the size and age composition of the allocations to minimize the total expected co sts incurred by transshipment, emergency shipment, and ou tdating over a finite horizon. This particular storage and distribution problem was formulated to address one of the current industrial challenges faced by various blood banks in the US. With limited supply and high outdating cost, the practitioners we interviewed were interested in knowing an efficient way to allocate units. Little research has been done in this area with most focusing on the derivation of myopically optimal policies and the evaluati on of certain allocation policies in the long run. The mul ti period allocation problem formulated in the present work is intended to fill the gap in the existing literature It is shown in Chapter 4 that a policy is myopically optimal for the rotation/ret ention /hybrid model constructed if it maximizes the probability of each allocated unit PAGE 75 75 being used at a depot. The myopic solution proposed in Prastacos (1978) in which units are allocated under a different objective and no transshipment is allowed is myopi cally optimal for the three models. D ue to product perishability and the randomness in supply and demand, the optimality of a myopic policy over a multi pe riod finite horizon is not straightforward To identify/construct an optimal allocation policy for th e multi period problem, we first derived a new criterion for myopic o ptimality in Markov decision processes. We proved that a Markov decision process with finite state and action sets h as a myopic optimal policy over an arbitrary finite horizon if (1) the Markov decision process is ergodic, that is, if the Markov chain induced by each stationary policy is irreducible; or (2) the Markov decision process is strongly multi chain with no transient states. The existence of transient states induced by a stationar y policy may prevent a finite MDP having a myopic optimal solution. This main theorem isolates a class of Markov decision problems that possess a myopic optimum, which can be of interest in various research areas where Markov decision problems are consider ed. Th e application of the main theorem to both the rotation and retention models is presented in Chapter 5. It is shown that there exists some myopically optimal policy that is optimal for the multi period rotation model regardless of the initial invento ry and the length of the planning horizon. However, as there are a number of myopic policies in the rotation case, the existence of a myopic optimal policy gives little information about its specific structure. An augmented model is thus constructed that t akes into account aging costs incurred by leftovers at the end of a period. We then invoked Theorem 3 1 and proved the existence of a myopic optimal policy for the augmented model, which is the unique myopically optimal policy. This particular policy is shown to be optimal for the original rotation model over arbitrary finite hori zons by converging PAGE 76 76 the vector of aging costs to 0 The retention model possesses a myopic optimum when the horizon starts with zero inventories. A simulation study was carried out to further explore the impact of the extent of inventory rotation performed in the system, the ratio of supply to demand, and the length of shelf life on system performance. Three performance measures were chosen in consistent with the models formulated: the total number of units transshipped during the whole planning horizon, th e total number of units sent by emergency shipment, and the total number of outdated units. Inputs such as the supply and demand distributions were estimated based on the empirical data obtained from the blood bank we studied at. A total of 3,600 runs of a 30 day simulation were conducted. Simulation results were analyzed using the Analysis of Variance tables. It is observed that the extent of inventory rotation, the ratio of supply to demand, and the length of shelf life all have a significant impact on t he total number of units transshipped during the horizon and the total number of units outdated, while the total number of units sent by emergency shipment is only significantly impacted by the ratio of supply to demand. The increase in the total number of transshipped units is minimal and almost constant for varying extent s of inventory rotation under the myopic optimal policy. That is, when the myopic optimal policy is adopted, the extent of inventory rotation to be performed in the syste m may not be a cr ucial factor in minimizing the number of units to be transshipped. The decision of a satisfactory allocation policy becomes urgent for a product with a high supply to demand ratio and short shelf life that is distributed in a less flexible system. Future R esearch To extend the current work, it might be of interest to model demand in a manner to take into account some physicians' requests regarding the freshness of the blood units to be used in a procedure. Demand occur s to each age category, which can only be satisfied by units in that PAGE 77 77 particular age category or units in younger categories. We might as well design a pricing scheme a blood bank can adopt to prevent over ordering from hospitals it services The blood bank we studied with currently offers full credit for returns, damaged/outdated or not. The hospitals pay for the units they use, but not f or most of the units outdated on their shelves This return policy, to some extent, encourages its clients to over order, since they take little risk if over st ock. Some non trivial problems can be addressed in this setting. PAGE 78 78 APPENDIX A FIGURES SUMMARIZING EMPIRICAL DATA Figure A 1. Daily fresh supply. Figure A 2. Daily demand at depot 1. PAGE 79 79 Figure A 3. Daily demand at depot 2. Figure A 4. Daily Demand at depot 3. PAGE 80 80 APPENDIX B TEST OF ANALYSIS OF VARIANCE ASSUMPTIONS A B Figure B 1. Test of assumptions of normality and homogeneity in variance for the total number of transshipped units. A) test of normality, B) test of homogeneity in variance. PAGE 81 81 A B Figure B 2. Test of assumptions of normality and homogeneity in variance for the total number of units sent by emergency shipment. A) test of normality, B) test of homogeneity in variance. PAGE 82 82 A B Figure B 3. Test of assumptions of normality and homogeneity in variance for the total number of outdated units. A) test of normality, B) test of homogeneity in variance. PAGE 83 83 LIST OF REFERENCES Baafat, F. 1991. Survey of literature on continuously deteriorating inventory models. The Journal of the Operational Research Society 42 27 37. Blackwell, D. 1962. Discrete dynamic programming. The Annals of Mathematical Statistics 33 (2) 719 726. Brodheim, E., G. P. Prastacos. 1979. The long island blood distribution system as a prototype for regional blood management. Interfaces 9 (5) 3 20. Ceti nkaya, S., M. Parlar. 1998. Optimal myopic policy for a stochastic inventory problem with fixed and proportional backorder costs. European Journal of Operational Research 110 20 41. Chazan, D., S. Gal. 1977. A Markovian model for a perishable product inve ntory. Management Science 23 512 521. Cohen, M. A., W. P. Pierskalla. Target inventory levels for a hospital blood bank or a decentralized regional blood banking system. Transfusion. 19 444 454. Derman, C., E. Brodheim, G. P. Prastacos. 1975. On the evalu ation of a class of inventory policies for perishable products such as blood. Management Science 21 1320 1325. Dhrymes, P. J. 1962. On optimal advertising capital and research expenditure under dynamic conditions. Economica New Series. 29 (115) 275 279. D irickx, Yvo M. I., L. P. Jennergren. 1975. On the optimality of myopic policies in sequential decision problems. Management Science 21 (5) 550 556. Fries, B. E. 1975. Optimal ordering policy for a perishable commodity with fixed lifetime. Operations Resear ch 23 (1) 46 61. Goyal, S. K., B. C. Giri. 2001. Recent trends in modeling of deterioration inventory. European Journal of Operations Research 134 1 16. Heyman, D. P., M. J. Sobel. 1984. Stochastic models in operations research Vol 2, Dover Publications, Inc. Mineola, New York. Hirsch, R., E. Brodheim, G. P. Prastacos. 1976. Setting inventory levels for hospital blood banks. Transfusion. 16 63 70. Huh, Woonghee T., G. Janakiraman. 2008. Inventory management with auctions and other sales channels: optimality of (s, S) policies. Management Scienc e 54 (1) 139 150. Huh, Woonghee T., G. Janakiraman. 2008. (s, S) optimality in joint inventory pricing control: an alternate approach. Operations Research 56 (3) 783 790. PAGE 84 84 Huh, Woonghee T., G. Janakiraman, J.A. Muckstadt, R. Rusmevichientong. 2009. Asymptotic optimality of order up to policies in lost sales inventory systems. Management Science 55 (3) 404 420. Ignall, E., A. F. Veinott Jr. 1969. Optimality of myopic inventory policies for severa l substitute products. Management Science 15 (5) 284 304. Jennings, J. B. 1973. Blood bank inventory control. Management Science 19 637 645. Johnson, G. D., H. E. Thompson. 1975. Optimality of myopic inventory policies for certain dependent demand process es. Management Science 21 (11) 1303 1307. Karlin, S. 1960. Dynamic inventory policy with varying stochastic demands. Management Science 6 (3) 231 258. Khang, D. B., O. Fujiwara. 2000. Optimality of myopic ordering policies for inventory models with stochas tic supply. Operations Research. 48 (1) 181 184. Kochel, Operations Research 33 (6) 1394 1398. Lamond, B. F., M. J. Sobel 1995. Exact and approximate solutions of affine reservoir models. Operations Research 43 (5) 771 780. Levin, D. A., Y. Peres, E. L. Wilmer. 2008. Markov chains and mixing times American Mathematical Society. Providence, RI. Liu, L., Z. Lian. 1999. (s, S ) continuous review models for products with fixed lifetimes. Operations Research 47 150 158. Lovejoy, W. S. 1990. Myopic policies for some inventory models with uncertain demand distributions. Management Science 36 (6) 724 738. Nahmias, S. 1975. Optimal ordering policies for perishable inventory II. Operations Research 28 (4) 735 749. Nahmias, S. 1982. Perishable inventory theory: a review. Operations Research 30 (4) 680 708 Nahmias, S., C. P. Schmidt. 1985. (s 1, s) policies for perishable inventory. Man agement Science. 31 719 728. Nerlove, M., K. T. Arrow. 1962. Optimal advertising policy under dynamic conditions. Economica New Series. 29 (114) 129 142. Pedergruen, A., P. H. Zipkin. 1984. A combined vehicle routing and inventory allocation problem. Opera tions Research 32 1019 1037. Pierskalla, W. P., C. D. Roach. 1972. Optimal issuing policies for peris hable inventory. Management Science 18 603 614. PAGE 85 85 Pierskalla, W.P. 2005. Supply chain management of blood banks In Operations Research and Health Care International Series in Operations Research & Management Science 70 103 145 Kluwer Academic Publisher, Norwell, Massachusetts. Prastacos, G. P. 1978. Optimal myopic allocation of a product with fixed lifetime. The Journal of the Operational Research Soci ety. 29 (9) 905 913. Prastacos, G. P., E. Brodheim. 1980. PBDS: A decision support system for regional blood management. Management Science 26 (5) 451 463. Prastacos, G. P. 1981. Allocation of a perishable product inventory. Operations Research 29 (1) 95 10 7. Prastacos, G. P. 198 4. Blood inventory management: A n overview of theory and practice. Management Science 30 777 800. Prastacos, G. P., A. Pedergruen, P. H. Zipkin. 1986. An alloc ation and distribution model for perishable products. Operations Research 34 75 82. Puterman, M. L. 2005. Markov Decision Processes: Discrete Stochastic Dy namic Programming JohnWiley & Sons, Inc., Hoboken, New Jersey. Rosenfield, D. B. 1992. Optimality of myopic policies in disposing excess inventory. Operations Research. 40 (4) 800 803. Shapiro, J. F. 1968. Turnpike planning horizons for a Markovian decision model. Management Science 14 (5) 292 300. Simposon, K. Jr. F. 1959. A theory of allocation of stocks to warehouse. Operations Research. 7 797 805. Sobel, M. J. 1981. Myop ic solutions of Markov decision processes and stochastic games. Operations Research 29 (5) 995 1009. Sobel, M. J. 1990. Myopic solutions of affine dynamic models. Operations Research 38 (5) 847 853. Sobel,M. J. 1990. Higher order and average reward myopic affine dynamic m odels. Mathematics of Operations Research 15 (2) 299 310. Sobel, M. J., W. Wei. 2010. Myopic solutions of homogeneous sequential decision processes. Operations Research 58 (4) 1235 1246. Veinott Jr., A. F. 1965. Optimal policy for a multi p roduct, dynamic, nonstationary inventory problem. Management Science 12 (3) 206 222. Wang, Y. Z. 2001. The optimality of myopic stocking policies for systems with decreasing purchasing prices. European Journal of Operational Research 133 153 159. PAGE 86 86 Weiss, H. J. 1980. Optimal ordering policies for continuous review perishable inventory models. Operations Research 28 365 374. PAGE 87 87 BIOGRAPHICAL SKETCH anagement from Fudan University in June 2006. Interested in pursuing an academic career, she sta rted her doctoral study in the D epartment of Information Systems and Operations Management at the University of Florida in August 2006. She received her Ph.D. in Business Administration from the University of Florida in August 2011. 