<%BANNER%>

Stochastic and Dynamic Sequential Decision Problems with Postponement Options

Permanent Link: http://ufdc.ufl.edu/UFE0044117/00001

Material Information

Title: Stochastic and Dynamic Sequential Decision Problems with Postponement Options
Physical Description: 1 online resource (158 p.)
Language: english
Creator: Feng, Tianke
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2012

Subjects

Subjects / Keywords: optimization -- postponement -- sequential-decision
Industrial and Systems Engineering -- Dissertations, Academic -- UF
Genre: Industrial and Systems Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Postponement naturally arises as an option for decisions under uncertainty, as it enables the acquisition of more information for better decisions. The value of postponement options has been widely identified in finance and operations management. However, the research on postponement option in the sequential decisions, especially in operations settings, is relatively few. This thesis includes the postponement options into decision-making and studies its value in three sequential decisions settings, i.e., Sequential and Stochastic Assignment Problem (SSAP), Dynamic Stochastic Knapsack Problem (DSKP), and Dynamic Stochastic Loading Problem (DSLP), involving both theoretical analysis and design of algorithms for practice applications. The SSAP and the DSKP have been well studied in the literature and have wide applications in logistics, finance and health care management. We analyze the optimal policies for these two problems in the special case through stochastic dynamic programming approach, and develop effective heuristic policies for the general case. We believe that this model has drastic implications on the real options and the revenue management literature. In particular, the research results on these two problems shed lights to the DSLP. The DSLP arises from the airfreight industry, in which the ground staff need to make decisions under a dynamic and stochastic environment. The problem is by nature a combinatorial problem, which can be shown to be PSPACE-hard for special cases and moreover is indeed not well-defined. We illustrate the limits of the standard optimization methodologies in dealing with this problem and develop a viable computation framework, based on the insights found out from the study on the SSAP and the DSKP.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Tianke Feng.
Thesis: Thesis (Ph.D.)--University of Florida, 2012.
Local: Adviser: Hartman, Joseph C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2012
System ID: UFE0044117:00001

Permanent Link: http://ufdc.ufl.edu/UFE0044117/00001

Material Information

Title: Stochastic and Dynamic Sequential Decision Problems with Postponement Options
Physical Description: 1 online resource (158 p.)
Language: english
Creator: Feng, Tianke
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2012

Subjects

Subjects / Keywords: optimization -- postponement -- sequential-decision
Industrial and Systems Engineering -- Dissertations, Academic -- UF
Genre: Industrial and Systems Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Postponement naturally arises as an option for decisions under uncertainty, as it enables the acquisition of more information for better decisions. The value of postponement options has been widely identified in finance and operations management. However, the research on postponement option in the sequential decisions, especially in operations settings, is relatively few. This thesis includes the postponement options into decision-making and studies its value in three sequential decisions settings, i.e., Sequential and Stochastic Assignment Problem (SSAP), Dynamic Stochastic Knapsack Problem (DSKP), and Dynamic Stochastic Loading Problem (DSLP), involving both theoretical analysis and design of algorithms for practice applications. The SSAP and the DSKP have been well studied in the literature and have wide applications in logistics, finance and health care management. We analyze the optimal policies for these two problems in the special case through stochastic dynamic programming approach, and develop effective heuristic policies for the general case. We believe that this model has drastic implications on the real options and the revenue management literature. In particular, the research results on these two problems shed lights to the DSLP. The DSLP arises from the airfreight industry, in which the ground staff need to make decisions under a dynamic and stochastic environment. The problem is by nature a combinatorial problem, which can be shown to be PSPACE-hard for special cases and moreover is indeed not well-defined. We illustrate the limits of the standard optimization methodologies in dealing with this problem and develop a viable computation framework, based on the insights found out from the study on the SSAP and the DSKP.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Tianke Feng.
Thesis: Thesis (Ph.D.)--University of Florida, 2012.
Local: Adviser: Hartman, Joseph C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2012
System ID: UFE0044117:00001


This item has the following downloads:


Full Text

PAGE 1

STOCHASTICANDDYNAMICSEQUENTIALDECISIONPROBLEMSWITHPOSTPONEMENTOPTIONSByTIANKEFENGADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2012

PAGE 2

c2012TiankeFeng 2

PAGE 3

IdedicatethistoallwholentmeahandwhenIwasindifcultyinmyadventureofacademicresearch. 3

PAGE 4

ACKNOWLEDGMENTS IwouldliketogratefullyandsincerelythankDr.JosephC.Hartmanforhisguidance,understanding,patience,andmostimportantly,hisfriendshipduringmygraduatestudiesattheUniversityofFlorida.Hismentorshipwasparamountinprovidingawellroundedexperienceconsistentwithmylong-termcareergoals.Heencouragedmetonotonlygrowasanoperationsresearcherandanappliedmathematicianbutalsoasaninstructorandanindependentthinker.Thefouryearsofexperienceinhisgroupmademeamuchindependentresearcherandgrewmycouragetodefendmyownopinions.IwouldliketothanktheDepartmentofIndustrialandSystemsEngineeringoftheUniversityofFlorida.Withoutitshelp,Iwillnotbeabletoseekmyinterestinthisareaandthismaybecomemylifelongregret.Ialsothankthemembersofmydoctoralcommitteefortheirinput,valuablediscussionsandaccessibility.Inparticular,Dr.Geunes,Dr.Momcilovic,andDr.Guanusuallyallocatedtimeformefromtheiralreadyincrediblytightschedules.Moreover,IwouldliketothankDr.Sapra,whoismyrstacademicadvisorinourdepartment.Hetriedhisbesthelpingmebuildmymathematicsfundamentalsandstrengthenmyskillsintheoreticalanalysis.AlthoughhisdecisiontoleavetheUniversityofFloridagreatlyfrustratedme,thisexperiencemademestronger.Finallyandmostimportantly,IthankmyfamiliesforallowingmetobeasambitiousasIwanted.DuringmyPh.D.years,IonlywentbacktoBeijingonceandmymothernevercomplainedbecauseofthat.Ithankmyelderbrotherandsister-in-law,forwhenmymotherwasdiagnosedwithcancer,theyassumedtheheavyburdencompletelybythemselves,evenwithoutlettingmeknow.Withouttheirsupport,Icouldnotfocusonmystudyandsailthroughmypainfullyearnestoralsummerin2007.Moreover,whenIsufferedfromthedepartureofmyrstadvisorandwhenmystipendfailedtocovermylifeexpenseshere,theygenerouslygavemenancialhelp. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 8 LISTOFFIGURES ..................................... 9 ABSTRACT ......................................... 10 CHAPTER 1INTRODUCTION ................................... 12 1.1PostponementOptionsinSequentialDecisions ............... 13 1.1.1TheSequentialandStochasticAssignmentProblem ........ 13 1.1.2TheDynamicandStochasticKnapsackProblem .......... 13 1.1.3DynamicandStochasticContainerLoadingProblem ........ 14 1.2FeaturesofthePostponementOptionandRelatedMethodologies .... 15 1.3ChapterOutlineandSummaryofResults .................. 17 2THESEQUENTIALANDSTOCHASTICASSIGNMENTPROBLEMWITHPOSTPONEMENTOPTIONS ............................ 19 2.1IntroductionandMotivation .......................... 19 2.2LiteratureReview ................................ 20 2.3AssumptionsandModelDenitions ...................... 22 2.3.1StateVariableS ............................. 23 2.3.2Policies,StateTransitionsandDecisionEpochs ........... 23 2.3.3ExpectedProtandDynamicProgrammingEquation ........ 25 2.4HomogeneousResources ........................... 26 2.4.1OptimalPolicywiththePostponementOption ............ 26 2.4.1.1Case1:m=1 ........................ 26 2.4.1.2Case2:m2 ........................ 28 2.4.2TheBenetsofPostponement ..................... 39 2.4.3NumericalExamples .......................... 40 2.5HeterogenousResources ........................... 43 2.5.1Properties ................................ 43 2.5.2HeuristicsforHeterogeneousResources ............... 45 2.5.2.1Heuristics1:Raisingthresholds .............. 46 2.5.2.2Heuristics2:Samplingonfuturearrivals .......... 48 2.6Conclusions ................................... 50 3THEDYNAMICANDSTOCHASTICKNAPSACKPROBLEM .......... 52 3.1RelatedResearch ............................... 55 3.2ModelandAssumptions ............................ 58 5

PAGE 6

3.2.1ModelDenitions ............................ 58 3.2.2ARelatedMarkovDecisionProcess ................. 62 3.3TheFiniteHorizonDSKPwithPostponementOptions:ASimpliedVersion 64 3.3.1Case1:m=1 ............................. 64 3.3.2Case2:m2 ............................. 67 3.4TheFiniteHorizonDSKPwithPostponementOptions:Extensions .... 70 3.4.1DelayCost ............................... 70 3.4.1.1Theupdaterulewithentry-barrier ............. 71 3.4.1.2Optimalpolicy ........................ 73 3.4.2StopOption,RejectionPenaltyandStructuralProperties ...... 78 3.5NumericalExamples:BenetsofPostponement .............. 80 3.5.1ImpactofTimeHorizon ........................ 82 3.5.2Impactofc ............................... 82 3.5.3ImpactofDelayCostsandDelays .................. 88 3.6DSKPwithRandmlySizedItems ....................... 89 3.7Conclusions ................................... 90 4THEDYNAMICANDSTOCHASTICLOADINGPROBLEM ........... 92 4.1IntroductionandLiterature ........................... 92 4.2Motivation .................................... 95 4.2.1FuelSavings .............................. 95 4.2.2ValueofDelayingaFlight ....................... 97 4.3ProblemDescription .............................. 98 4.4StochasticDynamicProgramming ...................... 100 4.4.1Formulation ............................... 100 4.4.1.1State,decisionepochs,andnotation ............ 100 4.4.1.2Dynamicprogrammingequation .............. 102 4.4.2Properties ................................ 104 4.4.2.1Combinatorialnatureandstructuralproperties ...... 104 4.4.2.2Impactofcosts ........................ 105 4.4.3ComplexityandLimitationoftheDPFormulation .......... 105 4.5OtherApproaches ............................... 108 4.5.1Multi-stageStochasticProgrammingandRobustOptimization ... 108 4.5.2Online-optimization ........................... 111 4.5.3SimpleHeuristics ............................ 112 4.6SolutionApproach ............................... 113 4.6.1ApproximationwithTwo-stageMixedStochasticIntegerProgramming(TSMIP) ................................. 114 4.6.2NumericalExperimentsonIdealConditions ............. 117 4.6.2.1Experimentsettingandbenchmarkloadingrules ..... 117 4.6.2.2Numericalresults ...................... 118 4.7TechnicalConstraintsandApplications .................... 120 4.7.1TheImpactofTechnicalConstraints ................. 120 4.7.2AnApplicationExample ........................ 122 6

PAGE 7

4.8ConclusionandFutureWork ......................... 125 5CONCLUSIONS ................................... 127 APPENDIX APROOFSINCHAPTER2 .............................. 129 A.1PreliminaryLemmas .............................. 129 A.2ProofofTheorem 2.4 ............................. 130 A.3ProofofTheorem 2.6 ............................. 132 A.4ProofofTheorem 2.7 ............................. 133 BPROOFSINCHAPTER3 .............................. 135 B.1ProofofTheorem 3.3 ............................. 135 B.2ProofofTheorem 3.2 andTheorem 3.4 ................... 137 B.3ProofofLemma 3.2 .............................. 148 B.4ProofofTheorem 3.5 ............................. 149 CPROOFSINCHAPTER4 .............................. 152 C.1ProofofLemma 4.1 .............................. 152 C.2ProofofLemma 4.2 .............................. 152 REFERENCES ....................................... 153 BIOGRAPHICALSKETCH ................................ 158 7

PAGE 8

LISTOFTABLES Table page 2-1Thresholdvaluedifferences:z(m,n))]TJ /F3 11.955 Tf 11.95 0 Td[(w(m,n). ................... 41 2-2Averagevaluesofjobsassignedtoresourceswithoutandwiththepostponementoption. ......................................... 48 2-3Jobassignmentswithoutandwiththepostponementoption. .......... 50 4-1DesignofQ. ..................................... 118 4-2DesignofQ. ..................................... 120 4-3SlotPositionsofMD-11Freighter. ......................... 124 4-4LoadingPlan. ..................................... 125 8

PAGE 9

LISTOFFIGURES Figure page 2-1DiscountedprotimprovementwithX10Beta(5,2)(rightskewedF,homogeneousresources). ...................................... 41 2-2DiscountedprotimprovementwithX10Beta(2,5)(leftskewedF,homogeneousresources). ...................................... 42 2-3DiscountedprotimprovementwithX10Beta(2,5)(leftskewedF,heterogeneousresources,andHeuristics2). ............................ 49 3-1Valuesofgi(r,t),fori=1,2,...,15 ........................ 81 3-2ThresholdsoftheDSKPwithandwithoutpostponementoptions. ....... 81 3-3Expectedprotofexistingmodel( KleywegtandPapastavrou ( 1998 )):V(m,t). ............................................. 83 3-4ExpectedProtofexistingmodel( KleywegtandPapastavrou ( 1998 )):V(k,t))]TJ /F3 11.955 Tf -413.78 -14.45 Td[(V(k)]TJ /F4 11.955 Tf 11.96 0 Td[(1,t). ..................................... 84 3-5Expectedprotofthenewmodel:gk(0,t))]TJ /F3 11.955 Tf 11.95 0 Td[(gk)]TJ /F5 7.97 Tf 6.58 0 Td[(1(0,t). ............. 85 3-6P.D.FofR=20,Beta(1.5,3.0). ........................ 86 3-7Percentageofprotimprovement. ......................... 87 3-8ImpactofHoldingCostonProtImprovement. .................. 87 3-9ImpactofDelayCostonProtImprovement. ................... 88 3-10ValueofDelay. .................................... 89 4-1Threedoorcongurationsforacrafthull. ..................... 98 4-2HullcongurationofB757-200Freighter. ...................... 103 4-3DecisionTreeoftheGeneralDPModel. ...................... 106 4-4A1NContainer .................................... 117 4-5DistributionofCargoCGDisplacementw.r.tSlotCenter ............. 119 4-6ImpactofQ ...................................... 120 4-7ImpactofTippingConstraintonBalance ...................... 122 4-8ImpactofTippingConstraintonWeightDistribution ................ 122 4-9SlotCongurationofMD-11Freighter ....................... 123 9

PAGE 10

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophySTOCHASTICANDDYNAMICSEQUENTIALDECISIONPROBLEMSWITHPOSTPONEMENTOPTIONSByTiankeFengAugust2012Chair:JosephC.HartmanMajor:IndustrialandSystemsEngineeringPostponementnaturallyarisesasanoptionfordecisionsunderuncertainty,asitenablestheacquisitionofmoreinformationforbetterdecisions.Thevalueofpostponementoptionshasbeenwidelyidentiedinnanceandoperationsmanagement.However,theresearchonpostponementoptionsinsequentialdecisions,especiallyinoperationssettings,isrelativelylimited.Thisdiscussionincorporatesthepostponementoptionintodecision-makingandstudiesitsvalueinthreesequentialdecisionssettings,i.e.,SequentialandStochasticAssignmentProblem(SSAP),DynamicStochasticKnapsackProblem(DSKP),andDynamicStochasticLoadingProblem(DSLP),involvingboththeoreticalanalysisanddesignofalgorithmsforpracticalapplications.TheSSAPandtheDSKPaddresstheissueofallocatingalimitednumberofresourcestodemandsarrivingsequentiallyovertime,withtheobjectiveofachievingthemaximumprot.Optimalpolicieswiththresholdstructureshavebeenwellestablishedforthemintheliteratureandhavewideapplicationsinlogistics,nanceandhealthcare.Thepostponementoptiongreatlycomplicatestheanalysisandtheresultingproblemsarehardingeneral.Weanalyzetheoptimalpoliciesforthesetwoproblemsinspecialcasesthroughstochasticdynamicprogramming,anddevelopeffectiveheuristicpoliciesforthegeneralcase.Numericalexperimentsshowconsiderableprotimprovementfromthepostponementoption.Webelievethattheseresultshaveimplicationsonrealoptionsandtherevenuemanagement.Moreover,theyshedlightonthestudyoftheDSLP. 10

PAGE 11

TheDSLParisesfromthecargoloadingoperationintheairfreightindustry.Intheprocessofcargoloading,theloadplannerneedstoachieveagoodbalanceofthecargocontainers,whilerespectingtechnicalconstraints.Traditionalresearchassumesthatallcontainers'weightinformationareknowninadvance,whileinpractice,theloadingprocessstartswhenasmallportionofcontainersarrive,duetothevalueoftime.Ateachmomentinthisprocess,thegroundstaffmaydelaytheloadingandwaitformorecontainerstoarrive.However,fewanalyticalresearchonthisproblemisavailableintheliteratureandthegroundstaffmainlyrelyonempiricalrulesand,sometimes,lucktoachievethetargets.Theproblemishardandweillustratethelimitsofthestandardoptimizationmethodologiesindealingwiththisproblemanddevelopaviablecomputationalframework,basedoninsightsfoundoutfromthestudyontheSSAPandtheDSKP. 11

PAGE 12

CHAPTER1INTRODUCTIONPostponementnaturallyariseasanoptioninthecontextofnanceandoperationsmanagement.Theliteratureinnanceusuallymentionspostponementwiththeexibilityinrealoptionsandarguesitsprot-enhancingcapability.Asstatedby Triantis ( 2000 ),realoptionsareopportunitiestodelayandadjustinvestmentsandoperatingdecisionsovertimeinresponsetoresolutionofuncertainty. AmramandKulatilaka ( 1999 )and Trigeorgis ( 1996 )illustratethebenetsofpostponementinreducinguncertaintyandincreasingprot. McDonaldandSiegel ( 1986 )modelthetimingdecisionsinirreversibleprojectinvestments. McGrath ( 1997 )pointsoutthetimingdilemmasfacedbycompaniesininvestmentdecisionsinnewtechnology:movinglatemayleadtothelossofcompetitiveadvantagewhilemovingearlycarriesgreateruncertaintyofinthemarket. GrenadierandWeiss ( 1997 )developamodelofoptimalinvestmentstrategyforacompanyfacingasequenceoftechnologicalinnovations.Thefoundationalworkofpostponementinoperationsmanagementcanbeattributedto Bucklin ( 1965 ).Theauthorstudiestheroleofpostponementinmarketingchannels. FeitzingerandLee ( 1997 )illustratetheeffectofpostponementtorealizecustomizationinsupplychainsthroughacasestudy. DingandKouvelis ( 2001 )analyzetheroleofdelayingcapacityallocationashedgingagainsttheexchangeraterisk. HuchzermeierandLoch ( 2001 )studytheoperationalvariabilitiesinR&Dmanagementandtheirimplicationsondelaycommitments.Acomprehensiveliteraturereviewofpostponementinoperationsmanagementispresentedby VanHoek ( 2001 ).Thestudyofpostponementinsequentialdecisionsislimited,toourknowledge,thoughinmanycases,theoptionofpostponingadecisionisoftenavailabletothedecisionmaker.Similartosituationsofinvestmentandsupplychain,uncertaintiesexistindynamicandstochasticsystems,inwhichinformationarrivessequentiallyanddecisionsshouldbemadewithoutafullrevelationofinformation.Thepostponement 12

PAGE 13

optionenablesthechancetogathermoreinformationtomakebetterdecisions.Itisalsonoticedthateveninthenanceandoperationsmanagement,wherethestudiesaboutpostponementoptionsabounds,nemathematicalmodelsorformulationsofpostponementarenotrich.Manystudiesarequalitativeorempirical,ascanbeseenin VanHoek ( 2001 ).Thisdissertationstudiespostponementoptionsinthreesequentialdecisionproblems:1)SequentialandStochasticAssignmentProblem;2)DynamicandStochasticKnapsackProblem;and3)DynamicandStochasticBalancedLoadingProblem.Thersttwoproblemshavebeenwellstudiedinoperationsresearch,whilethelastonearisesfromarealproblemintheairfreightindustrythatisnotmentionedmuchintheresearchliterature.Inthisdissertation,westudythebenetofthepostponementoptionandhowtoexploitthisbenetinthesethreeproblems. 1.1PostponementOptionsinSequentialDecisions 1.1.1TheSequentialandStochasticAssignmentProblemTheSequentialStochasticAssignmentProblem(SSAP)haswideapplicationsinlogistics,nanceandhealthcaremanagement,andhasbeenwellstudiedintheliterature.Itassumesthatjobswithunknownvaluearriveaccordingtoastochasticprocess.Uponarrival,ajob'svalueismadeknownandthedecisionmakermustimmediatelydecidewhethertoacceptorrejectthejoband,ifaccepted,toassignittoaresourceforareward.Theobjectiveistomaximizetheexpectedrewardfromtheavailableresources.Theoptimalassignmentpolicyhasathresholdstructureandcanbecomputedinpolynomialtime.Inreality,thereexistsituationsinwhichthedecisionmakermaypostponetheaccept/rejectdecision,suchthatmorevaluableresourcesareallocatedtomorelucrativejobs. 1.1.2TheDynamicandStochasticKnapsackProblemTheDynamicandStochasticKnapsackProblem(DSKP)isdenedasfollows:AsequenceofitemsarriveaccordingtoaPoissonprocesswithresourcerequirements 13

PAGE 14

andassociatedrewardsforeachitemmadeknownuponarrival,whereuponthedecisionmaker,withlimitedcapacity,mustimmediatelydecidewhethertoacceptorrejecttheitem.Anyunutilizedcapacityincursaholdingcostuntilthedecisionprocessterminates.Theobjectiveistomaximizetherewardearned,withoptimalthresholdpolicieshavingbeendenedinearlierwork(e.g., KleywegtandPapastavrou ( 2001 )).TypicalapplicationsoftheDSKPmodelincludedispatchingtransporters,schedulingofbatchprocessors,assetmanagement,andinvestment.Theintroductionofthepostponementoptionenablestheexibilityofmakingdecisionsbetweenarrivalsofitemsinthesituationwithdeadlines,besidesitsexibilityincollectingmoreinformationforbetterdecisions.ThismakesthepresentedDSKPmodelbettersuitedfortheapplicationsinwhichtheopportunitycostofholdingresourcesisnotnegligible(e.g.,sellingrealestateandcapitalbudgeting).Moreover,thismodelcanpotentiallyshedlightonthestudyofrealoptions.Realoptionsanalysisonlyconsidersasingleinvestmentopportunity.Inthecontextofourknapsackproblem,onecanconsiderthedelayoptionwithaportfolioofprojects,consideringtheirinterdependencywithregardstoabudgetingconstraint. 1.1.3DynamicandStochasticContainerLoadingProblemTheDynamicandStochasticContainerLoadingProblem(DSLP)arisesinthecontainerloadingoperationoftheairfreightindustry.Theobjectiveistoachieveagoodbalanceofthecargocontainers,whilesatisfyinganumberofconstraints.Mostliteratureconcerningthecontainerloadingproblemassumesthatthegroundstaffwaituntilallcontainersarrive,withweightsknown,andthenstartloadingcontainers.Inpractice,however,thetimevalueoftheairfreightindustrymotivatestheloadingprocesstostartfarbeforethearrivalofallcargocontainers.Andthisdenesthedynamismandstochasticnatureoftheproblem.Formally,theDSLPisdenedasfollows:Agivennumberofcontainersofidenticalsize,butrandomweight,arriveaccordingtoastochasticprocessintime.Eitherbefore 14

PAGE 15

oratagivendeadline(whichmaybebeforeallofthecontainershavearrived),thecontainersmustbegintobeloaded,oneatatime.Theorderinwhichthecontainersareloadedisconstrainedanddenestheultimateobjectiveofbalancingthecontainerloadalongagivenaxis.Inthissituation,theimmediatedecisionassumptionutilizedinthetraditionalSSAPandthetraditionalDSKP,mayyieldratherpoorbalance,asitsimplyloadsacontainersatitsarrival.Inpractice,however,thegroundstaffusuallychoosetopostponeloadingacontaineratitsarrivalsuchthatabetterbalanceisachievedandtheconstraintsaresatised.However,thecurrentpracticeisgenerallybasedonrulesofthumbderivedfromexperienceanditsperformanceisnotquitesatisfactory,aswillbeillustratedinthisdissertation. 1.2FeaturesofthePostponementOptionandRelatedMethodologiesThethreeproblemsstudiedinthisthesissharesomecommonfeatures: 1. Asequenceofitemsarrivesaccordingtoastochasticprocess; 2. Theattributesofeachitemareunknownbeforeitsarrival; 3. Adecisionshouldbemadeforeachitem(e.g.,acceptedorassigned),witharewardorcostassociatedwithitsattributesincurred; 4. Thereisanitetimehorizon.Asmentioned,theSSAPandDSKPhavebeensolvedforthecasewhenthedecisionaremadeuponanitem'sarrival.Thisdissertationallowsthedecisionmakertodelaythedecisionforsomeamountoftime.Aconsequenceofthisrelaxationisthattheitemswiththeirdecisionspostponedformaqueue.Comparedwiththesituationrequiringanimmediatedecisionuponanitem'sarrival,thequeuesignicantlycomplicatestheanalysis.Fromtheperspectiveofdynamicprogramming,thequeueenlargesthedimensionsofthesystemstateandthedecisionset.Thus,thetractabilityoftheproblemsuffers,asaresult. 15

PAGE 16

Thethreeproblemsbothpossessdynamicandcombinatorialtraits.Inoperationsresearch,thistypeofproblemisusuallyformulatedwithtworelatedapproaches:Markovdecisionprocesses(MDP)andmulti-stagestochasticprogramming(MSP).Herewereplacethenameoftherstapproachbythemoregeneraltermofoptimalcontrol.Thedevelopmentofthetheoryofoptimalcontrolcanbetracedbacktothecalculusofvariations.TheMaximumPrinciple,discoveredbyL.S.Pontryagininthe1950slabeledoptimalcontrolasadistincteldofresearch.Oneofthemaineffortinthiseldisthestudyoftheoptimalityconditionsundervariousspecicproblemsettings,besidesthestudyonregularityandstability. Vinter ( 2000 )presentsadetaileddiscussiontothesetopics.AnotherapproachtoestablishtheoptimalityconditionisthroughdynamicprogrammingorHamilton-Jacobi-Bellmanequation.Theoperationsresearchcommunityfavorsthedynamicprogrammingapproachandastate-describedformulation,whiletheeldofcontrolengineeringprefersthemaximumprincipleandaninput-and-outputformulation. Whittle ( 1996 )motivatesaninsightfuldescriptionontheconnectionbetweenthesetworepresentations.Asstatedby Vinter ( 2000 )and Whittle ( 1996 ),thoughtheoptimalityconditioniswellestablished,solvingtheoptimalcontrolproblemisextremelydifcultandincludesaspectsofanart.Computationalmethodsaredevelopedinbothareas,including SuttonandBarto ( 1998 ), Bertsekas ( 1995 )and Powell ( 1994 ).Usually,theapplicationoftheMDPsisrestrictedbytheexponentiallygrowingnumberofstates(knownasthecurseofdimensionality).Approximationandheuristicsalgorithms,includingthelearning,nero-networkandstochasticgradientmethod,requirealongtimeforthesolutiontoconvergeandthequalityofthesolutionisusuallynotguaranteed.Anotherlineofrelatedresearchcomesfromdynamicschedulingproblemsinthequeuingtheorycommunity.However,theproblemsstudiedinthisdissertationdifferfromthequeuingmodelsintwoaspects.First,queuingtheoryismainlyconcernedwithwaitingtimesofcustomersorqueuelengthsinaqueuingsystem,ascanbeseen 16

PAGE 17

in HeymanandSobel ( 2007 )and Serfozo ( 1999 ).Second,thedynamicschedulingproblemusuallyassumeaninnitehorizonandtheresearchersaremainlyconcernedabouttheperformanceofthesysteminsteadystate,asdiscussedby ChenandYao ( 1993 ), Wein ( 1992 )and GajratandHordijk ( 2000 ).Theproblemsinthisdissertationstudythetotalcostorrewardofatransitional,notsteadystate,environment.Thesecondapproachisthemulti-stagestochasticprogramming. Ruszczynskiand ( eds )presentadetailedsummaryonthistopic.Multi-stagestochasticprogrammingcanbemathematicallyrelatedtostochasticdynamicprogramming,buttheformeradoptstheresearchresultsofthemathematicalprogrammingcommunity.Itshouldbenotedthattheapplicationofstochasticprogrammingingeneralrequiresaxednumberofdecisionstages.Intheproblemsdiscussedhere,however,thenumberofstagesisstochasticallydeterminedbythecurrentdecision,futureinformationanddecisionsinfuturestages.Insummary,thedifcultyofthepresentedproblemsareill-suitedforcurrentsolutiontechniques,whichhasledtothedevelopementofnewtechniques. 1.3ChapterOutlineandSummaryofResultsThisdissertationisorganizedasfollows.Chapter2discussestheSSAPwiththepostponementoption.Theproblemismodeledthroughstochasticdynamicprogramming.Itisshownthatforthecaseofhomogeneousresources,thereexistsanoptimalpolicypossessingathresholdstructure.Basedonthispolicyandtheinsightsderivedfromthiscase,aheuristicalgorithmisderivedforthecasewithheterogeneousresources,whichisthencomparedwithotherheuristics.Numericalexperimentsindicatethatsimpledecisionrulesofpostponementmaynotresultinasignicantimprovement.Instead,thetheoremsderivedfromthecasewithhomogeneousresourcesarevaluableindesigningeffectiveheuristicschemes.Chapter3mainlystudiestheDSKPwiththepostponementoptionforthecasewhenallitemsarehomogeneousinsize.TheproblemisrstmodeledasacontrolledMarkov 17

PAGE 18

process,inwhichitisfoundthattheoptimalpolicyofarelatedMarkovdecisionprocessisalsooptimal.Thenanoptimalpolicywithathresholdstructureisfound.Weutilizethispolicytoderiveeffectiveheuristicalgorithmtosolvethecasewithrandomlysizeditems.Chapter4discussestheDSLP.Wendthatitisdifculttoderiveoptimalpoliciesbearingathresholdstructure.Wediscussthepossibilityofapplyingapproximatedynamicprogramming,multi-stagestochasticprogramming,andon-lineoptimization,andpointoutthelimitationoftheseoptimizationmethodsinsolvingtheDSLP.Then,wederiveaheuristicalgorithmbasedonthetwo-stagestochasticmixedintegerprogrammingapproachtosolvetheproblemanddiscusstheapplicabilityofthisalgorithminrealworldcargoloadingoperations. 18

PAGE 19

CHAPTER2THESEQUENTIALANDSTOCHASTICASSIGNMENTPROBLEMWITHPOSTPONEMENTOPTIONS 2.1IntroductionandMotivationTheclassicsequentialstochasticassignmentproblem(SSAP)isdenedasfollows.ThereareMresourcesavailable,havingqualityvaluesp1,p2,...,pM(or,asaspecialcase,Midenticalresourcesallhavingthesamequalityvaluep).Anumberofjobs,N(typicallygreaterthanM),arriveaccordingtoastochasticprocessandhavevaluesX1,...,XN.ThesevaluesareknownaprioritobeindependentlyandidenticallydistributedwithknowndistributionfunctionF.Uponthearrivaloftheithjobattimet,thevalueofXiismadeknownandthedecisionmakermustimmediatelyacceptorrejectthejob.Ifajobisacceptedandassignedtoaresourcewithqualityvaluepj,theassignmentoccursimmediately,yieldingareward(t)pjXi,where(t)isadiscountingfunction.Declinedjobsarelostforeverandassignedresourcescannotberecalled.Theobjectiveistomaximizetheexpecteddiscountedrewardofallassignments.Whiletherearemanyapplicationswheretheassignmentmustoccurimmediately(orwithaminimaldelay),suchaskidneyallocationorpacketswitching,thereexistmanyapplicationsinwhichthedecisionmakermaypostponetheassignmentdecision.Forexample,intheclassicalsecretaryproblem(oftenreferredtoasthelabormarketanalysisorjobsearchproblem),theofferneedsnottobeimmediate.Rather,anumberofcandidatescanbeinterviewedbeforeadecisionofferismade.Fromtheperspectiveofoneinterviewingforanumberofjobs,theymaywaitformultipleoffersbeforechoosingone.Inmanyassetsellingorbiddingprocesses,multiplebidsmaybereceivedbeforeoneisaccepted.Finally,itmaybebenecialinproductionsettingstodelaytheassignmentofajobtoamachinegiventhepotentialofmorelucrativejobarrivals.Itshouldbeclearthattheoptiontopostponecanbringhigherrewardsandreducerisk.Ontheotherhand,ifsolutionsaretimesensitive,postponementmayreducetheamountofthereward.Thedecisionmakermustbalancetheseissues. 19

PAGE 20

Toourknowledge,noresearchhasbeenpublishedontheSSAPproblemwheredecisionsmaybepostponed.Inthisthesis,westudythevalueofpostponementassumingthatjobsarriveaccordingtoaPoissonprocessandweincludeadiscountfactortoacknowledgethatrewardsmaybedelayed.Undertheseassumptions,wemakethefollowingthreecontributionstotheliterature: 1. ModeltheSSAPwiththepostponementoptionforhomogeneousresourcesandderiveanoptimalthresholdpolicy.Interestingly,thispolicyisonlydependentonthehighestvaluedjobinthequeue; 2. ModeltheSSAPwiththepostponementoptionforheterogeneousresources,investigatetheoptimalpolicyandproposeheuristicsolutions; 3. Illustratethebenetofthepostponementoptionwithbothhomogeneousandheterogeneousresources.Thischapterisorganizedasfollows.AfterabriefreviewoftherelevantliteratureinSection 2.2 ,weintroducenotation,assumptionsandourmodelinSection 2.3 .InSection 2.4 ,theoptimalthresholdpolicyfortheSSAPwithhomogeneousresourcesisdevelopedthroughinduction.Wecomparethispolicywiththeoptimalpolicyrequiringanimmediatedecision.Conditionsunderwhichpostponementissuperiorareidentied.InSection 2.5 ,theSSAPwithheterogeneousresourcesisdiscussedandnumericalexamplesillustratingthebenetsofpostponementareprovided. 2.2LiteratureReviewIntheirseminalworkontheSSAP, Dermanetal. ( 1972 )studythecaseofN=Mwithoutadeadlineordiscountinganddeneanoptimalthresholdpolicy.Specically,attheinitialstagewithresourcevaluesorderedasp1pM,thesupportofFcanbepartitionedintoMnon-overlappingintervalsandtheoptimaldecisionistoassignthejobvaluedatxtotheresourcevaluedatpiifxfallswithintheithinterval.Moreover,thethresholdvaluesusedtoobtaintheseintervalsonlydependonMandFandcanbecomputedinO(M2)time0.Assignmentdecisionsforlaterarrivalsfollowthesame 20

PAGE 21

rulewithareducedM.Theauthorsshowthatthethresholdstructureholdswhentheassignmentrewardcanbegeneralizedasf(x,p). Albright ( 1974 )extendstheworktoinnitejobarrivalsanddiscounting. Sakaguchi ( 1984a )extendstheworkof Albright ( 1974 )tothecaseinwhichjobsareniteinnumberandarriveaccordingtoanon-homogeneousPoissonprocess. Sakaguchi ( 1984b )permitsthenumberofavailableresourcestobeunknown. Albright ( 1976 )studiesthesecretaryproblemwherethebestMsecretariesaretobeselectedfromNavailablesecretaries.Themodelinthisworkassumesthevalueofthesecretariescomefromdifferentdistributions,withtwosuccessivesecretaries'valuesgovernedbyaMarkovChain. Nakai ( 1986a b )studiesthecasewiththedistributionofresourcesvaryingaccordingtoapartiallyobservableMarkovprocess. Albright ( 1977 )considersthecasewhentheparametersofthedistributionofthejobvaluearenotfullyknownandallowstheparameterstobeupdatedthroughaBayesianmodel. Kennedy ( 1986 )permitsjobvaluestobedependent. Righter ( 1987 )studiesthecasewhereeachpersonhasindependentdeadlinesandcomparesthismodelwiththediscountmodel. Righter ( 1989 )permitsthearrivalrate,thejob'svalue,andthevariabilityofjobvaluestochangeaccordingtoindependentMarkovprocesses.Additionally, AlbrightandDerman. ( 1972 )and Saario ( 1985 )studytheasymptoticresultsoftheSSAP.Thedynamicandstochasticknapsackproblem(DSKP)withhomogeneoussizeditemsstudiedby KleywegtandPapastavrou ( 1998 )iscloselyrelatedtoSSAP.Bysettingthecostsappropriately,thethresholdpolicydevelopedthereholdsfortheSSAPwithhomogeneousresources. NikolaevandJacobson ( 2010 )extendedtheresultsoftheSSAPandtheDSKPtothecasewitharandomnumberofarrivals.AvarietyofapplicationsutilizetheSSAP.Forexample,theresourcecanbereplacedbyhousestosellandjobscanbereplacedbypurchaseofferswithrandomvaluesarrivingatrandomtimes. Elfving ( 1967 )usesthisinterpretationforthecasewithM=1. McLayetal. ( 2009 )applytheoptimalpolicyoftheSSAPtotheallocationof 21

PAGE 22

screeningresourcestopassengers.ThestudiesoftheSSAPalsobringinsighttotheorganallocationprobleminhealthcaremanagement,including Zeniosetal. ( 2000 )and SuandZenios ( 2005 ).Thevalueofpostponingdecisionsiswidelyidentiedinthenance( Trigeorgis 1996 )andoperationsmanagement( VanHoek 2001 )literature.Innance,thisisgenerallyinthecontextofrealoptions.Asstatedby Triantis ( 2000 ),realoptionsareopportunitiestodelayandadjustinvestmentsandoperatingdecisionsovertimeinresponsetoresolutionofuncertainty. AmramandKulatilaka ( 1999 ),and Trigeorgis ( 1996 )illustratethebenetsofpostponementinreducinguncertaintyandincreasingprot.Whilesimilar,thepostponementperiodinthesesituationsisgenerallyoveramuchlongertimeframe(months,quarters,oryears)thaninourenvisionedapplications. 2.3AssumptionsandModelDenitionsWiththepostponementoption,uponthearrivalofajob,thedecisionmakerhasthreechoicesuponthearrivalofajob:(1)Rejectthejob;(2)Acceptthejobandassignittoaresource;(3)Postponetheaccept/rejectdecision.Ifthedecisionistopostpone,(1)and(2)areavailableinthefuture.Theproblemendswheneitheralljobshavearrivedorallresourceshavebeenassigned.Inthispaper,westudytheSSAPwiththepostponementoptionunderthefollowingassumptions: 1. ThevaluesX1,...,XNaredistributedasi.i.drandomvariablestakingvaluesin[0,1],withadistributionfunctionF(thisinformationisknownapriori); 2. JobsarriveaccordingtoaPoissonprocesswitharrivalrateandthereisnodeadline; 3. AtmostMresourcescanbeassignedandatotalofNjobsarrive; 4. Rewardsarediscountedcontinuouslybyapositivefactor,i.e.,(t)=exp()]TJ /F8 11.955 Tf 9.3 0 Td[(t); 5. Theinter-arrivaltimeofajobisindependentofitsvalue; 6. Alldecisionsareimplementedinstantaneously. 22

PAGE 23

Notethatthecaseof=0reducestothestaticproblem,asonecouldwaitforalljobs(postponedecisions)andthenmakeassignmentsatnopenalty.Wedonotconsiderthiscase.WemodeltheproblemasaMarkovdecisionprocesses(MDPs).Withthepostponementoption,aqueueofjobsismaintainedinadditiontothequeueofavailableresources(asinthetraditionalSSAPwithimmediatedecisions).Clearly,thisqueueofjobsgreatlycomplicatestheanalysis,asitsignicantlyenlargesthedecisionspace. 2.3.1StateVariableSGiventhenumberofremainingresources,m,andthenumberofjobsyettoarrive,n,attimet,thestateofthesystemisdenedasS=(xm,pm,n,t).Specically,xm2Rmrepresentsthevaluesofthejobsthathavearrivedbuthavenotbeenacceptedorrejected;similarly,pm2Rmrepresentsthequalityvaluesofresourcesthathavenotbeenallocated.Essentially,xmdenesaqueueoflengthmwhereuponjobarrivalsareinsertedaccordingtotheirvaluexsuchthatxm=(x1,...,xm)andx1x2xm;pmdenesaqueueofavailableresourceswithpm=(p1,...,pm)andp1p2pm.Letushenceforthrefertoaqueuedjobthathastheithsmallestvalueas`jobi'andanavailableresourcethathasthejthsmallestqualityvalueas`resourcej'.Avalueofxi=0signiesajobwithvalue0orthecasewhentherearelessthanmjobsinthequeue.Att=0,mM,nNandxi0,8i.DenoteSmasthestatespacegivenmavailableresources. 2.3.2Policies,StateTransitionsandDecisionEpochsUponthearrivalofajobwithvaluey,thejobwiththelowestvalue(x1ory)isrejected(andremovedfromthequeue,xm)asitshouldbeclearthatitcannotappearinanyoptimalsolutionwithmresources.Iffewerthanmjobsareinthequeue,thedeletionofx1=0merelyentailsremovingaplaceholder.Furthermore,withthedelayoption,itshouldbeclearthatitcannotbeoptimaltorejectanyjobuponitsarrivalifitsvalueisgreaterthanx1,asonecanretainitinthequeueanddeletex1.Wedenethe 23

PAGE 24

twooperationsofinsertionanddeletionasupdateanddenoteitwiththeoperatorU,where U(xm,y)=8>>>><>>>>:(x1,x2,...,xm),ifyx1(x2,...,xi,y,xi+1,...,xm),ifxiyxi+1,i=1,2,...,m(x2,...,xm,y),ifxm
PAGE 25

decision,eithermornisreduced.Thedecisionprocessterminateswheneithermorngoesto0.Ifm=0andn0,theremainingjobsthathavenotyetarrivedareignored.Ifm>0andn=0,thedecisiontopostponeisnolongerbenecialasalljobshavearrived.Decisionepochs(whenadecisionismade)occurwheneverthestatechangeseitheratthearrivalofanewjoborwhenajobisassignedtoaresource.Asaresult,jobsmaybeconsecutivelyacceptedandassignedtoresourcesatthesamepointintime,eachbelongingtodifferentdecisionepochs,sincetherstassignmentchangesthestateandposesanewdecisionproblemtothedecisionmaker.Moreover,afterpostponingandreceivinganewarrivingjob,thedecisionmakercanchoosetopostponeagain. 2.3.3ExpectedProtandDynamicProgrammingEquationDenePasthesetofpolicieswiththepostponementoptionand2Pastheoptimalpolicywiththepostponementoption.Forthesakeofconvenience,wedenetheexpectedprotofstate(xm,pm,n)asVm,n(xm,pm).Similarly,wedenotethedecisionruleasDm,n(xm,pm).Thedynamicprogrammingequationselectsthemaximumbetweentheexpectedprotofpostponinguntilthearrivalofanewjobandtheexpectedprotofimmediatelyacceptingajobfromthequeueandassigningittoaresource:Vm,n(xm,pm)=maxZ10Vm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F16 10.909 Tf 5 -8.84 Td[(U(xm,y),pmdF(y), (2)hrA(xm,pm)+Vm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F16 10.909 Tf 5 -8.83 Td[(A(xm,pm)i,where= +,Aistheoptimalassignmentrule,andrAistherewardwhenapplyingtheoptimalassignmentrule.TheboundaryconditionsaregivenasV0,n(xm,0)=0, (2)Vm,0(xm,pm)=mXi=1xipi. (2) 25

PAGE 26

Eq. 2 correspondstothesituationwhenresourcesaredepletedandEq. 2 correspondstothesituationwhennojobshavearrived.InEq. 2 ,thedecisionmakersimplyappliestherearrangementinequality(Theorem368, Hardyetal. ( 1934 )),whichstatesthatitisoptimaltoassignthehighestvaluedjobstothehighestvaluedresources,respectively. 2.4HomogeneousResourcesWithhomogeneousresources,weassumep1==pm=1withoutlossofgenerality.Also,theassignmentruleAonlyspeciesthequeuedjobstobeaccepted.Thus,inthissection,weuse`accept'tomean`acceptandassign'.Furthermore,wecanremovepmfromS.Weestablishtheoptimalthresholdpolicies2PfortheSSAPwiththepostponementoptioninSection 2.4.1 .Then,wecompareittothepolicyoftheSSAPwithoutthepostponementoptioninSection 2.4.2 .Finally,weillustratethedifferencebetweenthetwopoliciesthroughnumericalexperimentsinSection4.3. 2.4.1OptimalPolicywiththePostponementOptionTheoptimalthresholdpolicyfortheSSAPwiththepostponementoptionisestablishedthroughinduction.Werstestablishtheresultform=1andthenextendittothecasem2.Inthefollowinganalysis,weusetherightderivative,f0(x+),ofafunctionftostudyitsmonotonicity,sincetheexpectedprotfunctionsdonothavecontinuousderivatives.ThisapproachisnotcommonlyseeninstandardtextsandthuswestateitinLemma A.1 (AppendixA). 2.4.1.1Case1:m=1Deneasequenceoffunctionsfg1,n(),n=1,2,...gon[0,1]suchthatg1,n(x)=maxnEhg1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(max(x,X)i,xo, (2)whereg1,0(x)=x.Aswillbeseenlater,functiong1,n(x)istheexpectedprotofhavingoneresource,njobsyettoarrive,andajobvaluedatxonhand.Indeed,thetwotermsinthecurlybracketsofEq. 2 correspondtothetermsinEq. 2 .Moreover,g1,n(x) 26

PAGE 27

convergesasngoestoinnityandwedenethelimitasg1(x).Lemma 2.1 presentsthecharacteristicsofg1,n()thatareutilizedintheproofoftheoptimalpolicyinTheorem 2.1 Lemma2.1. Letz(1)bethevalueofxsuchthatx=Emaxfx,Xg.Therootiseasilyseentoexistandbeunique.ThefunctionsequencedenedinEq. 2 possessesthefollowingproperties: 1. g1,n(x)isstrictlyincreasinginxforx2[0,z(1))andn=1,2,...; 2. g1,n(x)isstrictlyincreasinginnforx2[0,z(1)); 3. g1,n(x)>xforx2[0,z(1))andg1,n(z(1))=z(1)forn=1,2,...; 4. 1>g01,n(x)>g01,n+1(x)forx2(0,z(1))andn=1,2,...; 5. g1,n(x)!z(1)asn!+1forx2[0,z(1)); 6. g01,n(x+)isnon-decreasinginxforx2(0,1)andn=1,2,.... Proof. Itiseasytoseebyinductionthatg1,n(z(1))=z(1)foralln,andalsothatg1,n(x)=xforxz(1).Also,Eg1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(max(x,X)isstrictlyconvexinx,andsog1,n(x)>xforx<>:x,xz(1)g1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)F(x)+R1xg1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(y)dF(y),x
PAGE 28

Thus,g01(x)=g01(x)F(x).Thisholdsonlywheng01(x)=0,since<1and0g01(x)<1forx
PAGE 29

straightforwardforthecasewithm>n,becausejobmwillinvariablybeassigned,evenifallnjobsthatareyettoarrivehavevaluesgreaterthanthatofjobm.Knowingthis,onemayaswellassignjobmimmediately.Thecasewithmnismuchmorecomplicated.Theoptimaldecisiondependsontheprotadvantageofthepostponementoption:Gm,n(xm)=Z10Vm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1U(xm,y)dF(y))]TJ /F14 11.955 Tf 11.95 16.86 Td[(rA(xm)+Vm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,nA(xm)=hVm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x1,...,xm)F(x1)+Zx2x1Vm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(y,x2,...,xm)dF(y)++Z1xmVm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x2,...,xm,y)dF(y)i)]TJ /F14 11.955 Tf 11.96 13.27 Td[(nVm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n(x1,x2,...,xm)]TJ /F5 7.97 Tf 6.58 0 Td[(1)+xmo, (2)whichisindeedthedifferenceofthetwoexpressionsinthedynamicprogrammingequationdenedinEq. 2 .ItcanbeshownthatthesignofGm,n(xm)onlyde-pendsonthevalueofjobminthequeueandthereexistsauniquez(m,n)suchthatGm,n(x1,...,xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,z(m,n))=0,whichdenesathresholdformakingthedecision.Moreover,theexpectedprotisseparablewithrespecttocomponentsofxm:Vm,n(xm)=Pmk=1gk,n(xk),wheregk,n(xk)isasequenceoffunctionsthatcanbeestablishedrecursivelywithg1,n(x)denedinSection 2.4.1.1 .Finally,asngoestoinnity,gm,n(x)converges,withthelimitdenotedasgm(x).Theseresults,includingpropertiesoftheexpectedprotfunction,aresummarizedinTheorem 2.2 Theorem2.2. Givenxm=(x1,x2,...,xm),m2andn=1,2,...,theoptimaldecision,Dm,n(xm),istoimmediatelyacceptjobmifxmz(m,n)orotherwisetopostponeuntilthearrivalofthenextjob,whereform>n,z(m,n)=0andformn,z(m,n)istheuniquevaluesatisfyingGm,n(x1,...,xm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,z(m,n))=0.Moreover,0=z(m,1)==z(m,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1)
PAGE 30

Theexpectedprotis Vm,n(xm)=gm,n(xm)+Vm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n(xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1).(2)wheregm,n(xm)=8>>>><>>>>:xm,xm2[z(m,n),1]hgm,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(xm)F(xm)+R1xmgm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(y)dF(y)i+R1xmhgm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(xm))]TJ /F16 10.909 Tf 10.91 0 Td[(gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(y)idF(y),xm2[0,z(m,n)). (2)Also,(@=@x+i)Vm,n(xm)=g0i,n(x+i),i=1,2,...,m,andsatises0g01,n(x+1)g0m,n(x+m)g0m,n+1(x+m)1. (2)Finally,limn!+1z(m,n)=z(m)andlimn!+1gm,n(x)=gm(x).Inparticular,gm(x)=z(m)forxnhasbeendiscussed.Jobmshouldbeaccepted,regardlessofthevaluesofthenjobsyettoarrive.Denez(m,n)=0andgm,n(x)=x.Theremainderisstraightforwardforthecaseofm>n.Thecaseofmnismuchmorecomplicatedandwerstbrieyoutlineourapproach.Theproofusesinductiononmwiththecaseofm=2asthebasestep,sincethegeneralizedcasesofm2signicantlydifferfromthecaseofm=1.Ineachstepoftheinductiononm,weestablishtheresultsthroughanotherinductiononn.Specically,ineachinductionstep,theproofisseparatedintotwoparts: (i) Decisionrules,thresholds(Eq. 2 ),andexpectedprots(Eq. 2 andEq. 2 ); (ii) Propertiesofthepartialderivativesoftheexpectedprots(Eq. 2 ).For(i),weshowthatforxm2[z(m)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),1],Gm,n(xm)<0andforxm2[0,z(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)),thevalueofGm,n(xm)onlydependsonxm.Moreover,forxm2[0,z(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)),Gm,n(xm)is 30

PAGE 31

positiveatxm=z(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),andismonotonicallydecreasinginxm.Asaresult,thereexistsauniquez(m,n)2[z(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),z(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))suchthatGm,n(x1,...,xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,z(m,n))=0.Theproofof(i)requiresthepropertiesofg0m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n(x+).Therefore,ineachstepoftheinduction,weanalyzethepropertiesin(ii)afterestablishing(i).Ascanbeexpected,theconclusionsforthecaseofm=2requirethepropertiesofg01,n(x+)inLemma 2.1 .Belowisthedetailedproofforthecaseofmn,followedbythecaseofinniten.Forthesakeofconvenience,wedenoteVPOSTm,n(xm)astheexpectedprotofpostponingandfollowingpolicythereafter(therstexpressioninthecurlybracketsofEq. 2 ).Westarttheanalysisfromthecaseofm=2.First,considertheoptimalassignmentruleA.Whenm=1,theassignmentruleAissimplytochoosetheonlyjobinthequeue,whilewhenm=2,therearetwochoices.SinceV1,n(x))]TJ /F3 11.955 Tf 12 0 Td[(xisnon-increasing(byProperty 4 ofLemma 2.1 andTheorem 2.1 ),x1x2indicatesV1,n(x1))]TJ /F3 11.955 Tf 10.46 0 Td[(x1V1,n(x2))]TJ /F3 11.955 Tf 10.45 0 Td[(x2orV1,n(x1)+x2V1,n(x2)+x1.Therefore,Aacceptsjob2.Considerthecasewhentherearetwojobsyettoarrive(n=2).SubstitutingV2,1(x2)=g2,1(x2)+g1,1(x1)intoEq. 2 ,wehaveG2,2(x2)=hg1,1(x1)+x2iF(x1)+Zx2x1hg1,1(y)+x2idF(y)+Z1x2hg1,1(x2)+yidF(y))]TJ /F14 10.909 Tf 10.91 12.11 Td[(hx2+V1,2(x1)i=hg1,1(x1)F(x1)+Z1x1g1,1(y)dF(y))]TJ /F14 10.909 Tf 10.91 14.85 Td[(Z1x2g1,1(y)dF(y)+Z1x2g1,1(x2)dF(y)i+hx2F(x2)+Z1x2ydF(y)i)]TJ /F14 10.909 Tf 10.91 12.11 Td[(hx2+V1,2(x1)i. 31

PAGE 32

NotethatVPOST1,2(x1)=Eg1,1)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(max(x,X)=hg1,1(x1)F(x1)+R1x1g1,1(y)dF(y)i.WefurtherrewriteG2,2(x2)asG2,2(x2)=hg2,1(x2)F(x2)+Z1x2g2,1(y)dF(y)i+Z1x2g1,1(x2))]TJ /F16 10.909 Tf 10.91 0 Td[(g1,1(y)dF(y)+VPOST1,2(x1))]TJ /F14 10.909 Tf 10.91 12.11 Td[(hx2+V1,2(x1)i (2)=hg2,1(x2)F(x2)+Z1x2g2,1(y)dF(y)i+Z1x2g1,1(x2))]TJ /F16 10.909 Tf 10.91 0 Td[(g1,1(y)dF(y))]TJ /F16 10.909 Tf 10.91 0 Td[(x2+nVPOST1,2(x1))]TJ /F16 10.909 Tf 10.91 0 Td[(V1,2(x1)o,whereg2,1(x)=xandtheexpressionintherstcurlybracketsofEq. 2 isindeedVPOST2,2(x2).Forx2z(1),sinceV1,2(x1)VPOST1,2(x1)andg1,1(x2)=g2,1(x2)=x2,G2,2(x2)hg2,1(x2)F(x2)+Z1x2g2,1(y)dF(y)i+Z1x2g1,1(x2))]TJ /F3 11.955 Tf 11.96 0 Td[(g1,1(y)dF(y))]TJ /F3 11.955 Tf 11.95 0 Td[(x2=()]TJ /F4 11.955 Tf 11.96 0 Td[(1)x2<0.Forx20.Also,forx2
PAGE 33

Theoptimaldecisionisthustoimmediatelyacceptjob2ifx2z(2,2),orotherwisetopostponeuntilthearrivalofanewjob.Notethatforx2z(2,2),V2,2(x1,x2)=x2+V1,2(x1);forx22.Additionally,supposethatg02,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x+2)<1forx2G2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x1,z(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)). 33

PAGE 34

ByProperty4ofLemma 2.1 ,g01,n(x)decreasesinnforxg1,1(z(2,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)))]TJ /F16 10.909 Tf 10.91 0 Td[(z(2,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)+Z1z(2,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)hg1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(2(z(2,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)))]TJ /F16 10.909 Tf 10.91 0 Td[(g1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(2(y)idF(y)=G2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x1,z(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))=0.Also,notethatlimx2"z(1)G2,n(x1,x2)<0.Therefore,thereexistsauniquez(2,n)2(z(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),z(1))suchthatG2,n(x1,z(2,n))=0.Forx2
PAGE 35

Sinceg02,1(x+)=1>g01,1(x+)forx2.Supposethat(i)and(ii)holdform)]TJ /F4 11.955 Tf 13.13 0 Td[(1.Inparticular,supposethatforx
PAGE 36

hypothesis),itfollowsthatGm,m(0)=Z10h)]TJ /F16 10.909 Tf 5 -8.84 Td[(gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,m)]TJ /F5 7.97 Tf 6.58 0 Td[(1(0)+y)]TJ /F16 10.909 Tf 10.91 0 Td[(gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1(y)idF(y)>0.Second,Gm,m(xm)isstrictlydecreasinginxmforxm
PAGE 37

Wenextshowthatforz(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)0.Notethatforx2[z(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),z(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)),gm,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)=gm,n)]TJ /F5 7.97 Tf 6.58 0 Td[(2(x)=x,anditfollowsbyEq. 2 andLemma A.2 thatGm,n(xm))]TJ /F3 11.955 Tf 11.95 0 Td[(Gm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(xm)=Z1xmhgm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(xm))]TJ /F3 11.955 Tf 11.96 0 Td[(gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(y)idF(y))]TJ /F8 11.955 Tf 11.96 0 Td[(Z1xmhgm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(2(xm))]TJ /F3 11.955 Tf 11.95 0 Td[(gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(2(y)idF(y)>0.SinceGm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x1,...,xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,z(m,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1))=0(bydenition),theconclusionfollows.Therefore,thereisauniquexm=z(m,n)in(z(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),z(m)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))suchthatGm,n(xm)=0.Functionsgm,n(xm)andVm,n(xm)canbeeasilyestablished.Tocompletetheinductionon(i)forthecaseofgeneralm,weneedtoshowthatforxm.Notethatforx
PAGE 38

Now,consider(ii).Basedonthepreviousdiscussion,itisstraightforwardthat0g0m,n(x+)1.Wenextshowthatg0m,n(x+)>g0m)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n(x+)forx
PAGE 39

istoimprovethequeuexmbyupdatingitwiththearrivalofanewjob.Thenumberofresourcestobeallocated,m,reectstheeffectofdiscounting.Thedecisionmakerismoreconservativeinpostponingwhenmincreases.Ontheotherhand,thenumberofjobsyettoarrive,n,measuresthepotentialforimprovement.Thedecisionmakerismoreselectivewhenacceptingajobasnincreases.Moreover,themonotonicityofg0m,n(x+)inmindicatesthattheimprovementonxmgivesthelargestprotincrease. 2.4.2TheBenetsofPostponementForthesakeofcomparison,weintroducethemodelfortheSSAPwithoutthepostponementoption,thestudyofwhichisabundantintheliterature(e.g.,Section2,3of Karlin ( 1962 ), Dermanetal. ( 1972 )and Sakaguchi ( 1984b )).Thesystemstateisdenotedas(x,pm,n),wherexisthevalueofthejobonhand.Decisionepochsoccurwhenjobsarrive.Whenajobvaluedatxarrives,ifthedecisionmakerimmediatelyacceptsthejobandassignsittoaresourcewithqualityvaluepi,thenarewardpixisreceived.Thestateofthesystemthentransitionsto)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(y,(p1,...,pi)]TJ /F5 7.97 Tf 6.59 0 Td[(1,pi+1,...,pm),n)]TJ /F4 11.955 Tf 12.05 0 Td[(1,whereyisthevalueofthenextarrivingjob.Ifthedecisionmakerimmediatelyrejectsthejob,thestatetransitionsto)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(y,pm,n)]TJ /F4 11.955 Tf 12.45 0 Td[(1.WedeneIMasthesetofpoliciesthatrequireanimmediatedecision(nopostponementoption)and astheoptimalpolicyinIM.Additionally,wedeneD m,n(x,pm)asthedecisionruleandV m,n(x,pm)astheexpectedprotofstate(x,pm,n).Moreover,denethethresholdvalueusedin asw(m,n).Unlikethesituationwiththepostponementoption,withpolicy ,onlyonedecisionoccursatatime,sincenoqueueofjobsismaintained.Indeed,itcanbeshownthatasngoestoinnity,w(m,n)andV m,n(x)converge.Wedenotethelimitsasw(m)andV m(x),respectively.Below,werephrasetheresultfortheSSAPwithoutthepostponementoption.Asweassumehomogeneousresourcesinthissection,weremovepmfromthestatedenition. Theorem2.3. Givenanarrivingjobvaluedatx,mremaininghomogeneousresources,njobsyettoarrive,andmn,theoptimaldecision,D m,n(x),istoimmediatelyaccept 39

PAGE 40

thejobandassignaresourcetoitifxw(m,n)ortoimmediatelyrejectthejobifx<>:x+Pm)]TJ /F5 7.97 Tf 6.58 0 Td[(1k=1w(k,n),x2[w(m,n),1]w(m,n)+Pm)]TJ /F5 7.97 Tf 6.59 0 Td[(1k=1w(k,n),x2[0,w(m,n)).(2)Moreover,w(m+1,n)w(m,n)w(m,n+1),8m1,n1.Comparedwithpolicy ,theadvantageofallowingpostponementwithpolicyisobviousforniten.Intuitively,givenanarrivingjobvaluedatx,mandn,thejobshouldberejectedifx2(w(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),w(m,n)),accordingto .However,itislikelythatthenextarrivingjobisvaluedlessthanw(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)andinthiscasethedecison-makermayregretrejectingthejobvaluedatx.Thepostponementoptionhedgesagainstthisrisk.Also,sinceIMP,Vm,n(0,...,0,x)V m,n(x).Forinniten,evenwiththepostponementoption,thedecisionmakerwillneveracceptjobsretainedinthequeue,sincetheirvaluesarelessthanz(m)(Theorem 2.2 ).Thus,thereisnobenettokeepingjobsinthequeueandthepostponementoptionhasnobenet.Wesummarizetheresultsinthefollowingtheorem. Theorem2.4. Withnitemandn,Vm,n(0,...,0,x)V m,n(x).Specically,policystrictlyoutperformspolicy underthefollowingconditions:(1)m=nandx2(0,z(m,m))(withz(1,1)=z(1)),(2)n>m=1andx2[0,z(1)),or(3)n>m2.Finally,inthecaseofinniten,z(m)=w(m)andVm(0,...,0,x)=V m(x). Proof. SeeAppendixB. 2.4.3NumericalExamplesToillustratethebenetsofpostponement,experimentsareconductedforrewardsXa,whereisabeta-distributedrandomvariablewithparametersand.Assume 40

PAGE 41

=1,=0.001,=2,=5anda=10.AsshowninTable 2-1 ,thenumericalresultsindicatethatz(m,n)w(m,n). Table2-1. Thresholdvaluedifferences:z(m,n))]TJ /F3 11.955 Tf 11.95 0 Td[(w(m,n). m,n12345678910 14.13.43.02.72.52.32.22.01.91.820.02.92.62.42.22.01.91.81.71.630.00.02.22.32.11.91.81.71.61.540.00.00.01.81.71.91.71.61.51.450.00.00.00.01.51.51.51.51.51.4 Also,asmgrows,thedecisionmakerismoreconservativeinpostponementandthedifferencesbetweenthethresholdvaluesofthetwopoliciesnarrow,deningadecreaseinthebenetofpostponement.Figure 2-1 showsthepercentageofimprovementintheexpectedprotofthepostponementoptiongiventheaboveparametersforvariousmandnvalues. Figure2-1. DiscountedprotimprovementwithX10Beta(5,2)(rightskewedF,homogeneousresources). Anotherinterestingobservationisthatthebenetofpostponementisnotmonotoneinn.Intuitively,whenthenumberofjobsthatcanbequeuedissmall,soisthechanceofpostponement.Asngrows,thedecisionmakercanretainmorejobsinthequeueandthebenetofpostponementincreases.However,asngrowsfurther,thebenetofpostponementdecreaseswithn!+1,asdiscussedinSection 2.4.2 41

PAGE 42

Figure 2-2 illustratesthepercentageofimprovementintheexpectedprotwiththesameparameters,exceptthat=5and=2.ThisillustratesthatthepostponementoptionismorevaluablewhenthedistributionofXisskewedtotheleft.Intuitively,whenapplyingpolicy ,thedecisionmakermayregreteitheracceptingalessvaluablejobwhenreceivinghighervaluedjobslater,orrejectingavaluablejobwhenencounteringjobsvaluedevenlesslater.Policyreducestheregretinbothsituations.Thisnumericalexperimentshowsthatthelatterregretismoresignicanttothedecisionmaker,possiblybecauseofthediscountingfactor. Figure2-2. DiscountedprotimprovementwithX10Beta(2,5)(leftskewedF,homogeneousresources). Finally,thepotentialofthepostponementoptioninhedgingagainstundesirablearrivingjobsmayleadtoalowervariabilityoftheprot.Tothisend,givenparametervaluesof=1,=0.001,=2,=5,a=10,m=9andn=20,theaverageprotanditsstandarddeviationover10,000runsforpolicyis37.72and4.39,respectively.Forpolicy ,thevaluesare36.65and4.51,respectively.Thoughitisinsufcienttoconcludethatpolicyresultsinalowervariabilityoftheexpectedprotthroughahypothesistest,wefoundthattheprotofpolicyhasasmallervariabilityinmostexperiments. 42

PAGE 43

2.5HeterogenousResourcesWithhomogeneousresources,thepostponementoptionreducestheregretofrejectingavaluablejoboracceptingalessvaluablejob,comparedwithjobsarrivinglater.Withheterogeneousresources,thisbenetmaybeampliedbythedifferenceinresourcevalues.However,theSSAPwithheterogeneousresourcesismuchhardertoanalyze.Inthissection,werstanalyzepolicy 2IMandpolicy2Pandidentifythedifferencebetweenthem.Then,weproposetwoheuristicsfortheSSAPwithheterogeneousresources,basedontheinsightsobtainedfromtheanalysisontheSSAPwithhomogeneousresources.Additionally,weusenumericalexperimentstoillustratetheperformanceoftheheuristics.Itshouldbenotedthatthecaseofm=1istrivial.Inthissection,weonlyfocusonthecaseofm2. 2.5.1PropertiesConsidertheoptimalpolicy 2IMwithheterogeneousresources.TheliteratureonSSAP(withoutthepostponementoption)showsthatpolicy followsasimplethresholdstructure,asdescribedatthebeginningofSection 2.2 .Theexistenceofthethresholdstructureforpolicy canbeinterpretedasfollows.Givenajobwithvaluex,mresourcesandnjobsyettoarrive,ifthedecisionmakerassignsthejobtoaresource,thentherearem)]TJ /F4 11.955 Tf 12.53 0 Td[(1jobstobeassignedinthefutureandthedecisionmakershouldcomparexwiththeexpectedvalues(afterdiscounting)ofthesem)]TJ /F4 11.955 Tf 12.81 0 Td[(1jobs.DenethesevaluesasE(X(i,n)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1),i=1,2,...,m)]TJ /F4 11.955 Tf 12.34 0 Td[(1,withE(X(i,n)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1)astheithlargestone.Withpolicy ,itturnsoutthatE(X(i,n)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1)=w(i,n).Thus,theoptimaldecisionfollowstherearrangementinequality.Forexample,ifx>w(1,n),itisoptimaltoassignthisjobtoresourcem.Below,werephrasethisresultinTheorem 2.5 43

PAGE 44

Theorem2.5. Givenpm=(p1,...,pm),njobsyettoarriveandajobvaluedatx,theoptimaldecisionis D m,n(x)=8>>>><>>>>:reject,xw(m,n)assigntoresourcei,w(m)]TJ /F6 7.97 Tf 6.59 0 Td[(i+1,n)w(1,n),(2)andE(X(i,n)m)=w(i,n),fori=1,2,...,m.Unfortunately,policy2Pdoesnotpossessasimplethresholdstructurewhenresourcesareheterogeneous.TheupdatingruleUremainsthesame,whiletheacceptanceandassignmentruleAiscomplicatedinthiscase.First,considertheoptimaldecisionsunderextremecases.Givenastate(xm,pm)andnjobsyettoarrive,ifxm=(0,...,0),thentoimmediatelyacceptajobandassignittoaresourceisequivalenttoabandoningthisresource,andthusisnotoptimal.Ifxm=(0,...,0,1),thereisnobenettopostponeduetodiscounting.Thus,itisoptimaltoimmediatelyacceptajobwithahighvalueandassignittoaresource,asinthefollowingtheorem. Theorem2.6. Givenxm,pm,andn,andxi)]TJ /F5 7.97 Tf 6.59 0 Td[(10.Withpolicy,theexpectedprotisV2,1)]TJ /F19 10.909 Tf 5 -8.84 Td[(x2,p2=maxp1x2+p2,p1+p2x2,p1hZyx2ydF(y)+x2F(x2)i+p2hx2F(x2)+Zy>x2ydF(y)i.Besidesthersttwoexpressionsinthecurlybrackets,whichcanalsobeachievedbypolicy ,thethirdexpressionistheexpectedprotofpostponement.Itcanbereadily 44

PAGE 45

veriedthatwhethertopostponedependsonthevaluesofp1andp2.Next,considerthesamesituationexceptthatx1>0andx2
PAGE 46

thecorrespondingvaluesare48.91and6.05,respectively.Theprotimprovementis2.86%.AnotherwayofmeasuringtheperformanceoftheheuristicpoliciesistostudytheupperboundofVm,n.DeneYjasthediscountedvalueofjobjthatarrivesattimeTjwithvalueXj(i.e.,Yj=e)]TJ /F15 7.97 Tf 6.59 0 Td[(TjXj).Moreover,deneY(i,n)astheithlargestvalueinfYjgnj=1.Then,theidealassignment(assumingmn)yieldsPmi=1piY(m)]TJ /F6 7.97 Tf 6.59 0 Td[(i+1,n).Obviously,thisprotisnotachievableinthecontextoftheSSAP,foritisnotpossibletodecideY(i,n),beforealljobsarrive.ConsideringTheorem 2.5 ,wehavemXi=1piw(m)]TJ /F6 7.97 Tf 6.59 0 Td[(i+1,n)Vm,n(0,pm)mXi=1piE[Y(m)]TJ /F6 7.97 Tf 6.58 0 Td[(i+1,n)]. (2)ThevalueofE[Y(m)]TJ /F6 7.97 Tf 6.59 0 Td[(i+1,n)]canbecomputedthroughsimulationandsampling.Intheaboveparametersettings,theupperboundisPmi=1piE[Y(m)]TJ /F6 7.97 Tf 6.59 0 Td[(i+1,n)]=49.56,deningamaximumprotimprovementof4.23%. 2.5.2.1Heuristics1:RaisingthresholdsInthissection,wedesignapolicy~2P,withasimilarstructuretopolicy asdescribedinTheorem 2.5 butwithhigherthresholds.Weshowthatpolicy~yieldsahigherexpectedprotthanpolicy .Inthissection,weonlyfocusonthecaseof2mn.Specically,withpolicy~,thedecisionruleformnisdenedas D~m,n(xm)=8>>>>>>><>>>>>>>:postpone,xmv(m,n)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1acceptandassignjobmtoresourcei,v(m)]TJ /F6 7.97 Tf 6.58 0 Td[(i+1,n)m)]TJ /F5 7.97 Tf 6.58 0 Td[(1v(1,n)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,(2)wherev(m,n)m)]TJ /F5 7.97 Tf 6.58 0 Td[(1=w(m,n)andv(i,n)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1with1im)]TJ /F4 11.955 Tf 12.19 0 Td[(1bearsthesamemeaningofE[X(i,n)m)]TJ /F5 7.97 Tf 6.58 0 Td[(1]exceptthatpolicy~isfollowedandtheinitialstateis)]TJ /F4 11.955 Tf 5.48 -9.68 Td[((0,...,0),pm)]TJ /F5 7.97 Tf 6.58 0 Td[(1.Inparticular,v(1,n)1=g1,n(0). 46

PAGE 47

Theorem2.7. For2mn,V~m,n)]TJ /F4 11.955 Tf 5.47 -9.69 Td[((0,...,0,xm),pmV m,n(xm,pm)andv(i,n)mw(i,n)fori=1,2,...,m,wherenisniteandtheinequalityisstrictform
PAGE 48

Table2-2. Averagevaluesofjobsassignedtoresourceswithoutandwiththepostponementoption. ResourcesAveragediscountedvaluesofjobsassignedunder Averagediscountedvaluesofjobsassignedunder~Improvement p9=2.05.645.660.36%p8=1.85.005.040.80%p7=1.64.554.611.17%p6=1.44.224.230.39%p5=1.23.913.961.12%p4=1.03.643.691.29%p3=0.83.393.482.72%p2=0.63.183.386.30%p1=0.42.973.3612.93% 2.5.2.2Heuristics2:SamplingonfuturearrivalsAsillustrated,apolicysimilarto withincreasedthresholdvaluesisnotquitesatisfactory.Inthissection,wedeneapolicy2Pthatutilizesz(m,n)andsamplesfuturearrivals.Also,wemainlyfocusonthecaseof2mn.FollowingdecisionruleDm,n,therststepinadecisionistoseewhethertopostpone,dependingonwhetherxmislargerthanz(m,n).Ifthisisnotthecase,thedecisionmakerwaitsforthenextarrival.Ifthisisthecase,thedecisionmakeracceptsthequeuedjobwiththesmallestvalue.Thesecondstepistodeterminewhichresourcetoassigntheacceptedjob.TheassignmentrulefollowsasimilarstructuretoD (Theorem 2.5 ),exceptthatw(i,n)arereplacedbyw(i,n)+(1)]TJ /F8 11.955 Tf 11.96 0 Td[()E[Y(i,n)]with01.Intuitively,w(i,n)+(1)]TJ /F8 11.955 Tf 12.22 0 Td[()E[Y(i,n)]isanapproximationoftheexpecteddiscountedjobvaluethatisassignedtotheresourcewiththeithlargestqualityvaluebyfollowingpolicy.For=1,w(i,n)+(1)]TJ /F8 11.955 Tf 12.72 0 Td[()E[Y(i,n)]=w(i,n)andpolicyisverysimilartopolicy .For=0,w(i,n)+(1)]TJ /F8 11.955 Tf 12.3 0 Td[()E[Y(i,n)]=E[Y(i,n)]andpolicyperformspoorly,sinceE[Y(i,n)]istheidealexpecteddiscountedjobvalueassignedtothejobwiththeithlargestqualityvalue,whichisnotachievable.Forsome2(0,1),policycanachieveahigherprot.Itshouldbeclearthatbysimplyapplyingthisassignmentrulealone,withoutthepostponementoption,policyisinferiortopolicy 48

PAGE 49

Figure2-3. DiscountedprotimprovementwithX10Beta(2,5)(leftskewedF,heterogeneousresources,andHeuristics2). Itcanbeseenthatpolicyutilizestheinsightsfrompolicyinthesituationofhomogeneousresources.First,theconditionofwhetherxmissmallerthanz(m,n)servesasathresholdofpostponement.Second,whenacceptingjobs,policychoosesthequeuedjobwiththesmallestvalue.Theintuitionbehindisthattoretainjobminthequeuemayyieldalargerprot,asindicatedbythemonotonicityofg0m,n(x+)inm(Theorem 2.2 ).UndertheparametersettingsfromSection 2.5.2 ,thispolicysignicantlyoutperformstherstheuristicpolicy.With=0.55,thesamplemeanandthesamplestandarddeviation(over10,000runs)oftheexpectedprotare49.13and6.00,respectively.Policyyieldsaprotimprovementof3.11%.Indeed,policydominatesthebenchmarkpolicyforavarietyofnvalues,asillustratedinFigure 2-3 .Also,wecanconcludethatpolicyresultsinasmallervariabilityoftheprotthroughahypothesistestwiththesignicancelevelof0.05.Thoughitishardtoevaluatetheperformanceofpolicyintermsoftheclosenesstooptimality,wecanbetterunderstanditsperformancethroughthefollowingexample.Considerthecaseofx2=(0,x2),p2=(p1,p2)andn=1,inwhichwecanndtheoptimalpolicy,.Theprotimprovementincreasesasp2)]TJ /F3 11.955 Tf 12.6 0 Td[(p1grows.Intuitively, 49

PAGE 50

theexpecteddiscountedvalueofthejobassignedtoresource2underpolicyislargerthanitscounterpartunderpolicy ,whiletheexpecteddiscountedvalueofthejobassignedtoresource1issmallerthanitscounterpartunderpolicy .Inotherwords,policyachievesahigherprotsinceitutilizesadditionalinformationsuchthatresourceswithhighervaluesareassignedtobetterjobs.Indeed,Table 2-3 illustratesthatpolicyachievesthesameobjective,thoughnottotheoptimalextent. Table2-3. Jobassignmentswithoutandwiththepostponementoption. ResourcesAveragediscountedvaluesofjobsassignedunder AveragediscountedvaluesofjobsassignedunderImprovement p95.646.067.45%p85.005.306.00%p74.554.795.27%p64.224.373.55%p53.914.012.56%p43.643.722.20%p33.393.462.06%p23.183.210.94%p12.972.980.34% 2.6ConclusionsInthischapter,thesequentialstochasticassignmentproblemhasbeenextendedtoincludethepostponementoption.Specically,thiscaseallowsfortheanalysisofproblemswherethedecisionmakercandelaythedecisiontoacceptorrejectajobuponarrivalforsomeperiodoftime.Thetradeofftobeconsideredisthevalueofdelayingthedecision,allowingmoreinformationtobegathered,againstthedeclineinvalueofacceptingajoblate,duetodiscounting.Thereareavarietyofapplicationswhereanassignmentcanbedelayedinordertogathermoreinformation.Forexample,mostinterviewingprocessesallowforadelaybeforeoffersaremadeand/oraccepted.Similarly,multiplebidsforthesaleofanitemorpropertymaybereceivedbeforeacceptance. 50

PAGE 51

ItwasshownthattheoptimalpolicyoftheSSAPwithhomogeneousresourceshasathresholdstructure.Despitedeningthestatespaceaccordingtoavector,theanalysismerelyrequiresconsiderationofthemaximumvaluedjobonhand.Thesituationwithheterogeneousresourceswasalsoexamined,whichissignicantlymoredifcult.Astheanalysismustconsiderboththevaluesofthejobsandresourcessimultaneously,itisdifculttondanoptimalpolicypossessingasimplethresholdstructure.Twoheuristicpoliciesareproposed,basedoninsightsderivedfromthecasewithhomogeneousresources.Numerically,policy~,whichadaptsthethresholdpolicyfoundintheSSAPliterature(i.e.,policy )byraisingthresholdvaluesdoesnotworkwell.Policy,whichsamplesfuturejobarrivalsandemploysthethresholdz(m,n)derivedforthehomogeneousresourcescaseachievesthehigherprotthanotherheuristicpolicies.Webelievetherearesignicantextensionsofthisworkthatareworthyofconsideration.Assumptionsconcerningthearrivalprocessofjobscanberelaxedtoconsiderothercasesinadditiontohowjobvaluesarediscountedovertime.Clearly,thecasewithheterogeneousassetscanbeexploredfurther,aswouldbethecasewhenresourcesbecomeavailableovertime(accordingtoanotherstochasticprocess).Additionally,whilethedelayoptionmaybefeasibleinanumberofsettings,theremaybealimitastohowlongadelaycanoccur.Thus,itwouldbeinterestingtostudythisprobleminlightofadeadline.Infact,ithasbeenfoundthroughnumericalexperimentsthatgk,m(x)maynotbesignicantlyhigherthanw(k,m)ifxissmall,indicatingthatsomejobswithlowvaluesinthequeuedonotcontributemuchandthuscanberemovedfromthequeue.Inotherwords,itislikelytoachieveahigherprotwithmoderatelevelofdelay. 51

PAGE 52

CHAPTER3THEDYNAMICANDSTOCHASTICKNAPSACKPROBLEMTheknapsackproblem,whichaddressestheallocationoflimitedresources(capacity)toitemswithknownsizetomaximizeexpectedprothasbeenextensivelystudied( Kellerer,H.andPisinger ( 2004 )).TheDynamicandStochasticKnapsackProblem(DSKP),asdenedin Papastavrouetal. ( 1996 ),assumesthatitemsarriveaccordingtoaPoissonprocess.Eachitemisdenedbyarewardandaresourcerequirement,whicharemadeknownuponarrival.Accepteditemsareplacedintoaknapsackwithknowncapacity,andneitherallocatedcapacitynorrejecteditemsmayberecalled.Thereisalsoaknowndeadline(possiblyinnite).Moreover,thedecision-makermayterminatetheprocessbeforedepletingthecapacity.Thisproblemhaswideapplicationsintransportation,scheduling,andnance. Papastavrouetal. ( 1996 )presentedadiscretetimemodelforthisproblemandderivedoptimalthresholdpoliciestothisproblem.Theauthorpointedouttwofutureextensions:extendthepolicytocontinuoustimesetting,andincludetheoptionofrecallingrejecteditems. KleywegtandPapastavrou ( 1998 )and KleywegtandPapastavrou ( 2001 )presentacompletesolutiontotherstextension.Thesecondextensionhasnotbeenaddressed.Itshouldbeclearthatifadecisionmakerhadtheabilitytoholdanarrivingitem,essentiallydelayingthedecisiontoacceptorrejecttheitem,thenhecanmakebetterdecisionswithmoreinformation,asmoreitemsmayarriveinthedelayperiod.Inpractice,therearemanysituationsinwhichthisisfeasibleoritmaybepossibletopayafeeinordertodelaythedecision;indeed,arequestforaresourceisusuallyansweredwithapromiseofwhenthedecisionontherequestwillbemade.Theapplicationofthepostponementoptionmaybeoperationalorstrategic.Typicalexamplesinoperationalsettingcanbefoundinlogistics,inwhichthedispatchermaydelaycapacityreservationrequeststhatarenotveryattractive,aspointedoutin Spivey 52

PAGE 53

andPowell ( 2004 ).Moreover,insomebiddingprocesses,suchassellingadvertisingontheradio,onecanoftenaffordtopostponedecisionswhenallocatingtimeslots.Typicalapplicationinstrategicsettingscanbefoundininvestmentanalysis.Considerthefollowingscenario:Armhasallocatedaleveloffundingforinvestinginstart-upcompanies.Overtime,anumberofrmsarriverequestingfundinganditisexpectedthatmorewillarrivethanfundscansupport.Whileitmaynotbepossibletomakeastart-uprmwaitindenitelyforadecision,itisgenerallyacceptabletodeliverthedecisionatsomelaterpointintime(sayafewweeksoramonth),asallnancialdatamustbecarefullyreviewed.Inthistime,theinvestmentrmcanscreenotherstart-upcandidates,providingtheopportunitytogathermoredataandmakeabetterdecisionwithregardstotheirentireinvestmentportfolio.Infact,theinvestmentrm,ifstillundecidedaftersomeperiodoftime,maypayastart-uprmsomefee(forsustenance)tofurtherdelaytheinvestmentdecision.Inthischapter,weexplicitlyanalyzethesituationwhereadecisionmakermaydelaytheaccept/rejectdecisioninadynamic,stochasticknapsackproblem.Furthermore,weimplicitlyidentifythecostorvalueofdelayingthisdecision.Itshouldbenotedthatwiththepostponementoptions,anarrivingitemthatisnotimmediatelyacceptedmaybere-consideredlater;thisindeedaddressesthesecondextensionspointedoutin Papastavrouetal. ( 1996 ).Moreover,thismodelhasimplicationsontherealoptionsanalysis.Inrealoptionsanalysis,techniquescommonutilizedtovalueastockoptionareutilizedtovaluerealinvestments,suchasbuildingaplantorpurchasingequipment.Thetruevalueofaninvestmentisthesumofitstraditionalvalue(generallycomputedasthenetpresentvalue)anditsoptionvalue,wheretheoptionvaluecapturestheadditionalworthofoptionswithregardstoaninvestment.Acommonrealoptionistodelayaninvestment.Thishasworthbecauseadelayaffordsthemanageranopportunitytogatheradditionalinformation,suchasmovementsinpricesorpotentialdemand.Butitmustbemade 53

PAGE 54

clearthatrealoptionsanalysisonlyconsidersasingleinvestmentopportunity.Inthecontextofourknapsackproblem,onecanconsiderthedelayoptionwithaportfolioofprojects,consideringtheirinterdependencywithregardstoabudgetingconstraint.Toourknowledge,littleresearchhasbeenpublishedonthepostponementoptioninoperationalsettings. SpiveyandPowell ( 2004 )pointoutthepotentialbenetofpostponingassignments.Inthischapter,werststudytheDSKPmodelwithhomogeneoussizeditemswhenthedecisionmakercanpostponetheaccept/rejectdecision.Surprisingly,wefoundthattheoptimalpolicyofthenewmodelstillpossessesathresholdstructure.Then,wediscussthepotentialextensionsofutilizingthethresholdpoliciesforthehomogeneouslysizeditemscasetoderiveheuristicsalgorithmforthecaseofheterogeneouslysizeditemscase.Specically,wemakethefollowingcontributionstotheliterature: 1. ModelthenitetimehorizonDSKPwithpostponementoptionsandhomogeneousitems; 2. DeriveanoptimalthresholdpolicyoftheDSKPwhenthenumberofarrivingitemsisniteorinnitewithaholdingcost,delaycost,andtheoptiontoterminatethedecisionprocessearly; 3. Extendtheoptimalthresholdpolicytothecasewithdelaycosts(inotherwords,rejecteditemscanberecalledatacost); 4. Illustratethebenetsofpostponementthroughnumericalexperiments.Thischapterisorganizedasfollows.InSection3,weintroducenotation,assumptionsandourmodel.InSection4,weexaminetheDSKPmodelonlywithholdingcostsforunallocatedresources.TheoptimalpolicyandtheassociatedstructuralpropertiesofthenitehorizonDSKPwithhomogeneousresourcesaredevelopedthroughinduction.InSection5,weinvestigatehowtoincorporateothercostsandthestopoptionintothemodelandidentifyconditionsforthethresholdoptimalpolicyandtheassociated 54

PAGE 55

structuralpropertiestohold.Thebenetsofpostponementareillustratedthroughnumericalexamples. 3.1RelatedResearchTheseminalworkoftheDSKPisperformedby Papastavrouetal. ( 1996 ),inwhichitemsarriveoveranitenumberoftimeperiods. KleywegtandPapastavrou ( 1998 )studytheversionoftheDSKPwithhomogeneous-sizeditemsincontinuoustime.Moreover,theyintroduceholdingcostsforunallocatedresources,arejectionpenaltyforrejecteditemsandanoptiontostopreceivingitems.Anoptimalpolicywithasimplethresholdstructurewasdeveloped.Thismodelandtheoptimalthresholdpolicyareextendedtothecaseofrandomlysizeditemsby KleywegtandPapastavrou ( 2001 ). NikolaevandJacobson ( 2010 )extendthemodeltothecasewhenthenumberofitemstoarriveisrandom. McLayetal. ( 2009 )applytheoptimalpolicyoftheDSKPtotheallocationofscreeningresourcestopassengers.Papastavrouetal. Papastavrouetal. ( 1996 )notethatrecallingrejecteditemsisapossibleextensionoftheDSKPmodel.Thisisinessenceequivalenttodelayingtheaccept/rejectdecisionsandisthebasisofthischapter.TheDSKPiscloselyrelatedtotheSSAP.Byappropriatelysettingthecostsintroducedby KleywegtandPapastavrou ( 1998 )andforbiddingthedecisionmakertoterminatedecisionsbeforethedeadline,theinnitehorizonDSKPwithhomogenouslysizeditemsreducestotheSSAPwithhomogeneousresourcesandinnitelymanyarrivingitems.ThereareseveralotherproblemsrelatedtotheDSKP.Ingeneral,theStochasticknapsackproblem(e.g., Deanetal. ( 2008 ))considersthesituationwhenthesizesand/orrewardsofitemsareunknownuntiltheyareinsertedintotheknapsack.Thesequentialselectionproblemisdenedasfollows.Ani.i.d.sequenceofnon-negativerandomvariableswithknowndistributionistobeinspected.Uponinspection,thevalueoftherandomvariableismadeknownandthedecisionmakermustimmediately 55

PAGE 56

decidewhethertoacceptorrejectthevariable.Theobjectiveistomaximizethenumberofacceptedvariableswiththesummationofthesevariablesbeinglessthansomespeciedamount.ThisproblemcorrespondstoaspecialcaseoftheDSKPwithrandomsizeditems,homogeneousrewardsandnocostsorstopoptions. Coffmanetal. ( 1987 )proposeanoptimalthresholdpolicy. RheeandTalagrand ( 1991 )discusstheasymptoticbehavioroftheproblem. Luetal. ( 1999 )developatime-constrainedcapital-budgetingproblem,whichcanbeviewedasageneralizedversionofthesequentialselectionproblemoraspecialcaseoftheDSKP.Inthisproblem,projectproposalsofndifferenttypesarriveatafundingagencyrandomlywitharatei,i=1,2,...,n.EachtypeischaracterizedbyadistinctivecostciandabenetBi,whichisrandom.Uponthearrivalofaproject,thedecisionmakermustimmediatelydecidewhethertoacceptandfundtheproposal.GivenalimitedamountofbudgetBandanitetimehorizonT,thedecisionmakerselectsprojectstomaximizetheexpectedprotbytimeT. Herbotsetal. ( 2007 )discussedadynamicorderacceptanceproblem,inwhichprojectproposalsarrivesequentiallywithdeterministicinter-arrivaltimes,whileprojectcharacteristicsarerevealeduparrival. PapadakiandPowell ( 2003 )tackledthebatchdispatchproblemwithrandomarrivingrequestsbyapproximatedynamicprogramming.Postponementnaturallyarisesasanoptioninnance(e.g., Trigeorgis ( 1996 )and AmramandKulatilaka ( 1999 ))andoperationsmanagement( VanHoek 2001 ).Innance,itusuallyappearsinthecontextofrealoptions,asstatedby Triantis ( 2000 ).Inoperationsmanagement,theconceptofpostponementhasbeenstudiedinmarketingchannels( Bucklin 1965 ),masscustomization( FeitzingerandLee 1997 ),capacityallocation( DingandKouvelis 2001 ),andR&Dmanagement( HuchzermeierandLoch 2001 ).ThesedecisionsettingsdiffergreatlyfromtheDSKP,asthepostponementmentionedhereisgenerallyoveralongertimeframe(months,quarters,oryears)thaninourapplications.Also,thisliteraturemainlyfocusesoncasestudyorqualitativeanalysis.Mathematicalmodelsonpostponementoptionsarerelativelyrare. 56

PAGE 57

Ofthemathematicalmodels,twoarequitetypical. McDonaldandSiegel ( 1986 )studytheoptimaltimingofinvestmentinaproject,withthebenetsandcostsfollowingBrownianmotion.Ourresearchalsostudiesthebesttimetoacceptorrejectanitem,butweconsiderthearrivalsofanumberofitems(orprojects)andtherewardandcosts,oncemadeknown,donotvaryovertime. GrenadierandWeiss ( 1997 )modeltheoptimalinvestmentstrategyforarmconfrontedwithtechnologicalinnovations.Inthisresearch,technologicalinnovationsarrivesequentially,withthearrivaltimeandthevalueoftheinnovationtothermuncertain.Eachinnovationconnectstootherinnovationstothatarriveinthefutureatdifferentcosts.Theobjectiveofthermistodeterminetherighttimetoinvestinordertoadoptinnovationsatlowercosts,whilekeepingacompetitionadvantage.Thisproblemresemblesourresearchintermsofthestochasticarrivalsandtheabilityofthedecisionmakertodelaydecisions.However,theDSKPconsidersdifferentcostsandthecostsincurredbyoneitemdonotdependonotheritems.Moreover,thisresearchonlyconsiderstwoinnovationarrivals,whiletheDSKPusuallyinvolvesalargenumberofitemsandanitetimehorizon.Inourmodel,arrivingitemsmayformaqueue,suchthatthemodelcloselyresemblesanonlinebinpackingproblem.Inthebinpackingproblem( CoffmanandStolyar ( 1999 )),fractionsofasingleresourceareallocatedtoasetofstreamsofrequests.EachstreamischaracterizedbyadistinctvalueofresourcerequestsandisgovernedbyaPoissonprocess.Thesystemoperatesindiscreteunitsoftime.Eachfullledrequestoccupiestheallocatedresourceforaunitoftimeandreleasesitthereafter,leavingthesystem.Unfullledrequestsformqueues. GamarnikandSquillante ( 2005 )providenewmethodstoanalyzetheproblemunderageneralclassofschedulingpoliciesandyieldsstabilityconditionsandstationarydistributions. Gyorgyetal. ( 2010 )studythecasewhenthereismorethanonebin.ThisproblemdiffersfromtheDSKPmodel,whereallocatedresourcescannotberecalled. 57

PAGE 58

3.2ModelandAssumptionsAsstatedpreviously,withthepostponementoption,thedecisionmakercanaccept/rejectanitematanytimeafteritsarrival.Tocapturethesefeatures,werstformulatetheproblemasaControlledMarkovProcess( FlemingandSoner ( 1997 )).Theresultingmodel,however,isdifculttoanalyze.Thus,byrestrictingthedecisionepochs,wereducethemodeltoaMarkovDecisionProcess.Itturnsoutthattheoptimalpolicyofthismodelpossessesathresholdstructureandthispolicyisalsooptimalfortheoriginalmodel. 3.2.1ModelDenitionsIntheDSKPliterature,thedecisionmakerhasthreeoptionsuponthearrivalofanewitem: (1) Rejecttheitem; (2) Acceptandloadtheitemandfulllitsresourcerequirement; (3) Stopreceivingitemsandsalvageallunallocatedresources.Thestop-and-salvageoption(Option(3))isadistinctivefeatureoftheDSKPmodel.Forexample,noairplanewaitsatanairportsimplybecauseaseathasnotbeensoldandnoteverytrucktravelsatfullcapacity.Likewise,managersmaynothavetosellallassetsavailable,regardlessofthepopularityoftheassetsinthismarket.However,theoptionofchoosingastoppingtimecomplicatestheanalysis.Inthissection,weignoreOption(3);inlatersections,weincorporatethisoptioninadifferentapproach.Wereplace(3)bythepostponementoption:todelaytheaccept/rejectdecision.Ifthedecisionmakerchoosesnottoimmediatelyacceptorrejectanitem,allthreeoptionsareavailableatanytimeinthefuture.Theproblemendswheneitherallitemsarriveorwhenavailableresourcesareexhausted.Theobjectiveistomaximizetheexpected(discounted)prot.Notethatthedecisiontoacceptandallocateresourceisimplementedinstantaneously.WeformallydenetheDSKPmodelbelow.Forconvenience,weabbreviateacceptandloadasload. 58

PAGE 59

Wenextformallydenethemodel.ThearrivalprocessisalmostthesameasinthetradditionalDSKPmodel.DenoteT2(0,1]asthedeadlineforloadingitems.DenefAjg1j=1asthearrivaltimesofaPoissonprocesson(0,T)withrate2(0,1).DenefRjg1j=1asani.i.d.sequence,withRjastherewardofarrivalj,independentofAj.LetFRbetheprobabilitydistributionofRandXasthesupportofR.AssumethatE[R]<1andX[0,1).DenoteM(t)asthenumberofunallocatedresourcesandN(t)asthenumberofitemsyettoarriveattimet.NotethatthemodelsintheDSKPliteraturegenerallyassumeN(0+)=1.Inthischapter,weallowforanitenumberofarrivals.Aconsequenceofthepostponementoptionisthatthearrivingitemsyettobeloadedorrejectedformaqueue.Denoter(M(t))astherewardsoftheitemsinthequeuegivenM(t),andrequirethatr1r2rM(t).Letushenceforthrefertoaqueueditemthathastheithsmallestrewardas`itemi'.Withoutlossofgenerality,weassumethatthereareexactlyM(t)queueditems,sinceinthecaseofmorequeueditems,onlythelargestM(t)ofthemneedtobeconsideredandinthecaseoffewerqueueditems,andthevacantplacescanbelledwithri=0.Thus,uponthearrivalofanitem,atleastoneitemshouldberejectedimmediately,whichmightbearealitemormerelyaplaceholder.Wecombinethetwooperationsofinsertionandrejection,termedupdate,anddenoteitasUA.Atanytimet,thedecisionmakerdecideswhetheraqueueditemshouldbeloadedandifso,whichoneshouldbeloaded.Form=M(t)andn=N(t),denoteDasthedecision: D(r(m),n,t)=8>><>>:1ifanunallocateditemistobeloaded0ifdonothing.(3)Alongwiththeloadingofanqueueditemisthechangeofthequeue,denotedasUL()andtherewardearned,denotedRL().Forexample,ifitemlisloaded,UL(r(m))=(r1,...,rl)]TJ /F5 7.97 Tf 6.59 0 Td[(1,rl+1,...,rm)andRL(r(m))=rl. 59

PAGE 60

DeneA:=ffAjg1j=1:0><>>:Dhr)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(M(t),N(t),tiRLhr)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(M(t),N(t),tiif0t
PAGE 61

whereTsisthersttimewheneitherM(T+s)=0orN(T+s)=0andTsT.Thentheexpectedprotunderis VDSKP=EZTs0e)]TJ /F15 7.97 Tf 6.59 0 Td[(dY())]TJ /F14 11.955 Tf 11.96 16.28 Td[(ZTs0e)]TJ /F15 7.97 Tf 6.59 0 Td[(hcM()+cqe0r)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(M()id,(3)wherethersttermrepresentstherewardandthesecondtermrepresentstheholdingcostandthedelaycost.Theobjectiveistondthemaximumexpectedprot:V=sup2DPDSKPVDSKP.Comparedwithpoliciesrequiringimmediatedecisions,policiesallowingpostponementdonotrestrictdecisionepochstoarrivaltimes.Moreover,therewardfromanitemthatisloadedmaynotcoincidewithitsarrivaltimetherewardprocessandthearrivalprocessaredecoupledintime.ThesefeaturescharacterizetheDSKPasastochasticoptimalcontrolproblem.Itshouldbeclearthattheprottobeobtainedinthefuturedependsonthequeuer(M(t)),thenumberofunallocatedresourcesM(t),thenumberofitemsyettoarriveN(t),andtimet,ratherthantheitemsthathavebeenrejectedfromthequeueorloadedintotheknapsack.Therefore,withthesystemstatedenedas(r(M(t)),N(t),t),theDSKPproblemwiththepostponementoptioncanbemodeledasaControlledMarkovProcesses.ByBellman'sprincipleofdynamicprogramming,theoptimalvaluefunctionV(r(M(t)),N(t),t)satisesV(r(M(t)),N(t),t)=sup2DPDSKPEZ(t+h)^Tste)]TJ /F15 7.97 Tf 6.59 0 Td[(dY() (3))]TJ /F14 10.909 Tf 10.91 14.85 Td[(Z(t+h)^Tste)]TJ /F15 7.97 Tf 6.59 0 Td[(hcM()+cqe0r)]TJ /F16 10.909 Tf 5 -8.83 Td[(M(t)id+V(r(M(t+h)),N(t),t+h), (3)withtheboundaryconditionas Vr)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(M(Ts),N(Ts),Ts=8><>:e0r)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(M(t)ifN(Ts)=00ifM(Ts)=0,(3) 61

PAGE 62

wherethedecisionmakereitherloadsallitemsinthequeueifN(Ts)=0orabandonsallitemsinthequeueifM(Ts)=0. 3.2.2ARelatedMarkovDecisionProcessNotethatitisnotstraightforwardtostudytheDSKPmodelaboveduetothedifcultyincharacterizingr(M(t)).Thus,weintroduceanotherclassofpolicies.Insteadofmakingdecisionscontinuously,thesepoliciesrestrictthedecisionepochstomomentswhenthequeuechanges,eitherthroughthearrivalofanewitemandthesubsequentupdate,orwhenanitemfromthequeueisloaded.Specically,uponthearrivalofanewitemandthesubsequentupdate,thedecisionmakerdecideswhethertoimmediatelyloadanitemorpostponethisdecisionbyacertainamountoftime.Fortheformerdecision,M(t)decreasesbyone.Inthecaseofthelatterdecision,N(t)decreasesbyoneifanewitemarrivesduringthepostponementperiod.Ifthereisnoarrivalduringpostponement,thedecisionmakerloadsanitemfromthequeueandM(t)decreasesbyone.Inotherwords,thepoliciesdescribedaboveentailaMarkovdecisionprocess(MDP)anddecidewhentoapplytheloadingrule(ULandRL)afteranarrival.Specically,givenanarrivalattimetwithrewardrandM(t)=mandN(t)=n,policy2DPMDPplansapostponementperiod[t,t+t).Byfollowing,thesystem(r(m),n,t)eithertransitionsto(UA(r(m),r,n)]TJ /F4 11.955 Tf 12.01 0 Td[(1,t+),n)]TJ /F4 11.955 Tf 12 0 Td[(1,t+)orto(UL(r(m),n,t+t),n,t+t),wheret+isthearrivaltime,dependingonwhetheranitemarrivesduring[t,t+t].FollowingeachdecisionisareductioneitherinM(t)(t=0ornoarrivaloccursduring[t,t+t])orinN(t)(anarrivaloccursduring[t,t+t]).ThedecisionprocessterminateswheneitherM(t)orN(t)goesto0.IfM(t)=0andN(t)0,theremainingitemsthathavenotarrivedyetareignored.WhenM(t)>0andN(t)=0,thedecisiontopostponeisnolongerbenecialasallitemshavearrived.Asdecisionsareimplementedimmediately,thedecisionprocesshereallowsmultipledecisionstooccuratthesamemomentintime. 62

PAGE 63

Theobjectiveistondthe2DPMDPthatmaximizesexpectedprot.ItshouldbeclearthatV(r(m),n,t)V(r(m),n,t)asDPMDPDPDSKP.Byconditioningonwhetheranewitemarrivesduringpostponement,V)]TJ /F19 10.909 Tf 5 -8.84 Td[(r(m),n,t;t=Zt0e)]TJ /F15 7.97 Tf 6.59 0 Td[(e)]TJ /F15 7.97 Tf 6.59 0 Td[(ZXVhUAr(m),r,n)]TJ /F18 10.909 Tf 10.91 0 Td[(1,,n)]TJ /F18 10.909 Tf 10.91 0 Td[(1,t+idFR(r))]TJ /F16 10.909 Tf 10.91 0 Td[(Cpost)]TJ /F19 10.909 Tf 5 -8.84 Td[(r(m),d+e)]TJ /F15 7.97 Tf 6.59 0 Td[(te)]TJ /F15 7.97 Tf 6.59 0 Td[(thrl+Vr(m)nflg,n,t+ti)]TJ /F16 10.909 Tf 10.91 0 Td[(Cpost)]TJ /F19 10.909 Tf 5 -8.83 Td[(r(m),t,wherelistheoptimalitemtobeloaded,r(m)nflg=(r1,...,rl)]TJ /F5 7.97 Tf 6.58 0 Td[(1,rl+1,...,rm),andCpost(m,t)isthepostponementcostfunction(theholdingcostandthedelaycost),denedasCpost)]TJ /F11 11.955 Tf 5.48 -9.68 Td[(r(m),t=Zt0e)]TJ /F15 7.97 Tf 6.59 0 Td[(hcm+cqe0r(m)id=cm+cqe0r(m) (1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F15 7.97 Tf 6.59 0 Td[(t).TheboundaryconditionisthesametoEq. 3 .Bytheprincipleofdynamicprogramming,theoptimalpolicysatisesV)]TJ /F11 11.955 Tf 5.48 -9.68 Td[(r(m),n,t=supt2[0,1]Vr(m),n,t;t.TheexpressioncanberewrittenasV(r(m),n,t)=supt2[0,1])]TJ /F3 11.955 Tf 13.15 8.09 Td[(cm+cqe0r(m) +h1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti+Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ZXVhUAr(m),r,t,n)]TJ /F4 11.955 Tf 11.96 0 Td[(1,t+idFR(r)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)thrl+Vr(m)nflg,n,t+ti. (3)Forconvenience,wedeneanotherpolicy2DPDSKPbasedon,whichplansapostponementtimet,afterwhichisfollowed.Correspondingly,theexpectedprotisV(r(m),n,t;t)=Zt0e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)ZXVhUAr(m),r,t+,n)]TJ /F4 11.955 Tf 11.96 0 Td[(1,t+idFR(r)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tVr(m),n,t+t)]TJ /F3 11.955 Tf 13.15 8.09 Td[(cm+cqe0r(m) +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti. 63

PAGE 64

WhethertopostponedependsonthesignofthederivativeofV(r(m),n,t;t)withrespecttothepostponementtimet,ormorepreciselyonthesignofG(r(m),n,t+t)= +ZXVhUAr(m),r,n)]TJ /F4 11.955 Tf 11.96 0 Td[(1,t+tidFR(r))]TJ /F3 11.955 Tf 11.96 0 Td[(rl)]TJ /F3 11.955 Tf 11.96 0 Td[(Vhr(m)nflg,n,t+ti)]TJ /F3 11.955 Tf 13.15 8.09 Td[(cm+cqe0r(m) ++1 +@ @tVhr(m)nflg,n,t+ti. (3)IfG(r(m),n,t+t)>0fort2[0,T)]TJ /F3 11.955 Tf 12.43 0 Td[(t),thedecisionmakershouldpostponethedecisiontotimeT,deningtheoptimalt=T)]TJ /F3 11.955 Tf 12.77 0 Td[(t.IfG(r(m),n,t+t)0fort2[0,T)]TJ /F3 11.955 Tf 11.96 0 Td[(t),itisoptimalforthedecisionmakertoimmediatelyloaditeml. 3.3TheFiniteHorizonDSKPwithPostponementOptions:ASimpliedVersionInthissection,westudythenitehorizonDSKPwithpostponementoptionsconsideringonlytheholdingcostonunallocatedresources.DeneanupdateruleUAasfollows:Uponthearrivalofanitemwithvaluer,theitemwiththelowestvalueisrejected: UAr(m),r,n,t:=UAr(m),r=8>>>><>>>>:(r1,r2,,rm)ifrr1(r2,,ri,r,ri+1,,rm)ifrirri+1,i=1,2,,m(r2,,rm,r)ifrm
PAGE 65

Theorem3.1. Givennitemsyettoarrive,aqueueditemwithrewardr1andtimet,(i)thereexistsw(1)(r1)2[0,T]suchthattheoptimaldecisionistopostponeuntilw(1)(r1)ift>>>><>>>>>:r1ift0Zt0ZXe)]TJ /F5 7.97 Tf 6.59 0 Td[((+)g1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1UA(r1,r),t+dFR(r)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tr1)]TJ /F6 7.97 Tf 19.31 4.71 Td[(c +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tiift>0, (3)wheret=w(1)(r1))]TJ /F3 11.955 Tf 11.95 0 Td[(t,g1,0(r1,t)=r1,and w(1)(r1)=8><>:0,if +hr1FR(r1)+Rr>r1rdFR(r1)ir1+c +T,if +hr1FR(r1)+Rr>r1rdFR(r1)ir1rdFR(r)i)]TJ /F3 11.955 Tf 11.95 0 Td[(r1)]TJ /F3 11.955 Tf 24.32 8.09 Td[(c +.ItisstraightforwardtoverifythatG(r1,1,t+t)ismonotonicallydecreasingwithr1.Thus,thedecisionmakereitherimmediatelyloadsthe(only)queueditemifrrorpostponesotherwise,whereristherootofG(r1,1,t+t)=0.Thisdenesw(1)(r1)asEq. 3 .Property(i)holdsforn=1. 65

PAGE 66

Now,supposethatproperty(i)holdsforn)]TJ /F4 11.955 Tf 12.67 0 Td[(1withn2.WhethertopostponedependsonthesignofG(r1,n,t+t),whereG(r1,n,t+t)= +ZXg1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1UA(r1,r),t+tdFR(r))]TJ /F3 11.955 Tf 11.96 0 Td[(r1)]TJ /F3 11.955 Tf 24.31 8.09 Td[(c +.Bytheinductionassumption,fortw(1),g1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r1,t)=g1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(2(r1,t))G1,n(r1,t)=G1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r1,t);fortg1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(2(r1,t))G1,n(r1,t)>G1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r1,t).ItfollowsthatthethresholdpolicyholdsanditiseasytoverifyEq. 3 .TheproofofProperty(i)iscomplete.WenextshowProperty(ii).Themonotonicityofw(1)(r1)followsbydenition.Consider(@=@r1)g1,n(r1,t)witht0,(@=@t)g1,n(r1,t)<0fort0, 66

PAGE 67

(@=@t)g1,nisstrictlyincreasinginr1fort0,thereexistsauniquethresholdvaluer,whichsatises +[rFR(r)+RXrdFR(r)]=r+c +andservesasthelowestrewardthatcanbeloaded.Inthischapter,wediscussthepolicyintermsofthresholdsintimeinsteadofinreward.Also,g1,n(r1,t)iscontinuousbutnotdifferentiableatr.Thiscausestechnicalinconveniencesinarguingthemonotonicityoffunctionsthroughthesignoftherst-orderderivative.Later,weusethesignoftherightrst-orderderivativeinstead. 3.3.2Case2:m2Next,wediscussthecasewithmorethanoneresource.Weonlybrieyintroducetheintuitionofwhytheoptimalpolicypossessesathresholdstructure.AformalproofisgivenintheAppendix. Lemma3.1. Forcontinuousfunctionfn(x),n=1,2,denedon[0,a],whichisnon-decreasinginxandn,andf0n(x+)f0n+1(x+),thenforanyx1x2:fn(x2))]TJ /F3 11.955 Tf 11.95 0 Td[(fn(x1)fn+1(x2))]TJ /F3 11.955 Tf 11.95 0 Td[(fn+1(x1).Whenf0n(x+)>f0n+1(x+),theaboveinequalityisstrict.Ingeneral,wehavethefollowingtheorem: 67

PAGE 68

Theorem3.2. For(r(m),n,t)andm2,9w(m,n)(rm)2[0,T]suchthatthedecisionmakershouldimmediatelyloaditemmiftw(m,n)(rm)andpostponetow(m,n)(rm)otherwise.ThecorrespondingexpectedprotisV(r(m),n,t)=Pmi=1gi,n(ri,t),wheregm,n(rm,t):=8>>>>>>>>><>>>>>>>>>:rm,ift0Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ZXgm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1UA(rm,r),t+dFR(r)+Zr>rmhgm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(rm,t+))]TJ /F3 11.955 Tf 11.96 0 Td[(gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t+)idFR(r)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)trm)]TJ /F6 7.97 Tf 19.32 4.71 Td[(c +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti,ift>0, (3)wheret=w(m,n)(rm))]TJ /F3 11.955 Tf 11.96 0 Td[(tandw(m,n)(rm)=mint:nZXgm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1UA(rm,r),tdFR(r)+Zr>rmhgm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(rm,t))]TJ /F16 10.909 Tf 10.91 0 Td[(gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r,t)idFR(r)o)]TJ /F16 10.909 Tf 10.91 0 Td[(rmc +,t0 (3)Moreover,ifw(m,n)(r)>0,w(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(r)
PAGE 69

Theorem3.3. PolicyisoptimaltotheDSKPmodelforanymandV(r(m),n,t)=Pmi=1gi,n(ri,t). Proof. SeeAppendix B.1 Theoptimalpoliciesandthecost-to-gofunctionpossessthefollowingstructuralproperties. Theorem3.4. Ifc(n)=cn,thevaluefunctionandoptimalpolicytotheDSKPwithpostponementpossessthefollowingstructuralproperties: 1. Fort2[0,w(m,n)(rm)),gm,n(rm,t)isstrictlyincreasinginnandrmandisstrictlydecreasingint; 2. Fort2[0,w(m,n)(rm)),(@=@r+m)gm,n(rm,t)2(0,1)andisstrictlyincreasinginrmandmanddecreasinginn; 3. Fort2[0,w(m,n)(rm)),(@=@t)gm,n(rm,t)isstrictlyincreasinginrm; 4. w(m,n)(rm)isnon-increasinginrm; 5. Fort2[0,w(m)(rm)),gm(rm,t)isstrictlyincreasinginrmandisstrictlydecreasingint; 6. Fort2[0,w(m)(rm)),(@=@r+m)gm(rm,t)2(0,1)andisnon-decreasinginrmandm; 7. Fort2[0,w(m)(rm)),(@=@t)gm(rm,t)isnon-decreasinginrm; 8. w(m)(rm)isnon-increasinginrm. Proof. SeeAppendix B.2 Intuitively,theexpectedprotisdeterminedbyrewardsofqueueditems;ahigherprotisachievedifthequeueditemspossesshighrewardvalues.Thenumberofitemsyettoarrive,n,determinesthepotentialofrewardsthatcanbeearnedinthefuture.Withalargern,therewardsofitemsinhandaffecttheexpectedprotless,sincethereexistsalargerchanceofreceivinghighrewardingitemsinthefuture.AnotherfactordeningtherewardpotentialofitemsyettoarriveisT)]TJ /F3 11.955 Tf 12.08 0 Td[(t.Thisexplainswhygm,n(rm,t)decreasesint;alongerT)]TJ /F3 11.955 Tf 12.05 0 Td[(tbenetsmorewhenqueueditemsarelowinreward.The 69

PAGE 70

numberofresourcesinhand,m,dilutestheeffectofn.Withmoreitemstobeloaded,morediscountingandholdingcostsmaybeincurredandthusthedecisionmakerislesspickyinloadinganitem.Thisexplainswhy(@=@r+m)gm,n(rm,t)decreaseswithm. 3.4TheFiniteHorizonDSKPwithPostponementOptions:ExtensionsInthissection,wediscussthefeasibilityofincorporatingothercostsandtheoptiontostopreceivingitemsbeforedepletingallresources,asthesecostsbetteraddresstheconcernsinapplications.First,weincorporatethedelaycostandtheoptionofstopping.Finally,wediscussthedifcultyofincorporatingtherejectionpenalty.Inthissection,werestrictourselvestothecaseofN(0+)=1. 3.4.1DelayCostForameaningfuldiscussion,werequirecq<1.Aswillbeshownin 3.4.1.1 ,thedelaycostinvalidatestheoptimalityoftheupdateruleUA;itmaynotbeoptimalonlytorejectthequeueditemwiththeleastreward.Thisproblem,however,canbexedbyintroducinganothersetofthresholdstoUA;uponanarrival,itisoptimaltorejectallqueueditemswithrewardslowerthanthesethresholds.Foranalysis,weconsideranewclassofpoliciesthatrestrictthenumberofitemsthatcanbereceivedwithoutallocatingaresource.Afteraresourceisallocated,thisclassofpoliciesfollow,theoptimalpolicy.Denotethistypeofpolicyas^(k),wherekisthemaximumnumberofitemsthatcanbereceivedwithoutallocatingaresource.Followingthisdenition,itshouldbeclearthatV^(0)(r(m),n,t)=rm+V(r(m)]TJ /F4 11.955 Tf 10 0 Td[(1),n,t).Moreover,askgoestoinnity,^(k)convergestopolicy.SimilartoSection 3.3 ,deneV^(k)(r(m),t;t)astheexpectedprotofpostponingtgiven(r(m),t),whereV^(k)(r(m),t;t)=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ZXV^(k)]TJ /F5 7.97 Tf 6.59 0 Td[(1)hUA(r(m),r),t+idFR(r)d+hrm+V(r(m)]TJ /F4 11.955 Tf 11.95 0 Td[(1),t+t)ie)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t)]TJ /F3 11.955 Tf 13.15 8.08 Td[(cm+cqe0r(m) +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti. 70

PAGE 71

Similarly,thedecisionofwhethertopostponedependsonthesignof^Gk(r(m),t),where^Gk(r(m),t+t)= +ZXV^(k)]TJ /F5 7.97 Tf 6.59 0 Td[(1))]TJ /F3 11.955 Tf 5.48 -9.69 Td[(UA(r(m),r),t+tdFR(r))]TJ /F14 11.955 Tf 11.96 13.27 Td[(hrm+V(r(m)]TJ /F4 11.955 Tf 11.95 0 Td[(1),t+t)i)]TJ /F3 11.955 Tf 13.15 8.09 Td[(cm+cqe0r(m) ++1 +@ @tV(r(m)]TJ /F4 11.955 Tf 11.96 0 Td[(1),t+t). 3.4.1.1Theupdaterulewithentry-barrierTheupdateruleUAdenedinSection 3.3 rejectsitemsuponarrivalsbyremovingtheleastrewardingitemfromthequeue;asthereisnobenetofimmediatelyrejectinganitemuponitsarrival.Then,thedecisionmakersimplychoosesbetweentheoptionpostponeandtheoptionimmediatelyload.However,thisupdateruleisnotnecessarilyoptimalwhenthereisacostforkeepingitemsinthequeue.Mainly,foranitemwithasmallreward,thebenetofkeepingitinhandaccuredbyhedgingagainsttheriskofencounteringitemseveninferiormaynotcompensatethedelaycostincurredduringpostponement.Toillustratethis,considerthecaseofm=1.Dene^g1,k(r1,t)astheexpectedprotbyfollowingpolicy^(k)andtheupdateruledenedinSection 3.3 .Whethertopostponedependsonthesignof^G1(r1,t+t)= +ZX^g1,0maxfr1,rg,t+tdFR(r))]TJ /F3 11.955 Tf 11.96 0 Td[(r1)]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr1 +,where^g1,0(r1,t)=r1.FollowingtheanalysisinSection 3.3 ,itiseasytoshowthat^G1(r1,t)=0hasauniqueroot,r.Wedenew(1)(r1)=0forr1randw(1)(r1)=Tfor 71

PAGE 72

r1>>>><>>>>>:r1iftw(1)(r1)ZT)]TJ /F6 7.97 Tf 6.58 0 Td[(t0ZXe)]TJ /F5 7.97 Tf 6.58 0 Td[((+)^g1,k)]TJ /F5 7.97 Tf 6.59 0 Td[(1hUA(r1,r),t+idFR(r)d+r1e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.59 0 Td[(t))]TJ /F6 7.97 Tf 13.15 5.7 Td[(c+cqr1 +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.59 0 Td[(t)iift
PAGE 73

3.4.1.2OptimalpolicyItisnotquitestraightforwardtodirectlyincludetheoptionofimmediaterejection.Theexpectedprotofpostponingtheload/rejectdecisionbytamountoftimeisV^(1)(r1,t;t)=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)hr1FR(r1)+Zr>r1rdFR(r)id+maxnr1,^g1,1(0,t+t)oe)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t)]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr1 +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti,where^g1,1(0,t)istheexpectedprotofimmediatelyrejectingitem1:^g1,1(0,t)=max0tT)]TJ /F6 7.97 Tf 6.58 0 Td[(tnE[R]Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)d)]TJ /F3 11.955 Tf 24.31 8.09 Td[(c +1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)to.NotethatwhenE[R]>c,themaximumoperatorintheaboveexpressioncanberemovedandthedecisionmakershouldkeepwaitingforthenextitem.Toavoidabusingnotation,wekeepthisassumptionanditshouldbeclearthatitdoesnotaffecttheanalysisonthresholdpoliciesbelow.Tosimplifytheanalysis,werstconsideraspecialoptimalstoppinggame:Aplayerwithapieceofassettosellwithfacevaluer1attime0.Theplayermaysellthisassetatanytime,earningitsfacevalue;orwaitforabonusopportunitytosellit,whichmayappearbeforethedeadlineT,accordingtoanexponentialdistributionwithparameter.Thebonusopportunitydenesabonuspricefortheassetbyafunctionf:ifr1r,f(r1,A)=r1andifr1r1,whereAisthearrivaltimeofthebonusopportunityandrisathresholdvalue.Supposef(r,t)isstrictlydecreasingintandnon-decreasinginr,f(r,T)=r,andE[R]>c.Moreover,suppose(@=@r)f(r,t)1andisnon-decreasinginr1.Finally,tostayinthegameincursaholdingcostcperunittimeandtokeepr1inhandincursadelaycostcqr1perunittime. Lemma3.2. Forthegamedescribedabove,theplayeratmostneedstoconsiderthreeoptions: (1) immediatelyrejectanitem; (2) immediatelyloadanitem; 73

PAGE 74

(3) postponetheloadingdecision.Moreover,thereexistsathresholdpolicydenedbythreshold()andthresholdw():foranytimet, ift>w(r1),theplayershouldimmediatelyloadtheinitialitem; iftw(r1)andr1>(t),theplayershouldpostpone; ifr1(t),theplayershouldimmediatelyrejecttheinitialitem.AndthecorrespondingexpectedprotisV(r1,t)=8>>>>>>><>>>>>>>:r1ift>w(r1)Rt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)f(r1,t+)d+r1e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t)]TJ /F6 7.97 Tf 10.5 5.69 Td[(c+cqr1 +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tiiftw(r1)andr1>(t)Rt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)f(0,t+)d)]TJ /F6 7.97 Tf 13.15 5.7 Td[(c+cqr1 +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tiifr1(t),wheret=w(r1))]TJ /F3 11.955 Tf 11.96 0 Td[(t. Proof. SeeAppendix B.3 Now,returntothecaseofm=1andk=1.Clearly,thedecisionissimilartothegamemetionedabovewithf(r,t)replacedbyRX^g1,0(maxfr1,rg,t),with^g1,0(r1,t)=r1.SimilartotheanalysisinLemma 3.2 ,wedene1,1(t)=ZT)]TJ /F6 7.97 Tf 6.59 0 Td[(t0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)E[R]d+r1e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.58 0 Td[(t))]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr1 +[1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.59 0 Td[(t)],^w(1,1)(r1)=8><>:0ifr1rTifr1
PAGE 75

1,1(t).Thentheoptimalexpectedprotis ^g1,1(r1,t)=8>>>>>>>><>>>>>>>>:r1ift^w(1,1)(r1)andr11,1(t)ZT)]TJ /F6 7.97 Tf 6.59 0 Td[(t0ZXe)]TJ /F5 7.97 Tf 6.59 0 Td[((+)maxfr1,rgdFR(r)d+r1e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.59 0 Td[(t))]TJ /F6 7.97 Tf 13.15 5.7 Td[(c+cqr1 +[1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.58 0 Td[(t)],ift<^w(1,1)(r1)andr11,1(t)1,1(t)ifr1<1,1(t),(3)Itisstraightforwardthat^g1,1(r1,t)satisestheconditionsofLemma 3.2 .Byrepeatingtheargumentabove,wedene1,k(t)=ZT)]TJ /F6 7.97 Tf 6.59 0 Td[(t0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)E[^g1,k)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r,t+)]d+r1e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.59 0 Td[(t))]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr1 +[1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.58 0 Td[(t)],^w(1,k)(r1)=8><>:0ifr1rTifr1>>>>>>>>>>>>>><>>>>>>>>>>>>>>>:r1,ift^w(1,k)(r1)andr11,k(t)ZT)]TJ /F6 7.97 Tf 6.58 0 Td[(t0ZXe)]TJ /F5 7.97 Tf 6.59 0 Td[((+)^g1,k)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(maxfr1,rg,t+dFR(r)d+r1e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.58 0 Td[(t))]TJ /F6 7.97 Tf 13.15 5.7 Td[(c+cqr1 +[1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.59 0 Td[(t)],ifr11,k(t)andt<^w(1,k)(r1)1,k(t),ifr1<1,k(t),(3)where^g1,k(0,t)=RT)]TJ /F6 7.97 Tf 6.59 0 Td[(t0RXe)]TJ /F5 7.97 Tf 6.59 0 Td[((+)^g1,k)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t+)dFR(r)d)]TJ /F6 7.97 Tf 19.31 4.7 Td[(c +[1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)(T)]TJ /F6 7.97 Tf 6.59 0 Td[(t)].Itisstraightforwardtoshowthroughinductionthat1,k(t)ismonotonicallyincreasingink.Thenifr11,k(t),r11,k)]TJ /F5 7.97 Tf 6.59 0 Td[(1(t)1,1(t).As1,k(t)is 75

PAGE 76

boundedforanyk,itfollowsthat1,k(t)uniformlyconverges,say,to1(t).Themono-tonicityof1(t)intindicatesthatifanitemcanenterthequeue,itisoptimaltoretainitinthequeueatleastuntilthenextarrival.Therefore,load/rejectdecisionsstilloccuratarrivalsepochs.Withthe1-barrierUA,thedecisionprocessisindeedtheMDPdenedinSection3.2.Similarly,denew(1)(r1)=limk!1^w(1,k).Itisalsostraightforwardthat^g1,k(r1,t)uniformlyconverges,denotedas^g1(r1,t).Also,itiseasytoseethatthemonotonicitypropertiesofthestatevaluesstatedinTheorem 3.1 holdfornitek,exceptthatthestrictmonotonicityshouldbereplacedasmonotonicity.Thesepropertiesareinheritedby^g1(r1,t).Finally,thestatevalueV(r1,t)=^g1(r1,t).Wegeneralizetheresultsinthefollowingtheorems: Theorem3.5. Supposecq<1andE[R]>c.Theoptimalpolicyistoconsiderthebestiteminthequeuerm.Ifrmm(t)andtw(m)(rm),theoptimaldecisionistoimmediatelyloadrm.Ifrmm(t)andtw(m)(rm),theoptimaldecisionistopostponeloadinguntilw(m)(rm).Finally,ifrm>>>>>>>>>>>>>>><>>>>>>>>>>>>>>>>:rm,ifrmm(t)andtw(m)(rm)Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)gm(rm,t+)FR(rm)+Zr>rmgm(r,t+)+gm(rm,t+))]TJ /F16 10.909 Tf 10.91 0 Td[(gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t+)dFR(r)d+rme)]TJ /F5 7.97 Tf 6.58 0 Td[((+)t)]TJ /F6 7.97 Tf 12.1 5.38 Td[(c+cqrm +[1)]TJ /F16 10.909 Tf 10.91 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t],ifrmm(t)andt
PAGE 77

Bythemonotonicityofk(t)int,onceanitementersthequeue,itisnotremoveduntilthearrivalofthenextitem.Sincek(t)k)]TJ /F5 7.97 Tf 6.59 0 Td[(1(t),whenanewitemarrives,itislikelythattheitemoriginallyseatedinthek-thpositioninthequeuecannotbemovedtothe(k)]TJ /F4 11.955 Tf 12.3 0 Td[(1)-thposition,ifitsrewardisbelowk)]TJ /F5 7.97 Tf 6.58 0 Td[(1(t).Moreover,thedecisionmakermayremovemorethanoneitemfromthequeue.Inotherwords,itislikelythatthedecisionmakermayputaniteminthequeue,keepitforaperiodoftimeandrejectitevenwhenthenumberofitemsheldislessthanthenumberofunallocatedresources.This,tosomeextent,iscounter-intuitive,sincethedelaycostsforkeepingtheseitemsinthequeuearepaidinvainanditwouldbetterhadtheybeenrejectedearlier.Notethatthisconclusionistruebasedontheinformationrevealedsofar;thedecisionmadeatthepreviousepochisoptimalbasedontheinformationavailableatthattime,andissurelylesswisebasedontheinformationatpresence.Actually,whenanitemismovedtoapositioninthequeuewithalowerindex,itusuallymeansthatitshouldstayinthequeueformoretime;thisimposesahigherrequirementontherewardofanitem,formoredelaycostswillbecharged.Itremainstoshowtheoptimalityofthispolicy.First,itisstraightforwardthatthethresholdbywhichtoloadanitemremainsoptimal,forthedelaycostdoesnotaffecttheproofofTheorem3.Specically,ifindicatesimmediateloading,thereisnobenettopostpone.Ifindicatestopostpone,itisbesttopostponeuntilthestoppingtimeifnoarrivalcomes.FollowingasimilarproofofTheorem 3.3 ,itisoptimaltorejectanitembythe-barrier.Thus,thepolicyequippedwiththe-barrierupdateruleUAisindeedanoptimalcontrolpolicyforthecontrolledMarkovprocessdenedinSection4.1,asstatedinthefollowingtheorem: Theorem3.6. Ifcq<1,thepolicystatedinTheorem 3.5 isoptimaltotheDSKPmodelwithpostponementoptions. 77

PAGE 78

3.4.2StopOption,RejectionPenaltyandStructuralPropertiesAsnotedearlier,theDSKPmodelrequiringimmediatedecisionscannotbemodeledasanMDPinastraightforwardwaywhenthestop-and-salvageoptionisincorporated. KleywegtandPapastavrou ( 1998 )introduceastopdecision,I(m,t),intotheDSKPmodel,withmandtdenedsimilarly.Ifthedecisionmakerdecidestostopreceivingitems,I(m,t)=1,andtheunallocatedresourcesreceiveasalvagevaluev(m).Accompaniedwiththisdecisionisastoppedstateandaninnitetransitionratetothisstate,whichcomplicatestheanalysissincemostresultsforMDPsassumeboundedtransitionratesbetweenstates.Toavoidthisdilemma,theauthorsallowthedecisionmakertoswitchthedecisionprocessonandoffmultipletimes,ratherthanstoponlyonce.Inotherwords,thedecisionmakercanresumetheprocessuponanarrivalandreceiveitemseveniftheprocessisterminated.TheresultingMDPisobviouslyarelaxationoftheDSKP.ItisshownthatthereexitsanoptimalpolicytothisMDPmodel,whichisalsooptimalandadmissibletotheoriginalDSKPmodel.Implicitinthismodelaretwoassumptions.First,itispossibletosalvagetheunallocatedresourcesatanytime,sincein KleywegtandPapastavrou ( 1998 ),thesalvagefunctionv()isindependentoftime.Second,ifoneresourceistobesalvaged,thenallresourcesthathavenotbeenallocatedmustbeimmediatelysalvaged.Wewouldarguethattheoptiontostopreceivingitemsandsalvagingtheunallocatedresourcescanbegeneralizedtoallowingasubsetoftheavailableresourcestobesalvaged.First,considerthecasewithnodelaycost.SimilartotheMDPmodelproposedby KleywegtandPapastavrou ( 1998 ),weallowthedecisionmakertocontinuereceivingitemsevenifaresourceissalvaged.However,insteadofrequiringthedecisionmakertosalvageallunallocatedresourcesatatime,thedecisionmakermayconsidersalvagingasubsetofmresources.Afterthis,thedecisionprocesscontinuesandsomeresourcesmaybeallocatedbeforethesalvageofanotherresource.Whenaresourceissalvaged, 78

PAGE 79

thedecisionmakersimplyreceivesarewardasifanitemwithrewardvisacceptedandassignedtothisresource.Inotherwords,byassumingthatv(m)islinear,wecantreattheunallocatedresourcesasitemsheldinthequeue,withr(m)=(v,v,...,v)0astheinitialqueue.Whenanitemarrives,ifitsrewardislessthanv,theitemcannotenterthequeueandisrejectedimmediately.Astimeelapses,ifnoitemarrivesuntilt=w(m,n)(v),thedecisionmakershouldsalvagearesource.Afterthat,thedecisionmakermaypostponesincet=w(m,n)(v)w(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)(v).Inthisway,itislikelythatthedecisionprocessstopswithoutallresourcesactuallybeingallocated.Tosomeextent,ourmodelallowsmoreexibilitywhensalvagingunallocatedresources.Inthetruckingindustry,atruckmaynotreceiveafullloadatonelocationandthedrivermayreceivecallsreservingcapacityfromanotherpick-uplocation.Similarly,managerssellingassetsmaynothavetoexposeallavailableassetstothemarket.Whenthedeadlineiscloseandthenumberofofferstoarriveissmall,itmightbewisetoonlyputsomeoftheassetsonthemarket,salvagingtheremainingthroughotherchannels.Otherwise,itislikelythatsomeresourceswillndnobuyersbutincuropportunitycosts.Infact,evenwhenapplyingtheDSKPmodelintheliterature(e.g., KleywegtandPapastavrou ( 1998 )),V(m,0)isnotnecessarilythehighestprotthatthedecisionmakercanachievegivenmresources.Rather,max1kmfV(k,0)+v(m)]TJ /F3 11.955 Tf 12.03 0 Td[(k)gisgenerallyhigher.Ourmodelsimplyincorporatesthisconsiderationdirectly.Now,considerthecasewhenadelaycostisadded.Inthiscase,theonlydifferenceisthatthefunctionsequencegm(r,t)needstoberedenedasgm(r,t)=gm(v,t)forrv.Note,however,thatitisnoteasytoaccommodatetherejectionpenaltyintothemodel.GivenM(0)=0,thequeueconsistsofitemswithreward0.Whenanewitemarrivesandisaddedintothequeue,thenoneitemwithvalue0istoberemovedfromthequeue.However,thequeuingmodelpresentedinthischaptercannotdistinguishwhetheranitemremovedfromthequeueisvaluedat0.Wewouldarguethatthisdoesnotaffecttheapplicabilityofourmodelinpractice.Inmostcases,therejectionpenalty 79

PAGE 80

representsthelossofgoodwill,whichisusuallyincurredwhenapriceofferabovethefairvalueofanassetisrejected.Infact,thethresholdvalue(minimumrewardthatcanbeimmediatelyaccepted)itselfcanberegardedasafairpriceandthereisnorejectionpenalty.Thefunctionsequencegm(r,t)possessesthestructuralpropertiesdenedinTheorem 3.4 Theorem3.7. Ifc(m)=cmv(m)=vm,p=0,andcq=1,theDSKPwithpostpone-mentoptionspossessesthefollowingstructuralproperties: (i) Fort2[m(t),w(m)(rm)),gm(rm,t)isstrictlyincreasinginrmandisstrictlydecreas-ingint; (ii) Fort2[m(t),w(m)(rm)),@gm(rm,t) @r+m2(0,1)andisstrictlyincreasinginrmandm; (iii) Fort2[m(t),w(m)(rm)),@gm(rm,t) @t0andisnon-decreasinginrm; (iv) w(m)(rm)isnon-increasinginrmandisnon-increasinginm. Proof. TheprooffollowstheproofofTheorem 3.5 .Theintroductionofvdoesnotaffecttheanalysis. 3.5NumericalExamples:BenetsofPostponementTotesttheeffectofpostponement,wecomparetheprotyieldedbyourmodelandtheprotyieldedbythatof KleywegtandPapastavrou ( 1998 ).Assumingthat=0,=1,XU(0,20),T=25,M(0)=15,N(0)=1,c=0.25andcq=0,theshapesofthefunctionsequencegm(r,t)arepresentedinFigure 3-1 .Theexpectedprotgivenr,t,andm,isPmi=1gi(r,t).Figure 3-2 showstheacceptancethresholdsobtainedbythemodelwithoutandwithpostponementoptions.Comparingthetwogures,itcanbeenseenthatwithoutthepostponementoption,thedecisionmakeracceptsitemswithrewardhigherthan4.5given15resourcesattime0.Withthepostponementoption,however,thedecisionmakerrequiresarewardgreaterthan6.0forimmediateloading.Theimprovementinprotwiththepostponementoptioninthisexampleapproaches4.0%(Theactualimprovementisdependentontheapproximationmethodchosentocomputegk(r,t).). 80

PAGE 81

Figure3-1. Valuesofgi(r,t),fori=1,2,...,15 AWithoutPostponement BWithPostponementFigure3-2. ThresholdsoftheDSKPwithandwithoutpostponementoptions. 81

PAGE 82

Intuitively,postponementisbenecialbecausethehelditemshedgeagainsttheriskthatrewardsofitemsyettoarrivearelessthanexpected.ThisriskdecreasesasTandincrease.Supposethatthedecisionmakerholdsanitemvaluedatr1,T=25,and=1.Roughlyspeaking,itcanbeexpectedthatmorethan20itemswillarrivebeforethedeadline.Inthiscase,theprobabilityofalltheseitemsbeinginferiortothehelditemisnegligibleandthusthebenetofpostponementissmall.Otherfactorsthatsignicantlyreducethebenetofpostponementarethediscountfactor,holdingcostc,anddelaycostcq.Theeffectofthediscountfactorisclear(Chapter2).Theeffectofcandcqlieinthattheydecrease(@gk(rk,t)=@rk).Asaresult,thebenetofpostponement,gk(rk,t))]TJ /F3 11.955 Tf 11.95 0 Td[(gk(0,t),decreases. 3.5.1ImpactofTimeHorizonIntheDSKPliterature,suchas KleywegtandPapastavrou ( 1998 ),numericalexperimentsassumearelativelylongtimehorizon,suchasT=100.Suchalonghorizonrevealsthemodel'sbehaviorinboththeinnitehorizoncaseandthenitehorizoncase.Suppose=0,=1,c=0.25,cq=0,RU(0,20),M(0)=15andv()=0.TheexpectedprotwhenT=100isalmostthesameasT=5000.ConsiderabledifferencesappearwhenTdropsbelow50.Inourexperiments,thebenetsofpostponementareobviousonlywhenTislessthan35.Ontheotherhand,whenthetimehorizonisquiteshort,onlyafewitemsmayarrivebeforeT,whichalsoreducesthebenetofpostponement.Ingeneral,postponementisbenecialwhenTismoderate. 3.5.2ImpactofcTheholdingcostcmotivatestheloadingoption.Intuitively,whencislarge,boththeimmediaterejectionoptionandthepostponementoptionleadtohighercosts.Itwasnotedthatthepostponementoptionimprovedtheprotbynearly4.0%intheearlierexample.However,thiscomparisonwasconductedbetweenthemodeldevelopedinthischapterandthemodeldevelopedby KleywegtandPapastavrou ( 1998 ),whichare 82

PAGE 83

notequivalentmodels.Forexample,whenitisclosetothedeadline,theprobabilityofhavinganarrivalbythedeadlineislowandsoisitsexpectedreward.Thus,duetoholdingcosts,itmaybebettertoabandonsomeresources(e.g.,salvage).Considerthecaseof=0,=1,T=25,RU(0,20),andc=0.25.Theexpectedprotofthemodelby KleywegtandPapastavrou ( 1998 )isgiveninFigure 3-3 .Asseen,thereexistcaseswhenV(m,t)
PAGE 84

Figure3-4. ExpectedProtofexistingmodel( KleywegtandPapastavrou ( 1998 )):V(k,t))]TJ /F3 11.955 Tf 11.95 0 Td[(V(k)]TJ /F4 11.955 Tf 11.96 0 Td[(1,t). KleywegtandPapastavrou ( 1998 )noteasufcientconditiontoavoidthisphenomenonbydeningc(m)=c.Analternativeapproachistodevelopanimmediatedecisionpolicythroughourmodel,whichassumesc(m)=cm.Whensettingcq>1,thedecisionmakeronlyresortstoimmediateloading/rejectingdecisions.ThecounterpartofFigure 3-4 isgiveninFigure 3-5 .Itisalsointerestingtointerpretthisphenomenoninanotherway.Theoptimalpoliciesofexistingmodels( KleywegtandPapastavrou ( 1998 ))werederivedfromtheHamilton-Jacobi-Bellman(HJB)equation(underN(0)=1):)]TJ /F20 10.909 Tf 9.68 7.38 Td[(@V(m,t) @t=ZrV(m,t))]TJ /F6 7.97 Tf 6.59 0 Td[(V(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,t)r)]TJ /F14 10.909 Tf 10.9 8.84 Td[(V(m,t))]TJ /F16 10.909 Tf 10.91 0 Td[(V(m)]TJ /F18 10.909 Tf 10.91 0 Td[(1,t)dFR(r))]TJ /F20 10.909 Tf 10.91 0 Td[(V(m,t))]TJ /F16 10.909 Tf 10.91 0 Td[(c(m).Thesolutionofthispartialdifferentialequation(PDE)isnotstraightforwardduetotheholdingcost.Asisknown,thesolutiontoaPDEissensitivetotheboundary 84

PAGE 85

Figure3-5. Expectedprotofthenewmodel:gk(0,t))]TJ /F3 11.955 Tf 11.96 0 Td[(gk)]TJ /F5 7.97 Tf 6.59 0 Td[(1(0,t). condition.Inliterature(e.g., KleywegtandPapastavrou ( 1998 )),theboundaryconditionisassumedtobeV(m,T)=v(m),whichiszerointhisexample.Notethatinthiscasetheoptimalstoppingruledoesnotplayarole.Assumethatm=1,c>0,andV(1,t)>0.ItmaynotbecorrecttosetV(1,T)as0.Indeed,itisalsopossibletodeveloptheoptimalpolicyofourmodelthroughtheHJBequation,whichcanusuallybederivedfromtheprincipleofdynamicprogram-ming.However,theresultantHJBequationisafree-boundaryPDE,whichisnoteasytosolve.Specically,thedecisionmakershouldpostponeuntiltimew(m)(rm),given(r(m),t)andt
PAGE 86

First,wecomparetheexpectedprotofourmodelwithandwithoutpostponement(betweencq=0andcq>1)overdifferentdistributionsofR.Twosetsofparametervaluesaretested,sharingthefollowingparameters:=0.001,=1,c=0.25,cq=0,m=15,andv=0.TherstparametersettingassumesR=20,whereBeta(1.5,3.0).Theleftleaningp.d.f.ofRisplottedinFigure 3-6 .Improvementsof3.02%areexperiencedinthiscase,asgiveninFigure 3-7 .ThesecondparametersettingassumesRU(0,20),c=0,and=0.005.Improvementsofnearly3.0%areshowninFigure 3-7 Figure3-6. P.D.FofR=20,Beta(1.5,3.0). Second,wecomparetheexpectedprotofourmodelwithandwithoutpostponementoverdifferentcvaluesunderthesamedistribution.Here,R=20,Beta(1.5,3.0)andthevaluesofallotherparametersremainthesameexceptthatcvaries.Figure 3-8 illustratestheeffectoftheholdingcostontheprotimprovementofthepostponementoption.Interestingly,cdetermineswherethemaximumimprovementisachievedinadditiontoitsvalue.NotethatinFigure 3-8 ,weuseabsoluteprotimprovementinsteadofrelativeprotimprovementtogiveaclearerview. 86

PAGE 87

Figure3-7. Percentageofprotimprovement. Figure3-8. ImpactofHoldingCostonProtImprovement. 87

PAGE 88

3.5.3ImpactofDelayCostsandDelaysDelaycosts,cq,arechargedtoitemsinthequeue.Asindicatedpreviously,benetsofthisonlyaffectthepostponementdecision.Whencqislarge,thepostponementoptionisnegligible.Figure 3-9 illustratestheeffectofthedelaycostontheprotimprovement.Comparedwithc,theprotimprovementoverthetimehorizonexhibitsauniformchange. Figure3-9. ImpactofDelayCostonProtImprovement. Aninterestingissueisthevalueofdelay.Weanalyzethisthroughsimulation.Foreachcq,thethresholdsareobtainedandadoptedtocontroltheloading,postponement,andrejectiondecisions.Foreachitem,thedelaytimeisdenedasthedurationbetweenthearrivalofanitemandthemomentwhenitisloadedorrejected.Foreachsimulationrun,thedelaytimeisaccumulatedoverallitemsthatactuallyarriveduringthedecisionprocess.Theaveragedelaytimecorrespondingtoeachcqistakenastheaverageofthedelaytimesof5000runs.TheprotimprovementthroughpostponementisplottedagainsttheaveragedelaytimeinFigure 3-10 .Theresultsareintuitive.Theprotimprovementisessentiallyconcavewithrespecttothelevelofdelay,indicatingthattheinvestmentindelayleadstoadiminishingmarginintheprotgain. 88

PAGE 89

Figure3-10. ValueofDelay. Thedelaycostcanbeusedtoeitherrepresenttherealcostsofholdingitemsinthequeueorbeutilizedasatooltocontrolthedelaytime.Suppose=1,T=25,cq=0.025,m=15,v=0,=0.001,c=0andR=20,Beta(1.5,3.0).Theexpectedprotwithandwithoutpostponementare127.31and129.93,respectively.Ifweonlyusecqasatooltocontrolthedelaytime(i.e.,notcountedintheprot),theexpectedprotis130.58andtheaveragedelayis7.88,fromasimulationof5,000scenarios. 3.6DSKPwithRandmlySizedItemsInthissection,webrieydiscussthesituationwhenthearrivingitemsareofheterogeneousandrandomsizes.Theapplicationmotivationofthisproblemincludethecapitalbudgeting,capacityreservation,andrevenuemanagement. KleywegtandPapastavrou ( 2001 )foundathresholdpolicyforthisproblem.However,whenthepostponementoptionisincluded,itisnotlikelythattheoptimalpolicyhasathresholdstructure.Itiseasytoseethatforn=0,theresultingproblemisastaticknapsackproblem,whichisNP-hard.ItisthusdifculttocharacterizetheexpectedprotfunctionVforn>1.Werstintroducethemodel,whichutilizesthenotationof KleywegtandPapastavrou ( 2001 ),andthenproposeanheuristicalgorithm.SupposethereareKtypesofitems, 89

PAGE 90

witheachhavingofrewarddistributionasFkRandsizeasmk(k=1,2,...,K),respectively.Moreover,assumethattheprobabilityofanitembeingoftypekispk.Insteadofbuildingupthedynamicprogrammingequation,wedecomposetheproblemintoKsubproblems,eachofwhichisaDSKPproblemwithhomogeneouslysizeditems.Inotherwords,wecanallocatecapacitiestoeachofthesesub-DSKP's.Specically,denotetheexpectedprotfunctionofeachsub-DSKPasfk.Theoptimalallocationcanbeachievedbysolvinganon-linearknapsackproblem:max:Xkfk(xkmk,rxk,n,t) (3)s.t.:Xkxkmkmxk2Z,8k. (3) 3.7ConclusionsTheDynamicandStochasticKnapsackProblem(DSKP)withhomogeneousitemsandthepostponementoptionwasdenedandanalyzed.Whenthepenaltycostp=0andthedelaycostcq<1,theoptimalpolicypossessesasimplethresholdstructuredenedaccordingtothetimeperiodsuchthatthedecisionmakershouldpostponetheloadingofthebestiteminthequeueuntilthereward-basedtimethresholdvalueisreached.Whenthisoccurs,thebestiteminthequeueisloaded.Furthermore,generalstructuralpropertiesofthecost-to-gofunctionareidentied.Also,thismodeladdressestheextensionoftheDSKPmodelby Papastavrouetal. ( 1996 ),whichallowsarejecteditemtoberecalledatsomecost.Moreover,themodelwithdelaycostscanbeusedasanapproximationofthesituationwhenthereisapostponementdeadlineforeachitem.Thereareavarietyofapplicationswhereanassignmentcanbedelayedinordertogathermoreinformation,especially,inrevenuemanagement.Considerahotelwithseveralclassesofroomstobeallocatedthroughpricingstrategies.Ineachclass,there 90

PAGE 91

areseveralnearlyhomogeneousrooms,withspecialpropertiesjustifyingcertainpricedifferentiationsoftheclass.Correspondingtoeachclassthereisacustomersegment,offeringdistinctprices.Duetothevariabilitiesofpriceanddemand,roomsofahigherclassmaybesoldtocustomersofalowersegmentandviceversa.Ifthepricevariabilityforahighclassisrelativelyhigh,pricearrivalsfortheroomsofahighclassmaybeundesirableandsomeroomsinthisclasscanbegiventoalowerclass.Inthiscase,thedecisionmakersalvagesroomsinthehigherclassbysellingthemtocustomersinthelowersegment.Indeed,buy-upandbuy-downpolicieshavenotbeenresolvedintherevenuemanagementliterature,sincesegmentationisusuallynotperfectinthedesignoftherestrictions(inhotelcases,theclassesarephysical)andthepricedifferencesbetweentheclassesarerarelythatdispersed( TalluriandvanRyzin ( 2005 )).Itwillbeinterestingtostudythesegmentationstrategythatcanbeadjustedaccordingtorealtimedemands.Anotherissueworthyofnotingisthedelayexperiencedbyhelditems.Indeed,numericalexperimentsindicatedthatgm,n(rm,t)maynotbesignicantlyhigherthangm,n(0,t)forsomevaluesofrm.Thismeansthatitmaynotbebenecialtoholdanitemifitsrewardisnotlarge.Indeed,m(t)helpseliminateitemsfromthequeuewhichdonotcontributemuchtotheprot.Theseindicatethepossibilityofearningahigherprot,withalimitedamountofdelaysonitems.Particularly,itwillbeinterestingtostudythecasewhenitemshavedeadlinesforloading.Clearly,itwouldalsobeofvaluetostudythecasewithheterogeneousitems,asitisclearthatitemsofdifferentsizemayarrive.Thisaddsanotherdimensiontotheproblemandgreatlycomplicatestheanalysis. 91

PAGE 92

CHAPTER4THEDYNAMICANDSTOCHASTICLOADINGPROBLEM 4.1IntroductionandLiteratureThischapterismotivatedbyproblemsexperiencedinthetransportationindustryDistributioncompaniesoftenaggregateshipmentsanduseaircrafttoexpeditetraveltime.Thisisaccomplishedbyloadingpackagesintospecializedcontainerswhichtintothehullofanaircraft.Thegoaloftheloadingoperationistobalancetheload,fromfronttoback,overtheplane'swings.Moreover,inloadingtheplane,whichisgenerallyfromtherear,caremustbetakentonottiptheplane.Similarapplicationsarisewithcargoshipsandlolling.Inourapplication,thecontainersareloadedwithpackagesovertimeandthenweighed.Hence,thecontainersarriveforloadingaccordingtoastochasticprocess,uponwhichtimetheirweightsbecomeknown.Asdeparturesofloadedcraftaregenerallyscheduled,thereisadeadlineastowhenloadingmustcommence.Thismayoccurevenbeforeallofthecontainershavearrived.Additionally,theremaybeabenettoloadingthecraftearlier,asitensuresahigherprobabilityofanon-timedeparture.Thisisespeciallycriticalintheovernightshippingindustry.Inthispaper,weexamineageneralcaseofthisproblemandexaminesomespecicsolutionsforcertainaircraftcongurations.Theaircraftloadingproblemisnotnewtotheoperationsresearchliterature. CochardandYost ( 1985 ), Martin-Vega ( 1985 ), Amiounyetal. ( 1992 )haveexaminedtheproblemwithdirectconcernstoweightdistribution. Thomasetal. ( 1998 )examinetheloadingproblemwithrespecttoshearandcenterofgravityconstraintswithintegerprogramming.Intheirthreephaseapproach,therstphaseisconcernedwithidentifyingafeasibleassignmentofcontainers. MongeauandBes ( 2003 )explicitlypointouttherelevanceofthebalanceofcargocontainerstothefuelefciency:adisplacementofthecenterofgravityoflessthan75cminalong-rangeaircraftyields,overa10,000kmight,asavingof4,000kgoffuel.Theauthorsdevelopanon-linear, 92

PAGE 93

integerprogrammingformulationtominimizefuelconsumptionduetounbalancedloadswhilemeetingstabilityconstraints. LimbourgandLaporte ( 2011 )pointoutthatthebalanceofthecargocontainersalsoaffecttheefciencyofanaircraftwithrespecttoitsaltitude,maneuverability,rateofclimb,andspeed,anddevelopeamixed-integerlinearprogram.Inparticular,theauthorsintroducethemomentofinertiaastheobjectivefunction,whichgreatlysimpliestheoptimization.Thesepreviousresearchworksgenerallyassumethattheproblemisstaticinthatallcontainerweightsareknownattimezeroandthedecisionisthesequenceinwhichtoloadthecontainers.However,loadingcontainersistimeconsuming.Aspointedoutin LimbourgandLaporte ( 2011 ),itusuallytakesasignicantlengthoftimeforanexperiencedloadplannertoloadallcontainersforBoeing747. Thomasetal. ( 1998 )statethattostreamlinetheprocessandtoreducethegroundtimeoftheaireet,thegroundcrewwillusuallystartpackingplanesbeforeallthecontainersarriveattheramp,andingeneral,aplaneisone-thirdtoone-halffullbythetimethegroundcrewknowsallofthecontainerinformation.Inthischapter,weexaminethedynamicaspectsofthisprobleminthatloadingmaycommencebeforeallcontainershavearrived,asnotedin Thomasetal. ( 1998 )andwitnessedinourexperiencewithapackageshippingcompany.Inourknowledge,thecontainerloadingprocessisasfollows:Packagesenterafacilitywheretheyaresortedaccordingtonaldestinationandloadedintocontainersaccordingtonaldestinations.Theloadedcontainersarethenmovedtoaweighingstation.Onceweighed,theymovetoastagingareawheretheycanbeloadedontothecraft.ThroughathoroughconversationwithaheadofcialoftheUPS,whoworksintheairfreightoperationdepartmentformorethantwentyyears,wefoundthatinpracticeforanaircraftwithfortyslots,theloadingprocessstartsusuallyaftervetotencontainersarrive,inordertoshortenthegroundtime.Moreover,itisconrmedthattheimbalanceofcargocontainersmayresultinunloadingandreloadingofcontainersanddelaythe 93

PAGE 94

ight,thecostofwhichcouldbeprohibitivelyhigh.However,despitetheimportanceoftheDSLP,theUPSisusingsomeexperience-basedrulestohandletheproblemandtheresultsarenotquitesatisfactory.Astheheadofcialpointedout,theresupposetobemoreanalyticalworkdoneforthisproblem.TheDSLPissimilartoanumberofproblemsintheliterature,includingthedynamicandstochasticknapsackproblem( KleywegtandPapastavrou ( 1998 2001 )and Papastavrouetal. ( 1996 )).Inthatproblem,itemsofrandomsizeandvaluearriveovertimeuponwhichthedecisionistoacceptorrejecttheitem.Inourapplication,thereisnodecisiontoacceptorreject,butratheradecisiontoloadornotandinwhichorder.TheDSLPcanalsobeviewedasanassignmentproblem,ascontainersareassignedtoholdsinthecraft.Thesequentialstochasticassignmentproblem( Dermanetal. ( 1972 )and Albright ( 1974 )etc.)isconcernedwithassigningjobs,whicharriveoneatatimewithvaluesthatbecomeknownuponarrival,topeople.Theobjectiveistomaximizetheexpectedtotalrewardwhichisbasedontheindividualpairings.Inourapplications,thepairingsmustnotoccurimmediatelyandadditionalconstraintsgoverntheassignments.However,withouttheseadditionalconstraintsandassumingthedeadlineistimezerosuchthatcontainersareloadeduponarrival,theproblemsareequivalent.ThedynamismandstochasticityoftheDSLPgreatlycomplicatethebalancedloadingproblem.Indeed,standardoptimizationmethodsdonotwellapplyfortheDSLP,duetothetighttimeconstraintinpractice.Wethusintroduceaheuristicalgorithmthatcangenerateloadingplansofhighquality.Thealgorithmcanserveasaviabledecisiontoolforvaluingthetrade-offbetweenreducinggroundtimeandbalancingthecontainers.Thischapterisorganizedasfollows.Section 4.2 presentsafuelsavingmodeltoillustratetherelationshipbetweenthecargobalanceandthefuelconsumption.Section 4.3 describestheproblemindetailsandintroducesnotation. 94

PAGE 95

TheDSLPisanalyzedfromtheperspectiveofdynamicprogrammingandotheroptimizationmethodsinSection 4.4 and 4.5 ,respectively.Section 4.6 utilizestheinsightsobtainedfromtheprevioustwosectionsandpresentsanheuristicalgorithmandtestsitsperformancewithoutconsideringconstraintsinpractice.InSection 4.7 ,wediscusshowtoincorporatetheseconstraints. 4.2Motivation 4.2.1FuelSavingsThemainmotivationforbalancingaplaneistoreducefuelcosts.Therangeofanaircraftisinuencedbyanumberoffactors,includingenginethrust,weight,aerodynamics,lift-to-dragratio,andthelocationofthecenterofgravity(CG).ChangingtheCGalterstheaircraft'saerodynamicsandthusrequiresightvariationstomaintaintrim.Notethattrimightisachievedwhenliftequalsweightandthetailpitch-momentisbalancedbythewingpitch-moment.ShiftingtheCGaftofanoptimalcongurationincreasesthemomentarmforthewingwhiledecreasingthemomentarmforthetail.Consequently,anincreaseinthetailliftisrequired(achievedbyincreasingtheelevator)tomaintainbalanceofthepitchmoments.Anincreaseintaillift,addedtotheunchangedwinglift,resultsinthetotalliftbeinggreaterthantheweightandthus,theangleofattackmustbereducedfortheaircrafttoreducethetotallift.Thisinturninuencestherangeperformance,whichweapproximatehere.Denexw,xt,andxeasthedistancefromtheaircraft'sCGtotheaerodynamiccenterofthewing,tailandelevator,respectively.DeneLasthelift,withLwoastheconstantliftfromthewing;Lwasthevariationofwingliftwithangleofattack;Ltasthevariationoftailliftwithangleofattack;andLeasthevariationofelevatorliftwithangleofattack.Notethatistheinitialtrimvalueforanangleofattackwhileeistheinitialtrimvalueforelevatordeection.Shiftsintheangleofattackandelevatorinorderto 95

PAGE 96

trimaremeasuredbyaande,respectively.Finally,WisthetotalweightoftheaircraftandistheshiftinthelocationofCG.Assumetheaircraftistrimforsomeinitiallocationofcenterofgravity.Thisdenesthattheaircrafthasnounbalancedforces: 0=Lw0+Lw+Lt+Lee)]TJ /F3 11.955 Tf 11.96 0 Td[(W,(4)norunbalancedmoments: 0=(Lw0+Lw)xw+(Lt)xt+(Lee)xe.(4)AvariationofintheCGleadtoperturbationsintheangleofattackandelevatordeection,denedinEq. 4 and 4 ,respectively =Lee(xe)]TJ /F8 11.955 Tf 11.96 0 Td[()+Lt(xt)]TJ /F8 11.955 Tf 11.96 0 Td[())]TJ /F4 11.955 Tf 11.96 0 Td[((Lw0+Lw)(xw+) Lw(xw+))]TJ /F3 11.955 Tf 11.95 0 Td[(Lt(xt)]TJ /F8 11.955 Tf 11.96 0 Td[()+(Lw+Lt)(xe)]TJ /F8 11.955 Tf 11.95 0 Td[()(4) e=Lw+Lt Le(4)Therangeoftheaircraftisproportionaltotheratiooflifttodrag.Theliftofmostaircraftaroundtrimisalinearfunctionofangleofattackandalinearfunctionofelevator: L=L0+L+Lee,(4)whilethedragismorecomplicated.Itvariesasaquadraticoftheangleofattackandlinearlyintheabsolutevalueoftheelevator,or D=D0+D2+Dejej.(4)Dening_Wasthefuelburn,Tasthrust,Vasvelocity,andinitialandnalweightsW1andW2,therangeoftheaircraftisdenedas R=VT _WL DlnW1 W2(4) 96

PAGE 97

Fromourapproximation,themodelpredictsthata10footperturbationinCGfromtrimmaycauseareductioninrangeof4.7%.ConsideraNewYorktoLosAngelesightof2500miles.Thisreductioninrangewouldleadtoanestimatedincreasein587gallonsoffuelrequiredtomakethenaldestination.Atanestimated$4.30pergallonofjetfuel(REFERENCE)andfuelburnof5gallonspermile(REFBOEING),theincreasedcostofthesinglelegightisover$2500.Fora6760mileightfromNewYorktoToyko,theincreasedfuelcostisestimatedatover$6,830.AsthelengthofaBoeing747is230feet,notethatthisperturbationintheCGismerely4.3%ofthelengthoftheaircraft.Assumingalinearrelationship,a1%errortranslatesintoroughlya1%increaseinfuelcosts. 4.2.2ValueofDelayingaFlightAspointedoutearlier,thesimplestsolutionforthebalancedloadingistowaituntilallcontainers'weightinformationisknownandsolvetheresultingquadraticassignmentproblem.However,thetimeconstraintintherealworldoperationexcludethispossibility.Forexample,intheUPShublocatedattheOrlandoInternationalAirport,thegroundstaffsusuallystartloadingwhenlessthantenoutoffortycontainersarrivesuchthattheairplanecantakeoffatthescheduleddeadline.Togiveasenseofrelevanceofthetimevalue,wecitehereinternaldatafromtheUPS:thedelaycostofaightrangesfrom$500to$1000perminute,dependingonlocation(e.g.,FloridaorNewYork).Itshouldbenotedthatthereexistsapotentialcorrelationbetweenthebalanceofthecargoandthedelay.TheFederalAviationAdministration(FAA)imposesarequirementthattheCGofanaircraftmustfallinarangeforthesakeofsafety.Iftheimbalanceofthecargocontainersexceedsacertainlevel,containersshouldbeunloadedfromtheaircraftandreloaded.Thisprocessistimeconsumingandsignicantlydelaysights. 97

PAGE 98

4.3ProblemDescriptionThecontainerloadingprocessisasfollows:Packagesenterafacilitywheretheyaresortedaccordingtonaldestinationandloadedintocontainersaccordingtonaldestinations.Theloadedcontainersarethenmovedtoaweighingstation.Onceweighed,theymovetoastagingareawheretheycanbeloadedontothecraft.Thedecisionsassociatedwithloadingacraftareinherentlydependentonthecraftitself.ConsiderthehulldrawninFigure 4-1 withthreedifferentdoorlabels. Figure4-1. Threedoorcongurationsforacrafthull. Thereareintotal7slotsinthisaircraft.IftheaircraftonlyhasacontainerbaydoorinpositionA,thenthecontainersmustbeloadedfromtherear,inslot7through1,inorder.IfthedoorislocatedatpositionB,theneitherslot1or7maybeloadedrst.Onceslot1islled,thehullislledfromtherear.Althoughitmaybepossible,forthiscongurationthatacontainerplacedinslot1maybesubsequentlymovedtoaslotintherear,wedonotallowthismoveasweassumethatonceacontainerisplacedinaslot,itremainsthereasitislockeddown.Finally,Cgivesthegreatestnumberofchoicesinthattheaircraftmybeloadedfromthefrontorbacksimultaneouslywithslot4beinglledlast.Notethatthepositionlocatedatthedoorisalwaysloadedlast. 98

PAGE 99

Largeraircraftwhichcanttwocontainerssidebysideinthehullleadtomoredecisionpossibilities.However,theassumptionthatonceacontainerisinposition,itdoesnotmove,holdsasacontainmentbarprohibitsmovementofthecontainersfromonesideoftheaircrafttotheother.Givenahullconguration,therearetwoconcernswithloading:(1)balanceduringintermediateloadingsuchthattippingorlollingdoesnotoccurand(2)nalbalance.Foragivencraft,afulcrumpointisdesignatedastheaxisforbalance.InFigure 4-1 ,positionCisalikelyfulcrum.Thus,ifAisthelocationoftheloadingdoor,caremustbetakensuchthattheloadsofcontainersplacedinslots7,6and5(astheymustbeloadedrst)donottiptheplane.Foraircraftapplications,oncetheseslotsaresuccessfullylledandtheplanehasnottipped,thecraftcannottipforfurtherloadingasthefrontwheelsprovideadditionalsupport.However,thismaynotbethecaseinallcraft,suchasships,whichmayrequirebalanceineitherdirection(suchthatloadingalllightcontainersinthebackandheavyinthefrontmaynotbefeasible).Thenalbalance,forFigure 4-1 isdenedasthedifferencebetweenthecontainerweightsinslots5,6and7versusthosein1,2and3.Thus,foragivenproblem,thehullcongurationandfulcrumdenethetippingconstraintswhilethehullcongurationanddoorlocationdenestheloadingprocedure.Furthermore,thedoorlocationandthearrivalprocessdenethefeasibleloadingdecisionsovertime.Inpractice,therequirementonthedeviationoftheCGofthecargocontainerscouldbeveryrestrictive,typicallylessthan20inches.Moreover,therearealsoothertechnicalconstraintstobeconsidered,whichmaketheproblemevenmoredifcult.Inthesesituations,theremaybeadditionaltippingconsiderations(fronttobackandsidetoside).Thedecisionmaker(thegroundstaff)needstomaintainaqueueofcontainersdynamically,consideringthebalance,thetechnicalconstraintsandthetake-offdeadlinesimultaneously.Inthisresearch,wemainlyfocusonthebalancealongthelongitudinal 99

PAGE 100

axisoftheaircraftandthesituationwhenthecargodoorlocatesatonesideoftheaircraft(AorB). 4.4StochasticDynamicProgrammingNowweturnourattentiontotheproblemwherecontainersarriveovertimeandmustbeloadedcompletelybeforeadeadline,T.Itshouldbenotedthattwouncertaintiesarisewhendynamicsareintroducedintothebalancedloadingproblem:(1)arrivaltimesand(2)containerweights.Inthissection,weintroduceastochasticdynamicprogramming(SDP)formulation,sinceitnaturallytssequentialdecisionproblems.Though,aswillbeshownlater,itturnsouttobeverydifculttosolvetheproblemthroughthisformulation,evenwhenapproximateschemesareemployed,importantinsightscanbederivedforthedevelopmentofmoreefcientalgorithms. 4.4.1Formulation 4.4.1.1State,decisionepochs,andnotationForthegivencraft,Ncontainersarriveaccordingtoastochasticprocess,withthearrivingtimepotentiallyoccurringatTtimeperiods.IfucontainershavenotyetarrivedattimeT)]TJ /F3 11.955 Tf 12.03 0 Td[(u,exactlyonecontainerarriveseachperiodfortheremaininguperiodswithprobabilityone.Ineveryperiodt
PAGE 101

directioninthecraft.Forfeasibility,theweightfromoneendcannotexceedalimitatanypointintime,suchthatjbtjB,atanytimetandBisgiven.TheobjectiveistoassigncontainerstoslotssuchthatbTisasclosetozeroaspossible.TheremaybeadditionalobjectivesinthatadecisiontoloadbeforethedeadlineTisbenecialtoensureon-timedeparturesorevenenableearlydepartures.Similarly,acostmaybeassignedtolatedepartures.Wediscretizetimeaccordingtotheamountoftimeittakestoloadacontainerontothecraft.Itisassumedthatatmostonecontainercanbeloadedatatimeandwefurtherassumethatatmostonecontainercanarriveineachperiod.Thisfollowsfromtheloadingprocesswherecontainersmustgenerallyqueueforweighingbeforebeingreleasedtothecraftforloading.Thenotationissummarizedas bt=scalarbalanceofhullattimet;rt=positiverewardforcompletingtheloadingprocessattimet;At=setofemptyslotsattimet;St=setoffeasibleslotsassignmentsforacontainerattimet;Qt=setofcontainerswhicharequeuedandmaybeloadedattimet;Xt=setofcontainerswhichhavenotarrivedbytimet;Lt=setofcontainerswhichhavebeenloadedbytimet;B=maximumimbalanceatanytime;T=periodatwhichthecraftmustdepart;N=totalnumberofcontainerstobeloadedandthenumberofslotsonthecraft;W=numberofweighttypes;w=weightoftype;p=probabilityoftype;pt=probabilityofacontainerarrivingattimet;Usingthisinformation,thedynamicprogrammingrecursionfollows. 101

PAGE 102

4.4.1.2DynamicprogrammingequationDenethefunctionalequationas Vt(b,Q,U,S)=minimumexpectedcostwhencrafthasbalanceb,containersinthesetQareeligibletobeloadedintoslotsinthesetS,containersinthesetUhavenotyetarrivedattimet,andchoosingoptimalloadingdecisionsthroughtimeT.Ateachdecisionepoch,sayt,thedecisionmakereitherloadsacontainerinthequeue,Q,ifjQj>0,ordoesnothingandpostponestothenextdecisionepoch,t+1,atwhichtimethedecisioncanberevisited.Ifonecontainerisloadedtoaslot,bothjQjandjSjdecreasebyone.Ifthedecisionmakerchoosestopostpone,therearetwopossiblesituations,dependingwhetheracontainerarrivesatthebeginningofthenextdecisionepoch.Inthecaseofanewarrival,udecreasesbyoneandthedecisionmakerhasmoreoptionsinloading.Itshouldbenotedthatnotallemptyslotsarefeasibleforacontainertobepositioned.Forexample,considerthehullcongurationofB757-200FreightershowninFigure 4-2 .Thereisonlyonerowofslots,numberedfrom1to15.Itshouldbeclearthatslot3(theoneatthemaincargodoor)shouldbeoccupiedlast;otherwise,slots4through15cannotbereached.Inotherwords,ateachdecisionepoch,thesetoffeasibleslotsforloading,S,maynotbethesameasthesetofemptyslots,A.Withthedenitionofstate,decisionepoch,andstatetransition,thedynamicprogrammingequationfort
PAGE 103

Figure4-2. HullcongurationofB757-200Freighter. t=T)-240(jUj)-241(jQj,postponingresultsinadelayedight,thustherstexpressioninthecurlybracketsshouldbeexcludedinEq. 4 .Ifthedeadlineisahardconstraint,thepostponementoptionisinfeasible;ifnot,apenaltyshouldbeintroducedtodiscouragedelay.Moreover,ifTeisthelatesttimebywhichthecraftmusttakeoff,whereT
PAGE 104

4.4.2PropertiesTechnologiestosolvesequentialdecisionproblemsarerichintherealmsofMarkovDecisionProcesses(MDP)andApproximateDynamicProgramming(ADP).Ingeneral,optimalpolicieswithsimplestructurescanonlybefoundinalimitednumberofproblems.Forhardproblems,itisdifcult(e.g.,PSPACE-hard)tocomputeoptimalpoliciesandsuboptimalpoliciesorheuristicsarestudiedinstead.Inthatcase,structuralpropertiesmaybeutilizedtodevelopefcientalgorithms.However,itisdifculttondusefulstructuralpropertiesfortheDSLP.Theonlythingthatisfoundisthetrade-offrelationshipbetweenthedelayandtheimbalance.Aswillbepointedoutlater,thequadraticnatureoftheobjectivefunctiongreatlycomplicatestheanalysis. 4.4.2.1Combinatorialnatureandstructuralproperties SmithandMcardle ( 2002 )presentaframeworktostudythestructuralpropertiesofthecost-to-gofunctionofanite-horizonMDP.Ingeneral,ifanitehorizonMDPsatisesthefollowingthreeconditions: (a) thecost-to-goofthelaststageandtherewardfunctionsatisfyacertainpropertyP, (b) thetransitionprobabilitiessatisfyastochasticversionofP, (c) theoptimizationoperatorpreservesP,thecost-to-gofunctionsofallearlierstagessatisfythisproperty.MDPswithoptimalthresholdpolicesthatsatisfythisframeworkarerichintheliterature(e.g., Scarf ( 1960 )).Theanalysisbecomesmorecomplicatedwhenaproblemhasmultidimensionalstate.Thestochasticallocationproblemstudiedby PapadakiandPowell ( 2007 )isoneofthefewworksutilizingstructuralpropertiesofmultidimensionalMDPtodevelopefcientalgorithms.Giventheexistingliteratureonthemulti-dimensioncase,theexistenceofpartialorderiscriticalindevelopingstructuraloptimalpolicies.However,asindicatedEq. 4 ,thedecisionofthenalepochisaquadraticassignmentproblem,whichiswellknowntobeNP-hard.ItisdifculttostudytherelationshipbetweenQandSandthevalue 104

PAGE 105

function.Andthus,itisnoteasytondapartialorderordominancerelationshipamongstates. 4.4.2.2ImpactofcostsDenescalarfunctionscdt(b,Q,U,S)andcbt(b,Q,U,S)asthecostofdelayandthecostofimbalance.Clearly,Vt(b,Q,U,S)=cdt(b,Q,U,S)+cbt(b,Q,U,S). Lemma4.1. Forapolicyandastate(b,Q,U,S,t),Vt(b,Q,U,S)isnon-decreasinginCimbandCd.Specically,withincreasingCimb,cd(b,Q,U,S,t)increasesandcb(b,Q,U,S,t)strictlyincreases.Also,withincreasingCd,cbt(b,Q,U,S)increasesandcdt(b,Q,U,S)strictlyincreases. Proof. SeeAppendix C.1 Infact,whenmorepenaltyisimposedonimbalance(increasingCimbalance),theoptimalpolicymayleadtoanincreasedcdt,sinceonemayusethepostponementoptionmoreinordertomakebetterloadingdecisions,andviceversa.Indeed,experimentresultsofsmallsizedproblemindicatesthatwhenoneofthemincreases,cbtandcdtbothincrease. Lemma4.2. Thewaitingoptionreducestheimbalanceatthecostofincreasingthedelay.Theimmediateloadingoptionaffectsoppositely. Proof. SeeAppendix C.2 4.4.3ComplexityandLimitationoftheDPFormulationTheabovemodelallowsforanon-stationaryarrivalprocessbyassumingpttobedependentont.Ascanbeexpected,whentimeapproachesthedeadline,thegroundstaffmayallocatemoresortingcapabilitytospeedloading.Moreover,itensuresanitehorizon.Second,thismodelfullyincorporatesthewaitoption.Thegroundstaffcanchoosetowaitconsecutivelyoverepochsandaqueuedcontainermaybeloadedifnocontainerarriveafteronewaitsforseveralepochs. 105

PAGE 106

However,thisexibilityisachievedatthecostofagrowingnumberofstates.Toseethis,wepresentthedecisiontreeinFigure 4-3 ,whichcorrespondstothehullcongurationofFigure 4-1 .IgnoringthesizeofS,thenumberofstatesisintheorderof(MWN),whereMisthenumberofpossiblevaluesofb,whichcouldbeaverylargenumber.ForB757-200,whereN=15,thisnumberislargeevenforW=5.ForB747-400,whereN=30,thenumberofstatescouldbeintractablylarge. Figure4-3. DecisionTreeoftheGeneralDPModel. ADPiswidelyregardedasaneffectivetooltosolvelargescaleproblems.Inourcase,severalapproximationschemescanbeapplied.First,notethatbcanbeeasilyboundedandthuswecandiscretizethevalueofb.Moreover,notethatalongqueueresultsinaveryhighdelaycostandstateswithalongqueueinthedecisiontreemaybeunnecessary.Wethusimposeanupperboundonthequeuelength,qmax;whenjQj=qmax,onecontainermustbeloadedfromQ.ForB757-200,inwhichcontainersareformedinonerow,theonlyfeasibleslottobelledateachdecisionepochisthe 106

PAGE 107

innermostavailableslot.Inthiscase,Scanberemovedfromthestate.Then,thenumberofstatesisintheorderofO(Mqmax).Asanillustration,supposethatT=30,=5,andM=1000,thenumberofstatesisaslargeas3055515,whichismorethan93millionstates.Aproblemofthissize,ifsolvedthroughADP,usuallydepletes2GBcomputermemoryandusuallytakesseveralminutestogenerateadecision.This,however,isnotacceptableinpractice,whichonlyallows10to20seconds.Infact,thedifcultyofndingadecisionpolicythatisdependentontherevealedinformation(weightsofarrivingcontainersinourproblem)forastochasticcombinatorialproblemhasbeenwellidentied.Intheliteratureofstochasticknapsackproblem(e.g., Deanetal. ( 2008 )),thepolicyisnamedasadaptivepolicy.ItisshownthattondanoptimaladaptivepolicyforstochasticknapsackproblemsisPSPACE-hard.SincetheDSLP,afterrelaxation,canbereducedtoatwo-dimensionalstochasticknapsackproblem,theDSLPisindeedPSPACE-hard.ThisprecludesthepossibilityofndingapolynomiallylargedecisiontreefortheDSLP.AnotherissueassociatedwithADPisthatitrequiresdeningstates,whichinturnsnecessitatesthediscretizationoftheimbalancecausedbyloadedboxesandweightvaluesofqueuedcontainers.This,however,affectsthequalityofthesolutioninourcase.NotethatinstandardMDPmodels,thetotalrewardcanberewrittenasPtRt,whereRtistherewardearnedateachdecisionepoch.Errorscausedbydiscretizationmaycanceloutsincetheobjectivefunctionislinear.Inourproblem,however,theobjectiveisquadraticinnature.Theapproximationerrorsmaynotnecessarilycancelout.Forexample,considerastatewithjUj=1.AscanbeinferredfromEq. 4 ,theexpectedvalueofthisstateisthelinearcombinationofanumberofabsolutevalues.Inthiscase,theapproximationerrorswithineachoftheseabsolutetermscannotbecanceledout.Indeed,wetestedtheADPapproachontheB757-200case,withotherparameterssetasabove.TheADPgeneratesgoodsolutionsifthecontainers'weightstrulyfollow 107

PAGE 108

discretedistributions.However,whenitcomestocontinuousdistributions,theADPapproachingeneralworksworsethanheuristics,besidesitslowspeedindecision.Theonlysolutiontothisissueistorenethediscretization,whichresultsinintractablylargenumberofstates. 4.5OtherApproachesAsanalyzedabove,theADPapproachisnotsuitablefortheDSLPduetoitson-linenature.Inthissection,webrieyreviewthreemethodologiesappliedtosequentialdecisionproblems:(1)multi-stagestochasticprogramming(MSP),(2)on-linestochasticandcombinatorialalgorithm(OSCA),and(3)simpleheuristics.Thedifcultiesofapplyingtheseapproachesarepointedout.Finally,asimpliedproblemispresentedtohelpinterpretthecomplexity. 4.5.1Multi-stageStochasticProgrammingandRobustOptimizationInthissection,wepresenttheMSPformulationoftheDSLP.ThoughtheMSPformulationsuffersasimilarcomputationalbarrier,duetoitssimilaritytotheADPformulation,itbetterillustratesthecomplexityoftheDSLP.Later,wewilldesignaneffectiveheuristicalgorithmbasedonthisanalysis.Wedenotetijastheassignmentdecisionvariableforstagetanddenotetasthedecisionofwhethertoloadacontaineratstaget.SinceafterT,thereisnocontainersyettoarriveandallqueuedcontainersshouldhavebeenloaded,weassumethatallcontainersareloadedafterstageT+1.Also,tijandtarebinaryvariables,exceptthatT+1isthenumberofqueuedcontainers.Tomodelthearrivaluncertainty,weintroducearandomvectorIt=(I1t,...,INt),whereIit=1ifcontainerihasarrivedatstagetand 108

PAGE 109

Iit=0otherwise.Withthesenotation,wedenetheMSPformulationoftheDSLPasmin:CimbEjT+1Xt=1tXi=1Xj2AtWidjtijj+Cd(T+1)]TJ /F18 10.909 Tf 10.91 0 Td[(1) (4)Xj2Sttijt,i=1,...,N,t=1,...,T, (4)Xj=2Sttij=0,i=1,...,N,t=1,...,T, (4)NXi=1tijt,j2St,t=1,...,T, (4)tX=1Xj2StijIit,i=1,2,...,T,t=1,...,T+1, (4)Xj2ST+1T+1ij=1)]TJ /F6 7.97 Tf 15.19 13.63 Td[(TX=1Xj2Stij, (4)NXi=1T+1ij=1,8j2ST+1 (4)tij2f0,1g,t2f0,1g,i=1,...,N,j=1,...,N,t=1,...,T+1. (4)Inthisformulation,constraintsEq. 4 andEq. 4 statethatatstaget,containerscanonlybeloadedtofeasibleslots.ConstraintEq. 4 statesthatatmosttcontainersshouldbeloaded.ConstraintEq. 4 statesthatnocontainershouldbeloadedbeforeitsarrivalandcanbeassignedforatmostonce.Atthenalstage,theDSLPreducestoastaticbalancedloadingproblem,thefeasibleregionofwhichisdescribedbyconstraintsEq. 4 andEq. 4 .Sincewehaven'texplicitlydenedStandthusthisformulationisincomplete.Notealsothatduetothedifcultyinadjustingcontainers'positionswithinaircrafts,Stdependsonpreviousdecisions.ForanaircraftwithslotscongurationassimpleasB757-200,Stiscomposedoftheinnermostavailableslot.Foranaircraftwithtworowsofslots,sayB767-300,ittakesmorecautiousnesstodecideSt,sincepreviouslyloadedcontainermayblocklatercontainers. 109

PAGE 110

However,evenforthissimplecase,itisnoteasytosolvetheproblemthroughstandardMSPalgorithms,likethenestedBender'sdecomposition,giventhescarcecomputationaltimebetweenepochs.Mainly,accordingto Shapiro ( 2006 ),thecomplexityofsolvingamulti-stagestochasticprogrammingproblemisexponentialinthenumberofdecisionepochs.TheDSLPisusuallycomposedoftensofdecisionepochs.Therefore,standardMSPalgorithmsarenotwellsuitedfortheDSLP.Onewaytoreducethecomputationalcomplexityistochangetheobjectiveasminimizingtheimbalanceinthebadscenarios.Studiesinthisareausuallybearthenamerobustoptimization(e.g., Ben-TalandNemirovski ( 1998 )).Themainargumentssupportingrobustoptimizationinclude:1)inpractice,dataisincompleteoritisdifculttoobtainaprobabilitydistributionondata;2)itisusuallycomputationallyintractabletondanoptimalsolutionwithinareasonabletimeframe.Othermeasurescanbefoundin ChenandSim ( 2009 ).Byappropriatelydesigningacoherentmeasureoranapproximationondatauncertainty,onecangreatlysimplifythecomputation,whileobtainingasolutionthatcanavoidunacceptableoutcomes(likeveryhighcostsorinfeasiblesolutions).Themainargumentagainsttheworstcasemeasureisthatthesolutionisusuallytooconservative.Toxthisproblem,scholarsintroducewaysthatcansystematicallyreducetheconservativeness(e.g., Ben-Taletal. ( 2004 )).Recentstudiespresentedregretasameasure,whichtosomeextentbalancesthesolutionqualityandcomputationalcomplexity.However,theseresearchresultsdonotdirectlybenetourresearch.Usually,agoodpolicygivenonescenarioispoorinanotherscenario.Forexample,supposethattherearenarrivingcontainers,withtherstn)]TJ /F4 11.955 Tf 11.69 0 Td[(1ofthemverylightandthelastoneveryheavy.Inthiscase,agoodpolicyistoloadtherstseveralcontainersupontheirarrivalssuchthatthereismoretimeforthepostponementoptionandthelastheavycontainercanbeloadedtooneofthemiddleslots.However,thispolicyisundesirableinthecase 110

PAGE 111

whentherstcontainerisveryheavyandtherestcontainersareverylight.Inotherwords,itisnoteasytodenethebadscenarios.Theregretanalysisalsodoesnothelpinreducingthecomputationalcomplexity,sincetheimbalanceitselfisameasureofregret. 4.5.2Online-optimizationThedynamicbalancedloadingproblemalsofallsintheareaofon-lineoptimization.Asdenedin Grtscheletal. ( 2001 ),onlinecombinatorialoptimizationproblemsarecharacterizedbymakingdecisionsbeforecompleteinformationaboutthedataisavailable.Moreover,theonlinealgorithmmayindeedberequiredtodeliverthenextpieceofthesolutionwithinaverytighttimebound.Inthiscase,theseproblemsarealsocalledreal-timeproblems.Inotherliterature,however,researcherssimplyuseonlinetomeanbothonlineandrealtime.In HentenryckandBent ( 2006 ),itisalsonamedasonlinestochasticandcombinatorialproblem.Aspointedoutby Grtscheletal. ( 2001 ),onlinecontinuousoptimizationproblems,likecontrolproblems,havebeenwellstudied,whiletheresearchonitscombinatorialcounterpartisneglectedtoalargeextent.Mainly,thehistoryofresearchinthiseldonlytracesbackto25yearsago.Unfortunately,theresearchresultintheonlinecontinuousoptimizationproblemscannotbeappliedinthecombinatorialcase,justastechniquessolvinglinear/non-linearprogrammingproblemscannotbedirectlyusedtosolvediscreteoptimizationproblems. Karlinetal. ( 1988 )suggestcomparinganonlinealgorithmtoanoptimalofinealgorithm,whichlaydownthefoundationofcompetitiveanalysis.Athoroughdiscussionofonlinecomputationcanbefoundin BorodinandEl-Yaniv ( 1998 ).Thisapproachtriestoanalyzetheperformanceboundofheuristics.However,thecompetitiveratio,ameasureofalgorithmperformance,maybemisleading.Forexample,competitiveanalysisisatypeofworst-caseanalysisandmaybeoverlypessimistic.Moreover,thecompetitiveratiomaybetoocoarseacriteriontovaluealgorithms.Also,algorithms 111

PAGE 112

foracomplicatedproblemliketheDSLParedifcultforderivingperformancebounds.Therefore,thecompetitiveanalysisarenotwellsuitedfortheDSLP.Anoteworthydiscoveryintherealmofonlinecombinatorialoptimizationafterthecompetitiveanalysisistheonlineanticipatoryalgorithms(OAA),presentedby HentenryckandBent ( 2006 ).Ingeneral,thisapproachrelaxestheanticipatoryconstraint.Withthisrelaxation,themulti-stageproblemreducestoasingle-stageproblem.Toevaluateanaction(option),onecansimplysimulatefutureunknowninformation,solveadeterministicoptimizationovereachrealizedpath,andsummarizetheoptimalvaluesoverallpaths.TheauthorsprovethatarelativelytighterrorboundcanbedevelopedforOAA.Moreover,theysolvedseveralclassesofonlinecombinatorialoptimizationproblems,inwhichthisconstraintcanbe(approximately)relaxed.Notethatrelaxingtheanticipatoryconstraintisaverystrongassumption.Aspointedoutin HentenryckandBent ( 2006 ),OAAbettersuitsforsituationsinwhichthenon-anticipatoryconstraintscantoalargeextentberelaxed.Obviously,thisassumptionisinappropriatefortheDSLP.Innumericalexperiments,wendthatOAAgenerateslowqualitysolutions. 4.5.3SimpleHeuristicsTothebestofourknowledge,theDSLPisnotwellsuitedfortheoptimizationframeworkpresentedinliterature,especiallywhenthetightrequirementoncomputationtimeisimposed.Itisdifculttoestimatethevalueofadecision,fortheamountofimbalanceisknownonlywhenallcontainersareloaded.ForacomplicatedandpracticalproblemastheDSLP,researchersusuallyapplyheuristicsandtesttheirperformancesthroughsimulationexperiments.Forexample,onemaysuggestvariantsoftheheuristicsdevelopedby Amiounyetal. ( 1992 ).SincethestaticbalancedloadingproblemisNP-hard,theauthorstendtooptimizeanalternativeobjectivetoseekaninvertedVshapeintheweight 112

PAGE 113

distributionofthecargocontainersoverslots.Inotherwords,bypositioninglightcontainersattheendsofanaircraft,whileleavingheavyonestothemiddle,onemayachieveagoodbalance.Asimilarapproximationistominimizethemomentofinertiainsteadofthetorque( LimbourgandLaporte 2011 ).Theresultingobjectivefunctionislinearandefcientalgorithmsareavailable.Whenthebalancingproblemisintroducedintoadynamicenvironment,however,itisverydifculttoapplythesetechniques,sinceintheDSLP,theweightinformationisincompletewhencontainersareloaded.Inconclusion,itisdifculttondeffectiveheuristicsfortheDSLPbearingsimpledecisionrules. 4.6SolutionApproachFromthediscussioninprevioussections,itcanbeseenthatthedifcultiesoftheDSLPstemfromtwofeaturesoftheproblem.First,theimbalanceisnotknownuntilallcontainersareloaded.Thus,toevaluatetheperformanceoftheactionofloadingacertainqueuedcontainer,oneneedstoconsideralargenumberofweightrealizationsofthecontainersthathavenotarrivedyetandsolvetheproblemovereachstochasticpath.ThisfeatureiscausedbytheobjectivefunctionoftheDSLP,whichisquadraticinnature.Second,thedecisionofwhethertowaitforthenextarrivalindeeddenestheactionsetorthefeasibleregionfordecisionsoffutureepochs.Theattempttobestcontrolthesizeofthepostponementqueuesignicantlyincreasesthecomplexityoftheproblem.DuetothecomplexityoftheDSLP,itisdifculttoderivestructuralpropertiestofacilitatedesigningalgorithms.Thus,anapproximationalgorithmisdesignedtohandletheproblem.Tohandletherstcomplexity,weproposeanalternativeobjective,throughwhichtheDSLPcanbeapproximatedasatwo-stagestochasticmixedintegerprogrammingproblem.Theresultingproblemismucheasiertooptimize.Tohandlethesecondcomplexity,weintroduceaqueue-sizebasedruletodecidewhetheroneshouldpostpone. 113

PAGE 114

Inthissection,weforthemomentignorethetechnicalconstraints.Wewillillustratehowtoincludetheseconstraintsinlatersections. 4.6.1ApproximationwithTwo-stageMixedStochasticIntegerProgramming(TSMIP)ThedifferencebetweentheDSLPanditsstaticversionliesinthatintheDSLP,loadedcontainersareconnedtotheirslotsandinprinciplecannotbemovedduetothelimitedspacewithinanaircraft.Withthisconstraint,queuedcontainerscannotbeloadedtoanarbitraryslot,sincetheymayblockthepathofcontainersarrivinglater.Toavoidthesecomplexities,wesimplifytheproblemthroughthefollowingtwosteps.First,wedesignanalternativeobjectivefunction,byrestrictingcontainersyettoarrivetocertainslots.Thisideacomesfromtwoobservations.First,whencontainersyettoarrivearereservedtocertainslotsandareloadedupontheirarrivals,thetorquegeneratedbythemcanbeeasilycomputed.Second,accordingtheSSAPdescribedinChapter2,toholdthebestresourcesinhandmaynotdeviatemuchfromtheoptimalpolicy(theheterogeneousSSAP).Intuitively,someslotspossessahigherbalancingpower(e.g.,slotswithlargerdisplacementvalues)thanotherslots(e.g.,theslotsinthemiddle).Therefore,wepartitionthequeuedcontainersintotwosets:therstsetistobeloadedbeforeallfuturecontainersarriveandthesecondsetiskeptinthequeueuntilallfuturecontainersarrive(i.e.,thenalstage).Formally,deneStdasthesetofslotsreservedtothesecondsetofcontainers.NotethatStdshouldbeselectedinsuchawaythattheloadingofthesecondsetofcontainersatthenalstagewillnotbeblockedbythecontainersloadedpreviously(e.g.,theoutermostslots).DeneStnxastheslotsclosesttoloadedcontainersatstaget(e.g,theinnermostcontainers).Withthispartition,wecanapproximatetheproblemasatwo-stagestochasticmixedintegerprogram.Intherststage,weplanslotreservationsfortherstsetofthequeuedcontainersandthecontainersyettoarrive;inthesecondstage,therstsetofqueuedcontainersandthecontainersyettoarrivehavebeenloaded,andweassignthe 114

PAGE 115

secondsetofqueuedcontainerstotheremainingslots,withtheobjectiveofminimizingtheimbalance.Itshouldbeclearthatthisloadingplanissurelysuboptimal,sinceitignorestheexibilityinarrangingtheloadingoffuturecontainers.However,wecan,toalargeextent,compensatethelossofexibilitybyonlyloadingonecontaineraccordingtothisplan.Inotherwords,wesolvetheDSLPbysolvingasequenceoftwo-stagestochasticmixedintegerprograms.Second,weintroducearuletodecidewhenacontainerhastobeloaded.Specically,deneQ(u)asthemaximumnumberofcontainersthatcanbekeptinthequeue,whereuisthenumberofcontainersyettoarrive.IfthecurrentsizeofthepostponementqueueexceedsQ(u),wemustloadacontainer.ItshouldbeclearthatQ(u)=ureectsthechanceofdelayingtheight.WithahighvalueofQ(u)=u,itishighlylikelythatwhenthelastcontainerarrives,therearestillquiteafewcontainerswaitingtobeloaded.Indeed,policiesoptimallydecidingwhethertomakeanimmediatedecisionortoposponehavebeenstudiedintheSSAPcontext(Chapter2).ThoughtheSSAPwithpostponementoptionsissignicantlysimplerthantheDSLP,itisverydifcult,ingeneral,tondanoptimalruleforpostponement.Moreover,numericalexperimentsshowthatthepotentialimprovementinoptimalitycausedbythepostponementoptionisnotquiteappealing(lessthan10%),besidesthecomputationaldifcultytooptimizeit.Thus,webelievethatitwillnotaffectthesolutionqualitymuchbydesigningasimpleruledescribedasabove,theperformanceofwhichcanbeenhancedthroughtrial-and-errors.Dene1ijand2ijastheassignmentdecisionvariableoftherststage.Denebtastheexistingtorquecausedbyloadingdecisionsbeforestaget.Denebf(!)asthetorquegeneratedbythefuturecontainers.Here,!representsarealizationoftheweightsofcontainersyettoarrive.Theformulationofthetwo-stagestochasticmixed 115

PAGE 116

integerprogramisStage1:min: (4)s.t.:Xj2Stnx1ij1,i2Qt, (4)Xi2Qt1ij=1,j2Stnx, (4)1ij2f0,1g,i2Qt,j2Stnx,E[f(1,!)]. (4)Stage2:f(1,!)=min: (4)s.t.:Xj2Std2ij=1)]TJ /F14 11.955 Tf 13.7 11.36 Td[(Xj2Snx1ij,i2Qt, (4)Xi2Qt2ij=1,j2Std, (4)bt+bf(!)+Xi2QtXj2Stnxwidj1ij+Xi2QtXj2Stdwidj2ij, (4))]TJ /F3 11.955 Tf 21.92 0 Td[(bt)]TJ /F3 11.955 Tf 11.96 0 Td[(bf(!))]TJ /F14 11.955 Tf 12.48 11.36 Td[(Xi2QtXj2Stnxwidj1ij)]TJ /F14 11.955 Tf 12.47 11.36 Td[(Xi2QtXj2Stdwidj2ij, (4)2ij2f0,1g,i2Qt,j2Std,0.Inthisformulation,Eq. 4 andEq. 4 denetheassignmentpolytopeforstage1;allslotsinStnxshouldbelledwithcontainers.Also,Eq. 4 istheoptimalitycut.Stage2computesthetorquegiventheloadingdecisionsintherststage,theloadingoffuturearrivingcontainers,andtheloadingdecisionsinthesecondstage.Instage2,Eq. 4 andEq. 4 statethatallremainingqueuedcontainersshouldbeloadedtoslotsinStd.Finally,Eq. 4 ,Eq. 4 andlinearizetheobjective,whichisquadraticinnature.Uponthelastarrival(say,atstaget),therearejStdjcontainerswaitingtobeloaded.In 116

PAGE 117

thiscase,jStdj)]TJ /F4 11.955 Tf 18.76 0 Td[(1containersaredelayed.Theremainingproblemisastaticbalancedloadingproblem.Aftertheloadingplaniscreatedbysolvingtheaboveproblem,onlyonecontainerisindeedloadedaccordingtoit;inthenextdecisionstage,theabovedecisionwillberevisited.WenamethisalgorithmasTSMIP.ThebasicintuitionbehindtheTSMIPistondanobjectivefunctionfallingbetweentheMSPapproachandtheapproachonlyconsideringtheworstcasescenarios.Intermsofcomputationalcomplexity,thelatterissurelyeasierthantherstone,whilethesolutionqualitymaynotbedesirable.TheTSMIPtendstooptimizeacomputationallyaffordableobjectivefunction,withsatisfactorysolutions. 4.6.2NumericalExperimentsonIdealConditionsInthissection,weuseBoeing757-200Freighterasanexample.WecomparetheperformanceofTSMIPwithsomeheuristicswithsimplerules. 4.6.2.1ExperimentsettingandbenchmarkloadingrulesTheslotcongurationofBoeing757-200FreighterisgiveninFigure 4-2 .Inthiscase,N=15.Ateachstage,wecandesignStnxastheinnermostslots.Also,ifatstaget,u=0andQ(0)=3,wecanletStd=f1,2,3g,sincethedoorisnearslot3.WeassumethatWN(2100lb,500lb).Figure 4-4 showsthedimensionsoftheUPSA1Ncontainer.Moreover,assumethatonlyonecontainercanbeloadedperbatch,taking Figure4-4. A1NContainer 117

PAGE 118

50seconds,andtheaverageinter-arrivaltimeis100seconds.Withthisinformation,wedesignQasshowninTable 4-1 Table4-1. DesignofQ. uQ(u) uQ(u) 04 8614 9624 10734 11744 12855 13865 14875 158 Withthisdesign,inmostcases,itishighlylikelythatwhenthelastcontainerarrives,thereareintotal4containerswaitingtobeloadedandthedelaytimeis2.5minutes.Withthissetting,westudytheperformanceofthealgorithmover1,000runs.Tomakeacomparison,wedesignseveralheuristicsasbenchmarks. UBLBAssumethatallfuturecontainersareeitherveryheavyorverylight.Eachqueuedcontainerwillbetriedontheinnermostavailableslotunderthesetwoscenarios.Foreachscenario,wesolvetheresultingstaticproblem,andtheperformanceofloadingacontaineristheaverageimbalancedtorqueineachscenario. AVGWAssumethatallfuturecontainersweighattheaveragevalue.Thentheproblemissolvedasastaticbalancedloadingproblem,withtheconstraintthattheinnermostslotistakenbyqueuedcontainers. LUCKLoadacontainertotheinnermostavailableslot. 4.6.2.2NumericalresultsThedisplacementoftheCGofthecontainersisshowninFigure 4-5 .Amongthefourheuristics,UBLBperformsfarworsethanotheralgorithms.Inparticular,toconsidertheextremevaluesoffuturecontainersisevenworsethantoloadcontainersupontheirarrivals,indicatingtheinsufciencyofonlyconsideringthetwoscenariosdenedinUBLB.Ontheotherhands,AVGWisnearlythesameasLUCK. 118

PAGE 119

Figure4-5. DistributionofCargoCGDisplacementw.r.tSlotCenter TSMIPsignicantlyoutperformsotheralgorithms.WithQdenedasinTable 4-1 ,theaveragedisplacementoftheCGofcontainersyieldedbyTSMIPis2.49inch.Moreover,thelargestdisplacementoftheCGislessthan24inch,whileinotherheuristics,sayAVGW,thisvalueis76inch.Itshouldbenotedthatinpractice,anequivalentlyimportantconcernbesidestheaveragebalance,istheworstcaseimbalance.Whenaverypoorimbalanceoccurs,itislikelythattheresultingCGoftheairplane(afterloading)exceedsthesafetyCGlimits(forwardoraft).Inthissituation,containershavetobereloaded,whichsignicantlydisruptstheschedule.Moreover,TSMIPusuallyuses5to7secondstogeneratealoadingdecision,whichisacceptableinpractice.Additionally,bydeningthedeadlineas26min(1minmorethantheaveragetimeforallcontainerstoarrive),theaveragedelayis2.35min.Inmanyscenarios,thetimeforallcontainerstoarrivemaybelessthanthedeneddeadline,indicatingthepossibilityofreducingthegroundtimeandanearliertake-offtime.Finally,weillustratetheimpactofQonthedisplacementoftheCG.ConsiderthetwodesignsinTable 4-2 .Clearly,Q1(u)
PAGE 120

underthesethreerulesarepresentedinFigure 4-6 .ItcanbeseenthatthoughQ2(u)maynotyieldahighlevelofdelay,HeuristicTSMIPcanbalancetheaircraftverywell. Table4-2. DesignofQ. uQ1(u) uQ1(u) uQ2(u) uQ2(u) 03 85 05 8713 95 15 9723 106 25 10833 116 35 11843 127 45 12854 137 56 13964 147 66 14974 157 76 159 Figure4-6. ImpactofQ 4.7TechnicalConstraintsandApplications 4.7.1TheImpactofTechnicalConstraintsInthissection,weincludethetwoconstraintscommonlyseenintheairfreightindustryintoouralgorithm,namely,thetippingconstraintandthecombinedweightlimitconstraint( LimbourgandLaporte ( 2011 )).Thetippingconstraintensuresthestabilityof 120

PAGE 121

anaircraftwhencontainersarebeingloaded.ThemagnitudeofBvariesoverdifferenttypesofaircrafts.Here,weassumeB=40,333,333lb-ftfortheexperimentsettingdenedinSection 4.6.2 .Notethatthisvaluecorrespondstothesituationwhenhalfoftheslotsinthetailsideoftheaircraftarealllledwithcontainersweighing2,250lb.Then,weaddaconstrainttoStage1ofEq. 4 ,requestingthatthesummationoftheexistingtorqueandthetorqueyieldedbythenextloadedcontainershouldbelessthanB.Throughnumericalexperiments,wefoundthatthereareindeedcasesinwhichinfeasibilityoccursintherststageproblem.Notethatinthiscase,wecanchoosetowaitforfuturecontainers.Anotherwaytohandlethisproblemmoreeffectivelyistorestraintheweightvaluesassignedtotheslotsatthetailend.Notethatthisisindeedthecombinedweightlimitsconstraintmentionedin LimbourgandLaporte ( 2011 ).Duetostructuraldesignreasons,itisnotallowedtoputveryheavycontainerstotheslotsatthetailendandthenoseend.Formostaircrafttypes,theweightlimitofthetailslotsareusuallysignicantlymorerestrictivethanthatofthenoseslots.Inourexperiment,wetendtorstconsidertheweightlimitconstraintforthetailslots.WetestedtheimpactofthetippingconstraintundertheQ2denedinTable 4-2 .Itturnsoutthattheseconstraintsdonotplayasignicantrole,asshowninFigure 4-7 .Moreover,werandomlypicktheweightdisruptionsofsomesimulationruns,showninFigure 4-8 .ItcanbeseenthatTSMIPloadsrelativelylightercontainerstotailslots.Surprisingly,thisheuristictendstoleaveheavycontainerstothemiddleandcontainerweighingnearlythemeantothenoseslots.Apossibleexplanationisthatsincesomelightcontainersarerestrainedtothetailslots,onewaytoachievethebalanceistoputcontainersthatarenotheavytothenoseslots.Inotherwords,theremightexistacorrelationbetweenthecombinedweightlimitconstraintandthetippingconstraint. 121

PAGE 122

Figure4-7. ImpactofTippingConstraintonBalance Figure4-8. ImpactofTippingConstraintonWeightDistribution 4.7.2AnApplicationExampleInpractice,on-timetake-offishighlyprioritizedoverthecombinedlimitconstraintandthebalance;thegroundstaffinprincipledonotconsiderthebalanceifitdoesnotaffectthesafety.Accordingtothedata,asignicantportionoftheloadingplansstayrightatthesafetythresholds.Planswithgoodbalanceareusuallyachievedwithviolatedcombinedweightlimitconstraints,whichmaydamagethestructureoftheairplane.Inthissection,weutilizethedatafromtheOrlandoHuboftheUPStotestouralgorithm. 122

PAGE 123

Inthisexample,weconsidertheMD-11freighteremployedbytheUPSOrlandohub,withitsslotcongurationillustratedinFigure 4-9 .InMD-11freighter,therearetwodecksofcargoslots.Theupperdeckcontainstworowsofslots,labeledbyLandR,representingtheleftsideandtherightside,respectively.Thelowerdeckcontainsonlyonerowofslots,labeledwithP.Therearetwocargodoorsinthelowerdeck,forslotsP1-P6andP7-14,respectively.Andthereisonlyonedoorfortheupperdeckslots. Figure4-9. SlotCongurationofMD-11Freighter Inthisexample,weassumethattheCGoftheaircraftlocatesatthecenterofslot7Lor7R.WebelievethatthisassumptionisacceptablesincetheCGofanaircraftshouldbeclosetoitsmainengine.Takingthispointasthereference,thepositionoftheotherslotscanbecomputedbasedonthedimensionsofeachslot,asillustratedinFigure 4-9 .ThedetailedpositionsofslotsarelistedinTable 4-3 .Here,weignoreslotsP7-P14,sinceinthisexample,thecontainersloadedhereareallempty.Theaverageandstandarddeviationoftheweightvaluesare2467.06lband1308.93lb,respectively.Giventhedataweobtained,itisvalidtoassumethatthe 123

PAGE 124

Table4-3. SlotPositionsofMD-11Freighter. indexL(inch)R(inch)P(inch) 1750750665262562557735005004894375375401525025031361251252257008-125-1259-250-25010-375-37511-500-50012-625-62513-75014-838 weightvaluesarenormallydistributedasN(2400,800).Also,thearrivalprocessofthecontainersattheOrlandoHubisdifferentfromthearrivingprocessassumedinpreviouschapters.Thecontainersarriveinbatchesevery5to7minutes,with5containersineachbatch.Besidesthecombinedlimitconstraint,itisrequiredthattheweightassumedbyslots1L-5Landslots1Rto5Rshouldbegreaterthan8,000lb.Moreover,theweightassumedbyslotsP1-P6shouldbemorethan12,000lb.Alsotheliftingmachinecanloadtwocontainersatonetime,takingaround1minute.Here,weassumethattheintervalbetweeneachloadingis1.5minute,inordertocompensatethetimeconsumedindecisionmaking.Indeed,thealgorithmusuallytakes10secondstogenerateadecision.Inthisexample,weimposeaconstraintthatthequeuelengthshouldbebetween6to10,whichisverysimilartotherealcargoloadingprocessattheOrlandohub.WedeneStdasslotsP1-P6anduseourstochasticprogrammingbasedheuristictogeneratetheloadingplanshowninTable 4-4 .Withthisloadingplan,theimbalancedtorqueis978lbinchandthedisplacementoftheCGofthecargois0.012inch.Itcan 124

PAGE 125

beseenthatouralgorithmsmartlyloadlightcontainerstothenoseandtailslots,whilerespectingtheadditionaltechnicalconstraintsmentionedinthepreviousparagraph.Theloadingprocessendsat42.5min.Ifallcontainersaredirectlyloadedupontheir Table4-4. LoadingPlan. IDArrivingTimeWeightLoadingTimeSlot IDArrivingTimeWeightLoadingTimeSlot 15334013.512R 1720455729.510L25304220.06R 1820246432.55L35118521.54R 1920213223.02R45209018.58R 2020219431.07L55184512.014 2125334828.012L610412839.5P4 2225130735.52L710182212.013 2325280728.011L810269313.511R 242594035.51L910204220.06R 2525203931.08L1010118821.53R 2630238434.03L1115266617.010R 2730260339.5P31215229317.09R 2830279134.04L1315363029.59L 2930366737.0P51415204832.56L 3030266841.0P21515208728.011L 3135149041.0P11620150223.01R 3235202237.0P6 arrivals,theloadingprocessendsat36.5min.Theextra6minutestradedusabettercargobalanceandabetterweightdistribution.Itshouldbenotedthatweassumeatightinter-arrivaltimeofbatchesinthisexample;theactualdelayedtimecanbeevensmallerinpractice. 4.8ConclusionandFutureWorkInthischapter,westudiedtheDSLP.ThefeaturesofthisproblemrelateittobothmathematicalprogrammingandMarkovdecisionprocesses.However,thestandardoptimizationapproacheswellestablishedinthesetwocommunitiesdonotapplyinthiscase.Wethusdevelopatwo-stagestochasticmixedintegerprogrammingbasedapproachforthisproblem,withaframeworkcapableofincorporatingdynamicprogrammingandoptimalcontrolbasedheuristics.Numericalexperimentsillustratethesignicantpotentialofthisalgorithminreducingthegroundoperationtimeandthe 125

PAGE 126

fuelconsumptionforairfreightcompanies.Webelievethatthisworkalsohaspotentialapplicationsintheshippingindustry.Toapplythisalgorithminthefuture,webelievethefollowingdirectionsofresearchshouldbeconsidered.Firstofall,sincetheoptimalitycutsofthesecondstagegreatlyaffectthesolutionqualityandcomputationaltime,itisnecessarytoconductresearchingeneratingmorepowerfuloptimalitycuts.Second,moredataisneededtochecktheweightdistributionofthecontainers.Inthecaseofnon-normaldistributions,efcientsamplingalgorithmswillbenecessaryintheTMSIP-basedheuristic.Third,itisfoundthattheintegralityconstraintscanbelargelyrelaxedateachdecisionstage,barelyaffectingthealgorithm'sperformance.Thereasonsforthisphenomenonisstillunclear.Fourth,asshowninSection 4.7 ,containersattheOrlandohubarriveinbatches,whichdiffersfromthearrivalprocessassumptionemployedinotherexamples,whiletheheuristicseemsinsensitivetothisdifference.Itwouldbeinterestingtofurtherstudytherelevanceofthearrivalprocesstoloadingdecisions.Finally,itisworthdevelopingsomesuboptimalpolicieswithsimplestructuresthroughmachinelearning,whichwillgreatlyenhancethealgorithm'sapplicability. 126

PAGE 127

CHAPTER5CONCLUSIONSThemaincontributionofthisthesisistheformalstudyofthepostponementoptioninseveralsequentialdecisionproblems,includingtheSSAP,DSKP,andDSLP.Specically,optimalpoliciesarederivedandanalyzedforthersttwoproblems.TheinsightsdevelopedforthesetwoproblemsareutilizedtofacilitatethesolutionoftheDSLP,whichislongencounteredinthelogisticsindustry,yetwithfewresultsintheliterature.Underthehomogeneousresourcesassumption,theoptimalpoliciesfortheSSAPandtheDSKPpossesselegantthresholdstructures.However,itisdifculttocharacterizetheoptimalpoliciesoftheSSAPandDSKPingeneral.Theintuitionobtainedfromtheoptimalthresholdpoliciesinhomogeneouspreviouscasecanbeusedtoderiveeffectiveheuristicsforthegeneralcase.Thenumericalexperimentsindicateaconsiderableprotimprovement.Theseresultscanbedirectlyappliedinthedynamicpricingmodelsandevenshedlightonthestudyofrealoptions.Inparticular,atechnicalbarrierontheapplicationoftherealoptionslieinitslimitationinhandlingasingleinvestmentopportunity,whileourresearchmayprovideinsightsforvaluingrealoptionsinaportfoliosetting.InthesettingoftheDSLP,itisdifculttoanalyzethevalueofthepostponementoption.Indeed,thevalueofpostponementatonedecisionstagedependsonhowcontainersareloadedinfuturestages,whichisinessenceaquadraticassignmentproblem.Therefore,theanalysisofthepostponementoptionisatleastashardassolvingaquadraticassignmentproblem.Asaresult,wesuggestthatonefocusesonoptimizingtheassignmentdecisionanduseaheuristicruletomanagethequeueandthepostponementdecisions.Tosummerize,wendthatthepostponementoptionmayresultinsignicantimprovementintheobjectivefunctionofsequentialdecisionproblems.However,the 127

PAGE 128

postponementoptiongreatlycomplicatestheanalysis,thoughpolicieswithsimplestructurecanbederivedforspecialcases.Ingeneral,analyzingthevalueofthepostponementoptionofasequentialdecisionproblemisatleastashardassolvingthestaticproblematthenaldecisionstage.Moreover,asindicatedintheSSAP,topostponearbitrarilymaynotbenetandmayevenhurtprots.Thethresholdpoliciesinspecialcases,iftheyexist,arecriticaltodesigningeffectiveheuristics.However,whenthereisnosuchguidance,itisdifculttodecidewhentopostpone,asillustratedintheDSLP.Inthefuture,wesuggestmoreresearchindevelopingpolynomialtimeapproximationalgorithmstoevaluatethevaluesofthepostponementoption. 128

PAGE 129

APPENDIXAPROOFSINCHAPTER2 A.1PreliminaryLemmasBeforeestablishingtheoptimalpolicyform=2,werequirethefollowinglemmas. LemmaA.1. Letf(x)beacontinuousfunctionon[b,c]andforeveryx2[b,c),therightderivativeexistsandisnonnegative:f0(x+):=limh#0f(x+h))]TJ /F3 11.955 Tf 11.95 0 Td[(f(x) h0.Thenfisnon-decreasingon[b,c].Moreover,fisstrictlyincreasingon[b,c]iff0(x+)>0foreveryx2[b,c). Proof. FirstweextendthedenitionofftoRbyassumingftobesmoothinR[b,c]andf=0forxc+1.Sofissupportedin[b)]TJ /F4 11.955 Tf 11.96 0 Td[(1,c+1].Let(x)=8><>:c0e1 jxj2)]TJ /F22 5.978 Tf 5.75 0 Td[(1,jxj<10,jxj1wherec0ischosensuchthatR1)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)dx=1.Observethat2C1(R).For>0wedenen():=1 (=).ThenweobservethatZ+1(x)dx=Z1 )]TJ /F22 5.978 Tf 7.78 3.26 Td[(1 (x)dx=Z1)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)dx=1.Letf(x)=RR(x)]TJ /F3 11.955 Tf 11.95 0 Td[(y)f(y)dy.Achangeofvariablegives f(x)=Z+1(z)f(x)]TJ /F3 11.955 Tf 11.96 0 Td[(z)dz=Z1)]TJ /F5 7.97 Tf 6.58 0 Td[(1(z)f(x)]TJ /F8 11.955 Tf 11.96 0 Td[(z)dz.(A)Thenf(x)convergesuniformlytofover[b,c].Indeed,for0<<1weobservethatf(x)=0forxc+2.UsingRR(x)dx=1andEq. A wehavef(x))]TJ /F3 11.955 Tf 11.96 0 Td[(f(x)=Z1)]TJ /F5 7.97 Tf 6.58 0 Td[(1(z)(f(x)]TJ /F8 11.955 Tf 11.96 0 Td[(z))]TJ /F3 11.955 Tf 11.96 0 Td[(f(x))dz.Thenthecontinuityoffimpliesthattherighthandsidetendsto0as!0. 129

PAGE 130

Thesecondpropertyoff(x)isthatitisC1,whichisimpliedfromitsdenition.Foreachx2[b,c),f0(x)=limh#0f(x+h))]TJ /F3 11.955 Tf 11.95 0 Td[(f(x) hZ1)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)limh#0f(x)]TJ /F8 11.955 Tf 11.95 0 Td[(z+h))]TJ /F3 11.955 Tf 11.95 0 Td[(f(x)]TJ /F8 11.955 Tf 11.95 0 Td[(z) hdz0,aslongasb+xc)]TJ /F8 11.955 Tf 12.12 0 Td[(.Infact,ifwemakefincreasingon[b)]TJ /F5 7.97 Tf 13.32 4.71 Td[(1 2,b]and[c,c+1 2]weobtainthatf0(x)0forallx2[b,c](isassumedtobelessthan1 2).NotethattheinequalityintheequationaboveisbecauseoftheFatou'slemmainrealanalysis.Weprovethatfisnon-decreasingon[b,c]bycontradiction.Supposethereexistsx1,x22[b,c]suchthatx1f(x2).Bytheuniformconvergenceofftof,wehavef(x1)>f(x2)forsmall,whichisacontradictiontothefactthatfisnon-decreasingon[b,c].Theremainderfollowsdirectly. LemmaA.2. Forfn(x),n=1,2,...denedon[0,1],whichiscontinuousandnon-decreasinginxandn,andf0n(x+)f0n+1(x+),thenforanyx1x2,fn(x2))]TJ /F3 11.955 Tf 11.95 0 Td[(fn(x1)fn+1(x2))]TJ /F3 11.955 Tf 11.95 0 Td[(fn+1(x1).Whenf0n(x+)>f0n+1(x+),theaboveinequalityisstrict. Proof. Itsufcestoshowthecaseoff0n(x+)f0n+1(x+).Deneq(x)=)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(fn(x))]TJ /F3 11.955 Tf -419.83 -23.91 Td[(fn(x1))]TJ /F14 11.955 Tf 12.71 9.69 Td[()]TJ /F3 11.955 Tf 5.48 -9.69 Td[(fn+1(x))]TJ /F3 11.955 Tf 12.71 0 Td[(fn+1(x1).Notethatq(x1)=0andq0(x+)0.ByLemma A.1 ,q(x2)q(x1). A.2ProofofTheorem 2.4 Proof. Werstverifythethreeconditionsunderwhichpolicystrictlyoutperformspolicy .WithLemma 2.1 andTheorem 2.3 ,itiseasytoverifyCondition(1)form=n=1.Basedonthisconclusionandthemonotonicityofg1,n(x)inx,Condition(2)canbeestablishedthroughinduction. 130

PAGE 131

Next,weshowthatVm,m(0,...,0)=V m,m(0)andVm,n(0,...,0,0)>V m,n(0)formV m,n(0)formVm,m(0,...,0)=V m,m(0)=V m,m(x).Forxw(m,m),Vm,m(0,...,0,x)x+Vm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,m(0,...,0)>x+V m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,m(0).Thus,underCondition(1),policystrictlyoutperformspolicy .Condition(3)canbeveriedsimilarly.Finally,considerthecasewithinniten.Itsufcestoshowthatw(m)=z(m).Theproofisbyinduction.Thecaseofm=1isobvious.Supposez(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1)=w(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1).ByEq. 2 ,z(m)=g1,1(z(m))+Z1z(m)hgm)]TJ /F5 7.97 Tf 6.59 0 Td[(1(z(m)))]TJ /F3 11.955 Tf 11.96 0 Td[(gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1(y)idF(y)=g1,1(z(m))+Z1z(m)]TJ /F22 5.978 Tf 5.76 0 Td[(1)hz(m)]TJ /F5 7.97 Tf 6.58 0 Td[(1))]TJ /F3 11.955 Tf 11.95 0 Td[(yidF(y)=z(m)]TJ /F5 7.97 Tf 6.58 0 Td[(1)+g1,1(z(m)))]TJ /F3 11.955 Tf 11.96 0 Td[(g1,1(z(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1)).Bysummingz(2),...,z(m)asdenedabove,wehavez(m)+m)]TJ /F5 7.97 Tf 6.58 0 Td[(1Xk=2z(k)=m)]TJ /F5 7.97 Tf 6.58 0 Td[(1Xk=1z(k)+g1,1(z(m)))]TJ /F3 11.955 Tf 11.95 0 Td[(g1,1(z(1)))z(m))]TJ /F3 11.955 Tf 11.95 0 Td[(g1,1(z(m))=m)]TJ /F5 7.97 Tf 6.58 0 Td[(1Xk=1z(k))]TJ /F6 7.97 Tf 11.95 14.94 Td[(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1Xk=1z(k). 131

PAGE 132

BytheinductionhypothesisandEq. 2 ,z(m))]TJ /F3 11.955 Tf 12.24 0 Td[(g1,1(z(m))=w(m))]TJ /F3 11.955 Tf 12.25 0 Td[(g1,1(w(m)).Sincex)]TJ /F3 11.955 Tf 11.96 0 Td[(g1,1(x)isstrictlyincreasingforx2[0,z(1)),z(m)=w(m). A.3ProofofTheorem 2.6 Proof. Itsufcestoshowtheconclusionforthecasewhenx1x2
PAGE 133

Supposetheconclusionholdsforthecasewithmandn)]TJ /F4 11.955 Tf 12.92 0 Td[(1.Bytheinductionhypothesis,ateachofthem+1statesafterajobarrives,itisoptimaltoimmediatelyacceptjobminthequeueandassignittoresourcem,i.e.,VPOSTm,n)]TJ /F19 10.909 Tf 5 -8.84 Td[(xm,pm=nVm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F18 10.909 Tf 5 -8.84 Td[((x1,...,xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1),(p1,...,pm)]TJ /F5 7.97 Tf 6.59 0 Td[(1)F(x1)+Zx2x1Vm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F18 10.909 Tf 5 -8.84 Td[((y,...,xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1),(p1,...,pm)]TJ /F5 7.97 Tf 6.58 0 Td[(1)dF(y)++Zxmxm)]TJ /F22 5.978 Tf 5.76 0 Td[(1Vm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F18 10.909 Tf 5 -8.84 Td[((x2,...,xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,y),(p1,...,pm)]TJ /F5 7.97 Tf 6.58 0 Td[(1)dF(y)+Vm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F18 10.909 Tf 5 -8.83 Td[((x2,...,xm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,xm),(p1,...,pm)]TJ /F5 7.97 Tf 6.59 0 Td[(1)F(xm)o+pmVPOST1,1(xm)2.Considertherstarrivingjobvaluedaty.Ify>v(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)1,thisjobshouldbeassignedtoresource2,withtheexpecteddiscountedvalueofthejobassignedtoresource1asv(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)1.Ifw(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)
PAGE 134

yv(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)1=w(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1),thisjobshouldbekeptinthequeue.Thus,v(1,n)2hv(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)2F(w(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))+Zv(1,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)1w(2,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)v(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)1dF(y)+Z1v(1,n)]TJ /F22 5.978 Tf 5.75 0 Td[(1)1ydF(y)i>hw(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)F(w(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))+Z1w(1,n)]TJ /F22 5.978 Tf 5.75 0 Td[(1)ydF(y)i=w(1,n),v(2,n)2hv(2,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)2F(w(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))+Zv(1,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)1w(2,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)ydF(y)+Z1v(1,n)]TJ /F22 5.978 Tf 5.75 0 Td[(1)1v(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)1dF(y)i>hw(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)F(w(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))+Zw(1,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)w(2,n)]TJ /F22 5.978 Tf 5.75 0 Td[(1)ydF(y)+Z1w(1,n)]TJ /F22 5.978 Tf 5.75 0 Td[(1)w(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)dF(y)i=w(2,n),wheretherstandthethirdinequalityfollowsinceinthecaseofy2(0,w(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)],notalljobsinthequeuearevaluedat0.Itiseasytoseethatv(i,n)2isindependentofp2.Supposev(i,n)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1w(i,n),i=1,2,...,m)]TJ /F4 11.955 Tf 12.2 0 Td[(1.Theanalysismimicsthecaseofm=2.First,itiseasytoseethatv(i,m)m=w(i,m).Supposev(i,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)mw(i,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1).Itisnothardtoconcludethatv(i,n)misnolessthanw(i,n)bythefollowingequation:v(i,n)mnv(i,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)mF(w(m,n))+v(i,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1F(v(i,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1))]TJ /F3 11.955 Tf 11.95 0 Td[(F(w(m,n))+Zv(i)]TJ /F22 5.978 Tf 5.75 0 Td[(1,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)m)]TJ /F22 5.978 Tf 5.76 0 Td[(1v(i,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)m)]TJ /F22 5.978 Tf 5.75 0 Td[(1ydF(y)+v(i)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1F(v(i)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)m)]TJ /F5 7.97 Tf 6.59 0 Td[(1)o (A)>hw(i,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)F(w(i,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))+Zw(i)]TJ /F22 5.978 Tf 5.76 0 Td[(1,n)]TJ /F22 5.978 Tf 5.75 0 Td[(1)w(i,n)]TJ /F22 5.978 Tf 5.76 0 Td[(1)ydF(y)+w(i)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)F(w(i)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1))i=w(i,n).Also,v(i,n)misindependentofpm. 134

PAGE 135

APPENDIXBPROOFSINCHAPTER3ThischapterpresentssomeoftheproofsoftheDSKPwiththehomogeneousresources. B.1ProofofTheorem 3.3 Proof. First,considerthecaseofm=1.First,wearguethatifitisnotoptimaltoimmediatelyloadr1given(r1,n,t)for2DPDSKP,thereexistsat>0suchthatforany00andany0r1rdFR(r)d+e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)hV(r1,n,t+h)o+c +(1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)h).Notethatthederivativeoftherighthandsideis(+)e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)hn +r1FR(r1)+Zr>r1rdFR(r))]TJ /F3 11.955 Tf 11.95 0 Td[(V(r1,n,t+h))]TJ /F3 11.955 Tf 24.31 8.09 Td[(c +o,whichisclearlynon-positiveasV(r1,n,t+h)r1.Ifw(1)(r1)=T,supposeanoptimalcontrolpolicyrequiresthedecision-makernottoloadr1givenn=1fromttot+h.Then,requiresthedecision-makernottoloadr1untilT,forotherwiseV(r1,1,t)
PAGE 136

Followingthesameargument,whenitisnotoptimaltoimmediatelyloadrmat(r(m),n,t),thereexistst>0andanoptimalcontrolpolicyrequiringthatforany0n.Followingthesameanalysisabove,itcanbeshownthatV(r(m),n,t)=V(r(m),n,t)form>nandthatfortw(m,n)(rm),thereexistsanoptimalcontrolpolicy,whichbehavesas.Nowconsiderthecaseofm=nandt
PAGE 137

Bytheinductionhypothesis,itfollowsthatV(r(m),m,t)=)]TJ /F16 10.909 Tf 15.6 7.38 Td[(mc +[1)]TJ /F16 10.909 Tf 10.91 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)h]+Zh0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ZXV(UA(r(m),r),m)]TJ /F18 10.909 Tf 10.91 0 Td[(1,t+)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)h[rm+V((r1,...,rm)]TJ /F5 7.97 Tf 6.58 0 Td[(1)0,m)]TJ /F18 10.909 Tf 10.91 0 Td[(1,t+h)]<)]TJ /F16 10.909 Tf 15.6 7.38 Td[(mc +[1)]TJ /F16 10.909 Tf 10.91 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)h]+Zh0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ZXV(UA(r(m),r),m)]TJ /F18 10.909 Tf 10.91 0 Td[(1,t+)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)hV((r1,...,rm)0,m)]TJ /F18 10.909 Tf 10.91 0 Td[(1,t+h)=V(r(m),m,t),wherethesecondinequalityfollowsbytheoptimalityof.Butthiscontradictstheoptimalityof.Thus,fortm. B.2ProofofTheorem 3.2 andTheorem 3.4 Proof. Wewilluseanestedinductionapproachtoestablishthetheorem,withthecaseofm=2asthebasestep.Foreachinductionsteponm,weestablishthetheoremthroughanotherinductiononn.Alsoforeachinductionsteponn,werstshowtheoptimalpolicyandtheoptimalobjectivevalue.Then,weestablishthepropertiesoffunctiongm,n,whichwillbeutilizedintheproofoflaterinductionsteps.Withtwoitemsinthequeue,theoptimalloadingruleisnotstraightforward.Notethatsince(@=@r+)g1,n(r,t)1,g1,n(r1,t))]TJ /F3 11.955 Tf 12.78 0 Td[(r1g1,n(r2,t))]TJ /F3 11.955 Tf 12.77 0 Td[(r2,forr1r2.Thus,g1,n(r1,t)+r2g1,n(r2,t)+r1andatanytime,itisoptimaltoloadtheitemwithhigherrewardifonedecidestoloadanitem.Below,weusethecaseofn=1andn=2toillustratethebasicidea.Letr(2)=(r1,r2).Considerthecaseofn=1.Notethattheitemyettoarrivecanatmostreplaceoneofthequeueditems.Thereforeoneofthequeueditemsshouldbeimmediatelyloaded,deningw(2,1)0.Thus,V(r(2),1,t)=g2,1(r2,t)+g1,1(r1,t),whereg2,1(r2,t):=r2. 137

PAGE 138

Thecaseofn2isverydifferent.TheexpectedprotbypostponingtisV(r(2),2,t;t)=Zt0ZXe)]TJ /F5 7.97 Tf 6.59 0 Td[((+)VhUAr(2),r,1,t+idFR(r)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)thr2+g1,2(r1,t+t)i)]TJ /F4 11.955 Tf 21.18 8.09 Td[(2c +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti.WithV(r(2),1,t)=g2,1(r2,t)+g1,1(r1,t),V(r(2),2,t;t)isseparable:V(r(2),2,t;t)=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)f2(r2,t+)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tr2)]TJ /F3 11.955 Tf 24.32 8.09 Td[(c +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti+V(r1,2,t;t),wheref2isdenedasf2(r2,t)=g2,1(r2,t)FR(r2)+Zr>r2hg2,1(r,t)+g1,1(r2,t))]TJ /F3 11.955 Tf 11.96 0 Td[(g1,1(r,t)idFR(r).Fortw(1)(r2),w(1)(r1)w(1)(r2)byTheorem 3.1 ,andthust>0maynotbeoptimaltoV(r1,2,t;t).Moreover,inthissituation,g2,1(r,t)=g1,1(r2,t)=r,andthusf2(r2,t)0,9w(2,2)(r2)2[0,w(1)(r2))suchthatG(r(2),2,w(2,2)(r2))=0.Thethresholdpolicyisestablished,deningtheexpectedprotasV(r(2),2,t)= 138

PAGE 139

g2,2(r2,t)+g1,2(r1,t),whereg2,2(r2,t):=8>>>>><>>>>>:r2,ift>w(2,2)(r2)Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)f2(r2,t+)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tr2)]TJ /F6 7.97 Tf 18.26 4.3 Td[(c +h1)]TJ /F16 10.909 Tf 10.91 0 Td[(e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)ti,iftw(2,2)(r2),wheret=w(2,2)(r2))]TJ /F3 11.955 Tf 11.95 0 Td[(t.ThisveriesEq. 3 .Next,weshowthat(@=@t)g2,2(r2,t)<0andincreaseswithr2.Notethatfortr2h@g1,1(r2,t) @t)]TJ /F8 11.955 Tf 13.15 8.09 Td[(@g1,1(r,t) @tidFR(r).Since(@=@t)g1,1(r,t)isnon-decreasinginr,itiseasytoseethat(@=@t)f2(r,t)<0andisstrictlyincreasinginr.Moreover,fort
PAGE 140

ofthecasenisseperable:V(r(2),n,t;t)=Zt0e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)fn(r2,t+)d+e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)tr2)]TJ /F16 10.909 Tf 22.43 7.38 Td[(c +h1)]TJ /F16 10.909 Tf 10.9 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti+V(r1,n,t;t),wherefnisdenedasfn(r2,t)=ZXg2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1UA(r2,r),tdFR(r)+Zr>r2hg1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r2,t))]TJ /F3 11.955 Tf 11.95 0 Td[(g1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t)idFR(r).Iftw(1)(r2),tw(1)(r2)w(2,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(r2)(inductionhypothesis)andg2,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r,t+)=randg1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r,t+)=rforrr2.Thus,fn(r2,t)doesnotchangewithtandf2(r2,t)
PAGE 141

Thethresholdpolicyisestablished,deningtheexpectedprotasV(r2,n,t)=g2,n(r2,t)+g1,n(r1,t),whereg2,n(r2,t):=8>>>>><>>>>>:r2iftw(2,n)(r2)Zw(2,n)(r2))]TJ /F6 7.97 Tf 6.58 0 Td[(t0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)fn(r2,t+)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(w(2,n)(r1))]TJ /F6 7.97 Tf 6.59 0 Td[(t)r2)]TJ /F6 7.97 Tf 16.65 4.7 Td[(c +h1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)(w(2,n)(r2))]TJ /F6 7.97 Tf 6.59 0 Td[(t)iift0fort
PAGE 142

Moreover,sincew(2,n)(r2)(@=@r+2)g1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r2,t)andthus(@=@r+2)g2,n(r2,t)>(@=@r+2)g1,n(r2,t).Finally,itcanbeveriedthat(@=@r+2)g2,1(r2,t)isstrictlyincreasinginr2andthus(@=@r+2)g2,n(r2,t)possessesthesameproperty.Next,considerthecaseofgeneralizedm.Weestablishthefollowinginductionhypotheses: (i) Itisoptimaltoloaditemm)]TJ /F4 11.955 Tf 11.96 0 Td[(1; (ii) Theoptimalpolicyandtheobjectivevalueholdform)]TJ /F4 11.955 Tf 11.95 0 Td[(1; (iii) Forim)]TJ /F4 11.955 Tf 12.18 0 Td[(1andt2[0,w(i,n)(ri)),gi,n(ri,t)isstrictlyincreasinginnandriandisstrictlydecreasingint; (iv) Forim)]TJ /F4 11.955 Tf 12.07 0 Td[(1andt2[0,w(i,n)(r)),(@=@r+)gi,n(r,t)2(0,1),isstrictlyincreasinginr,decreasinginnandincreasingini; (v) Forim)]TJ /F4 11.955 Tf 11.96 0 Td[(1andt2[0,w(i,n)(ri)),(@=@t)gi,n(ri,t)isstrictlyincreasinginri; (vi) Forim)]TJ /F4 11.955 Tf 11.96 0 Td[(1,w(i,n)(ri)isnon-increasinginri.Werstshow( i )holdsm.Forr1...rm,V(r1,...,ri)]TJ /F5 7.97 Tf 6.59 0 Td[(1,r,ri+2,...,rm),n,t)]TJ /F3 11.955 Tf 12.13 0 Td[(risnon-increasinginr,since(@=@r+)gi,n(r,t)1.Therefore,V(r1,...,ri)]TJ /F5 7.97 Tf 6.58 0 Td[(1,ri,ri+2,...,rm),n,t)]TJ /F16 10.909 Tf 10.91 0 Td[(riV(r1,...,ri)]TJ /F5 7.97 Tf 6.58 0 Td[(1,ri+1,ri+2,...,rm),n,t)]TJ /F16 10.909 Tf 10.91 0 Td[(ri+1)V(r1,...,ri)]TJ /F5 7.97 Tf 6.58 0 Td[(1,ri,ri+2,...,rm),n,t+ri+1V(r1,...,ri)]TJ /F5 7.97 Tf 6.58 0 Td[(1,ri+1,ri+2,...,rm),n,t+ri.Thustoloaditemi+1isnoworsethantoloaditemi.Theoptimalityofloadingitemmfollowsbyrepeatingthisargument.Now,weshowthatProperty ii ,Property v ,andProperty vi holdsform.First,considerthecaseofm>n.Evenifnitemsyettoarrivearehighinreward,thequeued 142

PAGE 143

itemswiththem)]TJ /F3 11.955 Tf 12.17 0 Td[(nlargestrewardsshouldbeimmediatelyloaded,deningw(m,n)=0andV(rm,n,t)gm,n(rm,t)=rm.Now,considerthecaseofmn.Weestablishtheresultsthroughinduction,withthecaseofm=nasthebasestep.Theanalysisisverysimilartothatinthecaseofm=2.NotethatV(r(m),m,t;t)=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)fm(rm,t+)d+e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)trm)]TJ /F3 11.955 Tf 24.32 8.09 Td[(c +[1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)t]+V(r(m)]TJ /F4 11.955 Tf 11.96 0 Td[(1),m,t;t),wherefmisdenedasfm(rm,t)=rmFR(rm)+Zr>rmhgm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1(rm,t)+r)]TJ /F3 11.955 Tf 11.95 0 Td[(gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t)idFR(r).Fortw(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(rm),gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t)=r.Byincreasingt,fmremainsthesameandpostponementonlycausesmoreholdingcostanddiscounting.Fort>>>><>>>>>:r2ift0Zt0e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)fm(rm,t+)d+e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)trm)]TJ /F6 7.97 Tf 16.65 4.71 Td[(c +h1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tiift>0,wheret=w(m,m)(rm))]TJ /F3 11.955 Tf 11.96 0 Td[(t. 143

PAGE 144

Beforeprocedingtothecaseofgeneralizedn,weshowthat(@=@t)gm,m(rm,t)<0andthatincreasesinrm,fortrmh@gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1(rm,t) @t)]TJ /F8 11.955 Tf 13.15 8.09 Td[(@gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,m)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t) @tidFR(r).Clearly,(@=@t)fm(rm,t)isstrictlyincreasinginrm.Tocompletetheproof,weneedtoshowthatw(m,m)(rm)isstrictlydecreasinginrm.Fortgm,n)]TJ /F5 7.97 Tf 6.58 0 Td[(2(rm,t)and(@=@t)gm,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(rm,t)strictlydecreaseswithrm.NotethatV(r(m),n,t;t)=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)f(rm,t+)d)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)trm)]TJ /F3 11.955 Tf 24.32 8.09 Td[(c +[1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)t]+V(r(m)]TJ /F4 11.955 Tf 11.95 0 Td[(1),n,t;t). 144

PAGE 145

wherefnisredenedasfn(rm,t)=ZXgm,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1UA(rm,r),tdFR(r)+Zr>rmhgm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(rm,t))]TJ /F3 11.955 Tf 11.96 0 Td[(gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t)idFR(r).Iftw(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)(rm),sincew(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(rm)w(m)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(2)(rm)bytheinductionhypothesis,gm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t)=gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r,t)=r.Thus,fn(rm,t+t)doesnotchangewitht;postponementdoesnotbenet,buthurtingtheprotbyhodingcostsanddiscounting.Iftgm,n)]TJ /F5 7.97 Tf 6.58 0 Td[(2(rm,t)andthusfn>fn)]TJ /F5 7.97 Tf 6.58 0 Td[(1.ItfollowsthatG(r(m),n,t+t)>G(r(m),n)]TJ /F4 11.955 Tf 12.32 0 Td[(1,t+t),orG(r(m),n,w(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(rm))>G(r(m),n)]TJ /F4 11.955 Tf 12.44 0 Td[(1,w(m,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1)(rm)).Similartothecaseofm=2,wedenethethresholdasfollows: IfG(r(m),n,w(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(rm))<0,w(m,n)(r2)=w(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(rm)=0; IfG(r(m),n,w(m,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1)(rm))0,w(m,n)(rm)istherootofG(r(m),n,w(m,n)(rm))=0.Anditisstraightforwardtoshowthatgm,nincreasesinn.TocompletetheinductiononEq. ii ,itremainstoshowthat(@=@t)gm,n(rm,t)<0andisstrictlyincreasinginrm.Fortrmh@gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(rm,t) @t)]TJ /F20 10.909 Tf 10.73 7.38 Td[(@gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t) @tidFR(r). 145

PAGE 146

Itiseasytoseethat(@=@t)gm,n(rm,t)<0.Moreover,themonotonicityof(@=@t)fm(rm,t)withrespecttormfollowsfromthatof(@=@t)gm,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r,t)and(@=@t)gm)]TJ /F5 7.97 Tf 6.58 0 Td[(1,n)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r,t).Tocompletetheinduction,weshowthatw(m,n)(rm)isstrictlydecreasinginrm.Fort
PAGE 147

Second,ifw(m)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)(rm)tw(m,n)(rm),1=(@=@r+m)gm,n(rm,t)>(@=@r+m)gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n(rm,t).Ift@gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1,n)]TJ /F5 7.97 Tf 6.59 0 Td[(1(rm,t+) @r+m,theconclusionfollowsfromEq. B .Finally,considerthecaseofinniten.FromtheproofinTheorem 3.1 andTheorem 3.2 ,(@=@r+m)gm,n(rm,t)iscontinuousanddecreasinginn.Since(@=@r+m)gm,n(rm,t)0,(@=@r+m)gm,n(rm,t)ispointwiseconvergentforanytand(@=@r+m)gm,n(rm,t)uniformlyconverges(7.13of Rudin ( 1976 )).Denote@gm(rm,t) @r+m=limn!1@gm,n(rm,t) @r+m.Notethat(@=@t)gm,n(rm,t)isdifferentiableinrm.Sincethearrivalswithinasmallintervalisnite(since<1)andE[R]<1,(@=@t)gm,n(rm,t)isuniformlybounded.Moreover,itcanbeveriedthat(@2=@t@rm)gm,n(rm,t)isuniformlybounded.Thus,(@=@t)gm,n(rm,t)isequicontinuous.ByArzelaAscoliTheorem(11.28of Rudin ( 1987 )),itfollowsthatthereexitsasubsequenceof(@=@t)gm,n(rm,t)uniformlyconverges.Denote@gm(rm,t) @t=limn!1@gm,n(rm,t) @t.Tosee(v),notethat@gm(rm,t) @r+m=Zw(m)(rm))]TJ /F6 7.97 Tf 6.59 0 Td[(t0e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)@gm(rm,t+) @r+mFR(rm)+@gm)]TJ /F5 7.97 Tf 6.59 0 Td[(1(rm,t+) @r+mFR(r)d+e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)(w(m)(rm))]TJ /F6 7.97 Tf 6.59 0 Td[(t)>0. (B)Thus,gm(rm,t)isstrictlyincreasinginrmfort
PAGE 148

B.3ProofofLemma 3.2 Proof. First,weeliminatetheoptionofpostponingrejection:ifnoarrivaloccursduringaperiodofpostponement,t,rejectthequeueditem.Comparedwiththeoptionofimmediaterejection,thisoptionhedgesagainsttheriskofencounteringlessvalueditem,whileincurringahigherdelaycost.Bycontradiction,supposeitisnotdominatedfor(r1,t).Denetheexpectedprotofpostponingrejectionbytas(r1,t,t).Then,(0,t,0)isthetheexpectedprotofimmediaterejectingr1.Sinceg(r,t)decreasesinrandE[R]>c,(0,t,t)=(0,t,0)foranyt2[0,T)]TJ /F3 11.955 Tf 11.96 0 Td[(t].Clearly,(r1,t,t)=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)f(r1,t+)d (B)+(0,t+t,0)e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)t)]TJ /F3 11.955 Tf 13.16 8.08 Td[(c+cqr1 +h1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)hf(r1,t+))]TJ /F3 11.955 Tf 11.96 0 Td[(f(0,t+)id+(0,t,0)e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t)]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr1 +h1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti (B)Leth=minft:(r1,t,t)0,t0g.Bythecontradictionassumption,h>0.Moreover,wearugethath
PAGE 149

Next,weanalyzetheoptimalpolicy.Denetheexpectedprotofpostponingloading:(r1,t,t)=Zt0e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)f(r1,t+)d+r1e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)t)]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr1 +h1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)ti.Itcanbeshownthatthereexistsathresholdrdeningwhethertopostponeloading.Also,itcanbeshownthat(@=@t)(r1,t,t)=0hasauniqueroott.Notethatthereisnoguranteewhetherrislargerthan(0,t,0).Bydening(t)=(0,t,0)andw(r1)=8><>:0ifr(0,t,0)tifr>(0,t,0),theproofiscomplete. B.4ProofofTheorem 3.5 Proof. Thecaseofm=1hasbeenstudiedinLemma 3.2 .Here,weconsiderthecaseswithm2.First,considerthecaseofm=2.Fork=1,whenanewitemarrivesduringpostponement,thedecision-makerhastoloaditem2,bythedenitionof^(k).Thus,withoutconsideringtheimmediaterejectionoption,theexpectedprotisV^(1)(r(2),t;t)=Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)nr2FR(r2)+Zr>r2r+g1(r2,t+))]TJ /F3 11.955 Tf 11.96 0 Td[(g1(r,t+)dFR(r)od+r2+g1(r1,t+t)e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t)]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr2 +1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t+V(r1,t;t).Itiseasytoseethatiftw(1)(r1),postponementprovidesnobenet.Ift
PAGE 150

decisiononlydependsonr2and^G1(r(2),t+t)= +nr2FR(r2)+Zr>r2r+g1(r2,t+t))]TJ /F16 10.909 Tf 10.91 0 Td[(g1(r,t+t)dFR(r)o)]TJ /F16 10.909 Tf 10.91 0 Td[(r2)]TJ /F16 10.909 Tf 12.1 7.38 Td[(c+cqr2 +.As(@=@t)g1(r1,t)isnon-decreasingint,^G1(r(2),t+t)isnon-increasingint.Thus,thereexistsa^w(2,1)(r2)in[0,w(1))specifyingwhetherthepostponedloadingdominatestheimmediateloading.Dene^w(2,1)(r2)=infft:^G1(r(2),t)=0,t2[0,w(1))g.Withouttheoptionofimmediaterejection,thedecisionmakershouldimmediatelyloaditem2if^w(2,1)(r2)torpostponeotherwise.Itshouldbeclearthattheoptionofimmediaterejectioncouldbeoptimal.Toestablishtheresults,weadoptasimilarapproachtothecaseofm=1.Indeed,ift<>:r2if^G1(r(2),t)0r2FR(r2)+Zr>r2r+g1(r2,t))]TJ /F3 11.955 Tf 11.95 0 Td[(g1(r,t)dFR(r)otherwise.Correspondingly,bydening2,1(t)=max0tw(1))]TJ /F6 7.97 Tf 6.59 0 Td[(tnZt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tf1(r2,t+)d)]TJ /F3 11.955 Tf 24.31 8.08 Td[(c +[1)]TJ /F3 11.955 Tf 11.95 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t]o,theexpectedprotofplayingthegameattimetwiththeasset'svalueasr2is^g2,1(r2,t)=8>>>>>>>><>>>>>>>>:r2,ifr2^w(2,1)(r2),r22,1(t)Zt0e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)f1(r2,t+)d+r2e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t)]TJ /F6 7.97 Tf 10.49 5.7 Td[(c+cqr2 +[1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t],ifr2<^w(2,1)(r2),r22,1(t)2,1(t)ifr2<2,1(t), 150

PAGE 151

wheret=^w(2,1)(r2))]TJ /F3 11.955 Tf 12 0 Td[(t.ItfollowsthatV^(1)(r(2),t)=^g2,1(r2,t)+g1(r1,t).Moreover,itiseasytoseethat2,1(t)indecreasingintand2,1(t)1(t).Theaboveargumentcanberepeated.Briey,for(r(2),t),itcanbeshownthatthereexists^w(2,k)]TJ /F5 7.97 Tf 6.58 0 Td[(1)(r2)^w(2,k)(r2)w(1)(r2)suchthat^Gk(r(2),t+^w(2,k)(r2))=0,where^Gk(r(2),t+t)= +n^g2,k)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r2)FR(r2)+Zr>r2^g2,k)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r)+g1(r2,t+t))]TJ /F3 11.955 Tf 11.96 0 Td[(g1(r,t+t)dFR(r)o)]TJ /F3 11.955 Tf 11.96 0 Td[(r2)]TJ /F3 11.955 Tf 13.15 8.09 Td[(c+cqr2 +,whichisnon-increasingint.Moreover,wecandene2,k(t)asfk(r2,t)=8>>>><>>>>:r2if^Gk(r(2),t)0^g2,k)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r2)r2FR(r2)+Zr>r2^g2,k)]TJ /F5 7.97 Tf 6.59 0 Td[(1(r)+g1(r2,t))]TJ /F3 11.955 Tf 11.95 0 Td[(g1(r,t)dFR(r)otherwise,and2,k(t)=max0tw(1))]TJ /F6 7.97 Tf 6.59 0 Td[(tnZt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)tfk)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r2,t+)d)]TJ /F3 11.955 Tf 24.31 8.08 Td[(c +[1)]TJ /F3 11.955 Tf 11.96 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t]o.Correspondingly,theexpectedprotofplayingthegameattimetwiththeasset'svalueasr2is^g2,k(r2,t)=8>>>>>>>><>>>>>>>>:r2,ifr2^w(2,k)(r2),r22,k(t)Zt0e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)fk)]TJ /F5 7.97 Tf 6.58 0 Td[(1(r2,t+)d+r2e)]TJ /F5 7.97 Tf 6.58 0 Td[((+)t)]TJ /F6 7.97 Tf 9.68 5.38 Td[(c+cqr2 +[1)]TJ /F16 10.909 Tf 10.91 0 Td[(e)]TJ /F5 7.97 Tf 6.59 0 Td[((+)t],ifr2<^w(2,k)(r2),r22,k(t)2,1(t)ifr2<2,k(t),wheret=^w(2,k)(r2))]TJ /F3 11.955 Tf 13.12 0 Td[(t.ItfollowsthatV^(k)(r(2),t)=^g2,k(r2,t)+g1(r1,t).Theuniformconvergenceof2,k,^w(2,k),and^g2,kareratherstraightforward.Moreover,2=limk!12,k1. Theproofforthegeneralizedmcanbedoneexactlyinthesameway. 151

PAGE 152

APPENDIXCPROOFSINCHAPTER4 C.1ProofofLemma 4.1 Proof. Underpolicy,anarbitrarystate(b,Q,U,S)ultimatelytransitionsto(b0,Q0,;,S0)ofsomeepocht0,thevalueofwhichisestimatedthroughEq. 4 .AccordingtoEq. 4 ,theincreaseinCimbalancedoesnotaffectcdt0(b0,Q0,;,S0)andtheincreaseinCdelaydoesnotaffectcbt0(b0,Q0,;,S0).IfCimbalanceincreases,allstatesimmediatelyproceedingstate(b0,Q0,;,S0)haveincreasedcb,withnon-decreasingcd.ItfollowsthattheincreaseinCimbalancedoesnotaffectcdt0(b0,Q0,;,S0)andcbt0(b0,Q0,;,S0)isnon-decreasing.Thesecondconclusioncanbeestablishedthroughasimilarargument. C.2ProofofLemma 4.2 Proof. First,considerstate(b,Q,U,S)witht+1=Te)-234(jQj)-234(jUj.Inthenextdecisionepoch,thewaitingoptionisinfeasible;otherwise,thedeadlineconstraintwillbeviolated.Clearly,withtheloadingoptionchosenatepocht,thequeueatepocht+1isarealsubsetofthethatobtainedbychoosingthewaitingoptionepocht.Inotherwords,foreachscenariopathafterepocht+1,thereexistsmoreexibilitytobalancethecontainersifthewaitingoptionischosenatepocht.Therefore,theloadingoptionresultsinahighercostofimbalanceandalowercostofdelay.Then,considerstate(b,Q,U,S)witht+1
PAGE 153

REFERENCES Albright,S.C.OptimalSequentialAssignmentswithRandomArrivalTimes.Manage-mentSci.21(1):(1974)60. .AMarkovChainVersionoftheSecretaryProblem.NavalResearchLogisticsQuarterly.23(1):(1976)151. .ABayesianApproachtoaGeneralizedHouseSellingProblem.ManagementSci.24(4):(1977)432. Albright,S.C.,andC.Derman.AsympoticOptimalPoliciesfortheStochasticSequentialAssignment.ManagementSci.19(1):(1972)46. Amiouny,S.V.,J.J.BartholdiIII,J.H.V.Vate,andJ.Zhang.Balancedloading.Oper.Res.40:(1992)238. Amram,M.,andN.Kulatilaka.RealOptions:ManagingStrategicInvestmentinanUncertainWorld.Boston:HarvardBusinessSchoolPress,1999. Ben-Tal,A.,A.Goryashko,E.Guslitzer,andA.Nemirovski.AdjustableRobustSolutionsofUncertainLinearPrograms.MathematicalProgrammingSer.A99:(2004)351. Ben-Tal,A.,andA.Nemirovski.RobustConvexOptimization.Math.ofOper.Res.23:(1998)769. Bertsekas,D.DynamicProgrammingandOptimalControl.Belmont,Mass.:AthenaScientic,1995. Borodin,A.,andR.El-Yaniv.OnlineComputationandCompetitiveAnalysis.NewYork:CambridgeUniversityPress,1998. Bucklin,L.P.Postponement,SpeculationandtheStructureofDistributionChannels.JournalofMarketingResearch2(2):(1965)26. Chen,H.,andD.Yao.DynamicSchedulingofaMulticlassFluidNetwork.Oper.Res.41:(1993)1104. Chen,W.,andM.Sim.GoalDrivenOptimization.Oper.Res.57(2):(2009)342. Chockalingam,A.,andK.Muthuraman.AmericanOptionsUnderStochasticVolatility.Oper.Res.59(4):(2011)793809. Cochard,D.D.,andK.A.Yost.ImprovingUtilizationofAirForceCargoLoading.Interfaces15:(1985)53. Coffman,E.G.,L.Flatto,andR.R.Weber.OptimalSelectionofStochasticIntervalsUnderaSumConstraint.Adv.Appl.Prob.19:(1987)454. 153

PAGE 154

Coffman,E.G.,andA.L.Stolyar.BandwidthPacking.Algorithmica29:(1999)70. Dean,B.C.,M.X.Goemans,andJ.Vondrak.ApproximatingtheStochasticKnapsackProblem:TheBenetofAdaptivity.Math.Oper.Res.33(4):(2008)945. Derman,C.,G.J.Lieberman,andS.M.Ross.StochasticAssignmentProblem.ManagementSci.18(7):(1972)349. Ding,Q.,andP.Kouvelis.OntheInteractionofProductionandFinancialHedgingDecisionsinGlobalMarkets.TechnicalReport,WashingtonUniversityinSt.Louis.. Elfving,G.APersistencyProblemConnectedwithaPointProcess.JournalofApplidedProbability4:(1967)77. Feitzinger,E.,andH.L.Lee.MassCustomizationatHewlettPackard:thePowerofPostponement.HarvardBusinessReview75:(1997)116. Fleming,W.F.,andM.Soner.ControlledMarkovProcessesandViscositySolutions.BerlinandNewYork:Springer-Verlag,1997. Gajrat,A.,andA.Hordijk.FluidApproximationofaControlledMulticlassTandemNetwork.Stoch.Models.35:(2000)349. Gamarnik,D.,andM.S.Squillante.AnalysisofStochasticOnlineBinPackingProcesses.Stoch.Models.21:(2005)401. Grenadier,S.R.,andA.Weiss.InvestmentinTechnologicalInnovations:AnOptionPricingApproach.JournalofFinancialEconomics44:(1997)397. Grtschel,M.,S.O.Krumke,J.Rambau,T.Winter,andU.Zimmermann.CombinatorialOnlineOptimizationinRealTime.TechnicalReportZIB01-16,Konrad-Zuse-ZentrumfrInformationstechnikBerlin.. Gyorgy,A.,G.Lugosi,andG.Ottuscak.OnlineSequentialBinPacking.J.Mach.Learn.Res.11:(2010)89. Hardy,G.H.,J.E.Littlewood,andG.Polya.Inequalities.Cambridge:TheUniversityPress,1934. Hentenryck,P.,andR.Bent.OnlineStochasticandCombinatorialOptimization.Cambiridge,Massachusetts:TheMITpress,2006. Herbots,J.,W.Herroelen,andR.Leus.DynamicOrderAcceptanceandCapacityPlanningonaSingleBottleneckResource.NavalRes.Logist.54(8):(2007)874. Heyman,D.P.,andM.J.Sobel.StochasticModelsinOperationsResearchVolumeI.NewYork:McGraw-Hill,2007. 154

PAGE 155

Huchzermeier,A.,andC.H.Loch.ProjectManagementUnderRisk.ManagementSci.47(1):(2001)85. Ilhan,T.,S.M.R.Iravani,andM.S.Daskin.TechnicalNoteTheAdaptiveKnapsackProblemwithStochasticRewards.Oper.Res.59(1):(2011)242. Karlin,A.,M.Manasse,L.Rudolph,andD.D.Sleator.CompetitiveSnoopyCaching.Algorithmica3:(1988)79. Karlin,S.StochasticModelsandOptimalPolicyforSellinganAsset.K.J.Arrow,S.Karlin,H.Scarf,editors,StutdiesinAppliedProbabilityandManagemnentScience.Stanford,California:StanfordUniversityPress,1962. Kellerer,U.Pferschy,H.,andD.Pisinger.KnapsackProblems.Berlin,Germany:Springer,2004. Kennedy,D.P.OptimalSequentialAssignment.Math.Oper.Res.11(4):(1986)619. Kleywegt,A.J.,andJ.D.Papastavrou.TheDynamicandStochasticKnapsackProblem.Oper.Res.46(1):(1998)17. .TheDynamicandStochasticKnapsackProblemwithRandomSizedItems.Oper.Res.49(1):(2001)26. Limbourg,S.,andM.SchynsandG.Laporte.AutomaticAircraftCargoLoadPlanning.J.Oper.Res.Soc.1. Lu,L.L.,S.Y.Chiu,andL.A.CoxJr.OptimalProjectSelection:StochasticKnapsackWithFiniteTimeHorizon.J.Oper.Res.Soc.50(6):(1999)645. Martin-Vega,L.A.AircraftLoadPlanningandtheComputer:DescriptionandReview.ComputersandIndustrialEngineering4:(1985)357. McDonald,R.,andD.Siegel.TheValueofWaitingtoInvest.QuarterlyJournalofEconomics101:(1986)707. McGrath,R.G.ARealOptionsLogicforInitiatingTechnologyPositioningInvestments.AcademyofManagementReview22:(1997)974. McLay,L.A.,S.H.Jacobson,andA.G.Nikolaev.ASequentialStochasticPassengerScreeningProblemforAviationSecurity.IIETransactions41(6):(2009)575. Mongeau,M.,andC.Bes.ASequentialStochasticPassengerScreeningProblemforAviationSecurity.IEEETransactionsonAerospaceandElectronicSystems39(1):(2003)140. Nakai,T.ASequentialStochasticAssignmentProbleminaStationaryMarkovChain.MathematicaJaponica31:(1986a)741. 155

PAGE 156

.ASequentialStochasticAssignmentProbleminaPartiallyObservableMarkovChain.Math.Oper.Res.11:(1986b)230. Nikolaev,A.G.,andS.H.Jacobson.TechnicalNoteStochasticSequentialDecision-MakingwithaRandomNumberofJobs.Oper.Res.58(4):(2010)1023. Papadaki,K.,andW.Powell.AnAdaptiveDynamicProgrammingAlgorithmforaStochasticMultiproductBatchDispatchProblem.NavalRes.Logist.50(7):(2003)742. Papadaki,K.,andW.B.Powell.MonotonicityinMultidimensionalMarkovDecisionProcessesfortheBatchDispatchProblem.Oper.Res.Lett.35:(2007)267. Papastavrou,J.D.,S.Rajagopalan,andA.Kleywegt.TheDynamicandStochasticKnapsackProblemwithDeadlines.ManagementSci.42(12):(1996)1706. Powell,W.ApproximateDynamicProgramming:SolvingtheCursesofDimensionality.NewJersey:John&WileySons,1994. Puterman,M.L.MarkovDecisionProcesses:DiscreteStochasticDynamicProgram-ming.NewJersey:John&WileySons,1994. Rhee,W.,andM.Talagrand.ANoteonTheSelectionofRandomVariablesunderaSumConstraint.J.Appl.Prob.28:(1991)919. Righter,R.L.TheStochasticSequentialAssignmentProblemwithRandomDeadlines.ProbabilityintheEngineeringandInformationalSciences1:(1987)189. .AResourceAllocationProbleminARandomEnvironment.Oper.Res.37(2):(1989)329. Ross,S.M.IntroductiontoStochasticDynamicProgramming.NewYork:AcademicPress,1983. Rudin,W.PrinciplesofMathematicalAnalysis.NewYork:McGrawHill,1976. .RealandComplexAnalysis.NewYork:McGrawHill,1987. Ruszczynski,A.,andA.Shapiro(eds).StochasticProgramming,HandbooksinOperationsResearchandManagementScience,10.Amsterdam:Elsevier,2003. Saario,V.LimitingPropertiesoftheDiscountedHouse-SellingProblem.Eur.J.Oper.Res.20(2):(1985)206. Sakaguchi,M.ASequentialStochasticAssignmentProblemAssociatedwithaNon-homogeneousMakovProcess.MathematicaJaponica29(1):(1984a)13. .ASequentialStochasticAssignmentProblemAssociatedwithUnknownNumberofJobs.MathematicaJaponica29(2):(1984b)141. 156

PAGE 157

Scarf,H.TheOptimalityof(S,s)PoliciesintheDynamicInventoryProblem.Math-ematicalMethodsintheSocialSciencesStanfordUniversityPress,Stanford,California. Serfozo,R.F.IntroductiontoStochasticNetworks.NewYork:Springer-Verlag,1999. Shapiro,A.OnComplexityofMultistageStochasticPrograms.Oper.Res.Lett.34(1):(2006)1. Smith,J.E.,andK.Mcardle.StructuralPropertiesofStochasticDynamicPrograms.Oper.Res.50(5):(2002)796. Spivey,M.Z.,andW.B.Powell.TheDynamicAssignmentProblem.TransportationSci.38(4):(2004)399. Su,X.,andS.Zenios.PatientChoiceinKidneyAllocation:ASequentialStochasticAssignmentModel.Oper.Res.53(3):(2005)443. Sutton,R.S.,andA.G.Barto.ReinforcementLearning:AnIntroduction.Cambridge,MA:MITPress,1998. Talluri,K.T.,andG.J.vanRyzin.TheTheoryandPracticeofRevenueManagement.NewYork:Springer-Verlag,2005. Thomas,C.,K.Campbell,G.Hines,andM.Racer.AirbusPackingatFederalExpress.Interface28(4):(1998)21. Triantis,A.J.RealOptionsandCorporateRiskManagement.JournalofAppliedCorporateFinance13(2):(2000)64. Trigeorgis,L.RealOptions:ManagerialFlexibilityandStrategyinResourceAllocation.Cambridge,MA:MITPress,1996. VanHoek,R.I.TheRediscoveryofPostponementaLiteratureReviewandDirectionsforResearch.JournalofOperationsManagement19(2):(2001)161. Vinter,R.B.OptimalControl.Boston,MA:Birkhauser,2000. Wein,L.DynamicSchedulingofaMulticlassMake-To-StockQueue.Opes.Res.40:(1992)717. Whittle,P.OptimalControl:BasicsandBeyond.NewYork:Wiley,1996. Zenios,S.A.,G.M.Chertow,andL.M.Wein.DynamicAllocationofKidneystoCandidatesontheTransplantWaitingList.Opes.Res.48(4):(2000)549. 157

PAGE 158

BIOGRAPHICALSKETCH TiankeFengwasborninBeijing,China.HeattendedtheBeijingUniversityofTechnology,majoringintransportationcivilengineeringin1998.HereceivedhisdegreeinBachelorofEngineeringin2002andwasinvitedintothegraduateprogramintransportationengineeringwiththehighestgraduateentranceexamgrade.There,heobtainedaMasterofEngineeringin2005.Then,hedecidedtopursuefutureinacademiaandwasadmittedbytheUniversityofFloridaasaPh.D.studentfortheDepartmentofCivilandCoastalengineeringin2006.Oneyearlater,hefoundthathisinterestsweremorealignedwiththeDepartmentofIndustrialandSystemsEngineeringandhetransferredtothatdepartment.There,heworkedbothonsupplychainmanagementandstochasticoptimization. 158