Citation
Exact and Heuristic Approaches for Integer Knapsack Problems

Material Information

Title:
Exact and Heuristic Approaches for Integer Knapsack Problems
Creator:
He, Xueqi
Publisher:
University of Florida
Publication Date:
Language:
English

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Industrial and Systems Engineering
Committee Chair:
HARTMAN,JOSEPH C
Committee Co-Chair:
PARDALOS,PANAGOTE M
Committee Members:
GEUNES,JOSEPH PATRICK
HAGER,WILLIAM WARD

Subjects

Subjects / Keywords:
knapsack

Notes

General Note:
In this dissertation, we consider several integer variants of the standard binary knapsack problem, and propose new approaches in different areas of interest, exact algorithms and heuristic algorithms. First, we develop a hybrid approach which combines a dynamic programming approach with a branch-and-bound algorithm to solve the unbounded knapsack problem to optimality. The intermediate solutions of a forward dynamic programming are utilized to derive upper bounds of profit and valid inequalities which depict the relation of consumed knapsack capacity and total profit earned in any feasible solution. By incorporating these inequalities into an equivalent integer programming formulation of the unbounded knapsack problem, the problem can be solved through a cut-and-branch framework. Extensive experiments are conducted to show the effectiveness of our approach. We next extend our approach to the multidimensional unbounded knapsack problem, where more than one constraint must be followed. Dynamic programming with lists is implemented in consideration of memory requirements, while inequalities are relaxed to make the trade-off between efficiency and effectiveness. All the problems studied so far have deterministic input data, however, in real life, many cases contain incomplete or uncertain data. Therefore, we next examine the robust binary knapsack problem with profit uncertainty and propose exact dynamic programming algorithms with a space reduction technique. Finally, we broaden our research from a linear problem to a non-linear problem. A reactive GRASP heuristic, which utilizes a group of greedy algorithms, is designed to tackle the quadratic binary knapsack problem.

Record Information

Source Institution:
UFRGP
Rights Management:
All applicable rights reserved by the source institution and holding location.
Embargo Date:
8/31/2018

Downloads

This item has the following downloads:


Full Text

PAGE 1

EXACTANDHEURISTICAPPROACHESFORINTEGERKNAPSACKPROBLEMS By XUEQIHE ADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOL OFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENT OFTHEREQUIREMENTSFORTHEDEGREEOF DOCTOROFPHILOSOPHY UNIVERSITYOFFLORIDA 2016

PAGE 2

c 2016XueqiHe

PAGE 3

Tomyfamily

PAGE 4

ACKNOWLEDGMENTS Iwouldliketorstexpressmysinceregratitudetomyacademicadvisors,Dr.Joseph C.HartmanandDr.PanosM.Pardalosforthecontinuoussupportofmyresearch.Their guidancehelpedmeintacklingchallengingproblemsandmademyPh.D.possible.Besidesmy advisors,IwouldliketothankDr.JosephP.GeunesandDr.WilliamW.Hagerforservingon mysupervisorycommittee,andfortheirinsightfulcommentsandsuggestions.Iwouldalsolike tothankDr.YongpeiGuan,Dr.Jean-PhilippeRichard,Dr.J.ColeSmithandallprofessors inDepartmentofIndustrialandSystemsEngineeringforthegreateducationwhichlaidasolid foundationformyresearch. IwouldliketothankallofmyfriendsatUF.Iwillneverforgetthewonderfultimethatwe spenttogether. Finally,Iwouldliketothankmyparentsfortheirunconditionaltrust,constantsupport andsacricethattheyhavemadetoprovidemewiththebest.Iloveyou. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS ................................... 4 LISTOFTABLES ...................................... 7 LISTOFFIGURES ..................................... 8 ABSTRACT ......................................... 9 CHAPTER 1INTRODUCTION ................................... 11 2DYNAMICPROGRAMMINGBASEDINEQUALITIESFORTHEUNBOUNDED KNAPSACKPROBLEM ................................ 13 2.1MotivationandLiteratureReview ........................ 13 2.2DominanceRelations ............................... 14 2.3DynamicProgrammingApproach ........................ 15 2.3.1FundamentalDynamicProgrammingApproach ............. 15 2.3.2ModiedDynamicProgrammingApproach ............... 16 2.4ValidInequalitiesDerivation ........................... 16 2.5ComputationalResults .............................. 21 2.5.1InstanceGeneration ........................... 22 2.5.1.1Randomcases ......................... 22 2.5.1.2Realisticcases ......................... 23 2.5.1.3Hardcases ........................... 23 2.5.2ResultsSummary ............................. 24 2.6ConcludingRemarks ............................... 28 3DYNAMICPROGRAMMINGBASEDINEQUALITIESFORTHEMULTIDIMENSIONAL UNBOUNDEDKNAPSACKPROBLEM ....................... 29 3.1MotivationandLiteratureReview ........................ 29 3.2DominanceRelations ............................... 31 3.3DynamicProgrammingApproach ........................ 32 3.3.1FundamentalDynamicProgrammingApproach ............. 32 3.3.2DynamicProgrammingApproachwithLists ............... 33 3.4ValidInequalitiesDerivation ........................... 35 3.5ComputationalResults .............................. 38 3.6ConcludingRemarks ............................... 39 4EXACTDYNAMICPROGRAMMINGAPPROACHFORTHEROBUSTKNAPSACK PROBLEMWITHPROFITUNCERTAINTY ..................... 40 4.1MotivationandLiteratureReview ........................ 40 5

PAGE 6

4.2DynamicProgrammingApproach ........................ 42 4.2.1DynamicProgrammingRecursion .................... 42 4.2.2StorageReductioninDynamicProgramming .............. 46 4.3OtherExactApproaches ............................. 53 4.3.1AMILPFormulation ........................... 53 4.3.2IterativeSolutionofNominalKnapsackProblems ............ 54 4.4ComputationalResults .............................. 55 4.4.1InstanceGeneration ........................... 55 4.4.2ResultsSummary ............................. 56 4.5GeneralizationoftheRobustKnapsackProblem ................ 60 4.5.1DynamicProgrammingApproach .................... 60 4.5.2ComputationalResults .......................... 62 4.6ConcludingRemarks ............................... 67 5REACTIVEGRASPHEURISTICFORTHE0-1QUADRATICKNAPSACKPROBLEM 68 5.1MotivationandLiteratureReview ........................ 68 5.2GreedyAlgorithm ................................ 70 5.2.1AlgorithmSchemes ............................ 70 5.2.2GreedyFunctions ............................. 71 5.2.3NumericalStudy ............................. 72 5.3GRASPFramework ............................... 74 5.4ReactiveGRASP ................................. 75 5.4.1GreedyAlgorithm ............................. 76 5.4.2ConstructionofRCL ........................... 77 5.4.3ItemSelectionfromRCL ......................... 78 5.4.4RestartMechanism ............................ 79 5.4.5ConstructionPhase ............................ 80 5.4.6LocalSearchPhase ............................ 81 5.4.7IntegratedReactiveGRASP ....................... 82 5.5ComputationalResults .............................. 83 5.6ConcludingRemarks ............................... 89 6CONCLUSIONS .................................... 91 REFERENCES ........................................ 93 BIOGRAPHICALSKETCH ................................. 98 6

PAGE 7

LISTOFTABLES Table page 2-1Computationaltimesummary(milliseconds)for ............... 25 2-2Computationaltimesummary(milliseconds)for ............... 25 2-3Computationaltime(seconds)ofspecialinstances .................. 27 3-1Analysisofsimplydominancerelationinrandomlygeneratedcases ......... 32 3-2Summaryofexperimentsforthed-UKP ........................ 38 4-1Averageruntime(seconds)onvarioustestinginstancesfortheRKP ........ 58 4-2Averageruntime(seconds)onvarioustestinginstancesfortheG-RKP ....... 65 5-1Relativeperformanceofgreedyalgorithmsonrandomgeneratedinstances ..... 73 5-2Solutionaccuracyofseveralheuristicalgorithms ................... 85 7

PAGE 8

LISTOFFIGURES Figure page 2-1Graphof valuesandenvelopeinequalities. ................... 19 2-2Graphof valuesandenvelopeinequalities. ................... 21 2-3Graphof valuesandenvelopeinequalities. ................... 22 2-4Initialgapof cplex andproposedmodelsforpartialtestingcases. .......... 27 3-1Feasiblestatesfora2-dknapsackproblem. ...................... 36 3-2UpperconcaveenvelopeofpointsinFigure3-1. ................... 36 3-3Projectionofpointsto2-dplaneandtheupperconcaveenvelope. .......... 37 3-4Runtimeforthed-UKP. ............................... 39 3-5Initialgapforthed-UKP. ............................... 39 4-1IllustrationoftheDPrecursion. ............................ 46 4-2AbinarytreerepresentingtherecursionstructureofAlgorithm4.3. ......... 51 4-3TestinginstancesfortheRKP. ............................ 57 5-1Computationaltime(seconds)ofseveralheuristicalgorithms. ............ 88 5-2RuntimedecompositionforGRASPheuristics. .................... 89 8

PAGE 9

AbstractofDissertationPresentedtotheGraduateSchool oftheUniversityofFloridainPartialFulllmentofthe RequirementsfortheDegreeofDoctorofPhilosophy EXACTANDHEURISTICAPPROACHESFORINTEGERKNAPSACKPROBLEMS By XueqiHe August2016 Chair:JosephC.Hartman Cochair:PanosM.Pardalos Major:IndustrialandSystemsEngineering Inthisdissertation,weconsiderseveralintegervariantsofthestandardbinaryknapsack problem,andproposenewapproachesindi!erentareasofinterest,exactalgorithmsand heuristicalgorithms. First,wedevelopahybridapproachwhichcombinesadynamicprogrammingapproach withabranch-and-boundalgorithmtosolvetheunboundedknapsackproblemtooptimality. Theintermediatesolutionsofaforwarddynamicprogrammingareutilizedtoderiveupper boundsofprotandvalidinequalitieswhichdepicttherelationofconsumedknapsackcapacity andtotalprotearnedinanyfeasiblesolution.Byincorporatingtheseinequalitiesintoan equivalentintegerprogrammingformulationoftheunboundedknapsackproblem,theproblem canbesolvedthroughacut-and-branchframework.Extensiveexperimentsareconductedto showthee! ectivenessofourapproach.Wenextextendourapproachtothemultidimensional unboundedknapsackproblem,wheremorethanoneconstraintmustbefollowed.Dynamic programmingwithlistsisimplementedinconsiderationofmemoryrequirements,while inequalitiesarerelaxedtomakethetrade-o!betweene"ciencyande!ectiveness.Allthe problemsstudiedsofarhavedeterministicinputdata,however,inreallife,manycasescontain incompleteoruncertaindata.Therefore,wenextexaminetherobustbinaryknapsackproblem withprotuncertaintyandproposeexactdynamicprogrammingalgorithmswithaspace reductiontechnique.Finally,webroadenourresearchfromalinearproblemtoanon-linear 9

PAGE 10

problem.AreactiveGRASPheuristic,whichutilizesagroupofgreedyalgorithms,isdesigned totacklethequadraticbinaryknapsackproblem. 10

PAGE 11

CHAPTER1 INTRODUCTION Knapsackproblemsformafamilyoffundamentalcombinatorialoptimizationproblems. Allproblemvariantsdesireasubsetofitemswhichyieldthemaximumtotalprotwithout violatingthecapacityrestrictionoftheknapsack.Thereisvastresearchonknapsackproblems duetotheirbroadapplications.Inpractice,theyareappliedtocuttingstockproblems[ 1 3 ], capitalbudgetingproblems[ 4 5 ]andrecourseallocationproblems[ 6 7 ].Theyalsocommonly occurassubproblemsinmanyintegersolutionmethodsoftheoreticalresearch. Thefamilyofknapsackproblemsisbuiltuponthestandardbinaryknapsackproblemwith di erentassumptions.Thebinaryknapsackproblemisthesimplestproblemwhereasingle constraintneedstobesatisedandeachitemcannotbechosenmorethanonce.Whileinthe boundedknapsackproblemandtheunboundedknapsackproblem,weincreasetheavailability ofeachitemtypetoagivenpositiveintegernumberandtoinnitycopies,respectively.Other knapsackfamilymembersarethemultipleknapsackproblemwhereitemsarerequiredtobe packedintoseveraldisjointknapsacks,themultiple-choiceknapsackproblemwhereitems areselectedfromdisjointsets,themultidimensionalknapsackproblemwhereatleasttwo constraintsareconsidered.Withnewobjectivefunctions,anordinarylinearknapsackproblem couldbecomeanon-linearproblem.Andwhenparametersofitemsareuncertain,wewouldlike tosolvearobustversionofknapsackproblemtoeitherguaranteethefeasibilityofasolution orachievethehighestprotintheworstcase.Moreover,aspecialcaseofthebinaryknapsack problemisknownasthesubsetsumprobleminwhichtheprotofanitemequalsitsweight. AllknapsackproblemsareinthedomainofNP-hardproblems.Tosolvetheseproblems foroptimalornear-optimalsolution,agreatnumberofalgorithmsortechniqueshavebeen developed.Themainexactapproachesincludedynamicprogrammingandbranch-and-bound. Greedyalgorithmsanddi! erentheuristicandapproximationalgorithmso! ermoree" cient alternatives,especiallywhendealingwithlarge-scaleproblems.Conceptssuchascore, dominance,preprocessingareintroducedtoassistthecomputingprocess. 11

PAGE 12

Thisdissertationcoversseveralknapsackproblemsandproceedsasfollows.InChapter 2 wedevelopanewhybridapproachfortheunboundedknapsackproblem.Andweextenditto themultidimensionalunboundedknapsackprobleminChapter 3 .Chapter 4 studiestherobust knapsackproblemwheretheprotofeachitemisnotexactlyknownbutbelongstoagiven interval.InChapter 5 ,areactiveGRASPalgorithmandnon-reactiveGRASPalgorithmsare analyzedandcomparedforthequadraticknapsackproblem. 12

PAGE 13

CHAPTER2 DYNAMICPROGRAMMINGBASEDINEQUALITIESFORTHEUNBOUNDEDKNAPSACK PROBLEM 2.1MotivationandLiteratureReview Theunboundedknapsackproblem(UKP)describesthefollowingcombinatorial optimizationproblem:Given typesofitems,itemsoftype haveprot ,weight andunlimitedsupply.Theobjectiveistodeterminethenumberofcopiesofeachtypeofitem thatshouldbeplacedinaknapsack,withcapacityof ,tomaximizetotalprot.Here, and ,arepositiveintegers.Thisproblemcanbeformulatedas: maximize subjectto # Z (21) TheUKP,whichwasprovedtobeNP-hardbyLueker[ 8 ],hasbeenstudiedbroadlyinthe integerprogrammingliterature.FloudasandPardalos[ 9 ]providedacomprehensiveoverviewof thisproblem.GilmoreandGomory[ 10 ]andCabot[ 11 ]proposedalgorithmstosolvetheUKP basedonbranchandbound.Byderivingupperboundsanddeningacoreproblem,Martello andToth[ 12 ]presentedanalgorithmforlarge-scaleUKP. Variousdynamicprogrammingapproacheshavealsobeendeveloped.Classicdynamic programmingalgorithms,whichprovideexactsolution,havebeenintroducedbyDantzig[ 13 ], Bellman[ 14 ],andGilmoreandGomory[ 15 ].Inordertoreducethesearchspaceandsolution time,severalimportantpropertiesoftheUKPhavebeenstudiedandintegratedintodynamic programmingapproaches.GilmoreandGomory[ 10 15 ]proposedthenotionsofdominance andperiodicity.Basedonpartitioninginnumbertheory,YanasseandSoma[ 16 ]provided amodieddynamicprogrammingapproach.MartelloandToth[ 17 ]introducedmultiple dominancerelations.Andonovetal.[ 18 ]presentedanewdynamicprogrammingalgorithm, EDUK,basedonanewdominancerelationtermedthresholddominance. 13

PAGE 14

Bycombiningbranchandboundanddynamicprogramming,severalhybridalgorithms havebeenintroducedtosolvetheUKP.Poirriezetal.[ 19 ]providedanalgorithmto incorporateinformationobtainedfromapplyingbranchandboundapproachtothecore problemintodynamicprogramming.MartelloandToth[ 20 ]presentedamixedapproachto obtaintheexactsolutionofthesubsetsumproblem(SSP),whichisaspecialtypeoftheUKP. Thepurposeofthischapteristoillustratehowvalidinequalitiescanbegenerated fromsolutionsofintermediatestagesofadynamicprogrammingalgorithmtoimprove computationale" ciencyofsolvingtheintegerprogrammingformulationoftheUKPwith traditionalapproach.ThisapproachwasproposedbyHartmanetal.[ 21 ]forthecapacitated lot-sizingproblem.Unlikeotherhybridalgorithms,utilizingbranchandboundapproachto assistadynamicprogrammingalgorithm,weincorporateinequalitiesderivedfromdynamic programmingintotherootnodeofabranchandboundtree,sinceintermediatesolutions ofdynamicprogrammingprovideusefulinformationtostrengthentheintegerprogramming formulationoftheUKP.Itisthersttime,tothebestofourknowledge,thatsuchamethod hasbeenappliedtotheUKP. Therestofthischapterisorganizedasfollows.InSection 2.2 ,wereviewtheconcept ofdominancerelationsfortheUKP.Section 2.3 describestheclassicdynamicprogramming methodforsolvingtheUKPandhowwecanreviseitwiththedetectionofdominance. Section 2.4 introducesourapproachtoobtainvalidinequalitiesfromthedynamicprogramming method.Section 2.5 discussescomputationalexperimentdesignandshowstestresults.Finally weconcludeinSection 2.6 2.2DominanceRelations DominanceisanimportantstructuralpropertyfortheUKP.Severalresearchershave contributedtothistopic,suchasMartelloandToth[ 12 ]andAndonovetal.[ 18 ].We summarizeknowndominancerelationsasfollows: 1. Itemtype simplydominates type ,if and $ 2. Itemtype multiplydominates type ,if and $ ,where # Z 14

PAGE 15

3. Asetofitemtypes collectivelydominates type ,if ! and ! $ where # Z .Simpledominanceandmultipledominancearespecialcasesofcollective dominance. 4. Asetofitemtypes thresholddominates type ,if ! " and ! $ where # Z and # Z Dominancerelationshelpsimplifytheproblem.Theysetanupperboundforthenumber ofcopiesofanitemtypeintheoptimalsolution.Ifitemtype iscollectivelydominated,itwill generatelessprotandconsumemorecapacitycomparedwithacombinationofitemsinset Therefore,itwillnotappearinanyoptimalsolutionforanycapacity,andwecandiscardit.If itemtype isthresholddominated,forasimilarreason,themaximumnumberofcopiesoftype inoptimalsolutionis % .Detectingdominancerelationscouldbeconsideredasastepof pre-xingvariables'valuesandpre-processingtoreducethesizeofproblem,whichwilldecrease thecomputationalcomplexityoffollowingprocedures. 2.3DynamicProgrammingApproach 2.3.1FundamentalDynamicProgrammingApproach SeveraldynamicprogrammingapproacheshavebeendevelopedfortheUKP.Garnkel andNemhauser[ 22 ]introducedseveralfundamentaldynamicprogrammingrecursions.In considerationofhowvalidinequalitiescouldbegenerated,thefollowingforwarddynamic programmingrecursionisappliedtoourproposedmethod:Let bethemaximumtotal protthatcanbeachievedin( 21 )withknapsackcapacity usingonlytherst typesof items,where # { } and # { } .Withinitialconditions & and & ,thedynamicprogrammingrecursionisgivenby: # $ $ % $ $ & if < maximum { % } if $ (22) TheUKPcouldbesolvedbyllinga tablewithrecursiveequationgivenabove. Eachcolumn # { } ,representsastage.Whilecell representsastateand recordsthebestprotthatcanbeachievedbytherst typeitemswhen unitsofcapacity 15

PAGE 16

havebeenused,or .Thevalueincell showstheobjectivevalueoftheoptimal solution.Thecomplexityofthisdynamicprogrammingalgorithmis .Whensolving large-sizeproblems,thisalgorithmcouldbetimeandmemoryspaceconsuming. 2.3.2ModiedDynamicProgrammingApproach Now,wewillillustratehowthedetectionofdominancerelationscanbeinvolvedin dynamicprogrammingandbenettheprocess.Thedetailedprocedureislistedbelow: Step1. Identifythemaximumweightamongallitemtypes. Step2. Applythedynamicprogrammingalgorithm,whichismentionedabove,forall itemtypesuntilcapacity reachesthemaximumweightidentied. Step3. Examinestate foreach .If > ,thenitemtype iscollectively dominated.Weaddtheupperboundconstraintof totheintegerprogramming formulationandremovetype (stage )fromthedynamicprogrammingprocess. Step4. Completethedynamicprogrammingprocessontheremainingstagestothe knapsackcapacity 2.4ValidInequalitiesDerivation Although,fromapracticalpointofview,theadjusteddynamicprogrammingprocessruns fasterthantheoriginalone,theyhavethesamecomplexityof .Whenthecapacityof theknapsackortheamountofitemtypesislarge,itmaybecumbersometoexecutetheentire adjusteddynamicprogramming.Withpartialdynamicprogrammingcompleted,informationon intermediatesolutionscanbetranslatedintoe! ectiveinequalitiesfortheintegerprogramming formulation( 21 )oftheUKP.Toassuremeaningfulsolutions,wesortitemsindescending orderofprotability,i.e.: $ $ $ Itshouldbeclearthatanitemwithhigherprotabilityismorelikelytobenetthetotalprot ofknapsackwhenthecapacityisxed.Andaccordingtoperiodicity,anotherwell-known propertyoftheUKP,themostprotableitemplaysadi!erentrolewithothertypeitems,i.e., itistheonlyitemthatcontinuouslycontributestotheoptimalsolutionwhencapacityisabove athresholdvalue.Therefore,wewouldliketoplacethemostprotableitemastherststage. 16

PAGE 17

Lemma 1 denesavalidinequalityfortheUKPbasedonthedynamicprogramming solutions. Lemma1. Forany ,thefollowinginequalityisvalidforallfeasiblesolutionsof ( 21 ): (23) Proof. Bythedenitionof ,wehave Because constituteafeasiblesolution,theysatisfytheproblemconstraint: Therefore,togetherwiththefactthat isanon-decreasingfunctionofcapacity ,we concludethat Thus,( 23 )isvalidto( 21 ). Thisinequalitydenesanupperboundontheprotthattherst typesofitemswould contributetotheknapsackinallfeasiblecases. Secondly,weintroduceanothervalidinequalitywhichdenesanupperboundforthetotal protoftherst typesofitemsasafunctionoftheirtotalweight.Dene and .Thereexistsapoint correspondingtoeachfeasiblesolutionand allofthesepointsareonorbelowthestepfunction (refertoFigure 2-1 -Figure 2-3 ), because canalsobeinterpretedasthehighestprotobtainedwhenthetotalweight oftherst typesofitemsisnomorethan .Notethat isnotnecessarilyaconcave functionof .However,wecoulddenetheupperconcaveenvelopeof byusingaset ofinequalities .Let and representtheslopeandinterceptofthe -thinequalityof 17

PAGE 18

respectively.Andtheseinequalitiesareinthefollowingform: (24) Sinceallfeasiblesolutions satisfyinequality( 24 ),theupperboundonthetotalprot asafunctionofthetotalweightcanbedenedasfollows: (25) or % (26) Sincetheknapsackhasacapacityof couldbeanyvaluebetween0and .Tond theupperconcaveenvelopeof ,weneedtoexamine points.Thisprocesscouldbe simpliedwiththefollowingtwolemmas. Lemma2. Ifthecapacityoftheknapsackislargeenoughsuchthat $ ,thentherst inequalitywhichdenestheconcaveenvelopeof hasslope andintercept ,or: (27) Proof. Recallthatwehaveorderedtheitemsbydecreasingprotability.Because recordsthebestprotundercapacity andthehighestprotperunitisgivenby , aswellasthetotalprotforanyfeasiblesolution,shouldbelessthanorequalto Therefore,inequality( 27 )holds.Also,wecouldconrmthat( 27 )istightanddenesthe rstinequalityoftheconcaveenvelopeof ,sinceitpassesthroughpoints and whichareonthefunction Lemma3. Ifthereisnoitemthathassameprotabilitywithitem1,thenthesecond inequalitysegmentbeginsfrompoint ( ) ( ) 18

PAGE 19

Proof. Theproofisstraightforward.Becauseallpoints ( ) ,areonthe rstinequalitysegment andallotheritemshavesmallerprotabilities,thesecond segmentwillbeginatthelastpointon( 27 ),i.e.,thepoint ( ) ( ) Thus,wecouldobtaintherstinequalitydirectly,andtoderivetheremaininginequalities, onlypointsbetween ( ) and ,whicharelessthan points,needtobeinvestigated. Thefollowingexampleillustrateshowwecancreatethevalidinequalitiesthatwe introducedabove. Figure2-1. Graphof valuesandenvelopeinequalities. Example. ConsidertheUKPofthreeitemswithprots ,respectively, andweights ,respectively,andtheknapsackhasacapacityof15.This problemcanbeformulatedas: maximize subjectto # Z 19

PAGE 20

Theoptimalsolutionis withatotalprotof34.Notethatnoitemtypeis collectivelydominatedinthisproblem. Solvingtherststage, ,withdynamicprogramming,wend ,suchthat themaximumprotthatcanbeachievedbyusingonlythersttypeofitemsis30.Thevalid inequalitycorrespondingto( 23 )is: Figure 2-1 ,whichdepicts asafunctionof ,illustratesthat isnon-linearandisan upperboundforallfeasiblesolutionswhenconsideringonlytherstitemtype.Wecanderive theconcaveenvelopeinequalitiesforstage1accordingto( 25 ).Connectingtotalweight0 through12denes: whileconnectingtotalweight12through15denes: Fortherststage,thevalidinequalityobtainedaccordingto( 25 )iseitherredundantora duplicationofvalidinequalityobtainedaccordingto( 23 ).Wegenerateitheretoachievethe completenessoftheexample.However,whensolvingproblems,validinequalitiescorresponding to( 25 )willnotbederivedfortherststage. Aftersolvingstage2, ,wend ,andtheinequalitybasedon( 23 )is: Figure 2-2 includesinformationofthersttwoitemsandtheconcaveenvelopearedened withinequalities: " 20

PAGE 21

Figure2-2. Graphof valuesandenvelopeinequalities. Afterstage3, ,problemhasbeensolvedtooptimalitywith .The inequalitydescribedby( 23 )is: andtheconcaveenvelopeinequalitiescapturedbyFigure 2-3 are: " 2.5ComputationalResults Inthissection,severaldatasetswhichinvolvedi! erentweight-protcorrelationare generatedtodemonstratethee! ectivenessofourproposedmethod. 21

PAGE 22

Figure2-3. Graphof valuesandenvelopeinequalities. 2.5.1InstanceGeneration 2.5.1.1Randomcases Inaccordancewiththerelationshipofitems'weightsandprots,threetypesofrandom datasetsaregeneratedandtestedwithourpresentedapproach:uncorrelated,weakly correlatedandstronglycorrelated.ThistestingschemewassuggestedbyMartelloandToth [ 17 ].Inallthreecases,weightsarerandomlyselectedamongtheinterval .Inthe uncorrelatedcase(UC),protsarerandomlypickedfrom ,sothatthedatasetsof weightsandprotsareindependentofeachother.Fortheweaklycorrelatedcase(WC),prot isobtainedrandomlyintherangeof % ,where and aretwopredetermined numbers.Finally,inthestronglycorrelatedcase(SC),prot isxedas whichis linearlydependenton Inourexperiments,instanceswith and areexamined.Therange ofitems'weights variesbasedonthetotalnumberofitemstoallowmore weightchoiceswhen isbig.Specically,forallrandomcases,wesetparametersasfollows: 22

PAGE 23

,and when ,while and when .Foruncorrelatedcases, and when ,while and when .Forweaklycorrelatedcasesand stronglycorrelatedcases,wehave and when ,while and when 2.5.1.2Realisticcases Althoughtheaboverandomcases(uncorrelated,weaklycorrelated,andstrongly correlated)createpossibledatasetsfortheUKP,theydonotfrequentlyhappeninreal worldproblems,sinceitismorecommonthatanitemwithaheavierweightismorevaluable. SinhaandZoltners[ 23 ]rstintroducedarealisticscenariowhereaheavieritemyieldsmore prot.Tobeginthisdatagenerationprocess, weightsand protsareselectedrandomly fromtheirownfeasibleintervals, and ,andsortedindescending order,respectively.Theweightvalueandtheprotvalueatthesamerankbecomeapairof propertiesofanitemtype.Attheend,theorderofitemtypesisrecoveredaccordingtothe originalsequenceofweights.Hence,if ,then andweights isinrandom order. Weruntestsforrealisticcases(Real)withthesameparametersettingsastheuncorrelated randomcases. 2.5.1.3Hardcases Andonovetal.[ 18 ]describedthreehardcasesfortheUKPconcerningdominance.To constructthesedatasets,rstly,chooseweightsrandomlyintherangeof ,where and arepositiveintegersandsatisfy < < .Thensortweightsin increasingorder,andapplyoneofthefollowingthreeequationstodeterminetheprotsof 23

PAGE 24

items.Finally,rearrangeitemsaccordingtotheoriginalorderofweights. (28) { ( " ) } israndomlychosen, (29) ( " ) % israndomlychosen. (210) Hardcasessatisfying( 28 )-( 210 )aretested,respectively.When ,weset and ,andwhen ,weset and .Hardcasewith( 29 )islabeledasHard2,whilehard casewith( 210 )islabeledasHard3.Inexperiments,weuse # for and # for Duringthetestonhardcaseswithformula( 29 ),wefoundsomespecialinstancesthat CPLEXtookextremelylongtime,morethan20seconds,tosolvecomparingwithgeneral instances,wherethesolvingtimearelessthan0.1seconds.Wealsotestourmethodonthese instances. 2.5.2ResultsSummary Wetestedourproposedapproachforeachcasediscussedabove.Thedynamicprogramming algorithmwasappliedtoasubsetofthereordereditems(stages)andinequalitieswere obtainedandaddedtotheoriginalintegerprogrammingformulationtosolvetheUKP.For comparison,aclassicdynamicprogrammingapproachandabasicbranchandboundapproach wereapplied.Wetestedverandominstancesforeachcase.Allexperimentswereconducted onapersonalcomputerrunningWindows7witha2.60GHzCPUand8.0GBmemoryusing CPLEX12.6. Weincludedthefollowingmodelsinexperimentsandcomparedthecomputational e ciencyofsolvingeachofthem: dp :dynamicprogrammingapproach; bb :branchandboundapproach; cplex :formulation( 21 )solvedbyCPLEX(defaultsettings); 24

PAGE 25

protu :validinequalities( 23 )togetherwith cplex ; envu :concaveenvelopeinequalities( 25 )derivedinthelastdynamicprogrammingstage togetherwith cplex ; envuinc :concaveenvelopeinequalities( 25 ),whicharederivedthrougheverystage, addedincrementallytothestandardformulation( 21 ). Theintegernumberaftermodel protu envu and envuinc representsthenumberof dynamicprogrammingstagesexecutedinthemodel. Table2-1. Computationaltimesummary(milliseconds)for modelUCWCSCRealHard2Hard3 dp 66561313 bb 16181924148836 cplex 202620242824 protu10161512191218 protu20161614181516 protu30121613181215 envu10151313171617 envu20141312141915 envu30141311152214 envuinc10161412171617 envuinc20151311141515 envuinc30151312151617 Table2-2. Computationaltimesummary(milliseconds)for modelUCWCSCRealHard2Hard3 dp 187194189202440456 bb 3036308672585 ** cplex 342829295747 protu10272120224131 protu20252225244333 protu30252327265241 envu10212228223932 envu20252326264634 envu30242127265642 envuinc10221922213744 envuinc20222227254837 envuinc30222229245851 25

PAGE 26

Table 2-1 andTable 2-2 summarizesaveragetimeconsumptionforeachmodeloneach testingcaseexceptthehardcasewith( 28 ).Thistypeofproblemisalsoknownasthesubset sumproblemandcanbesolvedbyallmodelsintheexperimentinashorttime.Inthetable, 'indicatesthatthereareinstancesrequiringmorethan300secondstosolve.Resultsshow that dp approachisverysensitivetothesizeofproblem,boththenumberofitemsandthe capacityofknapsack.Whenthereare100itemswithrelativelysmallknapsackcapacity,ithas thebestperformanceoverallothermodels.However,whenthenumberofitemsincreases(as aresultofhowwedecidecapacityvalue,thecapacityincreasesaswell),thetimespenton ndinganoptimalsolutionincreaseddramatically.Thetestingresultsalsoshowtheadvantage of dp approach,i.e.thecomputationaltimesarestableforproblemsofsimilarsize.For bb approach,itissensitivetotheinputparameters,suchastherelationshipbetweenitems'prots andweights.Ithaslongcomputationaltimesforhardcaseswith( 29 )and( 210 ).For cplex althoughitalsotendstohavelongertimewhenproblemsizegrows,butunlike dp ,thetime increasesmoderately.Ourproposedmodelshavegoodperformanceonaverage.Formostof casestested,theysolveproblemsfasterthan bb and cplex .Andcomparedto dp ,theyareless sensitivetothesizeofproblem. Besidescomputationaltime,initialgapisanotherimportantmeasurement.Theinitial gappresentstherelativedi! erencebetweentherstintegersolutionexploredandthebest upperboundderivedwhentherstintegersolutionisfound.Asmallerinitialgapimpliesa tighterlowerboundand/oratighterupperboundhasbeenfoundtofacilitatepruningthe searchspace,eliminatingcandidatenodes.Therefore,usuallylesstimeisspentonexploring nodesofthesearchtree.Forbrevity,weonlypresenttheinitialgapof cplex andourproposed modelsinthestronglycorrelatedandrealisticcasesinFigure 2-4 .Resultsdemonstrate thedynamic-programming-basedinequalitieshelptoimprovetheinitialgapinmostof theexperimentalcases.The envuinc model,whichisclosetothe envu modelbutderives inequalitiesincrementallyforeachstage,hassimilarresultswiththatof envu 26

PAGE 27

A B C D Figure2-4. Initialgapof cplex andproposedmodelsforpartialtestingcases.A)Strongly correlatedcasewith100items.B)Stronglycorrelatedcasewith500items.C) Realisticcasewith100items.D)Realisticcasewith500items. Table2-3. Computationaltime(seconds)ofspecialinstances instancecplexprotu envu envuinc 122.450.07-5.510.07-5.470.05-2.05 242.180.10-0.110.06-0.120.07-0.09 372.840.05-0.080.07-0.210.08-0.16 4145.950.06-0.070.07-0.110.10-0.20 5188.870.23-1.870.23-1.070.15-0.24 65086.510.04-0.070.05-0.180.07-0.08 78095.910.19-0.300.25-81.660.23-26.96 Table 2-3 illustratesthesolutiontimeof cplex modelandourproposedmodelsonsolving specialinstancesfoundduringinstancegenerationofhardcases.Withvalidinequalitiesadded totheoriginalintegerprogrammingformulation,theseproblemscanbesolvedinaveryshort time. Notethatourexperimentalresultsindicatethatexaminingmorestagesofthedynamic programmingdoesnotalwaysprovidebetterperformance.Becausethereisapotential 27

PAGE 28

trade-o !betweenobtainingacomputationaladvantagefromabetterinitialsolutioncausedby trimmingthemodelwithmoredynamicprogrammingbasedinequalitiesandacomputational costoncalculatingtheseinequalities. 2.6ConcludingRemarks Inthischapter,wepresentedanewexactapproachtosolvetheunboundedknapsack problemtooptimalitybytakingadvantageofbothdynamicprogrammingandtraditional integerprogramming.Accordingtotheinformationobtainedfromthepartialexecutionof dynamicprogrammingrecursion,wederivevalidinequalities( 23 )and( 25 ),whichhelpto tightentheintegerprogrammingformulationandobtainbetterinitialboundsoftheoptimal solution.Therefore,afteraddingdynamic-programming-basedinequalities,thesearchspaceis prunedmoree"cientlyandthetimespentonexploringthesolutiontreeisreducede!ectively. 28

PAGE 29

CHAPTER3 DYNAMICPROGRAMMINGBASEDINEQUALITIESFORTHEMULTIDIMENSIONAL UNBOUNDEDKNAPSACKPROBLEM 3.1MotivationandLiteratureReview Weconsideradirectgeneralizationoftheunboundedintegerknapsackproblem(UKP), whereasetofitemtypes, { } ,andaknapsackof dimensionalresources, # Z ,aregiven.Eachtype item, ,hasprot ,andisassociatedwithaset ofattributes { } ,where indicatestheconsumptionofitem onthe -th resourceoftheknapsack.Thereisanunlimitedsupplyofeveryitemtype .Theknapsack providesnomorethan unitsofresource .Theobjectiveistodeterminethenumberof copiesofeachtypeofitemstobepackedintotheknapsackwithrespecttoallresource capacitiestoyieldthegreatestprot.Theresultingproblemisknownasthemultidimensional unboundedknapsackproblemor -dimensionalunboundedknapsackproblem(d-UKP),which canbeformulatedas: maximize subjectto # Z (31) whereallparameters , and ,for areconstantintegers. Furthermore,toavoidtrivialsolutions,weassumethatprot isastrictlypositiveinteger, isallowedtobezeroaslongas > ,and > foratleastone value.Also, toensureanon-emptyfeasibleregion, $ forall and .When ,theproblemisreducedtothesingleconstrainedunboundedknapsackwhichwestudied inChapter 2 .Therefore,ingeneralweconsider $ forthed-UKP. Thed-UKPiswellknownasastronglyNP-hardintegerprogrammingproblem[ 24 25 ]. Inmanyliterature,itisnotconsideredapropermemberoftheknapsackfamily,butaspecial caseofgeneralintegerprogrammingproblemwhereallconstantsarenon-negativeandall constraintshavethesameinequalitysigns.Consequently,onestraightforwardideatosolvethe 29

PAGE 30

d-UKPistoapplyanyintegerprogrammingapproachsuchasbrand-and-boundanddynamic programming.PirkulandNarasimhan[ 26 ]generalizedtheoptimaltheoryforthed-KPintoa branch-and-boundprocedurefortheexactsolutionofthed-UKPwiththexedintegervariable upperbounds.Ozden[ 27 ]combinedthetechniquesofpreprocessingandastategeneration schemetodevelopanexactalgorithmforthed-UKPwithfewresourceconstraints.However, thecomputationalcomplexityincreasessignicantlyasthenumberofconstraintsincreases. Becauseofthehighcomplexityofsolvingthed-UKPtooptimality,severalheuristics andapproximationapproacheswereproposed.Chandraetal.[ 28 ]brieydescribeda polynomial-timesolvable % # -approximationalgorithmforthed-UKPwiththefact thatatmost decisionvariablesintheLPrelaxationofthed-UKPwillbenon-zeros.Akcay etal.[ 29 ]denedanewgreedyfunctionusingthee!ectivecapacityandadjustedsearching stepsizeintheirgreedyheuristicmethodforthed-UKP.MeyeraufderHeide[ 30 ]proposeda polynomiallinearsearchalgorithminthecontextofrandomaccessmachines.Therstprimal gradientmethodfortheapproximatesolutionofthed-UKPwasproposedbyKochenberger [ 31 ]. Comparedwiththeintensivestudiesonthed-KPwhichisconsideredastheclosest siblingofthed-UKP,literatureonsolutionapproachestothed-UKPseemtobeinadequate. Tohandlethisproblemwiththenotionofaknapsack,weextendedandmodiedour proposedapproachtothesingleconstraintunboundknapsackproblem.Computational resultsdemonstrateitse!ectiveness. Therestofthischapterisorganizedasfollows.Section 3.2 discussesthedominance conceptind-UKPtogetherwithnumericalstudies.InSection 3.3 ,weproposeanewdynamic programmingapproachwithlistswhichwillbeincorporatedintotheexactsolutionapproach forthed-UKP.InSection 3.4 ,thevalidinequalitiesderivationprocessisdescribed.Section 3.5 presentsthecomputationalresults.Finally,WeconcludeinSection 3.6 30

PAGE 31

3.2DominanceRelations TheconceptofdominancerelationswasoriginallydevelopedfortheUKP.However,it couldbegeneralizedtothed-UKPeasilywiththeconsiderationof attributemeasurements, insteadofasingleweightmeasurementintheUKP,foreachitemtype.Wesummarizethe dominancerelationsforthed-UKPasfollows: 1. Itemtype simplydominates type ,if $ and & 2. Itemtype multiplydominates type ,if $ and & where # Z 3. Asetofitemtypes collectivelydominates type ,if ! $ and ! & ,where # Z 4. Asetofitemtypes thresholddominates type ,if ! $ and ! " & ,where # Z Asthenumberofconstraints intheproblemincreases,thee" ciencyandbenetsof examiningitems'dominancerelationsdecrease,becausemoreattributemeasurementsneed tobeveriedtodeterminewhethertheirrelationsatisesallrequirementsofthedominance relation,whichmaybetimeconsuming.Takethedetectionofsimplydominancerelationasan example.Thecomputationaltimeofcomparingattributes'valuesamongallpairsofitemsis ,whichincreaseslinearlywiththenumberofconstraints.Thisisalsotruefordetecting otherdominancerelations.Ingeneral,thesequenceoftimespentoncheckingdominance fromshortesttolongestissimplydominants,multiplydominance,collectivelydominanceand thresholddominance.What'smore,therewillbelessdominancerelationsamongitemsdue tomoredominancerequirementstoreach.Inthecaseofrandomlygeneratedcoe"cients, fortworandomlychosenitems,theprobabilityofonesimplydominatingtheotheris whichdecreasesexponentiallywithincreasing .Moreover,anumericaltestoninstances wherecoe" cientsarerandomlygeneratedhasbeenconductedtoinvestigatethepercentageof simplydominateditemswithdi! erentnumbersofconstraints.Inthetest,wehave # { } # { } and # { } .100instancesare 31

PAGE 32

generatedforeach value.TheresultsareshowninTable 3-1 .Iftherearetwoconstraints, morethan80%itemsaredominatedandcouldberemovedfromlaterconsideration.Thus, thepre-processingofdominancerelationsise"cientinshrinkingthesizeofproblem.Butas increases,thepercentageofsimplydominateditemsreduces,hencethesizeofproblemcould notbereducedmuchbypre-xingvariablevaluesofdominateditemstozero.Whenaproblem hasmorethan15constraints,pre-processingisnotbenecialatall. Table3-1. Analysisofsimplydominancerelationinrandomlygeneratedcases 95.887.242.77.30.20.0 99.396.467.611.40.70.0 Ingeneral,withlarge ,moretimeisspentondominancedetection,butlessdominated itemscouldbefoundandremovedfromproblem.Therefore,withd-UKP,weonlydetect simplydominancerelationamongitemsforproblemswithnomorethan10constraints. 3.3DynamicProgrammingApproach 3.3.1FundamentalDynamicProgrammingApproach WecanextendthedynamicprogrammingapproachonUKPtod-UKPeasily.Eachitem isregardedasastage,andeachsetofconsumedresourcesinalldimensionsofaknapsack isregardedasastate.Anintermediatestage problemforthed-UKPcanbedescribedas follows: maximize subjectto # Z (32) 32

PAGE 33

ThefollowinggivesthestraightforwardextensionoftherecursionfortheUKP,obtaining theoptimalobjectivevaluetothestate inthestage : # $ $ $ $ $ $ % $ $ $ $ $ $ & if < forsome # { } maximum { % % } if $ forall # { } (33) FortheUKP,wellupa tabletostorestatevalues.Throughdynamicprogramming recursion( 33 ),a dimensionaltablewithsidelengths , isused.Thus,the timeandspacedcomplexityis ,whichiscomputationallyexpensiveevenfor medium-scaleproblemswithasmallnumberofconstraints.Todealwiththisissue,dynamic programmingwithlistrepresentationisimplemented. 3.3.2DynamicProgrammingApproachwithLists Becausethedynamicprogrammingfunctionofthed-UKPisastepfunction,similar withthatoftheUKP,alistcouldbeutilizedtosolelycontainstateswherethedynamic programmingfunctionvaluechanges.Thisideawasappliedforsingleconstraint0-1knapsack problemsinHorowitzandSahni[ 32 ]andmulti-dimensional0-1knapsackproblemsinBalevet al.[ 33 ].Followingisourproposedgeneralizationofdynamicprogrammingwithlistsforthe d-UKPwithaccommodationfortheinequalitiesderivationprocess. Duringthedynamicprogrammingprocess,alistofstateswillbecreatedforeachstage. Everystate representsafeasiblesolution,where iscapacity consumptionin recoursedimensionsand isthecorrespondingobjectivevalue.For computationale" ciencyandeaseingeneratingvalidinequalitiesinthenextstep,states arerequiredtobestoredintheorderofincreasingobjectivevaluesinlists. Therstlistofstates, ,isformedwithallfeasiblenon-negativeintegermultipliersof itemtype ,i.e.: { ' } ( / ) 33

PAGE 34

where iscolumnvector Forotherstages,thegenerationof $ isbasedonstatesin .Givenastate in ,newstatesaddedto are { ' } wheremultiplier isanynon-negativeintegernumberandkeepingnewstatefeasible. However,withthisstategenerationscheme,anorderedlistmaynotbeobtaineddirectly. Tohaveanorderedlist,asortingprocessisrequiredwhichwillcost time,where isthelengthofthelist.Toachievingcomputationale" ciency,insteadofsortingafterthe listbeingcreated,wesuggestkeepinganorderedlistatalltimesduringstategenerationwith assistanceoftwopointers and indicatesthestatethatcurrentlyisinusetocreatea newstate,while pointstothelocationwherenewlygeneratedstateshouldbeinsertedinto thelistregardingitsprot.Bothpointersareinitializedatthebeginningofthelist which copiesallstatesin .Thenewstategeneratedis ,where isthe statepointedby .Sincestatesinthelistareorderedaccordingtotheirprots,tondthe correctplacetoinsertnewstate, needstomovedownthroughthelistuntilndingtherst statewithahigherobjectivevaluecomparedwiththenewstate'sprot.Thenthenewstateis insertedbeforethestatelabeledby .Afterwards,moving tothenextstateandrepeating thisprocedureuntil isattheendoflistandnomorefeasiblestatescouldbeproduced. Becausebothpointersonlygothroughthelistonce,thecomplexityofbuildinguplist is Similartotheconceptofdominancerelationsamongdi!erentitemtypes,therearestates consideredtobenon-promisingduringdynamicprogrammingprocess,i.e.state isa non-promisingstateif,forsomeotherstate $ and .Thesestatesdonot leadtoanoptimalsolution,thereforetheycouldberemovedfromthelisttokeeplistsshort andreducethecomputationaltimeandmemoryspacerequirement.Becausethedominance relationislesscommontoappearbetweenstateswithbigprotdi!erences,examiningall 34

PAGE 35

statesinthelistisredundant.Therefore,wecouldpre-xabandwidth andonlycompare newstateswithstatesthatarewithin stepsawaywheninsertinganewstateinthelist. Thedynamicprogrammingapproachwithlistsissummarizedwithpseudo-codeasfollows: Algorithm3.1. Dynamicprogrammingwithlists 1: createtherstlist: { ' } ( / ) 2: for upto do 3: copystatesfrom to 4: initializepointers: % 5: while < | | do 6: 7: if in then 8: while do 9: 10: endwhile 11: insertnewstate at location 12: examinedominatedstates 13: endif 14: endwhile 15: endfor 3.4ValidInequalitiesDerivation WeproposedtwotypesofvalidinequalitiesfortheUKPinChapter 2 .Bothofthemcould beappliedtothed-UKP. First,toderiveinequality( 23 ),theupperboundoftotalprotsfortherst typesof itemsisneeded.Sincewehaveorderedlistsofstates,theupperboundistheprotassociated withthelaststateof Second,tocreateinequality( 25 )forthed-UKP,theupperconcaveenvelopeofpoints in dimensionalspaceneedstobecalculatedinsteadofthatin2dimensionalspace 35

PAGE 36

fortheUKP.Eachattributerepresentsonedimensionandtotalprotrepresentoneadditional dimension.Figure 3-1 and 3-2 showpointsandtheupperconcaveenvelopeforaproblemwith twoconstraints. Figure3-1. Feasiblestatesfora2-dknapsackproblem. Figure3-2. UpperconcaveenvelopeofpointsinFigure 3-1 Obtainingtheexactupperconcaveenvelopeasvalidinequalitiesiscomputationally expensive,evenforlow-dimensionalproblems.Tosimplifytheprocess,singleattribute dimensionwillbeconsideredatatime.Inotherwords,theexactconvexenvelopeisprojected totheplanespannedbytheselectedattributeandthetotalprot(seeFigure 3-3 ).Then inequalitiescouldbeconstructedwiththesameprocessasthatfortheUKP.Thisisrepeated forallotherattributes. 36

PAGE 37

Figure3-3. Projectionofpointsto2-dplaneandtheupperconcaveenvelope. Example. Considerthefollowingthree-variable2-UKP: maximize subjectto " # Z Afterthedynamicprogrammingprocessforstage2,weobtainalistofpairedfeasible solutionsanditsobjectivevalueasillustratedinFigure 3-1 .Thelaststateinthelistis ,thustheprotupperboundeddenedforthersttwoitemsis: Togeneratetheothertypeofinequalities,insteadofcalculatingtheexactconcaveenvelopof allstatesshowninFigure 3-2 ,weprojectstatestooneattributedimensionatatime.Figure 3-3 displaystheprojectionoffeasiblesolutionsontotheplaneoftherstattributeandthe 37

PAGE 38

totalprot.Theupperconcaveenvelopofprojectedpointsisdenedwithinequalities: " 3.5ComputationalResults Ourapproachistestedonrandomcasesofthed-UKPwith100and500itemtypesand dimension2,3,5,10and15,respectively.Protandweightsofeachitemtypeareuniformly generatedfromrange .Capacitiesforallattributesarexedto300.Allexperiments wereconductedonapersonalcomputerrunningWindows7witha2.60GHzCPUand8.0GB memoryusingCPLEX12.6.Averageresultsof5trialsareshowninTable 3-2 .AndFigure 3-4 andFigure 3-5 provideamoreclearviewoftheimprovementofourapproachincomputational timeandinitialgapmeasurement. Table3-2. Summaryofexperimentsforthed-UKP ndinitgap(%)time(ms) nodes cplexourscplexourscplexours 10022.321.9414731.670 0 34.163.90165.3331.330 0 59.178.2219157220 1024.3414.40287.67188.33771.67747.33 1520.8719.84410.33322.333833.33295 50021.460.84175540 0 33.091.29208540 0 58.155.59180730 0 1027.114.3714451006113959520 1525.6221.18171813581400610883 Resultsindicatethatourproposedmethodforthed-UKPhelpstoenhancee" ciency byreducinginitialgapsandcomputationaltimesformedium-sizeproblems.However,as thenumberofconstraintsgrows,theadvantageofthismethodislessened.First,aswe mentionedintheprevioussection,when becomeslarger,feweritemsaredominated byothers.Therefore,fewervariablevaluescouldbexedduringthepre-processingstep. 38

PAGE 39

Figure3-4. Runtimeforthed-UKP. Figure3-5. Initialgapforthed-UKP. Second,dynamicprogrammingtakesmoretimetobeimplemented.Third,thesecondtype ofinequalitiesforthed-UKPisobtainedbyrelaxing % dimensionsoftheexactenvelope, hencetheseinequalitiesbecomelessaccurateindeningthefeasibleregionwhen isalarge number. 3.6ConcludingRemarks Inthischapter,weextendedthestudyinChapter 2 tothemultidimensionalunbounded knapsackproblemwhichisangeneralizationoftheunboundedknapsackproblem.Twotypes ofvalidinequalities,protinequalitiesandenvelopinequalities,weregeneratedaccordingto theintermediatesolutionsofamodieddynamicprogramming.Theexperimentalresults demonstratedthee! ectivenessofproposedapproachonproblemsupto500itemsand15 constraints. 39

PAGE 40

CHAPTER4 EXACTDYNAMICPROGRAMMINGAPPROACHFORTHEROBUSTKNAPSACK PROBLEMWITHPROFITUNCERTAINTY 4.1MotivationandLiteratureReview Inpractice,manydecisionprocessescanbeabstractedascombinatorialproblems,making adecisionbetweentwoalternativechoices.Theclassicknapsackproblem(KP)isoneofthe mostimportantandfundamentalproblemincombinatorialoptimization.TheKPcanbe denedasfollows:givenaset of items,eachofthemhaspositiveprot andpositive weight .Aknapsackofcapacity istobelledwiththeitems.Allparametersareinteger. Theobjectiveistopickasubsetofitemsthatprovidesthehighesttotalprotwhiletheirtotal weightdoesnotexceedcapacity .TheKPiscommonlyformulatedasthefollowinginteger linearprogramming(ILP): maximize subjectto # { } (41) TheKPisawell-knownNP-hardproblem,andithasbeenstudiedformorethana century.Basicintegerprogrammingapproachessuchasbranch-and-boundanddynamic programmingwereimplementedontothisproblem.What'smore,advancedconcepts,such asthecoreconceptandcoverinequality,weredevelopedtotackletheKPmoree"ciently.In practice,evenlargescaleKPscanbesolvedtooptimalitywithinashorttime. TheproblembeingstudiedinthischapterisavariantoftheKP,whereuncertaintyon inputdataisconsidered:theexactprotofeachitemisnotknowninadvance,butbelongs toacertainrange isthenominalprotofitem ,andatmost itemscanhave protsdeviatefromtheirnominalprotstoanyarbitraryvalueintherange.Constant is usedtoindicatethelevelofconservatismandisassumedtobeapositiveintegerlessthan .Notethat,if ,thisbecomestheclassicKP.Foreachitem,theweight isa 40

PAGE 41

determinatevalue,therefore,thefeasibleregionofthisproblemisthesameasthatoftheKP. Theobjectivebecomestondthemostrobustsolution,asetofitemsthattsthecapacity ofknapsackandyieldsthemaximumprotintheworstcasescenario.Forafeasibleitemset + ,theworsttotalprotiscalculatedas: minimize ! \ ! # | | $ Manyreal-lifeproblemsrequiretakinginputdatauncertaintyintoconsideration.One applicationoftheaboverobustproblemisthecapitalbudgetingproblem.Thereareseveral investmentprojectstobeconsideredsubjecttoabudgetconstraint.Theexpenseofeach projectiseasytoestimate.However,therevenuewhichisusuallymeasuredinpresentvalueis unclear,sinceitinvolvesbenetearnedinthefuture.Inthiscase,wecouldevaluateapossible revenueintervalforeachprojectandmodeltheproblemasarobustknapsackproblem,helping obtainthebestinvestmentdecision. Duringrecentdecades,problemswithuncertaintyhavedrawnmuchattentionfrom scientists.Mulveyetal.[ 34 ]proposedamethodthatincorporatesascenario-baseddescription ofproblemdataintogoalprogrammingformulations.Amodelthatsolvesuncertaintyinlinear programmingwasintroducedbyBen-TalandNemirovski[ 35 ].Thesolutionofthemodel needstobefeasiblewithprobabilityatleast % # .BertsimasandSim[ 36 ]introduceda newapproachthato! erscontrolonthedegreeofconservatismwithaspeciednumber Theyprovidedsolutionson0-1discreteproblemsandapplieditontheknapsackproblem. Poss[ 37 ]introducedanovelmodelofrobustcombinatorialoptimizationwithvariable budgeteduncertaintyandconductedexperimentsshowingthattheirmodelreducesthe priceofrobustnesscomparingtothebudgetuncertaintymodel.Foradetailedsummaryof theoreticalapproachesandapplications,refertoBertsimasetal.[ 38 ]. Inthedomainofknapsackproblems,multiplevariantsofproblemsunderdatauncertainty havebeenproposed.KlopfensteinandNace[ 39 ]addressedthechance-constrainedknapsack problemandbuiltthetheoreticallinkbetweenthisproblemwiththerobustknapsackproblem 41

PAGE 42

withweightuncertainty.Validinequalitiestothecasewhereknapsackitemshaveuncertain weightswereproposedbythesameauthor[ 40 ].MonaciandPferschy[ 41 ]proposedtherobust knapsackproblemwithweightuncertaintyandanalyzedtheworst-caseperformanceratiowith respecttotheoptimalsolutionvalueoftheclassicKP.Theyalsoextendedtheirresearchonto acontinuousversionoftherobustknapsackproblem.Adynamicprogrammingapproachwith aspacereductiontechniquewaspresentedbyMonacietal.[ 42 ]forthisproblem.Another attractivevariantistherecoverableknapsackproblem(seeBusingetal.[ 43 44 ])inwhich bothweightsandprotsaresubjecttouncertainty.Theaimofthistypeofproblemistond asolutionthatisnotnecessarytobefeasibleatrst,butwithcertainlegalmoves,feasibility wouldbeachieved.Legalmovesincludetheremovalofatmost itemsandsometimesadding newitems. Althoughtheproblemwestudiesisaspecictypeoftherobustknapsackproblem,we usetheabbreviationRKPtoindicatethisproblemthroughoutthischapter.Thechapteris organizedasfollows.InSection 4.2 ,weproposeadynamicprogrammingalgorithmwhich imitatestheclassicdynamicprogrammingrecursion(seeBellman[ 14 ])butmodiedtothe worstprotscenario.Also,twoapproachestoobtaintheoptimalsolutionsetaregivenwith consideringreducingtheamountofspace,sincewhenthesizeofproblemgrows,itmaycause computationalissues.WereviewotherexactalgorithmsoftheRKPinSection 4.3 .Section 4.4 describesourexperimentaldesignandshowsthecomputationalresults.Finally,weextendthe dynamicprogrammingapproachtothegeneralinstanceoftheRKPinSection 4.5 4.2DynamicProgrammingApproach Inthissection,weproposeadynamicprogrammingapproachtosolvetheRKPto optimality.Moreover,tocontrolthememoryspaceconsumedtocomputetheoptimalsolution set,astoragesavingtechniqueisimplemented. 4.2.1DynamicProgrammingRecursion Monacietal.[ 42 ]providedadynamicprogrammingalgorithmfortherobustknapsack probleminwhichtheweightofeachitembelongstoagivenrange.Weextendtheiralgorithm 42

PAGE 43

totherobustknapsackproblemwithuncertaintyintheprotsoftheitems,thecoe"cientsof theobjectivefunctioninthemathematicalformulation. Let bethedi!erencebetweenthenominalprotvalueandtheworstprotvalueof item ,i.e. % .Withoutlossofgenerality,weassumethatitemsareorderedby non-increasing values,i.e. $ $ $ .Itemsaresequentiallyconsideredtobe selectedduringthedynamicprogrammingprocess.Forasubsetofitems + represents theindexofthe -thitemin if | | $ ,otherwise istheindexofthelastitemin .The RKPresultrevealstheworstcasebehavior.Therecursiondesignedforthisproblemisbasedon thefollowingobservation. Lemma4. Asubsetofitems isselected,toobtaintherobustprotoftheknapsack,items withindexnumberslessthan contributetheirworstprot andotheritemscontribute theirnominalprots Proof. TheRKPconsiderstheworstcasescenario.Thus,theobjectivefunctionvalueofa feasiblesubset iscalculatedas: ! % maximum ! # | | $ Sinceitemswithsmallerindiceshavelarger values,toachievethehighestvalueofthe secondtermintheaboveequation,itemset shouldcontainitemswithindexnumberless than .Thentheequationfor canberewrittenas: ! % ! $ ! $ ! > Therefore,Lemma 4 holdsunderourassumptionthatitemsareorderedbynon-increasing values. Underourassumption,worstprotsareaddedtotheobjectivefunctionfortherst itemsselected,whilenominalprotsareaddedforthefollowingitems.Comparedtothe 43

PAGE 44

well-knownBellmanrecursion[ 14 ]developedfortheclassic0-1KP,anextraindex isneeded torecordthenumberofitemshavingbeenselectedwiththeirworstprots.Let denotethebestprotvalueachievedwhenknapsackcapacityis ,itemset { } isunder considerationand itemsarepackedwiththeirworstprotvalues.Thedynamicprogramming recursionoftheRKPisdenedasthefollowingformulas: # $ $ % $ $ & % if < maximum { % % % % } if $ for (42) # $ $ % $ $ & % if < maximum { % % % } if $ for (43) Tostarttherecursion, isinitializedtobe %, ,for and .Whenlessthan itemshavebeenpackedintotheknapsack,equation ( 42 )isusedtodecidedwhetheritem shouldbechosentoobtainthetemporarybestprot foreverycapacityvalues.Whenatleast itemshavebeenpacked,equation( 43 )isused tomakethedecisionfornewitems.Inboth( 42 )and( 43 ),therstterminbracesgives thebestvaluewithoutselectingthenewitem ,andthesecondtermisthebestvaluewhen includingitem inthesolution.TheoptimalsolutionoftheRKPcanbefoundwith: % maximum { } (44) andthecapacityoccupiedis % .Theoptimalsolutionistheonethatachievesthehighest protamongallsubproblemswhere itemsareconsideredandthenumberofitemspacked withreducedprotsarenomorethan .Note,if < itemsarewithreducedprots,italso meansthatonly itemsareselectedinthesolution. 44

PAGE 45

Figure 4-1 illustratesthedynamicprogrammingprocessofasimpleRKP,where itemshavenominalprots ,reducedprots ,andweights ,conservatismlevel andknapsackcapacity .Theseitemshave beensortedaccordingtothe values.Thegureshowsadirectednetworkthatwecan onlymovefromanodeatlefttoanodeatrightifthereisanarcinbetween.Nodesinthe graphrepresentfeasiblesolutionsoftheRKPinstance,andarcsbetweentwonodesarethe extraprotsobtainedbymovingtothenewnode.Therearethreetypesofarcsinthegraph. Thedotedarcsleadingtonode havezeroprot,correspondingtothecasethatitem isnotchosenandthereisnochangein and .Thiscorrespondstotherstcomponentin recursion( 42 )and( 43 ).Thethinsolidarcsfromnodes % % % tonodes haveprot .Thistypeofarconlyappearswhen ,i.e.,thenumberofitemsthatchange toreducedprotsdoesnotreachthelimit.Thiscorrespondstothesecondcomponentof recursion( 42 ).Thelasttypeofarc,athickarc,hasnominalprot andisbetweennodes % % and wherenomoreitemswithreducedprotscouldbeselected.This correspondstothesecondcomponentofrecursion( 43 ).Thereexistsatleastonedirected pathfromtheoriginnode toeachnodeinthisnetwork.Takenode for example,toreachitfromtheorigintherearetwopaths.Iftheprotofanarcistreatedasthe lengthofthearc,thenthetotalprotcollectedatanodeisthemaximumlengthofpathfrom theoriginnode tothatnode.Therefore,theRKPcouldbetransformedintoalongest pathproblem,i.e.ndingthelongestpathfromorigintothedummynode .Sinceadotted archasprotzero,thisisequivalenttondingthelargestprotamongnodes atthe laststage,where and -correspondingto( 44 ). Tondthesolutionset,wecantracebackfromthestatewithoptimalsolutionvalue. Thedynamicprogrammingapproachisimplementedbyiteratingonall and valuesand storingoptimalprotsforeverystate .Thus,thecomputationalcomplexitiesofboth timeandspaceare 45

PAGE 46

Figure4-1. IllustrationoftheDPrecursion. 4.2.2StorageReductioninDynamicProgramming Ahighstoragerequirementisoneofthemaindrawbacksthatpreventsdynamic programmingapproachfrombeingimplementedinlarge-scaleproblems.Thesolutionof eachstateneedstobestoredinordertocalculatethefollowingstates'valuesandtraceback forthesolutionset.Notethatinrecursion( 42 )and( 43 ),thebestsolutionvaluesofstates where areonlyusedincalculatingvaluesforstateswith .Therefore,when iteratingonanew value,thebestvalueintheoldstoragecanbeupdatedwiththenewone. Inthiscase,only spaceisrequired.Tomakesurethateachitemisselectednomore thanonce,thealgorithmiteratesonthedecreasingorderof and .Thepseudocodeisshown inAlgorithm 4.1 .IfweonlywanttondtheoptimalsolutionvaluefortheRKP,thenweare nished,withanalgorithmoftimecomplexity )andspacecomplexity .Ifwe needtocomputetheoptimalsolutionset,anarrayofsize couldbeassignedtoeach state.Andinthisway,thealgorithmwillstillcost storagespace. Algorithm4.1. RKP Value 1: for upto do 2: for upto do 3: -%, 4: endfor 46

PAGE 47

5: endfor 6: 7: for upto | | do 8: for downto do 9: maximum { % } 10: for downto do 11: maximum { % % } 12: endfor 13: endfor 14: endfor 15: return maximum Toreducethestoragerequirementofdynamicprogramming,Kellereretal.[ 45 ]developed arecursiveapproachwhichisbasedonthedivideandconquerstrategyforaspecialtypeofthe subsetsumproblem.Afterwards,ageneralizationofthisideawaspresentedbyPferschy[ 46 ] fortheKP. Theoverallstrategyispartitioningthecurrentitemset intotwodisjointedsetswith equivalentsizes,i.e. | | | | ,aslongasthecurrentproblemhasasizeofmorethanone item.Andtwosubproblemswith and aresolved.Theoptimalsolutionsetisobtainedby collectingsolutionsetsofproblemswithonlyoneiteminvolved.Inordertoapplythisstrategy ontotheRKP,acounter isneededateachstate torecordthenumber ofitemsthatarefromset andareinthesolutionsetof .Let % bethecounter valueassociatedwiththeoptimalsolution % .If % < ,thenthecurrentproblemisdivided intoanexactk-itemknapsackproblem(E-kKP)withitemsin andaRKPwithitemsin If % $ ,thenthecurrentproblemcanbepartitionedintoaRKPwithitemset andaKP withitemset .Thefollowingobservationensuresthatourpartitionschemewillsolvethe originalRKPtooptimality.ThedetailedalgorithmisdescribedinAlgorithms 4.2 and 4.3 47

PAGE 48

Lemma5. If % < ,thentheoptimalsolutionsetoftheRKPconsistsofthesolutionsetof anE-kKPwithprot andcardinalitybound % on andthesolutionsetofaRKPwith robustlimit % % on .If % $ ,thentheoptimalsolutionsetoftheRKPconsistsofthe solutionsetofaRKPwithrobustlimit on andthesolutionsetofaKPwithprotvalues on Proof. Sinceitemsin havesmallerindexthanitemsin ,togetherwithLemma 4 ,itis easytoverifythecorrectnessofthislemma.Let denotetheindexofthe -thitempackedin knapsack.If % < ,then # ,and % itemspackedin arewiththeirreducedprots. For ,therearestill( % % )itemscanbeselectedwiththeirworstprotsanditisstilla RKP.Note,inthiscase,thereisachancethatlessthan itemsareselectedintheoptimal solution.If % $ # ,onlytherst items,whichareallin ,addtheirworstprots. ThisbecomesanewRKPwithreducedsize.Itemsin willbepackedwithnominalprots Algorithm4.2. Solve RKP 1: callRKP Value andobtain % % ,and % 2: % -. 3: callRecursion % % % 4: return % and % Algorithm4.3. Recursion % % % 1: if | | then 2: if % > then 3: add to % 4: endif 5: return 6: endif 7: partition into ( | | / ) and ( | | / ) | | 48

PAGE 49

8: if % < then 9: solveE-kKPwithparameters % % ,obtainingarray % 10: callRKP Value % % % ,obtainingarray % 11: nd suchthat % and % % % 12: addthesolutionsetofE-kKPfor % to % 13: let % bethecountervalueassociatedto % 14: callRecursion % % % % 15: else 16: callRKP Value % ,obtainingarray % 17: solveKPwithparameters % ,obtainingarray % 18: nd suchthat % and % % % 19: addthesolutionsetofKPfor % to % 20: let % bethecountervalueassociatedto % 21: callRecursion % % 22: endif 23: return InAlgorithm 4.3 ,besidestheRKP,weutilizetwospecialknapsackproblems.Oneof themistheclassic0-1knapsackproblem.ThesolutionsetofKPcanbecomputedin timeand spacewiththestandarddynamicprogrammingrecursion,andin time and spacewithrecursionstrategy.Theotherproblemistheexactk-itemknapsack problemwhichrequirestoselectexactly itemstolltheknapsackandtheadditional constraint( 45 )isaddedtoclassicKP. (45) TheE-kKPcanbesolvedoptimallyin timeandspacewithnominaldynamic programming.Capraraetal.[ 47 ]provedthatthesolutionspacerequirementofE-kKP canbeimprovedto incorporatingarecursionstrategy.InouralgorithmfortheRKP, 49

PAGE 50

wewillapplytheseadvancedalgorithmswithrecursiontosolvetheE-kKPinline 9 andKPin line 17 inthealgorithm 4.3 tokeeptheleastspaceconsumption. Theorem4.1. Algorithm 4.2 ,adynamicprogrammingalgorithmwithaspacereduction technique,computestheoptimalsolutionvalueandtheoptimalsolutionsetoftheRKPin timeand space. Proof. Thetimecomplexityofthisalgorithmcontainstwomainparts.First,callthe RKP Valuefunction(Algorithm 4.1 )toobtaintheoptimalvalueofaRKPwithparameters and whichconsumes time.Second,buildtheoptimalsolutionsetbycalling theRecursionfunction(Algorithm 4.3 ).ForeachcalltotheRecursion( % % % ),itwill triggeraRKP Valuefunctionwhichrunsin | | / % time.If % < ,wewillsolvean E-kKPintime % | | / % whichisdominatedby | | / % .Otherwise,if % $ ,we willexecuteaKPin | | / % time,whichisdominatedby | | / % aswell.Ineach Recursionfunction,wegothroughtwoarraystondthe and values,where % whichcosts time.Overall,thetimecomplexityis | | % Insummary,therecursionprocesscanbedepictedbyabinaryrootedtreewithsome relaxationmentionedabove(seeFigure 4-2 ).Inthegraph,eachnodeislabeledwith % value intheRecursionfunction.Duringtheentireprocess,thetotalcapacityisunchanged,i.e. % | | Therefore,thetimeconsumptionforAlgorithm 4.2 is: | | | | | | / | | whichisin Toprovethespaceconsumption,wenoticethatinbothRKP Valuealgorithmand Recursionalgorithmtableswithsizenomorethan arelledandallspaceissetfree thereafter.Thus,weprovedthatAlgorithm 4.2 runsin space. 50

PAGE 51

Figure4-2. AbinarytreerepresentingtherecursionstructureofAlgorithm 4.3 Now,letusconsiderthedynamicprogrammingrecursiveformulas( 42 )and( 43 )again. Theindex isimportantsinceinthesolution,itemwhoseindexissmallerthan ,hasworst prot addedtotheknapsack.Whileitemwithlargerindexispackedwithnominalprot.In thecasethatlessthan itemsareselected,allitemsareconsideredwiththeirworstprots. Basedontheseobservations,weproposethefollowinglemma. Lemma6. Ifthenumberofitemsselectedintheoptimalsolutionisnolessthan ,the optimalsolutionsetconsistsofthesolutionsetofanE-kKPwithparameter and reducedprot onitemset { } andthesolutionofaKPwithnominalprot on itemset { } Iflessthan itemsareintheoptimalsolution,thentheoptimalsolutionsetisthesame asthesolutionsetofaKPwithreducedprotsonallitems. Proof. Therststatementiscorrectsinceitsimplyfollowstheprocedureofthedynamic programmingrecursion. Wewillprovethesecondstatementbycontradiction.First,theoptimalsolution % % % oftheRKPisafeasiblesolutionoftheKPconstructed.Assumeabettersolution 51

PAGE 52

existsfortheKP,i.e. % < If " isafeasiblesolutionoftheoriginalRKPwithabetterobjectivefunctionvalue. Thiscontradictsthefactthat % istheoptimalsolutionvalue.If > isstilla feasiblesolutionoftheRKPwith > % Thisisacontradiction. Toobtain ,asinglestorageisassociatedtoeachstate torecordtheindex ofitempackedduringRKP Valuealgorithm.Let % bethe valueassociatedwithoptimal solution.Iftherearelessthan itemsintheoptimalsolution,set % % .Besidesalgorithm Recursion,Algorithm 4.4 providesanotherapproachtondtheoptimalsolutionsetbasedon Lemma 6 Algorithm4.4. Separation % % % 1: if % < then 2: solveKPwithparameters % 3: addthesolutionsetofKPto % 4: else 5: solveanE-kKPwithitemset { % } andparameters % ,obtainingarray % 6: solveKPwithitemset { % } andparameters % ,obtainingarray % 7: nd suchthat % and % % % 8: addthesolutionsetofE-kKPfor % to % 9: addthesolutionsetofKPfor % to % 10: endif 52

PAGE 53

11: return BysubstitutingRecursionfunctionwithSeparationfunctioninline 3 ofAlgorithm 4.2 ,we presentanewapproachtondtheoptimalsolutionsetoftheRKP. Theorem4.2. ThedynamicprogrammingalgorithmwithSeparationfunctionsolvestheRKP tooptimalitywithtimecomplexity andspacecomplexity Proof. ThemosttimeconsumingstepistocallRKP Value,whichtakes .TheKPand theE-kKPcanbesolvedin and ,respectively,withtheadvantageofrecursion strategy.TheRKP Valuefunctionconsumes space.AndtheKPandtheE-kKP takeup and storage,respectively. Therefore,thisdynamicprogrammingalgorithmsolvestheRKPin timeand space. 4.3OtherExactApproaches 4.3.1AMILPFormulation BertsimasandSim[ 36 ]proposedawell-knownapproachtoformulatetherobust optimizationproblemasamixedintegerlinearprogramming(MILP)problemwhilederivingthe dualproblemoftherobustcomponent.WhenapplyingthisapproachtotheRKP,theMILP formulationisobtainedinthefollowingsteps. First,theRKPcanbeformulatedasamax-minproblemasshownin( 46 ),i.e.weare tryingtomaximizethetotalprotintheworstcasescenario. maximize subjectto # { } (46) 53

PAGE 54

where isthecorrespondingrobustminimizationproblem,whichisdenedas: minimize % subjectto # { } (47) Inthecontextofrobustoptimization,( 46 )isreferredtoastherobustcounterpartof thenominalproblem( 41 ).Then,sincethecoe"cientmatrixofconstraintsinMINP(X)is totallyunimodular,theoptimalsolutionoflinearrelaxationoftheMINP(X)istheoptimal solutionofMINP(X).Therefore,bytakingthedualofthelinearrelaxationofMINP(X),robust counterpart( 46 )couldbereformulatedasthefollowingMILPproblem: maximize % $ % % subjectto % $ % % # { } $ $ % $ (48) Comparedtothenominalproblem,theMILPformulationhas additionallinearvariables and extraconstraints. 4.3.2IterativeSolutionofNominalKnapsackProblems In[ 36 ],theauthorsalsostatedthatwhenonlythecostscoe"cientsaresubjecttochange, therobustcombinatorialoptimizationproblemcanbesolvedbysolving original deterministicproblems.Thus,theoptimalsolutionoftheRKPcanbeobtainedbysolving 54

PAGE 55

deterministicKPsasshownin( 49 ). % maximum { } (49) wherefor maximize ( % % ) subjectto # { } (410) In( 410 ), isdenedtobe0. Alvarez-Mirandaetal.[ 48 ]improved( 49 )byreducingthenumberoforiginalproblemsto besolved.Theyprovedthatonly % deterministicproblemsneedtobesolvedtond theoptimalsolutionoftherobustproblem.Withthesameassumptionasourapproach,items aresortedsothat $ $ $ .Andtheoptimalsolutionisgivenby: % maximum { } (411) Thecomplexityofthisapproachhighlydependsonthee"ciencyoftheKPalgorithm implemented.IfdynamicprogrammingisusedtosolveeachdeterministicKP,thetime consumedforndingtheoptimalobjectivevalueandthesolutionsetis % and spacerequiredis % withaspacesavingtechnique. 4.4ComputationalResults Inthissection,wecomparethecomputationale" ciencyofdynamicprogramming algorithmsproposedinSection 4.2 withtheotherexactapproachesfortheRKPdescribedin Section 4.3 onanextensivesetofinstances. 4.4.1InstanceGeneration Tothebestofourknowledge,therearenobenchmarksestablishedfortheRKPwith protuncertaintyintheliterature.Therefore,basedontheschemesdescribedin[ 49 ],we 55

PAGE 56

consideredthefollowingsixclassesofrandomlygeneratedinstanceswithaccommodationon uncertainprots.TheseinstanceclassesaregraphicallydisplayedinFigure 4-3 Uncorrelated(UN):Nominalprots andweights arerandomlyselectedinrange .Theseinstancessimulatesthecasesthat and areindependent.Therobust prots israndomlychosenin Weaklycorrelated(WC): valuesarerandomlychosenin ,andeach is generatedin { % / } / israndomlychosenamongintegers Stronglycorrelated(SC):Each israndomlygeneratedin ,and / israndomlychosenamongintegers Inversestronglycorrelated(IC):Nominalprots israndomlygeneratedin ,and { / } israndomlychosenamongintegers % / Almoststronglycorrelated(AC): israndomlygeneratedin ,and isin / % / / / israndomlychosenin .This caseissimilartoSC,butwithsmall"noise"on Subsetsum(SS): israndomlygeneratedin and israndomlychosen amongintegers { % / } Intheexperiments,di! erentsizesofproblemsweregeneratedtotestourproposed dynamicprogrammingapproaches,DPRandDPS,andotherexactapproaches.Weexamined instanceswith and # { } ,andinstanceswith # { } and .Foreachcombinationofinstanceclassandproblemsize, instanceswere tested.Therefore,wehad360instancesintotalfortheRKP.Finally,foreachinstance,we testeddi! erentvaluesof ,tobespecic, # { } 4.4.2ResultsSummary Toshowthee!ectivenessofourdynamicprogrammingapproaches,thefollowing algorithmsweretestedwiththetestbedmentionedearlier: MILP:themixedintegerandlinearprogrammingformulation( 48 ). SKP:theapproachsuggestedbyAlvarez-Mirandawithsolvingseveralclassic0-1KP ( 411 ). DPR:ourproposeddynamicprogrammingapproachwithsolvingRKPrecursively. 56

PAGE 57

A B C D E F Figure4-3. TestinginstancesfortheRKP.A)UNinstances.B)WCinstances.C)SC instances.D)ICinstances.E)ACinstances.F)SSinstances. DPS:ourproposeddynamicprogrammingapproachwhichseparatesRKPintoaKP and/oranE-kKP. Tosolvetheclassic0-1KPinapproachSKP,weappliedalgorithm combo whichwas introducedbyMartelloetal.[ 50 ]andconsideredasoneofthemoste"cientalgorithmsfor solvingexact0-1KP.Thecodeispubliclyavailable.MILPweresolvedwithCPLEX12.6. ExceptSKPwhichwascodedinClanguage,allotheralgorithmswerecodedinJavaon ThinkpadX230witha2.60GHzCPUand8.0GBmemory.Atimelimitof1000secondswas appliedtoallexperiments. 57

PAGE 58

Table4-1. Averageruntime(seconds)onvarioustestinginstancesfortheRKP Instances ClassMILPSKPDPRDPSMILPSKPDPRDPSMILPSKPDPRDPS 100100UN0.0581.126 0.0010.001 0.0770.802 0.0020.002 0.0490.4600.005 0.002 WC0.0590.785 0.0010.001 0.0840.7780.002 0.001 0.0470.4260.003 0.001 SC0.0470.954 0.0010.001 0.1040.828 0.0020.002 0.0600.4780.006 0.002 IC0.0470.840 0.0010.001 0.0410.8120.002 0.001 0.0350.4580.004 0.002 AC0.0740.977 0.0010.001 0.1240.907 0.0010.001 0.0510.5080.004 0.001 SS0.0630.832 0.0010.001 0.0880.772 0.0010.001 0.0620.4250.004 0.002 500100UN0.1184.916 0.0010.001 0.1714.9030.005 0.003 0.0904.3800.013 0.007 WC0.0984.1660.002 0.001 0.2473.9490.005 0.003 0.0883.6370.012 0.006 SC0.0854.091 0.0010.001 0.2253.9920.004 0.002 0.1093.5020.014 0.007 IC0.1024.173 0.0020.002 0.0724.1980.003 0.002 0.0573.6940.012 0.004 AC0.0885.128 0.0010.001 0.2505.0110.005 0.004 0.1344.8500.014 0.007 SS0.3164.1760.002 0.001 0.5824.1410.009 0.003 0.2113.7370.012 0.005 1000100UN0.1439.8440.003 0.002 0.22910.0440.010 0.004 0.1339.5740.022 0.012 WC0.1198.0710.002 0.001 0.2617.8460.013 0.004 0.1237.5060.020 0.009 SC0.1238.1910.002 0.001 0.2728.6310.006 0.004 0.2188.3050.018 0.012 IC0.1268.661 0.0020.002 0.1228.2720.005 0.003 0.0847.9750.015 0.008 AC0.17410.438 0.0020.002 0.3159.9680.008 0.005 0.2299.1200.019 0.011 SS0.7488.085 0.0020.002 1.6368.0850.006 0.004 1.1907.6300.019 0.010 58

PAGE 59

Table 4-1 .Continued Instances ClassMILPSKPDPRDPSMILPSKPDPRDPS MILPSKPDPRDPS 5000100UN 0.43355.5820.009 0.005 0.63652.6500.025 0.015 18.39169.8900.072 0.037 WC 0.51140.3020.007 0.004 0.81240.8820.023 0.013 105.23940.6140.091 0.069 SC 0.37341.8960.008 0.005 0.98541.5630.022 0.016 6.77141.2740.069 0.065 IC 0.64042.7780.007 0.004 0.66742.8900.019 0.011 0.33642.7800.057 0.024 AC 0.39148.8890.008 0.005 1.22650.4560.026 0.017 7.77452.0340.082 0.071 SS10.00241.2750.013 0.004 8.96746.1860.033 0.020 7.77848.7080.067 0.032 50001000UN 0.50450.5320.037 0.025 0.93451.4230.205 0.108 13.73450.4980.901 0.635 WC 0.51943.0570.041 0.023 0.89146.5990.180 0.092 9.28544.0670.977 0.761 SC 0.77149.0290.033 0.029 1.07655.0620.225 0.129 4.02052.6530.907 0.570 IC 0.64944.3390.032 0.030 0.41743.9310.155 0.065 0.45441.3520.518 0.232 AC 0.76153.7880.051 0.036 1.11254.4260.234 0.134 6.32153.7250.908 0.592 SS14.13444.6100.045 0.019 44.74543.6270.241 0.111 53.88245.7481.134 0.360 50005000UN 0.60851.2640.169 0.117 0.87150.6241.107 0.641 152.88040.5355.178 4.086 WC 0.76647.9190.176 0.096 1.16359.0451.165 0.541 28.71249.0809.436 5.921 SC 0.67853.8670.144 0.112 1.07651.0991.350 0.706 52.06646.8606.384 4.192 IC 0.67242.2890.134 0.1060.337 43.3260.8250.345 0.337 43.2773.7581.671 AC 0.79145.5140.154 0.116 0.98644.0911.347 0.768 5.85341.8206.774 4.392 SS16.93342.7680.206 0.089 41.76746.7822.238 0.786 49.09444.1948.589 2.849 59

PAGE 60

Table 4-1 summarizestheaveragecomputationaltime,inseconds,offouralgorithms forallinstancesizesandclasses.Theshortestruntimeforeachcaseishighlightedinbold. Theresultshowsthatinmostcases,proposedalgorithms,DPRandDPS,outperformthe othertwo,MILPandSKP.MILPhasonlytwobestperformancerecordswheretheinverse stronglycorrelatedinstancesweregeneratedwith ,and or 50,respectively.AweaknessofsolvingRKPwithMILPisindicatedbytheresults,thatis, MILPisverysensitivetotheinputdata,theweightsandprotsofitems.Forsamesizeof problems,theruntimevariessignicantlyamonginstanceclasses.Forexample,withproblem size ,and ,problemsinclassSSneed16.933secondstobe solvedonaverage,whiletheaveragesolvingtimesofproblemsinotherclassesarenomore than0.8seconds.Asprovedearlier,bothDPRandDPShavetimecomplexity .And thiscouldbeobservedbytheresultsthatthecomputationaltimesofthesetwoalgorithms increaselinearlyonthesizeofproblems,suchasparameters and .Althoughtheoretically, DPRandDPShavethesametimecomplexity,theexperimentsillustratethatinpractice, DPSismoree"cientthanDPR.Andwhentheproblemisofsmallscale,theyhavesimilar performance. 4.5GeneralizationoftheRobustKnapsackProblem Inlightofthedynamicprogrammingrecursion( 42 )and( 43 )developedfortheRKP, inthissection,weconsideramoregeneralrobustknapsackproblemwithprotuncertainty andproposeadynamicprogrammingapproachtosolveittooptimality.Insteadofhavinga xedconservatismlevel fortheprotsofentireitemsintheRKP,weconsideracasethat itemsarepartitionedinto disjointgroupsandalevelofconservatismisassociatedtoeach group.Wenamethisproblemthe"groupedrobustknapsackproblemwithprotuncertainty", orG-RKPforshort. 4.5.1DynamicProgrammingApproach Weassumeitemsineachgrouparesortedaccordingtonon-increasing values.Let bethe -thgroupofitems,and istheindexofthe thiteminGroup ,forallgroups 60

PAGE 61

and | | .Noticethat,tosolvetheG-RKP,wewillsolveoneRKPwith foreachgroup.Therefore,basedonthedynamicprogrammingrecursionfortheRKP,we obtainthefollowingrecursionformulaappliedwithinagroup : # $ $ % $ $ & % if < maximum { % % % % } if $ for | | (412) # $ $ % $ $ & % if < maximum { % % % } if $ for | | (413) Tobegintherecursionfortherstgroup,wehave %, ,for and .Forothergroups,beforeselectingitems, whencapacityis ,theinitialvalueisthebestvalueobtainedinthepreviousgroupwithtotal weight whenconsideringallitems,i.e. %, ,for and maximum | | ,for .The optimalsolutionvalueoftheG-RKPiscalculatedby: % maximum { | | } (414) andthecapacityconsumedis % Ifonlythesolutionvalueisrequired,thenwecouldstophereandreturn % value.The dynamicprogrammingprocessisconductedintime | | | | where isthelargestconservatismlevel,andinspace ,sincestoragecanbe setfreeaftersolvingtheRKPineachgroup.Ifwewanttheoptimalsolutionsetaswell,we justneedtond % ,thecapacityconsumedbyeachgroupintheoptimalsolution,andsolve theRKPwithcapacity % forgroupjwiththeapproachintroducedinsubsection 4.2.2 .To 61

PAGE 62

obtain % ,anindicator isassociatedtoeachstatetotracetheinitial valueofthis state.Recallthatthesolutionof isbuiltfromthesolutionof ,therefore,theinitial valueisnotnecessarilytobe0,liketheKPortheRKP.Itcouldbeanyvaluebetween0 and .Startfromthelastgroupwiththeoptimalcapacity % ,applyingthefollowingsimple algorithm,wecouldget % valuesforeachgroup. Algorithm4.5. Determine % foreachgroup 1: % 2: for downto1 do 3: % % | | % 4: % % 5: endfor 4.5.2ComputationalResults Todemonstratethee"ciencyofthetwoproposeddynamicprogrammingapproaches, wewouldliketocomparethemwithanexactapproachbasedonaMILPformulationforthe G-RKP. First,weformulatetheG-RKPasamax-minproblemasfollows: maximize ! subjectto # { } (415) where isthecorrespondingrobustminimizationproblemforitemgroup ,whichis denedas: minimize ! % subjectto ! # { } # (416) 62

PAGE 63

Takingthedualproblemofeach ,aMILPformulationoftheG-RKPisnally obtained: maximize % $ % % subjectto % $ % % # # { } $ $ % $ (417) TheMILPformulationinvolves binaryvariables, continuousvariablesand constraints. Insubsection 4.3.2 ,wedescribedhowtheRKPcouldbesolvedbysolvingagroupof knapsackproblems.Indeed,thisapproachappliestotheG-RKPaswell.Basedontheresult ofAlvarezetal.[ 48 ],aftersolving % nominalknapsackproblems,the solutionoftheG-RKPwillbefound.Considerthee" ciencyofthisapproachandtheresultsof experimentfortheRKP,wewillnottestonthisapproachasacomparisonfortheG-RKP. Welistthealgorithmsthatweretestedandcomparedinbelow: G-MILP:themixedintegerandlinearprogrammingformulation( 417 ). G-DPR:ourproposeddynamicprogrammingapproachwitheachRKPsolvedbyDPR. G-DPS:ourproposeddynamicprogrammingapproachwitheachRKPsolvedbyDPS. Sixclasses(UN,WC,SC,IC,ACandSS)wereconsideredintheexperiment,eachwas testedwithproblemsize and # { } andsize # { } and .Foreachcombinationofproblemclassandproblemsize,10problemswere generated.Intotal,360instanceswereexaminedinourstudy.Toaddtherobustconstraints toeachinstance,itemswerealwaysdividedinto10groupswithequalsizeandweconsidered threecasesforthe valueineachgroup: ,and wasrandomlygeneratedin 63

PAGE 64

range .AllalgorithmswerecodedinJavalanguagewithCPLEX12.5onThinkpadX230 witha2.60GHzCPUand8.0GBmemory. Table 4-2 showstheaveragecomputationaltimeofthreealgorithmsforalltesting instances.Thebestsolvingtime(s)achievedineachscenariois(are)markedinbold.Compared toG-MILPapproach,G-DPRandG-DPShavemuchbetterperformance.Therangeofthe averagesolutiontimeofG-DPRandG-DPSare0.001-0.449secondsand0.001-0.450seconds, respectively,whiletherangeoftheaveragetimeofG-MILPis0.033-105.359seconds.The runtimedi!erencebetweenG-DPRandG-DPSisnegligibleinalltestingcases.Sincethese twoapproachesbreaktheG-RKPintoseveralsmallersizeRKPwherethesolutiontimeof G-DPRandG-DPSarealmostthesameasshowninTable( 4-1 ).Thecomputationaltimesof G-DPRandG-DPSmainlydependonthesizeofproblem,suchas and .Andthee!ect ofinstanceclassisinsignicant.FortheG-MILPmodel,theruntimeisnotonlydecidedby thesizeofproblem,butalsoreliesontheinstanceclasses.Forinstance,when ,and ,thetimevariescasebycasefrom0.322to105.359.AsPisinger[ 51 ] described,thenominalKPismoredi" culttosolvewhenthecorrelationofitems'protsand weightsbecomesstronger.WeobservedthesamerulewhensolveG-RKPwithG-MILPmodel. TheinstanceclassesSC,ACandSSaretendstobethehardestproblemstosolve:thetime consumedisrelativelylongerandtheruntimeofthreeSCinstancesandfourACinstances exceedthe1000-secondtimelimit.ComparedtoG-MILP,tosolveG-RKP,G-DPRandG-DPS aremuchmoree"cientandstableforallinstances. 64

PAGE 65

Table4-2. Averageruntime(seconds)onvarioustestinginstancesfortheG-RKP Instances # ClassG-MILPG-DPRG-DPSG-MILPG-DPRG-DPSG-MILPG-DPRG-DPS 100100UN 0.057 0.0010.001 0.057 0.0010.001 0.071 0.0010.001 WC 0.049 0.0010.001 0.040 0.0010.001 0.048 0.0010.001 SC 0.064 0.0010.001 0.048 0.0010.001 0.072 0.0010.001 IC 0.033 0.0010.001 0.031 0.0010.001 0.045 0.0010.001 AC 0.080 0.0010.001 0.052 0.0010.001 0.071 0.0010.001 SS 0.061 0.0010.001 0.095 0.0010.001 0.065 0.0010.001 500100UN 0.123 0.0020.002 0.098 0.0020.002 0.1290.003 0.002 WC 0.1050.002 0.001 0.084 0.0020.002 0.111 0.001 0.002 SC 0.143 0.0010.001 0.117 0.0020.002 0.127 0.001 0.002 IC 0.064 0.0010.001 0.057 0.0020.002 0.058 0.0020.002 AC 0.163 0.0010.001 0.123 0.0020.002 0.128 0.001 0.002 SS 0.241 0.0020.002 0.199 0.002 0.003 0.192 0.001 0.002 1000100UN 0.170 0.0020.002 0.1840.004 0.002 0.191 0.0030.003 WC 0.163 0.0020.002 0.167 0.0030.003 0.173 0.0020.002 SC 0.222 0.001 0.002 0.221 0.0030.003 0.269 0.0020.002 IC 0.102 0.0020.002 0.0760.004 0.003 0.083 0.002 0.003 AC 0.234 0.001 0.002 0.227 0.0030.003 0.277 0.0020.002 SS 0.663 0.0020.002 0.514 0.0030.003 0.448 0.002 0.003 65

PAGE 66

Table 4-2 .Continued Instances # ClassG-MILPG-DPRG-DPSG-MILPG-DPRG-DPSG-MILPG-DPRG-DPS 5000100UN 0.5180.006 0.004 1.5890.024 0.021 1.1690.015 0.006 WC 0.4980.008 0.007 1.3860.012 0.009 1.1490.011 0.009 SC 0.0040.004 69.4820.012 0.010 84.2970.012 0.009 IC 0.547 0.0040.004 0.3110.010 0.009 0.5070.007 0.006 AC 0.0040.004 55.0510.011 0.009 84.951 0.0080.008 SS 35.327 0.0090.009 9.9560.016 0.011 10.6710.008 0.007 50001000UN 0.5230.030 0.028 11.469 0.070 0.078 3.6430.050 0.044 WC 0.4990.023 0.023 10.561 0.063 0.067 3.3420.045 0.041 SC 5.175 0.022 0.024 75.967 0.068 0.070 1.4350.051 0.046 IC 0.450 0.030 0.031 0.322 0.057 0.059 0.345 0.0440.044 AC 0.684 0.0250.025 105.3590.076 0.071 1.8310.046 0.043 SS 20.1860.036 0.034 45.1960.100 0.094 23.728 0.0600.060 50005000UN 0.5010.119 0.117 1.6860.305 0.304 0.947 0.211 0.214 WC 0.498 0.108 0.116 1.6540.303 0.301 0.912 0.193 0.196 SC 3.149 0.109 0.116 24.227 0.316 0.325 1.748 0.190 0.200 IC 0.456 0.100 0.103 0.2700.260 0.241 0.345 0.176 0.177 AC 2.729 0.115 0.117 0.311 0.310 4.0820.218 0.215 SS 43.213 0.189 0.195 51.706 0.449 0.450 32.8680.320 0.310 66

PAGE 67

4.6ConcludingRemarks Inthischapter,weintroducedanewrobustknapsackproblemwithprotuncertaintyand agivenconservatismlevel.Adynamicprogrammingrecursionisproposedtondtheoptimal solutionvalueoftheRKP.Toobtaintheoptimalsolutionset,twoalgorithms,DPRandDPS, wereproposed.Moreover,therecursionmechanismofdynamicprogrammingisconsideredfor bothalgorithmstocontrolthememoryspaceoccupied.Wetestedtheproposedapproachona groupofrandomlygeneratedinstanceswithseveralcomparisons.Thesedynamicprogramming approachescanbeeasilymodiedforthegroupedrobustknapsackproblem. 67

PAGE 68

CHAPTER5 REACTIVEGRASPHEURISTICFORTHE0-1QUADRATICKNAPSACKPROBLEM 5.1MotivationandLiteratureReview The0-1quadraticknapsackproblem(QKP)wasrstproposedbyGallo,Hammerand Simeone[ 52 ]anditisageneralizationoftheclassic0-1knapsackproblem.TheQKPdenes thefollowingproblem:aknapsackofcapacity and itemswhereitem hasapositiveweight .Inaddition,wearegivenan non-negativematrix where standsfortheprot earnedifitem isincludedintheknapsackand istheprotearnedifbothitem and item areincluded.Allparameters, and ,areinteger.Withoutlossofgenerality,we assume > .TheobjectiveoftheQKPistondanitemsubsetwhoseoverallweight doesnotexceedtheknapsackcapacityandmaximizestheoverallprot.Thisproblemhasthe followingintegerprogrammingformulation: maximize ! subjectto # { } (51) TheQKPhasbeenintensivelystudiedduetoitswideapplicationingraph-theoretical problems(maxcliqueandweightedmaximumb-clique[ 53 ])aswellasintelecommunication problems[ 54 ],inlocationdesignproblems[ 55 ]andcompilerdesignproblems[ 56 ]. DuetoQKPbeingNP-hardinthestrongsense,nodynamicprogrammingapproach existswithpseudopolynomialsolutiontimeunless .Numerousupperboundshave beenpresentedassistingthebranchandboundprocesstosolvetheQKP.Galloetal.[ 52 ] proposedagroupofupperboundsfortheQKPwiththeconceptofupperplanewhichisa linearfunctionsatisfyingthatitsvalueisgreaterthantheobjectivefunctionvalueoftheQKP foranyfeasiblesolution.ALagrangianrelaxationofthequadraticknapsackproblemwas studiedbyChaillouetal.[ 57 ]andtheydeterminedthebestLagrangianmultipliervaluebased onthesolutionofasetofmaximumowproblemsinanetwork.BillionnetandCalmels[ 58 ] 68

PAGE 69

proposedupperboundsthroughlinearizingthequadratictermandthensolvetheLP-relaxation oftheproblem. Intheinterestofe" ciency,variousheuristicalgorithmswereinventedfortheQKPto obtainnearoptimalsolutionwithinashorttime.Twobasicstrategiesofthegreedyalgorithm whicharealsoapplicabletotheclassicKParetheprimalheuristicandthedualheuristic.In theprimal,feasibilityismaintainedthroughouttheconstruction,iterativelyaddingnewitems whichyieldthehighestprotifselected.Whileinthedual,startingwithaninfeasiblesolution anditerativelyremovingitemsthatdecreasetheleastprot(see[ 58 ]).HammerandRader [ 59 ]presentedLEXprimalheuristicwhichisbasedonthebestlinear -approximationof theQKP.AdynamicprogrammingheuristicwasintroducedbyFomeniandLetchford[ 60 ]. ApproximationalgorithmsweredevelopedfortheQKP.RaderandWoegingerproposedfully polynomialtimeapproximationschemes(FPTAS)fortheQKPwiththeassumptionthatall prots arenon-negativeandtheunderlyinggraphisso-callededgeseriesparallel.Di!erent FPTASforthesymmetricquadraticknapsackproblem(SQKP)havebeenproposedbyKellerer andStrusevich[ 61 ],andXu[ 62 ]. TheGreedyRandomizedAdaptiveSearchProcedures(GRASP)iscommonlyapplied oncombinatorialoptimizationproblemsduetoitssimpleimplementationbuthighqualityof solution.ThisapproachwasrstdescribedbyFeoandResende[ 63 ]anditisalsoknownasa semi-greedyheuristic.Forasystematicintroductiononthemethodologiesandapplicationsof theGRASP,werecommendsurveypaper[ 64 ]. OnesuccessfulimplementationofGRASPontotheQKPhasbeenproposedrecentlyby Yang.etal.[ 65 ].Theyexecutedexperimentsonbenchmarkinstancesandtheresultsshowed thee! ectivenessande"ciencyoftheirapproachwhichoutperformedoneofthestate-of-art heuristics-Mini-SwarmproposedbyXieandLiu[ 66 ].Inspiredbytheremarkableresults ofapplyingGRASPontheQKP,inthischapterwestudytheperformanceofGRASPwith di!erentconstructionschemes-theprimalandthedual,andpresentareactiveGRASPwhere 69

PAGE 70

theselectionofgreedyalgorithmisself-adjustedaccordingtothequalityofsolutionspreviously foundundereachgreedyalgorithm. Therestofthischapterisorganizedasfollows.InSection 5.2 ,wereviewthefundamental greedyalgorithmsandstudytheprimalanddualschemes.TheframeworkofGRASPis introducedinSection 5.3 .ReactiveGRASPheuristicinvolvingseveralgreedyalgorithmsis proposedinSection 5.4 .Section 5.5 discussesresultsfromcomputationalexperiments.Finally, weconcludeinSection 5.6 5.2GreedyAlgorithm Greedyalgorithmsweredevelopedbasedonasimpleideathatmakesthelocaloptimal decisionateachstagetargetingtoobtaintheglobaloptimumattheend.Greedyalgorithms donotguaranteetoderivetheoptimalsolutionandeventhenalsolutionisoptimal,itcannot generallybeproven.However,becauseofitse! ectivenessande"ciencyinsolvingdi"cultand large-scaleproblems,greedyalgorithmshavebeenwidelystudied. 5.2.1AlgorithmSchemes WecouldroughlydividethegreedyalgorithmsfortheQKPintotwotypes:theprimal andthedual.Intheprimalscheme,wemaintainthefeasibilityofthesolutionthroughoutthe constructionprocess.Theconstructionprocessmaybedescribedasfollows: Stepone:Thealgorithmstartswithanemptyitemset orequivalentlyhavingall decisionvariable Steptwo:Amongallitemsthatarenotin andhaveweightssmallerthantheresidual capacityoftheknapsack,choosetheitem withthehighestprotabilitytoentertheset orset Stepthree:RepeatSteptwo,untilthereisnoitemthatcanbetintotheknapsack. Note,anitem'sprotabilityiscalculatedwithaformulacalledthegreedyfunctionandwewill discussitindetailinSubsection 5.2.2 Thedualscheme,whichisthecounter-processoftheprimalscheme,startsfroman infeasiblesolutionandstrivestowardsafeasiblesolution: 70

PAGE 71

Stepone:Thealgorithmstartswithaddingallitemstotheset orequivalently havingalldecisionvariable .Thisisaninfeasiblesolutionbecauseof theassumptionthat > Steptwo:Withinset ,choosetheitem withtheleastprotabilitytoleave orset Stepthree:RepeatSteptwo,untilthetotalweightofitemsin doesnotexceed capacity .Afeasiblesolutionisfound. 5.2.2GreedyFunctions Agreedyfunctionisdenedtoevaluatethebenetobtainedbypackinganunselected itemintotheknapsackorthedamagecausedbyremovinganitemfromtheknapsack. Followingissomenotationthatwillbeusedindeningthegreedyfunctioninthissection: thesetofallitems, { } thesetofselecteditems, { # | } thetotalprotobtainedbyselecteditems, " thetotalweightofitemsinset Thegreedyfunctionmaybenaturallydenedas: ! % / # (52) Thisfunctionevaluatesthepossibleunitvalueofeachitem.Notice,thisfunctiononlydepends onparametersofitems,therefore,the valueisxedforeachitemthroughoutthegreedy algorithm.Ifthisfunctionisimplementedinagreedyalgorithm,itcanbecalculatedonceat thebeginning,savingprocessingtime.However,amajordrawbackofthisfunctionislacking accuracy,becauseitiscomputedwithrespecttoallitemsintheQKP,regardlessofwhether ornottheyappearinaparticularsolution.Again,theprot countsonlyifbothitem and item areinthesolution.Takingthecurrentsolutionintoconsideration,astraightforward 71

PAGE 72

modicationofthegreedyfunctionis: # $ $ $ % $ $ $ & / if # \ % \{ } / if # (53) Intheprimalgreedyalgorithm,protabilityiscalculatedforunselecteditemswithformula intherstcasein( 53 ),whileinthedualalgorithm,protabilityiscalculatedforselected itemswithformulainthesecondcase.Greedyfunctions( 52 )and( 53 )aredenedfromthe perspectiveofitems.Sincethecapacityoftheknapsackisbounded,tomaximizethetotal protintheknapsackistotendtomaximizetheprotperunitweight.Hence,thegreedy functioncanalsobedenedas: # $ $ $ % $ $ $ & / if # \ % % \{ } / % if # (54) 5.2.3NumericalStudy Inthissubsection,wecomparetheprimalschemeandthedualschemewithgreedy function onagroupofrandomlygeneratedQKPinstancestostudytheirperformance. Eachinstancecontains200items,valuesofprots andweights arerandomlygenerated intherange .Thedensity oftheprotmatrix,whichisthepercentageofnon-zero elementsintheprotmatrix,belongsto { } ,andthecapacityofthe knapsackissettobe ,where fallsinvedi! erentranges , and andinallcasescapacity is greaterthanthelargestweightofitems.100instancesaregeneratedforeachcombinationof valueand value. Table 5-1 summarizesthenumberoftimesthateachschemeyieldsthebestobjective valueandthenumberofties.Weobservethatrstthesolutionsobtainedwithprimalgreedy algorithmanddualgreedyalgorithmarenotalwaysthesame,therearegreatchancesthat 72

PAGE 73

Table5-1. Relativeperformanceofgreedyalgorithmsonrandomgeneratedinstances v dPrimalwinsDualwinsTies 0-20%25% 48 2626 50% 39 1744 75% 28 1260 100%15 4 81 20%-40%25% 63 2413 50% 31 1851 75% 20 8 72 100%19 7 74 40%-60%25% 60 3010 50% 35 2243 75% 22 9 58 100%14 5 81 60%-80%25% 58 2814 50% 36 2638 75% 17 1172 100%11 4 85 80%-100%25% 34 1749 50% 27 1756 75% 11 1277 100% 8 8 84 onealgorithmoutperformstheother.Second,neithertheprimalgreedynorthedualgreedy absolutelyoutperformstheother.Forexample,among100testinginstanceswithparameter ,and # ,in60instancestheprimalalgorithmndssolutionwithbetter objectivevalue,in30instancesthedualalgorithmproducesbettersolution,andtwoalgorithms obtainthesameobjectivevaluein10instances.Inaddition,astheprotmatrixbecomesmore dense,thenumberoftiesincreases,whichindicatesthereducingbenetofasinglealgorithm. Noobviouse! ectofcapacityrangeontherelativeperformanceofalgorithmsisobserved. Acriticaldrawbackofpuregreedyalgorithmsisthatineachiterationtheymakethe bestdecisionforthecurrentstate,andthiswilleasilymisstheglobaloptimalsolution.A straightforwardimprovementistoinvolverandomness. 73

PAGE 74

5.3GRASPFramework TheGreedyRandomizedAdaptiveSearchProcedures(GRASP)whichwasrstproposed in[ 64 ]isamulti-startoriterativemeta-heuristic.Withineachiteration,itperformsa randomizedsemi-greedyalgorithmwhichgoesthroughtwophases. Therstphaseistheconstructionphaseinwhichafeasiblegreedyrandomizedsolution isgeneratedbyiterativelymakingone-stepchangestothecurrentsolution,eitheradding anelementtothesolutionsetorremovinganelementfromthesolutionsetdependingon thegreedyalgorithmschemeapplied.Theelementbeingchangedisrandomlyselectedfrom arestrictedcandidatelist(RCL).TheRCLstoresasetofwell-rankedcandidateelements accordingtoagreedyfunction.TheRCLaddsvariabilitytothesolutionoftheconstruction phase.Theiterationstopswhenaddinganymoreelementwillviolatethefeasibilityofthe problemintheprimalschemeorafeasiblesolutionisfoundinthedualscheme. Thesecondphaseiscalledthelocalsearchphaseinwhichiterativeimprovementsare madeonthesolutionobtainedfromtheconstructionphase.Theneighborhoodofthesolution isexploreduntilnomovewillyieldbettersolution.Thelocalsearchphaseguaranteesthata localoptimumsolutionisfound. TheoutlineofGRASPisillustratedinAlgorithm 5.1 usingpseudo-code. Algorithm5.1. BasicGRASP 1: inputinstance 2: setbestsolution % tobeempty 3: while stoppingcriterionisnotmet do 4: constructionphase 5: localsearchphase 6: updatebestsolution % 7: endwhile 8: return % 74

PAGE 75

5.4ReactiveGRASP DiversicationandintensicationaretwomainprinciplesofGRASPalgorithms. Diversicationstrategiesencourageexplorationofuninspectedsolutionstoavoidbeingtrapped intothesamelocaloptimum.Therefore,thequalityofthebestsolutionfoundisimprovedby examiningmorelocaloptimalsolutions.Intensicationstrategiesencouragefocusonregionsof thesolutionspacethatcontainshighqualitysolutionsinhistoricalrecords.Thus,theattractive areamaybeexploredmorethoroughlyandthee"ciencyofthealgorithmisimprovedaswell. GRASPismulti-startprocess,everyiterationisindependenttoothers.Hence,itis possiblethattwoiterationsyieldthesamesolutionaftertheconstructionphaseorleadtothe samesolutionafterthelocalsearch.Toavoidrepeatedworkandincreasediversityofsolutions andthechanceofobtaininghighqualitysolution,penaltyfunctionsorrewardfunctionsare oftenapplied.Tobee"cientandintensive,amemorystrategyiscommonlyimplemented. Forinstance,FleurentandGlover[ 67 ]utilizedalong-termmemorymechanisminGRASP constructionphasetoconstructanewsolutionbasedonelitesolutionsfoundsofarforthe quadraticassignmentproblem.PraisandRibeiro[ 68 ]proposedareactiveGRASPforthe TDMAtra"cassignmentproblem,wheretheyusesparameter tocontrolthediversity,i.e. thelengthoftheRCL.The valueisrandomlyselectedfromasetofprexedvaluesandthe probabilityofselectingeachvaluedependsonthequalityofsolutionspreviouslyfoundwiththis value. Inourproposedapproach,insteadofself-tuningthevalueofaparameterwhichtothe bestofourknowledgeisthemainapproachofmostreactiveGRASPheuristics,itadjusts thegreedyalgorithmimplementedintheconstructionphaseofeachGRASPiteration, includingboththegreedyschemeandthegreedyfunction,accordingtothequalityofsolutions previouslyfound.What'smore,thememory-basedmechanismintroducedbyYangetal. [ 65 ]isapplied.Thefrequenciesofitemsthathavebeenselectedinpreviouslocaloptima arerecordedandutilizedtocreateanewfeasiblesolutionintheconstructionphaseofthe 75

PAGE 76

followingiterations.AdditionalnotationthatwillbeusedinillustratingtheGRASPapproach are: thebestsolutionvaluefoundsofar. theworstsolutionvaluefoundsofar. theaveragesolutionvalueobtainedwithgreedyalgorithm ascalerevaluatingtheperformanceofgreedyalgorithm theprobabilityofapplyinggreedyalgorithm thevectorofdecisionvariables. # { } .If ,the -thitemisselected and0otherwise. thesetofunselecteditemsthatcantintotheknapsackwithrespecttothe selectedset { # \ | } thelocaloptimumfoundiniteration anintegervector,whichequalsthesummationoflocaloptimaintherst % iterations, " the -thelementof ,anditisthenumberoftimesthatitem hasbeen selectedintherst % iterations. therestrictedcandidatelistatthe -thinneriterationofconstructionphaseof the -thGRASPiteration. theindexofthe -thiteminthe ascalarusedwhencalculatetheprobabilityofitembeingselected. theprobabilityofchoosingthe -thelementof 5.4.1GreedyAlgorithm ExceptrandomselectionofitemsfromtheRCL,theconstructionphaseofGRASP generallyfollowstheprocedureofgreedyalgorithms.Asfarasweknow,mostGRASP heuristicsdevelopedsticktoaspecicgreedyalgorithmthroughoutheuristics.Inourproposed reactiveGRASPapproach,bycombiningthetwogreedyschemesandthegreedyfunctions 76

PAGE 77

and ,weemployfourgreedyalgorithmstosolvetheQKP.Atthebeginning ofeachGRASPiteration,agreedyalgorithmisrandomlyselectedtouseintheconstruction phase,addingvariabilitytosolutions. Theprobabilityofeachgreedyalgorithmbeingselectedisself-adjustedbasedonthe qualityofsolutionspreviouslyfoundundereachgreedyalgorithm.Itiscalculatedwiththe followingformulas: % % (55) " & (56) Equation( 55 )estimatestheperformanceofeachgreedyalgorithmonsolvingthecurrent instance,where isaninputparameterwhichcontrolstheconvergencespeedtothealgorithm thatyieldsthebestaveragesolutionvaluessofar.Algorithmwithbetterperformancewill receivehigherevaluationscore,whichresultsinahigherchanceofbeingselectedinanew GRASPiterationwith( 56 ). Todeterminethegreedyalgorithm,acontinuousvariable israndomlydrawnfrom range .Thenthegreedyalgorithm % willbeimplementedinthenewiterationif # " < # .Thevaluesof areinitializedtobe1forall greedyalgorithmssothattheyhaveequalchancetobechosen.And isre-evaluated every & GRASPiterations. 5.4.2ConstructionofRCL Duringeachiterationintheconstructionphase,anunselecteditemisrandomlychosen fromRCLtobeaddedinto intheprimalschemeoraselecteditemisrandomlychosen fromRCLtoberemovedfrom inthedualscheme.Theconstructionphasestopswhenno unselecteditemscantintotheknapsackintheprimalscheme,i.e. ,orafeasible solution isfoundinthedualscheme.TheRCLcontainsallitemcandidatesthatwillbe consideredduringthecurrentiterationwhere + intheprimaland + inthe dual. 77

PAGE 78

ToestablishtheRCL,twomethodsarecommonlyused.First,wecoulddenetheRCL byitslength,becausetheRCLstores | | itemswiththelargestgreedyfunctionvalues.The lengthof controlstherandomizationofthesolutionconstructed.WithlongerRCL,the diversityofthesolutionincreases,becausemoreoptionsareavailabletoconstructthesolution. WithshorterRCL,thequalityofsolutiontendstobehigher,sinceonlyafewmostprotable itemsareinthelist.Second,theRCLcanbebuildbyquality,i.e.itcontainsallitemswhose greedyfunctionvaluefallsintherange % % ,where and arethemaximumandminimumgreedyfunctionvaluesofitemsevaluatedand isascalar between0and1.Parameter controlsthequalityofitemsintheRCL.Note,if ,only itemswiththehighestgreedyfunctionvaluegetachancetobeselected,andthisbecomesthe puregreedyalgorithm.If ,allitemsevaluatedareintheRCL. Theadvantageofusing todenetheRCListhatwecouldhaveabetterideaofthe qualityofitemcandidates.However,thediversityofsolutionsishardtoguaranteewitha xed value.Becauseitems'greedyfunctionvaluesarenotevenlydistributed,sometimes, although issettobealargenumber,onlyoneorfewitemsareintheRCL.Therefore,in theconsiderationofbothdiversicationandintensication,wewouldliketoestablishtheRCL withrespectto withinitialvalue .Andifnobettersolutionisfoundduring & iterations, thevalueof isincreasedbysetting ! ,where isapre-xedvalue.Bydoingso, moresolutionspacecanbesearchedintheconstructionphase. 5.4.3ItemSelectionfromRCL Ineachiteration,anitemisselectedfromthecurrent .Toencourageexploring solutionsthathavenotbeenreachedpreviously,itemsthathavebeenselectedlessinhistorical solutionsshouldhaverelativelymorechancetobeselectedfromRCLintheprimalscheme andhaverelativelylesschancetobeselectedinthedualscheme.Foritem in ,the probabilityofbeingselectedisdenotedas ,and .AftereachGRASP iteration,alocaloptimalsolutionisobtained,andintegervector isusedtorecordthe frequencythateachitemhasbeenintherst % localoptimums, ,i.e., 78

PAGE 79

" isthetimethatitem hasbeenselectedinprevious % iterations. iscorrespondingtotheindexofthe -thiteminthe .Therefore, and % % arethenumberoftimesthatthe -thiteminthe wereincluded inorexcludedfrompreviouslocalsolutions. Intheprimal,theattractivenessofselectinganitemshouldbeproportionaltothenumber oftimesthatitisnotinlocaloptimums,andalsotoallowitemsthathavebeenselected % timesintheoptimalsolutions, % isusedtorepresenttheattractiveness.Thus,the probability ,tochoosethe -thiteminthe atthe -thconstructioniterationofthe -thGRASPiterationis: ! % % / # (57) Inthedual,theattractivenessofselectinganitemshouldbeproportionaltothenumber oftimesthatitisinlocaloptimums.Andalsotoallowitemsthathavennotbeenselectedin thepreviousoptimalsolutionstobechosen, isusedtorepresenttheattractiveness. Thus,theprobability is: ! / # (58) Inbothcases,wehave .Tostarttheselectionprocess,afractionalnumber israndomlydrawnfrominterval .Let " betheintervalofthe -thitem intheRCL.Thenif fallsintheinterval " ,thenthe -thiteminthe or theitem isselected. 5.4.4RestartMechanism Asmentionedearlier,ateachiterationoftheconstructionphase,di!erentgreedy algorithmmaybeapplied.Intheconsiderationofintensicationandthee"ciencyof theGRASP,thelocaloptimaobtainedinpreviousGRASPiterationscontributestothe 79

PAGE 80

constructionofnewsolutionsinthefollowingGRASPiterations.Alsothevalueofparameter whichcontrolthequalityorthevariabilityofsolutionisincreasedevery & GRASPiterations. Therefore,therestartmechanismisnotsimplyrepeatingtheGRASPprocess,butbringing incrementalrandomness,thusalargersearchingareatothealgorithm. Yangetal.proposedacoreconceptin[ 65 ].Weconsiderthesameconcepttorepresent itemsthatappearinallpreviouslocaloptima.Theseitemsareinallelitesolutions,implying thattheyhave"important"contributionstotheprotofknapsack.Therefore,weallowthese itemstostayinthenewsolutionoftheconstructionphasebyaddingthemtothesolutionset beforetheconstructionphasestartsintheprimalgreedy.Orweforbidthembeingremoved fromthesolutionset inconstructionphaseinthedualgreedy.Note,thereisstillachance thatthecoreitemswillbetakenoutofthesolutioninthelocalsearchphase.Andweneedto updatethecoreitemset periodically.Inourimplementation, isupdatedevery & GRASP iterations,thesamepaceasevaluationofgreedyalgorithmsandthechangingof value.In GRASPiteration & isaninteger,if % ,item isaddedtothecore andwill stayforthenext & iterationsbeforenextexamination. 5.4.5ConstructionPhase Givenaparameter ,asetofcoreitems andaselectedgreedyalgorithm,weexecute theconstructionphasetobuildafeasiblesolutionoftheQKP.Thepseudo-codeofthe -th iterationwiththedualscheme(thegreedyfunctioncouldbeeither or )is shownbelow: Algorithm5.2. Constructionphasewithdualscheme: 1: initialization: , " %, and 2: while > do 3: for # \ do 4: calculatetheprotabilityofitem withrespecttothegreedyfunctionselected 5: update and ifneeded 6: endfor 80

PAGE 81

7: create containingallitemswhosegreedyfunctionvaluesfallintotheinterval % % 8: calculate 9: for # do 10: / 11: endfor 12: randomlydraw fromcontinuousrange 13: let betheindexofitemin ,suchthat " < .Removeitem fromset % 14: 15: endwhile 16: return feasiblesolution ThealgorithmoftheconstructionphasewithprimalschemeissimilartoAlgorithm 5.2 ingeneral,but isinitializedtobeanemptyset.Moreover,thewhileloopfromline 2 toline 15 continuesaslongas / ,andformulasusedtocalculatetheprobabilityofselection betweenline 8 andline 11 arereplacedbyequation( 57 ).Vector issettobe ,sothatin therstGRASPiterationitemsinthe haveequalprobabilitiestobeselected. 5.4.6LocalSearchPhase Solutionsobtainedfromtheconstructionphasearenotnecessarilyoptimal,evencompared toitsneighbors.Tofurtherimproveafeasiblesolution,localsearchisexecuted.Inlocal searchphase,weattempttoimprovethesolutioniterativelybyperformingasimpleoperation. Thesimpleoperationisdenedasa"ll-up"or"pairwiseexchange"movementwhichwas presentedbyGalloetal.[ 52 ]andwasoriginallyproposedbyPeterson[ 69 ]foracapital budgetingproblem.Duringthe"ll-up"operation,unselecteditemswhoseweightislessthan thecapacityresidualareiterativelyaddedtothesolutionset,whiletheexchangeoperation replacesaselecteditemwithanunselecteditemifsolutionreceivedisvalidanditsobjective 81

PAGE 82

valueishigher.Duringthelocalsearchphase,thesolutionissuccessivelyimproveduntilno bettersolutionexistswithone-stepoperation. Theresultofalocalsearchphasedependsontheinitialsolutionandthesearch technique.Thecommonlyusedandeasilyimplementedtechniquesarebest-improvingstrategy, rst-improvingstrategyandk-optstrategy.Someadvancedsearchstrategiesincludesimulated annealing[ 70 ],tabusearch[ 71 ],geneticalgorithms[ 72 ]andneuralnetworks[ 73 ].Fora systematicintroductionoflocalsearch,wereferto[ 74 ].Consideringthecomputationale!ort, wewillapplythesimplebest-improvingstrategyinourGRASPapproach,andthelocalsearch phaseisdescribedasfollows: Algorithm5.3. Localsearchphase: 1: 2: while do 3: apply"ll-up"and"exchange"operationsseekingforbettersolutionsintheneighborhoodof 4: if bettersolutionisfound then 5: update tobethebestsolutionfound 6: else 7: 8: endif 9: endwhile 10: return localoptima 5.4.7IntegratedReactiveGRASP Incorporatingallblocks,thepseudo-codeofproposedmemorybasedreactiveGRASPis givenbelow: Algorithm5.4. MemorybasedreactiveGRASP 1: Initialization: ! , %, , , 82

PAGE 83

2: while < do 3: for & to & do 4: randomlychoosegreedyalgorithm fromthefourcandidates 5: constructionphase: 6: localsearchphase: 7: updateLB,WorstLBand 8: endfor 9: if > then 10: 11: 12: else 13: 14: ! 15: endif 16: updateset basedon 17: 18: endwhile 19: return BestLB InAlgorithm 5.4 ,BestLBisthebestsolutionfoundsofarthoughtheGRASPalgorithm. AndthealgorithmisterminatedwhenthereisnoimprovementtoBestLBduring consecutive restartedGRASPiterations. 5.5ComputationalResults ThissectionpresentsthecomputationalperformanceofthereactiveGRASPalgorithm describedinSection 5.4 .AllalgorithmprogramswerewritteninJavaonaThinkPadX230PC witha2.60GHzprocessorand8.0GBmemory. ThreeGRASPalgorithmsandonedynamicprogrammingbasedheuristicweretestedin theexperiment: 83

PAGE 84

:Anon-reactiveGRASPapproachwhereasinglegreedyalgorithmwiththe primalschemeandgreedyfunction isutilized.Otherstepsareidenticaltoour reactiveGRASPapproach. :Anon-reactiveGRASPapproachwhereasinglegreedyalgorithmwiththe dualschemeandgreedyfunction isutilized.Otherstepsareidenticaltoour reactiveGRASPapproach. :TheproposedreactiveGRASPapproachwithrandomselectionofgreedy algorithmamongfourcandidates. :ThedynamicprogrammingheuristicpresentedbyFomeniandLetchford[ 60 ]. ThetestinginstancesarethebenchmarkinstancesgeneratedbyBillionnetandSoutif[ 75 ] wherethenumbersofitems | | are100,200,300andtheprotmatrixdensitywhichspecies thepercentageofnon-zeroelementinthematrixvariesamong { } Theseinstancesarepubliclyavailable.Theauthorsusedtheformat" "tolabelinstances, where isthenumberofitems, isthedensityofprotmatrixand meansthe -thinstances withapairof and values.Theoptimalsolutionofeachinstanceisalsoprovidedandwill beusedtocomparetheaccuracyofdi!erentheuristics.Note,somedatasetsaremissingfrom theon-linesource.Theyareinstance and .Foreachinstance, 100trialsaretestedoneveryGRASPapproach. Table 5-2 presentsthesummaryoftheaccuracyofthefourheuristicapproaches.We evaluatetheaccuracyperformancefromtwoaspects.Ononehand,werecordthesuccessratio (SR),whichisthepercentageoftrialswhoseoutputsolutionisthetrueoptimalsolutionof theinstanceamong100trails.Ontheotherhand,theaveragerelativegap(GAP)between theoptimalsolutionandthealgorithm'sbestlowerboundobtainedisrecordedinpercentage fortrialsthatdonotreachtheoptimalsolution.Inotherwords,thevalueincolumn"GAP"is calculatedby (theoptimalsolutionvalue-thebestlowerbound)/theoptimalsolution value. Fromtheresult,rst,weobservethatsolutionsofallthreeGRASPheuristicsare moreclosedtotheoptimalsolutionthanthatofthedynamicprogrammingheuristic.The averageGAPvaluesareshownattheendofthetableandthemaximumGAPvalueover 84

PAGE 85

Table5-2. Solutionaccuracyofseveralheuristicalgorithms SRGAPSRGAPSRGAPGAP 100 0100 0100 06.79 100 0100 0100 09.35 811.8 -1100 0999.3 -39.28 924.0 -3100 0971.5 -35.05 100 0100 0100 08.86 100 0100 0100 02.64 100 0779.0 -2100 011.40 100 0100 0100 08.63 843.1 -2992.1 -3941.2 -25.68 100 0100 0100 08.01 02.0 -1100 0100 02.79 861.0 -2100 0916.4 -30.19 03.1 -1 01.8 -1762.3 -22.07 100 0100 0100 00.79 100 0100 0100 00.09 100 0100 0100 04.11 100 0100 0100 06.31 893.3 -2100 0100 00.84 100 0100 0100 01.16 961.0 -3863.6 -3922.1 -31.38 100 0100 0100 00.00 100 0100 0100 00.62 100 0100 0100 01.26 121.6 -1123.4 -2667.6 -30.62 100 0100 0100 00.63 100 0100 0971.0 -30.04 621.3 -2727.6 -3954.1 -42.37 100 0100 0100 03.51 100 0995.4 -4993.9 -40.32 100 0100 0100 02.41 774.8 -3992.1 -4984.2 -40.02 100 0931.4 -3824.2 -30.29 100 0100 0100 00.11 100 0100 0100 00.06 100 0100 0100 00.17 100 0100 0100 01.85 100 0100 0100 00.13 100 0100 0100 00.00 01.5 -1 01.5 -1901.5 -20.39 85

PAGE 86

Table 5-2 .Continued SRGAPSRGAPSRGAPGAP 100 0100 0100 02.60 06.3 -3981.3 -4981.3 -43.20 100 0100 0100 02.82 100 0502.1 -3961.7 -45.14 726.5 -3621.6 -2794.2 -34.72 100 0100 0100 011.59 100 0100 0100 05.07 100 0100 0100 03.75 100 0100 0100 06.95 100 0100 0100 01.83 91.4 -2506.3 -3534.8 -31.13 100 0732.6 -3100 00.48 100 0100 0100 01.90 100 0100 0100 00.54 100 0913.2 -4100 00.77 955.2 -4991.0 -4100 03.06 100 0100 0100 00.49 100 0100 0100 02.18 100 0992.1 -5100 02.63 683.2 -3132.3 -2415.3 -30.64 100 0100 0100 00.41 02.1 -1 02.1 -1951.0 -20.06 100 0100 0100 01.17 100 0100 0100 00.20 100 0100 0100 01.73 985.3 -4100 0100 00.92 100 0100 0100 00.03 474.6 -2694.0 -3911.1 -30.20 100 0100 0100 00.07 01.7 -2 01.7 -2100 00.01 100 0964.6 -3100 00.02 977.3 -3100 0984.8 -30.00 100 0100 0100 00.35 100 0100 0100 00.07 100 0100 0100 00.00 100 0100 0100 00.05 09.3 -3 38.6 -3 78.6 -30.15 100 0996.5 -5100 00.08 100 0100 0100 00.19 86

PAGE 87

Table 5-2 .Continued SRGAP SRGAP SRGAPGAP 100 0100 0100 07.86 100 0 673.4 -3 981.2 -41.87 100 0100 0100 01.20 893.9 -2 721.1 -3100 03.73 100 0100 0100 06.18 100 0100 0100 03.56 100 0100 0100 010.21 100 0100 0100 03.56 100 0100 0100 03.18 100 0100 0100 00.78 100 0100 0100 00.94 843.5 -4 786.4 -4 892.4 -41.24 100 0100 0100 01.56 100 0 983.4 -4100 00.57 100 0100 0100 00.15 100 0100 0100 01.55 100 0 863.2 -4100 00.83 100 0100 0100 01.42 863.4 -2100 0100 00.63 Average 87.91.5e-289.19.0e-396.11.3e-32.36 allinstancesfor , and is0.31%,0.21%,0.023%and 11.59%respectively.Second,onlyconsiderthethreeGRASPapproaches,theaveragesuccess ratiosandGAPvaluesof and areclosetoeachother,insomeinstances overcomes ,whileinotherinstances, overcomes Thereactivegreedyalgorithmstrategyof helpstobalancetheresultsofGRASP heuristicswithxedgreedyalgorithmtomakethealgorithmmorerobustunderallinstances. SincethereactiveGRASPtendstoapplythegreedyalgorithmwhichfoundbettersolutions morethantheothers.Insomeinstances,bettercomputationalresultsarederivedby Forexample,oninstance and reachtheoptimalsolution62and 72timesrespectively,while reaches timesandwiththesmallestGAPvalue.The alsoyieldsthesmallestaveragevalueofGAPamongallheuristicalgorithmsshowing itsgoodperformanceoverallcases. 87

PAGE 88

Figure5-1. Computationaltime(seconds)ofseveralheuristicalgorithms. Figure 5-1 presentsthecomputationaltimeofthesefourheuristicalgorithmsonalltesting instances.Tobebrief,instancesarelabelbynumbersfrom1to97intheordershownin theTable 5-2 ,whereinstances1-39have ,instances40-78have ,and instances79-97have .Theruntimehaspositiverelationshiptothenumberofitems intheproblemforallalgorithms.Therankingofcomputationale" ciencyfromhightolow is , and .Theruntimedi!erencesbetween and or aresmallerthan10secondsinalltestinginstances.Figure 5-2 showsthetimespentontheconstructionphaseandthelocalsearchphase,givingaclearidea ofthecompositionoftheruntimeforeachGRASPheuristic.Intheconstructionphase,the timeconsumedisrelatedtothecapacityoftheknapsack.Sincetherearemoreinstancesin benchmarkshaverelativelysmallcapacityvaluecomparingwith tendsto runmoreiterationsthan tobuildafeasiblesolution.And containsboth theprimalandthedualschemes,itstimeoftheconstructionphaseisinthesecondplace ingeneral.Inaddition,weobservethatthelocalsearchtimeofthereactiveGRASPisthe longest.Thisismainlyduetotheneighborhoodexplorationoffeasiblesolutionconstructedby theothertwogreedyalgorithmswithfunction 88

PAGE 89

A B Figure5-2. RuntimedecompositionforGRASPheuristics.A)Constructionphase.B)Local searchphase. 5.6ConcludingRemarks Inthischapter,weintroducedanewreactiveGRASPapproachforthequadraticknapsack problemwithapplicationofseveralgreedyalgorithms:thecombinationoftheprimaland thedualgreedyschemeswithtwogreedyfunctions.Theselectionofgreedyalgorithmis self-tuningbasedonthequalityofpreviouslyfoundsolutionsundereachgreedyalgorithm. TheperformanceofthisreactiveGRASPtogetherwithotherheuristicsarestudiedwith numericaltestsonbenchmarkinstances.TheresultsdemonstratethatthereactiveGRASP 89

PAGE 90

whichimplementstheselfadjustmentmechanismhasthemostrobustperformancewithlittle longersolutiontime. 90

PAGE 91

CHAPTER6 CONCLUSIONS Inthisdissertation,severalvariantsofknapsackproblemsarestudiedandsolvedwith eitherexactalgorithmsorheuristicapproaches. InChapter 2 ,wepresentedanewhybridapproachtotheUKP,wherepartialdynamic programmingissolvedtoderiveprotupperboundsandvalidinequalities.Theseinequalities helptightenthecorrespondingintegerprogrammingformulationandreducethenumberof nodesexploredinthebranch-and-boundtree.Experimentswereimplementedonanextensive testingpoolwhichincludesproblemsinrandomcase,realisticcaseandhardcase.Resultsshow thatourapproachiscompetitivewithCPLEXsolverandseveralotherexactapproaches.There isatrade-o!betweenthetimespentongeneratinginequalitiesandthebenetobtainedfrom inequalities. WeextendedtheapproachdesignedfortheUKPtosolvethed-UKPinChapter 3 .To betterdealwiththemulti-constraintknapsackproblem,twomainmodicationsareadapted. Firstly,toavoidthelargememoryspacerequirementduringthedynamicprogrammingprocess, alistrepresentationtakestheplaceoforiginal dimensionaltableformat.Secondly,we relaxedthevalidinequalitytoachievecomputationale"ciency.Theapproachwastestedwith randomlygeneratedinstances.ComparedtotheCPLEXsolver,itimprovestheruntime,initial gapandnumberofnodeexploredinthebranch-and-boundtreeformostinstances. Inreal-worldproblems,itiscommonthatinputdataareuncertain.Thus,inChapter 4 we consideredarobustversionoftheknapsackproblem,wheretheprotofitemisnotxedtoa numberbutbelongstoaknowninterval.Aconservatismlevelisusedtocontrolthenumber ofitemthatyieldnon-nominalprot.Dynamicprogrammingapproacheswithspacereduction techniqueareintroducedtohandlethisproblem,andextendedtosolveamoregeneralproblem inwhichitemsaregroupedupandaconservatismlevelisassociatedwitheachgroup. Chapter 5 studiedthequadraticknapsackproblemwhichisingeneralmoredi" cultto tackle.WeintroducedareactiveGRASPheuristicandanalyzeitsperformancewiththree 91

PAGE 92

heuristics.Experimentswereconductedonasetofbenchmarkswhichareavailableon-line. ThereactiveGRASPalgorithmproducessolutionwiththehighestqualityinmostcases. 92

PAGE 93

REFERENCES [1] H.Dyckho! ,"Atypologyofcuttingandpackingproblems," EuropeanJournalof OperationalResearch ,vol.44,no.2,pp.145159,1990. [2] R.W.HaesslerandP.E.Sweeney,"Cuttingstockproblemsandsolutionprocedures," EuropeanJournalofOperationalResearch ,vol.54,no.2,pp.141150,1991. [3] A.Lodi,S.Martello,andM.Monaci,"Two-dimensionalpackingproblems:Asurvey," Europeanjournalofoperationalresearch ,vol.141,no.2,pp.241252,2002. [4] H.M.Weingartner,"Capitalbudgetingofinterrelatedprojects:surveyandsynthesis," ManagementScience ,vol.12,no.7,pp.485516,1966. [5] G.L.NemhauserandZ.Ullmann,"Discretedynamicprogrammingandcapital allocation," ManagementScience ,vol.15,no.9,pp.494505,1969. [6] E.L.LawlerandJ.M.Moore,"Afunctionalequationanditsapplicationtoresource allocationandsequencingproblems," ManagementScience ,vol.16,no.1,pp.7784, 1969. [7] G.R.BitranandA.C.Hax,"Disaggregationandresourceallocationusingconvex knapsackproblemswithboundedvariables," ManagementScience ,vol.27,no.4,pp. 431441,1981. [8] G.S.Lueker, TwoNP-completeproblemsinnonnegativeintegerprogramming .Princeton University.DepartmentofElectricalEngineering,1975. [9] C.A.FloudasandP.M.Pardalos, Encyclopediaofoptimization .SpringerScience& BusinessMedia,2008. [10] P.C.GilmoreandR.E.Gomory,"Alinearprogrammingapproachtothecuttingstock problem-partii," Operationsresearch ,vol.11,no.6,pp.863888,1963. [11] A.V.Cabot,"Anenumerationalgorithmforknapsackproblems," OperationsResearch vol.18,no.2,pp.306311,1970. [12] S.MartelloandP.Toth,"Anexactalgorithmforlargeunboundedknapsackproblems," Operationsresearchletters ,vol.9,no.1,pp.1520,1990. [13] G.B.Dantzig,"Discrete-variableextremumproblems," Operationsresearch ,vol.5,no.2, pp.266288,1957. [14] R.Bellman,"Dynamicprogrammingprincetonuniversitypress," Princeton,NJ ,1957. [15] P.GilmoreandR.Gomory,"Thetheoryandcomputationofknapsackfunctions," OperationsResearch ,vol.14,no.6,pp.10451074,1966. [16] H.H.YanasseandN.Y.Soma,"Anewenumerationschemefortheknapsackproblem," Discreteappliedmathematics ,vol.18,no.2,pp.235245,1987. 93

PAGE 94

[17] S.MartelloandP.Toth, Knapsackproblems:algorithmsandcomputerimplementations JohnWiley&Sons,Inc.,1990. [18] R.Andonov,V.Poirriez,andS.Rajopadhye,"Unboundedknapsackproblem:Dynamic programmingrevisited," EuropeanJournalofOperationalResearch ,vol.123,no.2,pp. 394407,2000. [19] V.Poirriez,N.Yanev,andR.Andonov,"Ahybridalgorithmfortheunboundedknapsack problem," DiscreteOptimization ,vol.6,no.1,pp.110124,2009. [20] S.MartelloandP.Toth,"Amixtureofdynamicprogrammingandbranch-and-boundfor thesubset-sumproblem," ManagementScience ,vol.30,no.6,pp.765771,1984. [21] J.C.Hartman, I.E.Buyuktahtakin,andJ.C.Smith,"Dynamic-programming-based inequalitiesforthecapacitatedlot-sizingproblem," IIETransactions ,vol.42,no.12,pp. 915930,2010. [22] R.S.GarnkelandG.L.Nemhauser, Integerprogramming .WileyNewYork,1972, vol.4. [23] P.SinhaandA.A.Zoltners,"Themultiple-choiceknapsackproblem," Operations Research ,vol.27,no.3,pp.503515,1979. [24] M.R.GareyandD.S.Johnson,"Aguidetothetheoryofnp-completeness," WH Freemann,NewYork ,1979. [25] E.Y.-H.Lin,"Abibliographicalsurveyonsomewell-knownnon-standardknapsack problems," Infor ,vol.36,no.4,p.274,1998. [26] H.PirkulandS.Narasimhan,"E" cientalgorithmsforthemulticonstraintgeneral knapsackproblem," IIEtransactions ,vol.18,no.2,pp.195203,1986. [27] M.Ozden,"Asolutionprocedureforgeneralknapsackproblemswithafewconstraints," Computers&operationsresearch ,vol.15,no.2,pp.145155,1988. [28] A.K.Chandra,D.S.Hirschberg,andC.-K.Wong,"Approximatealgorithmsforsome generalizedknapsackproblems," TheoreticalComputerScience ,vol.3,no.3,pp. 293304,1976. [29] Y.Akcay,H.Li,andS.H.Xu,"Greedyalgorithmforthegeneralmultidimensional knapsackproblem," AnnalsofOperationsResearch ,vol.150,no.1,pp.1729,2007. [30] F.MeyeraufderHeide,"Apolynomiallinearsearchalgorithmforthen-dimensional knapsackproblem," JournaloftheACM(JACM) ,vol.31,no.3,pp.668676,1984. [31] G.A.Kochenberger,B.A.McCarl,andF.PaulWyman,"Aheuristicforgeneralinteger programming*," DecisionSciences ,vol.5,no.1,pp.3644,1974. [32] E.HorowitzandS.Sahni,"Computingpartitionswithapplicationstotheknapsack problem," JournaloftheACM(JACM) ,vol.21,no.2,pp.277292,1974. 94

PAGE 95

[33] S.Balev,N.Yanev,A.Freville,andR.Andonov,"Adynamicprogrammingbased reductionprocedureforthemultidimensional01knapsackproblem," EuropeanJournalof OperationalResearch ,vol.186,no.1,pp.6376,2008. [34] J.M.Mulvey,R.J.Vanderbei,andS.A.Zenios,"Robustoptimizationoflarge-scale systems," Operationsresearch ,vol.43,no.2,pp.264281,1995. [35] A.Ben-TalandA.Nemirovski,"Robustsolutionsoflinearprogrammingproblems contaminatedwithuncertaindata," MathematicalProgramming ,vol.88,no.3,pp. 411424,2000. [36] D.BertsimasandM.Sim,"Robustdiscreteoptimizationandnetworkows," Mathematicalprogramming ,vol.98,no.1-3,pp.4971,2003. [37] M.Poss,"Robustcombinatorialoptimizationwithvariablebudgeteduncertainty," 4OR vol.11,no.1,pp.7592,2013. [38] D.Bertsimas,D.B.Brown,andC.Caramanis,"Theoryandapplicationsofrobust optimization," SIAMreview ,vol.53,no.3,pp.464501,2011. [39] O.KlopfensteinandD.Nace,"Arobustapproachtothechance-constrainedknapsack problem," OperationsResearchLetters ,vol.36,no.5,pp.628632,2008. [40] ,"Coverinequalitiesforrobustknapsacksets-applicationtotherobustbandwidth packingproblem," Networks ,vol.59,no.1,pp.5972,2012. [41] M.MonaciandU.Pferschy,"Ontherobustknapsackproblem," SIAMJournalon Optimization ,vol.23,no.4,pp.19561982,2013. [42] M.Monaci,U.Pferschy,andP.Serani,"Exactsolutionoftherobustknapsackproblem," Computers&operationsresearch ,vol.40,no.11,pp.26252631,2013. [43] C.Busing,A.M.Koster,andM.Kutschka,"Recoverablerobustknapsacks:thediscrete scenariocase," OptimizationLetters ,vol.5,no.3,pp.379392,2011. [44] ,"Recoverablerobustknapsacks: ( -scenarios," NetworkOptimization ,pp.583588, 2011. [45] H.Kellerer,R.Mansini,U.Pferschy,andM.G.Speranza,"Ane"cientfullypolynomial approximationschemeforthesubset-sumproblem," JournalofComputerandSystem Sciences ,vol.66,no.2,pp.349370,2003. [46] U.Pferschy,"Dynamicprogrammingrevisited:improvingknapsackalgorithms," Computing ,vol.63,no.4,pp.419430,1999. [47] A.Caprara,H.Kellerer,U.Pferschy,andD.Pisinger,"Approximationalgorithmsfor knapsackproblemswithcardinalityconstraints," EuropeanJournalofOperational Research ,vol.123,no.2,pp.333345,2000. 95

PAGE 96

[48] E. Alvarez-Miranda,I.Ljubic,andP.Toth,"Anoteonthebertsimas&simalgorithmfor robustcombinatorialoptimizationproblems," 4OR ,vol.11,no.4,pp.349360,2013. [49] H.Kellerer,U.Pferschy,andD.Pisinger,"Knapsackproblems.2004." [50] S.Martello,D.Pisinger,andP.Toth,"Dynamicprogrammingandstrongboundsforthe 0-1knapsackproblem," ManagementScience ,vol.45,no.3,pp.414424,1999. [51] D.Pisinger,"Wherearethehardknapsackproblems?" Computers&OperationsResearch ,vol.32,no.9,pp.22712284,2005. [52] G.Gallo,P.L.Hammer,andB.Simeone,"Quadraticknapsackproblems," Mathematical ProgrammingStudy ,vol.12,pp.132149,1980. [53] K.Park,K.Lee,andS.Park,"Anextendedformulationapproachtotheedge-weighted maximalcliqueproblem," EuropeanJournalofOperationalResearch ,vol.95,no.3,pp. 671682,1996. [54] C.Witzgall,"Mathematicalmethodsofsiteselectionforelectronicmessagesystems (ems)," NASASTI/ReconTechnicalReportN ,vol.76,p.18321,1975. [55] J.Rhys,"Aselectionproblemofsharedxedcostsandnetworkows," Management Science ,vol.17,no.3,pp.200207,1970. [56] E.L.Johnson,A.Mehrotra,andG.L.Nemhauser,"Min-cutclustering," Mathematical programming ,vol.62,no.1-3,pp.133151,1993. [57] P.Chaillou,P.Hansen,andY.Mahieu,"Bestnetworkowboundsforthequadratic knapsackproblem," CombinatorialOptimization ,pp.225235,1986. [58] A.BillionnetandF.Calmels,"Linearprogrammingforthe01quadraticknapsack problem," EuropeanJournalofOperationalResearch ,vol.92,no.2,pp.310325,1996. [59] P.L.HammerandD.J.Rader,"E"cientmethodsforsolvingquadratic0-1knapsack problems," Infor-InformationSystemsandOperationalResearch ,vol.35,no.3,pp. 170182,1997. [60] F.D.FomeniandA.N.Letchford,"Adynamicprogrammingheuristicforthequadratic knapsackproblem," INFORMSJournalonComputing ,vol.26,no.1,pp.173182,2013. [61] H.KellererandV.A.Strusevich,"Fullypolynomialapproximationschemesfora symmetricquadraticknapsackproblemanditsschedulingapplications," Algorithmica vol.57,no.4,pp.769795,2010. [62] Z.Xu,"Astronglypolynomialfptasforthesymmetricquadraticknapsackproblem," EuropeanJournalofOperationalResearch ,vol.218,no.2,pp.377381,2012. [63] T.A.FeoandM.G.Resende,"Aprobabilisticheuristicforacomputationallydi"cultset coveringproblem," Operationsresearchletters ,vol.8,no.2,pp.6771,1989. 96

PAGE 97

[64] ,"Greedyrandomizedadaptivesearchprocedures," Journalofglobaloptimization vol.6,no.2,pp.109133,1995. [65] Z.Yang,G.Wang,andF.Chu,"Ane!ectivegraspandtabusearchforthe01quadratic knapsackproblem," Computers&OperationsResearch ,vol.40,no.5,pp.11761185, 2013. [66] X.-F.XieandJ.Liu,"Amini-swarmforthequadraticknapsackproblem," IEEESwarm IntelligenceSymposium ,pp.190197,2007. [67] C.FleurentandF.Glover,"Improvedconstructivemultistartstrategiesforthequadratic assignmentproblemusingadaptivememory," INFORMSJournalonComputing ,vol.11, no.2,pp.198204,1999. [68] M.PraisandC.C.Ribeiro,"Reactivegrasp:Anapplicationtoamatrixdecomposition problemintdmatra"cassignment," INFORMSJournalonComputing ,vol.12,no.3,pp. 164176,2000. [69] C.C.Petersen,"Acapitalbudgetingheuristicalgorithmusingexchangeoperations," AIIE Transactions ,vol.6,no.2,pp.143150,1974. [70] A.Drexl,"Asimulatedannealingapproachtothemulticonstraintzero-oneknapsack problem," Computing ,vol.40,no.1,pp.18,1988. [71] F.Glover,G.A.Kochenberger,andB.Alidaee,"Adaptivememorytabusearchforbinary quadraticprograms," ManagementScience ,vol.44,no.3,pp.336345,1998. [72] B.A.Julstrom,"Greedy,genetic,andgreedygeneticalgorithmsforthequadratic knapsackproblem," Proceedingsofthe7thannualconferenceonGeneticandevolutionary computation ,pp.607614,2005. [73] Y.Xia,"Anewneuralnetworkforsolvinglinearandquadraticprogrammingproblems," NeuralNetworks,IEEETransactionson ,vol.7,no.6,pp.15441548,1996. [74] J.K.Lenstra, Localsearchincombinatorialoptimization .PrincetonUniversityPress, 1997. [75] A.Billionnetand E.Soutif,"Anexactmethodbasedonlagrangiandecompositionforthe 01quadraticknapsackproblem," EuropeanJournalofoperationalresearch ,vol.157, no.3,pp.565575,2004. 97

PAGE 98

BIOGRAPHICALSKETCH XueqiHewasborninBeijing,China.Shegraduatedwithabachelor'sinindustrial engineeringatBeijingInstituteofTechnology,Beijing,Chinain2009.Inthesameyear, shestartedhereducationintheDepartmentofIndustrialandSystemsEngineering(ISE)at UniversityofFlorida.ShereceivedaMasterofScienceinISEin2011andwillreceiveher Ph.D.fromthesamedepartmentin2016. 98